Antrea Community, 17 Jan 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Antrea Community Meeting 01/17/2023

Description

Antrea Community Meeting, January 17th 2023

A

Good morning, good evening, good afternoon and I could do whatever time it is for you, thanks for joining uh today's community meeting uh today is a Wednesday January, the 11th and uh in the agenda for today we have a review of the support for wireguard in a multi-cluster environments.

A

If I'm not mistaken, I believe that I'm, correct and xiajing is going to uh lead. The discussion is that correct am.

B

A

Right perfect, so go ahead! Changing uh over to you.

C

Okay, uh let me show my screen.

C

C

Okay, let's start uh today's topic is I'm dramatic, cluster skill tunnel. uh We we were in pain. We were implemented this on screen tunnel by the welcome tournament cluster, so.

C

C

Apple keynote has corrected my my spell hello. Everyone welcome to join the Android community and today, I will introduce the secure tunnel in the intro multi-cluster. Okay, let's start.

C

Firstly, let's review the architecture of the entromatic cluster in dark gateways on in the cloud in each clusters, such as your node C and another gateways. There are all the gateways are responsible for the closed class traffic.

C

um So if, if one port in a cluster wants to access to another port or access to a service in other cluster, it will forward to the for the package to the Gateway under the Gateway will communicated with each with other Gateway by the genift tunnel, such as just like this, um for example, if a port in the port a it um it will uh it, it will um assess the service in class B.

C

Firstly, firstly, it will enforce the packet to the notices on the Gateway and the Gateway well forward the packet to to the remote Gateway by Gene, if ton of just like this um and when, when the traffic gets to the class B the Gateway for for the package to um corresponding Port.

C

This is the current Implement tension of the math cluster, but it is a bit. um It is based on the junior tunnels and just generate generate Network encapsulation on technology, but it is pity that is. This is not a secure tunnel, which means it's that it cannot guarantee the on security of the traffic. Okay.

C

Sometimes we are focused on the security of the traffic. For example, um in one company there are some departments in different cities and once it um different cities for different Services. If, uh if once once, one team want to access to other teams the service, um maybe the traffic, it will go through on public network. So in this situation we will focus on the on security of the traffic so in in order to solve this, there are so there are many.

C

There are many secure tunnels or there are many open sources of Cortana such as such as Opera God, open, VPN, um l2tp and so on, and why we choose well God or it is because that um in some workout uh it's easy to use. We just need a endpoint and a public key or we will. We can just tell you uh in other node as PR, for example. We we don't. We don't need, maybe uh some other handshake or some other operations.

C

We can just add up here at either a node as a peer in in local okay. So it is very simple and easy to use um what's more on, it is in the Linux kernel. So we don't need to install extra on extra softwares or some other advantages. uh It is minimal attack.

C

It has minimal attack, service and high performance in some in some benchmark test, the workout it may be the best, maybe the best, secure tunnel in performance so as it has so many advantages, so we choose well I got to implementing our our design.

C

Oh yes, this is a desired architecture, um not too much difference with the previous one. Just um just we replace the genift tunnel with our garden, but actually we do not replace without we just in cap again you have the genetic package into workout and then the gateways will communicate with each other's by Wagga tunnel foreign.

C

Responsible for other closed cluster traffic, but in the in cluster, the architecture will not change or why we choose. This architecture is that we are trying to avoid too much unnecessary, edit addition of the flows and the architectures. So we just encaped the web genetic package into well God that we want to do so.

C

In this section, I will introduce the details of the design in.

C

I think maybe the why God the Wagga tunnel in Android market cluster can be divided into four steps. The first step is, you need to initialize the workout interface on Gateway nodes. Once we once we are annotated, a note as a multiple multi-class Gateway. We need to create network interfaces of our God in such as maybe workers, zero or maybe workout.

C

Thank you, the workout interface is responsible is is responsible for the work of traffic. In addition, we cannot, we cannot add the. um We cannot use the obvious attack command to add the workout interface to ovs, because it is a layer, 7 interface interface once we so.

C

The first step is initialize the workout interface when the next one on once we annotated or not as a multiplication.

C

um The the multiple card, the multi-class controller, well, will watch the Gateway and the export export some class information to others um such as Gateway information, subsiders and others.

C

It means that one Gateway one class to info in import, so um so we we can consider that we can start.

C

We can consider that um one Gateway is one workout pill, so the next step is ADD workout pills. uh We can we implement this by the with the help of crd class class info import.

C

After we add added all the appears, maybe not all the appears, but other peels. We need to generate corresponding obvious flows. What such a um you know, our current implementation, one subsiders on it. It will be one maybe one or some, maybe not one- some flows to uh corresponding clusters, uh so one one cluster will be if one cluster was added as a PO that we need the controller we need to generate the corresponding or vs flows. After all, foreign.

C

If we want to use web app to communicate with other clusters, we need to you when you, when you use the help of um post Network. So when you, um the the last step, is to generate the static Road in on the Gateway node.

C

There are some steps, such as clean and update, but this this is not the key step, so I didn't mention. I didn't mention example. Here.

C

um um This this, these two flowchart is, is a controller process. The the first one is is if weight, if weight is able the record or maybe wait or if we disable the workout on the controller process, will be like this. Firstly, we need to init the well the smart cluster, such as maybe class set, and the class clamp and foreign.

C

Clusters um was added to um to a class set we need to and we need to create, get gateway to uh to enabled close-class traffic uh create Gateway. Is we just annotated a node as a Gateway, once our Gateway was created on the controller well well to to two things: the first one, the controller will export the local class, the local cluster info to other clusters, and then it will import other class clustered info and the add flow to the remote Gateway if it is, but if we enable if we enable the workout.

C

There are some more steps after we create Gateway. uh We we also need to export the cost info, um but if, uh since the workout is enabled, we need to initial, you need to workout interfaces such as, inter uh network interfaces and others, and uh the the workout interface is up we need to when we will generating some public generated public key um public key and the export the public key to other clusters. So.

C

So the local cluster can be added as pure by other clusters. The last step is how to host a static, Rook root. We just mentioned before just this is this step? Oh sorry,.

C

Another apologize in podcast info once we imported a cluster info. It means that there is a cluster. There is a remote Gateway, so if we enable workout, we should add them, as appears and the add flow to remote. So there are maybe four four extra steps to to initial workout and the LPS.

C

So let's continue. Oh, this is a detailed detailed example to initial workout. In the current in the current process, foreign.

C

If something changes in Gateway or in class info import, Gateway Club on the cluster info import are two crds are two crd. So for my cluster, if something changed, there is a key. The key is a dummy key just to um just.

C

We just need the key to trigger the process. So if something changed, we will include a key on the this. One is process next at work atom. It is a kubernetes um kubernetes functions for kubernetes Control Function if uh it will try to get the key from the queue and if the key is okay, sorry yeah.

C

If the key is okay, um the controller, the controllers will list all the class info import and add flows for each cluster. uh We I have mentioned it before one one cat away. It means one class to info import, so it means that under one cluster, only has one Gateway. So one class info import means one cluster, so we will at least all the um class info import and other flows to add flows for them.

C

It means that we will add flaws for every maybe every member clusters, but in the in the in the new process. If we enable the wire guard, there are something um yeah, something new foreign.

C

Function if something changes in Gateway, we just um we just include a Gateway key and it means it is our damage tool or we just need a Gateway key to um triggers or process. But if something changed in the class info import, we were not into the damage key, but the class inverse name. Why we do this?

C

It is because that um it it is because that we should appears for each class each cluster and some sometimes, if we, uh if a remote cluster is restart- or maybe you know, um control map classical controller is crashed.

C

um The the workout public key will be regenerated, so we should update the appears, but if we use a dummy key, we should list all the.

C

Update appears, um it is not maybe.

C

It will cost too much time, so we uh in this situation, we will include the class info important name on about the process. Next work, next work items function.

C

If, uh if a key is popped in the queue and the workout is enabled, we were initialize workout interfaces just to mentioned before this is the first step I have mentioned before. I have mentioned before. Just like this.

C

But if this key is not a Gateway key and the workout is enabled, it means that one uh there is something changed in classic info importer, so we should do something, maybe just other LPS update appears on the airflows. So if the, if the key is not get working, it means that there's a there is a info Imports in the queue so under.

C

If the uh interface in workout interfaces is not ready, it means that which will in the workout interface is not in it to initialize. Maybe we should include again and wait for the interfaces, but if the key interface is ready, we should get the corresponding class info import and what got Pure as and the Earth flows questions.

C

The last step is update, Hostess that load. um So in this process, we we will add, uh where God, plus, with the help of class info import. So there must be some change in class info important. We should export our public key or our work, output and other information in this CRT. Now we will mention this later on. The next step is update. Add an update appears, and this one. This one is this step.

C

I have just just mentioned a cast info importer: it means a remote Gateway, so under a remote and a cluster only has one Gateway, so um a class info important is means. It means a remote cluster.

C

So if there's there is something changed on the update in the class info importers, the controllers were up will try to update the peers.

C

So let's continue.

C

um This one is about the flows and static Road. The workout is a layer, 3 technology so and we should use use. If we want to use this technology with and always ovs Maybe, we should. We use the help of host Network. So, firstly, um we were, as we did not. We need to add um other rules for the hostage Network.

C

um But um in the current in the current implementation, also also all the class traffic will be uh will be sent sent to the portrait contractor tunnel zero. This part is responsible responsible for other cross-class traffic and it is the other side of this. This port is a genief tunnel, so our in our in our implementation, we should try to hijack or maybe hijack, not the correct or the other.

C

um It is okay. We should hijack the on the traffic out out of the out of the control tunnel zero um interfaces, so we use a package Mark in in this flow.

C

This is the previous flow and this is a current flow. The difference is just this action. We will load a load of value to a package Mark package Market registered.

C

um If, if we enable workout, we should Mark the package as a workout package, so we should add. We should add a mark.

B

C

We there are the only difference between this between the all the flows and the new flows. After we add this Mark.

C

Just like this, um if it will check the mark and look up or the look up the entromat class table, as for the Android cluster table, the diff, the default on device of this this table is the Android multiple cluster workout interface. We just you need to we just created before so in this. um In this way we can, um we can do the minimal change of the minimum change of the flow. We just need to change one flows for the map cluster, and then we should we will not change other flows.

C

um Just just follows: we just need to add or actions so in.

C

The output of of the output of this flows uh of in the output of the traffic um close class traffic is entrance zero under the control, math class workout interface well hydrology traffic, so we can communicate it with other clusters by well God, where God turning that's the implementation.

C

So, let's, let's continue. um This is the change of CRT, not too much change. Only in the cost info import crd has been changed. um We we add a new field in this CR CID. It named well God and, as you said by this crd, we just need to export the public key and support.

C

Workout is easy to use, so we can just use these two with. We can just add, use this to um arguments. uh Actually we just we need to know we need to get to ip2, but the Gateway, IP and the cell settings has been export exported before so we just need to add a new field.

C

Okay, let's come to the summary.

C

The first summary is that the floors are almost the same in the as the current current uh implementation or which, which we, just only we just need to do. uh We need to change, is, is.

C

C

Is this flow? Is this flow in L3 forwarding table.

B

C

And the stack the second summary: the output Port is untrusted unsure tunnel zero. So we need our IP rule on the hosted to hedge on the traffic, the IP row and the static Road, um the second, the third one, the gateways at exports, the well-guard public keys on the on the port by CRT class, the info importer, because we just need to add a new field in the series definition.

C

um The last one this is, the crd class info in portrait is responsible responsible for exporting the Public Public key to other clusters once or once or new, on class info importers, or maybe a class info importance has been updated. The controller will try to add a new Pure or appetitors appear.

C

So package traffic in the left, the left one is uh is the current is the current implementation. uh We we will, if the pet, the the package from Port A to Port c, will go through um the will, go through the virtual pair and the Android tunnel zero and it will be forwarded to the Gateway once the package when the pack reaches the Gateway. So it will go through uh obvious pipeline too, and uh the ovs flows will.

C

The obvious flows well um cases of the destination Gateway by the by the destination Center to choose which one which Gateway is the remote Gateway and it will forward forward the packet to the remote Gateway and uh you- and this is the third obvious pipeline. Finally, we reach the policy um in in our implementation. There's no, not too much difference.

C

The only difference in is that on the Gateway node, we will add a well-guard interfaces once or package comes out from the Android tunnel zero. It will be in kept Again by the web guard interfaces and the and be sended to remote Gateway by wireguard by wild guard tunnel. That's the difference not too much.

C

In this step, in this implementation, we we just need to do the minimal change of the flows. The architectures and the crds We Will, We Will, which only we just we need to do, is create a new interfaces and modify one flow and and added on you and I added a new field in one crd, not other, not too much on changes of the architecture. So that's why we choose this implementation with.

C

C

A

B

D

A

Lost you for about 20 seconds so yeah. While you are discussing this slide, we lost you for the past 20 seconds.

C

But also I I I do not care hear. Clearly we 20 seconds.

A

uh You were not Audible for the past 20 seconds, so maybe you can just repeat your last sentences so be great.

B

E

Yeah I think uh I I think for the last uh search 30 seconds. I heard it clearly we saw as your network issue I don't know, but judging coughs a lot so you may we may lost a two or three seconds: okay,.

A

E

A

Was just it was just my networking issue in that case it's not a problem. Sorry.

C

That should we repeat it again: no.

A

No, that's. Okay. Go ahead. Sorry I said my apologies for the interruption.

C

So, let's come to the demo, um this demo will show the difference between um between the current um implementation and the workout. We will check the um how how the traffic come to the remote Gateway and the remote class tabs I'll enjoy the chat.

C

Okay, um can you see my screen I open a terminal.

F

C

The other Duty, oh sorry, I, think this. This one will be okay.

C

Oh yeah, okay, let me know now.

F

C

Yeah, okay, Central I have created two cluster, um two control cluster and traumatic cluster before so it will maybe save some time the first one on the the KE command. It means that it means that uh it's it's the cluster under the KW command. It means rest class.

B

E

Judging looks like your terminal is transparent and I can also see the background.

E

B

G

E

The information you input in the terminal.

C

E

Or you may just close it.

B

C

C

um um In this one on format, cluster there's three cluster: there are three clusters uh which means which named cluster list cluster is the cost West. um Okay,.

B

C

There's two memo clusters, east and west, and there is a server in the cluster West.

C

Is the engine server under our client in class East.

C

Under the cluster, the cost was exported our service to Cluster is to we can we can get the steps now. This one is I, just the the service exported by the West cluster.

C

B

To adjust the flow.

C

Okay: let's try to access it. Let's try to focus on remote service, oh okay! We can see that this. If we disable the workout, the traffic between clusters will go through the genift tunnel. Just like uh I have TC just like this.

C

It will go through the genift tunnel and actually, actually the the first step is to forward the forward to the package to the Gateway node and the second one. The second step is: is the Gateway node for the connector to the remote Gateway that that goes through the that that is able disabling um workout situation so there's another there's another other cluster enable the workout. Let's see the difference.

G

C

There's also a server on the client and the server is also a hundreds and in the cloud in the cluster East.

C

Because I can't.

C

So, um on the let's check the service: okay, the Android Multiplex is the service exported by the West cluster that try to try to access it.

C

Oh the command Kew is. It means that class to Easter with well with workout.

B

C

It it it, it works. Well too, let's, let's, let's.

C

Check the traffic okay.

E

So, judging when you say the KW is the least class with white guards. Actually uh you are moving to another demo clusters right.

G

E

G

E

Show us the in the islands of the command to let us.

C

Understand what exactly yeah, because it um every command to use the the arguments uh could configure. It is too long.

C

Let's check the interfaces, uh we can see that there is a networking phase named work, MC workout zero. Under that.

C

Okay, that's good again.

C

We can see that the workout, the traffic will go through the workout, the workout zero interfaces. Let's get more information about this. Oh.

B

C

Let's trace the significant genome interfaces, we can see that the traffic goes through the Journey. The Geneva interface is true and.

C

And uh By the time and the TTL we can find that it will go through the genetic interfaces first and then in cap in calculate but encapsulated by the workout interface. Again, this one is actually continued package.

C

Okay, let's check the cluster cluster info importers.

C

C

This one is on East cluster and uh okay. Oh, let's check with our workout on client. We can see that there is. There are some some traffic transport by the wagon.

B

D

C

As for the class info importer CR, we can see that there is a New Jersey news, Field named workout in the spec, and it will export the public key and support to other peers. And, let's compare in the previous, uh if we disable the workout the workout, we can check the workout in.

C

We can check the class info important.

C

Okay in this situation, there is no well God just know where God felt in if we disable the workout the workout on in multiple cluster, but in this, if it, if we enabled this it will, it will export as a port and the public key to other peers.

C

Okay, okay, let's see.

C

Okay, that's all my want to sell um so the workout, the workout over the genev and the workout is.

C

In this implementation, we just need to add: we just need more modify, one flows and uh once the ad is added on either field to one CR. This and that's my sure, thank you.

F

Can you show the IP loot configured on the cluster with workout enabled okay.

C

C

Well, oh sorry, we won't I forget to mention uh we will add a new cluster Rule and.

C

C

Hi, um can you use my screen? um That's what you want to say is. Is that what you want to say, yeah yeah.

F

Yes and uh you have a loot in table 10 to redirect the traffic to White Gardens device.

C

Yeah, yes, the default device in table 10 is the worker and control math cluster workout, zero.

F

Okay, uh is this uh what, when we, when we Mark the package with this specific Mac, do we filter the traffic must be close, Port traffic across cluster traffic, okay to do yeah? What's the condition of of this Mark and oh, it will be on every packet or it will be okay,.

C

um Sorry um I will show you. I will show you, though. First.

F

Six I joined to the meeting late, so maybe this this you already answered the question.

B

F

Sorry, for that.

C

No, no, no! No, no I, I think this. This question should be answered on. Should we um what we added this Mark by the subsider, so, oh sorry, by the subsider or what I think.

C

And in this in this um in this flow, and this flow will match will match the remote subsiders are just like this. If under the subsider is the cross, Class 12, subside, I, don't think uh maybe on that is Inc in cost traffic. Well will be marked, will be added well below this Mark.

F

Okay, let's see, uh does this Mark for D2 has any special mean, or it's just an example.

C

uh Just an example: not not mine, especially.

F

Okay got it. Thank you actually,.

E

E

Do we have any rule for this one and in number we should follow.

F

We already use the list, eight bits and I think you could start with the next one and then maybe you, you use some constant to reserve, for example, how many bits you will use to avoid the future conflict.

D

uh So uh I have a question on like whether this is like internode. We will be using wire guard or it is only uh like, inter cluster.

C

Oh sorry, I didn't have clearly. Can you repeat your question again, I'm sorry.

D

Yeah, my question is like uh for internode traffic right like we are using ipsec uh if I, if I am right uh and for inter cluster uh you're, applying to use uh wireguard right yeah, that's correct right.

B

C

On in cluster, we, maybe we sometimes we we will not use the exactly. Maybe we can just choose to um do not uh use.

C

That that we think um in cluster, maybe we can, we will not use, not use any um tunnel. We just used to live.

E

Yeah I think uh for maybe I, can ask to answer this question for in cluster traffic for now, I think. uh Currently we support gym in uh only common tunnel, unlike the Geneva GRE Etc, and we actually don't support why God tunnel, when we have multi-class to Future enabled which means if we like to you, know for the we want to add weigard for the cross-class traffic. We will not support that at uh you.

E

Can you can see that it won't work if both multi-cluster and the in cluster use wire guard you know in for in cluster traffic wire guard is also considered as a tunnel type just like the genev. So if the user changes the uh use the wire for the in-class traffic, it means the cross-cluster traffic won't work.

E

But uh yeah for now, I think we haven't decided to support both in-class and cross-class tracker figures or why God, as you know, for the cross-class traffic we actually in encrypted the tunnel, the traffic right so the Geneva the cab packages, and uh if this wagon it means a wire gun based on why God I, don't think that will work yeah.

D

Okay and what about like, uh like we are like choosing IPC, we are not choosing ipsec, we are choosing wire guard uh so uh like because ibsec like in some other conversation like it is fips uh standard like federal information, right processing, standard and wireguard is not so there I think we might have some kind of a regulator. Regulatory compliances for some customer.

C

Yeah, actually, this um this feature is coming from our customers requirement and the customer is want to use regards so actually, this is why we use.

D

Yeah I mean yes, many customers are looking about. Federal customers want like a 5ps compliance like which is like ipsec, so.

E

Yeah I think we may start with a wire guard, but we may do have some play in the future for the ipsec as well and yeah. It's also in our to-do list.

D

Okay, I mean there was one uh discussion where, like they wanted to use anxia uh because it was like ipsec, uh their image. Ib set compliant image, and uh so they give like preferenced uh preference to Andrea for that so yeah, especially Federal customers, yeah.

E

Yeah: okay, thanks good to know that maybe yeah we can prior to that, that's, but it depends on the whole development to plan yeah, not.

D

G

G

For me, so uh do we need to disable the RP filter on the well God interface, if I remember correctly I during the previous test, we need to test the IP filter to zero in order to oh.

B

G

C

Actually, we don't I, I I got a question and actually actually we don't need you to disable. um I did nothing when we use it well, I got it uh across um cluster, but it works well.

G

Okay, good to know that, um so so you mean uh that the original Road uh that's strange. So uh how can that happen? If we, if the default route without the pack Mark is not.

C

um If we do not use our impact mark and it will go through the genetic tunnel direction under the workout, where I got no traffic.

G

But but we don't need to disable that we filter on.

C

On patch on, maybe it is set to one as default. I remembered, not zero and not true uh chapter. This argument on the uh it is decided to one.

G

This thing one means the strict mode so yeah for um yeah, maybe maybe we can check it off like yeah. Thank.

C

A

All right are there any other questions.

F

More than two clusters in the class set, do you have multiple flows and uh do we need to use different marks for in the open, Flow loss and to match different marks for different uh remote clusters in the IP Lua table?.

C

Yeah I got a question: um okay, um I think. Maybe we we do not use the uh we do not need a different mark, because the Mark is just on to um is is just is just to match the IP Rule and under to the table to the ultramatic cluster table, um but for more than one member clusters are, I did um actually I didn't test the test, this situation, but I think maybe um maybe the word god.

C

Oh I, think it it. We don't need, because um that.

C

um Because uh um the destination of why God can.

C

uh Because the destination of the remote, uh the packet is the remote Gateway, so the workout, which well um will send to different uh different appears for different Gateway I, will check this I will double check this. This situation offline, but I, don't think I think this is more than one member cluster and more than one plus it will works. Okay,.

F

Okay, thank you. So at least we need two floats uh in this last three for one table to one. Each of them match one remote cluster right, because uh Network destination condition is different for different cluster.

G

C

F

B

F

I just want to say: I just want to want you to say: uh if uh there there should be one OpenFlow law for each remote cluster. uh Do we consider we just use how more IPA rule in the host Network to match the destination giveaway IP? So we don't need this Mark and we don't need the open Flow loss, I'm, not sure whether it could could work. But if the purpose of the Open Floor lure is just to identify traffic to remote clusters, though I guess, we could also use destination IP as well.

F

Did we consider that.

E

F

E

I think uh charging only added the mark load action for other other fields on this flow, it's actually required by a common multicast traffic. So it's uh if you mean that we can replace this flow with for white guard, but uh it may works, but I don't think it will work when we have no wire guard.

E

If we remove this rule.

F

You you mean this rule already exist, which.

B

F

The other one actually oh I, see I, see I, see they make sense. Yeah.

C

um Yes and and I think maybe I wish we use this way on on the we should add, which we should. We can add the impact Mark to the package. um Even the workout is disabled, and so it will not of there's no influence for the um cross-class traffic. So I think this is more, maybe more convenient convenience convenient. Yes,.

B

E

I think there are no more questions here.

A

Good run because also there is no more time we are right top of the hour. I thought we have a little bit more than one minute left, but since there are no more questions, I believe that uh we can conclude today's meeting unless uh the team wants to bring up any other topic for discussion waiting for a few more seconds.

A

And I believe that this might be all for today, so thanks everyone for joining and, most importantly thanks, jejing for this great presentation and demo- that's been uh extremely informative. I have to be honest with you. I still have to digest everything that you presented here, but it looks great.

A

So thanks thanks a lot for this and uh and we'll meet again in two weeks time uh for the next community meeting, we will be back to the normal timing of Monday Tuesday 5 a.m, Greenwich GMT and uh that's all for today, thanks everyone for joining and see you in two weeks time.

F

Thank you. Bye thanks, cheers, bye, bye,.