Antrea Community, 11 Apr 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Antrea Community Meeting 04/11/2022

Description

Antrea Community Meeting, April 11th 2022

A

So good morning and good afternoon or good evening, this is uh the anthia community meeting and today is uh tuesday april the 12th, of course, for some of you it will be still monday april 11th uh for today, as you can already see on the screen, we have a very interesting presentation from chan about a proposal for traffic control capacity and uh very interested to hear about it. So without waiting. uh Please change. Go ahead.

B

Okay, thank you zarato and thanks everyone for joining the meeting and I'm glad to have the opportunity to present the proposal for traffic control capacity.

B

Let's first talk about the motivation and the requirements.

B

We want to enable users to monitor network traffic passing in or out of a set of ports for several scenarios, for example when troubleshooting.

B

If we want to see what traffic a port receives and the sense currently the currently the the the common ways uh requires, you have tcp dump installed on the nodes and you and you have the privilege to plug in the node and learn the command to capture specific interfaces, and you will need to know the mapping between the nodes between the ports and the network interfaces it used.

B

Another way is that you, you need to run a demon set which have a host network demonstrate a dim set with the host network mode ports, and you need to run that that port on specific node and as there you need the privilege to capture the traffic from specific interfaces. So it's not quite easy to for for troubleshooting.

B

So one use case we think, is that if we can mirror the traffic or for ports to specific destination, then it will be quite easy to debug such network issues and another use case is for analysts.

B

We have a an issue from user. It requires that we can mirror traffic of specif specific ports to us destination while uh tunnel, and the issue requires uh uh we excellent tunnel.

B

The traffic can be married right to uh to a destination while uh yes, lance tunnel, and I think- but I think the use case can can be more- that we can use any kind of tunnel uf using the er span protocol to to send the to mirror the traffic and another use case is we want to enable users to deploy their own network protections while mechanisms such as dpi or you just use a traditional ips software well for ports to with fine grained control over network traffic?

B

Today there is no way to deploy a network protect protection mechanism and to to to intercept the north to to intercept the east west traffic. Maybe you could deploy such uh software or hardware at the edge node of your network, but you cannot intercept the class and the in-class traffic, so it would it will. It would be great if we can provide such capacity and reserve.

B

Is that, as mentioned about um in? If we don't have this capacity, user can only intercept the north south traffic, but not is the west traffic, so we propose to add a traffic control api using the id and the traffic control. Api accepts clients request about uh the the desired behavior of phone users to control the poor traffic with open flow lures.

B

The api is designed to be generic, providing a magnesium to specify the ports to to simplify the poles, whose traffic should be selected and the direction of the traffic and whether the traffic should be mirrored or be redirected under the destination to redirect or mirror the traffic in the future.

B

We can also provide other options, for example the protocols or for the traffic we're interested in, and maybe the product number of the traffic or the even the ip range of the traffic and to support, depending on the different user cases, we need to support different kinds of destination.

B

For one use case, I mentioned the request: the user requires a wax run tunnel and that is created on each node. So one possible traffic destination is a wasteland tunnel and we can also support other tunnel tabs and even the yeah span tunnel.

B

We can also support a local port for the scenario that the network analyzer is running on each node directory, so we can just create an o open with internal port and redirect the traffic to that port. Then the network analyzer could listen to the internal part directory and the product could be can also be a physical, physical, nic or a waste pair if the application is running in another named network namespace.

B

So for port traffic for the kind of the traffic traffic destination of four kind, we have two attributes. One indicate whether this is an internal os internal port. When it is true, it means that we need to create the port and when it is missing, otherwise the port must already exist.

B

For example, the port is a fake technique or is a user-created us device, and the peer name is used to represent the name of the pr device from which the traffic will be sent back to os. It should only be set for a direct action for me reaction. This is not needed, because we only need to mirror the traffic to the traffic destination, but for redirect action we need to because there is no copy.

B

The traffic is directed directly redirected to the traffic destination and after a network firewall process it it may drop it or it may allow it. So even it will need to send the traffic back to the pipeline after it process it. So we need to know which device it will come back for so that's the purpose of this field and for remote kind of for remote uh network analyzers, and we provide two kinds of destinations.

B

One is while the normal panel, so it could be wasteland, gnu and gi, maybe even st, and we need user to provide the remote ip of the tunnel and the id of the tunnel.

B

Then we will create a tunnel device based on this opportunity and this attributes and for some well-known traffic mirror protocol. We can also support it, for example, for espn tunnels. We provide a specific type for its arguments, for example the remote ip tunnel and the washing of protocol.

B

The other three fields are the ear span tunnel specific.

C

Do you think it's possible we can? We can have some other solution to pre-create the port, so here we don't need to automatically.

B

C

B

C

Traffic control policy.

B

Yeah uh in this proposal, both scenarios are supported. If users don't want to create their own product, meaning that they don't want to deal with anything related to open with switch. Otherwise they want, they need to mount open with way open with switch database and the circuit, and they need to manage open with switch. So that's one scenario, so we provide this internal uh property. You can set it to true.

B

Then we will create the port, but for some existing interfaces you mentioned user can pre-create them and where we are not, we will only attach them to open with switch. For example, user can create a web device or just leverage a basic, unique directory.

B

Then we, our code, were responsible for attaching this device to open with switch so that it can receive the traffic. Is. This? Is the latter? Your your your case.

C

So you mean uh so how that device is specified by by it's defined by name.

B

Yes, this, this is a common attribute of all traffic destination. It basically basically means the important name being open with switch.

B

So if you have created a virtual device- and you can just create put the name of the device here and specify internal as force.

C

Okay, uh I just feel it's little somewhat straightforward to follow, um but I think the other thing that um I think it might be possible if later you have much. If you have some matching criterias, uh probably it's possible. Multiple traffic control policy can can share one port.

B

Right yeah, yeah yeah, but that requires that the the destination with same name should have same properties.

B

For example, you cannot create one name with internal cell2 or under another, and another traffic control policy using the same name, but it is a ton, is a wax land tunnel traffic destination that that will not be supported if they have same prop uh properties.

B

C

Yeah I just mean uh in that case, since it's easier to you know understand if you see a the port must be pre-created now in the traffic control points I just I just gave you a response.

B

So you mean changing the description and the name of the fields to indicate that it should be.

C

Actually, I was thinking, uh should we see uh the product must be pre-created? That's my question.

B

um But that's that may be not convenient for other users. Yeah.

C

My major uh issue here like if, if you have multiple policies here, one port, it's a little, it's a little confusing and pretty easy to make a mistake. Since it's not um explicit it's the the way to experience say I want to share the port right. So.

B

I can think of of why use the case of in kubernetes that you have. You can create a host path, volume and the host path. Volume can be type of directory and a file and or even circuit file and the different ports can use same host path volume. But if you create one volume in one port saying this volume should be directory and in another port you see this port should be a file, then they they will have same situation they conflict.

B

So so when designing this, I also think about that. If we want to avoid such conflict, we could create a separate traffic destination crd so that we can refer to this destination and share it, but I thought it may make the api more complex complex. So I just use inline structure as a volume amount.

C

Okay, um sure you, I got a point, uh let me just think about it. um At least I feel probably we should think about how to restructure the uh fields a little. I think it's a little.

C

At least it's not very straightforward for me to understand another question for tono what what does tunnel? I d mean.

B

uh It just means the wing I id uh that this is a generator open, yeah in different protocol. It means different uh concept.

C

Okay and you're saying we will automatically create one tunnel port.

B

C

B

The point of such properties is to prevent users having to.

B

To operating open with switch directory, otherwise user, how to create another port and mount the same database and the circuit in the airport and the manager open with switch.

C

Okay yeah, I got it.

B

Yeah, yes, yeah, as you can see from this issue, what they do is that they match the same open with rage bridge, and I think that may be too complex or maybe error-prone for users.

B

If and the requirement is actually simple if we know the destination and we can create it directly, not how to be done by users.

C

Okay, actually, I originally was thinking uh another way that you can have, uh I'm not sure it's it's it's it's uh appropriate that we will not follow in theory, you can also have a configmatic or something to define what the problem want to create.

B

Yeah, that's one way yeah, but I I I also discussed this with only and when she uh that that's actually my original proposal, but I feel if we I was not thinking about conflict map. I was thinking about the configuration of android agent, but that when we use the one to use it, we need to use it where need to configure the configmap and restart android agent. It's a little different from a config map describing the tunnel this destination or a conflict map for all possible traffic destination.

B

um But but I I think this mode should be a little more friendly to users. They can describe all um all desired behavior in a single api, not something in a config map, something other things in a society sure that we could discuss.

C

Yeah, I got it again. I think my my only important if later reason is policy will be like security policy. You can have many policies and then, with different matching rules for the share one point, then it's become little. I don't know at least I feel it's probably not very straightforward to try to share the port. um Let's, let's let me think about that, and then we can do a flight on it.

B

Okay and then I have two examples of how the resource should be configured and what impact it will have for traffic for traffic. For example, we have two parts: originally they communicate with each other directly.

B

If we want to monitor the traffic between them, we could create a traffic control object and we use port selector to select this ports and with directions specified to both, and the action is mirror means that we want to mirror the traffic and this destination is a gie tunnel with remote ip set to this iphs.

B

Then, after this resource is created in the cluster, we will create a gi tunnel and each node in on the bridge of each node, and we will install some open flow loss to copy the traffic and send it while the gi tunnel, so that the network analyzer running on this remote node can receive the traffic between this ports.

B

And another scenario is redirect same uh wire and power to talk to each other directly or previously and after we created this traffic control object. This is redirect web app and we use pod selector to choose this post and the action is redirect and the destination is called local ports.

B

uh One part the receiving port is named, firewall, 0 and internal set to two means. We need to create this uh open, this pose as a os internal port, and we will also create another port os internal port named the firewall one, and then user click could deploy network firewall in in line mode, meaning that it could receive one receive traffic from one interface and decide whether you want to drop it. If we drop it, the traffic will not send back to the pipeline.

B

If it allows it, it will send the original packet back to os pipeline while another interfaces this. This is a common mode used by several traditional ids ips softwares such as snort and srikata.

B

After this is created, the traffic will first be redirected to the uh this device and received by the network firewall and then, if it allows it using the bike, sending traffic back to the bridge and redirected to its original destination.

B

And about the implementation, the the uh the roof workflow is like. Such a user will create a traffic control cr, while the api and enter agent will watch the traffic control api and guess the creation event.

B

After that country agent will handle the traffic control resource. It will create or update the port for tunnel destination if it does not exist or if it changes anywhere. It will filter ports running on its own node, based on the specifying the label selectors and get their open flow port ids, and then it will call open flow interfaces to install or uninstall lures, which mirrors or redirects traffic of the selected ports and for display implement data implementation.

B

For me, for mirroring, we have two choices: one is leveraging the openwave switch mirror function directory, however, it is hot based, so, for example, in this, in this example, it creates a mirror object and specifying traffic.

B

Centering is a zero and the iso one or coming from e0 and user 1 will be mirrored to user 2..

B

As you can see from the arguments, there's no way to specify which proctor to mirror and no no port number is not supported.

B

So if we want to how fine grain the control over the capacity, then this way is cannot meet our requirement and we can also have flow based. Memory means that we just.

B

Update the open flow loss to output the traffic we are interested in to one more destination, so the benefits of such approach is that we can filter traffic based on protocol port and iphones and because there is no native support for traffic redirect traffic redirecting if we use flow based mirroring, we can unify the approach for mirroring and redirecting the traffic classifying code will be shared. The only difference between these actions is that how the action, the open flow action or for uh of their corresponding for the corresponding traffic control apis objects.

B

So currently, I think this approach is more suitable.

B

And uh how was a simple demo about the mirroring action? uh Let me can you see my console.

C

Yeah, I can see.

B

It yes, okay, so currently, I'm in.

B

I have a cluster and I want to uh mirror traffic uh between.

B

Let me see this one between this, this port and.

B

This port, they are running on same node, those are vm and 0 1, so this is east west traffic and I want to get the traffic uh mirrored to another node while at a giant tunnel. So I have pre-created the tunnel. Maybe let me share it here.

B

Oh, I created a wax line tunnel, so this is the remote ip. This is another node, and then this is on the other node, as you can see from the ip.

B

This is the tunnel endpoint ip and I have created.

B

Link for this tunnel.

B

And then I will capture traffic on this channel device.

B

Then let me send the smp request. I I have to send three smp requests and then, as you can see now, I can receive the exactly same traffic on another node received on the tunnel device. Of course, we can also receive it on the physical, uh the physical interface directly, but it's encapsulated.

B

This is three in your package. Packets we received and also we can have.

B

I also create a local port which can receive the traffic mirror traffic so like this is running on the same note as the source and the destination part.

B

When I ping the destination, I can receive the same traffic on the mirror: local mirror port, so because in the demo I only uh mirrored a single direction. So we only see the outgoing traffic, not the incoming traffic.

B

Okay, that's the demo and the current plan is. We want to support all directions, including in ingress, egress and both and all actions, including mira, and redirect, and we want to support the listed traffic destinations, such as the local port, with run genius.

B

And in this washing we just want to make it basically work, so we will not do some fine-grained traffic con filtering in next release. We will support traffic filters like based on protocol port or iphs.

B

That's the current plan, and I also have some open questions. First thing is about the api name. Currently I name it traffic control, but I think, maybe is too large for what the api actually does, because in linux, traffic control tool can do many things mirror and traffic is only one feature of it. So another proposal I have is quite a mirrored is corresponding to tc mirrored function, or maybe we could use two different apis for mirroring and redirecting separately so that it this object.

B

Api name can be more specific for its for its purpose, and another question is about what should we do for uh directions, whether where the traffic, whether we should treat it state, 4 way, meaning that when direction is ingress, should it cover the should it cover incoming traffic? Only or should it cover the outgoing response?

B

Traffic for this incoming traffic, which is similar to network policies, always should just treat it as state stateless when it is in ingress, with only we only mirror already directed invoice incoming traffic and the user need to specify both if we want to get the corresponding response, traffic and uh happy to get discuss these options here and help your appearance, and that's almost all for my part and any questions.

A

China, I don't have a good option about name about.

A

I just think that we should not use uh something which maps a specific utility or maybe not something that is mapped to dc mirrored mostly because most people don't even know what it is, but something like traffic controller perhaps is kind of nice. Then you know naming everyone can have a different opinion.

A

The other question is, I looked at the design, looks really cool, but my question is in andrea and to end testing how we are going to validate. In your opinion, this functionality in the ci.

B

Yes, I think it's doable.

B

For validation, for example, to edit the mirror feature, we just need to uh run long long, remote for free. This can even be a kubernetes node. We are managing the ci is managing and it could uh we could capture traffic or just count the package number of specific interface to see whether it received the traffic.

A

uh In the mirrored okay and uh that net, for instance, something that counts, this thing accounts the pocket. It's not some piece of software that we have to develop. We can use some existing utility and we get that pretty much for free right, yeah cool thanks.

B

And for redirecting, I think, maybe sam, we could create such an object in in a test, and then we just use a os pair device to connect this to interfaces. Then the original package would will still go back to the os pipeline directly without being dropped. Then we can count whether the number of the packets are same as the number of the traffic between the two ports.

A

Okay, understood thanks.

C

Yeah, I think later, when you add a filter, then it sounds like you should do stateful right. Otherwise, the filter is a little hard to manage.

B

uh You mean when, when a partner, the a protocol port number is specified like yes, you.

C

Say a tcp, some tv doesn't import 80.

B

Yeah yeah, I'm not sure how how the traditional span function handle this scenario, but one.

C

Way, I think responsibility is stateless. uh That's smiling it's it's more like a switching feature.

C

Okay, and actually, I think the stainless one has some benefits too. uh I mean if you just look at switch, uh the performance can be better, but I'm not sure it might much our uh I mean that. Does that matter in in our case or not, uh do you think if we do things, the performance can be better.

B

I think so because currently you went to support state four we need to. We need some actual status to be come to be stored in connection track. Otherwise we don't know whether this is a response traffic. If it is, if we only need to support stateless, I think we don't need to start anything in city.

C

B

To resolve the problem of the use case, when port number is specified, I think user can still say that the direction should be both and when it, both under a port number, for example, 80, is specified.

B

We just check whether it exists in either direction.

C

Yeah, probably that makes sense.

C

Okay, now, of course, for redirection, uh no, not really for the internal portal case, uh let me see for the internal port case. Why do you think we should auto create the internal port? I think if it's internal port likely should be marked by some other solution, consume the traffic right.

B

uh You you assume the the other solution of the opera, the other application. uh We have mount open with switch resources and created the port by himself.

C

Now, for example, in your case I mean even you even you create a port, the other search you need to know about you you're saying that, since the.

B

You they, they should give the name right, so they can listen.

C

To the name, I I think it's uh problem for mirroring it's it's clear, but for the rederived case you have a peer port you're, seeing that part is created by who is created by the.

B

It it's also this attribute will determine the way determining whether we will create the two devices. If it is internal, we just create two internal parts. If it's not internal, we'll assume that two devices already exist.

C

C

Another one that for for tunnel actually people don't really care about the name of the tunnel. Part right.

B

uh Yes, yes, it is to keep the unique uh for, for example, the scenario that multiple yeah yeah you you make, you make sense, but shouldn't we have a name or we just create a render name.

C

Okay, hopefully I need some time to think.

B

uh An m is still good because if, if they do have multiple policies- and they want to share.

B

At least using same name and specifying same arguments can avoid multiple parts in single bridge.

C

Yeah, I think, for internal points, it's easier to understand, since it's really a port, so you should have a name for tunnel.

B

Channel also maps to uh os yeah.

C

I know I know but again.

C

C

To avoid any conflicts.

B

Yeah, maybe maybe that depends whether our documents will be clear for that. If user just follows the documents, the guide and they, they should understand this. This name, for example, for for cuban export you will also need to. You will also need a volume name.

B

Actually what your name is only used to identify the the volume when you use it.

C

For that volume needs help to be identical. Of course, pause.

C

In your case, uh the actually you you actually have a requirement, everyone share. The same tunnel must share the same name.

D

Yeah in that case I was, I was going to ask the question: have we entertained the option of having the destination to destination as a separate crd itself? I mean I feel like there's enough information for it to be a standalone resource, and then, if we were talking about us some tunnel, we can definitely, you know just define a crd. Just referring to this is a destination created for this tunnel and the other resources can just refer to it by name just like the cluster group.

B

Yeah, yes, I agree, I actually thought about it. Yeah.

B

I think it's just the two ways how we present, how we avoid conflict and how we present the apis to user. I think that is acceptable, yeah yeah, but we we couldn't discuss it.

C

For tunneling series, if you want, you can see the flow-based tunnel.

B

Right, uh I'm not quite sure because uh because when we set the when we when we output the package, if we we want output of the package to the original tunnel and another tunnel, which there is only single tunnel metadata right when using flow based tunnel, we need to set the remote ip to the tunnel metadata.

B

Then I'm not sure whether it works. If we want to output outputting the package to two devices, both of them have no remote ips specified.

B

So in my test, I I specified the audio.

B

I'm not sure whether the approach you you mentioned the world.

C

Okay, arizona.

C

Okay, uh before, let's think about.

D

C

A little um yeah, I just feel it's a little inconsistent between different types and even for mirror, and the redirect is a little inconsistent. I mean you have this pure name inside.

A

C

A

D

But the name is outside.

C

The is in the directing under the traffic, that's instructed if our father, I think these are balances. Probably probably we can discuss often.

B

A

Is there a yeah? No, no other question for me any other question from the community.

A

I guess we can interpret this silence as a no. I guess uh I mean. I think this was a great presentation, a great new addition to andrea features, uh really looking forward to see it in uh in the first uh in andrea 1.7, according to the road mapperchon listed today. So thanks a lot for this presentation, sean- and uh I guess this is also all for today, unless anyone has something else to bring up before. We finish the call I'll just wait, 10 seconds to see.

A

If anyone has anything you'd like to discuss in this last two minutes.

A

Perfect, it seems that therefore it's all for today, so thanks everyone for joining and uh we'll meet again in two weeks time on tuesday april the 26th thanks again for joining- and I wish everyone a good evening a good morning or a good afternoon-.

B