Antrea Community, 1 Mar 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Antrea Community Meeting 03/01/2021

Description

Antrea Community Meeting, March 1st 2021

A

A

So welcome to today's instance of the andrea community meeting today is tuesday march, the 2nd or well still monday march, the 1st. If you are in the united states and for today we have two topics in agenda.

A

The topics are and not necessary in this order cube proxy removal design for linux and uh which hang and hon liang will uh introduce this design. Then we have a demo uh about the flow aggregator and this demo will be given by srikar and eo.

A

um We don't have a strict order and therefore I would like to ask you whether uh zurich neo would like to go first or whether, instead, you want to first discuss the design of the cube proxy removal removal, if you don't have any preference I'll pick a random one, I'll wait 10 seconds to express if you have any preference.

B

We don't have any preference.

A

Sounds good, uh so I will say: let's keep the demon, which is usually lighter and nicer to the end, and let's start with the reviewing the design proposal for the removal of the cube proxy.

A

Does that sounds good to you.

C

A

Perfect so I don't know who's going to present either way chance or hong kong or both of them, but please go ahead.

D

Okay, so it stop.

D

Hello, everyone: this is a presentation about the design of replacing cookie proxy and it's linux only not including windows.

D

Here's all the lan. There are three paths, motivation, work, items, design.

D

First, the motivation: if a curvy proxy is removed, the implementation can sell a lot of cpu cycles and memories and the entry proxy takes all charge of service traffic and we can have more control over service traffic.

D

We still need the following abilities to do the replacement first during the installation. The entry is to access to kubernetes service towards resources, thus some components you actually need to able to access to cove kubernetes api servers directly.

D

Second, the ability to serve cluster ips service or host node until proxy does not have the ability yet so since anti-proxy will take out of the service serving service will not be available until the first thing of entry proxy.

D

So we need to make sure our sub components rely on service to be waiting until entry proxy is ready. First, some options should be compatible with copy.

D

Proxy, the kubi proxy may not exist when we are installing entry in the filter. This will make services not work during the installation. Meanwhile, actually needs to connect to the kubernetes storage towards resources.

D

To overcome this issue, anti-agent and entry controller should be able to access to the kubernetes api server directly.

D

In the current implementation, we use the in-class copy config to set up the connection of the kubernetes client in the replacement design. We need to change the server address from the service address to the ip address connecting to the kubernetes again server.

D

There is a pr 173 file for supporting this yeah into agent. We also need a similar pr for the entry controller.

E

Sorry to disrupt, uh I know you haven't attended that you will rearrange the job order off for components in an agent right because they rely on the availability, but uh could answer controller depend on the answer agent service ability as well?

E

We we could first write an answer. Agent stand, then enter controller can continue, use this service address to connect cuba, api.

D

Server, I'm sorry, I.

E

Mean I mean you have an item that uh it says components in anti-agent will be they'll start their bus drop order will be re, arranged right so that other services, other modules that rely on um android proxy will start. After that I mean. Could this apply to enter controller tool because enter controller can rely on android agents to provide this service access so that it doesn't have to connect to the api server using the using the iphs.

D

I got the other meme.

F

Would would upgrade work in this scenario because I remember we always upgrade intro controller first right.

E

E

This doesn't turn it doesn't impact this. I think it doesn't impact the upgrade process because uh it still needs.

E

I think android agent doesn't really doesn't especially entry proxy doesn't rely on the net policy connection to start right, so um android controller is not running android plus it can still start.

F

D

D

Myself, android controller can rely on the services implemented by your entry proxy, but antoine controller relies on the anti-controller connects to kubernetes. Api server is also okay.

E

Yeah I just want to discuss whether there is it is necessary to introduce another configuration to android controller. If this can, if this is not a wireless, not a wireless service issue, sure you, you can always add, make it connect to a pass server directly.

E

Okay- and I understand.

D

Maybe we should just discuss the introduction part not including the entry controller part.

D

Here we go sure: okay, next, the android agent, the android agent, the entry, client and entry agent use the entry service towards internal network policies, and we make the entry client to be initialized after the first sync of entry proxy, as the android service is ready.

D

Okay, next, currently before routing to os the service traffic, is manipulated by asp table items. These items are under control of curvy property.

D

What we expected is that service traffic can bypass could be proxy and be routed directly to oes pipeline okay. Next, firstly, we use fpsat to match the cluster ip address, protocol and port the maximum time complexity is over. Since we use an ipsec with a hashtag in copy proxy. Once a cluster ip is created, the related items will be appended to happy tables and its matching time. Complexity is all.

D

In the mango table of ip tables, we created a custom chain that marks packets matching the ip set just mentioned with marker 0x f2. The chain should be referenced in both output and prerooting change.

D

We add an ip rule to make the traffic marked 0x f2, go to a specific routing table named andrea that we added the routing table of entry, has only one default, unlinked route which forwards all traffic tool and to deduct zero. The traffic will be processed by os pipeline later note that uos in oes pipeline table 20. We also need to add a flow in response to the russell ips erp request.

D

In that table we match the traffic marked 0x f2 and the master editor.

D

In summary, we can see that first, the traffic is marked in mango table. Second, with an ip rule, the traffic will be loaded in a specific routing table, not the default routing table. Third, the specific routing table has only one special default load. The traffic is rotted the tool and to dw0 by this rod item first, the packets are masqueraded in that table with entry, gw0.

D

In os pipeline table 41, the class ip will be master and the central group integral in the group. The traffic will be distributed by loading and point ip address to an exam ruxin.

D

This is the dependency of the sub components near adjacent from these dependencies. We can get the bootstrap outer.

D

And the last part, further options are compatible with copy proxy for the red ones. We consider supporting term in the future for the black ones. We have alternative options and we have employee implemented the great ones.

D

Graham sorry, the green ones, all right any.

D

E

Could you go back to the uh the happy tables and appear lure.

E

E

uh Could you go back to the ivy tables and ip root lure configuration part, so do it do not help? uh I I think uh uh yeah? Yes, yes, well, were there were the packages being added in the host network.

E

What will it, what not it does it's not or did not.

D

Ask and the packages the package source address should be masqueraded with the with the ipl address of entry, gw0.

E

Oh okay, so you too is for all traffic or specific.

D

Traffic space, specific traffic marker with 0x f2.

E

Access traffic or some special service access traffic, like a harping traffic.

D

Oh sorry, the hacking traffic is from port is that I mean the traffic from a power toolkit to its service.

E

This is from host to class ip lite, so the original iot is already the host ip right. Why do we need a mass grid help.

D

uh This is for host to uh to a service up here to this user for host to service traffic.

E

Yeah yeah, I mean from localhost right from from.

C

So in chinese for return traffic to return to the source house that guys dynamics.

E

uh I I think this is from localhost and not from remote host right. This is not for no port traffic. Sure.

C

But the the backend can be a remote and remote node.

E

uh Yeah, but this this is, this is doing s net right. Yes, I just mean that the uh source ip is already this host ip right and needs to be mask, radiate.

C

The new how the return traffic will come back mean a written chef if, if the back end is on a remote host.

E

uh The the destination ipo for the return traffic is already the the hostel itself and.

C

That will go the host night, one for another one when, but it will not go to the open research.

E

You mean it will first, it will use the no ip not as a right.

G

E

Yeah right really, I think we use the ip load to choose the load right. It will always use the node iv. I I didn't try it, but I I don't know how the look, how the source ip is selected for this situation.

C

Oh people are sure that hong.

D

I think uh I think the song, I think it doesn't matter what the source ip address is, because what whatever it is it is, it will be masqueraded with the anti-gw0. The ip address.

D

The traffic and the traffic from localhost will be rotated will be routed by the ip law and then then it will be routed to actually deliver zero.

H

Yeah holidays agree, so I I think means if the endpoint is is on a host network. So uh the endpoint ip is the node ip.

H

So uh if we do not do the master rates on the request, uh the reply, uh the the reply packed for the ripple effect, uh the packet will be uh forwarded to the uh to the client directory and the source option will not be the service ip.

D

Yes, yes, that's that's what I want to see.

E

D

D

Okay, I think I don't need to sell it again.

E

It may be. My real question is uh how this society is selected when it starts the when, when the host access the cluster ip service, why it's always this node ip, not the getaway, app? I think that that's I want to figure out. Maybe it's determined by the a lot table, or maybe it's.

C

I think that depends on if you, if you solve it, to the to the know that you're not if you don't burn, then the police.

D

I think uh I didn't, I didn't have a test about the butter. I think, if the destination is the service ip address, I think the holster will choose ip address from from the internet, not the entity.0, but I didn't have a tester to.

D

I don't know how, because that he was actually testing.

E

Sure I could we could test it later if without this as neta action, whether the source app will be the gateway ip or the node ip yeah.

H

Okay, yeah. I have some experience on the windows node, so for the windows. uh If we don't buy the source ip for for our request, uh the source ip will be chose to follow uh follower. Rfc there is rfc for the windows; basically it will. uh It will choose source ft uh for, according to the rule, table.

E

Yeah, that's what I mean here, because the root table tells it should go. The entry get awaiting right. Oh get away, though sorry.

H

Right for the windows, we will find the uh the nearest uh the nearest neighbor neighbor of are most match the root table already rule and, uh for example, uh if the, if the that's ip match the default rule deposit, a rule, I think it will choose the node ip and the source id.

E

H

I think linux may be some difference, but I think they must be as similar.

E

E

C

I have one question: uh if we either what is iv table source and uh I'm users do we do we have to stop kobe proxy or even could build this running uh until process still intercepts the the traffic the host to service traffic and the load points on the low project.

C

Yeah, I I mean um once we add this like tables and uh ip's, also rules. um If we don't stop, could we probably say uh what will happen?

C

Where could we probably implement the class ib service traffic or the driver still go to android.

D

Proxy first, um I I first repeat your question. I think you want to question that if we stop for the entry proxy.

C

If we don't stop could be proxy.

D

Okay, what will.

C

D

And it doesn't matter because, uh with you, the insert of we use the insert accent before the kuby proxy item could be proxy epitable items and it won't, it won't be heated could be probably could items.

E

E

Maybe also we need to check where the queue plots are used insert because if you use insert 2, then after restarting itself, it could insert before our loss.

D

C

Probably we don't need to disappear, um but I wonder uh if we start to add more and more habitable zoros.

C

Would that be the you know, uh approach we want to go for the long run, or should we still try to put more forwarding logic into openly switch?

C

For example, is uh s night singer she will try to enhance our prevalent hundreds in open research.

C

Which one we start on, I mean, for example, it's not always routing table thing and you.

D

Mean this? Yes, this one! Yes,.

C

Yes, because we I so we are adding more and more table tours. Let's I'm little, I start to be a little worried. We. Finally, we relax relying on live tables too much.

C

And how could we uh declare, we are openly switch based.

D

I think goes through the battery handle the acid- that's not in the happy tables.

D

I don't have a better idea to how to handle to masquerade in oas.

D

Maybe I don't. I don't answer our question, but.

C

I think in your case, definitely if you want, we can do doing openly switch for snap, but we just need to handle the limitation that you cannot do 19 twice in openly switch. I think, anyway, is looking at some solution for windows. Maybe we should consider together again, I I I didn't release things through uh for that, since I saw more and more different table rules reading I started wondering: should we continue that out or we should do, we should reconsider to put more logic into opening switch.

C

Probably we can decide more plan.

D

I think the traffic data routed to open raceways should be with the with the crdr of called ip address.

D

I I don't know what will happen if we do as last night, the open open.

D

E

I think, ideally, the society should be the uh gateway ip directly right.

C

uh Chain only thing that if you find the socket to the load api, I don't know what.

E

You you mean it readily select a source ip or it always uses a default interface. Ip no.

C

I'm not saying it's very different, I mean if you, for example, if you create a socket and bond it to the ldip directly.

C

I know people typically don't do it, but if they are actually ban live here, I I I think it will be uh using the loadout with the source type.

C

Maybe we can we can, we can do some experimentation.

E

C

And the actually, I really want the idea uh is that.

C

I think we talk about is that I'm still thinking could the tc be simpler to redirect traffic, I mean we don't need to add any results, uh but we if we have a way to redirect traffic directly from the node ip to the to the openly switch interface.

C

uh Yeah, but probably uh I I can also do science parents and we can talk more. I don't really tried anything yet so I'm.

I

Not really sure.

C

It works so long.

I

Engine is a tool support sctp, also.

C

I

Yeah yeah, sorry to have a source should have should support three protocol, tcp udp and the sctp yeah.

C

uh Tc, actually is a low level things like more like layer, 2 level. So, according to recap, on the s3 protocol, let's release all.

C

D

Okay and cruncher.

A

I don't have any question I just wanted to understand. If you have not already mentioned it, whether you already have a poc for this design.

D

D

Exactly so, you could actually call it source code or.

A

Yeah, I mean any sort of source code that validates this design.

D

This this this will be ready for and well we can talk about it.

I

Yep, yes, yes, but we don't have a valid testbed for now, but it should be. Is yeah.

A

Sounds good thanks.

A

So is there any other question on the q proxy removal design.

A

It appears that you know there is a quite a lot of discussions going on here, especially regarding this conversation that we had on this recipe and I'm sure that the conversation will continue, probably on github, if necessary, if needed, we can also bring it up in the next office hours meeting, so I'm sure there will be much full of apps to this design. So thanks a lot for sharing it online and we change and- and if there is no other point on this, I think we can move to the next topic on the agenda.

A

Just wait, uh 10 seconds 15 seconds to verify that we are good on this front.

A

All right, so I think that it's now time for the flow aggregator demo, so I don't know what's going to present, if uh sirica eo of both of you, so please go ahead.

B

Yeah, I I'll uh you will be presenting the demo I'll just introduce uh give a bit of context. uh So, a few months back, we have demod flow exporter, uh which runs uh which connects from every node uh of uh every node in android cluster uh to uh uh external flow collector. So we developed a flow aggregator, which is uh uh which is a which runs as a deployment in the anterior cluster. All the nodes stop to flow aggregator and we do correlation and aggregation uh and then send it to external flow collector.

B

So if young wing and you had worked on the demo I'll uh give the stage to them.

J

Okay, thank you for your car. Oh, I will start first so.

J

A

J

J

Our flow equator is a new feature integrated into our network visibility feature and in the past we only support full exporter feature in nfl that exports connection at agent to an epiphany explode collector explo uh to sending the flow records.

J

However, in this implementation we will have two flow records, one from the source node and another from the destination rule of a sim giving connection.

J

So we have add a new kubernetes service which is called flow aggregator and all the flow exporter as each agent connects to the flow aggregator and send flow records to it, and the flow aggregator will do a combination and fill out. Some missing information also add some status statistics into the final aggregate flow records and then flow aggregator will send the aggregate fpfix records to our external flow collector to a further more fluid of flow visualization using the erk flow collector.

J

This is a brief framework of our workflow aggregator design. For example, there are three uh three or more flow agent. A entry agent for exporter which exports the flow records to the flow aggregator.

J

The first part of the aggregator is called is called a collecting process which is used to collect the flow records come from the flow exporter. Then the intermediate process will combine these flu records, fill out. Some missing, coordinate, infos and also add some statistic: calculation into the aggregate flow records and then using the exporting process to send all the aggregate flow records to the external epifix flow collector.

J

This is the supported functionalities in our flow aggregator, which most important one is absolutely aggregate for records by the factory for flow key, which are source ip source, port destination, ip destination port and also the protocol.

J

Also, we do some status calculation and added into the aggregate flow records also have as the single stack fpv6 support in both flow exporter and aggregator, and also all the connection between the flow exporter and aggregator is protected by the trs security.

J

Also, some new diagrams and dashboards are added into kibana, which will be shown in the demo. Soon, uh nice yeah you, you will give a uh show of the demo setup and also the configuration file.

G

Yeah sure, thanks you mean.

G

G

um Can you guys see my screen.

F

J

G

Yeah um now I will demo the flow aggregator and its visualization here is a setup for the demo. In the configuration file, we enabled flow exporter in entry agent and specify the external flow collector in the flow aggregator tls in the flow aggregator and in the entry agent and for aggregator are enabled by default, and we we use iperf to simulate the port to port and portal service traffic, as shown in this diagram. This is our demo setup for the ipv4 cluster.

G

We have five nodes and several ports on each node. There are seven servers, um import um font and, for example, for server 1-1. There are 5 clients talking to it, client, 1-1, 1-2 and 1-3, 1-4 and 1-5 to send traffic to it. So we will cover both inter and intranode traffic. Here.

A

uh Yeah sorry, quick question regarding it's a very stupid question of me: client, two one. There are three of them. Are there three pods on server two or they are running on the nodes, because.

G

A

Are they are outside of the diagram? So I don't understand where.

G

They are yeah. There is some problems because there is no, I didn't assign any node affinity to it, so it will randomly assign to any one of the five nodes. Okay, so that's.

A

G

Yeah, thank you and um the server is actually a service to expose the iperf server port. So later in the visualization, you will see server 1-1 in both port to port and port to service traffic.

G

I also applied three ingress network policies and three egress egress network policies um to the client server ending with one so server dash, 1-1, 2-1 3-1 will have ingress and client 1-1. 2-1 and 3-1 will have the e-grass network policies and we have a simplified ipv6 setup. I will show you later yeah. Let's move to the.

G

Dashboard, so this is the dashboard.

G

Compared to the last demo, we add the nodes throughput and the network policy dashboard and also add some graphs in the port to power flow and power to service flow. So let's go to the port to power flow.

G

So before we have the sankey diagram to add some line graphs to show how to power flows, for example, I can filter the flow by the destination full name. If I want to use the server 3-1 and it will show the traffic that have server 3-1 as a destination port and it's worthwhile to mention that in the past we cannot resolve the remote destination information, unlike destination, porn, name and namespace, but with integration with flow aggregator, the destination information can be filled.

G

So, for example, for this web server 1-1 there are five um client sending traffic both internal and internal. We all have the destination information.

G

Additionally, we add this port to port flow key to this onto our visualization. It's like a semantic version of five tuple, which is composed of the source, port name and port destination, port name and port and the protocol.

G

So there are four graphs we newly added. They are grouped by this port to port flow key. So the last two graphs show the throughput and reverse throughput of each flow, and the right part is the cumulative bandwidth for this to port flow. For example. If I say this, let me choose one of this orange flow. You can see that it's very stable has a very stable throughput and the corresponding cumulative bandwidth will show a smooth growth.

G

Similarly, to this port to port flow, we also have the portal service flow here.

G

The only difference is that the destination we, instead of using the destination port and the port, we use the destination service name and the service port. So the flow key here we also use for destination. We also use the destination service and service port.

G

The graph here is similar to the portal port. One.

G

We have the throughput, reverse throughput and accumulated bandwidth here.

G

Let's move to the note2 pro dashboard.

G

In the note, 2 super dashboard first, the flow diagrams um are showing the showings in a straightforward way, the traffic between the nodes or internal, and it also has a line line graph to show the aggregated node throughput. Over time.

G

um We introduced uh the heat map here to visualize the aggregated, total ingress and the egress throughput, which is helpful for user to identify like which node will take a line share of the bandwidth, as shown here on this node anyways. Two three zero share, a line um majority of the traffic and the rest of the the rest of the notes are basically having the similar traffic here, as you can see from the darkness of the color.

G

The next feature we have this network policy.

G

The next network policy in this sankey diagram you can see there are different colors of this flow. The the flows are grouped by network policy here so for these yellow ones you can. If you hover over, it will say that the ingress policy is mp1-1 and all of them are mp1-1, and this brown one will will be the 1-2 instead, so they are color-coded and I can filter the ingress and egress network policy by its name and namespace. If I only want to know the 1-1 network policy, it will fill out.

G

So, as I mentioned, the mp1h1 is applied to the server 1-1. So you can see all the traffic file traffic to the 1-1 are filtered out here.

G

The pine chart here shows the distribution of the traffic group by the network policy, and this is the namespace.

G

Yeah, that is basically a network policy. um What to what we need to mention here that we only support network policy that will allow the traffic um the dropped and denied connections are planned to be realized with pacting in the future that is ipv4 cluster. We currently also support the ipv6 single stack.

G

The dashboards are similar to ipv4, but with some ipv6 fields like in this overview.

G

Previously we have the ipv4 address now for ipv6, we have the ipv6 address and in the flow records we also show some ipv6 information here, yeah. That is basically what we want to demo for this features.

A

Many things that was a very nice demo, it was uh very entertaining to see only one question. You mentioned that it's a single stack ipv6 is this because dual stock will come later or is there any specific challenge for supporting dual stock.

J

Yeah thanks for the question yeah. Currently we found that uh in the dual stack cluster, the flow exporter is failing to retrieve the service community service information. So we are still debugging on this, uh so we market it as uh working on items right now,.

A

All right, thank you.

J

G

You any other questions.

F

um A quick question on this: um I know you have mentioned that you know the deny action. Support has not been yet realized for for the full aggregator um so for for the uh network policy that actually has allowed rules. um Does this work right now for both the kubernetes network policies and the entry native policies already.

G

I think so yeah.

F

Oh cool all right.

B

B

And I just want to add that we plan to enhance the network policy information with the rule name and which, with uh with granular information about network policy, currently uh we can we can. We are planning to enhance that part.

B

F

Yeah, that's gonna, be really nice um yeah, and also I wanted to try to understand like since, if we wanted to support some sort of like visibility for for the denial rules, um how would those like actually show up on on these? Thank you diagrams, because for the deny traffic there won't be any sort of like load generated right so like it would be. Would it be like a like a very, very thin line in terms of you know how how the traffic looks or are we thinking something else here.

G

I haven't think about that yet, but maybe we can show a graph that only had the deny rules.

F

Okay, it makes makes perfect sense.

G

Yes, it's just.

F

uh Just a question that I wanted to throw out: it's.

G

Hard to cooperate in this graph yeah.

F

Yeah, it makes sense.

A

A

All right, I believe that it's all for this demo, so many thanks to you and yumming for presenting it that have been uh extremely interesting and we are now in uh open, disc, open discussion and.

A

Anything else that uh we would like to bring for open.

A

A

Any other topic you like to bring.

A

A

All right, it appears then, that uh we are at the end of the meeting. I would like to thank you. Thank you again, today's contributor contributors, and I think we can stop the recording here and we'll meet again in two weeks time. I just want to also give you a reminder that we now on tuesdays at 2 p.m. Pacific time we have a gentria open office, sorry office hours, meeting where you can attend to either ask or answer any sort of questions that community members might have about andrea.

A

So thanks everyone for attending and have a good one.