Antrea Community, 8 May 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Antrea Community Meeting 05/08/2023

Description

Antrea Community Meeting, May 8th 2023

A

Good morning, good afternoon, good evening, thanks for joining the systems of gentria community meeting uh today is a Tuesday May, the 9th and uh the agenda. For today we have a couple of topics, um an up, a general update on the status of the 1.12 release, and these will be led by China.

A

And then we have a a conversation about a change for Windows in kubernetes 1.26, where user space is being reusable, support is being removed from Q proxy and therefore the entry proxy will take over responsibilities, such as all the responsibilities which are currently held by the cube proxy and I. Think that perhaps winning will take care of discussing this. That that is that correct.

B

A

Perfect, okay, so do we have any other topic for today? Any announcement you would like to make before we start with this agenda foreign.

A

Please go ahead.

C

uh Wayne, could you start faster as asking you, let's, uh let's introduce the progress of and sell Pro sale on Windows and then I could have the status I'm still preparing some list of the status update.

A

Sounds good to me.

D

B

D

B

D

um Thank you Val. um Actually, this topic is about so youth entry proxy, all on Windows to replace comparency user space mode. um The motivation includes that the major reason is that kubernetes has dropped the QPS user space mode on Windows since version 1.26.

D

So that means that we cannot use qparency user space and mode in future, and so we have to switch to ensure entry procl. That is because there's all the occupancy has another mode in kernel modes on Windows, but the ensure cannot work with with that modes. The reason it says and chair needs to use open waste wage extension on the HS Network, which was used for running um pause but Focus wind kernel modes. It requires a webp extension. It is different from OS extension and for the windows interests Network.

D

It requires that only one extension can be enabled on the single Network. So that means we have to switch to control CL and if we want to enable Android, Pro CL on the Windows nodes, the key configurations include that so visually enable entry process, visual Gates and we need to set policy off like as true effects. Entry proxy is already enabled on Windows for many releases, but for entry proxy proxy all enablement.

D

It is a new comparison if we want to to enable the process of feature and a second change in the configuration is that we have to set the cube or APS server or write with the address of the qbps server. Here we cannot use the cluster IP service of the kubernetes service. We must ensure that ensure agents can use this address to access the API server directly.

D

Vip address for the API server and the file if we want to enable Android process, one normally so I want to highlight in this community meeting is that we used to hit a Windows. We thought easily in unsure version, 1.10.0 release after process enabled um for for the stack current status of the iso.

D

It is fixed now, but the the uh I want to highlight it here is because I want to introduce some observations and articles, and the page is about it, so that if anybody in in the future hit the same issue, maybe it can switch the release versions to to a newer new release um for the install. The observations include that we absorb results happen on the Windows node, randomly with entry agent restart case.

D

uh In fact it may help it may happen one way restart and share agent, and we see that entry agents is started running in the story starts case, and we also observe that. So when we capture package on the server side on the kubernetes service or ensure service, we saw that some TC TCP acknowledgment packets are not captured and on the server side here, the TCP ack packets are not for the handshake stage, but so for the payload sck and a second.

D

A third observation is that so we found that several service related Rod entries are lost on the Windows host. After some troubleshooting, we found that the root cause related is on the normally so that so the windows, Service rules, May Pro, may be broken on the nodes, and here I paste the issue.

D

There is a link, the the request that privacy will have two routines to process the logging entries and once used to add the new service related routes and another is used to delete the steel Roots because the the turbo routines run in parallel, and so in some cases the new added results may be removed by mistake, but by the the other routines. So in that case, the service related results and missing.

D

Then the service traffic sent from Android agent to kubernetes service of children, Insurance Service, maybe not able to to find the routing entries and maybe not forward it correctly, and by now we have two patches to fix the the easel and the first one for the 4470 is already merged in the uh unsure release of 1.11, and it is also backplated to the previous versions of privacy releases from the 1.10 and the second one is recently fixed attention and by now it is also merged in merge in the main branch and the back potted patches are submitted and ready for review, um then about the next stop a step to to enable process.

D

All on Windows includes this. Firstly, we need to enable process out in Windows manifest, and the second is that, so we need to run tests with process of like Flex in the CI. That is the member contents and any questions.

A

No question from me, then: any question from the rest of the team.

B

May I know why this issue will cause windows bsld. It sounds.

E

B

It should not cause any yeah. Yes,.

D

B

I mean this kind of one configuration yeah yeah.

C

Sometimes I could answer this question. We we had a salary, but we haven't validated. It must be that great but like let me share our investigation. First uh on Windows in release, 1.10.0.

C

uh When Edwin mentioned, there was a an issue that some loose may be deleted by mistake and, however, today we also found that the issue, or was also reproducible in 1.11.1, which already takes the routine deleting problem, uh uh but but that that's without the latest patch I made last week. So and after after that, patch it's merged. The network was not repressed used so far, so perhaps it's uh related to the traffic that is being redirected to open with switch but was not handed because anthroprocy hasn't installed.

C

The open flows for the service IPS on Windows, the OS user space runs no no didn't run, as doesn't run as part of the Android port. It's a standalone process, so we thought that U.N will restart entry agent. The service flows should be there and should be installed by the last round, should still be there and should be able to continue processing the traffic to the service.

C

However, after some tests we found that after Android agent starts up and the service flows are gone and I found that it's because the service group we have a service group coming up on Android agent, startup and that operation cleans up all Service Groups open, Flow groups in open with switch, and you know the service entry proxy depends on the service group and even the open flows referring to that groups will be directed deleted. So it didn't work as we expect every time Android agent starts up and there is a Time window.

C

The service, the the traffic to the service IP, will not be processed correctly and will be treated as normal traffic towards external IPS, because they are not poor IPS and they are not node IPS. So we we treat them as an external IPS and perhaps the final, the the final action of that those traffic will be again redirected to and took it away, interface from open research.

C

In this situation, there may be a circle that, in the hosted network name space, we install loots to redirect traffic towards the service IP to open with switch, but in open with switch before answer plus it's ready. We redirect the traffic back to the host Network namespace and we don't know whether this cause problems on Linux yet, but on Windows. This may cause the some very high CPU usage and the manual usage and eventually leads to system crash.

C

That's our server, but we haven't plugged it yet, and maybe we we should also check our Linux houses, will Behavior and the the key of the latest patch is that we only generate the traffic towards the service IP after answer plus is ready. This guarantees that uh if the anti-agented uh generates a package to service IP, it will be redirected to open with switch and the new open switch. It will always be processed correctly and netted to one of the endpoint.

B

I got the explanation thanks a lot.

C

And perhaps this may indicate some other potential issues, because we in the latest release, 1.11 and all release after that we use a single bigger cluster, IP, say IDR to as the destination IP of the loot for service IPS, which means that, for example, if the services IDR we got, we calculated is 10.96 dota 0.0 and if the, if the agent or any port tries to access one service IP uh one one IP in that block. But it is not a real service. Ip. The package may be Circle.

C

We maybe maybe looked in a circle between host Network and the open world switch still yeah, and uh we need to verify what uh what's the account. What's the consequence and if it could lead to another system. Panic.

C

And uh that's one General issue: we need to further investigate under one. Not another potential issue is that previously we thought that uh uh the open waste the open flows will be kept, will will be kept on and their agent start, but it seems that it will be removed. So if some traffic needs the the flows installed by Android proxy, we need to ensure the all of them must be run after enter browsing.

C

That applies to linear size. Well,.

C

Okay, I think that's of the the the part of this problem, any other questions.

A

I think your answer was very exhaustive, but uh let's see if there is any other question.

A

Three two and the time's up. Okay, so we know uh what's the status here and what we need to do and uh I have to say it's probably not the feature that we were looking to develop about. uh It doesn't seem that we have any any choice here. We have to do it all right, so maybe we can move to the general update on the 1.12 release, Channel.

C

Sure, let me show my screen thanks. Okay, currently, uh we still have 20 PRS in progress and I made some list based on the uh areas and, let's take a look at each area.

C

The first is two PRS about anthropology.

C

The first one introduced support for external IP support in Android, plus I. Think this is the only missing part from functions perspective or currently answer.

C

Has we I think we just missed this handling this field and it's not by intention and uh and follow the problem by running some kubernetes config Network tests, the the each service could have a one one or more external IPS, in defined in its spec and in Q crossy, they also or proxy, and these IPS, and by this patch 12 plus it will do the same and I have reviewed this patch I think there's no risk for this one. It's a simple change uh and another is a behavioral change in anthropology.

C

Previously we, if external IP, is if, if the learn, bands or not report, says external traffic policy is local, we will always. uh Let me see yeah we were always uh sorry. It is cluster.

C

uh Remind me was the previous behavior and there was the latest Behavior we we want to change. I didn't quite remember the detail.

E

S not in the core.

C

Okay, then, let me simply describe the the change is about how we handle the traffic from Port and from external uh I. Think this patch changes the behavior to align with the cube losses that it always learn and balance the traffic from local port to external IP or node port to the to the whole cluster.

C

Instead of uh respecting the the external traffic of the service, Eastern traffic policy of the policy, the the reasoning behind it is if there is no short circuit circuiting, uh the the packets will reach the external loan advance and we are eventually be distributed to endpoints across the whole cluster. So when Andrea policy does proxy the external IPS, you should do the same instead of only browsing to local endpoints, I think and it's the current Q process behavior and is defined in Signet Network test and the second major error is narrow policy.

C

We had a fourth Futures Improvement in this era and the first I think is to support IP based group, Association, quality, API and uh unfortunately, I I didn't get enough time to review this part. So far, and it's um on my list- and perhaps I will do it in today- yeah and uh any reviews from the other reviews and maintenance are welcome and currently I think we we, the dispatches, are lockable reviews and the second is for user requested requested feature: adding low label log label to the.

A

C

C

And so do the so. The part I think this is to um keep in line with some networking firewall products. That uh user could also match the source ports in net policy API- and this is the last one- is to implement the um propose, the future by Upstream Network policy, with two apis I think and according to Yang I think the fastest race should be Pro prioritized and the last one may need more more review effort, because it's a big change.

C

And um the third area is the multi-cluster, as I saw that most of the patches are upload, and perhaps it's just a while taking effect before the remain for for these tools and and I I guess the first one is a test for the last one and.

C

Land and do do what do you think about the second patch? Do you think it's in good stage, I.

E

Think uh changing left a few comments. I have adjusted them, I think not a big challenge, so I suppose it should be no risk.

C

Okay, thank you, and the first area is about cutting up some text that and some for some minor improvements in Wireless area. The first is to switch to packaging two, to make the code easier to maintain, maintain and to be more extensive reviews.

C

um The second I think it's uh it's Improvement in windows, clean up strip to fix some uh some issues. Awkward in some scenario, I guess.

C

And I think this one yeah this doesn't introduce new features but to make the code more robust in some scenario that we didn't need even need the if some API service and web hooks and the last one is a big Improvement in the memory usage of financial agent. It removes some unnecessary cash for anti-agent and I. Think the Implement is about uh 100 megabits and more megabits memory reduction reduction with this number of open Flow entries.

C

But it's a big change, so we should review it carefully.

C

Yeah and Nest Android Error is Windows uh I, just confirmed that this is a preparation for containerizing OS in Windows, but not planned for this release.

C

So I think merchandise in this release is a best effort and that didn't affect the release. Note who you any user features- and we have just mentioned this this one previously.

C

And Nest is about test. We have three PRS for newly tested for these packages again and um I guess they have been differed several times from, for example, this one it has been nearly half a year. So maybe let's try to include them in this release and I see this one has already been upload, so no, there should be no risk for this one and the last one is a documentation Improvement for using Android with Rancho, Crafter, I I, guess no real risk for this one.

C

B

C

Yeah, it's a small change. Okay, that's the current status of we will enter release, one daughter, cow, yeah, I, I, guess we'll. uh We we need to improve the efficiency of reviewing under spend more time on it. I will also spend more time in the following two weeks. Currently we have two days or two weeks and one day left for for the release.

C

Any questions about the release.

A

No, that your description was uh very, very clear. Let me see, let's see if anybody else from the team has any question, especially if you ever any PR in Flight. That was not a list here so that you believe should be included to release plan.

A

Okay, it seems that then, the discount description of the status of the 1.12 release is very clear for everyone, which means that we can move to open discussion. If anybody else has a any topic that you would like to propose, or they would like to bring up a poor discussion with the team.

A

Our way to a few seconds in case anybody has a proposal.

A

Three two one zero, and so it might be all for today. So it seems that uh this might be the end of today's meeting and uh I would like to thank uh wening and China for bringing up the topics for discussion today, and uh we will meet again in two weeks time. So thanks a lot for joining and I wish everybody a great evening or a good afternoon.

B

A

You thank you bye.