Antrea Community, 31 Aug 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Antrea Community Meeting 08/31/2020

Description

Antrea Community Meeting, August 31st 2020

A

For today uh we don't have as many topics as recently on the agenda. However, we, uh the topics that were proposed on the community channel, are a conversation about uh entry or proxy uh default support in entry 010. Sorry, I expressed myself in a terrible way.

A

I mean to see, make entry a proxy the default option in uh entry 010 and then the other topic that has been proposed is a backward compatibility for cmp amp apis, so um recording has started so perhaps we can start the conversation on uh andrea, proxy and weather should be made default.

A

uh I believe that uh wagehang performed some experiments related to the andrea proxy resource usage. So perhaps we change you can start sharing your experience.

B

uh Yeah sure I I didn't get the schedule for today this meeting, but I used tests for the american consumption of the entrepreneurs, with entrepreneur proxy enabled and with with services the angel. The answer agent cost about uh 50 megabytes yeah. So I think the memory consumption is reasonable.

A

Okay, and one question is: is the increase linear with the number of service, or did you observe a different relationship.

B

And as average service memory, consumption increased very little of about one or two kilobytes.

C

Okay, when you say the memory cost is 50 megabytes for 2000 services. Is that total, or is that in uh as compared to q, proxy.

B

uh Compares to q proxy yeah, the increase part of the android agent yeah.

C

Okay, so it's really you're really comparing uh entry proxy, disabled and empire proxy enabled, and you observe that entry agent is using an extra 50 megabytes when you enable.

A

It okay and this extra memory- consumption anyway, is static right, it's, so it does not increase over time. Unless, if you keep the number of services, constant memory usage doesn't increase over time or doesn't depend on the traffic being handled by the proxy. It remains constant. Is that correct.

B

A

C

And how does that compare to the uh memory consumption of the agent without entry? A proxy? Is that, like?

C

Is that, like a big increase, or is that.

B

Is always not the actual prophecy enabled? The memory is constant?

B

A

uh I think controlling questions was like uh you. You told us that it's uh like 50 megabytes more but in percentages like 10 20, more of the cube proxy, or did you do? Did you calculate this.

C

I meant compared to the agent when you don't have entry, a proxy enabled like easy, and I I understand that it depends on the size of the cluster and probably on other things. But he's like in your benchmarks was the agent using like 200 megabytes, and then you enabled qpr. You enabled an entry a proxy and it was like 250 megabytes, or is that very different from that.

C

Well, I guess my point is 50 megabytes. The number you gave is like an absolute increase in memory usage. um But relatively, is it like a big increase, or is that, like a 20 increase.

B

The percentage- and uh I run this this test in my uh vacuum- test value without any other resources like policies or whatever, so um the flash uh agent costs about uh five five mega miles like uh I remember.

D

B

Okay, so increase power is totally proxy cash.

A

Okay, thank you very much uh for this update.

A

Is there considering this information is? Is there any feedback uh regarding whether it is okay to enable entry proxy by defaulting zero time, or perhaps you think you we may need to collect more information before making a decision.

B

So now I think you know we are absolutely. We need the android proxy document which I'm working on and any other things. I guess books will have some their opinions.

E

I I personally feel probably we need more scale tests and probably the dive tests are not enough, and I know the webmail qe team are also doing some scale tests for a week. We can wait for the test results.

C

Yeah, I'm hoping we can enable this by default at some point, because I mean right now, we that's the only thing we support for uh non-encap uh traffic modes right. So it's a bit weird that we, this is the only thing we support for non-encap traffic modes, but for end cap. It's still like something you have to enable.

A

Yeah, that's uh that's a good point. Okay, so the feedback is to uh wait for, uh let's say more uh precisely, more accurate measurements at scale to get a better understanding of performance and memory footprint skill, but our goal is still to enable it for the reasons that have been listed by antonin.

A

Is there anything else that you like to bring up about the entry proxy.

A

Okay, uh which means we can probably move uh to the next topic about uh uh backward compatibility of cmp amp apis, and I think that at least according to the zlak channel happycheck has a couple of proposals to discuss so abhishek. Can you can you go ahead? Please.

D

Yeah thanks so much um so as you guys, as everyone knows that uh in the previous release, we released tears in the form of static uh form like this direct tears, uh which essentially mean that you know there are no resources created for them.

D

uh It's uh it's just a string as part of the enum in the cluster network policy and anti-network policy uh spec, and at the time we also made it clear in the documents that we will remove this and supersede this stat these static tiers by tier crds, uh so that users have a flexibility of creating uh as many tears as they want.

D

uh You know, instead of those five static, tiers that we've introduced and and and if and you all know, that this is a alpha uh release which is uh behind a feature gate and the default is false.

D

So as as as we move towards the tier crds, you know the there was a question that was raised uh last week about how do we manage the the upgrade, especially that the five static tiers? Do we consider that andrea at initialization uh creates five year crds corresponding to those static, tiers.

D

Or do we you know considering? This is an alpha feature. Maybe we do not carry forward that you know logic in andrea, keep it clean and, uh and then you know just starts starting you with with the trc ids. uh So essentially there are, you know two proposals. One is uh that uh considering this is a feature-gated sorry, this is gated behind a feature feature and it's an alpha feature. We we decide not to create these uh tiers by default.

D

We only create the uh the the default tier, which is the application here for all the policies which do not have a tier in their uh spec or the other. One is that we do manage this uh upgrade by at this. At startup we create one tier corresponding to each of those five static tiers, so five crds uh or five cr custom resources on startup.

D

Now the logic essentially is uh you know not very different from creating one default year versus five five tiers. uh It is just that. Do we want to handle uh this upgrade, and should we also um carry this code and how long shall we carry this and then you're kind of forcing users to uh users the five tiers which they may or may not be using? So that's that's the question to the to the team or the community.

D

So anyone has any opinions on that. I know. Cody has one.

D

Have I made the the problem clear and uh you know in general my opinion is that uh for any you know ga release or you know, any release which is uh widely used or feature which is widely used. We should make sure that the upgrades are seamless and without any incompatibility changes are introduced.

D

But considering this you know it's just one release and it was a you know. It was alpha feature plus no. Unless we have like users who are actively using the tiers uh static tools. uh Maybe then we do consider that. But if that's not the case, then maybe perhaps we can start cleaning with the tlc ids.

D

So any opinions.

F

Hello abhishek.

F

Do we consider that that we continue keeping this static, tl uh constants, as as a hard coded tails and not non-managed by users? I know there is a similar case for kubernetes priority class that you can use some api to define user defined priority class, but you can also specify two special priority class. One is a system node, critical and another is a system cluster critical and they are mapped to two to priority uh consistent consistent.

F

So I wonder if we could do the same- and I I remember that in the beginning, we don't want to use it to delete some details right. So I think this could solve that problem too.

D

uh Yes, so uh so we are already we are. We are going to be creating one uh tier at startup, uh which is the default year, for let's say the application here, and we will not allow users to delete that the validation hook will ensure that any system created tiers any updates or any deletes to them will be. You know controlled by that validation. Who can it will be rejected?

D

So we definitely will do that for one year.

D

The one of the proposals that I had was that you know if we want to continue with these static tiers, you know and seamlessly add them into scrds, then maybe we instead of creating one system here we create these five system, ps corresponding to each of those strategies and then.

G

D

Add the logic in the validation web book uh to ensure that users do not delete them.

D

That can be done, and you know doing it, for one tier versus five tiers is is not is, is the same effort, so it's not in terms of effort, it's not any any different, so I just want to know whether do we want to create five tiers or do we want to handle five system tiers or lesser number of them, or one is enough.

F

I mean not creating the ids for this file tiers, but just hardcode them in encode, like the priority cluster name system, not critical.

D

So if I, if I do a get on these, uh you know as a user, if I want to see what are existing tiers, then if these are hard coded, then I don't as a user I do not know about that, is that is that correct, or for you.

F

Yeah yeah, that will be that need to be some well-known uh tails that we need to document that, if you just want to use this tales, you could just reference them in your policies. If you want more, you can define your own that that divides the tiers into two categories: user defined and the system defined.

H

The challenge is going to be the overlapping priorities. How do you set a priority on a static tier? If you also have user-definable tears that you may want to fall in between the static tears.

I

Yes, uh I think do do we still have the uh constraint on the 10 overall tiers that we're trying to we're trying to constrain that.

D

uh So you know, I think, from cody's past experiences he mentioned that you know the the general usage of tiers uh is about five to seven and at max maybe around ten. uh You know correct me if I'm wrong, but.

H

D

Yeah uh on the priorities we are, you know trying to keep. Maybe you know up to 255 priorities, uh but maybe we start small. We start with 10 and then increase the number of tiers. As you know, more tiers are expected by users or requested by users, uh but coming back to cody's point, I think uh you know having the the tiers in the crd format gives you, you know. If you do a get tiers, you you see all of them and you see all of the priorities that have been previously allocated.

D

uh Instead of going through the documents and going through, you know what are the allocated prior or what are the reserve priorities and which priorities I can use. uh So that is one advantage that I see about. You know having them as uh tears, also having them as custom resources uh instead of a hard-coded principle.

H

Another option is, we can always create those tiers and our user can decide to delete them if they wanted.

D

H

Another option is, we could always create the five static tiers using you know as a as a default mechanism using the crds, but a user could decide to delete a tier if it wasn't needed so out of the box.

H

You would always start with the five tiers.

I

um Cody, I think your voice is breaking.

D

H

Sure what's happening here, maybe my internet's messed up.

D

If I heard you correctly, I think you you're saying that we allow oh sorry, we manage, or we create these five tiers, but allow users to delete them. Instead of you know not, instead of making them read only correct.

H

C

Yeah they could delete them, I guess in theory they could also change the value mapping for those tiers.

H

Exactly that would also give flexibility for priority, or something like that.

C

And uh yeah- and I guess if we look at chan's analogy right with a priority class uh even for the system, cluster critical and system note critical. It seems that they do create api resources for those. uh If I'm not mistaken, um I don't know if the user can delete them, but they do create those uh those api resources just like we would create crds for those static, tiers.

C

I guess I wouldn't see the rationale of creating like a single one versus creating all five of them. I don't know why you were considering the the single one solution in the first place, the single one.

D

Is uh you know mainly for the default consumption uh wherein like if you had previous cluster network policies, uh wherein you know before the existence of tears? uh So they don't have any reference to it here, so they they kind of fall into a single bracket or a single tier, and uh you know you do want to show that there exists one tier at least and your all. Your existing policies have been. You know classified as part of that tier and they're all at the bottom of the of the ordering priority.

D

That's the only reason why I would instantiate.

I

D

Of one one tier.

I

Yeah another special thing about that here is that for any of the cmp or emp that doesn't specify a tier in the in the spec yaml, it will go to the default here.

C

Yeah, I see- and you said there was still a limit on the number of tiers and is it pretty small or is it pretty large.

D

So at the moment it's part of the validation schema. We can always update that uh the upper we, you know you know as as we map tiers to you, know, obs tables. uh There are only x number of tables, which is a small number. uh I think 250 to 55 something.

G

D

um So we we don't expect the tiers to be used. You know that many uh tears to be created. uh It would be a small number.

C

D

C

The five static tiers by default- uh sorry, if we auto automatically create those five tiers by default. Does that mean that we force the pipeline to be five tables longer um unless the user deletes the crds and we saw there were like some performance issues with that? No, I don't.

I

Know, no, I think we as of now the power resigner can handle more than a tier in the table, so we're we're. Basically thinking of you know for the ingress in the in the default uh in one of the cmp tables we handle the default tiers and all the other rules in all the other tiers can go into a single table and we already have a pr for that. So that shouldn't be the issue. Yeah.

D

Yeah, the reason why we have the single table for the default here is because we expect most people not to be using them, so they most of the policies do end up in that one here.

C

Okay, well, I kind of feel like yeah, maybe creating those five tiers is maybe both convenient and uh maybe the right thing to do and that's my personal opinion- and uh I don't know if you want to say okay, you can delete all five tiers or you can just delete those four tiers and you cannot delete the default here and going back to chang's analogy once more. I just checked and you cannot delete like those two priority system. Cluster critical and system- note critical so that you can visualize them.

C

But I guess I don't know if there is like a mission controller or something, but you cannot delete them when you try, you get a an error, so I don't know if you want to do the same thing for the default tier.

C

That's another.

H

Another advantage of going with with the five tiers out of the box is, I think, as patterns evolve in the community on common network patterns for for setups. Very much in the same way like helm, charts would right for deployments. uh Common security patterns could follow those tiers.

H

um Hopefully you know, hopefully they would be compatible across more more systems if we, if all systems started with the same set of default chairs,.

C

That's also tricky right. That means we may have to adapt our static, tiers.

H

Maybe maybe um just hoping that we may have some convention or standard.

D

There, okay, so.

F

I'm not able to update or delete the uh default priority class.

C

Yeah I checked as well yeah I get an error when I try to delete them, but I can. I can visualize them. So I guess it's like in our case. That would be the same as creating those crds and I guess we should let the user delete them, except maybe for the for the default. One.

D

All right, I think, that's fair enough- just want to make sure that uh you know everyone's on board with with this approach or one or the other approach so seems like uh most people on the call prefer to have one-to-one mapping to the static tier to the tcrds, and then perhaps we can make sure that only the default here is something that is read. Only the other tiers can be deleted by users.

C

Yeah- and I think it's kind of like mapping to what cody said, I think it's good from like a documentation and a user experience. Point of view, because we can, we can describe security models built for those like five tiers and kind of. Like assumes, there's going to be they're going to be there in like a standard installation of entry.

D

Okay, I think that answers my question, um which is good. I guess then we can also claim that we are not going to be backwards incompatible, so.

C

Well, that assumes that it's not much more work for you, but I assume, since you are preparing to create the default here anyway, creating those five tiers and just making sure that the default one cannot be deleted is not too much of an issue.

D

Yeah so yeah that is already part of the pr it's just now, adding more more uh tiers to be created, but I'll just make sure that they are not read only so. It should not be too much work.

E

Subscribe so before change, do we already have a default here on.

D

uh So the default here is, I was going to map the application here to the default here, which would be the lowest priority here.

E

So um then the name is the fixed or assuming the name can be changed.

D

The name can be uh changed. uh We we will handle internally, but currently uh I mean, with the current static tiers release, the application tier is the name that is already part of it. So um so we you know.

G

D

If we have a mapping to the trcrd, then I was thinking of using the same names. uh But but if we have a different name, then we we explicitly call that out that the static tier name was uh renamed to you, know xyz as a tiercrd, so.

E

So basically I mean, after upgrade um we will have four sorry, five pre-credited tiers and uh giving t repeated default here for later. You can remove other tiers and you can rename mapkins here.

D

Renaming would not I mean no, no, you cannot rename the cid it's like any other resource, like it acts as a key right. So um the.

E

Place they were, we.

D

E

This app killing here right.

D

Yeah, so I was thinking, maybe as part of the this, you know upgrade process. We we decide that you know. Maybe application here is not the right name for the default to be used, and we want to rename it to something else. Then we create a one, the tier corresponding to application tier, and we we name it. uh We don't name it application, we name it something else, and we call that out in the documents, but but then, once that name is chosen, we keep that name.

H

Is there, is there an issue with naming it default here.

D

No, no such a technical issue as.

D

D

Ginger, did you have a particular name in mind.

E

Name is, uh I was thinking if it's default here, maybe you can just call it default item for that. I think the thing that since you have other tears, I forgot the names, I'm not sure they are. I mean I list this other names, much default, yeah or not.

E

I don't. Maybe it's not an important thing.

D

I mean no, I guess it's, you know it's important, because it's user fixing so I mean, but that we can. We can decide that on the pr.

E

I

um I also have a another question going off cody's point. uh I think uh one of the initial you know concerns cody had was sort of like if we create all those five static tiers and when user wanted to create custom tiers, um they they need to be able to sort of like insert into two or so uh originally created tiers in terms of priority. So is that uh does that mean that we probably want to create all these five uh initial tiers with priorities?

I

That's you know spaced out, for example, 10 20, 30 or something so it is always possible for user to you know, using get crd to query the actual power, dissolve those tiers and be able to always insert a new tier um in between those yeah. I I was.

D

uh You know if, if that was the plan, I my my proposal would be to space them out from zero to 250. Some, you know between between those digits.

I

Okay, thanks for clarifying.

C

uh What was it going to be like a float like for priority, or was it going to be an integer? This would be.

D

And I mean integer, I mean the priority range would be 0 to 255 and we will reserve- maybe some latter, not half a latter. You know last few uh priorities for future use and maybe first few for future use so that you.

G

Know if, at all, we.

D

Have some some extensions that we do in the future? We have some priorities at both ends. Result sounds good.

E

So I was still thinking uh the reason we want to pre-create the five tiers just because we want to keep the comparability of course versions right. um But if we rename the the tier then still break the capability.

D

We can, you know internally handle the name, you know. Let's say application is now default, so any any cluster network policy which was mapped to an application here. uh We we convert them into a default. Here I mean we can handle some of those uh changes.

E

Understood by the from user perspective, it's still a change right.

D

Yes- and there is a description field for the trcrds that I've added- maybe we can add some additional information on that, but but yeah I mean it's, you know as long as we document that maybe the user doesn't have to take any action on that. You know it's uh from from users perspective. They don't really need to take an action, except that know that there's a change of names uh in the other case where we do not create these, then the user needs to.

D

G

D

Create new crds and map the the policies into those new cids. That would be a new action on usb.

E

C

But, uh based on the conversation with that, I wouldn't say that backward compatibility is the only reason here I mean that's a plus, but I think that was the the point that uh cody brought about like kind of like uniformizing the security model, and I think it's just more convenient also for users to have those pre-existing tiers.

E

That is true, um but if we go that model, then it's underneath the switch. We keep the default tier special from others. It means you, I don't know the reason. It's called.

H

The reason it's called default tiers. We have to have a tier right for any policy to land in at least at least one, and so we could. We could document that that's not capable of being deleted. I think that that's a, I think, that's a reasonable balance to strike.

C

And I do think also that I agree that in that in, if you look at it, this way, in that sense, I mean default here, makes more sense than applications here, maybe as a.

E

Name sure, um but if you want to have five frequent tears, then maybe I'll be in here sounds.

H

Better by the way, um if don't all, of the resources have some type of a grid tracking them anyway, so we shouldn't be opposed necessarily of somebody renaming it. Maybe the the default you're renaming it to another tier.

H

The problem is, I guess, with the binding right, so we're binding by name for the policies.

B

D

Right got it yeah, it's like a typical clusterscope resource. You know where the names are unique and updating. The name is as good as creating a new new resource. Okay,.

H

Understood got.

D

It I mean there is an alternative of, like you know, having those five tiers created, as is the crds, and then we create an additional default here. So we have those application and defaults, but but it's too much complexity.

C

D

C

Head actually, when I said when I think default here, I kind of feel like the tier where the communities network policies go so under all the tiers we define for entry are specific policies, so maybe application is not that bad.

D

Okay, so at least at least, we have a consensus that uh we move forward with recreating those tiers and then uh uh and then maybe you know which ones are system one with. What are the names we can probably talk on the p on I mean you can have a conversation on the pr.

D

C

Thanks evan uh salvatore, there were two quick things I wanted to bring up before we uh we stopped the meeting I mean assuming. No one else wants to bring up anything. uh I just wanted to ask if we intend to support endpoint slices in entry, a proxy, uh I think that's a question that has come up and I think in starting with communities. 1.19 endpoint slices are enabled by default in in the cluster and in cube proxy.

C

So did we intend to implement endpoint slices support.

A

We should yes, I guess we should keep up with the anything any change that is, uh you know in kubernetes api. We shall not have unsupported features.

A

I don't remember if they were promoted to beta or if they were already gay.

C

Because they went through several stages right. What I know now is they're enabled by default with q proxy and before that I think the api was enabled, but unless you explicitly enabled it in q proxy, it was not that's.

G

C

A

Yeah I mean we should support them anyway,.

D

But do we know that was it? Is it part of the 1.15 or 14 release, because I think uh one of the reasons why we are not moving towards new resources is that you also want to support kubernetes 1.15.

C

Yeah, I don't know if that case is a bit different in the sense that we can have like a feature gate for endpoint slices or I don't know if we can try and detect. If the I guess, we cannot try and detect. Oh yeah, I guess we can try to detect if the api is active.

C

I think you can basically, my point is: I think you can support those endpoint slices in entry or proxy without breaking uh support for older communities versions.

D

Okay, so we do plan to like have a flag or something like that to use either endpoint slice or the endpoints.

C

uh I think I don't think it's an either. uh I think you should still monitor you. You should still watch both apis, but I didn't. I didn't look into it too much.

C

So so yeah I opened an issue on github about this. We can continue the conversation there. uh I don't know if uh hui chan has thought about it already or or not,.

G

Yes, I think we can can look into the issue to see whether some blocking breaking issues.

C

C

Thank you and the second thing is more of a bookie bookkeeping item uh salvatore. Do you know if we sent like an actual calendar, invite for the meeting on the mailing list that we need to update because of like the one week shift.

A

I don't think we have an invite on the calendar on the calendar. I don't think we have, but I will verify it and, if not we'll add the new one with the with the new calendar yeah, but I don't think we have it only one. On the uh on the main kubernetes, you mean the main kubernetes meeting list right.

C

uh No, I meant, like um I don't know if, when you created the meeting, you actually added to the list of attendees uh the mailing list, so that everyone got a notification. I actually don't remember, because I know that sometimes we have like a couple more people attending and I I'm wondering if.

A

There is surely there is surely no reminder sent from zoom automatically uh for this. uh The me, what I need to change is the let's say the zoom meeting schedule, which is probably not relevant, because at the end of the day you can start the meeting whenever you want, but the meeting is scheduled. uh uh You know, for instance, for the to be next tuesday and then that that should be changed, but in terms of our automatic notification strategies, we don't have anything because you know being a community meeting.

A

We don't have a an attendee list, so there is no way to notify a single person, and I don't know honestly if there is a way to send a message to the mailing list. That's something that I can try send a message to the schema to the meeting list about uh the meeting schedule. That's something that I can try if it works.

A

But anyway, our next meeting now is going to be on february, 14th, sorry february, it's september, 14th and you have you have love on your mind salvatore. No, no, you know what it is. It's just like september september is the month where school begins and it looks like january pretty much that's why it's the year is starting right now anyway.

A

So, yes, the next meeting will be on september 14th and then we'll go with the usual schedule of having a meeting every other week, uh starting with the september for september 14th um yep, and that will be it. It will also play nicely with the holidays in china.

A

Abhishek just pointed out that endpoint slides are not yet ga. There will be ga 1.20 and I believe that it's important for us to support them when they become ga. So let's say that it may be. If it's not 400 10 400 11, they should be supported, in my opinion, all righty, and uh is there anything else for today's meeting? I will try and see if I can send notifications about meeting schedule to the developer developer mailing list.

A

uh I don't know if there is a way for setting up also integration with this lock channel I'll, try and do that too and right anything else for.

A

Today and and no then, which means that I perhaps can stop the recording now.