Antrea Community, 26 Oct 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Antrea Community Meeting 10/26/2020

Description

Antrea Community Meeting, October 26th 2020

A

So uh the very scary noise from zom is reminding you that the meeting is being recorded and um so for today we have two topics on the agenda both proposed by abhishek.

A

So the further these comments, the first one, is a proposal to add a baseline tier, and uh for this we have already an open issue which is issue 1362 and the second are potential solutions for sorting anterior native policies based on higher priority and policy priority, which is uh issue 1388..

A

I'm going to copy these two topics on the chart for everyone's benefit. You can also find them on our slack channel and given that abhishek propose both topics, it makes sense to let him talk about them. Abject uh the floor is yours,.

B

uh Thank you for uh sharing the screen just to uh go through the issues and uh maybe one proposal uh for the first issue. uh So ever since we introduced the anti-native policies and uh tiering crds in the in the past couple of releases, we have had some users trying them out and we have already received some feedback in terms of what they would like to see in the policies or entering in those crds.

C

B

Few feedback, a couple of things actually more than two that were reported, uh two of them are on. You know, github issues, some of them. I have made maintaining myself like a list of things which I'll open up issues for them as well. So between those uh feature, requests for ask, uh I thought you know these two.

B

These two issues are, you know, probably worth tackling soon, so the so the first one is you know raised, although it's this issue is raised by jay, but the actual uh actually just comes from.

B

Essentially his use case is that okay, maybe maybe I'll back up a little. uh You know with the anterior network policies and uh gearing what we introduced is that an ability to for administrators to write network policies which will override a developer policy or a developer return kubernetes network policy uh so that the tiers in which we create are always uh evaluated and enforced uh before the kubernetes network policies are.

C

B

So so this use case is actually uh for something uh to be done. The other.

C

B

uh That is, uh they want uh to have some guardrail uh rules for the cluster. Some default cluster wide policies uh to be enforced, but they want to allow namespace owners or developers to write their own network policies to overwrite those clusters for policies. So this is something that we have not yet uh or our tiering does not solve.

B

So so that's the use case. So I think if we go back to the diagram you know on this community talk that we have. uh Essentially, this is the layout of the land that is, you know we uh provide a capability for users to create their own tiers.

B

In addition to that, we also have. uh We also have some system generated uh default, tiers that are created, uh or, I should not say devotees I. I will say that we have five tiers that are created uh when anterior controller is initially initialized, uh so for default consumption of your of the administrators, but they can also create their own tiers. Now the thing is that if the cluster administrator decides to create their own tears today, they can only create tears which will be enforced before the kubernetes network policy table.

B

So in this case now we have like three tiers in the in the in the diagram. uh As you can see, the emergency tier takes precedence over every other tier, so any policy which is created in this tier will be first and post. uh So if there is a perfect match on the rule, uh the rules action will be uh taken.

B

uh If not, then it flows to the next year and all the way to the application tier which happens to be the default year, where, in all the all, the policies which do not have an association with the tiers are accumulated.

B

Once all the all, these tiers are evaluated and there is no match, then the the traffic will be matched uh against kubernetes network policies which are developer written, and these are all upstream policies and eventually, when none of these traffic pattern or rules rule matches today, what we do is that we either allow or drop based on whatever the baseline goes on, so so in general, uh if there is no, if the pod is not selected by any of those network policies, then the traffic will be allowed.

B

So essentially, the ask, but by the community users is that we need another tier which can be created and policies.

C

B

Associated with that gear, after the kubernetes network policy, so I updated this diagram to include a tier called a baseline tier.

B

This will also be a system generated like or the proposal is that we would like to create this baseline here at startup, and I don't think they should be. You know we should allow users to create more than one baseline here. You know or provide that flexibility, because I think for this particular use case, a single tier below the kubernetes network policy probably solves all the use cases that would be out there, uh so the so.

B

In short, the proposal is that we add a baseline here when android controller is initialized and then allow cluster administrators to now associate policies either to their higher higher, ordered, tiers or or to a baseline.

B

So so I hope I mean if there are any questions on the user story or on the proposal. What do you guys think is this a viable option? Is it something that we can achieve.

B

Any feedback, any question.

D

So any uh uh layer, any tier would have like a default, uh allow rule and uh basically any denial would apply on each one of those tiers.

B

There is essentially a collection of policies and in each policy there are rules uh so that so, when anterior controller starts up, a tier will be created and it will be essentially empty. So let's say if a user does not create any policy or associates it with the baseline here, so this will be empty, so that would essentially mean today's behavior like the way the policies are today uh that is by default. The the traffic will be allowed uh because you know in kubernetes cluster every podcast should be able to communicate with each other.

B

uh So so, if there is no policy associated with baseline here, there will be no change in the traffic pattern.

B

Did that answer your question.

D

B

B

Any other questions suggestion on whether this is the right approach or maybe a question to people on the call is that do you guys think we should allow the flexibility to create more than one baseline here or you know, a system generated baseline here at npr. Startup is good.

B

B

Okay, silence: I will take it as.

E

B

um So I think the use case that uh foreign brought up was that uh he would like to create uh cluster-wide default denial policy and with certain exceptions for communication from parts to the code, dns parts and cube system namespaces, and he would want the namespace owners to be able to allow traffic on top of this denied traffic. So so, if, if the namespace owner has not created an allow rule or rather created a policy to allow some traffic, uh the default behavior would be to deny that traffic.

B

F

Yeah, I think sorry, so I think that's uh something. That's also coming up uh on the cluster network policy discussions we had last thursday is that because kubernetes allowed pause to talk to pause by default. So if the baseline tier is created under the equivalent cell policy layer, then I agree that in this tier it probably doesn't make sense for people to create a loud rule, because the pods uh communication patterns will be allowed by default.

F

But you know um I don't know if that's some hard constraint that you want to impose on this tier or if people just specify allow rules in this tier, it would simply have no effect.

B

Yeah so essentially, I think uh you know if you have an allow rule in the baseline care. It's really not.

B

It's really not doing anything special because by default you still allow traffic, so uh it I think, I think, by adding the baseline here, you're just giving more control for an administrator to write policies, but at the end of the day, what kind of policies they write is something that we would not. I mean.

E

If you don't have a larger just one tier is good enough right. You can create more tiers down. It will help you yeah.

B

B

So so you are agreeing that we should only. We don't need to really uh provide a way for our users to add more tiers like to to the baseline after.

E

One reason for if you have denied a law, then maybe makes more sense to have multiple tears. I'm not sure you do have do we have any other reason you want to do our bike for your tears that you have different persona to create different businesses. I don't know.

B

E

B

Then that's true, but that's that's really not a use case that comes often or at least the user who tried to uh bring up. This issue did not come up with that kind of a use case. uh Okay,.

E

B

E

B

Good also, I think we don't want to restrict the kind of policy that goes in a baseline, tier versus other tiers. Then you would have to do a lot more checks.

B

I mean we can give the ability to create our rules. It's just that. You know, there's really no use. No, it's not the right use proper use of that.

A

And abhishek, I have a simple question on this design.

A

The current uh proposal will put you in a situation where a kubernetes network policy can override the baseline here, meaning that probably the cluster, the baseline tier as a policy to say to deny traffic on a given port, and then you override it with a kubernetes network policy to instead allow traffic on a given port. Is that possible.

B

With the current design or with the new proposed design,.

A

With the new proposed design.

B

Yes, I think that is what we are trying to uh achieve with this. So.

A

B

Yeah, so so, basically, if you have let's say in this uh diagram, you have a a and easy. Let's say this is used to deny traffic for port 8080 and and now as a cluster. Sorry as a name space owner I for my main space, I create a network policy z, which is essentially this thing, but it allows certain parts to allow traffic on port 880 then for those parts it will be allowed and for other parts you know it will not be enough.

A

Yeah. Okay, sorry that, then I I was thinking, I was thinking about a different use case where you were using this baseline here for uh for having a baseline security behavior that users could not override status over apps. It's a different use case.

B

Yes, so if you want something which cannot be overwritten by network policy, then you can create policies in either of you know any of the tiers that are hired. I see.

E

Actually, another one is that um you know people are talking about. They want some default behavior, but isolate the namespace right. That means inside and space you can communicate freely. You cannot do any course. Names course names risk making um for such kind of a policy. uh It cannot be defunded with all kind of policy expression.

B

Yeah, so yeah go ahead, so, like I mentioned at the beginning, is that there are multiple feature: requests that come in so the so this this was one. The one that you mentioned is the other, uh wherein people want a very simple way to write a cluster scope policy, which kind of the effect is that my intra name space, all in international space traffic, is allowed, while, inter name space is not allowed, and I would like to have that as a baseline policy, so so with the baseline here.

B

I think we provide them to have such kind of policy, but our policy is not yet expressive enough to do that, uh like a namespace isolation, so I think we can. We can add, on top of it afterwards and have our cluster scope policies the the ability to express mean space isolation in an easy manner. So maybe we can decouple those two stories did. I did. I make sense.

E

And, do you mean that policy can be on any team.

B

Yeah I mean I haven't yet thought through how to implement uh the namespace isolation, uh but we can, we can think about it in in a separate user stories. What I'm saying.

E

B

But do you are you suggesting that we should? We should do the name space isolation only for these uh baseline tiers not does it make sense for higher order, tiers.

E

No, I I didn't think about yet I'm just asking: what's.

B

Okay, yeah, I think I think we need to uh have our policies be able to express that in a very simple fashion. Today it is not yet the case. Okay,.

G

Regarding having uh allow rules in the baseline here, I had like a question. What, if I want to define like a baseline policy that says, unless the developer decide to prevent this, maybe allow all of my parts to talk to core dns pods right, and I can do that using like a clustered.

B

Workflow policy does not have explicit denial, yet so anything that they will be creating would be an allow and then.

G

Well, yeah, but if I, uh if I did find a policy that selects pod a and say point a can only talk to part b, I mean part a becomes isolated right, and so it cannot talk to other parts. Even though I don't do an explicit deny just doing an allo means if.

B

So so the the thing is that, with this we are kind of altering the expectation of a kubernetes network policy.

B

I think right because if you are part, a was supposed to be isolated, but as a administrator, if the isolation rule is uh uh enforced after the baseline tier and the baseline here add like a then we are not truly. We are altering the behavior of that community's network policy.

G

B

So, in this case,.

G

It it does seem that I would only have deny roles in the baseline here, because.

G

Because the other way I prevent traffic otherwise with a humanities network policy, is by making a pod isolated, basically with an explicit network policy and then, as you say, if I allow uh some traffic in the baseline here using analog rules and I'm basically changing the implicit behavior of the kubernetes network policies, yeah yeah.

F

Yeah, so I feel like in in this particular case, the uh the cluster ming might be better off, creating that policy, not in the baseline tier, but in some tier, that's above kubernetes network policy. If what they want to enforce is something like always, you should always be allowed to talk to core dns.

G

Well, yeah, but uh then the same thing is that the developer cannot override it right. What I was saying was that uh you should allow pods to talk to core dns unless explicitly denied. Basically, but you cannot express you cannot deny it with the currencies network, both.

H

Yeah yeah, that's.

G

That's the thing yeah, so it does seem to me that only deny policies make sense in the baseline here.

B

I mean users can break allow policies, it's just that it's not really going to uh have much of an effect on the on the cluster.

G

Yeah I mean it's hard to see how it would have like any effect right, because if you don't have any kubernetes network policies, then your baseline here is not useful. And if you do have kubernetes network policies, we don't want to be able to negate their uh the isolated behaviors that they provide right. So.

B

Yeah, so I think an alternate to this would be that we do not have a baseline here. We have a global configure option. Yeah.

G

That was going to be my next point. If the only use case we have is default, denies.

B

Yeah so so there we could introduce a couple of uh option values. One is uh you know deny, which is you know you just deny any traffic and the other is uh you just allow the namespaces which which gives you the default.

C

B

Of namespaces, but but does not allow cross, namespace talk and the third one is default, allow which is essentially, you know the current.

I

B

Now yeah, the only thing is that it doesn't give you like an expression, a way to express very specific rules like I don't want certain traffic to be allowed by default, but maybe the administrator name, space so name space owner can change that. So that's an alternative which works for most use cases, but maybe not all right, but I don't know if there exists a use case which, uh which needs to be that specific for for a baseline sphere.

B

I guess when you're already aware of the config option based.

E

uh Yeah first thing is basically is more flexible yeah. You know you can control prevent space and based on.

B

That's true, I mean if it's a cluster config option, then it's for the entire cluster, with the baseline tier. You have the ability to maybe write policies for certain.

E

B

And leave out certain other namespaces.

E

And the for its landspace as long as I stopped over, I think probably makes it more sense to be based on only otherwise very strange. I mean I fixing up here, you're saying uh landslides. Actually you mean uh you allow water in international space traffic that will override the other, the the policy below the tier. So it's it's! It's pretty.

E

I I I think it's very useful.

B

Maybe, as a maybe as a cluster owner, you want to isolate namespaces and you don't want crosstalk based talk, even though kubernetes network policy does allow you to do that. But okay, maybe you want to have them as silos and yeah depends on the use case of the class.

E

But they are not saying a name specifically you. You are saying uh sorry, probably I I I mean a different thing. I mean the behavior of actually I mean two behaviors isolate them space and allow internal space um for I just mix them uh yeah you're right. If you just want that, so you can still do it. Okay make sense.

B

So so it does make sense, it seems to have give the flexibility to the administrators with the baseline.

E

I don't know how useful it is. I just mean I can imagine.

G

Yeah, because, right now, based on the use cases that people have presented to us, these are mostly like the configuration parameters you presented right, allow within a namespace by default, uh but not to other namespaces or default deny. So those are the actual user requests we've had, but I agree that if we make we can make a uh acnp more expressive to satisfy those use. Cases and the baseline tier semantics become more useful.

G

And more powerful, just having like configuration parameters.

B

All right so yeah, I guess maybe that could be the next uh next issue which we can open up for for the uh uh name, speed, isolation, expression, and maybe you can think about how to how to do that with our policies.

B

uh If no other questions, then maybe we move to the next uh issue.

F

Hype check just one more very bookkeeping thing, um so I remember for the tiers. We have the numbered priority associated with it and higher the number, the lower the power lee so for this baseline tier. What are we thinking in mind for the for assigning the power for that here?.

B

So as part of the tiering crd- and we left a range as reserved like a higher, very high range- and maybe we can use one of those reserved priority numbers as as one for the baseline, which would be known, which will still be reserved and which will be known to the system. Agent and controller.

B

Would that satisfy.

F

Yeah that works for me, um but I mean I'm probably I'm just thinking you know for for that reserve range. Maybe you want to. We want to pick some in between values in case we in case we wanted to add. You know more baseline tests.

B

In the future, yes, I think I think, right now, we have about five five, four or five reserved uh priority numbers.

G

Yeah and if I try to create a tier above 250 right now, which is, I think, is the default tier priority right now. The the web book is going to give me an error, correct, okay and right now we're going to keep that behavior. If we introduce the baseline here as well, so I won't be able to create it here between the default here and the baselines here right, correct.

B

So yeah, maybe we can discuss the exact implementation. What number to use in the pr um the next one uh is. You know it's more cosmetic, not much functional behavior, so this one is about, as you guys know, that you know, as you all know, that, for the cluster scope policies we have priorities which help us order. A policy precedence over the other policies uh higher the lower the number uh value short value of the priority means it's. uh It is of higher order.

B

Similarly, the tiers are also ordered based on priorities, so there's a lot of priority numbers going on around right and we, since we don't have a dashboard or a way to visualize things. uh You know mostly people use to view those policies. So you know this is very good as part of the issue. These also provide an example.

B

uh Essentially if he does a get for the.

B

The sorts by the priority field, uh he gets uh all the all the clusters of policies or and candidate policies uh based on the increasing order of the priority number. But they are you know. If you have policies spanning across multiple tiers, uh you know it will only consider this uh the policy priority number. It doesn't consider the the tiers order, because the tier is in you know in the form of its string name right.

B

So so what we are seeing is that, although pa1 is shown at the top, so now as a user, I would think that this is the highest ordered policy, which is not the case, because you know the pa1 belongs to application tier as we. As you know, the application tier is one of the lowest uh priority tier, while the emergency tier is the most highest priority. So if we see that pe1 which belongs to emergency, has is technically the most highest ordered policy.

B

So so what happens here is that the output of a cube cutter is not really giving me the correct picture now. Now there are a couple of ways to solve this uh problem, one, I think uh young. Maybe you want to talk about it because you also kind of uh proposed similar uh thing. I think I had an offline chat with chan on this about adding a new field uh on on a effective priority. Sort of uh that that we can have.

B

uh Essentially what the effective priority would do is that it will take the tier name and it will take the policies uh priority number and then it will.

B

You know normalize this into another priority field or a number uh which will become the part of the the anti-native policy spec, and then we can sort on that one field, um so you know, I think I also spoke about this approach with chan uh offline, and uh I think uh you know maybe china, you wanna talk about why you were kind of against, or you did not really like. That was because uh now we have like two or three priorities associated with with the single policy.

B

uh So sorry any feedback on so far uh on on this issue before okay go ahead.

G

Well, I was just gonna say I was also thinking about the status thing, but it kind of seems wrong, because I'm ordering I mean I'm adding to status something that's entirely determined by the spec. In a way right I mean it feels weird that I have to rely on status to sort those objects, those resources when, in the end, their priority is like completely determined by the specification I've provided for them.

G

B

So for the status, I think I I initially proposed that I think tran also had a suggestion that maybe, instead of the status, we kind of do this in the mutating web uh itself. So, as part of we introduce our book for mutation, which you know before it's passed to the controller it just takes in the tier. Your priority takes in the policy priority and then generates it in the spec itself and adds it to the spec instead of adding it to this as part of the status.

B

But essentially it will still result in the new field.

G

Yeah I mean personally, I understand the appeal of being able to do it with cube ctl, but I kind of feel like that's also why we have nctr in the first place, being able to do things like this.

G

That that's a really good point too, so I feel I mean it would be nice to be able to do it with cube ctl using the built-in features, but at the same time I feel like. We also want to make nctl better and like a to-go place for users, and so maybe we should not go out of our way too much too, to like define some solution just to have it work with uh uh cuba, solutions that would add complexity right, because now we would like mutate, the mutate, the resource and so on.

B

Yeah, I think we are already planning to add a mutation for the rule names. But but yes, it's just another value to be added.

F

um One thing I actually also want to bring up is that uh I might be hardly wrong about this, but uh does the end cuddle only work on the on the agent containers or for the container controller agent as well.

G

Now that would be the remote mode where you connect to the controller, so you can download nctl on your laptop right and give it cube config. Just like you would cube ctl, and then it connects to the ngr controller through the community's uh apis. Okay,.

F

I see yeah the reason I'm asking this is that uh when I was discussing this with object- and he brought up a point where you know, if we have um ac ampere or amp enforcement uh pods on different nodes, then we probably want to you know.

F

uh We don't really want to look into the policy that on that specific node right, we want to look at all the policies and figure out their relative priorities, rather than just you know, looking at what's on the node, so um this has to be done somewhat like globally, in order to work.

B

Yeah, I think it should be possible.

G

However, maybe I'm wrong, but I.

B

G

Isn't it true that today, nctl is only gonna, use the control, plane, api and query the internal policies, or is it able to get the actual crds.

H

G

B

And the policy priorities both should be part of the internal network policy uh spec. So we should be able to finalize the priority.

F

B

The only problem is then displaying with the tier name might be a problem.

G

Yeah, but my point was right now: if the user actually wants uh the crd and not the internal resource right right now, nctl only gets you the internal resource, it doesn't query the crd from the current disappears. It only queries the internal object from the controller.

H

Yes, I think so maybe we need to rewrite the the command name, because it will be confusing.

H

I mean this is srd network policy.

F

Yes, but from my point of view like as long as as long as then, we are given the names um with the relative order. You know if users want to actually look at the crds or from the program message. Sorry, the kubernetes perspective um they can always you know, use the name and the kubeco command to actually inspect those resources as long as in encode, we provide them with the sort of like the ordering he is. uh You know looking to see, but yeah that's my opinion on this.

G

I

G

True yeah yeah go ahead and check.

B

Yeah I was, I was going to say as long, I think yeah as long as there is some sort of way between the two cuddle, outputs and control output. I think the user will be able to figure that out. It's just going to be a two-step process if he wants to get more information about the policy.

B

The other proposal that I had was probably you know requires a upstream change. uh It's a potential, uh something that we can. You know if the upstream folks thinks that it is something that we can add and it's valuable to the output output is today, the sort by uh sort by uh filter criteria picks only a single field, and it's also on that.

B

I don't know if there will be happy to accept something or I don't know whether this is very easy to perform, but we provide a list of fields uh and the sort happens on each of those multiple fields based on you know the priority in which they are sent in so, for example, if I were to do cuddle, get a anc, scmp sort by spec, dot tier commas for spec dot priority, then it first sorts on the spec dot tier and then next output would be uh starting on uh spectre priority, and then that would give us the correct order.

B

uh So, but that would mean an upstream change in cubecode.

G

Don't we have the same problems, though, where the tiers are going to be sort sorted by a lexicographic order.

B

Yeah, I think we, if we.

G

Because, in the policy theory all.

B

You have is a tiered name right.

B

Yeah, so we might have to mutate it back to the priority priority.

G

Sorry either my internet was bad or your internet was bad. I couldn't hear you for a.

B

Second, here.

G

B

Okay, so I I said that maybe we need to mutate the tier name back to your priority um and then user use that output.

A

I don't know you know abhishek. I have the feeling that every proposal to solve this problem is a little bit over engineered. In my opinion, yeah I mean.

I

H

A

Commenting to this issue report, I will say you use, I will never use awk and sort.

A

To filter the output so that it so that you know he, it displays nicely.

B

Yeah or you, you could still cover that, but uh yeah yeah, but I think the ant cutting is kind of like a good.

B

A

B

Gives more value to our cli.

F

Yeah, I agree, and I I don't think that would be too hard to implement, given that we have all the information handy already in in the store. So.

B

Yeah yeah, I think we don't need to pull anything.

A

So I hate to interrupt on this topic, but uh we have only 18 minutes in the meeting and another fairly large topic for today's agenda. So I would like to ask you if there is a any final question on this issue.

A

So um the as some of you know already uh ray in the past few months, three weeks has been working on an operator for andrea and his work now is a very mature stage. So uh you would like to present uh what is this operator for andrea, how it works and what are its benefits so ray? Are you ready.

I

Yeah, I'm ready yeah.

I

Okay, so can you see my screen.

G

I

Okay, so next I will uh show a demo about the entry operator uh before the demo. I will have a brief introduction about the anterior operation, for example, uh what the anti-operator can do and how it works.

I

Okay, let's start from from the introduction.

I

Okay, uh the uh the main uh task or main target of the anterior operator is to provide a automatically way to deploy and show plugin uh unto plugging resources. The resources uh includes the entry and controller deployment. Then tray agent demonstrates config maps and other resources.

I

Besides the deploy the deployment, uh the operator uh will will also have the ability to monitor the configuration changes uh from the user and apply the configuration change to the entry resources.

I

And also the entry operation, we can monitor the entry resource or entry object status uh and to maintain maternal resources uh to the to a desired state.

I

uh For example, uh if we have, uh if we uh delete the actual controller or uh summer resource uh by mystic, the anti-operator will um will uh monitor the the changes and will create the resource bank again uh to meet the desired states, and the last task is when operator is running. It will update the related status according to the operator reconciliation uh process or results.

I

uh For example, uh if the operation is working on created resources, it should uh update some uh updated status and processing uh to some results to show the status of the operator.

I

So uh it's the many function of the entry operator.

I

Okay, so this is about the uh components of the android operator. The entry operation uh consists many two two components uh deployment. The entry up uh the entry operator runs as a deployment, and the deployment will will include the entry resource template manifest.

I

This template is from the anterior all-in-one yammer file actually operator will use this template to generate the entry resource specs and on the operation. We also have a custom resource uh which name the uh the entry install this. uh This case of uh resource is for configurations uh for the entry.

I

For example, the user uh can specify the android configurations in this resource uh because, for example, the anterior controller config on the agent config the signal configure of the entry image image version.

I

This config will be right to the uh to the anterior config maps by the entry operator. Just like we added the uh configure maps directly.

I

And here is the example of the entry install uh custom resource, uh for example, uh in this bike we can see here as a configs with just a list of above, for example, we can set the agent config in this uh this option in the entry original config option and set the control config in the controller config option, uh and also we can set the image we want to use by the entry and the last one is the addressing eye config.

I

This configure will be used by the an actual agent to be installed on the nodes.

I

Okay, so next, let's say uh how enter operator works. uh Azure operator uh will just works as a controller. It will uh run the reconciliation logic to uh to monitor the change of the mindset, change of config and apply the user config to uh entry resources.

I

For example, we just have shown that we have a custom resource named entry store. We will set our entry related configs in this custom resource and for om shifts. It will have a additional class network config, which we can set the position, the service side and other options.

I

uh So these two file, these two files, uh will contain all the configurations needed by deploying by deployment. So the operator will use this config options and render this options in the anterior dot. Emo template file and generate the entry objects by spec, and this. This template file is just a copy of the entry ammo from the andrea upstream uh and just when the operator will need to uh replace the configs in this template uh and and and last uh after we generate the own entry object spike.

I

The operator will uh apply these specs to the api server and to create the entry objects.

I

So this is the configure reconciliation.

E

For the yamaha template, why why we can yama template it's from the upstream? Oh, it's like you, maintain it in until operator, repo.

I

uh This is from the andrea uh report from the release release sites yeah, okay,.

I

uh And, besides the config reconciliation, uh I feel uh we also run the portal reconciliation logic. It will monitor the states of the entry controller deployment and the entry agent inside. uh If, uh if this, if the resource is deleted, uh the entry will recruit the object again.

I

And during the reconciliation process, uh I'm sure we'll uh we'll we'll show we'll set the states of the of the process are the results to some resources to show uh the current process process of the operator. uh Here we list three releases, three resources uh which which will record the states.

I

uh The first one is the class operator processing status. uh It will show uh if the uh algebra entry operator is processing, processing, uh config change or other things, and if the operator is in degraded states.

I

We can take an example, for example, we just make a configure change and, as the operator will process the config change and during the this stage uh the processing status of this results will be true.

I

It will mean that the operator is working on the process and another is the entry um configuration realizations test.

I

We can remember that uh we set the anterior configs in the entry install uh custom resource, uh so uh this this this this status will record if the configuration is realized or not, uh and the last one is the custom network status. uh This is this resource is used in the openshift environment.

I

uh In openshift, there is a environment uh we can set the uh cluster skillful network configuration in this resource. uh For example, we can see that uh here we set the cluster slider here and we set the service slider here and after the operation the operator has reached the realized status.

I

It will update, update this customer resource uh to uh record the current status uh currently realizes. This, uh for example, we can see uh here uh is a 100 additional uh this option as I'm the mtu. uh It means uh the the operator has currently set the m2 uh as 1040 and 400.

I

Okay, uh here is the a brief introduction of the entry operator, so any questions.

C

Hi hi ray uh I'd like to know: when will the cluster operator shows the degraded state? When will the degrading status is true.

I

uh Okay, uh the one key is that, uh for example, um uh uh we can assume that if we have a wrong configuration for the entry, uh in this case, the anterior agent of the controller may cannot be created successfully.

I

So uh in this case we will show we will see that the entry cannot be cannot be deployed properly uh successfully. And if this, if the keys are seated for a long time, uh the entry operator will show will set this degraded status and force and true.

I

I

Okay, uh next stage, I will show the demo of the operator on openshift four, and here are the steps I will show I will list in the demo. So, firstly, we will show the classes go next local configuration before we deploy openshift cluster and the second step. We will use the configuration which, as I said, and to deploy the upper shifted cluster and after we have installed an objective cluster. We can, we can test or verify the entry operator functions.

I

For example, we can up update some configs of the entry and to see if it will take effect, and the third step is that we can. uh We can delete some anterior resource and to see if the operator will create it again uh and and in the last step uh I will show if the anterior network can work properly in the shift in openshift environment.

I

Okay, I just recorded a video for the demo start, the demo.

I

Okay, firstly, we will set a custom scope network configurations for the openshift cluster.

C

Right actually.

I

C

See the ppt yeah.

I

Okay, sorry, let's let me switch.

I

I

Okay, okay, see the video screen.

E

Yeah, I can see.

I

I

Okay, uh firstly, uh I will edit the cluster scale of network configuration for the cluster.

I

We will use a file named install.

I

Config.Yml and uh we will set the netting networking option.

I

The field uh we need to sign is to one is to change. The network type here is ncp, uh where we will change it to anterior.

I

And for us for the other options we will, we will keep it as original, for example, the position and others, and here we can see that there is platform field this field. This option is for the configuration of vcenter.

I

Here we change the network type to entry and save it.

I

Okay: next, we will use the openshift install command to create the manifests for the deployment.

I

After we, after we execute this command, we will come. um We will have some manifest files. uh This this director will consist uh the yaml files uh or the yamaha resources uh which needed to be installed to the to the cluster, uh because we want to install the entry operator uh during the bootstrap, so we will need to copy the entry operator yaml files to this manifest directory.

I

So, let's copy it.

I

We can see we will copy the essential files, for example, the name space, the operator deployment and the real binding mos.

I

Okay, next, I will use another command, the create ignition configs and after we execute this command, we can see the direct contact change. We can see here. We list the files in in this directory. uh We can see. uh There are three uh one, two three ignition config files, this three config files is used, are used to deploy uh openshift cluster virtual machines.

I

uh It will uh each of the files uh will contain the virtual machine config and the network network configurations for the virtual machines, for example, the bootstrap dot. Itunes is for the bootstrap virtual machine and the master and the master file is for the mass node and the worker worker.itune is for the work node.

I

We will we will need the virtual motion config files when we deploy the virtual machines.

I

Let's go to the next step yeah. uh If we want to deploy or openshift our class, we will need to download the installer, which is provided by openshift to help install the.

I

I

um The opposite of the installer will use the terraform for the deployment, so we will need to copy the virtual machine configs in the terraform terraform config file. Here is the terraform dot tf vars.

I

I

Okay- and we can see in this file, we have already set some options. uh For example, the the top lines are the options for the v sphere, this very center options we want to use and we can see the the middle lines. It includes the control, plane, cons and the computer computer.cons, and this line is bootstrap ignition url.

I

This url will point to the photoshop data engine and the file we just created, and the next line is the control plane in acne ignition.

I

In this line we can copy and we can paste the control plane virtual machine configs here.

I

And the next line is the computer ignition option. We will copy the will copy the work node configuration here and in the bottom line. These are settings for the ips of the virtual machines.

I

Okay, let's start to copy the virtual machine.

I

Configurations.

I

Okay, take the master node, as example. We can just copy the content of the master configuration file and paste it here.

I

The other nodes are will have the same operations. I will skip it.

I

Let's go to the next step, so after we have set the virtual motion configurations, we can use the terraform to apply the configuration to apply the configurance and create the cluster we can use the apply apply command to create other virtual machines in the.

I

I

Okay from the log, we can see that the terraform is creating the vms.

I

If we open the ui web ui of the center we can see here is a directory on the wp user, provisional infrastructure.

I

We can see from the tasks the vmware is in creating okay, the credit process will take a lot of time, so the openshift has also created other provider commands to which the bootstrap completes. We can use this command to with the with the bootstrap of the cluster.

I

Complete okay, it will cost about 30 minutes. We can skip this. This.

I

Time, okay, after four well, we can see here uh it's. It is now safe to remove the bootstrap resources, so it means the strap stage of the cluster is complete.

I

Now, let's say the known states of the of the cluster.

I

Firstly, let's get the status of nodes I'll be using oc, get nodes. We can see here. We have successfully applied the sixth node cluster and every no every stage of nodes is ready, so uh it is written now it means it also comes from the initialized ci plugin.

I

Next, uh let's see the entry operator stills, uh so we can see here uh we can see that the status shows and now currently the processing in force and it means uh countries the states of the states of config have reached the desired state.

I

And degraded flank is false.

I

So let's check the anterior entry ports.

I

Okay, we can see that uh the entry agents and the entry controller acts greatly successfully.

I

uh And then, let's see the status of the glass network.

I

Okay from here, we can see that currently, the status of the cluster network has meets the spike of the in the network and currently the uh class network. Mq is.

I

1450, so next we can verify the function of the entry. For example, in this step we can change the mtu of the entry and to see if the change will take effect.

I

I will add the entry install custom resource and we can, in this custom resource, we saved all the configurations for the entry.

I

Okay, I will add it to the untraditional configuration and the set the default mtu to another value, to a new value and save the save to the option and at the same time I will monitor the status of the entry operator in the bottom.

I

You can see uh currently the processing in force and then I will save the configuration.

I

Okay, after the configuration change, we can see the processing how switched to the true it means the entry operator is working on change. The configuration- let's wait for well.

I

Okay up somewhere, we can see the processing from switchbank to force, and we, if we release the ports we can see, the agent is, is created recently, because uh if we change the mtu, it will need to recreate agent.

I

So from this step we can say that the entry abroad algorithm can monitor the config change and and apply the config change successfully.

I

Yeah, if we see the class operate class network status, m2 has been changed to a new value.

I

Okay, so next step, I will delete, delete the deleted entry uh controller deployment and to see if the android will monitor the change and recruit the deployment automatically.

I

As we can see, currently the entry controller deployment is used unless let's delete it.

I

We will delete the deployment.

I

Okay, currently, the device has been deleted and we can see there are change in the processing it has been switched to to true. It means the operator has monitored the change.

I

And yes, we can monitor the deployment change here.

I

And with four wheel, we can see that the operator has recreated the android operate on the entry controller again.

I

So uh from this step, we can show that uh if the entry resource is deleted uh by uh by user or by some uh some other way, the operator will monitor the change and recruit the results again.

I

Okay, here is the next is the last step in this step I will show I will want to verify uh the function. The function of the alternate uh networking uh we we will test if the uh network negativity between between port and between port and the service is, is as as expected.

I

So, let's I will apply some ports and.

I

I

Okay, we can see on each node, we have created a hello world pod and let's test the connectivity between between the different.

I

Nodes, uh I I select the two parts from the different node and the user command for the test.

I

Yes, here we can see the say this is the output, hello communities, so it means um the the connectivity uh between the node is successfully.

I

Next, uh let's check the connect connectivity between port and service.

I

Here we have just created a hello world service. I will use a product to outsize this service.

I

Let's use the class ip okay. We can see that the port also can access the service successfully. So this step means we use the actual operator to deploy a function of function. Normal attribute, login,.

I

Okay, uh that's all for the demo.

A

Thank you very much ray and uh we are quite a bit over time. So is there any question on for ray on this demo.

E

I do have a circle for that. I'm not sure I should ask here or I should ask.

A

It depends if I think you know in the interest of everyone's time, if it's a quick question, please go ahead. If it's like something that you believe requires, you know more conversation, perhaps we can have it on luck.

E

Okay, maybe let me start with um several quick one, so um so first do is support killing off of the andrea I mean: uh do we have a way to delete entry and cleaning up all the resource and the state we.

E

Installed or changing our arena.

I

uh Do we need to delete all resources.

E

I

E

Yes, and uh probably even better, if we can remove any state, um live diagnostic, like ib tables, whatever.

I

Currently, uh we we only supported to deploy, to delete the uh deployment and the different sides of the entry.

E

I

Yeah, if we did some other resources, however, we didn't monitor the the other change of the resource.

E

Okay, probably I should ask- is that a typically a responsibility operator to do the cleaning up of the resource in the state when we uninstall the single.

I

uh The cleaning up uh so so, how do you uh do the cleaning up to remove the uh every every result of the entry.

E

uh Actually, I'm not sure what openshift operator or other operator typically do. I'm just wondering uh do we know any other operator or do we know openshift operators support some cleaning up. We are like wrong.

A

Junction from what I know, some application operators when you delete the operator uh they they perform, they perform this kind of cleanup like uh deleting every resource that was managed by the operator, but other operators.

A

When you delete the operator, they just do nothing. So you know all the resources that are managed by the operator are just left there. I don't know if this is the correct, behavior or not, but we find both in this case, in particular, when you install the operator with this process in the openshift cluster.

A

The problem is that I don't think that once you remove antria openshift will do anything to install another cni solution, so basically the cluster will be left without any cni at all, got it.

E

Because we do have issues still there in england report about cutting ups, should we be able to use operator for that.

A

We can surely think about it, I, but not in the overshift scope. I think I don't know right all right. What do you think about it? That would be like, uh I believe, an explicit. uh There will be something that the operator will have to explicitly do because you know, if you just delete the operator deployment, then you know the the operator pod will die and will not collect the resources. So there must be an explicit cleanup action, probably.

E

Got it um another question, that's do we? Are we able to support multiple versions of andrea by the operator.

I

uh Currently, we, the version of the operator just uh just part of one version of of andrea it is, it is a one-to-one religious relationship.

E

Okay got it. That means for every new entry release. You will need a new operator.

I

Yes, yes, because uh some options will be different between different uh actual version.

E

And then for the for the master, uh I mean untrained master, uh the monster, the beats from master punch. um Probably we don't have a way to track the chain right.

E

What's up, okay, so basically we're seeing uh just after rv until version released, we have a new operating motion, but for the master, probably we don't mean we, we probably don't have a way to track the chaining on the master.

I

uh Yes uh in serious uh series: if the options have no change, uh it will be kind of used can be used.

E

Okay, uh yeah, probably my last one. uh Probably we don't need to conclude here. uh I've started wondering for what's kind of useful feature. uh Do do do do. Do you think it makes sense to put that into android repo itself? I mean, if we're able to decouple some openshift specific thing just for generic operator functionalities.

E

um Is that possible? We, um I don't know it's possible now, it's like. Could we try some some generic functionalities and the maintenance and share ripple, and then that means even uh even from unsure itself, we we have some useful operator features um first in a second.

E

Probably it's about it's. It's either easy easier to track the change in anterior itself. I mean if we, if we put anything on, depends on the anterior um config poles, some other behaviors, it's probably easier to maintain that in 90 percent.

I

uh Yes, currently, we want to uh to provide the comp compatibility for the standard communities environment in current rebel and uh for further things you have mentioned. uh I think, for the first step we can consider to expose some functions or some data structures from the anterior repo. We can directly use the this exposed structures uh for in the entrepreneur entry operator.

I

uh How do you think.

E

Okay, it probably actually I didn't symbol that much yet, and probably probably let's discuss more on this one- and this is really wrong with the round of 10 today,.

I

A

So one very, very quick question for me is: do you think this uh installation process with the entry operator for openshift can work also on other platforms like aws as your gcp, or do you think there will be some blockage blocking issue there.

I

I think there's no block issues. We just need to do some code clean to to set the code control on other platforms to knowledge to depend on the overshift.

A

Perfect. Thank you all right is there any final question on the entry operator waiting 10 seconds so.

A

All right, so thanks for staying with us for a little bit longer than expected, it's been a great presentation from roy also thanks a lot to abhishek and uh young for the discussion on network policy and financial cluster network care network policy topics, and I would like to thank everyone for attending and uh remind you that uh the we will update meeting times for uh the next meeting until the next change of daylight saving time in the next spring, and the new meeting time will also be updated. Indian tria readme.

A

So thanks everyone for attending- and I wish everyone a good evening good morning or good afternoon,.