Kubernetes WG Multi Tenancy, 30 Jun 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 20200630 Kubernetes Working Group for Multi-tenancy

Description

Xiaoning Ding will walk through multi-tenancy features of Arktos (https://github.com/futurewei-cloud/arktos), and see if the community is interested in working together to upstream some of them to Kubernetes

A

Okay, hi everybody and welcome to our regularly scheduled multi-tenancy working group for kubernetes. Today, we're gonna have a presentation on arctos and kubernetes multi-tenancy and then Rodolfo is going to talk about CRD management and multi-tenancy.

B

Okay, hi everyone: this is Johnny I'm, going to walk through the Cobra, tiny, small, the tenancy Actos. This is this feature that you're in a walk from Chan, Oh Hawaii and me they also hidden a meeting today.

B

This is what I plan to talk about today. First I will give you a quick background and overview on the project, so you guys didn't know. What's what's the context, I know what we are doing and then I will walk through our design.

B

I will give a quick update on the current status. I know what's next and then lastly, I will ask you a demo.

B

So here is a background is an open source project where we talked for you Actos to build a really large-scale cloud infrastructure.

B

This project is a open source and it's licensed under Apache 2.0 Actos is derived from kubernetes at the village diamond core design enhancement by deriving from cognates I mean why we take the kubernetes codebase and applied our design enhancements on that so far where we are mostly working on this key husband. Firstly, the native vm, orchestration multi-tenancy and the scale bridge and some cloud age scheduling for today's talk.

B

I will only talk about the multi-tenancy part, but if you are interested in any other features, just feel free to check out our github repo and all the design documents and the PR codes are all there. It's all public so can I.

C

Just ask one sort of background question before we go on.

B

C

Yeah, it is our twist intend it to be like a research platform or a production platform. Are you just looking at experimenting with features that might be up strained, or is this supposed to be a product in its own right? Oh.

B

Yeah, actually, it's kind of in-between we are all, probably all from which we were based in Seattle. We ourselves a kind of research group, but this project will be a standalone project. We also looking for industry adopters, because only only by getting feedback from the real real product environment we can. We can move forward, so it's kind of in-between. We we tested some ideas here, but we also want to be adopted in the real in the real production environment. Okay, thanks and for all the features into the community community is interesting.

B

We would be very happy to absolute work with the community to upstream them. That's also one reason the. Why wait? Then we reach out to the multi-talented working group.

B

Well, first, one some feedbacks establish establish a connection with you guys having some regular technical discussions and if we gather interested in any multi-tenancy features, we were happy to work with you guys, absolutely okay. Thank you.

B

And this is an overview of our multi-tenancy vision over our our regions. We want to enable multiple organizations to safely and easily share a physical cluster infra. We all know that they are different models. Our model Helen see. What we implement is a is a kind of what you cluster model. We know in the communities. There is also a project to cut what your cluster the difference here is. Our vertical is based on one copy of control plane, it's kind of beauty in changes, making changes where we don't deploy separate control plan for each Talent.

B

Instead, all the talents that share one running control plane and they are all isolated. If we look at the diagram, our left side, we can see there are two talents, each of them access the physical cast as if they own the physical cluster exclusively, and they are also using the natural kubernetes api.

B

In fact, for these talents, they're probably even cannot aware they are on the multi-tenancy environment, it's all transparent to them. They feel like they are using the original kubernetes in an exclusive way.

C

It's a question: is this virtual cluster, the same as the virtual cluster project that yeah.

B

That's what I! Just briefly you know when we are aware there and there's a water crust in the working in the modern sense of working group. The.

D

B

We have some design principle I do like to come back to this later because after we understand our design, probably it's easier to understand this design was.

B

Our hood is concept cutter space. This space is actually a new layer in the resource hierarchy, so with space we will break the whole I guess server into different pieces. Each Tanana has a header space and the inside his space he can has, he can have always resources and each API object will be located in one space and only in one space here, I put a tool. Examples you can see on.

B

The left side will have a pod, which is a namespace scope, object right and the PV is a non namespace scope object and they are both under on the space called QR. To do some of historical reasons. In the actual code implementation we use tell in the for space, are they actually the same? We probably were changing and changing naming later and on the right side. You can see how physically these objects are stored in ethnicity and to help customer developers easily to tell which space object is located e.

B

We introduce a new field in object, metadata section which is here. You can see the example, so this will use space to isolate talent and resources. This is what this is and also the most Muslim basica see how the space and Hanan together initialized.

B

We have a beauty in space. This is basic all the system space, so it's kind of read is a reserve, the name, any other tenants cannot create a space color system.

B

Oh all the system, level resources, like notes or some system cluster configurations. They.

A

B

The system space and we introduced a new idea object called talent here is an example Yama fire for the talent object.

B

Whenever a talent object is created in a system space, our talented controller, which which watches the talent object, we were kicking and it will create a corresponding space for this talent object, and it will also initialize this space with this default namespaces and also are the default objects. So this space will look like to to to a tulip element where look like the Li, hua kubernetes connection. So.

B

Here we and objects created in the system space and the correspondingly we initialized the three spaces for these three talents and the inside of the inside each space. You can have all the original kubernetes resources, regardless their namespace, our own namespace scoped.

B

E

Quick question: hmm does the.

E

Do these essentially spaces allow you to also create tenant resources of your own, or is it a flat hierarchy? Oh.

B

Yeah, for now we are focusing on virtualized kubernetes features. We didn't the introducer and new features like a hierarchy. Namespace. We didn't you introduce that, but the technically for the hierarchical namespace increment, it's probably can be implemented. It can be used inside a wall, space I.

E

Was specifically talking about the tenant resource that you have here? Yes, right so is: if I'm in a space, is the tenant kind available to me to create another one, and is there also a system space inside that space or are.

B

Those spaces, but it's a flat, it's a flat. Any object is visiting one space under the tell system. Space is just a Wang, it's just the first space that created by the system during class, the bootstrap. They are all flat and they are known they are no nested spaces. Okay,.

F

G

Joshua, oh yeah,.

G

Now I will say that, so what is the progress will create creating your namespace? Can you create it if you actually or you have to go through Saturday for the namespace of space, namespace, regular name.

B

You can create a namespace seeing any in any space here here. You can sync our space as a virtual kubernetes community, I guess.

G

I, don't quite understand why you need a CRD I mean. Why do you need a controller? Because if you have a building matter, I do either controller wait. Do that I, don't quite understand why you have to use a controller to create all the namespaces?

G

That's the purpose of of that controller.

B

This controller is a responsible for the space initialization and and cleanup, because when we create a space for telander, it's not an empty space. We also create a default name spaces like a kuba system default and some some other resources which I can talk about later. So.

B

We have now in our space and we can create a space for for talent and the talent that can create always resources inside a space, so alternates our isolation. But there is a big problem. Well, after we introduce this, we actually create api compatibility right because for now for other HR resources, you can see the resource. Url has changed. We introduced new way. I did inserted a new layer there to solve this problem. We introduce a new concept: cutta cutta, cutta half. This is really a key part in our design.

B

This, and this help us makes the whole IP I like about the compatible and make all our changes: trans almost transparent, 210 on the user's the idea. Actually, it's a visa is single. We we modified that we modify the on the aina pointer any pointer, handle. It seems. Our cache server user can use a full path to access the API object or they can use a short path to access object.

B

If this endpoint of resolution part could couldn't find as a target space in the UI, oh, it will get that information from the from the access credential associated. With this request.

B

This diagram example yeah. This.

C

Implies that every user it can only be a member of exactly one tenant. Yes,.

B

Because user for now, in our model a user belongs to and that teller is a part of the identity of the user. And can I.

C

Let's say logout of one tenant and login to another like in I switch contexts.

C

B

You can you can you can switch a contacts on the client side when you use so use a different credential or different different contacts to indicate that you belong to allow the tenant this resolution we automatically resolve tool to the other tenant space. Okay,.

C

So so like today, if I want to switch from one cluster to another, I can use the same credentials because I'm still me like there's no change of identity yeah, but in this model you actually need separate set of credentials for each tenant that you remember. Okay, yes,.

B

For for the multiple talents, the yin wonk identity- that's that's not addressing our current design way. We plan to start in the next stage. For now we assume we want any one user you know to one tenant is a very good point: okay,.

C

Thanks yeah yeah, like that's one reaction, I have right now is that I would want to operate like I would be a member of multiple tenants simultaneously. For example, like imagine, I'm a super admin and I'd want to be able to where's that only needed for this short path. Resolution like if I am yeah.

B

That's only for the pad resolution, you can always use the full path plug. That's any designer your space I see.

C

B

C

What every user has a default.

B

Yeah yeah, and if you are, if your user belong to a system, talent, neither characters as any I need that space. That's our policy. Okay,.

C

B

In this diagram, I give an example: we have a 2, regular talents to our energy to their. The issuer. I can request that you use the original IP I. The access resource same object for the pure in namespace and s1, but after this short pass a resolution, it will be automatically change to the full path to the corresponding object in in different spaces, but the tool to this Helen uses these are all transparent, which means they can use all their original communities API and the tools like a cupola and some old stuff. The.

B

Shuttle has the implementation heavily relies on the access control part. The way we did with.

G

Go back, I just have one question. So what do you think there are some other components like you know if.

G

B

Mean our controllers right, yeah, yeah yeah, like yeah.

G

B

I can I will comes at a net size, but for now I can I can quickly explain for all our controllers. They are running using a system credential and the second is to watch all the objects across across spaces.

B

Okay, it's also transparent or Twitter well, but.

G

But what if they want to use the short path.

B

In fact, they are using their initial pass, yeah. Well, when they least watch using clinical their most of the time they just append a new parameter called a telethon. All there were lists all the objects. If they want to access a specific yeah yeah, then they need to put it at ten. In the name is a a icon, but.

G

How could we figure out there, let's say cooperate? How does it oh.

B

That one using the API object, then let me give you example: let's say you have a job controller Aqualad right it. It needs to watch all the thoughts object, so it is still use the original code. But in the list watcher API call it use a additional parameter, correct and all the work at all the objects. And then we know when the process is in the video object. It knows which object belong to a specific talent, because we have this field. Okay,.

B

In the access control part for authentication, we support all the authenticators. We just add a new field in the corresponding Authenticator, for example, and in the third, where user will use a see him for the tenant name and the for talking and and look here. We append an additional field to indicate which tenant this this user belong to, and we put it in as the last field, so it can be compatible with the original identity definition format.

B

So so, after the after this change, we we know what Talent I use a belong to and in other, in the authorizing part. For now, the ROI is very simple: if I use a belong to system talent, it can access any space. But if I use a belong to a regular talent, it County access its own space.

B

We plan to support across talents access, but unless another time yet where we are working on that know all the other access control, the SDR performed by the current communities, access control mechanism like the rule and the clustered rule, so the code is changing here, actually is not big. It's very it's a very small. We just add some additional checks in the Bekaa authorizer or iraq. Riser like this like this. So am sorry.

E

Continue, oh, let's make.

B

B

You can me, do you have a cousin, yeah.

E

I was just gonna wait until later, but um if you're, okay with s and now a little curious about how the list watch works in the situation where you have custom resource definitions in different api's from the system scope, all.

B

You mean you mean that's the ideal intelligence spaces. Nothing is easy. Yes, so.

E

I have CR these infinite spaces and I'm at the system, scope and I'm going to make a request over all the tenants for a qvk. Did you think the list watch or something they could all have different schemas right? Yes,.

B

I have actually I have a long slide that that he came to for CID. We Dakota.

H

B

I will talk about that in you. I got a slicer thank.

H

B

Second, so this is a the general access control. I know this is a diagram, how we do the resource, access, combining access, control and the short pass it. So it's basically a person. We first check you for the target space here. If it is specified not if it's not, then we try to give in further space from the user identity.

B

Then, if it's already inferred, then we just check if it's across tenant access and then we go to the normal access control could pass, and with this rule and and cluster rule, we can talk about DDL later. If we gotta interested hey.

I

One question: so a lot of te kubernetes are back is all based on sort of users and groups. Yes, and here we are introducing the tenant explicitly in the authentication, yes away, so so what you are, is it going to be based on the LDAP.

B

Groups yeah: we did in the changes that apart we're only first up for user identity and come daniel has a you username and the group right. We still keep that we just add a third property which is tender and the you know a bag. We keep all the original code and we just add additional checks on this telling the property I.

I

Guess where I was going was sort of some sort of potential mismatch between the tenant and the group. Let's keep going, maybe I'll formulated. Okay.

B

And then now we talked about the clinical and the controller changes Putin or I talked about earlier was about the changes in API server right for, for the clinical, we add the tenant as an additional parameter in most of the CID methods. This part. Actually, we can optimize the design since the in the latest version. We introduced the context in communities clinical, and we also in defining a special and impendent at all. It's like a namespace at all, so you can.

B

So if we are, if for the controller, is running as a system credential it can cross, it can perform across space Lister watch for for all the controllers and schedulers, then they need to make some small code changes this here at least some example code. It's the kind of typical changes. This changes this all these changes are very similar. They are not the non related to the controller submissive logics it's mostly about.

B

When you listed up there you're interested objects, you use talent, all not all, so you can't get all the objects from our spaces and then, when you do a certain time, we need to object the queering, updating or correlation. For example, in our controller, you probably need to correlate a replica set object to its post object right previously, only based on namespace and the label select button. Now you need to put an additional comparison, compares a this talented field. That's all all that all the control the changes are very similar.

B

Actually, we already finished the change along about the 15 controllers. It's a pretty quick.

D

Do any of these spaces have ownership over all the objects to think absolutely what.

B

Happened me for sorry, I didn't get last part if you do in his face when we deleted space. Currently, we are following the same philosophy when we delete our name space, all the objects were already deleted.

B

There are some probably telling the data retention policy stuff, but the weight fit as that as a kind of outside module. Well, we don't plan to to do the inside in the kernel.

J

So this tenant all so that will really like allow to list all the tenants cross tenant or is there a way to selectively like define which tenant can be watched or listed? Oh.

B

First, first ETA: it depends on your credential right if we do not have permission to list all. Of course you are not getting the resource and the second. If we want to selectively list some resources from some spaces, you can add this is the watch filter based on the the field in object, the meta.

B

Is this one yeah you can, you can add a list of what you filter based on this field?

B

B

On this slide, I want to talk about how talent against self manage themselves, because we want to enable our talent Academy to manage its own resources of virtualized without a turning to the last item in every time.

B

So we kind of we reinterpret some concepts, for example, for the for some regular operation like creating namespaces the resource resources, the parts they are already isolated by space, so there are no problem and tenants can also reuse. This, like a class, draw or class the rule of binding or resource quota, it's all same to the tenant, Adam II, but these objects will only apply to the to its own space.

B

For example, if for a talent, admin creates a class the rule binding, this will only apply to the namespace within its own space, it's kind of transparently limited to its own scope, but but, as this whole thing's the transparent to the tenant at me, he doesn't know this has been limited. He thought his working on the whole entire cluster.

B

So, with this approach a tenant admin can has the full authority for its own virtual cluster. It under need to it under need to turn to the cluster admin.

B

And I've gotta mention a talent at halendar, a demean or a telling. The user can also install CID, and this were only applied to this talent and it will not impact other talents, and this helping I can introduce more details about you.

B

Previously, we have a question about the CID supported right in our design. We we already we're already implemented and the way we actually divide us the ideal into two different types of Saudis. First, we cover the screaming. First, we call the Patel in CID. This is all backward compatible. You can Italian that can install any existing CID and operators. Without any modifications you don't need to change the Yama fires or operator codes or recompile operator codes. It simply runs as little war. It was before, but only inside the telling the current Italian space.

B

If we look at the diagram on the left side, let's say we have three talents and each talent install are different CID. This society will only be visible to the current talent. It will not impact other talents but way during the implementation we realize there are some see ADIZ. There will be commonly used by all talents. For example, now some virtual networking solutions are based on status and some storage solution, also based on status. This is very likely to be used by other talents.

B

Technically, we can still have deployed them as a as a patella society, but imagine on a large cluster. Let's say: if we have 1,000 talents, then we need to deploy 1000 the copies of these operators. You know operators and the CID division. So to optimize this we introduced a new concept called a system CID for system CID. You only need to deploy one copy in the system space and then you after you test after they tested. We find it is working fine.

B

You apply a special annotation on the list, a ID, and then it will be automatically visible in all the talent spaces. The good thing is that, with this, with this system CID, you only need to deploy one copy. You don't need to run the operators in in all the talent, talent, spaces look yeah, but the requirement is for the operator. It'll need to do some slight um code, modifications to use the tenant on all to watch all the CR objects in different namespaces in different spaces.

B

This is very similar code changes for our bill team controllers, I.

B

Don't know if I addressed on the previous question know about the idea where or if they are more question about CRT yeah I.

E

Think a little bit well, the the question was particularly about in the left for tenancy, Rd. Yes, let's say: I have a controller or a user at the system, scope that is doing a list on tenant all for particular GVK.

E

Now you can't really be guaranteed that the schemas of those CRTs will be the same, even if the dbj is the same across the.

A

E

Right and I imagine their situations where a system CRD shares a gbk with a tenant that has the same DBK already installed at the tenant scope, yeah.

B

I think yes, this, if we were combined if it leads to what you talent at all tend at all for this, there could be a schema schema. Conflicts. Are differences, uh change, Chinese, online change, walking on this part and change I, don't know if we think of this case, hey.

F

Yeah, this is Chen and I'm the one who's working on CRT. Now it's like we are, we are working with so once we have a system CRD. We think like it's a it's, the one that what takes a control code. Basically, the system's the re work covers the preparing the CRT, so you, you won't see the pertinency ID. That's that's! That's our design! So far, yeah.

B

Yes, I understand locations even on the Patel and is the ideal situation user. The potentially can list the watcher or the CIDR see, are using tenancy or tenancy.

F

Id, like a test that a tenant user only have the visibility to its own space, so you don't need to make a change to your operator or CR, but, like oh, you can see it's just just this resources. The definition in your own space well.

B

What, if user, let's say, system user least least of this unit or tenant at all, that.

F

One is like so the system CRD as the desist antenna. Now, when he solicits listed out there, the customer resource he will see the custom resources and all the tenants, unintelligible yeah. That's that's it! So that's our behavior now yeah.

B

F

Think you're concerning about their the steamer conflict right. Is that it's a question so so, basically it so I would I say if, once we have the system CRT bases that one the designs the system one would be will take control because we want to once the system tenant deploy its the RT. We wanted to be consistent across the whole cluster, so it were kind of over once over.

C

Lightly and it correct me if I'm wrong, but I, think the question Nick was asking is let's say the two tenants have installed different versions of the same CRD, but they didn't like actually record them as different versions, so they they both happen to install a pertinent Ciardi called foo, and then the system user lists all objects that are of type food. In that case that the resulting list that you get will have resources from two different spaces, with two different schemas. Okay,.

F

C

F

Thank you. Thank you. Thank you for clarification, I call big question. Now it's like okay, the solution is like like two tenants, regular tenants. They have their OCR D and they they can different. They have their freedom to define their ceoddi so or same gbk. They have different schemes. In that case, you see in the picture on the left after the system. Tenants does not have idea on this. Cr D so for such tenant only see our T's.

F

We don't have their least watch because in system space you see there is no such a corresponding owners. Only tenants cuz. In that case, that's always said. Autonomy of your tenant, the regular tenant, have totally control. There is the RV's.

E

Yeah, so it's it's! It's a similar situation for like sub resources like if we used an analogy like sub resources of you can't do a list of the status across the set of resources. You have to get the entire objects. It's a similar situation here. I can't do a list across the seer the see ours for several tenants. Oh yeah.

F

Yeah I would say that's a very good comparison. Yeah I, like it yeah.

E

Okay, so then that makes sense to me. Alright! Thank you. Sorry for Google I.

B

Just realized I have used the 40 minutes, do I have more time or should I speed up. I probably need probably yeah.

A

It seems like people were really interested in the presentation, so maybe we'll just finish it and we'll rodolfo. If it's okay well have the conversation you wanted to have it. The next meeting is that okay sure.

B

Yeah because I see that the other topic is about the CID and ammonia tenancy, and maybe we can combine combined together. If you guys have more questions regarding CRD, we can discuss all the COS here. Yes, I hope that we can dress up like.

C

A lot of the the issues would be overlapping because I think what Rodolfo wanted to talk about was the possibility of adding names based CRTs to kubernetes, as it is today and I think you've run into a lot of the same kinds of issues like what happens. If I do a list across multiple names faces now, I'm gonna have different, schemas and different namespaces that that's I think it's good to talk about both of these but yeah I. Think at this point we won't have enough time to do rudolfo's, okay,.

B

Yeah sure we can I think we can discuss in more detail in the next meeting. I expect this will be kind of a long time kind of communication or collaboration. This is just today just our initial effort to establish the connection with you guys, and we want to have more discussion later.

B

And the so far we have talked about the control planning our design, but they are also isolating in the happening in data playing for the wrong time. For the wrong time, I mean the OS kernel. Isolation will use the cutter toast about that and the for storage. So far, there are no changes, because the most restorative providers already have provides the volume based isolation.

B

The most complicated part is in network because so far the kubernetes has a flattened network model. Each part is reachable from another pod right and relied on manova policy to the to the restitution. But when you have a multiple multiple talents sharing one single physical cluster, you don't want to the talents and worry about this IP allocation or IPA conflicts.

B

So we introduced a new multi network model and currently we are still working on that, and so today, I I don't have a time on to too much detail to share. We can discuss later. You forgot interested, but there's a key idea is that we introduced a new API object called a network. This is a kind of abstraction how an isolated Network this API object, type like it's like. As a network policy object, it can be implemented by any external network providers.

B

For example, you can use a VPC based service to implement this networker object or you can use some other updated, updated the flatten that program and politely to improve this and.

C

Our networks one-to-one on spaces or or attendance or you.

B

Know ensure yeah, there are kind of one too many in so each tenant had a space and in each space they are the default. Network object created and the talents can create a more natural objects based on each needs and.

C

What happens if two tenants want to communicate with each other? They just have to use a non kubernetes networking.

B

Yeah yeah for now, for now, we only supported this through the public are in public IP to communication that we haven't reached into that part is across Canada. Okay,.

K

Well, didn't the network attached definition kind of do some of this.

B

Listen adore gotta not to object definition for now. No.

K

No, no I know what you're introducing a new object, but there is something called a network attached definition that seems like Malta's and other things are relying on right. So there's already a network object. That I think gives you some of this comes for you. You could provide IP space and static routes and unique DNS and defaults, and things like that and you can associate them with namespaces. It felt like you could always use that and expand it to into to support spaces.

B

Yeah maybe week, yes, thank you for station I think we can check a more detailed on that apart, yeah.

A

B

Compare that okay, so for nano apart, it's a it's more complicated and we can talk about market here later for now, it's the kind of just over introduction on this apart yeah.

G

I think, whenever working the biggest problem is your your car psyche can be the same or different tenants, but the nodes are shared. So thank.

B

You and talking about class type, I or type II class.

G

Type II service, oh yeah,.

B

Yeah, yes, services, a tree, KISSING Hana in our model, each a so is also belong to a network, and each network can defines a class IP range. They are all kind of I did.

G

Yeah I know, but the problem is inside know that you know the IP table is not Canada. Well, unless you change that no.

B

No now we do not require an IP table could proceed depending on which network provider we use like, for example, now we use a V PC based provider to implement and an object on another way. It's actually not the traffic. Another come Institute proxy. It's going through the leads to provide a specific car.

I

It is the primary goal: they're wanting to have uniqueness of IPs or as long as there is a network isolation, it doesn't matter whether each one have its own IP allocation, like example, if if there was not, if there was a network shared between two tenants or two spaces, as long as the isolation was guaranteed between two different tenants, does it matter whether they're sharing a network or not? Oh yes,.

B

This is a good question because in our design we want to support a mostly demanding scenario which, in the public cloud in that a scenario each attendant shouldn't be aware of each other. So if you are sharing Wang Network, but you using some policy to limited access, one tenant I can still feel like as a existence of other talents, because some IP yeah, no, they are not. They are not available some idea available. They need to know which IP range it can it can use when it can detect something on a or two level.

B

In our solution we want to do the most strict, fixed isolation, email, I, Tula. There are two levels isolated, so.

I

Just one quick suggestion there could be that your overall multi-tenancy solution looks pretty promising, but it could have either shared or multi network model as two variations, so that if somebody doesn't have to necessarily have the multi network model in order to do the rest of your stuff, yeah.

B

Yes, these are well to the kissing actually for them network apart, our network is defined as a CID. We are wondering if this can be a standalone effort, just as they described some people probably want to use multi network, but the time the necessarily use the for multi-tenancy model. We are working on this hardly.

I

As somebody else said earlier, there is a multi network work going on in the networking sig, so you should look at that. Yeah.

B

We probably need to I know talking in some more thank you for, and this is our current status and next one quick.

L

So how are you associating the networks with it? Reports or the workloads are just sitting at an interface level, or are you insinuating at board level? Oh.

B

For all the like objects, like a pause service and networkers and policy objects, for now, we are using a special label who is label annotation I forgot to associate this object in visa with attach the network.

H

This is home way. I am working on the networking part of actors. It is an able.

H

B

Initially I, we wanted to use a new field to associate these objects under the work, but because we want to make this mot Network a kind of a stand along effort. So now, if I use name label so.

L

Label or annotation sorry yeah you're, using annotations to specify which network attachment definition to attach or you.

B

Know now we are using a. We are using labels. Okay,.

H

By the way, user neighbor has a one more advantage. It works very well with network policy existing my own policy. If we want to review them.

I

Okay, yeah I think a lot of things here that we can go into further detail in future. Calls yeah.

B

Yes, that's a big part of what I sing it, because a lot of the discussion took of all that it has okay.

C

One sort of high-level question which is to compare this with virtual clusters, and it seems to me and tell me if I've, missed something. The biggest difference between this approach in the virtual cluster approach in terms of features is that this approach maintains some kind of concept of sharing between the different clusters. You can have a super admin who can, by making calls to the same endpoint can see stuff from the different spaces, whereas with virtual clusters there's essentially no sharing at all like there's.

C

No systems, there's no equivalent of a system spaced, but other than that they appear to me to be quite similar. Is that yeah.

B

Difference is this is a cluster across the talent and management. The other big difference is I, think the probably more related to the resource consumption or something because let's say you have a larger class, you have solvents or talents, and you know we're just sharing wrong kind of one copy. Ups of the way, all right, we do not know how to solve all kind of separate the small control plane between talent, I'm, not I'm, not saying that that's bad I mean the pros and cons are for both approaches. Yeah 24 different situations.

K

Are we saying the difference between this and the other virtual cluster? Is it may be you some simplistic way of saying is in your MA.

B

Yes, it's a one deployment with the Unicode made. You could change into support, multi-tenancy the other model user. You deploy us more, basically, a small class controlling for each pennant yeah.

C

And that's I understand the difference in implementation. I guess what I was wondering is what are the features that fall out of that implementation decision and I? Think that the the answer is that in virtual clusters there is no concept of sharing, because each each API server is kind of a world unto itself. Where is in your model because they guess are assured, you've been able to add these features to allow the system namespace. Yes, yes,.

B

And we have potential to do all this kind of cross, telander sharing our management and stuff because they are using one kind of physical, okay.

M

Faith at all sounds description. The majority differences between everything.

G

And we share the same power arm with the networking. I probably asked with because yeah by the way, including our answer, one question. So how do you handle the old object thing? Come.

B

On yeah, this is this a very good question. For now we still keeps an OD in system namespace, so.

G

That's always a problem because not because you suppose a user can add annotation, so you are kind of escalate escalation. You the privileges, you that a user name space to change your system, namespace object, but anyway, I know: we've shared kind of sharing the same problem, but you.

B

Know, regular regular Helen, the users do not access as a node object, but as there are some kind of inconsistency because there they can still see the node name like the pod or some pod feels yeah.

G

It's kind of we before we do press, publish your Sara murder, we do it in different ways. We we create a virtual object inside the pen and soul. We have a one level of abstraction. Oh yes, yeah I'm, there I. Would this kind of Laura and counts the part? So would you trust the consume more resource? The benefit is, is dedicate dedicated control plan and you probably have smaller. You know the Duster you know can kind of radius of the the failure.

G

Radius is smaller because one can and see if they do something is get on. This only gets over not a whole thing country. Where you, your model, definitely is more efficient in terms of resources, but the costly is you have the shame in a tester where you have you probably reintroduced compatibility issues and you probably have to make sure other plugins have to make some certain changes. That is efforts that that integration efforts that I would say, may you know, have a may introduce the issues when people want.

G

You actually use it, but you need I think from multi-tenancy perspective. Some of the change or changes is the one that we have longed for a long time to be change upstream, so did you ever consider the upstream? You leverage some of your ideas. Yes,.

B

Way as I mentioned earlier today, we will first want to have the initial touch base with you guys and establish equation, and you guys.

G

B

We understand this is kind of a challenge, a challenge, a walk because not power, as you can see, not saw changes. They are kind of building changes like in the point of resolution, so we expect this kind of long term effort that if we want to upstream this feature other way are, we know we're.

C

A workgroup and, and so I think this is the good the right place to start this discussion, because the machinery sig would want to come to us and say well, hey what do you? What do you think does this because of course, machinery is one part of this change, but there's a lot more to it. There's also networking.

A

C

Many other and storage many other aspects of this as well, so so yeah, no I, think you did the right thing coming to us, not sure what the next steps are. I think.

I

This is really good because in theory this is in fact true multi-tenancy. It appears very in the shared control plane, but practically there are some ok. Scalability could be another concern because you're, presumably having one at CD with one control plane and it sort of has to capture the state of all tenants right, yeah.

B

It was actually well I listed earlier, second abilities, a lot of bigger and feature we are working on. We introduced a multi-point, DTD constants and we partition a cache servers and then we make a controllers active actual mode. So this that's another part, but if, if we got an industry, I can talk about them in others. You have.

I

A working demo, yes.

B

Yes, we already use the wrong and let me use several moments to quickly show a demo and for all the other details, even kinda, interesting I think we can discuss the King email are in the you know, subsequent meetings, I, don't think we can carve out details everything more meeting, mm-hmm.

M

B

I prepare a simple demo. You can see. I have I, have two terminal windows to act as the two different talent, two talents. They are talking to a same physical crust. The only difference is, you can see. They are using different cube controller config. Why is the tenant one while telling two.

B

So initially, then they look the same. They are using their under two different talents. Now, let's say if I create I. First first demos demonstrates that I create some resources in with one Talent. Firstly, you can see it's all transparent to the tenant users. It's like telling the human cannot feel the difference. It's just a user user, same command, Anais and Caesar, 17 and I say if I I create a space, a.

B

Seasoned even met named space, but on the other side on the other tenant is they are the same and now I create some parts, and the replica says how this stuff, just like what I, what I'm using we are using the native kubernetes cluster I.

B

Forgot, let me see it was a demo file name I.

B

Create a pod I create a deployment you.

I

Create the part in the cube system, namespace of one of the tenants, then.

B

B

One week reading the trip system right.

K

I

Doing this o or that's.

B

Yeah, you can see it's a crate. Is there.

I

But not in the other one's name, tube system on the right side, no.

B

No, if I said, if I get the time.

B

Yeah this is our total isolated. If I get a all the pods, then the path on the on this talent is all isolated, and if we on.

I

The left side, the kids, vanilla pod, is both in cube system and in default. No.

B

No, this is this. Pod is the one I created earlier if on M space, and if we look at this Yama fire, it's awesome. You don't need to make any changes. You steer you the crippled control for you, you don't feel anything difference.

B

So this the first thing I want to demonstrate how customer hotelling's isolated and how this changes are transparent to customers and there's. The second last thing I want to demonstrate is a some management work. Let's say if I now I'm acting as a talent at me now, if I want to.

B

B

This is all this time. I've got a.

B

Rocket and I prepare this to demo in a hurry I'd. Let me blame country, what's the what's the context name I'm using Oh tenant, another Johnny, okay,.

B

So you can see now, if I'm using tenant a demean together, pods I get a vicodin.

I

B

Okay, actually, as you can I just created,.

I

mmm I feel there's something we.

A

Changed the name to coupe system things, this way, yeah.

H

Yeah, the namespace change it you to cool system, let.

B

B

Oh I see I, see let necessary reason. Okay, I see cat it because I already created in name spin, now I'll create a wine in the default namespace you can see. I had a pause, but if I'm using regular talent, a user I.

B

Cannot get as a resource so now I'm 10, demonstrator a management tasks, I created the class, the Ruhr binding and at around, to the permission to the to the user.

B

If we see the disapproval binding, it's all the original kubernetes thing: we don't you don't need any change. This rule binding curanto, the user shunning the permissions. So now, if I select a path, I will be able to see it.

I

Sorry I missed the early part of your discussion. How do you guarantee uniqueness of the cluster rules, names and the clustering binding? First.

B

This all particular CF is a regime one space, so each attendant I can't create a creator. The Ruhr binding on raw objects without worrying about name conflicts right, including.

I

Cluster rules, yes,.

B

Everything MC rule rule binding cluster, drew a class of ID and the second in our a Beck authorizer. We only interpret this object through the current space, so so customers still use the same object like a clustered rule across the row binding, but the other one we're only taking effect on the current space, and this is all transparent to the to the total ends.

B

Okay, so 10 is now tenant. Other means I can do the self management, then that only they can manage their tenant space. They don't need to worry what about an epochal to the class. That means- and the last part is I- can create a I quickly put on some CID definition from the kubernetes community. Certain documents.

B

Now I'm creating see adding another in this talent and I can see this Seattle gets created and on this talent. If I try to deceive nothing and I can continue to create a thearc custom resources.

B

B

Can see these objects get created, see our objects can create.

B

Content yeah, this is a custom result gets created, but if I try to create this CRE sauce, other talent.

B

Into a company error, because there are no CID definition visible in that space they're all they are all these talents are totally isolated and they are transparent, they're they're, the walk as if they are working with the reason native kubernetes.

I

Greatly PA resources, sorry cutting APA resources, I.

I

Mean API resources make sure there's a hyphen between.

C

Somebody speaking very quietly an idea.

B

Yeah Chan, you probably need to come closer to the mic.

F

Sorry it's! This is Chan. Okay, so Johnny I said you just Cuba city power sources. On the left side, you should not see your contact, but you.

B

F

Side because we have Ciotti of chrome, tab, okay,.

B

F

We can't you look rap to see it.

B

This is a car on its.

F

Last in next to a last one, you see the crown to have.

F

B

A contact on tabs, let me on the left side.

B

B

Ok, this is all my demo on today. Talk.

I

Yeah very, very cool stuff, good stuff, thanks yeah.

B

Yes, we know they are not sour kind of mana tendencies, a complicated area and they're. Not so details under you because of it take is a native approach to make combination multi-tenancy. So it's a very challenge it for probably more upstream this. So today it's just about what you need effort to establish regime with you guys what? If we want to have more discussion in the meetings and.

F

Shani, maybe you want to share the last slide. There are some useful links there. So, oh.

B

Yeah sorry I forgot yeah, and this is some links. I can I can share the slides link in the email group, and this is our kind of project, repo design documents, roadmap and all this other stuff feel free to check out. It's all open, okay,.

A

Yeah, thank you yeah. Let's continue this conversation in slack and on the on the mailing list and if.

K

A

Have a lot of questions that come up? We can discuss them at our next meeting too, but yeah. Thank you for this presentation. It was great and.

B

Thank you for all the time. Thank you all for your time,.

A

B

A good question of feedback boy.

D

Alright, thank you.