Kubernetes WG Multi Tenancy, 3 Dec 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 20191203 Kubernetes Multi-tenancy working group

Description

- Go over proposal for Account Quota : Paul, Chaitanya and Shikha ( IBM)
- Kubecon retro
-- New OPA plugin using Core DNS! Ryan to follow up
-- Cruze’s multi-tenancy overview
-- Videos are up now!

A

Hi everybody and welcome to this week's multi-tenancy working group session. So today we are going to have a presentation from an tina IBM on their proposal for account quotas and then we're gonna. Do a coop con retro chica. Are you up.

B

Gosh, yes, I'm on awesome, so we we presented like few weeks back I, think it's not recording, and so how we are our use cases. Ibmc use cases on multi-tenancy. One of the use cases that we have is to get the account coder. So once you create the accounts which which we heard was CR debase and I'm trying to get the whole in from all the information on it.

B

That may be welcome back to you as to how was the status of it, how we can adopt it, but once you get the counts created or one of the requirements that comes to us is, how can we manage administer the account coder and that's that's what that's what we started looking into and we have a proposal that would like to review, will start on the implementation and would like to contribute back and figure out how and where we should start contributing account. Coder all. Are you online? Yes,.

C

B

So we have a slide deck but or we do have a repo where we posted the original use. Cases we'll add this document that Paul is going to present in the repo and share the link with you all as after the call so Paul. If you want to take through.

C

All right, can you tell me if I'm sharing the correct screen, you're.

B

D

Your wall of it.

B

C

B

C

New Mac had to ask me for some permissions, just a second okay.

C

All right, can you see PowerPoint now? Yes,.

A

C

All right, so this is a pretty simple slide deck that I just put together based on our design. So obviously, as she could describe when we're talking about multi-tenancy and having multiple tenants sharing the cluster, we really need to make sure that one tenant can't create so many workloads and things that they deny access to the cluster to the other tenants. So, from our perspective, the need is pretty obvious that we want to be able to control that as a cluster administrator to define yeah I've got five accounts that I've on-boarded to my cluster.

C

Each one's not allowed to consume more than five gigs of memory or certain number of milliseconds, or is it that sort of thing? So do this we're looking at doing this with resource quotas to reuse and leverage that ability already there in kubernetes? So as your tenants create new namespaces of their own, create new workloads of their own? We really just need to make sure that every namespace that they create defines a resource coda that limits that namespace to say a certain percentage or part of their total account quota, that's available to them.

C

So the cluster administrator would define an account quota says: everybody gets five gigs of memory as namespaces are get created. Resource quotas get created in those namespace that never total up more than five gigs per account, no matter how many namespaces they create, so it would be resource quota based. It would be based on enforcing. So that's what resource quotas do I, don't think it's possible at this time to define a soft resource quota at the structure. Barista quota is just the hard CPU limit or the hard memory limit.

C

Welcome is a soft option available yet, but it could be possible in the future and then, of course, it be extensible and by way of resource could is we actually started looking at this originally just from kind of like a pod memory perspective, but we also have requirements where an application may be defining other types of custom resources, and you want to make sure that no tenant creates more than 10 of those custom resources across their namespaces, so that actually became possible in communities, one that 15, which I think came out in June this year, where you could actually define a resource quota on object, counts for custom resources, which really kind of cements.

C

The idea that we want to use resource quotas for account level enforcement as well so real, simple, high-level design, we're going to define account quotas. We have a CR D for account quotas. It's going to look almost exactly the same as a resource quota, but it'll be defined at the cluster scope. We're actually going to have another CR D for account configs, which will bind the account quotas to specific accounts. So you can say account a uses.

E

C

Quota account, B and C, use that quota and separate it out, because you could have different levels of tenants based on how much they want money. They want to give you or whatever it happens, to be how much you like them.

C

So we are actually currently using our own iam provider to define accounts, since the kind of official kubernetes doesn't have the accounts yet, but obviously we want to make sure we design and implement so that, as the kind of official kubernetes accounts become available and accepted, we can instead of looking at our own custom, iam provider, or we just say, yeah, we'll just look at the accounts that are defined in kubernetes and then each namespace, an account must have a resource quota with a specific name.

C

If it's not there, we're to make sure it automatically is created with a controller and for.

E

Every attribute.

C

Of the account quotas, such as the limit on memory, we have to make sure that the sum of the memory limits and all those resource quotas for an account are always.

E

C

Than equal to the account quota, value and resource quarters also have scope selectors. So we have to make sure that if you'd find a quota.

E

C

The account level that says here's the scope, selectors all of the resource they get created in that count, also need to have that one.

C

So the orange tech tier is pretty much. The exact word-for-word was on the previous slide, so this just kind of going into. What's the implementation to actually make this happen, so we're gonna have a namespace controller that looks for creation, doesn't really care about updating or deleting namespaces utility namespace you're fine they're, not gonna, make break quotas as a result of that, but if you create a new account or a new namespace as a tenant, this controller will be notified after the namespace.

C

That created automatically kick in, create a resource quota in that namespace with a specific name and that's pretty much, it doesn't have to do any other calculations. It has to make sure that that resource quota exists in that namespace now. The second half of this is that we would have a resource quota admission controller, looking at creation, updating and deletion of resource quotas across the cluster.

C

So when that new resource quota gets created, if it's that named one that is specific to multi tenant enforcement is going to say, aha you've created one, it doesn't actually specify all the attributes that your account quota needs. It doesn't specify the memory limit, for example, so it's going to calculate or not to calculate so much, but it's going to check and see if it has that attribute, if it doesn't have it it's going to set it to zero. Actually, this is a slight typo. I've got this correct in other slides.

C

We do a quick check here to set it to zero for that attribute, so you created a resource quota without specifying a value. By default. It's just going to go and say yep zero. You created your namespace. It's not allowed to have any resources in it until you go back back and edit it again. If you do set a value in it, the admission controller is going to actually calculate. Is that value that you've specified that's greater than zero? Is it actually summing up all of your resource quotas?

C

It is still less than your account quota so that the tenant administrator who's creating this namespace who can go in and edit this resource quota. This will prevent them from editing that resource quota and setting their memory limit to a thousand gigabytes of memories available in this namespace, or something like that, if that's too high and again, if you're deleting the quota, it's been us about that and tenant administrators have the control of everything in their namespace, their administrator role there.

C

But the admission controller will make sure they don't actually delete the resource quota that supports enforcing the account limits and then periodically what we want to kind of resync to make sure that something didn't slip through. While the admission controller pod was stopped or something like that, and then kind of the final piece of this that we're still thinking through some of the details is what happens when the cluster administrator actually applies.

C

A new quota to an account that didn't exist before or reduces the quota value for an account and actually makes it so that count is kind of now in violation. So you used to be allowed to have 50 gigs, but now you're only allowed to have five gigs well, I've got a whole bunch of pods. Already running I've already got a bunch of all these resource quarters that say I sum up to 50 gigs. How do I make sure that something bad doesn't happen in this case, so we talked through some options on our side.

C

We said well what, if we kind of randomly picked the resource quotas in their namespaces and reduce the value until the account quotas actually met, it didn't really satisfy kind of randomly picking namespaces to clobber, wouldn't really make anybody happy kind of proportionally reducing all the namespaces has the same problem: I could have some workloads and some Dame spaces they're, really critical and that the tenant had their druthers, they'd, say I, don't want to reduce those I want to reduce these ones over here which are for development or whatever it happens to be so.

C

The conclusion we came to is that we really can't watch the changes to the quotas themselves. The account quotas and say when you reduce the account quota do something about it, but we need to find some way to inform the cluster administrator. You just apply the new quota value and these three accounts, or these ten accounts are now in violation. You need to go talk to them or decide yourself what things you want to clobber or tell the tenant.

C

You need to clobber something in the next 24 hours or I'm, going to start doing it or whatever the administrator decides, is the best choice, action based on their actual customers. So this one we're still kind of talking through some options here on the last slides I. Have it some questions which perhaps we get some feedback on what could be done there?

C

So this is a simple diagram showing where all the different pieces, so obviously we've got tenants down here with namespaces they create and the resource quotas as the namespaces get created. It triggers the namespace controller, which sticks the quotas in once the quotas are there. The resource quote admission controller makes sure that they have the appropriate values based on the quotas that are defined.

C

For that account, so all these different pieces and again currently we're going to be using our I am provider, but we would structure this so that we could work with the actual CRD based accounts going forward as well, and then this account quota controller probably won't end up in the final implementation because of the problems with actually doing something about reducing the account quotas after the resources already consumed.

C

So, for example, if your tenant cluster administrator creates a new account and said you can have 100 milliseconds, but just a number, your account user would actually log in create a namespace X. The namespace controller would create the resource quota in it. It would just be created empty and then the resource quote admission controller receives the admission request, mutates it to set limits CPU to 0 M and approves the admission request. So it's a second or two after that names is created. It's going to have the limits in it.

C

So after creation, zero megabyte or megabytes milli, CPUs isn't very useful. So after creation, the tenant user who has administrator permissions in that namespace can edit that quota and say yeah I'm going I know. My total account code is hundred know that what I paid for I'm going to make this namespace consume. Half of that and set my quota 250 mililiters, the emission controller says yep. That's just dandy is still below the total account quota and that changed go through when you create a second namespace. It's very similar.

C

It's the exact same thing you create the namespace, creates. The quotas puts the quota in as zero milliseconds allowed, and then the attendant administrator can decide all right I'm going to reduce the quotas in my other namespace to 25 milliseconds and then increase this one to 75 milliseconds. Does that still blow my account total or whatever balancing act? They decide to do based on their understanding of what the names basically be used for and how much it needs.

C

As I said earlier, account. Quotas are pretty much exact copies of resource quotas. They have the exact same structure except they're, defined by CR D at the cluster scope, rather than a particular namespace. So they'll have spec card limit CPUs by card limit memory, thought the scope selector same thing: that's provided for resource quotas, alright and that's basically the idea. So a namespace controller, a resource quota, admission, controller, CR DS to support those and define the limits.

C

Do you want to go in there apply those to a cluster, and you can make sure that the tenants in there don't step on each other's toes too much. There's still the issues of well one tenant can create a namespace called X and the next one can try to create a namespace called X and it won't let them I. Think I've heard that discussed before on a couple of these calls that that's a known limitation, but as far as resource consumption.

C

This should allow a cluster administrator to make sure that one tenant doesn't just go in there and create something that kind of takes everything and prevents the other tenants from doing their work.

C

So obviously we have a haven't implemented all of this, yet we're working on some pieces of it already, but we do have some open questions and definitely like any feedback. Anybody has on these, so first is I know. Obviously we can say there's a specifically named resource quota that we create called tenant in s quota or whatever name. We pick I'm scuse me whatever name we pick, but theoretically.

C

We've also seen that we could just say we're going to look at any resource coulda in all the namespace and if there isn't fun, we'll create one with the specific name. But if the tenant creates more resource quotas in the same namespace, we could sum all of them up. We couldn't really see any benefit from trying to look at all of them, but we'd welcome feedback on that. I'm, not sure if there's a have acknowledged, kubernetes pattern, where it's discouraged to use specifically named resources like that, but that's something we're still mulling ourselves.

C

We're still kind of thinking about what exactly to do with quotas that are reduced by the cluster administrator and I. Think the best answer is that we need to kind of find some way to informed in this cluster administrator, which kind of is this next one here, because we know that resource quotas in a namespace, you can say the limit of memory or whatever. When you do a coop control describe of that resource quota. It tells you oh yeah. The limit is five gigs of memory and it'll.

C

Tell you the current consumption level, so we're we're not quite sure how that happened. Yet we're doing trying to do some investigation on our site. But somebody knows how that happens. If there's some sort of webhook involved, where you can intercept the described, call and do a lookup of the cache of memory, consumption or CPU consumption in a namespace and add that back into the response, even those not part of the actual resource quota, we would want to do the same kind of pattern with account quotas. We're just not sure how that works.

C

Yet and then there have been some discussions about our Chrome namespaces on this call. I've only been on the call for a couple of sessions now, but there was an interesting presentation on hierarchical. Namespace is the first time I, joined and I. Don't think that hierarchical namespaces would have any direct impact on the account level quotas, because, as the tenants create namespaces, whether they're hierarchical or not, it's going to create them with a zero limit for memory and CPU and then lieutenant just gets to choose how they balance it out.

C

I, be able to think of any way that we would try to automatically say: oh yeah, if it's a sub namespace that we would inherit the quotas of the parent because they can't really inherit their cumulative. So from my perspective, I, don't think. There's any impact, but I'm sure others on the call have been thinking about hierarchical. Namespace is a lot more than I have in my testing different ideas and kind on a similar vein.

C

I haven't heard any plans, but if there was a plan or concept for accounts and sub accounts and sub accounts, that also could influence the design here that we would want to take that into account before we got too much further to see how the resources could be divvied up between sub accounts as well.

C

All right, then one more slide and then I'll come back to the question slide and see. If anybody has any other questions, they want to add to it or if anybody has the answers.

B

C

Write ahead, question.

B

On the technical side for design first before we go to the delivery favor any Tasha or anybody any questions on the design part- and my hope is that, once we have the account creation done and I, don't know where the status of that one is. This will kind of gel with that and it'll be just another extension to that. But that's the part I'm. We need to make sure that we are aligned on.

F

What account are we talking about these.

B

Are the multi so once you create a much multi-tenancy environment, the each tenant is referred to, as account is.

F

This specific, no.

B

F

An answer key yeah.

B

I think I think it's the it's I've heard it other places as well. Yeah, but you've been calling an account and tenant interchangeably, but you can call it tenant tenant quota as well. Okay,.

A

Yeah I actually originally had the same question, but then, as you went into it, I figured it seemed like it. It seems like you, don't actually have any sort of external account or it's not mapping to anything. That's called an account. It's more focused on just how you're managing quota yeah.

B

And it's for a tenant right.

C

A

B

Have to be yet it's.

C

Just the terminology we've been using internally a tenant is the person. The account is what they have on the cluster, but yeah. It's the same thing.

F

B

F

So I'm, the CEO of hierarchical namespaces and one of the integration points we were thinking of doing is is basically having the limit that all of the quotas in a child's namespace should sum up to less than that of the parent namespace. So there it wouldn't. We wouldn't use the inheriting mechanism, which we've used, which is kind of like the default behavior of hierarchical namespaces, but you could certainly use it to implement a very similar kind of behavior where, instead of summing all namespaces across a tenant, you would sub.

F

But all you would sum up all namespaces beneath a parent, namespace and so on and so forth. Recursively downwards. That was kind of the way. I was thinking that we would integrate the two and that and if we we will be talking about this later about having the tenant operator or the tenants here, do you actually use article name spaces under the hood so that each tenant would have its own root namespace, in which case these two would actually devolve to be the same thing.

F

Any thoughts on that.

C

So in that case, if your kind of parent namespace is given a quota of one gig,.

B

Of memory, yes,.

C

And the child namespaces.

F

C

Sum up to be a gig of memory, wouldn't that mean, if you have access to them, you've actually got access to two gigs of memory.

F

No I believe that all so first of all, not all objects are inherited from parents to children and, secondly, I'm pretty sure that resource quota, if you have multiple in the same namespace, I'm, pretty sure it's the intersection, so you get the minimum of what any of them to find other rights.

F

So if you had a gay.

C

You end up a scenario where the checker.

F

Hidden, oh I, think I see what you're saying what you're saying is that oh I'm, assuming that what you would put in the parent would be your new resource quota, not the the built-in kubernetes one and then what actually gets applied in any one namespace I suppose including the parent, would have to sum up to what is in the the parent, namespace I think I see what you're saying what here's I have a parent and let's say ten children's they've got a good memory and the parent in your quota and then, let's say a hundred megabytes in each of the children.

F

What I would have seemed to have happen, then, is that the controller would actually enforce a local limit of zero megabytes of memory in the parent because the children have claimed them all. Does that make sense right.

C

So basically, you're saying that, instead of how I described it as a kind of a cluster level quota definition, you would put the quota as a CRD resource in the top-level namespace for a tenant. That's.

F

C

And then that tenant could create sub namespaces and those are all because they are sub. Namespaces of the parent would be able to find that brief, that account level code, our tenant level quota and make sure that all the sub ones sum up to less than that one and then.

E

The parent namespace.

C

Itself would have zero enforced. You can't put anything in the parent well,.

F

It could put you could put something in the parent, but then it would subtract from what's allowed in the children. Okay,.

F

C

It would, although I'd worry about letting tenants, have kind of administrator abilities to edit and create things in the parents, because then they'd be able to potentially edit. The actual accounts is health as well. Obviously, other things could prevent that and with an admission control or something like that. That would be my concern with using that model, rather than kind of literally defining it at the cluster level, did only the cluster admin could see and edited.

F

Okay, I can think about that. Yeah.

E

This sorry I can so I think I believe are the article level for that American name. Space with the parent-child relationship, he'll, probably be still be an admission controller at that level to enforce to ensure that the children doesn't modify what the parent can actually give and over over you over subscribe for what the pin can actually also give. So that could be an add-on consideration at a point.

E

C

And with the hierarchical namespaces is a tenant who has access to a kind of a sudden namespace? Can they create additional sub namespaces under that initial one, or can they only create additional namespaces under the true parent test? There's like two levels or could it actually be in levels deep.

F

It can be and levels deep as our current yeah yeah yeah.

C

And that definitely complicates trying to figure out how to use some up the resource quotas well,.

F

That I mean to a certain extent, if you have, if you have the account quota as we'll call it or yeah they kept quota in a in a single namespace. You don't actually have to worry about the hierarchy beneath that namespace. You simply have to worry about the total number of namespaces beneath it.

F

It's not correct yeah I'll say you have to.

E

C

It out, but if you have a say, a sub namespace with a particular resource quota in it and then the tenant creates another namespace under that one concept when I started thinking about it, I was thinking. Okay. Well, the tenants going to want to divide up the resources in the parent but into the sub namespace and treating it essentially as a flattened tree kind of takes away some of the value of it being hierarchical, because, yes, because essentially flatten the tree and to sum up the recorders in all the namespaces, but in doing see sorry.

F

C

Couldn't think of a good way to handle that ain't cuz, it's partly just my conception. If I was a tenant and I've got a namespace with 100 gigs of memory available in it and I want to create sub namespaces each one with 50 mm-hmm.

F

C

Didn't quite I couldn't quite map out how a user would expect to create one reduce the parent 50, make the child 50 and then create another child 50 reduce the parent to zero and then, if you're kind of one level up from that and you're the one who gave that namespace to a particular user. Originally, you gave all my hundred gigs of memory.

C

Well now it's got zero gigs of memory in it. Why is why did that happen? It seemed conceptually hard to manage, but you could simplify this by saying if we're going to treat it as a as far as resource quotas go just treat it as flat.

C

It doesn't matter how many layers deep your namespaces are it's just flat, but the under the entire account, but in that case it didn't seem to intersect with a clerical namespaces as much just to find the account quota at the cluster level and apply it to the tenant or to the account yeah.

F

I think if I were to map the the sort of agency idea to what you're talking about what you're asking for, is the ability to put what we're calling an account quota in each of the hierarchical namespace and have it only apply to that subtree. And you know like that, is something great.

E

F

And then we're using the built-in resource corta like the kubernetes more, is an implementation detail. It's the only reason we're still in the because we don't want to go write our own. Let's say universal admission controller is that well.

B

F

So yeah I guess what I would say then, is that when I was talking about, the flattening I was assuming that at a certain level beneath the tree, you wouldn't have additional account quotas, but you could have different resource coders and at that point you can just flatten it. But if you also want to have an account footage, some of those in.

E

The hierarchies then.

F

Then you you, obviously don't flatten it that you can't fight in at that level. As you say, you would have to do some kind of hierarchical summation. So beneath that you would have to make sure that I feel like at this point. We either need a whiteboard or a dock.

B

And Paul and Pasha I got a drop but Paul I'll sing with you later.

C

Okay, thank you good yeah, and that's really. Why I had this question here about sub accounts, because if we wanted to have account quotas below other account, quotas below other account quotas, that's when it get starts to get complicated and I wanted to see. If we were leaning that direction as a community worth, we were really looking to go that direction. Yet I.

G

C

G

My perspective, it really only gets complicated if you take the onus on yourself, but to me you're, providing an account to an end user and you give them a chunk of resources. And, however, they want to do that up amongst us, they're their sub resources or sub tenants or whatever that's kind of on them, and they have and as long as all the resources under that account don't exceed the limit.

E

G

It doesn't matter ain't.

C

Yeah, it just takes a bit more implementation to be able to think about that. Yes, your tenant can create an account, enter an account quota of their own and any namespaces below the namespace, where they create that one use that account quota I can kind of picture the implementation, but it does get complicated to think about the.

E

I think one of the question is how deep, how deep is the errant clean space? How many negative.

F

Well, I think that their current assumption is that this would be mainly applied to the tenancy Rd and not not to hierarchical namespaces, although, as I said, those two might be devolving into the same thing where tenancy is basically a thin layer on top of it on top of agency or not a thin layer. Actually it's what say a medium-sized layer, but if you were to do that, we could simply say that I mean it is actually quite simple to stop a tenant from creating one of these things themselves.

F

So I do is you you don't create a role binding but allow them to create or.

E

F

This kind of object, so that way, only the cluster, the cluster administrator, who is also creating tenants, can.

E

F

This thing up and then nobody else can create them, and so that is one solution to that problem.

C

F

What is that? How does that sound to you like it sounds like that could be an initial implementation anyway,.

C

E

C

I I think that having can hierarchical, tenants as well as hierarchical namespaces, definitely would be more than they would want to fight off in the first pass and while I think nothing we've discussed would prevent that in the future. We certainly could prevent it in the first pass, with role bindings. As you say,.

F

Useful place to start or any of the is sanjeev on, or they they, the ones who've been doing the most on the tenants ear to you, because I'm representing a course the HMC, although of course were all one big family, but it would be nice as somebody who works more closely with the tendency. Rd were here to talk about them.

A

G

I've been doing most of the PRS for to Tennessee, ready and I. Guess. Development is kind of slowed down significantly over the last couple months, because we're kind of waiting for agency and how it goes and how we can incorporate it so right now we basically have the POC that we've basically been endeavoring for a while I mean yeah, I, guess yeah I, don't know how I think this is I mean.

G

This is exactly something that we wanted to implement, impotency repeat but again, and all these these issues and questions that would have come up if and it prevented us from doing that, and so we were kind of waiting to see what agency gives us and what we would need to apply on top of it. So.

F

Given that does it make sense to at least think about how a single level of what we're now calling account quotas might work in the context of HNC, because that would also suit the needs of the tenant CRD, which only supports a single level of named spaces underneath tenants and then, if we later need to extend that to be fully hierarchical.

F

We've got a fairly straightforward path to doing so, straightforward in concept, but infancy, really an implementation sure.

G

Yeah I mean I, guess my one way to think about it. That I'm from a hierarchical perspective is if the parent does not have an account. Kota than a child cannot create an account Kota, and in that regard, the only thing you're doing is subdividing the parents account Kota and the sum of all of the children can't you know, exceed the parents for an account quota perspective. That kind of allows you to have as many are as deep of hierarchy as you want, but simplifies some of the implementation. I think great yeah.

F

And we'll need to make a couple of changes to agency to support this, but definitely.

G

F

Those are changes we were already planning on making, which was to allow you to be able to configure what what objects get copied down and what which ones. Don't you know, was for exactly this kind of reason like right now we're copying resource quotas from some parents of children.

F

But if this becomes a thing, then we probably would want to stop doing that. We'd want to turn that off I've been either cluster wide, or certainly within the certainly within the any subtree, with an account quota, and then you could and then and then that responsibility would fall to the account quota controller to make sure that everything sums up properly right.

F

C

Extent that brings me to my next slide. The last slide I had was kind of how we were picturing delivery, because obviously right now we're not working with the tenant CRD or anything like that. We're using our identity and access management system, which has an account concept already to do our accounts so have our first pass. If we were implementing, this will be to work with that, and then we would work on. How do we make sure we can inform the user about the or the cluster administrator about the current usage of each tenant?

C

And then once the kubernetes native accounts, multi-tenancy CRD is more widely adopted or as soon as it's kind of officially adopted, we would want to make changes to work with that, but not quite sure what the schedule is for when I kind of win would tenants, erd move kind of out of alpha or beta or whatever it's currently in to say, yeah. We should be developing against that at a certain time,.

F

It sounds basic friends saying it sounds like that is more.

F

It's kind of bond holds to discover where we go with H&C and the agency I'm thinking we'll start getting to some kind of real States in the first half of next year. Hopefully before that, but it's hard to start to kind of stay on it right now.

C

Yeah I think that's essentially the same timeframe that we were looking at we're hoping to have at least some parts of phase 1 implemented in our own stuff in the first quarter.

C

So it sounds like it would probably be appropriate for us to proceed with that implementation for our own side and then continue working with community, obviously to get feedback, get put demos out there and adopt the other pieces as they start coming in and making it. So we can actually contribute this back to be used outside of our own company.

C

That's the impression I get so like I said I've only been on a couple of these calls so definitely feel free to correct me. I.

G

Mean I think that's great I. Think addition, if you I mean if you don't want to wait and you see some areas of the tenancy OD where you guys think is your bit to help move it along. That's also greatly appreciated you.

F

Need accepted so what's the path forward, are you gonna continue based on the tenants, you're you're gonna have another look at each and see if it's the latter I can certainly give you guys and help you out on that. If you want to stay with the tenancy review, that's also fine, but one but yeah. Just let me know what you, what are you thinking or do you need to discuss it? Yeah.

C

I think, obviously it I think we'll need some changes to what we've spelled out here in this PowerPoint, so I'm not sure, yet whether we would kind of go directly forward with putting the account level or tenth level quota in the namespace.

C

That would be like the top level namespace for HNC, or you recall, I we're currently working with our I am provider for the account definition right now so I'm thinking. Perhaps we would look at say the account config that we described where we bind a particular cluster level quota to a particular 10, our account. Maybe that should be the resource that gets put in the top-level namespace for HNC, so you could actually have one quota defined at the cluster level that shared among multiple tenants.

C

It's just the binding to that quota is what goes in the agency top-level name space, rather than exactly how I have it laid out here so I think we'll definitely need to work together and adjust what we're doing so that, as we get further down the line, it doesn't have to go undergo major changes to work with everything else in the community.

C

At that point we obviously want to start early, so yeah I definitely be happy to meet with you again directly and kind of work through some of these design decisions before we get too much further along okay.

F

um That sounds good. You know how to reach me on slack and whatnot I.

C

F

My slack name to the best of my knowledge is the same as my real name, which is Adrian Ludwig. That's how I show up to myself anyway. It's not that then it's Eli doing there's something similar I'm, also on github as the Adrian blood one.

C

Okay, I will see if I can reach out to you that way. I do see your name in the meeting, so hopefully.

E

C

Will be able to follow that chain and she can might if I run into problems. So hopefully she could we'll know how to do that as well. Yeah.

F

Plus my name, my email address is all over the design box too, and you.

E

F

The rest of this week is a little bit busy for me, but the week after it shouldn't be that bad and the week after that, the one before Christmas is even better I have very little to do as I will be traveling but still available. Okay,.

C

We'll be in touch. Thank you. Okay,.

F

C

Alright and don't think I left groom on my last slide for quote anything else, but are there any other questions or other areas that we haven't discussed that somebody like to throw out there before it falls out of active memory.

C

A

C

How stop sharing? Thank you very much thanks.

A

Paul yeah super interesting overview cool, so I had said that we were gonna. Do koukin could con retro as part of this meeting, but we only have 15 minutes left so I guess I'll. Just at a high level say you know, kind of open the floor up and just ask was anyone on the call who got any who went to any great sessions who took away anything really great from kook on that they'd like to share with the group learned about any new projects or just wants to sound off on anything?

A

The platform is yours, so yeah.

F

There was this awesome panelist named Tasha at one of the events.

A

G

I did actually find something quite useful that pretty recently occurred, enos implemented a new plugin for OPA policy decisions based off of DNS requests, which I think could be super powerful from a multi-tenant perspective. These.

E

Call having worked requests.

G

A DNS request, so DNS lookup core DNS, can can send that request to OPA to make a decision on what to return or whether or not to allow that. Basically, that request.

E

G

So I thought that was actually quite interesting and worth looking into further. Those might the most exciting thing I saw I. Think.

A

Okay, cool yeah, I wonder if we could get him to come and present on that to our group. Oh.

G

Yeah and if not I've been playing on the pillow.

A

Cool yeah yeah, put yourself on the agenda Ryan and then see if you can can convince the person to come and if not you're up great.

G

A

Cool anybody else there.

F

Was a great talk by Cruz, formerly GM Cruz, about how they do multi-tenancy and I attended the second I'd say 2/3 of it, I missed the the beginning. It was packed.

F

I know you, you asked a question at it, and I need to go back and read I jest everything from the YouTube video and then maybe that would be another good one, actually that maybe this is a ideas that can get a list of some of the multi-tenancy related videos and a small quick summary of some of them and post them somewhere. Yeah.

A

That's a really good idea: I usually dig out ours and add them to our list and then sometimes I'll add other videos too, that are highly related just to there to our YouTube channel but yeah. If you wanted to do a blog on it, that would be cool too. We.

F

A

Know like a medium post, you know.

F

Do we have a medium account? No.

A

But we could, you know the ropes.

F

Are ulster cool yeah.

A

Awesome well yeah, so for everyone who didn't make it out, the videos have all been posted already and I have not done any multi-tenancy curation to our YouTube account yet. But if you watch one and you're like this is really awesome, multi-tenancy content definitely shoot it over and I can add it to our YouTube account.

C

Speaking of adding things to account, I've had a couple of requests from the slide deck that I went through today, with the easiest way for me to post that twits. The group usually.

A

If you put it on Google, slides and then drop the link to Google slides in our meeting notes documents that should do the trick. Okay,.

C

I think I've got you only access to the meaning, that's things so I just need to click to quick yeah request access. If.

A

You join the mailing list and you're logged into your Google account that's member of the mailing list. Then you should have edit access.

C

Okay, thank you. I'll try to do that and.

A

The way to join the mailing list is just to go to groups google.com and then search for multi-tenancy working group and then just click. The join button.

A

Cool okay! Well, thanks everybody great meeting a great overview. Paul! Thank you for going through that with us and.

E

A

We'll see y'all in two weeks, if not before,.