Kubernetes Storage SIG COSI, 6 Aug 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes SIG Storage - Object Bucket Provisioning Review Meeting 20200806

Description

Meeting of Kubernetes Storage Special-Interest-Group (SIG) Object Bucket Provisioning Review - 06 August 2020

Meeting Notes/Agenda: -
https://docs.google.com/document/d/1KTh1y9klby64t7btNULtxLWDkRC9SAWE-SZnJeFZqug/edit?pli=1#heading=h.5xeufhnfakeh

Find out more about the Storage SIG here: https://github.com/kubernetes/community/tree/master/sig-storage

A

That, just as a quick brief introduction, this is sort of our third focused review meeting where we're seeking um approval of the cap get it merged, get to uh the opportunity to use the official repos for the uh prototyping and and and and future code. um The kep is big and um it's long, it's difficult to go through it all. In a meeting.

A

um There were some great points brought up at the previous review, which have been addressed and should be visible in the current version of the kef, um and we said, came up with the idea that maybe a a simple focused slide deck that summarizes the cap will make it easier to digest and get through and with uh with five to ten minutes left at the end of the meeting were were striving to get to the point where we could do a vote.

A

At least a straw man vote, see where see where people are are leaning toward how close they are to approval all right. Sid. Are you um co-host.

B

A

B

um May I suggest that if we are going to put together some sort of material to help people understand at a high level, the cap and the associated functionality that it be grounded in the core use scenarios uh that are talked about at the beginning of the cap.

C

That's exactly how it's done. I mean that's how I've done it today, um I'm glad you actually brought that up, because that was my intention behind making these slides.

C

Main thing is, I think, in order to give a high level complete picture of what this proposal is doing from the perspective of one, the user, that's going to consume it and to the vendor, that's going to write the driver for it. um One of the things we I felt like it was hard to cover last time was the grpc spec, which is the contract between the vendor and uh kubernetes bucket apis.

C

So let me start by sharing my screen. Are we recording niching yeah.

D

E

F

E

D

Thank you. No problem.

C

Okay, so uh I'm I'm going to start out uh this slide that really really uh in a basic way quickly, we'll jump into the details um I've taken uh for for explaining uh some of the workflows. I've taken the simplest, successful use case uh and and uh gone forward at that. You'll you'll see it shortly, so so we're defining a new standard, I'm starting with the absolute basics, just like the definition for anyone that wasn't a part of this uh all these months.

C

Just so just so, we clearly explain what you're doing it's a standard for consuming object, storage and kubernetes. It's basically like csi is a standard for file and blocks we're creating a new one for object, storage, we're calling it container object, storage interface, cos si or cosy yeah, and uh just to reiterate what object storage is I'm sure we all know what this is it.

C

It stores data as immutable logs within organized into buckets uh and bucket here is going to be a primary unit of management in in in our system that we are building, we provision deprovision grant access all at the bucket level.

C

Now we are going to do it across multiple object: storage, vendors. uh In a vendor agnostic fashion and uh let's get into what these uh four things mean, provision grant revoke and deprovision.

C

So, let's start with provisioning, provisioning is basically creating a bucket and providing it to, uh but just creating a bucket or or checking the bucket exists in the back end. um So what's the back end here, let me okay, so so you've defined three uh uh separate components here.

C

uh The one on the left is the user or admin middle is the cosi, uh which is our system, and third, is the back end, which is the cloud or the object, storage provider that actually creates the bucket the workflow is a user admin creates a bucket or requests a bucket to be created, and the system understands this request somehow and then communicates at the back end to provision this bucket.

C

So, let's, let's look at what the contract between the user and the user or admin and and cozy, so we've we've created a so one second, so cozy runs within kubernetes and the declarative standard way of communicating with kubernetes is through the objects, kubernetes objects and we've created uh in order to model how user admin requests a bucket we've created a bucket request resource.

C

Now this resource has fields required to request a bucket in the back end as a name. It belongs to a certain namespace, whichever the user is originating, the request from and our namespace the user has access to a protocol which says what kind of object storage protocol do. I expect about bucket prefix some form of prefix to say this is how I want the bucket name to start and a bucket class name.

C

I've. I've created a dummy kind of use case where the name of the bucket or the name of this request is something called profile. Pictures under the namespace profiles and um the bucket class name is write one's readmany to to explain a scenario where you create profile pictures. You read them many times.

C

You don't delete them, though, so here we showed that there's a bucket class name bucket class groups, a bunch of bucket requests uh based on certain parameters in this in in a bucket class, we have a provisioner set, which is the actual vendor specific code that will go and create the bucket in the back end again, protocol is set here which tells you what kind of protocol we expect the bucket to uh serve.

C

Then we have something called this: the anonymous access mode, which sets the access policy on the bucket itself. A bucket can be either a public worker, so everyone can read and write or read only bucket self explanatory read, write or private where by default nobody gets access to it unless access is explicitly granted to it.

C

A release policy says what should be done with the data in the bucket after this bucket is deleted from kubernetes, not the back end. Just kubernetes allowed namespaces is uh the list of namespaces kubernetes namespaces. We want this bucket to be provided to so when requesting the bucket um so say for in this example, a profile picture bucket would ideally only be relevant in this made-up scenario, uh say it's only relevant for teams that work with profiles, and let's say they- they use an install profiles, and so you set the allowed namespaces to this.

E

A quick question on that: why is the uid there so we've.

C

So all these weeks, this was something- and I had as a question were worried about the scenario where someone deletes you a namespace and then creates another one by the same name.

E

Yeah, I think in that case we wanted to apply by name not necessarily the specific instance.

C

I was uh yeah, I I thought the same as well, um I'm explaining it exactly as uh we've uh we've put in the cap. One of the reasons being I I came in uh kind of midway so uh in, but I completely agree with you. It should be just the name and if someone does do that, where they delete the namespace and bring it back in, it should apply to the new one. So.

A

And we got feedback from nicholas on this too thanks nicholas assan. What you're saying then, is really it's the string of characters, the name that matters not an instance of the name space.

E

uh Right in general, that's what we care about in kubernetes, there's very few cases where we actually care about the uid where we care about the uid is things like the persistent volume claim and persistent volume objects where the specific instance actually matters, but outside of that, it's generally the name. That's.

A

Great, I mean it makes it simplifies, it makes it easy. So we will. We will um include that suggestion.

E

Cool. Thank you.

D

Absolutely um then, we have sorry a had a one question um so so in general, in kubernetes, uh there's a concept of kubernetes admin, creating a namespace and uh assigning some kind of a storage quota and stuff like that. So um how can a could have been limited for objects uh created using this object class?

D

um Is there any quota management going on here.

C

Right now I don't know so, someone answering please go in. I.

B

Was just saying that's out of scope right now, but that could be an enhancement at some point in the future.

C

Yep, okay, yeah, so parameters here is a generic map string string uh structure that lets a cloud provide a vendor specific parameters to pass be passed in. um In this case. I've just set the region, however, like if you look at gcs, there's a project id and or azure has a project id and another container name.

C

Each each cloud provider has its own set of parameters. That's modeled in this structure going forward.

G

Allergies, for can we just go back to that? Yes, thank you. Can we get a clarification on what the purpose of the uid is? Please.

C

Oh we're going to remove the uid. That was this question. We just had a second ago.

G

So there will not be any uid on any of these bucket requests.

C

But yes, the original intention was uh here that we have a list of allowed namespaces. The idea was only the namespaces that we've deemed uh as safe for uh you know safe for providing uh access to this bucket, or these classes of buckets um should be allowed to do them uh should be allowed to access them. However, um our concern was if we set the allowed namespaces to a bunch of namespaces and someone goes ahead and deletes those and recreates a new one with new credentials. They shouldn't be allowed access to these buckets.

C

However, that's that's non-idiomatic to kubernetes, so.

B

C

Go ahead and remove it.

B

Yeah, just just so everyone's clear, what this is stating is any namespace, with the name matching what's specified in the bucket class can gain access to the stored, object, storage, credentials.

G

Yeah yeah, okay and then the um namespace can have multiple labels as part of the metadata correct.

C

We're not yes, that is correct, however, in this case, we're not selecting a namespace by labels, we're specifying the names uh directly explicit. I think that should be documented.

A

Yeah: okay, what's confusing there john.

G

Just the um whether or not the bucket class could be tied to a namespace.

C

Yeah, I think we can correct that in the documentation for sure uh we can. We can add a statement clearly saying how we're going to represent this um we're going to move forward with this. Thank you for your patience here. Oh absolutely, no worries um so uh so earlier. We saw how the user admin communicates cozy once the bucket requests and bucket class are provided to the system, because it goes ahead and processes it before translating it to a request that the backend understands and what it does is it takes the bucket request, object.

C

I've just put the relevant fields here and then it takes the bucket class object again put the relevant fields here. Basically, all the fields and respect for both of them and I've marked the bucket class fields with orange and bucket requests in green and my my mouse scroll bar is super sensitive um and I don't know how to fix it. It's the apple magic mouse. I'm just going to go ahead with this, so the bucket object is created by the system.

C

This bucket object has the orange set of fields from the bucket class and then the other set of fields from the bucket request or they're auto generated. So the provisioner is from the bucket request.

C

The release policy is from bucket class anonymous access, modbus class again there's a bindings field we'll get into it. In a few slides um allowed. Namespaces remove the uid parameters comes from the class, and the protocol structure generates a bucket name based on the prefix. That's provided and adds, as you ready to it, and then, when it, when it first gets created the status of the bucket.

C

It is it's not available.

B

And the intent of the prefix is that, if two bucket requests come in using the same bucket, prefix they'll both be assigned globally unique names, at least in the context of this, is that correct.

C

E

One minor optimization we might want to make is on the status.

E

I know we talked about uh the uh using conditions, uh but instead of conditions we may want to just use boolean values here things like bucket ready and then you could add additional booleans in the future for any other kind of binary state that you want to add. As long as it doesn't invalidate the previous state.

C

E

C

Shang also mentioned it, and uh I I think that makes sense. um We just went ahead with the usual way of doing things uh with the conditions. However, there's no reason why we couldn't do the.

E

Direct yeah, that's fine. I think uh we could fine-tune that in api review. It's not a huge deal right now.

A

I'd like to call out just for people looking at the official cap that under conditions, the reasons that have been stated are are not actually what we were thinking of doing and then, if we eliminate conditions all together, it's it's as long as you know that those reasons were a first pass and not what we actually think right now makes sense.

C

Yeah I'll go with them: jeff, okay, thanks yeah yeah, uh all right. So so we saw what cosy did so. It takes the bucket request in the bucket class and creates he creates a bucket object. Now, then, the bucket object is passed along to the vendor side of the the back end or vendor side of the system where um we use a grpc protocol.

C

The grp pc protocol defines a contract between cosy and the vendor library of the vendor driver or the provisioner, and here's what the protocol looks like and- and these are the fields that that driver requires to go ahead and create a bucket in the back end. So it needs a bucket name um needs a bucket context for different cloud providers.

C

We have a different set of options that need to be passed in, say, for instance, project id and azure, that's captured in this bucket context and then the anonymous access mode, which is access, control, set on the entire bucket.

B

And we have, and the map string string is in essence saying you have one or more key values where key is a string and the value is a string yeah.

E

Is a bucket context here mapped to the parameters from the class? Yes, okay, uh could we consider renaming this two parameters to match that.

C

Yeah, absolutely, I think, that's that's not yeah. That's definitely possible cool. You can do that. Yeah, uh hey is, is the shrinier jeff taking notes on these uh suggestions? If you could do that, that would be great.

H

Yeah aaron and screen taking notes: okay,.

C

So uh the fields uh for uh you know filling in these fields. For the example we just showed. um The bucket name is the auto generator one profile: pics dash, uuid uh bucket context, parameters, uh region, usc, spawn and then the anonymous bucket access mode set to private, which is what we wanted in the bucket object.

B

And at this point the dispatching is already determined that, for example, it's going to an aws s3 provider. Yes, so that is decided based.

C

On the provisional that is set.

B

Yep yeah, so the invoking of the correct provider provisioner specifically what happened at the earlier stage and the correct fields will then be passed down. Yes and what happens if um a provision or a back end can't be found that maps.

C

Then the bucket object remains and bucket available set to false.

B

Indefinitely um and it will keep trying until such time as the back end uh comes into existence.

C

Yeah back in that that advertises the specific provisioner uh comes into existence. Yes, okay, good uh and the response, if everything is successful, is an empty uh response. If there is an error, there's a separate error channel in the rpc, along which an error would be passed, um I'm going to go ahead without going into the errors. Since I'm only explaining the easiest simplest successful case here, we will get into the errors as the second step of this presentation, um but for now we'll move forward with the provisioning.

C

So once that successful response is returned by the vendor, provisioner back to the cosy system, we go ahead and set the bucket available to true on the bucket object, indicating that the bucket has been provisioned in the back end and then what we do is percolate that back up to the user by setting bucket available to true in the bucket request, object.

B

Okay, so the um that single, successful indication uh going back, that's kind of implicitly synchronous in the grpc protocol.

I

C

How do you mean the grpc protocol simply just uh informs that it's done and then there's a there's, an asynchronous uh set of steps that leads to this.

B

Okay, so that was uh that was the question that I had. If my provisioner takes 10 minutes to set everything up, um will the grpc call just be left open for 10 minutes, or will the gp rpc call return immediately um and then there's some other indications, some asynchronous indication that then triggers the bucket available being changed to status.

C

Oh that way, um so the grpc request has a timeout. So let's assume that the timeout expires, how much every time it takes to create the bucket is longer than the timeout. Then another status is returned and a bucket available is, uh you know, continues to be false, but then we keep retrying. So when error status occurs, we push the request to the back of the queue uh and uh the queue of requests that need to be served and uh when it gets retriggered again say the bucket is created.

C

At that point we get back there. A response saying already exists.

B

That pattern's fine, with one caveat the provider, needs to be able to determine whether a subsequent request is exactly the same as an earlier request. It's still working on.

C

Yeah, so so that's that's done using the bucket name.

B

Yup, which is guaranteed unique for each bucket request,.

E

I think uh somewhere in the spec, you just need to say that all the grpc calls for uh cozy need to be item. Potent.

J

Yeah- and we have to be specific about which fields must you know or can be dependent on for item potency, yeah yeah.

B

J

Think this is pretty straightforward for.

B

That but I agree that that needs to be documented.

A

Yeah- and you know david- it's a good point- you brought it because um the bucket name can be the same for different requests. There could be a green field request and right behind it, a brownfield request and they're both the same bucket name, right and they're, trying to do green to brown and um and so a driver could get the same bucket name provisioning. The bucket food takes a long time. You can get food coming back again, but this time it's a brownfield request to food right yeah we do have to handle that.

C

Yeah, it's item yeah. We will document that.

C

So yeah at this point is where provisioning ends. uh Provisioning is strictly creating the bucket loan. uh You know in case it doesn't exist and- and that's the simple use case, if shown here and at this point provisioning ends and and any application or any user that wants to consume. This bucket has to first request for access to this.

C

With the second workflow, the grant access workflow, um I can hear someone's background noise, okay, so uh uh the grand access workflow and this in in in for this workflow there's a there's, an additional component, which is the application that's going to consume this bucket once it's granted access so just shown in the arrows here. The user talks to cosy to request a request, access for an application, cosy talks, the back end gets the access and then from the back end it it takes the credentials and gives it to the application.

C

The application can then use the credentials to uh make object, requests, uh uh object, requests of uh different sorts.

B

So did we cover where the credentials get passed from the back end too cozy.

C

That's going to be covered in the next few slides.

B

C

So just like, when requesting a bucket and requesting an access, we've used the object resource type bucket access request. Bucket access request is a namespace resource which resides whichever user requests. The the bucket access gets. You know creates this object in their name space.

C

They specify the name of a bucket request. In order to point this, access request to a particular bucket and access types can be grouped into a class and and that's captured in the bucket access class, and it's definitely.

B

This is a green field or brownfield, or both.

C

uh This would be for for greenfield definitely for brownfield. uh If we're doing static provisioning, then we wouldn't need this, but if you're doing dynamic provisioning at the green field or brownfield case, we would.

B

Okay, so how do we map from the profile, pics name specified here to the profile, pics hyphen, something auto-generated um earlier, the bucket name? You mean yeah.

C

Yeah, so uh the bucket request- uh I I think I did not capture this, but once the once the bucket is provisioned. Let me go back.

C

C

Oh yeah, so so the bucket request has a uh I'm not captured this year. So bucking request has a bucket instance name or a bucket name uh in in the cab we get into it, uh and that is a pointer from the bucket request to the bucket. So uh the the name for the.

B

Oh, I thought we have a single layer of abstraction. The bucket name is a protocol specific construct with respect to the kubernetes environment. You wouldn't ever refer to bucket name directly. You'd refer to this um other name that you used to access it. Is that correct summary: uh when you mean other name, you mean the one: that's not shown in the bucket request that you were just alluding to.

C

Yeah yeah, so the name would be pointing not to this bucket will be pointing to the bucket resource.

B

Yeah the whole object yeah. So then, technically, the only time you would actually have the bucket name exposed is through the information made available to the application.

C

uh Yes, the credentials yes, so so there is a there's, a tiny error here. um I don't know if it's worth pointing out, but quickly I can show you. The name of the bucket is actually derived based on the bucket request and the name space.

B

I

B

um Now is namespace, something that you can have more than one a bucket.

C

Request, a single bucket request is confined to one namespace. If you have another namespace with a with a similar bucket request, then the name of the bucket the corresponds will be different.

B

Okay, so we're restricting it that a bucket request can only be in the context of a single namespace.

B

Yes, you would have to use a brownfield request if you wanted to share across uh two name spaces. Yes, and that way, let's go ahead. Glory.

I

Can you can you um concatenate them with a dot because it's like it's, I mean.

C

uh Wouldn't that make it more uh dns style.

C

I know like it's the same. You don't have dots so.

I

That's um I mean I'm just saying it's a typical use case and, yes, you have uh dots and names because you'd have dots in urls. So that's why.

C

Right right: um okay, yeah: we can look into it, but I think uh the convention. Sorry do you mean in the name of the resource, or do you mean the name of the actual backing bucket? I mean the actual.

I

Bucket, I mean.

C

I

See yeah, that's definitely the name. I mean.

A

Oh no, but the names are a thing: if you're, concatenating, namespace and name, you have to use a legal character and that character as a separator and that character could be in in.

A

If you're trying to be able to break apart the name space from the name visually any delimiter, we pick there is a case where that doesn't work.

C

A

Legal yeah, that's all I'm saying the convention is to use dash in kubernetes, at least that's the that's. What I see most often yeah same here.

C

And, and as soon as I know, dot is not legal in the names. I might be wrong.

E

um A quick comment on the bucket resource object name, I think you're concatenating, the bucket request, name and namespace for that.

G

E

I think it may be worthwhile following the model for pvpc I mentioned. Uid is not important in general. This is one place where it might be worthwhile to use the uid.

E

That way you can differentiate between two different bucket requests. uh We may want to think through whether we want to do that.

E

We did decide to do it for pv pvc and it might be worth figuring out if we want to follow the same model here or not. Basically,.

C

I think that requires a longer conversation.

D

A

Yeah we can note that down for now, yeah it's just after the notes assad. We have this one. This many to one mapping between brs and bucket instances does that. Does that mapping there change your opinion, any.

E

uh Off the top of my head, I'm not sure I think I need to think more about it.

F

Is the uid something readily available to that persona? That's going to be using it in this scenario, though,.

E

The uid of the request should be available to the component. That's going to be creating the bucket object. So that's true. Okay, it's as readily available as the name.

F

F

C

All right going into bucket access class, the bucket access class groups, a bunch of bucket access types.

C

A provisional field is set here to specify which provisional uh understands this axis class and again this has a set of parameters, a bunch of opaque parameters that are passed into the back end and then policy actions, policy actions define how this bucket is going to be accessed by anyone consuming using this bucket access class.

C

um In this case, I've written out a spec access control spec for aws s3, where you can write and read so put object and get object are allowed and everything else is denied. So this is right once read many times, which is right once read. Many is, is the access control we wanted to provide for this bucket. Based on the example I explained and yeah. This is the.

C

This is an example that shows that.

E

Any questions about this, this policy action- is it going to be consumed by any other part of the kubernetes system, or is it only for pass through to cozy? It is uh passed through.

E

So if it's only for pass through to cozy.

C

Should we make it an opaque string, yeah, I'm actually, uh so again we had a conversation about this uh jeff ion string. Yesterday, however, again we wanted to make sure that when we bring about changes, we want to include everyone. I think that's a valid suggestion and yeah it's something we can add yeah, so that.

B

Would be moving it into parameters? Yes, so.

A

It could be in parameters or it could be called out, but a just a single string, a json string, most likely right.

E

The idea being that in general, if, if you have a field that is only consumed or is only passed through to the underlying driver, there is no need for us to make it a first class field. Yeah.

K

E

We make it a first class field is when a kubernetes component needs to operate on it.

A

Yeah, that's a good, that's good side. Thanks.

F

I agree because the other effect that it would have is that the user may assume that it's validating these two against each other, like it's going to be up to the driver to make sure one of the um a lot of resources isn't on both the allowed and deny list and we're not doing that with the actual cosy piece.

C

Yeah that confusion could happen. Yeah yeah.

B

This becomes really important when you have multiple brownfield requests for the same bucket.

L

I'm actually kind of curious how this works with the. How this would work with the service account part yeah.

C

So the service account use case is where um I mean the access policy would still look the same. It's just how the access is. Provisioned is what changes.

C

What I mean by that is, is a set of credentials passed into the application, or uh is uh the application identity somehow known, or uh you know any request coming from that application with that identity should somehow be given access like in aws. You have instances having iem roles, or you know even pods,.

M

The problem is: is that the identity you're mapped to on the server you might map to that identity uh from several places, and so therefore, theoretically from those different places, you could also all access the same brownfield bucket and I'm trying to figure out. If I'm talking myself into a hole here.

M

So so I I guess um yeah so this this is. This makes the assumption that a particular bucket access can be uniquely crafted at this point and that if you therefore have multiple cases of that user trying to access that bucket, that's kind of its own problem- and I guess there's no way around that and it doesn't matter that these policy actions- I don't think make that problem worse.

F

But it doesn't solve it either like you're, saying that if I have granted service access a role access to, I don't know a particular object in one, and it happens to be the same name and the other I'm going to automatically get access to it. What do you? What are you saying, andrew.

M

um So so the the original mechanism uh that was in the cap was this idea that you would mint a new server-side user for every one of these. In that case, that implies that every bucket access can have a separately defined set of accesses.

M

But in the case where you're doing a service account mapping, there is no longer a guarantee that every single access is uniquely minted right. It could be that oh, the access is already defined there, or at least the user is already defined there. So I'm not minting a new user on the other end, um but what it implies is that if I craft multiple cases, even different clusters, same cluster whatever, where I'm asking for the same service account to have access to the same bucket, I can have conflicting permissions.

B

So is that up to the provisioner, or do we put any restrictions on that.

M

Yeah, I don't see how I mean the source.

K

Yeah, if you use the same one and it had more restrictive permissions on the second request or third request, you'd be locking out the first request.

L

Well, yeah I mean effectively, the problem is the source of truth, for the policy is not kubernetes.

K

But it's the last affected policy right. So if you keep dumping in new policies every time you create a new instance with the same service account it's going to be last one wins or you.

M

You basically have uh you know, it already exists kind of thing, and then you leave it up to the driver to deal with that or something.

B

Yeah, I could see one driver trying to match to an existing and another driver. Just saying I will create a new user for each request or.

M

I mean the new user approach makes sense if you're doing the sort of credential minting, but if you're not using a credential minting. If you're using the gcp or amazon approach of doing service account mapping, then the credentials are already known. The users on both sides are known. The only thing that you're minting is the particular access and, but even then, if that access, if an access binding, already exists for that user.

M

On that object, then I think the best thing to do is to just say, there's already a binding for this user on this object and because modifying it, when it already exists, seems to be fraught with danger. Yeah.

F

Is this something that we need to go to security with saad like, I feel like we're, creating a whole of possibly accessing object data in buckets that we shouldn't be. I don't.

L

Well, no, it's just that we probably need to make the policy actions mutually exclusive, with the use of a service account.

E

Right, yeah, yeah and it wouldn't hurt to yet say goth to take a look at it. But yes agreed.

M

um But okay, so the policy actions are unfortunately as articulated. These are. Are these really generic? I mean these look specific to s3.

B

And that's why we wanted to move it into parameters, because these are uh specific to the type to the protocol.

M

And- and so I guess my point is once you've done- that I don't know how you can at eddie's level deconflict anything you've just got to leave this up to the back end.

B

To handle it, each back end is going to provide in its documentation what parameters it can handle and what the syntax, the semantics of those parameters are.

L

So just anyone doing an s3 quasi driver just needs to make sure to not not permit both or something right.

F

We're saying each cozy driver for each provider will be responsible for handling that collision.

C

Only one service account.

F

And there will be only one service counsel that you said sorry.

C

Yeah yeah, what if we tied it to the service account instead of trying to understand the policy itself?

C

That is, if a second bucket access class, we have a service account field in the bucket access class and if a second request comes in for the same service, account uh in a different bucket access class be rejected.

M

So there's a couple of of scenarios right, so one of the possible scenarios is that um I am crafting the complete access out of band right, so I've I've. I've got a service account on the on the cloud side that I've created. I've set up roles for that, so I've got all of my permission in place. For that account to get to that object.

M

Then I then I set up a workload, identity or the equivalent aws to map a kubernetes service account to that, in which case there is really nothing for a driver to do potentially right. One possibility is the driver, could just say: oh it's already there and that I I don't think you would necessarily want that to be an error.

M

You might want to control whether it's an error, but but that seems like a perfectly reasonable, because otherwise you're walking down the path of having to say we want to construct every possible um mode of access in the kubernetes object, and I don't know how you deal with sharing in that case, because then you end up with conflicts expressed at the kubernetes level.

A

And andrew what you said is actually an example of driverless. I mean all this done on the back end out of band. Is you don't need the driver for that access.

L

Yeah- and so I driver is still a useful thing right, so we don't want, we don't want to make. You know. I don't know.

M

Sure I'm just suggesting a a mode, um and this gets back to the whole problem of sort of greenfield versus brownfield right in greenfield. You really want to automate all of it. You want to automate bucket creation. You want to automate if necessary, um service account minting. You want to automate credential minting.

M

You want to automate passing the credentials into uh into the the client side into the app, but all that stuff you may have to support kind of from scratch and then there's various modes on top of that, like workload, identity or the equivalent, where uh I'm not minting service accounts, but I may be minting just credentials, um but then you could even say I'm not minting service account or credentials. I'm just.

M

um I have a mechanism here that says this is how I'm accessing it and um but it's effectively a no-op, because all of that access is already plumbed in. I just think we want to be able to support all of those, and I don't think it's too hard.

M

It's just one of the things we got to do is to say something in a bucket access class or something that says: hey, I'm expecting this to have already been handled or overwrite on you know or don't override, or you know what I mean some kind of of deconfliction policy statement at the class level. I think, does it.

D

K

So you're saying, if minting a an account or a user or a service account, then you would want to implement these policies. But if the account already exists or if you're using a service account that should policy action should be ignored.

M

And further you could even assert at the bucket class whether or not such an account must or must not exist. On the other hand, right right.

L

Yeah yeah that would work yeah. I.

H

Mean I like I like.

L

Like like, I I like the like, I like the service accounts and I love, but I also like the idea of being able to um like like set set initial policy and have and have a service account and, uh like you know, effectively kind of like a arn roll or whatever to be minted, be minted.

L

For me, um you know that that that that corresponds to the policy right, so that I can I can you know I can I can craft this, this bucket access class, cr or whatever, and and it you know, if it's minting, if it's minting something new it it. You know it, it creates the the I am roll or whatever it was necessary. That would be mapped in here. I'm not sure. Maybe not. I don't know.

K

Let me leave that to the driver, then, to identify if the access exists, ignore the policy action if it doesn't exist, you're, creating a new service account or new user and credentials follow the policy action as part of the bucket creation.

M

Right right, I think that's one option, which is to basically treat it as a um effectively a kind of um item, potency kind of statement. That says: look I'm going to add this access, but if you've already got access, that's fine, but then the question becomes. How do you handle conflicts?

M

Do you reject something whose policy actions don't match? What you currently have, I mean that's a possibility, and then, if you don't provide these policy actions at all, then effectively you're making some statement about then right I mean I, I think, with just a little bit of fleshing out yeah. We could. We could come up with something reasonable here that that isn't dangerous.

F

It shouldn't be dangerous and it should be able to be solved consistently if you're, trying to go from to handle identity between two different ways that other clouds handle identity within.

K

Right, I think it has one at the driver level, though.

L

I I mean it, it might be as simple as just if it's, if we're minting something new, then then you can impart policy and you create a new iam that only like a new iam user that only uh kubernetes can only this specific kubernetes can use and then basically then you know that particular kubernetes is the only one. That's going to be able to update that policy object and- and you could you know you, could you could share that with you? Could you could also share that bucket with with the application user?

L

That would be distinct from the I am user for kubernetes right and then and then, if you have a if it's an already existing and you're, not minting a user or something, then you just don't allow policy updates at all.

F

We um articulate these different usage patterns more so that we make sure we're not creating an issue or do we not. I.

M

Think I definitely it that it's time to do that and probably can't do it in the eight minutes we've got here, but yeah.

F

Come on, eight minutes is like.

C

Yeah, I I mean, I think suggestion and, and I think we can figure it out uh in the future, but the issue is only when there are conflicts with uh two different market policies pointing to the same uh service account.

M

I I think it's a little bit broader than that, and let me see if I can't articulate it.

M

What I imagine is several levels of action you might take in response to a bucket class right. One action is and- and I'm going to use, I'm going to try to use generic terminology and- and- and you know, hopefully it ports to the different platforms. So one action is I'm going to mint a user in response to this.

M

On the on the bucket side, then I am going to add to that user permissioning to access this bucket um and and I'm assuming by the way the bucket's already there. This is the bucket access, so so I'm going to add a user and I'm going to set permissions for that user.

M

So that is a question of what I'm going to provision on the bucket side. I may also then uh mint some sort of access, token or credentials for being that user in the context of an api call, because I need to do that, you know coming from a client. So that's one of the.

I

Areas that I imagine why isn't that just the provisioners I mean, why does it? Why does the cozy controller need to be aware of it? Isn't.

M

I

Just you know configuration parameters.

M

I'm sorry I'm describing a scenario, I'm not saying who does it I'm suggesting.

C

No, I just I understand.

C

Is it one second? So we have like five minutes left. I think this is something we need to talk in a little more detail. We'll write this down for the next meeting.

F

C

I want to finish this uh and you know, come to a just get through this. To be honest, uh there's only like two or three slides left and uh I think uh it'll paint a picture of what the response looks like and then that'll give us more context uh after after you know we go through and make this bucket access request call.

C

So I'm going to quickly just go through this, and so that we have a picture of everything. That's going on um so with this flawed model. uh Okay, so so you have to use an admin request. Cozy cozy goes and creates a bucket access, object, copies over fields, and then it goes ahead and calls the back end call this parameters again bucket name. So here we have something called as a principle, which is what I want to introduce you to.

C

So what we've thought of, as the principle is the uh unique identity of who the new user is, that gets access, or if it's an existing user, that you want to update or provide new access to you pass in the principal's unique id. The access policy is the access policy we had earlier and.

B

I would assume that that's point. The bucket already exists: yeah, okay,.

C

um So access policy is a direct copy of whatever is then the bucket access class bucket context is again a copy of. What's in the parameters uh and the name we've said, we've set the principle empty in this case, and the response is creates a new user gives that as the principle so far, what we start is the credentials file, content and credential file path will decide how this access is provided to the application. In case it's minted credentials in case it's a service account.

C

uh Only the principal field should be set, and you know the other two fields should be empty. uh That's about it.

C

Yeah, that's about it.

B

um So the driver specifies the path to the credentials.

C

Yeah, based on whoever the the driver is, the credential resides at a certain default path. Okay,.

B

And um will will there be some guidelines to make sure that a driver can't stomp over uh something else.

B

Guidelines yeah.

C

C

All right, so I think we should. We should write down things that we want to discuss for next week, uh specifically around bucket access class and uh jeff. Did you want to take it forward? Now? We have like three four minutes left um conclude.

A

It's another good set of suggestions. um Can we get a straw man vote on how people are feeling right? Now we haven't done that, yet it would just be interesting. There's 16 participants on the call right now.

E

I think this is a this is a really productive meeting. I like the uh provisionary, ass or the provisioning aspect of it. I think that's solid. It's looking good from the access perspective.

E

I think it's close, but we need to probably go through the flows for both a service account that already exists using workload, identity and uh minting a new service account walk through both of those flows just to make sure that they're handled properly, and then I think the third thing we need to walk through is the binding logic, uh because that will be that's going to be crazy here, because we're proposing something completely new, which is a many to one binding. So I want to make sure we review that as well.

B

And there's some additional um additional grpcs that we need to review.

J

Yeah yeah I'll just echo that that I think these these review meetings are extremely productive and we should keep going as long as people keep having questions and- and I don't- I don't see any reason to hurry towards having it be done. I mean if there is a a reason like a deadline we have in mind. Maybe we want to go faster, like we should just schedule more time per week to do these kinds of meetings two hours three hours whatever it takes, but we should take the time.

C

Completely agree with you on that: um how do you all feel about joining us on our engineering meeting on monday at 11 pst? um We can I really want to go over this bucket class design.

C

That would be a good time to do it.

J

You said 11 pst.

A

Yeah, it's the same zoom link. Isn't it.

C

Yeah, it is the same zoom link. Yes, it's a sig.

A

Storage, uh you know ben you mentioned urgency and I agree we want to get it right. I mean there's no point in rushing and and getting it wrong, but what what I'm thinking is that if we get this pr merged, we can start using official repos and we can get some prototype. You know pre-pre pre-alpha code out there and start using it, and I think when you have it in your hands and you're using and you're trying to write a driver, things are going to come up that we aren't. We may not just discover.

J

That seems like a weird bit of administrivia, where we're trying to merge the design just so we can write some code like that's backwards and there's got to be another way to get code written before the design merges. That's.

C

True, we can work on our own repositories. Code yeah. I don't think uh I don't think we're really uh gated on having this merged before we can start writing code, but I think, having uh you know it be more official, uh would would certainly not hurt us um anyways. So I think I think the the next step is really to get nailed the design of the bucket access pipes and then yeah, hopefully I'll see you all on monday at 11.

C

um jeff do you think can? Can we send an invite to everyone somehow.

A

E

I I think, if you update your calendar, invite to include sig storage that'll. Do it the same storage, mailing.

E

List sounds good all right thanks folks,.

B