Kubernetes Storage SIG COSI, 23 Jul 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes SIG Storage - Object Bucket Provisioning KEP Review Meeting 20200723

Description

Meeting of Kubernetes Storage Special-Interest-Group (SIG) Object Bucket Provisioning KEP Review - 01 February 2018

Object Bucket Provisioning KEP -
https://github.com/kubernetes/enhancements/pull/1383

Find out more about the Storage SIG here: https://github.com/kubernetes/community/tree/master/sig-storage

A

Okay, hi everyone uh thanks for joining this kept review. This meeting's a little bit different on a little more exciting, maybe than the previous ones, um um but we're going to be seeking um approval to merge this cap and um that that goal might not be realized this meeting, but that's what we're that's, what we're hoping for, or striving for um and so to accomplish, that we're going to go through the meat of the cap, but hopefully at a high level. So we can actually get through it in an hour.

A

But I don't want to suppress technical questions because obviously we are seeking approval and uh and that the cost of that can be. We need to go deeper into something that um a person has a concern about or isn't clear or whatever it is. So we can ask technical questions and if we don't get through it, all this meeting will go on um we'll just continue the next one.

A

So um my agenda for this meeting is to give you a quick intro go through use cases which are important, because you have to agree to the use cases and then believe that the kep, as we propose will meet those use cases we're not missing an important use case or focusing on an unimportant one, we'll go through the apis. We have the grpc spec in this cap and um sid, whose technical lead has some diagrams we can share.

A

If that helps, explain certain things, we'll we'll figure that out as we go, I wanted to give you some very brief stats on the kept first, I don't know which review meeting this is, but we've had dozens of review meetings.

A

We've had six or seven, maybe maybe more sid stand up meetings on mondays, where we do code, we do more technical discussions on the design and code.

A

We've done some presentations this. This cap's been a live thought I don't know. Six plus months uh sounds.

B

About right, yeah.

A

Recordings are available on the six storage youtube channel. We've had what did I count? 16 uh reviewers on the cap so far, 50 something commits um staying there for history, they'll get squashed during emerge, potentially, but they're there every commit commits visible um and so on so there's. The point of this is: there's a lot of activity and it's it's had a lot of eyeballs on it um and many iterations name changes and so forth that you expect in in a larger project like this.

A

So I pushed up the latest changes to the pr uh late yesterday afternoon early evening, so many of you might not have had a chance to look at it we'll go through this now um and let me know if you're able to follow my screen, because I I need feedback, whether I'm scrolling too fast. I have a another.

A

I have a daughter at home on a zoom meeting right now, so I don't know how great my um home network connection will be for this, so I'm getting down to use cases now, so I'm just scrolling through you, you guys can read that stuff. Why we're doing it and so forth? uh Some vocabulary. There's terms used in the cap and we we define them up front.

A

So it's clear what we're talking about, but I think use cases matter and, and we divided it between two people, admins and users and and admins- want control, usually and users want flexibility right. So there's usually a tug, a tug attention in these kinds of in in these two different use users.

A

um So for the admin we want them to um be able to manage buckets with familiar uh cue, cuddle and other tools that you are used to in a kubernetes environment and the second one is. We recognize the name spaces about the familiar boundary where security rules are enforced and applied to, and so we want an admin to be able to for sharing buckets, to be able to uh allow or restrict namespaces um that's sort of the unit that we look at in kubernetes.

A

We also not stated here, but our design um does allow minimum rbac capabilities for pods running in different name spaces, with a sense that the vendor pod is the least trusted pod in the cluster. You know you don't know what they did, and so you don't want that. Vendors, the driver pod- I'm talking about you, don't want that pod having access to your whole cluster. So we, our design, allows that pod to stay in its own name space for that vendor.

A

The google g of aws azure doesn't matter, um but but they don't need our back rules that grant them access into any other name space other than their own um and then from the user use case. Point of view we're our user we're thinking of as the developer, not the person running an app, but the person in charge of developing the app and using the object store and that developer wants familiar manifests.

A

They want to deploy to their bucket oriented app in the same way as any other app, and they want to use familiar tools from familiar looking animals or deployments um etc, and the first two bullets there cover that requirement or need, and the last one I'm not exactly sure I'll sit. So you can chime in I'm sort of thinking it's a foundational api in the sense that hey.

A

If we have these apis, we can build other uh neat things on top of them, which is true, you could do a object, backup, cloning, snapshotting other tools, that's not just a user use case that can be a vendors or as well even an admin.

A

um So those are the use cases that we are focusing on and we believe our kep satisfies.

A

I'm just pausing for questions.

C

So I have one question: um do we have the concept of um buckets having different roles, I.e, a read-only role, um a write exclusive, where only one app is able to write to that bucket uh being coordinated through this.

A

I'm going to give you the a high level answer, and if it's not good enough, sid can chime in we we have. um We have a a provisioning set of apis and in parallel we have an access request, access control set of api. We have two of them, um but we are not trying to be an iam system or something like that. So what what we do define in in one of the apis is uh related to access modes of read only read write right only and we have a public.

A

Is it a boolean sid? We have a public setting.

D

For yeah, so so we have uh so we support whatever the underlying uh cloud provider or backend object. Storage requires um what uh jeff is talking about. We call it anonymous access mode where we give like catch-all uh authorizations per bucket. Some, like you, said, uh read-only read, write or write only or private, where you have to specifically add explicitly add a credential before it can start being used.

C

Right so that that makes sense uh less about iam, more about scheduling where, if I have multiple applications that can't run simultaneously, because they both require say, for example, exclusive rights to a bucket. Is that something that kubernetes is going to be able to then correctly schedule? As a result, the information uh provided through this uh interface so in in such a case.

D

Where uh so, we uh ourselves like kubernetes itself or cozy, um will not understand that both of them are right-only roles. uh We let the back-end take care of that, so both will be scheduled and one of them wouldn't be able to uh access right right to that bucket.

A

Yeah dave, I think the short answer is no to your question and um the back end wouldn't have anything to do with pod scheduling that could, if that, if that actually is important, you know there are scheduling, plugins that can be added and so forth, or it could be another cap, another enhancement down the road, um but this.

C

Kept wouldn't do that got it. I know that there's some equivalent functionality around volumes, um but yeah that.

A

Yeah, that's a different use case.

C

A

Especially um local local persistent volumes where they need they need to know that information uh we don't have. We haven't, given that actually any thought really in this cap, the scheduling side of things.

C

And I'm assuming that there's been uh some thought about where you'd spin up a local uh pod, specifically to provide local object. Storage.

C

A

What do you mean by.

C

Sorry go ahead.

A

Well, don't know, sid you better answer that go ahead.

D

I'm trying to understand.

C

uh What what what do you mean by local object? Storage, uh where, for example, I uh spin up a pod that then uses a volume to then expose an object interface? You know people do this quite often in development patterns or using, for example, a min io or other lightweight object stores. So they just need an object, storage, uh local object, storage, scratch, space or persistent store yeah. We we support.

D

That, uh however, the mechanism by which we support that is slightly different, um you you'll get to see it just in the next section. I believe um yeah.

E

Yeah, actually, I think if I understood the question right, you're talking about implementation of the object store, and I think that this cap is almost entirely about the kubernetes side accessing an object store. So if you in fact an object store that is implemented in kubernetes, it's all hidden behind the driver. How do you do that is not? I think part of this yeah.

C

So not about the implementation, but about whether the cap includes enough functionality for an implementation of an object store to then expose itself through this camera yeah, and I think I I think, as you go through here. Hopefully that's yeah cool. Well, I don't want to take too much more time. Thank you. Yeah.

A

Right and save a question if it hasn't been answered well enough, um reach out to us, add a comment in the pr, for instance, for this kept and we will address it there. um Okay, so I'm scrolling down please raise please.

A

Let me know if um my home network is messing up and you can't see this stuff, so I'm going through the apis and, like I mentioned there, there's sort of some parallel apis and a very high level, um a user requests a bucket we're calling it a bucket request, andrew we we've iterated on this name a lot um and I won't go through the history of it.

A

But this is its current version, which we think is the most meaningful and least ambiguous, because when you say the word bucket, what the heck are you talking about the s3 bucket um or are you talking about a kubernetes custom resource named a bucket and um it gets confusing? So this is a bucket request and it is in the user name space and um it is there uh asking for access to an existing bucket or asking for access to a new bucket that they're going that will be created for them.

A

um It's important to know that all of our apis have finalizers on them. uh Even a user-created um resource we will cozy will add a finalizer. We want to orchestrate the entire tear down of these objects and coordinate it with the object store. Underneath um this I'm not going to go through individual fields. But if something raises your eyebrows ask you: can the user can basically state what prefix they want added to an otherwise random bucket name?

A

There's a concept of a bucket class similar to a storage class and um statuses that you're familiar with um the other thing to call out here is you can um and sid. I changed this after our conversation, so this is. This is going to be unexpected to you, sorry, um you you um right now we don't have a way where you can fully define the bucket name sid. We did last night, but we don't now, um but what what you do is you can? What? How do you access brownfield from a bucket request?

A

You know: how do you identify that bucket um and it's and it's done through field number seven there a bucket instance name. I added the word instance to it uh because bucket name, what is that is that your s3 bucket name or is that a kubernetes resource, so I called it bucket instance name and I think you can see the definition of it, but basically, in the brownfield bucket request, you name the kubernetes custom resource, which is called a bucket. You may it's it's cluster wide, so you don't need a namespace, you name it there.

A

How did that get there? How did the bucket instance get there? That's an admin step, it could be automated, but it's not automated by cozy there could be a separate tool chain that does it or an admin creates it by hand like they did in the old days with pvs, and you referenced that bucket instance in your bucket request and that's how you get brownfield, you don't need a bucket class in the brownfield case.

A

The bucket instance, which we'll go through next, has all the information uh needed by the provisioner and driver uh protocol.

A

That's just saying um you know we don't have portability across different object stores, uh it's not like posix or iscsi, so um so an application that works on as yours protocol uh doesn't necessarily work in uh google cloud and so forth, and vice versa. And so um we require that the requesters specify what protocol their app is talking and uh and and and for greenfield new buckets, uh a bucket class is required and that bucket class uh defines the protocols it supports and we match them.

A

So if you want s3 and the bucket class says as your blob, it's a mismatch. Your pod, your bucket request, is not satisfied. uh It doesn't get deleted or anything. It's typical declarative api, not satisfied we'll back off, but keep trying we don't ever give up. um So this is what the user sees when they want a bucket or access to an existing bucket.

A

So what is this bucket instance that for brownfield is created manually potentially and for greenfield it's created by cozy, and this is what it's cluster-wide, and this is what that api looks like um and there's a couple of things that are important here. One is that these fields are mostly populated by the bucket class and uh and the response from the driver.

A

um That's where we get this information now, if it's brownfield, the admin had to come up with this, they have to. They have to put most of this in themselves. They don't have to put labels in in the finalizer, um but the rest of this information.

A

They need to add, when they're, creating an ins, a a representation of an existing bucket, that they want to grant access to their developers.

A

um So a couple of a couple of points here we have a release policy also defined in a bucket class which you'll see next, it's the standard stuff, you know retain the bucket or delete the bucket delete's important. When would kubernetes ever delete a bucket in brownfield? The answer is: never it doesn't matter if you've closed it up and put delete, reclaimed policy of a release policy of delete. We don't delete brownfield bucket content ever we'll clean up kubernetes crs around it, but we aren't going to touch the content.

B

When would we wait, why not so in the case that I have uh manually constructed bucket and I put release policy delete, why would we not delete it.

A

Well, I guess out of fear um just you know, I know, there's a very strong sentiment to don't with code that can have bugs in it to never delete the user's data. Now there was one time so sod, it's a fair question, the one time we would delete. It is a green field bucket with a delete policy of a release policy of delete, which is not a default and no more accessors to that bucket, because, unlike pv pdc, we have a one-to-many relationship.

A

One bucket can be requested access to by many bucket requests.

F

A

When, when all the brs are are sat, are done are closed, deleted that bucket could be will be deleted. Not good is for green or brown, but for green we said if the release policies delete we'll, also delete the content we'll at that point, we'll call the driver with a delete, uh grpc delete, call and let that driver actually clean up everything.

C

Will there be a mechanism by which uh code can be added that executes before the delete.

A

Oh interesting, a pre-delete hook.

C

Yeah so, for example, I could have a policy to take a backup or to move something to glacier, etc. Just before it gets deleted.

A

We don't have pre or post hooks in our design.

D

I think that's a good point, but um as of right now we don't have it. However,.

C

Nothing would stop us from adding in the future. Okay, there's a bunch of use cases around that if there's more interest in drilling down into what the enterprise use cases are for having a pre a post delete.

A

Yeah I mean I yeah I I can see that it's that's a good suggestion.

G

I have a question when I look at the bucket. How can I tell if it's green or brown field.

A

Looking at the bucket instance here you uh oh uh sid, did we say: there's an annotation yeah yeah. I don't.

C

D

Right yeah, so we would. We would have an annotation saying created by either by us or you wouldn't have that field if it is brownfield.

G

And we need this annotation only for to distinguish between green and brownfield.

H

G

For it is there is, it makes sense to create just the first class field, for it I mean like if this annotation sounds a little bit like like hacky.

B

Untyped yeah um well I'd go the other direction. I'd say why even try and differentiate.

G

I think I agree with your thought I it just as, as we said like if I don't like that, probably another kind of idea that release policy only is related to greenfield, but not brownfield and it kind of it's. Why should we have it here or should we name it something else or.

I

Any any greenfield could become a brownfield for the next instance.

H

Sorry go ahead, would it just be brownfield, buckets might have been created and used for something else outside of the kubernetes ecosystem, so the life cycle of that bucket should have consideration and that's where we want to differentiate.

B

Right so I think the way I look at it is it: by default, the release policy should be retained, right, like if somebody doesn't put it in, or the use case that you mentioned are in by default. It'll be retained, nothing happens to it if you delete, but if I am, for example, uh you know copying over a bucket and that bucket was dynamically provisioned.

B

I should be able to retain that dynamic deletion uh effectively. If I copy over this object,.

A

Yeah, I mean delete's a tough one sod. You know, um there's also this green brown use case that andrew first brought up where you, where you provisioned a new bucket, but now you prove it a bucket request provisions in new bucket, so it's greenfield, but you want to share it so now, the other accessories, the other bucket requests, become brownfield requests to this new new bucket and they're sharing it.

A

In that case, it seems like um that bucket's whole life has been under control of kubernetes and if the release policy was deleted in that situation and all the accessors are are gone, it seems like we could delete it, but to me saad, if you have some existing bucket out on s3 somewhere and you've created a abstract bucket instance to represent that bucket, and you put the release policy as delete in your bucket instance um that bucket could have been around for years. I mean: do you really want kubernetes to delete it.

B

I may right by default, I would say: let's not do that right. That's why release policy default should not be that yeah, but if I want to allow that behavior because, honestly, that's what the api here says right, you have a release policy if I set it to delete and then it doesn't actually delete in some cases. That's really odd. We if the api allows you for statically.

E

Provisioned pbs can I.

B

Do that again,.

E

Can I set a delete uh whatever the retained delete policy I forget the name. Can I have that automatically deleted on a statically provisioned pv.

A

Yeah, it's a fair question.

G

I think so because how.

A

G

Like look by looking at the pv, how you can tell if it was statically or dynamically provisioned.

F

uh You have to go change the division policy and then you can delete it.

G

Right yeah, but otherwise there is no difference between statically and dynamically provisioned pvs right right.

B

uh Giannis has a question.

J

Sorry, I don't think they're out anyone, so yeah, I'm wondering on what's happening on the following case. In one namespace, you create a green field right bucket and somehow you get the credentials, get the access connection and you go and create it as brownfield or another namespace, and then on the um on the green field. You delete it right and then, if you go and delete the contents, then what's happening on the other namespace, where they point to.

E

It's reference: it's a reference, counted references to the same bucket, so it isn't deleted to the last one to the last round.

C

It's it. That would only be the case if the it was if it was going to the reference count system. I think, what's being asked, is if someone created a whole new object that just happens to have the same information in the protocol configuration pointing to the same bucket.

A

Oh two bucket instances pointing to the same actual bucket yeah.

C

And then that's probably don't do that, but um you could get into a.

F

C

Where it gets deleted out from under another application, yeah, I mean tvs as well.

G

Well, it can happen anyway right somebody can just go to the admin console of the actual cloud provider and delete it there right. So we kind of probably.

A

C

A

G

C

Is well if you don't want that happening, don't grant delete to that those credentials, that's being used to access it.

I

Yeah, fair enough too, going back to the original part. If the, if you're, attaching as brownfield and you're the last operation in the chain, you should be able to delete as part of the closing operation.

D

Yeah to stay symmetrical with how pvs work right.

D

You could yeah you, you could you're saying if, even if it's a brownfield, um if the release policy is set to delete, we should be able to delete it yeah. I.

A

Mean I think, assad's saying honor to honor the release policy. That's what he's saying and um because there can be times when that is actually what the admin wants done. So why aren't you doing it? And I think it's just a very conservative.

A

Decision we were thinking about being, very being very, you know, risk-averse when it comes to deleting.

C

Data, if the delete policy has to be explicitly set for brownfield uh buckets, then that setting of that release policy to delete is an explicit indication that you want that behavior yep. I.

A

Can't argue that it is it's just you know, and kubernetes basically doesn't try to keep someone from shooting themselves in the foot. So you know, okay. Is it okay to move on from that? I think. There's good points raised there. um I, like the pre post, hook, ideas. I I'm okay with us honoring release policy.

D

Yeah, we don't need any major design changes in order to yeah. This is a decision that you know it's more of a use case decision, um whatever we agree on that can be satisfied.

A

I want to point.

D

Out to point out this access modes.

A

Here that I've highlighted just it's there.

D

Someone was stalking study, oh.

K

There was a question about what happens with statically provisioned pvs, and I think we kind of glossed over the fact that pvs may not have a provisioner at all, in which case there is nothing to to delete them. If.

F

K

Added you know by administrator, then deleting them won't cause anything to happen because kubernetes didn't create them. Companies might have no idea. There might not be anything on the other end of the delete rpc to call for a pv at least correct. I don't know if we're gonna have a corresponding case here, where I created a bucket that I pointed to something and there's literally no driver. That knows how to do create or delete it. Just only knows how to do attach.

A

Well and andrew has proposed a use case which seems to have been lost andrew in this cap. So just so, you know of stat a driverless, in other words, to sort of uh jump start this whole idea of using kubernetes for buckets. um You don't have a driver at all.

K

Or you have a driver that implements attach but not create and delete, or something that that's.

E

Whatever attached the access piece but not.

A

The not the provisioning you could you could. You could have that and andrew over various iterations when I was going through this pretty carefully the last few days, although driverless is defined in our vocabulary, and there was one reference of static provisioning. There's no examples, there's no workflows that actually support it.

C

When I read through the cap, I got the impression that driverless was was just implicit in terms of that. If I am an application, that's aware of this, I can use the kubernetes apis to look at these objects that are accessible to me that define the buckets and then I can just use that to go, for example, and directly talk to aws or azure blob, etc. Yeah.

A

And david, that's absolutely right and one part of this whole life cycle was we stripped down the kept to bare bones saying all we really need is a foundational api and you can build anything on it and the automation that's described here isn't even needed. I mean it's nice, but it's not a requirement. They it. If you have a good set of apis, you can do what you want with them, but we have come back to uh re, adding back in automation to this cap.

C

I I don't disagree with that. I just wanted to to make the point that, um as an outsider coming in and reading the specification, it seemed at least clear to me how an application could use this without requiring a driver in their pod. Okay, thanks, um I'm just pointing.

B

Sorry, quick question uh is somebody taking notes, there's a lot of good feedback here I want to make sure it's captured somewhere.

A

Saw that so I forgot to say that at the beginning, I'm not able to run them to drive this thing and take notes. I'm not I'm not good enough parallel processing to do that.

L

I can go back and listen to the recording and document all right, aaron thank.

A

You aaron, so we have anonymous access modes here, highlighted um as described um what I, what I want to show you is this is there's, um and the next resource is the bucket class, but basically um in brownfield uh you don't have a bucket class, so the admin would say what name spaces can use this bucket. It's really that simple um for for the green, slash brown use case, an admin would create a bucket class and in the bucket class they can say what name spaces can use this bucket.

A

uh The protocol we've gone over.

A

Here's our proposal of phases and- and you guys that are have done this stuff before we are trying to find the balance between not making it look like we're a state machine um or have all these states, because we know we know that you don't go from a to b to c in kubernetes and we don't want a logic predicated on what phase we're in, but we believe from a debugging logging trying to figure out where it got stuck point of view uh that these phases are reasonable for the life cycle of a bucket, um and so this is our proposal on phases.

A

We're you know it's probably best to get the feedback on that in the cap. It's easy.

F

I think the if you look at the api conventions it says uh faces, this pattern is deprecated. You should not be using this anymore. Oh.

A

Great, so what is it all conditions now, shane.

F

This is the the convention yeah, I can yeah. I can.

A

Add that in the cap I know I've seen this coming. um Okay, good to know, we should use the we shouldn't use, something that's actively being deprecated right now.

C

Would it help if we split this into whether it's accessible for an application, because creating deleting and deleted all represent conditions where an application cannot or should not use that bucket.

A

Well, true, and because the oh, we still don't have the application manifest in here darn it. That was something on our to-do list, um because the application specs uh the application manifest will just will define a csi volume.

A

We we are that pod can't run until um a csi. Adapter pod tells the cubelet, through a grpc interface, that it's okay to launch the pod, now yeah, so we'll synchronize. That way. Right now, if that was what you were getting at.

C

Yeah, so if there's, if there's a way that an application well, there's two aspects, there's whether kubernetes even launches, um but the second one is as an application. I may want to be able to see whether I can use a given bucket.

A

Okay- and we don't have that second one- there's not a is it okay, now type mechanism.

C

A

To make it at the wrong level.

C

Because, if our goal here is, is that, during the run time of the application that's attached to that bucket, it should always be accessible. Then that actually represents an error. State yeah.

A

Yeah, I want to draw your attention to something: that's new in the cap under the.

M

A

M

Hey, I was wondering: do you have the um I didn't notice if that's, including also a status for the um bucket and also the bucket axis?

M

Is that, like you mean the the phase guy right, so the phase or conditions? But in any case, are these represented in the cap, so that there's uh stay states, different statuses uh for bucket access and different statuses for buckets right, there's like different status for each one right, oh yeah,.

A

So this this is the bucket instance. So it's it's on the provisioning side, api, not the access side, so it it it's not it's not directly representing access states. um Although you can't bank bind to a bucket until which we haven't covered. Yet until the access uh instance, the bucket access has been instantiated.

D

So guys we do have uh separate phases for buckets and the access apis uh and and uh they're representing the cap. Here, um it's just uh further down below um yeah, we'll be talking about it. Yeah and.

A

We're like more than halfway through the meeting uh shoot. um Okay, I gotta just throw this out because it's a change and they're. It may not be um accepted. Acceptable labels used to just be a label, just a decoration kind of thing that gives some information to an admin. Hey like this was created by cozy, but now labels actually have a purpose.

A

um The label is our linked list if you will or our connection to the binding. So we have a label here. This is the proposal where the label does the mapping it maps the bucket request or the bucket access uh it's it's technically. uh The key of the label is the bucket access instance name, and the value of the label is the bucket request name. I guess it would be namespace, plus name and and and the set of those shows you all your bindings isn't.

F

There references.

A

Would have been nicer, except you can't use owner references on cluster scope, objects.

C

Is there precedence for using labels this way or is it used as a free form uh in a field elsewhere in kubernetes.

A

A fair question which I don't know the answer to I.

K

Don't think labels are meant to be used in that way. I think we'll get into trouble with api reviewers if we try to make labels have semantic meaning.

E

Yeah, I thought the remote could just use for selectors that that was the primary intent.

A

Yeah that that is correct, so, okay, um what would you do to try to if you can't use owner references is what we wanted to use? What would you do to represent these this mapping? Why.

E

Wouldn't they just be first-class attributes of the bucket.

A

Have a a slice inside the bucket object itself, the bucket instantly.

E

Yeah yeah and by the way, if you make provisioner optional, then that's the driverless case right.

A

Yes, yeah, you know what there was a reason we didn't do this, but do it the way you just said andrew- and I can't remember right now, though I it probably what it might not have been a good reason.

E

It's probably to because then you don't support delete if you do that, but I think that's exactly how you solve that issue. If you don't have a driver, you can't delete it so boom.

A

Yeah: okay, okay, uh a bucket class, has the role of a storage class. I'm just scrolling down here. It's admin created cluster scoped. um There's nothing particularly interesting here. um It's stuff you've seen before a bucket cl an object store, can support more than one protocol azure. Does that gcs does that? And so we have a list of supportive protocols um and that's kind of and the additional name spaces beyond the name. Space of the bucket request.

E

References you you've got some comments about this on the kip. I don't remember exactly what the resolution was, but it is there a reasonable alternative that just says: do a bucket class for each access protocol and rather than trying to fold them all into a single class.

A

Yeah, you know, I don't. I didn't see that as a current comment, but I remember discussing it and that the answer is no there's no reason. Since we have don't. We don't have that many protocols, there's no reason that um you need a list of supportive protocols that I can think of and in fact, there's a reason not to do it in that it makes default bucket classes, which we support um a little.

C

Weird yeah, so there's there's the question: if I have multiple choices, how do I make the choice.

A

Your your bucket request says what protocol the app speaks.

C

So if I have two different bucket classes, both of which have the same protocol, how do I choose between them? If everything else is the same,.

B

Your request has both a class and a protocol option. First, we need a request. You can say so so.

C

We're going to mandate that class has to be unique. You can't have two bucket classes with the same class identifier, correct and that class has to be specified in the request. So therefore, it's unambiguous, yes.

I

C

I

There would be.

E

C

Resolution issues with having multiple protocols listed, um prevent people from having to mangle their classes. Let's say you have a provider that has two protocols: they'd have to have class protocol a class protocol b. If you couldn't have a list.

A

C

That's the case, so that's.

A

The trade-off david.

G

C

A

B for gcsa versus one class that lists multiple protocols, what what's what's better.

E

um I I just want to point out that somebody said something which is a little concerning, which is that you have to have a class. Oh.

A

Never mind for.

E

Greenfield andrew just for greenfield, okay, right, okay, that that's yeah; okay! Yes, that was my.

F

The other question about the protocols: uh do we have to have them as first class fields? They cannot be like part of the parameter or something part of the. What I was just saying that the particles can can they be part of the parameters like? Oh okay, do they have to be first class? Maybe there were some questions in the past that I didn't catch them. Maybe look.

E

At it, we actually model the the api parameters somewhat. So, if you look in the bucket structure, each of those different supported protocols have their own unique fields so to not explicitly sit put them here and try to hide that behind opaque parameters.

G

Doesn't really match.

E

With the idea of exposing the interfaces to those protocols, yeah.

B

I can expand a little bit on that too. Basically, if you think about it, if you hide it behind parameters then effectively, we need to have a standard way across all protocols to figure out what you know bits of information. We need to expose up into the container for the container to access that bucket and there's no generic common subset of fields that must be exposed to a container.

B

So what we did was say: okay for s3, for define a protocol such that, regardless of the back end, we can say: okay, if it's s3 or if it's gcs. These are the subset of information that must be provided that the container will surface up.

C

We've also blended credentials in we don't have a separate credentials, object.

A

Okay, so that is the provisioning side, the provisioning api. I don't think we're going to get to go to the workflows, but please look at them at the near the bottom of the cap, just above the grpc spec. That shows you uh access apis and provisioning apis in parallel and what the what a typical workflow would look like, um and I just we aren't going to get to cover this review meeting. I I was wondering uh sid, if you wouldn't mind going through the access apis.

A

You and john spent much more time on these than I did I'll, go ahead and scroll the screen and keep sharing. But would you mind uh running the dialogue.

D

Oh yeah for sure um can I can I share my screen. That might be easy for me.

A

Sure, let me unshare here we go.

H

uh Give me one.

A

A

And just as a timing alarm, we are like uh 12 minutes left.

D

B

One four two: six.

A

D

Can someone give me host access.

F

I thought you are today.

D

Yeah I just joined from my computer. uh I'm gonna hang up from this one.

F

Okay: let's see that you are a co-host.

D

D

uh Can everyone.

F

Hear me: okay, let me make you hear you just.

E

Can't see anything.

F

You, oh like that. I think I maybe did the wrong wrong. One.

F

Okay, can you share now.

C

F

C

D

D

D

Okay, so so uh uh jeff just went over how the buckets are provisioned um and once they're provisioned. uh We model uh the access to it uh in in such a way that each workload gets its own set of credentials to access the bucket and consume the objects in it. Now the having a separate set of credentials, rather than shared set of credentials for each bucket.

D

um Let me let me explain that a little more clearly, so when we create a bucket, we don't create one credential object for that bucket and ask all the workloads to share them. Rather, we generate a new set of credentials for every workload that requests the requests to consume. That bucket and the way we model that is using the bucket access uh objects.

D

So there are three objects here, uh just like uh in in case of buckets, so you've got a bucket access request, a bucket access class and a bucket access object.

D

The bucket access request is similar to the bucket request. Here. This model is a user requesting access to a particular bucket.

D

It has fields that that request uh the credentials to be put into a certain a secret name or a service account name. This seeker will contain the credentials and it points the bucket that it's interested in and then the list of credentials that it wants is specified in the bucket access class, I'm going to quickly jump over there.

D

The bucket access class defines what kind of actions we allow. We model um these actions uh here uh to be like the iam policies. uh For instance, um star allows you to read, write and delete, update or whatever. There are no updates here, but rewrite and delete. uh You could do a list only access or get object, which only lets you allows you to get the objects, but not list them or put objects, which is right only and we can do that for by default.

D

It's deny all and whatever we add, uh gets added here and going back to uh the bucket request. The user makes a bucket request by creating this object, and then cozy uses the bucket request in the bucket class to create the bucket access object, which is a representation of credentials.

C

For that bucket, so I really like this um as an application. If I wanted to have a green field bucket created and then be accessible, I would this create a bucket request. Any bucket acquis access request is that correct, yeah.

D

Yeah, that's pretty much it at the high level. uh This is what we do all of these fields here they support that workflow. I can get into any of the fields that we have given the amount of time we have left, I'm keeping it kind of short.

D

Any more questions.

D

Okay, um jeff did you want to continue or.

G

Well, we've got you you have. You seem to have protocols in bucket request class. Is it required, but bucket access, class, yeah support.

M

G

D

Do we need um this is to say that.

D

So so, uh when we looked at uh the policies, the access policies for different object stores, they vary quite a bit between the s3 style and the azure style of doing things. um So if we define a policy action for azure uh or for s3, you don't get the same level of granular control uh with azure at least not as far as we've we've explored. um That's the reason we had that.

C

So I can make a the the link between the bucket access uh request and the bucket itself um is that, through the name or how? How are they linked up.

D

uh The bucket access, so um it is through the uh yeah bucket, the name of the.

C

Bucket, which is got it yeah, so you could have a conflict where supported protocols is indicating a value. That's inconsistent with the bucket uh that it's linked up to exactly.

B

Could we just use the bucket instance protocol.

D

So if you have a set of access policies that cannot be applied on the bucket, this would tell.

C

Us wouldn't we be able to tell either way like, for example, if we had azure style access policy actions, and it was then related to a bucket. That's set up for protocol s3. Wouldn't that allow us to determine that that's not going to work. uh These are allow and deny fields.

D

Are strings at this point.

C

D

So it's not type. uh How would is that what you mean.

C

I just was wondering whether there's sufficient information to not require us to specify the protocol here.

M

D

If you didn't specific, it should be fine. The the the request would go to the back end and you know you'll get invalid argument.

C

Back so again, where would it be? Where would it be validated um currently, while creating the bucket access object? So.

D

C

Access object is cleared by cozy, does the does creating the.

D

C

Access object require accessing the corresponding bucket object at any time.

D

E, um yes, we we create the axis for a certain bucket. So yes,.

C

Yeah, so we already have that information from.

D

There we do uh it's just to make sure it's more like input, validation at this point having the support protocols. uh It's uh it's not.

C

More than that yeah couldn't we just validate from the bucket, though the protocol specified.

B

I guess the problem is kubernetes doesn't allow multi-object validation to occur, but thinking through that, uh I don't know if we want to get into the business of encoding all this validation logic into the kubernetes api.

B

uh You know we're probably going to drift from what the storage system actually wants. So maybe it's better to let the storage system do it. Even though you don't get the immediate gratification of a failed object created.

E

With a web hook on the bucket access request, sorry can you be the question I I get. I'm still, I'm still not quite following the the problem of just so part of the issue.

E

Here is supported protocols, there's a plural that isn't good enough, because the actual access of the bucket of the the actual protocol of the bucket is what matters right, yep, um and so so you have to read that through anyway right in order to be able to validate that this, this path works right so and then the question of where that gets validated. I guess I didn't quite follow the discussion about you can't do multi-object validation. Why are there multiple objects that.

B

Have to be validated, I think one of the proposals was to dereference the bucket instance object and figure out what protocol it was using, and you can't do that. You know in a web hook validator, you can't.

E

B

Kubernetes, api validation, you can't for an admission controller you may be able to, um but that's additional work.

A

Yeah- and I mean a fair point on that andrew I mean obviously it's a better user experience. If there's going to be an error that you catch it sooner rather than later um and and web hooks could could facilitate that we don't have any web hooks as part of this cap and andrew. Would it be okay that that would be a non a later than mvp um feature?

A

E

I completely think it's okay, I guess my my concern is that uh it seems like the argument for putting supported protocols. Here is a kind of belt and suspenders, except that I don't think you're really getting the belt you're, not actually blocking some of the mismatches that could happen.

K

D

A

That's true: um we have two minutes.

D

I'm not opposed to taking this field out. It actually causes more confusion than you know helps um yeah. We we can go forward at this point um yeah. We can discuss this in a future meeting and decide how we want to go forward with that. That's.

A

That we still have the 15 participants, no one's dropped yet with a minute left. So thank you, but let's take a a sod. Can we take um a reading? A pulse, um there's been some good points. Brought up. They've been captured in the video aaron has is going to look at the video and create notes. Sid has some notes from the beginning, um so we'll we will we'll aggregate those together. Where do we stand right now in, in your guys's opinion, on on this cat.

B

I think this looks good. um Let's uh maybe address the feedback offline that we think we have consensus on and then let's do another round next week, uh and maybe we can deep dive into the this cozy spec itself.

A

Yes, the grpc, yeah, okay, so another meeting same time next week and assad you mentioned resolve offline. Some of the points that have been made in what form on on the cozy on the.

B

On the cap, github pull request, yeah.

A

Okay, because I we we have that uh cozy uh slack channel now.

B

Okay, that could be used as a point of uh a place to discuss if you want offline yeah.

C

Yeah, okay, great news about the slack channel- I don't know about this: okay, I'll get myself on that too.

A

Okay, great, let's do it on slack; initially, the pr is cluttered right now, with so many comments. um I I intentionally didn't resolve the comments because I wanted to preserve them, but maybe I need to clean that up too.

B

There's a mailing list as well right. Yes, you could potentially use that if you want dedicated threads.

A

All right everyone one minute late! uh Thank you. Please. Please look at the cap and add your comments there. I do try to respond to every question or piece of feedback on the cap um and we will shoot for finishing this up a week from now with the intention of getting approval on merging a quick question. What's the name of the slack channel?

A

Oh, oh yeah, that's good! Everyone should know that it's uh so it is quizzing. Okay, yeah! It's sing, dash storage, dash, cozy, sweet.

B

Thank you all right, take care folks. Thank you.

A

Thanks everyone, thank you, jean for recording. Thank you. No.

L