Kubernetes Storage SIG COSI, 4 Nov 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes SIG Storage - Object Bucket API Design Meeting 20211104

Description

Kubernetes Storage Special-Interest-Group (SIG) Object Bucket API Design Meeting - 04 November 2021

Meeting Notes/Agenda: -

Find out more about the Storage SIG here: https://github.com/kubernetes/community/tree/master/sig-storage

Moderator: Sidhartha Mani (Minio)

A

A

Yeah so we're trying to figure out im style, authentication at least have a clean story around it uh before before bringing them again.

B

Okay, yeah, uh I think uh I think there was uh some deadlines. um Let's see, yeah the cold freeze deadline coming. I think it's. What is that code free 16th november 16th and then, and then there is this? uh um There is this deadline november, 23rd that that is basically for us to uh submit some entry to show up in the major major things for 1.23 release, but yeah. So, let's see if we can get it in before that, because otherwise we can't.

B

We can't add this to the major themes. I like this one to show up as the in the release blog.

A

Yeah, I I agree: okay, so yeah. Let's try to get hit that deadline. We have. We have about uh what 12 days now still.

B

Have a couple of yeah weeks, but then it's holidays right so right.

A

Yeah nobody's gonna, yeah, okay, so uh yeah, let's start off uh today, so in terms of the overall plan, if we figure out uh workload, identity based authentication, if we have a clean story around it, I think things will move very fast, because that was that was the big thing that that tim wanted addressed uh it's. It seemed very important for uh google cloud and I would presume other cloud providers also would say the same thing so far.

A

We've mostly you know, had vendors uh non-cloud vendors, uh making uh progress uh on this and and for us all, uh I would think uh access key secret key based authentication is, is the easiest step. Sometimes we run in air gap environments. So um you know these.

A

These tokens are good enough for us, but but in in places like google cloud, where it's public I I can understand why they would be more focused on more advanced types of authentication, so uh yeah, let's get this figured out and we'll get back to tim again and we should be able to make it this time all right.

A

So I spent some time um trying out workload, identity, based authentication, and uh here let's see yeah. So it's it's very straightforward. It's it's what we guessed uh it works like uh last week when we were talking. um So what I would like to do is explain how this works, and uh I think this will help us model the api and answer some of the questions we had last week, much better.

A

So what what kubernetes does in or what gke does google kubernetes engine does is. First, you have to enable something called workload identity, uh but but they recommend that you know all kubernetes clusters um that are run by gk enable this. uh If, if there are workloads that talk to the gke itself, google cloud itself for, for example, like buckets, so um what it does is it creates. uh It creates a iam account for every service account that you create.

A

So basically, if a pod runs as a particular service account, then that pod is mapped to a particular identity inside google cloud, and if that identity has the permissions to access a bucket, then this part also gets permissions to access the same bucket.

A

So there's a lot of text around it, but that's basically what I just explained is basically what happens? Okay, so once you enable it, I want to quickly show you something um yeah you you can enable it by default, it's gc metadata. uh Otherwise, it's gk. What this means is um there is. There is something called as a metadata service that runs uh whenever you're running in the cloud um you can. You can query the metadata service and based on who the source is that's querying it. um The result is given um according to that.

A

So if I'm quoting from node one- and I ask for node name it'll, say node one, if I'm going from node two and ping the same api as for the node name, it'll say no two, um so when you, when you enable workload identity, then the workload metadata gets set to something called gke metadata.

A

If not, it gets set to gc metadata. This gke metadata is important. If you want to do workload, identity, style, authentication, the gke metadata is the metadata service that understands the mapping between service accounts and kubernetes and im accounts in google cloud okay. So that was a that piece of information is important, um even though it's slightly uh tangential.

A

So once it's created, uh what you can do is, uh uh let me see yeah you can just put uh you know, take a service account that you created, and uh you know, provide it to a pod and the pod can query the metadata service.

A

Metadata flavor equals google and it can just query it and when it queries it, it gets a response accordingly. So.

A

So yeah, so you know you get service accounts and, uh let's see all the gke or google cloud, sdks already know how to work with this. So if you take any of the sdks for talking to google uh storage uh buckets, um they would just be able to talk. If, if you already map the service account to the im account for the bucket, then you know any sdk. That's that's built by google will just know how to use this um yeah. This is this is what the metadata service looks like.

A

uh These are the apis it supports. um So um so, if we were to create a pod and using that part, if you were to query uh something like this, um then you mean any of these apis. You would get the right um account tokens or whatever to yeah. This is the token um to actually talk to the backend.

A

There is one limitation I can go over it, but so basically the the access model is, you have an im account and you enable workload identity uh to map that im account to a service account kubernetes and if a part runs with that service account it has uh that it has access to the backing.

A

C

Hey sorry, I came in late. um I did have a question in terms of provisioning, so I think that means when you provision a bucket, you need to provision it with using that identity as the owner of the bucket. Oh.

D

C

A

Right, no so um bucket life cycle and and identity life cycles are are independent, so bucket need not be provisioned with a owner. The organization under which the bucket falls into that's the owner of the bucket.

D

C

How do you give access to this service account.

E

D

Like you need to be able to configure.

C

D

Say like this service account can access it.

A

Right so there has to be a an admin uh user and a user with admin level permissions. It can be a it can be a service account or it can be a you know, real user, um and they would have admin permissions to update and create iam roles and imm identities, and and they would they would have to create a service account in kubernetes, and there is a way uh google cloud provides to map this kubernetes service account to the service account in google cloud and once that mapping is done uh it's somewhere here yeah.

A

So you create a service. Account im service account like that, and then you add: oh yeah, you create im policy binding and the policy binding binds. The google cloud service account to the kubernetes service, account.

E

So uh to be a bit clear like there's, no, the cosy does not have a load over here. It should.

A

Be managed by admin: well, no, okay, so that is also so. Let me go over the lifecycle, so bucket provisioning will happen through cosy where it will create a bucket cosy in order to run it. It needs some privileges to do it and we expect cozy to run as an admin that can create buckets and also uh you know, provision access for those buckets, so provision.

A

Credentials for that bucket, so cozy's job here would be to create this iam policy binding, which will bind a service account in kubernetes to a iim account in google cloud, and once that binding is done mount the service account into the pod that is requesting access and now that part will have access to talk to the bucket.

C

So I think, does that mean in terms of provisioning?

C

We need to have some parameter to for the pod service account that we want the provision on behalf of.

A

Yeah so um or we create the service account from scratch um rather than reuse, something existing. um I'm not sure. That's that's what we need to discuss right now.

C

Yeah, I would look at um so actually csi. We just added an equivalent feature called csi service account token, um where basically well. This doesn't happen at provisioning time, but it happens at mount time.

C

But during the mount call we pass in the pods service account so potentially for cozy. There could be some equivalent kind of thing, um at least for at the uh time when we modify or set the access policies.

A

Right right that that's exactly when we would need it. So uh is this something that's going to happen at no, no, no stage volume or like not published volume.

C

Yes, in csi, it happens.

D

F

uh Publish I see and and is this uh is- is this meant for.

A

Like how is it used.

C

So this is meant for there's some csi drivers um that are actually like uh secret stores. So, like there's, a csi driver that that basically acts like kubernetes secrets, but it actually gets the secrets from vault.

G

C

An external secret manager.

G

C

So what happens is when the pod does amount? We pass in the pod service account to the csi driver, and then the csi driver can authenticate to vault using the pod service. Account token.

E

C

So I think it's it's a little different for the cozy yeah.

E

C

Right for cozy, we'll want it earlier at at the um the time that we set up the access policies right.

A

Right, so we we um so so far the model we've been following is uh cozy would have a static set of credentials which, which would be, which would be like getting admin privileges across the board, to create buckets to do everything related to buckets, uh rather than a particular service account or a particular key for that particular operation, um so is doesn't csi also have something like that where, when, when a new volume needs to be provisioned say from the cloud, the csi drivers controller would would have permissions to talk to the cloud say if you're on amazon create ebs drives and also attach them to the nodes that that you wish them to be attached to.

A

C

That is the same model. I think the the main difference here is that object, storage and- and I think that's still the same like I think.

A

C

Cozy driver still needs to sort of have admin privileges to manage the buckets.

E

C

Think one more aspect to it is adding additional owners or access to the buckets.

G

A

Exactly so another question I have, and I'm not sure about this- so is it okay to create new service accounts each time a pod wants access to a bucket, or should we latch on or add permissions to an existing service account?

A

The reason I ask this question is if we were to create our own service accounts and then, and if the user wanted a different service account for their part, for whatever reason, um wouldn't we be conflicting.

C

Yeah, I mean, I think it is better to have like it could be as part of provisioning like we require the user to give us the service account that they want to use for that.

A

Yeah so provisioning or during workload creation.

C

During I think during whenever we sorry, I I forgot the latest, but we have like a we have a um an api for access control. Is that right.

F

C

Policies, so I think, maybe as part of that access policies api, we also have a field to say: hey. Here's a service account to like allow access to.

A

Yeah that makes sense yeah, that's that's where I would. I would think it goes to okay. So so we take service account as an input, and then we do the cosy magic to do the iam policy binding and the service account goes back into the part.

C

I would also um one thing to clarify the the cozy magic to do the binding I I assumed that would have to be done by the driver yeah and not like a generic cozy component. Okay,.

A

Yeah yeah, so the cozy's role here would would simply be to let the driver know I want. I want you to do the I am binding to a service account. Rather than give me uh access and secret keys.

A

So so this is, this is where we left off the discussion last week.

E

Yeah so last week it was kind of a decided like the driver will provide everything and the application or like it can pick up whatever it wants right.

A

Yeah yeah I'll, explain I'll, explain what uh jeffen is saying here, uh so one of the so so this is the overall architectural workflow. We decided just just what we just said now um the service account comes in as an input or something like this earlier. The service account comes in as an input and cosy.

A

Does the binding of the service account to a cloud using the driver service account to a cloud entry and the service phone goes into the part and the application is presumed to have the necessary facilities to use this uh service account based, authentication um and and as far as official cloud sdks are concerned. Now all of them support this style um if you're using a vendor sdk.

A

um That is that is not the case. So if you are moving from on-prem to cloud- and uh you know, let's say you you're using min io now you're moving to google cloud, I mean they're incompatible protocols, s3 versus gcs, but but let's say, google cloud suddenly started. Supporting s3 um min ios sdk will not know how to talk to google cloud's metadata service to uh leverage the service account cell authentication.

A

So that's that's one limitation to know of which is I mean the whole point of this is uh portability uh and we say we allow portability as long as the protocols are the same, but we are adding another.

A

You know star to it saying um disclaimer to it, saying that we allow portability as long as the protocol is the same and the access mechanisms are supported by um are understood by by the sdk. So there is this reliance on on the application sdk's uh facilities here, and we want to model that in the api. Somehow we want to model this in the api.

A

Where we say here here are the authentication mechanisms this driver provides, or this particular environment provides, and uh there needs to be some explicit contract to to denote that or we can even it's up for discussion.

A

I would say.

A

So say, for instance, there is a there is a client. There is an application that requests access to the bucket.

A

The first question is: should it be able to say hey, I want uh access key secret, key style authentication or should be completely transparent to that workload, so developer who's, creating the workload should they be able to say here's the style of authentication. I would need for my workload.

A

Anyone if they have any thoughts, feel free to pitch. In the second question since silent, I'm gonna go to the second question. um If, if it's not the application that says uh hey, I want I want workload, identity versus access, key secret key- uh maybe maybe this this field should be a part of the bucket class or the bucket access class.

C

I guess since since this does require like the application to have the right sdk like I, I'm okay, putting it at the application level, api mm-hmm, it's something they have to like consciously.

D

A

Right right, um different, you were saying right.

E

Last week we had a.

A

Go ahead, sorry go ahead.

E

Yeah like last week, we don't know the input part right, so this is a dependency from the user to provide the service account right. So in case of access, key and secretary user did not don't want to provide anything, but we can automatically mount that information to the port. But here we are expecting user to give the service account details right, like it is expected from the user part right. So we are.

A

E

That field for that.

A

Yeah yeah, so so one of the arguments last week was- and this is this- is a good argument really if we want to be portable um and if we had this particular requirement on authentication style, um we wouldn't really be portable because you'd have to change the authentication style when going to a different environment.

A

So shouldn't this shouldn't and- and we do the same with like pvcs and pvs, where the pvc doesn't say um what kind of file system is going to be on the on the on the pv. That's that's the configuration that comes from the storage class.

A

uh Similarly, pvc doesn't also say whether it's a read write once or rewrite many type of volume. Again that comes from the storage class. um That's one of the ways pvc stays um portable.

C

Oh sorry, I think, read, write read, write mini is on.

D

A

Tpc object; okay, I might be mistaken there then. um Oh really.

C

Yeah I mean, but that so like that also requires, like application. Awareness right, like a application specifically has to request.

C

If they want like a read, write once or read, write many because it's it it will impact the way that they deploy their application.

A

Right right, because you know sometimes you'll need to do right, locking if you're, using nfs or something yeah, that's correct, all right, so huh that that is confusing to me, because um what, if I were to move between a provider that loves rewrite many versus another, that doesn't.

C

Yeah that I mean that's, uh it is a problem right like in with pvcs and pb. Today we have a bunch of um we. We do have a bunch of features that are dependent on the volume driver to support, like snapshots, for instance like if not all, drivers support snapshots. So if someone creates a pvc and then takes a snapshot and tries to take a snapshot, but the driver doesn't support it they'll, you know get some error or something but like they don't really know.

C

Until unless they've read the driver documentation uh to see what it supports.

A

Interesting, okay, so so pv portability is dependent on multiple things. Then it's it's dependent on um yeah, well, actually just one which is a rewrite mini or volume mode. It's portable. As long as uh the volume mode is, you know, supported by the driver right, correct, okay, understood, okay, interesting! So if we were to put the authentication mechanism, okay, so that that argument would also apply for the protocol, then s3 or gcs, um and so far the argument for protocol has been.

A

You know that it should be in the class object should be something that the admin sets based on based on the driver. um I'm trying to remember why we made that decision. uh Jeff. Do you remember.

A

H

I said, I'm sorry, could you repeat the questions so.

A

The question is: why did we decide to put protocol inside of the class? Yes, I know we went back and forth on it a few times.

H

Yeah we did go back and forth and at one time we thought it was a list of protocols in the class and the br. The bucket request could just name one of them, um and then we went away from that idea.

H

E

Question because.

H

The driver that you're that you define in the class um implies a protocol, so I don't know, I think it would just seem like it was an obvious attribute, um and maybe that was all the thought it was given. Okay, so in the cap we had some notes about it.

A

um Yeah, we will know why we put it there, but but the reason we are going back to protocol is because um this this, this authentication style, isn't very different from the protocol field.

A

Yeah protocols are not file systems, the bucket is not formatted, so yeah we have protocol inside of the uh developer, control resource and tim was suggesting. The protocol should go into the class and his reasoning was: the class represents the provider plus parameters.

A

The provider has intrinsic support and non-support for protocols no need to list protocols. Okay, this this field also works the same way.

A

Memoryless protocol in bucket class, but even that seems overkill to start, we can always add to the api. We can never take it away, interesting, okay, so.

A

Any thoughts about this argument that tim is making.

A

So all he says is a field like protocol, something that is intrinsic to the provider.

A

Or the driver should not go into a user defined resource.

C

C

The application needs a specific protocol right because they, the application, has to like include a specific sdk.

C

So yeah, I kind of feel like it's like, if say you import say like a g cloud: gcs sdk, but then you go to an environment where you only have s3 like I. I assume the application has to know that the provider can't support it. Somehow.

A

Yeah I mean this. This is one of the challenges uh with with um trying to go off the pvc approach uh or or trying to retrofit some of the things we did with pvcs and object storage. um The data api is is different between different providers with pvcs, it's always posix or pvs.

A

It's always posix read write calls are the same, regardless of who's, providing the file system or block device to you, um and- and in this case you know, there's two things: data api changes like s3 versus gcs and also this authentication mechanism to talk to the back end changes.

A

That being said so so far, we we've we've modeled this, as if uh those details are always hidden from the user.

A

The admin is expected to provide the user with the storage class name, and this storage class is assumed to always provide a bucket um that the user expects. So if the user expects s3 buckets, then the storage class is supposed to provide some s3 compatible bucket that might be from any vendor could be from from aws or any any other vendor out there.

A

That's kind of the model we've been following.

H

I mean one reason to have protocol in the bucket class is for drivers that support multiple protocols. Then the driver could grab. Could the the sidecar can pass the protocol down to the driver, so the driver knows which protocol it's speaking.

C

But I guess does.

C

Does the driver care which protocol like is there actually different things? The driver would do if the protocol changes, because it's I assume the driver like creates an endpoint.

E

And the endpoint.

C

Can support any of the protocols.

F

Yeah, that's interesting.

A

um So sorry I mean: are we discussing hey jeff, there's a lot of echo? Okay, I viewed it now wait. How did you say that family, okay um anyways um uh so so, are we talking about uh having one driver, support, multiple protocols.

C

Yes, I think that was the scenario that jeff was talking about.

H

That's what I was saying, I think it's that's a low or probability use case, but it it is possible and google may have may do something like that. um Yeah I mean gcs supports both gcs.

D

H

Oh okay, we did have a thought michelle when we at one point the bucket class had a list of protocols and what we decided for simplicity is that in the case say of google and they support multiple protocols.

H

You would just define multiple classes so that we were doing one class per protocol even with the cloud provider, but.

C

I guess now it's like the the uh is it a list versus like a single field.

H

Yeah now it is a single field and I think what we're discussing is do we need it at all in the bucket class? No.

A

No, I I don't want to get distracted, so um the bigger question is: uh do we do the same for authentication model or not? I I don't know if we should go back and question the protocol argument. Yet unless we have a strong case um for for for choosing not to have or having a list uh and and the same argument, you know whatever we decide for authentication method would apply to protocol as well. But I can see that these two being uh you know symmetric, I would say.

C

A

C

Think my thoughts, sorry, I have to um leave early, but I think my my main thoughts are that, like what like, the higher goal we want to achieve is an application wants to request like a specific protocol and authentication mechanism, and the driver needs a way to say if it supports it or not.

C

I from an api perspective. I don't know if having a field in the bucket class is necessary for that, like you could just pass the parameters down to the driver and the driver could return an error. um That's sort of my.

D

A

Cents on this um cosy needs to do some orchestration based on the authentication mechanism you ask for so, if you ask for access keys and secret keys, because you would request for that, get create a secret object out of it and mount that secret into the part when it starts up, but the service account it does it differently.

A

C

Yeah, I mean that's fine, uh I I'm saying like. I think it's fine, if, like in the bucket object, there's like some field there for authentication method. What I'm, I guess not clear on is whether or not you actually have to have a field in the bucket class, and I I'm thinking you don't.

A

Oh okay, to to denote, if uh to denote the authentication mechanism, you mean.

G

A

Interesting uh and and you're saying that that would just be uh defined in the uh in the bucket request of something that the user requests right.

C

Yeah- and it would basically just get passed through to the driver and the driver can error if it doesn't support it.

A

So question of portability here um is that a concern at all here like based on this approach? Can we um yeah? Is there a concern with this approach? Not this portability important but more. Like um does this in this model? uh If you wanted to move from an environment that allows authentication mechanism a to an environment that doesn't allow it but doesn't support it, are we now not portable?

A

C

Yeah, but I think you can argue that this is similar to like any like volume feature we have today right, like you, could be using volume snapshots in one environment and move to another environment where snapshots doesn't work, so I think it's a similar boat. I think what the maybe the important part is is like, if say, the feature capabilities were equivalent on both environments.

C

Can you still use the same api.

B

Yeah, so I guess the difference between this and the snapshots is snapshots like optional right like particle, everybody has to specify some protocol right, so this is like a required field.

C

The authentication meth method could be required, but, like it can default to you know something else.

A

The the easiest one access cases.

A

Yeah yeah, I like this. Actually it's a very simple and it's similar to how pvcs work with uh volume modes all right. So, let's, uh let's revisit uh what the decision from last week was different. Do you remember what we decided.

E

So last week, like the decision was to like a driver is the holder like says it can support lot of protocols and when the application is mounted with a bucket, I mean bucket claim. uh The driver will give all the information on the port or the application like for the, for example, in this case for identity base, it will be the service account for the secret and access key. It will be saved in a file like that and the application can pick any one of it. Really if you it's like that,.

A

But was there any representation of uh the authentication mechanism in the class objects or or in the uh bucket request, or bucket access class or bucket access request objects.

E

uh Okay, let me check the notes like I guess. I have written something: okay, uh yeah, the storage class. Have the list of authentications and driver can pick the authentications from the list. Something like I see.

A

Yeah, that seemed a little weird to me.

E

uh So yeah that's! uh This is number three like which was discussed last week.

A

Yeah, for the sake of portability, but for the sake of saying that it is portable, we, we were kind of going down this road where we were saying. Oh, we list everything that that this application supports, or this driver supports the driver is free to choose one. I would think that is even less deterministic than um not having the field at all.

E

So I guess it's better to check with tim again like. Can you ask him, like I don't know like well.

A

E

Some authentication like whether it's in glass or like whether it is on the cozy driver or whether it's on the.

A

Application yeah so the way he's thinking about it. You can see in this comment itself. um He says even having the protocol in the bucket class seems overkill. um He's thinking about just transparently passing it through, like, like michelle, was suggesting.

A

So we don't even need to represent it anywhere either the class or the user. Well, the user objects somehow and just pass it along. So how would that look? Like sure would do you have any thoughts on what that would look like where how the user would say? This is the authentication mechanism that I want.

A

She talked about, okay, so yeah. So that is something that we need to figure out. um How would the bucket access uh look like when they say I want access, keys or secret keys versus when they say I want?

A

A

So who's here on the call.

E

So we still have bucket taxes class right. It's still the right authentication people protocol is the highest level like at least for me and authentication is the next level under protocol right. So as calendar process, the protocol will be staying on storage class, and can we have something on the bucket access class? Saying authentication mechanism will be part of this, like this authentication, like this worker class, I mean bucket access class supports this authentication and all the bucket workers created from that bucket access class will have that authentication or something like that.

A

I see um yeah so so what you're saying is um you you want to define on the bucket itself uh that this thesis authentication mechanism supported? Is that what you're saying I mean the bucket access class or something like that yeah yeah? So that is what we discussed last week right that we'd have a list of uh authentication mechanisms written down in the class object, and that's.

E

Back at class rate, like I mean, there's two like this bucket class and bucket taxes, class right or like bucket two couscous. I don't know.

A

So I don't know if it makes sense to put it in the bucket class, because authentication is separate from the bucket life cycle as in to the same bucket.

E

I don't think it won't be part of the bucket class, so this bucket access class right, okay, which is the life like which represents the life cycle for the authentication right. Basically, it's not the life cycle of the bucket right.

A

Right right so you're saying we would we would specify the authentication mechanisms like multiple in there is that what you're, saying or just one.

E

Just one like something.

A

Okay, so yeah that's what we were talking about just now um and what michelle seems to think is: maybe we don't even need to specify it there.

B

Sid I'm confused, I'm looking at your cap looks like the protocol right now is in the bucket requires spec and then team has a comment they are saying. I thought this move to class. Is this yeah? What what is the current state.

A

No, we want to.

B

So now we're talking about moving the front, the class to the uh bucket request, but it's already there. So I mean I mean yeah.

A

We're moving to classing so.

B

But michelle was just saying that it should not be in the class right. That's what she was talking about earlier. So.

A

What what tim is saying is: if you're going to have it, we should have it in the class. What michelle is saying is: um why even have it in the class, we don't need it um so michelle's thinking we can pass it uh as a parameter in the in the bucket request, object.

B

That's what what we have right now right in your current cap.

A

Yeah yeah, so the these two are completely opposite of what each other is saying right, like tim, wants it either in the class um no need to list protocol in the bucket request, so he doesn't want. The request may want a list protocol in the bucket class he's okay with putting in class, but he sees even that is overkill to start so yeah we need to. We need to. um There are no there's no wrong answer here.

A

It's just different styles. I would rather go with what tim is saying right now, because tim has been more involved. He knows this more deeply, um so so going accord by this we would have uh you know a list of authentication mechanisms in in the bucket access class object. If you were to go this route of how tim is thinking about it.

A

Similar to protocols we'll have authentication mechanisms, and that would be in the class object, and when someone wants access to a bucket they would refer to the class object and based on what we spoke last week we said yeah, so that part is still confusing to me. So so, first before you go there shane uh does it make sense, is whatever we talked about just now.

B

Keep it including the class yeah.

A

Keep it in the class yeah.

B

I'm I'm fine with that. I was just thinking. This is a different uh because enemies. I was using snapshots of example. I thought this one is like everybody has to specify this and then does has this uh portability issue? I think if it's in the spec.

A

Right right, exactly if it's in the class, then it's not there, the portability is easier. My only concern here is, you know. An application also plays a role here, uh unlike unlike with pvcs and pvs or unlike with snapshot.

A

The application has a role to play here. If, if it is an s3 bucket, then um you know the application needs to have an sdk that that knows how to talk to s3 the gcs bucket. The application needs to have that too. So, right.

H

So this is the whole problem that we don't have um a posix standard for bucket access, and so the the sdk is defining. What is defining the protocol inherently and and the application uses the sdk and cosy has no idea what sdk an application is using that a workload uses. So so there's always a chance, no matter what we do, no matter where protocol gets defined or no matter how a driver name is used that the application is out of sync with the underlying bucket right, and we can't.

H

I don't know how we can uh prevent that.

A

Yeah, I don't know if we can, I mean we could do one thing where you know we. We create a three different bucket types like it's. It's a it's not just bucket, there's no bucket anymore, I'm just like hypothetically going there. I don't. I don't know if you should even go down this route, but one possible solution is to standardize three different types of buckets: uh s3 bucket gcs buck in azure. Buckets that way. There's there's no single! You know generic bucket. It's just one of these three.

H

It is, I mean it's, I don't I don't know if we want to go down there either, but one advantage of what I hear you say is that um it is it's honest. It acknowledges that there's not portability between the different bucket types.

A

Right right, I'm saying maybe the abstraction isn't bucket itself, but rather a bucket the specific type of bucket.

H

A

That would that would solve this problem. Actually.

A

uh That would solve this. uh This problem with uh you know having needing the application to define, define the data protocol, but I don't even know if we need it because see if you look at the other resources in kubernetes, um say.

A

Let's see, say, services um services only behave in in one way right. They just route your request to one of the pods. um Let me try and remember this: um does service do tcp level load, balancing yeah it does because it's an ip tables. huh Let me think about a different resource where there could be a possible mismatch.

A

I think I think I think I think we need to looking at how pvcs do it looking at how pvc models the volume mode, because application needs to know uh if if a volume is rewrite many or rewrite once um the pvc allows you to specify the volume mode shouldn't uh shouldn't our bucket uh access request or bucket access now. But that's what we're calling it now um model authentication mechanism, because it's very application. Centric.

A

Does that make sense, so you know when I think about it more, I feel like we should. We should go down the path that michelle is suggesting.

A

It seems to me like it is better to have protocol and what is that? Authentication mechanism in the bucket access object.

A

Well, protocol would be in the bucket request, and uh authentication mechanism would be in the bucket access object.

A

And then we would have perfect portability right.

A

Okay, so yeah go ahead. Jeff.

H

Well, I just saying I I agree with your division: the separation of the protocol, if that ends up being defined being on the bucket creation side and the access being in the bucket acts uh and and the authorization or token uh mechanism being in the bac bar side.

H

That seems natural.

F

A

Yeah jeff, I'm hearing an echo so yeah, no, it's gone so so. Where do we put it though? The the question is: should it be? It should be in the class object. We should be in the user, controlled object and, as I think about it more, I think it should be in the user controlled object. It should be in the bucket request for protocol and bucket access for uh the authentication mechanism.

H

If we move more of these properties into the user objects, then you I, you start losing the value of having the classes. I think.

H

How do you, how do you well, then, a class really just has a driver name and a and parameters a map, maybe deletion pilot.

H

That's a good point: yeah you're, right, xing,.

A

Yeah so so class would be um provider plus parameters for that provider, except here the application is involved right. So my my reason for not putting it in the class is because there is a there is a role for the application to play here and, and that should be modeled somehow right or are we saying like it can't be in the class right?

A

This class is controlled. The application is developer, control.

H

Right, that's the that's. The conflict. Sid is the the workload picks, an sdk, which we know nothing about, so they are involved in this.

A

Yeah, that's what's making me think. Maybe we should. We should go down this path of uh uh saying that uh um protocol you know remain so what michelle is saying is is very similar to what I just said, but but with another um uh step further she's saying don't even model it, don't don't even have um protocol in the as a separate field or don't even have authentication mechanism a separate field just pass it through blindly.

A

Right that does make some sense to me. Well, well, um now that I think about it, even that that might not be such a good idea.

A

Like like, if because because then you know we'll have we'll have vendors creating their own specialized protocols, so you know um mineo won't do this, but you know we might we. Let's say menu, creates its own mineo protocol, which is basically s3, but we just call it minio and then now we're suddenly not portable between minion s3. We would never do.

A

That makes no sense, but I'm just just saying we would uh we might I mean this is what saad was worried about initially when when when we were discussing about having a an opaque set of fields inside the bucket request or in the user control resource- and we said that's not a good idea, because we would end up with vendors implementing their own proprietary ones,.

H

A

H

Protocol altogether, it's not in a br, it's not in the bc addresses that issue. Doesn't it.

A

So how would how would you model it then shouldn't it be specified somewhere? Tim is saying no just to be clear.

H

If it matters the driver can return the protocol.

A

Okay and what about authentication then same story.

H

Maybe I I need more time to think about it.

A

Yeah I feel like. Maybe if you write it down uh and- and you know maybe wrote down thoughts about it- it'll be clearer.

H

Right I agree. Xing had mentioned that in our last meeting that if we could get something written down, we could post the link to sig storage, cozy and and then we have some ability to digest. It better think, do a little research and make comments. But, okay, it's it's all. You know we hear it and there's a recording, but it's it's hard to give it a lot of thought. I think.

A

I agree okay, so I will take up that burden because I I I want to write it down just even for myself um and and uh and yeah. Let me let me um let me give it a try. Let me see what thoughts come up when I, when I write it down, maybe you know things will get clearer.

H

That sounds great sid, okay,.

A

And then I posted on six storage, uh cozy.

A

Yeah all right, so I look forward to your feedback there. um I think if we just figure this out, we should be in a good shape uh and, and we can, we can convince him to push this along.

A

um Yeah, that's it everyone! Okay,.

B

All right, thank you.

A

Thank you. Thank you. Are you saying something awesome said.

E

I hear a question like if you haven't yeah.

A

E

So I should have asked this before, like you have mentioned how the authentication it means I am, identity, authentication works for gc gcs right, so will it be the same for s3 as well or like yes, yeah uses.

F

The data service.

E

Yeah yeah, okay, I like it, you also have the service account and you need to bind to a account in the yeah. I mean something like that.

A

Yes, that part is almost exactly the same. Yes, okay, yeah yeah sure bye. All right. Thank you. Everyone! Thank you. Thank you. Yeah bye,.