Kubernetes Storage SIG COSI, 1 Oct 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes SIG Storage - Object Bucket Review Meeting 20201001

Description

Kubernetes Storage Special-Interest-Group (SIG) Object Bucket Review Meeting - 01 October 2020

Meeting Notes/Agenda: -

Find out more about the Storage SIG here: https://github.com/kubernetes/community/tree/master/sig-storage

A

Okay, so um last week, where we left off, we were talking about what monday, where we left off. We were talking about uh how the development is coming along. um As of uh as of uh uh today, uh we've started working on the sample. Provisioner um we've got a new developer sergeant, helping out with that um and uh development, as of right now is, is going at a good cadence and and it's moving forward, um I would say uh we're still on track to uh do the demo in a few weeks um again.

A

The way I was defining the demo was as a development milestone.

A

That is, I I want to have a high quality code, not prototype uh level code and and uh have it with testing in place and do the demo, as as the end point of finishing a certain number of tasks.

A

So the demo is going to include the simplest use case of creating a green field bucket, uh creating a bucket access for that green field bucket and then giving that bucket to a part by by through the node adapter.

A

So this demo covers all three components of of this system.

A

Moving forward, we have some good news, so my talk for cubecon 2020 was accepted. We're going to be talking about cozy um and and yeah it's going to be exciting and I we we will definitely get more uh visibility, probably more contributors after this stock. I think that'll be good.

A

So the talk we originally submitted was with the john cope, and I I don't know uh if everyone here knows john koh, but he was leading this group before I came along um and uh when we both were here at one point, we decided to submit this, uh I'm not sure if john is going to speak at this stock. This time, uh however, if john cannot make it uh jeff who is taking out jon's version will will uh help us out with that.

A

So it'll be me and jeff, if not me and john or john, and I um going to the next thing, so uh the deadline for cap merging is is coming up it's very very soon, so I want to spend a majority of the time today going over anything left in the cap. That needs to be fixed.

A

We got a bunch of feedback this morning from uh andrew. Thank you so much andrew uh and uh you know I would like to go through them and answer the questions that are there or figure out. uh If there's anything, we don't know that we need to answer in a different way, so I'm going to open up the email so um so andrew. I haven't had a chance to go through this. It came in like what seven minutes ago. I know sorry about that.

A

It's okay, um so are these? uh Are these mostly uh like? Are there any technical issues, or most of them seem to be about just the wording or several rewarding related.

B

um Probably the biggest, um I have two sort of general questions that are technical questions and they're. Not necessarily. I think you got a wrong comments. They're more like I don't quite understand, okay, the first is around anonymous access mode.

B

um The the driver, the the cap says basically, oh yeah. This basically doesn't have any effect, um in which case I I I don't know if that statement means that really or uh yeah yeah. So that's what I was kind of trying to to get at there. Okay, so I yeah. Let me open it up uh and I can quote you the actual, no.

A

Okay, yeah, please.

B

B

Oh, that's right. This one right, yeah nope, does not reflect or alter the backing, storage, apples or iam policies.

C

Right yeah, uh I was on mute andrew. um You know this is a carry over some. I think um it's john's, it's john wrote it and I'm not sure I fully understood how it was used and I sort of as I was refreshing, the kept didn't really focus much on there. My understanding was, if you don't have a more formal policy. This is a way to say: uh what's the access for generally open buckets for anonymous access, non-user, non-specific user type, access policies? What what?

C

What do we want as a catch-all policy? But I'm not sure if my interpretation is correct.

B

Yeah, I just don't understand how this would be. I mean I I could understand if um if if, if this actually should be reflected in setting public read-only access to the bucket, but that line below it says: oh no, no we're not going to do that. So that's I'm just confused! That's all because it's just decoration, I'm not really sure. I understand why it's there.

A

Okay, yeah, that's yeah one.

A

One answer to this is yeah: maybe it shouldn't even be there, but I think we're trying to say that uh generally in object, storage, uh you know there is, I think, four different ways of setting access uh or, or you know, doing, access management and uh accols, iam and- and there is this catch-all uh bucket level uh policy um which is anonymous, access which does not follow the im way or the echo way, or uh I think there is another way to do it um sts, I think token service.

A

um So so I think that's what we're trying to say technically, it doesn't use articles or im policies to achieve this. Okay, so so I think it's just the.

B

Way the node is worded, then yeah.

C

So the note to me andrew reads if you're using anonymous access, if you set that um in the bucket class, it does not impact any of the underlying object stores policy.

B

Well see, that's what I thought it meant too, but that's not what sid just said. I mean we're.

C

B

What he said it would implement the underlying access policy, it would change it, it just wouldn't be apples or iam, it would be hey. General public reader write said.

A

Jeff, I didn't hear what you said. Can you repeat that.

C

Yeah, I guess I guess I see the source of compute, I'm saying that if the you, if a bucket class describes, defines an anonymous access- and so it's reflected in the bucket instance- object- which is what we're looking at here- the the cosy itself isn't going to make any kind of driver call. So what the grpc will pass in this private, public or public settings. The three public settings are private right, we'll pass that down to the driver.

A

So you're saying you won't be creating a bucket access thing for it. That's what you're saying.

A

We would have to pass this along during the create bucket call right.

C

A

Okay, sorry, maybe I didn't understand you fully, could you can.

C

A

C

You just mentioned bucket access. This isn't going to do anything with bucket access right.

C

No we're just looking at um anonymous access is in is in the bucket class, so we're talking green field buckets only and um it's copied into the bucket instance right. So.

B

C

Talking about it andrew's, looking at our definition and um oh the definition, it doesn't have the note in bucket class like it does, for the bucket instance. If you go to the bucket instance and you look at anonymous access description, um number six you'll see a note after it says, does not reflect or alter the backing, storage, apples or iam policies.

C

um What was the explanation for it? Well, I think john was trying to say we aren't going to that. He doesn't expect that a driver would alter any underlying axles or iams if you're using anonymous axis, which seems like a reasonable thing, but we don't have any control over what the driver does.

A

No, no yeah. I don't think that statement is correct. um So so that's like saying: if anonymous access mode is set, then we don't do anything with eccles or iom policies correct.

A

That is. We just want to.

C

Is isn't that what you're saying? No, I'm saying that in the app in absence of more restrictive policies we would default? We would. uh We would expect the uh anonymous access to be followed.

B

Right but but following anonymous access, I mean there's two parts to this. One is imagine this is applying to greenfield. How do you implement that right? That's the first problem, the second problem, which is what sid just kind of mentioned, which is okay. What are the implications on bucket access when you've got anonymous because you don't need any kind of credentialing if anonymous access is available right, and so it almost strikes me that this ought to be moved. This whole concept ought to be moved into bucket access.

B

If we're going to do it and figure out how to fold that in there as an alternative protection mechanism, I think.

C

B

I think it should be. I mean you could say like have a mode that says we're not going to touch anything, but then I don't think there's a lot of value in saying we're not going to touch anything but trust us. It's public.

A

What do you mean we're not going to touch anything? Do you mean like so? Are you saying how this interacts with the bucket access itself.

B

Right so so um right, presumably if it's anonymous access, that means it does not require any kind of credentialing, either the identity, mapping or downloaded credentials right. So fine, that's, okay! I don't know, I don't argue with that as a potentially desirable thing. My question is: how do you model that?

B

Because in that case, what's the purpose of a bucket access request right, so you got to figure out what do we put in a bucket access request? If we're just relying on the fact that something is already public.

A

So yeah um I mean there's still reasons to have. um You know credentialed access, um like for rate limiting, say uh if you want to go over the default rate limits you want to be. uh You know an authenticated user.

B

I'm sorry sid, I didn't mean there's no point in having credential access. I meant in a world in a scenario where the workload wants to access something anonymously right. So how do we say that I don't think it makes sense to put that particular access question on the base bucket itself. It seems like that really ought to be an option that can be put in the bucket access request.

B

Whether that option is honored or not is a kind of a different question.

A

Right yeah, I think what you're saying makes sense in terms of the workflow. um It doesn't model what the clouds do. Today, though, the the anonymous access itself is a policy. That's for the entire bucket and and the the layer of abstraction is the bucket. uh Where this is set um a bucket access.

A

um Cannot, you know, you know you can have multiple different bucket accesses that that actually.

B

Yeah so so I take your point so so, let's look at it this way. So fine we want to make this private basically means. Don't do anything in this case right. The three different publics would reasonably be expected to then at the time that you are provisioning this bucket to set these public access modes and yeah your point being: is this isn't specific for user? So it shouldn't be on bucket access, but that's fine. I get that, but then what do you do for bucket access when you want to access a bucket?

B

That is a public bucket. How? In other words, there has to be an option that says I'm going to be accessing this bucket. Give me the end point for it, but don't try to give me any credentials and don't try to set up any kind of workload, identity, mapping or anything like that, because I just know that I'm going to be able to access it anonymously.

C

Yeah, what you're saying is that it seems like we need to um expose this concept of anonymous access in the bac as well.

C

D

Think it's a kind of a class type for access.

B

Yeah and and further, I think that, based on what I the way, the way, what that I heard is that setting this public read-only public right-only public read write does have an impact on the back end. It is expected to implement that on the back end. So this note here is a little misleading. It isn't misleading because yeah it doesn't impact apples or iam policies, but it does move back to back end.

A

Right right, so I think I think what we can do is to move forward. um I think one we should definitely take this note out. I don't think it clarifies anything and it definitely adds confidence.

C

The note can be removed, you're right because it doesn't, it just confuses. um I think I think, because we still want this concept of anonymous access.

A

Yeah yeah I'm going into that, so okay, so about that, um I think I think modeling anonymous access is something which might need a I'm, not sure if we need a longer discussion, uh but I think it's worth having it. um I think I think this is so so earlier we talked about how you know we were sure there were going to be some changes that will come into the cap even after the cap goes through.

A

This is the first thing you'll have to address, and I think this this anonymous access is with the current workflow it'll still work except you know you will go through the credentialed way of doing things. However, in order to really model anonymous access, I think I think we can have that discussion after the the kept deadline after we uh go through with the with the cap process.

A

uh I am not sure you know it adds too much value at this point um to to try and change things uh with this right now.

B

Well, I I would, I would say I don't know if this is the point you're trying to make, but I don't feel like removing anonymous access mode at this point is a deal breaker and doing it to remove some of the ambiguity and then adding it later, wouldn't be a horrible thing.

A

I think so I don't think there's much ambiguity, um maybe I'm getting it wrong. But uh if the worst case scenario is uh there's going to be an anonymous access mode and credentials are always going to trump that, regardless of what the anonymous access mode is.

B

But what's the point in being able to provision an anonymous bucket if you can't access it anonymously.

A

uh So one of them could be, you have a you, have a like a website, the workloads that have credentials right to it, but the public can read from it. Anyone can go to the website, see its contents. um So the.

B

Workload, I'm provisioning a bucket for public access, but none of the workloads in kubernetes will actually be accessing, because the primary reason for public buckets is to back websites all right. That's fine!.

A

Right yeah so yeah to move forward. Let's remove this note and I think uh yeah we can have a lot longer discussion about this in the future um right next right after you know, the the cap gets reviewed and hopefully merged. um Let's go to the next comment.

A

Oh brow field: what.

B

Oh yeah, that was um there's some small typos and things yeah.

A

Good catch, uh I think, gcs calls it uh gs. Okay got it. We should fix that.

B

And then we're using azure blob and- and I don't know if there's a url format for azure blob storage, but is the protocol azure blob in that like in the case of gcs, the protocol is gs colon just like s3 colon, so I'm wondering if there isn't an a b colon or something like that. I don't know yeah uh hey chris. Do you think you could uh check that out.

A

C

Chris, um I'm not sure chris is on the meeting. He had a conflict during the first half of this meeting.

A

Okay, no worries uh yeah. We will look into it for azure um storage, apis bucket request, parker instance name. You want to make sure there's a discussion of the binding logic for brownfield case.

B

Well, I can mention that, because that's that is my other one of my other question areas. um So let me let me up level just one thing. As I was reading the the cap, I was very much trying to put myself in the position of an app developer, trying to decide which of these fields to use, and I would say, as a meta comment, it is not always obvious why I would use a particular field, whether it was optional or not, and what I should supply to get.

B

What behavior- um and I know this is an authorship issue, it's less of a technical issue, but um it's difficult to fully reason about things when it's not super clear what the intent is and- and sometimes I feel like- I'm not even sure that keph itself has consistency of intent. The language is a little bit different in an access versus uh you know or the you know the namespace versus non-name space or etc.

B

But um as far as this particular comment, though, I think that the answer is: we can control namespace level access to uh from the bucket side, and so it is sufficient in the bucket request side to just name the bucket and then as long as your name, space is allowed you're, the one that gets to buy in the bucket, as opposed to forcing the bucket to to have a back reference in order to make the binding happen.

B

So I I think I think I got that right and and I'm okay with that, so we can move past that one okay, so to address.

A

The um the consistency issue, basically, so we have a lot of contributors, who've been working on this over a long time.

A

I would say it's getting more and more consistent from where it was before, and uh you know this is how it goes when lots of people are working on it with very different backgrounds, and you know different uh experiences.

A

It starts out this way, but but eventually it will become very clear in terms of the language and how how the cap itself is worded sure this wasn't.

B

Intended as a backward looking statement, this is just a forward-looking statement like it would be good to to to read this with an eye to be very clear about what the intent of the fields are and indication to the user, for why they would use fields or not right.

A

Yeah, I think uh you know we'll we will have definitely have to write good documentation, uh which is a part of our milestone um and also the api review process is going to get into all of this. um So I think uh we we're uh we're, set up to go through this uh uh at a higher higher detail and- and you know, get it right so.

B

My last substantive.

A

B

um Comment slash question has to do with a confusion in the bucket access request over.

B

Basically, how you support the various scenarios, the various scenarios being workload, identity, uh a driverless kind of thing which says hey. I got everything defined here. You don't need to invoke a driver at all and the third being. I want you to actually go mint. What you need to mint on the server and give me credentials back so the case where you provide the um um sorry. Let me I have to look at the actual um cap here um when you provide the secret. The service account name that I think that that feels fairly clear.

B

The problem that I have is with the access secret name, because I don't know what that's supposed to be. You say this field is used used by whom the user supplies it, because this is a bucket access request. But what should the value be?

B

And this right, if I'm wanting you to mint a secret and return it to me? Is this the name of the secret? I expect you to populate, but if that's the case, I thought we were providing the secret in the form of the of the ephemeral volume. So why do we need a secret?

B

So I'm just confused about how one uses this field in the case, where I'm wanting this the driver to meant the secret so.

A

So uh I think this would be for static brownfield jeff. uh Could you answer this question yeah? That's. I.

C

Don't know if I'm on you now we can hear you, okay, um yes, I'm reading the description. I think that I agree that this is. uh This is a case where I already have access to the back-end store and I'm putting my access tokens in a secret in my own name space and I'm supplying that secret.

C

Although why do I need to do that?

C

um Because my workload can just consume the secret through environment variables, it's my own secret uh or or through a as a file mount any either way. So I'm not I'm not sure.

B

It's absolutely confusing by the fact that it implies that you have to specify either a certain account or a secret name. I think maybe the two are mutually exclusive, but you don't necessarily have to provide one or the other and that wasn't clear in the language right. That's.

A

The that's the right answer, so I think this has been left over from again from from the previous, uh I think, yeah so yeah. So this is good. This is how it gets removed. um So uh yeah, I think we can. We should remove this field because we added we added the minted secret name here, which is what holds the actual secret and in a static brown field case. The admin would create the bucket access with the secret provided here not here.

A

That would clean it up to me, so we should this access secret name is, is remnant of the previous uh discussions and design. I think that should be removed. I'm glad you found that.

B

A

That was it okay, great okay, can I ask.

B

The follow-up thing.

A

B

Sorry wait. uh Andrew did you, were you finishing, I don't think other than the knits, but the knits are. You know, we've already kind of talked about. Most of them.

D

Go ahead, guy, hey, so I just want to make sure that do we still have any way of like completing the automation in an environment, variable consumption by the workload versus a volume, or is it always going to be a volume.

A

uh In this case, I think volumes can be projected as environment variables.

D

All right, so that's that's how it would be yeah, so the workload can define to take a volume and create environment from it. Is that.

A

Yeah yeah yeah.

C

But from cozy point of view, guy we are just going to write to the volume for the endpoint information and the uh credential tokens um and the and the app will have to consume those whether it can use projected volume. I don't know that concept exists, but I don't know if that works for csi driver volumes or not node.

A

Adapter will not do it. No director will just write the file yeah, but projected volume should be able to put that as environment variables into the environment.

D

And about the the actual format of what it gets to the volume? Is that uh something that can be changed per the protocol or the provisioner? Or is that okay.

A

Yeah it's set by the provisioner um and and, and uh you know the provisioner sets it- creates a secret from it. That uh then later on gets mounted through the csi um into into.

C

The part and and guy the driver documentation will have to describe the format right because the well I'm assuming the driver now the driver will generate the secret for us well yeah. The sidecar does that.

D

Right so I'm wondering what what is the translation between the secret and the volume? I think if that's specified I didn't see, but maybe.

A

So no translation between signal volume. No, we don't. We don't make any changes to it. We shouldn't make.

D

Any changes to it um so it becomes just files flat file names that can be mounted to wherever the workload decides.

A

Yeah, it's a single file that gets mounted into wherever the workload decides, yeah desires.

E

Does it get saved into a secret object as well.

A

F

C

All right so a secret in the provisioner's name, space we have a secret and our our csi node adapter, also called a csi driver. um Reads that secret and writes it to the volume in the pod um csi volume section right.

C

Are you asking or he's telling jeff I'm saying that's my understanding of how it works, because I wanted I I wanted to make sure that we aren't cozy, isn't creating or copying a secret into the user name space. It's not, and I wanted just to make because that one one of these earlier iterations we were doing that side and we're not doing that got.

A

C

A

Okay moving forward, um so how much time do we have? Okay, so so we'll carry out the changes right away actually um and- and uh it seems like most of all- of the changes so far uh are not major and we can do them quickly.

A

um I want to acknowledge that the next thing we'll talk about possibly starting next week, is modeling anonymous access a little bit more clearly and how it kind of works alongside the current im style, access or bucket access that we're doing the next step is after the cap gets, reviewed and merged would be api review so about the api review.

A

So I just want to find out what that process looks like, and uh uh what would we? What would we need to do um in order to uh you know, get that started um just as a question just says, uh you know trying to figure out what to expect as the next step.

A

So can someone um sad or shane or anyone who knows about this um help me understand that.

E

Yeah, so my understanding is that you apply a label on the cap, saying that you're ready for api review and I'm not sure if it's you manually or somebody who's going to do the api review ads to adds you to a api review project board and one of the api reviewers will get assigned to review your cap.

E

And then it's basically like a code review, uh and you know, they'll, go in and place comments uh and questions as they have about the api and give their sign off on it.

A

Got it: okay, okay, that's great, and so who adds the.

F

Label, uh one of us can do that: shangri-probably, okay, good once you're ready. Just let us know, and we can do that all right sounds good.

A

Okay, yeah, I think I'm a little clearer now um yeah, when we, when, after we get merged with the provisional status um and after we finish the modeling of anonymous access and anything else that comes up, I think we can start the api review right after that.

F

uh Makes sense yeah.

C

Xing had a comment related to api review or someone did that the expectation is that we're showing them uh go structures, not yaml or maybe.

G

You can keep both. I think some people will be asking for the yamaha as well, but normally the api reviewers look at your the api to the way you you write your api. Obviously.

C

So we need to just take structures, copy and paste structures from the code that is being written right now and paste that into the cap for the api reviewers to see.

A

No, I think, we'll just point them to the code.

G

No, no, it has to be in the dark they're not going to go. I.

A

See I see okay got it all right, um all right, so that's all I've got for today. um If the uh does anyone have any other questions.

A

Great okay, um so, let's uh so we can end the meeting now uh we'll carry out the changes very quickly and uh yeah. Please uh do a final review. um Jing saad uh and everyone else here and uh you know, give us, give us feedback uh and we'll be able to uh respond to that. Also right away.

E

We'll do thank you. Thank you.

F

H