Kubernetes Storage SIG, 6 Apr 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes SIG Storage Meeting 2023-04-06

Description

Kubernetes Storage Special-Interest-Group (SIG) Meeting - 06 April 2023

Meeting Notes/Agenda: https://docs.google.com/document/d/1-8KEG8AjAgKznS9NFm3qWqkGyCHmvU6HVl0sk5hwoAE/edit#heading=h.77kmba8m60ax
Find out more about the Storage SIG here: https://github.com/kubernetes/community/tree/master/sig-storage

Moderator: Saad Ali (Google)

A

Okay, today's uh April 6 2023. This is the meeting of the kubernetes storage special interest group. As a reminder, this meeting is public, reported and posted on YouTube. uh Today on the agenda, we've got a few items. uh If you have anything additional that you would like to discuss, please feel free to add to the agenda. The link to the agenda document is, in the calendar, invite uh first up we're going to go through the 127 planning spreadsheet and get a status update of where uh the tasks are that the Sig committed to for 127.

A

uh The most recent past deadline on April 4th was docs completion for 127 and blog PR ready. uh So hopefully we got those done, and the next upcoming deadline is the 127 release itself, which will be happening next week, followed by kubecon the week after so with that I'll go ahead and switch to our spreadsheet and we can get uh get a status update for the items for 127..

A

So first we have recovering from resize failure. uh Do we have a month on the call?

A

It doesn't look like it. uh Anyone have a status update for this item.

B

uh I think it has play slipped to the next release. Okay, so.

A

I'll say: push to 128.

A

We'll get that moved over. Thank you, Jan. The next item is issues related to assuming Mount volumes or Mount points.

A

Any update on that one.

A

Okay, I'll mark that, as no update uh moving on to the next one, we have volume group, API, um Shing.

C

uh Yeah, so uh we do have several PR's out for review uh implementation of the controller the web hook and also uh in the CSR driver host path, uh and also the blog PR is ready for review.

C

uh There is also dog PR, that's the in a CSI docs repo, that's also uh I, think that's still need to be marked as a working progress, because we need to get a sidecars release. First got it.

A

Okay looks like it's on track to be able to be released along with 127. yeah uh cool. Thank you. Shang next is provision volume from uh volumes from Cross namespace snapshot. Pvc continue off of work. uh I think this has been no update for a while. uh Anyone have an update on this one.

A

Okay, we'll go ahead and Mark that one as no update I suspect we'll move that one to 128.

A

uh next is CSI volume, Health, uh adding additional metrics and or events. um This was also no update anything new from anybody on the call on this.

C

It's still no update. Okay,.

A

uh Next is change block tracking design um anything new on this one.

C

Yet still uh design discussions doing pocs.

B

A

And next is the new rwo access mode, adding end-to-end tests to move to Beta uh remaining item here was blog post anything new here.

C

I'm not sure if you show us talking.

D

um I mean the blog post is um under review and ready to go. I think we can probably just Mark the system.

A

A

Okay, next item is runtime: assisted mounting um is deep on the call by any chance. Nope.

A

Okay, I'll mark this one as no update.

A

uh Next is CSI proxy for Windows transition to privilege containers uh any update on this one I think last status here was. This is going to be on pause unless someone else takes over uh binary blob, not open sourced question mark.

B

A

That is no update.

E

Oh one, quick question about this: uh I thought the feature was already Jade and 126.. So what is the work at this point? Is this on the storage provider side, or is this something uh uh something different.

A

um Anybody want to give a summary for the current CSI proxy work. Michelle Leon.

B

E

B

Wanted to update all the CSI drivers out there, uh Andrew kubernetes umbrella.

D

I think, though, also the I think um I don't know John if you're familiar with, but I think there was someone that had some potential security concerns about the um the new model about the privileged container model. So I don't know if we need to work that out. First, like I, think code I'm, not sure if the code is actually merged. Yet.

E

Got it um yeah, I I, think the the reason I was asking was because if it is individual CSI drivers that need to transition I think that's a longer process right is: is there code in the CSI site that needs to be.

A

Done I think if what I remember correctly was uh the CSI proxy is out, you can use it uh no problem yeah. uh The challenge was the CSI proxy relies on effectively sitting as a binary Exposed on the host without a container, and so uh the uh change here now is that apparently Windows supports privileged containers and uh we want to follow a similar model to CSI on Linux I. Think for security related concerns, so you can think of this as the evolution of CSI proxy that's effectively going to replace CSI proxy with proper.

A

uh You know, privilege, containers, um I, think that's the the additional work to be done here. If I remember correctly, uh please correct me if I'm wrong, uh Jan Michelle.

F

So is the intent to make a better CSI proxy that runs as a container or is the intent to replace the CSI proxy entirely and have CSI drivers do what it does already.

A

I forget the discussions on this. My recollection is that we wanted to replace the CSI proxy. The CSI proxy was a stopgap solution since yeah didn't have privileged containers and windows now does have privileged containers so to.

F

Work on the Sig storage side would be getting Mount utils to be able to mount stuff on the Windows right. Correct, okay, cool yeah.

D

And I'm, just taking a look at the CSA proxy code base, the plan was to basically turn CSI proxy into a library and then um the CSI drivers can import the library sort of like the mount retails um that we have today.

F

It wouldn't just be in the existing mount utils.

D

I think the plan was to make it with in the CSI proxy library and not the mount utils.

E

Okay, uh yeah no uh I think that's helpful. The reason I was asking was because, if, if some of the store, if somebody today uh decides to make some changes on the storage provider side to to support it, it sounds like they will still need this work before before. This can really work is that is that accurate.

D

So I think the.

A

D

In terms of terms seems like the code: is there um the code is there, although I think it's not fully released, I see this um V2 Alpha tag, but I haven't seen an official V2 Tech, yet so I think we can um we'll need to follow up with uh Mauricio who was handling this about the current status.

E

Okay sounds good.

A

Thank you, oh good question. uh Next up we have node expansion Secret uh any update on this one.

C

So humble said, uh the blog PR's ready for review it's been reviewed. Stock has already merged nice yeah.

D

A

Okay, I'm gonna go ahead and Mark this one as done I think it's pretty much there.

A

uh Okay, next is SC Linux relabeling using Mount option; CSI driver API change.

B

So uh all these marriage, the documentation, has marriage, then block, is on review thanks.

A

Yeah I'll go ahead and Mark that as done as well.

A

Okay, next, uh these were placeholders for future releases. 128 129, uh AB, AWS EBS was done. uh Then this was a placeholder and Azure disk was done. Let me make sure that's marked correctly. This was done, so it's done.

A

Okay, so then uh ceph RBD, uh beta off by default uh tests were merged. uh Anything new on this one.

C

This is I think it can be. Market is down.

D

A

Okay, I'll go ahead and Mark this as done.

A

And we'll move the rest of these to the next release: uh control volume, mode conversion between source and Target, PVC uh docs done block pending.

C

So yeah Rolex that is not going to write a blog for this is nothing new, uh so this one can be marked as done as well. Yeah.

E

A

Thank you. Shang next uh was tracking item for 128, so we'll skip over that other items were dropped here. uh So we got two more address. Issues PVC is created by stateful set, will not be Auto removed.

A

um All code was previously merged, blog post, pending and Doc spending. Anything new here.

A

Okay, yeah I, don't see Matt on the call. So let me Mark this one as no update looks like it's getting close, uh since all we needed was the blog post in the docs, so I'm going to mark this as partially complete and finally, we have volume. Expansion for stateful sets uh I'm guessing no update on this one.

D

I've been seeing a lot of active discussion slack about this. Oh.

A

D

But I think yeah. It's still very much in design phase got.

A

C

So the one and.

A

C

There's one before there's the robust bowling manager, reconstruction I think that one.

A

C

Down right, yeah, yeah.

B

Everything is merged, including documentation. There is no block.

E

A

That one is done.

A

And I guess there was one more down here: known: uh non-graceful, node shutdown, ga128 tracking nope, so this is 128, so no no worries there. uh Okay, so we can go ahead and switch back to our agenda doc. uh We've got two items here: one is from Jordan one from uh Shane.

A

Let me see if Jordan's on the line. uh I don't see him so we'll skip over that for now and come back to it.

C

Yeah, so I brought this up in it's just today's the data production when group meeting, uh so we were moving this uh more remote conversion feature to Beta in 1.27.

C

Normally, if it's an entry feature, you know we will have the feature gate by default, but this is the output tree, so we have a flag controlled on the side, cars. uh So we're thinking that we want to enable that by default in 1.28, so to give people more time because for Backup backup vendors who actually rely on this more remote conversion in their backup workflow, they actually need to make changes to support us.

C

If we make this enable ID for then it's going to fail, they need to make any code changes so just want to bring this to everyone's attention.

C

So yeah basically added this new field in the um in a volumes, natural content and it's enable then the source volume mode it will be set to the body mode of the original PVC. And then we compare it when we created a PVC from one snapshot. If they are not the same, then we will see if there is this annotation added to the one snapshot content. If it's not there, it's not that you're. True! Then it's going to fail.

F

What do you mean by n128? Do you just mean the first release of the sidecars.

C

After 128. right so basically this will be because we we do async release Rising. We really set cars after the kubernetes release, so so the next release will be like after 1.1.27 release will be cutting release, but at that time um I don't think we're going we're ready to make this change in our sidecar. Yet so.

F

Yes, you just mean the first sidecar release after 128, because I was gonna say like.

C

F

Unofficially, synced or anything like like there's no 128 version of the sidecars.

C

Yeah yeah I know it's uh yeah. Basically, it's a release after the 1.28. Yes,.

E

uh Can you talk a little bit about the specific use cases that this is trying to address like what what kind of uh customers.

C

Yeah I think this is more for backup, vendors, I, uh I, don't know if it I think driver normally. Don't have this use case themselves, so Backup backup drivers they want to do efficient backups. They want to be able to use the change block change block tracking. So what happens is if they have a PVC provisioned?

C

That is the that is the file system mode when they do a backup, they may um you take a snapshot and then they create a PVC from that woman snapshot when they create that PVC they actually change that mode to Roblox.

C

This way they could retrieve the change of blocks, um for you know for more efficient backups and then copy the data yeah. That's the that's the reason they I know. There are a few backup vendors doing this, and then there is some potential security concerns uh when you just you know, if we allow anyone to make this make this change, there could be some problems.

C

um So that's why we added this only allow authorized users to do this. Conversion.

E

Got it so uh the reason I was asking this question was because it wasn't clear to me if this was a request that was coming from like customers who are using kubernetes and who are looking to do backups or or if it was something that uh that's being requested by, as you mentioned, like a backup vendor somebody who offers backup capabilities and, and they want to offer that in a standardized way. So this.

C

Feature itself is not requested by backup vendor it's just that backup vendor has this use case right. They want to do this conversion. This use case already exists, but then, when we look at this, this workflow we identify and there is some potential vulnerabilities.

C

Even there is some kernel box, then it could be exploited by malicious users. So this feature itself yeah, it's more like make this a more safe way right. So it's it's a super. It's a use case that is is it is useful. It's a right so.

E

That use case applies even to somebody who's running their own cluster and who wants to backup the data on that cluster uh on.

C

E

Volumes and things like that, and they want a more efficient way of doing it. Essentially.

C

Right, yes, okay, got it.

A

uh Okay, I don't see Jordan, but we can try to cover his item. So uh road map to n minus three node control, plane SKU support um what is in uh in progress. Six storage features. Cleanups rely on this sorry.

D

I could maybe give some background on this, because I think I'm a little familiar with um what's going on, but um we're basically proposing to extend the cubelet to control plane version SKU um from n minus 2 to n minus 3., and um that way it would allow folks to potentially only need to upgrade their nodes once a year um and so I think Jordan's question was mainly around. Are there any storage features that might be impacted um by extending by um you know, making this change.

A

Yeah, that's a good question. We definitely have uh you know our controllers that interact with each other right. The attach detached controller runs on the control. Plane sets bits that then the Mount controllers will consume.

A

That said, whether it's n minus two or n minus 3, since we haven't been making massive changes on that code, base shouldn't really matter.

D

Yeah, the the main thing that matters is the CSI migration and the removal of the code, because there we do have some special interaction between control, plane and cubelets for CSI migration, um and so our current plan was when CSM migration is GA. We can then remove the code to releases afterwards, um so I think. If say they extended the skew supported skew to three. Then that might potentially mean we have to delay the removal of the code.

D

One More release, but I think it really depends on which release they plan to uh start supporting this with.

F

Well, I think our answer to them right is that we need to wait until at least 126 has dropped off, because we did that for AWS right 125 was the ga and that 127 Was the removal.

D

Yeah and so I think it's going to depend on each each specific, CSI migration when they go ga um I think a lot of the ones that went GA in 125 or 126 should not be impacted. I think what we need to look at are the ones that have not gone GA yet and what are their timelines.

F

Junction yeah I mean we could just wait an extra release on those right, yeah.

D

We can but I think it has. uh It has sort of trickle-down effects with the whole um dependency removals that overall, um the project is trying to work on so I think um that might impact, maybe the the like. Basically, our timelines might impact or my influence when the kubernetes project wants to move to This n minus three call.

E

A

Cool any other areas, We can brainstorm in terms of uh things that could be impacted by this.

A

Okay, I'll go ahead and tag Jordan on this and then uh we'll see if he comes back with any more questions. uh Thank you Michelle. Thank you folks for the discussion on that uh any other items to discuss today.

A

Okay, uh if there's nothing else, we will get a half hour back. Thank you folks,.

C

A

Take care bye. Thank you.