Kubernetes Data Protection Working Group, 9 Aug 2023

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes Data Protection WG - Wi-Weekly Meeting 2023-08-09

Description

Meeting of Kubernetes Data Protection WG - 09 August 2023

Meeting Notes/Agenda: -

Find out more about the Data Protection WG here: https://github.com/kubernetes/community/tree/master/wg-data-protection

Moderator: Xiangqian Yu (Google)

A

Hello, everyone today is August 9th 2023. This is the kubernetes data protection meeting, uh I think today we don't have other things on the agenda. I think, namely we probably just want to give and do an update on the CBD uh cap, so Yvonne or car Prasad. If you want, if you guys want to talk about it,.

B

uh Yeah we've updated the cap with the feedback from the cigar team. I believe Prasad has updated his demonstration prototype do.

A

You are you going to share this look and make you a co-host yeah.

B

Prasad, probably all right, yeah.

A

C

Cool um yeah so uh based on previous discussion, I have updated the blue today, um yeah I would like to show the demo and we can walk through the sample implementation um yeah.

C

So yes, let me know I mean we can go through the design again, if needed on very high level, and then then you can go through the demo.

A

Yeah go ahead, you want to do it or you, okay, I'm, also making car not sure who wants you present. I'm. Sorry.

C

Oh yeah I have shared my screen. Is it visible? Yes, cool yeah I can on very high level. I can just talk about um what we discussed last time, so the yeah. So this is the new approach. um I think it's I, don't know how many iterations we had but I I. Guess it's 8th or ninth um iteration, so so the new as per the new proposal. What we are doing is we are we are taking.

C

We are making use of token request and token reviews API uh to basically generate the token, as well as validator token and uh to you know, achieve the authentication authorization uh without having the extra controller we had in the previous proposals.

C

So the if I talk about the flow of the end-to-end flow look like um so the backup client would basically discover the snapshot meta service CR for for the for the driver um and the snapshot metadata service CR contains uh the required information for uh for calling the grpc endpoint. So it has basically audience stream CS certificate and address of the site card service uh to which we'll be making the GRTC call to get the change block, met, editor and.

A

Yeah, so what will be the audience audience train can give an example.

C

Yeah, it could be any random uh stream or.

B

You could use the DNS name, I mean it is really the characteristic of the string is. It should be distinct and unique to the service, so using the DNS. This name for as a string could also be you know, a valid value.

D

Yeah, it can't just be a random one right like if it matches another CSI driver. Somehow by coincidence, then you know it would mess up the authentication flow, so it has to be unique to the CSI driver. So one of the recommendations is like DNS name, okay, but it's something that you know. If you imagine like user using a Helm to install it can be something that the user provides override. But by default we can, you know somehow extracted from the DNS name or something yeah.

C

Yeah, it has to be a unique per driver. That's right! um Yeah I mean we can decide on format. It could be DNS name as well to to make sure that it's Unique or for each driver um yeah. So once clients gets uh the the required information based on the audio stream.

C

It generates the token um so backup client can use token request API to generate a token for for the given audience uh it will so we can also specify the expiry time and bound object reference, but once it once once, we got once once it gets the token.

C

uh The next next step would be uh calling the grpc endpoint um with the given parents, um so we have to specify the audio scope token we generated in the first day as well as the you know, other information like namespace based snapshot, Target snapshot um and the the external snapshot metadata, which is deployed as a sidecar to the CSL driver where the the CSR Tower. Here we have the service provider implementation of uh getting the change block metadata. uh So the the site car basically performs the authentication using the token review API.

C

So it validates the token it has passed.

C

uh It also validates if uh it also validates, if the token or the client identity, that we received that we uh kind of faced using the token review uh is if that identity is allowed to uh access the volume snapshot, resources and the authorization uh we check using subject access through API, so basically token review provides user identity and that user identity we use to perform authorization check using subject access review if the user is allowed to, you know access the volume snapshot, resources, um yeah and then the the client-side.

C

um This communication is really obviously uh TLS, uh so the Cs certificate we got from the snapshot, metadata Service uh it it used to validate server at the client side and yeah. That way we have PLS as well as of the the. We have also validated client using tokens, reviews and subject accessories. Api.

E

um Yeah once it's done.

C

The sidecar calls the service provider implementation of uh CBT and streams the response back to the client. So this is the kind of end-to-end uh workflow I tried to summarize. uh If you have any questions around these, we can talk but um yeah. Otherwise, I can jump into the demo.

C

I'm not sure which screen is visible, but should we jump into the demo again.

F

We're looking at your web browser with GitHub.

C

Sorry, it's awesome, so this is where I have added all the Prototype source code um in the redmi Havel said it states like how to deploy this end-to-end prototype and given an example how a sample client can be used to get the change. Locator um I'll talk in brief about the services, so we have external network data service uh which, which is uh which is deployed as a side card to the service provider. Excuse.

A

Me sample drugs. It's uh it's like a fake driver. Is there like a real driver that.

C

A

uh Okay, so that means you don't you're, not really taking snapshot or.

C

uh We are taking snapshot, uh we are just mocking the gate. Delta um Handler. Basically, we implemented the get Delta RTC. um Okay,.

A

Okay, that's fake, okay, yeah.

C

Yeah only only that API, otherwise we are using in this demo. We are using the host part sales at driver to create.

A

It is a real Drive yeah.

A

Doesn't really know how to retrieve that, so you, basically that part is fake. Is that what you mean.

C

A

C

Right um so yeah, so as we see in in the as we have already seen in the design, these two containers uh are these two Services. Basically, uh this we see in the diagram, so external structure and metadata is the site car to the service provided implementation, which is sample CSS activity service, which marks the gate, Delta grpc handle and external snapshot. Metadata services are obviously responsible for uh proxying the call to uh service further implementation, sidecar and Returns.

C

The response response back also performs authorization or authentication checks, uh using token reviews and subject access reviews right. um So I'll briefly talk about the deployments.

C

So I have um yeah I, have it uh namespace CSR driver where.

C

Where I have installed postpath uh CSR driver, as well as the um external snapshot media data service, which has two containers, uh one is the Excel receptor metadata as well as um as well as the you know, the get Delta uh grpc Handler or we can call it. The service provided implementation to get the change block metadata, and uh there is one more namespace uh CBT client, where.

C

Backup client application I will be using, for you know, for this demo to get the change block metadata. There is one more namespace called database.

C

We have created a PVC, so if we can, we can assume this as a application namespace, where we have some uh applications installed. Some database is installed. uh I've also uh have also created couple of snapshots on the same same PVC, so you can see there are like I guess around four snapshots created on CSI PVC, um okay and I have created the snapshot metadata service resource for this driver for the host path, driver.

C

C

C

Right, if you view the content or if you see the Yaman manifest for this resource, um we have added it under the space we can see, it has address audience. Obviously, this is I have used standard stream, but we'll have to we'll decide on the specific format. We can use the TRS names itself and it tells our CSS CS certificate, um uh all right so um yeah. So we can work to the client. You know what what what specific things client does um in order to get the change block metadata so.

C

um Yeah I'll first render I'll first run the demo. Then we can go through the source code and figure out uh what how the implementation would look like. So in the.

A

uh There's a question in.

E

C

C

Can actually put us yesterday directly into space.

C

Any any I mean. Is there any concern about yeah there's.

F

Plenty of examples of there's plenty of examples of CI certs being embedded in kubernetes objects, they're, not Secret at all, so you just have to handle the encoding as the base64 encoding or something.

C

And very common example configurations or new trading web configurations like these specific certificates, along with the endpoints there.

C

All right, uh so what we'll do is we'll on uh for the snapshot that we have just listed so um I just.

C

I'll just executed to the CBT client for.

C

C

Client part I have there is a there is a CLI I would say which we have implemented. uh It takes this snapshot, uh Target snapshot as a parameter, along with other information like client, namespace, the application name, space and um yeah about the token Mount. We will come to this in a bit but yeah. So further.

C

These two pairs of snapshots.

C

uh We will try to call. We will try to get the change of metadata so base snapshot. Is uh this one? There is a target snapshot. The application name. Species database and the client needs to the backup. Client needs resistivity client, and uh we have also mounted service account into the CBT into the spot, which is uh CBT, client service, account and yeah. If you execute this command, what it does is it first discovers the uh Tower name from the snapshot right once it gets the driver name.

C

uh It search for volume, snapshot, metadata service resource for the driver, uh the driver resource parts, uh so once it it founds the it finds the um data service for this driver uh it. It gets the information like uh the Cs certificate, as well as audience and the address for the audience. uh It creates the token token request, API and in the response we get the token along with the expiry time.

C

Obviously, and then uh we use all these parameters to call the uh it's called the grpc endpoint, and this is the sample for mock change of metadata information that we have implemented on uh on this sample. Csr travel, which is yeah, which is streamed back to the uh the lack of client. Basically.

C

And if you see the logs of the external metadata service.

C

Right so there are, two containers will recruit one by one. uh This is the external snapshot service, which acts as a proxy right um yeah. So if you see the flow here, it gets the request. First, with the token and other information uh snapshot based snapshot, name, Target snapshot name and uh the max wizard parameter for pagination.

C

um So what it does is the Department of proxy service or the site external snapshot. My data site car does. Is it first discovers the selection data service for the driver um to to face the audience team so once it uh once it gets the audience stream. It calls the token review, API or with the token it received in the request, as a request parameter and the audience we faced from the uh the snapshot media data service.

C

So token review basically validates the token if this token is valid for this audio, if it's targeted for this audience right. So in the response we get authenticate true, uh we also get the user information uh along with the username, uh the group, and uh you know their information yeah statistical groups. It belongs to ID and all this stuff also. It also Returns the audiences um that we can again validate validate against the one we received in the snapshot. Metadata service object, uh I, guess it.

C

This returns all the list of audiences, for which uh this token is generated, and we can obviously match if, if it, if it's generated for multiple audiences, we can um it it validates these. The audience will be passed in this request.

C

If, uh if, if this uh audience team belongs to the list of audiences, if there are multiple, basically it it matches with any one of uh the audiences for which it generated, we don't know we don't have to go into that, but yeah I'm, just I, wanted to share that uh yeah.

C

So once we once it gets the user or user identity it um excuse me, it calls subject: access review apis to validate if the user can access to one in snapshot, resources and if it validated that and then it uh yeah it tries to catch the snapshot, handles or snapshot IDs for the snapshot names. So it looks for volume, snapshot, resources, only snapshot, content, resources finds out the IDS and then uses those IDs um should have logged third request.

C

um What what how it calls the CSI driver standpoint, but yeah so based on the um after, after converting the parameters like after mapping, the snapshot names to the ID and um it calls the CSI drivers grpc endpoint, to get the actual change of metadata and proxies the response back to the back to the client.

C

And if you take the CSR drivers uh logs so yeah, we can see here the request uh how the request to look like at the CSL driver side, uh and it sends it so it sends back the responsibilities service and then the response is streamed back to the client right.

C

um So this is one way how client can generate token. There is another way as well, so using projected volumes.

C

So client can either use token request apis to generate token or they can use the uh projected volumes um to mount the service account token into the Pod. So this is a. uh We are basically.

C

The the token can be generated by cubelet and mounted to the Pod uh when we create the Pod. So.

B

It becomes a bound token so, and the Pod dies is invalidated automatically.

C

Exactly exactly so, if the Pod is deleted, uh the token becomes uh invalid, automatically and client doesn't have to. You know, call this all apis to discover audience and create the token request. Api create token and use that token right, you can just read the token from the mounted path uh and directly call the GRT statement. So uh yeah we'll quickly write that as well so I'll create this Backup backup core in the same namespace, yeah I guess we already seen the specs of the spot.

C

uh We are using the same sample client, which in which we have a sample. uh We have a CLI, grpc clients CLI um it. It mounts. The CBT client service account and we are using projected volumes here to Mount this uh Mount. The service account token, with this audience right uh to to this this path. Basically, so, let's see what happens when we create this pod.

D

At the end of the day is really just up to the backup client how they want to get hold of the token from the kubernetes API server. Yes,.

C

C

All right, so we can see these are CBT client job for training.

C

um Will you quickly check if the service account token is mounted.

C

Right yeah, so we can see. The token is generally are mounted here um so the same pli that I had created for this demo. It also has these two flags uh which will tell it to uh use the projected token from this path. Okay, and if we at the parameters are same like Source at the base, snapshot, name, Target snapshot, name, duplication, uh namespace, the client, namespace and uh yeah list of the parameters are same.

C

If you call this, you can see uh if you go to the workflow right uh yeah, so it discovers initial steps. You may send to get the service address, as well as your certificate when it's uh snapshot metadata service and we uh we discovered it based on Twitter name.

C

uh The defense here is, uh instead of creating the service account token, uh using the token request, API. We are directly reading it from this path and using the token, to uh call the jrpc endpoint on this address right so yeah. These are the two ways uh the client can use these apis and uh yeah. All the source code is uh here in this repo external software. We have server implementation as well as client implementation um again quickly, go for it like yeah,.

D

I mean from the north, it's the first sorry just one quick thing so the for what is worth like um one of the good things with the um projected service account token is that, like uh I appreciate, like the backup driver, doesn't specifically need, like a r back permission, to call the token review. um Sorry, the token request API. Whereas if you invoke the API directly, you need the r back permission, but with the projector service account Define. This back like um appreciate that they don't need to specifically have the r back permission is.

B

That that's right! Okay, oh that's! Pretty cool! There's an also another really good thing about it. Is that cubelet keeps that token alive. So.

C

B

If you, if you have a backup uh taking a long time, let's see this, you know something breaks and you restart using the starting offset Etc, just really the fresh token in and you're good to go so.

D

B

D

Thing we got from the sick off group is like. There are multiple ways to generate this token, so you can even keep cuddle like. It has like a create token thing that you know for what this work user can use um so anyway. So basically, the message is like there are many options to do the uh API together, API, so API server, request token in comparison to say previously. If we have to you know like roll our own, like token management uh service.

A

Does this expire? Does it like exist automatically yeah.

D

A

Does but then there will be a new wall will be generated automatically yeah.

B

So there's an expiry period you can specify, and but every time you read it from the file Google, it will make sure that you get a fresh token if it's expired. Yeah.

D

If you called, if you call it, if we talk, call it if the backup server called it, the token requests API directly, the backup server is, is responsible for refreshing it. But with the projected service account uh token, then cubelet would um refresh it for the service account. So the backup software will be seamless to the backup software. Okay.

D

But again these are just multiple options: uh the backup software can, it doesn't affect the CSI driver. Let's say it's just many options for the backups client, which is the world that I live in. So this is like really.

A

Good yeah, but it's still yeah, those are those in the capital. You talked about how.

B

That no not really I mean I mentioned it briefly, that you can use the projected volume. But you know this is a kubernetes service, so people.

A

Can yeah, but it's probably still right for the completeness okay I just mentioned you don't have to go into deep details, even including some links would be helpful.

B

All the a lot of links to everything, yeah, okay,.

C

Cool yeah, so, as Evan mentioned, uh if you want to create token using token request, apis uh we'll have to add these rules um and bind these rules to the services counter. The the client backup, client right it needs access to uh this service accounts token sub resource, but yeah. With projected token, we don't need these specific permissions.

C

E

C

Can quickly go through the service.

D

C

D

So before that, um does anyone have any questions? Does anyone have any questions about the demo and the design so far before we dive deeper into the code?

D

Does it make sense any red flags.

A

I think this is great much better than the previous approach.

B

Much simpler, too yeah.

D

I think, like yeah I, think we uh I just feel like finger crossed I feel like we are at a point where we can just really scope this to be the alpha. You know, release and I know like there's still a little bit of requirements here and there like what ifs here and there but I think if possible like. If we can just say hey, this is going to be Alpha and anything else will be Beyond, Alpha D1. It would be. That would be great, um but anyways um yeah. You want to proceed.

F

Well, I I wanted I wanted to ask how many uh organizations do we know of that are likely to implement clients and or Surfers of this API, just as proofs of concept right, independent implementations.

F

Because because that that'll be the that'll, be the really positive signal when other people start using it and saying like yeah, this works. So if we identify who those parties are uh that'll help us get to the next step, I think.

D

Yeah I can't speak for other organizations, but at least for us on for Dell, like um I, live on the client side, so for us there's definitely the needs for it and Tom who's on the call like uh he's like the CSI driver, server side um I think he has some interest as well right Tom in CPT yeah.

D

So we would be definitely you know. It definitely serves on easy. Something like this um become available.

F

Yeah yeah what I'm thinking is like once you have two different client representations and two different server implementations, and you show that they're all interoperate, like that's the gold standard right once you get to that point. You like you'd, say this is let's pour the concrete on this and say it's done so I. Look forward to that, but please continue.

C

um Yeah, since you mentioned about two different service implementations, so basically the sidecar implementation, the the proxy service that will deploy as a site car that will be common for all. That will be maintained by Community.

E

The service yes, will.

F

Yeah I mean I'm, not I'm, not talking about the community code. Yes, we want one copy of that, but you want to have like a couple of different vendors implementing like storage implementations of this and a couple different backup providers, preventing client implementations of this and showing that it all works and then and then that's that's your sort of proof. Proof by example that that it it's the right answer.

C

C

uh All right so yeah I just wanted to quickly walk through the um dharback configuration for the the the Excel snapshot metadata service, um it for token to call token reviews and subject access review apis. It needs uh permissions for um the service account that this site car uses uh needs these rules to be bound. uh The token reviews, as well as the subject, access reviews, uh resources uh along with I, guess, almost all the sales side drivers have. These have permissions to one of snapshots and volume snapshot.

C

Contents um yeah I mean we can we can assume that it will. uh It will already have the success um yeah. Additionally, we also need to add access, or we also need to add rules to provide access to snapshot, metadata Service uh resource um uh yeah.

C

That other other than yeah I guess the permission to volume, snapshots and PVCs that that we can assume that almost all this is a set hour will have access already.

B

If the complete.

C

B

Permissions is in the cap, so you know request. Please read it and give comments on the thing and including typos, and you know, battery written cover stuff. So.

C

um Yeah we can quickly go over the code. um I guess the flow is from the logs and from the demo I guess. The flow is a big, uh clear, um but I'll just walk through the code to reiterate over that um yeah when the client calls get Delta RPC right for of this proxy service, um we have a get Delta Handler, where we first perform the auth authentication checks in the Authentication for authentication the request. We first discovered the snapshot metadata Service. uh We first discovered the driver name um yeah for this service.

C

It it already has driver name set as a parameter or EnV variable. uh Since it's uh it's deployed as a sidecar to the CSR divers itself.

C

Where we get the audience team and then we call the authenticator um to authenticate this request and in the authenticator we basically to jump to this yeah. We are calling token 3 view apis um service, token and audiences as a as a parameter to this API and based on the response, uh we return, if the if the request with with the token with a given token, is valid or not, uh once it's valued yeah, once the um yeah I went to the once, you make the token review API we are in the response.

C

We also get the user identity, uh which we written and.

C

Too much sorry about that, so we got the user info from the token review API and we use this user info to perform the authorized authorization checks and in the authorization checks. What we do is uh we called subject access review apis and we do it to check if the user has access to volume snapshot resources.

C

So you pass the verbs like it has access to a gate, the snapshots, uh the olive sub short name, spaces, uh the gvr information, the user info and um yeah, and in the response which we get the uh in this course we we get the if whether the user is allowed to access the given resources in the given namespace with a given verb or not so once it passes. uh The next next step is.

C

Server dot yeah, so the next step is um the next step is straightforward. We initialize the GRTC client.

C

Client with the Unix protocol and the grpc call is made over the Unix socket.

C

So we call it the CSI drivers in point. uh We.

C

We call the drivers get Delta RPC and uh we stream the response back to the client here. So we have a stream, for we have a basically we are getting streamed from this yesterday driver and we are sending it back to the CBT client, okay. So it's kind of proxying the response from the CSR driver to the backup.

C

I hope that makes sense. um If you look at the client, we do.

C

We start from here.

C

uh Yeah so, uh based on the parameters right, we call setup security access. What we do here is we first find the driver name for the snapshot it has passed. uh Then we get the snapshot. Metadata service object for the driver from which we get the we get the audience, and then we um we call. uh We create the security token. So if we have passed the flags like use mounted token, we read it from the file.

C

Otherwise we call creates a service account token method, where we are basically calling the token request, API with a given audience and given expiry uh yeah in the response we get the token and we use. We use that token. Then.

C

To perform the.

C

To to call the grpc endpoint to get the change of metadata, so here we populate all the parameters like security, token namespace based Target snapshot, names and uh yeah. We make the grpc call, we read the read from the stream till we get the end of file uh yeah. So this is the high level workflow and um we have, as I said, we have a. We have mock the get Delta RPC for for the driver and if you look at the implementation, it's again, we are doing nothing.

C

We are just we have implemented handled for get Delta. uh We are just logging, the apparentress, and this is a mock response. Basically, uh we are returning for each call.

C

uh This is where uh the service providers will have to implement their own logic, to get the to get the change of metadata and stream. The response back to the proxy service.

C

Yeah, that's all I had to show and any questions.

C

G

Hey Prasad, so um this is my first meeting so I apologize if any of this has already been discussed, but just looking at the design so far and the code as well um for for background I'm working on a backup restore of VMS, which are surfaced in kubernetes as custom resources.

G

So a lot of this can also be um generalized for other resources, but it looks like this is really specific for persistent volumes. There are a few references to volume snapshots, for example, in the code, um so I guess my question is early on. uh Did we discuss um to make it generic enough so that any backup, server or backup provider can implement the, for example, get Delta API and support zpt, for example, because.

E

The technology is.

G

Available for VMS as well right.

B

Yeah, that's actually awesome. That's a very interesting point. uh I think it's possible right. We can explore on future meetings, but you know, essentially, you saw prasad's sidecar well, the sidecar is built over a library and and the main program right. We can always expose every piece of the code and all the rest of the stuff is grpc. So if you have a service which is grpc as long as you can connect to it, there's nothing in this design that stops you that says it has to be.

B

uh You know uh a specific thing like a snapshot or something it's. It's still abstract enough that maybe the discovery mechanism of how you discover the grpc service can be tailored differently for say your use case.

G

E

G

Definitely go through the Gap in details uh to uh see um or try to spot anything that is specific here uh for volumes, but, for example, I saw a few references of um like volume snapshots, for example, so volume doesn't really um map uh one to one for a VM right, VMS have either disks um or I'm sure they can have um volume surface as well, um but yeah. So those are the references that I'm talking about.

G

They kind of tie this workflow to volumes specifically, which is fine, if you have already discussed it, but I just wanted to erase this.

D

I think, like um so far, we haven't discussed that but, like um I think for the uh really like um I think some part is generic, but I think some part will still be like specific to snapshots. I.

D

Think, as you know, right the getting the Deltas between the snapshots and the Deltas of uh you know, like uh a VM, for example, is like the parameters uh everything it would be different right instead of like um at this point like, instead of trying to like um shoot on it to fit like um every single, like potential storage um entities out there I think like the focus will be on snapshots so, but you know like if you can help us to flash out like.

D

Oh, you know like that in order to get the Deltas between like uh VM images like uh this was what the uh inputs uh parameters or requests will look like then I think that would be um definitely like um interesting, um but I think it's unlikely that we'll fold that- and it's unlikely that we will make this cap like a b-o and NL type of um get Delta for anything that you can think of cyber from API yeah.

F

I I think this is the right primitive. You know to just focus on how do you get diffs of individual volumes and individual snapshots, because everything else is built on top of that right? Vms VMS can have multiple volumes and multiple snapshots, but a VM snapshot at the end of the day consists of volume, snapshots and other stuff, and if you're, trying to efficiently back that up being able to go down to the individual volumes and get the diffs is extremely valuable. Yeah. So I.

A

But the VMS has static disks and those are the parts that cannot be contained included in this existing design. What's.

F

A

uh Maybe I'll measure it and explain that.

G

So uh uh so I mean there are. There are a few different types of disks um that are available out there in the industry just type, so um one of them are um like a first class or individual disks which can be managed or surfaced in kubernetes uh as well, um and their life cycle is not tied to the life cycle of a VM and when Shing says static disk. That means a disk that comes with the VM and its life cycle is again tied to the VM.

G

So if you delete the VM or if you snapshot the VM, that disk gets snapshotted or deleted with the VM um is.

F

There, a CSI is there a CSI driver associated with that.

G

There isn't okay, so.

F

G

What, if so, I guess what I what I was uh referring to is like you, you mentioned snapshots right. So, yes, snapshots work for the VMS as well, and if, um if this was genetic enough, that it was calling a snapshot API on a resource which can be a volume or a VM or a pod or anything, um sorry snapshots, don't make sense for parts. But you get the idea any runtime engine, um then an implementation could be calling the volume snapshot. All the VM snapshot.

C

G

Need to go and read the cap, but that's kind of what I had in mind.

F

I I just want to understand what what snapshots a static disk today, if anything,.

G

So nothing in kubernetes. Yet, but if you have a VM, if you have a VM, you can snapshot the VM pretty much. Every hypervisor supports that and that is going to take a snapshot of the vm's runtime, as well as all the disks that it has.

A

So then, basically, when you take a VM snapshot, it includes not just all those PVC snapshots but also the static part yeah.

E

F

I'm hearing is so the hypervisor has its own snapshot. Logic that doesn't rely on CSI whatsoever right is that accurate.

A

Yeah, you can take a vmc which will snapshot everything.

F

Yeah I mean this proposal is for CSI so like that is that's not CSI. So, like that's, not our concern. I agree. Someone should address it, but like for what we're trying to do is figure out four things that do use CSI. How do you make it better and the answer is this proposal?

F

You know if, if they're, not, if they're, not using CSI, there's nothing, we can do right, they have to go figure that out.

G

E

I'm just trying to see if this was yeah.

G

Just trying to see if if uh it was brought up um early on and and so all of that exists at the hypervisor level, it's just a matter of if somebody tries to surface more like different types of runtimes at kubernetes, VMS being one of them. um How would data Protection work for them and then I think that's a valid um statement that this proposal focuses on on just volumes that are managed by CSI, um which not every volume in a VM is managed by CSI, so that.

F

That it creates it creates an incentive so that, if you do want to be able to take advantage of this kind of backup technology to make all of your volumes CSI volumes right I mean it. It creates a path for someone who's sufficiently motivated to to go, build the right system to take advantage of this, and anyone who doesn't implement it might get left out right that that's that's always.

F

The threat of these new new technologies is, is we're trying to encourage adoption by you know influencing both the but the the demand in the supply for the for the future.

G

Yeah so I see the logic there, but but there's like again, there are additional.

D

Challenges so kind of possible yeah, so yeah so I think, like um the the use case you define is, uh is valid in a sense it's valid in a sense again, there are folks on my team at Dell like who lives in your world, but I think for the, uh for all intents and purposes, for the scope of this cap. I think that is outside of the scope of this cab.

D

Csi PVCs and CSI snapshots like um yeah when you said like a Snapchat backup or snapshot like the entire VM I I was thinking along what you said. It was beyond just all the CSI PVCs that are attached. It's like the actual entire image entire state of the so so that is completely beyond the the scope of this club as far as I'm concerned at this point, um I think it's a good story and there's a value use case, because we have uh people on my team trying to solve those kind of issues. Okay,.

G

Understood so let me just go through the cap first um and then yeah I I'm, okay, with the fact that this is outside the scope of this cap um yeah and we'll see how we can address it. um Yeah.

D

G

A different camp or some other design.

D

Yeah and for what is worth right, even if you're part or containers it's like, backed by like Qatar containers or G, Vice or firecrackers, if there is a way to CSI all the volumes of that thing of that part attached to it, then you know the implementation is up to you that that interface, that Prasad show that get Delta. At the end of the day, you have to write something you know over that. You know so grpc from client to CSI driver and then another grpc call from CSI driver over Unix socket to the plug-in.

D

At the end, you have to implement that get Delta um grpc on the server side. So, regardless of like whether your back, VMS or whatever, if there's a way to CSI it, then you know you have to you: have the flexibility to to to to override and to write your own get Delta implementation. There.

E

G

D

Yeah I think for uh okay. I just want to be aware of time. I think, like um seems, like um you know, there's no, um there's no like glaring, like red flag. So far, I think we're gonna ship like um update, keep the cap up to date and then, hopefully we can um I guess. What's whatever the right term, you know terminology is opt-in for 1.29. Is it I? Think 1.28 is coming out in a week or two weeks three weeks, yeah.

A

I think next week, yeah.

D

F

The cap, so so is the cap accepted, or are we still working on that? Well.

A

We I think we still need to get that approved right, so so Ben, okay review the cap.

F

Oh I was just gonna say that the cap deadline is very shortly after the previous release. Yeah.

A

I should update that and then we can start reviewing yeah yeah.

F

Yeah we should push for that. This is at a stage where it's obvious that we should be pushing for the.

A

F

To be approved, yeah yeah.

A

uh What about the CSS back part? Do you need to do update of that TR I.

D

Think yeah I think we do yeah. That's pretty.

A

Prns that also takes for a long time, but hopefully it's not that complicated.

D

Yeah I think like uh I, think Carl has been. Oh, no uh yeah I think lucky has been very detailed, so it's all the grpc spec I think we can hopefully like um copy and paste most of it from the cap into the CSI spec on PR oud, um but for maybe a little bit more details.

D

um uh I'll add it.

F

To them, so so, just so I understand that the CSI spec PR is not up to date today, but you'll fix that soon correct okay. That would be great because, yes, that's another thing that needs to happen. That yeah as Shing says it's not fast. Yes,.

A

It takes a very long.

F

A

um Yeah but but.

F

I think I'm willing to help so so yeah.

A

F

That update and tag me.

A

Yeah I think the tab needs to go in first, because cap has a more uh you know: there's a deadline, that's written right, so it's not like the cinestack one. We have more time so yeah make sure the clap gets in before the.

F

Before the CSI spec is done, is the it has to be a provisional approval if you can get the CSI spec just done, and the cap condition.

A

Oh no, we approved right. We saw you there last time with the modify volume right. We need to think of that case, but then we we think it's okay, we don't have to. We don't have to uh block the cap. Cap can be.

F

um We don't have to block it but like we have to approve it with an exception that you know. If something CSI level changes, we have to update the cap slightly.

A

A

That we, you know as long as you know how the CSS pack would roughly look like. If you are okay right, you can you can we can merge the cap first and then uh yeah the thing, that's what what we did for the modified Walling, because that cat was emerged a long time ago. The CSI is back where I just merged right like a week ago or something.

E

F

A

F

Had a lot of very subtle disagreements, we got it all sorted out so.

A

Yeah yeah, that's good, hey sorry, I think we are on the topic hour. Oh yeah!

A

D

Bye bye, thank you.