Kubernetes Data Protection Working Group, 6 Oct 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: WG Data Protection Bi-Weekly Meeting for 20201006

Description

WG Data Protection Bi-Weekly Meeting for 20201006

A

Hello, everyone today is october 7th 2020. This is the kubernetes data protection work group meeting. So today we have quite a few things under gender. So let me share my screen.

A

So just I want to quickly go over some of the caps that are in review or be merged currently for the 1.20 release, and then we can talk about the data protection workflow white paper. uh So there are a couple of uh work streams that are going on uh so we'll see how much time we have and we will discover as much as we can so for 1.20.

A

um The enhancement freeze, which is cap merch date was yesterday um and then code phrase is november 12th, which is one month away. So there are quite a few caps that are relevant to this data production group. So that's why I listed them here uh snapshot. So we are trying to bring snapshot to ga 1.20 cap got merged, so that's good, but then we actually still have a lot of work to do um to meet the code freeze deadline, which is one other way.

A

We need to write quite a few e3 tests, so um I listed this three tests here, I'm actually looking for some help uh right now. Those three tests don't have any owners. So if you are interested- and if you have some time to work on them in 1.20 release, please let me know um and and then the next one is a container notifier, so uh we were trying to bring it to afr, but we didn't make it. uh There are some issues that we still need to get resolved in our design.

A

So I listed a couple of them here. I just want to talk about them briefly and see if there's any um comments from you guys, so uh so the container notifier uh we are trying to bring bring in this as a port, inline definition.

A

uh First to define what kind of command we can run in the container uh for our use case. It is quite as non-choice and then also a con, also a notification um api object, that is, for an external controller, to request the container notifier. So basically you request what comment you run um in the pod. So uh there are a couple of things that uh that we have some discussions in that a cab review.

A

uh One thing is uh how to decide what parts need to be selected for this for this choice, and initially we have the support selector.

A

There so if you have the selector and it's going to select the parts that meet the criterion, but I think uh if it's a selector, then that means the the controller, so the backup controller, who is actually um running this requesting the class, wouldn't know ahead of time what parts are selected because the part is going to be this uh cubelet or uh in first phase we're going to have this uh central controller, um the content notify controller, which is the one that is deciding.

A

What are the parts? That's that are selected so uh which I think it's really not not going to be practical for the especially for the choice use case. So um so I'm thinking that either we can. We can use a pod list which is more uh deterministic. Basically, a uh backup controller would be the one who can use a part selector or some other ways to determine what other parts that should that should get this choice command and then and then, in this uh notification object.

A

We just have a pod list instead of a four selector and and then there is another concern which is the the status update uh right now we have uh because initially we say we want to have this part selector. So now, if we say we have a pod list still there could be um of uh basically or you're getting the status update for all the parts and all the containers. That's a lot to know some concerns on the scalability.

A

um So uh for that, if we're using a pod list, I'm thinking, maybe we need to do a update uh using a slice of pot so that so the details. I don't really have any details on that and there's some some similar uh proposal for the end points the endpoint slides. So it's going to be similar to that.

A

um Just this isn't just something that I'm thinking and then another idea is to actually um have a one-on-one mapping between part and notification so that uh we will just send one request at a time for one pod, instead of uh sending a notification for a a list of parts.

A

So this way we'll simplify uh the the status update, part yeah. So those are the things that we are. We still need to sort out. um I don't know if there were any comments on this.

A

Okay, so uh yeah the I listed the the cap there, so you can take a look at the cap. I need to update that and and uh with this proposals, so right now we have part selector, and so I'm thinking that I will just uh write down all the alternatives and what are the corresponding uh status update that we we need if we use either part will be straightforward or a party list. Maybe we need to have a slice of pop pod slice or something which might mean that we need another api object for that.

A

So I have to take a look of the endpoint slides and see how that is defined.

A

um Okay, so that so that's that, so if you don't have any questions for the first two caps, I move on to the genetic data populator. I wonder if ben is here today doesn't look like he's there. Okay, yeah, so ben is actually working on this generic data populator and initially we were thinking about bringing to beta, but I think um we're going through this production readiness review now the the cap have this new requirement for production readiness reviews.

A

Actually the snapshot cap went through that so ben realized that okay, so he he's going to introduce this uh validation webhook for uh for the data populator to making sure that um it is something it is like a registered um external data source rather than you know anything uh so so for that uh we actually wouldn't need one more release. So I think right now uh you want to keep the scene alpha in 1.20, uh so that's uh that's the status for for this one.

A

A

And then the next one is the cozy. I don't know if anyone from cozy team is here.

A

If not, I just uh quickly talk about it. uh This is uh this is the to have a kubernetes api to be able to provision object bucket, so there are actually weekly meetings on that.

A

So the plan is to get the cab merged as provisional, so that you can start to do pocs uh having code ready and then getting getting ready for the next phase, which is to bring you to alpha so 1.20. This was stay as provisional and pocs.

A

I think the cap is not merged yet, but I think uh because this is provisional, so we actually uh does not have to um be getting getting merged strictly by this uh deadline.

A

uh If it's alpha, then we have to uh meet that deadline, but these things is provisional, so we actually can merge this one after the deadline, so hopefully we'll get that cap merged soon.

A

So that's a quick update of all the cabs that are going on any questions so far.

A

Okay, uh if not, then we're moving on to the data protection workflows. I is a phone here today.

A

Okay, I don't see him here, because I actually would like him to go over the application workflow because he added uh some update for the backup and this workflow there.

A

If he's not here, then I'm going to maybe talk about the cbt.

A

Okay, I can talk about speedy.

A

All right, so um this is the one of the things that we are talking about. um We had one meeting to discuss this. This is a really just a pretty early discussions, but I want to um get more feedback from this uh from this group here.

A

So uh so so we talked about uh why we need cpt in kubernetes, so we listed a few use cases so, but the first one is backup um for backup. Before we talk about cvt, we actually need to talk about in backup in general.

A

um So currently we do have uh apis in kubernetes snapshot apis, um but that does not really have any definitions in in those apis to say whether we want to back up back it up to a different location or not. So that's not there. So anju actually had a uh google doc on this.

A

He actually talked about this in one of the early meetings that we need to have a backup apis to handle this um so yeah, so the first one is to in order uh for um for backup vendors, to support this, we will need to be able to backup data to a different, uh uh a secondary storage. That is not your primary storage.

A

So that's uh that's the first one and then so what do we do if we don't have this? What is the uh currently? What what do you recommend to do, and so currently um we can have the data mover pod so um and then, basically, after taking a snapshot, we can create a pvc front snapshot and mount that to the data more pod and then move data to a secondary device away from your primary storage.

A

um So that's uh that's for the that's for the uh backup why we need this.

A

And then, then, and then also for for backup, if we are uh always taking the full backup, then the concern is, of course is not space efficient.

A

It takes longer um so and also uh there are, even though there are cloud providers that are providing ways to do incremental snapshots, um but that only works if, if it's on the same storage right so you're, it's all aws. So it's going from ebs to s3 it's handled underneath.

A

But if we want to backup this data to a different storage device, then this is not going to work. So we do need to have a cvt. um So here really.

B

Is the plan to have the cbd as a you know as part of the csi spec or.

A

uh So we haven't got to that point, so this is right now, I'm talking about the white paper part. So uh in the white paper we talk about um why we need it right. So if we, I think we need to list out the reasons why we need it and if without it, what is the workaround? What are we doing?

A

So that's why we have this uh search here saying that, okay, without that, what do we do today, right so think of phone actually added this one saying: um either we have to always do full backup uh or.

B

Communicate with the apis to get the cpt.

A

Yeah so otherwise we will have to uh directly use the um the storage uh api uh to get that information um right so yeah. So I think the the the problem is that then that means uh the state of mobile power will need to be dealing with uh different apis.

A

It's not a common api, so I think that's what's listed here so yeah. Do you see any other and other things that we should add in this uh over here.

A

So uh yeah, so basically the that's the thing: that's the the workaround. If we don't have this, then you just you need to have to deal with us individually.

A

A

Okay, do we have any other at any other use case, that we want to add, or any other reason that we we should add here saying that okay, we need to have a common api for this.

A

Or do you guys agree that this looks good.

B

And this looks good from my end at least okay thanks.

C

Sorry, the part I don't understand is yeah. So if a storage vendor supports incremental backup or incremental restore.

A

C

Then here we're talking about exposing that functionality through csi.

A

Yeah, so we haven't got that point, so basically, okay, so here is the thing right. So if you actually have every everything um so, for example, just use the aws, for example right, so you already automatically take incremental snapshots. So, uh but in that case this is basically all aws right, so aws they take care of this one internally, that's fine! But what? If we want to back up this data to your different, uh solid device from a different vendor right, so third-party um backup blender right, not not aws right! So that's that's.

A

uh What do we want to do here right? If you want to support a variety of storage, you want to be able to um backup from different storage vendors and like packet to your storage device, which is not the same as the primary storage. What do you do? I think that's the so that's the question here right. If you handle everything underneath, I guess that's yeah, then you don't need it because you have your api coming apart, handle this right. So here we are talking about.

A

um So if you want to back this up to a different um different location, a different uh storage device, then you would have to use that slow defenders api to to do that or you can also you can probably come up with your own. um Maybe a common!

A

uh You maybe have your own common api to to look at the diffs. That's also possible right. That's also possible too.

C

I guess the part that's hard for me to comprehend is like what's the incentive for on-prem storage vendors as well as cloud storage vendors, to expose these changes incremental changes through an api yeah and.

A

C

Do of the hyper skaters why these apis.

A

So we actually have not really get to the details of the api part. Yet so just right now we are looking at the reasons right why we need to do this, what a workaround, if we don't have this, uh so I think uh when we talk about this last time, I think there are someone was saying: yeah, yeah,.

D

Yeah so uh azure uh for azure disk that those apis are exposed.

A

Okay yeah, so we actually yeah. Actually, if you go to.

B

Azure aws, I believe the apis are there. I mean it's not part of uh kubernetes directly at all. You know, through their controllers, that they're building for kubernetes, you can still access them in a way.

A

So yeah, so basically, if we look at the whole support, so we have a section to say who supports this right. So this is a section that I need input from all of you, uh so we have aws aws actually have that they expose this and, as you have this, but I think uh uh I think the I think I see this uh uh lexi saying that the links listed here does not really uh does not list to the the change blog api. This is just uh some introducing this introducing the support.

A

So if anyone knows some api, is there a link to the api that we can add here? um Yeah. Please uh just add that information here, if you know, and for google, I think the this is a work in progress. They they don't have it, but they are working on it right. So we do know. At least you know this is three cloud providers actually um either have it or working on it uh yeah.

A

The sender is the little different singer, it doesn't have cbd, but it actually has a full backup service that does increment the backups automatically and then vmware we have. Those for vivos is dave here: okay, yeah so um yeah. So this is actually a api, so every story vendor that supports vivald actually already implemented those apis so yeah. So that's already a common api for different uh storage vendors. That's supposed to be well and then you know we have other other vendors who support this right.

A

So if you know any other uh storage vendor that supports this, please go ahead and add them here yeah. I think there are probably a lot.

C

Okay, can I ask the clarifying questions so like yeah, these vendors, that support incremental snapshots like um are these these snapshots portable, such that they can, for example, like a v-ball snapshot, can be converted into a non-viva volume or rds just lets. You store a vivo snapshot on a on an arbitrary storage backend like look. Oh that's part of the those.

A

Efforts to have portable snapshots for us or not so those are different apis, so I think the api. How do you have common api?

A

That's the next thing that we want to discuss right so right now we just want to say who actually have this and then, of course I think it's going to be a challenge to if we want to say: okay, that's how uh the api in kubernetes is just for this, and what is what does it look like right, but I think we've all is one example, because it actually have multiple so defenders. That's what's that right! So that's one example, I'm not saying, of course it's not going to be exactly the same as that.

A

This is the you know. The the way you express api is very completely different in this world.

E

Right, I'm uh certainly, uh I think you can add a little bit on that one. So I I did take a I did. I do research on these uh cbt um these and generally um the the component are similar like you for each of the chain block in each of this description, you will have like an offset and what the length of the data that is chain in that blocks and you have and then they specify like a multiple struct of that multiple of that in an array, uh so in general they are.

E

They are in that format in detail like each of them, maybe have a little bit here and they're a little bit different. Some of them do not. Instead of spell out the exact length, they would have a fixed length and then they have a bitmap.

E

The detail is a little bit different, but it can, I think it can easily convert from one another without losing the details that is, for block device and for the file system device, there's also a similar concept, but it specifies the file that changed and then you can have like, for example, uh in the struct of the like.

E

A short list, each of the item in the list for the file will specif specify the file path, the uh the the start of the file os, that of the file like you, know, file mode, what type of fridays access time recovers and so on so forth. So these are in general. They they they are. They they have. They have quite a lot of similarity between them I haven't taken. I cannot say I can look at all of them.

E

I look at some of the uh some of the api that being provide, or some of the um you know, internal from the omc api that not being exposed outside yet so I look at them and I see a lot of com common, so I think in our next step, when we have a meeting again, we will dig down to like how can we extract these common uh api these into into a common api to use? But that is the topic for maybe the next meeting. I guess.

A

Okay, yeah so yeah. If you can uh also just to add some here in the dark after the meeting, that'll be good.

A

Okay- maybe I can, can I sign this part to you. You can add something here.

E

Yes, please, aside to me, as a matter of fact, I already have a document uh to ready to be shared in our last meeting and then we started to say hey. Let's just talk about the reason for now, but I do have a. I do. Have the info. Okay, sorry, what is your email h-o-a-n-g?

E

Is the last name h-u-o, uh it's okay! I I I can talk.

A

A

So yeah, so please, uh you know, uh add, add to the list, if you uh no other other storage that has this for yeah the cheapest list. um Okay, so going back to here so uh island. Does that uh address your concern? Questions.

C

So it seems basically the goal like one of the end goals is to have a portable snapshot format such that, if a storage um vacuum incremental snapshots, those snapshots can be backed up to any arbitrary backyard, not necessarily of the same type of primary, but to anybody.

D

C

Seems to be part of this effort right, correct.

A

C

Want to have portable snapshots, yeah.

A

C

Enable volume to volume, migration, look, look, let's say volume, a um is a one type. Let's say: b ball volume b is not a different type of evs, and that can happen. Is that part of the end goal.

A

E

I I think we already do some of these portable when we create the p in the csi pv and pvc right. When we backing up um a pv, we can backing up from any started vendor.

A

Right and then you so basically, you can back up and then uh store at the same at the same storage device. Basically right.

F

We have, we have portability.

A

F

Volume data today that this would add portability of the snapshot implementation, so that you could literally switch your volume provider and keep your snapshots.

F

E

A

Okay, um so, and then okay, then continue with the restore restore. Basically, I think this really depends on the the backup part um at the restore time. We would still need to do a full, restore first and then layer incremental, uh restore on top of that from the incremental snapshots um and then and also partial, restore.

A

What is okay, so this is not strictly related to cpt right. I forgot who added this case. I mean yeah apache resource. It's like you, select, select some files, so this is a little tricky.

C

Because the way, the way most implementations do incremental, restore partial, restores based on data access patterns. So this is something in the data plane, so csi or the control plane is completely out of loop, and I don't know exactly how csi is going to do smart incremental, restore, um given that it's just a control, plane component.

F

It's not incremental. It's partial right like like, if you had a partial.

C

F

You could restore a block range or if you had a file system volume, you could restore a directory. Yeah like csi would have to become.

G

A

G

F

Like block ranges.

A

F

A

I yeah no, I actually I so the thing when you think about this. I don't want csi to do all of those work. This should be handled by the the backup vendors, but when you think about how to do that, yeah, I'm not. I haven't really thought about this restore case, yet um I think it's the back of vanderbilt.

C

This is a little tricky yeah. Even if we supply some rb2 range to csi, you know, let's say blocks one. Two 100 gets fetched by csi prefixed what if the application tries to access block 200. So there is a you know this joint there is a disconnect between what the csi may do or what the application needs. As far as data access.

E

Yeah this this, the restore, is a little bit more complicated or to the depend on jewelry store to a completely new or you restore on something is running right now right and then, when we do uh restore to a development in in a production environment that would have some part is running and using that persistent volume things are much more complicated than just that, and I don't think it might. I don't think it's allowed to uh be right to some pvc that currently is using by a specific part. I mean by some part.

E

The reason is that I think the volume when you create a pvc- you you create a mode of I think it have a you- have to specify the mode of um write. Read many or right read one. If, if you have a wrong permission there, then only the part that currently is using that pvc can accept that pvc, and if, for that case I mean you, cannot the backup sort. It cannot even do anything with the part that is with the pvc that currently being used.

A

Yeah, I think it's uh yeah, I yeah. I need to think about this one more actually yeah. Maybe this one, I'm not sure.

E

Yeah, I think we need to discuss.

A

Because I was when I was thinking about this, I was actually thinking about what you did from cinderella. Cinder actually has a complete backup service itself. It's it's on the data plane, it's a little different, so uh you think about yeah. This is done if we are only providing those apis.

E

But let's still have some, it still has some views, for example, if the, if the user agree that um to, for example, if I deploy an um a stateful set right- and I even though it's in the production I can, if I can, if the user allow me to scale down the stateful set to zero, which means that effectively remove the part, but still the pvc still there, then I can do partial or you know, differential, restore a few blocks there quickly and then restore the scale back.

E

The stateful set to whatever the current num, the previous number, and that can still can be. You know a good production of restore with the incremental restore, but you know we have to accept that the part have to be go away, for you know whatever period of time, it's still very quick compared to restore the entire uh new to entire new namespace with a new pvc. But it's still I mean that is something that is possible, but I agree that we need to think about this. A little bit more.

A

A

Yeah, okay, this one's a question mark yeah: did you think.

A

So uh yeah, so this one I I forgot who mentioned I does not look like this- is uh directly related to cbt, the partial, the oh you're, saying that it's it's still related because they probably already have something there just to do incremental.

A

H

So this is like pulling a single file out of a previous snapshot. If you will that, maybe you, you know just destroyed a single file in your volume, and you want to restore that.

A

Okay, this is more more relevant for file than block okay.

E

Yeah, I think uh david or someone in the previo in our meeting mentioned that this incremental backup the efficiency of us having a cbt mainly for the backup side, I'm not sure if it must benefit on the restore side. It's mainly efficiencies in the backup, because we do backup more frequently like every day or every hour restore. You know not that much so not sure yeah, but.

A

Then, how do you do this? So you you have those backup if user want to depict the incremental backup viewers. So then what do you do? So that's you. We still need to figure out. Oh yeah,.

E

The um the the the backup image actually synth is a synthesized image of the full backup right. It looked like a full backup all the time, but it helps it. We only um we only even though we only um you know you get the the change, but in the backup storage it should be a full backup. So you can restore every single time using that you know synthetics um synthesize, full backup image.

F

This is a problem for the backup provider to solve, like, if you put gene tracking in csi, everything else just becomes an implementation detail for the thing that does.

A

F

A

All right yeah: this is what it's going to do. If you know, if it's, uh if it can completely handled by backup vendor, then that's not a problem. I was thinking. If csi needs to do something, then that's really weird yeah.

A

A

Okay and then there's also a use case here for a snapshot based replication and hey john, since you are here and thank you jones yeah.

H

Yeah, I'm here.

A

Yeah you want to go over this, you see you. I did this use case.

H

Yeah sure so the the idea here is being able to use the change block tracking as a mechanism to implement, via some higher level controller, asynchronous volume, replication potentially between two sites. So what you could do is periodically take that diff and ship it to the remote site.

H

So you're then getting you know the efficient and efficient change list, basically between two snapshots um from your source and then applying that on the remote site, and so this obviously isn't a synchro. You know useful for synchronous, replication, but um async replication is a a commonly used thing and this would uh allow vendors to uh to build this facility fairly.

H

A

Do you have this implemented for ceph.

H

um So seth yeah I mean seth, does uh the rbd driver does do um snapshot based async replication, but obviously that's not the the change block and stuff is not exposed at the the kubernetes level. It's also limited to uh mirroring cepharbd to seth rbd, and so you know like a lot of like a lot of of what we're talking about here. um If we want to do uh or allow these diffs to be portable across storage back ends right, we need to to standardize that interface and and expose it.

A

Mm-Hmm, okay, but you do already have the the cbd apis for rpd already, okay,.

H

Yeah I put a link down lower in the document. I believe, okay, great.

A

Okay, right, okay, so yeah! So then, if we don't have this, then okay, it has to calculate manually each time between the source and destination. Then yes,.

H

Yeah, so for stuff, like file volumes, you can basically use rsync but block volumes. There's really nothing for an application to look at other than the individual. You know block data.

A

A

Okay, any questions for this use case.

C

But john, is this predominantly an optimization to reduce the amount of incremental changes that we transfer, or is this like you're, actually looking at this as a separate workflow.

H

um So, by putting the or by by getting the changes directly from uh to be exposed directly by the storage system, it would like say in the case of of file volumes, um allow us to skip the metadata scan before sending right so that for a file volume for a block volume, you know we there's really no good way to do it at a higher level.

H

I don't know if that was what you were getting at or not.

C

So basically, like rv, looking at continuous async replication as a workflow that is triggered by the users so like through kubernetes apis or is it just some process that happens in the background just to make incremental backup transfers more efficient? So because the deltas are now smaller.

H

um Does that make sense? I I guess I think, there's two parts to that right. One is kind of the the interface of what gets exposed at the kubernetes level, um and you know yes, I think it would be great for us to have a some sort of of interface. Probably not you know not something sort of standardized in in the cigs, but allow a vendor to produce a um an async replication controller right that would allow you to uh to configure async replication cross cluster within cluster.

H

You know cross storage system, whatever you know, the user wants to do to meet their their data protection requirements, and you know do that in a standardized way. Now um the the other side with respect to the efficiency of it, um you know, there's there's, certainly nothing standing in the way of today for a file volume just using rsync to implement that feature, um but it would be made even more efficient um by allowing you to by basically making it so that you didn't have to do the metadata scan right.

H

That rsync is doing um if you could get the change list directly from the storage system.

F

Well, isn't there also the issue of being able to read a snapshot like today? Kubernetes doesn't give you that so. Is that part of the proposal to be able to like directly read snapshots without cloning them to a volume.

A

But that would require mundane the snapshot right.

F

Well, I mean what.

A

Are you talking about.

F

If you wanted to do an rsync type thing, yeah, you kind of need to mount the snapshot or.

A

H

A

F

Metadata scan you're in the same boat.

H

F

Yeah, if you're.

H

If you're doing it manually today, yes, you do need to take the snapshot, turn it back into a volume. You know, attach it into a pod and you know go from there. So, yes, that is kind of inefficient. Today, um if we get the the change tracking, you know, I don't think we've gotten to the point where we figure out how we actually get the the data out right.

H

We haven't talked about the apis yet, but I think one of the benefits would be that you wouldn't need to create a well, hopefully wouldn't need to create a whole new pb um from these individual snapshots. In order to read the data out.

F

Yeah, what I'm, what I'm getting at, is like being able to directly read snapshots is a necessary, but not sufficient component of what you're asking for so like you could treat that as step. One.

A

But is that necessary though, if we have cpt, then we should.

F

Yeah, you still.

A

F

Read the blocks or the files once you know, which ones have changed in order to back them up.

A

But then you're supposed to be able to get that from the studio vendor right. You just should ask.

F

For those the point of doing this in csi is to provide a generalized interface so that anyone can do it. And then there needs to be a generalized interface to read the files or the.

A

Blocks, I think, that's like an additional one. I know that one we actually, I don't know if we still have that in your chart. I know I have that somewhere at one point, uh so that's like a new. This is actually another feature. I think actually.

E

That's the the very good point that the the weather, the cbt that we are talking about, is currently contained. The description of the chain, like you know the offset and the and the um the offset and the length of the chain, or it actually contained the data that it changed like, for example, if the chain is 200 megabyte uh then does the cbt provide the actual 200 megabyte of data chain itself or just saying it upset 200 and the uh the the length is 200 megabytes so.

F

That is great. I would think you would want the ladder because then you could implement.

F

You could implement just snapshot reading and leave the change block tracking up to some higher some other application like rsync to figure out and then or you know, or have a fallback mode for drivers that can't actually provide the change blocks but can provide read access to the snapshots. You can.

D

F

Something that was useful and then once you have that something adding in a change block tracking that just gives you the offsets or the file names could be paired with the ability to just read the snapshot. To do what you want.

E

That actually, some of the idea that we have um thought about it, uh but we haven't got to the point of of sharing it with the community. Yet we have to discuss a lot in our next meeting. I guess.

F

But I mean it seems like being able to directly read. Snapshots is way less complicated and controversial than the the details of the change block tracking.

A

uh Well, actually, not necessarily right, then you have pot once you have pod includes, then that's actually very complicated right. I think so far. Snapchat has been simple because uh it don't have to deal with uh attaching that to your partner.

A

Yeah yeah, but I think that that would need another. It's a it's not I actually didn't thought this would be required for cvt. I thought this is like an additional feature. um Well,.

F

In that case, then, the change block tracking has to include not only the what change but the data of the change which, which makes it a much heavier weight type of interface.

H

Well, we have the ability to read the snapshot data today. We just have to turn it back into a pv right. So isn't really this discussion just centered around whether we should provide a more efficient way of getting at the data in the snapshot.

A

Well, so this one, we can definitely talk about it, but I actually I thought, but not every storage would need this in order to get the uh cbt right. So that's why I didn't really thought this would be so you uh you are actually thinking. This would be a prerequisite almost.

F

Well, it depends whether the change block tracking depends.

A

uh But I I don't think this is required for average storage at least I think from our side. uh I don't think we need that.

E

So the common, the the api that currently provided by all the um started, vendors that I looked at uh cbt, do not contain the data. They only contain an description of the chain like what block had changed so and so forth. They do not including data whether it is possible to do that. I think it's totally possible that we can enhance our you.

F

Know my concern with my concern with including the data is that, like data interfaces have to be way more performance than metadata interfaces, and I don't.

D

F

Want to if you want to design like uh an interface, that's actually moving a lot of data around this based on grpc, um because that might just be.

A

Yeah, I actually don't. I think I want to avoid that because I don't want to get involved in the.

E

Yeah well, the one more thing is that, even though we want to do you know, including the data or not, it's have to up to the vendor the started vendor to support it right and currently the vendor the current all the vendors that currently support. You know this type of abt that exposed to us. uh I do not contain the data and.

A

E

Include that we.

A

Have what is that required for some vendors is that I mean I guess I need to understand this one, so I found so from your experience right, so you said those apis actually right now. Currently, that does not include data and then, uh but just with that, you will be able to.

E

The api provides also a way to get the data like, for example, if I, if this like vmware right vmware's, they you first, you get the chain block, and then you actually get the block in a different call right. uh It's just a two different call. I think it's on the same api, but it just do two separate call right.

A

E

H

So you're saying this is still.

A

Still required, basically, the the a call to get data is still required easily.

E

Yeah- and this is uh it's not really required for all, but at least uh it is something that currently in many of the api that.

A

What is not needed for all I mean, I guess that part. I want you to get something.

D

We need it right in what case. Wouldn't we not need the date? I mean if you.

E

Let's just say: um if I already have the pv my pvc mounted on my data mover block, I only need then then you know, I only need to know what block it is. Then, because all the data is already mounted there in such case we don't need to get it directly from the api. I only need say you know five blockchain. I have all the data mounted as a raw block on my part already, so I just copied those five blocks.

E

However, if you don't mount that pvc on the data mover, you have some other way, then yes, I agree at that point. It's better to have that data of the chain available in the api.

D

I see and then you have to mount that pv as a read. Only if you're creating a.

E

Well, like I said, there's a true possibility right two way to do that. uh I think um there's a good point that the data should be available should be there too yeah.

A

Okay, I think yes, I think this uh yeah. This is, I think, it's getting to the api.

A

Yeah, I think that's actually very important right so.

A

Yeah, that's that really got into the data path. So that's that's one reason I'm saying that not supporting that in csi, but we can take a look and see exactly what is needed and how to get there, um because I thought in last discussion. I remember we talked about that, we're just providing api, it's actually just the control path so um but uh yeah. So so we yeah. We will uh need to talk more about this, um but I think this is uh yeah. This is this is good, um but two more minutes left.

A

So, let's wrap this up. So another case we listed here is migration, but this one uh yeah it's not really strictly related to cvt. I think it's just okay interpretability, but okay. This is that this is like the portability piece right back up on one result. Another.

A

And auditing, uh I'm not sure, I'm just not sure what auditing is related to cbd uh anyone. I want to add something for this to use case here. I actually forgot: what is this auditing piece?

A

I'm not sure why we'll have this one so.

H

In the in the auditing case, you could just be looking at the stream of files that have changed um and basically getting a report for like security purposes, and that kind of thing.

A

Okay, all right, uh I think uh we have one minute left, so I think uh we can wrap up this and then we can continue to talk about this in another meeting. uh So going back to this uh yeah so phong, can you talk about this application workflow in next meeting, because I was initially uh going to ask you to talk about this today, but you joined a little late, so yes, I.

G

Did the backup.

A

And the restore yes, I think those are good yeah. So let's do that next time. um Okay, bye! Are there anything else.

A

All right, that's it for today. Thank you.

B

B