VMware Velero Community Meetings, 18 Aug 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Velero Community Meeting / Open Discussion - August 18, 2020

Description

August 18, 2020

Status Updates

ashish
Mostly preparing for 1.5 release and addressing any outstanding items, doc fixes and PR reviews.
I owe @areed another round of review on the exec restore hooks PR. Going to do that today.
During investigation of Issue #2789 added some documentation to our wiki on setting up standalone MinIO object storage using custom ca cert and self-signed certificates with https to use with Velero

nrb
Worked on adding the DeleteItemActions plugin type in https://github.com/vmware-tanzu/velero/pull/2808 and https://github.com/vmware-tanzu/velero/pull/2815
Reviews
Looking to review release process w/ Abbie in light of the v1.5 beta release

carlisia
On community support this week
Lots of PR reviews last week
wip (ServerStatusRequest to kubebuilder): https://github.com/vmware-tanzu/velero/compare/main...carlisia:c-status-request

Discussion Topics
Phuong Hoang: avoid backup PersistentVolumes not referenced by included namespaces when enable --include-cluster-resources

Use --exclude-resources=persistentvolumes?
We still want to include the PersistentVolumes being used by PVCs belong to the namespace being backed up but not including other PersistentVolumes
Plugin additional items may ignore this, but we can’t remember
Also want to include other cluster-scoped resources like ClusterRoleBindings
Sounds like we want OR selector logic support
One idea - use --include-resources and specify exactly what you want
Limitation: you have to specify exactly what you want
Action item: Phuong will investigate a proposal to enhance item selection.
stephanie

Velero 1.6 release planning (starting around v1.5 beta release)
We’ll share this around Sept 1 in ZenHub & GitHub
Will update roadmap.md, too
Between now and then, please :+1: any issues that you’re interested in seeing in the v1.6 release
Let us know what Velero code you depend on

Ashish: Velero Office Hours are coming, exact dates & times to be announced

Contributor Shoutouts
@a-mccarthy: add note about windows support
@runzexia: add kindfor func to get apiresource from gvk
@jenting: [velero] bump velero version to v1.4.2
@jenting: fix: pull request template not working
@cpanato: Update GH-pages branch
@cpanato: actions: update helm actions to use release 1.0.0
@jenting: Add Github issue template
@jenting: fix: custom CA not working

A

All right, hello, everyone and welcome to the valero community meeting for august 18th 2020. um we're gonna, get some status updates. We have some discussion topics and some contributor shoutouts.

A

So to start our status updates, we will go with ashish.

B

I don't have very much to uh mention. Instead of status updates, uh I was basically preparing for 1.5 release, trying to address any outstanding items that we have and dark fixes and pr reviews.

B

I know I owe andrew another round of review on the executive store hooks, I'm going to do that today and then, during the investigation of an issue that was opened, I found an opportunity to improve our documentation and I added some documentation around setting up a standalone in io instance with custom, cssr and https.

B

So if folks have questions or would like to refer that, please do and if you find any uh anything, any inconsistencies in my documentation, please point it out or fix it help us fix it. So uh that's all.

A

Okay for myself, I'm trying to get the delete item actions moved forward, um need to fix some uh review comments on 2808, which is just getting the plugin type implemented and then move forward on 28 15, which is actually getting it in vote.

A

That's in draft and right now, all it does is say: hey, there's, no plugins, so go forward with the old and current behavior um also been looking at reviews and issues. I was on community support last week and um later today, I'll be reviewing the release process with abby. um She has some documentation on the release process that she wanted to go over with me, um especially in light of the 1.5 beta release that we're going to try to get out later this week.

A

So um I think those are my main things so any questions, and so we skipped over questions for ashish.

C

C

I think carlesio might be having some audio issues. No one looks like from the chat.

A

I lost my chat, still no sound for the carlesia, I don't know. um Okay, so carlesi is on community sport. This week.

B

Oh god lucia's trying to speak. I suppose.

A

D

D

I don't think you guys can hear me, can you.

B

Yeah, I can hear you now. I can't hear you oh.

D

B

Yes, it is, it is.

D

Okay, sorry, I don't know what happens I anyway, I'm on community support this week. I had a lot of pr reviews last week and I'm still working on the server status request migration.

D

I, uh if you're, asking why I am too no I'm trying to uh so. I did. I did I the code from the cli that had to ping the server. I did it five different ways depending on who was helping me and I think I don't think it finally got to a to final version of it, which uses the api machinery library to just wait.

D

uh I pushed what I have I'm just finalizing the controller tests. um That's it for me.

B

Anything we can do to help out complexion.

A

Did the um change from the informers to the um the client listener?

A

D

Can't hear it again.

B

Oh, I was just asking if you needed any help with uh with with that stuff that you see you had issues with.

B

And nolan also had a question. She probably cannot still hear us.

B

Okay, I think I all came.

A

Yeah yeah, I know I worked with carly, say a little bit on this last week um and I know we made some changes, but I don't know, I don't know if those helped or not.

A

um Okay, um well we'll move on for to the discussion topics for now. um Carlesia, if you, if you need help, please feel free to ping us um uh so next uh phong. If you would like to um ask your questions.

E

Oh yeah, so um the uh I want to share one of the problems have in our backup work and I wonder if we can enhance our valuable backup to support us. So here is the problem that we in we back up when we backup a namespace. We also want to back up the cluster resource.

E

However, when we enable the cluster resource, we found that some persistent volume that is not currently used by that namespace is also being backup together with it and that's what we do not want. We just want to have to back up the persistent volume and the service suggested claim belong to the current namespace. That being that we and back up, so we are wondering if we can, when you, when we do when we turn on the include cluster resource, should we somehow enable us to not not include in that backup the persistent balloons?

E

I saw the suggestion of using excluse uh resources. uh We try that one too. I think when we turn on that persistent balloon. uh The problem with that is. We also exclude the persistent volume that being used with the pvc that we are backing up, so that is not exactly what we want. So I wonder if we can um enhance our federal backup to support such uh scenario.

A

So to clarify your question, you want persistent volumes that are referenced by persistent volume. Claims in the namespace, your namespace or namespace is you're targeting.

E

Yes, but not, but not just all the other persistent volume, because positive volume is actually a cluster resource.

A

Right right right, I know yeah, so um you can omit include cluster resources, so include cholesterol, resources. What that does is that includes all cluster resources, whether they're referenced by a namespa namespace object or not. If you leave off include cluster resources. So if you just say, valero backup dash include namespaces namespace, one that there's actually some valero plugins that walk from a pod to the pvc to the pv and include those as additional items.

A

So it will get referenced objects for you and you don't have to use, include cluster resources, but we do uh we.

E

Do want to back up some of the cluster resource, for example the cluster binding, plus the role binding and stuff like that? Okay, we do. We still want to back up some of these um cluster-wise uh resources.

A

So you want cluster wide resources, but you don't you want some, but you don't want all.

E

We want some, but we don't want the persistent volume. Okay, I what the the persistent volume from our part of you is somewhat a little bit um twisted, because it's not it should not belong to the cluster wide thing it should. It should be right, but you want to scope.

A

E

Yeah yeah, so that's. That is something that I was. um I would you know, that's the difficulty that we currently face right now. uh We we have a workaround which is really twisted, but we we don't want to. uh I think we want to have like a straightforward solution uh using valero.

E

Instead of you know what work around it. You know like, for example, we we one thing. One idea that we have is we just saying: let's just first backup just a cluster resource and then we another time we back up. You know everything else and that's kind of a little bit hacky from my point of view, but I wonder if we can do something like a flat or some some option that when we do include cluster resource, we can turn on and turn off.

E

Some of the specific type of you know cluster resource that we want order, one. So um I'm just bringing out idea here to see if we can do something like that. um I can dig into the code and propose make some proposal on the specifics. But I wonder to I should throw out the idea here to see um if you guys have any you know um suggestion or on any other way,.

F

B

I have a feeling that this would make the resource selection super complex. Would it wouldn't it be easier just to say, include resources and say cluster rule bindings instead, like call out the resource types specifically that you want to include in the backup that way, you know, what's exactly you and what exactly you expect in the backup.

E

um You you're saying that uh currently, because currently the include cluster resort is a boolean right, so it turned on and off. So instead of doing that as a boolean, we can say specify it as like. A like an array or specific type that you want to use. Was that your? Can I understand your idea correctly or your comment. Correction.

B

You know I I was just saying include: resources should just in uh in include resources. If you say cluster rule bindings and cluster roles, you should you should get what you want and instead of trying.

E

Oh, but I also instead of using the include cluster resource using the inflow resource yeah, yeah yeah. I think I think I I have to double check, but I think I tried that before.

E

Okay, if I put that in, if I, for example, if I put the cluster role, the result type that we want is the in the including resource, but I do not turn on.

E

If I don't turn on the the flag include cluster resort, that will not work yeah.

C

E

Maybe that's actually, maybe that's a very good idea, if, if we can the included source that should that would work, you know without the include cluster resort flag? I think that would be a best way to tackle this.

A

And right now, as it exists right now um include resources. It has to be an exhaustive list, so the downside there is, you have to list every single resource that you would want, uh which might not be a good idea for my case, yeah right.

E

Right right, if you've.

A

Got if you've got arbitrary, crds and stuff in there that's limited, that's limiting and something I wrote here is.

A

I think we have an issue here, but I don't want to take the time to to look for at the moment um right now, valero's all of every every um every selection flag, you add, is anded together, so it further refines.

A

It's not an or so you can't say I want include name spaces or this it's it's all anded. So um we were really at. Some point would like to get or selection uh and when I say selector I don't mean kubernetes selector, I mean like valero selection. um Excuse me but like if she like, if she says it, it complicates the selection logic, but I think in some cases the selection logic we want is complicated.

A

A

Yeah, um because, in this scenario it's like we want, we want all cluster resources, except for perspective.

E

Yeah not pointed.

A

To by a pvc in a namespace that is included in namespace yeah.

G

Like that that one it's already got way too long,.

A

G

Yeah, like I don't know how to.

A

Express that for both no.

G

No, no, I think a little less complicated might be. I don't want cluster resources claimed by other name spaces.

A

Yeah like putting knots.

G

A

It yeah but yeah. I think a proposal for this would be useful. um I certainly.

E

Wouldn't say no.

A

E

Yeah, let me let me dig out a little bit more and I create like a proposal uh design into uh and then I'm gonna propose to maybe next time we meet, but I just want to bring out here yeah we currently we can. We can use something like a label to select all the item that we want and we specify the label in our backup, but uh the problem is that is we still have you know some customers.

C

E

We have to label it and the customer might not have that. You know, um may not have have the label on the some of the items so but anyway uh I I, I think I'm gonna go back and do some design and propose to you guys. Maybe next time.

A

Yeah and and with pvs and pvcs like uh even if you label the pvc kubernetes, does not automatically transfer it to the pv and there's a lot of reasons. I've heard in storage for why it doesn't do that. uh So I wouldn't hold your breath for that ever happening.

A

um So yeah you can put a label on there. But again, then you have to like go put the label on there so that just moves the problem from valero to something else.

E

And the problem with is that we don't want the customer to have to do manual manually. Do anything right sure when the system they touch the pv level and so on. So for this we start to convert them into like a different role. Instead of just a user of the system right right.

A

E

Thank you. Thank you. I I will um work on it and I will propose this uh something.

B

Next time, and just just to finish on that, doesn't the exclude uh to say include cluster resources and say exclude resources, persistent volumes. Does that not work? I'm just curious.

E

You saying that's the absolute, oh, no, that doesn't work for us because we do want to include some persistent volume that we want. We are interested.

B

E

B

Yeah, so that's that's: when the walk from pvc to pv happens, the uh the pvs that are actually uh used are do get backed up. No, no, no, it got excluded.

E

As well, if that flat turned on it got excluded all the pv got included, even with additional items from the. Let me let me double check. Let me do some more um double checks on what we have test so far, but I think we did include that we do have that.

E

We do have tests on the ecl resistant balloon. Okay, yeah.

A

I can't remember I can't remember if additional items from return from plugins ignore this.

B

Yeah, I think they uh they we do the filtering and then everything that the plugins return just gets packed up. So that might work for you. It's worth trying out.

E

A

um Stephanie, you have an item for valero 1 6 release, planning.

C

I do thanks nolan, uh so we'll be starting the 1.6 release planning for valero following the 1.5 beta release date, which is hopefully heading into the end of this week. That way, it gives the team some time to wrap up the beta and all also gives us time to look at the 1 6 um priorities uh we'll be looking at 1 6 from a lot of different angles.

C

Like upstream kubernetes alignment bug fixes community asks in the prs and a couple of uh additional pipelines that you'll see identified on our roadmap and anything coming out of the 1.5 1.6 release planning. We will um provide some details on what that stack. Rank looks like uh in our the community meeting uh following that release plan.

C

So sometime around the uh I guess september, first time frame for our meeting and we'll share that, then uh we'll also reflect that uh on zen hub, so that everyone can see the prioritized list, uh we'll also triage that on the get issues and you'll start seeing things appear on get with a 1. 6 label will also provide any roadmap updates around the september 1st time frame um to the group as well.

C

So nolan that's just kind of september. First then github wrote up.

C

C

Well, I guess I should say also. um I would like to encourage the community um since we're, since you have kind of the time frame, that we're heading into our 1-6 release planning, since we're going to be doing that uh next week and sharing a lot of that on september 1st between now and then that gives you about two weeks to go ahead and upvote any issues that you're really interested in seeing included in that release.

C

um We included that capability a couple of months back, I'm going to use that to kind of get a temperature gauge from the from y'all. um As far as what you're interested in and any prs that you're really interested in seeing uh for the 1 6 release as well. If you haven't gotten them in we'd love to see them.

C

Also, thanks for taking the notes, nolan.

E

G

A

All right and then finally, uh we've got this standing issue. um I can take it carly's, you can take it. I don't mind either way.

D

You can take it just please click on it, no one, so we are make making a point. Letting people know that if you use fill any of, if you use valero as a library- and you are using the package, that is not one of these packages listed here- please let us know, because otherwise we intend on making uh quotes. That's not not even one of those packages uh unaccessible by moving them to an internal directory structure, so we just want to make sure we don't break anything for anyone.

D

Just leave a comments here on this issue.

A

Okay, any more discussion topics.

B

Not much of a discussion topic, but some sort of an announcement.

B

uh I think we mentioned this last week as well, that we are planning to have valero office hours, so just a teaser for that stay tuned for more updates around when we will about exact dates and time.

B

C

A

All right, moving on to contributor shout outs. um Last week we got a note into our documentation about windows support with valero.

A

This was actually some testing that steve had done so this is now in our docs very short. Note that don't really officially support it. We can back up some stateless windows applications, but rustic integration and backups of persistent applications and persistent volumes are not supported.

A

um So thank you abby for adding that um this next one was from runzexia, which was adding a kind 4 helper to our discovery helper, and this was to enhance their uh helm, backup plugin, and this is something that other plugins can take advantage of. So this is super helpful. So thank you for doing that and then the rest of these are all in the helm. Chart um one that is super helpful uh is genting updating the valero version to version 1.4.2 um super helpful um genting. Also, I'm just going to do.

A

Gen gentings, together, genting, updated, uh pull, request and issue templates um so that when making an issue or doing a pull request, uh they have a template that you can fill out that helps us review things. um There's also some updates to custom ca support in the helm, chart that genting took care of.

A

So thank you very much for all those contributions and then finally, carlos panato um updated uh some github pages stuff um to make sure that the readme and links all worked and updated our github actions to use the latest uh releases of the testing actions.

A

So thank you very much to all those contributors and we've got a bunch more um community contributors uh lined up that. I'm sure we will get them shouted out next week.

A

So any other topics before we wrap up this week. Otherwise it looks like a short.

A

A

All right I'll give folks about 29 minutes back and have a great rest of your day. Everyone hi everyone.

D