VMware Velero Demos and Deep Dives, 16 Jan 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Velero meeting: new CLI revamp

Description

In this session we discussed the upcoming removal of the current `velero install`, and what could substitute it. A more intuitive CLI UX is in the works. PR: https://github.com/vmware-tanzu/velero/pull/2202

A

But we don't ship it in the tarball or anything like that, so that that's my thought on that, so we don't necessarily have to include a dry run or if we include the dry run, it's not operating on local files.

B

Yeah I mean I would agree, I think there are definitely a few different possible paths. There I, don't know I, don't know which one makes sense yet, but certainly we can't. We can't ignore that use case. I mean you know what one option that I was mulling over is whether we should just have static yamo for the baseline install, because now that we've, you know if we go down the path of kind of making configuration a separate step, there really are far fewer permutations of that baseline install like it's.

B

It's basically just a matter of adding any of these optional flags to the deployment. So we you know we could just ship static, yeah more for the baseline, install and say you know, take these if you want them or or just use them to do the baseline, install and then have commands for the the configuration stuff anyway, I don't want to I, don't want to wrap a hole too much down. This I just definitely wanted to highlight that we need to be considering this use case. We can't ignore it as we go through.

C

C

All right so I would definitely welcome any any heads-up for pitfalls, because this is a drastic departure from the way the CLI structure. What structure right now, if this is civil I, think, is going to be really very intuitive for the users, so I personally am liking the way this is coming out so I, you know, haven't I mean with we do have client commands and server-side server commands mixed mixed up and.

C

We have to address the issues we were just talking about, but I haven't really seen any downsides. Downside of doing it. This way so I.

C

Wanted to get there.

C

Did any of you have any question that it was something you didn't understand from what I put on a document like I guess we'll start there.

A

I guess the the one thing I would ask is to have a time line spelled out for deprecation as we move things around I. Think the even acknowledging the downfalls of the install command I know folks are using it in scripts and stuff and I don't want to see it like break immediately as we move things so I'd like to give at least one release where we support both even I know it's gonna be messy, but so some indication in there to say this.

A

This is the plan for how long we support we straddle, and then this is when it would switch.

C

Yeah I think one thing that we could do is we could add these commands before our next main version and on the main version we stop supporting for their install whenever that happens, to be yep.

A

Yep over here is that and then yeah they're their install continues to do what I what it does, but it has deprecation warnings. Yes,.

C

It people that will give people time to mic to shift to this way of doing things, even even with whatever version this comes out for.

C

So I guess it's pretty straightforward, so the init command I might I might have missed other commands that would have to go under in it.

C

But the idea is that, just to the minimum necessary to get the letter booted out and then maybe the valorous ads and I wouldn't be against combining and it's these two things but I think I personally laughs this way they just be optional flags that people can tweak things that, for example, gets configured with a default value, but but people could go here whether assert and see what trans are that they can tweak and obviously the volar config is all about locations.

C

One thing that I sort of got creative with was the set default location. So if we look at, if we do have a little help, we will have no sorry if we do have the letter install help that will be there Eve. There are three flags there that we need to fulfill to put for that default location to get created, and it's a the bucket, the prefix, the plug-in name which now we call provider and there's our like dangling in the middle.

C

Amongst all the other flags and it's sort of hard to map that those things map to out deal the reason that they exist there is to map to so that the fault location kicking get created, which is some in day. Even it's optional. You might not want to have that created by you.

C

You might create bit on your own, so I think what I was thinking was just like just trying to make that very as explicit as possible, and then so I got creative with this flag here, so I don't have my heart set, but I was just thinking. How can we make us?

C

So one command would take care of that, and so, instead of having we could make the values like positional like the first value with the bucket buckets and the second is the prefix or we can just make it require that the the key has a specific name, but you know, but mainly there will be an indication that these are the values they're required to gather default location going, and you don't need to create a full location from scratch, because we will take care of that.

C

For you, but if there is a plugin that will map to the plug-in name here- and this is something that would definitely rely on documentation more so than the other flags they're more self explanatory. But the nice thing about this is this: take one flag boom takes care of everything, and also the true or false, for the snapshot location go here to I.

C

Think it's a bit. It might be a little a little hard to understand. I mean I like it. That's concise. This one flag takes care of of that yeah.

A

I think I think by making that by making the snapshot, location true/false you, you then make everything in there. True, you then make everything optional, because bucket prefix aren't applicable to snapshot location like I, like the idea of having a like just give me the default command. That just does that.

A

I'm, just not sure on the semantics for making it span both snapshot, location and backup. Location I'd have to think about that a little bit more yeah.

B

Like by way of contrast, you know currently, we have Alero backup location, create as a command, and then it has a bucket flag, which is a required flag, because that's a required field for a backup. Look, that's kind of a you know, a strongly typed CLI command, so to speak.

C

B

Then separately, there's a Valerio snapshot. Location create command, which you know requires any fields that are specifically required for snapshot locations.

C

Yeah but those cases you are deliberately creating a location, and here we don't know if the user is going to publication doing you know doing using through a config they they can still go and create a location with with the other flag up here, get back up now, where is it backup location, this connectors to create backup, location.

A

They could we use that in a day.

A

hmm I'm wondering if we could add a flag to that to say this is the default one.

C

Sorry, are you saying to.

A

C

C

There is a flag that sets. Is this one use, backup, location? That's the default, stop backup! We already have this flag is called default, backup, storage, location, I, just renamed it to use backup location. Is that, were you saying I'm.

A

Saying well, I think: that's isn't that on velaro server! Yes,.

C

A

So I was thinking for velaro config that we would say when I'm, creating this location, I give it the bucket I'm doing all this creation and then, if I pass the default the default flag, it then and goes updates the the deployment and adds the views yeah.

C

A

C

A

So that way, it's clear that one, it's a backup location instead of a snapshot, location and two. It does the default. Switching for you because I think, if we, if we just have the yeah, so it would you it would be similar to this and what are all the like like he was saying all these things could be strongly typed and required, but we could make it default and do the server update.

A

Because I think the the command that does both snapshot and backup locations, it just gets hard to balance the required fields between both of them. Okay,.

C

So I think I think we were, you were bringing up is actually two things. What you get about, having that make defaults flag when you we are creating I was actually checking if reason and a half that cuz I that's a great idea, we should definitely we don't have that, so we should should have of that. So we win, we have winner, we are creating a hash, but then there are a bunch of configurations that we also have to import now. This is so so, let's I'm gonna add that to the notes later.

C

So this is trying to address the intention of making the initial setup as quick as possible, and that's why we have like with the install now we have the ability to do like a default location for them automatically as long as they pass those two three from informations, with the back bucket name, prefix in the plug-in name. So if we get there, we can create the default location for them. So this is what this is. What it is trying to address now I could also so I could do.

C

We could do away with with doing this and just requiring required so few things we can do, remove this completely and require them to explicitly say the location from scribes instead of trying to give them a hand Wiki and keep this here and just like this, but I don't think this is workable, or we could keep this here just with the one set default, backup location and then do nothing or set a sacrifice to to get the input to find out. If the user wants to set up defaults, look natural location.

A

Yeah I think I think the Valera server also assumes, if there's only one backup, storage location, that's the default sure.

C

However, if Frieda leaves I, don't know the answer to that, actually, because let's say we have to back to locate back applications, one location a is the default if I don't if we delete that, I am assuming that the other location will become defaults automatically without the others are doing anything right.

A

And new backups will go there. Old backups should have the old.

C

A

One written, but then, though they can't get to it because it's deleted.

C

But I can always recreate it right.

C

So any preference I think I like keeping this, because it's sort of we our rehab, it might be handy for people who don't we don't. We won't have to require them to no more commands more flags to set up our rights with just these three things. We can create it automatically that we could remove the snapshot looking into a false and put that into a different set flag, different yeah.

B

So may I- maybe not following part of this, but it's oh, is this a separate command from the one that like? Is there also a command for just creating a location? That's not the default! Yes,.

C

Yes, so this here backup location, this creates the backup location. This is just alright. The velaro, the cap storage location, creates it would be this year, which would say rename to create so.

B

Both of those create a backup location, but the set default location also marks it as default.

C

So, oh right, so what this command would be, which.

B

C

C

Yeah yeah, this backup location flag is essential, is exactly is this flag here I mean these commands. That could be a comment too. We will figure that out later, but the intention is that that flag, slash commands is exactly the same as this. The lower back application creates ok.

B

C

This one doesn't have a flag to make make this one a default location, but that's something I think we should add.

B

Yes, I guess I'm wondering you know if we just added a flag to that and then do we need a second command.

B

Like every Atta flag to the first one to allow the user to specify this should be the default location. Does that cover? It seems like that, covers the functionality of that second command right.

C

Let me see here.

C

I'm trying to see all of the things that are not optional, nothing, he says optional, I'm, guessing.

B

That documentation here doesn't fully capture what's required versus optional, that I would think provider and bucket are probably required, and everything else is probably optional. Yeah.

A

The only thing I see that says optional is backups in Kiryat.

C

So I, don't remember to backup location via this command. Here are the only required things, the provider name, which is the plug-in name, the buckets in the prefix.

A

Prefix isn't necessary because you, you may store at the root of the bucket and for a sure, I think config is necessary.

B

Yeah so that I mean I just tried it out. The two required flags are provider and bucket for that command.

C

All right, so, in that case, I, don't see a value having this.

B

So basically, we we'd add a flag and option to the existing command or you know if we rearrange it to allow the user to specify whether or not that should become the default.

C

Yes and then probably.

B

Probably update the documentation, don't like the CLI documentation to actually indicate what's required versus optional, without having to run it. Yeah.

C

B

I think that makes sense.

C

However, with I said, I still could be nice to have this fire, which is a flag that would allow you to switch make the default. You know, switch the default location from one to another from.

A

The client side you mean without having to go edit, the deployment yeah.

C

A

I would agree with that. I think. That's.

C

Really was pretty handy, it cool yeah,.

A

C

One thing that I actually go ahead: I.

B

Was just hope you go.

A

Ahead that one all I was gonna say, was I I. Think the the Valero server flag is fine, the the flag that does it is fine and all the all I think that the client-side one should do is go to the the deployment and it if it's already there change the line. I put the name in and if it's not there at it, but the I don't think we should really need to change the the server one. So that's all I'm gonna say yeah.

B

I I agree, I would say you know we. We can keep in mind that it's an option to change how we like how the backend for that works, I mean if we want to move it to be a you know, a field or an annotation on the backup locations themselves.

B

I think we could consider doing that like there are a couple of different issues to think through, but they're solvable problems and the you know the it may be nice to move to that because then this command that doesn't have to go modify the velaro deployment.

B

It's just touching backup, storage locations, which I think have some value, just in terms of simplicity and kind of like clarity of what you know what each command is is doing, but we can. We can just keep that in mind and I. You know, I, don't know yet if it really makes sense to do or not.

C

Okay, so we'll scratch this. Does it make sense to keep I think the answer to do is going to be yes to keep a flag to indicate if there should be a default snapshot, location or not.

C

Well, I would think about it. Let me ask you a different question because I just realized I don't understand how this works so.

C

C

All right so for default, the default Becca, app storage location for me is trivial. I super understand that but I'm not seeing how we mapped to the volume snapshots only because it's there multiple of them. What are we really sitting in to the user when we say make snapshots location, a B and C the default ones.

B

So the idea here is that there is a default per provider.

B

So you know if you're, using EBS volumes and also court works volumes in your cluster you'll have you can potentially have multiple volume snapshot locations for each of those, so you could have two EBS locations and two port works locations, and so the user should be able to say to specify a default location for EBS and a separate default location for port works. That says, when I backup an EVs volume, the snapshot should go to this location and when I back up the port works volume, it should go to this location. Okay,.

C

B

It's on a per provider basis, so.

C

There is a restriction that there will be only one backup per.

B

The snapshots can only go to one place per provider for backup yeah I.

C

Meant to say, location, okay, that makes sense yeah.

A

And- and that was in preparation for replication, so we can write it too, like availability zone and then later we can have some replication logic that moves them around, but for now yeah we usually usually it's copying it into the availability zone, you're in with your compute nodes and then later on down the road. If we ever get to the replication, we would copy it out. Excuse me somewhere else, I.

C

Mean so back to the yeah.

A

So for your your question about turning turning a default on or off, is this related to the you, the dad fetch used volume snapshots yeah flag, so this one I think was memory serves. This is this is mostly for rustic users who were using rustic and not volume snapshots at all, so they were only getting file level snapshots and not using BBS or GCP, or you know they were on. They were on some some provider that didn't have a Valera plugin, so they could just turn that functionality off and not even worry about it.

B

Yeah so I think that's probably not relevant going forward because it's you know if you want to set up a volume, snapshot, location you'll, do that and if you don't there's nothing's gonna create one automatically for you.

C

Okay, perfect, so that's how it's done well, deprecated! This I would just remove this completely excellent.

C

So plug in this is straight forward, so Stevie actually had a comment to add the the secret variable from tracks that set us up here, and that makes some sense.

B

Yeah because they like the process of adding a plug-in you're, basically you add in an it container and then you potentially create a secret and you potentially add an environment variable to the Valera deployment. You might also add service account, annotations or pod annotations, depending on the method of authentication you're using, but all of that stuff happens to the velaro deployment or the rustic daemon set so I figured you know the command for adding plugins should probably capture all of those things and we'll just need to figure out exactly how.

A

And plug in der here is that, with that edit, the deployment as well.

C

Yeah, this is only on the server.

A

So would we remove.

C

A

The server I think it has to stay, doesn't.

C

So my preference is everything that we are. Let's say we shift. We end up shifting comments from the velaro server into this. Anything that gets shifts here. I would not, even if we keep the commands, so we can use it. I wouldn't display it to the user, so I would prefer not to have to your commands, have a one command in two different places, just to make him less confusion.

C

A

Well, I think the server needs to know where to look to load. The plugins is I think what I was getting at, so the server needs to know when it's starting up where, where to read when the init containers are all mounted, it needs to know where it's where those binaries are so I think it needs to have that flag still.

A

So that's that's. Like a communication point between this configuration and the server, but.

C

Can these not be set at a later point before I have Eddie the plug-in by after in it I.

B

Mean if so, if you change that to something other than the default, then you need to go change. Your NIC containers docker file to have it like copy its plug-in binary to a different location and change the Valero deployment to have a different, empty durval. You more mounted into a different location.

B

I'm not sure, that's something we need to expose as a config option, like that's, really mainly a development development config option. Alright.

C

So we'll leave from here.

A

Yeah cuz, there's velaro server, plug-in and I think we want to leave that one alone.

A

Because that's how the server itself loads plugins, that it uses to act on like restore item actions and backup, item actions and things like that? That's that's! Not user facing! That's all like internal! That's what we do when we're doing, restore and backup, and things like that. I'm.

C

Sorry, which one did you say because I only sleep like in terror.

A

Actually, I think I think it's actually hidden. So if you run a velaro server, plug-in I think Steve set the there's a flag in there from Cobra that you can say it's hidden, so you actually run it directly. All complain where is Valero.

C

I think I. Remember. Okay, so don't.

B

C

B

Plugins serve I. Think is what it is. Well I, don't know, I know what you're talking about and yeah it's an internal server thing that we shouldn't really be changing, but.

A

Yeah, I think I think that one in Valero server, space, plugin, derp I, think we can leave a loan just because that's that's not that's a developer thing, not in a really a user thing. Okay,.

C

Now, when we were talking about environment variables and secrets, should these two flags here be moved to the plugin I.

B

Think so, yeah no I mean their plug-in level of things.

C

A

C

A

Think that makes sense. I know Dylan Murray from Red Hat has mentioned. He thinks those should go under backup, storage, location, which I think that's a worthwhile conversation. I, don't know if I completely agree, but I'm willing to hear him out on it, but I think plug-in is a place. To start I mean that's probably less change to our architecture than his proposal.

A

C

B

C

I I wasn't now that I pointed that out. It makes for me it makes sense to be with a plug-in, because it's necessary for that plug-in to run the backup location is, is it's an abstract, is a concept where we're just saying: okay, this is the connector between what you like your backup in the actual connector here. So this is just this handle saying: okay, you can connect here, you can connect there, so the secret's out, in my mind, definitely goes together with the plug-in.

A

Yep I would agree with that I. His argument was that it's putting it all in the back of storage location condenses all the information that you need to get do it in the backup, storage location, but I I can see like putting it all with the plug-in condenses all the information you need with plug-in, but.

C

A

I would I would say more of it needs to be with a plug-in than it needs to be with the BSL, and you need the deployment. You need the information in the deployment anyway, because the whole deployment has to get to stuff that those processes need to get there. So I would lean towards starting with putting it with the plugins, because that's where they're at now, yeah.

C

The plug-in just depends on that to have holy for it not to go together right.

A

And we have to redesign a bunch of stuff to move move all that to the B s, l's and and and yeah I would rather start here then move it all to locations. So yeah yeah.

B

And I would so I would also throw out the environment variables here so so currently we we basically put an environment variable for all of our providers base velaro, deploy so hope. Mint has an AWS and VAR GC, p1 and azure one and I. It would be really nice to remove that and basically, as part of flora plug-in add, actually you know specify hey and I need to add this environment variable with this value. That way, we wouldn't have to have them hard-coded in in bolero itself,.

C

So that would go together like if I am doing the little plugin adds. So the addis is adding an image. Should we have a different flag to add the environment variable or variables that go with that image?.

C

Or whether would it be like a high level thing, we just we in any environment level that will go with any plugin.

C

You know what I mean I would yeah.

B

B

I'm not sure that I have a strong opinion yet on whether it should be. You know like a single command or multiple ones, but you know, as far as like getting a plug-in to be functional. It's adding the init container, it's creating and mounting the secret, and it's setting that environment variable. So I think you know we need all three of those things plus some annotation stuff and.

A

C

B

Under velaro plugin.

C

Could we have a config map where we will have contains, or they all the environment variables for each provider or plugin, that we can then fetch from and configure things.

B

Yeah that could be an option so set up a basically have the user, create a config map with those things and mount that in along with the secret I mean.

C

I think we can have a command just like cube, control access, the config file. We say: okay, set this environment vero, whatever name we give it and then things will be added entries will be added and removed according to the plug-in you're trying to add something like that work, work.

B

Potentially yeah I didn't I'd have to think about it, a little bit more, but that's it's definitely an option.

C

So what are you are talking about as far as annotations teeth are these new flags that were going to need, or is it for many of the like, for example, power, annotations.

B

Yeah so and we'll need to look at it a little bit I mean that you know the reason we have that pod annotations flag is basically to support the use case of of users who are using cubed I am which requires that you set certain annotations on the pod, telling it what like what I am account or what I am role to use, but that's that's really again, a a plug-in level configuration thing and the other one is the service account annotations, I, think I'm.

A

Trying to remember.

A

Magette exactly I think G. The two, the two plugins that take advantage of it at the moment are AWS and GCP and I. Forget which I don't I, don't think both use service account annotations, but.

B

A

B

Service account annotations is for gke workload, identity, yeah,.

A

But for definitely for AWS you can get around having to give them a secret file, but what you have to do is put it put annotations on the velaro pod, the rustic pods and any other pods. You want to have access to those resources, so it has to go on all and it like Steve said it's. It really is required by the pods I'm, sorry by the plugins, but it becomes like a velaro pod level. Thing like it goes on the velaro and rustic pods.

A

But really the requirement is from the plugin.

C

Okay, so so this obviously needs to go in there again. This flag is a annotations.

A

Yeah I, don't think that's required for an initialization.

C

All right- and this is just what uke you saying.

A

Right right now, it is yes all.

C

Right so so move these to under plugins, and then we will see what else needs to be done under changed for annotations now. This makes me think that the well right now, the only thing here that I see that needs to be set for initialization, is damage. I might me see something else.

B

Image isn't even required, I mean you'll. You know if you're, using an official release binary of velaro it'll automatically use the appropriate version tag. So I was gonna say when, when you went through the section before that personally I think in it and set can be combined. Yeah.

C

B

I just realize.

C

B

We're clearing so many of these other flags. Out of you know, the install command I think that the set that will actually remain here is relatively small, I think they're, probably all optional and so I think I think one command. It's probably okay, yeah.

C

Should we call it in it I.

A

Think I'm fine with that I think I think changing the name makes sense. Given that it's change the the functionality is changing. I think that Matt makes a clear signal that it's not doing the same thing. Yeah.

C

A

Think some behavior will have to change because I think what's gonna happen, if we we changed it today and it just you know: click the image on its gonna sit there and complain that I can't connect any it's like. It's gonna complain that there's no plugins, oh yeah,.

C

That behavior is gonna, have to change, but.

A

But in terms of just putting it on there, I think yeah I think this. This meets the criteria because we now have. We have default limits and requests for the CPU and memory, so people can tweak that if they need to and yeah the rest that it puts the puts the namespace, it puts the I guess I guess that would be the one thing if they wanted to put it into a different namespace.

A

C

A

Only thing that I would throw on there I'm.

C

Sorry say that again watch there's a recording, so I can watch it again, but.

A

So if, if they wanted to install velaro into a different namespace, called something besides velaro, they might want to end it into I. Don't know backup the namespace backup that might be an optional command on an it, because I'm assuming an it, would handle putting the CR DS on the server making a namespace and then putting the deployment in the namespace yeah.

A

So if they want to put, they want to put it in a different name space. It doesn't I, don't see that in this yeah and we.

C

A

Just say we don't.

C

We get that we get questions for there sufficiently so that I would. But you always have to look up the documentation to remember how that's done. Yeah.

A

Honestly, I may have left it out of here and just said: hey, if you want to change it, you selm, because the the intent for the install command was to make QuickStart easy, so I could have left it out. It's a it's under.

B

Global flags, it's a global option, I'm fairly sure it works properly it just. It applies for basically any velaro command, so it shows up under global. Oh.

C

A

Yes, if you just evil, oh yeah little flags namespace, in which Faliro should operate, yeah that that that flag is always kind of weird, because if you use actually it's like falero back up, create it puts back up in that namespace.

A

Right, which is sometimes confusing.

C

Do we wanna maybe consider moving there too in it.

A

Now that I'm talking through it I'm, not sure I.

B

Mean I, so I think we want to support the functionality like the lair Owen. It should support doing the base install into any namespace I'm, pretty sure that, as it is already like, we'll get that functionality by default. Assuming we read the value of the flag in it's just that, since every velaro command, like instantiates of velaro, api client and namespace is part of that. This flag is global because it applies equally to every single Valero's. He like Ament, yeah,.

A

I think you're right. It's. It gets inherited through the Cobra, the Cobra command, not to objects, but yeah. The Cobra commands a little funky but yeah. It.

B

Would be nice to you know not have it just like blend into the list and the help text here, that's kind of the main issue. Is it yeah this.

C

Is one flag? I will be okay duplicating in having it there because it works and having them repeat it under in it, because we do get a lot of questions about how do I make it work? Well, we'll just add a to the flag, yep.

A

And I even forgot, but honestly I'm, not sure if you can duplicate it with the way Cobra works. I know I'd have to go reread, but I think I think Cobra will be like. Oh you already added it or something like that, but in terms of functionality, regardless of the implementation details, I think in it should let you specify a different name space to put put the deployment in no.

C

I think you definitely worry, because it's a global flag, yeah, it's just that it wouldn't be so soft document to this before it using the CLI to students find out what's available, I mean it's under the global flags, because yeah, maybe because it nate, is going to have a much shorter list of flags. Maybe people who own miss that one yep.

A

That's that's very possible.

C

Consider yep: maybe we can clean up some of these global files. I don't know I.

C

A

C

Some of these I never use.

A

C

Was gonna look up how to change the namespace I totally I. Think now, I will I will remember forever see what else. Ok, so I think it's gonna work out nicely because then it's going to be installed earlier, then till they were in it's the little config the little plug-in a little rest. They can create backup, stuff and then so. I have little I think this was really nicely. You might have it at least.

C

This part here just how to document how to use for.

C

Alright I'm gonna change the window.

C

Did you have a comment? No one.

A

The the operator I'm not sure in Opera, so I'm, not sure an operator would be able to add plugins. If they all, then again you can use Valero. You have clustered cluster permission. So I guess it doesn't matter. Yeah.

C

I'm not very well versed on roles in the air was much like the scope of roles. Yes,.

A

C

Be fictitious way were just to give an idea, like sure.

A

C

It's very hands wavy. This is not official yeah.

A

I think in the overall, like scheme of it makes sense that, like the first, the top four are the things you do and then, after that you can do create.

A

You know, restore you know, do the backups do the schedules and then you can store so I think I think that overall flow makes sense. It.

C

May be config, okay, patience based on what plugins there are huh yeah.

A

A

Yeah with the way the permissions breakdown and I know, we've gotten asked this before and I think I think Cruz is working on some of this.

A

People want to see more.

A

Restricted permissions and if we can get to that point, I think I think this break it break down. Well, like will make more sense and I would love to get there.

A

But at the moment I I'm not sure if that line makes sense, just because Valero is is, if you have access to Mike, install plug-in you're, probably an administrator yeah.

C

C

A

Overall I, like the the recipe based approach, yeah.

C

It was more to illustrate like hey. This user only has one two three four steps: these other user has even less than that. I know we right now. We can't constrain ain't users that way alright. So I have to watch this video again for the discussion at the beginning, I to see what I have to do to address and he gets ops concern yeah.

B

I think I mean I think now that we've had a chance to kind of talk through all the flow like I would I would suggest you know going through it and thinking about it from the perspective of the user, who doesn't want to interactively be Madhi modifying stuff, that's already deployed in the cluster, but who wants to generate a complete set of Yam all that they can check in to get and apply at a later time, I think for the for Valero in it. That's very it's very clear that you can do that.

B

If you do it, you know dry run, oh yeah! Well, you get your yeah mo and you can check it in to get, but for some of the later steps around plugins and config and stuff. It's still not clear to me how that will work for those users, so I would I would just suggest trying to think through that and think about what the options are there.

C

So the difficulty here is because the plugins wouldn't have default values right they'll have to answer.

B

C

You say I think.

B

There are a few complexities like part of it is you know so like with a with valera? Essentially, how do you generate a Valero deployment, yeah mol? That includes all the plug-in information that you need, so the Indic container and any secrets mounted and any annotation set? How do you get to that point through you know this new set of commands, mm-hm.

B

Without actually applying it to the cluster, yeah and I I feel like we may end up needing a separate path for those users like maybe it's just two. You know two parallel paths that don't that aren't aren't the same, but I, don't I, don't know yet I just want to make sure we continue to think about it and don't ignore those users, yeah.

A

Like I honestly think like this becomes a I think it becomes at least three I, don't like saying it, but at least three distinct ways to do it.

A

There's this way, there's helm and then there's customize, where the plugins become like a layer on top of the base, so that what we're providing and generating in the plug-in repos is not something that you cube, can keep CTL apply, F, it's keep CTL, apply, K or customizing, and that that would layer on top of your, your your valaria or deployment I, don't know exactly what that looks like that's kind of spitballing but I. Think despite the maintenance burden, that's probably the path of least resistance.

A

Just because making these commands be able to operate on or operate against a cube. Api server and a local file seems like it's not worth the effort. Yeah.

B

My my head was definitely going somewhere along the lines of the customized thing and I think we, you know we went away from that with velaro install for a reason, but now that we've separated plugins out to their to their own repos and we're talking about splitting up the workflows like we don't have to maintain every permutation of like here all the different ways you can configure the velaro server and then hear all the different ways you can install AWS like we don't need every possible combination of those materialized out as yamo its we just have like here's the base stuff, here's the plug-in stuff you can mix and match it as you need it and I think, like the maintenance burden, isn't as bad as it used to be.

B

Certainly it'd be nice. If we didn't have to support three different ways of installing things, but I think it's maybe a little bit more manageable than it was Prior yeah.

A

Yeah, if we can recruit people to to help maintain those or give us examples like I'm Way, more open to having gamal examples.

A

So especially if we go the customize route, having messed with it that now, if we, if we're doing the layering approach with customize, that yam was way shorter, so applying the layer for a plugin I, think it's more or less, not not exactly, but more or less a diff that you're applying on top for for the files, you're modifying so I, think I think it's a little bit less for our plugins to say: here's the change you're making to the base deployment to add the image.

A

That's not as bad and I still think we can put. We can work on some release tooling, to make that more automatic for incrementing things and stuff, like that, but yeah I think we're gonna end up, at least for the time being. We're gonna end up with three distinct, three distinct ways of doing this at least and customized I. Think.

C

A

In the conversation, because it's in cubes ETL they get basically is built-in, so yeah.

C

So, oh just so, are we saying that we will ensure that the setup works with our CLI obviously inhale in customize, but we are not concerned about being able 12:00 p.m. to use with cube control or did I get that wrong. I.

A

Think supporting custom by supporting customized.

A

Customized is inside of cube control. So if you use cue, CTL Kay, that's using the customized built into cube control, alright.

C

A

C

We get customized if custom I. If we get get this structure the way they worship customize, then you will work for control right.

A

Yeah yeah, so I I would I would count that as supporting cube, CTL yeah keep CTL keep I'll just pronounce it all the ways. I love.

C

It, like you, have five people in the car and just.

A

Make them all mad, because.

A

Yeah they unfortunately there's helm, was kind of crowned for a little bit, but now like because cube CTL merged customized support in it's. It's that's now getting traction and we don't want to completely isolate those those folks by building our tools in such a way that we make it really hard to do dry, run, output.

C

C

We talked about this yes, yeah.

C

Yeah he has any where I was I was doing some Google searches. But if anybody has any operator, they want to tell me about because I don't I haven't used any operator I in check out I can't get them installed and check for how they feel like works. Yeah.

B

I, looked at a few different things this morning, I mean I. Looked at contour I looked at is, do I looked at cert manager, maybe one other most of them provide raw yamo.

C

B

And I think yeah like having the ability to get to raw yam, oh I, think is important and that's that's kind of what the nature of my other comments were about. I think we can't totally get away from yellow sto has a CL is do CTL that has a install command that allows customization of things using like a dash dash set flag. That's kind of similar to helm, but SEO is also super complicated and probably way more complicated than velaro is, but anyway be. You know, interesting reading.

B

If you want to look more at that and then cert manager is actually has a very similar flow to what we're talking about. As far as first you install the base, and then you configure in this case issuers, which are essentially like backends for issuing certs, so they have the two-step like first you install, but it's not yet usable until you then go configure an issuer.

B

They still they're using raw yamo. I think to do that stuff, but definitely interesting to see that they're, using kind of a similar to two-step, install and configure process. So, anyway, that's what I looked at this morning out, but throw a few notes. But you know I think it's helpful to try and draw inspiration from other tools where we.

A

B

And and that helps to make sure that we're not kind of a snowflake in the ecosystem, yeah and I was.

A

Gonna ask about that if either of you had encountered tools like that, specifically to say sir manager specifically, but with that specific characteristic that there's a base system, that's not really useful until you put some other back-end on top of it, because I think that's the the property that makes our our approach a little bit more complicated as compared like contour needs and envoy. But that's to my understanding. That's the one requirement!

A

So once you get envoy, you you're fine, but if you could swap it out, that makes it a more interesting question. So I haven't done as much research as that I'm curious to look at helm 3 in terms of that. That's more client-side at this point, but it's like adding the repositories.

A

Because they've made that two-step now so you have to add, add you have to add your repositories and then you can do your installs. So I need to look at that and think about that to you.

C

All right so I agree with your comment. No one I think it is this. This make sense, it's like the action and then whatever property for that action comes as a flag. My Steve it's differently, no.

B

I agree and that's you know from the beginning, like we, we asked at the very beginning of the project. We asked Joe beta for help on this, and you know basically recommended that we go with kind of cute control patterns which are very much like the command is the action and then flags are modifiers.

B

So I think and that's how things are structured now, I think it makes sense to try and stick to that pattern. Yeah.

A

I I do I do like that Valero breaks from cubes ETL and that for the most part we can flip the noun and the verb, like you, velaro backup creator really are a backup. That's one thing that drives me: nuts about cube control is I, can't do keep cute control, namespace get control get namespace.

A

So I would like to retain that just because it's a person, precedent we set in velaro, but yeah yeah.

C

I like that too, because you have two verbs to action and you shouldn't have to think you know, which is because it makes no sense which one should come first, it just two random verbs. All right, thanks in this yeah I, will highlights in a better way what would be removed from we're in that any more comments. I really appreciate this. It was super useful and we went way over time. So thanks so any.

A

C

A

Don't have anything else at the moment, I'm sure I'll, think about this more, but this has been to breeze well, yeah.

B

This was a really good session and I'll definitely spend some time thinking about the kind of the get up stuff and the customized stuff that we talked about. You know we should probably have a follow-up in a few days and just keep the discussion moving forward, but yeah. This was review I'm.

C

Gonna re-watch the video too I'm gonna stop recording. Now.

C