VMware Carvel Community Meetings, 10 Aug 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Package Authoring with kctrl Demo

Description

The team working on kctrl demo'd their work on the package authoring workflow during our community meeting on August 10. The alpha release of this will happen around August 16, 2022, so be sure to get it setup and give us all your feedback!

A

All right, moving on to the demo of package authoring workflows, stop sharing my stream.

B

Yep I'll I'll start sharing and I'll take you uh all through the demo uh yeah so I'll I'll. Just give a brief on the the package authoring workflow so how it all started.

B

So, sometime back, we started looking on to simplifying the package authoring workflow, because.

B

The when you author, a package, you basically need to know uh almost all of the cargo tools, and that was a barrier and we wanted to simplify that process so that the new and the like, the beginner and the experienced users we if they can come and create a good quality uh cargo packages. So that is how all of this simplifying of the package authoring has started, and we did a bunch of.

B

We floated around the survey where we asked how they envisioned the tool and it looks like because everything is happening on the cli, so the the cli was a way to go even for the package authoring commands as well and uh so coming on to the package authoring. uh So we envision that the package authoring it's kind of an iterative flow.

B

uh So we are uh package authors they they already have some kubernetes configuration which they would love to package and once they have the configuration they will like to template or add some overlays to it, which, which is an iterative process until they get it in the desired shape. So they will, you know, repeat over writing the overlay or the template part to get it right.

B

So once everything is in place, uh they would love to make a release of the uh of the package so that it becomes distributed distributable and can be consumed by the package, consumers so preferably they. They want to do it in a automated way, maybe in a github action or something like that right and uh yeah, so that that's how we have envisioned it, and now uh I'll be so now giving this in place. So we have uh introduced a couple of commands on the k control.

B

um One is like so as again the journey which I have told the commands are pretty similar in that way like we will uh see to first initialize a package.

B

uh So when we say initialize a package, uh what that command will do is it will create the required files and it will ask a bunch of questions, for example like where the kubernetes configurations are there and a bit more, maybe asking the package name so once it has asked all those basic questions, it will initialize the required files and uh the content and then um so as like, adding a template or an overlay that that's an optional step.

B

So that is where, but it is an iterative step, and that is where the dev deploy comes in. So we have not uh fully finalized on the name as of now as a dev deploy but uh yeah, but the the whole point of dev deploy is that um you don't need to have the cap controller running on your kubernetes cluster. uh But uh what will happen is the dev deployer command will give you the feel of cap controller running. So when you run the dev deploy, it will deploy everything locally.

B

In the same way, the cab controller will do, and you will have your application and up and running so. You can also like iterate over your configurations quickly and see that um the um they they are in the correct place and they are getting changed.

B

The way we want in the desired behavior and once everything is in place. All the pieces are in place, we'll see that we'll have a command called control package release which will help to release the package.

B

And then you can add those package and package metadata to the package repository and can run key control package uh wrapper release to release that package depository itself.

B

So uh that is the whole flow, so I'll be demoing that starting from initialization to the dev, deploy and do the release uh yeah and then we'll see. Based on the time, we can do the adding of the package to repo and releasing the repo or we already have covered it in the documentation which will be coming out so yeah.

B

um So what I have here is a simple app where I am a developer and I want to package my simple app for the distribution right, and that is the scenario and what I'll do is. uh I will run okay control um package in it. So.

A

B

A

Is start interrupting, so sorry, do you mind making the text just a little bit bigger, yep yep? Thank you. I don't mean to mess up your demo perfect.

B

Thank you yeah, so it is asking for the package reference name like the package name again, some basic information about how the package name should be, so let me give it demo dot karbal.dev and then it is asking from where it can fetch the kubernetes manifest.

B

So we have a bunch of options uh so this time, because I want to package my own application from my source code, so I'll say local directory and then it is asking which what configuration or what files need to be part of this package. So all of my configurations are present in the config folder. So what I'll do is I'll add the config folder here and, as I think like it will create the required files. So it has created the package build.yaml and the package resources.yaml.

B

So now, if we look into both of these files package build.yaml, this is how it looks I'll come back to it later on, but we'll. Let's look at the package resources.yaml file first, so it contains package package metadata and package install again. Some of the things are filled with uh with the values provided which are default values, so they can be changed by the users once they are changed, they will stay there, which means, when you are releasing the package at that time. These values, uh you will see these uh changed values.

B

So let me change the long description to let's say long description.

B

Right uh so now we have. uh I also want to bring attention towards the package and package install so when we said uh that you might want to add some overlays or templates and iterate over it multiple times, so how those configurations or how those uh will be picked up is by the dev deploy feature is uh via the package and package install here so dev deploy will pick up this package and package install and it will try to deploy uh it on to the kubernetes cluster.

B

So what I'll do is um so? I already okay, I have it here, so I in the config dev, I have build.yaml. So what I want to do is with the help of, uh let me hide it.

B

With the help of k build, I actually want to build my image, but at the during the dev deploy flow. I don't want to push my image to a oci registry, because it is still very much in the development space as of now right and- um and I want to add- let's say my- I already have an overlay which will add the resource requested and limit, and I want to add this overlay as well.

B

So what I'll do is I'll add the overlay to the config dev folder, and I will move this config dev folder uh to be picked up uh by the dev deploy. So for that? What I have to do is I have to come here and add it in the package of the package resources.

B

uh Alternatively, what we can do is we can run the package init again and add the config dev, so what it will do is uh it will pick the values which have already been provided, but we can then make changes to those values yeah. So what I'll do is I'll run the kctrl dev deploy now?

B

So, if I'll do kctrl deploy hyphen h, it has a couple of flags. uh So what hyphen hyphen k build build will do. Is it will allow the k, build to actually build your binary, and then we have the hyphen l, which is local so that we can use the local fetch source yeah, so we'll be using these two variables.

B

So let me do k control dev, deploy, hyphen b, hyphen l and my package and package install are present in the package: resources dot, yaml uh yeah before we move on uh one more thing: the package install it: has the service account, so this service account needs to be uh created beforehand.

B

uh Otherwise the dev deploy will fail. uh We will see that uh in the next release uh we we can create the service account but yeah. uh Today you have to create it manually, so I can show that I already have it.

B

If I'll do cube cutter, let's say so, I already have the demo service account, so let me just run it also. We can run it in the debug mode to see that uh what is going on to get more depth in there um but yeah, uh let me just run it in the debug mode, so we'll see that cable is actually building. So you can see that here the cable is actually building the image.

B

Yep and then it is on deploying that so the reconciled has succeeded, and if I do cap ls hyphen a so, I can see that my demo dot app has been installed and everything is there, but I do not have the cap controller in this cluster, so this is uh the dem. uh The part with dev deploy, dev deploy is purely optional, and then we move on to so now I have everything in place, so what I want to do is I want to make a release.

B

So what I do is as part of the release I want to act, also push my image to an oci registry which is getting created. So that is what this destination implies. Again, I will the tool will be using cable underneath and also I want to add my overlay.

B

So what I need to do is I need to go to the package build which we said will come back in a bit now package build, has two sections one: is this uh app spec section which contains your deploy and template, and then there is this export section, so um this uh deploy and template will get copied over to the releasable package. So we need to make sure that our overlay is present, which should get picked up by the ytt.

B

And I also want to pick my config release folder now uh the export will uh what today the package authoring it will. Actually, when it is trying to release a package, it will create an image package, bundle for you and what does that image package bundle will include, will be defined in this include paths.

B

So we'll add these also over here. Otherwise they will not be available to the ytt right.

B

So now, if I go in here and if I look into the key control package- release hyphen h again, uh one thing I want to highlight is we: have this version flag which is exported so that the key control command can be the release command can be reused in the automated flow as well. So let's say I give version 1.0.0.

B

Then it is asking the register url where that image package bundle will be pushed. So let me see.

B

So I am giving my docker account.

B

So again um it will run the k build, but this time I am not running it in the debug mode. So that is why we are not seeing that whole output, but what it will do is it will again build the binary push that binary now to the docker registry.

B

So that is what it is doing. As of now.

B

And then it will do the image package, it will create that bundle and it will do the image package push and we see that these are all the content of the bundle which we wanted to include and our artifacts are created in here. So if I'll do get off, this is the package metadata. So, as I said like, whatever the value will change over there, it will stay in in your even released package.

B

So this is with respect to the package metadata and if you look into the package.yaml.

B

So it has this image package bundle which was created as part of the release flow and then on the paths in the ytt and the k build, and it has also generated the value schema from the values dot yaml file, which was present in the config folder.

B

um Yep from this, so um the next step is to add these packages and package metadata to the repository. So if, if we are interested in going over there, we can do that. Otherwise, I can stop here.

B

C

Maybe, to make this a little bit more digestible, it's probably worth stopping over here I don't know that's at least my opinion.

B

Okay, yeah, and uh so we we, I have already raised a pr on the secret gen controller, uh where you know this package release, we can see how it is being used in a github action, so I'll recommend all of you to go through that.

A

Yep so row had a question. You said it's like running uh cap. It does the same kind of thing as cap controller like in the cluster. Is it like it or is it literally cap controller when you do the dev deploy.

B

I'll say uh it is like it like. um The underlying code is same, but we do not do any resync or reconciliation, so that is not happening, so it is just like one time deployment if you can think of doing with the cap controller.

C

That's fantastic. I.

A

Love the little flourishes there's little bits like where um the k build uh configuration automatically reaches in into the images yaml in the bundle like you don't have to remember. To include that. There's a lot of little things I saw like the overlays had both the the kind and api version in them like it was really neat, seeing like all those little easy to easy. Gotchas are just like covered just by using the tool.

B

A

You it's cool thanks for the demo, so.

C

Here's a quick thought for folks who work on cap controller that I just uh kind of connected is when we do development of cap controller.

C

um If you're, just focusing on that singular reconciliation area right, you could actually now run this thing directly.

C

This is once we combine the cli and the cap controller go mode, so you can potentially skip the deploying to kubernetes right. You can just use the you know the code that you're building, let's say as you develop cap controller, you could also be just executing it directly in the cli form.

A

It I mean it seems like that'd, be especially useful for testing uh like we could. We could pull a bunch of things out of our end-to-end tests. Is that fair statement, dimitri.

C

I probably wouldn't go as far as that, but it probably really depends on what kind of tests you're talking about right, because a lot of the tests are exercising the kubernetes based flow right and we still want to ensure that that's all correct right, uh but maybe you know focusing specifically on.

C

I don't know the deploy part or something like that. But yeah. I don't know just just the thought. We'll probably have some further discussions along the way.

A

Well- and I I you and you you did say this: isn't this isn't quite enabled yet because the way that you all did the depths right.

C

Right today, the cli and the cap controller controller. Do not they have two different goal modes and I think the plan was always to to get the merge together. So it's a single project.

A

B

uh One thing I'll say is uh way: we encourage uh all of you to try it out and share your feedback, uh because uh this just this is an alpha release. So the intention is also to get the feedback and try to incorporate it in the next releases.

A

So if I grab the latest now I'll have this, how do I get a hold of my copy of the cli.

B

Yeah, so uh we will be uh merging it uh this week and we will be cutting our release next week, um but yeah. If you really want uh there is already a pr. You can build that pr. I can share that vr.

A

All right, thank you. Roy any other questions comments uh I did want to say neil's comment out loud. Neil said this is really cool stuff, no more or at least less manual, yaml writing and getting the indentation wrong. So super cool stuff.