Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2021, 14 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Sponsored Session: Kubernetes Package Management Using Unix with Carvel, Helen George & João Pereira

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Sponsored Session: Kubernetes Package Management Using Unix Philosophy with Carvel - Helen George & João Pereira, VMware

Sharing of Personal Information with Sponsors In order to facilitate networking and business relationships at the event, you may choose to visit a third partyʼs virtual booth or to access sponsored content. You are never required to visit third party booths or to access sponsored content. When visiting a booth (e.g. by clicking on a third partyʼs logo in the Solutions Showcase or exhibitor directory, and any actions within the booth thereafter including viewing resources), when accessing sponsored sessions in the Sponsor Theater, or by participating in sponsored activities, the third party will receive some of your registration data. This data includes your first name, last name, title, company, address, email, standard demographics questions (i.e. job function, industry), and details about the sponsored content or resources you interacted with. If you choose to interact with a virtual booth or access sponsored content, you are explicitly consenting to receipt and use of such data by the third-party recipients, which will be subject to their own privacy policies.

Managing hundreds or thousands of workloads on a Kubernetes cluster is not an easy task. On top of that, dealing with various workflows to customize and deploy applications can pose even more of a challenge. No single Kubernetes tool is going to solve everyone’s problem, and as an operator you might be forced to use multiple kubernetes tools to manage a single Kubernetes cluster. But then there’s no guarantee that these tools are going to work well together seamlessly.

Carvel is a suite of open-source projects that aims to solve these problems. It provides a set of reliable, single-purpose, composable tools that aid in your application building, packaging, configuration, and deployment to Kubernetes. Carvel enables users to get software installed and keep it updated on a Kubernetes cluster in a secure, scalable, and automated way.

In this session, João Pereira and Helen George will explore how the whole suite of Carvel tools (ytt, kbld, kapp, imgpkg, kapp-controller) work and show how you can integrate them for managing packages on Kubernetes. We’ll look at a few concrete examples where Carvel can be used as a part of your toolchain together with Helm Charts, kustomize and http library.

A

Hello kubecon, we are super excited to be here today and today we'll be talking about kubernetes package management using unix philosophy, with carvalho a quick introduction of ourselves. My name is helen george, I'm a product manager at vmware. I've been with vmware for a little over a year. Now before that, I worked at pivotal labs, which was acquired by vmware and worked at thoughtworks, which is a software consultancy.

A

I've built products for a variety of customers from healthcare, travel, finance and retail.

B

Oh, I am an engineer on the carmel team, I'm also in vmware. For roughly a year I came from pivotal software, where I worked with cloud native, build packs and I helped build products like k, build and pack.

B

Okay. So let's see our agenda for today we will start with uh talking about the problem statement that uh brings us here today and afterwards we'll introduce uh carmel our tool suite afterwards, we'll see a little bit uh we'll see a demo of how my car will help with the problem that we are uh that we see today and then we'll be opening for some questions.

B

Do you want to take it away.

A

Yeah, so today's uh problem statement is as follow getting software installed and keeping it updated on a kubernetes cluster in a secure, scalable and automated way is cumbersome before we dive into solving this problem draw. Could you tell us why this is a cumbersome task.

B

Yes, definitely so kubernetes ecosystem has a lot of tools that help people do all the tasks needed and a lot of these tools do more or less the same things. So you have multiple tools for templating, multiple tools for installing software and so on, and if you are a person that is producing your software, you want to have access to all the tools that all these tools and use the ones that make more sense for you.

B

But if you are someone that is consuming software uh and if you have, let's imagine a hundred applications, you'd have to know all the tools from the ecosystem in order to manage the in your clusters. So this can be like a very cumbersome task and also another problem that we saw, that is very, very cumbersome is that the person that is consuming software will have to know somehow and have to download newer versions of the software and we're trying to address this uh with uh carvalho.

B

So what is carl? uh Let's talk a little bit about. Carvel is a set of tools, uh sweetheart tools that um do very targeted operations, and so they are very small, but they do one thing and they do it in a good way. uh What we're trying to do with carver is that you can using the uniques philosophy you can swap tools if you want to use carbon tools from if you want to use the karma tools or if you want to use other tools, we want to allow you to do that.

B

So all the tools are pipeable between themselves and they are self-sufficient. So, let's see, let's talk about which tools are we going to see here today.

A

Yeah, there are multiple tools within carvo, as drought mentioned, and we highlighted five of them here at the top, starting with cad controller, which is our package management tool. Ytt, which stands for yammer templating tool is for templating and overlaying. K-Build is used for building out container images and also it does image resolution into their digest format.

A

Image package is used for packaging image and configurations, and it also relocates those packaged image and configurations.

A

And finally, cap is our deployment tool, but our focus today is cap controller and, as the diagram suggests, cad controller allows software producers to choose a variety of underlying tools, whether that may be carval or any other kubernetes tools for managing the lifecycle of the clusters. You can easily swap and pick and choose tools there, but for the consumer software consumer, it only exposes cap controller, which simplifies the cumbersome task of using multiple tools for lifecycle management. As we stated in the problem statement.

A

So we're going to solve our problem statement in three steps. We will start by showing you how an individual can install a software that he or she developed onto their own kubernetes cluster and keep the software up to date.

A

Next, in scenario, two, we will show you how software producer can create a distributable artifact so that the consumer can install it to their kubernetes cluster, because oftentimes you're, not just using the software that you build, but what others have built.

A

Lastly, we will go through a more complex scenario where the software producer produces multiple various applications and different versions, and the software consumer can decide which application and version to consume.

A

So, starting with scenario, one where again, the software producers developing for their own and installing on their own kubernetes cluster, to solve this in carbo. One of the way to solve this is by using cap controller and a custom resource called app cr app cr is built on top of kubernetes api, so you might see some familiar yaml statements here and it specifies how to fetch, configure and deploy software to kubernetes cluster.

A

Let's take a look at these three keys in app cr fetch template and deploy.

A

First, the fetch key is defining which content you can get from where and the composability aspect of carvel lets you choose from either karvos image package, helm or git in our demo. We'll show you how to do it with git and the template key defines which template tools you would like to use. Currently we use helm, support, helm and ytt in the future. We would like to support other tools like customize or json it in the demo, we'll be using helm as a templating tool, and the deploy key defines the deployment tools.

A

We currently only support cap to install an application, but we are open open to adding other deployment tools as well. So if there's a tool, you would like us to support, please let us know now: let's take a look at these in action.

B

Yes, so let's take a look: let's take a look at uh our application, so uh we have one application here that we created some hello world that basically just writes to the pod. It creates a path and writes onto the logs just a message that is passed as part of the attributes here. uh So we do have a mcr that ellen explained before we're going to use a git repository where we have our helm chart and then we're just going to templatize use, helm template to create the manifest and then just install it.

A

Joel, what would it look like if, instead of using git here, if we could use um you know other tools like image package? Could you show us what that looks like.

B

Yes, definitely so I do have an example here of using an image package bundle uh and how this looks like, but before we dive into what it looks like. Let's talk a little bit about what is an image package bundle, so an image package bundle is an oci image that contains two sets of files just in setup files, so as they are represented here in blue and in yellow the yellow files are configuration files, so you can have in your oci image on your bundle.

B

You can have, for example, in our case, we'll have our helm chart and it will contain like our charts, yaml, the values yaml, our templates and so on. You can add whatever you want. There's no real structure here that you need to follow, except if, for example, use helm, you need to use the helm structure.

B

uh If you use ytt, uh you can use, there's there's some structure on ytt, but it's up to you how you organize this, so the other portion is those two files there that tell image package that this is a bundle and what is a bundle, a bundle. It is a graphos software. What does that mean? You can tell that one particular image like in this case.

B

Our bundle uh requires other oci images uh as well, so your software will be running in uh in an oci image and you can point it from this bundle um and everything gets tagged together and you can share this bundle all together. The images that contain the configuration plus the images that contain the software. So now that we talked a little bit and we saw what an image package bundle is.

B

We can just look at this and basically the major difference between the these two here between the app cr that uses git in the app cr that uses uh image package bundle is just the fetch, the fetch parameter, where we're just defining where to get this information.

B

So let's try to apply this to our cluster and see what happens just just so you know beforehand.

B

I created I installed cap controller on this cluster and I also created some namespaces and some service accounts, so we're just going to uh deploy this uh appcr uh into our demo on namespace and it is reconciling so image. The cab controller is reading our apps here and it's trying to bootstrap the application, so it is looks like it is finished, so we are expecting to see a pod running uh in that particular namespace that we saw up there. So if we take a look at it, yes, it is running, it is ready.

B

Everything looks okay. So if we look at the logs for this particular um for this particular application, we'll be able to see that we have a log line that comes from our health chart.

B

Okay, so let's, let's take a look at our next scenario.

A

Yeah in scenario 2, the software producer wants to share the application with the software consumer and this time we'll be introducing a different set of custom resources, called package cr and installed package cr. But before we dive in, let's clarify what we mean by a package.

A

A package is a combination of configuration, metadata and oci images. It makes it easier for the software producer to share the software with someone else and it informs the software consumer what software it holds and how to install it into a kubernetes cluster.

A

Now back to package, cr package cr app cr and adds additional metadata like public name and the version of the software app cr was designed for a use case where I write the software and I deploy it. Whereas package cr and installed package cr were designed with.

A

I write the software and new deploy use case in mind and, as you can see in the diagram package, cr is created by the software producer and acts as a definition of your application and the installed package cr, on the other hand, is created by the software consumer and it's used to instantiate the application.

B

uh Ellen, can you explain us why uh there's like this division between uh package cr and the installed cr.

A

Yeah there's a um division between package, ceo and installed package here, because we wanted to separate out the definition and the instantiation, and we think this uh the benefits is that it abstracts away the details of what and how to install an application for the software consumer.

A

The software consumer does not need to worry about how to fetch what from where and doesn't need to know the deep doesn't have to have that deep knowledge of what he or she is installing.

A

All you need to do is create an installed package cr, which references the package that you would like to install by specifying the public key here then the rest of actual fetching templating and installing will be taken care by cap controller based on the spec defined by the producer and the package. Cr and the consumer can further customize this. um You can also specify a version of the software that you would like to install or you can. The the consumer can provide data values for um for them to perform more granular customization of configuration.

A

So let's take a look at these in action.

B

So, let's take a look at our package, our package custom resource and see, as ellen mentioned, there's uh it's basically an app cr with a little bit more information to it. So this is what we're going to be applying to our cluster and, let's see what happens so when we run again the command to deploy to the cluster our package, our package, this is going to create a custom resource that is cluster-wide because you'll be able you'll, be you'll, be needing this access to this resource in order to install it to different namespaces.

B

So um now that we created it, if we look at what is like the most relevant information for this package, we'll see that we'll have a public name on it and we have a version on it. So these are the two things that we're going to need in order to instantiate this packet.

B

So, let's take a look at how might we instantiate this package, so we do have an installed package custom resource here and it is pointing to our helloworld.carval.dev in the version 1.00, as we defined in this package bundle, and as ellen mentioned, you can customize the you can customize this instantiation so in this case we're providing a secret that contains uh values that we want to have that customization and we will want to change the message that is printed by our hello world application to say message from the install package.

B

So let's try to instantiate this application and see what happens so we're going to deploy the application in our installed application into our demo 2 namespace- and this is pretty quick but right now, cap controller should be trying to do all the fetching and the templating. So we should have a pod. Oh it's already running, that's good, so our pod is already running. So if we look at the logs, it should say our new message. It was installed from an image package from an installed image package.

B

So in this demo here we are showing how the software consumers workflow can be simplified. If you leverage the installed package, uh this all package custom resource, because cap controller will take care of the fetching the templating and installing for you without you ever knowing that all that is happening, and that is very powerful.

B

So, let's take a look, and what can we do more here.

A

Yeah, so the third scenario is where the software producer publishes various applications in different versions and the consumer can decide which application and version to consume. Also, if there are any updates made available by the producer, the consumer wants to be made aware of those changes automatically here. We're going to solve this problem using two new things: package repository and package repository cr.

A

It's important to note. The difference between the two package repository is an artifact that the producer creates to group various packages together. While a package repository cr is a custom resource created by the consumer. To let cat controller know where the package repository is located.

A

A simple workflow here is that the producer pushes the package repository to an either oci image, http server or git repository and the consumer references that package repository in the package repository cr, which then the cad controller will watch for any update made to the packages within the package repository.

A

So let's take a look at this together.

B

Cool, so, let's see uh so prior to prior to this demo, I created a package repository and I have the sample of it in my disk. So this is our repository.

B

If you want and the the only you can store the package repository as an oci image as an image package bundle or in a git repository or as an artifact, can be downloaded like guitar gz from an http server, but the only thing that they need to have is a folder called packages, as we have, as we can see right here and inside of it, you will and all the custom resources all packaged custom resources that you want to make available to through this package repository.

B

So in this case, we want to make two new um package: custom resources available, the hello world version 1.1.0 pretty similar to the prior one and also the other different apps. So you can mix and match applications and versions. It's up to you, there's no real requirement from the cab controller side.

B

So, as I said, I already pre pushed this as an image package. Bundle again. I did this because you can have bundles that contain that point to other bundles, making this a graph of software and bringing all this uh all this software to what uh whatever you want right so so before we before we do anything in our cluster. Let's take a look and see what are the packages that we currently have available in our cluster.

B

So if you remember correctly on the pry on the previous um on the previous step, we did create a package custom resource. So here we go, it has version 1.0.0.

B

uh So let's create now a package repository that points to a custom resource that points to our package repository. So what does that look like?

B

It is a very simple thing where you just define where to find it and then cap controller will be able to just look at it. So in our case we have an image package bundle and it is in this particular uh repository and it is tagged because uh if you do any change, it should be updated there. So, let's deploy this particular. um Let's deploy this particular custom resource here.

B

Let me just pick up here so say: yes and now, if we try to find out which packages we have so cap controller is going to you see like there's already two new packages here, so cap controller will process this custom resource and it will try to find what what are the package custom resources that are part of this package repository so two new applications, uh two new packages are shown up here.

A

What if um the consumer wants these updates uh made aware automatically, instead of manually, checking and pinging how? How can we do that.

B

That's a good question so like in here also, I have you know in my disk. I do have another repository here. Another package repository that contains two more versions of the software that we had before. So we contained the version 2.0.0 of the yellow world and the 0.0.2 of the other app, and what I'm going to do is I'm going to create an image package. Bundle I'm going to push it to this location and to this tag and we'll be able to see that image that cap controller is able to uh process that information.

B

So, basically, now I'm just picking up that folder in my disk and then I'm just creating a bundle with it and pushing it to the registry so uh now image package when it is time again to check to see if there's a new. If there's something new on this package repository- and this happens roughly every 30 seconds- it should give us all the new packages that we are looking for.

B

So we need to win a little bit and there we go so now we saw that our consumers can can just rely on cap controller to get updates when new versions are are sho are added to the package repositories, so this concludes our uh this concludes our demo of this of this scenario, and we are showing exactly that. It is much easier and it's much more automated for the um for the package, consumers to have to be notified and to start using new versions of this topic of these applications.

B

So what did we learn here today?.

A

Yeah today we learned um by demonstrating that package management workflow is much simplified for the consumers. There's this consistency for consumers, because he or she does not need to know the underlying tools that the producers used. While it provides flexibility for producers, because you can still pick and choose whichever tools that you would like, and we've also showed how various custom, resources and concepts that we've introduced today on how it leverages standard apis like kubernetes, api and registry api, and it comes the benefits of leveraging standard apis.

A

Is that because they're well known standard apis is easier for new users to learn how controller works and also reach the benefit of standard apis reliability?

A

Finally, we saw how different custom resources and concepts follow unix philosophy and they have well-defined boundaries and separations of concerns.

A

So, to recap, all the different resources and concepts that we learned today, image package bundle is a packaging format that is not tied to the type of the content and it acts as a shareable unit package is a combination of configuration, metadata and oci images where it's represented by the package cr and it tells cat controller where to find the kubernetes manifest and how to template and install those manifests.

A

We've also learned in scenario: three package repository is a collection of packages that are grouped together that helps distribution of multiple software and versions and package repository cr is used to point cac controller to a package repository so that the consumer could get that automated update and finally installed package. Cr is used to install a particular package which ultimately results in installation of package resources onto a cluster, so that wraps our talk today. Thank you for joining our session.

A

You can find more information about our tools on our website carbo.dev, and you can ask questions or ping share your thoughts or opinions and feedback to us by just messaging us on our slack channel called carvel within the kubernetes workspace. So, thank you. We'll see you at the q a.