Cloud Native Computing Foundation Kubernetes Forum Sydney 2019, 12 Dec 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Full Stack Cloud Native Packaging – An Intro to CNAB - Ben Griffin & Clarence Bakirtzidis, Microsoft

Description

Join us for Kubernetes Forums Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Full Stack Cloud Native Packaging – An Intro to CNAB - Ben Griffin & Clarence Bakirtzidis, Microsoft

Cloud Native applications require more than deploying Docker containers. At the very least, we need to configure infrastructure such as databases, load balancers, and networking for our application to run. To further complicate matters the use of heterogenous technologies and tools might be required.

Wouldn't it be great if there was a technology that took care of all this for us? Cloud Native Application Bundle (CNAB) address these challenges by securely packaging your artefacts and toolchains required to run your application.

In this talk we will demystify CNAB by looking at the problems it is trying to solve, diving into the spec and various implementations including Docker App and Porter, and how tools such as Docker Compose or Helm can be incorporated into CNAB. We’ll round out the presentation by applying what we have learnt through a real-world use case.

A

So welcome we're going to talk about full stack, fired native packaging in intro to see now my name is Ben Griffin I am a edutainment or specialist, focusing on open source in cloud native technologies. Work is Microsoft in part of the global black dog team. Everyone.

B

I'm Clarence Packard cedar so similar to Ben I, also work in the global black belt team across Asia, focusing on cloud native solutions.

A

So cloud native applications are generally made up of many different parts so who, in the room here, for example, is using tools that we just play here for your hand, up yep and that's congruent with what we heard this morning through Cheryl the CNC F, with her Paul, also with Andrew from IBM. You know we have our cloud native applications that are complex, they're hard to deploy, there's lots of moving parts. So what we need to do is need to look at. How do we manage the middle where the infrastructure tooling?

A

How do we manage it moving to multiple clouds? How do we ensure that, when we're deploying our cloud native that that the admin knows how to do this work for us there's loads, our main knowledge kept up in? How do you deploy these correct I'd complex, distributed applications? So docker has helped by simplifying some of this process by allowing us to package up our deployments for our real web apps, but there's still a lot of many moving parts that we need to still configure and manage. So how do we simplify the delivery of these applications?.

A

So who's been in the situation where you're deploying your cloud native, app and you've got to go to github and you've got to follow a big laundry list of items right, we've all been there right, github or we go to an internal wiki or something like that. We've got to follow this whole big list. Now, when we're doing that, you know, we've got to follow of setting up our infrastructure, setting up a DNS sending up. You know SSL keys and things like that, not to mention its application in question as well.

A

So we incur these type of problems where we've got lobsters desk, we need to follow and because we got with lots of steps and we're human we're primed to make mistakes. Of course. The other thing we need to consider is when we're following these steps, the person has written them all of their domain. Knowledge- or you know, brain is not like leads on to the page properly and might miss certain things. They might say you know configure DNS and you just go. How do I do that or create a SSL key or SSH key?

A

How do I do that? I don't know. So you know there is a certain amount of domain knowledge as well required with setting up these de Reynaud apps as well. So what we want is a repeatable way of having an end-to-end process that will just work for us every time. The same way now contain the soul. Part of this problem, of course, but you've still got other moving parts that we need to worry about. Again.

A

We still have the infrastructure component that we need to set up we're still going to be like in this instance, we're still going to think about. How do we install helm? You know how do we install like kubernetes, for example? These are quite complex things. How do we set up TLS? So what if there is a way that we could package all this logic application our application logic into a single bundle that we could then verify and know exactly what it's going to be doing and have this repeatable every single time?

A

So imagine if we can install this artifact into the single tool, so you know introducing that could be thought, but no not quite. But what we're trying to say here is that you can just have your application. It's tagged. It's got a version and here's my application go and run it for me. So wouldn't be nice if we had someone who could do that for us. So this is where we're going to introduce you. The cloud native activate in the cloud native application bundle so see now.

A

So she knows the packaging and installation installation specification. There aims to safely distribute application repeatable way. Over and again, it's um agnostic specification that has been created jointly with Microsoft, actually called pivotal, Red Hat and Konami and docker, of course. So the idea is, it facilitates bonding up and installing of our application, including the infrastructure so such as cloud storage such as DNS or a load balancer, it also can be prefer graphically signed as well, so it can be verified, which is important.

A

So let's talk a little bit about what a seen that bundle is. So we have our application image or a pin. Is you know, essentially, this is just really a docker image, no different to what you do today when you're creating a docker file and creating a docker image. We have something called an invocation image now this is a docker image, but this contains all the tool change needed to install your application, such as you know, terraform was your shell, scripts or stuff to do with.

A

You know, helm whatever it could be that your application needs. We can install into this invocation image. You can add like a progressed database, you know all the schemas and things like that. We could run to setup as well, so it also contains all the configuration data and metadata and templates and things so think of this bit like an msi of hacky exchanges from a window as well or think of this, like is a bit like app gets like a thing for some linux. We also have something called a bundle descriptor.

A

So it's a lot like a docker image again like a dock of specification image, it's a JSON file, but what this bundle descriptor does. Is that basically lists out in the in JSON format? What the invocation needs the invocation image needs to run. It lists all the application dependency. It needs to run from the app images, the definitions in inputs and outputs as well, that you can specify and finally, it can be signed. Now.

A

All this goes into is seen a bundle, and this can be bundled up essentially and then shared with an OCI compatible registry such as docker hub or in Asia, as in container registry etc.

A

So, let's talk a bit about what the implication image is. So, as I mentioned, it's essentially a doctor, a docker file or docker image, but it has a very specific strap, as you can see, on the right hand side. So in this case we're saying they're using like, as you see, Li or using arm templates or if you're from AWS CloudFormation helm templates. All these things can be actually dumped into this file structure for our application to run.

A

Finally, what you can do is you set like an entry point into this invocation image, and so therefore, what is it going to run? Is it going to install update and these things- and this was something we'll explain in a moment finally, we're going to talk about? How do we implement a scene out or the cloud native application bundle, so we've got Porter, which is an open source project created by Microsoft.

A

Porter is part of a joint development foundation, which is part of the Linux Foundation, and essentially it package up all your artifact and tool chains. So we've talked about you know terraform home scripts, etc into a format, and then it uses something some mixin towards where it talked a bit about later how they work. Finally, what you can do is you can use border to just deploy our application a bit like what we saw before with clipping. We also have docker app, and this is dropped. Its implementation of the scene app specification.

A

It works among the same type of you, know, docker app, workflows so quite similar to how you do your docker image workflow at the moment, so you created a definition. You build the image and then you share that in to like say, docker hub or something to that effect.

A

Now today, we as I mentioned we're going to focus on Porter and what we've done here at Microsoft on that night I'll hand it over to Klaus.

B

Thanks Ben, oh, you can just leave it for now cool, so porter. So it's been mentioned. There's a few different fermentations of a see notification. All this tooling, to support working we've seen our bundles, so port is the one that we're going to focus on.

B

So what is Porter so Porter helps you it's tooling to help you create this scene, a bundle, you know see, know the specification. If you look at what's contained in there, it's designed to be a machine-readable. It's designed for machines, Porter kind of simplifies the process of building these inner bundles. It's designed for developers and humans to read the great thing about Porter. Is it's not only about packaging up just your application, but also the tool chain to deploy your application and configure any supporting infrastructure needed once you have these bundles?

B

You can actually then share them, so you can push and pull them from a docker registry like docker hub, for example, and quite interesting thing is a use case where you might want to have a self-contained bundle that actually contains all the connected pieces. To think of, like a big tar ball that you might want to share into an air-gapped environment, so there's a use case for that where you can actually support.

B

You know a self-contained installer and lastly, second last I should say: Porter can be used from the command line, but there's also scope for building graphical user interfaces and self-contained install that self runnable stall installers as well, and you can extend Porter so as you'll see later, that we have a lot of existing modules called mix-ins. That kind of help you to install your applications and deploy them. But you can also add your own extensions supporter.

B

So how do you work with Porter this two main personas I work at so as a developer, you're going to create a new directory and then run Porter create, which is going to scaffold you a number of initial files that your edit and, once you add the steps that you need to deploy your application. Then your run Porter build now. You'll notice, with these commands, there's no additional arguments because we're running them within the scope of a directory where we're developing on the source code.

B

Now, typically, we deploy applications they're going to use some resources that need credentials so think of for cloud. You might have some access keys for databases, you might have some passwords, etc. They're not going to be stored in the bundle, obvious reasons. So there's a step to ascend. We generated a credential file, that's stored on your machine, then you can install the application once you've built the bundle and if it doesn't do if it doesn't meet your expectations, then you can iterate on that.

B

So then you can make some changes to the code build again and then install as necessary and once you've got something you're happy with. You can tag that and have it pushed up into say something like docker hub now, as a consumer you're not going to necessarily have the the source code that was used to build the bundle. So let's say Ben over here is built. This application and Ben says: hey Clarence, try out my app.

B

So he gives me a name and a version number now I want to know what this app is and what the requirements are in terms of credentials and parameters. So I can actually run a command called Porter, explain and port. Explain will show me some information and we'll see this shortly in a demo to make it more concrete, I'm going to generate the credentials necessary and install the app using that specific version and and tag now, there's also scope for upgrading applications.

B

So, when updates come, I come in update my application and finally from done with application. I can have it uninstalled, and the great thing here is as a consumer I don't care about the tooling, so it was using terraform and civil helm, whatever you want to use internally. I just know how to use Porter, and this can actually be used with other tools. So if I use Porter to build the scene that bundle I could actually use like docker, app or another tool to install the bundle.

B

Now, when you create the initial port of project, you get three files, as you can see here, the main one and the one that's sort of mandatory is port Adamo and that's sort of where the magic happens in terms of the steps you want to perform. Now, if one of the mix-ins doesn't do what you need- and you don't want to create a mixin, you potentially can add additional binaries or other files needed during your deployment process into the docker file.

B

So you have the opportunity to customize the docker file, that's generated, but if you don't need to do that, you don't have to touch the docker file and once generated for you on the fly when you build the the bundle.

B

So if you look up here, this is a sample of a port, a yellow file. I'm not expecting you to read every line of it, but I'm highlighting some of the important bits so declaring what big scenes want to use. So here we're using an exact mix in which is kind of the easiest one to use when you can't or not the easiest one. But let's say you can't find a mixing that does a higher level abstraction. Then you can fall back to exactly and execute any task you need.

B

Kubernetes mixing knows how to apply, manifests to a kubernetes cluster, and here we can declare what credentials we need. So for kubernetes you typically, you need a cube, config file, so we're not going to package that up. We expecting someone to supply a path to where we can find that here we have some metadata. That explains what our bundle does and that it's namin version and also what the invocation images and the tag is. The bundle itself and you'll see there's actually two parts.

B

Once we publish this image or this bundle, here's the installation process the uninstall, and they could also be an upgrade path in here, which we don't have on this slide. And lastly, we've got outputs so, for example, if you're provisioning a web service- and we want to know what you know- the IP or fqdn or the domain name- we can output that for the user to then try the app out now with mix-ins.

B

These are the ones that are currently available, but you can you can create your own and contribute those back so, for example, helm lets. You apply home, charts cubanía to your play, gooeys manifests and you can, as you can see, we have other ones for different cloud providers and terraform and the beauty here is these encapsulate the knowledge of what tools do I need to install? Are there any additional steps that are needed, for example, for using helm? You need tiller in your cluster.

B

Well, the helm mixing would actually go in, install and configure tiller for you, so you wouldn't need to know how to do that as a consumer.

B

So with the mix-ins, you can also get hold of outputs and you can use those outputs in subsequent steps. So, for example, if I provision the service and I have an IP address, I might want to add that into a load balancer in a subsequent step, so I can actually reference that output and likewise with helm, for example, we might have produced, say a database in this case MySQL and I want to get hold of the password from the secret that was generated.

B

Okay. Now, let's look at a demo app, so this is called spring music. It's a spring food application that is containerized a deployed on kubernetes now interesting thing here is get to in a second is that it uses kubernetes. It could be any kubernetes cluster and it also deploys database into digitalocean, so Postgres database and I've already packaged this up. So that's the C NAB reference to pull that down. So what I'm going to do?

B

It's going to switch out to mirroring modes? I think this is going to be a bit easier for me.

B

B

Right, so what would this we'll kick off the installation now you know how live demos I do have a backup recording. So let's kick off installation and it'll come back to the slides and look at what the architecture for this bundle looks like.

B

So here you can see, I've published my bundle and you notice there's two parts: there's the invocation image, which is the second entry and then the bundle itself, so bundle is actually a JSON file, but because registries used to OCI compliant I open container initiative compliant, you can actually push other files or layers up to those registries. So in this case we can push the JSON file, which references additional images.

B

So the first thing I want to do is I want to understand. What's in this package, so I'm running the explain command. So the explain command is telling me spring music. It runs on kubernetes and it uses digitalocean Postgres and here I can see some credentials that I'm going to need as well as parameters. Now the parameters have some defaults that I can override and at the end I can see. Output is going to be a service IP address.

B

The next step is to create some credentials.

B

Now the bundle itself specifies the credentials it needs and you can supply these in different ways. So for digitalocean I'm going to supply these via the environment, variables, I.

B

Explained these are a bit further shortly.

B

B

Secret, so these are the three pieces I need for generation and for cuba. Nettie's I'm, just gonna use a cube. Config file from a file pass so just paste that in to config, so now we're almost done. We have a credential file created, but I need to actually have those credentials available, because the credential files, essentially a pointer to those so the first thing I need to do- is I need to source those credentials from a credit from an environment file.

B

Alright, so not going to show you the credentials, but I'll show you what the what essentially is in there. So you can see here. I've got the three credentials: I'm sourcing into my environment and lastly, I need a kubernetes config file now, because I'm using aks in Asia I'm, just gonna tell it. Please give me the coop config file for that cluster. I already have running cool, so now we can actually run the installation, so the installation, you need a name, the bundle you wanted to install.

B

Then you can see the dash C is the credential file reference that we configured and then I'm overriding some parameters because I want my database called, for example, a certain thing in this relation right. So that's gonna run through a few steps now in interest of time, let's switch over to the video- and it's always good to have one of these, but we can see now it's running terraform Terra forms doing its thing.

B

Alright, what's going to happen up at terraform, is we're gonna provision a database and then eventually after the database has been provisioned.

B

As you can see here, we're an helm, so helm has have created a deployment, a pod and a service in kubernetes and now there's a script waiting holding for the IP address. The public IP address to be available.

B

So if you fast forward ahead, we've got the IP address and you can actually ask port ID like show me some of the outputs.

B

So here we're saying, show me what Spring, music's outputs we've got an IP address, and then we can paste that into the browser and in a moment as you'll, see we'll have the spring music app up and running so that didn't really require any knowledge on my part, I'd had to provide the credentials now, as you see it's still running in the background, but we'll leave that just want to show you one more thing before we cut back to the slides.

B

So because I'm using digital ocean I got a command-line tools. I'm clearing for the databases show me yet there's a postscript database and I'm also querying for some blob storage that was created. So let's have a look at that now in the diagram in case. There's some confusion as to what was happening.

B

So I, as a user ran, Porter I said: hey Porter I want to install this bundle. This bundle comes packaged with a few mix-ins, as well as instructions on how to install the application over on the right. We've got digitalocean.

B

So the first thing you want to do is create essentially like a blob storage account, because, if you're familiar with terraform, it creates a state file that represents the state at the state of the infrastructure is provisioning. You need to store that somewhere safe, so we store in blob storage. Then we run terraform.

B

It stores its remote state and then terraform creates the Postgres database in digitalocean next we're using an existing kubernetes cluster. Now we could have created the Bund or we could have told the bundle when we designed it to create a cluster for us. So that's a design decision or you might have an option that says, skip creating cluster. In this case it assumes as an existing cluster. So it runs Hjelm install we get a pod service.

B

The pod pulls down an image from docker hub, that's already been previously published and that's basically an example of a bundle. Now the bundle could be more complex than this. It could be simpler. It's really up to you in your imagination, what you can produce so here I'll pass over back to Ben.

A

So what we've seen here is that deploying our fayed native application he's going to be a complex task. 10 lakes, this remote involving many many moving parts.

A

So we're going to look at reducing that complexity and make it easier so that scene that allows us always a specification that it specializes in how do we actually reduce that complexity and then we're using a tool such as Porter or docker up to be able to implement a simple way of you be able to deploy a cloud native application that is quite complex to deploy and as we saw with clients, we had something that had helm in there and had terraform in there it had a Postgres database. It was recruiting something in digital ocean.

A

Now me as a user I just have to log in create my Porter and away I. Go so I, don't have to know anything about how helm works and I know how to provision a Postgres database in digital ocean. So, as you can see, it simplifies a deployment process quite significantly in terms of resources that you guys can have a look at people. Even look I should say so. You've got the CNO specification.

A

As I said, this is open source, and so, if you look at CNRI, oh we've got porter supported is eight and we also have some working examples in Azure as well, so that you can have a look at as well as pack your bags application, which is the simple thing tutorial run through from zero to hero and finally, in Asia, we have some QuickStart examples that you can use as well, which is linking to Oh tolling. Let's give it a.

B

Base a very sometimes it with three minutes, sometimes with ten minutes. Let's have a look for 50, so it's still going, but um are there any questions before we conclude yeah.

B

Yes, so, for example, if the mixing does what you need, for example, if you're using say certain manager in kubernetes to generate certificates with, let's encrypt or for example, then certainly you could provision that step. If it was in a cloud. Let's say it's in in a shake would use a an arm template if it's AWS CloudFormation, it's really up to you. You can put whatever you like and then there are pre-existing mixing.

B

So it's really is there a mixing that supports what you need and if not, you can always fall back to an exec command or you could create a mixin yourself. Never.

B

No, so you would actually install all the things you need in the cluster as part of the bundle, so the consumer shouldn't need to have those things pre-installed. You know the description can say. For example, you need to have a kubernetes cluster, but that's it so, for example, the demo we use it doesn't assume this even helm installed. It will install tiller for you I'm, going to setup our back roles necessary for helm to work, so you should be self-contained, and if we, if necessary, we could have it generate the whole cluster from scratch.

B

All you need to do is provide your access keys to your cloud provider of choice.

B

A

About from an air-gapped environment, what you could do, then is you could basically export all that detail out into like a tar, Zed file and then basically push that into an air-gapped environment that everything soft contains. So you think about it being a it's thick bundle versus something being a thin bundle. It's going to access the Internet and pull everything down for you. So it's up to us cuz, we're saying the application designer who creates all the bundle to have all that info and you as the consumer. You don't need to know that they worked.