Cloud Native Computing Foundation App Delivery Special Interest Group, 15 May 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: CNCF SIG App Delivery Air Gapped WG 2020-05-15

Description

CNCF SIG App Delivery Air Gapped WG 2020-05-15

A

I will give people a couple minutes to uh come in before we get started in the meantime. You can add yourself to the attendees on the agenda that I just posted in.

A

A

A

All right, it looks like we have uh quite a few people, so we can get started um hello, everyone. This is the uh air gapped working group from cncfc gap, delivery, friday may 15th um our agenda is posted in chat, uh feel free to add yourself as attendee. Today we have a demo from carolyn from microsoft for cnab and porter, um so I'll.

A

Let her take over and in just a few minutes, um but before that, I just wanted to see if anyone had any other announcements or um parking lot items for after the demo to discuss in today's meeting.

A

Great well carolyn, I see you here so um feel free to take over uh uh final note that this meeting is recorded. So don't say anything you don't want on the internet forever um and with that uh caroline, it's all yours.

B

uh I'm carolyn van sike, I work on the davis labs team under the azure group with microsoft, and uh I've been working for a while on something called cloud native application, bundles uh which delightfully shortens the cnab, and specifically I've been working on the implementation of that so cnab's a specification uh there's a couple different implementations of that you may have heard of docker app is one uh and another.

B

One is called porter, and I wanted to show this to you today and it's kind of relevant to your group, because uh it understands how to work with air-gapped networks.

B

You able to see my screen.

B

Yep looks great okay, um so I I know I don't know how much time we have for the demo, but um I just want to explain at a really high level.

B

What's cnab and porter, I'm going to kind of use them a little synonymously they're, not quite the same, but uh what it lets you do and it takes everything that you need to deploy your application and puts it in a bundle and allows you to version it and be able to distribute it using uh oci registries docker registries and be able to bring it over air gaps, for example, and then be able to work with it using a single command.

B

So, even if inside your bundle, you may have many different tools, maybe you're using docker inside your bundle, maybe using helm, terraform, maybe cloud specific tools like you're, using a gce or you're using aws azure, or something like that you're using a whole bunch of different bash scripts.

B

um You're able to work with your applications deployment as a logical unit and be able to put all the logic for managing not only the deployment like the installation, but upgrade maybe discrete uh actions that you up. That you do on that deployment like doing a dump of logs or like database dumps things like that, um checking the status and the health of your system um prove it anything like it'll run any command. It'll do anything it's whatever you put inside of it.

B

It's just a packaging system with versioning and security, and things like that, um and it allows you to work with it as a single unit.

B

So what I wanted to show specifically was how to take a bundle that already exists and it references things on a network. You know references things in docker hub, for example, and then be able to move that bundle over an air gap and then be able to rehydrate it and then use it again.

B

So this is an example of a porter bundle. He uses yaml.

B

Maybe you're happy and if you're sad, and uh so this defines my bundle that I made just for y'all called whale gap and uh it sneaks a whale size bundle through an air gap and uh it's designed to be published, like I said, to an ocr registry.

B

uh So this is set up by default, to publish to docker hub here it goes to get porter whale gap and its version, so this is set to be 0 1.0 and I'm going to skip over some of this stuff and just kind of go to the meat of what is this bundle do?

B

This bundle has because, like bottles by itself, just packaging unit, so what's inside of it inside of it, I've put helm, um you can use anything inside your bundle, you don't have to use it's not tied to kubernetes whatsoever, um but I decided to use home and uh it's heavily templated.

B

um So I'm deploying a chart that I made called whale gap and my chart's right here on my file system, and it has a couple parameters that I'm passing in through my bundle itself and that's something that's kind of built into cnn is the ability to pass to the bundle as like as a user. I could pass a parameter to it and then it can make it all the way through into the bundle.

B

So I can alter what it's doing when it's doing the installation- and the other thing you know is helm, needs a cube, config file, it needs a credential. So that's the other thing I've done. Is I've defined that my bundle is going to need when it's installed a cube, config file, and it says where, where it's going to uh find it so add installation it's going to install this wheel gap thing and then it also knows how to upgrade wheel gap and knows how to uninstall it.

B

I could have had a whole bunch more, but we're trying to keep it really small for the demo here.

C

um A quick, dumb question, of course,.

A

No dumb questions, uh so when you say helm is it assuming that the helm binary is on the disk similar to um the the cube config file and cube control? Is there or do you bundle it with it?.

B

Yeah, so that was the next thing is so porter and bundles itself. Bundles, bring everything you need in order to install your application inside of itself um right now, like a bundle, can have multiple different runtimes, um but at the moment it's just docker containers, so a bundle has um two components to it. It has a definition of what the bundle is.

B

So I said, there's yaml, but it actually comes down to this. This is cnab right now, like you saw porter before this is cena, and this is the definition of what the bundle is, but the other part of what makes a bundle is the docker file. The invocation image essentially, is the installer that goes with it. It's a considered utility docker container that has helm. It has cube ctl.

B

It has certificates that you may need scripts. The charts file, like all my uh files, anything else that I may have needed to do my installation all get shopped into this uh container here.

B

So this is a generated docker file. Porter generates it part of cnav generation is not that's why I met like cnab's one thing: porter's another, but uh so porter generated this docker file for you just based on the fact that you said mix and helm boom.

B

If you'd wanted to like make an artisanal docker file like more power to you, but so it made it for you and installed helm, and you were to get it. You could have told it which version you wanted.

B

It can auto, detect what you're using and get one for you um and also put on cube ctl to uh it, needs it just for a couple extra things, um and it will also copy all the files that are on your in your directory, so it automatically brought in the charts that you had so that helped answer your question.

A

Yeah, that's perfect thanks, yeah, okay,.

B

So the idea with bundles in general is that literally everything you need to run. It is right there. So as long as you have a cnab tool, so it could be. There's a couple out there, like I said, docker app, there's porter, there's um duffel.

B

um They all know how to run these they're. They work because there's a spec. They work with all of them. As long as you have one of those tools on your file system and you have the runtime in this case, it's docker, um you have everything you need to run a bundle, no matter what's inside, that bundle.

D

So I have a quick question, so the mix-ins here so helm is a built-in mix in for a porter. Then.

B

Yeah it is, anyone can make a mixon. um You know, we've made a bunch that are like uh obvious like we could have. Kubernetes we could have um docker compose is a new one. um Is it terraform.

D

B

uh We don't have ansible, yet that'd, be a cool one to have um that's that no one can make them. It's just a binary that understands how to talk on standard and standard out.

E

Actually there is an ansible mix-in that somebody else has contributed. Sorry there is an ansible mix-in that someone else has contributed. It's just not in the uh standard location. Oh.

B

Really do we have it listed in our little search, yet I'm.

E

Quite sure we don't, but uh I know where it is and we'll wired it up.

B

Add that, for me, ralph that'd be great. You betcha yeah yeah, so anyone can make one um there's no such thing as like an official one or an unofficial one. um If it. If it's on your file system, it'll work.

D

Basically, the mix in, though, is a kind of a client-side or build time plug-in then, and then, at the run time inside the docker image it gets generated, then you don't have to worry about the mixing anymore, because it's all built in it kind of sucks it in. I guess, there's.

B

Two levels of mixing: actually um here's two binaries for every mixin there's a client side which understands, for example, darwin linux, mac, and it understands how to do things like injecting these lines into your docker file, because it knows like this is what it takes to get helm inside of a docker file.

B

um But then, when you're actually executing this install part, this is happening at runtime and you still need that helm. Binary. uh The mix in binary.

F

B

Acts as an adapter between understanding cnab, uh because there's there's some templating going on there's some environment variables, there's things that are specific to the specification going on inside of this uh this docker container, and then it then handles making calls navy to a command line tool potentially to an end point like. Maybe it's talking directly to a cloud provider or it could be doing something just as like. I've made a mix in that just prints things to the terminal uh like to the console.

B

um It could do anything actually um so there's always two pieces to it. uh A client side like a build time at a runtime got it.

B

And that's that's a porter specific thing, um not a cnab thing. It just kind of helps. You make a uh a bundle quickly using existing tools out there. Otherwise, uh you could. You could make a bundle just by writing a like a really big bash script, but the nice thing about porter, like the reason why we did it this way.

B

Is it gives you a lot of inspectable metadata that you could use to enforce or do quick, uh find, replace and and kind of set policy, and things like that, um it's more fun to work with this way, instead of having to like uh reinvent the wheel over and over again, it has a lot of built-in air handling and kind of smarts understanding uh how bundles work and more desired state configuration, as opposed to maybe uh commands that it's fine to fail versus if I'm gonna mash upgrade over and over and over again until it works.

B

Maybe you want it to work a little bit differently inside a bundle versus as an imperative tool.

D

So sean was asking a good question, so do the the mix-ins for install and upgrade need to be the same kind same technology.

B

D

The mix-ins for install and upgrade need to be the same. So do you have to use helm for both of them in this case, because you could use kubernetes for the like delete or something or sure, yeah, okay,.

B

You have full control over everything that happens under install or upgrade. um Porter has no opinion. um It's porter at its heart is like a workflow execution engine um and it'll just do whatever is in the yaml.

B

um It has a couple like linters to make sure that you don't really shoot yourself in the foot, but um it doesn't understand what helm is, for example,.

D

It doesn't use kubernetes right, so these verbs like install and upgrade those are those are porter specific verbs. Those are those.

B

Are specific to cnab actually cnap understands what install upgrade and uninstall is. You can also define your own custom ones like you could make one called status.

B

If I had my indenting right, you can make one called status and you could do like a helm command here um or you could just you could just go into bash for example, and start doing your own things.

D

So if I wanted to create a backup operator or something like that, I could create.

B

Yeah, you could do your own things from here uh and then you could then just call these commands yourself. um The the idea behind cnav is it's extremely flexible? It's whatever you need to be able to do. There's very little, there's absolutely no opinion about what you have to do or understanding about what any of the tools are inside of your bundle.

D

B

So I want to talk a little bit about the air gap part about this, because one thing I'm doing here that you don't always see with uh helm, is uh I'm passing in some extra parameters, some set parameters here, I'm giving it very specifically um what repository and what digest to use. uh When I do my home install and that's for a very particular reason. When I move my uh bundle across the air gap, I won't have access to docker hub anymore.

B

I'm going to have to be able to install by pointing to a repository a registry link, that's on the other side of the air gap. So I need to be able to swap that out.

B

um So when you're writing like helm, has one way of doing this depending on what you're, using like some other technology, you need some other way to do this little switcheroo.

B

um So that's how I'm handling this with home cnab has built in the concept, though, that for images you need to be able to do this, switcheroo I'll, be able to copy images from one registry to another, and it has this baked in called image relocation mapping.

B

So in the images section I can give like a tag to my image, and uh so my image is called will say d here and I just say it's a docker image and it's originally coming from carolyn vs, wales id and here's the digest and I'm using digests, because I don't trust tags tags can be forced pushed and from one day to the next right I could be getting different content. I don't want that. So.

D

Can you also, can you also add the tag for like documentation purposes.

B

I could use the tag um like as a comment. I guess um porter supports just putting a tag in and will resolve the digest for you, um but at the spec level it just supports the digest um yeah.

B

So what this lets me do is that when I put this in here um when I make the bundle, it will put this image, it'll grab it from the the registry and it'll actually put it inside the bundle too, so the whole thing's portable. So let's take a look and see what that that actually looks like. So when I run porter, I have this command archive and I want to make there's different ways to work with bundles. I'm showing you a way called: stick bundle.

B

This isn't really the normal way, but this is the air gap way, so I'm going to make whale gap and when you archive it it just makes uh a gza file for you and I'm gonna get it from the registry where this is pushed. So it's that get porter wheel gap, zero one! That's that's! Zero! Okay, um I'm not gonna run this command, because this is a demo and it actually takes a little bit of time because it's 500 megabytes. um So it's right here actually well gap, tgz!

B

Okay, so let me copy this over somewhere. Let's move this over to temp real, quick and let's just take a look at this and see what it looks like.

B

Oh, let's make my direction first.

C

B

B

Let's see what gets put inside of this thick bundle so first, this is the cena part. It's the definition of the bundle. Okay and again, this is just the translation of everything that was in that yaml file to the standard, uh spec format and has everything about it. Like the credentials all the parameters, the description, and it also has a listing of what images are used inside of this bundle like this is what we were just looking at a minute ago.

B

The other piece that's in this uh zip follow is a listing of all the different uh manifests. Sorry of all the different images that are contained here, and we have two different images: uh one is the invocation image or the installer for the bundle. Okay, and this is something that's always present, um with any bundle and, as you can see, it originally came from uh docker hub right. This is where I had published it from and when I archived it, that's where I pulled it from, but the other thing it has is remember.

B

I had that images section inside my bundle so because of that, when I archived it, uh it also pulled down the image. I was using inside my bundle so that I'd be able to bring it over the air gap. So it's okay, I'm you're using carolyn, vs whales id in this specific digest.

B

So then in these blobs are actually all the layers of all the uh docker images I'm using, and this lets me move it across the air gap and then republish it somewhere else to another registry.

D

Is there a way to mirror a a bundle to from one registry to another, rather than having to copy it to tgz and it as an animator intermediary.

B

Yeah yeah um there's a copy command, so I could have done porter copy and then given it, two different tags and it'll, actually just shuffle them over. So I could rename it from one to the other. So I do up here. um I can give it a source and a destination. Is that what you're asking about yep, okay, yeah and it'll just rename it and bring everything over and everything it's referencing from one to the other.

D

So I'm trying to figure out how do you so one of the problems? One of the things I was going to demo next week is something called scopia which were yeah all lamenting on the uh portability of digests.

D

So how do you so you you? Basically what you said earlier was the digest is a parameter. The the the digest was primarized, and so is the image name, and so I'm assuming that you either you must be updating those, because the digest is changing.

B

D

B

Why we're injecting both yeah.

G

Okay, yeah can, I just add something real, quick.

F

D

This conversation.

G

Technically, we rely on the fact that, uh between on the same registry, the content digest should not change if you push the same content, but we do have the option for the user to accept the fact that, if the die just changes, the actual digest of the whole bundle is going to change and essentially is going to be a different content. If, if something changed in your bundle, so um it's essentially uh by default.

G

If any digest is different, this is gonna error out and tell you that something is different in the bundle you can choose to override that and it will generate a different bundle. Does that answer the question yeah? I think so.

G

The dldr is that you can copy bundles across any compliant registry, and if the content digest is the same for every single image, in your bundle, the content digest of the bundle is going to stay. The same.

F

So do you require then schema 2, v2 images. I assume.

F

Could you repeat that sorry, sorry, you require them schema 2, v2 images like the the docker images to make sure that, like when I'm pushing between registries, that the shop doesn't change.

B

I'm in trouble with v1- I just tried that yesterday and it wasn't working.

F

That might be why.

B

F

Had to make a new image.

B

It was v2 in order to get it to work. Okay,.

F

Yeah no, I was just wondering if that was a constraint. That's all.

B

Yeah, I don't know if it was an intentional constraint, but it doesn't work.

B

Yeah because I found a wonderful image that used whales, it was like four years old and I'm like perfect I'll use it, and then it didn't work. So I had to make a new one to get my wheels on.

D

So is this uh multi-architecture enabled so can I have fat manifest for the porter images and have different architecture groups for that or do they have to be tagged independently.

B

Trying to think about that, I know that porter right now is specific to one architecture, because we've had no asks for anything other than one architecture. Radu do you know.

G

Up when pushing the bundle to a registry uh architecture is taken into account, so it's generated uh in in the bundle you can specify an index that points to a multi-arch image. I'm not exactly sure if at front time, that's hooked up yet, but technically it should support it.

B

D

Yeah, so basically, if you're going to do a copy, then you're going to probably just you're going to have to do a copy multiple times for each architecture and it probably doesn't do a recur. You know full tree copy of all the architectures, I'm imagining that's kind of the default behavior of a lot of tools.

D

I'm just curious. I don't have to answer right now.

E

Which is the default behavior you're, the full copy tr for multi-arch, or just one almost.

D

Yeah most tools ignore they like. If you look for you, you get a fat manifest back or manifest index it'll go ahead and grab the architecture of your docker daemon and it'll just grab that one and move that across specifically.

G

The tool that pushes a bundle to an oci registry is multi-architecture enabled, so it will copy all architectures of an image across registries if they're present, but I'm not sure about the runtime bits. Okay, yeah.

B

That's the question is words kind of stumped on the runtime. um I have to maybe look it up and get back to you. It's not a question of like. Do we not want to support it? It's this question of like did it get coded yep yeah.

E

I would expect it to work uh properly on all our architectures in the future. Whether it does now or not, is a great question. We should test it.

E

I have another.

A

Semi-Unrelated question: do you preserve the original layers um of the images you're pulling in when you're generating the bundle itself as in um like? If I have, if I have you, know, 20 bundles and they all have the same exact like you know three images or you know whatever um at the bundle level?

A

Do I get that the merging of those layers or will it differentiate because they're pulled in separately? You know, I guess it's for me. It's. You know right now, our docker images when we pulled them off into like a tar ball to transfer air gaps, it ends up being to be terabytes because we don't achieve any. You know um uh we don't minify until we you know rehydrate, I guess, and so from a bundling perspective. If we can bundle, uh you know and we're achieving something similar.

A

I'm I worry about then now we're saving that same docker image 20 times, instead of once when we you know when we pull it over.

E

Are you you're talking about like so if two different images share a certain subset of layers? Are we getting multiple, multiple copies of those layers? Well, so well, I guess.

A

um uh Yeah, I guess well it'd be more yeah like if you're, if I'm taking. Let's say you know alpine into two different bundles um uh are and then the the bundle itself turns into an oci compliant. You know image or whatever. If I push that into uh you know, v2 registry um are the images or the layers from that alpine deduped um or because it's been bundled it gets.

A

You know it gets kind of messed up, and so they have different layers and therefore I don't achieve you know so that that alpine is essentially taking double space, because it's wrapped in a bundle.

B

So we we preserve the original layers. um If you take a look here um these, these are the layers that came out of the registry and we don't alter them.

D

Yeah since you're not putting them in subdirectories for each image, you're, naturally going to use the same cached layer right if it's already there, I assume yeah.

G

The assumption, though, is that, when you've archived the bundle you're ready to distribute it to the airgap environment, so you're, not as much for at least the current thinking is that the moment you archive it is prior to using it. If you want to move a large number of bundles between two different environments, then you might want to look into running a registry in your private environment and pushing.

A

Right so yeah and that's yeah, that's what I guess I'm asking so we do run the registry. So when I, when I carry all those bundles over and I pushed all the bundles to a registry, um do I get that that layer advantage.

G

Yeah, that's a good question when you push the bundle uh to an oci registry, we just copy images. So if the layers are there or if even complete images are already in the registry, we don't copy them anymore. Yeah, okay,.

A

So the image layers are at the same level as the bundle layers is. I guess the question yep.

D

I think the important thing technology wise you know layer layers are portable between docker registries. It's just the manifests.

A

The top manifest for them that are right, yeah. I guess I was just wondering if the the layers for the images were being like. You know, they're they're, embedded into the bundle until they're lost is, I guess what the question was. Yeah.

G

Yeah this this ties into how our bundle is represented in an oci registry which I'll will not get into very much detail, but it's an oci index ultimately and it points to multiple artifacts, gotcha yeah.

B

It's not something specific so, like I think, uh most or all of the layers already here. So if I try to do this, it should go pretty quick, because all the layers are there.

A

B

See, instead of taking like really long time, because I said it was 500 megabytes, um it went really quick because the layers were there, it recognized it and then only had to push like one thing.

E

So then, also from a holistic standpoint like from a higher level standpoint, it's really easy to realize that when you decide you're going to drag a whole bunch of layers across an air gap, the package is going to be big. You don't have a way to avoid that, so you you need to. Oh for sure, it's on the it's it's the transport layer.

E

That's that's the big thing and once you jump into the registry, you get the basic registry behavior that you see right so, hence, if you have a bastion server, you won't have that problem, because you can.

A

Just do a copy, yeah yeah.

E

A

I guess from a bundling perspective, you you're you're transferring a significant, more amount of data if two bundles share the same image, because they've been tarred separately right. Yes, you know yeah.

B

um If you don't mind, I wouldn't want to just complete the last little yeah.

A

B

I'm bringing this up uh how the kind of switcheroo works when we come across the area, um so I did the publish here and so previously I was on that other registry. I can't really do like a full like isolated network here, because I'm demoing to you online, um so I moved it to another registry. Okay and I have a kubernetes uh cluster here. um That's pointed to use uh this registry, and um so I'd like to do then is do a porter install and I'd like to give it uh my creds.

B

um It has a it's already pointed to use this credential as my cluster and then I want to install this bundle.

B

ah Wow, that's.

E

Really it was without saying that was going to happen.

B

Don't say that to me.

E

ah It's just the demo gods.

B

I'll make it.

B

Work, oh, I know what happens sorry about that. It really helps if you say that you're installing a tag.

E

B

Let's try that again: okay,.

E

Okay, see now now now that having messed it up and then putting the tags, it's making it obvious that it works.

B

Yeah, so this is what we want to highlight here is that we did the switcheroo okay, we uh told it what the new digest was when we did the helm install, and we also told it. This is the new repository we want to use um so that it's not pointing anymore to um docker hub, it's grabbing it off of the new registry on the other side of the air gap, and I just want to prove it to you. So let's take a look at the pod that it spun up.

B

Okay, so let's do it.

B

Let's take a look at this and we can see.

D

So when you public, when you published this porter image to your local repo, it injected the new digests as it discovered them that no never mind they're, they're, embedded.

B

Yeah they're embedded in that that archive right, yeah.

D

B

When I ran that publish command uh porter publish if we do a little bit scrolling here, um I ran quarter, publish and what it did is it went through every single one of those those blobs right. It went through the manifest and it went here are the images that are inside of this, this zip file, and then it published those to the registry on the other side. um So it grabbed it went through all the different layers reassembled. It made an image and pushed it all back up.

D

All right, so it's preserving the manifest then so it is preserving the digest. Then no, the.

C

Digests are specific to the registry. No, it.

G

If, if okay, if, if the registry is oci compliant, they should preserve the digests, and in this case it's, uh I think, it's azure container registry, which does preserve between docker hub and acr they're, both oci compliant. In this perspective, and most of the time all most registries strive to towards keeping the digest. So I think, in this case, they're actually preserved the image digests.

D

Okay, so anyway, the install process is what's actually pushing the images into that registry, so they could be consumed by uh the cluster, be pulled.

C

The payoff here yeah the publish.

E

Install just tells uh just tells helm to tell kubernetes to pull them.

D

I thought you just said: publish only published the porter image, but you're saying it's also publishing the embedded images yep. It's.

B

Everything that's used uh by your application, so it includes that application image.

D

They get translated to the same like name spaces and all that kind of whatever paths path. Components are in the image names, yep, okay,.

G

Yeah, that's where the relocation map that caroline was mentioning yeah. I got it. Okay,.

B

So, just to prove to you that it actually worked and we have a functioning whale gap on the other side. Let's come over here.

B

Take a look at it, and this is working.

A

B

um Yeah, it did rename it um so previously it was called my image that was running.

B

uh My my image was called um like caroline vs wheel gap, and so now it's get porter azure acr wheel gap, so it did rename it. It took like the registry swapped out the registry's names but then preserved the final path. Part.

D

Yeah, so basically, if someone's going to build a mix in their, your technology is going to have to have basically top-down awareness of all the image references. So it can do this proper translation, then. Likewise,.

F

D

Going to have to publish all of their of their images as values.

B

So the mix and didn't need to understand this. um The templating handled subbing this in um so the mixin just needed a way to accept the templated values.

B

uh So helm was pretty easy because it had set right and it just allowed me to to insert repository and digest um depending on what else it is you're trying to do it.

D

One thing you have to have both right, so your back end technology has to be built in such a way to accept those as parameters and the mixin has to be able to accept those parameters and pass them on correct.

B

Yeah, you need to be able the way to send these through all the way yep yeah.

B

It would be really great if mixins could understand what was being used and be able to help report this back to porter, so that you're not having to fill this out by hand. But uh you know with helm it's kind of like you know this information could have been anywhere, so you kind of have to type it in yourself, but.

D

But you, you kind of dictate the fact that the repository and the digest or the tag all have to be in independent fields. They can't just be one field and parsed out or anything like that.

B

um It's separate because of how the chart was written okay. um So if I take a look at my chart, real quick and I go to my templates deployment here.

D

They're trying to figure out where they relate where, where the magic is for the automatic, where the api.

B

Yeah, this wasn't magic, it was just it just had to do with like this was how the chart was written. It had it split into a repository and it digests if it was a single value that I could have put in like. If I go look at the values here. If it had been just a single thing, then I could have done a single thing, but most charts have it done this way instead, so you can easily swap out just the version used. You know just the tag.

B

um Honestly, most charts are written by tag instead of digest too, which is problematic, but.

D

Yeah I've yeah we've just gone down an entire rewrite of all of our software. To not do that anymore. We're just doing an image. Ref yeah.

A

So that that actually was one of my questions, do you have to use digest in this? In this regard? Like do we have to basically go back and change all of our our things to use the tags, or will it preserve the tags um you you could have.

B

A

A

B

Could have done that and porter would have figured it out for you. You just lose the extra assurance.

A

Right so it doesn't.

B

It may have been force.

A

Pushed right, it doesn't take the uh the tags with the manifest when you grab a digest.

B

um So what what porter will do here is it will at build time when you build the bundle it will resolve this tag to the digest and then when it builds the bundle, because everything turns into a bundle at the end of the day, um it has to be resolved to a content digest so it'll make it an immutable digest for you right.

A

Right yeah, so it's easy to get. This is the tag right, but when it's so you so when I so when I, when I rehydrate this in my air gap, environment is what does it? Does it add the tag to the upstream registry or not? No, okay. So then my helm chart would fail because it would need it would need to use a digest instead of a tag.

B

The whole chart wouldn't fail because um it still would have this tag. Oh yeah, I see what you're saying.

G

That's a really good point and it's a rather easy fix on when we're pushing the bundle. Yeah.

D

You have to tag it with something, though, don't you otherwise they're both summer registries will just automatically delete it. If they're untagged.

G

Well, it is a reference to an oci index after you pushed it.

E

But it's not uh not not of the tag form that the original repository had it and so the the real. So this is a great uh thing to point out uh sort of at the stage we're at right now. The expectation is that, if you're pulling something from online across an air gap that you're doing that with some desire to have confidence, it's precisely the same artifact, and so the original implementation is based on digest.

E

Because that's the only way we can give you any kind of verification or validation that you're really installing precisely the same thing that you had in an online environment um and tags, just don't give you that help they're. Just they just don't give you enough confidence. But, alternatively, you know you could uh we could?

E

Actually you know you could re-tag them, um but we're expecting that it's, or at least at least the in the current situation, we're sort of expecting that the bundle owner would actually understand that, because you have to sort of build it a certain way to make it a thick bundle so that the bundle owner would understand that modifying the chart inside the bundle to accept digest. Would probably be part of their workload and.

A

Their work club, where my concern is is um um this is a huge issue. We've had in general is many. Many operators will hard code tags into go code um that, especially if they're helpers, like you know, I need busybox 1.2.8, all right 1.28 and it's hardcoded, because that's the helper and who doesn't have busybox, and so they don't allow you override that you can override a lot of the other ones. But if that tagged, you know image doesn't exist, it doesn't have it's like you know.

A

Oh, I need to do a bash script or whatever, and so that's an issue. We've we've ran into a lot. They actually had to open up a bunch of pr's, but it's just you know it's something that it's a bunch of extra work. If you yeah no.

E

That sounds horrible and if you have individual people that I need to go beat up to do the right thing like that sounds just awful: do they usually give you the ability.

G

To change the registry, but not the tag or digest.

A

But we've even well often it'll just say you know busybox with no registry, and it assumes that it resolves and we've solved that using you know we use container d and we do mirrors so, like you know, docker.io and whatever goes to our registry, so yeah handle it. That way, and we also you know, turned off force pushing on our registries. You know in in our in our environment, so we have some resemblance of you know it hasn't been forced pushed but yeah.

A

It's it's not perfect, but it's just you know it's more of a I'm. Just trying to understand like we can't adopt this until we we basically go all in with digest is what it sounds like.

G

That's a really good point yeah and it just we've, never actually had the requirements for a digest for the pushed images for the relocated images until now. So it's a rather easy fix on on our side when pushing, I think.

B

It's a large change honestly to add the tag.

G

Now and an earlier version of of the pushing functionality actually tagged images we just didn't port it over because it no one actually needed it until now. So thanks for pointing that out, yeah.

F

um I have a quick question. um I think you mentioned something when you showed the bundle json uh that that was. Is that the that's the actual spec, that cnab conforms to and porter creates? That is that, is that I'm understanding. So when you said that at the spec level, it only supports digest. That's what you that's. What we're kind of talking about here is that yeah.

A

You could do a bunch of stuff.

F

With tags, but the speck in this bundle, json doesn't have any ability to do anything with tags. Is that okay, yeah.

B

Thank you, yeah porter porter has a tag there and then it resolves the content items for you and then that's what it's using to fill this in, um but because we don't have this.

B

This is what we want to add.

F

Yeah- and so this probably means that, if I'm using a different implementation, I think you said like duffel or one of the other ones, that I could only expect to use digest right, like okay cool, it's.

G

The only way of making sure you're running the same content.

F

G

C

E

Stop trying to auto complete for me there we go yeah um super great conversation, because that's it. It's very interesting to hear the problems that you run into such that we can think about how to handle them.

B

That's that's all I had to show by the way. I know we didn't really touch much on like how cnab works in general, but I hope uh we filled in enough of the blank. So you see how the air gap scenario.

C

A

I mean I think this has been awesome and I really appreciate it. Do you I mean, could do you know enough to give a quick rundown like porter versus the competition like you know what you know, how do you, what what did you guys? You know? What's yours, you know um interpretation of the the spec versus others and and how there's differences. Sure.

B

um So, like I said, there's there's three uh public uh implementations right now, um there's duffel, which is the it's the reference implementation of the specification.

B

um So that means that it's, it's not really it's more intended to vet the spec and not really intended to be like uh it's not really actively developed I'll put it that way rato. You could check me if I'm wrong um people work on it when we're changing the spec. You know what I mean and then we'll make changes and make sure that it supports the spec. But beyond that there is an ongoing development on it um and the way duffle works.

B

Is you edit, the docker file yourself, so whatever you need inside of that installer invocation image you put in there um and then you're, given a bash script, a run script and whatever you put in there, that's what it'll do uh when it comes to running, install upgrade or uninstall or custom action, um but there isn't any um like auto magic or like building blocks like porter or docker app.

B

um But at the same time like there's, since there's no opinion you can do whatever you want. um The other one is docker app, which is very similar to dr compose. Basically, um I haven't used it terribly much, but that's the way I think about it. It's more focused on the deployment of services that were already defined using docker compose and then so it conforms to the spec and everything, but deploying um infrastructure isn't really it's not really focused on that.

B

So deploying, um for example, like a say, like an s3 bucket, uh isn't really what it's intended for. I'm not even sure if you could do that to be honest with docker out. Someone fact check me there right ralph, that's not what it's not really good one of its strengths right awesome!

B

um So then, then, there's porter and porter is really built around being able to reuse um and adapt existing devops tools and existing tools that we use to do everything, and it is it's essentially like I said, a workflow execution engine to be able to plug them all together and adapt them so that you can pass inputs like parameters and credentials um and outputs between them. So if one step with an install created a cube config, you can pass it along to the next one, or maybe it made a database connection string.

B

It's basically allows you to pass information between them and kind of string together a series of steps between all your different tools that probably weren't meant to work together and hopefully make something a little bit easier to manage and deal with air conditions edge cases and make something a little more robust than a monolithic, bash script, um and that's kind of the intent is that, instead of making something that replaces or is a giant wrapper script around like column and terraform and qbctl, it gives you a little bit more easier ways to work with them.

B

I think you said this.

D

B

Actively developed because I get paid to work on it.

D

So I think you said that this at the beginning. I want to make sure so I think the porter executable the client tool that does use docker directly right, so you have to have docker installed because it interacts with that.

D

B

Needs to be able to connect to a docker uh connection either, so it cannot anything is a local docker socket um or it needs to connect to a remote one. So, for example, we run porter in azure cloud shell, um and at that point it's a remote docker engine that we connect to okay,.

A

And, and is it have to be docker? Is it any it's a what I guess would podman work is the question.

B

It just needs to be oci. Sorry I say docker when I should stay. I just.

A

I just want to clarify, um and so I guess my next question is: if no, I don't want to take everyone's time. So does anyone have questions because I've asked a lot.

A

Okay, cool, um so is there any concept of like um so well? So when you, when you deploy a bundle, um does it um does it have any like chaining or is there any dependency management with it of um so like one of our our, we wrote an internal tool right because we essentially need istio installed pretty early on, because it has a mutating web point and if you don't have the mutated web hook before you install things, nothing is mutated. um Does bundle bundles, have any kind of concept of that or not really yeah.

B

Yeah, um so we have the only one that implements it is porter. At the moment um we have a beginning of a spec for dependencies in the cnam, spec um and so porter lets you specify one level deep who you're depending on so like my canonical example, is I have a wordpress bundle and I depend on my my sql bundle.

B

For example, uh it will run my sql bundle first and then pass outputs up into the wordpress bundle, because it needs that connection string, you know um and then it will then use that and then tie them together and then you can work with them uh as a unit, um but we're trying to bring that to more mature level.

B

um So I'd say the next over the summer we're going to be working to really bring uh more scenarios to the dependency spec being able to have more level, I'm able to order your dependencies and be able to satisfy dependencies from something that we say installed weeks ago uh and things like that, um there's a little bit of work. We need to do that's happening right now to be able to enable these scenarios. But yes, sorry long answer. Yes,.

B

If anyone's interested by the way, porter is looking for contributors or just people who have ideas like everything you're seeing right now, it's super exciting for me and if you want to hop on our slack we're going to ncf slack um or we just want to like open an issue and drop ideas or leave comments, um we really want feedback or I don't know anything. Any engagement you're interested in uh would be amazing because I don't know you're you're doing the things that we're really interested in learning more about.

D

Yeah, so just one more time, I have to ask a dumb question because it may be missed at the beginning. The relationship of porter to cnab is the same team as he said, you're working on two independently or different people completely.

B

um So my team helped start found something pick a verb, um the cnap specification, uh but it was collaborative between a whole bunch of different companies um and a number of people on my team are maintainers for the cnav specification myself included uh and then I'm the found creator of porter.

B

um So okay, you're, just really into c now and making bundles porter just exists to be an implementation of cnab. Essentially.

D

Okay, except I wrote my competing version of this, but I had to do it at the same time. Cnab came out, though it wasn't quite ready for the scenarios we had. So it's interesting because you and I have taken different angles and so mine's mine's called the case specification. Call it application uh call whatever it. Matt stands for container applications for enterprises anyway, so it's out there on git too, but it has a whole different perspective on how um what what the opinionated parts and the unopinionated unopinionated parts are um so yeah. It's interesting.

D

It's like an 80 overlap. Maybe let's take a look at this more closely again.

B

Yeah, I I think this is an area that really does need a lot more innovation and people just using it, trying it in different environments in production and seeing what really works when, like you meet reality, if that makes sense- um and it's definitely like this is v1, um I think v2 will look very different.

G

Speaking of things, people should be using. I just want to point out that there's also a security spec that we're actively working on so, if you're interested in that part, essentially, we integrate with the tough and in total upstream specifications and we're working on having working implementations for both of these. So just if you're interested in this feel free to ping us in the cnap, channel or tough or in total channels for the.

E

Purposes of air gap roddy, you should probably describe just a high level set of features that come even to air gap, environments.

G

I'm not exactly sure whether we have the time for that or not that's why I just wanted to briefly point it out if we do have the time.

E

I'm happy to basically, you can do supply chain signing and then bring that metadata along with you, and if you have the right uh access to the original metadata server, you can. You can validate the supply chain and arbitrary.

G

Yeah essentially, there's two parts of it: one which is signing the bundle itself using a notary service just like just like. You would sign a regular container image, we're just using note 3 in the same way as docker content, trust, so you're signing a bundle in the same way and then the second part, which is, if you have ins. If you have a supply chain layout using intoto, you can bring that up into the note 3 metadata as well, so you can have both of them specifically for air gap.

G

You still need to have access to the original notary instance where you push the signature, but that's something that's going to change with note, 3 v2, if you're interested in that, I'm very happy to chat trishank, and I was in the call as well.

G

A

You thank you for the summer.

A

Great, does anyone have anything else before we close out.

A

Awesome well, thank you very much carolyn and everyone else from microsoft that chimed in. We really appreciate it. This is interesting um and uh chris, are you still uh able to demo next week or yeah? They can scope. You will.

D

Be much easier, it's really small compared to this. So um if you have other topics, I would.

A

Definitely pile them on okay sounds good. Maybe we could have a follow-up discussion now that people can play around with porter and see what people think great thanks. Everyone have a great weekend.

C

C