VMware TGI Kubernetes, 11 Feb 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: TGI Kubernetes 185: Cartographer

Description

00:36 Welcome to TGIK
01:35 Introductions
04:58 Week in review
12:42 Why Cartographer?
32:37 Demo: basic supply chain
01:12:28 Wrapping up

Join David Espejo, Rasheed Abdul-Aziz and Waciuma Wanjohi who will kindly introduce the audience to the Supply Chain choreography pattern and how it's implemented in the Cartographer (https://cartographer.sh) open source project.
See you there!

A

All right seems like we're live well, hello. Everyone welcome to tgi gate episode. 185 if you are just joining us, feel free to say hi in the chat, it's a bit of tradition to kind of say where you're signing him from so if you feel comfortable doing so, please do it. We usually have a pretty big audience from all over the world, so good to see you all.

A

Hey john from minnesota, welcome well friend, welcome cora. Also one of of our favorite developer advocates out there welcome from new york city, okay, that's great cool! um Well, I'm dave espejo espejo, if you're wondering why my twitter handle and github user etc, david mirror is because my last name is pejo.

A

It's spanish for mirror, so I'm kind of david mirror also so I'm davis peco your host for this episode, I'm an open source community manager at vmware, where I have the chance to support open source projects and other initiatives, and today I'm joined by two of my wonderful colleagues, rashid and washuma, were maintainers for the cartographer project. Welcome guys, would you like to introduce yourself please.

B

uh Sure uh rasheed abdullah a maintainer engineer on cartographer and when I get to my part of the show I'll tell you a little bit more about myself uh over to yoshima.

C

Yeah hi, I'm wishuma wanjohi, I'm also a maintainer on cartographer. um I'm looking forward to today tgik a great time to learn new things, and I just learned something new david. I had no idea, that's what your last name meant, uh nor why your github handle was the way it was so uh I wan. I know I need to learn more spanish. Now.

A

Cool got it: okay, that's great uh yeah, hello, pete from minnesota, olaf from denmark, that's great and sam, hey from new york city, uh also, scottsdale, arizona.

A

Welcome awesome uh for folks who are just joining dji cakes is a weekly screencast, where we aim to learn together by playing around with technologies and tools and the cloud-native ecosystem, formerly host by some big names in the kubernetes community.

A

I'm not aware I'm trying to fill up big shoes by being a host today, nine posters, intramural winky winky, so we'll see how it goes right. um Remember that let me post this here uh remember the show notes are in the hacking dealing that I just share in the chat and the goal is to crowd outsource news of the week, so feel free to add news and links to the notes.

A

What we do after the episode is to add minutes and check the notes to github to the djik repo, so everyone can go back and refer to the episodes archives. So without further ado, let's get to the news. Let me share real quick here.

A

Right share my screen all right. I hope you can see my screen without issues. That's correct.

C

Yes, yeah cool.

A

Thank you yeah, I'm kind of new to streamyard, so I had to ask cool. We can review core kubernetes. uh The secret release team is working towards the 1.24 release schedule for april 19th and um last week. Last friday.

A

The enhancement freeze is in effect, since last friday. uh One of the interesting things to notice is that 25 enhancements were removed from milestone.

A

You can track that here in the enhancement sheet and yeah majority of the enhancements removed from milestone are under the sick, node and some others from the six storage scope um yeah. I was curious investigating, for example, these two.

C

A

That are similar. You know: providers trying to move in three uh storage drivers to the csi spec uh why they were removed from milestone if their production readiness review was approved. Well, in both cases where the authors themselves pushing for next release for 1.25, we have some others that also didn't meet the pr process. Remember that this is a and yeah process established by the seek release team to ensure that a new enhancement new feature is safe to operate in production.

A

Okay, there's a new blog post. This one is interesting because it's spotlight into seek multi-cluster.

A

uh It's kind of the transcript of the conversation uh with one of the members of this special interest group and all the kind of the origins of the group and priorities scope and all that. So it's interesting.

A

Great um this one is one of my favorites. uh The kubernetes community office hours will be next wednesday february 16th. uh This is a uh you know. Monthly live stream when, where you can, you know, take questions and of experts from the community who will most likely answer the question or will help you after the fact to find an answer so um yeah if you are just getting started with kubernetes or if you have pressing questions, and you would like to have an expert to give you some clues.

A

This is a good space for that they don't have access to your clusters. So live debugging is not allowed, but it's it's useful. Okay um for the cloud native echo system testify sec. A use-based company turn the witness written to public last week. What what's witness it's a really interesting feature?

A

It's well, it's not a feature, it's kind of a framework or attestation system or engine to verify the interior of the build process from source to target.

A

It has some interesting features here, um definitely worthy to take a look. It was released like last week, cool uh cluster api 1.1 released. uh It has like 24 new features, I'm not a cluster api guy myself, but it definitely looks like a awesome release with a ton of enhancements and new features. So if you're, a closer api user check it out, please cool um the continuous delivery foundation, which is another foundation under the linux foundation umbrella.

A

They will have the conference in real life in austin it's the first week of june. If I remember right and the cfp side is down, but the the counter, the call for proposals for papers is open. It will be open until next friday um february 18. So if you are planning to attend or to submit and anything around, these topics feel free to do it before the final deadline.

A

And if you want to submit something around cartographer, just let us know we can help cool um this. One is really interesting, at least for me, if you want to break into open source uh and don't know where to start turns out that the cncf has been participating in lfx, it's a program where different, open source projects establish ideas or opportunities for for mentees to participate in.

A

So, um if you, for example, if you have golang or kubernetes or cli skills, you could help the litmus scales project to develop new commands or features for the ctl cli, along with integration tests. For it. It's it's. You know there are many opportunities there, 42 to be precise and the deadline to um know to apply for them is again february. 18 next friday, so feel free to check it out.

A

And finally, the technical oversight committee from cncf approved the archiving of the open tracing project.

A

uh Where are you my friend there you go well, it's you know, archiving open source projects, it's not a day-to-day task, but it's the sign of a healthy environment and in this case it was because the open tracing and open sentences projects merged into open telemetry, and it was the original intention of the maintenance to archive open tracing.

A

So it was approved.

A

So it will be moved to the list of archive projects.

A

Right, um if you feel that something's missing here, if you have some news anything to share just feel free to add it there, as I mentioned it, will be checked into the github repo, so it will leave there right, okay! So um without further ado, I will move to uh the intro to cartographer by my good friend, rashid.

A

Let me stop sharing my screen.

B

Thank you david. I will start sharing mine.

A

Meanwhile, well we have alex from northern california yeah too warm to sunny uh martin from netherlands um and yeah rashid virginia welcome. I.

B

Know that guy all right, hi folks, so I did say I'd- introduce myself a little more um and so in introducing you to what cartographer is.

B

I thought I'd take the tack of why I'm excited for working on cartographer and so around. 42 years ago I wrote my first computer program on an apple 2 and maybe 10 years after that about 32 years ago. I did my first professional work as a software engineer and I don't believe in the appeal to authority, but I do have some cautionary tales from years of experience in software engineering.

B

And I think I think, we're all aware of the not built here syndrome, all right, there's a specific one, and that is uh engineering um everything because you must have full control and your software project, whatever it is, is, is a snowflake that cannot be uh contained by traditional practices.

B

These are these. Are um these are stories I've seen and I've been a perpetrator of.

B

I can't use that language uh I'll make up some arguments for why you should not use this language or this platform, and it will be because I really like this other one- or this is something that only we can. We can only do on this other one.

B

These are not uncommon stories and in my time, in my time as an engineer, I've worked for small startups trying to produce something quickly, get it out the door work for small startups, who have to work with enterprises and deal with significant bureaucracy in the enterprise sometimes necessary, sometimes not, um and then there's also the significant bureaucracy that occurs in your startup or your small business.

B

Quite often because it's assumed that you need to adopt uh the at-scale solutions that large businesses use to be. As successful as them, so I see a lot of those sorts of things happen and classically it would be. um It would be about needing metal servers being able to run your uh your product.

B

Well, actually, I'm jumping the gun a little bit there. We used to distribute everything on cd and disk right, so you wouldn't be able to run your product. Your user would be running your product. One huge value that came up with the advent of the internet and internet distributed services.

B

Applications was that you could actually have much better control over versions that your customers were using because they're connecting to your applications online now using online services and as a major jump forward, I mean it really was, and uh it just meant that you were maintaining uh not multiple versions and historical versions.

B

Maybe 10 years old of your software, you were maintaining the one that everyone could use, perhaps a couple of previous versions uh because you're using web services uh as your way of distributing uh your product, uh be it a website or a business service that people could connect to or both monolithic, microservices.

A

B

Matter the thing is: is that when you were producing those services and you're trying to distribute them or host them at the time, certainly, let's say going back, maybe 15 years.

B

At this point I was still personally working with metal a lot so working with physical servers installing software installing your operating system installing services to back your applications, databases, you were dealing with dns networking email you had to configure sendmail so that you could distribute email as part of your web, app as a great example of just a horrible side, job that you would have to work out and maintain and and then, if your business grew.

B

So typically, if you were in a small business, then the developers had to worry about those things as much as they did the actual app. They were. Writing it's hard to focus on the app you were developing, because a lot of your time was spent doing infrastructure and uh as those companies grew, you would start to separate, and my experience was with a um I hired this wonderful guy, this guy john jonathan hoskin, who came to the company.

B

I was working at and started building out our infrastructure, our physical infrastructure, so that we could deploy our applications and we started to have the same struggle. A lot of people had, which was the developers were building applications that worked on their machine but did not work when we handed it to jonathan or whoever. That person was you've all got some ops guy at some point who needed to deal with developers, configuration and get it working, and there would be things like apache configurations or iis configurations, because you didn't have your own web server.

B

There was a web server that was running the company's uh applications online and, and you would sit behind that, and so there would be certain configurations that apache would have to match to work just the way that it did on your machine as a developer and so jonathan being a smart fellow. He was realized that he could use virtualization and he actually had uh um he had.

B

You know a couple of blade servers already set up where we were able to actually deliver him uh operating system configurations which were usually done with like chef or puppet or ansible, or those style of things, but directly on the semi-metal, so the virtualized metal of those virtual machines, so we're still setting up apache and we're still setting up a thin mail and whatever else we needed and making sure that networking was when firewalls were configured properly and then we'd finally install the application and make sure all the applications dependencies were installed and then- and it was around that time- this is before containerization.

B

In uh to my knowledge, anyway, um heroku uh came along with this thing, called the 12 factor app now, and it was a game changer for us. So I recommend taking a look at uh 12 factor app. I forget what the url is. Unfortunately, it's not going to tell me how to say there we go 12factor.net.

B

um This is a game changer this site. It's not always obvious that it, I guess in in the modern world uh how different this was and that's what I wanted to to point out. It's something so different uh to what we were doing at the time. This is about developing an app and other technology. Technological advancements came along at the same time that made this incredibly easy to do. First of all, was the twisted framework in python and then a lot of other languages following suit.

B

Basically, your application and the framework that you're writing against has its own web server and handles web requests, so joke apache out of the the loop immediately.

B

So that was really helpful because you were now actually talking directly to a web app and you didn't have to configure uh um like apache iis, that sort of thing and the other, I think, really valuable thing here and it's all valuable. I encourage people to have a look and understand why this was great, but um the other really really important thing for me was backing services. The idea that uh that your backing servers are attached resources.

B

It doesn't matter how you get them so long as they're the right resource um and in the case of heroku they actually gave you your backing services. You wanted my sequel, you got my sequel, you wanted! um I forget what else they had memcached, for example, right. It was as a service, and so those configurations were something you separated from your app design.

B

There's a lot more to it, but I think it's a really good start to think about how heroku push changed the world. So um this changed my world. What changed very quickly was instead of me worrying about all.

C

B

That was, um I would write my application rails. Quite often it was versioning at the time. I would write a rails web application.

B

um I would have a script to set up a local version of my database and I would design my application, based, especially with the support of rails, to be portable, truly portable, not portable, on like architectures.

B

We all knew that we're going to target linux, that's usually what we're going to do, but we made sure that if we had a linux machine handy, we could definitely run the application, and I would write my app and focus on writing my application and it was amazing to have all of that configuration lifted off of my shoulders and to be able to hand this application to anyone in the dev team them instantly be able to run it to the operators.

B

Let them instantly be able to run it and then deploy it, but they were deployed to heroku, um which didn't work for many reasons. We'll talk about why heroku is a bit of a limitation, but first I'll just talk about that experience.

B

You had your scm managed repository of code, you added, uh if you were using git, you added another remote which was heroku with your own account, the team account or you would just add, get if you're using a different scm, and you would push your app and heroku would look at your application and say well. This looks like it conforms to one of my build packs, which is um I I think you know the core technology here is a a bill.

B

Pack picks up your um application and because it's con constrained by the 12 vector app, it turns that into a disk image that can be deployed anywhere.

B

We didn't have access to that that built image, but heroku would host it for us um and there are other conventions that made sure we could attach to services. This might all be, very, I don't know commonplace for you all now, but uh that was amazing. It's a huge change.

B

Basically, how did I configure my app I configured my app by writing it the way I wanted it to be uh so long as it was constrained to the 12-factor app and I didn't have to set up configuration variables all over the place and special cases for different infrastructures that it might run on, because it was going to run in a virtual machine and so long as I could get it to run.

B

That way once I could continue to get it to run that way, and then containers come along and they've made that even simpler these days. So now we have that ability um with something that's very easy to move around. We don't need a virtual machine to run it on. We can drop it into docker and do a docker run, so that was the huge benefit of heroku um and then clown foundry came along because heroku has some downsides.

B

What are some of the downsides? It's publicly hosted. uh It lacks supports for air-capped environments, uh it's backed by private ip. It's not it's, not um it's not an infrastructure that a company can have full control over.

B

So it's tied to heroku's physical infrastructure and it's also quite forceful about its compliance with the 12 factor app as much as I talk about it. Not everyone wants a stateless service backed application. Some people want services on disk or they want to use lambda functions, or there are other ways that you might want to write your application.

B

This goes back to the not built here thing. Sometimes you actually do need to build something here. There is some sort of infrastructure or design that calls for more than what you have today.

B

So cloud foundry tried to address some of these things.

B

And I I hope some people have seen cloud foundry in action, but uh it goes from the git push to the cf push with a little manifest file, sometimes to set certain setup variables. Configuration variables that are are needed to get it started and running um cloud foundry also used build, packs and extended them. It made build packs available in air gapped environments so that you could pull dependencies only from within your network to avoid man-in-the-middle attacks uh so that offline mode was really helpful.

B

It's largely open source, much easier for companies to adopt and not worry about vendor lock-in.

B

It supported multiple iss uh like a lot of them and generalized internal infrastructure such as vsphere. So you could run it on local machine infrastructure. You could run it on one of the cloud providers.

A

B

Generally, the applications would deploy the same in all of those places which is is pretty epic. um I I think so anyway. I hope you all agree, uh but then cloud foundry has a few gotchas. um I think one of them would be bosh.

B

uh It was an infrastructure adapter and it was a good one, but uh it did not see significant adoption and uh it certainly had a learning curve, which was not huge for developers. I don't think it was even all that um insurmountable for operators, but I think for uh platform, integrators people providing services add-ons and side cars.

B

It was difficult and- um and I think that kubernetes has really taken- that place has become the you know, the industry standard uh and it's a really good declarative language for a declarative system for describing infrastructure and even where to put your applications.

B

um Also cloud foundry represents the last mile effort of your software supply chain. um It's simply from source code, built image deployed and there's some compliance and control there. That's just it's absent. It doesn't help with ensuring that vulnerabilities are maintained and patched from the software it did with it did with the underlying images that was great, but it didn't check the software and ensure that it was continually integrated with new cve patches and it didn't support special compliance rules or specialist supply chains. Specific behaviors that your company wants to use.

B

It did not support that. It had a very particular path to prod and that's what cartographer is trying to.

A

B

That's a great comment. I like that one so so cartographer cartographer is now sorry. I'm on my wrong.

B

Cartographer is now trying to bridge that final gap give us a way to go from development being able to build and rebuild your application. Much like it would appear in production, which a huge value add that heroku's paradigm gave us, but also go through constrained and controlled supply chain to production. That means that operators can collaborate and divide their labor on how to get things built uh to meet the enterprise requirements or the business requirements and keep it away from the developer. The developer should still have a very simple manifest to describe their app.

B

Their app should conform in basic principle to whatever build pack we select for. If that's the the route we choose to use, there's other ways to build apps, but it gives it gives the the operators and the and your business team the opportunity to ensure that uh your applications go through a process of ensuring the security compliance, um and, uh I don't know like socks.

B

Compliance is a classic example to go over boundaries from development into staging and into production, and it tries to remain uh simple in the fact that it used native cate objects which which humor I'm sure is going to explain a little bit more clearly than I have, but it it will print out. It will stamp out native kate's objects um and follow them uh instead of being edge, triggered when events happen just looking for changes.

B

So if something produces a new output along your supply chain, other objects that have been watching it will pick them up and I think that's an important journey and I think that's the oh carlos asked the question here: cartographer using bosch, no cartographer is using kubernetes behind the scenes. It is kubernetes as a kubernetes controller. You'll see more of that with washuma in a moment, and no there is no bosh.

B

That's the, I think, that's the point here is that we're trying to give that same experience but make it expandable and use modern infrastructure uh tooling, especially kubernetes and native kubernetes objects, to implement the steps along the way. So with that, I think I'll hand over to washuma. I think uh he can give you much more detail now.

C

All right, but.

B

I want to make sure can.

C

Everyone hear me all right: oh.

A

C

Now let me make sure that I can share my screen.

C

All right, um I'm gonna go off script for just a moment, um because I I love that question like oh, is this. uh You know heroku bosch what's going on here um and I wanted to make a call out to a presentation that I watched at kubecon this year. That really spoke to me um and uh this is hold on.

C

I want to make sure that I get his name right, david, daniel bryant of ambassador labs, I'm just talking about looking at his dev career and the cognitive load that he was going through, and so you know he shows he starts in 2000 and the cognitive load goes up and then, like the cognitive load, really dropped, and he made a huge call out to that. Heroku cf experience that rash was speaking of, and then, after that you know, we moved to kubernetes and there's so many advantages to cates.

C

But one of the disadvantages is that it does raise the cognitive load on our developers, and so what we are, what we're looking to do is make sure that we keep the power of these tools, uh keep the dynamicism of this community and, at the same time unlock that amazing dev experience.

C

um So, let's, let's talk cartographer, how does it work hi, I'm I'm wishima, I'm I'm a dev and I like devops, uh and so I've got myself a little bit of code that I've written here and what I want to do. I want to get this running I want to. I want my customers to be able to hit my amazing hello world app. They need to hear hello, and so there are a couple of steps that I'm going to want to do.

C

I want to get my source code into my kids cluster.

C

I want to take that source code and I want to turn it into an oci image, and then I want to deploy that image on uh nodes in the cluster that are going to run and serve, uh and so you know the first thing that I'm going to do.

C

I'm the devopsy guy in my organization, I'm gonna just do this by hand, uh and you know the thing that I'm keeping in mind too is. I want to simplify my life and I want to simplify the life of everyone after me, so I I'm making the decision. What I don't want is a lot of bash script that I'm going to have to maintain. I want to find tools that are purpose-built uh by individuals in the case community to do all of these steps that are represented here.

C

Well, I'm doing this by hand, and so the first thing I do is I go out and I find uh the git repository uh custom resource.

A

C

Yes, oh let me make this yeah make this a little larger.

C

All right there you go.

A

C

So I'm going to go out and I'm going to grab the git repository custom resource, I'm going to fill it out, and I know that when I deploy this to my cluster, it's going to go it's going to grab my grab my source code, it's going to expose it in my cluster to any other resource that would like to use it.

C

Eventually, it's going to create this status and in this status I can see now this url that's custom for my cluster. I can also see oh here's. Here's the newest revision on the branch that I specified here. Okay, that's great! Now I want to go on to my next step. I want to build that image.

C

I am going to go to another amazing tool, kpac image. If you were not here for tgik last week there was a wonderful, uh deep dive into kpac. We made a call out at the beginning of the show decorah you can hear why we are such big fans of her and she can tell you why we're such big fans of kpac here, I'm going to define my image and I'm going to take this url here and I'm just going to drop it straight into the definition of this.

C

This specification into the spec and once I do that, I'm going to submit it to the cluster, the kpac reconciler is going to pick it up. The controller is going to pick it up and it's going to reconcile it. It's going to output a status, it's going to tell me yep conditions, ready, true and there'll be a new image available.

C

Let me know where that image is, and the next thing I'm going to need to do is grab that location and I'm going to create a deployment and I'm just going to copy and paste just as always.

C

Well I, and once I do that I'll be able to expose the the ports of that deployment to the outside world and people will be able to uh to use the to use. My app wonderful, so I've just solved one problem for one person: I've gotten my code into production. What I would really like is to one simplify this for myself, there's a lot of copy and pasting here uh and two I'd like to simplify it for everyone else.

C

Well, the first thing that I notice, uh as I go on that journey, is that what I was defining uh there's a lot of stuff. That's just gonna remain the same. You know the timeouts on this git repository, not gonna need to change that a lot. uh Some of the definitions of you know the failed, build history limit things like this are not concerns that are going to vary by project to project. Those are things that we can really set policy for my entire organization. That's exactly what I want to do.

C

So what I'm going to do is I'm going to take this resource and I'm going to wrap it in a template.

C

That's going to be uh here and I have uh these fields and I'm going to then template out uh those fields that are going to vary now before I dive a little bit deeper into this. I want to one step back and say: okay, that's great! I can create all these templates, but really what's my goal. My goal is this file rash showed us earlier, the cf manifest and it's 10 lines and that's our goal too.

C

We want to give developers, here's 10 lines, really simple, that you can submit to the cluster and your app will be available. It's going to go through a defined path to production, it's going to be approved by the experts in our organization that know about that know about operations, and you won't need to care. You push it and it exists.

C

I in here one of the things that I'm going to want to know I'm going to want to know where's the source code, we'll talk about these other fields in a bit. uh So let's go and take a look. Let's jump out graphically and.

A

C

A

We have a question: have you really written all that by hand.

C

uh Have I so so this? Yes, I've had to write this behind before I try not to but we'll uh you know. The whole goal again is that I want to write this once and then I want to be able to reuse it end times, and I want other people to be able to reuse it end times. I don't want anyone else to do this. That's our entire goal!

C

um Thank you. So, let's come back here and let's talk about you know, what's going to happen, I've shown you these two types of resources, uh you've gotten a quick glimpse of one. Is this workload. The workload is uh owned by the developer and it's going to own its own logical supply chain. There's going to be a set of resources, a set of objects that are going to define that path to production. No surprises here. Those are the exact same types of objects that you just saw. It's going to be a git repository.

C

It's going to be a kpac image, it's going to be a deployment right and then there's going to be an operator who owns the blueprint of all of those things.

C

That operator is going to be doing that right once use a million times process they're going to define a supply chain blueprint, they're, going to define a set of templates that are called out in that blueprint, and what will happen is whenever a developer submits a workload that workload is going to pair up with is going to be selected by a supply chain blueprint and the cartographer controller will start stamping out those resources.

C

As the first object is created, it's almost certain that uh other objects will depend on it, and you know every supply chain is different. uh We're just describing uh that super simple path to production that we saw here, but I it can certainly get more complicated and at the end of this, we'll be pointing to some resources where you can see how you can iterate forward. You can see some examples of that iteration.

C

uh So the we'll have a resource. That's going to be stamped out once that resource is stamped out uh following templates, we'll will say: hey. There are values there that I want that I'm going to use and then those objects will be stamped out and eventually, at the end, you'll have completed your path to production. Each app will have followed this predescribed route to user availability.

C

uh So that's the high level view. That's where we're going before we jump jump into the template. There's just one other thing in our documentation that I want to want to call out. I mentioned there are templates. What does a template? Look like at a high level, a template spec is going to have one. The definition of the template makes total sense. Two there'll be some params optionally, we'll see an example of that and three there'll be some output paths.

C

These are going to be where the, where the author of the template says, hey that object that you just created. Here's uh here's, what values to expose to the rest of the supply chain.

C

There are a couple of different types of templates and the templates are typed, because different templates are going to be responsible for exposing different values. If you're writing a source template. The expectation is that it is responsible for providing source code or verifying source code, and so what it is going to provide is it needs to expose the url path, the url and the revision path of that object.

C

Similarly, if you create, if you're, stamping out an oci image, you're going to want to specif that resource is going to specify on its status somewhere, hey here's where the image lives and so you'll use an image path to define that. So, let's, let's dive in, let's see an example of some of these.

C

There we go so here we have that git repository that we were looking at before and suddenly we can see that those fields that we said can be hard-coded are hard-coded and these other fields have callouts to the workload object. That's the same workload that we were just looking at here. So let me say there is a metadata metadata name, hello.

C

I know that I'm going to when I apply this to the cluster, when cartographer gets going on that workload, it's going to stamp out a git repository with the name hello, because that's what's on my workload. Similarly, it's going to have some value work, spec source get url. You can see that right here, no surprises.

C

And off it'll go it'll stamp that out, I well that's great, this one's pretty straightforward, uh but then suddenly we get down to our cluster image template. That's wrapping that kpak image that we saw before and we see a little bit of complication and in particular I want to draw your eyes down here in our previous object. We just saw call outs to workload, but now we have a call out to some sources, and you know we're we're uh right. So far, all we've done is define some templates.

C

How do these templates know which one they're going to talk to? Well, it's that final object, that I've just made brief mention to the supply chain, the supply, the blueprint of what the supply chain is going to be. Let's take a look at that really quickly here we have the supply chain object and it has at the top level two fields that are super important.

C

One is a selector uh because we need to this supply chain needs to define here are the sorts of workloads that I'm going to work with, and the other is resources- and this is saying: okay here are the templates that are going to be stamped out. Here's how they rely on each other, here's how they can uh here's, how they can communicate.

C

Let's start with the resources and uh rash david, I'm going to ask you to hold me accountable, make sure that I come back to the selector, so I can talk about it for a moment uh here we have uh the first step, uh the first resource in our supply chain, it's a source provider, and this is the name that all the other resources will use to refer to the work that goes on here uh and then there's a template.

C

Ref no surprises here, uh it's a cluster source template and its name is source, and if we go back, I'm sure you'll not be source super easy. I if you've worked with object references in case this will seem very familiar. That's the goal.

C

So the first thing that's going to happen in the logical supply chain is that it's going to take the workload it's going to take the values from there. It's going to make them available to this template and the object is going to be stamped out once the next. It's going to go and say: hey I've got my next resource, it's an image builder, and we see it has another template ref the cluster image template.

C

uh We have taken just a quick peek at that uh it is also providing some values uh that are going to be made available to that image. First is params. Second sources uh sources. As we said, there's there's this name match we're, making sure that uh at this level we can pair them up, and then we specify, what's the name, that the template calls out uh to this value with.

C

So here we say source, so we've already talked uh quickly about the fact that a template exposes some values and a source exposes a url in a revision. Well, as I go back to my demo, templated.

C

As I come back here, I can see that my image is now saying.

C

I expect that there's going to be a list of sources that are provided to me, I expect that one of them is going to have the name source and from that I want to get the url, uh and so there we have that copy and paste that I was having to do by hand. Now it happens by cartomagic uh you'll, be unsurprised to learn that our next template is going to do much the same, except now the the location of the image that's going to be exposed on the status here.

C

ah Actually, let's make a call out to that uh when I was doing that copy and paste, I was going to some field on the object. I saw here's my kpac image. It has a status field and it has a latest image field, and so that's where I was copying and pasting from well now.

C

I've put that into code in my template I said: hey, this is going to create an object and that object will have a status latest image and that's where you can find the definition of the image location that all the other steps in the supply chain are going to want you to expose the the deployment you'll be unsurprised to see makes a call out to that. It says: hey in the supply chain.

C

There's going to be my resource, it's going to refer to me the template, and I expect there's going to be some images provided- and here we see images is providing a solo image and we make it available uh now some things when we get to the dev experience. We're gonna simplify this some syntactic sugar that we've made available is you can refer to this by just image, because there's only one image being made available and the image type makes uh only one field the image field available so rather than images.image.image.

C

You can use that shortcut, but uh if you want to, if you want to spell everything out, make it very easy to track, uh you can use that whole uh definition. Okay. Well, that's how things are being passed uh from template to template. uh Another thing that we I kind of alighted over was that in our supply chain we saw this field params.

C

So let's go and look at the cluster image template. It does indeed have a field here where we are templating out some parameter, and what this represents is the supply chain.

C

The template author is available to put in here is the best information that I have about what I think this value is going to be, and so they say for my for the params, I'm going to give a value image prefix and I'm going to provide some default value and then that's uh that parameter value is something that can be overwritten at the supply chain level. Perhaps the uh in this organization, the individual who's, an expert at building images.

C

That knows exactly how kpac works is say: I'm going to defer to the folk, the person operations, who's going to be uh responsible for saying, hey. Here's where I want all of our the images in our organization to live uh and they're able to do that by saying, hey, you know, call out to to this default supply chain, and when you want to vary that image, prefix value, you can specify a value on the parameter at the supply chain level in a very similar way.

C

If the, uh if the operator said you know what I actually want to give the developer ownership of that piece, they can just go and in the same way just rename this default and that allows the developer to define params on that workload.

C

But for now, let's stay here, we're just going to say this is this: is the location of our image registry, okay, and now that last piece that I I was worried that I might not get back to that, I might forget about, cannot forget how do workloads get matched up with supply chains ultimately happens based on the selector.

C

uh We see uh that there is some label that is expected for the types of workloads that are going to be made by the supply chain, and you can see here some of the some of the philosophy. This is the supply chain that should uh apply to all workloads that are are web apps, and so we expect that, there's that your organization is going to have a similar path to prod for all of those uh different applications uh as rash started out they're, not all special snowflakes.

C

We should, let's standardize, let's make things easy for ourselves, uh make it easy for our operators, because then we can start really honing in on the value uh at our customer level. uh So when I go back to my workload, you'll notice, one of the fields cannot be forgotten. Is the label needs to be on this workload and that label should match some selector of some supply chain? That's been applied in the cluster all right now.

C

The last thing that I want to call out on. Oh, there are two things. There are two fields that I've not explained on this object at all. um One is just some this build and here we're just saying in the same way that there were some variables that we wanted to pass on to uh objects in our supply chain for our kpac image, we're just saying hey, you know there might be some build environment variables that we want to pass on. For this example, I'm just saying hey for my go.

C

I want it to be hard linked and specified easy peasy ready to go. The last is this service account? I we are defining. As I said, we've got this supply chain that can stamp out all these objects in our cluster and uh cartographer is, would technically be able to create anything anything anything. uh Well.

C

We want to make sure that your organization has uh the keys to your kingdom, uh can uh keep things uh locked down lock tight, so we, your operations team, is going to be responsible for make sure that there is some service account that has the rights to create and manage the sorts of objects that are going to be created in a given supply chain.

C

Here we can see an example of that. It's nothing crazy. We've got our service count. We've got a role binding to some role, and here we see that role- and you know it just says: hey there kpac it's going to create an image, make sure that it can list create, get update, etc. Right, I I said nothing controversial. There.

A

Yeah uh sorry shuma there is a multi-part question from carlos.

C

A

Hey by the way, carlos welcome being a long time friend of dj kids. um So let me put this here.

A

Oh, this is one part of the question.

C

ah Carlos is carlos you're. Speaking to my heart, um I see him well, there's a it's a multi-part question and I haven't gotten in the second part, uh but the first thing I heard was uh what, if what I'm looking for is some git ops flow, uh where I'm not looking to push uh code in my cluster, uh and I think that is that that makes total sense uh in this example- we're not going to get to that.

C

But I'm going to point to resources that describe exactly the flow that you're talking about, because our hypothesis is that, yes, you will want to or many organizations will want to build in one cluster that has all these resources that have all this power to analyze code and create, create images, create uh yeah, create images, uh and then you'll have a separate cluster that you want. That's just the public facing cluster that has much less on it. It's just hey.

C

I've got my kids, kids config, I'm going to drop it in there it's going to uh serve uh so one david. As I said, I kind of read half the question and I you know I was off on a direction. uh Have I responded to a question from carlos, or is there more that uh yeah.

B

There's more questions, I have been responding in line and I think.

A

B

I think I've left carlos and perhaps other people um uh in a in a position where they believe what I was saying, because I could understand how you would see this was that cartographer replaces heroku push, and I think uh I think I wasn't clear enough there.

B

What cartographer does is it provides you a way to create heroku push like experiences short, simple, dev experiences to go through complex uh but malleable uh pipelines or, as as we like to look at them supply chains to get to production, and what we're describing now is how you would architect those paths to production.

B

This is a complex scenario that even even the simple one, with three steps where there is a lot of templating, but the idea is that it can be reused by all of your developers when you're happy with the the structure of it and what you're doing is at an organizational level describing the path to production such that uh developers only have to push the replication, but it can go through varied different ways to production. So it's not cartographer itself is a way for you to build a platform that gives the experience of cf push.

B

Heroku push, I hope that helps a bit.

A

C

A

Just let us know cause.

C

So uh and now we get to the to that fun part. What does that? uh What does that analogous experience? Look like we've gone through the work of the operator defining all of these uh pieces and congratulations. You guys are all now experts. You can go back to your org and build that logical supply chain. What are your developers going to do? Well, let me bring up my cluster and I'm going to first just cut my workload. You'll see nothing no surprises here.

C

I'm going to take this and I'm going to apply it to the cluster, so k apply, dash, f workload.

C

That's that's our analogous cf push. uh Take your uh take. Your workload, apply it to the cluster, and now I'm going to uh I'm going to just try describe this cube. Cuddle get o dash, yaml my workload, and I know that my workload is named hello and great. I look at my status and I see supply chain ready, yep. True resources submitted missing value at path. Oh no, it's not! Oh! No! uh This is probably one of the biggest gotchas that folks have they uh submit and they expect that instantaneously. Everything will be done.

C

Supply chains take time if 2021 2022 has not taught us anything. It's that supply chains can take some time what's happening here is that we can see there's a value that latest image that we're waiting for from our kpec image object, I'm going to go ahead and do a k tree on the workload.

C

And I can see. Oh look, it's already beaten me. I I can see the git repository object that was created. I made a prediction early on that I'm going to have a git repository object named hello. Here I can see it. I can see my image object named hello. We've got those names again because that's what the template says uh and now the that kpac uh image has completed and having done that, that final step, that deployment step has occurred here uh and that's ready.

C

So if I do my check in on my workload again, I can see yeah that top level that top level ready condition is true. That's great. I want to hit my deployment so I'm going to keep cuddle port forward type deployment there and forward 8080 to my local machines, 8080 and I'm going to swap over to there right there. There we go, I'm going to go ahead and curl building with card code is easy. That's the experience that we want to give all of our devs.

C

Well code doesn't stay static. You know this is great for today, but what about tomorrow? I'm going to change my code and I wanted to go straight to the same uh supply chain. Well, all I'm going to do is uh here's here's that uh amazing hello, world uh application, I'm going to say I'm glad I spend fridays with tgik.

C

I'm going to.

C

I'm going to commit this.

C

And I'm going to push it up.

C

Okay, I'm going to start port forwarding for a moment and I'm going to take a look at my uh my tree. What I'm expecting to happen is that my git repository that piece- that's an expert at checking for updates in a git repository, is going to notice that change that's been made and it's going to spin up and it's going to start exposing a new value to the supply chain. The moment that new value is exposed, I'm going to see a new.

C

The image is going to be updated. That image is because of the way kpac works. uh It's going to create a new build and once that's been completed, the image will be exposing a new value and it will. The deployment will pick that up. Let's go ahead and see.

C

Looks like that has not been picked up there we go so here we can see that new build happening while it spins. uh Let me take a step back for a moment because at the beginning I said sure it's great. I can create a supply chain for myself, but really what I want to do is I want to create a supply chain for somebody else.

C

So here I want to talk here. I have my friend I'm sitting here, I'm a go developer. I've got this, go hello world app, uh but I've got uh my good friend who's sitting in the at the desk. Next to me at the zoom screen next to me this year, and I want to tell them hey, I can get your code to production super easy and they say. Oh no, I'm writing this job app, it's totally different from yours. I say: no, no. I've got special built tools.

C

I've got these custom resources that are experts at each piece that they do and they know how to speak. Your app's language kpac can work with a go app. It can work with a java app. So let's go I'm going to cut out a new workload definition.

A

C

A

Before that, there's a couple of questions here, first, one.

C

A

The change you just push flux will detect the change and update the status of the kit source custom resource.

C

That is correct, yeah, let's go ahead and take a look at that git repository.

C

uh So the spec has remained the same uh and we can see that the last transition time is just a moment ago. um I.

C

Yeah other than saying like oh yeah, we we just saw it occur. I I don't know what else to say other than yes, there's a great resource. That's purpose built for just this.

A

And also are these builds text tecton crs.

C

No, they are not. We love tecton for certain things.

C

I love tecton for making sure that I can run tests because the tests that I have on my application may well be very different from the tests that I write for a different application, but what I don't want to do with tecton is write all of that bash to do step, one step, two step, three step. One being you know, get all the things uh get all the source code that I need from xyz place, uh step two uh build my image, step, three etc.

C

So these these resources are are, I think they are all cncf projects david. You may know better, so please fact check me or anyone in the comments. Please fact check me, but these are other uh resources, other controllers that have been developed in the kubernetes universe that we find super valuable.

C

Now all that said, can you work with techton? Yes, we love techdot, and we, uh if you when we go through, we won't get to today, but in our uh repo. In our examples uh we talk quite a bit about how you can best use. Techdon pipeline runs and techton task runs in your supply chain.

C

All right cool, taking a pause there if they're follow-ups, please hit hit us with them. um uh So yeah I've got. As I was saying, I've got this uh new workload. It's really similar. This is going to be another web application.

C

It's going to call out to the same service account. That's been provided for the operators to say hey. We know that the supply chain is going to create some objects. uh We're going to we're going to hand you that reference, so that you can see your workload will be able to go at it. This one build environments. Doesn't it doesn't need to provide anything special to kpac and that last piece? The piece that really does vary is hey. There's some new uh application at this at this new url.

C

So I'm going to apply it.

C

And we'll see that it's going to spin up and it's going to look quite similar to what we saw before capex going to work on it, the git repository having been so quick at its job. uh While we wait for that to work, uh let's check in on our previous.

C

Our previous workload, I see the image repository is ready. True, I see the deployment is ready. True, I want to keep cuddle forward again and I want to see what is being exposed to my customers. They've got their brand new hello world message. They are super happy.

C

C

I don't suppose not super quick, so as uh as my friend's java app uh spins and creates, I I'm gonna come back to it. I don't want to take it on my word that this is going to work.

C

I want to just mention very quickly a few things. ah Here's that here's that java application. um Where do you go to learn more, I I I hereby certify you, you guys are experts. Now you guys know the basics of how to create a supply chain, uh as you want to move forward in your journey, as you want to hear about. How do I do that multi-cluster journey? How do I do work with resources like techton that don't have quite the same update mechanisms that other other crs do that place is cartographer.sh?

C

You can come in you drop into the doc into the documentation. You can see the architecture docs that we've been looking at today, you'll be able to see it for each of these custom resources. uh The work that's been put into you know just explaining what are all the different fields. uh What do they do? What are their purposes uh and you'll be able to go on to and to our examples directory?

C

That's a link here and uh this first example. uh You've essentially done this uh source image app. uh The next place that we recommend that you go is to learn about another custom resource, that's part of the cartographer universe, runnables and then you'll be able to use that runnable to expand that first supply chain to something that's doing testing with tecton.

C

uh After that uh you know, there's a quick uh conversation about how do we then do that get ops workflow? I I want to build my I want to build that image. I want to define what that kate's resource is going to look like, but I want to define it. I want to actually deploy it somewhere else. Well, here's where we talk about. How do we do that writing to get, and then how do we use a separate resource, a delivery resource that is a little less powerful that has just one job deploy this resource?

C

You can read all those examples uh here. You know behind uh behind any product is an amazing team. uh The resources that I've just been describing have been made by uh the folks that you see on the screen, and I want to give a big shout out a big thank you to all of them and you'll see that there's a big question mark at the bottom, because this is an open source project and it's going to thrive based on the contributions of the community.

C

We want to put something out there, that's incredibly valuable and anytime. You think you know what would be incredible is if it did. Xyz come talk to us uh there. uh When you look at the readme of cartographer. There are call outs to when we have our meetings. How rfcs are submitted uh we want to. We want to engage with you. We want to hear how is cartographer, helping you, how can it be better, and I want to check in and see?

C

Oh, my goodness. Well, I'm gonna, I may leave us here. uh Just wondering was wishima telling the truth is my kpac image ever going to be built? I I I have a hypothesis yes, but I don't want to hold you here uh for that, so I'm going to hand back to david uh and uh perhaps uh at the end, when kpac finishes, spinning on that uh maven app I'll jump back up.

A

Sure enough, he wouldn't be dji cave. We didn't have any sort of issues so not for the medal. Okay, that's great! Thank you. Thank you for joining, uh so anyone else have any questions so far and yeah for sure it's open source prs welcome any form of contributions, both code and known code contributions are welcome, we'll be happy to see you around the community and share here link for the community resources and okay. There is no anything else we would like to say. Thank you see you in further episode goodbye thanks.