Red Hat OpenShift The Level Up Hour | Red Hat Livestreaming, 1 Dec 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: The Level Up Hour (E53) | Pipeline as code

Description

Pipeline-as-code can help devs with quicker and more consistent releases, but how do you get started? In this episode of the Level Up Hour, we'll dive into pipeline-as-code, review its advantages, and teach practical techniques that can help you help simplify building a CI/CD pipeline. So join us to learn more as we introduce, discuss, and teach you about the DevOps pipeline-as-code concept. We'll share our experiences and discuss some of the tools and practices needed to automatically run pipelines from an application code repository.

Follow us:
Scott McBrien https://twitter.com/stabby_mc
Randy Russell https://www.linkedin.com/in/randy-russell-0312243
Jaafar Chraibi https://twitter.com/ChraibiJc

What is the Level Up Hour?
Every Wednesday at 9am ET we stream live demos on how Red Hat Enterprise Linux (RHEL) users and admins can benefit their organizations and improve their careers by learning how to use containers, Kubernetes, and Red Hat OpenShift.

Learn more at:
https://red.ht/leveluphour
http://red.ht/3ayHlHI

How to participate in the The Level Up Hour:
Watch on YouTube every Wednesday at 9amET
Chat with us on Discord https://discord.gg/5VMVGJt
Check out our GitHub repo https://github.com/level-up-hour/

About OpenShift:
Red Hat OpenShift is an open source container application platform based on the Kubernetes container orchestrator for enterprise application development and deployment.
https://www.openshift.com/
https://github.com/openshift/

#RedHat #OpenShift #tluh

A

A

B

Morning, good afternoon,.

C

Good evening and welcome.

B

To another episode.

A

Working on open shift- and sometimes this is so.

C

C

On top of the red hat portfolio,.

B

Good morning, good afternoon and good evening, you have lurched into another episode of the level up hour. So thank you for joining us. Please make sure to like subscribe and share it.

B

Let everybody know that we're out here on the air, uh I'm randy russell director of certification at red hat, so shout out to all those red hat, certified professionals out there and I'm joined by my co-hosts jafar chiribi and scott mcbrian greetings, gentlemen and hi welcome to, and so today we have a very interesting uh episode in which we're going to talk about pipelines as code okay and to help us untangle that is shimul bhujna. Who is the lead architect and developer of guess what pipeline says code?

B

So I think this is going to be uh a very interesting episode. So, first of all, let's, let's talk a little bit about pipeline as code jafar. That's uh you know I you know, I think I'm going to back it up a whole lot and say well, you know, I think most of us at this point in time are familiar with the concept of a ci cd pipeline.

B

You know continuous integration, continuous, continuous deployment, where you have your application, the new feature, the bug fix whatever, and it is whisked seamlessly and effortlessly through test and stage all the way to its glorious end in production. Right, uh so help me understand what we're getting at when we talk about about pipeline as code.

C

Okay sure sure, so again I wanted to thank small for joining us here today, um so he will be explaining a bit more. The new features that we are offering on openshift but to to to give some background to this. So we all know that traditional ci cd tools have been evolving since uh we started using that kind of tools to provide some kind of um scripted automation to to to do your pipelines.

C

So basically writing what we call domain specific languages where you have some scripting specific scripting, like groovy or whatever specific languages, was used for that specific ci cd tool.

C

So basically, for example, if we speak about openshift and jenkins, we've been providing something called support for jenkins files, which basically is a a groovy script that orchestrates your ci and cd steps throughout the different environments, but as the kubernetes ecosystem really boomed and I would say, exploded and it started to become the de facto uh infrastructure to run apk containerized applications. So I'm saying kubernetes is the intra. uh Probably some some people will throw some some rocks or tomatoes at me, but basically yeah.

C

We can consider that it's the the new, the modern uh infrastructure to run those applications and something that has emerged uh about two years ago, which is, uh could we think about kubernetes also as a ci or cd tool. So basically, instead of relying on a dedicated ci tool like jenkins or I don't want to call out those tools because they are great and they are good at what they do, but like jenkins, gitlab or travis or whatever those tools.

C

um So the goal was: can we have something that is relying solely on kubernetes as the cicd engine? Can we extend kubernetes to understand new concepts of ci cd and since the introduction of what we call custom resource definitions in the kubernetes api that allowed us to create some new concepts that were not existing in kubernetes before in a simple way, so the the key thing behind it is in a simple way. So, basically, you can now tell two kubernetes here's a new concept.

C

It's called a pipeline, here's a new concept, it's called a step, here's a new concept, it's called a task or whatever and basically now kubernetes starts to understand how to trigger uh and run pipelines natively by orchestrating containerized actions uh or tasks.

C

And thus we don't need those uh dedicated cicd tools anymore and one of the drawbacks, I would say well not drawbacks, but one of the things that you had to take care of was the administration uh of those tools. So now, since you are switching to kubernetes, that's, I would say less administration. So that's the first, it's more uh portability, because now you can have your ci running on any kubernetes cluster. That has those concepts installed.

C

So basically, it's based on an upstream project called tecton and the great thing with tecton is that it's all of those uh great cicd vendors players coming together to come up with some sort of standard, so the the standard way of of pragmatically describing what a pipeline is or what we call pipeline as code right. So how can we write yaml files that translate into pipeline execution, so that was the whole idea behind tecton, so that was to provide a kubernetes native cicd uh ecosystem.

C

B

Right, so if you have a, if you have a now that it's kubernetes native you get the the benefit of not having some integration points that you might have, otherwise you have the consolidating down to a smaller number of uh technologies or products in a sense and potentially a smaller number of vendors, which is also sometimes a benefit, uh and then it is. It is actually something. That's native.

B

I there's something ironic in this in that I think a lot of the world has come to kubernetes and to openshift with the desire to build out their their cicd pipelines, and it can be effective for that, but it has not actually been a native capability from the outset. So it's really interesting to me that now the purpose that so many uh organizations come to openshift is actually going to be supported. Natively. Would that be a fair statement to make.

C

Yeah, yes, definitely, and so the interesting thing is that um so, of course, kubernetes is known for its scalability. uh You can now orchestrate thousands of tasks and and pipelines, because kubernetes allows you to have that scalability you can dedicate, so you could do that before with the traditional ci tools by using what we call agents. So basically you had to deploy agents you had to administrate them upgrade them and all all all of those kind of of topics, and also one of the the things was say you wanted to integrate.

C

Now with a new set of uh tools. um It's so either there's a an existing plugin that you can rely on uh and you just uh take it and use it, but you are limited to whatever the plugin offers or you are a massive java guru and you can, for instance, create your own pipeline.

C

I have some colleagues who did that but like they are crazy guys, but they are very talented guys, of course, uh but now with with the kubernetes and text on uh way of doing it, it's basically, you are creating container images that contain all the tools that you need and then, through the yaml steps you are referring referencing those images and say this is the command that you need to execute in this. By putting this specific image. So now it becomes much more much easier to extend the capabilities of your ci cd ecosystem.

C

Basically, as long as you can put it in a container, it's going to be fine with your it.

B

Is it is ready to go so um we actually already have a question in the chat and uh shmuel. I think uh a perfect introduction to your participation here would be to put you on the spot, so she can't ask the question is okay? How do we manage pipeline as code when we have hundreds of micro services and, in brief, the answer is.

B

uh The proof is that.

A

You're doing uh integration testing is that uh so first, uh so first like we need to maybe to explain like what's uh what's pipeline has called I mean uh uh java has done like a great uh introduction like to take down pipeline and how you specify pipeline like in, jumbles and define your your testing and the steps, the different steps for the testing, but the pipeline has code itself.

A

What it does is that you have uh you have a repo history like where you have some codes inside it. uh Micro services, for example like she was mentioning and- uh and you want part of the development process. So when you're doing a change to your code, you won't like to have your pipelines living with it, so you don't like to to to be able and that you that you will change that's happening in your code like reflect. What's inside your pipeline, so historically, and a lot of ci systems uh does.

A

That is that, like you installed your pipeline before and uh whenever there is like a code change, then you test it. If you need to make a change to your pipeline, you go on the server. You change the background here with pipeline escort. The basic idea is to be able like to have your code inside uh your your to your pipeline inside your code.

A

So whenever like, for example, like you're, adding like a new component that needs like different kind of testing different steps inside your pipeline, then part of the pr that you're going to spend. Do a two cent like it's going to pick up like that cell change and and it's going to uh take effect with with the change of your code. uh So the question of street is like how you do that like when you are like.

A

So, as I said, like the the the repo history, like it's very tight for the change that you have inside the pipeline. So the pipeline reflects the code, and how do you do that like when you have like multiple micro services uh and usually like? Usually, it goes beyond pipeline code? Is that it's how you're going to test your micro services so to be able to test your micro services?

A

There is different techniques to do that, like integration technique, integration, integration test and uh and when you have like multiple macro services, you usually uh use contract testing, so you have contracts between different uh components, microservices and you just like making sure that they work to each other.

A

So, when you're going to work, your pipeline has code pipeline uh you're going to have like integration testing of those microservices, but together and every time you're doing like a pr to your code like it's going to run those uh integration tests, making sure that your change is not integrated.

C

Yeah, so thanks thanks a lot small. That's um that's uh some, some great insight and if, in fact, this this question alone uh can be discussed for like the whole afternoon, because it's it's it's a very complex topic. Actually, um I I did a webinar on that specific topic. It's like ci cd uh for microservices. uh It was a few few months back.

C

uh We can try to find the link and share that afterwards, but I think it's also a good topic to just because uh it's yeah there are a lot of things that you need to address, both from an administration standpoint and from a a developer standpoint or from a devops standpoint. So so, basically, one of the key aspects is because you are managing hundreds of services.

C

These type of new pipelines allow you to have some genericity something that can be reused across the similar applications. So, for instance, if your application landscape is composed of 30 of java, apps and 50 of node.js apps or whatever, so I'm throwing random numbers here, you can try to factor the java apps into a similar pipeline. That will allow you to just point out to a different java repo and then build out the application with the same exact pipeline and then deploy that microservice somewhere.

C

So what you really need to avoid at all cost is, if you have 100 micro services to create a hundred different pipelines. That's that's something that doesn't scale it's, not something that you will be able to manage. So the key topic here is factor your pipelines and have something that is generic and that can be dynamically adapted to your micro service. So one pipeline definition can serve 30 micro services, for example. That's one of I would say the key aspects.

C

Of course, uh what uh smwell mentioned integration testing contract uh testing things using things like mocks, when the microservices that you rely on are not still available in the testing process, so yeah? There are many many techniques that we can maybe discuss and yeah smile. I think you wanted to say something um now.

A

I I just wanted to add uh to your to your to your answer. uh Is that it's like there is this like, as you said, like there's, so many things like we can talk about uh for uh for micro services, but there is as well like. How are you going to structure your code like? Are you going to have like multiple representatives are going to have like a mono ripple? Are you got like a multiple repos branch together, and the complexity is going to be enormous, like different.

B

A

Way so a lot of people goes to monorail, monorepo kind of structure to be able like to uh to solve those kind of problems and, uh and then yeah, if you, if you start like to have like a hundred uh android micro services, uh you'll need to think about like how you're going to uh move it like to a monorail structure. If, because the complexity can be like really great, if you are, if you don't have that.

D

Yeah, you don't want to have 100.

B

Examples of uh pipeline as code is basically right what you want to end up, not with.

C

Yes, yeah exactly.

B

C

The earlier example.

B

So you you say: okay well so they're, maybe at 60 40 java to node.js what you're hoping is that in the perfect situation. Maybe you have two approaches to pipeline as code. One is your node.js approach. One is your java approach. Maybe you have to subdivide it a bit more, but again you don't want to have a hundred approaches, and that might be the real challenge is that in theory, it would be possible to do that right.

C

Yeah yeah, so so also one of the key aspects of this pipeline as code or tecton is the reusability of tasks. So, basically you can define shared tasks that users can can can reference.

C

So basically, you don't have to reinvent the wheel for pushing uh an image to a registry or for building code or for doing integration, testing, etc. So it's all there. It's ready on your shelves. You just have to reference it and say: please use this task in my pipeline and that's how you can have some genericity and that's how also you how you can lower the barrier of administration when you need to make something evolve.

C

So if everything is spread out into the 100 pipelines, then whenever there's a change, you will have to manually commit that to each and every one of those hundred pipelines.

C

If you are uh taking the shared um approach, shared tasks or whatever you want to call it, you can factor the change in into a single place and then it's reflected in the 100 pipeline that are going to to to reference it. So there are many things that you can use to.

C

I would say industrialize the way that you build your pipelines and actually would be uh happy to have another episode around this uh one of the other things that we can leverage uh or I don't know if yeah we can say, leverage something but use as a leverage is uh git ups. Also in the picture.

C

It can also simplify a lot of those things and it can also address what schmuel mentioned, which is this notion of monorepo or multiple repo, where every microservice is going to have its own um code, and then you have, if you are using githubs, then you have to think about where you're going to store your github's resources uh and and and stuff like that.

C

So organization and standardization are key uh topics that need to drive the conversation uh and, of course it's not gonna happen uh like from just one guy deciding something it needs to be taken into a devops approach. Where you have everyone uh thinking about the end, um I would say the end goals and the the best way to standardize the code, the repos, the pipelines, uh the testing approach, etc. So yeah, I think we have uh spoken largely enough about that covered the marketing services yeah. Exactly so.

B

Let's back up a little bit and, and you know for the for the benefit of uh stabby and myself here, what needs to happen if you have for an open shift administrator to.

D

B

Capability manifest itself in in a in an open shift environment. Or is this something that that the developers simply start doing and it works as though by magic.

C

Yeah so uh uh of course that's the end goal is uh to to make it uh appear, magical for the developers and make their life easier, and I guess shmuel can can talk uh in more details about the the end goal of this. But let's, let's dissect that into uh several uh things. So I would speak about the openshift pipelines.

C

Shmuel can speak about the pipeliners code. I will just rephrase what he said to make sure that everyone is on par in terms of understanding what it does. So, as an admin first thing is, you will have to install the openshift pipelines operator.

C

Openshift pipelines is our downstream productized supported offering, which is included in openshift, based on the tecton upstream project that we spoke about, so it allows you to basically define pipelines as yaml and kubernetes native resources. Okay, so first thing you need to do is install the openshift pipelines operator.

C

Luckily, we made that very easy for our openshift users. So as an admin, you can either go through the I would say operator hub and install it or you install it through command line. So I can show you that once we go to the demo section and then some things need to happen for the pipelines as code, so just to make sure I'm going to rephrase what schmoyer said so once we have the pipelines installed.

C

What we want to allow our users to do is to ship the yaml of their pipeline in their application and don't need to worry about creating the pipelines in openshift beforehand, because if you are using the traditional way, your code, even if you commit and push your changes nothing's going to to to happen unless you have a web hook somewhere that triggers a pipeline that you have already configured before. So it takes several steps before this can happen.

C

So let's say this is the traditional way of doing it. I create my pipeline in openshift, using yaml or using the ui so because we have a ui builder, where you can basically draw your your your ci cd pipeline, then you create a webhook that your code repository will use whenever you trigger some events like a git push or git pull, or something like that, and all of that needs administration.

C

You need to configure all of those things so now, switching to the uh tkn or pipelines, that's called sorry way of doing it and well, if you would like please to explain uh how it works and yeah what needs to be done.

C

That would be good.

A

uh Yes, so uh so, as you explained, uh pipeline has caused like plugs into openshift pipeline.

A

uh So it's a feature on top of uh openshift pipeline that you can add and uh to make it uh to to make you uh to be able like to do your ci directly uh from your github from other vcs providers like source code.

A

So currently partners code is, uh is a is a dev preview feature.

A

So it's not yet integrated inside the openshift pipeline operator, but we are working toward it and it's currently that the way it works like it's available inside the github organization of achieve pipeline and where, like you, have a way where you have like some restrictions like how to install so, the thing is like there's two parts to it like how to do the install uh there is the easy part which is like applying uh yaml files inside the cluster like to get that capability and- and the second part is, is to be able to say, like to your source code provider like github, uh that all the all the the events happening inside your repo.

A

It's going to it's going to go like to pipe to the openg pipeline cluster. So you need to be able to say that so to do that, uh uh like the main way like the the main way is to create like a github app, so that github app is going to be central to uh to to pipeline as code and it's how uh you're going to manage all the interaction between pipeline as code and your source code.

A

So what I mean is like, whenever, like a developer, is going to send a pr uh pipeline code is going to see it uh and it's on the cluster is going to see that there is uh that tecton directory inside your source code, repo with a pipeline inside it, and it's going to see if it's possible to uh to uh to like if the cluster wants to uh like as information about it.

A

So if we are able to to run it or not, and then, uh if the user as well is a load as well like to do that and then it's going to run. It run the pipeline as defined inside your source code. So at the point of time of your pr and then it's going to report like it's going to do a nice repost of all the things that has been uh has been failed or has been uh what has been successful and uh with linked to the logs to the openshift console.

A

So that's the basics of it, and there is multiple features of pipeliners code around that to make it easier to uh to develop a partners code pipeline and which is uh which sits on top of openshift pipeline.

A

So so we have like a different like capabilities like uh like, for example, we have like a what we call like shut ups capabilities, so you you're able like, for example like when you have like an issue with your infrastructure, and uh the backline is failing due for that reason, and you can change just like do like a slash retest inside your pr itself, and uh it would like restart like the the process, and you can do like different things like for uh allowing people to by default.

A

Like should I like to make sure that uh we we don't allow anyone like using your infra? So not everybody can send a pr for anything, but uh so I, but someone who's alone, like can just do like a slash okay to test and automatically like you'll, have a like allowing this there's other features as well uh to make it easy, so by default, uh something that's not possible to do uh with openshift uh with openshift pipeline. A vanilla chip pipeline is to be able like to get tasks.

A

That's not clusters task or that's not like tasks inside your uh you're inside your namespace, that's already installed, but with sparkline escort you're able like to say, like I want to uh pick up that remote task uh from somewhere else or from the hub to the hub, which is like a catalog of all the tasks that's been.

A

uh That has been uh that has been contributed by uh by by contributors and ultimately, it's going to uh you can use it inside your pipeline without having like to pre-install it or to references from somewhere else, and uh and that's the other question that we had previously that we were talking about is like it makes it like as well, even more easier to share your different tasks, uh because you can just like have a simple url now and you don't have to install it.

A

So that's the that's some of the features that you have with spike linus code and, I think part of the demo. uh It's going to be a bit more clear like how the the flow looks like and how you can you can do a pr and iterate into it and after uh be sure that it's green or not.

C

Yeah sure so uh thank you uh well for for the I would say this transition should we maybe go to the demo to to.

D

C

Yeah, so uh you know it's gonna be more visual and afterwards we can talk about what happened in the background.

D

C

Exactly yeah live demos. What can what can go wrong right, yeah, all right so first step is I I need to succeed in sharing my screen. So that's the first step of a successful demo. uh Can you guys see the screen correctly.

C

A

Yes, we can yeah.

C

Yeah, it's good, okay, okay, cool, so um yeah. Let me try something real, quick here. Okay, so I have an openshift cluster here, where I have a node.js application so where the goal here is to keep it very simple: uh let's have a look at this app. Okay, let's see our openshift starters, it says version 2.32 by blind just go okay. This uh application runs in a namespace where, if we look at pipelines, I have no pipelines defined.

C

So traditionally, if I wanted to have ci cd for this application in this project or in another project, the first prereq would be to have a pipeline that is defined in the namespace and then I would execute the pipeline by creating what we call backbind runs and if we refer back to what I said about having something that is generic, the the pipeline definition is the generic uh aspect of it, and the pipeline run injects some some data to say: here's, the git repo that you should use as an input.

C

Here's the output image that I want you to generate and stuff like that. So, okay, we have our image. We see it's version 232 here and let's have a look at our uh repo so to to speed up things uh for the demo. I have already created a pull request and the flow that we are going to look at is I have this specific, full request that is open right and basically I can use it as a conversation with all the developers.

C

So whenever I'm committing new change to my application, I want some ci cd process to be triggered automatically, so we can iterate, etc, and once we are all satisfied, we can ultimately merge the pull request into the main branch, or something like that. So I have a new branch, which is, I would say, our feature range and afterwards I want to merge my code changes into the master branch or sorry we should say main branch, but I haven't updated the terminology since then. So, okay, let's make some very quick changes to our code.

C

So I'm opening up my ide here and basically you can see. I have version 2 33 and I'm going to say it's the level up our level of power change. So now I'm gonna commit my changes here say changes. I I I like how how it sounds it's funny. So.

C

And then I push my my changes and wait for something to happen magically so as a developer. This is what I do so, let's uh watch what happens uh in here as the the comets get uh you know coming into the repo, so this will trigger some magical things that we are going to talk about in the in the background, and now you see in the github interface, I have some indication that the pipeline as code has been triggered and that it's running a ci cd uh pipeline.

C

So the interesting thing is that directly from openshift I can see the status I can see that it's running etc, and I can also look at the details directly from here. So when it's finished I'm going to have the summary. If I wanted to check the logs directly from my terminal, I could just run this command and oh I'm not logged in so I need to log in there first, but let's go back to the ui and have a look at the pipeline.

C

So so, here from the developer perspective, I can see that it's running it's been triggered recently, it's doing it's doing the build steps etc and just to make it easier for me to to consult to to to see.

C

I have added this notion of pipeline runs into the developer perspective, so just a very quick tip for people who are looking at this so say you are using a concept of, I would say, image stream image stream tag that doesn't exist by default in the navigation bar on the left side, you just say add it to navigation and then on the developer. You have a new item that shows you the things that you that you need.

C

So that's what I did for the pipeline run, and I see now that the pipeline is being executed here. So, uh for the sake of time, let's go back to some of the previously run instances, and actually what happens is once the pipeline has succeeded.

C

The pipeline has called feature from openshift updates github, with the results the build times for each step etc, and I can have a direct link from here to the task. uh If the logs are still there, if they haven't been erased, etc, so you see that directly from the github, I see the status I get directly to the logs, uh etc. So very nice feature. I love it uh well very good job, guys uh it's much easier for developers to trigger the pipelines.

C

I don't have to create my web hook in the repo and select the event etc. So very nice and thank you for making our life if I'm impersonating the developer life easier, um all right. So uh other things uh exist in this uh in this extension, so let's uh go back to how it works.

C

So, basically, within my repository of my application, I have some files that have been defined, so I have the the pipeline as code that you can see here and it basically describes the um steps that are going to to to be executed. You see that I have the first task. That is fetch. Repo second task is build the application to to store the container image and uh last task is going to be deploying the application somewhere. I can see it, but uh yeah deploy the the app so very simple steps uh in my pipeline.

C

Now, whenever I trigger uh an event, so on our case, we were speaking about a pull request right. So the pull request. Sorry, it's not this fine, it's a different one!

C

uh Yes, so I believe it's this one, so the pull request uh in um sorry the pipeline run injects the data that I need for the pipeline to run. Basically, it tells where the repo is the branch I'm going to pull the the code from etc what output etc.

C

So here I've done some static things, but basically, what's very interesting here is on what type of events am I going to be triggered so here in my code, in my application, repo, I have a file that says whenever you intercept a pull request configured for this specific repo and we're gonna, see how we we configured that uh please trigger this ci cd pipeline run and the pull request needs to happen on this specific branch. Okay, you can also trigger on different events like a push.

C

If I'm doing a git push then trigger the pipeline, and if I'm doing, the uh pull request to a qa then trigger a subset of the pipeline. If I'm trigger deploying to production, maybe do something more elaborate, etc. So, uh basically you are defining the the how like uh so the when, which is basically on what type of events are you going to react and you are defining the what is going to happen in the pipeline, so basically that's the magic behind it. You have so there's some kind of structure.

C

We spoke about this notion of structuring your repo. If you want to provide some kind of magic and that's basically, um what we we decided is that you have to to ship a dictum folder inside your repo, and this is what is going to be triggered so uh now before. All of those this magic can happen from the developer perspective.

C

Some other things need to be done in the background and that's basically what we are going to try to cover for the remaining uh time. uh Just before that, some very nice features that I like uh that small element so yeah we've seen that the pipeline has finished everything looks fine. It's all green, all good! You see that it's been run just now. We are not uh like uh smoke doing smoke and mirrors here.

C

It's live stuff and now say I I have published something else or whatever, and I want to retest, basically a specific pr, so I just uh submit the retest command command and the backlash code is going to intersect that and basically it's going to trigger the pipeline again, even if I didn't uh specifically make my commit or whatever so very nice feature, I love it uh again. Shmuel spoke about some more advanced workflow features where you can approve the prs like okay to whatever okay, to test. I think or something like that.

C

So yeah, that's I mean I love it. It's advanced, it's not something that we uh provided with the with the previous uh tooling. We didn't have that. I would say two-way integration where from uh openshift you do some stuff in github, for instance, and then get your results back uh in openshift itself. So it's a two-way uh integration uh and I just love it.

B

So if so, if we go for a moment, uh jafar love the demo. We did have a couple of questions come in that uh that I thought we might want to turn our attention to uh briefly here, and so uh um the first question was: is it possible to deploy the resources to multiple name spaces or are pipelines limited to uh deploy resources in the name space which the pipeline is deployed?.

C

B

So that's a great question.

C

No, no! That's the architect.

A

Of this, no, I don't, I don't mind either way, but uh it's just that, uh maybe something that we didn't spoke about, because it's a bit lower level is that um is that, like when you have uh like the way it works?

A

Is that uh you, you have uh like to be able to say that uh your repo is going to be handled by python, has code on that cluster and on which namespace uh we have what we call like a repository crd, so a resource, a custom resource called people history, and that's going to say that I want to be able like to handle that repo and uh coming so whatever like. There is going to be events coming to openshift pipeline and coming from that repo it's going to be tested in that namespace.

A

So in that case, it's like when you start like you only it's only going to stop on that namespace. But then, if you want to use like to a different namespaces, uh you can have part of your partners code step. You can have like a deployment step or like something else. That's going to be able to deploy another domain space, but then you're only allowed to do that if you're, like some rules, are back rules to say that uh this user is allowed to deploy on that other namespace.

A

Because obviously you don't want to allow to do a to have everything to allow to have access to everything and on the customer.

C

A

Correct that's like, uh and so basically.

C

Yeah, that's that's a mix of um yeah.

C

You are going, for example, to use service accounts, dedicated service accounts to do subsets of your pipelines and whenever so, for example, uh for me uh in a previous demo, I was doing the build in a different namespace, and then I was deploying in this namespace, but just for the sake of simplicity and avoiding to switch back and forth, I put everything in in the same name space, but what you have to do is to to give that service account the correct role to be able to do whatever it needs to do in that namespace.

C

So, for example, if you want to to build and push the image to a specific.

C

Namespace, then you have to give it the edit role or something like that uh or if you want to deploy uh you can you you would need to to give it the specific rights using the the openshift role based access stuff. So.

B

So I think the the answer- the answer, I'm discerning- is that it is yes, uh but it's not something that it's going. It's going to actually require a bit of you, know: user and access control and you're going to want that because to the to the earlier point, you don't want it to be the case that something can simply be be leveraged in the next thing. You know there are things coming into your pipeline that you never expected to have coming into your pipeline right, especially.

A

With events coming from the internet, you really don't like to let everything happen like that and we sold the finals and all that.

D

Yeah, that might be a problem, so we had another question.

B

Too, uh going back a little bit, the uh the deliciously named bacon fork uh was posing the question of what kinds of things can you check for, and this was while we were, you know in the demo, and so we might have to rewind our brains a bit here. uh What kinds of things can you check for, and I I guess I would say it probably depends on the particular point in your pipeline- that you're talking about.

C

Yeah, so I I'm not sure I completely understand the question, but basically you are defining uh in your pipeline, whatever events or whatever tasks you want to to to trigger, and this is uh what is going to be checked within within the pipeline. So if, for some reason a specific task fails so meaning one of your checks, because basically a pipeline is a set of checks and and actions. So I'm checking I'm doing something. Then I'm checking that what I wanted happened if it happened, then I go to the next task.

C

If not, then I fail miserably my pipeline and say: okay, so please go ahead and make changes to whatever so yeah. Each step is going to be, I would say, a check in itself because it's using text on you are going to define uh whatever tasks. You are going to reference tasks that you have built on your own or that you have uh pulled from the public uh hubs and then you you, you are going to use them in your pipelines.

C

B

C

I understood the questions correctly but yeah. That's my honest uh answer to what I understood. They're.

D

Sticking with them.

A

Yeah sorry I was, I was just adding me like. Maybe if you didn't know the question and the question was about uh what kind of events pipeline has code can handle, and uh but okay like- and uh in that case I mean that's, uh that's a good information too, like we handle the events of uh push and put request. So so that's the two events and that's ending and, uh and you can add another on the different uh different branch like, uh like you say: men master whatever and uh release tags uh if you wanted.

A

So if you like to launch a pipeline, uh whenever you push to a release for a tag, then you can handle that events too. So I just like to add to it in case that was what the user was meaning to to ask.

B

All right so scott, you actually had a question um and you know you you are on air. You do get to just speak up and ask them you don't have to type them in the chat. Dude.

D

Indeed, but I I wanted to make sure that I didn't forget to ask it uh and then also you know it's kind of in the middle of jafar's demo and I want to interrupt his flow there.

C

So so, just just sorry, uh I said I was going to show also some of the cool things that they built for the admin side of it. So there's still uh another uh set of demos that I was wanted to show, but just go ahead these and I promise we will come back I'll, try to to to push whatever I can in the demo.

D

B

You're gonna have to talk.

D

Really fast get to ask my questions.

B

D

Yeah go ahead, so you know rolling back to the beginning. um I just have a more rudimentary question about the integration with github. So did you guys set up a web hook so that the operator is triggered by the web hook or does the operator just like do something? Thank.

C

You thank you very much uh scott for for the question, because that's exactly what I was going to show so all right, the decla, the clever guys. So I I hope it's gonna work because I haven't I'm switching to a completely different cluster that doesn't have it installed uh and I hope it's gonna work.

B

A new live demo.

C

Yeah new live demo; let's, let's take our risks again and what could possibly.

B

C

And see if we can make it alive so all.

B

Right well so let's go to the let's go to that demo. uh We did have some more questions and hopefully we can maybe come to them at the end, but I am here.

C

B

C

All right so so first thing um so as a prereq, as I said, uh we need to have the uh the openshift pipeline operator mod and that need needs to happen um from so you can do it for, for example, from the operator, so openshift operators.

C

So I see that we have the pipelines operator that should be installed yeah. So we see that it has been already installed in this cluster, but the pipeline as code shouldn't exist or if it does, we are going to delete it like savagely deleting so I'm on a different cluster right and let's remove it and show you how no that's not what I want to remove.

C

I want to delete pipelines as code and it's not there. So that's good all right.

C

So that's what I thought, because I I had taken care of that now, let's go back to our terminal and we have some nice uh command line called tkn pack and tk impact is for techton, so we have the tecton cli, which is tkn, and we have, I would say, an extension of it called tkn pack for pipeline as code, and one of the options that you have is the bootstrap and the bootstrap is, is uh basically taking care of all of those background things that you said uh that you you mentioned scott and that you need to do as an admin is basically enable the feature on the cluster.

C

So, let's try it and uh hope that schmuel did some good job on on that. So first thing yeah, no pressure, I mean if it doesn't work. That's not my fault right, so I have to mention speaking of that that shmuel has been amazing in in terms of support. So, okay, let's go ahead and install it, and basically we were on slack and I'm trying some some funky things, and I I ping him on slack and say you know uh the application or this stuff doesn't work. He says: okay, please, please remain there.

C

He makes some some some weird changes on his like code. He pushes and and asked me to to try like 20 minutes afterwards and it works so amazing.

B

So basically he is your ci cd pipeline.

C

Yeah, exactly no, I mean he is making changes to the pipeline as code uh plugins.

A

Code publishers could use python as code to do the development, so it goes much faster. The feedback loop.

C

So um yeah we're going to create a github, app called cla app and it says: okay, I have detected that there's a a route that is going to intercept all my events etc and look at the cleverness of these guys. They have created an automated way for us. Oh my god. I have to log into github and I have my two-factor stuff, so yeah, okay create the github app for me and whoa. It's it's there magic!

C

So now I have my github application that has been created for me, and this is basically what is going to intercept all of those uh funky events. So first thing: that's conversation we had yesterday if you are using uh so basically here the the app is here before using it in the repo. Let's have a look at what it some of the things.

C

So yes, it it points back to the uh interceptor that we have on our cluster right and now it says enable ssl verification verification.

C

I am not using trusted certificates on this uh home lab, so yeah, I'm just making sure that I'm, I would say, yeah not using that that setting now, if I go back to the main.

C

C

I can now install the application.

C

And I can choose what repo I want to install it on, so I can say push it to all my repos or only to some specific repos, and then it's going to intercept those events and it's the equivalent of creating all the web hooks for the concerned repos, uh except that it's one single entry point that you create and that can take care of that that magic behind the scenes.

C

So all went well for the uh enablement of the feature on the on the new cluster. And if I check for as code, I should have a new project here, and maybe I have also my running pod in there. So yeah great, the interceptor is working fine and now, if I use that application in a repo, it's going to create the pipelines on this new cluster.

C

Even though I haven't created pipeline definitions on my namespace.

A

D

A

Yeah, just just quickly, because I know we're going to go out of time- is that uh that there is a lot they like. We put strap like the thing like makes. It easy is that uh it makes like it makes a lot of manual steps like automatic. So it's going to create like a secret on the cluster and it's going to manage and uh do a lot of things.

A

uh So that's that's a quick way to get you started so the end goal actually for the as a product so like is to make it part of openshift pipeline, but tk impact bootstrap is like it's mostly like if you want to get started and uh and uh and uh to get started and be able like to to uh to try to try it on, and uh there is as well like a old manual step. That's that's! That's frequency in documentation.

A

If you really wanted like to go that way, and uh hopefully we are working to get the operator. So a lot of the steps are going to be are going to be handled by the operator, but some part of it like the creation of the github apps, which actually we implemented this, because we had a lot of because before we didn't have this, but a lot of people were making uh uh we're, not checking like the right uh uh case like when creating the apps.

A

That would like it's really tedious uh manual process, but uh that's so that's why we we made it uh so it takes like. Yes, I had a lot of reports and people tagged me about it. So, uh but uh nice like it makes like literally, if you want to try on uh pacquiao, you just need to go to the webpage and download the cli ticken pack bootstrap and as long as you have uh openshift cluster and and I github github account, then you'll be set to go.

C

And the pipeline's operator, okay, so.

D

So yeah, let's make sure.

B

We've got the list together here: you're going to need the pipelines operator and then we're also going to need was that package again.

C

Cli, which we can.

B

Yeah, so if we can maybe drop that.

A

B

Chat just so that people know, because I think, there's going to be a lot of interest in uh in being able to do this, uh you know being able to have that single view into this within openshift to manage it within openshift. I think just the more I think about it, the more beneficial it starts to appear, so uh we did have a couple of other quick questions. um First of all, as as always uh you know, there's the question about what are the resource requirements to implement this?

B

Is it going to be part of my same name, space or cluster.

A

So that's the the beauty of it is that there's no there's no resource requirement extra. I mean uh I mean the.

B

Way we design the incremental there's no incremental resource requirement beyond the open shift deployment. You already have.

A

That's correct, so open pipeline as code is a is, is implemented as openshift pipeline. So there is no demands they're all like using like internal components from openshift pipeline, which is a triggers component uh task pipeline. So uh so you don't have like you.

A

Don't need like to have uh to uh to have like a special like deployments, and it's going to like demons and things, and so it is that uh it's only going to react to events coming from outside and it's going to use like uh what we call an event listener like a tecton trigger image listener, which is built in inside a bench pipeline to be able to handle those events. So there is no extra requirement uh resource. We can do it.

B

Great, and so uh I think another question that we had was: uh can you share with us a holistic view of each stage of the pipeline, and so I'm wondering, if perhaps we have uh you know a link or web resource that we can point someone towards.

C

Yeah, so I would say so so sorry to phrase it phrase it that way, because I'm not a native english speaker, but I don't think the content of the pipeline itself is meaningful, because basically, I've done a very simple pipeline to show like build uh like uh build. The code, uh build the container image and deploy so basically very, very simple stuff.

C

But we have a lot of other more sophisticated and more meaningful text and pipelines that we could show you, maybe if it makes sense as another episode where basically we do all the stuff like ci, we use sonar. We use uh unit tests, we use integration tests, we use code coverage and you know all of those funky stuff and security code scanning, and so.

B

They're, do it I mean basically, there could be a lot of different stages. That and that's the beauty of this is that you are, with code literally defining what all of those stages might be, and it might be that you have a simple, a very simple, naive, one-stage model. uh You know for the purpose of a.

D

B

Or something a bit more than that or you could actually have something that has quite a few different stages, because it's required by your particular app or governance, or you know, compliance whatever kinds of things might be in the picture that need to go through a much more staged process. So I I guess it's in a sense. The answer to that question is: is the holistic view is whatever you choose to actually build as you're playing.

B

Very cool stuff: well, uh let's see do we have any other questions or closing comments. It looks like you.

A

Just need to add uh something is that about uh openshift console. I don't think java had a chance to show it uh because uh it's part of openshift49, so on openshift pro 9 on the openshift console, we have an integration like a visual integration of the the repo histories like of batman haskell. So basically uh you you, if you have like batman, has got installed on openshift 409 you'll get a.