keptn User Groups, 19 Apr 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Cloud native Delivery with Keptn. Breaking down your monolithic Jenkins Pipeline

Description

Jenkins is one of the most powerful automation tools while Keptn is the emerging control-plane for cloud native delivery, Day 1 & Day 2 operations orchestration. In this user group we have Rene Forstner from NTT Data walking us through the benefits and steps to breaking down complex monolithic Jenkins pipelines into individual orchestrate-able components. Rene shows us how this reduces complexity of your custom scripts while leveling up your pipelines with automated observability-driven orchestration.

The event will be hosted by Oleg Nenashev who is a long-time Jenkins contributor and maintainer. It's going to be a great opportunity to discuss anything Keptn and Jenkins, join us!

RSVP here: https://community.cncf.io/events/details/cncf-keptn-community-presents-cloud-native-delivery-with-keptn-breaking-down-your-monolithic-jenkins-pipeline/

A

Are going live on youtube.

A

A

So yeah we should be ready to go so thanks. Everyone uh who joined us- um and uh I just want to to introduce our next uh user group session. So today we talk about captain and jacob's interactor ability, especially about cloud native delivery. So rene from ndt deity is one of active captain users. Rene has already presented once at captain user group, and today we will have a really nice presentation about what you can do when you combine two powerful tools together.

A

So during the presentation, if you have any questions so be streaming on youtube, there is a chat, use this chat to ask any questions and for the rest, we also join captain on slack. You can ask questions today, anytime, and we will be happy to help you, okay, so that's it from me and yeah. Remember the floor. Is yours.

B

Cool, thank you very much olek for having me again for today's skip music group, as has already introduced um today. We will talk about um captain and jenkins, so it's glad to have you here as well, because you're way more experienced with jenkins than I am because of your past. So I'm glad that you'll have some some cool user group today.

B

So what do we would like to talk about today?

B

So what our key takeaways is quite simple, um we'll just take a look, a simple look at how we integrate cap nut system type jenkins problems at all and how we can let captain or history choose pipelines, and we will learn how we can reduce complexity within our jenkins problems to those integrations and even just putting some improvements, because when we're already touching our patterns and configuring, some some captain stuff within our jenkins will improve our patterns with, with captain triggered quality gates just out of the box with with the functionalities we have.

B

So this is what we would like to talk about the next case, 30 to 40 minutes or something like this, and um hopefully there is something for everyone of you to to get out of it. So before we start one one picture of our legacy pipelines or why we did this.

B

As you can see here in the past, we had some some kind of chained pipelines where we have multiple pipelines in place and each problem is triggering each other pipeline and we've pretty much set up everything within jenkins, so from building the container from building the software itself, building the container pushing into a registry and doing the deployment uh triggering tests and stuff like this, have it all done within one single pipeline and then changed to another pipeline, getting into a staging environment and then again, chained because kind of a daisy chain to get it deployed to our production environment and the the issues we had here, or that we had a lot of issues and a lot of problems.

B

This is why we mentioned or where we moved to to let captain orchestrate our pipelines, because it was quite hard to troubleshoot. I have some some shortlist why we did this in um after the presentation and we will walk through it in the in the presentation.

B

This is already my almost my last slide because I love doing it within the tools and I do not want to waste the time or get to sleep with just doing some. Some um slide show so that this last slide is. This is our current architecture. So what we've built? It's quite simple: we have um build up several very smaller pipelines with our build button, which is just building our software, and this build pipeline. If you want to, is already triggering captain. So what happens then? Is that we will trigger our captain delivery.

A

B

And captain then fully orchestrates our deployment workflow, our delivery, workflow from the deployments to any given environment is its death environment or staging or production environment.

B

Captain will trigger our test problem, which runs some tests and captain will uh evaluate our test results in pentacles with uh it should trigger the same sequences for any other stage like going from depth to staging going from staging to production and so on. So this is what we will look into today. We will look on both ends. We will look on jenkins and you will look on capital what we have configured and how this is working and we will start with um looking into jenkins.

B

So if you have never integrated kept into jenkins, it's quite simple: there is a library available from um from the captain open source project, the captain, jenkins library, um so sorry for being a little confused. There is within the slide that I can show it to you and we will post it in the chat afterwards as well. There is a link with your reboot, which has all the urls all the libraries and all the repos.

B

You are seeing within this demo collected in one single repo, and you will share the link to this repo within the chat and it's also within the slide deck available. um So.

A

Basically, there.

B

Is a jenkins library available for for a cap library available for jenkins and it's quite a simple and straightforward configuration to get started with captain jenkins um for not sure where my champions right now is that slow?

B

So, basically, you have to configure some environment variables. Of course you can put the token in um in the secrets as well. This is just for testing purposes and demon purposes. I've just configured this in an environment variable we need to configure our captain bridge and our captain endpoint, which takes all our cloud events and um for the communication, and we will configure our captain library just configured as a global python library. You could, of course, configure this directly within your pipelines as well.

B

Therefore, just configured for dedicated libraries, um I will just configure our capital library in my demo. In my case, this points to my personal jenkins reaper, because I did some changes on the captain champions library for testing purposes. There's a currently pull request open. So this will be available. I guess within the next few days you can the public repo, of course as well. So this is not that much configuration we need to do within jenkins to get just jenkins integrated with captain and now.

A

Yeah renault sorry for interrupting here there is a report that on youtube site is quite blurry. If you could zoom in a bit so that it's more visible, it would be great.

A

So uh yes, so so just uh make the fonts bigger so that everyone.

B

A

B

So no check this is this is getting recorded as well. If this is just a live stream, so we'll just show the screens again for a short of time, and if it's recorded, you can click on pause of course afterwards. So we need to configure our environment variables for the bridge and the captain endpoint and as well for a captain api token. This could be a secret as well for demo. I have it here configured directly and we need to configure uh pipeline libraries.

B

I've done it for a global pipeline libraries that I have it available in all of my pipelines. Of course you can configure this directly in your pipelines as well, or we just configure a library for for captain integration and point towards the repository of the the official captain jenkins library, which is um public available github. So this is all the basic configuration we need to do.

B

The more interesting stuff, I guess is within the patterns itself, so again, assuming a little um so I've already mentioned that we started with having one large pipeline. This is my rachel jenkins file and then we have some build some smaller pipelines for the integration for getting this stuff integrated together, and we have just quite some easy steps and some some good going steps within our problem to just trigger captain. So basically, what.

A

We're doing is.

B

A plane we always trigger some events: either we trigger it out of jenkins to get captain start something, or this is when we look into our capital configuration captain is triggering an event um which is then pushed by a red book towards jenkins and jenkins system. Just answering our triggered event. So it's quite simple: we send a deliver delivery triggered event. So what we want to do is for our project workshop. This is when we will switch to captain.

B

We have a service cards and we will trigger our delivery for the image this pipeline just has built. So if you take a look at jenkins and take a look at our build pipeline, so we have the cars built by plane.

B

Which has just some some few steps, as you can see just just before we started the meeting. I screwed up my pipeline a little so, but I was able to fix it, and so we have quite a small pipeline where we just build our software push it to registry, and then we will trigger the captain delivery sequence for having captain um delivering our our software to whatever stage we have configured here.

B

So if you take a look at captain right now, I will start with taking a look into the shipyard file, um which is basically just telling which sequences we want to have for which stages. So it's quite some straightforward shipyard, as you might see it often doesn't matter from themes or production environment. We have three stages: development, staging and the production stage, and within our stages we have just a delivery sequence right now, which is um consists of a few tasks like a deployment task and a test and an evaluation task.

B

We do pretty much the same for each and every stage so for for death, we don't need to do the same for staging and to do the same for production. There is just one difference and because for the depth stage we do not have any trigger on event, so we somehow need to trigger the death stage. This is what's done within our jenkins bar plan, where we sent the delivered trigger event, so we do this for our dev stage and our staging and our production stage are triggered once the previous stage is um was successful.

B

So this is quite a simple, straightforward, checkpoint file and if you take a look into um captain, so just have a managed environment. So usually this is blue from it's somehow green or crackers, and we can see here the deliveries and the stages.

B

So what I will do now for demon to prove it's working, I will just trigger a sequence, and then we just follow up how it works through for jenkins and just started the build and usually, as you can see it shouldn't, take that long getting the captain delivery sequence kicked off.

B

A few minutes ago, therefore, it's a little bit slow but better slow than nothing.

A

A

Is it the usual execution speed or does it get situated by uh just local the environment.

B

I think it's in in general, it's running a little faster, quite a small kubernetes cluster, because over all of the stuff contain the rest. So my jenkins is containerized jenkins will spawn my build containers, and so, if, if I'm not running it frequently, my build containers are not running, so it takes some time to getting the containers up and running, but it's already so the build um is success. So you can see here if I'm just going for for my console output of my logo.

B

So here is the last step which is reflecting the build we push the stuff to my registry and then we can see that kept is triggering the sequence.

B

So this is where we can see what communication happens with and even pointing directly to captain, and if we're going to captain, we can now see the delivery sequence of cards has started, so you can see. um What's was reflected in the shipyard. So this is our delivery segments.

B

We are starting with the deployment, and this was that fast, the deployment that this seems to be already done, because we already see all the test sequence, and this is what gets quite interesting, because the deployment sequence was triggered by um by captain torx jenkins and how is this working um also quite a cool stuff with captain, because within the uniform you have a web hook service where you can register various web hooks, and this is all the magic it takes.

B

So what we have done is that we just created some some subscriptions for webhooks for our different tasks. We have defined in our shipyard so like a test, triggered a deployment triggered, and this is something for something different for for getting another demo. So um with the web box, you can do some quite some granular configuration for which captain event and right now we're doing just some some standard deployment with test and delivery and stuff like this.

B

Basically, you could name your event whatever you want to and then just subscribe to that book to your event, give some meet information like I have done for this project sock shop and for which stages I wanted to subscribe this event and if you take a look at our web hook, it's quite simple.

B

We just call in my case is a post because of of uh multi-branch pipelines, but you can you can just get your champions, so just um put a get request in there and you just build just trigger that jenkins bubble with some metadata. Like what stage do we want which test strategy in this case? Do we want to have in some kept needed data?

B

And if you take a look at jenkins again, we should now see our deployment problem is still running already finished. So it's already finished, so our deployment by plan was just triggered by captain forty-swipe. So this is the quite cool stuff because we have just defined our sequence within the shipyard or we have our tasks and we can captain. We just say, for which task do we want to trigger which jenkins pipeline- and this is the problem which just ran a few seconds or a few minutes ago?

B

Where are we? Let's take a look at the console output again, where we're just deploying our stuff to a given namespace and then send back the finish event to captain once our deployment is finishing captain then.

B

um You see the deployment event is finished at screen, then we will trigger test event the test event itself. We have a web book subscribe to it and the web hook is starting another jenkins pipeline. It's our test button. There are some chain meter tests running it's a load pipeline and then we will trigger our installation.

B

So you can take a look at our deploy pipeline. You can see again it's quite simple- to have jenkins out of captain integrated, doing some some deployment stuffs on the plumbing configuration and, at the end we just trigger again captain until captain that um we have successfully deployed our software and we want you to put the stuff to the next stage.

B

Just also take a look into the jenkins file, how this is configured within jenkins, so just a jenkins file deploy pipeline in my case where we can see the parameters again so the stage the captain contacts the trigger at the end and image id. Those are all the information we just gather from captain through already available metadata, then send a finish event to captain.

B

Tell captain it's quite simple and the finish event of the events deployment, getting the captain context and the trigger id that we can find in sequence. Again we just put in some meter data and it's also vertical, so our deployments are to you or whatever media data you want captain.

B

If you take a look at the deployment event, we see the labels we have um put in within champions and also, if you take a look at the meter data here at the event payload, and we can see our customized event payload, which we submitted for jenkins.

B

So this is quite some some cool features and great feature, because if you need this data in any other system in any other environment, in any other tools, you can just put it in your payload of your deployment or whatever event, and then you can access it on captain with some use some variables.

B

So this was the deployment. Then we can already have started a test event which is triggering another pipeline, and then we have in our case it's regulated with dynamics, and then we have uh implemented a quality gate with some evaluations.

B

We can look at those evaluations on how this build was going. You can see there is something yellow, because we have got some some issues, but still not red, so we're going for our next stage- and this is all- and this is the cool thing here, because we take again the uniform.

B

um We have just looked at the deployment trigger event which triggered our deployment pipeline. Then we have triggered our so we have triggered the test. We have subscribed to the test trigger event and we just trigger our testing pipeline. So it's basically almost the same web hook, looking at our jenkins or just triggering the test problem again with some metadata, whatever we want to. So we can add a lot of metadata here from the previous steps. What was in the payload there and triggering again the pipelines?

B

So if you not take a look at our test problem, this is already finished as well, and we can just see that it just.

A

B

Some chain meter tests- and this is quite some- some important topic, because there are multiple different features within the jenkins powerpoint, because you can just trigger a captain quality gate out of the pipeline. But what we did is that you just again send the finish given to captain and let captain orchestrate all the other stuff.

B

So it's quite some simple. We will look into this step in a second, I will just run some simple chain meter tests.

B

Let's take a look at the problem: script itself. We just run some jmeter tests and getting the event finished back to captions. So again, our parameters just some stage, some some kept information and we didn't change that much within our popular. We just added two steps. One of them is quite interesting because we will always update our slis and slos so we'll we can look into this if this is required and never just have to find what captain should evaluate.

B

So we just define our matrix, which should be should be monitored and evaluated, and we just define our objectives so what criteria we need to knit. So we just add those resources together. So if we change something because there needs to be a change, we easily add those resources. Those update resources are ready to up for our pipeline. We will execute our virtual meter tests and afterwards, just like we see in the deployment pipeline, we will send a finished event this time.

B

It's a test event and not a deployment event with the payload we have here so with for for the test step, our payload, which is required for the next step, is a start in the stop start and end time frame that kept disable and triggering the quality gate to say, which time period do we need to check? When did the test take place, to get the quality get triggered properly and get the evaluation done for the for the respective time frame?

B

So if we take a look at our tests appear, you can see the workbook service started. This was our subscription and you can see uh once it's done and if you take a look again at the payload here, we can see our customers payload, which we put in the jenkins with the start and the end time frame and we hand this over to our evaluation phase, where this is some technical new features where we just evaluate our defined slice and slos, and once two stages is finished.

B

As you remember, our shipyard, the staging stage will be triggered once the delivery is finished. We are going to the staging doing exactly the same system, 100 evaluation and once this is triggered, we are going for production and within the production of also some delivery. We can actually only have the deployment step triggered by um within the delivery sequence, and then we do or we start an evaluation. Five minutes after deployment has finished to get checked. If there are some issues within our deployments. This is nothing related to jenkins anymore.

B

Basically, you could bring in some some kind of rollbacks.

A

Sorry so, basically, in this case, uh you just uh do follow up evaluation for deployment. But what happens uh if this evaluation fails? uh Can you fall back uh with from this pipeline.

B

um So there is, there would be a different or basically, we could fall back from um with the patterns as well, because we have somehow parametrized it and can can work with the image ids. So this is something which would work in this case because we can just again um with if you define in our shipyard. I think I have to find it yeah.

B

If you define a rollback sequence or a ruleback task, um you can just again register webhook and or subscriber web hook with a given pipeline, and once the captain is triggering the rollback task due to the failed um evaluation which we have here, then you can just kick off your pipeline having the image at the in place having the image in the meditator and roll back to the previous version. This is this is some use case which should work. It's currently not implemented, and this team would be harnessed.

B

But basically this is some some use case, which will work and taking a look at our.

B

Small slice deck um so from becoming from going from here so have all within jenkins, and all days are chained to have it orchestrated by captain.

B

There might be some some questions why we should use this or why you should do this because in first place it might look a little bit overhead, but um just within our example, we reduced lines of code within the jenkins file by more than 30 percent.

B

This is because we had a lot of redundancies, because we had a lot of redundant pipelines and a lot of redundant patterns which, having the deployment of staging, which almost looks similar like the deployment to death or to production, but with some different parameters and there's some different stuff and just having this orchestrated with cap, which is okay, we won't build powerplant, which builds our our piece of software.

B

We have one deploy pipeline and just with a few parameter and triggered by captain, we tell what we need to deploy where and when and the same for testing. Of course. So we used a lot of code. We reduced a lot of complexity within our pipelines so just to to get rid of redundancies within code and, of course, as we're already dealing with um adapting our jenkins buttons was quite easy to build in quality gates. In the past, we had some some performance for things.

B

I can show it, of course, but just usually, as this is um for sure some, but some some common use case where we just compare manually or just compare with some high thresholds here, our builds and.

B

And um doing some manual performance analysis, we even managed to increase the quality by just implementing quality kits with just one easy line, because we already know that. Are we already triggering captain or sent back to finish the captain? So we just trigger the quality gate. We can keep just updating the slides and the solutions in the background, so it was just one single one, easy step for stuff which was done previously manually and the last.

B

But not least, thing is- and this is quite some important- um is ease up the integration with any other tooling. So because, of course, jenkins is quite powerful and there are, I assume there is almost no plugin which is not available for integrating stuff within jenkins, but with captain or with with the stuff we have built- it's quite easy, for example to just for the deployment just unsubscribe to web hook and let, for example, helm, do the deployment and then again working with the desktops with jenkins.

B

So it's quite easy with having this orchestration in place that we decide. Okay, the delivery sequence is triggered through our champions battles, because we still want changes to build our software and push it to any registry into the trigger or by our delivery sequence and for testing purposes or just for production.

B

We want to to move the deployment to help and not get deployed or get our stuff applied for jenkins, or we move our test sequence, our test tasks to any other tooling, so we just um deactivate the web book towards jenkins and point it to any other tools or with the out of the box integration of captain. So with having this orchestration in place, it's for us quite easy to exchange some building blocks with some other two links and jenkins. This is within the previous.

B

It was quite hard or to get the right, plugin working or to work with with curling and triggering post and stuff like this, and then some are trying to validate the result. This is not all done by captain and we can easily replace some some single steps.

A

Yeah speaking of tools, uh there is a question from the audience, so the geometer uh evaluation you run in the demo. How does it happen? uh Do you use gmeter service for captain, or is it just a part of jenkins execution flow? In this case, it's just.

B

A part of the jenkins execution, so this is really built of a meter container virgin meter working node for for my jenkins problem and have just some some common tests defined between- and this is. This is running slowly with with jenkins and germany itself, so we do not bring any information about the test or the test result itself to captain. You just tell captain hey captain. We have finished our test event. We have started it's time x y set and we have ended at time on abc, please validate or please go to the next phase.

B

We just say we finish our test event and captain as it orchestrates.

B

Our our sequence is the verb that, once the test phase is done, we will go further with the evaluation step and in our case our services are monitored with time address because, of course, can monitor me files and let the evaluation against data output promethouse, but we do not put any recorded data from from chain meter um towards captain. This is slowly done with the monitoring with the observability of our services and we just trigger the evaluation sequence.

B

With the information we have collected during our previous steps, like the deployment stuff.

B

The configured sls and slos- and it goes with our test time frame yeah, which timer should be related on.

A

The junk inside do you use plugins, like performance, plugin or maybe x unit plugin, because yeah they could be used to unify this uh advantage. Duration flow. If there is a pipeline library.

B

um Yeah, basically yeah, we are we're using the performance, signature, plugin.

A

B

We have used it in in the past and have almost manually compared our pills, so this is still here. The performance report is still available because I've just not excluded it from a pipeline, so it's still available, but to be honest, we don't use it anymore, escape evolutes, already everything and also some some something I guess which is on the plus side, because you do not need to get rid of this.

B

So you can have this in parallel in common and still whether they did on your own still look at your performance reports and there you of course have a comparison as well, so just cut some basic stuff with the previous build, and um you still can have this in place and still use captain in advance to get this validated automatically.

A

Yeah, that's nice, and uh so, when uh preparing to send the event, uh how do you extract the data on the jenkins site? Do you suggest dtepa.

A

Sorry, so how do you extract data from jenkins, so basically, this report to send it uh back to captain? Do you use a custom groovy pipeline or do you access this data through rest, api.

B

um Basically, this is currently this is not pushed or captain at all right now. I might think it is okay. You could, but I'm not sure about this. um You need to look into this, but basically you could add something like this, where you just put in the cabinetry source to get the files towards the captain, repo or- and this is um also always possible- is put it in the event page, for example.

B

So if you just have some some kind of a raise or some kind of information, you always could put this into your event payload and which is then, if you take a look again in jenkins, so my simple: it's.

B

So if you want to put in some some additional information here, we can put this in here and if you take a look at the web book service, for example, and just going for this one.

B

To show what you can do here, and here you can then access all the data which is a radio so like there was a data, there is a deployment. Then we have um some labels. We have some. Also, all the information which is somewhere defined within within your payload or within your data can be accessed again through um within captain, and you can put this in again within all that you want to trigger. So if you want to create another way for forgetting stuff, you waited or something like this.

B

For example, you then just can put in your data here within within the the url or the configuration. This is not only working for netbooks, but somehow the most common case.

A

A

Rob uh john mentioned multiple times in the chat that he would also be happy to see a webhook integration with jenkins for common cases, because, yes, we standardized jenkins api. It would be really nice if you could use a rest api, uh so you basically throw web hooks, etc and for jenkins. For some cases we also generate basically open api or swagger like you prefer, and it means that if we do some ux improvements uh on the on the captain side, we could actually easily integrate through web hooks.

B

A

B

I really love the backpack concept, which is in place right now, because it's first of all is of course fast and it's so easy to adapt and so easy to configure with just saying. Okay, for my definite, my staging environment, I still want my deployment to be done through jenkins, but from a production environment.

B

I would like to trigger another bay poke or to do the deployment for helm, for example, because I have some some canary deployments and they do not want um to to get rid of or to get somehow screwed up with the with the files. I can just do it for helm with reproduction, so this is quite cool and because what we are doing during the deployment- it's quite simple it just it's all running on on kubernetes.

B

So what we are doing is not only simply almost some kind of stupid, because I just set some values. So just going for my channels, which I want to have deployed and replace some placeholders, we place some some variables and then do a simple cubesat apply. So this is basically not the best deployment for demo case. It's it's quite cool, but this is the case where it's just it seems quite easy, because this is changing the summer oversized for just doing these three lines of code to get something deployed.

B

There is a better choice, but if I want to test this- or if I just do not want to get completely rid of jenkins, because I've already done so much stuff, it's quite easy with captain and with the orchestration to decide what steps I want to move from changing so any other tool. Some won't move within jenkins from one person to another, so just creating new additional pipelines and then move step by step towards captain orchestration. This is quite easy with the web book service, which is currently available with the captain.

B

So I really love the concept of doing this, and for us it was a huge impact. All for these three points do not look that great, but reduce the lines of code reduced the time of troubleshooting, because our pipelines failed quite often because of just some typos or some some abilities, or we not we're that good with with coding our pipelines error-prone.

B

So it helped a lot also the quality gates, of course, some some quick win or some easy win, and most of the integration with any other tool to play around to show some some use cases and just getting some um some more stale configuration within our within our environments.

B

You can take a look, I'm not sure if how much bills are still in there. But if you take a look at my old pipeline, which was so basically, you need to say that this is a demo applications on production. So we can destroy anything here. um But if you take a look at my demo pipeline, you can see how often it fails for the place where it's somehow when screen and then we just have some timeouts or there.

B

We have some reasons: they've airported, and we have this daisy chain where we need to deploy the staging out of of this pipeline. So we need to go to our next, but when we need to check when there feels something, what is what has failed and why it has failed and from the staging bubble we might then deploy or return might and trigger the deploy pipeline.

B

So it's um always. It was always hard to follow up what jenkins doesn't and what we are doing here right now. This is way easier with having just the orchestration full captain and just having my three stages and having some some clear sequence. You can just see when they feel something what failed what happened.

B

So it's really helped us a lot in getting stuff done faster.

A

Yeah, so the main pre-condition for such architecture is not passing data between stages on the drinking side, which already makes it a kind more manageable.

B

A

Don't have a context and all context you have is either stored in artifacts uh on a captain database and you can easily restart stages, etc. So there is a pending proposal from johannes to have automatic, restart and ui for restarting stages in captain and yeah.

B

So basically, my my jenkins problems are, I would say um they were stateful, so you need to remember some of the state in order to get all the sequences triggered correctly and now they're, not stateful anymore. So it's just some sequences, just some plain stupid pipelines which are just doing some stuff and then they're closed again and if they fail in the middle, there is nothing to troubleshoot anymore, because I can just trigger the dedicated sequences, dedicated tasks and just start where I want to start and not always work for the complete pipeline again.

B

So it's it's really cool and, as you can see within the jenkins file, it's quite um it's a one-liner more or less to get.

A

To get this integrated.

B

With just one simple same finished event, then captain will take care of what to do next and I do not need to take care of this within my jenkins files anymore.

A

Yeah, that's really cool um one question from the chat: do you use uh monika, monitor ranker's code in your environment.

B

um Yeah, I'm basically I'm not using to use it in this environment. I'm I'm not sure, don't use it in this environment, but technically I'm using it quite often, but I'm also using it. This is not sure if this is the the right place here, I'm also using terraform for configuring dynamics, because um so because dyna because terraform is the one for me, is one of the most important and plus on on the terraform side. Is that it's stateful? So I can destroy my configuration.

B

I can update my configuration easily and I'm aware of what's configured right now and also I can integrate it into pretty much everything else or anything else because of just the standard of terraform, but this is somewhat up to the to the use case. Of course, um there are a lot of places where monaco's, which way better tear terrapin.

A

So the question was coming: basically, is whether you have an example when kept and jenkins geometer and monaco are used together. So basically, when you can manage everything as code, I do not have such a demo but yeah. Maybe we can follow up later and finally,.

B

A

So please ping us on captain slack and we will try to find something. I guess we already have it uh somewhere from monaco or in captain tutorials.

B

A

And a question for me: so uh what other integrations do you currently like in jenkins or in captain, uh to make it uh more efficient for your use cases.

B

What's this a question for smear towards the chat.

A

No, it's it's a question from me. So basically, what would you like to see in captain or in jenkins to make your software delivery pipelines even better, even more manageable, whatever whatever so any wish list.

B

This is quite a hard question for me, because I've um um implemented already my own jenkins libraries and just what would have uh yeah basically I've adapted this and finished event, but there is already pool just open um what legs and depression.

B

Hopefully, someone from the chat is already answering your question because I'm still thinking about what was what was the hardest part in the last few days and.

A

Yeah, if uh please feel free to bring up these comments in the chat, I know that the rob is there. So probably we will get a list of 100 services he would like to see, preferably yesterday but yeah. This is how the community works.

A

A

Actually, there is a question uh from rob.

A

So you have the cloud event status and result basically hard coded. So have you done any logic or would it be an enhancement idea to translate jenkins status to cloud event, values automatically.

B

um Basically, this should be yeah. This is hardcoded in my so result and status is hardcoded in my in my my team environment.

B

Basically, this should be already possible at least the status, because no, it's not possible right now, because the status success is only done once. The problem is completely finished, and this is one of our pattern steps. We do not have this status available right now um so yeah, but this would be.

A

Some interesting.

B

Thing I'm implementing the same finished event in a postpartum step, which runs always use. The button fails having this in place at the post step, because this is one issue I have right now if my problem still fails or something for some reason, um captain is not aware of it, so this is where you can see some some abroad cases. This is where my deployment just stuck because jenkins failed, or our tank and scratched for whatever reason and within the web hook services.

B

I always wait for my service to response, so this is something you can configure within the web book services and for for changing software with it. It's important that you mark send finish event by the web hook, receiver and in case the web procedure is not sending anything. um Your sequence will stack not sure if, if it will start in in an unfittingly state- or this will somewhat- but I've reported this manually.

B

So this is something I need to update my demos or we need to think about um to have some kind of best practices where we implement a post action within our public, which all this trigger kept. This and finishing, and then we always already have the status of the problem we could put in some actual status and get the.

A

B

Proceeded because right now, I need to import it manually if my pipelines fail for some reason.

A

Yeah, maybe that is actually alternative way, because it's true that jenkins doesn't set the status right away through api. At the same time, you can access status for particular stages and even steps through api. So this how it gets visualized, for example, in blockchain in pipeline graph, and the results are open. Telemetry plugin, which sends information about your pipeline execution, basically has open telemetry traces.

A

So what could be done today? In theory, instead of relying on event sent to captain directly? We could uh access data submitted to monitoring system at the moment, and also there is a project called cd events which actually intends unifying different events for csd systems and rankings is one of the prospective adapters.

A

We actually have already cloud events plugin, so maybe through this plugin, we could provide more observability in what happens on the jenkins side and propagating the captain in unified standards so that we can process it and without much magic logic on required, requiring development.

A

Technically in jenkins, maybe uh well, the was the desired approach, uh but if you develop us uh your pipeline's encrypted pipeline, there is also try catch block. So in theory, if anything happens, you can use, try catch and catch will catch anything including syntax errors uh in your pipeline.

A

uh So maybe it's one of the ways to send a captain event in the chip mode, because you just catch a failure, send it back to captain and then exit from the pipeline.

B

Yeah this disorpers and cases you can see here I mostly use or mostly script blocks, which is basically not the best case. You can do because this will um do not throw any error or not not any error, but you will of course, see a lot of errors but yeah. This is. This is some great approach.

A

I have to admit that I'm still a fan of scripted pipeline.

A

It's a topic for hollywood in the jinx community.

A

uh But yeah uh in principle, you can do that and in uh scripted pipeline you can also have handlers uh for failing events, so you can also integrate uh some fallback logic.

A

Okay, so it looks like a really nice pipeline and yeah thanks a lot for demonstrating today, because um yeah there are many integrations between captain and jenkins and basically, you either use them as quality gears, whether you trigger, let's say captain from jenkins to evaluate quality gate, and in this case we have opposite integration when captain is actually used as main orchestrator, and we have jenkins rather is automation, engine that automates particular pipelines and particular steps in the main pipeline, while jenkins does sort of while captain does the most of orchestration and benefits from its event driven nature and all the metrics we receive.

A

So both ways of these integrations are totally legit depends on your use cases, but it's really cool to see this integration, because this is what is not always preserved uh presented. So many people go with opposite direction because it allows to reuse the pipelines as much as possible, and here it tends to see that actually adopting another way simplifies and improves your pipelines.

B

Yep, so if there is, if there are any additional questions or additional topics, we want to discuss just drop a message within like just trying to get this like channel, where all I get myself are also available. Drop us a hint. If you can do anything else for you or you need to discuss further. I've already learned again a lot by myself, while improving my pipelines for for some resilience and getting their rank back to captain.

B

In any case, I will do an update on my demos and hopefully we'll be able to somehow present this again with an updated and even better solution.

A

B

And what yeah? I'm just sorry, what rob just mentioned from the jet some some right updated the pass in in the in the end. Of course, we've just done this demo with jenkins, because there's a library within jenkins as well available, but basically it doesn't matter if from a kept perspective, if you have jenkins on the other end or if you have any other ct tooling, um which can be triggered by a web hook, you can of course do this with any other tool as well.

A

Yeah, it's pretty nice and yep. I think that uh it's a very legit way of integration and uh for me, yeah, I've been using the jenkins for ages and I still believe that jenkins is one of the best automation tools but for event, driven orchestration yeah this somewhere. When I wanted to use captain so and yeah, there is no native way to do it in junkies. Unfortunately, so I think that such combination could be nice.

A

We also have a google summer of code project, I'm not sure whether it happens this year or not, which would integrate uh jenkins pipelines as a part of uh captain execution plane. So there is a project called jinx file, runner which basically packages chang's pipeline executor as a docker image, and you can execute it anywhere and the idea that you can take a captain job executor service, put jenkins pipeline execution right there.

A

If you don't need the jenkins uis et cetera, it's in gt case and then basically, you could do similar pipeline, but by mostly using captain capabilities and by not having fixed the junkies instance, which also can protect you from some mistakes from some run-time issues, and I guess in the case of this basically stateless junkies pipelines.

A

It would be quite helpful.

A

So yeah it's maybe a bit more short right now, but uh well it's one of my topics for a hackathon for me. If we don't have a google summer of code project.

B

A

Maybe we will have, let's see, actually the deadline for jsoc applications is just in five minutes. So soon we will know what projects we might have this year.

A

Yeah, so going back uh to the presentation, uh so thanks uh a lot and yeah rob says that if you have more ideas, how to do web hook integrations, etc. uh Please contact him because he would be happy to help his uh having this integration and guidelines listed on the artifact hub and yeah. I think it would be beneficial for everyone to have this opportunities.

A

Cool okay, so thank you and your closing course.

B

Thank you very much for having me.

A

Thank you. uh We are always looking for news pickers and yet, if you use captain, uh please feel free to join us. Any presentation, uh no slides needed uh just any use case, your success story, or maybe not so success story with a lot of deep dive and, if needed, we'll make sure to bring captain maintainers to there so that you can say everything you think what needs to be improved in captain. So any feedback is appreciated.

A

Yeah thanks everyone.

B

You very much guys have a nice evening, wherever you're located.

A

Yeah, so I'll stop the broadcast. If you could uh stay for a few minutes renee. I would appreciate that. Okay, thanks that everyone who watched it I'll see you in one month.

B