keptn Keptn Tutorials, 22 Apr 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Tutorial - Keptn Quality Gates with Prometheus

Description

What we will cover in this tutorial:
- Setup a demo with Prometheus and Keptn
- Trigger Keptn Quality Gate evaluations
- Manage SLOs via the Keptn

Find the tutorial here: https://tutorials.keptn.sh/tutorials/keptn-quality-gates-prometheus-08/

Learn more: https://keptn.sh
Get started with tutorials: https://tutorials.keptn.sh
Join us in Slack: https://slack.keptn.sh
Star us on Github: https://github.com/keptn/keptn
Follow us on Twitter: https://twitter.com/keptnProject
Join our Keptn community newsletter: https://keptn.sh/community/newsletter/

A

Hi everyone and welcome to this video on captain qualitygate with promethos. This video is based on a tutorial that you will find on tutorials.captain.sh, and you will also find the link to this exact tutorial in the video description.

A

What we're going to do today is to install captain quality gates and have a demo application up and running, hit it with some load and then execute the captain quality gate.

A

We will experiment with different versions of this demo application and we will take a look at how the captain quality is evaluating based on the different versions and which version we should promote to production and which version we should definitely not promote to production everything I'm going to show you is prepared in a demo installation for you and you can just execute the stem installation on the vm.

A

I will do this. I will start the demo installation right now in the background, and while this is going to be installed, I will explain you what is actually happening, so the tutorial itself is available as set as on tutorials.captain.sh, and the tutorial is basically explaining how you can use the character quality gates.

A

uh If you already have some kind of deployment or delivery workflow, but you just need a way to measure the quality and evaluate the quality of your services- and you can do so by using the data from prometheus and slo and sli based quality gates from captain- and this is the part of this tutorial. This is the main part of this tutorial.

A

Actually, so we're going to use the potato application for the uh as our demo application we're going to use hay, which is a very simple to use, load testing, cli, we're going to use captain plus the parameter integrations of captain to trigger quality gates and, of course, where you're going to use the captain's bridge to take a look. What is actually going on so while uh the installation is running in the background, let's take a look. What you would need to also run this tutorial on your machine or on the vm that you would bring?

A

I have tested the tutorial on the amazon ec2 instance, with amazon limits on it with a t3 large instance. This is what works best in my experience, so if you can make sure to bring this instance, this tutorial will also work fine for you. Please also make sure to open uh port 22.80 and port 443 for interacting with your vm and make sure you also have pseudo rights to execute all the needed steps of this tutorial.

A

Once you can bring this machine ssh into this box and get started first part is to install a couple of prerequisites. You can just copy this whole block and execute the whole block. um It will download and install git curl jq, just a couple of utility tools and functionalities that we're going to use in this tutorial and at the last part here it's going to clone or download the captain on k3s sample repository and we'll check out the latest release.

A

So with this, uh you will get all the latest sources and we're going to make sure when we install it. We have all the latest and greatest so before you're going to install it. You just need to export one cert email that is needed for let's encrypt, uh as everything that we are going to install, will also have a certificate, and you can just use the tutorial that is or the email that is given here or you can use your own email address. Both will work.

A

Hopefully- and here you have the installation instructions, so you can just copy and paste this installations instructions and while my installation is already running in the background, I would just briefly explain what we are actually explaining we're using the flat control plane here, because we only want to install the captain control plane. We don't need the full execution plane of captain because we're only using the captain quality gates in this tutorial.

A

I've chosen provider aws because I'm running running my ec2 instance in aws. Of course, I've chosen the flag with promethvoice. So during the installation procedure there will also be a prometheus instance installed and configured with captain.

A

We also want to have a prometheus demo, because we want to have everything ready once the installation is finished and we don't want to install the demo afterwards. We also need kit that is a ui for it very similar to github or gitlab, where we can take a look at all the um all. The resources that have been added to the captain managed repository um yeah, we're going to use, uh let's encrypt, to have the valid certificates and we're going to disable their login for the captain's bridge.

A

So we can just enter the capital switch without any logging needed. This is just for the demo purposes. These are the exact instructions that I'm going that I was going to use in the background for my installation. So while it is still running, let us take a look. What is actually installed?

A

First, we will get the captain control plane. We will also get premium voice, plus the prometers integrate integrations that are needed for captain. We will get the potato demo application already installed um or already deployed and also exposed. uh We will have gentile. We will have the captain's bridge, of course, and we will have the first run of the quality gate already executed. So once we enter the captain's bridge, we already see the first quality gate evaluation.

A

So, at the end of the installation procedure, this is what you will get and let me take a look at my installation procedure. It's not there yet, but what we can already see what's going on is that right now it's generating the traffic for the potato application, so this will be running for around 90 seconds because we need to generate some traffic for our first quality gate evaluation. So the script is already quite fast into the quite far into the process. We already have the url for our potato application.

A

Actually I can copy and paste this already to take a look. Now. The script is just moving on.

A

uh It's basically now finished with the quality, um with the load test and right now the script is finished, so we will see their url to our captain api, the url to our captain's bridge. There is no username or password needed in this case. We also see an api token for captain. We see the git server, the user forgives the password again. So we'll see a lot of things.

A

Don't worry. I will kill this instance right after this tutorial right after this video. So no need to copy the api tokens to mess with machine. It won't be available afterwards. So let me just open the captain's room here we go so we are in the captain's bridge.

A

Let me just enable the automatic check and let me take a look at the projector's quality gate, so we can see the first run of the quality gate was already successful.

A

Let us take a look actually at all the sequences that have already been executed, so we created service, we have configured the monitoring and we already can see the first part of the quartergate evaluation. Let me just move this here. We can see we got 100 of the score and we have a couple of slis here and we can see everything that was executed. We can see this here in our quality gate evaluation, so this is great. We can already see that captain did evaluate the quality of the rotated application that was deployed during installation.

A

So if you want to take a look on the actual quality gains, we can do it right here show us the slos that we are going to use. So we can see the response. We are taking um care about the response time in seconds for the main page we are taking here about the request, throughput the go routines and also the response time just for information purposes here, so we can see. Oh everything has been evaluated. We got enough points. Everything was fine to actually get a thumbs up in with green colors on this column.

A

Gate evaluation.

A

So let us uh take a look in our second quantitative version here in the tutorial everything is is described, so you will get all the information also here, if you want to take a look in the kit instance, kt instance, we can do so by opening again this link if we lost it or if you still have it open, you will also find the git server right here.

A

So let me just open this here we go click on explore and you will find the captain management repository and I'll just switch into my quality git range, and I have here my hello service, that's the potato head application and I can see the configuration for promptvoice and I can see my slo file. So let us take a look first in the prometheus folder, so we can see here a couple of different service level indicators that we are going to use and they have a couple of placeholders.

A

So this is the part where captain will make sure once you use this file for a specific service to replace all these placeholders and use the actual service, the project in the stage so later on, when we will fetch the data from from ethos, it will make sure to fetch the correct data.

A

Okay and we can also take a look at our quality gates. Here is our slo file and we can see here all the criteria we have for the captain quartergate and we could also change it. So if you want to change it, you can do so by uh logging, first logging into the git repository and then changing it here, the login credentials.

A

You will find them right here at the end of your installation. Here are the login credentials for the gta instance. Okay. So now we've seen the um the demo application is installed, it was already load tested and we already did the navigation of the quality kit. So let us now deploy a new version.

A

Actually, the potato head comes with different versions and let us take a look at the new version so before we go into this, let me just open up retrieve the url of the potato head and let me open up the potato head application just go ahead here. Here we go here. We have our rotated application in version 0.1.1, and uh let me just open this once we just leave this open here and let me just go back to my tutorial and deploy a new version. So in this case we are deploying version 0.1.2.

A

We can just do this by copying and pasting into our terminal and it will do a huge ctl and just exchange the image of our deployment. So it's not very sophisticated and uh for all the interactions with cube ctl inside your vm make sure to uh always prepend it with k3s, because we built the k3s, the keys, managed version of cube ctrl. So here we go. If we're going back and just reloading this again, there should be a new version of our potato head and we can see it's not a version serial 1.2.

A

So so now we want to evaluate this version. We will also hit it with some load and then evaluate it again with our captain quality gates.

A

Okay, so the instructions how to hit it with some load are actually given here, and I said everything you can just copy and paste it from the tutorial into your vm. So what I'm going to do here is, I will just copy this. It's already running great, so what it's going to do?

A

It will execute the hey binary, which is our tool for the load tests. We are running this only for 90 seconds. If you want to run it for a longer time, you can just do so by changing it to let's say 10 minutes 100 seconds. Whatever is the time you want to execute your load tests, the binary would at the end, we will give you a summary of the load test and you can experiment how long you want to run it or also in how many concurrent connections you want to open for the load test.

A

So in my case, I'm just running it for 90 seconds with 10, parallel executions, and uh here I'm just um fetching the ingress of my potato application. So that's basically um fetching the the url where the potato head is exposed, as this url is generated dynamically. We're just fetching this here. With this statement, and with this we are now executing for 90 seconds, some load on our tailored application.

A

It it is monitored with promote funds. Basically, it's um exposing data to promoters and prometheus is scraping this data um every uh 60 seconds, I believe, and so after those 90 seconds, we of course have some way to to evaluate with a captain quality, as we already have some data here, we go, we already have the load test is finished and we already have the answer on the status quo distribution. So we know all of our 450 requests responded with a status code 200.

A

That means everything was fine, so now that we know that we could reach the application we want to know how did it behave?

A

What's the quality of this evolution and we can just trigger a captain quantitative evaluation by copying this line here, pasting it into our terminal, and I just want to have an evaluation for my hello service, which is my potato head application and for the quality gate stage for my promethos in my prometheus project- and I will do this here for the last three minutes. You can also do it just for the last two minutes. uh I would just uh do it basically for the time frame of three minutes.

A

This is the data I will be querying or captain will be querying and evaluating. So let us go back to the captain's bridge, I'm just moving back here and I can see. I just did this evolution. Let me take a look. I did not get a very high score on this, so what happened- and I can see the response time for the main page- is now uh around two seconds. So there is actually a very high delay uh in this um demo application for this exact version.

A

So now it takes two seconds for the main for from the main page um to to respond and to render. We can also evaluate this. If we just go to the browser hit f5, we will also see it takes quite long for it to come up, but this is not how we want to do the load test. We want to do the load testing, of course, with some tools, and we just used heavens, um because it takes so long for the application to to respond.

A

We could also not satisfy our uh the request throughput, so this dropped by more than 80 to previous runs. Also response time is quite high in the goal routines. We just have it here as an indicator. We do not really care about this as long as this. In this example, we should say it has to be: has they have to be less than 100 and we're fine, but you can now experiment with the qualitative by yourself, so you can go into the git manage repository.

A

You just take a look here and you can experiment. What is uh what are the criteria? The captain quality gate uh should evaluate, and uh what are the criteria that you allowed to pass where you want to receive a warning which uh sli or as long as you just want to show up as a for informational purposes, so you can now go ahead and experiment with this application, uh as the prometheus is set up, captain is set up for you and you have the demo application up and running.

A

As said, there are three versions of the demo applications version, 0.1.0, 0.1.1 and 0.1.2.

A

So we could also now go ahead and again deploy the previous version by just exchanging the image name and run another test of the quality gate. Everything. What we did is also described in this tutorial. If you have any troubles, there's a short troubleshooting guide and if you have any feedback, please reach out to us via twitter via slack, join us in our slide channel on slack.character.sh and give us some feedback. What are you doing with captain quality gates.