keptn Community videos, 12 Nov 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Automated Quality Gates with Keptn @ Amplify2020 - 2020.11.12

Description

To keep up with the pace in continuous integration and continuous delivery, the automation of quality gates is inevitable. We want to show how Keptn can be used to achieve such an automated quality gate by integrating performance tests.

Presentation from Christian Faustmann @ Dynatrace Amplify2020

Useful Links:
------------------------------------
Keptn: https://keptn.sh

SRE-Playlist(Google): https://bit.ly/36P6gTB

Performance Matters Playlist: https://bit.ly/3pHOB98

Visit & Follow us:
------------------------------------
Homepage: https://triscon-it.com

LinkedIn: https://www.linkedin.com/company/triscon
Twitter: https://twitter.com/triscon_it
GitHub: https://github.com/triscon

Sections:
------------------------------------
Intro: (0:00)
Triscon: (0:15)
Performance Tests: (1:00)
Autonomous Cloud: (2:07)
Keptn: (4:24)
Keptn - Quality Gate: (6:20)
SLIs, SLOs, & SLAs: (8:04)
Demo: (9:04)
Outro: (14:18)

A

Hello, my name is chris, I'm from triscon, and today I'm going to talk about how to automate quality gates with captain and about the huge benefits that come along with it.

A

First of all, let me quickly introduce triscom for the ones who haven't heard about us before we are dealing with all aspects of performance engineering and particularly, we focus on performance testing.

A

We support our customers by setting up, as well as utilizing their apm solutions. In addition, we offer several workshops and trainings for various topics related to performance. Engineering. Here is a list of some of our clients and partners.

A

As you can see, we are dyna trace, as well as new to his partner, as I mentioned before, we are strongly focused on performance testing if you haven't heard about it. Yet let me briefly describe the benefits of performance tests.

A

Obviously, the main goal of performance tests is to avoid crashes and to have a stable system. Often there are errors in software that only occur under heavy load, so you actually need a load test to reveal those errors with a load test. We simulate many users accessing the system at once, and you can reveal errors that you wouldn't see in any other tests.

A

Apart from stability, performance tests are crucial to find out if your software is fast enough. In addition, performance tests also indicate whether you need more or even less harder resources which can potentially save a lot of money.

A

One of the new buzzwords in the industry is autonomous cloud, but instead of hyping it even more. Let us go through the steps which are necessary to achieve it.

A

Autonomous clouds means that the cloud can take care of itself as our baseline. We assume that you have already a monitoring system in place like dynatrace or prometheus.

A

The next step would be to automate quality here. Quality here belong automated testing, continuous performance analysis and quality gates afterwards comes the automation of the deployment and all operations of the cloud.

A

We saw, on the other hand, that many customers automate their deployments without or partially automating quality.

A

We see automated unit tests automated end-to-end tests, but we usually do not see any automated performance tests, but why is this the case?

A

The reason is the following: the evaluation of unit or end-to-end tests is simple. It can be either pass or fail in performance tests. This is not a simple decision, as you have hundreds of tests being executed. At the same time and for the evaluation you have hundreds of metrics to consider if the tests have passed or failed.

A

So complexity is much higher and here comes captain into place because with captain we can evaluate these hundreds of metrics automatically and compile combine them into a final score.

A

So, let's talk more about captain captain is an event-based control plane for continuous delivery and automated operations for cloud native applications. That's a lot of information packed in one second sentence. So, let's break it down.

A

Captain follows the cloud event specification. Therefore, you can easily integrate it in your existing pipelines or monitoring solutions or receive events from captain to trigger deployment scripts or run automated tests.

A

Continuous delivery covers everything till deployment and then the deployment itself, which can then be manual or automated.

A

This is a rough sea and there is a lot to cover here, but captain can support us on every step.

A

Here at automated operations, we will focus on uh the other side of captain that he can react on events too. So on our production side, we see now that we have a monitoring in place in that case final trace. So let's imagine we have an issue with a application.

A

Dynatrace will inform captain about a problem, and so captain reacts here with a rollback of this application, especially application version, and so it moves back to tefland.

A

Let us next focus on captain quality gates.

A

Here we have already development staging and the production area and we have a running version, one in production, so let's put quality gates into place, so they check if a software is ready to promote to the next stage and now, let's imagine we want to um roll out a new version version two for that example.

A

So, first of all we have to test now on a unit level and performance in a component scope.

A

Here you see, everything is fine and version 2 gets promoted to staging, but at the staging area we test on end to end, and here we have a performance issue, so the quality gate prohibits us for moving forward into production.

A

So now we take a look at version: 3. version 3 has the improvements which we gathered on version 2 or the problems of version 2.. So now the testing in the depth area is everything. Fine again we move forward to staging, and here in the staging, everything is fine too, and the performance is satisfied, so the quality gate lets us promote version 3 to production and this with confidence, and we are sure and everything everything is fine.

A

But how actually does captain know what to analyze in those quality gates here come slos into place? So but let's have a short look into the sli's slos and slas, especially service level. Indicators is something you can measure like error rate or response time and the service level objectives is only boundaries to those slis and slas are penalties when those slos are not met.

A

So it's a short reminder: you can think of slice will drive those slos and those slos will inform the slas.

A

So, let's have a look, how it's done in real life and we will do it in our demo.

A

So before I start our pre-recording, let's have a look at the dashboard.

A

This is a dynatrace dashboard and we have here our metrics, which we will use in our captain evaluation.

A

As you can see up up above here, we have our shortcuts to the most important things, our application, the asia pipeline, which will trigger our load test, which is handled by new load, and we will see it in nearly web which orchestrates all load generators, and then we have our captain bridge. This is where we see the final evaluation.

A

So, let's start and jump into the demo.

A

First, we will trigger our.

A

A

So here on the dashboard, we take a additional emphasis on the annotation here. This is where we define those slos for the captain quality gate here at the triscon testing turf. This is our demo application for our recruiting purposes. We have only a simple registration and, as you can see, there are already many users registered. This is from a previous load test.

A

Our pipeline is now at run new load test, so we move into new load web, and here you can see one moment here- you can see that everything gets uh configured and we sped this. uh We yeah accelerated a little bit, and here I will pause a little bit. So you can see the blue line here is our simulated users, which are now register, registering new accounts on the testing turf. But, as you also can see, we have a lot of errors here, so we can imagine that captain will not allow this in our production.

A

A

A

So new load is now done with our load test, and now we will inform captain that a load test has already been done and we will say when does it happen from here till here? This is a timestamp. We will see we would see down here, but it's not very important, but we will now trigger it. Oh, that was unfortunate.

A

So we move back.

A

So exactly here we are now in our pipeline and we will send this captain event with the start and the end date or the end date for our load test.

A

Then we move to our captain bridge, and here we will see in a moment all our slis or all our metrics. I will stop pause here a little bit. You see all the slis.

A

These are all coming from the uh the dynatrace dashboard and, as you can see, our test here has a problem uh with the error rates, so it gets marked red, but the important part is on down below here you can see here. Is this the final score? In that case it is 88 out of 100 and the result is fail, so the squad, the quality gate, would would prohibit us to move forward to blackjack and as we can now compare it to a previous run.

A

As you can see, this is everything green and we have 100 out of 100 and of course this passes our quality date. So, let's emphasize it, we didn't have to analyze anything by hand. Everything we defined previously in our finance dashboard and now captain automatically analyzed all our metrics and give us a final result. So this makes our life a lot lot easier.

A

Okay, so this is done.

A

So I hope I could give you a good introduction to the captain quality gates. If you would like to collaborate, do not hesitate to contact us. If you have any questions, please let us know now in the live chat.