keptn Community videos, 21 Jul 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SRE Webinar - Data & Observability Driven DevOps & SRE with Keptn by Andreas Grabner.

Description

The SRE (Site Reliability Engineering) Practitioner course introduces ways to economically and reliably scale services in an organization. Site Reliability Engineering benefits in a multitude of factors, and is being implemented in various organizations across the world. Watch our webinar to know more about SRE on the Topic "Data & Observability Driven DevOps & SRE with Keptn" by Andreas Grabner.

To know more about our SRE Foundation Certification, visit: https://www.gsdcouncil.org/certified-site-reliability-engineer-foundation
Connect with us on LinkedIn: https://www.linkedin.com/company/14409844/admin/
Your review can grab our viewers' attraction. Please leave your precious review at https://g.page/r/CfVxKucPnM4TEAg/review
Join us on Instagram: https://www.instagram.com/gsdcouncil/

Don't forget to Like, Comment, and Share the video. Also, Subscribe to our channel for more updates on upcoming videos.

#srewebinar #sitereliabilityengineering #srewebinarseries #srecasestudy #srewebinarvideo

A

Great to see so many people trying to learn about things that I'm very passionate in my life, because over the last 20 years since I've been in the industry, I always try to not only learn things but also teach things, some things that I learn and it's great to see. So many people been motivated in learning site, reliability. Engineering, my name is andy grabner. This is a typical setting here.

A

For me, I'm in my kitchen, as you can see- and I well today, is not about my kitchen today- it's really about showing you stuff that I think, will help you as you're adopting s3 best practices. Now let me start sharing my screen.

A

uh As was mentioned, uh we will have a dedicated time for q and a later on. But if questions come up, you know feel free to use the chat, feature and zoom already type in your questions or later on. I think we can open up the mics and then have a discussion.

A

So again, if you want to follow up with me later, there was the linkedin group that was mentioned earlier. You can also directly connect with me on linkedin. If you want to, who am I my name- is andy grabner, I work for dynatrace. I've been there for more than 14 years now, in the capacity of the I call myself a devops activist, but really I'm a deaf rel, so developer relationships, I'm the link between our user community that wants to know more about observability and building high performance applications and also with our product team.

A

On the other side I represent, captain captain is uh since last week, a so-called cncf incubating project. If you're familiar with cncf the cloud native computing foundation, that's the foundation that um kubernetes gave birth to right. So it's a it's. The foundation that hosts projects like kubernetes, argo, techton, helm, istio and many others, and we are now and with we I mean the captain uh community- is also now a so-called incubating project. We made it from sandbox to incubating and the next step is hopefully graduating.

A

You will hear me talk a lot about captain because I really believe we built captain to make it easier for you to adopt and adapt devops and sre principles around automation, but with a very special focus, which means putting observability into the driving seat. Looking at monitoring data observability data to then drive your automation, and this actually brings me to the first polling question.

A

I want to launch the first poll. Hopefully this works well, the first poll it just gives me a little bit more insights about who you are and with you I mean the attendees right now, and the first question is: what's the state of observability in your organization or if you're, currently, maybe uh switching organizations and learning something new? What do you think the observability status in companies you work with or for is? Are you monitoring uh every environment all the time? That means everything from production back to development?

A

Do you only monitor business, critical applications to your only monitor infrastructure? Maybe you just don't know, and that's also completely fair, to say. I also understand that monitoring and observability are terms that might be a little a little bit loosely defined. What does it really mean to get observability?

A

Can it just be an availability monitor that monitors if an application or infrastructure is available by doing things or is it something where you may use what the industry has called for many years? Apm application performance monitoring? What do you do? Full stack visibility? Are you looking into open, telemetry, prometheus, there's so many cool things out there, especially in the cloud native space, so I can see now it's great to get some answers here. It seems the majority is monitoring every environment all the time. This is great. I think this is what you want.

A

You want to see what's happening at every step along the way from the application life cycle, the next majority. The next bucket is, we only monitor business, critical, apps, that's uh 33, that's the third, that's great! As fraction is, we only monitor infrastructure and one person doesn't know. That's also fine, all right. Let me uh close this poll there's going to be two more polls later on, uh but thanks for this, uh this is great insights all right.

A

So today uh let me go and actually talk to tell you what I'm gonna talk about the way I've structured my talk, kind of like in three acts.

A

The first talk is: what is captain, what are the key use cases that we bring to you as sre engineers as devops engineers as platform owners, or somebody that's just interested in it, and also who is adopting captain because there's a lot of open source project out there and therefore the question is: is this worth your while to look into it, or is this just like a kind of we call it a kindergarten project right with nobody, people playing around I'd show you that's not just playing around large organizations around the world are adopting it.

A

The second act is: why did we build captain which problems? Does it really solve in a unique way? And why wouldn't you use just other tools for captain and then the third act is going deeper into how captain actually works the architecture, especially how you can extend it and integrate it with your existing tool?

A

Sets because captain is not replacing tools, as you will see, but captain is connecting your observability with all of the other tools that you're using in your organization, deployment, testing tools, notification tools, monitoring tools, it connects them all together to make sure that the systems that you're deploying are actually adhering to your sre best practices, meaning that your systems are aligned with your slos.

A

So which brings me to the first act. What is captain uh the best way for me to explain what captain is to actually take a quote from tadash tarash is a performance engineer at facebook? I think facebook is a well recognized organization. He is taras has been at facebook for many many years he's been involved in many large scale and uh performance and scalability project within facebook, and he made a very nice quote.

A

He invited me two years ago to speak at his performance conference and he said when, after he saw captain there, that captain feels like a reference, implementation of google's site, reliability, engineering and the site reliability workbook book. So I think that's actually a great quote from somebody that is living and breathing performance and reliability day in and day out at an organization like facebook. But what does this really mean?

A

Why did harass come up with this quote? Well, because at the core of what we do at captain, we are using so-called service level objectives to continuously evaluate application and infrastructure, to make sure that your abs and your infrastructure always stays within your desired state. Typically healthy.

A

Now, before I put out this animation to show you what this is and go into a live demo, I want to quickly switch to a second poll because I actually want to know if you are as an audience familiar with the concept of slos service level objectives, and I would like to know: are you familiar with it? Meaning yes, you're implementing slos? Already you heard about it, but you're not using it yet or no you've never heard of it. It's a completely new concept.

A

That would be very interesting for me to know, because then I also know how much more details. I need to give you in terms of explaining what the slos are all about. So I'll uh keep this poll open for another couple of seconds, because I roughly have an idea on how many people voted earlier.

A

So let's make sure that we get everyone, give everyone the chance to enter the poll so right now it seems the majority has heard about slos some already implementing it, but I would say about 80 percent, definitely know what slo's about 20 don't know never heard about it.

A

So you will hear a lot about slos today it is one of the founding or one of the foundations of site, reliability, engineering, uh but that's great, I really like it that uh 20 of you saying yes, you are already implementing slos 60, yes heard of it, but not using it yet, and hopefully I will inspire you today with the talk to use captain to start your slo journey. All right, let's end the poll now, I think we're good got a good overview. So how does captain work?

A

Captain does a lot of things, but at the core captain evaluates, as it says, on the top, the state of your applications and also your infrastructure. You can also expand to your end users, anything that can be monitored with your observability platform and the way you are telling captain what is a healthy system. What is a healthy app? What is a healthy user experience we're using the concept of service level objectives? It's really simple!

A

In captain terminology, what we have is we have a so-called lighthouse service and you will see and hear a lot about our terminology, which is a very alliance to what the kubernetes community is using. So you will hear a lot about uh things that reference to nautical terms like the lighthouse. The lighthouse is always watching up out for you and it's trying to give you clear passage into a safe harbor. So in captain you, we also have a so-called lighthouse service. It's a core component. When you install captain and what the lighthouse service does.

A

It looks at a config file, one of the configuration files where you as the user of captain as a site, reliability engineer. For instance, you can specify what the objective is of your system to be considered. Healthy, you can see here is a list of slis error rate. Jvm memory count db, calls security critical, so that means you can specify hey.

A

I have a system that I'm observing, like the error rate of my system, the amount of memory that is used. The number of database calls we're making uh whether there are any critical security issues, and you can not only specify that these are indicators that you're interested in, but you can also specify pass and warning criteria.

A

So what do you consider the system to be in order? For you say that this is healthy, a healthy system? Yes or no? So what captain does it allows you to define your criteria, your objectives?

A

Now the question is: where does this data actually come from? The data comes from your observability platforms. We call them in captain terminology sli providers.

A

This could be data from prometheus, from open telemetry or from tools like dynatrace data dog, new, relic sonar cube falcon any type of tool that is, observing the quality or the operational aspects of your system and depending on which tool you're using and you're connecting to captain. You can specify how to query how to extract things, like the error rate, the jvm memory, the number of critical security issues the database calls. So what you can see here are two examples.

A

If I would use prometheus as my observability platform, I would specify my individual queries using promql, which is the prometheus query, language to query error rate jvm memorands on.

A

If you would use dynatrace as a platform, you would specify the dynatrace metrics expression what metric should be pulled in so the point is we have a nice separation of concerns here on the top you specify what type of objectives do you have for certain indicators for certain metrics and then on the other side, you specify how you can query and extract those metrics, and what captain does captain orchestrates these two? It basically reaches out to these tools, says: hey, hey prometheus. I need these values or hey dynatrace.

A

I need errate database called jvm memory for a particular app. Then these tools report back to caption the data and then the lighthouse service takes that data analyzes every single metric, every single sli, as we call it against the objective, gives it a rating, green, yellow or red, and then we calculate a total score on the top.

A

Now let me show you this a little bit in action, how this can look like, or a good example of how I've seen people use this in a very common example. How people start with this? Is they typically specify a list of slis and typically they range similar? What we saw earlier, but maybe going now into a little bit more detail. They typically cover some performance aspects like response time failure rate, that's kind of like how stable is your system right?

A

How many requests to your system that is currently are currently processed are successful versus failed? Then you may have some specific critical transactions in your application that you're hosting. In this case, I have specified the login login as a feature, and if you have a new version of your app, you may run some tests against it.

A

This is why I call it test step login, measuring the response time of a new bill and also test the login number of service calls, if you think about it, if you're testing, let's say a micro service or any other service, that is providing your login feature like an authentication feature, you might not only be interested in the number in how fast it responds and what the failure it is. But you might also be interested in how many back-end service calls this service does right.

A

You try to authenticate, and maybe this service connects to a database to validate username and password. Maybe it connects to another back-end authentication system and you want to just monitor and see if the number of database calls or back and service calls kind of stays within what you expect last one critical security vulnerabilities. I think we've all learned from log for shell that security is very important. So, every time you get a new build, you want to also validate that you did not include any new security vulnerabilities.

A

So with captain you can specify a list of slis, then remember you specify your objectives. Objectives can be specified in two ways. You can either specify an absolute value, for instance, response time. The 95th percentile should be faster than 100 milliseconds in order to be considered good or 250 milliseconds in order to be considered in the morning range, but you can also combine it with a relative threshold. So, for instance, if you look at the test step, login number of service calls here.

A

I specify that it should be plus zero percent, which means it should not increase by anything. It should not increase between bills. The number of service calls should be stable. We'll give another example test step login response time. I say I want it to be faster than 150 milliseconds, but also I don't want the response time to increase by more than 10 to my current release, for instance, so you specify your list of slis your objectives and then what captain will do?

A

It will calculate a total score across all of these slis and then you can specify what should be your total score to be considered a good, build or a good health environment and a warning range, so you can specify overall score goals. So now, how does this work in action? Now? If you have set this up, you can say captain. I want you to evaluate my service that I've just deployed. Maybe I've built number one and please I just deployed it into an environment.

A

I ran some tests and now, please evaluate, let's say the time frame and the test trend last 10 minutes. What captain does it reaches out to your observability platform? Pulls in all the metrics compares them remember. The lighthouse service compares it against the thresholds grades every single metric and, in this case build number one. Everything seems to be in the green range, which means we're getting a hundred percent success rate.

A

We get a hundred percent total score because by default, every individual sli contributes if it's green uh one-fifth to the overall score, which is a hundred percent, so everyone would contribute 28 maximum. If it is yellow, it would contribute half of what's possible so 10 and if it's red zero. So let's assume this is great. Build number one is green developers, bring you a new, build you deploy it, you run some tests and then you say: hey captain.

A

I want you to evaluate build number two now with the automated integration into your observability platform and the automated validation against your slo. In this case, we could see hey response time and overall failure rate increased. So something is definitely not going into the right direction. We're getting penalized here. But, most importantly, we have an architectural regression here, because the number of back and service calls from the login service that we're testing, which is very critical for us, went from one to two calls.

A

So, all of a sudden, a feature that has always made one call is now making two calls and we have defined an sre rule, a resiliency rule where we say we don't want that number to increase, because there must be a really good reason why the authentication service, the login, all of a sudden, makes more back-end calls.

A

So if this happens, then we're getting penalized and we're only getting 50, and we are directly passing this back to the developer, to your git tool, to your jenkins to your github action whatever it is that you use currently in your automation, tool belt to kick off a build to deploy, and then you could kick off the evaluation captain. The point is quick feedback.

A

Quick feedback means developers can address this really easily. So maybe the developer says ah I know what I made. I did wrong. Maybe I need to update the library or I mean to fix the code, so he fixes the code gives you a new build performance is good again.

A

He fixed the architectural regression, but all of a sudden there is a security vulnerability that came in because maybe the developer, in order to fix, build number two, he updated to a library, remember log4j that heises no vulnerability in this case. This is no good either so we're again getting penalized, not a good, build but fast feedback.

A

If you integrate this into your ci cd, you always monitor and you always observe- and you always evaluate your desired state, which is it should all be green, so fast feedback developer can fix it build number four comes along. Hopefully everything is clean. Again, that's kind of the idea. Now this was nice in powerpoints and I always say nobody believes if it's just powerpoint. So this is why I want to give you a quick overview of my first demo of how this looks like all right. So let me switch over to this is captain here.

A

So by the way, what is captain captain, as I said, is a cncf project. You need to install it on kubernetes right now it is a we have helm charts. We have a captain cli, where you can install it on any kubernetes cluster. We also have some tutorials, where you don't even need a kubernetes cluster where we are creating one on the fly for you and you can play around with it, but in general, when you have captain install what you have here is the ui. We call it.

A

The captain bridge because remember captain is a captain of a ship that helps you to steer your apps your services, your infrastructure, into the safe haven of a green. It will always always always healthy haven. um You can define automation projects right.

A

I have multiple automation projects here and I want to focus on the release, validation, automation project because I want to in my first demo show you the core feature that everybody starts with when they're, starting with captain captain can do much more, as you will see later on, but at the core captain does really well with automating the validation of your slos. So what you see here, I have a release, validation project.

A

um I have a stage called production and I have different services. Services could be my applications could be. A micro service could be a monitor if anything, any any name. We just call it service and every time I have a new build, and maybe I deploy that build, and I run some tests either manually automate it then I want captain to evaluate my quality, so what I've done earlier, as you can see here just before I kicked off the webinar, I kind of kicked off captain.

A

Actually, let me kick off a new sequence, so you can see how easy this is. So I can say trigger sequence. I can say I want to do an evaluation for my. I have three services here, I'm using this one in stage production. Then I click on next. Now I can specify what type of time frame I want. I just go with a standard frame of 30 minutes. That means I'd say kept. My application has been running for 30 minutes. Now you please evaluate it.

A

You can also give a start and stop, but you can also give time frames whatever you like. You can also do some labels build ids and one is a very specific one. One two three five, I think, is the one I need to use, because this will later show up nicely in the ui. You can also do executed by andy uh during uh gstc webinar, and then I could also say something like. I don't know external link.

A

Let's assume I'm not kicking this off manually, but maybe I'm doing this to the api, and maybe I'm kicking this off from your jenkins, your jenkins dot, your work, dot, com right and it could be your pipeline, so you can specify here name value pairs. We call them labels and they allow uh regular strings or also links. Now, if I trigger the sequence, what you will see if everything goes according to my plan, captain is now triggering that sequence. Remember the lighthouse service, the lighthouse service is doing its job.

A

It is looking at the slo definition that is behind the scenes. In a git repository, I also get to see build one two, three five executed by andy. These are all the things external link.

A

Also, what captain does in my case, because I'm using dynatrace in the background it is pulling that data in now, in my case, from a dynatrace perspective, um I we have a special integration with tools like datadog and dynatrace and prometheus for dynatrace. For instance, you can actually specify a dashboard.

A

So what you see here this is a dashboard that I've created in my observability platform, and this is kind of the definition of my slos, so I can either do it in config files as you've seen it earlier in my slide deck, but with a more visual approach. You can also just specify your dashboard and you put your thresholds on these charts and then really what captain is doing with that integration.

A

It is pulling in the data from that dashboard and comparing it against the thresholds that are also specified on the dashboard.

A

So this will probably take a minute, or so I can go back to the previous execution just to show you how this looks like, and I can also go back and see all of the results right. These are all the results that are that came in here. We go one two three five, I think that's the one I executed.

A

You can also see the actual values that were pulled in so for every metric. On that I specified I get to see the value that was pulled in the change, maybe to the previous, build and so on and so forth, and also how every metric every sli was compared against the criteria and what the score was now. You can also see something interesting here that I didn't mention earlier.

A

You can specify weights so by default, every metric, every sli is weighted equally, so contributing equally to the total score, but you can also change it. You can say a certain metric, a certain sli might be five times more important than others right. You may have different criteria that say memory usage. Your performance is more important than something else. You can also mark certain metrics, as you see here, in bold as a key metric, which means if this metric fails, it basically overrules the final result.

A

A good example would be if your key criteria is to not introduce any new vulnerabilities. You can say the vulnerability metric is a key criteria. I don't the best performance. The best stable system is no good if it has vulnerabilities. In my case, I have the aptx specified as a quick criteria. Aptx is the so-called user experience index. I want my users to have a great experience in my monitoring tool like observability tool. Language can do this. For me. Let's go back.

A

Let's see my previous execution, you can see here it just finished with a score of 81. That's great! I can see it here now. In my result, I got 81. That's awesome. What I also have- and this is the nice thing about captain- and you will learn more about this as I go into the architecture.

A

Captain is an event driven system, which means I've also configured to push events when captain finishes an evaluation, for instance, to an external tool like slack, so you can see here, 4 pm, that's my local time. I push that event over with the final result, and everything is here right. So captain is very flexible here. Captain itself is an event driven system. These are all events.

A

Evaluation get sli, there's always a starter and a finished event, and later on. You will see that you can also subscribe to those events and push it to external tools. Captain is really connecting all the tools together all right, so this is kind of my first kind of demo and the idea is trigger this from any of your existing tools. Whatever tools you currently use to build to deploy, then you can trigger it directly from there and then get the the result back to your system.

A

The other option that you have, if you're not fully automated, yet you can do it like. I did I just manually clicked on trigger evaluation, and then I got the result back even pushed to my slack once the result was finished, all right, so the bigger picture. I want to talk about the bigger picture, because there is always a bigger picture, because this is just the start really. What we do with captain is much more now we're putting slos into the center, where you can specify slo service level objectives to really describe your.

A

I call it almost the business desired state of your apps and infrastructure, because that really means you always want your system to be green right when a new build comes in. You want to make sure that new build only makes it into production. If everything is green and you also want to make sure it stays green in production, so with captain you can define the slo captain itself gets triggered and becomes active based on two kind of type of events. Either.

A

If you have a new, build or let's say a github exchange event using argo cd jenkins, there's a new build and you want to make sure that new build gets tested, evaluated or if a production alert comes in. Let's say you are prometheus, you have an alert. Your dyna trace your data dog. It detects a problem, it can also trigger captain, say: hey captain we're currently out of our slo. We need to do something now. What does then captain allow you to do?

A

It can react to those events, and it can do two major use cases. One is it can orchestrate a sequence of tasks as we call it to deliver the promotion of an asset. So if you have a new build, if you have a new container, a new version captain can be used to assess the situation before deploying then instruct the deployment tool like jenkins argo gitlab, to deploy. It can also, then instruct tests to be executed and then afterwards validate if everything is still green, based on the slos. If everything is green, it can promote.

A

Otherwise it can stop the promotion there or even roll it back. So captain on the one side can orchestrate delivery promotion. The other major use case is it can orchestrate what we call day, one and day two operations. That means, if something is wrong in production and you get an alert because your slos are violated. So let's say your prometheus sends an alert.

A

Then captain can orchestrate what we call a so-called remediation sequence, where it can assess the situation about the problem that was reported and based on the assessment details right, collecting more information about your slos deciding what action to then trigger. Maybe it can then trigger another pipeline to roll back if it's a bad, build scale up scale down afterwards. It always validates again. If the system is back at a healthy state- and it can do this in multiple iterations, you can specify multiple tasks that should that caption should trigger.

A

If something, if, if it goes back to green, that's great, then the problem is solved. If not, you can escalate at the end. The point, though, is so we're putting slos in the middle, we're getting it from the observability platform, and then we can orchestrate sequence of tasks that are triggered by events like a new deployment is available or a problem in production.

A

Most important- and this is a mistake that I made in the very beginning when I talked about captain captain itself- does not deploy. Captain itself does not test. Captain itself does not monitor captain orchestrates and connects to your existing devops tools, because you have great tools out there that can do these individual jobs pretty well, but what we are doing with captain.

A

We are putting your observability tools into the center and then we are connecting your automation. Sequences that you can specify and captain with your existing tools to let them do their job when the job needs to be done and the way we do it we're doing it through an open source standard. This is another great benefit of being an open source project. We are driving an open source standard to connect tools to an orchestration tool.

A

Like captain we are using so-called cloud events, it's a project in the cncf and on top of cloud events, there's a new project called cd events. Cd stands for continuous delivery and we already have a bunch of these tool: vendors on top on board that they are implementing this particular standard. Why do we do this? Because we want to avoid that. We have multiple tools like multiple deployment tools, multiple testing tools.

A

They basically do the same task, but they all have a different api, a different way how to automate them- and we have understood that this is a pain point right now in building automation and building orchestration, to connect all of these tools, because they all have proprietary apis so through the open source community. We're standardizing the way, we're communicating with these tools and let them be part of an automation sequence.

A

That sequence is triggered and centered around slo store, because in the end, you always want to make sure that whatever build comes in, it is green based on your slo specification and in case something happens in production. You want to bring the system system back to green kind of your slos are kind of like your north star, your business desired state all right.

A

So, um as I mentioned earlier, you can trigger captain for many tools and it can then also orchestrate uh many other tools, and this is just what I want to show you in a second a quick use case. So let me go back to my example: here: release validation. I have a different project here. That's called delivery demo before I show you this, I want to show you behind every project. We have a git repository that contains all the configurations.

A

So when you start setting up a project, the first thing you need to do is you need to specify git repository. You need to specify what we call a shipyard file. I will go into cheaper than in more detail in the rest of my presentation, but this is where you specify what we call sequences: there's a delivery sequence and there's certain tasks.

A

You can have multiple sequences defined that have different tasks here and you can also connect them like have a production stage with the delivery sequence that gets triggered if other sequence is finished, so this also allows me to do multi-stage delivery. So if I go back to my captain project now and into my let's say my delivery demo, actually let me go to my rollout demo, that's even more interesting here. What I have is when I push a new build in staging you can see here.

A

Captain is actually doing a so-called delivery sequence and then it's doing monaco. It configures my monitoring system. Monaco stands for monitoring is code. Then it triggers a deployment. This could be helm, for instance, or it could be jenkins anything it validates. If the system is running it, then we have an integration here with argo, where we're letting argo actually do a blue-green deployment or a canary deployment, and then we do the evaluation.

A

Then we even have an approval in the end where I can ask the user for permission to really then promote this build here from my staging environment that was just evaluated by captain with 100 into the next stage, and the next stage for me here is, if I go to the top right, is production right and in production you can see here. I have a delivery canary phase, one canary phase, two canary phase, three we're doing even my canary rollout orchestration when every phase we are basically allowing argo to do.

A

A roll out of the next canary set then run some tests. Some low cost tests, some cheap meter tests, whatever testing tool you have and then the evaluation. So you can see here. Captain can do much more than just evaluating. You can specify sequences of tasks and you can trigger them either. When you have a new deployment or when you have a problem production, then captain can also orchestrate sequences and can then talk with the tools and get them yeah get them in there.

A

All right demo number two now quick thing who is adopting captain. I want to just tell you that we're not the only ones to talk about and have demos. We have a lot of people adopting captain for the evaluation of a new release, and very often we see it in combination with automated testing. For instance, like mike, is doing here with automated performance testing. There is a we have a youtube channel as well. A captain youtube channel, so you can see.

A

All of these recordings point here is mike is using captain to automatically analyze the results of his loadrunner tests that he runs for every build saving him a lot of time because he doesn't have to manually not analyze the results for him he's using the same dashboard based approach that I've showed you earlier. Captain analyzes it gets in a feedback back the next use case, and this is very dear to my heart. I live in austria, center of europe and one of our first adopters was a reificent software.

A

They are building the software for for most of the austrian banks for the online banking and what they are doing again, they're using captain for the evaluation capability for every build that gets built by jenkins in their case, right jenkins, once jenkins is done with the building and the deployment and running, I think some selenium tests they're then calling captain and say please kept now you go off and evaluate the results and then give me a score back to jenkins.

A

Another example, which I also like a lot, is kind of like the multi-stage orchestration process like what uh will volusion is doing, they have git that is pushing the pipeline. That is then creating an artifact in jfrog, and then captain takes over when the new artifact is available and is then orchestrating a multi-stage deployment by integrating or triggering all of these tools using arc of what it's great for.

A

It's deployment using chain meter what it's created for testing using dynatrace, what's great for monitoring and then kept orchestrating everything last use case I have here from a user is from png.

A

They are using this for all the remediation, so they're getting alerts from their observability platform, and then they use captain's capability to specify so-called remediation sequences, where you can specify for what particular problem that is reported from a monitoring tool. What type of action should be triggered and in their case they are triggering uh python scripts and the most important thing is captain is then always evaluating after these actions. If the system is big at a healthy state, that's it. So why did we build captain? I think this is.

A

Hopefully you understand now what is captain? But the question is: why did we build captain? We built captain because well clearly, if you look at it, you could do this all of yourself right. You could build an orchestration and automation platform that let's say, uses jenkins or any other tool, because there's so many great automation, tools out there. You can build this all yourself with custom coding, custom scripting, but then you also need to take care of connecting all of these tools that you want to integrate in that process.

A

Like your testing tool, your deployment tool, notification tool, your security scanning tool and so on and so forth, you can do all this, but you need to maintain all of these automation scripts and also these integrations and we've seen that this sometimes becomes really challenging like one of our early adopters christian.

A

He was also building all of these automation scripts with his tool of choice, and sometimes these automation scripts became more complex than the microservices that were actually deployed and tested, and they didn't do this just for one pipeline, but it did this for almost a thousand cicd pipelines, and this is just challenging also we internally at dynatrace.

A

The reason why dynatrace is behind captain is because we also ran into the same problem that our existing pipelines that we used, while they were great in the beginning for monolithic applications as we were moving to microservices every application team had their own little requirement for what tools to use for a particular task. So they all started off with a template pipeline but then made their own modifications.

A

So we ended up with a lot of different permutations, a lot of copy and pasting and in the end we had a lot a lot of duplicated code and code complexity in our pipelines, which were very hard to maintain by people like dita, and this is why we said we need a new approach to automation, to automation in the cloud native space, because the old approach just means a lot of technical debt. So my conclusion is the left.

A

One is what we do today, but it doesn't scale because eventually, a lot of people that maintain pipeline scripts, that became become very monolithic and very hard to keep them from breaking, and so we thought we need a new approach with captain. So how does captain work? I think you've seen it, but I want to explain it again in a little kind of architecture overview.

A

First of all, if you want to get started with captain check it out, go to captain.sage or the captain community, but initially the problem, how we came up with what we have is really what you saw earlier.

A

We saw that internally and with many of our users, we had pipeline scripts or automation, scripts, classical automation, scripts like jenkins and others. These are great tools and we don't replace jenkins, but I think it's time to rethink of for what we use these tools for so we've seen like with the christian example earlier automation, scripts that were more than a thousand lines long, because they had hard coded integrations and tasks, prepare, calling a tool deploy, calling a particular tool with a very specific integration.

A

So what we said instead of having everything hard coded in one big automation, script, we need to remove those dependencies.

A

We need to allow you to separate the process from the tooling and the configuration so now in captain you can specify on the left the sequences that should be orchestrated when a certain event happens and your build is there or in the problem, happens in production on the right side. You have your tools, you may have one monitoring. You may have multiple monitoring tools. You have different testing tools. Maybe then we want to make sure that no configuration lives somewhere hard coded in an automation.

A

Script but stays somewhere version controlled, and then we connect all of this again together using eventing, which is the same thing that software engineers do as they are breaking monolithic applications into more service oriented architectures where they have individual services and they chain them together to fulfill business processes through events and event subscription models.

A

So to give you an example, mike the first example, I showed you earlier from a real user performance, automation evaluation. So he has a sequence: that's called performance and does a deploy of an artifact a test and then the evaluation.

A

So what he can do is, if you said, captain uploaders, you can say captain. I want to trigger the performance sequence, because I have a new image, my app version 2 in staging. So what now captain does it goes through that sequence definition? It says the first task in there is deploy.

A

So what captain does it sends out a standardized event and says I need a tool that can deploy this particular image in a particular stage with let's say a particular deployment strategy, and you can see here through the arrows to the lines that you can have one or multiple tools subscribe to those events. Why would you have multiple tools? Well, first of all, you can pick your deployment tool of choice for that particular environment.

A

To say, I take the deployment job, but you may also want to push the notification about that deployment to select what would email whatever it is. So you can subscribe to the same events and then every tool can decide if it has an active part in it doing a deployment or maybe just notifying people as the deployment is done. Captain goes to the next task. It's test and does the same thing sends out a request to say I need a tool that can do a test may have a testing tool.

A

Does the test any external tool will do and then at the end, there's the slo evaluation and that you've seen this order in the demo. It will send the get sli pulls the data in from the monitoring tool. The lighthouse does the evaluation comes up with a score, and then everything is good.

A

So terminology perspective we're talking about the shipyard file we're talking about the uniform. This is the tools that are taking part of an orchestration sequence. Then we have the config and the config repo, and then we have cloud events here and I hope I hope this makes hopefully a lot of sense for you now. I am also conscious about time, because I want to make sure that you guys are you get all of the uh the time to also ask questions now.

A

Let me go into what I've explained to you is, I think what we're doing is we're bringing opinionated cloud native automation to all of your projects, while captain itself runs on kubernetes, it's an event-driven system that allows you to get rid and reduce the complexity of your existing automation and orchestrate it for you.

A

The key point, though, is we are bringing and putting observability slos. This is the core concept of site reliability, engineering into the automation. We always make sure that your systems are operating the way you want them to operate, that they're always resilient, and therefore we look at slos and you can either I always say you can either take the left, the red pill or the blue pill or the green pill, whatever the matrix from the movie.

A

The analogy is right, and I also have a more management friendly slide version of this same thing, but you can either decide to uh spend a lot of time in maintaining your pipelines as they are, or you can have a look at captain. Now most people start with integrating slos into their existing pipeline.

A

As I said right, if you have an existing automation tool already like jenkins, like gitlab harness, you can see all the names up there and you already deploy and maybe already run some tests then start with using captain for just the slo evaluation.

A

But if you want captain can do much more right. You can automatically execute your test as well or trigger your testing tools and you can integrate with a bunch of different tools for different mains. You can also build your own integrations.

A

Now before I open it up for questions, I want to also say thank you to dynatrace for driving the innovation, because they have been doing this obviously for their benefits, so that their in internal development teams have a better way of deploying and and managing automation. But I'm really happy that they are contributing this project to the cncf for the greater good of the devops and s3 community.

A

In cloud native with this, I truly hope that what I said made sense that you saw a little bit of captain and now I would like to open it up for q, a.

A