keptn Keptn 0.8.0, 21 Jan 2021

Previous Meeting

⏯

youtube image

►

From YouTube: Tutorial - Keptn 0.8.0-alpha - Shipyard Spec

Description

keptn/spec 0.2.0-alpha: https://github.com/keptn/spec/tree/0.2.0-alpha
Shipyard Spec: https://github.com/keptn/spec/blob/0.2.0-alpha/shipyard.md

---

Learn more: https://keptn.sh
Get started with tutorials: https://tutorials.keptn.sh
Join us in Slack: https://slack.keptn.sh
Star us on Github: https://github.com/keptn/keptn
Follow us on Twitter: https://twitter.com/keptnProject

A

Hi everyone and welcome to another video in this video we're going to talk about the shipyard specification, that's being implemented in captain 080 alpha based on the captain spec020 alpha. Please note that some things that you see here might break for the final version of captain zero. It's your alpha, but I believe that most of the things that we're talking about here will be valid for the final version, all right, uh just a quick intro in case you don't know captain itself yet please visit the website.

A

Captain.Sh take a look at the source code at github.com, slash, captain, slash captain and, in short, captain is an event-based control plane that allows use cases like continuous delivery, automated operations and quality gates, okay, uh also again disclaimer. This is work in progress. uh Please expect braking changes, but unless you see a new video uh like for zero, eight zero final, I'm pretty sure this video will be valid.

A

So, while implementing use cases like continuous delivery, quality gates and remediation or self healing with captain, we always felt the urge to allow users to not just modify existing workflows but also define their own workflows.

A

Our goal was to create to not create another workflow engine, but we wanted to create a customizable event-based control plane with continuous delivery and automated operations in mind.

A

So what you see here is our shipyard format that we've had with captain 0.7 and older, or with captain spec 0.1 point x, this shipyard format defined multiple stages and as well as the way how to deploy on the stage like the deployment strategy here or a test strategy for tests and even approval and remediation strategy.

A

This format allowed users to define a set of stages and for each of the stages, specify the set strategies. However, you couldn't customize it like what, if I want to add another step or what, if I want a different deployment strategy and how do I implement it, and what do I do and which service is affected? So all questions that came up, but you haven't been able to answer them with this format.

A

In addition, like all the tasks and workflows behind the deployment and testing and evaluation and remediation, they were defined implicitly and it's better to have them defined explicitly, so you can actually modify them, which is also one of the key features that lacked in this specification and, last but not least, the stages linked here, it's always from top to bottom, and you haven't been able to do anything else, so you can't do parallel stages. You can't skip a stage.

A

Yeah wasn't possible to fix that. We are introducing the new shipyard format which is introduced by captainspec020alpha, and we've also described it in this captain enhancement proposal in case you're interested. So essentially, what does this new spec look like? First of all, it looks a little bit like a kubernetes manifest, but it definitely is not. It's just meant to look like it, because we think the format of a manifest is well known to the users of captain, or at least it should be- it's not a custom resource.

A

Yet we might go that way in a later version, but not at the moment. So it starts to look quite similar, like the old format like you can define stages in here, but other than that the similarities will change quite quickly.

A

So, uh first of all, basically we define a stage, but within that stage we then define a sequence. uh For instance, this is an artifact delivery sequence with sequence, which then consists of several tasks like deployment, test, evaluation and release.

A

Each of those tasks is then executed by a captain service or even a third party service, and then yeah. You just go back and continue with the next stage with the same sequence.

A

One of the main ideas of the shipyard file, however, is to be technology independent. Therefore, we don't configure things like file names or technology, specific parameters within the shipyard file. So, like the property here deployment strategy, it still says deployment strategy direct, we don't care who deploys it at this point like helm, service might deploy it, but there might also be a different service for deployment. We just want to have the attribute deployment strategy and that's it.

A

The same is true for tests with digimeter service. We don't actually say geometer run the tests in the functional mode. We just say we want to run tests with functional tests in mind. We will see this in an example later, so how does the old shipyard format compare, or actually, how do you migrate from the old shipyard format to the new one? First of all, there will be a cli command to upgrade the old chip yet to the new format. But essentially this is what it looks like.

A

So the stage name stays as it is deployment strategy is now a property of the deployment task test strategy is a property of the test task and, in addition, the existing artifact delivery or configuration change. What it was called in the old version is now made as an explicit sequence here with the aforementioned tasks.

A

This is also true for the other stages, but it might look a little bit different depending on what you have defined like remediation strategy or approval strategy.

A

Okay back to the task itself: now we have a task called deployment. We have a task called test, and this is a major difference now to the new version here, the deployment is being triggered with sh.cap.event.deployment triggered cloud event and test is being triggered with a similar cloud event.

A

We here take the name of the task and put it in the middle of this cloud event type, and so essentially, let's say I want to trigger tests. What does the cloud event? Look like it's a simplified version here. So, if you're familiar with cloud events, then you should be fine. If not, please take a look at the captain spec. There is many of examples of the cloud events in there.

A

For now, what you need to know is the test itself. The name test is being put into the type with a source of shipyard controller, because shipyard controller is now the central point where you will send yeah, where you will trigger certain tasks and the test strategy itself is within the data payload. This is a reserved space with the same name as the task.

A

We also introduced a new set of cloud events, so, for instance, for each task. We have sh.captain.taskname.js.

A

And status changed as well as finished. The rule of thumb here is shipyard. Controller always sends the triggered event, and some captain service that responds on this event needs to respond or acknowledge even with a started event, it can send an optional status change and it must send a dot finished event in the end when it's done, this is also being visualized in this little sequence. Diagram here again, shipyard controller sends the dot triggered event to nuts. This is retrieved by some captain service wire nuts.

A

The captain service needs to acknowledge that it has retrieved the triggered event with a dot started event, which is then retrieved in the end by shipyard controller, and the shipyard controller takes a mental note that the task has started in the meantime. The captain service- that's reacted, is doing whatever it does. It might send an optional status changed event for yeah. Just for your information like hey, I'm working, but give me like another minute, or so it's not required.

A

This is just optional and when it is done with whatever it was doing, it sends the finished event, which again is retrieved by shipyard controller via nuts, and then shipped controller can say. Okay, I'm done um so. What does this look like in terms of cloud events? Again, we've already seen a triggered event. The triggered event is being sent by shipyard controller with an id and an sh captain context.

A

This id needs to be put into the triggered id of the started event sent by helm service. This edge captain context needs to be copied. Obviously, and then this is just going to be sent in the meantime the service is working, and once it's done, it needs to send a deployment or not finished event. Again, this is being sent by helm service.

A

This again has the trigger id and the sh captain context. The the trigger the d is really important. I can't stress this out enough, because this is the way for shipyard controller to link a dot triggered cloud event with the dot started and dot finished cloud event.

A

Also, what's nice to see here is the data attribute which has a deployment attribute, which has some properties set by helm service. This is going to be transferred back to shipyard controller and going to be transferred to uh the other services later, which we will see.

A

Okay, so this deployment finished event is now retrieved by shipyard controller. Shipped controller looks into the shipyard and it sees okay after deployment. We have test. So, let's trigger a test by sending a test.trigger, sorry test the triggered event and again, the shcap context needs to be copied, because this is one sequence and the sequence is identified, but is kept in context.

A

We copied the data of the deployment event that we had, so this is just copied over and we're adding the test field here with test strategy functional because it was defined here as a property.

A

And then everything starts working again in this case in geometer, service jmeter service will have to center dot started event, run its test, send the finished event, and so on.

A

There are some restrictions I want to talk about for task names, please make sure they're lowercase a to c. I think you can use uppercase, but I just don't recommend it because it just doesn't fit into the style. You can also use numbers if you really want to as well as dots and hyphens. Please don't use any other special characters or spaces or unicode characters, I'm pretty sure things will break.

A

There are some task names that are reserved for captain internal usage, like action, approval deployment, evaluation, release remediation test and some other internal tasks that I don't want to bother you about. You can use them if you really want to, but I recommend defining your own tasks.

A

So what are the key benefits of the new shipyard format? Basically, you have the way to separate sequences now from stages or even have multiple sequences per stage. You can have individual and custom tasks in every stage. In every sequence, you can link stages and sequences together, like you want to, and you can have multiple parallel stages, which we will see in an example later.

A

Okay, that's it for now. This is the shipyard specification if you're interested in more please subscribe to this channel. There will also be a video about examples for this new shipyard format.