keptn Developer, 21 Jan 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Keptn Community Meeting - January 21st, 2021

Description

Meeting notes: https://docs.google.com/document/d/1y7a6uaN8fwFJ7IRnvtxSfgz-OGFq6u7bKN6F7NDxKPg/edit

Learn more: https://keptn.sh
Get started with tutorials: https://tutorials.keptn.sh
Join us in Slack: https://slack.keptn.sh
Star us on Github: https://github.com/keptn/keptn
Follow us on Twitter: https://twitter.com/keptnProject

A

B

A

And welcome to this edition of the captain developer.

B

A

um Thanks everyone for joining um as usual, please put in your name on the attendee list, so we can keep track of smds and also please follow our cncf code of conduct. That basically means please be nice to each other in our community and development meetings. We got a couple of agenda items for today. If you have anything more to add, please add it to the agenda or edit to the agenda for next week. If you cannot cannot address it for today,.

B

A

The first agenda item is already assigned with my name. um I briefly talked about this. As you might know, we have already been participating in the linux foundation, mentorship program that was previously called the community bridge and we had a mentorship for two mentees, and this mentorship program is now basically renamed to.

B

Lfx mentorship.

A

And the captain is planning again to participate in this program. The applications period just started just opened up today for projects and we will basically participate with one or two projects. So please reach out or check the the cncf mentoring page on github, if you're interested in joining us or becoming a mentee in the captain project, and we hope we have some very interesting.

A

I'm pretty sure we have some very interesting projects that we can offer and where we offer mentorships on your first journey into into cncf projects into in probably also into open source contributions.

A

So we will have more updates in the in the coming weeks, uh as we are also developing uh our uh project descriptions, but just as a heads up, um we already um working on this um next agenda item is.

C

Christian just um note, I think we should mention that ankit, our mentee from the last quarter uh published a blog post this week about his experience in this program for everyone who wants to be a mentee, please read the blog post.

A

um I will link it here. uh It's very nice blog from akit, you might have. You might know him from the developer meetings as well, thanks for mentioning this giannis.

A

So I will just also add this here. Previous experience.

A

Report here we go. um Okay. Are there any other questions or.

B

No question but.

A

um Remarks on this first chant item- okay, all good! uh I will be heading over to christian. uh Do you want to share a screen question, or do you want just to um highlight it here.

D

I think it's enough if I just talk about it quickly. um So, a couple of weeks ago we have published captain zero, etier alpha and obviously we've published release notes. But if you just want to have a quick summary of what's new in captain 080 alpha, then there is two youtube videos.

D

I recommend watching uh the first one linked in here is basically uh the highlights, based on the release, notes that we've created with captain uh with this version and the second video in this list is a quick introduction into the new shipyard specification that we've published with zero eight zero alpha.

D

Both videos are targeted for developers not necessarily end users. So if you're just an end, user hang tight until we have the final release, but if you're a developer or if you're building integrations with captain, then those two videos are for you.

A

Thank you. Are there any questions on those videos.

A

Not yet okay, cool.

C

So I will be handing over.

A

Probably to johannes.

C

Yeah also just a quick announcement. uh Currently there are a couple of discussions going on around captain project and also in regard to the 0.8 release for the final version, and if you want to contribute or also want to bring in your opinion on certain topics, please feel free to follow those discussions and to post a comment there. We are very help happy to see other ideas, thoughts and concerns that we then can also consider when building certain features.

C

The first discussion is about how execution plan services should behave and how they should get their events.

C

Then, today we started the discussion about whether it's necessary to have a mapping between a service and a sequence, or we should provide smart default, and the third discussion is about um the situation that the task that about the behavior of a task sequence and whether it should fail based on a failed task or not yeah. Please also take a look there.

C

Thanks to christian who provided the first, the first or the introduction into this discussion. Please take a look and feel free to post a comment.

C

All right, yeah, there's um jung. You brought in this item, but locust.

A

uh It's uh right now we are basically starting the discussion. It's um we are in the process of uh starting an integration for uh locust, for which is a load testing tool um similar to jmeter, for example, and uh we are reaching out in slack to find the folks that would be interested in joining us in the development we have already found some, and uh so we are initiating this. um We will start uh first gathering the requirements and then starts coding in the next weeks.

A

This should already be a service that should be developed for captain 0.8, so this would be a great start to.

B

A

Get familiar with the version of captain- and I think I have it in my clipboard here so um this.

B

A

Right now the discussion uh that we are having- and we also you- can also find the link to the git repository where we will start development um so yeah. I think that's everything for the for for this agenda item and the next one is an open question from the last meeting.

A

Should we streamline the versioning of the spec with the captain versions, uh very good question: um did anyone already has uh established, or do we already have some conclusion how to answer this question or we just keep it for now? We keep it as an open question.

A

I think the question came up because the spec is now developing, sometimes with spec version 0.2, but captain has 0.8.

A

Actually, in my opinion, it makes sense to have different versions of the spec and the implementation um as we, if you progress with minor versions, uh we won't change the spec for for minor versions, probably so it can live with two separate version: definitions, in my opinion,.

C

I think so too, because we had this discussion also on the level of the services, and we also decided um to keep the services separated, even though it's uh you need, then um a matrix that tells you which version of service is working with with which version of captain.

C

But at the end or at some point in time, it will not be possible to have everything aligned at the same version and it will run away.

C

Yeah, based on that, I think that we will keep two separate versions.

D

I I agree on that in principle, uh but we should make sure the versioning we use is semantic.

D

So with the changes we've done now with captain 080, we obviously also made a jump in the minor version of the spec, which is fine. But let's just say we at some point release captain 1.0 then I believe the spec should also be 1.0, because the spec in that sense is to be considered general availability and and production ready. I don't know how to call it, but I guess you know what I mean.

D

So uh maybe we need to align at some point, but that might create confusion again whatever, but this this could be the one way that we go with the spec version.

A

Okay, but if there are no objections, I I try to find the or to write down the conclusion um in general: keep it with separate versions, but for uh major versions.

A

We have to align uh g captain zero, implement spec, something like this that make it clear.

C

Or when there's a breaking change in the specification, then this breaking chain should also be reflected by a breaking change of the captain version. So to say.

C

When we jump from spec one dot 0.1 to 0.2, then it's a breaking change in captain.

A

E

A

Breaking change.

A

B

Make sense should we keep it like.

A

D

Yeah sounds sounds right.

A

Cool, so we can kind of close this question then, if there are no more questions on this, I will hand over to christian to share his captain service template.

D

Thanks, okay, so my contributions for the last couple of days. They start with the captain service template. I have uh taken the task to make sure that the new cloud events are also available in the new cap or in the existing captain service template, and this task has proven to be quite some work, because in our eventhandlers.go we had handler functions for almost every task, that or for almost every cloud event that captain 0.7 had and now with the introduction of all the started and dot finished events and even some more cloud.

D

Events in captain 0.8. There was quite some work involved, so I'm not gonna go for all the details in here. um I'm just gonna go through the rough edges uh of the things that I had to do, and I just want to mention that there will also be a documentation for the people or for all the developers that are writing integrations based on our go service template um on how to migrate from the old version to the new version.

D

um So the most important change- and this is also kind of a summary of what's new in captain 0.8- is that we have migrated to a newer version of cloud events. We are now on cloud cloud events, version 1.0, which is reflected in the cloud events. Sdk go v2, um so we're not the only ones that are mixing spec versions or that have mixed spec versions versus releases of libraries.

D

So that's good to see here, but that's this change and then, while in the old version of captain, we rely on go utils package lib for many of our helper functions and cloud events. We have introduced a new package uh called package, lib v020, which includes all the structs names and utility functions. You need to implement the new cloud events basically and just as an example, I'll go down to the test triggered event. This is what the chest triggered event.

D

It's kind of an event that didn't exist before to be honest, uh but this is what it looks like now. You basically pass in a captain handler. You have your incoming cloud event and you have your cloud event. Data which is a test triggered event data, and then you can start processing the event like you did before all right. There's a lot more changes into this pull request. So, if you're interested into the details, please take a look into this pull request.

D

It should be quite complete uh or if you just want to wait until we have the documentation, then that should be a lot easier, because this pull request obviously covers, like all possible cloud events uh that you need to go through and not just yeah, not just the ones that you might need all right. um Any questions so far on that or any remarks.

D

Cool the next one I want to share with you is that we've created a new github action. Actually, this is the first github action that we've created uh officially under the captain uh organization. This action is a helper for our internal or for our github ci pipeline that we're working with and all it does is.

D

It extracts the branch name based on the pull request that somebody is doing and the logic is as follows, or actually, let's, let's jump into the usage example here you can extract the branch name by using the github action like this, and then it provides outputs wire, the github actions, outputs feature and those outputs are the git commit hash branch slack so a short and cleaned or sanitized version of the branch name and the branch name itself, and if this is a pull request that this action is running on, then this will be the pull request id or rather it will be pr pull request id.

D

So um this this helps for automation like I want to build a docker image for a pull request, and I want to publish this docker image. Then it needs to have a good name or a proper version, and this this helps for generating such a version.

D

And basically this is the action and the one that's more interesting to look at is the pull request of our actual ci pipeline, so we're using this action right now in production.

D

Basically, what I did is this is the the bash script that was in our workflow for ci and with captain itself we obviously have the mantra of let's not write a bash script or let's, let's not write a script, but uh let's rather make something containerized or something testable, something reusable.

D

uh This is exactly what we did here. We basically just say hey. I want to use this github action that the captain team has published, and with that I also want to come to the conclusion. If you want to use such an action, you can obviously use our action. This is. This is basically publicly available for everyone, so this is not just for us.

D

Everyone can use it and if you google, how to extract the branch name from a github action, you will come across well, I guess he will come across this code and maybe this code- and we just pack this into an action basically.

D

All right any questions on that.

D

Cool um with that I'll hand over to florian all right. Thank.

B

B

Okay, today, I have uh three items to show you. The first one is a pretty cool extension for the distributor. So, as you know, the distributor up until now was mainly responsible for subscribing to captain events. So, for example, a deployment finished event or evaluation finished event or triggered events as well. In case, you want to write a action executor and perform a certain task.

B

That's been scheduled by the shipyard controller and the distributor is basically the best way to do so, so you can run it as a side car within the same pot as your cabin service. So that's the the best practice that we recommend to have them within the same pot and then the distributor will forward the events you are subscribed to to the cabin service.

B

Now, what we've added to the distributor is a function to act as a proxy to the captain api.

B

So that means, for example, if you are operating a cabin service outside of the actual cluster, where captain is running on, you can use the distributor to forward or to send api requests to the distributor, and then the distributor will take care of communicating with the captain api. So that means you only have to configure the access credentials to the captain api.

B

So the captain api, endpoint and, of course, the captain api token in the distributor, but you don't have to handle all that credential credential information within the captain service, so within the captain service, if you run it within the same pot as the distributor, you can just access the those uh free api services within captain via those urls. So, for example, if you want to communicate with the mongodb mongodb delta store, you can then access that via http localhost, 8081 and slash mongodb, datastore or for convenience. We also provide different uh synonyms for those services.

B

So, for example, slash data store or slash event store will also be mapped to that api service and the same for the configuration service. So this one is available under those three paths and the shipyard controller will be available via slash shipyard and slash shipyard, controller and yeah.

B

um Also, as part of this pro request, I've tried to um to rework all the environment variables and see which ones are not necessary anymore and which ones were maybe duplicated for similar purposes, and now that list of parameters is up to date and as you can see, it is quite long, but fortunately, for most of the usual use cases, it is sufficient to only set a very small subset of them. So here we have added two examples of how you might want to use them.

B

So the first one is, if you are running your captain service within the.

D

B

You basically only need to set the pops up recipient, so that is the hostname of your captain service, the port. It is listen, listening for events on and then the path and, of course, the pops up topic, and also the good news is, if you uh of course run your captain service within the same and, for example, use our goal service template that christian just showed us.

B

You also can leave those three parameters here empty because by default the distributor will be set up to correctly send incoming events to to the default uh host that and important path that the service is listening on all right, and the second scenario is: if you want to operate your cabin service outside of the cabin cluster, you obviously need to know how to communicate with the captain api.

B

So in that case, for the distributor you need to set the captain api, endpoint and the api token variable, then you can also set the interval in which the distributor will call for new uh triggered events, but that can also be left empty. So we have a default of, I think, 10 or 30 seconds. I think it's 10., so you don't necessarily have to set that then yeah. Those three parameters are the same as above so again.

B

If you use the goi service template and run within the same pot, you can leave those empty and then, of course, you need to set a pops up topic. The only limitation uh right now that we have here. If you run the distributor and the kept service outside of the cluster, you cannot use nuts wildcard syntax.

B

So in that case you have to specify each topic or each event type you're interested in specifically, and you can separate you can pass multiple topics by separating them which, with a comma all right, are there any questions about that.

A

uh I do have one it's a pretty amazing, pretty big thing. uh I I do have one question uh on the on the beginning of when you shared the different urls for the same thing, um if I was just wondering if it is a bit confusing, especially for the mongodb data store, since first we have the mongodb data store, but also data store is available and then event store, but written a little bit differently with the hyphen.

A

If it's kind of introduces a little bit of con or might introduce more confusion, then it might help uh if it's because data store and event store could mean kind of different things for for different people, that the data store is something like where you really log some some data or put some data and the event store, is obviously something where you only have events and maybe- um but it's just my opinion that maybe keep the the um the names that we usually had and I think we always called it the mongodb data, so I just go with mongodb datastore.

A

um That's a good point.

B

The reason behind this why I did did it like that is we do have an open discussion about renaming the paths to our core or to our api services, and here I wanted to maybe think ahead a little bit and in case we do, for example, rename the mongodb data store to slash data store. uh You can already use that path and do not have to update it afterwards.

B

So, but maybe it is a little bit confusing so.

A

Okay, yeah, maybe I uh um I take a look at the the ongoing discussions uh and uh take a look. What is the outcome of this and then we can maybe streamline.

D

E

I'm also on young's opinion here and actually, if you now take multiple routes here, you are reserving those words or routes for for future development. So if you really want, then a url url, which is called just config or configuration or data store.

E

This changes behavior then because it's already reserved for the mongodb event store. Basically, so I would not go with multiple paths here, but what's a good idea is to replace them. In my opinion, the word mongodb data store with event store and then it should just be event store.

B

Yeah, of course, we can also remove those options and only provide one path.

C

So yeah at the end, it should be aligned with uh the api and- and this is what we have ongoing and- and we are talking about discussing about- but um what I don't like is exposing a technical details, and this is, for example, where we have the slash mongodb data store. Why should we expose the information that internally we are using a mongodb?

C

This should not should not be part of the of the end point, but this is topic of another story. Sorry,.

E

What we can maybe do here really for 0.8, switch basically to event store, which is the clean path so to say, and have it for backwards. Compatible compatibility have also the mongodb data store still, which is just redirecting, of course, to the event store and in the next version, then we could get rid of that. Mongodb data store path because.

B

E

Recently, you've mentioned it's, it's exposing technical details and also for the other stuff. We don't do that. For example, the configuration service is just a configuration service and not the I don't know git service or something like that.

A

I like this idea.

B

All right, then, let's continue so. The next issue is quite a small one. So, basically, here the requirement for that was uh since we now are switching to different event types, and uh one of them is the evaluation.finished which will now replace the previous evaluation done event.

B

We needed to remove all the references to the old event types so, for example, in our integration tests, we had several places where we still reference those evolution done events and in this pr I basically went through all the places where that occurred and removed the references to those events. So nothing too spectacular here, but are there any questions about that.

B

All right, if not, let's continue with the last one, so this one is also not a really big pull request or a big issue. So basically, the refinement here was that whenever a test sequence is triggered, so when a new task sequence, the triggered event is coming into the shipyard controller.

B

It needs to check the version of the shipyard of the project. This incoming event is related to, and, of course, if the shipyard is not does not have the correct version, the shipyard controller should exit with a task sequence that finished event with a status of errored and results failed. So also the code here is not that complicated.

B

We can have a small, a quick look so basically, first we try to get the shipyard and then what has been added here is the shipyard controller will then validate the shipyard version and in case it does not match with the expected version, it will send out the finished event.

B

So that's, basically all there is to that issue. Are there any questions about that.

B

All right, if not, let's have a quick look at the working items, because there have been a few bugs. I think this fixed as well in this uh week, so the first one was an interesting one, so that was a memory leak in the shipyard controller and that was caused by yeah.

B

Actually, I discovered that when I was developing this new uh functionality for the distributor where it's port open the triggered events by the shipyard controller, there are noticed that the shipbuild controller for each request tried to create a new mongodb connection and didn't close it properly, and I fixed that with that pierre pr and also what was where dynatrace helped me to validate the the fix I've made is. uh I could use it to to have a look at the memory consumption of the service, so that is before the fix.

B

So, as you can see here very nicely, the memory grew then the part crashed because of an o m error. Then the memory grew again and that kept on happening. Then I made the fix and after that it looked like this. So for of course, at the beginning, the memory grew a little bit because it needed to establish the connections to the mongodb, for example. But then, even though the number of requests.

B

Was steady and there were more and more requests coming in the memory? Consumption stayed the same. So that's again, a very nice example of how uh dynatrace enables you to or enables us to self-monitor captain and actually use it to to develop captain and have an eye on performance issues.

B

All right and then I think there was another one exactly so.

B

We also discovered a small bug in the helm service where, in the deployment finished events, it was not sending um in the deployment finished events we do have the property deployment uri local, which is the uri within the cluster, where the service that was just deployed is reachable, and in that case we discovered that the port within the uri was not set correctly, so it was actually hard coded to 80, and I fixed that with that pr. So basically, what I did here is, for example, when we have this uh this service definition here.

B

So this is the kubernetes service specification.

B

uh We actually look at the created kubernetes service resource and go through all the parts, and then we take the proper port. So in that example, it is 80, but for example, if the created service is exposed at port 9000, for example, then the helm service will take that the the uh part that is defined here all right. I think that's it for my side, so I'm handing over to.

E

Bentham, thank you. Let me share my screen.

E

um First question: is this uh big enough to see that's good? Okay, um I've I've. Only I've picked only one uh issue which I think is worth mentioning today.

B

E

I've worked on this in the last week and I think the pull request is still open, but I can show you that one and it's uh about a new, I think very convenient feature, uh especially in development phase, where you can use the cli to basically watch the event stream of some of the commands you've issued via the cli.

E

What do you mean by that? Let's look at my environment here I have a locally compiled version of my cli.

E

And on my cluster, I have just one service: the cast db from the examples running here in stage dev and uh let's just.

E

Let's just send a new artifact for this service. You all know, of course, the send event new artifact command.

E

What's new here, you can actually provide here a w flag, which means a watch flag, and if you do that, you can actually watch the events which are resulting from your our command. So you see the cli is not terminating here. It is basically continuing to uh pull for events which are resulting you see just one when event came in here additionally- and I think that was already the last event.

E

So this is useful uh if you just want to yeah, take a look. What events are firing in the system when you do certain stuff on your cluster, where the cli.

D

Ben, yes, do you have jq installed on your system? The command line will check you.

D

Maybe perfect. Can you just use the command that you used before and pipe it into jq.

E

D

Yeah, let's see if that.

E

Okay, that's probably because the first line says something about incompatible version, blah blah blah.

D

Yeah, probably we will, we would have to check. I think if we print it to std error, then uh only the the json payloads uh will be passed, but uh we should take a look at this because uh with jq this gets an awesome, formatting and colorized, and everything makes.

E

D

Nice for demos.

E

Yeah, that's actually another topic. We need you had to address, uh so this one should uh probably not go into that standard into that stream so that we have machine processable output actually, but I think there's an issue for that. um What you can do here is because you can also output this uh the yammer in the yaml format, which is, in my opinion, much cleaner than the json output, so I've just issued here the same event event new artifact with the armor.

E

As you see it's quite fast already finished again um it outputs the stuff in yamaha. um What else can you do um sorry for interrupting.

A

I think it's pretty cool, but the only thing uh or that might be missing is a kind of a delimiter so that jq or or in yammer vomiting.

B

A

They know when the event starts an event ends because I think, um maybe in the payload, the the uh json payload just a comma between the events or maybe have it as an array and then a comma between the events. So you can process it and in three dashes here between the events. But I I'm not 100.

E

Sure but that's a good point. Thank you.

E

Where was I uh yes, um you can actually use that double you flag, also in other commands. If I take a look at captain, um let's.

E

Captain event here.

E

Captain get event um we need to pass in a captain context for which the events gets filtered. um Let's just look at this one.

E

And it should give us the exact same list here afterwards, so.

D

E

D

This fetches all the previous events and then keeps on watching for new ones. Right. Yes, yes, yeah and.

E

What you can also do, actually you see this one, the default behavior of this command or of this flag is it will fetch events uh for an infinite amount of time? Probably you want to stop after five seconds.

E

You can do that with by the minus minus watch time flag, and now it should actually stop after five second seconds. It did stop here so uh where's this command else available or this flag. um I think it's the send new artifact command, which we already saw, then the get event- and you can also issue this one, this uh this flag with the start evaluation command.

E

So it should also be reflected in the help. Okay, good and.

C

There is a typo, an event.

E

E

Sorry, okay, like this, you see here the description of this flag, minus dot or minus double minus minus watch, print event, stream or here the time out which defaults to max int or something like that, and it's in. Second, um that's, basically it. I think this feature is especially when you're developing stuff and wanted to try out things and see what events are fired in the system. It's quite useful.

E

I already use it to trace so to say the events when I develop a new feature, so this was incorporated already in the cli, as I saw the book request for this is still opened.

E

The main logic for this is implemented in go utils when you are developing a service, and you want to use this feature, you want to just uh repeatedly boil for events filtered by some filter.

E

You can use the implementation here. I've updated the readme file in the go youtubes. uh Until now, it was only possible to query events from the event store. This was done like like, like shown here, you basically creating a event handler.

E

You create a filter and then you ask the event handler to fetch the events which are matching the filter. Now, what you can do is uh you can wrap so to say so to say the event filter with the event watcher. You can create it like. Like this new event watcher.

E

You can, of course provide it with an event filter like the same like above you can provide it with an interval so so that you can control how often or in which interval that watcher will fetch your five seconds.

E

So this stuff is all optional. If you, if you don't provided their defaults for that, you can provide it with a start time.

E

So the oldest event will then just fetched from this timestamp here, and you can also provide a timeout, for example, 50 seconds. If you have a watcher, you can just start watching. You provide it with a go context and it will return you and go channel, hear all events, and you can just loop over this channel to get the events then we'll also provide you. It's not shown here a cancel function where you can actively cancel their polling.

E

If, if you need to very nice, that's how you can use it, I think it's a good component. If you want to support such a use case, that we have now something which we can reuse and which is basically bulletproof, and because there are lots of things which can go.

B

E

uh We should not do this every time from scratch, so please reuse this event watcher or extend it if you, if new uh features, are needed here and that's exactly how it's always implemented or incorporated into cli, it's basically just this code. That's it!

E

Okay! That's from my side.

E

If there are no questions, I will stop my sharing.

A

Cool um yeah thanks everyone uh and thank you. uh Are there any other comments. Questions this community.

D

A

D

Just to add to the discussion about check you, I just checked, you don't need a comma. Jq works. Fine, it's really just a log message at the beginning. So, okay.

E

D

A

Okay, so if there is uh nothing to add here, then uh thanks everyone for joining me and uh see you all again uh next week, next uh thursday, and uh please everyone uh go ahead and try out the alpha release and watch the videos from christian. uh That will give you um a good head start on the new format of the shipyard.

A

Okay, thanks, everyone see you next week, bye.