keptn Community, 22 Apr 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Keptn Community & Developer Meeting - April 22nd, 2021

Description

Meeting notes: https://docs.google.com/document/d/1y7a6uaN8fwFJ7IRnvtxSfgz-OGFq6u7bKN6F7NDxKPg/edit

Learn more: https://keptn.sh
Get started with tutorials: https://tutorials.keptn.sh
Join us in Slack: https://slack.keptn.sh
Star us on Github: https://github.com/keptn/keptn
Follow us on Twitter: https://twitter.com/keptnProject

A

Hi everyone welcome to this edition of the captain community and developer meeting it's april 22nd, uh and uh we have uh quite an agenda for today. uh I think.

B

It's been a long time since we had a.

A

This long list of uh agenda items.

B

So I try to keep it short from my side.

A

uh First part, please add yourself to the attendee list and please follow the the cncf code of conduct here, since captain is in cncf project.

A

First part that I want to announce is that we have sent out the survey um the last couple of weeks on the captain newsletter and we now rework this and we are coming up with a new newsletter for it. So please make sure to also subscribe to the newsletter.

A

The link is here and uh the cadence will be now at bye, weekly cadence, with all the latest information on the captain project with latest releases, latest blogs, tutorials uh everything you want to know about the project about the captain project new integrations, for example, everything will be.

B

A

Newsletter, so please make sure to subscribe to this newsletter. The link is already live. uh We put it in here live today. uh Please subscribe via caption.sh.

A

Also, please update your captc installations. We we've been in contact with the quite some captain users, and some of them are still running um about seven. Something releases. Please make sure to update to the latest version. uh Maybe the latest version is not even possible because it's just being released right now, um but if you're have a dot, seven dot, something running, uh please make sure to upgrade to 0.8 um and please make sure to keep an updating captain version. This helps.

B

A

Supporting you also and the last thing I want to announce here, the other thing, so you can just read it in the meeting document, but we are looking for a new code owner on a very important project of the captain country organization. It's the notification service right now.

A

The notification service is a little bit left behind, but we know that there are some um organizations out there and some folks out there that are using the notification service and if you like, to contribute uh to captain- but maybe you feel not ready yet for captain core, then this notification service is the perfect place to get started uh right.

A

Now it's basically covering three different notifications for microsoft teams, for slack and for webex, but if you feel that you're an expert on microsoft teams or you're an expert on slack- and you just want to have this one functionality being implemented- and you can take ownership of this- please reach out to us- we're happy to assist you here um if you want to or if you want to help us in upgrading the notification service and also then maintaining the notification service for your favorite notification uh platform.

A

That would really help the project and really help this notification service. So if you, um if you want to participate here, please let us know you can also just put a comment here in in the chat I see quite some folks are joining today. So this is kind of my ask for today and now I've taken four minutes of our meeting time. I hope that's still fine, I'm already handing over to johannes the other things you can read them, but my two asks for today.

A

I really sign up for the newsletter that will be the first newsletter on monday. It will have great content. It will feature the community rockstars and all the latest uh updates on the project, and the second ask is uh please: uh if you are interested in contributing to captain, maybe the notification service is for you.

A

Okay, with this, uh we can skip the other two parts um and I'm handing over here to johannes for the release. Notes. um Do you want me to click on this, or do you want to share your screen? Let.

B

Me share your screen.

B

B

uh Yeah you can already mentioned it uh today is a release day and we are excited to get the captain 0.8.2 release released.

B

The last pieces are currently getting finished so, and we are confident that we will get it out the door by tomorrow and, let's just take a look at the key announcements of this release, um it's a patch release with not that many um feature changes but uh really cool things that we as a team, have accomplished.

B

First, with this release, we did some changes in captain, especially in the helm chart of captain, so that we are now able to deploy captain using captain. um This should also help us as a project to drive the vision of having captain's delivery tool as we are using than it by our own and new features or new um ideas. We can then test out based on our implementation uh and the entire ecosystem, and the entire community can take a benefit of that.

B

A great outcome of this was definitely a github action that has been implemented. I linked it here. It's the git of action send event, and this one allows you to send a captain event uh using github.

B

This is the closing the closing link of having your ci, which will be obviously on github, linked to your captain deployment.

B

Next, to that, we also did some improvements in the captain core in the sense of hardening captain components, and one important note on that is that captain services are now or containers are now running as non-root users to make it more secure, and some changes have been made in the role in the airbags of kunitz and on the bridge side. There are several several enhancements when it comes to the service screen, as well as to the sequence screen for the sequence screen.

B

um We are now able to better visualize uh multiple sequences that are executed in one particular stage. They are now separated and makes it easier to follow the sequence execution and in the service screen, as already have been shown in the last review, but just let me mention it again: um there we now show not the list of executions that happened in the past, but rather their list of deployments that are currently running for this particular service.

B

This should also make it easier to see the behavior to see the status of your service and to get an understanding what is currently deployed or where an evaluation happened, all right yeah. I have to add a screenshot for that, but this will happen tomorrow and please take a look at all the bug, fixes and minor changes that have also been implemented in this release and will then be shipped by tomorrow and, as usual, to upgrade your captain installation.

B

Please use the captain cli with the command captain upgrade and distance automatically takes your captain or lifts it from a odot 8.1 to an 0.8.2 version, all right yeah. I don't want to take too much time on that, as we will see a couple of features I mentioned here also now in the review- and this is the point where I want hand over to a little bit showing us what she did on the notification bar in the captain's bridge.

C

um Hi, so I have to admit um I just needed my cluster now I can't get my local, uh my local environment running, so I have to skip for now, but um the summary is the update bar is now uh darker blue, with white.

D

The poor request- and I added the screenshot there- maybe.

C

Oh yeah, so oh that would be great because.

E

E

Do a single screen, okay, so this is. This is now the new notification bar and yeah.

C

Yeah, oh so so, basically yeah the colors changed and- and what I also did is that there is only one notification uh left uh before I think there were perversion, one notification bar. So if you were more than one version behind there would be like four or six or something like that. So I think that's more a bit more condensed. Now.

C

Yeah thanks for sharing.

B

And the link is also working now correct. Yes,.

E

B

There I get to the upgrade documentation perfect, because this is what the user wants. He wants to get directly forward to the instructions to follow great.

E

Okay, so I will go on so my part uh is the new uh the adjustments to the chart type of the evaluation uh before it was not possible. uh Yeah there weren't any just about.

E

There was just one thing, one as slow shown in the over overlay, and now there are all active uh slos shown so response time or response time, uh 90, etc.

E

Additional to that, uh the value also shows uh two decimals instead of zero and also the the y-axis on the on the right uh changed, so that also lower values uh are more visible and not with an interval of 50 or something yeah and that's basically, that's that's. Basically it.

B

um Claus you have here very, very long, sli name. Can you click on that? Yes,.

E

How does it behave it? It's adjusted so yeah, it's the greater one. So when I disable the greater one, you can see more the uh one with the less values.

B

And also the entire name is shown in the overlay in the tooltip, correct, yes, yeah and also.

E

uh To mention now the display name is shown in the chart instead of the metric name.

E

Okay, that was that was the message, so I will hand over to a question.

A

Thank you so sharing.

A

Okay, um so, as you heard, we are working on a cra two release and I have contributed two quick bug fixes to it not really noteworthy to show then the next one I just want to quickly talk a little bit about it. um We were alerted about the codecof.your security issue. This is all public, so you can go and read it up. If you want to um essentially the way we use code cough is a bash script that we're loading um wire, curl and then execute this.

A

This bash script and kodkov has notified everyone, essentially that this bash script was compromised for a couple of days and that environment variables were sent to external services and yeah. So what I did is I verified which environment arrivals did?

A

We did we passed to this script or at least uh which one did we have in the um github action and whether any tokens credentials or whatever were among those environment, arrivals and luckily, for us, our setup um was uh in a way that we didn't leak, any um security, relevant credentials or anything um I can actually.

A

I think I can show this here because I tested this, why I have pull requests, um and this is basically the way that uh that they said that you can test it with. So just instead of calling the bash script just check with and which environment variables you have and if the action is still here. Let me.

A

A

So here's the coverage report yeah there we go there, you can see which environment variables we have- and I think uh 99 of them are set by github automatically like the selenium, whatever we don't use, selenium, github path and etc, and I think somewhere down below. There is some things from us from us.

A

Although I think in in this case there is not and like. This is one way to verify it and the other way to verify. It is obviously to look at the source code of this action and see if we pass any um credentials as environment variables, and also did that. Then the answer was no like. We don't pass anything here.

A

um Another one also related to github actions. We had a contribution by jos, josh um and what he basically did is. He enabled pull requests, or at least the ci workflow for pull requests on various event types which looked fine when this pi was merged, but it turned out that it led to duplicate, runs in the ci system uh on certain events. So, for instance, if I open a pull request and I label it immediately- I would get two runs, and this is not just true for when I do it.

A

This is also true when a bot like dependebot does it uh so we get many many duplicate, ci runs, which is just uh not worth the trouble, and I looked if it is possible to detect duplicate, runs and then cancel the second run. But the answer is this is not easily possible, so um I have basically I had to revert this change and I have removed the types setting here at all.

A

So now the pull request, cip, is running when the pull request is opened or when a new commit is pushed, which was the default case before any questions. So far on those two issues or um tasks.

A

Okay, uh right and last but not least, um I did some small contributions, mainly cleanups or um some ci task updates on some captains and books repositories, which is also not really worth to show, um because this is really like one or two lines of changes for like 10 repositories and that's about it and yeah. With that I'll hand over to amin.

D

Thank you very much. um I'm going to present uh the pro request regarding the beta sequence visualization when having more than one sequence uh in a stage. Actually I showed you that one already uh in the last community meeting, it's yeah in the case that we have multiple sequences configured in one stage. Previously all the tasks were just listed and yeah us user had no idea, but the next sequence started and now, with the yeah new feature or uh in the new bridge.

D

um You can see, for example, in yeah this case we have a failed delivery and the rollback sequence is a separate sequence that was executed on the stage and yeah. You see those two sequences.

D

What I'm showing you uh now, what has improved since last time is, as you can see, when I click on staging the last sequence that was executed is expanded automatically, for example, when I check the def, so the last sequence here is expanded automatically, and what you also can do is, for example, if I'm interested into that task, as you can see, when I click through the tasks, the link, the deep link is also adjusted, and when I use this deep link again, it will open that sequence and focus that task so expand the sequence where this tasks is task is in and also expands that event there yeah.

D

Also you have the option now to have a look at the um the payload of each event, and you can also have a look at the payload of the sequence event, so the sequence triggered event and also the sequence finished. One is here uh at the very bottom, so you have uh yeah insights also into the sequence, triggered and finished event payloads from this screen here, yeah.

D

um That's regarding this pull request, and one more thing that was um uh yeah done uh during this sprint. uh We had that bug uh that was reported. um Actually, uh this was an issue in the core when, for example, for tests, multiple tasks were executed, the the sequence was continued on um the next stage.

D

uh Now, uh if the test fails um or if one of the tests at least fails, the sequence will stop there. So actually, this was fixed by um yeah. The pull request here from band and uh now that the sequence or that stage is marked as yeah failed and um yeah. No uh further uh sequences will be executed uh there, um yeah that this verified, so uh that works now as expected.

D

um Yeah. Actually that's all from my side and I'm handing over to johannes.

B

All right, then, I'm back again. uh What do I have to share? I also touched some code in this sprint, um not not really called but more manifest. First of all, I splitted the kubernetes whole captain configure bridge into two purposes, one purpose for the secret management and the other one for um restarting a pod. Why was this necessary yeah? Because we had an um security assessment and there we got there twice, that it makes sense to split up this role into two components, because there are two responsibilities uh baked into.

B

As I said, um the pot handling as we have here and then the secret handling as it's covered by this part- and this is important- and this is the best practice to have all secret related.

B

Permissions also in a separate role- and this is what I did- I created a separate wall role, one for managing the bridge credentials with these permissions and another one for managing the bridge. Part that is allowed to get a get the port from the kubernetes api api deleted and also delete the.

B

Collection, um I think not, I think, can't really show that it's just uh what um changes in their role of of kubernetes and the second one was very similar.

B

Here we had the situation that the captain captain configuration service had permissions. He actually did not require he had all permissions to read a secret or to manage a secret, but he just needs uh read access, and what I did here is I removed.

B

Let me just show you: um I created a captain, get secrets, role and um yep assigned it to the kubernetes uh to the configuration service so that he just has the permissions to read and not to delete or update or do any other things with services, as he does not require that.

B

All right yeah, these two were changes on the on the health chart on the captain manifest, and then I want to bring up an additional one.

B

I think that one was overlooked a little bit, because we also did changes on the daily version check, and let me just show you that why was this required, because the last time we rolled out captain when we rolled out captain auto8.1 a couple of users kind of asked, why they don't see the notification on the top and we, the only reason, was that they declined our daily version check mechanism and yeah.

B

This was kind of a reason why we thought that it would make sense to make it more prominent for users, especially for new ones, and to also accept it or to accept this feature and to make sure to get notifications when a new version is available.

B

This also should help to make sure all cap users update their captain environments and to have them on the latest greatest version.

B

Here you get now the recommendation that you should accept it, and some changes have been made here in the text as well as here on on the um username. So here in the top right corner. There are also some changes made in the text as well as year. This toggle is enabled by default and not disabled anymore.

A

I have a recommendation or an idea here now that we release every four weeks, or at least that's the plan, how about we notify the user? Let's say six weeks after the installation, hey by the way, you've disabled the version check. If you want to get blah blah blah- and please enable it again because somehow we can, we can estimate that there is a new version.

B

um Correct um and thanks for bringing it up, because I forgot that we do that now, after a couple of days, we delete the configuration order. We we again tell the user, uh you don't have this enabled when you do that you get notifications about new releases. How many days is it.

A

B

A

Weekly week, okay, let's, let's see how users respond. That would be annoying. For me to be honest if it is weekly, uh but let's say every four weeks or something because that makes more sense for our release schedule, but let let's.

C

See how you react.

A

To it, correct.

A

I think it's a really cool feature that we now enable this, um as people were not sure uh how to enable it again, and uh I think it really helps them.

B

A

Yeah, if we get uh responses that it's uh uh it might be annoying for someone, we can always change it.

A

B

All right and then um since lauren is not here today, he asked me to show what he did in this sprint and yeah. Let me go over his issues. First of all, he worked on a github action that allows us to set up a captain installation on a kubernetes cluster.

B

We call this one, always the master, cabin or main captain, as in the future. This particular cabin will take care of doing the delivery, and let me just show you what florian did here, what he was working on.

B

That one is still not.

B

Merged oops, I'm sorry, I'm I'm on the wrong repo.

B

Yep, for some reason, I'm on the wrong one um that one um I messed up with the links. As I said here, he pro as an indie captain organization. He provided a github action that prepares who need this cluster on tke and then installs captain on there and additionally, you get all the features and scripts like installing istio.

B

Then we also decided that we want to have on this cluster an environment that behaves like a remote execution plane and with this script you can set up a remote execution plane.

B

Then um this one, as the name already says it, sets up the captain project with doing a captain grade project antenna captain create service, and there are then yeah. That's basically it setting up the kubernetes cluster installing captain on top and then setting up the project and service so that we are then ready to trigger the first deployment.

B

And the other action that florian worked on was then setting up self monitoring on this particular kubernetes cluster and we decided to go with dynadress for doing that, and this is actually linked right here.

B

You can also reuse that action when you want. um You can bake it into your ci pipeline, and so you can then deploy dynadres on your cluster and also set up or deploy the diameters sli service and the dynadris service, as they are used by captain, so that you have done a full monitoring available and, as I said, that one is currently not merged, but let's just um take a look. What he did yeah he provided the action itself that takes a couple of input.

B

Parameters like the dynadris tenant api token pass token and then an option whether to enable or disable self monitoring or the name of the project. There's version of the dynadress service and time address sli service and based on these parameters. It then sets it up uh the monitoring and monitoring on your environment and um yeah. This is basically basically done by this deployed: dynadris dot, sh script.

A

By the way, because you just have this open line 30 to line 38, this is a good example that people can improve on not criticizing here. Just saying it could be improved because you here have the api token and the pass token as an environmental arrival, and that would be an attack vector that was used for code coverage, dot io.

A

In our case, I guess it's not that much of a deal, because the deploy uh dynatrace's h script is in the same repository as what is environment is defined. But the moment you load something from an external service here you basically risk exposing such variables, so I'll actually keep that in my mind and uh see if, uh if we can improve something here or at least recommend it to to florian.

B

Thanks for bringing it up, yeah uh security is always an important point to consider and when we can improve especially at these two lines, then uh we should do that.

B

Christian, do you know how to address that? One.

A

Well, for one, you don't need it as an environment arrival. You could pass it as a parameter into the script. That only is that's like 30 percent better, but not 100 percent better and the other option I need to investigate. uh It basically depends uh who needs the secrets, and maybe you can handle the secret outside of the action somehow okay, but this is really. This is now within the action itself.

A

So uh unless there is any code loaded afterwards shouldn't be a problem, but as soon as we load some code from outside, obviously um it could be a problem so right now this is safe. um That's what I want to clarify, but we need to make sure it will be safe in the future. When we do some changes as well.

B

Got it thanks all right yeah and with these two actions in place now, florian also got or I went one step further because he combined everything into a github workflow, and this one now allows us to create a cluster, install cabin setup monitoring and trigger the first delivery. And let me show you that, based on his repository.

B

D

Has it currently.

B

On his private uh account, but will be moved then into captain, but what we can see here in the main dot jamal of this work workflow I mean it's really straightforward. As I said, it's first set up captain in the sense of creating the cluster and installing then this particular captain version. This is, then, the captain that takes care of doing the delivery.

B

Then he pulls in the second second github action, which sets up the monitoring on that particular environment, and here he passes in the token and the tenant and the api and the pass token as we currently discuss. This can be improved in the next step.

B

This action updates a couple of resources to make the deployment then unsuccessful already, and therefore shipyard will be added, as well as a load test for gmeter and last but not least, this is the final task.

B

This action triggers the delivery by sending a captain cloud event to this particular captain installation.

B

This is a typical event and the outcome is then, as we can see here in this environment, we have a dev, hardening and production environment. Additionally, an environment that behaves like a remote execution plane.

B

Just to let you know um this one should be from a logical point of view on this order, and I think we are currently working on an improvement on that, so that um this production environment is closer to that because it it kind of groups or should be in this particular group, and what we have now right here is a deployed captain installation in depth hardening and not yet in broad, because here we set up an approval process where someone has to take care of accepting it and then triggering the delivery.

B

And let's just take a look into this delivery process because, as I said, um the script also uploads uh jamita scripts and they are executed, and we have also an evaluation, then are triggered to get more insights about the currently the data store. And here we are taking a look at the cpu time. Error rate io timeout response time as robot, and as we also want to integrate this particular setup in our ci, so that we have then daily deliveries.

B

We get more and better insights into captain itself, so that we also can improve there and also be more also get informations when something get worse, yeah and I'm really looking forward. This is, I think, the last missing piece on that story to also link the github action. I showed you right here that one with our nightly build so that we can deliver a captain on a daily.

B

B

All right yeah, this brings me back to our agenda and now we are actually at the end with the last item and are now open for comments, discussions or feedback. Please feel free.

A

Wow there is, it was a lot of content, really cool progress, um so right now there are no questions coming in. So from from my side, just the kind of the ask please sign up for the newsletter. We will start on monday uh to send it out and it will always have the information on the captain project like also on the latest releases. So if you want to stay up to date, please make sure to subscribe to the newsletter the link.

B

Is also in the chat.

A

And you will find it also on captain.sh community. um You will find that the newsletter there um if there are no more questions coming in. I think we can close it a little bit earlier and I'm really looking forward to the new release to execute my captain upgrade tomorrow and everyone that is also running a 1.8 installation of captain or all the installation of captain make sure to upgrade and to always get the latest and greatest of captain.

A

The team is really working hard to provide you the best experience and therefore make sure to also receive all the new features.

A

Then thanks. Everyone, I think, uh for the other parts um we kind of uh also can keep it for the community focus meeting for next week. So we are closing it here and thanks everyone for joining, see you all next time.

B

A