Add a meeting Rate this page

A

Hello, eric hello.

B

Hello: everyone, hello, hello, jurgen. How are you hi? I saw a boy hi eric stuff.

B

Let's wait a couple of minutes to see if someone else joins. Okay,.

C

Probably alois will join a little bit later, but he has another conflicting uh meeting. um He just wanted me to announce that he might join later, um but.

A

uh Yeah he's involved.

B

On.

C

On slack, so we can also update him later on, but it's.

B

Nice.

C

To meet you both.

B

Same same same, hopefully, we can start like collaborating now. uh I think that so far we are just kind of like been trying to organize the work and uh we are really pushing hard to get something up and running. So I think that in the following like couple of weeks, it would be great if we can just create the most minimalistic thing, with the vocabulary that we are trying to define and then just iterate and refine. And you know these are just basically like the basic first steps into like interoperability.

B

So.

C

Sure, and how do I pronounce uh your first name- uh is its salah boy.

B

That's all.

C

Right.

B

My first name is mauricio okay, yeah he's here the other one is a nickname. It's just a nickname, yeah from when I used to play counter strike. When I was a kid.

C

Got it okay,.

B

So hey mathias.

C

Hello.

B

um I'm not entirely sure who else is joining. I think that viva is also joining. uh He was really interested in working on the tecton side with andrea. So maybe we just need to wait a couple of minutes uh more uh you can just for you to know. I mean we try to keep these kind of meetings as short as possible. uh This is more like a sync to make sure that we tackle all the you know, all the all the things that are happening and uh just to move things forward in general.

B

It's just a thing: it's not a meeting just to have a meeting, and uh we usually tend to discuss here like what's going on. What's what are the next steps uh and the things that we talk in a slack about like okay? How do we organize ourselves? So we can see some progress.

B

Eric do you want to give another like another angle of that, like what do we do here like in these meetings.

A

No, I absolutely agree, and now the focus is as much as you said it's on the poc here to see if we can get anything uh ready for the silicon before silicon to get to like capitalize on the attention that we will get from there to get more people into this work. So yeah, I'm looking forward to that as well.

A

Yeah before we focus on the poc we could like yeah. Sometimes we brought up a couple of discussions from github and focus on that a bit etc. But now the focus will be on the plc, because that's the short term goals.

A

Do you know if.

B

Emily is joining today.

B

He doesn't seem to be here, which is strange because he's always early. uh No, he wouldn't he wasn't joining today. Okay, that's good! Okay! So then I think that, well, maybe let me give up just to see if he's joining. uh I think that he was really trying to push some stuff forward as well. So.

B

Let me see if he joins us.

B

In any case, uh let me see if I can share the link for the meetings like the meeting notes, and we can take some notes because the conversation is being recorded, so I think that it should be okay if we also take some meetings and meet ignorance and can like record what's the plan, what are the action items?

B

uh Let me add, I think that you can add your names in there. Let me copy so today is first of june.

C

Right.

B

ah And yeah topics here.

B

So one big topic is the poc every pr use case and.

B

Okay, I definitely can't share my screen if people want to see I'm not entirely sure what's going on here in my laptop, but let's see if this works, I need to I'm in a new laptop, so I might need to close zoom and enter again.

A

There we.

B

Go yeah be back in a second. I need to close to my entire game to be able to check.

D

It.

D

Hi everyone.

A

Hey look: marriage is just reconnecting yeah. There is.

B

Sorry about that, I'm back welcome all right uh and then we have bebop now hi viva, hey mauricio. How are you doing sorry for pinging you in slack, but uh I think we wanted to start and it's.

D

Good to have you here, if you thank me, so I I just uh right now perfect.

B

Thank you for joining us. Let me share the screen, uh and unfortunately, andrea is not here. Andrea is usually more on the technical side, and he kind of like also is looking at the pull requests eric. I really appreciate your comments and be well as well.

B

I can add here the link to the pull request, which should be in the repository.

B

I think that we have that pull request somewhere, and I think that I wanted to what I wanted to bring into the discussion is: uh what are the next steps to manage this pull request? I appreciate all the comments. I think that I've closed most of the comments already uh I'm happy to go back again and check.

B

Yeah, I think that I managed to okay, so I have very comments here.

A

I only had minor ones good, very little.

B

Yep, so this one is done and let me add this link into the document.

B

uh And that's pretty much it, so let me see go back again, so the idea of this pull request uh jurgen. Just for you to know it's a pretty simple go library uh that is creating the cloud events mapping uh for um for the events that we have here in this repository in sig events- I don't know if you have looked at this repository before.

B

So if you haven't looked at this basically, this is where we are just having the discussions about what are they?

B

I don't know why it's not going to code here, but if I go to code here inside this directory vocabulary draft. This is where we kind of like trying to define what are the events that we want to focus at the beginning, and if you take a look at, for example, here we have a bunch of of you know of common like continuous integration, events like about bills and test cases and artifacts being packaged and published.

B

So what what we wanted to do is we wanted to see if we could have a library that allows us to create these events and to admit these events and probably what viewer, what viva and andreas is doing people if you can put collect the link of the of the tecton controller. I know that there is an issue now where we are discussing that in tecton right.

D

Yes, uh so there is, there is an issue made for the cloud events controller specifically, and there is a poc which I'm making for emitting cdf events or using the controller. So currently we are working using a pipeline run and if the pipeline redone will have like an annotation which says like okay, uh this cdf event needs to be sent if whatever happens on this pipeline run.

D

So we are currently working on that and uh I have scaffolded the pipeline run controller up outside outside the or tecton pipeline itself, not in the main repo, because it has to go as like an experimental first and then it will be merged into tecton.

D

So my next step would be to like uh include the annotation reading thing. So today I worked on the scaffolding part and next I would work on that thing. But okay, I'll just share the issue or actually I'll just apply it here. One second, so.

B

Yeah, if you can share it here in the chat that will be great, so we can add it here and also if you can share the link of the of your kind, like your repository, where you're working on the controller, that's great as well.

D

So I will be uh creating uh so I'll so I'll be creating a pr to like the tecton cd, experimental, repo.

B

Whenever you have that pr, please put it like that in the slack, so we can all take a look.

D

Okay, yeah I'll do that.

E

And.

B

Also,.

D

I'll also.

B

Go tomorrow.

D

Okay,.

B

Once andrea's.

D

Back and he.

B

Yep definitely, and I'm I'm guessing that from that pr, you're going to be consuming the library right.

D

Yes, I'm.

B

Hoping to so that's that's why. I think that we need to discuss when to merge this, because I think that this is the same that we did like.

B

I think that we should kind of try to follow the same approach that we did with the vocabulary like documents, it's like to merge it like weekly, with very small content and then can, like you, know, keep iterating on it until we get something that works, and we will mark it as a draft or something, but at least we can consume it from on from other projects like, for example, in this case tecton, and then we definitely need to figure out what to do on the on the captain side.

B

Right and again, this is just because uh we are uh trying to create a poc. We can definitely go to jenkins, for example, or to other projects, and just pretty much do the same right, because we have also the jenkins.

B

We have the jenkins uh google summer of code, uh a student that is working on something pretty similar, so we need to uh definitely guide them to kind of like follow the same approach that we're following with tecton and with whatever we do with captain to listen or consume events.

B

Does that make sense everyone like eric and matthias that's kind of like a conversation kind of like I think, okay, okay,.

C

So from what I've seen from the uh just briefly, the the events that you defined here very similar to the events that captain has internally. So I think that should be really possible to align these events and probably in the first step, we we need to kind of have some some small translation service that just translates service. uh The.

B

Events from.

C

One implementation.

B

To the.

C

Other implementation.

B

And I think that maybe jurgen that's kind of like what we can work together. So if we have andrea and viva working on the tecton side, we can create this small translation layer at the beginning.

C

Yeah.

B

And then we see if captain is interested in just adopting that internally or whatever like we can create an experimental component in the same way that tecton is doing.

C

Yeah, I think, as a poc, this kind of external or this additional component that will be necessary for the plc, but in the long run we we should um think about how to kind of merge the specifications and come up with one specification, as it's also shown in the in the figure that eric shared in the dislike, where it's really. This cd events.

B

Perfect perfect- and I think that yeah, that for me, makes a lot of sense. I mean that's easy to create uh and I think that we can just get it done pretty quickly and uh if techno is kind of like, including that into their pipelines, we can just have a kind of like an end-to-end thing that we can show pretty quickly um and that's why. I think that the next next kind of important conversation that we need to have is like.

B

What's the use case right, we know that tecton is running pipelines and they are kind of like annotating their stages or connect their tasks into emitting cloud events. Now what do we do from the captain side right like? What can we actually do from it and what kind of events would we need for doing that and uh yeah by looking at the captain docks, I saw kind of like that.

B

You have like the auto remediation kind of thing, and I wonder kind of like what that involves and kind of like how do we trigger that and how do we can like configure like the actions here? Let me see I was looking at kind of like this documentation, which I I'm hoping that that's kind of the one that I should be looking at for kind of like what do you? uh How do you define your remediation and then how do you kind of create actions for them right.

E

Again, hello for my side, sorry, hello,.

B

Hello.

E

Cncf meeting, at the same time where I had to present no worries, um I think we can do two things so one of the ideas when we work with pipelining tools, we usually use highly generic pipeline and uh kept very often parameters these pipelines. So that's what we do in the shipyard file to give an idea.

E

You might have a definition on how to define a production tenant or you roll it out, but captain would then tell you which candidate actually wants to deploy or how you generally handle the stage and that captain will handle which dates to deploy. So for us, a pipeline is actually not a stage for us a pipeline depending on what it's doing is actually what we consider a task in a sequencing captain.

E

It's almost like a bit of inception, but we would trigger do a deployment for us and we would work with what we call micro pipelines from from an idea more or less so you dissect pipelining what they're actually doing, whether it's a test execution. It's a deployment, it's an environment creation, whatever you wanted to do it and the pipeline itself is only parameterized and what what captain then would do it calls ecap.

E

We call it the service, but technically it's exactly this event where we push it where it supplies all these parameters, then, to the to the underlying pipeline and and to your point, we can either do this for a pipeline for a deployment for a test for a a promotion. I think one one thing that we one of the reasons why we built it. That way is that we can easily add additional tasks that the original pipeline knows nothing about.

E

I can, for example, add a chaos testing stage or I can change my promotion mechanism from manual to automatic or the other way around, depending on what I want to do. Based.

C

On a.

E

Quality gate evaluation and the pipeline code stays in completely untouched, so this is handled totally independent of the pipeline.

B

So let me let me try to understand that a little bit better. So let's say that we have a pipeline running in tecton. It's emitting some events right, like it's a building pipeline, it's building a set of artifacts, including, let's say a docker container.

D

And.

B

Like that's, my question is more about that. Like should we focus on on building pipelines or on deployment on deployment firefox? Maybe we can just create a deployment pipeline and then just decorate that with captain right like add more stuff, on top of it,.

E

Yeah, so we're not building anything I mean technically, you could do it with a custom sequence, but the idea is usually we get a new artifact event like something new is here to deploy or create them. Okay, then, usually this event moves into captain captain and looks at what we have in the shipyard file, which is the strategy more or less how this specific application or a service should be deployed.

E

It's usually based on a per project basis, and then captain picks and choose what it wants to do with this uh specific application, obviously might be different from an application. After from an application component to a database, to maybe an ios application, whatever it is, and then based on the sequence, what you call a sequence, what is defined, it will then call other services which again can be pipelines.

E

Okay, so we would have to build pipeline, calling captain captain looking up. How should what should I actually do with the specific artifact? Also right now, one example: we we're using here assuming an ecommerce provider.

E

So throughout the year, you kind of encourage people to do fast experimentation, but during black friday and cyber monday you're not encouraging people to do experimentation because you're making 60 of your revenue- that's not the best time of the year to try out uh that new card application that might fail. So you might just by the switch digital rounding captain or you add, an additional security test policy.

E

One of the areas is also especially if you look into the finance and health care environments.

E

They kind of need to ensure that the delivery workflows follow a certain practice and they don't want to have everything encoded and still want to give people freedom, so that sequence would be encoded in captain what needs to happen, and then people still have the freedom to kind of do the individual steps, so we build it, send it to captain captain picks what it wants to do, whether it's test deploy validate the validation of the suv pieces you have in there in captain this again would be calls to different, smaller tactile pipelines.

E

Again, we would just define the order of what we want to do, because for certain stages, we might want to call a performance test for others. We want to call a security test for others, we run the performance test and the chaos test at the same time again, this is all something I don't need to encode in the pipeline. This is dependent on maybe some policy. So that's one way, it's more as a back and forth communication between the two.

B

Perfect yeah.

E

I think micro pipelines, that's what I like to call them like sharing micro services with micro pipelines, they did an orchestra and and more or less captain is x, is acting as the mesh in here that kind of links to things properly together, but also to your earlier point. The pipeline for us is also a service. So when we do the remediation, we can also call the pipeline so assume that we want to scale something up or change an environment configuration based on the problem.

E

We again would emit this event, where we have a specific on this kit again trigger attack on pipeline. If it's, if it's running they're there as well.

B

I think that that makes sense. So basically, at that point, uh when you kind of run and deploy by what's kinda like the event in captain, that needs to be emitted to say something was deployed like new service or something like that.

E

New artifact event right now,.

B

And what what happens when you have like a new locker image? How is that different, or is it the same.

C

It's the same.

B

Okay, perfectly so indeed, yes, so this one is like uh this is okay, new local image, let's say or new film thing whatever, and here like when you deploy things. It's just like you have another artifact, which is probably a new service running.

E

Yeah and then captain would like a which up which environment should I start to deploy the thing to then triggers the proper execution. Usually we always run quality gates, so we validate against slis and slos.

C

Yep.

E

Again, it's something you don't need thinking in including your pipeline. This is like a built-in functionality and if.

C

It works if it.

E

Gets automatically promoted, it will then trigger again, for example, a deployment pipeline, but this time give it a different target because it knows if it works in this environment, we push it to to the next one.

B

uh For for that, like for environments, you have as well like events there like uh what are kind of like the environment. Events like every time that you create an event, do you need to notify captain that they send you a vendor? Like? Can you say a new environment available.

E

uh No, we would we have, we would send it to the pipeline. We would tell you for example, first time I want service a deployed to death, it's it's like part of the metadata of the event and the next event. They would game when to have service, a being deployed to say uh to do pre-production and there's other metadata like we would send you which type of deployment we want to use. Be please deploy it for me, please direct deploy.

B

It for me, okay, so I think that that's super helpful. So in general, what captain is doing is basically parameterizing these micro pipelines and this defining, which ones yeah, okay, great and now. I think that that's why I need jurgen to help me up to set up kind of like something pretty simple around this right. So we can definitely assume that we will have some pipeline some tecton pipelines that are going to build stuff and then I'm going to deploy stuff, and then we build this component.

B

That says, you know every time that we send like a new artifact event from from the cdf events, we are going to emit the one that is from captain and then we are going to see how the pipelines are triggered. If I can set that up, that will be awesome.

C

And then we can in between, uh like you already have which environment should I deploy this, then we could also make use of the captain quality gate. um That can be like the next step. uh In the first step, we maybe kind of like do not do the quality gates, but just have a multi-stage uh environment and then there's a subsequent step. We can add the quality gates, and this should not change anything on the text on site. So that's.

E

That's also the pretty part.

A

Here, yeah.

E

The nice thing about this demo would be you, for example, add a staging captain and you didn't don't touch any pipelines, it's just parameterizing those pipelines differently, and then you see that deployments are running differently. Changes are running differently or you say well nobody's running a security check in here. I want to add a security check in here.

E

It makes sense, and the advantage is you have like, like one of the major arguments when we talk to the bigger companies is that you have a real separation of concerns, because the sre or whoever is responsible for the delivery process. Only austin owns this delivery process. That's a separate file, the devops engineer owns all those micro pipelines, and then wall is like lego.

E

They are comp they're combined at run time, but people are only responsible for what they want to do, and the individual developer just says what what artifact they want to deploy, but don't have ever access to the other components uh in there. That is one of the major drivers that are not all stuffed into one thing together, but yeah.

E

I think the easiest way is to have what I mentioned: the translation service, where we translate from the captain event, which is an sh captain, started event into a start test event with the proper metadata which we didn't send to a tactile pipeline. We just need to ensure that we get the that's. Also when you remember when we had this discussion yeah, we said like we need to start it. We need a finished event, because this then feeds back into the the execution, bits and pieces yep.

B

And I think that that's the main like so there are two main reasons for the poc: one is the cd like the silicon and the other one is to refine the current events that we have, so we actually make them real right with this scan like iteration of creating a plc making sure that the events kind of like fit the purpose and then going back and redefining the events and the properties and making them more concrete, so yeah. So if we can uh work on these, I'm I'm more than happy.

B

I think we kind of have something there that we can actually and we can just get it done and then just present and then see what people think so.

E

Yeah and then again, we can have very nice use cases like, for example, you get the rollback for free, because if captain deploys a new version to an environment sees that it fails, it automatically gets gorillas back to the old version. Without you having to implement the rollback.

B

Perfect.

E

So that yeah and again what we don't do, I think, where tekton really comes in nicely, we don't have like traditional step based workflows. It's really the eventing engine.

C

That you're getting.

E

Here and I think with jurgen, that's also why I brought him here. He has built lots of our integrations and use cases so perfect. He knows captain in in and out and how to get these things done.

B

Great great folks, eric mathias viva, any questions.

A

Yes, so my understanding or one question first, this new artifact event would that be sent from the tectum pipeline or is that something that captain sent somehow it.

C

Should be sent from tacton to captain to trigger captain it's basically the first trigger to capture the captain starts it's it's working.

A

Yeah, so this actually answers my second question, so we will actually have techdon triggering captain through messages and captain triggering tecton through messages or events. In that case,.

B

Something like that, I'm not entirely sure. If tecton has that so basically tecton we can, we can build something. To do that. I mean tecton is not listening for events in that way, yet at least not for our events. So maybe what viva needs to implement at some point, or we need to have that translation mechanism as well to figure out what pipeline to trigger when we get a captain event for triggering pipelines.

D

What we talked about last week about some kind of conversion broker or something yep like adapter, that.

B

Yeah yeah. I think that this is going to be bi-directional for now.

E

So I think this is also very important because we don't want to build the ci system. That's definitely not what what we want to do.

E

We don't want to build anything that already exists, but this event orchestrator does not exist and that's why the focus was there perfect.

D

So uh this might be part of the uh another poc that I made last week, which is uh which isn't completed because the uh cdf events, library is still uh kind of yet to be done. But this might be part of the poc I built last week for like the conversion broker, um we could probably we can discuss more on how that would work.

B

Yep, I think that well, in that sense view I don't want to kind of like add you too much on your plate. I think that if you help and if you work with andrea on the controller on the tecton side I can take, I can take a like the initiative on with jurgen on the captain side for this translation, and then we can see if we can first of all use connect your plc use any other library and making sure that again we don't we don't.

B

We do not reinvent anything that is already there like out there. We can just basically use go and use.

D

No, I was, I was hoping if, uh like the thing that I made before, could uh like help you right now, so I like.

C

You could.

D

Just pick it up, probably like the thing I hate before or like probably work like a head on it maybe are.

B

The other link there yeah, I will link like as an example.

D

Like it's a very crude thing that was made, but it probably could help like the idea that, like it was built around.

B

Perfect. Thank you very much. Fella.

B

Matias, uh you know viva, do you have any other specific question about kind of like this kind of use case and they come like the initial steps.

D

No, but this looks good I haven't uh seen captain as much. I have to still try it out once at least great.

A

So, just to be clear, we will need to to either write some small component or add some support in tecton to actually be possible to trigger through cd events.

A

I think.

C

So yeah.

A

Yes, when will we decide specifically how to do that or more completely how to do that, whether we build something into takedown or whether we write some small of event, listener that just sends the correct start command to take them on. However, that would work just so yeah. It doesn't fall between the chairs so to say.

B

That's a very good question and I think that it's all about this kind of translation component that first is going to basically listen. So basically, this one is the one that's going to pick up cd events and create captain events and as soon as we know how to do that doing it. The other way around shouldn't be that difficult right, like making sure that we go and create, for example, pipelines. It shouldn't be that difficult, because it's just basically using the tecton library to trigger to create new pipelines.

B

New pipeline runs that can be hacked pretty not pretty easily. It might take a week or so, but I think that we can just get it done and then see. Where do we put these components? I know that the component that is building viva it makes a lot of sense to have in tecton itself, but all the other things like the this translation layer.

B

As you know, as jurgen was saying before, let's keep it out of captain first to seek like how it looks like and how it works, and then we will see where I mean where it can live. Maybe in the cdf repository as well or close to the events.

B

um There is not too many other topics from my site to discuss. I wanted to definitely have a quick chat with jurgen to kind of again get get started with the captain side to see if I can install something or if we can define how to build that component and and where uh you know- and I do not have any other topics to discuss. Kinda like in this meeting, because I can definitely see that we are making progress, uh and I wanted to mention eric you.

B

I think that you should kind of like send a pull request with the diagram right like definitely if we start getting diagrams in the repository. That would be a good good thing to see.

A

Yes, and on that topic, I would just like to to make sure I have questions. Sorry answers to my questions. I think I have the most of.

C

This.

A

But I will just go through it as I post it in slack, so we will have both way communication now, as we define it, so some stuff will be will trigger the importance and stuff will trigger techno stuff. So I will make sure to to add that uh I still had. I know transport is the wrong way, but broker like what broker will be used to actually get these events between tekton and captain, or will we just pick something when we're ready for it.

B

um Yep, so my suggestion was to start like with http and probably use the k native broker.

B

If we can use that or we can use kind of like direct communication right like we can just send, for example, from tecton the component that it's generating the cd events can have a sync which is basically going to be an url uh where it's going to be send events p http there will be a component that will pick up those events and then send another cloud event to another http endpoint in captain that can be kind of like possible, as well kind of like no need for broker like message messaging broker there in the middle.

B

uh But we can also add, like uh you, know, okay native broker there, and it should work. It shouldn't be any different.

A

And that then, if we use a direct http endpoint, presumably this is a small detail. Perhaps, but captain will receive that event as just raw json or whatever, and then it will take that raw json and go to our event. Library and say: can you please decode this for me, so it wouldn't use the go sdk. The cloud events go is the case to like set up some sort of event receiver and then receive it through there, it's more, the captain would receive it directly and then decode it using the library.

B

No, that's where the translation like component is right, so tecton is sending to a component that has this library and understands so. Tecton has a component that will create cd events right and that's going to be sent to another component that it's in the middle, transforming from cloud events using the go sdk to captain events. So there is another component that basically has the our library, plus the captain library to be able to the translation, and that component is the one that is going to send to captain already captain messages.

B

So captain on its side doesn't need to do any translation.

A

All right, so so, as far as captain is concerned, there is no difference. It doesn't matter.

B

Exactly so, this diagram, at least in my head, will have two more components: one that is closer here to tecton that it's basically the one that by is creating which will read annotations and send cd events, and then, here closer to captain there was, there will be kind of this translation component that is going it's receiving cd events using the go library, sdk and sending pure captain events to right, gotcha, good, yeah, good, good.

E

Stuff, I call this in in the mid term. We would agree all on the same event, so this component will simply more or less go away.

B

Exactly that's the whole purpose, just to get rid of all these crazy components and just make sure that the the products kind of like interact, natively yeah. I think it just.

E

Shows greatly how things can work together. That's I think, that's really nicely, and then I think that you will get by the way as a side effect. That jurgen can then also show you get you get more or less end-to-end tracing we're not yet using trace context in there for historic reasons, because we started some of this work.

E

So there's still the captain context, which is planned to be replaced by race context good, but you will see the whole sequence tracing also then in in captain, if the proper events are sent, I think it's also a nice side effect.

B

That's pretty good yeah and I think that's if we can include well whatever we can, including like in the in plc, the more to show kind of like visibility, wise and all that stuff that's great.

B

uh But for now let's keep it simple, just interaction and and see if we can make it work like in a single environment,.

E

Yeah yeah, I think once we get it to the point where you say: okay, there's a new image that I've built here. It is um update your charts run it and then you go back to tekton and deploy it and then for the next run we add an additional station. It gets deployed to that stage as well. I think that's something that people will find really nice without having to touch a pipeline at all and just eggs.

B

Exactly.

E

Sounds good.

B

Good stuff, I'm happy, we are making progress, that's really good! uh All right! uh Folks! I think that's pretty much it. uh Let's keep it short. Let's make sure that we don't kind of like do meetings for having meetings only so if you guys want to catch up at some point in slack feel free to ping me or ping each other and let's push this forward.

C

Sounds great uh mauricio, do you want to stay on this call and swing a little bit on me? If I can show you how captain works, or do you want to do this? Another time.

B

No, I think that, let's do it now and people if you want to stay, I mean I'm happy to do it in this call and whoever who wants to stay can stay. Basically, the main idea here is just getting started with captain and trying to discuss this a little bit more in detail. That sounds like a.

D

Good idea, thank you.

B

Let's do it.

B

I will stop sharing, so if you want to share.

C

Yeah sure um so let me just share.

C

Just a second, so I think you already um saw that the tutorials and saw a little bit of captain, but I just make sure to make this full screen here um and what I just want to show. You is like uh how captain um orchestrates all the different tools and how also where uh techno can fit in so this is one of our public demos.

C

I can also share the link in the chat, so you can also uh have a look and what's especially interesting, uh and I think which will help a lot in our pc. Is it's just, for example, the potato head? It's a ctf um demo project. Let's just focus on this um for the.

A

Moment.

C

The the important part I think for our poc and where I want to what I want to highlight, is the sequence screen here. So if you go into the into the potato head project into a sequence screen, we can see a whole delivery sequence that was uh being executed and the what we can also um see here where we can really take a look- and we can use this for um for for debugging or for learning how it's stunning captain is.

C

We can take a look at all the different cloud events that are actually sent from captain and and send two captains and from captain and, of course, it's a little bit different for the city events, but it's also. It might also be a bit similar. So this is the complete payload of a cloud event that is sent to captain, and this is the basically the payload that we have to create this.

A

Is the event.

C

We should create um once tactone does the ci part and, uh let's say, comes up with a new image. So this is a um just the the container image and the version- and this is um sent to captain so captain knows this is the image uh that should be deployed uh which project which service service, which stage um and uh then some other information.

C

That is just a meeting information for for captain- and we do have this here, for example, here in our production stage, so that that it is deployed to production was actually sent from the hardening stage. So we have a multi-stage delivery pipeline. It starts in in our hardening stage. We have a delivery sequence and the sequence consists of a couple of different tasks. So we have a deployment, a test, an evaluation we get the sli, but for our purpose we could, for example, just focus on deployment.

C

We can do a deployment when the deployment is finished. We go back to tecton and do another deployment in in production or we can do whatever we want whenever we want to add some test here. We just added in the shipyard file, but we don't have to touch the pipelines in tactum, for example, and how to how the tests are triggered. Again, it's very similar.

C

It's uh we will have all the data on the artifact itself and the tests are triggered just with a test triggered event. So this is how how the events are working. It's always the task and the trigger and then inside the task we can see who is actually consuming this event. In this case it was a cheat meter service. This, for example, could be attacked on service or um we.

A

Could even.

C

Send this to tecton and detect and start and uh and then finish its its work. So this is how we can see how captain orchestrates uh the whole delivery sequence uh and I think what alois also mentioned earlier was uh we have a complete trace and how one artifact goes through the whole system. So we can see that this version uh was deployed. First in hardening there was this delivery sequence. It was evaluated with 100 of the quality gate. It was then released to the next stage and the next stage was production.

C

We can see the whole for the whole flow again.

C

um So this is the. uh This is one of the the public demos we have for captain, where we can take a look, how the the events look like and uh which events we have to send and which events we can send back from captain um there are more, but this is just for the for a delivery sequence for, like, let's say, maybe a typical delivery sequence with deployment tests evolution. uh We can see all the events here.

B

So you have a single sequence there right so that it's called delivery. uh Yes, okay, can you can you show us kind of like the definition of that sequence,.

C

Sure um I would go back to the project here and here it was the potato project and we can take a look at all the resources that it's managed here by captain. um It's also publicly available. You can also take a look at this and the shipyard defines all the sequences and it's very easy. uh This is the whole definition, so I have in hardening and in production I have the delivery sequence in hardening.

C

Maybe I can open it up again here, just to kind of to see what was really going on um in hardening. We do have um the delivery sequence. I I just have one sequence: there could be more sequences, but I.

A

Just have.

C

The delivery sequence here and it starts with the deployment. The deployment strategy is blue-green on the microservice level, uh then I have some tests. uh I want to execute some performance tests. What's not specified in captain is which tool is responsible for this. uh This goes into another concept, which is the captain uniform the captain's uniform.

C

So if you ever want to move from che leader to locust or artillery or k6 or whatever, we don't have to change our process, it's just the tool. We need to change, so we need to execute the performance test, and then we have the evaluation and the release and all these tasks we can see here.

C

uh They are also shown here. It's the deployment.

A

The test evaluation, if.

C

You look very closely. We can see the get sli, which is right now, a part of the evaluation, because we need to go to some sli provider, fetch the data and then do the evaluation. This is not necessary to um to be defined in the in the shipyard, but everything else.

A

Is.

C

Is a one-to-one weapon? It's basically everything that's happening and for a subsequent stage this one, the delivery sequence, will be triggered when uh hardening uh when a delivery finished event occurs from the hardening stage and if we want to define some filter here, um so this one only is triggered when it's passed but for let's say a rollback sequence, we could just say it should only be triggered when the delivery finished but failed. So the universe was finished.

C

It was not stuck stuck or anything, it was finished, but the result was a failed result of the quality gate. Then we could do something like a rollback or whatever.

C

So this is how we can define and how we can also uh make a whole flow of different uh delivery, sequences or other sequences even going through different stages like production or harm.

B

That makes a lot of sense where, like then, the tasks defined? What is what kind of deployment means? So where do we define what events are going to be sent kind of like in this in this sequence? Basically, so there are a couple of.

C

Tasks that, um let's say our our reserved keywords in captain uh this can be found in the captain documentation. But it's basically deployment test evaluation release.

C

What else remediation.

D

Is one.

C

But if you want to um write your own, let's say um security scan or something like this, you can define it and the capital will create an event which has the type that say hardening dot security scan and whenever we want to start this, this task it will be hardening.securityscan.triggered and whatever tool is responsible for the security scan that says sleek. It will answer with a start event and once the security scan is finished with the finished event, and then capture will move on with uh with the next phase.

C

The important part here is that the other tools they do. They actually do not even know that there was one additional task added. So if you want to do a if you want to do it before test or after test uh for the tools it's completely transparent, they don't need to know and for the deployment um usually in captain, we can do deployments with helm, but we could also replace the whole helm part, for example, with with tecton and also do deployments with with tecton just.

B

Yeah yeah, which tecton can be using helmet at the same time right like so it doesn't really matter what tool you use, but what I'm interested in here is okay by looking at this car, like sequence, how do I so I'm guessing that if these are reserved words, this basically means a set of events that are it's going to be emitted somewhere right and how do we configure that like? Where are those events going to be emitted and how do we integrate their uh yeah? We with our stuff.

C

Yeah, um so all the events uh we do have a specification of all the events.

B

Yeah, I think that we have seen that before yeah.

C

So these are all the events, but I said um like, if you add some.

E

I think if we show the subscriptions, that might be helpful, so a service which, in your case, would be a task or a pipeline actually subscribes to an event with parameters. Yeah, that's what the uniform more or less is.

B

Okay, great great so, basically definitely yeah. We will need to create an uniform.

C

So, for example, we can take a look at a simple service. That is um how the subscription mechanism is working. So we do have templates for this, but um just to see like here we can see for which events I'm sorry, uh which events that this integration is uh is listening for. It's the litmus integration, it's starting um chaos tests and it's it's listening for test.triggered or also test.finish, because in a dot finished event. If it comes from some other test integration, we can also stop the chaos.

C

Let's say, performance tests are finished and we will receive these events uh in the litmus integration. Then we say: okay performance tests are already finished. We can also stop now chaos because it's not needed anymore, um but here we can define it. um Captain sends this to uh we're using here as the um as the messaging system we're using here nets- and you can just subscribe to this and listen for this kind of events.

B

Yep, perfect, okay yeah that makes sense so so yeah in in general. I think that I do understand the idea we will need to build. So basically what we will need to do is we will need to create kind of like a repository with the shipyard definition for the sequence that we want to have like a pretty simple sequence.

B

Then we will need to go into a captain installation and basically import that project. Or do you can like add that project to captain? How do you do that.

C

um You add the project to captain, so it's the uh as I showed you, the potato head, you don't add the source code or whatever to captain, but you add um you add the project and only um meet information, not meet the information, but the help chart the test files remediation instruction. This is what you add to captain. It's, basically, the uh all the information that's needed for deployment, but not the source code itself.

B

Okay, okay, good, so we will so basically we create that we have it. So that's the next question. So should we work with like uh something like this, like a public captain instance for the plc, or should we kind of like try to kind of install a setup where we can have tecton and captain all working kind of locally.

C

uh Yeah, I think, for the poc uh coming up with our only installation where we can really mess around the thing. That makes sense um if we want to like uh exchange some tools or exchange some uh some components.

A

And.

C

Also do some debugging there.

B

Perfect yeah, so definitely so if we can basically then have like a very minimal, because in order to run these projects, these definitions, these connect sequences, we need to have basically what like the control plane for captain right. Yes, we have the control plane. Then we can. Then then we can just basically create these services and subscribe to events. That captain is meeting right.

C

Yes, we would need to install captain, uh create a project and uh create one service uh like uh the project is the uh is.

A

A logical.

C

Representation of one or more micro services that we can add, so there has to be at least one service that we add to a project, and then we can already uh invoke the first uh sequence.

B

Sounds good and you mentioned that you're working on something for me. Eq.

C

That I.

B

Can probably run run on kind as well right.

C

Yeah, I'm working on a quick start guide on the k3d um okay, and uh this makes it really easy to run captain also locally uh kind might also work. It was just not working on my machine because kind needs a little bit more resources than uh k3d, but I can share the same installation instructions and um probably I can even try.

B

Yep, I can give it a try and see if I can get it working and then I can start playing around, then we can start basically defining these kind of the sequences and how to connect things together. I think that that would be great yeah very stuff, and I think that for for the rest of the team, I think at the end like pushing for merging the library.

B

So we can start using the library, even if it's buggy or if it's not complete or if it's not covering all the events, but just having the pull request merge. What we can do is we can definitely make reference to that library from other components, something that by having it in a pull request, it's going to be a little more tricky.

B

So I think that, for the next kind of like event, c events meeting, I think that we should just try to have that discussion and kind of try to reach a decision on when to manage that.

B

Jurgen, thank you very much for the intro. Is there any? Is there something else that you want to share from? From that point,.

E

You can, it might make sense, maybe to show a remediation workflow as well. What it looks like I mean it's not part of that core use case, but just for completeness and also because mauricio asked for it yep, because.

C

It actually works.

E

Exactly the same, it works, delivery and operations works exactly in the same way. I think that's why it's good to see it.

C

So let me just find the loading find the right project for it. So let's think I won in.

D

This workshop.

C

And in cards, so he also he again. We have a multi-stage uh environment here and I think regulation has defined in uh production and the I can open.

C

Let me just sorry: first open the shipyard file and then let's have a look in the master.

C

So it's again defining all the delivery sequences and having a look in production, uh we can even see a rollback sequence.

C

uh So once the if the liberating production is finished but fails, it will be rolled back and what we also have is uh sorry.

E

To stop here, but you can, for example, see that these are individual sequences like every stage has its own sequence, which also allows you to manage them separately before we had everything in one, but here's either actually every stage or more or less every part of that delivery has its own sequence. It could be, for example, also a subset of your tenants that you want to move there. Sorry, you again, I think it's a good idea to share this year.

B

I feel that's good, so that's kind of makes me wonder: what's happened first here like delivery and then roll back only if uh that's triggered by that event right and the same with delivery direct only if the events happen, that sequence is going to be executed.

C

Yes exactly so, this is only triggered on the delivery direct um if a delivery direct is finished from the staging stage here. um This is a way to also uh model it with different delivery strategies. So I have one service that goes with the blue green deployment strategy, and I want service that goes with a direct deployment strategy and to not kind of uh they are not getting interfered.

C

We can just use two different uh sequences. The one is called delivery direct and it only listens for delivery direct finish from the previous stage.

C

So if there might be some services that need a canary rollout and others need to be green and others be direct, you can simply uh build it and you can even have this, for this is the whole file, it's 86 lines of code and it's for a whole project, and you can even manage now two different delivery uh strategies and it's not really a large file. So it's still easy to understand.

C

Yeah, that's really good, and even with.

A

With this.

C

Rollback mechanism here and what we also have is this: uh the remediation file and the remediation file is basically triggered when there is a remediation sequence and um the remediation sequence is triggered when there comes an open problem. So if there is a problem or an alert, let's say prometha sends an alert or there's a problem. Ticket from dynatrace sends to captain. It will start the revelation sequence, and here we have, let's say a response, time degradation and we can map this problem type to a set of actions.

C

So in this case uh there is one action that should toggle a feature: uh we have a human, readable name here. um What should be done? The description and then also all the values that will be um will be sent to the uh action provider. So the action provider will integrate the the set of values.

C

What should be done? We can eat for other remediations just to give an example. This could be, um the action could be scaling and the value could be scale up and then like a percentage or or a value um or even like a restart and then with a value. Maybe a grace period for for termination of these kind of things for all of the feature flags. We have the feature flagging the name and the property, but this is really kind of uh yeah specific to the use case.

C

But again here we do not have uh the tools directly defined, so we could use uh unleash or launch darkly or split eye or different feature frameworks, and they would listen for this for this action and they would execute this action again by this triggered, started and finished um types of cloud events, so that captain again knows what's going on and if the revelation is successful and then once this action is done, captain can start to do another evaluation um with the slo-based quality gates and then can probably move on to another action that can be a sequence of actions or just um close.

C

The uh either close the problem, ticket or um yeah also um escalate. If there, if there was no action that could really solve the issue.

B

Perfect yeah, this makes sense yeah. This is great. This is great and the more that we can show the better right, like the more that we can use the tools, uh the better, so having kind of like an understanding of all these features will help us to. I think that go faster at some point like as after we go to uh the stage of kind of, like initially aiding, initially seizing all the components making sure that they can talk to each other and then building more and more advanced demos, so yeah. This is great.

C

Yeah, I think, once we have the mapping between the cloud events, we can do a lot of things. It's just that this uh part that you mentioned also in the in the diagram. uh That's uh that's the crucial part here, great great.

B

Any other questions what I don't know if this is still around nope, not for me good uh bebop, any questions.

D

uh No not from my side, I think I like to just go ahead and try it out.

B

Yep yeah yeah. We will be doing that and I will definitely be trying to document that for running tecton and uh also um captain in in probably in kind and probably I will take a look at k 3d as well just to see how it works, and I mean, if you guys are using that already. I can maybe take a look and see how different is from kind. It should be pretty similar. That's what I'm expecting so yeah great great, so jurgen.

B

Let's uh uh move it to the to the slack channel at some point and let's try to move these things forward. If you can share that kind of tutorial that you mentioned, I will give it a try.

C

Yeah sure happy thank you.

B

Thank you, everyone else uh from joining and let's, uh let's meet again on the cg events meeting, I think that's happening monday.

B

So let's uh talk there and and try to define the next steps, but in the meantime I will be uh pushing the library and the events and uh post about like the tech concert as well.

D

Yes,.

B

Thank you. Thank you again. Thank you.

D

For joining.
youtube image
From YouTube: CDF - SIG Events Meeting - Vocabulary Workstream - 2021.06.01

Description

For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/