Continuous Delivery Foundation Spinnaker Summit 2022 - Detroit, 4 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Events in Action With Spinnaker, Tekton and Keptn - Jalander Ramagiri & Adam Kenihan

Description

For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/

Events in Action With Spinnaker, Tekton and Keptn - Jalander Ramagiri & Adam Kenihan, Ericsson Software Technology

Many organizations use more than one CI/CD tools as part of their pipelines and most of the times the organizations implement their own glue code in order to make the tools work with each other. Due to these concerns, CDEvents project hosted by the CDF started working on a common specification for Continuous Delivery events based on CloudEvents hosted by the Cloud Native Computing Foundation (CNCF). CDEvents enables interoperability in the complete software production ecosystem, helping CI/CD tools and services to communicate with each other about occurrences in a CI/CD ecosystem. As part of its work, the community created a Go-SDK to introduce native CDEvents support to various cloud native CI/CD technologies such as Keptn and Tekton. A new Java-SDK has also been developed to bring other CI/CD technologies into the picture such as Spinnaker. In this session, we will talk about the interoperability challenges within the CI/CD ecosystem and how CDEvents aims to overcome those. We will also talk about Java SDK and our proposal to contribute CDEvents support to Spinnaker Community which enables Spinnaker to communicate with other CI/CD tools within the ecosystem.

A

Yeah welcome all so today we are going to discuss about events in action with Spinnaker techton and Captain yeah. So we are here. I am jalandar ramagiri software engineer at excellent software, Ericsson software technology and I'm, a member of CD events and we are contributing to Spinnaker as well.

B

A

B

I'm also a software engineer at Erickson software technology.

B

A

Yeah so coming to the agenda that we have today to discuss so we'll talk about little bit like in general. What is continuous integration and deployment? uh Just an introduction and I will talk about what is interoperability and versus integration, so we'll see like how different tools can communicate uh using interoperability or integration, which is best like we can. We can see with some examples and we will see what is CD events and how using CD events.

A

We can communicate different cacd tools within the cscd ecosystem and we will see events in action with cicd tools and, finally, the demo and reference says like we have a demo uh like where we are actually building a simple application uh where I am trying to deploy application like all with different cacd tools. All will talk to each other using CD events, so we'll have a cool live demo with that. uh So I'll ask Adam to take over here and talk about continuous integration.

B

Yep so I'm sure a lot of you already know what CI CD is continuous integration and deployment. um So it's just a process of continuous planning coding, building testing, releasing deploying uh operating monitoring.

B

um But it's it's a message to deliver your apps in a automated way, efficiently and safely, um and there's many different tools involved at each of these stages uh that we'll just we'll go through a few uh example ones now in a minute.

B

So this is kind of what a general workflow would look like. uh First I'll explain the CDF The Continuous delivery, Foundation, it's an open source Community. It hosts open source projects uh mostly related to cicd projects. So a general workflow will look like uh kind of your build. Your deploying staging you'll run some tests, you'll have a quality gate and you can then deploy in production.

B

uh So, for example, you might build in tecton and then deploy and staging also attacked in run some tests on it using Jenkins test Suites. You might have a quality gate, then with Captain and then deploy into production using Spinnaker.

B

So, uh just to explain interoperability versus integration, so integration is more like. So a few weeks ago, I went to Spain and there were a couple times: I needed to use Google Translate to communicate with some people, that's kind of integration, whereas my trip to the US so far, I'm from Ireland has been uh everyone. I've talked to has spoken English to me. So that's more like interoperability. They they speak the same language and they use a common language in order to communicate with each other.

B

So Tech, then, can communicate with captain speaking the same language, whereas integration often uses glue codes kind of like a self-invented systems in order to link these two uh tools together.

B

In order to enable this, we use events, so events enable interoperability between the different systems and tools, and we integrate this using Cloud events, which is a specification for describing event data in a common way, consistently accessibility and portably, and this is hosted by the cloud native Computing Foundation, um but using Cloud events with CI CD tools is a bit dicey. It has different formats and kind of inconsistent semantics, um specifically for CI CD tools, so we use CD events to accomplish this within CI CD tools, so just back to Commander yeah. Thank you.

A

Thanks Adam, like uh yeah, like Adam, said like here, uh we have cd events to enable interoperability between different cacd tools. There you can see right, tecton, Spinnaker, Jenkins Captain, uh but the cloud event has a different uh format to comply with the cacd specific context. So that's why uh we have a CD events. So this is the new project under CD foundation and it is a common specification for continuous delivery events, and this is based out of cloud event again.

A

So the event format looks something like this, so this is the one it can be. ah We can say this is like a cloud a CD event, and this has a specifications that is related to continuous integration and development. We can see that uh the black portion of the text is something uh so we can say that is the cacd specific context fit in uh Cloud event.

A

So this is the uh we can call as like CD event right, so we have uh in 3D events like we have a special interest group formed, so that is mainly focused on the uh creating a common vocabulary. So the common vocabulary is divided into these four specifications, so we can say that core source code, Version Control, continuous integration and deployment, so each specification is having like different event types.

A

The example is shown above is like type of one of the types of artifact, so that comes under continuous integration specification so, and we have many ah types that CD event can produce like pipeline run, started task, run, started task and finished. If you check the cover part so like we have different levels like wherein, we have cicd specifications that we can describe as City events so yeah and we have like some SDK supported as well.

A

Go and Java SDK and using this sdks different tools or systems can integrate with CD events so that they can produce and consume CD events using these sdks and the community is towards building more sdks and we have like Python and other sdks is coming on the way. Yeah foreign.

A

The CD event is so CD event can express something, a record or a daughter record that is specific to cacd context, so we can say something like a CD. Even can produce. Artifact is packaged right, so our pipeline runs started, like I explained in previous slide, so we have different types of CDA events, so that is related to pipeline test, or we can say something like service service deployed or artifact is published. So these are like different types of events that it can produce like.

A

These are the some of the types so that many comes from the uh the specification different, even types we can Define and these using these events like different platforms, cicd platforms are tools, can integrate using these events, and these are some companies engaged with events that are currently working with and so yeah.

A

This is how the CD events looks and it can produce the events the way uh the specification is described and using this like, we can enable interoperability using CD events uh between different cicd tools and, like Adam mentioned like now, systems can speak the same language, so that's the CD event and using that CD event, tools can communicate with each other.

A

All right. Let's we don't have any glow code here like Jenkins, is talking to tecton and tecton is talking to Spinnaker. So we will not deal with any integration part here, but it's all interoperability using CD events yeah. So the main part comes in the presentation, events in action with cacd tools. So we have a POC developed uh to uh see like how these tools will integrate with each other. So we have tecton Captain, Spinnaker and K native, even K native ending.

A

So we have like Tech 10 uh to build some Cloud native solution for building cicd pipelines and ah so in the POC. We are using Tech 10 to build an artifact right, so it will build an artifact and notify some other tool to proceed from there and we have a captain here. So this is like a life cycle management and it orchestrates different cacd tools, and so we can. We can have some quality checks here before deploying some applications into Productions and we can have some sequence to run in within captain and we have Spinnaker.

A

So that is known. uh So here we are like multi-cloud continuous delivery platform, so we can run the pipelines uh so in the POC. We are actually using this to deploy the application that is built by tecton, so Spinnaker will deploy that in production environment and we have K native Eventing. So this is the one used as a core part. This is used to enable event driven applications, and this works as a publisher subscriber model we can subscribe.

A

So this will basically have a trigger mentions here, so the components can subscribe to different different triggers and so that they can get the events whenever it happens in the K native Eventing. So we can see more about like what K native Eventing and these tools communicate with each other. So this is how it is. The the functional diagram looks the middle. You see like its K native Eventing, so it has a cloud even broker in the middle yeah.

A

So it has many triggers configured right now, so these triggers are works like a subscriptions to other cacd tools within this uh like POC, that we are going to show in the Remo. And so let's say here we have a tecton, so tecton is subscribed to that one of those triggers. uh So whenever this type of trigger this is the CD event of type trigger. So this type of trigger happens. So the rectang gets a notification and tecton runs, some jobs are doing some stuff and it can produce the event like once.

A

The job is finished are saying so that can be received by Cloud, even broker again and Cloud. Even broker uh is the one like holds all the triggers and whoever subscribed to it, and they will get notified back to the other cicd tool that we have.

A

So if you can see here, we have Captain here and so Captain also registered uh or subscribed to that type of trigger and Captain get notified whenever that type of trigger arises in the cloud events broker and so Captain also capable of producing a CD event right so- and here we have the final Spinnaker, so Spinnaker has a component called seed event controller, so that is the responsible for producing and consuming CD events, so we have a a trigger, so that is that will be subscribed by Spinnaker CD wind controller.

A

So whenever that type of trigger arrives into the cloud events broker Spinnaker get notification and runs the pipeline undo any job that in inside the Spinnaker that we wanted to and It produced the CD event back to Cloud even broker, so they can like. uh So this goes on. uh It's like a different cacd tools can produce and consume uh this CD event, and they can do the things without like with the interoperability manner, so yeah.

A

So here down, we have the cloud even player, so this is just to visualize all the events that is going in and out throw a native Eventing. So this is what we can show in the demo. Even so we can show in the cloud even player and Tech 10 captains vinegar how they communicate with each other. So we have a live demo hope that works yeah.

A

So before going to that, so we will just explain how the flow it is so that it is easy to understand. So we have three components here, like CAC tools, I can say uh tecton, captain and Spinnaker, so these three tools will communicate with each other. Now we are building an application. It's like a simple application that we are going into a stage where we do some quality check using captain and finally, Spinnaker deploys into production. So these are three steps that we can see in the demo uh like see.

A

Let's say we are using here tekton to build an artifact, so user triggers a build in tecton and once the build is successful, the Titan produces an artifact packaged CD event and that is subscribed by Captain. So you can see that captain gets notification and once the captain gets notification, so uh so Captain runs some manual sequence to approve the artifact and it produces an artifact published CD event and that is subscribed by Spinnaker here and once that even gets in the Spinnaker runs pipeline a deploy pipeline.

A

So once the deploy pipeline is green right success, so it will send a service deployed event and that event is again subscribed by Captain to marks the sequence as successful. So we can see this uh in demo.

A

Yeah I will share my screen.

A

A

A

Yeah I can run them, so this is the pipeline that we are going to trigger. um So this is the build artifact pipeline that runs in tecton yeah, so speak underscore.

A

Yeah, so uh we are just building an application by uh from this repo- and this is the branch I just wanted to put here- spin demo yeah, it's it's built just it's built a Docker image and it's pushing into this kind registry I think once the build is started, we can see the running logs here and how that is pulling the image pulling the source code from the repository that I mentioned here and yeah.

A

So here you can see the running logs of building an artifact, so we can also see in the dashboard of protectant, so probably that we have it here.

A

So this is the tecton dashboard where we can see the pipeline running instead, yeah the status of the pipeline is running right now, so we can also see like what is uh the running log says: it's actually pulling the source code from the repository and building a sample image, it's just application that it builds and pushed to the local kind registry.

A

So we can see like.

A

Yeah similar way, we have here the captain dashboard, so this has like one manual sequence that is added to approve the artifact once it gets and we have a CD sorry, the Spinnaker dashboard as well- and we can see the the deploy configuration has done here so in the Spinnaker. So we have ah so to take the latest image whatever is built by tecton and deploy as a Spinnaker service, so that we can see all the flow once the tecton build is finished.

A

So so we can see the tecton build uh status, yeah, it's it's green! Now it's success and the console also we can see the tecton build and it builds this image and pushed to local kind registry. We can match with the SHA ID of this image, to make sure everything is properly deployed right. So.

A

So yeah, so here we get ah the cloud even player. That I said like it has a visualization of all the events that is going in and out uh so mostly yeah, so we have artifact uh packaged, so the pipeline runs uh started and finished is from the protectant pectin pipelines. uh Once the tecton pipeline is finished or we could see, uh the artifact packaged event is sent, so this event is actually subscribed by Captain. So we should see. Captain should get a sequence now so notification.

A

So here you go so the captain gets a notification saying for an approval. So this is the shy image. Id, so I want to verify and approve for the deploy in production. Basically, so we can assume. So this is the image it's 2913 ending with. Is this the one that I want to upgrade? Approve it's two nine one? Three so I can go ahead and approve it's just a simple manual check, but nothing to do with more checks.

A

But what I can do like I get the notification once the artifact package is done, so Captain is registered, I subscribed for that event, artifact package, so I get this notification and once this is approved, the deployment jobs uh runs in. So we can see in this Spinnaker pipeline no jobs running right now, so this is the one we can approve.

A

So once this is approved, we could see the pipeline in Spinnaker running and we could also see the cloud even player what events are going in, so we could see the uh yes.

A

A

Yeah we can see the CD run pipeline started, so that is coming from a Spinnaker. The pipeline of Spinnaker is started is what it is saying. So you could see that pipeline is Success, so we could see an event soon saying that pipeline run is finished and and the service deployed event will be sent by Spinnaker.

A

And we can see also in Captain, so the delivery is started and the deployment is actually in progress.

A

So this is handled by tecton and Spinnaker together and we could see uh this is what is deployed actually so I name yeah. So we could see the delivery is marked as success by Captain so because Spinnaker again notified back to uh Captain saying that I am done with my deploy, so you can mark this deploy status as successful. So here it says the captain says deployment is successful, so we can verify. Even is that the right one Spinnaker is deployed yeah, so.

A

So it's actually deploying in a kubernetes spot. um The Spinnaker deployment is installed, I mean in 92 seconds back, so we can see that is up and running into second back. So when we can see, uh is this the image that is deployed just be built by tekton.

A

A

Having difficult managing with one hand, sir.

A

Yeah, so you can see this, this image is the one it's pulled from the kind registry. It's ending with four two nine one. Three is the one built by tecton and deployed by Spinnaker and in the middle we have orchestrator. That's the captain right. So we can even launch this application. Very simple.

A

I just need to do port forward here, because.

A

Sorry, bye with me yeah, so I can just access this application here that it's running yeah. So this is the image we just built in it's it's deployed here.

A

And so we could also see the events also here in the cloud events player, so that we could see the CD service deployed. Event is the one that is produced by Spinnaker.

A

So this is the even Json format that it looked like so yeah and we have the application deployed in running that's about the demo and we have some links here that talks about like what CD even signed the community links and interoperability and events different CSD tools. So we can I think we have the CD events page. So there we have like our slack and GitHub for contributions. So we are in CD even slack and Spinnaker slack as well, and if any questions we can reach out there and any questions here.

A

um Yes, we have only one mic.

A

Yeah now in this POC, like the part has taken tecton only is the source of the image starting. So maybe we can integrate like more uh to upfront uh for the tecton right, so it may be the jit Source when any check-ins happens. So that's what we wanted to listen for, the tecton will listen for so if any source code change happened or any change updated in the project, so that's that's. The source of it comes and tecton can trigger the build on that specific event.

A

Yeah for this privacy purpose like we are starting with the protectant and user runs the build manually.

A

Yep I hope that answers the question yep. Yes,.

A

uh Which exactly I didn't get the question? Sorry.

A

But before one is um yeah, okay, we have too many animations.

A

uh You your question is here in this slide: is it.

A

Yeah, maybe in this slide, uh so we see like the CD events. Controller is the one. So we have like end points created uh here, so that is the uh so I will show you in one thing, so that will clarify the uh your answer. I guess like see so so we have triggers.

A

So if you see these are the type of events that we have uh like events broker is the one that we talk about K native Eventing, and these are the API endpoints that will be mapped to I mean like whenever artifact is published to Spinnaker comes in let's say, example, so we have this API endpoint created ins inside Spinnaker, Echo component. So if.

A

A

Exactly yeah yeah, so even we can see this like how the types are mentioned here. So this is like simple, uh the filter mechanism that we have to listen to specific type of events.

A

So if you see here uh this, this is the broker that we are using, and this is the filter type. So whenever this is happening so and Spinnaker gets a notification on this URL.

A

Any other questions.

A

So I think yeah, so the previously starting we are uh having like one by one components integrated with this. uh So right now, I think initially the you can say the first release of POC is developed by community, so that is having a tecton and Captain. It's like only that tools are communicating with each other. So now this POC is like enhanced to add Spinnaker, and so we are like in plan planning to have like more tools added to this POC, so that we can prove many tools can work together in the entire cacd flow.

A

So but this is just starting like we are trying to integrate more and more tools.

A

I can't hear probably.

B

So you're asking how we're going to integrate or make GitHub interoperable with this system, kinda.

A

Yeah yeah, yes,.

A

A

B

It's a complication of having multiple different dashboards.

A

B

Whether it's practical facilitation of programming to the effects.

B

A

B

Again, thank you so much for listening.

A