Continuous Delivery Foundation Spinnaker Summit 2022 - Detroit, 4 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Let's Work Together! : CI/CD Interoperability using CDEvents - Vibhav Bobade, Uffizzi

Description

For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/

Let's Work Together! : CI/CD Interoperability using CDEvents - Vibhav Bobade, Uffizzi

The CloudEvents format was introduced to standardise event formats in the cloud ecosystem. This has allowed different event brokers to consider a single format for communication. The need for better interoperability between CI/CD systems is gaining importance, as companies continue to have multiple CI/CD systems in their environment. To facilitate interoperability between them in an event-driven fashion, the CDEvents community was born which would use CloudEvents as the base format for the CDEvents format as a standard for communication between different CI/CD systems. In this talk, the speaker will go over the CDEvents support for Spinnaker, and how to interoperate with other CI/CD systems using CDEvents.

A

Hello: everyone, my name, is vibhav I work as an open source, Integrations engineer at Fiji today, I'm going to talk about cicd interoperability using cdos.

A

So what do we mean by interoperability even interoperabilities, where multiple systems can work together to get the desired result? There are two types of interoperability: the first one is director structural. The second is indirect or semantic in the direct or structurally droplet, as the name suggests. That is the way they are working together.

A

The multiple systems are working together is the structure of each system is known by the other system, so a good example would be document close to kubernetes deployment, where the structural interoperability will be implemented by a tool like officei, where the conversion to or the deployment from dog compose or to like the kubernetes service will be done by office itself.

A

This is direct translation from one format to the other semantic interoperability, as the name suggests, is there is semantics between the systems are known by each other, and each system leverages those semantics to broadcast what they are doing in a Eventing survey, which is then understood by their resistance, and there is no need for each of the systems to know the format of the uh the working format of any of the other systems involved.

A

A good example of indirect or semantic interoperability is CDM and spec, which we'll talk about more today, multiple cicd systems, no or when they consume CD event specs or they translate their events to CD of aspect. These events would be known by other cic systems, so CI CD systems wouldn't have to really know what they need to do if they need to trigger like a pipeline on another one. All they need to know is how do I broadcast what I have done as a CD event, spec and then that's the end of it.

A

So once the ICD system just broadcasts the CD event and that CD event would be known by one of the cicd system, plus delve a little deeper into how we can make CI CD systems more interoperable.

A

One way to do it, which we're going to talk about more in depth today is Eventing. Eventing allows us to implement indirect or semantic interoperability without um uh in a really good way, without knowing the structure of each of the structural formats of each of the CI CD system and implementations. Hence we can be D, copper. If one CI CD system goes down, there will be others, and then um we won't have to rely on just one cicd system and considering it's decoupled.

A

We can still continue with our work that way to be for tolerant and once if the cicd systems themselves scale through to the architecture, we can still be sure that all of the Eventing will work through, because Eventing itself is a very scalable solution.

A

Now the problems are to come along with this is there are there will be like too many formats? Each system will give out different kinds of events which might not be known by the other system. There would not be a proper language that can be uh uh known by each of the systems to communicate between each other. So there will be a lot of inconsistencies now to go around this. We it we can use some a specification that is known by all of the cicd systems to talk between each other.

A

This is where CD events comes in City events gives descriptive events or prescriptive apis, and there are shared semantics which it provides, which can be used to talk between different CI series systems. There are different semantic buckets that we can think about that can be used to define different events in the life cycle of cic, for example, the very core of cicd is running pipelines and tasks, and there are events related to that there are.

A

There are events related to a source code, Version Control events which are specific to any repositories, branches or changes, for example, for PR's created on a repository or a repository state or a branch is created or deleted. All life cycles related to social Version Control tools. Then there is continuous and integration events such as builds tests, art facts, for example, the bill started uh where it got deleted uh or stopped tests started, completed, failed artifacts such as images being created or not, and then based off that there are continuous deployment.

A

Events as well, where environment is created for a specific service to run in or a service has been created in a specific environment.

A

Now, let's use what we've learned about City events and see how we can set up CD events for Spinnaker and some other cicd tools.

A

Some things to remember about while implementing this architecture is these are the things that we need to be aware of and see that we have before we go ahead with bringing the entire architecture together for all the CD events that we have. We need a way to propagate these events throughout the system.

A

The first thing we need is a broker, a broker which will take all the events and let everyone else know all the triggers in that system know that this event has occurred. For example, if a PR is created, the broker should know about this, and then let all of the triggers in the system know about this triggers which understand that they need to do something on this particular event will run the others they will not follow.

A

Other other things that we need to know is not all CI CD systems are supporting CD events, so they need to have providers, and it is better to create providers for a CI CD system than to completely onboard a new cicd system uh and migrate to a new one. You can just create a provider, and then all you need to do is carry out your new workloads workloads over there and with the old ones. You can still work on the old CI CD systems without breaking anything.

A

Now. uh What are these CD event?

A

Providers CD event providers would be Pro providers which convert normal events to CD events uh which see to it that uh events in those CI CD systems are aren't silently completed or failed, so so that on each of these CD events should be put out and the system knows which action to create, with which event what I mean by this is uh with CD event is created, on which uh action uh action taken by the cicd system now to understand better how CD events can be propagated through a system.

A

uh We are going to look at this example: architecture which uses K native for the broker and triggers uh then tacked on as one CI CD system Spinnaker as the other cicd system of easy as a previous environment, and we have cd events. Providers for uffizi and Spinnaker and gray over here mentions the CD events which are created, and then we also have a cloud events player which is being used to see all the events that are coming in and out of the system.

A

As you can see, the first thing that happens here is a change created event is create created now. What is this event? This event, basically, is uh when a pull request is created to GitHub or a mouse request created to gitlab.

A

This event should be created, and this has an ID, a source which is the link to the uh link to the repo itself or to the VR, and the repository uh where this is happening for, as you can see over here, the ID of the repository would be your odd name, slash the repo name, once the pull request is created and the change created event is sent out the keynote of trigger triggers or CD event uh via the official CD class provider, the CD lens provider or less office.

A

You know that it needs to create a preview environment for the pr where the pr can be previewed and a full stack environment can be tested which will be similar as the um similar as.

A

A

Created event is uh broadcasted by the broker. The first trigger knows that it needs to create a uh the first trigger knows that it needs to let the ufizi CD events provider know that this trigger has been created, and then the provider will let fizzy know that needs to create a preview environment for the pr itself, and this preview environment will send a environment created event once the environment is created, and it will let the broker know about that.

A

In the environment created a event, we will have the source of the context of the environment. We'll have the name in this case, so it will be a preview environment, so the name will be preview and the URL to the environment itself, foreign.

A

Now, when the user is done, checking the environment and seeing the full start preview of their application, they can now uh go ahead and you know review the change and then merge the change once a change, most uh change merged um event has been created. A CD event has been created.

A

This uh is sent to the broker and the second trigger, which is waiting on this event, uh will know about this, and the change event basically says that uh this is the source of the repo which has been merged, and this this is basically the event uh in other metadata. uh If this is a release event, what will happen is uh in this case. What will happen is a build will be created through tecton, so attacked on event.

A

Listener will pick up on this event and trigger a tecton pipeline, build and push a pipeline run, and this will, after the run, send a bill finished event uh which will uh set out the last trigger that we have over here. The last trigger will be C triggers Pinnacle uh to create a final deployment of the application.

A

uh It will take the if you can see here it will take the artifact ID from the build finished event and then send it over to Spinnaker and Spinnaker will take this artifact ID and then based on this artifact, it will create a new deployment.

A

Once a new deployment is created, the CD events provider for Spinnaker will create a service created event, and let the broker know about this and the service created event will basically have the end point of the service itself.

A

Over here you can see that the cloud events player gets all the events from the broker, so you can see that the cloud events player basically collects all these events, and you can see that it also acts like a source. So if you want to send a vessel broker, you can do it through the tolerance player.

A

Now, to give an example of how sounding events are triggered, we can mock one of these events into the broker here, I'm running a uh in the cloud device player in the tecton pipelines, um namespace I'm, going to run a event which says that my PR has been most and I'm going to send it to the broker which will trigger protect on Pipeline.

A

You can see that this is the event that I sent over here and detect on. Pipeline has been record, which will create.

A

A build and push it to the registry and then send a build finished event.

A

If I come over here, you can see that this is what these triggers look like.

A

This is a trigger that we've just triggered the trigger for Spinnaker will look something like this, where it will listen on under CD events build finish trigger and then we will trigger the Spinnaker Service. uh In this case it will be the Spinnaker uh Cloud events provider service, and this is what it looks like to trigger uh officeian moment, so it will be listening for like PR changes and then trigger the uffizi CD events provider.

A

You can find all of this in my Apple.

A

On wavy waves Pinnacle series, where you can see the ongoing work of creating a POC between all these systems together.

A

Now that we know how CD events can be used to interoperate between different cicd systems, we now know that we can use it to just interoperate and without having to move to the other CI CD systems. We can interoperate without with least amount of overhead.

A

If you are using a Spinnaker, you can continue using Spinnaker and still use other cicd systems along with Spinnaker and be able to interoper between these two, it's better to create a CD events provider for a CI CD system than completely onboarding a new one and moving to it.

A

So thank you for attending my talk. It was great to have you here and do go to my GitHub repo and go to the example. Spinnaker Series, where you will be able to see a proof concept of how to implement Spinnaker with a CD events of easy and technon see you soon.