Cloud Native Computing Foundation Kubernetes Community Days Bangalore 2022, 11 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Unleash CI CD with Tekton and ArgoCD

Description

Kubernetes is the preferred container orchestration engine to manage microservices to build & deploy applications. However, to effectively manage the application deployment across multiple environments is still a challenge. How do we handle such situations of deploying applications across multiple environments in a safer & transparent way? Is there a way to do it?

Yes we have ArgoCD which continuously watches different cluster objects & Git repositories to make sure resources are in sync & Tekton which is Kubernetes-native & provides a lot of ready to use resources which will be helpful for users to write Pipeline. Tekton & ArgoCD together is like a match made in Kubernetes Heaven. Let's team up Argo with Tekton. In this talk, you will hear how easy it is to leverage these two cloud native technologies together & build CI/CD solutions for your project. As part of this talk, you will hear - Intro to Tekton & ArgoCD - How we can build CI/CD using Tekton & ArgoCD - Live demo

A

Hello, everyone welcome to kcd bangalore 2022. Today we are going to talk about unleashed ci cd with tecton and argo cd little bit about ourselves, myself savita. I am working at red hat and with my colleague shivam, he is also working at red hat and we both are contributing to tecton project and you can find us on these below details moving on.

A

So this is the pretty much agenda we have it for today we are planning to cover tecton overview, its core concepts, then argo city overview and finally, a live demo on building and deploying news application in a different stage, cluster using tecton and argosy.

A

So, let's talk and understand about the overview about tecton tecton is a kubernetes based ci cd framework, which is open source project. uh It was initially started by google, red hat pivotal and many many other companies, and now it is part of pd foundation.

A

Petrol is basically used by developers who who who try to do eipd in their application development day-to-day and also used by platform engineers to develop ci cd system for their developers in their organization.

A

Tecton is a very, very big project. It has multiple sub projects which does a different operations based on the different different use cases, and today we are going to focus one of the two major sub projects of tecton are pipelines and triggers, let's understand about the type 1 pipelines.

A

So this is one of the sub project of tecton cd. It has its own core resources, something which is similar to like how we have in kubernetes like deployment or services.

A

The resources in tecton pipelines are step, task, pipeline task, run, pipeline, run and reset. Let's understand these resources working functionality and how they interact with each other.

A

So I have a scenario where I'm planning to clone the code, build the image, push it to the registry and deploy to the cluster. So this is just an end to end ci cd from source code to the deployment.

A

So to do this operation, I can use jenkins, but I want to do it using the tecton pipelines concepts. So, let's take a one of the very basic object of pepton pipeline step. A step is a custom resource which is very similar to container and which does some operation based on the provided information.

A

So, as I said like, I will be cloning building image pushing and deploying. So these are the multiple stages of operations I have. So I am writing a step to clone the code. Writing a step to build the image, push the image, get the digest and finally deploy it on the cluster.

A

Now these steps are at are like an individual container, so here there is only one that is okay, but when it have multiple such steps, then how do we handle the communication between those so to to to provide such functionality?

A

Tech and pipeline have another custom resource called task. The task is a custom resource which contains multiple steps and also try to execute those steps in a sequential order, and the output of one task can be given as input to the other task.

A

The input output, handling of thing is done in python, using a custom resource called pipeline resources and results.

A

So this task is basically a static template which will, when we create the task custom resource on kubernetes cluster. It will just apply, or we can say, create it. It doesn't execute any of these steps and if you want to execute those tasks and the steps inside those tasks, we have another custom resource called tasker.

A

So basically task run is a runtime entity which instantiate this task.

A

Now I am doing this source code manager, so so sport thing like cloning and deploying- let's suppose I have done for group, but similarly I want to do for other project of java, the project of python. So for all those things I have to execute multiple times of this task multiple times of this faster. It is kind of a tedious jobs, because I'm just repeating everything. So, instead of that, let's create a one and one big template which can combine all these tasks together and with single uh runtime unit.

A

It should execute everything so to do that we have a another custom. Resource called pipeline pipeline is a static template which contains multiple tasks and also pipeline controls which task to be executed then, and to run this pipeline template, we have a runtime entity which we call as a pipeline run. This is a another custom resource provided by tecton pythons.

A

So till now I have written it step yaml I have written a task camel. I have written a pipeline ammo and to run this pipeline. I I even I have uh wrote a pipeline run, yaml or I could have done to cli, but everything I have done manually so writing pipeline is something anyone can do, but executing that python should be something automated.

A

We cannot manually handle running of the pipeline every time whenever it is required so to to achieve that thing to make sure that, as we know that cs it is, some csd is used for automation. So if, if we do stuff manually that doesn't give any value for the crcd concept, so that's the reason tecton cd come up.

A

Having you have a another project called tecton trigger, so tecton trigger is basically a sub project which may its motive is to execute the tecton pipeline resources dynamically based on some event, even this tecton trigger also have some of the resources.

A

Its own resources like event, listener triggered interceptor, trigger template, binding and cluster trigger binding. Let's understand each of these resources uh diagrammatically.

A

So, let's suppose I have every pipeline set up. Everything is ready. Now I want to execute my pipeline based on some event. So, let's suppose I want to take event from the github right. So, let's suppose I I have written, I have a go application where I have configured the book so whenever I send a push request, pull request any of the request with the as I have configured the book so that event comes to this event.

A

Listener object, so it means in my cluster I have created event listed object which watching the event coming from the github repo. So that's the reason whenever a event comes to the event lister card it it do some operation. So basically, this event is not quad is the http I or it could be an https server which continuously matches the incoming event coming from the github or it could be gitlab with bucket any of the sources so once event occurred.

A

It passes to the trigger the trigger is a custom resource which basically thinks what to do with the event and then may it make use of the cluster interceptor, which actually does the payload validation, secret validation and some other processing part. So once all those things are success, it pass to the trigger binding, which is another custom resource way. It will do some modification to the input data or you can say payload once that is done. It will pass that information to trigger template. So the trigger template is a custom resource.

A

It is just like a blueprint which actually contains the pipeline run or task run and at the end it will just go and create those tecton resources. So this is how tekton trigger work and maybe like at the end, in a live demo.

A

If time permits, we will show how this trigger template looks like and how we will encapsulate the pipeline run or task run inside the trigger template. So this is about the tecton triggers now moving on to another, very big project of argo cd. So, for the time constant, I am not going in depth about the argo cd. Instead, I'm just giving high level overview of ergo so argos.

A

It is a declarative and continuous delivery tool for kubernetes and also it can be run as a standalone or it can be combined with the cicd workflow and the source of troops, for the argo is a gift repository and basically, argo is used as an automation tool for the application management application life cycle similar to the tecton. Argo also have have number of concepts, number of modules and resources, but today I'm just taking one resource that we are using in our demo, which is application.

A

Application is a custom resource which actually contains the information of the kubernetes resource. So as as shown in the uh diagram, you can see that kind application is a custom resource which contains the information of source. So, basically, the source contains the path. What will what is this part is nothing, but it's a directory inside this repo github.com tech on cdr go cd inside this repo. There will be a directory for create stage and inside this directory.

A

Kubernetes resource exists, so the I mean to say kubernetes resources deployment table service among ingress. If, if there are any so this will be deployed in the argo cd cluster.

A

Now I I will, I will be talking about another concept, which is simple, think is kind of a process which try to make sure that the application on the cluster is in sync, with the resources or resource object which are exist in the git repository. So this is the functionality of the thing.

A

uh That's all from the argo cd which we are planning to cover as part of this talk now I'll quickly uh start for for the demo. So in the demo part, what we are going to see is how I'll just explain the high level overview of the demo, which we are planning to do so we are planning to build and deploy news application.

A

So for that we have it. Two repositories, one is the application repository and one is the config. So this application repository contains the business project and config contains the yamas.

A

I I really have a two clusters development uh stage. Cluster dev cluster uh has installation of tecton and in on stage cluster we have installed us.

A

So whenever users and user merge pull request on this application repository as we have configured a webhook, so whenever a full request, merge pull request occur, it sends an event to take on triggers and based on the event it tries to execute the pipeline. This pipeline contain multiple tasks.

A

Those tasks are cloning, the code building the image deploying it to the dev cluster. Once the deployment is success, then it will send a full request to this config repository with the changed image information.

A

So before I we go and merge the full request from the config repository. Let's try to make sure that the deployed application is working perfectly on the cluster.

A

Now, let's, if it is working success, let's go to config repository, merge the full request. So once we merge the pull request, we see that the application which was deployed on the stage cluster, have different image compared to the the image which present in this config repository now here a sync concept arrives and it tries to make sure that the images present in the config repository and on the stage cluster are same, and then it will take that latest image and try to deploy automatically.

A

So this is how argo works with the help of sync and application pr. So till now we see the high level overview of publication, which we are going to demo now. I would like shivam to take hando to take out from here and start showing the live use cases of these uh over to you shivam. Thank you.

B

Yeah, thank you savita. uh I will be taking over the demo part uh swipes with the discus architecture of the demo. So we have two clusters. One is the dave another staging. uh We have techno installed on the dave cluster and organity on the seating part, and we have two code repository. One is for the application code. There is all the logic agreements for the application and another is for configuration, so uh we can also maintain just one repo, but for the demo we have maintained two separate one for code and one for configuration.

B

So so we will be doing some code changes then looking like how the pipeline is started, how it will deploy to update the dev instance and then creating a pull request to configure config repo and then how rpcd updates the staging instance. So for the due to time limitation. So we have installed all the uh require tecton and urban cd on the clusters and also it deployed application.

B

So in this repo you will find all the documentation uh like for the installation for the demo, and you can also perform this using a single resistor uh and, if just like, in a different name. These are two different instances, but uh we are targeting a scenario where we have two different clusters: one is the div and another is physics okay. So, let's start uh before we do some code changes.

B

I just want to show you a demo, so this is a simple newsdev application where you can search for news and it's just simple okay, so we have one dev instance and we have another staging system which is on a different cluster. Okay. So now we can just go to code. So this is like uh the news demo code repo and we are just going to do one change. uh So we are going to update this title here uh with a news demo to news demo by accountancy.

B

So I'm just going to do some replace here and I'm just going to commit that change and say, update a title and once I commit it, I'm going to push it.

B

Okay, so once this is changed, so we have two clusters right: one is uh we can have staging here and there we are. So I'm just going to see whether the pipeline is started or not.

A

B

If I do give yeah, we can see that pipeline is started, so this is the cli view. If we we also have a test on dashboard uh which is provided by tecton city, uh and we here we can see all the tecton resources, so we we have one pipeline installed and for the event which just we triggered that the push event a new pipeline run has started so here we can see the pipeline run. It is like we have a different task right, so we have cloning.

B

We are building the image and pushing to our image registry. Then we are updating our dev instance with the new image with the new changes and then we are creating a pull request, so we will see in a bit so till this happen. Let's see uh the different components which we perform for this demo.

B

So if we go here so we did uh some code changes where we pushed a new commit right. So a new event has occurred, so there is an event listener on the cluster. uh That is the text on event listener. So if I go to the dev and if I do cube, ctl get event listener, so there is an event listener which is configured which is listening for the event. So I have created a english for this and using that ingress I have configured a web book.

B

So if we go in our code, repo and in the settings in the web book, we can see this. This is the english ingress for that event listener. So, whenever an event occurs, uh github will send the event on that event listener. So the event listener will uh process the event and, as sabita explained, our different concept of triggers like trigger temperature and figure binding, which we have here which are already applied on the cluster but trigger binding, is the thing.

B

But we are fetching some of the data from the payload, so we want commit id uh the url of the repo from which the event is coming and uh the push at like when the time is pushed, and we are using that in our trigger template trigger template is nothing but uh where we define our pipeline run. So the push event occurs. The event goes to event list now. The event listener will use this template to create our pipeline.

B

So here we have defined our pipeline run and given some configurations to which the pipeline will be started, and if we go back to our tecton dashboard, we can see the pipeline is completed. So in the last task we can see a pull request is created right, so I will just open this. We will just go over here in a minute, uh so we have different tasks we have used. So most of these tasks are reused like we have used it from the tecton hub.

B

So you will find here in different tasks which you can reuse uh based on your requirement of your project. There there are. There is one task which we have wrote uh specifically for our use case, uh which you will find this in this code, this repo and now that uh we have created a pull request. So this is like a change in image for our staging instance, but before we want the uh before, we want to update the image for the saving instance.

B

uh Let's just go to our dev and uh like test our changes, whether it is successful enough, so now we can see the title is changed right, and this is our dev instance. So, and if we go to our senior instance, uh there is no change right. We haven't applied any changes there, uh so we are okay. Once we are okay with the changes, uh we will just go to the our full request and we will merge it.

B

So what this will do? This will update the image in our uh repo right. So these are the app manifests right. So argo cd, we have configured our cd to watch our uh application in this uh directory for this repo. So how that is created. So there are multiple ways to configure application. One is from the argos dashboard. We will come here in a minute and another way is uh using uh using yaml, so we created application yaml where we told you that this is my repo.

B

This is the directory which you need to watch and you need to maintain the resources on the cluster. uh So now we can so once we applied the uh applications here on the cluster. Argo cd will create all the resources on the cluster and will show us in the dashboard. So if we refresh here okay now we did some changes in the repo right and now adversity says this is out of sync that, because, uh in the report the image is changed right.

B

When we merge the pull request, uh the image is updated and now argo is saying: do you want to update uh the image? Basically the deployment? It says that this is out of sync, so there are two ways: uh one is manual sync or you can also configure as automatically so currently we have uh manual sync, so I will just do a sync here and argo will fetch the latest deployment and update it on the cluster. So now we can see. This is thing right.

B

So if I go to the staging cluster, if I do cube ctl gate parts.

B

And we can see the images uh bodies recently started right, 70 seconds now, if we go to our staging instance, which was not updated previously, now you can see yeah. This is updated and this is our staging instance. uh So this is how we automated the things. uh Some other steps are manual and some are automated. So if you see here uh just to recap, uh we did some code changes.

B

We had a event listed listening for the events which uh started a text on pipeline which uh cloned the code build the image and push it to the imagest, updated our dev instance and then created a pull request. So uh before merging the pulse request, we tested our changes on the dave instance. The test test, uh the changes were fine. We are okay with the changes, then we merged the changes uh in the repo and arco city was uh looking for the update.

B

It figured out that there is something changed and it was uh showing that out of sync, so we went and we said yeah you can sync now, because we are happy with the changes and it updated the change from the staging instance. uh So this is how you can use tecton and rbcd together to make your uh development and deployment uh uh quite easy and synchronized, uh and you will find all the information, the explanations and the installation steps in this repo.

B

That is techno dash, argo cd and if you can just try it out, you can just try it on one cluster. It is not required to have two clusters. You can just install technology on the same cluster and try it out in so you. You will have just uh two instance in different name spaces, and that would also work. So if you get any issues, uh feel free to create an issue here, and we will just help you out and yeah. Let us know your feedback here. Also, that's all from the demo.

B

Thank you for watching.