Continuous Delivery Foundation cdCon 2022, 24 Jun 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Continuous Software Delivery using Jenkins, Spinnaker and Terraform... Irshad A Buchh & Paul Roberts

Description

For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/

Continuous Software Delivery using Jenkins, Spinnaker and Terraform Cloud on Amazon EKS - Irshad A Buchh & Paul Roberts, Amazon Web Services

Customers running microservices-based applications on Amazon Elastic Kubernetes Service (Amazon EKS) are looking for guidance on architecting complete end-to-end CI and CD pipelines using Jenkins and Spinnaker. Jenkins is a very popular CI server with great community support and many plugins (Slack, GitHub, Docker, Build Pipeline) available. Spinnaker provides automated release, built-in deployment, and supports blue/green deployment out of the box. This talk will explore configuration of Jenkins on Amazon EC2 using Terraform Cloud. Irshad shall also discuss the creation of Spinnaker pipelines, a combination of stages that enable powerful coordination for Continuous Software Delivery. These pipelines can be started manually or can be automatically triggered by an event, such as a new Docker image appearing in the Docker registry.

A

Hi good morning, good afternoon, depending on where in the world where you are my name, is paul roberts. I'm a senior principal solutions architect here at aws and today I'll be presenting along with my friend ershad on the continuous software delivery, using jenkins, spinnaker and terraform cloud on amazon eks rashad. Do you want to say a quick hello, yeah.

B

Thanks paul so hey good morning good afternoon, whatever you guys are, my name is arshad bush, I'm based out of dallas texas, hi to everybody and um I'm a principal solutions architect at amazon web services. I've been here for almost three years and uh worked with very large strategic aws partners uh on their journey to the cloud happy to be here.

A

Thanks rashad for that, so we're super excited here to talk to you today about continuous software delivery, so we're gonna jump into it and so from an agenda perspective.

A

What rashad- and I thought we would do is we would create a scenario- that's going to dovetail into our overall demo, and so the scenario here we'll talk about just for a minute is we have a pet clinic web app and the the idea is is that we want to modernize this web app using. You know these open source technologies.

A

You know such as you know, terraform, spinnaker and kubernetes, and in this case it's amazon eks. So you know next after we set the stage for that. Next, we're going to do a quick overview on amazon, eks and then we're going to drop into you know an overview on continuous software delivery that helps with this modernization effort and then finally, we're going to walk through.

A

You know infrastructure as code and how this is a fully managed uh sas offering uh with the terraform cloud product and then and then ershad going gonna walk you through an awesome demo, showing you from end to end on how we modernize. You know our web application.

A

So first thing is jumping into our pet clinic uh modernization. Here you know we actually have you know a couple pets, that's that that are headlining our website, and this is what the initial website looks like so the initial pet clinic app. Here we have two rescue animals. This is you know we have a puppy and a kitten here, and you know over time you know, as our pet clinic has evolved. You know we're continually continually looking to rescue new animals and any of you that have pets at home.

A

You know it's, it's always a wonderful thing to be able to rescue new animals, and in this case we have we rescued a new dog. You know several months back and here's actually a a picture of that dog, and actually she happens to be my my dog that I rescued her name is tilly she's, a white german shepherd she's super nice, but going back to you know our pet clinic app. We want to make sure that you know we have the latest visuals on that site.

A

So first thing is: is that our web application, the ui? We need to go through a refresh, and you know typically, if anyone's done this, you know if you have to make manual changes that can be somewhat difficult. So how do we go through and automate some of these, these these processes? And then you know rolling out. You know software and these changes or even infrastructure changes can be manual, and so, as we've alluded to earlier, we're going to talk about leveraging.

A

You know spinnaker and the terraform cloud on how to make this happen, and you know oftentimes. You know these small, you know small medium businesses, they don't necessarily have a huge budget, and in our case you know, with this scenario our pet clinic. We don't have a huge budget, so we want to focus on using open source software. This really helps control our our costs and then finally, you know we've heard and we've learned all about.

A

You know how hyperscale companies like netflix and snap are deploying uh into production at scale, and we really want to be using their technologies and their and their methodologies, and so we're going to dive in a little bit deeper on about how they do that, and you know again the overall theme here for our business. Our business is, you know, pushing pushing changes to our web.

A

Application in production can be a bit of a challenge, and so we're going to show you that it's actually much simpler to do that using aws and various open source technologies.

A

So, let's jump into what it looks like to actually modernize the the pet clinic application. So first thing here is: is that we as the the pet clinic owners at rashad and I as the the mock pet clinic owners, we decided. You know kubernetes sounds really interesting to us and you know containerizing.

A

Our web application seems like a better way to go versus using you know, virtual machines, and the reason is, is that we really want to focus on the the portability that containers offers us, and you know when we think about you, know the portability.

B

A

Is where you know we want to look at using something like kubernetes and when we looked at using kubernetes, we said hey, you know we could be using kubernetes directly on amazon ec2, but we, you know we're only a couple guys that that are running this business and the last thing we want to do is manage the the operational overhead of maintaining these clusters.

A

So this is where amazon eks comes in amazon eks is a fully managed kubernetes uh offering from aws, and this is where you can, where amazon eks will fully manage your ec2 instances using manage node groups. So this handles all the patching, the full life cycling of the respective workers- and you know, with amazon eks.

A

This is you know the same as the upstream kubernetes we're not making we're not forking. Anything everything is is identical to what's happening upstream and what aws does is we actually manage the control plane on your behalf and so managing that control plane? What that really means is that we have you, know the api server.

A

We have the lcd uh instances and so aws handles that heavy lifting and really it's just up to you or us, in the pet clinic example, to actually deploy our workloads to these workers that are running uh with our within our managed node groups. The other benefit here with with amazon eks, is that you can deploy this across multiple availability zones, so everyone wants to make sure that their applications and web applications are fully resilient.

A

So, when we deploy across multiple availability zones, our cluster can sustain outages, which is super important, because you know we want people to continue to be able to adopt pets, no matter if there are any any outages so moving on.

A

I want to talk to you about uh the different components here, uh and some of the these components are what you normally see in uh customer environments when they're doing their own builds so number one. You know, jenkins is probably the most popular continuous integration uh platform out there and I would say second from a continuous delivery perspective. Spinnaker is arguably probably one of the most popular cd tools, open source cd tools as well. So what jenkins allows you to do I as a developer?

A

I want to do some builds or compilations or I want to do. You know some tests with my software or maybe I want to write a custom plug-in that does you know some testing. You know for security tests or scalability tests. So with jenkins you can build all that automation into jenkins and those tests and what spinnaker can do is spinnaker can iterate through a pipeline, this logical pipeline of different steps of what you want to happen during your your software uh delivery or your software deployment life cycle.

A

So what's great is, is that I, as a developer, I can say I'm going to check in some code to my my code repository in this case, we're going to say it's just using git, and you know once I do that spinnaker is going to see that that code that has been checked in and it's going to be, it's going to kick off a pipeline saying: look. We need to go package up this code. We need to. You know, place this code. We need to do a build.

A

We need to place this code into a docker image. We need to push that to our uh our image repository and then subsequently, we need to be able to deploy that into production and so automating automated testing and building and packaging and at scale like it can be kind of complex. But when you combine combined jenkins with spinnaker and kubernetes, it's just incredibly powerful. So with that, I'm going to pass you over to my friend rashad and he's going to dive a little bit deeper into the overall architecture.

B

Thanks paul really appreciate it so um right so now, let's, let's talk about some of the uh components that paul laid out as part of this conversation.

B

The first thing that I wanted to focus on uh is essentially you know the terraform, the infrastructure, automation and, and then here I wanted to talk about terraform cloud.

B

um Most of us are aware of the open source uh terraform, which um we have a lot of community, and then we have been using it uh for quite a bit of time.

B

We have a lot of partners and customers using open source terraform, but I wanted to take it to the a little bit to a high level, which is the often called terraform cloud which is think about this as a fully managed infrastructure as um as code platform uh and and then it has some of the features that you you know you will see, are very valuable for you as an end developer as a as a customer, starting with there's a concept of organization and workspaces.

B

If you are an enterprise, if you're a big company, then within your enterprise you will have organizations, you may have you know different units uh vertical units and then within those vertical units you will have a number of projects. We call them workspaces, so it becomes very easy for you to scale out these uh end environments, mapping into the the individual organizations.

B

The second key thing is you know: how do you manage the state now for the you in in the in terraform cloud? State management is done automatically for you in cloud and it's encrypted at rest, meaning you know, security uh is a big focus, so you you don't have to store um if you're a team of developers. Normally, you know we juggle around storing the state when we use open source terraform.

B

So uh here um a lot of that work is uh done by the terraform cloud, and then it has a a great user interface. Where you know you will get access to a lot of these different uh organizations and workspaces, and you you manage the infrastructures code in the form of your source, control repositories and tie them into these workspaces.

B

There's also a concept of private module registry. Where you know you will you'll have access a lot of modules that are built by you as an enterprise, and then you share those modules across the board, with all the organizations.

A

So it's a lot of.

B

Reuse at scale and then leveraging git ops principles, which means that your infrastructure code, which is the hashicorp configuration files right um that sits in source control repositories, maybe a git lab or you know, a choice of your own and then you can integrate those source control systems directly into terraform cloud and anytime. You make a change to your infrastructure code.

B

What it will do is you will have an option of running uh those changes and deploying those changes using get ops principles so pretty valuable and then one other thing I wanted to highlight is the policy as code. There is a option available in terraform cloud and terraform enterprise, which is sentinel now what sentinel does is it gives you the ability of enforcing your governance? For example, you may have amended of you know: enforcing tax um across the board venue provision, resources, taxes.

B

One way of uh you know doing uh creating resources you, you could do wonderful things with tech, for example, you could monitor costs of the different resources using tags, so policy is code using sentinel will enforce your developers, your end users uh to to use tags and that there are many other ways of many other advantages of using policies for central and production.

B

All right, so I just wanted to spend a few minutes here. This is the the workflow terraform cloud workflow on the left.

B

Where you see you, you know you as a developer, think about yours, you're, a part of the infrastructure team or you're an application developer, and you have access to those private module registries right where you create a terraform modules or you use turbo modules available in the registry and then what happens that you use your typical ide, what you start writing terraform code using those modules, and then you start interacting with your terraform cloud. There are a number of ways of doing it.

B

You could do through cli uh through user interface, uh there's a rest api also available. So there are a number of ways how you can actually interact as a developer of your terraform cloud and then the state 5 management.

B

uh As I mentioned earlier, it's done automatically for you uh in in the cloud so and then, when you provision um the infrastructure, what would happen is, let's say: aws you're you're, creating amazon ecas, which is part of this demo, which is what I have done or jenkins pseudo, which is running on ec2, the elastic computer cloud service.

B

So what will happen is then terraform will provision all of these resources for you and maintain the state for you in the form of state the you know it is like repository for you, so all the developers will have automatic access to that state. So you don't have to juggle uh between different um developer or console machines that normally we do in the open source, terraform, vault and then anytime. We make a change. What will happen is as part of the get ops principles.

B

Those changes to your infrastructure code will be picked up by terraform cloud, and then those changes will be applied to your uh infrastructure at scale.

B

All right, so this is the architecture that paul and I came up with for the demo and also you know the theme of continuous software delivery. Let me go from the left to the right and by no means this is the only way of you know, visualizing how your uh different uh kubernetes clusters um can be architected. This is one way of architecting your uh your infrastructure at scale. So, uh let's, let's start from the left, so we have a um ide called aws cloud9.

B

This is your id in cloud and you have your source control, uh source control. Let's say github will have your infrastructure code as well as your application code. So we're talking about uh you could have a mechanism of you know creating different folder structures or different. uh You know branches, it all depends. There are a number of ways of doing that, and then we have terraform cloud to provision the different uh components that we need.

B

We need jenkins for continuous integration, which is an open source, continuous integration server, and then what we have is when jenkins builds our core, so we're using enterprise java for this demo when it builds uh those enterprise, java archives and then, ultimately, we have uh docker files and um we'll have as part of the continuous integration, we'll have a floor where we create those darker images and then push them into docker hub and and then in the middle. What what you have is you have a spinnaker eks.

B

This is one of the amazon eks server or cluster, where all I have is pinnacle installed and running, which is the which takes care of the continuous delivery.

B

Now, on the right side, what you have is you have two separate clusters think about one as the uat cluster and the terminology that that I use you may have a different nomenclature for it and then at the bottom I have the production cluster, so the workflow is when somebody makes these changes on the code on the left as a developer right once the infrastructure is provisioned and now we're talking about the application code for us right now. This is the spring pack.

B

Cloning application that paul talked earlier, and if you make any changes, then those changes will be picked up from the source control by jenkins. It's a you know. It's a continuous part of the continuous integration. Those changes of objective by jenkins and jenkins will run sort of a pipeline. So there's a concept of pipeline and jenkins.

B

As most of you know, and then as part of the pipeline, where jenkins will do, it will build test your code and ultimately, it will create a a docker image which will be stored and pushed into docker hub and once a new tag of that image appears in docker hub. What's then, what will happen is spinnaker in spinning curve we have a concept of pipelines as well, which I will show you in the demo.

B

So a pipeline will pick up that new tag of the docker image and pipeline you'll get will start executing um and it's all automatic, it's it's it's like weaved in and it works from end to end now, once a spinnaker pipeline gets executed, then within the pipelines failure pipeline you will have a number of stages and I will be using three different pipelines for this demo, which I will show you shortly and what will happen is then. The first pipeline will deploy the changes code, changes into uat cluster, and then you can.

B

You could go into the us uat as a uit, tester or end user and see for yourself those changes and once you're satisfied and then you can, you can have a manual process and or approval to go and approve those changes to go from uat into production cluster. So there's a manual approval pipeline in in spinnaker.

B

It becomes very easy to manage this and then finally you will you will see those those changes appearing in the production cluster, as well as part of the empty envelope.

B

All right, so uh it's demo time, let me uh share my screen um and so that you know I could show that gmo.

B

All right, so, uh let's go through uh some of the core things that I have. um So this is the jenkins graphical interface you see here and I have a job as part of the job. You know I have some build history and then there are in different stages that you will see you have the checkout, the build application test, application, build docker image, push docker image, and then you know some of the other bookkeeping tasks. That's the jenkins interface.

B

Now this is in my source control. I talked about the pipeline, the jenkins pipeline, and this is how we actually codify this, and there are these stages. If you can look at, for example, build the application test application, you can write your stuff, your scripts, there again codify once for all, build docker image and then uh push docker image into docker hub. So um and finally, you know yeah getting rid of those extra images at the bottom, just to make sure that you don't run out of space.

B

So now all of these uh are are plugged into in the form of the jenkins pipeline. It becomes very easy for you, as an infrastructure part of the infrastructure team that you codify this, because this will be kind of an automatic process. So again you see those jenkins pipeline stages laid out very nicely, which I will show you when you know we trigger some changes.

B

You will see, you know how it typically works, and um now, let's let's go to the spinnaker, so we have three pipelines as part of this phoenix splitting her um you know uh set up here. We have deployed to uat, which I talked earlier uh and if you look at, um you know how it's configured um it gets triggered um through that new tag of the docker image appearing in the docker hub.

B

So right now I have configured um for a specific uh docker image and then once that gets done, and then we deploy these artifacts using gamma files again this isn't. This could be in the github. This isn't the github right now, there's a pacling.yaml which contains the definition of the deployment and services.

B

Similarly, there's a deploy to prod. This is another uh spinnaker pipeline. Where you know we uh deployed production. Now this gets triggered by a manual approval process that you see here, uh which I will show you in a bit now. What happens is when you approve manually, then the code gets deployed to production. um Again, deployment is done through the the configuration of the manifest file. This manifest file is in your source control in your github repository, so.

A

B

These are two uh pipelines. um You know you could create your own architecture. This is just like an example. The third one that I've used is the manual approval. Now, if this gets triggered by the deployed uat, when uh source changes code changes get deployed to uit, then this manual approval gets triggered and as part of the manual approval, you have two options approve or reject.

B

Think about you as an administrator, uh you know you have two options. If you approve, then that those changes will get deployed to production, if you reject those changes will not go to the production. So this is a continuous integration, continuous delivery at scale now, also you can get access to the ingress or the load balancer endpoints of the two services which I will show you. You know it's a good way of getting those links. um So I have. I have two environments right now uh running.

B

I have the uat, which is this one right now you will see and you'll see the old uh that shape, which we you know we want to modify and take the next level. That's uit and the second tab. You'll see this is the production right and again we have the same image right now. This is how we start. This is where we start, and now we go through the process of the software delivery.

B

How do we do it once we make a change because we have to make a change, so let's go back as a developer. This is my cloud nine and I have the front end uh page and then all I'm doing is I'm updating the source code via the new pad image, um which is a small change. The concept is to make sure how the changes uh are incorporated. So I save the change and then, as an end developer, I go back to uh doing the real step which is like you know.

B

I community changes um that there's one change that I see right now. I made these changes uh into the source control repository right, um uh nothing new there. This is, this is part of part parts of how we work as developers, builders and uh the the purpose is to show um how do we tie all of these pieces together, and so when you look at the developer, workflow um right now, what I'm doing is um pushing these changes into source control.

B

So once I have git push, then let's go and watch how these changes will be picked up by jenkins. Now jenkins is right now configured uh to pick up the changes from my source control uh repository and then here you see, uh jenkins, build a new bill, gets triggered right and the same jenkins pipeline gets executed, which has got six stages.

B

uh It becomes very easy to customize these uh these stages right. This is like one of the examples um I have simple tests, I'm using maven, because you know we're using enterprise java maven is a good uh tool uh to do project management uh from the perspective of video, doing enterprise uh application development using uh enterprise gel. So you can easily tie in maven into your uh jenkins pipeline so uh and and testing.

B

I have a simple test here uh right now, just to show the demo, but you can have all sorts of tests included as part of your jenkins pipelines, so we built a docker image and then we pushed the docker image into docker hub, and so the jenkins pipeline is uh executed and now, let's go back into the spinnaker to use an interface where you have to uh you know, look at all the pipelines that you will see and what will happen is because there's a new tag of the docker image at the docker hub that will be picked up by the spinnaker pipeline uh labeled as deployed to uat, which is like think about this as like a lower end environment right and once that change gets picked up, then here you go so you will see this pinnacle pipeline getting triggered, uh and then you will have a number of stages in spinnaker uh pipelines right.

B

So it was so quick. Obviously, because I have only two stages, so it's already deployed and now it gets dropped into the manual judgment.

B

um So I don't want to go directly approve this, but what I want to do first is see whether the the new change that the update to the to the path uh to the app is reflected in my uat environment. So I go into the uat environment. This is what it looks like right now and if I do a refresh at the browser level, you will see that you know my pet application is modernized right. Like paul said earlier, it reflects the change now. If you take a look at the production, you will still see.

B

Production runs the older version because I have not deployed to production. Yet I have a tight control using spinnaker pythons.

B

Now, let's go to the manual approval process and let's say I'm satisfied as an administrator and I go in and approve the changes so what it will do now uh it will execute deploy to prod uh pipeline. That's that's the pipeline, which is the which is tied in the last um piece to to the whole mtn flow, and um it runs pretty quickly.

B

It has got two stages and then, ultimately what will happen is those changes will now be automatically rolled into production as well so from the spinnaker uh username interface, you will get access to all of those endpoints so, for example, service endpoints and you know, load balance so.

A

This is the production.

B

Right now, this is what it looks like. Let's do a refresh and you'll see that the new change is reflected in the production application as well, so so this is a demo um that illustrates um to you how we can deliver software continuously, using some of the important tools that I that we talked about. We talked about some of the open source tools like jenkins and spinnaker, and we talked about the managed kubernetes service, which is amazon, eks and how you can tie all of these together to deliver software at scale and codify this.

B

You know because there is less manual intervention, so there is a chance of chances of errors, are reduced substantially and also it scales. Well, so you know, depending on how many times you check out code and then you know, work on those features coming directly from your end, customers, you can scale um extremely well using this methodology. I hope you like the demo um at this point uh paul and I are ready to take any questions.

B

If you have any questions around the architecture or around the different technologies that we used today, I'm happy to do that.