Continuous Delivery Foundation cdCon 2021, 6 Jul 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Jenkins Can Do It All - Jason Odoom, Clarity Innovations

Description

Jenkins Can Do It All - Jason Odoom, Clarity Innovations

There are many platforms and tools at our disposal as Engineers. However, sometimes the most popular tool isn't necessary depending on the size of an organization. What is most important are the concepts, techniques, and innovations that tool creates. With this idea in mind, we can explore these concepts in depth using Jenkins, an open source automation server that allow teams to deploy, test, and build code. But this is only a minor description of the power and efficiency Jenkins as a platform can provide. We will apply the latest CI/CD practices using Jenkins to prove that maybe you don't always need to use the latest industry darling to move quickly.

For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/

A

Hi, my name is jason rodroom and I'm a devsecops engineer at clarity. Innovations welcome to my talk jenkins can do it all so I'll. Try as hard as possible to not invoke the k word when I say the k word, it might get a little too much, but when I say the k word, I'm sure you'll understand what I'm talking about I'll. Try not to use that word as much as possible in my presentation.

A

So one of the big motivations for giving this talk was the space the current space they were in. uh You know in regards to uh container orchestration platforms and and their popularity and, of course, that one big popular uh platform, uh the k word and whether or not you know you need it, whether whether or not you need to use it.

A

I'm sure there are a lot of small to medium-sized businesses that are using these platforms and they most likely don't need to use them, not because it's bad, but because they're just not ready it's not whether they're running a micro service or a monolithic application. That's completely irrelevant when it comes to whether or not you're ready to run uh container orchestration platform. But what really matters is scalability?

A

Do you really need to scale how many users do you need to support? How many users do you have, and that's one of the big questions to ask when you're looking to decide if you can or if you should start using a container orchestration platform, but one of my favorite tools is jenkins, which is a cicd automation, server and it's quite powerful.

A

It can do a lot and it has a very rich plug-in system which provides it ex morph, which provides it more features and makes it accessible as well as the built-in internal features, uh for example uh the different pipelines, declarative pipelines and just the amount of things that you can do by just using using it. So let's get started so there are a couple problems, as I hinted at in the previous slide in the beginning. uh The first one is popular container orchestration platforms are ignited formal in industry and fomo is fear of missing out.

A

So that's the fear of missing out. That's when you see everyone doing something that you wish, you could do you kind of feel like you're left out, and you want to be a part of that. You want to be a part of that greatness. You want to be a part of that next big thing and that's a serious uh problem, because your your peers may be doing it, but you have to understand.

A

Maybe your peers are: are corporations and you're, just a business and you're still learning you're you're still growing, and you probably don't need to scale now. You probably don't need this, and you know you don't need this and there are alternatives that alternatives that you should definitely research before uh considering diving into into something as as big as a container orchestration platform, because it can get costly if you, if you don't do the necessary research.

A

The second point is using and managing container orchestration platforms when you don't need to scale is wasteful and costly.

A

This is it gets expensive. Besides, hiring engineers are gonna have to not only learn this but manage it as well as the different features that come with platforms such as these. You know you may think you know what what this platform is capable of, but it's it's. It has so much to offer that you will be. You might be overwhelmed, so it's really important to be prepared and understand what you're getting yourself into, and the third bullet point is using platforms for platform's sake as necessary.

A

Just because it's there doesn't mean you have to use it. It's not for everyone, don't use it just because it's there it's there to help, not only drive innovation but help fix your issues to help fix your problems. And if you don't have these big problems, like some corporations, do it's not really it's not worth using sure it's okay to learn it in your free time and understand what it is, what it does and maybe prepare yourself for the future.

A

But at the moment, especially if your small business- and if you don't need this, then you shouldn't use it, and that leads me to the last point lack of research into alternatives before implementing the latest and greatest you have to do your research before you decide to implement a new technology in your company or organization.

A

If you don't know anything about it, just what's out there in industry and in regards to popularity and how everyone's using it that's not good enough. That's not going to help you and your company grow.

A

Don't don't follow popularity because everyone's using it, you have to understand the technology first and then you have to most importantly discover if that piece of technology will help you solve problems in your organization.

A

So why jenkins jenkins is, in my opinion, very simple: it's really uncomplicated to configure very extensible via plugins and other built-in features. It has sufficient security features to allow different users to have access and limit their access.

A

It's also cloud agnostic, meaning you can use aws, gcp or azure to integrate.

A

So no trade-offs jenkins are scalable, supports containers, it supports automation, it is a cicd automation server. After all, it's definitely metrics capable, and it's very.

A

Configurable, so what does this mean so jenkins? If jenkins is missing a feature that your organization needs? You can always look for a plugin. There are so many plugins available to provide more additional features to jenkins.

A

So how do you determine if jenkins is right for you? So let's take what I said previously in regards to when to determine not to use the k word or uh a container orchestration platform.

A

So if you know, if you know that k word isn't right for you, maybe jenkins is right for you. Do you need high availability so in terms of scalability supporting many different users and having to be always available globally, especially, is this important to you and your company? Do you need to provide this feature now?

A

A

You can cross this. This first bullet point off. Do you need to manage a large amount of containers?

A

If you don't need to manage a bunch of containers or even deploy a bunch of containers, then you probably don't need to use the k. Word and jenkins is most likely your best choice, and also can you afford to manage it?

A

So if you can't, if you, if you don't need to manage or if you need to manage a bunch of containers, will it cost you.

A

This also, this is also a really important factor to consider. Also the last one. Does your app consist of multiple services.

A

This is this goes back to my first. The first point that I made in the beginning uh regarding, uh if your application is monolithic or microservice, which is irrelevant uh in regards to whether or not you should use the k word, but if you're, using different microservices and you're deploying them at scale, then also consider this.

A

All right, let's dive into jenkins.

A

So, as I mentioned I mentioned before, jenkins- is a cicd automation, server. It allows you to deploy apps to different environments as well as test them before deployments, it's open source and written java and supports many plugins.

A

Let's get started in this example, we'll look at the declarative pipeline, so in the clarity pipeline we basically have this syntax syntax. uh You know pipeline and we define our node being the the server that we build. Our application on as well as instructions for deployment later on in this presentation, we'll take a look at ansible tower and how it integrates into jenkins and how you can use ansible to actually support some really cool features.

A

We also, then, have the scripted pipeline, which allows you, which provides you more freedom, because in the scripted pipeline, you can create a more you can provide more instruction compared to the declarative pipeline in a scripted pipeline uh by the way is, is based on groovy, as is the declarative, but now you're able to code and actually define what you want to do and doesn't get in the way, whereas declarative you're, given sort of a bunch of syntax that you have to use, and you have very little room to sort of, innovate or or define what you need to do with your application or in your environment and scripted provides that freedom.

A

So do you, you have to do this for all pipelines, and what is this? Do you have to define each file each declarative, pipeline or script? The pipeline in your jenkins file, at least for every single stage in your environment, and the answer is no suppose we create a file a scripted pipeline by the way, it's important to know that this this code, here, whether it be scripted or declarative, would be in a file called jenkinsfile, which jenkins would parse through and and run.

A

But if we have different environments we don't have to. We don't have to write multiple jenkins files. We can rely on maybe two jenkins files or two to four james files uh at max and apply them to different environments or or teams within our organization, and then we do the way we do. This is via shared libraries. So there's here is an example for scripted and declarative.

A

So in the descriptive library we see here that we're importing our shared library our share library is in is at this location of this directory and we're using it in our development stage and we're basically importing code uh that can be used in our our jenkins file, and this is how it would look if you were to write this declaratively and it's similar.

A

So this allows us to reuse code without having to rewrite it multiple times. So we don't have to repeat ourselves, which is a very important.

A

Okay, so unleashing the power of jenkins, I should say so. One of my favorite automation tools is ansible. Ansible light jenkins is open source, it enables infrastructure as code and it's written, it's written. uh It uses yaml. So by writing. Yaml configuration files, you can provide it to ansible to to execute and jenkins, has actually two uh municipal plugins one being instable tower, which is a user interface for ansible. That allows uh people in your organization to create and run playbooks, and you can provide them the appropriate permission.

A

Think of it, as ins, think of it as a more friendly uh way for others to be able to interact with ansible as opposed to just using the command line and or giving them access to a server on the system to run playbooks or develop. Instable playbooks.

A

So before I move on to this next section, let's take a look at the jenkins file.

A

So in this jenkins file we have our declarative pipeline and I really want to combine ansible and jenkins at this point of the presentation, so we're using ansible and jenkins together, specifically asphalt tower where we define different job templates for each environment.

A

You know, like we have a dev, a job template for our dev environment, a test, job template for our test environment and a staging job template for our staging environment and in this example, we're using the jenkins choice uh plugin. So when you go to jenkins, there's a gui in your pipeline that allows you uh that provides a drop down to choose which environment to deploy too.

A

So an example: we have a node labeled cloud node, it can be labeled anything uh per the configuration of your jenkins server and we have a notification stage basically lets. You know that the build has begun and it started and it will send an email to whoever is in the email list of your organization. If you wish and then we begin to deploy or deliver the artifact for the deployment uh and then we do the same thing for the development, the testing stage and the station's stage, and also another option being delivered via tag.

A

If the artifact is tagged with a number and then at the end we send an email with a status. If it fails, then I'll send an email saying it fell. If it passed, then I'll send the email saying it passed when the internal ip equals is equal to the one that we've defined or specified, because we already know what the ip should be. Then we set the stage.

A

uh Excuse me: we set the environment to the appropriate uh node name or, or fact so, once we've done that, once we once we've established the facts, then we remove our old artifacts are all artifacts being the one we previously deployed, and I've just realized that I'm showing I'm way I'm going ahead and showing the atomic deployment configuration so really quickly. Let me switch back okay. So let's take a look at this configuration in this configuration. uh This is an example of an ansible playbook in this ansible playbook.

A

We are defining facts or variables for each environment, so in the first example we're set in effect for the dev server. If the dev server is has this ip address, if the ip address is equal to 10.1.101, then we know that it's the development environment and we're using the ansible default ip uh command to set or grab uh the internal ip address.

A

And, of course that's that's how we set the condition uh for for each environment in this playbook. So we know we're on the test and server or the state of the union staging server, because uh when we use the excuse me, we use the ansible default ipv4 command and we compare it to the internal uh ip or the ip that we specified here. If the condition is true, then it's going to set the environment appropriately next we're going to import dependency tasks, so this would be another playbook, an external uh playbook, in a separate directory.

A

That's gonna! Do uh it's gonna, do some additional things for us. uh Maybe we don't want to uh have like such a long file such as one playbook, so we wanna import separately, for example, and once we've done that, then we move on and we download uh you know our artifact artifact being our application.

A

So in this case we're downloading an artifact from nexus and it's going to download to our temp directory and then it's going to clean the the web route, where the application is being served, for example, for our www.html and once it's cleaned the web route, we won't be serving our application. Our application will actually be offline if a user tries to access it. They'll most likely get a 500 error and then, once you clean the web boot we're going to on un-archive or extract uh our artifact to the web root that being var wwe html.

A

What brewed here in the playbook is defined, uh and this variable is defined outside of this playbook uh globally in our uh ansible uh directory in our playbook. directory, and once we've done that, then we build the application and we compile the application, as I mentioned, to these node packages to be installed for our app once we've done that we add necessary configuration files, we add our database configuration.json template. This template ended in j2 for jinja, which allows us to interpolate different variables within our configuration file.

A

So the way variables work in ansible is by these curly braces and in in regards to template. We'd have this file that ends in j2 and you'd, see a syntax similar to this. That allows us to replace different parts of it with actual predefined variables in our configuration and then once we're done with that. We add another template, the environment, template and we add more templates for each environment if we're on stage and then I'll add station's own template.

A

Foreign test and it'll do the same thing for testing if we define it of course, and then replace the environment with appropriate server ip. This being, maybe you know, our application requires that a server ip is hard coded.

A

So we grab that via a variable server, ip, which we define above in this uh and in the facts and the ansible facts before we actually begin uh creating the instructions or the tasks, we define the facts for each environment. So that's, that's that's why it's important that we set this up before we make use of them uh later on.

A

And now one once we've done that once we've replaced the necessary uh variables and we've built our app next we're going to apply the appropriate server permissions in this case we're using apache web server. We will apply the appropriate and safest web server permissions and then we're going to start the app and in this example, we have storefront start from being. What? Where we're serving our our home page, our index.html and we're going to start that up and that's it.

A

Then we will reload apache and now it's accessible to our users and we're done- and this is cool, but the only major problem is. We took the site offline.

A

So when we deployed via jenkins, we deployed our app via jenkins uh jenkins, of course, used the ansible plug-in to make this happen uh and our jenkins probably defined. uh You know the ansible job template uh to use, as, as I briefly showed uh you here, so jenkins knew which templates uh to use which excuse me which playbook uh to run, uh but the uh and of course this is prayer environment.

A

But one of the biggest issues in regards to this is that, while we deployed we took our website offline, so now no one uh was able to access it.

A

If a user wanted to send us a message or buy a product, they wouldn't be able to until our our site finished uh deploying and who knows how long uh that took, of course, uh jenkins stores metrics uh in regards to how long a build uh takes, and we could use that uh to help uh decrease bill times if we, if we prefer, but not at the cost, hopefully of taking uh our site offline, where, where users can actually do anything and that's where atomic deployments come in app availability with atomic deployments, so atomic deployments are really cool.

A

Basically, uh they use sim links. So in this previous example, where we just took our site offline to redeploy a new one, that's how make the deployments make it possible. So we can continue serving a live website while deploying our updated or new website to the same host and once deployment has finished, we switch the same links and serve the newly deployed application.

A

So we avoid downtime by keeping the servers online while we're deploying, uh of course, this allows for pipeline expansion. We can do a lot with this.

A

I don't we, you shouldn't necessarily think about this as uh creating things from scratch, but this is one of those things where we have the freedom to do whatever we want, because jenkins allows us to through, through its configuration through uh the different uh plugins, that it has available and, of course, through uh ansible, and it's its many features and ease of use uh in my opinion, so this also allows us uh by the way, to roll back easily.

A

So in the case of of artifact of downloading artifacts from nexus, for example, uh depending on how long we choose to store artifacts in excess, we can later on configure a similar pipeline.

A

Let's call it a rollback pipeline, for example, and roll back previously deployed artifacts if something doesn't go well, but also here's another important thing to consider. If we're deploying an application via atomic deployments, where uh for where our site is online, uh it's continuing it. It's serving uh it's easily accessible to users. We've just deployed a new one. uh No one knows what's going on in the bank, obviously, except for the engineers. uh Users have no idea what's going on because they can still access the website and let's say that deployment.

A

That's in progress, somehow breaks something goes wrong. uh It makes its way from dev to testing all right and makes it the station okay uh and, and then it gets the production and something happens. uh Who knows? Who knows why? But something happens during deployment to production and the app fails.

A

The cool thing is this: biases time, because through the ansible playbook, we define in our our and I'm I'm skipping ahead really quickly, but we define this condition here. That says when so we're using this when condition available to us by instable.

A

So if our deployment fail or something failed, a file is missing, it won't, it won't complete, it won't continue and this can be up to us. Maybe the same link doesn't switch right uh and then we deploy again and start over, because at that point we haven't switched to the new uh deployed application, we're actually serving a copy of the previously deployed application, because what happens is every time we've deployed?

A

We mirror the current deployment to a separate directory that gets they get switched to during deployment. So this way we have every uh past deployment for us to serve to the user, and then we can wipe out a copy of that and then build the the current deployment or the current or new application and then switch sim links again to serve to our users.

A

So in this configuration this playbook this is the atomic deployment. We do the same thing that we do in the previous playbook. That does not support atomic deployments that takes our website offline, but the changes that we make here are crucial to actually keeping our site online. The first being we remove our current release, symbolic link, so these directories are are created beforehand. For example, var www.html is usually directory where you would serve what web content in this case.

A

We'd have, for example, for rw html release, nvrw html previous release, and then once we download the artifact, we would switch the symbolic link to in this example previous release, so where server appears released, and it's very quick and it's it's at the blink of an eye. No one barely notices anything and, of course, the previous release.

A

At this point, is it's just currently it's some it's a mirror of current release that was copied over uh clone at the end of the previous deployment, and once we've done that we start our app again, because this it's a different instance, it's being served from a different location, different to the one that was being served previously before we switched the sim links.

A

So in order to avoid any issues, we we start a new instance and we're using process manager 2 to handle that in this case, once we've done that we've removed our deploy, cache, we clear it completely in the pool cache being where we uh arc. We extract the the contents of our the archive that we downloaded from nexus and deploy cache, isn't meant to be persistent. It's just meant to it's. Just it's just a place for our app to sort of temporarily live.

A

While we we provision it and once it's it's good, once it's successful, we ship it out or copy it to a different part of the web server to be served later, and then we we removed the podcast, and we recreate it to avoid any uh permission or issues relating to you know, having old remnants of past deployments just to be on the safe side, and once you've done that, then we extract the contents and we begin to provision our application so, depending on the type of application you you'll know what ansible modules to use or how to provision the application, and here become true escalation, part, but anything that we do as root.

A

Of course, we take care of at the end by sending appropriate permissions, and we do this for for a couple of reasons, one being to to make sure everything runs runs. Well, there's no like there are no issues during provisioning.

A

um Of course you don't have to if you don't want to, but depending on the on the application and the modules are being installed, it may be necessary and in this example in regards to npm module, if you like to do that, there's certain uh features that you have to set. For example, let's say perm equals true or is true, and once we've done that we do a check here, and this is actually what I was referring to uh before, where we we check.

A

If, if uh the build has the deployment has pass- and this is just a simple one- it can be really complex depending on what you prefer, but in this example we check to see the inaccurate html, which is, of course the home page for the app in this case the microservice storefront we check if it exists. If it doesn't exist, then the user won't see anything once it's deployed and we won't allow. uh We won't allow for that. So we check to see if it exists.

A

If it is, then we store that sorry not if it is, but if it, if once we do the check, we store the value in a variable.

A

So we check to see if this index.html file exists, because this is our homepage. So if it exists, then we not necessarily if it exists. So once we do this check, we assign this check uh the value or the outcome of this stat using the ansible stat module to uh this. This variable storefront underscore index and we have a condition. So when you know when the foul or if the following is based on the condition, then uh we're gonna end the player, we're gonna fail this whole deployment and we're not gonna continue.

A

This is a nice way to build to to save time or or rather give us time to to to see how this happened to investigate before we redeploy- uh and this is actually the part uh if the configuration is not right, that's actually the part where things can get a little uh bad.

A

So, in the case you know, as you remember uh in the beginning, what we did was we switched assembling to the previous release and that's the one that we're serving now. So if we did the same thing for our new deployment and once we've done that, once we remove the current release directory, then we recreate it and then we just go through and set the appropriate web server permissions. We copy our deploy, cache uh directory to the current release directory because it's it's it's sort of passed. It's built.

A

It was built appropriately and now we can continue the process. We can provision the app and get it ready. You know move it to this directory, then get it ready to be served once the deployment is done.

A

We add our appropriate configuration files, remove any process ids from the previous release directory, that's being currently served and we stop all our services and then here's the part where we change the same link and we serve the current release of the of the application. And then we apply the appropriate web server permissions.

A

You can never be too careful and then we restart our application and then remove the previous release directory uh in preparation for uh the cloning, where we'll we'll clone our current release directory we'll copy the contents over the previous directory previous release directory, which will prepare us for the next deployment where that will be served.

A

So we can make way for the new version.

A

So now that I've showed I've explained some of the configuration for the atomic deployment.

A

We can move on.

A

All right, so, yes, that's how my deployments make version rollbacks easy and that's one of the issues that it provides besides uptime. Besides, you know, keeping your application alive and accessible to users. Thank you for coming to my cdcon talk. If you have any questions I'll be happy to answer them thanks.