Cloud Native Computing Foundation GitOpsCon EU 2022, 19 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Applied GitOps with Argo CD Autopilot Using Multiple Clusters with an Appli... Hannah Grace Seligson

Description

Applied GitOps with Argo CD Autopilot Using Multiple Clusters with an ApplicationSet - Hannah Grace Seligson, Codefresh

Using a GitOps controller, such as ArgoCD, to deploy applications allows you to deploy more often, execute rollbacks, and avoid configuration drift. However, what if you are an organization beginning to adopt GitOps and are new to Argo? What if you need to deploy hundreds of applications? Manage several clusters? How do you structure your directory? Or apply GitOps at scale for your organization? You can do all of this with Argo CD Autopilot, which provides an opinionated directory structure, allowing you to promote changes across environments with an ApplicationSet Controller for Argo CD applied by the Autopilot bootstrap. Autopilot enables organizations to simplify disaster recovery and quickly set up one-off environments or easily promote changes by leveraging GitOps. This tool works well for teams that are new to Argo CD but want to enable the ApplicationSet's automation and templating abilities to create, modify, and manage multiple applications simultaneously while also targeting numerous clusters with a no-brainer directory structure.

A

Hi there, thank you so much for joining me here at get ups con, where I'll be discussing applied, get ops with argo, cd, autopilot using multiple clusters and an application set.

A

My name is hannah sellickson and I'm a developer advocate at code fresh, which is a continuous deployment platform that supports both get ops and argo deployments. However, prior to being a da, I spent several years as a net developer, writing c-sharp and primarily designing and building rest apis for the insurance industry.

A

Later on, I took on a role as such as a coe lead, which is essentially a center of excellence lead. I made sure that our developer organization had the right tools, resources and trainings to execute their tasks efficiently. This also included vetting tools, monitoring workloads and then reporting back to our vp of I.t management and cto, with data that I've rather proved that we were working quickly and efficiently or we wouldn't, and we would have to come up with solutions.

A

So this is why I'm excited to be on the other side of the table nowadays, sharing with others these tools that can really help you so.

B

A

We're going to talk about why organizations are implementing git ops to their deployment process, and essentially, what does that mean and then we're going to identify some of the challenges that these organizations face when they are applying git apps to their workflow, then I'll, introduce you to argo cd, autopilot, we'll explain how it works and how it can help those organizations who are utilizing git, apps and using argo for their deployment process and then we'll give a brief demonstration.

A

So the infrastructure management growth. First, we start off with our programmatic automation, and this includes, like your powershell, and then we evolve into configuration management infrastructures code and then get ops. Get ups is becoming an essential part of the cloud operating model. It promotes devops, best practices to the management infrastructure, to improving the speed and quality of your infrastructure, service delivery and maturing infrastructure operations.

A

Making github's part of your cloud operating model enables you to standardize application and infrastructure management processes, driving both from definition, stored and version controlled repositories, so essentially your source of truth.

A

The search and popularity of get ops is due to the desire of speeder application deployment cycles. Organizations today want to be able to deploy their new features as quickly as possible to their customers. This improves business profitability and it makes everybody happy so. As a result, an increasing number of businesses are expressing interest in using this framework. Just this year in 2022, gitops is attractive because there's a small learning curve, it's fairly easy to set up and it can be adopted pretty easily.

A

It allows your teams to work faster, deploy their own code without the need of a heavy operations team, all in tandem with having boundaries put in place to ensure that those devs don't go breaking things and make unauthorized changes to your infrastructure.

A

Plus, it's important to mention that, because, just because you store your manifest and get through your manifest file in your git repository does not make your framework get-offs. Sorry getups combines an orchestration system kubernetes and your source control system with a monitoring software.

A

This is to ensure that we create a closed reconciliation loop in which an application running in production should always match its desired state. This is recorded in version control and your source control. Therefore, any deviation between the two will be detected automatically, so in today's evolution.

A

Tech industry cannot afford to wait several months to release a new feature and get in the hands of your demanding customers, like I mentioned before, get ops provides business value to those who adopt it and, regardless of your size, big or small, based on the fact that cloud applications can be updated faster without compromising the security or the stability of their infrastructure.

A

The benefits of doing this can sometimes be overlooked just due to ongoing debates of legacy, ways that lots of us can relate to when it comes to bigger organizations with clunky legacy systems in an outdated culture. However, based on research such as the devops research and assessment group, aka your dora metrics, there comes, several organizations have found that when it comes to delivery for a kubernetes or cloud app, there are four major variables that determine this organization's success and those are described as your software delivery and operation performance measures.

A

It's these four measures, frequency of deployment lead time for changes in updates, mean time to recovery and your change failure rate, which is otherwise known as the proportion of changes or updates that fail with get ops. It's a sure way to improve your organization's effectiveness against all those four measures, metrics that are providing critical and many business to market share and profitability.

A

So now that we've really kind of proved that hey, applying git apps to your workflow can really improve your business, your profitability and also probably the happiness of your organization and your teams and how they work so because they're working much more efficiently when you apply git, ops and you're, starting to explore best practices and making sure that you're executing this properly there do come a lot of chat challenges and those challenges can really be as simple as how do I handle multiple environments and apps and multiple clusters when I've containerized all of these services, and now I'm using uh you know a ci cd tool, and I don't know how to do this.

A

Also, when I need to promote changes to those multiple applications, and I need to also utilize different clusters, it can be a little tricky. So you know.

A

Maybe what you need to consider when you are thinking about these challenges are essentially the time constraint, engineers who are performing a lot of these tasks manually and just how expensive and inefficient that can truly be because the effort in transition to a github's workflow is super appealing, and you also have to consider like your testing and gatekeeping, and your regular maintenance processes that can lead to really expensive and also preventable environmental issues and outages for your organization.

A

So this brings up a question: how do we begin applying git ops to our deployment process, so we can deploy more frequently and reap those benefits. So it's no surprise. I'm mentioning argo is it. Argo is one of the most popular tools in the cloud native space and has continued to experience.

A

Rapid adoption argos has built in a git ops flow, and so that's why it's so appealing to organizations when they begin this, the shift in their in their culture and in their processes that they utilize argo, and I want to point out two major concepts when it comes to argo.

A

First, it's their application, which an article application is essentially what connects a namespace within your targeted kubernetes cluster, with the declarations of your desired state and your git repo, and this can be done by defining a lot of your configurations within a raw, manifest, a home chart or a customized overlay, and then the second major concept I want to mention is a project. A project in argo cd allows you to group your applications, which is useful when your organization has multiple teams. However, I also think of project as environments such as it could be regional.

A

It could be your staging your prod, and this is what's allow you to restrict what can be deployed and restrict what apps may be deployed too, such as your destination clusters and namespaces.

A

I don't want to get too thick in the weeds about these two concepts. However, these two major components of argo allow organizations to begin deploying their cloud native apps with git ops, quickly and efficiently. So, let's take a second to identify some additional challenges. When organizations are beginning to utilize argo, so two questions organizations have once they're on the get ups bandwagon and they're using argo. How do we structure our directories?

A

How can I enable argo cd to manage itself when organizations are first adopting git ops and begin leveraging open source projects and tooling, like argo, these basic concepts make others feel blocked, and it's because there's no there's. No one way to do any of these things. How do I structure my directories for multiple apps and environments, which structure will work best?

A

If I need to promote changes and then also raises the question of how can we install argo cd, so it manages itself or how can we install argo cd in a git ops way, because, honestly, trying to manually set up a way for argo cd to manage itself is not easy, because, essentially, what you want to do is you want to use argo as your orchestration engine, because it would be nice to let it rule itself through your git repo. Then it enables automation so that any change can show up on your cluster.

A

So, with these challenges, there's a way that we can kind of just at least have a foundation to answer them, or we have at least a roadmap that we can follow, and that's when I bring up autopilot autopilot is an open source project for the argo community that provides us with the tool to bootstrap argo cd.

A

It also includes an opinionated get op setup for your directory structure, and one thing I'd like to clarify about autopilot is that it is not a replacement for argo. It's simply an installer. You can use to install argo in a getup way. So, as I mentioned, this tool offers the opinionated way for organizations to kickstart, their github's continuous deployment process, and it allows them to add applications and simplify the process of promoting changes across those environments.

A

It also has a friendly command line tool for bootstrapping argo cd onto clusters, setting up your git repos, managing applications across environments and even simplifying disaster recovery. You can easily set up one-off environments or staging environments and promote changes across those in a straightforward get-ups way.

A

So how does autopilot do this? Well, like I mentioned there is a bootstrip command that you can execute that allows you to do this and autopilot saves operators times by installing and managing the argo cd application using get ops, providing a clear structure of how applications are added and updated all centralized from get creating a simple pattern for making updates to application and promoting those changes across environments and enabling better disaster recovery by being able to bootstrap new clusters with all the applications previously installed.

A

Autopilot is a tool which offers the opinionated way of installing argo, cd and managing git ops repositories.

A

A

So once committed argo, cd would do its magic and apply the applications to the cluster. An application can be added to a project from a public, git, repo or path or directory in your local file system on your machine.

A

Autopilot also communicates with the cluster directly, but only during the bootstrap phase, and this is where it deploys argo cd. After that, most commands will require access to your get apps repository and when you're, adding a project or an application to your remote cluster, the autopilot will require access to the argo cd server. So let's see that bootstrap command in action just give me a moment.

A

B

All right, so what we're going to do now is we're going to start by executing the argo cd, autopilot, bootstrap command, and so what this is going to do is it's essentially going to create our directory structure and our github repo and I've already assigned my github repo, as you can see, and I also provided it with a token so that has access to this. So now, I'm just taking a look inside my repo, where you can see the directory structure here that I had mentioned before, we'll just take a look.

B

This explains how to create an application.

B

And within this bootstrap argo cd customization file- this is where we're defining our resources. I'll just take a look in here now these are cluster resources.

B

And this is going to be the argo cd yml file.

B

Look through this folder.

A

B

Okay- and this is where our projects are defined, and it gives you an example of how you can create a project.

B

Let's get back to the terminal.

B

So when we complete the bootstrap command, it provides us a cube, cuddle port forward command that we can use to then access the argo cd ui. It also provides us a password that I can then use and sign in.

B

And now you can see essentially everything that we just created and just saw within my repo.

B

So, as you can see, argo cd is able to manage itself now with that command. I just created a project for prod.

B

We're going to go back into our terminal now, I'm going to install an application and assign prod to it.

B

B

Now I'm using a different cluster and I've created a project for staging and assigned the application to staging. This is now completed now we're going to take a look at the ui again and you'll be able to see my other application for staging still synchronizing.

B

Then we'll take a look at the project, so you can see the staging project there. We go.

B

Okay- and so this is just a brief example of how you can use multiple applications, multiple apps, multiple environments. Unfortunately, I don't have time to get too nitty gritty, but this is just a brief demonstration to show you how autopilot works.

B

Now we'll take a look into our file structure: okay, yep, there's prawn staging awesome. Thank you. Everyone! So.

A

Here's an example of a directory structure with aggro cd autopilot, I'm including a demo app and two projects and overlays for staging and production environments. However, whenever bootstrap command is executed, autopilot provides the following for you, a bootstrap folder, which contains all bootstrapping resources as shown and root.yaml file, which contains your application definition for projects folder a projects folder, which is essentially a logical grouping for of applications.

A

It's this project folder that can actually be used as a different environments. Like I mentioned for your staging your dev qa prod, you also have a clusterresources.eml file, and this is an application set controller. Managing your cluster resources folder by pointing the cluster resources in your cluster.json file. It uses argo cd, git generator file and manages the argo cd namespace in your kubernetes cluster.

A

An argo cd.yaml is an application definition to self-manage argo cd itself, so autopilot also has tons of commands, which you can use to add new projects, an application, which is why I mentioned those earlier. It creates and commits any changes made to your github repository or whatever repository you're using and ensuring that your desired state is applied by the argo cd operator.

A

So now, as you can see within my structure, I have multiple environments and for this demo application I can also create additional applications and leverage different clusters, but how well nowadays, argo cd incorporates an application set with its installation.

A

So what is an application set? Well, unlike an argo cd application resource that deploys resources from a single get repository to a single cluster or namespace, the application set expands this functionality. An application set, uses a template, automation to create, modify and manage multiple argo cd applications at once, while also targeting multiple clusters and name spaces.

A

The application set controller is now installed, alongside with argo cd within the same name space, and it creates multiple argo argo applications based on the application set custom resource. This is what ensures that your argo cd app remains consistent with your declared resources, and it essentially takes the application set and outputs one or more argo applications.

A

Well, fun fact something I didn't realize when I submitted my cfp, for this talk was that autopilot incorporates an application set and applying this pattern doesn't need to be a separate step when you need to create multiple apps.

A

So before I learned about this tool, I thought: wouldn't it be great if we could apply an app set after we install argo cd to set up our directory? Well, the maintainers of argo cd and the community agree that hey. It would be really helpful if we could install app set when we install our go cd, because we all are working with multiple apps oops.

A

So now that you've had your crash course of argo cd autopilot and seen a demonstration of how it works. Let's review some of the benefits autopilot provides an opinionated way to build multi-project multi-application system using get ops principles. It also creates a new git apps repository or you can still use an existing one that you have. It supports creating the entire directory structure under any path that you require you can, when adding applications from a public repository.

A

It also allows committing as either a customization that references. The public repo or as a flat, manifest file containing all of your required resources, and you can use a different cluster from the one argo cd is running on as a default cluster for a project or a target cluster for a specific application.

A

So argo cd autopilot, is still under active development and you can find it under the argo project. Argo labs and feel free to to check it out, learn more about it. You can also check out some existing issues and find ways to contribute.

A

I really hope that this helps you and your organization get started with your git ops workflow and makes things a little bit easier or at least provides an opinionated way of how you can move forward. Thank you. So much. Everyone have a great day.