Cloud Native Computing Foundation GitOpsCon North America 2021, 30 Oct 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: GitOps Cloud Resource Management - Brett Weaver, Intuit

Description

Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

GitOps Cloud Resource Management - Brett Weaver, Intuit

In this talk, you will learn about how Intuit is extending the support for GitOps to manage Cloud Resources. You will learn about Argo CloudOps, the open source project we developed to integrate Terraform and CDK using Argo Workflows to provide GitOps automation to manage our cloud resources. We discuss how this has provided greater security, automation and allowed us to extend Git as the source of truth for our cloud resource configuration.

A

Hello, my name is brett weaver, I'm an engineer at intuit and here to talk to you about an exciting new project. We are developing argo cloud: ops, intuit developed argo cloud ops to manage our cloud resources via githubs leveraging open source frameworks on multiple cloud providers. So what does that actually mean when you deploy a new service? You probably start with a configuration management framework to manage the resources for that service, for example, for terraform you create your main.tf and then run terraform apply at some point.

A

You may need to do some configuration prior to running terraform, staging artifacts or setting context variables to perform this pre-work. You may create a bash script to run prior to terraform apply in the future. You'll likely need to do something once terraform is completed and you might be running this on your laptop initially, and things are working great locally until you attempt to implement running terraform with minimally privileged credentials on a secure system.

A

This needs to be run against both dev and production, which have different configuration and access credentials over time. New services are developed which require their own configuration at this point. You've probably started running your configuration management framework on a build or cloud instance, an acquisition or two happens, and you have configuration which is done in different frameworks that you'd like to support using a single solution.

A

As the number of projects increases, you need to provide varying levels of access to teams who are responsible for managing those projects, as well as capture logs and audit trail.

A

You get this all working and eventually your business has requirements to run service and additional cloud providers. How do you onboard these new services with unique configuration management solutions for other cloud providers?

A

This is the situation where intuit found itself. After almost a decade of running our workloads in the cloud we developed argo cloud ops to solve the complex problem of managing our cloud resources across clouds and frameworks. It is guided by the following principles: argo cloud, ops is adaptable. You can get started in minutes and it can be expended to support known and unknown cloud providers and frameworks, as your requirements evolve.

A

A git ops, interface for managing resources.

A

Argo cloud apps is open source available in the argo project, labs organization and finally, enterprise scale, providing support for multiple projects and teams with role-based access control, while removing the need for direct access or admin permissions to manage cloud resources from continuous integration pipelines.

A

Why did we align to githubs as the interface for google cloud hubs? Getups provides a consistent developer experience, regardless of the framework. Our cloud engineers engage via known, git, ops, interface versus leveraging, multiple pipelines and tools for different cloud providers and frameworks get ops, pulls all configuration management from a git repository security controls and verifiable audit trail.

A

We are on a journey to consolidate to githubs as our interface for managing kubernetes and our cloud resources, providing a single developer interface to their entire service configuration.

A

In the following demo, we will download this example manifest from a trusted git repo and use it to deploy the following resources with amazon cloud development. Kit here is a quick demo of the experience using argo cloud apps to configure cloud resources. Argo cloud ops has the concept of projects which contain targets.

A

Access is granted to projects for confined user. In this demo we have created a single project, project 1 and are going to use it to apply changes to target 1.. We start by running the argo cloud up. Sync command sync accepts four options, which will tell argo cloud ops where to download the configuration within a trusted git repo for that project. This will be used to configure the target resources. In this example, we are using aws cloud development kit to configure resources in the target.

A

You'll see the sync command or return an id for the configuration being applied by argo cloud apps, which is run in a container managed by argo workflows. You can retrieve the logs for that workflow via the logs command.

A

You can see argo clouds that require shortcuts downloads the configuration from git and runs the configuration command you can use, get.

A

To get the status of a sync which can be used to trigger success or failures in a system calling over cloud ops, this was just a quick introduction for full setup guide on details on google cloud ops. Please check out the argo cloudups project in the argo project, labs, github organization.

A

Thanks for listening, we would like everyone to check out argo cloud apps and provide us feedback. Please reach out, and let us know if you think this can help your management, your organization's cloud resources.