Cloud Native Computing Foundation GitOpsCon NA 2022, 28 Oct 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Lightning Talk: Managing the ‘Git’ Half of ‘GitOps’: How to Structure... - Jim Sheldon

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Lightning Talk: Managing the ‘Git’ Half of ‘GitOps’: How to Structure Infrastructure Code Repositories - Jim Sheldon, Harness

With popular GitOps tools such as Argo CD and Flux CD, managing your Kubernetes configurations in Git repositories has never been easier. Unfortunately, often the structure of this code in Git repositories is an afterthought, which can lead to significant refactoring in the future.

This talk covers four approaches for structuring the code applied by GitOps tooling:
- Application and infrastructure code in one repository
- Separate infrastructure repository, multiple branches
- Separate infrastructure repository, directory-based
- Multiple infrastructure repositories, one per environment

With each approach, Jim will present the benefits and potential drawbacks.

A

Yes I'm, my name is Jim I'm, a developer Advocate with harness and yeah I'm here to talk about some different ways. We can manage our git Ops configuration and how to promote those changes between our different environments, so I'm going to share four different methods that I've seen teams adopt for different reasons, the first being an application where the application and configuration are in the same Repository.

A

Second splitting up the configuration into its own repository, where there's a branch per environment next again, another repository but a directory per environment and finally splitting up multiple repositories, one per environment.

A

So, let's start off looking at a repository, this is a node application where we got some application parts of it and then some parts that are specific to its runtime configuration um so I'm intentionally avoiding any file extensions. Here in case we could be talking about yaml, maybe we'll be talking about terraform. It was a great talk this morning about using terraform in get off workflows so intentionally not putting making this specific to any specific tool.

A

So our git Obstacle of choice, maybe Argo, maybe flux or something would come along and watch the development file for development, environment changes and production for production, environment changes. So anytime we need to compare differences between those two environments. We would be able to look in one place, so that would be kind of nice um and another Advantage would be that the application and configuration are all versioned together.

A

So if you're ever trying to reproduce a problem track something down in the future, you don't have to connect the dots between multiple locations to get to the bottom of it, but this could be challenging um where, uh if you'd have some kind of CI process right, that's watching this application you'd it would have to be smart enough to not trigger when you're changing your configuration files, you could get maybe end up in an endless loop there.

A

It could also be challenging if you you have to go to your team, that's managing your get Ops tool. Every time you have a new application right. Every single repository has its own application. This way it might not scale well um up. Next, let's look at splitting out configuration into a separate repository where there's a branch per environment, so changes would always originate in the development branch and then get promoted to the production by getting merged into production branch. So these two environments, then, are just only differing by time.

A

Production is always destined to have development at some point in the future, so um this can be challenging where, if you make changes in your development environment that aren't ready for your production environment, you might end up having to do some complex cherry picking to pick up the only changes you wanted so as you're trying to promote them.

A

So that's something to keep in mind. So next, let's um talk about a separate repository where there's directories per environment. This is very similar to the example we were seeing before from ring central I believe from Ivan. uh So it's good to see that they're, using that, so each environment gets its own directory and when we promote changes we are ideally just copying files drag and drop. Maybe um we should be able to do that. We shouldn't have to worry about the differences in these files. The promotion should be an operation, that's a copy operation.

A

The development in production, specific files will have changes in them that are meant to be different, but everything else. Ideally, we can just drag, drag and drop. This is what Argo recommends and I believe flux as well. Organizations like Adobe and Intuit are definitely using this method, and this is what my previous company we did.

A

Each team had its own repository like this, so it was definitely successful, then um something to keep in mind that I wanted to mention uh that, if you're concerned about security, with this kind of a workflow, multiple developers being able to change the development and production, something I just learned about is this feature in GitHub called code owners. It's apparently been in there for years.

A

I just found out about it last month, but with this you can do rules saying hey these people, or these teams need to approve changes to these files or directories before they can be merged. So that's worth looking into if from a security standpoint, you might be concerned about this kind of workflow uh yeah.

A

So, finally, let's look at uh splitting everything out with a repository per environment, so here it's the same idea where we have multiple environments and the ability to copy from one to the other is what we're, what we're aiming for a little more complex. Now that we have to have two repositories, we've got to make sure we're on the right branch in both of them before we start making any changes and trying to promote something. uh But that's another way if I've seen companies acquire other companies that are on a different git solution.

A

Maybe this is what you'd have to do in some interim solution to uh keep your keep them work, keeping the working together but being able to use their own git Solutions in the meantime until they all get on the same page.

A

So I wanted to highlight that, no matter what workflow you're going through, if you're, finding that you're getting dozens or hundreds of lines of changes and that you're trying to review as you promote between environments, um an artifact might be able to help you so Helm charts, maybe terraform modules. We saw this morning using the terraform operator.

A

Then these things can help you to maybe get to the point where your promotion between environments is aversion change instead of dozens or hundreds of lines. So that's never fun.

A

Finishing up here's a bunch of resources. These slides are up on the schedule, so hopefully you can find them easily. If you have any trouble, come find me. Let me know the first article is from Gaia code fresh named Costas. It's highly recommend checking out that article. It's got it expands on all these topics gives a bunch of recommendations and uh pros and cons. I wrote an article it's up on the harness blog, some links to the Argo and flux documentation and a recording from Costas uh from argocon last month is on the last link.

A

Finally, if you're interested in harness at all here are a few links to what we do and some of our Community Resources will be on the main floor next tomorrow and through the rest of the week. If you want to come, find us find our booth and with that I'll say thanks for coming thanks for listening and find me after. If you have any questions.

A

We we do have time for any question like a question or two. If anyone has any one wants to run to lunch, yeah.

A

um Thanks for that, it was good. Have you considered using uh git sub modules?

A

um I've never had great experiences with Git sub modules. Maybe I've been burned in the past, but no that it certainly could work I'm thinking of areas, nothing to do with Git Ops, where we've used sub-modules and it was challenging, uh but yeah I can't see why it wouldn't work. Yep.

A

Thanks everybody thanks Jim.