Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2020 - Virtual, 4 Dec 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: DevOps All the Things: Creating a Pipeline to Validate Your OPA Policies - Goran Osim

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2021 Virtual from May 4–7, 2021. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

DevOps All the Things: Creating a Pipeline to Validate Your OPA Policies - Goran Osim & Karpagam Balan, Booz Allen Hamilton

Open Policy Agent is quickly becoming the de facto tool for applying configuration governance as code to your Kubernetes clusters. It can be challenging to understand how to optimize your workflows after finishing the getting started guide. This talk will focus on how to streamline the validation of your Rego policies using unit, mock, and integration testing to validate your OPA policy changes against your application manifests prior to applying these changes to production using OPA Gatekeeper. This talk will feature a live demo of using a combination of Rego unit tests, using your application’s Kubernetes manifests as input data to your OPA test suite, and using Terratest to perform end-to-end integration testing to apply your proposed policy changes to an ephemeral cluster orchestrated. Once validated, policy changes will be deployed using a GitOps strategy to a live cluster.

https://sched.co/ekEM

A

Hello, everyone thank you for joining us on our talk, devops, all the things creating a pipeline to validate your opal policies. uh My name is guaranosum. I'm a senior lead technologist at booz island hamilton, I'm a big fan of corgis ramen, and occasionally when I have two kubernetes with me today, I have.

B

Hi, my name is karpagam balin, I'm also a senior lead technologist at booze allen, hamilton, I'm happy with spicy food in spy movies, anytime and love anything open source.

A

Awesome so today we're going to be talking to you about open policy agent carpenter. Could you please tell us a little bit about oppa for those viewers that might be new to the technology.

B

Sure, oppa or open policy agent is an open source, general purpose policy valuation engine. It takes any json json data as input and evaluates that against predefined policies or rules written in a declarative language called rego by fashioning policies and code users are able to decouple policy decision making from policy enforcement and oppa allows us to define rules that lay out how your system should behave across the stack, and this, of course relates to the devops stack as well.

A

Right: it's not enough to have these policies live in a vacuum right at its core. Devops is about streamlining value to customers by shifting lift quality assurance, which includes security as much as possible. Continuous integration has become a standard that many teams adopt automatically by executing unit tests and integration tests as part of their devops pipeline.

A

These these same practices can and should be applied to infrastructures, code configurations, code, configuration management and today we're going to demonstrate how to apply these practices when developing europa policies to ensure that things work as expected, early and often carpet gum. Could you tell us a little bit about some of the tools that we'll be using in our demo to address these stages.

B

So yes, there are many tools that wrap around the oppa engine to provide policy enforcement capabilities at various layers of the stack and various stages of the devops cycle, along with some of the policies that we've written today in rego, we are using some such tools to help automate and strengthen our development and enforcement process.

B

One such tool is the gatekeeper which is a kubernetes admissions controller. When gatekeepers install in the cluster, the kubernetes api server will trigger the gatekeeper admissions web hook to process the admissions request whenever resource in the cluster is created and based on the results of this evaluation made by the gatekeepers opa engine. The admission of the resource to the api server will be either granted or denied.

B

Another tool we will use is called conf test, which is also a wrapper around the opa valuation. Engine conf test allows us to statically, run validation tests against policies that we have written before. We even actually go install those application and deploy them.

B

This allows us to verify that the policies we are writing are evaluating as we expect them to and, of course, the manifest we are developing are complying with those policies.

B

Another tool we'll be using today in our demo is called constraint. That's k-o-n-s-t-r-a-I-n-t um an open source contribution by an organization called plex that translates generic rego policies into specific templates used by gatekeeper that are called constraint, templates and constraints.

B

This allows us to keep our policies as generic regula code and have different tools, evaluate and enforce the same set of policies at different points in the devops cycle.

B

Finally, we will use git pre-commit hooks to automate using contest and constraint together to automatically validate the manifest the developers are. Creating.

B

This process allows us to ensure that all code that we check into our application repos is conformant to our opa policies. Grant I think, it's time for a demo.

A

I think you're right. Okay, let me switch our view here, okay and just to set the stage a little bit. We do have a kubernetes cluster, that's running, and I think my my port forward timed out of course, demo gods. All right. We have uh just to set the stage. We have a kubernetes cluster, that's running, I have argo cd deployed into it with some applications that we're going to use to showcase.

A

Here we have gatekeeper already installed in the cluster, so it's already activated and ready to do its job, and I have a couple of empty applications here that are going to hold our constraints right, so they're empty right now and sorry constraint, templates, as well as the constraints um which is where they're going to live and we're going to see them get deployed.

A

And then I also have like a little demo app of components that we're going to test against these constraints to see them launch successfully, so just to set the stage there, and so now, if we go to the terminal, um I have our little repository here and we'll start right with the policies directory right. This is where we're centralizing all of the various regal policies that we're writing.

A

So, if I look into one of these here, I can see that we have the rego source actually in here right. So this is this: is the rigo code that we're using to write the various policies that we want to enforce in our cluster? So this is good, but this isn't you know to kubernetes spec right. We need actually something to load this into our cluster. So that's where that tool that carpet gun was talking about, comes into play, which is called constraint constraint.

A

Does the job of taking generic rego policy files like this and creating constraint, templates and constraint files for us, which we then can use to launch into our cluster and have um oba consume them right.

A

So, just as just to dive a little deeper constraint templates basically describe the rego that enforces the constraint and the schema of the constraint and then the constraint itself is the basically the definition of what we want to enforce and how right and so for that we've actually written this little uh wrapper script around constraint and all this does it does some cleanup if we try to run this multiple times, but it'll call constraint against the policies that we've written in our policies directory uh to generate the constraint, templates and constraints and then it'll place them into two directories that I have argo cd watching to be able to deploy those files right.

A

So let's actually do that. Let's call this script now all right, and so, if I do a git status, I can actually see that we've created a bunch of files. We've created several constraint templates and we've created several constraints to go with them, so I can actually add those right now demo commit and we can push those to our repository right and so argo cd. Seeing this commit now is going to pick those up, but just so, we don't have to wait for the default.

A

Argo um refresh to actually check our repository, I'm going to hard refresh it here to tell it hey. We've got some changes since we're a little short on time, and so argo cd has already picked out the constraints that are coming up. It recognizes them from the source code that we've committed and it'll start creating those custom resource definitions which opa gatekeeper will then consume right. So the constraint templates are there and there is a. There is a order that needs to be done in place.

A

The constraint templates have to be created before and a constraint can be uh created. The constraints depend upon the template to know how to instantiate right. So if I go back into my applications and I go into the constraints.

A

B

A

A

Demo, gods, why there we go we'll do the same refresh.

A

A

A

This is crazy, it looks it's like, it knows, we're.

B

B

One day it's whether it's just a network traffic, it's not able to refresh.

A

All right all right, sorry about that internet issues. What can you do anyway? So now we've seen that the constraints have launched right, so um these have now been loaded into gatekeeper and gatekeepers, actually ready to start enforcing these new policies that we've uh just loaded in using argo cd.

A

So going back to my little demo here, I actually have a have a directory that has some.

A

It has some files that will violate certain policies that we have running in our cluster. For example, uh we've made it so that um containers that try to spin up images using the latest tag are non-conforming right. You might wish to pin your applications that you're launching in the cluster on specific versions. You don't necessarily want to use the latest tag, because that can contain breaking breaking changes that you, you know, don't necessarily want to deal with until you go through a rigorous process of evaluating those changes with the newer versions.

A

Additionally, we've decided to enforce resource limits right. We don't want people to create deployments or containers that don't have resources specified and finally, we want to be explicit about not running containers as roots right. So you can see I've commented those out, but basically, if I try to create this deployment right now, it's going to fail for several reasons that I just covered. So let's, let's validate that right, let's do a queue. Control, create f, with container.

A

And we can see here, this is a response from gatekeeper, saying, hey uh your action is denied. For all these reasons, right images must not use the latest tag. uh Container resource constraints must be specified, um pod can pods can't run as root, so we can see that literally the policies that we just instantiated into our cluster are already being enforced. um But what, if you know what, if I'm not someone that has cube control access?

A

What, if I'm just a developer, that is trying to commit some code right, run it through a pipeline get it deployed into a cluster? I don't necessarily use cubecontrol to get my resources into the cluster. I just make a commit right, so I have that demo folder, that's being watched by argo to deploy applications. So what if I just took this container- and I moved it up to the demo demo directory- that's being watched by argo right and let's add that and let's try to commit it.

A

Let's just do a commit we can see here. Conf test is basically validating um what we're trying to commit in the demo directory against the policies that exist in our policies directory right. So not only are we actively guarding against new objects being deployed into the cluster, but we're also guarding against you know the development phase of developers trying to push objects or code. That's non-conformant right. So this is just a local git git commit hook a pre-commit hook. That basically tells us hey, you're, trying to commit something, but it's actually not conformant.

A

So let's actually fix it right, let's go into the demo directory and into the container that we have in here and hopefully, if I've made it easy enough. With some of these comments, we can easily change this.

A

Let's see 25, let's change the latest tag and specify a specific version. Let's uh uncomment the resources that we want to implement here and then let's bring in a security context and specify that we don't want to run this route that we want to run as a specific user right. So, let's write that let's try to commit it and let's see if the git com, oh, I messed up a spacing. Of course.

A

I believe I am one short.

A

All here, let's try.

A

B

I think the security context also needs to be moved. One more yeah.

A

Thank you very.

A

Much there we go. Thank you so now we can actually commit this. We can say adding uh sorry adding.

A

Right and now we can actually push this, and if we push it, argo should pick it up and we can go into our demo.

A

A

My port forward failed again. My internet is just loving me tonight.

A

Go into our demo. Apps do a hard refresh and we can see that um argo is going to start deploying that container, um since it's picked it up from source code. So that's great! So let's do one more test. I have one more file in here um in this directory. I have a namespace that we potentially want to create right, so this namespace is going to check for a specific label right. Let's say we have istio running in our environment. We want to make sure that istio injection is enabled right.

A

So we'll do the same test here, we'll try to create it and we should get back hey. You can't do this. You know you need to have the proper label installed on your um in your manifest right. So same thing: let's try moving it see if we can be sneaky and get around this and try committing it.

A

Same thing, hey, you know your manifest, isn't conformant, so we're going to uh not allow you to commit this. So let's fix it again.

A

A

Right all right and let's refresh one more time and there we go the name space that we have. There is getting created right. So that's good cool. Let's go back to our powerpoint.

A

Right so, let's go over what we just covered right: we've showcased, how you can centralize all your rego policies. You can extrapolate this to your own organization and that this allows you to keep all your rigo policies for all your various applications and one location in source control.

A

This location can then be leveraged by any of your pipeline tools that require those policies. We've showcased, a process that allows you to automate the creation of constraints and constraint templates, which are valid, kubernetes objects that gatekeeper can consume, as well as the seamless deployment of those resources with argo cd.

A

We've shown, you use cases that validate these policies that are deployed into the cluster, with uh by git ops, using argo, cd and finally, we've shown validation in the development phase right using conf test and git hooks to not only give validation feedback to application and manifest developers, but at the same time continuously validate the rego policies that exist in our code repository.

A

So what's next well, the get the git pre-commit hook. That we showed is local right. You can take the same validation test and make it a part of your pipeline. So that the manifest and policies are validated before deployment policies can be developed independently, centralized in one location and can be enforced, enforced uniformly across the devops lifecycle.

A

You don't want to test new policies or changes to existing policies in a live environment. You could set up a pipeline that employs terraform and validates with terror. Tests to generate an ephemeral, kubernetes environment, apply your policies and then run a battery of tests to validate against those policies.

A

When those tests come back, you can improve the merging of new policies to your code base and that's it. Thank you so much for attending our talk and we'll be around to take your questions in the chat thanks.

B