Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2020 - Virtual, 4 Sep 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Open Policy Agent Introduction - Rita Zhang, Microsoft & Patrick East, Styra

Description

Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Open Policy Agent Introduction - Rita Zhang, Microsoft & Patrick East, Styra

Come to this session to learn about the Open Policy Agent (OPA) project. OPA is a general-purpose policy engine that solves a number of policy-related use cases in Kubernetes and the wider cloud native ecosystem. During this session the OPA maintainers will introduce the project and then provide updates on the latest and greatest features to land in OPA and OPA Gatekeeper.

https://sched.co/Zex1

A

Welcome everybody to the open policy agent project, intro and community update.

B

Hey everyone. My name is rita zang, I'm an engineer at microsoft and I am one of the maintainers of the gatekeeper project.

A

And I am patrick east, an engineer at styra and a open maintainer.

A

Let's talk about the open policy agent, originally founded by styra in 2016 as open source donated to the cncf and is a sandbox project in 2018 since then is grown to become an incubating project in 2019 and is currently on track for graduation.

A

The contributor base is pretty diverse. We've got 30 plus companies, which includes both you know kind of the upstream usual suspects, as well as end users contributing back to oppa. There's with that, you know 150, plus developers. I think this slide, maybe even a little old I think it's grown since then.

A

The user base is also pretty extensive. We list a few on here. If you check out the adopters file in the open policy agent, repository you'll, see that there's a lot of kind of name brand, cncf, landscape customers.

A

So what is oppa? Why is there one like this? First and foremost at its core, you get rego a declarative policy language. This lets you concisely express policies in a way that lets oppa, evaluate them efficiently, with the language you're. Getting all these built-in functions that help you do very common policy things anything from math or you know, handling jots is a very, very common use case in authorization, as well as giving you a really nice sort of uh expression, like the syntax for expressing policy sort of logic.

A

It's you know it's tailored for that. It's 100 built to be a policy language. The cool thing is the performance optimizations that you get in oppa are pretty extensive. uh Not only can it do things like rule indexing, it can partially evaluate policies. This means that you can pre-process a policy decision ahead of time and then only process. The unknown parts at you know actual, like authorization check time this language, you get a runtime which can be used as either a library everything's in golang, or you can run it as a server.

A

So opa provides a little http server that sits in front of the runtime and allows you to basically run it very common as a side car, a host level daemon.

A

The other thing you get with that server and also available with the apis, but most people use it with the server.

A

Are the management apis, so the ability for oppa to basically give you like the the full life cycle management of of how these these agents are going to be running, so the bundle service, bundles, basically being a pack of policies and data that you can configure opa to go, pull down and load and use status apis for oppa sending status out logs everybody you know like for for authorization, it's very important that you keep track of what decisions were made.

A

Oppa has a you know, first class feature for doing decision logging uh that log service gives you that high fidelity uh record of what the agent has actually done, as well as discovery api. This thing essentially lets you do dynamic configuration and policy distribution. It's another bundle of configuration.

A

The last piece of this puzzle is: how do you make those policies? How do you you know optimize them? uh So oppa provides a set of cli tools for letting you build test. You know check everything you need to do to these policies and the you know the source code for them as well as integrations with vs code with soon to be intellij.

A

Lots of different editors plus functionality for doing tracing and performance analysis profiling. That sort of thing the last thing is the playground which I'm actually going to do a demo of in a little bit. So I won't go too far into that one right now.

A

So, at a high level, the open policy agent is basically decoupling. Your your service, uh whether that's a server. um You know some ssh, you know linux, pan, module uh or or you know, I think, a lot of these ones that we show in the example here are api authorization specific, but a lot of people use it for uh configuration testing as well. You know if, if you have some policy that you need to apply on a set of structured json, opa can do that. The cool thing here is the input can be any json.

A

Oh, it doesn't care it's it's general purpose, it's agnostic to any sort of you know domain. So if that input, that request is a you know: serialized uh terraform template, that's fine! If it's a jot from an api request. Also fine! You know it's! It's all up to the the policy that you write to. You know use that input so on the bottom half of this diagram.

A

Here you have the rego policy, of course, the data that basically gives you the ability to have external data for opus sounds really vague, and that's because it is in the sense that it can be pretty much anything you want. Opa provides an api to push data into it as well as bundling it with the policies and pulling it in the cool part. Here is that this lets. You do context aware decisions so say you have a api authorization request coming in.

A

You may have a jot on there with some information about the user, but it may not have information about other resources or the ability to connect that user. To you know some other group, that's maybe not listed in the job, so the cool part, then is you can integrate opa so that the data it has available at that time you could have pushed in those they'll, say, group, permissions or other attribute information.

A

The last thing here is the policy decision being any json value out, so oftentimes you're going to have a boolean. A yes, no was this allowed, but that doesn't have to be the case. It can be a set of changes, a set of warning messages. I think a common cases for kubernetes as an admission controller. If you wanted to do mutating decisions, oppa can return back a set of patches that need to be applied.

A

So let me show you a quick demo: I'm going to open up the playground, beautiful, okay, this is the open policy agent playground. The playground is a open public site. It's hosted by styra. um Everything here is, you know, ready to run latest versions of oppa yeah. I think uh you know if you're familiar with some of the like, like the go playground or other things. This is a sort of common language tool. Nowadays, um there's some instructions in here when you first open this up.

A

uh Basically I'd recommend anybody who does this read through the the sort of readme here at the top um crash course into rego policy? Syntax. uh This stuff starts with the the hash mark is going to be a comment, so the policy itself in this hello world that you get kind of you know out of the box with the playground.

A

It basically is defining a rule that says hello is false by default, and then it's saying that hello is true if the input dot message equals world, so we can see over here on the right hand, side the input box uh again kind of back to that diagram. The input is any json, the you know if, as long as you can serialize it into json, it can be an open policy agent policy input.

A

In this case, we have message as a field the you know, value being world. So you know the logic of our policy says that hello should be true. I will click to evaluate and view the results sure enough. What we find is that yes, hello is true.

A

This is a very basic example, so let me show you something: a tiny bit more complex just to give a feel for what these can look like the playground is is packaged with a bunch of, uh like example, policies that you can use to get started.

A

Let's look at a rule based example, uh again, there's a bunch of information here on kind of you know getting started that sort of thing. um What we're going to do is jump down and look at the policy itself. So what we've done is we've designed? We've we've specified a package so for each rego file, a complete module, as we call them, includes a package definition essentially saying this is the the path at which all these rules exist.

A

um We're going to for this example be showing a you know: it's our back for an api authorization use case, so the rule that we're interested in is allow by default. We're going to say allow is false, so we're sort of saying that if you know unless otherwise specified when we query for app dot rbac.allow, uh you know it should be false.

A

When we do want to allow is when the user is an admin. This syntax here is essentially showing that you have an allow rule. That's true. If user is admin is true, user is admin. Here is a rule which is defined down below so user is admin is true if there is some variable, I such that data dot user roles, uh and you know the lookup of input.user at some element I is admin. So what is all this when we're looking at data?

A

This is that external data from the from that diagram that I was talking about so you'll notice, there's another tab over here, a bunch of json data. Again anything json serializable. You can put into opa the user roles key, so data dot user roles, so user rules we'd see that they're all keyed by the user.

A

So we say user roles, look up, input.user our input, backup here in the input panel is we have users alice, so we can see that under user roles alice and then inside of the user roles, we find an admin. So what that means is there is some I in this expression where we can make this this ex line. 48 and everybody can see that. But if, if this line is true, then user is admin is true, and allow is true so checking that we evaluate this thing.

A

We can see that yes, indeed, allow is true and we can see that user is. Admin is true, um so this is just a really kind of you know. High level example of sort of how you can write these rules and what they kind of look like. I would recommend that everybody try out the playground and also go to the open policy agent documentation.

A

The documentation is backed by the playground. So all of the examples and and demos in there you can edit and change and open up the playground as well all right back to slides okay, so we've talked a little bit about what opa is, but how do you use it? Well, let's take a second look at the broader ecosystem.

A

Right now you have policy everywhere, whether you know it or not, um and your ci cd services. You have policy, you know when, when does something get deployed, when do rules get run uh when when do jobs, get run so um for kubernetes container management? It's obvious. We have. You know admission controllers. We have pod security policies. We have all this other stuff right, um apis, of course authorization.

A

um That's you know it's probably the biggest use case around databases filtering again authorization. You know what who's allowed to query for what data? That's the sort of questions and then also, of course, down in this bottom right corner public cloud. We have terraform shown here, but basically like how do you manage infrastructure and the policies around that? So you know, is a user allowed to delete all of the servers in the cluster? Probably not. You should have a policy for that right.

A

So all of these locations have an opa integration point and are prime candidates for it. All of these use cases have been, you know, sort of tried and true, um I think, the main takeaway here anybody who's interested in using opa and wanting to use it in any of those those uh you know places across the stack go check out the opa integration index.

A

There's a shortened link here. If you can see the url in the screenshot, it's the docs slash ecosystem.

A

This thing is a live list showing you, you know all the different integration points, plugins and other other tools that are built using opa to allow you to enforce these policies all across the stack in different places.

A

I want to take a second to call out. One of these integrations. um It's a separate tool, uses opa as a library comp test. I think at this point I would guess that most people have at least heard of this in the kubernetes. You know ecosystem.

A

If you haven't, I highly recommend you check it out uh this thing. If, if you're doing any kind of you know infrastructure as code or even just like, if you, if you find yourself using config files in production, you should probably be looking at contest.

A

You know you'll find a use for it at the end of the day. It basically lets you do policy tests against config files and this these config files can be anything from kubernetes manifest to terraform definitions. To you know several others here, and this quote that I took from the readme um check out their website. uh The docs are pretty good. I think the other thing to call out here is that the conftest project has officially moved into the open policy agent organization. So it's now an official part of oppa.

A

uh So that's that's really cool. I you know big shout out to the the conf test. Maintainers and everybody who's been involved in that project. It's very popular and with good reason, but we are at kubecon.

A

Virtually and so I think everyone here probably is interested in kubernetes, uh so let's talk a little bit about kubernetes and oppa.

B

Gatekeeper is a customizable kubernetes admission web hook that allows us to enforce policies and strengthen governance, to extend security and governance in your kubernetes environment. Now, let's take a look at some of the motivations behind the project.

B

If your organization has been operating kubernetes, you probably have been looking for ways to control what end users can do on the cluster and ways to help ensure the clusters are in conformance with company policies. These policies may be there to meet governance and legal requirements or to enforce best practices and organizational conventions.

B

Often the need for policy is discovered after your system is already running in production, so introducing new policies can be dangerous and can often be a breaking change. Does a new rule break a core piece of software? Can it bring down the entire stack with gatekeeper? You can preview the effect of policy changes in production clusters to prevent impacts on existing workloads.

B

So here's a timeline of the gatekeeper projects in 2017, the v1 of gatekeeper with q management, was introduced and in 2018 microsoft donated the v2 of gatekeeper to the oppa organization and soon after microsoft, google, styra and many others work together to bring the v3 of gatekeeper project and now, in august, 2020.

B

We are preparing for the project to go stable and we're going to talk a little bit more about all the changes that have happened.

B

So, let's take a look at how b3 works when uh when we deploy the gatekeeper project to the cluster, what happens is when an admission request comes in the api server then sends the request to the gatekeeper, webhook and the webhook then receives a mission review request and sends the data to opa for reviewing the request data against all the policies that are currently in opa and once the decision has been made, then the data is sent back to gatekeeper and then gatekeeper then sends the emission response as a deny or allow to the api server and the audit functionality runs periodically in the cluster and the data.

B

All the resources in the cluster are then evaluated periodically against all the policies that are currently in opa gatekeeper is a dynamic, configurable emission controller. So no reboot is required for any policy updates. The policies are configured via krm so that the policies are reusable parameterized to encourage reusability for common policy needs. The project is also built on top of keybuilder.

B

Here are some core functionalities of gatekeeper with the validating emission control. You can control what end users can do on the cluster. With the dry run feature, we can gradually roll out new policies so that we can gain confidence in the policies before enforcing them. We can create referential policies to evaluate new admission request against existing resources in the cluster.

B

We can write policies via configuration, not code, then this can be done by creating constraint templates, which are source code for rego rules and schema for constraints and their parameters, and the constraints can then be parameterized and easily configurable by admins.

B

The audit functionality allows us to periodically evaluate resources against these constraints, and this allows us to have ongoing monitoring of cluster state to help us detect and remediate any pre-existing misconfigurations and with metrics. We have prometheus endpoints to provide observability of the state of the cluster and how things are working within gatekeeper.

B

So since the last kubecon in december, we've added a lot of new functionalities to gatekeeper. So let's go through the list. First, we have security audit completed by cncf security audit and we've addressed all the security concerns.

B

As part of that audit and we've added new ways to exclude namespaces, you can either narrow the scope of resources for audit emission and sync within the config resources, or you can actually specify at the constraint level by adding excluded namespaces as part of the constraint and we've added a mid label validating web plug to lock down what excluded namespaces are actually allowed to be excluded.

B

Now, with past security policy, sigoth has identified the need to decouple enforcement and decision making and gatekeeper is referenced now as an alternative to kubernetes psps and we've added semantic logging, which allows us to get cluster-wide violating resources from audit and emission violations within the gatekeeper logs and we've added some audit enhancements, which allows us to audit resources using discovery client instead of relying on opa cache, and we now have standalone audit to separate the audit and the web hook component and we've enabled multipod deployment for webhook and we've added readiness tracker to ensure caches have been loaded before serving traffic and we've removed the dependency on finalizers.

B

So you no longer need to remove finalizers when, when you need to deploy a new gatekeeper and we've added the capability to match resource based off object, scope to support cluster scope objects. Now, let's take a look at a demo. We've used agile bank in the past for gatekeeper demos, agile bank is in a highly regulated industry, and both developers and admins are unhappy with gatekeeper.

B

We want to free up admins time managing all the policies in the cluster and we want to unblock developers by enable self-serving.

B

In the past few kubecons, we've shown you how you can create your own gatekeeper policies. Today we're going to take a look at how to use gatekeeper to create the same behavior as kubernetes psps. It's important to note that kubernetes psps has denied by default enforcement mechanism.

B

What that means is once the psp emission controller is enabled without defining any policies it will prevent any pots from being created in the cluster in comparison, gatekeeper has allow by default enforcement mechanism, so this allows us to gradually roll out the policies test, the policies and gain confidence before actually enforcing them.

B

So, let's take a look at how to use gatekeeper to disallow privileged containers from being created in the cluster. A privileged container is given access to all devices on the host. This allows the container nearly all the same access as processes running on the host here as an admin of agile bank. I want to be able to ensure that no privileged containers are installed in the cluster.

B

Let's see how we can do that by enabling kubernetes pop security policy.

B

So here in this cluster, I have psp controller enabled, and by default I have a privileged psp that already enables privileged containers in order to disable this behavior.

B

As you can see here, I have a psb that in disables privileged containers from running so next, I'm going to deploy this psp in the cluster and, as you can see, it is now enabled, and as a developer, I'm going to try and create a pod that contains a container that has the privileged security context enabled.

B

Let's see if this will block me from creating uh the pod in the cluster and this psp basically blocks it from doing so. Next, let's see if I remove that and make sure that I do not enable this privilege permission by default and it lets me create the pod correctly.

B

So as an admin. I think this is really good, but what if I want to test this behavior and test it before enforcing an impact users? I can use the dry run feature in gatekeeper.

B

So now in my cluster I already have gatekeeper components installed and next I'm going to deploy the privilege, container, constraint, template and next I'm going to deploy the constraint but in a dry run mode. So I can test this policy in the cluster before enforcing it.

B

So, let's take a look at what the template looks like um as we talked about it before the template actually contains the rego rules that will be executed by oppa and the target enforcement point is a mission kubernetes gatekeeper and within the rego, I'm basically telling oppa that the input review object for all containers and init containers.

B

If the security contacts privilege is enabled, then go ahead and disallow it so now, as a developer, I'm going to try to create a pod with a container with the security context, privilege enabled and because this is now in dry run mode, it doesn't actually block any updates in the cluster.

B

So let's take a look at what the dry run constraint looks like now, as we mentioned previously, the constraint basically provides parameters to the constraint template and here, as you can see, I'm creating a constraint with the kind ps, kubernetes, psu privilege container, which is defined here, and I am applying this policy on pods kind and enforcement action is dry run now as an ammon, I want to see how many violating pots are actually running in the cluster.

B

So let's take a look at the constraint and, as you can see here, I have total violation. Two, and here are the pods that are running with privileged containers enabled so at this point I can actually go and update these resources, and I have a pretty good idea of what the impact of this resource looks like.

B

So I am comfortable with enforcing the policy in the cluster now now I'm going to deploy the constraint again, but not in dry run mode, and what this looks like is pretty much the same as before, but the difference is, with enforcement action dry run removed, let's see as a developer, I'm going to try to create the same pod again with privileged contacts enabled and let's see if this actually blocks me from creating the resource and as you can see here, the gatekeeper webhook will deny this omission request and specify which policy is in action and why um it was blocked.

B

So that's a demo to demonstrate how you can use gatekeeper to roll out pot security policies and enforce the security constraints in your cluster. And next, let's talk about the current status of gatekeeper. It is currently beta and we're working towards moving into stable.

B

um So if you have any feedback issues, enhancements or requests, please open an issue or join us on slack or join us in our weekly community calls.

B

So here are some potential growth in the future that we're looking at obviously move the project into stable status and ensure that it's production ready and we are working on a feature that emit violations as kubernetes events, we're also looking into mutation and supporting external data and we're also looking at enabling more audit functionalities, more metrics and more policies so that the communities can share some of the common best practices and more developer tools to help us validate and test. Our policies.

A

Thanks for coming- and we have lots of places where you can come join us talk to the open maintainers, give us feedback feature ideas. Anything you need first and foremost. Github is where the projects are hosted and maintained. We do all of our collaboration there so check out the projects give them stars, follow them.

A

Also, the slack channel is very active. I think all of the maintainers are there every day all day as well as tons of end users. So if you have questions comments, concerns check it out. It's really good starting point for you and that's.

B

It thank you thanks. Alright, bye.