Cloud Native Computing Foundation KCD Berlin 2022, 14 Nov 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Goodbye Pod Security Policy - Hello alternatives - Mario Fahlandt, Kubermatic

Description

Back in December, it was announced to deprecate the pod security policy (PSP) in Kubernetes version 1.21 and to remove the API completely at version 1.25.

Website: https://www.kubermatic.com/

Organized by @Microsoft @kubermatic7173 @SysEleven

Thanks to our sponsors @CapgeminiGlobal, @gardenio, @sysdig, @SUSE, @anynines, @redhat, nginx, serve-u

A

Let's chat about policies and how policies work in kubernetes and why you need them. If you want to hear from me or see funny dog pictures, mostly on my Twitter just follow me on Twitter I work for kubomatic. We create two open source tools for multi-cluster management and kubernetes. Cluster management and I also organized the Google developer group in Munich for cloud.

A

So if you have a Munich drop by and say hello- and we are always looking for speakers, so if you're interested to give a talk on the Meetup uh feel free to ping me and uh yeah reach out to me, uh why do we need policies uh who of you is using policies in their cluster?

A

This is more than I anticipated, uh so congratulations uh you're, the uh the only few that are in the kubernetes ecosystem that are using policies.

A

um So we have the declarative style of kubernetes and it's getting more and more complex. So you have uh different people that use kubernetes clusters. You have the developers that just want the application to be running and they don't care why something is not working. They only care which port is open. How to do I get there and how do I uh do I expose my my service to the outside world? Then you have the operators on the other side, The Operators uh yeah they are unsure on. Is this really okay, what they are doing?

A

Why do you need to run a container with root access? uh Why are you? Why are you putting my my complete cluster in at risk and why are you working uh in a certain way, and so you have a couple of different interest groups that Collide on each other and uh have different interests on security inside of your own cluster.

A

So we need something in any way that we can create a rule set and make it easy for the people that are only consuming kubernetes and make it easy for the people that are operating kubernetes to put an uh security level uh onto your cluster for very specific Parts um of yeah, everything that runs inside your cluster and all that your cluster is doing uh I mean we have. We have some external tools that help us with this, and these are Helm where we can Define previous installations and specific configs, but they are General.

A

So they are not environment specific and they are not going into the deep part. So what we want to do is we either ideally want to use policy as code, so we want to make restrictions to our cluster with a certain written down style and because everything is CI CD now and everything needs to need to go into some Repository story and be reproducible. The idea would be to have restriction and policies as code and there we have different levels where we can put enforcements on.

A

So we have the first level that we can have that we can check our manifest before they're getting deployed. This is way easier because it fails already in your pipeline. The Next Step, where we can hook into is the admission level of the kubernetes API. So there we can either mutate things that go in there.

A

We can change the uh we can change parameters there or we can outrageously block something that goes in there and afterwards we have the possibility to yeah get a report, get a log and also get the yeah the feedback from what happened to the cluster or what? What went into the cluster so when we have a look and in how everything goes into your kubernetes cluster? So if you, if you say I, want a new deployment, what happens? What happens in when you do this? So basically we go from.

A

uh We send our request to the API Handler and then we go to the authentication. If yeah, our account is even allowed to do this, and then we are at the first point where we can potentially change something and where we can potentially alter the uh the command that goes in and there we are at the mutating admission.

A

So we can use web hooks to alter or test or check what goes in and what goes out and then we have the object, validation, schema and then we have also an a validation admission where we also can hook in. So this is important for the future. So to do this, uh we had in the past pod security policy who used pod security policy.

A

Were you happy using pod security policy? Why why.

A

Yeah, basically, that's it so um Port security policy was originally created as an inbuilt admission controller. It is in kubernetes for really really long time. This is what most of the people don't know. It was in kubernetes since uh 1.3 in a beta and it's beta since 1.8.

A

So it's been along with us for a long time, and it is basically you can Define it at kind of arbuck for pots, so you have like what can they do like you are not allowed to run as a privileged container or I can restrict you to use as a amount of specific host names. uh Namespaces I can restrict you from using networking types and networking and pods and volume types and how which of the file system you can use and stuff like this.

A

So from the idea, it was really really cool because yeah, you could restrict things and how did it look like so you created uh you created a resource, uh pod security resource, and when you create this, it does nothing it's just there. So for this example, we see uh it's, we want to uh forbid privileged containers, and so you defined it but to actually apply it to your cluster.

A

You needed to also create have a service account that had the rights to enforce it onto your specific namespace, and then everything yeah would go through this uh to this policy. The problem that we had is we created it and then it's either run it or don't run it. There's no test, there's no try, there's no there's only enable it and it works or it crashes your whole cluster and you're wondering. Why is nothing running anymore?

A

So this was the problem that we had. There was a long discussion and there were a long development phases and they really went the Pod security policy and uh they looked into what could we change and they came up with.

A

We don't change anything. We just kill the whole project, so Port security policy uh will be removed from kubernetes and uh the reason why we already heard some of the points. um The funny thing is. It was also really confusing because you had to enter it in a specific order or with a specific naming to uh so that you have like an order of policies that come from top to bottom, but this was not convenient at all, so you needed to name the uh the policies in a certain way so that you can use them.

A

So no new people to kubernetes could understand anything of this. uh You could not extend anything, so you could not add new rules like hey I want to prevent that people only pull from I want to force that people only pull from a specific repository.

A

uh I couldn't do this, and this is maybe something that you want to enforce, because you don't want to use that people use Docker Hub because of rate restrictions or because of security aspects, or something like this, and uh it could not cover all of the security concerns that you might had and the main problem that we face now it is deprecated and it will be removed with kubernetes, 125 and 125 is not so far away. uh So we need an alternative.

A

The good thing is the kubernetes project itself created an alternative, and the alternative was the prod security admission. Funny thing about this: it was PSP was deprecated in 121. and PSA got Alpha in 122.. So we basically, we are running one version without anything, uh that's not a problem, uh so we are now can use pod security standards um and they are way more easily defined. So they are. There are only three standards, you cannot change them at all and they have uh specific set and they are um like report.

A

Security policy is also restricted on a namespace level, so we have three run modes that we can use and force nothing works if the policy is not met, ordered the violations to this will be tracked in the audit log. The port is still created and one it will all it will only trigger. uh It will trigger a warning facing only to the user who the uh who deploys said yeah you're violating our pod security standards there, and these three, these three standards that we have is we can run it as privileged.

A

This is the yeah. Do whatever you want so use. You have access to everything. You have the widest range that you can basically use. Then you have Baseline. This is the minimum restriction. This is what uh the kubernetes project deems as this is okay for rights, for a container to run, and then you have restricted and restricted is basically the best practices for hardening hardened setup and everything is really really restricted to to the base level there and yeah. That's not so much, that's even less than we had with PSP, so we needed something else.

A

The good thing is there are a couple of projects inside of the cncf landscape that are taking care of yeah policies and all already took care of this before it went deprecated to have more options uh available. So what we want to have a look at is two of the projects.

A

The first project is uh open policy, agent, gatekeeper and the second project is Cubano, so the open policy gatekeeper project is a uh I would say the oldest project um that we that we have available, and uh the good thing is it's already in a graduated State, um but don't be mistaken. The graduated state of the cncf is not for the gatekeeper project, but for the OPA project, because you can run Opa also in other environments.

A

So if you have a if you're working in a big company, it might be that other parts of your company or also use open policy agent in different parts as your policy management tool and the good thing is, then you can just adopt those policies.

A

um It's yeah, as I said, gatekeeper is the sub-project of opa for kubernetes, and it's the good thing is that it has this unified approach. The only problem that we have is you need to learn. A new language and the language is legal. Regal is only there's, nothing, not no other use. It's just in Opa. You use this or you use nothing. The good thing is there are some user user libraries that you can also use, and how does something look like?

A

So we saw the snippet that we had with uh only privileged containers and- and now, let's have a look- how this looks like with opa.

A

So this is the same thing that we had with four lines of code uh with PSP. Is now in uh with written in Rodrigo, so you see the the Lego part um which, which you can find there and yeah it's it's not so easy to read.

A

So, if you're new to to anything uh today uh to the project world, this might be an issue for you, and uh you also have to Define where this rule, this policy that you created should be uh enforced onto the good thing is um you can use a you, can run it as dry run, so you can test your policies before they. You put them to uh you, put them to your cluster. uh You also have the possibility to mon uh to monitor them.

A

You also have to pull a possibility to get reports out of them and to just check what works and what don't work, and so we will get to another project uh which is I would say in a more open way for uh or in a more kubernetes way, um which is called Cubano.

A

Cubano is a project in uh currently cncf sandbox status, and it was specifically designed for kubernetes not as Opa just got adopted for kubernetes. It was designed for kubernetes, so uh it uses the kubernetes resource pattern and also the language pattern. So we go back to the declarative style and this is way easier for any of the kubernetes administrators to yeah yeah to adopt it. And the funny thing is um what means kubernetes in English.

A

Anyone yeah and Cubano is also Greek, because kubernetes is Greek and kuberno is also Greek and coriano is Greek for Gavin. So basically they've named the project as governing and to have a look there.

A

um We basically see where it hooks into so. We saw this before and we go into the mutating admission when the validating admission and we then have our uh our web Hooks and there we have our policy controller and we also have our uh generate controller and there we can also mutate what comes into our cluster.

A

So Cubana also have the possibility that you basically change the Manifest that goes into your cluster, so you can override things uh in the Manifest or in the deployments that go into your cluster, which makes it way easier to just enforce. Send us because you don't confuse your uh you, don't confuse a user that are just writing the deployments, because you just exchanged the things that it works in the end and.

A

How it works is, basically the uh you define a policy and uh you define what sure what rule it should have if it should match something if it should um have specific labels, if it should have uh your specific name spaces, and then you can mutate it, you can also verify the images and validate resources, and this also this goes down in a trickling way. So this is how we Define a privileged container, then with uh how we disable the privileged container with kuberno.

A

So this is way way easier to read than than opa um and it's yeah it's more. The way how how we should do it yeah and what should I do what should I use now?

A

um There are some key takeaways that, for which you should go so for the gatekeeper project, um because you can Define uh the whole policies and don't choose from an from an yeah from a set of specific policies. uh You can Define complex policies.

A

um Also, if you have Opa already running in your company another place, you can just adopt it and reuse it. This makes it easy and the OPA Community is bigger than any other community for policy management. uh The problem is, you need to learn, Rego the mutation web folks are or the mutations are not that easy to do, because they are only in an alpha State and um it I found found it harder to to get into Opa than to get into Cubano uh for Cubano. It's kubernetes native.

A

um The mutation is also not easy, uh but it's it's easier than in gatekeeper and uh it is faster to write because you can just select from a from a specific set of of policies that are already there and I never encountered the problem. That I cannot find a finished policy that I can just reuse.

A

The problem is, you cannot ex do everything like you wanted, because there's, no, no language and the community is smaller, so the talk was too long. I didn't listen!

A

If you you, uh if you don't use specific policies, go, you can easily go for Cubano if you already have a gatekeeper or are looking for a very, very specific policies. Go for for gatekeeper and um this tweet basically says everything: um I'm, not biased, but yeah I root for Cubano to be the better to be the bigger uh project in the end because it really took off in the last couple of uh in the last couple of months and for some bonus round.

A

There are also coupon coupon uh leverages webassembly, so you can write in every whatever language you want and compile it with webassembly. uh You can so codify your policies and there's also GS policy. Where you write your policies in JavaScript and the good thing is uh that what they do is they are the only policy tool that currently enables controller policies so where you enforce policies onto controllers and it supports the npm and JavaScript tool changes, so you can run tests and npm tests and stuff like this and yeah there.

A

You can find all of the projects and.

A

Thank you very much and goodbye from my dog.