Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2018 (Copenhagen), 7 May 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Multi-Tenancy in Kubernetes: Best Practices Today, and Future Directions - David Oppenheimer

Description

Want to view more sessions and keep the conversations going? Join us for KubeCon + CloudNativeCon North America in Seattle, December 11 - 13, 2018 (http://bit.ly/KCCNCNA18) or in Shanghai, November 14-15 (http://bit.ly/kccncchina18).

Multi-Tenancy in Kubernetes: Best Practices Today, and Future Directions - David Oppenheimer, Google (Intermediate Skill Level)

Kubernetes offers a set of features which, when used in the right combination and configured properly, enable secure multi-tenant clusters. But it’s not always obvious how to map particular multi-tenancy requirements to those features and configurations. As a result, people often resort to spinning up one cluster per user and/or workload, thereby foregoing the utilization and management benefits they could achieve by using a single shared cluster. This talk will describe a taxonomy of various multi-tenancy models that are possible on Kubenetes today, and how to configure the existing security/multi-tenancy features to satisfy each of those use cases. We will then describe some features that are on the horizon to provide even stronger, easier-to-use multi-tenancy in Kubernetes.

About David
David Oppenheimer is a software engineer working on Kubernetes and GKE at Google. He is co-lead of the newly-formed Kubernetes multi-tenancy working group, and was previously co-lead of the Kubernetes scheduling SIG. He has been working on Kubernetes since 2014, and prior to that worked on Google’s Borg and Omega cluster management systems.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

Join us for KubeCon + CloudNativeCon in San Diego November 18 - 21. Learn more at https://bit.ly/2XTN3ho. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

A

Okay hi, my name is David Oppenheimer I'm, a software engineer at Google and I'm. Also a co-lead of the multi-tenancy working group in kubernetes and I'm, going to be talking about multi-tenancy in kubernetes.

A

So probably a good place to start is to define what we're talking about when we say multi-tenancy and everybody has kind of a slightly different different definition. But the one we're going to use in this talk is providing isolation between tenants within a cluster now I'm, not sure that how much that actually helped, because I haven't really defined what we mean by isolation and what we mean by tenant and I haven't even explained why we were talking about multi-tenancy within a cluster instead of, for example, across clusters.

A

So we'll answer those first two questions in detail in a minute by walking through some examples, but first I wanted to quickly address the third question: why multi-tenancy within a cluster and for now until I, define what a tenant is? More formally, let's just say, a tenant is a user or application that you want to keep isolated from other users or applications in your cluster.

A

So the best way to answer this question of why we want intra cluster multi-tenancy is to look at the alternative, which is to rely on the isolation that you can get for free by running your tenants in separate clusters. So this is appealing because at first because you know the isolation doesn't require any kind of special multi-tenancy features within kubernetes, but it has some drawbacks.

A

So, for example, if you have a lot of tenants and for a very large system, you might have thousands or tens of thousands of tenants, for example, if you're running a software as a service or something like that, you're going to end up needing tools to manage those thousands or tens of thousands of clusters, and at some point you have so many clusters that, even with good tooling, it becomes unmanageable.

A

A second reason is that there's a resource efficiency issue with having so many clusters, because you have to pay the control plane, cost of the kubernetes master plus add-ons for every tenant in this, and in fact the control plane could easily consume more resources than one tenants. Application instance, if you're running a small application for each tenant, and so that's obviously an efficiency issue and another resource efficiency issue is fragmentation.

A

A pod can only use the free resources in the cluster that you submit it to, and so, if each of your running lots of very very small clusters, then you can't share any of the kind of unused space that the that the application isn't using. You can't aggregate that in the way you can in a large cluster and then lastly, you need to create a new cluster.

A

Each time you add a tenant and this can be slow and it may be difficult to do that on Prem and so, as a result, I believe that the cluster per tenant model isn't practical for a large number of multi-tenancy use cases, and that's why we have a lot of interest and work. That's gone into intra cluster multi-tenancy in kubernetes, and that's what the subject of this talk is so now. Let's talk about those first, two questions.

A

What are tenants and what kind of isolation do they require and we'll do this by looking at three concrete use cases and for each of them, I'll have one slide talking about control, plane or API server isolation, and then one slide talking about container or network isolation, because those are kind of two somewhat separable pieces of the isolation picture for different use cases, and then the pictures you'll see in this part of the talk, the rounded rectangles.

A

This is kind of an example of one you'll see them coming up later, are representing a shared cluster, divided into namespaces by call indicated by colors, and then those little icons are just supposed to represent different API objects running in those namespaces and then we'll also show various users that map to tenants the map to namespaces of the corresponding color and I'll talk a little later about why each namespace has exactly one tenant. But the basic reason is that the kubernetes multi-tenancy policies are generally set on a per namespace basis.

A

That's the granularity that most of those policies use and that's why people usually map one tenant into one or more namespaces, but a single namespace generally does not host more than one tenant. And lastly, as we go through these examples, kind of keep in mind that these are just kind of archetypes. There are very high-level examples, nobody's exact use cases, probably in a map exactly to one of these.

A

But these are some that we've seen in our experience that kind of representative of what people are doing so the first use case is enterprise and in the enterprise use case, we have users who are all coming from the same organization and so there's what we'd call semi trusted. Because you know you can you can't necessarily mandate exactly what they will do? They may do accidentally bad things, but if they do something truly malicious, you can fire them.

A

You kind of have some control over the users and in this enterprise use case often a tenant will map to a single department or team in the organization. So something that follows the organizational hierarchy and often we'll see three different personas in these kinds of enterprise clusters. So one will be the cluster admin who can, for example, create read, update and delete any of the objects in the cluster, including creating namespaces.

A

They can also manage policies, for example, setting per namespace resource quota based on some kind of internal budgeting, that the organization uses and also sometimes there's sort of delegated authorization model where the cluster admin can assign namespace admins who get to decide who has access to the namespaces. So the namespace admin may not be able to control things like the quota or the network policy which we'll talk about later, but they can decide who is a member of their team or department that they're responsible for the namespaces for and then.

A

Lastly, we have the regular users who can create, read, update and delete objects in their namespaces, except for policy objects that are the domain of the cluster admin, so that's kind of a typical isolation scenario for the control plane or API server. So what about the node and network for that? We need to think about what kinds of applications are running in the cluster, and so in this enter hypothetical enterprise cluster.

A

We say we have many different applications and, just like the users will say that the applications are semi trusted because the organization in theory kind of knows all the applications that are running. They either wrote them themselves or they at least have the opportunity to vet the code. But it may be open source code, and you know they obviously didn't check every line of code.

A

So there could be something malicious hidden in there, but there aren't external users on the Internet sending code in that's being kind of blindly executed, and so and this in this case the organization might be okay with the normal container isolation.

A

Just the features you get out of the box with a docker, for example, container or they might want to add additional restrictions like not allowing containers to run its route or use, hosts networking or run its privileged and we'll talk a little bit later about some of the kubernetes features that allow you to place those kinds of restrictions on containers and then at the network level they.

A

What kind of communication they want to allow between pods in this enterprise scenario depends on the application topology. So some applications are structured. Many are structured so that all of the pods of the application run in a single namespace, and so, in that case, you'd likely want to allow communication between pods in a single name space but prohibit communication between namespaces. On the other hand, sometimes people structured applications where they'll have one application tier per namespace.

A

For example, you might have your front ends running in one namespace and your backends running in a different namespace, and in that case you kind of want the opposite. You want to allow communication between pods in your front-end namespace in your back-end namespace, but you don't want to allow communication between pods within a tier, for example.

A

There's no reason that your web servers should have to talk to one another that your front-end web servers should have to talk to one another and in fact allowing them to do so could allow some kind of you know, privilege escalation, but allow someone's day to attack one node and then from there escalate the tech to another node.

A

So that's something that you actually would want to prohibit and then, depending on whether the application is serving purely internal users or also serving users on the Internet, you might want to allow or prohibit communication with the outside world.

A

So that's the enterprise use case. The second use case, we'll look at to illustrate the concepts of tenants and isolation, is what we call a kubernetes, a service or platform as a service, and the idea here is that we're talking about software as a service where the application is hosted, kubernetes, possibly in addition to extensions like producer supplied, CR, DS and controllers. So, for example, many of the platforms of service that are built on top of kubernetes.

A

They allow the users to interact directly with the API server and then the kind of value-add that they run on top of bare kubernetes is CR DS, for example, and a CI CD says it might be CR DS that represent like builds and and deployments and testing pipelines, and things like that. So that's the kind of application, specific logic that runs on top of the cluster and so the kubernetes as a service model, is similar to enterprise in that users are directly interacting with the kubernetes api server.

A

But there's the big difference is that you have untrusted users running untrusted codes, so this has very stringent requirements for control playing isolation and for node and network isolation.

A

So here you would want to allow users to create and create read, update and delete non policy objects just like in the enterprise use case. In addition, you might want to allow users to create namespaces.

A

So, in the enterprise use case, we kind of are assuming the centralized IT function where some cluster admin is creating the namespaces based on the organizational hierarchy, whereas here it's kind of a it's a software-as-a-service with kubernetes as the application- and you might want users to be able to walk up to the system, create a namespace and then start running their applications in that namespace. So it's more of a self-service, namespace creation model than the more centralized enterprise a provisioning model, and then you might use resource quota here based on how much the user paid.

A

Maybe they can buy more memory or CPU and then, in terms of the code that's running in the cluster, as I mentioned, it's untrusted code. So instead of relying just on the regular container isolation mechanisms, you might want sandbox pods a concept that was discussed in a couple of the talks earlier today, but that basically boils down to running something like Kotik containers, orgy visor or you could use possibly sole tenant nodes and we'll talk a little bit later in this talk about how you can do sole tenant nodes.

A

But the idea there is that you would only allow a single tenant per node so that if there was some kind of container escape, they couldn't affect other tenants, because there wouldn't be other tenants running on that node and then network wise. You would have strong network isolation between namespaces belonging to different tenants and then the third type of application, I'm going to talk about, is kind of another software-as-a-service model. But here, instead of kubernetes being the service, you just have a regular application.

A

That's the service, and so the user does not access the API server directly. They go instead, just directly to the application, so there's actually two kind to sort of subtypes of this model. One of them is illustrated on this slide and that's where you have a true multi tenant application, whether it's just a single instance of the application, that's shared by all users, and this actually isn't very interesting from a kubernetes multi-tenancy perspective, because the tenant concept is purely internal to the application and opaque to kubernetes.

A

So the cluster in kubernetes api are kind of implementation, details that are hidden from the user, and so you can see that the cluster or application admin interacts with the API server, but the users, don't they just talk to that's supposed to be a icon representing a load balancer. So, like you know the load balancer in front of the application, and then those are all the pods that implement this multi tenant application.

A

So the more interesting kind of software-as-a-service model from a kubernetes multi-tenancy perspective, is where your hosting mold instances of a single tenant application and each user or consumer gets their own instance of the application, and so this is illustrated on the slide by having the users go through proxy in order to deploy an instance of the application. So the user asks the proxy.

A

Please give me an instance: the application I'd like to start using your service and then the proxy talks to the API server. It's the API and tells it to create a new namespace populate it with the objects corresponding to the application and return and and create a load balancer and return. The IP address of the load balancer back to the user and then the user from then on just interacts directly with the application through that load balancer and doesn't talk to the proxy or to the API server.

A

And then, when they're done using the application, they would tell the proxy to delete the application instance and it would do the reverse of what I described for the provisioning step and so I think I covered everything on that's like so, oh yeah, and so then the code in this software, as a service model, is semi trusted, insert, which is similar to the enterprise use case, because the service provider sort of knows what code they're running they. They should have some understanding of the application.

A

But sometimes in this software as a service model, the service provider will allow application plugins. So, for example, one popular hosted application is WordPress, WordPress has a plug-in model, and so, if you were to run a WordPress says and allow users to supply their own plugins, then that would be untrusted code. And so then you would need those stronger.

A

Our container isolation mechanisms, I talked about earlier, like sandbox, pods or sold tenant nodes and then under the assumption that each application is confined to a single namespace you'd want to allow pods to communicate within a namespace, but not with other namespaces. However, I've illustrated here also something that sometimes you would do in any of these examples, but I just chose to illustrate it for this example, which is that balloon a face which you'll notice looks like it looks different from all of the other namespaces.

A

So the green, yellow and red namespaces are identical to represent the fact that they're all running identical instances of the same application. But then the blue namespace is running some kind of shared infrastructure, that's accessed by all of the application instances, and so in that case, you'd want to allow communication between all of the namespaces and that shared infrastructure and, like I, said.

A

That is a pattern that you can see in any of these use cases where you might be confining the application and users to a single namespace, except for their ability to access some kind of shared infrastructure.

A

So, to recap, the the the first part of this talk the discussion about tenants and isolation. We talked about isolation, the control plane, who can access the API server, what they can do. I talked about resource consumption quota and then for container and network isolation. We talked about how you could put restrictions on what code executing in container in a container can do which pods can share nodes with one another. That was the sole tenancy idea and the idea that you might need to restrict communication between containers in different namespaces and with the outside world.

A

So the second part of the talk now that I've sort of discussed the isolation requirements of these three different use cases. The second part of the talk is about the mechanisms and kubernetes that allow you to set the kinds of policies and restrictions that we've been talking about, and this isn't intended to be a deep dive into multi-tenancy policies. It's a very deep topic, and there isn't enough time to cover all of that, but I just kind of want to give you an idea of what words to go. Look up for the documentation to.

A

If your use case could benefit from some of the kinds of policies that I'll be talking about, and to make it a little easier to reason about these policies, I've divided them into two categories off the related and scheduling related. It's actually a little bit hard to come up with good categorizations, but these kind of are roughly a reasonable way to divide them, and so I'll start with the author related policies.

A

So this slide shows the authentication, authorization and admission control flow and kubernetes and I've grayed out everything except authentication, which is the first piece that we'll talk about and for authentication. The request that's coming into the kubernetes api server can come from a human. That's like the picture, the smiley face at the top or from an application that's running in the cluster in a pod.

A

That's like the gear icon on the bottom, and the requests when it comes in will include some credentials, and the idea is that the Authenticator takes the credentials and maps them to a user or service account, and possibly a group or set of groups and kubernetes has a variety of pluggable authenticators, including a web hook Authenticator that allows to you to write your own custom Authenticator, and so then the information that's returned by the Authenticator is used as input for applying the rback authorization policies which specify which group users groups, and/or service accounts can do which operations on which API resources and in which namespaces so, for example, to tie this back to the use case we talked about earlier.

A

Our back is what lets you say that only that a regular user can only create read, update and delete objects in their own namespace and can't manipulate policy objects. So those kinds of rules from those earlier examples are the kinds of control, plane, isolation, rules that that are back and authorization lets you specify and kubernetes has other authorizers.

A

Besides are back including the option of writing your own as a web hook just like for authentication, but today our back is pretty universally used in kubernetes clusters, so that's kind of the the main one that that is interesting to learn about. So the authorizer decides whether the request is accepted or rejected, based on the our back policy, and if the request is accepted, then it goes through admission control which can reject the request or it can accept it, possibly modifying it along the way and then persisting it in etsy d.

A

So, to give you an example of a policy that's enforced in the admission control phase, we'll talk about Todd security policy and pod security policy is an example of like I, said a policy. That's that's implemented at the admission control level. It lets you control. What a pod and container is allowed to do on the node by controlling security, sensitive aspects of the pod spec. So, for example, pod security policy lets. You say that a container can't run as root or can't run as privileged can't used host paths. Things like that.

A

It's also where you can specify a parmer and set comm profiles, and so the illustration here is this china, showing that you have a pod spec that comes in to the admission controller. The pod security policy decides whether it's rejected or if it's accepted it might be mutated and then written to SCT. The reason it might be mutated is that the pod security policy also can do defaulting in addition to deciding whether the privileges that the pod is trying to use according to the spec are allowed or are not allowed.

A

So. Lastly, you'll recall, in our sample use case that we wanted to place restrictions on which pods can talk, to which other pods across namespaces and network policy is what lets you do, that it allows you to control the communication between pods, actually even finer grained than namespaces of lets.

A

You control them based on label selectors, and it also allows you to specify address ranges that pods are or are not allowed to send traffic to or receive traffic from, and network policy is a little special because it near acquires support at the at the network provider level and there's a bunch of different network implementations that can enforce network policy for you.

A

So now, we'll move on to the scheduling, related multi-tenancy policies, the kind of goal or high-level goal of scheduling is to share the compute storage and working resources of a cluster and of individual nodes among pods and containers from different tenants, while providing resource isolation between them and I mentioned networking in there.

A

I'm not actually gonna be talking about networking, because kubernetes is not yet support network as a scheduled resource, but in principle, network scheduling is part of scheduling, isolation between containers so and to help explain the scheduling, related, multi-tenancy features, I'll, show them divided into each one and divided into two layers: a policy specification layer and a policy enforcement layer.

A

So you'll recall, we talked about resource quota in our use cases. At the beginning of the talk, for example, in enterprise use case, we said you might allocate quotas per namespace based on organization's internal budgets or in the software, as a service may be based on how much the consumer paid so resource quota lets.

A

You set the maximum CPU memory and storage consumption across all pods in a namespace and also lets you limit the total number of objects of any type within a namespace, so it could be pods or volumes or or anything like that, and it's enforced by the resource quota admission controller.

A

The next scheduling related multi-tenancy features that I'll talk about our request and limit, and so resource request is the amount of resources that the scheduler is going to reserve for a pod on the node and the guarantee that the scheduler gives you is that it will only place the pot on a node where the amount of free resources is greater than or equal to the sum of the requests of the containers in the pod.

A

This is a little bit tricky, because requests are actually specified at the container level, not the pod level, but scheduling is done at the pod level, but it's actually not that complicated. The way that the request for the pod is calculated is just by adding up the requests for the individual containers within the pot, and actually the same thing is true for a limit which we'll talk about in a second and the main takeaway, for a request that you should keep in mind is that it's very important to specify resource requests.

A

If you want to avoid performance interference between containers, because the scheduler packed too many pods on a node. So this allows you to prevent the scheduler from packing too many pods on a node.

A

The second important resource scheduling concept is limit and limit is the maximum amount of resources the container can use and if a container exceeds its limit, it'll be throttled in the case of CPU or it'll be killed in the case of memory, and the thing that is interesting about limit is that you can set the limit greater than the request and when you, if you do this, it allows scenario where the total limit of the pods on the node exceeds the nodes capacity.

A

And when that happens, we say that the node is over committed, because the containers, an aggregate can use more resources than the scheduler is reserving for them on the node and an over committed node is perfectly fine until the sum of the usages tries to exceed the capacity of the node, because it's actually the usage, that's the what what is going to decide whether everybody, although containers, can coexist safely on the node, and so if the containers somebody say, tries to allocate more memory on the node, but there's no more memory available, then kubernetes will kill one of the pods on the node.

A

In order to allow the pod, that was allocating memory to succeed or it might choose to kill the pod that was trying to allocate the memory and I don't have time to go into details about how kubernetes chooses which pod to kill when it runs out of memory.

A

But there's a policy around that and for CPU it'll throttle containers if they try to use more than the node has CPU resources available, and the reason limit is relevant to multi-tenancy is that over-committing nodes is useful in a shared cluster to maximize resource utilization, because it allows containers to opportunistically use the free resources in the cluster.

A

So for the next two scheduling policies. Instead of talking in great detail about the underlying mechanisms, I'm going to talk about them in the use cases, I've kind of called this meta policy layer or meta policy, spec that isn't really a concept that exists in kubernetes today. Maybe it would in the future, but mainly I'm, just using that as a way to illustrate the use cases that are driving the specific features that I'm going to talk about on this slide.

A

So the first feature is taints and toleration x', and this allows you to mark specific nodes as dedicated to particular users or groups of users or dedicated to really any arbitrary set of pods, and this is useful, for example, if you want to reserve machines with special hardware for a particular team in your organization that paid for those machines, so you get the benefit of being able to have a single shared cluster where some of the nodes are dedicated and some are available to anyone, and you don't need the administrative overhead of having to manage two separate clusters, one that has these special nodes with a special hardware in a separate cluster with the the general shared nodes that are available to everyone, you can run them all in a single cluster, but dedicate arbitrary partitions of the nodes to arbitrary sets of users and the machines here can be physical machines or virtual machines and when I was using that term machines.

A

So the second feature that is kind of related to to this is starting to fall off detains and toleration. X' is anti affinity, and this allows a pod to say which other pods it cannot share a node with, and actually it can be any topology domain. It can be zone, it can be region, it can be anything but sharing.

A

A node is usually the most common use case for this, and so, for example, you can use anti affinity to enforce sole tenant nodes, which is saying that a pod should only be able to share the node with other pods from the same user and not with or over tenant and not with pods from other tenants or you can use it for what we call exclusive pods, which means a pod cannot share a node with any other pods at all, regardless of whether they're from the same tenant or a different tenant.

A

And so, as you can probably imagine, this is useful for security, sensitive pods. If you don't trust container the container isolation boundary, it allows you to ensure that a security, sensitive pod will never share a node with possibly untrusted code.

A

So all of the features that I've talked about so far have been in kubernetes for a while I wanted to talk about one new multi-tenancy feature. That's currently an alpha called priority and preemption, and the best way to explain this feature is by analogy and analogy. I'll make. Is that I talked earlier about how you can set limit greater than request in order to over commit a node?

A

Well, you can also over commit a cluster by handing out more quota than there are resources on the nodes in the cluster or just by not using quota at all, which is actually something a lot of people do, and when you get into that situation, when you're, when you're in that situation, where you're either have no quota or you, the some of the quotas that you've handed out are greater than the size of the cluster.

A

You can end up with a situation where there, our pods are trying to use more resources than there are total resources in the cluster, and that's the situation that priority and preemption is intended to to help with so I'll illustrate this with a with a quick run-through of how this feature works. So here we have a cluster. It consists of three nodes: the nodes are running two types of pods: the size of the pods is proportional to the amount of resources that it's using. The two types of pods are high-priority or important.

A

Pods those are in red, those might be say.

A

Your web servers or your storage backends things that are kind of critical to say serving the serving pipeline, the user-visible kind of a pipeline for serving requests, and then we'll have the green low priority or less important, pods and those might be say, batch jobs or experiments that your developers are running and so the way this works and I'll start with how it works today and then explain what party and preemption does so today, there's no way, you can really distinguish these green and red pods, but we'll see how that comes up in a second.

A

So you have a deployment, that's the red, plus it's hooked up to a horizontal pot. Autoscaler your cluster chugging along happily the but then the horizontal pot autoscale are notices that there's a load spike for your red-pawed service, and so it tells the deployment to scale up that creates a new pod and it goes into the scheduling, queue and this pod can't run because all the nodes are full.

A

And so the thing you would like to happen is that the green, some of the green pods, would be killed in order to make room for the red pods, because the green pods were the most important than the red pods. And so that's what priority and preemption lets you do.

A

You can set explicit priorities on the pods and say that the green pods are less important than the red pods, and so in this scenario, the system would kill the green pods to allow the red pod to schedule and then, if the green pods were being managed by a controller, they would be recreated and go back into the scheduling, queue and wait to run later. So this feature is an alpha right now and by the way you might be wondering well in a multi-tenant system.

A

Won't everyone just use the highest priority, and the answer is well. We've thought about that, and so the way that that we avoid that situation is that resource quota is subdivided by priority. And so, instead of just saying, instead of having quota just say, namespace X gets Y amount of RAM, for example, which is the way resource quota works. Today you can say namespace X gets Y amount of RAM at priority Z, and so that allows you to limit how much high-priority resources a namespace can use.

A

So we just have a couple of minutes and so to close I just want to give you a quick peek into some of the ideas that are under discussion in the area of kubernetes multi-tenancy, and I just want to add the caveat that these might or might not make it into kubernetes they're, just at the proposal phase right now, but I just wanted to talk about them briefly and I divide them into two categories. Policy related and non policy related. So we'll stop we'll start with the policy related ones.

A

So one recent policy related effort is scheduling. Policy and scheduling policy is like pod security policy, but for scheduling features it allows you to specify which scheduling features a pod is allowed to use. For example, the features we've been talking about here, like anti affinity, priority toleration, things like that and what values those features can take on. So, for example, what toleration Zapp odd is allowed to use.

A

The second policy-related feature that has been proposed is something called security profile, and the goal here is to improve the usability of kubernetes security and multi-tenancy features. So the observation that this is based on- and this is a feature that we've been working on at Google- is that today to operate a secure, multi, tenant cluster. You need a cluster admin who has a pretty deep understanding of multi-tenancy and security and how the kubernetes policies that are used to enforce multi-tenancy and security rules work. And even if you do have that understanding.

A

As a cluster admin policy, configuration is error-prone and you need to build up tooling to apply those policies reproducibly, and even if you build that, tooling and and and have the right configurations, the policy configurations need to be updated as new kubernetes features are added, and so the idea of security profiles is kind of to democratize security policy configuration.

A

The idea is to create- and this is a kind of wordy, but it kind of captures all of the aspects of it where to create a small menu of versioned community curated policy profiles to enable turnkey cluster creation with desired security and tenant isolation, and the Everywhere part means that this will work should work anywhere so on any of the cloud providers should work using cube atom any any of the deployment tools, and so the idea is that you would run a command like cube atom in it, and you would specify a particular security profile that corresponds to your use case.

A

So default might be kind of like a secure by default. Cluster you can imagine or saz multi-tenancy might create the policies that give you what I described earlier in the talk in that example, and likewise, you could upgrade the cluster and the policies would work across clusters and also you could upgrade the policies themselves.

A

The profiles themselves, I, don't have time to talk in detail about how policies our profiles are structured, but basically they map down to command line flags, cluster scope policy objects, name escapes and name space scope, policy objects, and we are hoping to include this concept in kubernetes conformance. So you can be sure that the security profiles work the same way wherever you run wherever you try to use them. So I don't have time to talk about OPA.

A

There were a couple of sessions on that and I don't have time to talk about the non policy work either. Unfortunately, but you can look at the slides later and and see see what's going on there so to wrap up. We talked about various multi-tenancy scenarios that kubernetes supports today and why inter-cluster multi-tenancy is important and some of the ongoing work both related to policies and unrelated to policies more related to scheduling and, if you're interested in getting involved. We would welcome your involvement. There's a bunch of groups. Sig off is the main sig.

A

That's working on multi-tenancy, at least the policy parts. The scheduling parts I should have listed up here now. I realize is six scheduling and then there are working groups from all the tenancy policies, container identity and that's the information for participating in those groups. So thanks we I, don't know, there's no talk after this, so I guess I can take questions, but if they're not gonna, let us take questions because you ran over I'm happy to take them off the stage afterwards. Thanks.