Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2021, 14 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Resource Requests and Limits Under the Hood: The Journey of a Pod Spec - Kohei Ota & Kaslin Fields

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Resource Requests and Limits Under the Hood: The Journey of a Pod Spec - Kohei Ota, Hewlett Packard Enterprise & Kaslin Fields, Google

Let's learn how resource requests and limits work by illustrating what is really happening inside of your cluster - literally! CNCF Ambassadors Kohei Ota and Kaslin Fields will use fun hand-drawn artwork to walk you through understanding what your cluster is really doing when you set a resource restriction in Kubernetes. Engineers and their applications are going through a transformation. From running apps on a machine or VM directly, to running it in containers, and then to container orchestration via Kubernetes. During this journey, it is easy to become disconnected from what’s really happening on the underlying infrastructure. By learning what Kubernetes is doing under the hood, you can get a deep understanding for how Kubernetes manages compute resources. This talk will cover what happens, starting from setting the restriction on a Kubernetes Pod, then how these parameters will be delivered from the Pod spec to the Linux kernel layer through kubelet and CRI/OCI runtimes.

A

Hello and welcome to kubecon cloudnativecon eu 2021 virtual.

A

This is resource requests and limits under the hood, the journey of a pod spec, I'm caslin fields and I'm a developer advocate at google, I'm also a cloud native computing foundation ambassador and a member of the kubernetes special interest group for contributor experience.

A

I love to explain fun tech concepts with analogies and illustrations, so I do that on my website at caslin.rocks and you'll see a lot of that in today's presentation. You can always find me on twitter at caslin fields.

B

And I am kohei ota, I'm an architect at hewlett, packard enterprise, I'm also a cncf ambassador and in the communities community, I'm the owner of sick doc's, japanese translation. So if you ever speak japanese and wants to contribute to the quran's project, your per request are always welcome. I go by inductor on twitter and github.

A

So that's us, and let's get going today, we're going to use a fun analogy to help these concepts really stick and help. You remember them. So to start off our analogy for our talk today, I want you to think about your app being like a dog, and in this case your dog is going to be going into the doggy daycare.

A

I've used this analogy before as a way to help explain container primitives that will come into play here a little bit later, but the main idea I want you to get here is that your application, when you put it into kubernetes for kubernetes to run, is going to be running alongside lots of other applications being run in containers, and our goal here is to make sure that all of our applications have everything that they need to be able to run successfully.

A

However, apps don't always play nicely with each other, so resource requests and limits can help us make sure that they have everything that they need without getting in each other's way.

A

And since we're talking about kubernetes kubernetes is container orchestration, which means containers and applications at scale. So we're going to be talking about our doggy daycare and all of the many people and pieces that make up our doggy daycare and how how kubernetes takes care of our apps? Essentially so, let's dive into resource requests and limits.

A

First, here's a look at an ordinary pod spec you may have seen one of these before here is our app. Our dog is going to be within the image for our container that we're actually going to be running within our pod, and you can see below our container image. We have defined a resource request and a resource limit, so we're going to start off talking about the request and you can see here. We have a memory request defined of 64 megabytes and a cpu request of 250m there.

A

So let's look at what happens when we actually send this pod spec to kubernetes to run first thing I want you to think about is that requests are used by kubernetes for planning.

A

So in the doggy daycare analogy, this might be kind of like planning out how much space the doggy daycare is going to need in order to take care of your precious app. So let's go through the whole journey that that request goes through when it goes into kubernetes.

A

So you as the developer, you have your app that you love so very much and you know exactly what it needs and you want to make sure that the doggy daycare is going to do its very best job taking care of it.

A

So you want to put your app in a pod in the doggy daycare, so kubernetes says: okay, we can help uh take care of your app.

A

Our developer says that this app will need two cpu and four gigabytes of memory in order to run properly. Our doggydicker, of course, knows how to take care of apps and can make sure that that happens. So it makes a note of the app's needs to make sure that they'll be met once the application is actually running in kubernetes, and it does that by recording to etcd etcd is the key value store component used by kubernetes?

A

It functions as a single source of truth for the entire cluster, so any important key notes of how to take care of your app need to be reflected in cd.

A

So the next thing that needs to happen is we need to figure out which node there is infrastructure underlying this kubernetes cluster, so which actual compute node is this application going to run on? So that goes to the kubernetes scheduler the scheduler says. Oh, I see a new pod is coming in and that pod requires two cpu and four gigabytes of memory I'll take that into account and make sure that I put it on a node that can handle that.

A

So the scheduler is a kubernetes component. That evaluates nodes to assign a pod to resource request is one of the parameters that the scheduler takes into account when it's ranking which nodes could possibly hold this pod.

A

So now we've gone through the scheduler we have figured out which node we're going to place that pod on so now we need to talk to the cubelet, which is the kubernetes control plane component, which is running on the node and we kind of hand off our pod spec to that cubelet and say: hey, let go make this pod real, make it a thing, run the application.

A

So that's what our cubelet does.

A

So all in all, this is kind of what the journey of our request looked like.

A

A new request came into our api server on the control plane and that passed the the request off to the scheduler saying: hey, there's a new pod that needs to be assigned to a node, the scheduler figured out which node to assign that to, and then that got passed on to the cubelet which actually took care of running our pod.

A

Here's a little bit more information kohei is going to tell you about requests.

B

Thank you, caslin, okay. So let's talk about what cube scheduler does internally so cube scheduler.

A

B

Node resource usage- and it knows capacity based on the node metrics. If a node does not have enough resources, scheduler will not assign a part with the request value.

B

You can see the node resource information by executing cubicado describe node command. If you ever encounter a problem with the assignment.

B

So here's the summary of pod requests: request values are used when kubernetes creates a pod cube. Scheduler uses those parameters to evaluate nodes to match the pod requirement.

B

We will explain deeply later, but cpa request is also an important value to limit cpu resource in case the cpu usage is 100 percent.

B

If a pot uses more resources than the request, value and node resources are being used up. That means there is potential of pot eviction. Your pods can be removed from your node.

B

If you want to control the priority, you should understand about qos class in kubernetes.

B

The qos class is used when a pot is being created and evicted. Every pod has a qs class based on the request and limit values you set. There are three types: if you set the limit and request for all containers in a pod, then limit and request values are equal.

B

Kubernetes sets the qs class as guaranteed, which is the top priority. If you do not set any resource request on limit kubernetes labels, the pod as best effort, qos class automatically.

B

This is the least priority, so based on the priority kubernetes decides which pod should be removed. First, when a node resource is about to be exhausted,.

A

So now we're going to start talking more about limits. This is a look at our pod spec from earlier, and you can see that we're now talking about the limits section of our pod spec, where we have defined a limit on our memory and a limit on our cpu.

A

Main thing I want you to know about limits is that limits are for enforcing the rules. We want to say that this pod cannot go over this limit of of resources. So let's take a look at what happens when a limit comes into kubernetes the journey that it goes through to actually happen.

A

So, once again, we've got our developer with his precious app. Who says I want to make sure that my pod doesn't consume more than two cpu and four gigabytes of memory.

A

Kubernetes says: okay, we can limit your pods resource usage, so that gets passed off to the scheduler. Once again, it still needs to be assigned to a compute node. So this new pod needs to be limited to 2, cpu and 4 gigabytes.

A

The scheduler is going to make sure that the node knows about that limit so that it can enforce it.

A

So the scheduler has picked a node to run the pod on that's passed on to the node, and the node sees that there is a limit on the pod and the cubelet. The control plane component of kubernetes that's running on the node, actually needs help from another system component. In order to do this, so the cubelet talks to the container runtime.

A

That's actually going to run your containers for your pod, and it says I have a pod coming in that needs its resources limited our container runtime has the tools to actually limit uh the resources available to that container. So it says: okay, I can use c groups to make that happen, we'll dive into that a little bit more deeply here in a second.

A

So at a high level. I want you to remember that requests are for planning and limits are for enforcing. So now, let's dive a little bit deeper into limits and c groups. I was just talking about.

B

So, let's summarize, uh what resource limits does so resource limit is used to limit resources on a pod? So if a pot tries to use more resources than the value, you will encounter some issues.

B

If a container process tries to use over the limit of the assigned cpu, you will have the cpu throttling problem. This will not immediately delete the pod. However, the process gets stuck and the program has to wait until the cpu resource is ready.

B

If a container tries to use more memory than the limit, it simply gets out of memory error and the container will be killed by the ohm killer, kubernetes and the container technologies used. The linux kernel feature called c groups to implement the container resource limitation. We will explain more about this later.

B

Let's see how resource limit value is being used in kubernetes, once pod spec is registered to kubernetes cube scheduler fetches, the new pod specs then assign a node to the pod. You want to create keep scheduler as the node information in std, but the limit value is not directly used at this moment. Yet cubelet on each node runs the sync process to fetch.

B

The latest information of the pod that are assigned cubelet sees the limit value from podspec, then converts the cpu cores value into cfs period and quarter milliseconds then cablet calls container runtime interface to create actual containers on the linux host. Once container runtime gets the entry from cubelet. It executes the container creation by calling c groups.

B

Now, let's deep dive into the three parts here, cfs period and quota container, runtimes and cgroups.

B

When we talk about containers, it is actually implemented with linux. Kernel feature that is called c groups and namespaces namespace is used to isolated processed on linux. Host and cgrips is used to limit resources.

B

This time we talk about resource management on communities, so we are not going to talk about namespaces.

B

So, just like this cute diagram dogs, as in your pods, will try to fight over resources like toys and food. So we can solve that by allocating thousand food to each dog.

B

Let's see how kubernetes does in the technical aspect.

B

C groups is a linux kernel feature and you can limit resources such as cpu memory, network bandwidth or you can even combine those cpu request, value that you set in kubernetes will be converted to cpu shares in c groups. World cpu limit will be converted from cpu core numbers to cpu time value of cfs then stored in c groups as cfs period and quota.

B

You can also see those c groups value for your running pods on each node by seeing files under slash, sys, fs, slash, c groups. Directory memory limit is also stored in c groups.

B

One difference between cpu and memory here is that only cpu has the request value that is stored in c groups. So that also means that um memory it does not have request value in the c groups. I was really confused when I noticed this for the first time- and here is the reason cpu share is a relative value.

B

If there's four existing containers with one and a half half one and one cpu limit on each, they will share the whole allocatable cpu on the node. One cpu has enough space and there's no cpu limit. Cpu resource can be over committed, but when cpu is busy, the cpi request value will be used to keep the the application have enough. Cpu results for themselves if another application, with tcp request, is assigned to this node kubernetes recalculate this percentage and reassign all the allocated resources for when it's busy.

B

Now, let's learn about cfs cfs is a process scheduler in linux. It's the default linux process. Scheduler container isolation is based on c groups, but secrets actually uses cfs to implement cpu resource limitation.

B

Cfs has each cycle to process data and its default cycle period is every 100 milliseconds. That also means an application can be assigned to one cpu core, every 100, millisecond and cfs handles it.

B

If you set 500 m, which means a half court on kubernetes for cpu limits on your pod, spec cfs quota becomes 50 milliseconds, which means it can use 50 milliseconds out of 100 milliseconds of cpu time every period.

B

If you set 2000 m, which is 2 core, a cfs quota becomes 200 milliseconds, which means you can use two cycles for the process.

B

So what if you don't set any cpu limit? In that case, cfs quota becomes -1, which means unlimited.

B

And at last, let's look at container runtimes kubernetes has two types of container antennas and they are called cri analcia runtimes. You may remember the docker deprecation use. It was about sierra runtimes, but not ocean times when kubernetes tries to create a container cubelet executes, create container grpc call towards cr runtime.

B

When sierra runtime gets the create request, it executes ocr, runtime binary with the json file. That includes all the necessary information when creating a container such as c group limit parameters and so on. Then. Finally, ocr runtime is qc groups api inside now you have a new container running.

B

We are not going to any deeper about runtimes here, but you can think of sierra runtime as a runtime that runs at a kubernetes level and also runtime as a runtime that runs at linux kernel level.

B

If you are curious about container runtimes under the hood, I highly recommend checking out the maintainer trucks about cryo and container d, which are also cncf projects.

A

So, to close this out, I want to leave you with a tool to answer a question that you might have been asking yourself, which is so now. I know what requests and limits are, and I know why they're important and how they work. But how do I set the right requests and limits for my applications?

A

For this? I want to introduce you to pod, auto scaling. So kubernetes has two types of auto scalers available. The first is horizontal plot, auto scaling, which means creating more or fewer replicas of your pods to handle traffic. So if you have a big spike in traffic, kubernetes can use a horizontal pod, auto scaler, to spin up more uh replicas of your pod to handle that traffic.

A

If you're kind of in a lull of traffic- and you don't need so many pods- it can also scale down so that you don't have as many pods running uh and then there's also this vertical pod autoscaler, which is about changing the size of your pods.

A

When I first saw this, I was a little bit confused, because what does it really mean to change the size of a pod and in kubernetes? What that means is giving you recommendations for how to set your requests and limits for that pod.

A

So there are three modes for a vertical pad: auto scaler in kubernetes, and those modes are off initial and auto in off mode. The vertical pod, auto scaler, will monitor what's happening with your pod and how many resources it's using and it'll make recommendations for how you should set your requests and limits for cpu in memory, but it won't implement those for you. You would have to go in and check what those recommendations are and then implement them yourself.

A

Initial mode means that the vertical pod autoscaler will kind of watch. What's going on with your pods for a while and get uh some some recommendations on what you should set those requests and limits to and then it'll implement those for you once on those pods.

A

But then, any time after that, if you ever want to change the requests and limits of your pod, if it needs to change size, then you'll have to go in and check what the vertical pod autoscaler is recommending and then implement. Those manually then auto mode.

A

The vertical pod, auto scaler, will continuously monitor your pods and try to understand what they're doing make its recommendations and then implement those recommendations for you, which means that it will be continuously creating and deleting your pods when it makes a new recommendation, it will spin down the existing pods and spin up new pods of the new size. So that can be a bit disruptive and you have to make sure that you're managing that uh disruption, pod disruption, budgets and the like, um but that is also an option for vertical pod, auto scalers.

A

So the way vertical pot auto scaler recommendations work is they have a few different uh types of recommendations. They give target lower bound upper bound and uncapped target.

A

So the target is the value that the vertical pot auto scaler will use when resizing the pod. The lower bound is the lower bound number that vertical part, auto scaler looks at for triggering a resize. If your pod utilization goes below that, then vertical pod autoscaler will delete the pod and scale it down.

A

Your upper bound is the number that vertical pod autoscaler looks at for triggering a resize in the upward direction. If your pod utilization goes above that, then vpa will delete the pod and scale it up if you're in auto mode. Of course, this is saying and then uncapped target if no minimum or maximum capacity is assigned to the vertical pod autoscaler, then the uncapped target will be the target utilization for the vertical pod, auto scaler.

A

So let's review.

B

Congratulations: you've just been through the container request and limit bootcamp on kubernetes.

B

So let's see what we learned today, so pod spec is registered once you apply your pot manifest into kubernetes through cube api server then give scheduler fetches newly registered pods from scd and assign a node to each spot, referring to resource requests and cubelet fetches assigned pod in every sync period and calculate this between running containers and pod spec, then cubelet calls create container towards cr runtime.

B

After converting cpu cores into periods, then cr runtime executes ocr runtime binary to create a container with oc. I spec json oci runtime, manages c groups file system, so it can create and delete azure containers then. Finally, uh we also learned about pod, auto scaling and especially vertical part of auto scaler is not simply a resource that you can auto scale your applications, but also provide recommendations for your applications, resource requests and limits.

A

So thanks for joining us today to learn about pod requests and limits. I hope you learned a lot of fun new stuff. If you have any questions for us, please post them in the q a hopefully you've been doing that this whole time, and if you have any questions after this, do feel free to reach out to me on twitter, at caslin fields and.

B

Inductor again, thank you for joining us today.

A

Thank you so much.

B

Have a great day.