Cloud Native Computing Foundation KubeDay Japan 2022, 9 Jan 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Best Practices for Applications on Kubernetes for Both Frequent Updates & Stabil... Hiroshi Hayakawa

Description

Don't miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe in Amsterdam, The Netherlands from 18 - 21 April, 2023. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Best Practices for Applications on Kubernetes for Both Frequent Updates and Stability - Hiroshi Hayakawa, Yahoo Japan Corporation

In a modern architecture represented by microservices, applications are frequently updated and restarted. In order to keep the service running even during application updates, it is not enough to simply deploy to Kubernetes; a variety of efforts are required. In this session, He will introduce various practices to achieve both frequent updates and service stability at the same time for applications running on Kubernetes. Specifically, he will introduce practices such as container health checks, CPU and memory allocation settings, and how to write application implementations and manifests to achieve graceful shutdown. The presentation will go beyond just showing how to configure these settings, and will explain the behavior of resource and container management performed by Kubernetes, from which you can determine the best settings for your individual application. In the second half of the presentation, he will also introduce some unique practices used at Yahoo Japan to run a large number of Java application contianers. Tuning of containerized Java VM and automatic warm-up techniques will be covered.

A

A

uh I'm, looking for Fujitsu and uh responsible for developing Linux and open source software today, I take charge of chair of this beginner session.

A

Each speaker is given 30 minutes, 20 minutes for talk and 10 minutes for Q and A and uh I have a favor for ask. Please put put on your face mask during session. Okay.

A

So, uh okay, let's start our first speaker is ah Hiroshi hayakawa uh Yahoo, Japan Corporation. He is leader, engineer of kubernetes-based application platform in Yahoo, Japan, private Cloud division, and today's title is best practices for applications on kubernetes for both frequent update and stability.

A

Okay, here we go.

B

So good morning, everybody, my name, is platformed developer for Yahoo Japan me and my team developer March tenant platform using kubernetes, focusing on running web applications.

B

So we have been running and operating many web applications on such a platform, and today, I would like to share our findings with you, and first I will try to give you a little more background on this presentation.

B

From my perspective, as a platform developer, I think the most important characteristic of modern application is that they are frequently restarted.

B

In fact, there are many reasons to restart applications. For example, they may be updated to add new features, or they may be scared automatically or manually what they may be restarted for maintenance of infrastructure on which the application is running such as kubernetes.

B

So we need to view restarts of application as something that can happen at any time. Therefore, modern application must be able to work reliably under such frequent restarts.

B

And in addition to this fundamental requirements, we also want to achieve more. For example, we want to achieve a specific level of stability defined as a types level, or we want to achieve this in a larger scale. Application and I think achieving these things will help us all sleep well at night and get healthy quality of life lives.

B

So what specific efforts should be made to achieve it to consider this? Let's take a look at an overview of the life cycle of pots on kubernetes.

B

In this presentation, I will mainly consider Java applications deployed on kubernetes with deployment resource, but anyway the lifestyle group, such an application, roughly looks like this.

B

On the first deployment, several reports are newly launched later when we update the development resource supports, are replaced with new ones by Rolling updates, and at this time the new version of support is newly launched and the old ones are terminated.

B

And learning ports may stop and expect unexpectedly, for some reason and the new one will be deployed and you may want to add more ports. If the load to support growth, then again the ports are updated to the new version and it causes supports to start and stop.

B

As described above when an application is running, many things happen during its life.

B

How can we obtain application stability in the face of these policy events, I think we can simplifies the problem and think about it from three perspectives.

B

The first is when the Newport is starts, make sure the pot is well prepared to handle requests without errors right from the startup.

B

The second is when a pot is running, especially when the application load is high, unexpected Port outages tend to happen. It is necessary to prevent this.

B

The third is when a port is terminating to shut down a port without errors, you need to be aware of kubernetes support termination process and shut it down, gracefully.

B

In the life of an application, many things can happen, but by considering the three perspectives to enhance the stability of each board, it is possible to stabilize the applications as a whole.

B

So I will delve into each of the three perspectives in turn. First, let's talk about preparation in launch of launcher pots.

B

One seemed to be especially careful is that what support status become ready, requests from end users will come into support when a port is added to add it to already running pot grip, go to Rolling updates or horizontal scaling.

B

Once the boat is ready, the production volume of traffic will be sent from that point on, so the container containers in the port must be able to handle such volume of requests before the status of the boat becomes ready.

B

So what kind of preparation is required in a particular application.

B

Unfortunately, it is necessary to consider for each of your applications.

B

For example, if the application depends on a database prepare a connection pool for the database for Java applications, you may want to reserve an initial Heap memory or perform warm up to ensure the best performance of it.

B

So how do you prepare for kubernetes spot in an application to know that we need to understand what happens in a boat when it is launched?

B

The first step is to learn init containers. They are executed sequentially if multiple its containers exist. After that the main container is started and the post start lifecycle Hook is executed. At the same time when the posts start with finishes things three kind of pro three kind of proofs starts.

B

Afterwards, redness proof succeeds, the pot is determined to be ready and register sent to it. Therefore, we can use init containers and post start Slicer group for the preparation of containers.

B

Here is an important difference between init containers and post-start. Livestock works in its container is a process that is executed in the separate container before the main container is started.

B

On the other hand, the post start work is executed in the main container as soon as the main container is started, for example, for warm up of Java application, which needs to be performed by the main application is running post start. Work should be used.

B

This is an example of using posts such like hook to warm up a Java application. The post-start process is described in the lifecycle field on the right when written. In this way the post start work is executed in the main container. Here it slips for 10 seconds to wait for the application to finish starting up and then send requests to it, lovely speaking in Java applications.

B

The same code of the application same line is compiled from byte code to Native machine code after about 10 000 executions, so it sends requests ten thousand times since Readiness probe is always executed after the post start hook. When the pot is ready, the performance is high enough.

B

Next, we will discuss how to prevent unexpected pot shutdowns when puts are running.

B

If our put stops, kubernetes old hearing will start a new Port, but simply letting it to do so will not be stable enough, because if the container cannot be terminated, gracefully the in-flight requests will result in errors.

B

Also, if a port is stopped and high load, the remaining Port will have to endure the load before Newport starts when a port stops and a heavy Road, it is not expected that the remaining Port will be able to withstand the load, so our support will also stop one after another.

B

Therefore, we must avoid the situation where both stops working as much as possible.

B

This diagram shows the summary of class patterns. I have often encountered in my experience, running Java applications, excluding application bugs or misconfigurations. In most cases, High loads are beginning.

B

The high road triggers are CPU, one memory shortage in the port or a bottleneck outside the pot become apparent.

B

If memory consumption is too high, the kubernetes will perform oom kill. Also, if liveness blue fails due to insufficient CPU or hurt garbage collection associated with insufficient memory. This will oh this is. This will also bring the pot to hot if liveness proof is not well configured. Other next outside support may also cause a pot to stop.

B

It is necessary to make efforts to stop such crash scenarios.

B

To prevent CPU and memory shortage, it is necessary to perform load testing and tuning well.

B

Before before we get into the details, here is a quick review of kubernetes resource control, CPU and memory allocation for both can be set in resource requests and limits.

B

The value is setting resource request is the amount of resources that will be guaranteed to The Container can use and resource limits is a maximum amount of resources that can be used if the container used to exceeds the requests.

B

If a container's resource usage is about to exceed.

B

Resource limits kubernetes will control it to prevent it from exceeding them. In practice, Linux C grips control is used if the CPU is about to exceed the limits throttling occurs and the container is kept around the CPU usage settings are limits.

B

If memory uses sees the limits, the port is killed by omkir. At this time, signal handling is impossible on the application side, and the in in brightly case will result in errors.

B

In the case of Java application, there are many types of memory areas and you need to make them well tuned to keep the total memory usage within resource limits.

B

Unfortunately, the optimal value bodies from applications to applications and must be found by enough Road testing.

B

However, at the starting point, I recommend raising the Heap memory around to about 50 percent of the resource limits by default. The jvm sets Heap memory to about 20 to 30 percent of limits, so the amount of bleeding in the resource request and limits is often left over.

B

This may not allow the application to perform as expected and waste or gated memory, so it is recommended to pull it up.

B

The plug Max lamp percentage can be used to set the memory as a percentage of the resource limits.

B

Next, we will discuss how to prevent the port from stopping triggered by food. Garbage collection.

B

First I'd like to talk about the fact that constellations about garbage collection have changed since we learn Java applications in containers in containers. Resources are often allocated in a relatively small amount, so the crcc is often used since crtc takes a long time for full garbage collection.

B

It is easy for the liveness group to fail and cubelete to terminate support.

B

So if the third CC stop time makes your application unstable, please consider algorithms that emphasize response time, such as G1 GC, also another tip. Please note that, even if you just just increase the amount of hip memory, do not skip the test.

B

Increasing the amount of memory will increase the time it takes for full coverage collection and, if making a put more unstable than with less memory, has business mode.

B

Next, let's consider support stoppage that occurs when liveness will be sprung to failure. When you start the brightness group, what settings do you use?

B

This is a quote from the kubernetes documentation, which mentions that liveness proved detects that an application is running but unable to make progress, and it gives example of a Deadlock but I think it isn't easy to come up with other examples.

B

On the other hand, I think the liveness proof setting should be carefully considered, especially for application facing unpredictable laws.

B

If the application is overloaded, it may not even respond to the likeness group and if so, cubelet will decide that the restart is necessary and thumbnailing support, but in a situation where a large amount of traffic has to be processed, it is still better to keep support running than to restart it.

B

Therefore, it is better to live it as a default than configurable HTTP liveness group without much salt by default. As long as application process runs, liveness group is considered successful.

B

Finally, this is a simple tip, but it is also important not to make liveness groups dependent on components outside support.

B

For example, in the lightness group you check the connectivity of a database that application depends on such a configuration will cause support to restart when the database is unstable. Also it will to live. It will.

C

B

To improve the state of the database in these cases it is better that the port stays running and it gets to support result in an error circuit breakers Technique we may be, may also be effective. So please consider.

B

So, finally, we will discuss how to safely stop ports without causing user traffic errors.

B

Even before running applications in kubernetes, it was necessary to terminate applications gracefully. However, with kubernetes there are few more considerations in shutting down the port on kubernetes.

B

Two things happen: one is that the cube rate Terminator board, the other is that the cube proxy stop traffic to the pot and the port needs to be terminated after the traffic was stopped.

B

This diagram shows what happens when a port templates.

B

First, the cube proxy starts to stop traffic and, in parallel, the pre-stop rice cycle is executed. Based off. Work is similar to The, Post start work and it's performed in the main container at Exit time.

B

When this light sacrifice completes, it cubelet sends a sick time signal to the application and the application enters the termination process, there's no particular order in which the traffic stops and the port shuts down. So the application must be terminated before the traffic stop.

B

So use the Play Store PlayStation work to wait until traffic stops.

B

The Press stop work process cannot detect the stoppage, so I use a sleep command and wait for certain period when the application receives a sick term signal, it should 100 and exit gracefully.

B

This ensures that the application terminates after returning response to implied this in-flight requests.

B

This slide says same thing as a previous one, so I skip.

B

Here is an example example of an actual manifest in this example. The pre-stop life cycle is used to wait for 10 seconds foreign.

B

B

In many cases, sdks or programming, language or framework supports this, so you can use it. This is an example of spring boot. It is sufficient to write such a configuration in application properties.

B

I have shown you how to stabilize your ports from three perspectives: yeah. Of course, what I have described have is only what my team and I have experienced and I am sure there is room for you to start more I hope this prep I, hope this presentation helps you in your exploration and finally, I wish you all good night sleeps.

B

A

Thank you very much very much. Is there any questions.

D

Okay, thank you for your good presentations. I have one question so you mentioned so om killer. Yes on your presentation. So so I have a question: did you use a she group River? So when you got or makira right, yeah? Yes, have you ever been tried to use a she group B2, so I asked her because she group Brito has considered as sorry C group B2 consider so resource request of memory. So if you have a result to use C group on your application or something so I want to know, oh.

B

Sorry I in my work, I use only secrets to be one so I not experienced such a good functionalities of secret security. Okay,.

D

Thanks thanks, I am looking forward to seeing your result in the future.

B

B

C

Thank you for your presentation. uh I was wondering uh how how did you figure out that the optimal memory settings for I I, assume use uh springboot also and uh like, uh for example, the initial REM percentage and Max percent is like how do.

E

C

Figure out the optimal settings, and also like, uh should those be equal like this seems to be some debate on this also yeah.

B

I think the default settings of hip memory to 20 percent to 30 percent is a.

B

Because the first made JPM when there's no containers in the world- and it is a so.

B

It's not enough to run job applications in containers.

B

You know, containers are Oliva most cases, only one processing containers and um why I choose 50 percent.

B

The reason is I what Cloud native build box for Java applications and that product has a memory, calculator of java applications and this we, this memory, calculator, release, upper hip memory and users can specify a custom tunics.

B

So I think that this is the best I thought. This is the best to recommend fast start point for 50 percent, respecting made it back memory calculate it makes.

C

Sense yeah, thank you. So actually, we had recently uh uh optimization uh regarding this uh yeah uh also some failures, so we tried also a bit to uh uh high setting uh like a 80, 80 or something and uh then and ended up with.

B

C

Issues and currently uh uh we are using a 70.

B

C

uh So far, no no issues, maybe it's a bit too high, I guess 50- is safe, safe.

B

To start yeah, some some applications in yeah for Japan uses a I.

E

Think 80 percent.

B

Right some applications usage pattern settings so I think it is reasonable.

B

Oh thank you. Thank you very much.

E

Hi, a really great presentation, um I was curious. It seems like a lot of the configuration um can be set according to like how you do your load, testing, yeah, um so I'm, just kind of curious I know something that our company is struggling with, is figuring out the most optimal way to do. Load testing I just want to see if there's any uh tips or guidance that you could give for that.

B

B

In my company we also struggling about finding the.

C

B

A good Solutions, but but now.

B

We, our team, makes.

E

B

Testing or tuning activities with Subs application developers, sometimes yeah I I, have no idea. Another good idea. That's.

E

Okay, It's Tricky.

B

Yeah yeah, thank you.