Cloud Native Computing Foundation KCD Korea 2021, 13 Sep 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: [KCD KOREA 2021] 쿠버네티스 인터널스: 코드 레벨에서 보는 쿠버네티스 이야기 | 한우형

Description

쿠버네티스는 클라우드 환경에서 가장 성공한 컨테이너 관리 시스템입니다. 쿠버네티스는 다양한 오픈소스를 제치고 어떻게 성공할 수 있었을까요? 다양한 이유가 있지만 쿠버네티스가 클라우드에 적합한 디자인을 가지고 있기 때문입니다. 각 컴포넌트는 약결합을 가지며 낙관적 동시성과 캐싱을 통해 성능을 극대화 하였습니다. 이 발표에서는 쿠버네티스가 어떤 좋은 디자인을 가지고 있는지 공유합니다. 특히 쿠버네티스의 핵심 구성요소인 kube-apiserver와 client-go를 코드 레벨로 분석해봅니다. 이를 통해 쿠버네티스의 디자인 원칙을 이해하고 클라우드 환경에서 성공할 수 있었던 이유를 알아봅니다.

A

A

A

Am an older brother who made a presentation on Cover Netis Internals., First, I will introduce the presentation.

A

This presentation contains code level, analysis of Cuba, Netis, api server, client, Go.

A

In particular,, the reason why Cuba Netis was able to survive in the cloud environment.

A

Was also analyzed for considerations and best answers when designing a cloud system.

A

In this presentation, We.

A

Will analyze the core components,, the api server and the client code, in order,, and tell you what each function is and how it was implemented at the code level.

A

At, the end,. We will talk about the overall design of the Cuba Netis system and finish.

A

First, the cube mpi server Cube api server is a server that provides Cuba, netis, http, api. It.

A

Is also the only component that directly communicates with storage,, I.e. et cd. In, this presentation,, the three core functions of the cube api server and how they are implemented. The.

A

First function of the tube api server is that it.

A

Provides a watch that monitors objects for changes. You.

A

Can manage many objects efficiently without having to poll objects for changes. Hard, as shown on the left. Pod information is returned along with 205 Kwai.

A

However,. If you give a watch, call 1 query, as shown on the right,, the system is blocked at that moment and you can continuously receive changes from the father.

A

In. This case, a card is added and Also,. You can know that you have received a modified object.. You can.

A

Efficiently manage many objects by continuously receiving changes. By doing this, instead of polling, the object. The.

A

Second, function of the tube api server is concurrency control., For.

A

Example,, if the user's lee immediately changes the environment variable to the image tag as the part circle, and the user's P changes, the image tag to v and the environment variable to another,, either yu's I or yu's b provided. It should be updated.

A

In other words,. The image tag is the part cause,, but the environment variable should not be mixed. Like car. If, the update occurs like this, one side succeeds,, but the left one fails. The.

A

Way to solve concurrency control in Huber Netis is to use optimistic concurrency, that is,, the source version of d is read before writing, and the resource version is incremented. After writing.- Let's.

A

Say that the part circle has been obtained. Next,, since user b is also writing at the same time,, the resource version is obtained.

A

Since. There is no change in the resource version. Yet,.

A

The part circle is obtained. is automatically added to v. Now, user b tries to update. However, this source version. That user b has is a part source,, but the actual resource version is v2,, so this source version is different and the update fails. The.

A

Third feature of the cube api server.

A

Is that it supports object, lossless identification and storage. Over time, as the business logic, changes, objects can have new fields, added, changed or deleted. For example. Let's.

A

Say we have an object that has a field named name as the member's alpha circle. Over time. This object.

A

Will have separate fields for first and last names instead of name.

A

Therefore,, we generally provide a conversion function that can map a v1 alpha circle and a v1 alpha pendulum to each other, as follows.

A

However,, as new versions are added, more and more problems, arise. As follows,. All versions must support each other's conversion. Function.

A

If there are n versions, on square conversion, is required.

A

To solve this problem, Cobaretis has developed a.

A

Version called the internal version or the hub, version. It is managed. Internally. The internal version is the version that all versions must go through when converting between each other. Therefore,. The change between the part circle alpha circle and the v1 alpha frame never occurs directly,, but instead, through internal version, conversion. You can convert to another version. In this case,, when each version is added,, each version needs to add a new conversion flavor between itself and the internal version.

A

This is the conversion function between Deployment, Internal and v1 in the actual Cuban Netis code. It.

A

Can be seen that the function exists as a ruin. The.

A

Important thing here is that all the conversion logic must go to the wireless.

A

A

You can specify eva,, whereas in the function that turns back into b1 alpha circle,, you.

A

Can join these two with a spacebar to perform the conversion.

A

But such a lossless transformation is a very painful operation.

A

Should there be any errors during the splitting process. When writing the firm version. Function,. You have to be very careful with handling these exceptions. Therefore,. The best practice is to keep the fields undeleted.

A

For example, delete mine from v1 alpha two. Instead of doing it,. It is to keep the name as it is. Po Van Edison provides a hold loss conversion using the internal, or hub version,.

A

So how to save this object when Coba, Netis, actually stores, it, etc. There, are.

A

Several options. There may be,, but basically in Cover Netis,. It is stored in et cd before stories, and you can specify the version.

A

Here, the install function is a function that registers a group of apps as an api. You.

A

Can see how each version is registered, here. Register the internal version and pay attention to the v1 beta one v1 beta second v1 version.

A

And in the last line,. The priority between the versions is specified..

A

Priority is corrected in this order. In other words,.

A

Even if a deflation event is created with the beta one I grant,, it is internally converted to v1 through the hub version and then stored in at cd.

A

When saving the object of the site cd,, the internal version was used in the past. In fact, inside Cuba Netis,. An internal client called internal client was.

A

Used. All codes,, including ,, have been deleted.

A

So far,. We have looked at the three functions of the tube api server.

A

Next,. We will look at how the cube api server implements these functions. As. You.

A

Receive this quest,, you will pass through the default build handler. Jane. This is a chain of handlers with various functions.

A

First, there is a timeout handler that sets a 504 timeout for requests that are too long, and a 401 that goes through. Authentication. There is a handler that returns a 403. In addition,. Several other handlers are registered in a chain..

A

These handlers are easy to think of as modules,, so it follows the modular design, well.

A

Through these chains Through these chains, This quest- goes through the request: conversion process. Here,. We go through the conversion process from the external version to the internal version that we mentioned.

A

A

Is a case where you want to perform conversion on the b source from outside. In this case,? If you register a yew hf,, why call the corresponding function in the form of a hook in the middle?

A

Auro schema validation is performed.. Here,, schema validation based on open API, In other words,.

A

Validation such as, for example,, is it made up of a string or does the name contain 12 characters? Validation is performed, here. There are cases where.

A

Logic is required. In, this case,, you can implement a validating function externally through Bellet, atf and register it in the api server. In this way,. You can insert external phenomena into the api server without soldiers through the web. Book.

A

Next,, this quest goes through an interface called storage.. The storage interface plays a role in breaking the dependency with the actual DB of Cuba Netis. In, the storage interface,. There are basic crud,, machi, and version language. The buzzer also corresponds to the storage interface,, and this interface updates the object.

A

In this case,, the mi source version is used internally, and the source version is used to implement optimistic, concurrency. In other words,. The storage itself provides only interfaces for basic angular and cr glass and 7,, and the.

A

Actual implementation for them. can be implemented separately for each db.

A

Finally,, the 2t cd commonly used in Cuban netis, is implemented by an implementation called 2t cd3. Through this implementation,. The api server, is completed through the actual 2t cd.

A

In this way, Coburn Edison We, are reducing the dependence on storage through the interface.

A

Then, this time,: let's look at the 2t cd3 storage, implementation.

A

Here, we will focus on watch.

A

First, when the watch function of the storage is called, and chuck 5 routines are created., It is.

A

Transferred to the storage channel that returns, the response, channel, and etc. d changes can be detected through this channel. The implementation of this tcd storage.

A

Also includes this source version control for optimistic concurrency. If. You.

A

Look at the code, if the revision of the entry state is the.

A

Same,, then, you are updating the put, that is, the object.

A

Up to this point,, we have learned about the implementation of gas functions and non-implementation methods on the api, server.

A

That is,. We will look at the server side. Then, this time,. We will look at the client side in reverse.

A

Client Go is the Go library for calling Kubernetes http API directly. Calling, the http api is very cumbersome.. You have to go through the authentication and authorization process,, and you have to figure out the request and response directly by looking at the documentation,. And then you have to process things like marshaling,, mother, shirring,, etc. You.

A

Can call the Kubernetes API as easily as a client. It is.

A

Used for all components, except for the client and the API server.

A

Except for the server, components are nothing more than implementing business logic using Client Go.

A

In this presentation,, we will learn about Client Go as follows. The.

A

First feature of is resource. Caching. Client Go provides a client-side cache. Unlike server-side cache,. It is efficient because it does not have to go all the way to the server.

A

In ,: this problem is solved through list and sieve. At first,. All the information of the resource is read into the list, and then it reacts to the local cache by detecting changes through Machi.

A

Also, all applications that share the same client will share the cache.

A

This is effective only in the manager's pattern of making multiple controllers into one application. In. This case, be sure to do a deep copy before modifying to avoid polluting the cache. The.

A

Second function of the client is the indexing of resources.. Indexing is to pre-assemble resources with specific conditions. For example,. You.

A

Can pre-separate resources according to the gender field. In, this case, resources that satisfy the condition without 2 scans You can easily get a list of . In client Go,, you implement indexing very risky, I.e. you can rent any build. You want with any value you want.

A

Declare, an indexer like cache point indexer and index it with a specific field and value like index point pi index This indexing function is often used for labels and selectors. The.

A

Third feature of the proxy is that the edge plus level drip handles events. The client target. Has an import talking object that.

A

Specifies a number of specific Cuban, Netis objects. It, is an object that calls the handler when the object changes when it occurs.

A

Since, it is difficult to deal directly with problems such as cache consistency,. You need to use info instead of using it directly.

A

Level Drip is because event processing is used at the same time. In the case of edges,. The handler is realized only when there is a change in the resource. On, the other hand, in Level Drip,. The handler is executed, periodically. If, you miss a change in ,. The handler may not be executed, forever. So in Cuba netis,.

A

Therefore, in Cuba letis info, blank paper and level drip are used. Simultaneously.

A

Levels, where the handler is called periodically even without a change of employee, are used as a hybrid.

A

So far, I have talked about the three functions of the client,, so let's find out how these functions are actually implemented.

A

First,, the list and 7 are performed for the api, server. The, simultaneous list of information feed is performed only once, and after that, it is performed and filled. If. You.

A

Imagine creating a daemon set in a cluster, daemon ceci full scan is performed on all nodes. It.

A

Goes into the shared index: in-firm. The worst index info watches a lot of resources, and if there is a change,, it plays the role of adding an item in the work queue.

A

Here,, you can see that the name is shared index info gimpo.

A

Is that 2 clients will share the same cache within the application that uses the 2 clients Next,? It provides an index function. Dimensions stored in the cache can be indexed via an indexer. Reflector is a list and 7 via one of the resources Detects and inserts the item into the delta queue.

A

Delta q reflects the change of item in the shard cache and informs work q that there is a.

A

Change. It informs only the key of the item. In general,, a string in the form of a namespace. Ceci name is passed. From the controller,, the business logic implemented by the user is contained. Here,.

A

The worker queue is watched through a reconciler, or coordination. Loop. Via Arch One of the controllers wakes up and, of course,. This process is thread- safe.

A

At this time,. Since many items' keys are stored in work q,, the object is retrieved from the actual cache through client, revelation. The.

A

Darker object should never be updated because it is darker., The, lesser known.

A

Is that no request is sent to the actual api server? Even if there is only a castle in the controller.

A

It is only reading the value in the shared cache.

A

Cache Since. The consistency of the data is entirely the responsibility of the information simulation,. The controller can trust and use the information in the cache.

A

So far,, we have looked at the functions and phenomena of the client, nose and api server.

A

Then, summarize what you can learn from these designs.

A

First of all,. It will be helpful to see the Cuban netis design, n, overview., The figure below shows the process of creating a pod.

A

First,. The pode specifications to be created are passed to the api server through q, ct. The.

A

Scheduler and Cube Ellison 2 are constantly watching the pod. Object.

A

There is a change in the Pod,, so the scheduler and Cube l25 2 can be woken up,, but the Cube Ellison 2 Pod knows that the host is still empty in the pod specification and will ignore the incoming entries.

A

Make sure the post is empty and assign the incoming entries to a specific host The.

A

Pod now has a host field.

A

Because the card has changed,, it will be awakened once again via the Cube Ellison March The reawakened Cube Ellison will have its own host's container Create and update the status field on the Pod Now,. The Pod has posts and status as follows.

A

Looking at the Pod creation process, like this, in fact,.

A

Two things are involved in the Pod creation: process. The first is the api server and the second Each business logic is actually implemented as a client, code. In other words,. The container management of Cuba Netis is a.

A

Set of business logic, fields built up based on the server and client.

A

Now,, learn through the Cuba Netis design. Let's try to sort out the points that can be done..

A

The first is the messaging delivery method using like.

A

This achieves a very popular weakly coupled system in the cloud, that is, direct synchronous calls between components. Do not happen.

A

So each object is just watching over the waves and 7, doing its own thing and moving on. This. Weakly coupled system has several advantages.

A

Second,, if the consumer fails,, the sender, is irrelevant,, so the failure is accounted for. Finally,, since the upstream and downstream services are separated,. Responsiveness is enhanced.

A

And Samsung's is very good because if you call machi, it will block,. But if you use goroutine,, you can call indefinitely.

A

Next is optimistic: concurrency. Optimistic concurrency implements city control without past. In particular, container management,, which is the purpose of Cubanetis. The change of the object on the line of the line is very small. Therefore,. This optimistic concurrency method is very efficient..

A

Optimistic concurrency is implemented using the cd transaction function, and it can be said that it is very compatible with the 2t series.

A

A

This is a huge difference for a production, cradle You.

A

Can see that it provides version, control and low-level security at the system level Next is that it provides client-side caching for scalability, List resources and later change, as if This can be said to be a good choice that goes well with optimistic, synchronicity and ultimate consistency.

A

Finally, blank papers manage events in a hybrid form of minute and.

A

Level dribbling. This is the.

A

Presentation I prepared for today. If, you look at the design of Cuba Netis, like this, I, think you can see why Coba Betis, succeeded. There.

A

Were various open sources that create * business logic like Coba, Netis. In, the latter half, I think that the reason why Tees was able to succeed the most was because it had such a great design that was suitable for the cloud. As I.

A

Finished my presentation, I would like to promote it for a while. A.

A

Book called Programming Cuba Netis,, which will include the contents of today's announcement,, will be translated and published. Soon. Thank you for your interest. Thank you.