Cloud Native Computing Foundation KubeCon + CloudNativeCon China 2018 (Shanghai), 21 Apr 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes Love Machine Learning, Even on Private Cloud - Hui Luo, VMware

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Kubernetes Love Machine Learning, Even on Private Cloud - Hui Luo, VMware

Kubernetes has established as a good platform for machine learning workloads by extending support of accelerators like GPU, all major public cloud provider are offering GPU enabled Kubernetes services, but public cloud is not the only option for users. There are ongoing efforts from the community to make running machine learning workloads with Kubernetes on private cloud as easy as on public cloud. This talk is going to cover 3 major challenges that facing private cloud when enable GPU on Kubernetes. I will also demonstrate and discuss some of the projects that help to solve those challenges: 1) Private cloud usually needs to support a wider range of GPU types, in some case, to support heterogeneous GPU in one cluster 2) To support complex hardware topology like RDMA, NVLINK 3) GPU resource contention is usually very high when limited GPU resource shared by multiple teams

For more info click here: https://sched.co/FuLx

A

If you think we are in the era of machine learning, I think kubernetes is the right tool to democratize machine learning. Everyone should able to run machine learning everywhere and on-premises is a critical part of that wishing and that's my topic today.

A

Literally, a little bit about myself and software engineer, winger actor contributor to upstream kubernetes in area like device plugin, also an active contributor to Cal provider, vSphere and cluster vSphere that we were. We are doing two things around machine learning, the first we build intelligence into data center operations into products a second. We help customer to build infrastructure to run machine learning, including kubernetes.

A

So, first, let's take a look at the machine learning on kubernetes landscape. Obviously it's pretty big and a lot of development happening on coop flow, but underneath there's a common set of resources and GPU is a critical one. I believe it's a fundamental for machine learning. Workloads I'm gonna discuss some of their works from the kubernetes community to help you to manage this review resource more efficient and more manageable. In many respects,.

A

Here are the major aspects of the GPU resources on kubernetes and I. Do want to point out the first one, the life month, lifecycle management is have proved to be more challenging on premises because it has to support a variety of hardware's. There are others that are applied to both public cloud and on-premises.

A

Let's go through each one. Here's the Yama file, we all familiar in love. This is the part that's requesting one GPU by running coop, CTL crowd, you get a part deploy to the GPU caster and your container will get a GPU assigned when there's the GPU available in the cluster. It's very simple.

A

But before this magic happen, you do have to take care of all the yellow components in this picture. It's including from hardware to driver installation, container, runtime device plug in and even auto scaling you. Basically, you have two choice today: either you build automation by yourself, if you're a good engineer or you can look for the vendor solutions and I believe this is the area that vendor bring value through automation, to help you to manage all the components.

A

Let's take a quick look: there GPU resource sharing and isolation. You can use namespace and GPU coder to isolate the GPU resource for different users and also you can use the power priority cross and also curious to manage the mix set of workload. But you do have to note that it's not like CPU in kubernetes. You cannot request a fraction of GPU in kubernetes. That means the two part can now turn sharing a GPU at a time. This is the limitation today.

A

Gpu resource monitoring you get calls the core matrix like memory used duty cycles through C or wider today, and also the UI's compute devices our assignment proposal. Hopefully in the near future, we can get a much richer set of metrics for this GPU device.

A

Maybe you are comfortable with managing one type of GPU today for done for the k80, but inevitably you will start to deal with multiple type of Jib you in your custard, and you don't want to treat them the same because they have very different price points and capabilities. So we have the resource API proposal that can classify the GPU devices into different castes like high energy pu.

A

I call them go to GPU, so your part can say I need a high energy view, even without that you can use neighbor to label your notes differently, so you can manage a different type of GPU performance consistency. This area.

A

When we talk about performance consistency, consistency is really key because it's give you the per ticket predict, make you make make the performance pretty particular ball. You can use CPU manager and a huge page today to achieve a certain degree of performance consistency, but you further adjust their nominees and the device locality.

A

This apropos the new manager proposal in the community that going to adjust that, but on the hypervisor environments we can actually use properly sides that each node within the neumann boundary and also use the neighbor to achieve this similar degree of performance.

A

With that, I'm going to conclude my talk and you can, following the active discussion, seen the following working group and the six every week, and also you can find me on the github. Thank you.