Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2019 (San Diego), 22 Nov 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubeflow: Multi-Tenant, Self-Serve, Accelerated Platform for Practitioners- Kam Kasravi & Kunming Qu

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Kubeflow: Multi-Tenant, Self-Serve, Accelerated Platform for Practitioners - Kam Kasravi, Intel & Kunming Qu, Google

The kubeflow platform provides a self-serve multi-tenant platform on k8s for ML developers. Users can train their models using accelerated hardware in an isolated environment. Jobs can be configured and triggered from a notebook with no devops involvement. We leverage optimized libraries such as Intel® DAAL, Intel® MKL-DNN now included in tensorflow 1.14.+. Models can be monitored using Application CR deployed with kubeflow. All attendees can join the demo, create their own workspace and try out features. Attendees will walk away understanding how to run multi-tenancy on Kubernetes with kubeflow. Highlights: Self-serve multi-tenant workplace Workspace owners can share / revoke access System admin can reset access policy & resource quota per workspace Multi-tenancy service is transparent to other apps. A UI is available to simplify user experience.

https://sched.co/UaaI

A

Thanks for drawing our ox-3 today, we are going to introduce some new features of Google, including in multi-tenancy, self-served workspace and acceleration respects to smartly choose path to Rio close, so what we are going to walk through three. So today's feature is it's mostly around end users, so we are enable any others with a multi-tenant, self-served or workspaces or a team representative.

A

Imagine that you have a dynamically created and managed the workspace for each of your team members and we try to not set out in the universe created details. So cronin is: is it school with a town benefit, but apparently it has some disadvantages. I create a cat, so much like communities specific domain knowledge ease, which is kind of some habits, become a burden for end-users. We try to hide those yoga as much as possible.

A

Which one should I try to defy? I talk to the right, CP or GPU Oh No. Imagine you have a very large cutters. You have nose with different aspects. Some has like a memory optimized size commutation recognizes I have like a GPUs will help load ship to the load depends on the top ties you specified and finally, we have a like customized overlay, which is like a pre-configured.

B

A

For the for the Tom price and I say, I define a job template for this time, type of jobs for this type of template, I want like high GPU resources and that that family can be used to you. Simplifying for experience so briefly introduce like a cuckoo cuckoo is a as a platform for ML were closed and my products are coordinated. So everything in couple o is a great a native.

A

So our top priority is to resolve the interent problem from the time that we upload data to the work flow until all the way in that we served a moto API, our public or private endpoint.

C

A

Five minutes.

C

A

I'm benefiting about cuckoo, so the coop flow is a pure open source project. So all the technical details, all the demos we are going to do today, it's a available.

C

A

Yourself all night join the current using if you want, like we have like 200 of class members, actually have 20-plus organizations like joining the efforts.

A

Currently, it has like qualifiers, who called the cheeky NJ AWS and the ether, and it can also be deployed also existing community sponsors. So some briefly, each introduction about the available applications today so for experimentation and we have a Jupiter and a notebooks one is likely. You can spawn dynamic, no notebooks Ultima, 4 2004 artistry returning they are taking the board teleport.

A

C

C

A

You want to feel the tag for them or I all the way from they have labeling to training, to serving and the to monitoring the feedbacks and re-entering the loop I can build up a plans to appeal the I'm alois closed dead and we can marry dinner to record the melody how every steps of the pavement and for future store there are feast available.

A

Now, let's describe a bit about my dependencies for others. What was it like? A desired experience? We want, so all the others will operate on the same core natus classes, while users only have access to their own workspace by default, so they can only access the service. It's hosting hosted in your own work spaces and the work services are logically isolated by all means easier access. Is it's greatly across method? So take a closer look. Imagine we have two rulers while James and Sarah and imagine like see they are accessing services through the shared inquest.

A

German has its own box with, and Sarah has its own workspace and all the services works-based only available for four times, and all the services inside the service works with only accessible for Sara. It has all set up a workspace for a teammate. A team like this. This works with those kind of services are only available for the team.

A

That's the desired state. We want that cheap. We will construct Emily actually to illustrate how it works. So so some common challenges, I know like a crisis in Mali Tennessee has become like a hot topic and sound share. The common challenges include like say: first in person, we need to define the clear definition for the EULA workspace like what is the user workspace and how we want to build the access control around it.

A

It's a second in the challenges that the Cornett is empower network is transparent by that I mean if you create a core native service and you deploy them within the cursor so and any any part in the cluster can access this service by default and the sturdy the access control around the traffic through the sharing West. Imagine we have a acquaintances cancer early when we expose it to sera ingress or your the general practices.

A

We use one ingress for wine clusters so but it, but now we have like a dynamic number of users and they were shared listening where how we kind of do the access control around the sharing West. That's a interesting.

A

Problem to solve, and a nice nice thing to have to have is about a workspace access sharing, ReWalk, it's like imagine, I'm cooperating with my teammates I want to invite my teammates to access my my services. It would be really nice that if I can do it like a smooth and easy and finally, our policies, roles and parents involved or for all the power steps need to be like inconsistently. It should all operate with each other who have access to what resource or not.

A

So now it comes to our current current of choices. So so inside Google we have a new see our color profiles to manage the access, control and policies. So we define a user workspace as a namespace and the builder access control around it. The access control to nemesis is its control. Sir illuminates our air policy. It's like all the standard ways to control I believe is twice its employed for most project and the second we leverage is tool to control linklater traffic's, so so issue has a. Has.

A

An issue are back to control the encounter Traffic Manager. So it's you posit and assert. We leverage I think we epoxy, so this idea, web proxy is different across like culturally different cooperator have their own access identity. Proxies I could depend on your public qualifier or if you are working on ramp, you might need to set up your own identity proxy and the issue again. Is your policy on the bag to apply.

A

Okay, apply the airbag check against the existing, it's your horses, so we also enable the works with accessory and self-service or the crew flow. You is so we'll cover this tool in the demo later and finally, the cocoa profiles here, together with these controllers responsible for the consistency of different parties involved in the process. So one nice thing is: we also offer.

A

Buggy her face, which you can define your own custom, logics to control the resources within your organization that doesn't hosted inside the communities process by that I mean like imagine whenever something like our stories are cloud, the policies that we don't want to.

A

We cannot host inside the coronaries concert, but we want to update it per may. Add all know users, so you can implement your own logic to say, additives for, remove this or add a unit or tel aviv's or remove. Don't you have it like your status? We are going so why are examples later so a quick diagram explaining how it works. So imagine there are. There are two toys. You learn obsessed the services within cornetist fasters. So, while is definitely the most common, maybe so routers and another is so Kolkata.

A

They are going through different in point. So by that I mean like, for example, when, when you make a cut over passed, the real past actually goes direct to the kunais master proxy IP address and when you are making a request, routers that requested controller share the new West directory. They are actually different, different paths. If you so so repressed going through the powders, it was access was controlled by each. You are back while the request going through kubernetes proxy directly that one is controlled by the combination aback.

A

These two have similar names for their different scenes and they are like a controller at the same times through the profile, shares and users name. Is this: the user works with definition. Adultery is like a newspaper and all the requests if they are allowed code, the others own workspace and the services inside it so.

B

Yeah, so the key flow profile, as we stated in the beginning, was intentionally meant to be simple as part of uh self-serve paradigm, so the user is able to just specify the name which becomes the workspace name or the namespace name, and and then they just specify their name as the owner, and so this facilitates data scientists creating more than one type of workspace where they can run whatever they want training or inference, and so there are two ways to create a profile.

B

One is through the young, a file and invoking cube Caudill and the other is through the UI know. If you look at the left, you'll see that the profile is slightly slightly more complicated and that a plugins field was added. This is added as after the profile controller gets the profile, and it adds in this case the the user's identity, which maps to a GCP service account.

B

In this case, we have a pathway to allow GCP cloud resources to become available in the workspace transparently to the user by by adding to the profile, and obviously you wouldn't want to allow this plugin to happen by the cue flow user. You'd want it to happen within the backend side so that it can be so again, you have a choice of how you get started. How you create your workspace, you can either use a llamo or you can use the UI, and once the profile has been created, you have an option of inviting others.

B

You know, typically within a namespace, you probably want some peers with graphs, slightly less permissions or a slightly weaker or back model, to either observe or to partake in the work you're doing whether it's a training or inference, and so, as the owner of a profile, you can bring up the UI there's also equivalent you've got all commands and you choose a contributor.

B

That is your peer that you want to invite into your workspace and then, after you add them there, they show up within the UI as a a member- and this is not unlike a number of other UI- is where you can have in tribute as easily so switching gears slightly now that you have a workspace set up and you're a data scientists. The next question is: how do you get your training or your inference, job deployed and deployed to the correct node?

B

So this is the second feature we are integrating into cue flow and that is as training and inference to become more and more common, as Hardware chips become more and more varying with multi node distributed capacities. How do you best deploy your job to the right place and, and so we're doing we're enabling this by basically offloading this task from data scientists?

B

If the data scientist chooses to insert a string that indicates, for example, they want their a job executed on a GPU, we do not. We do not prevent that from happening. The idea here is that we provide some extensions that would allow vendors or DevOps to provide that sort of information, and so, as you saw two slides ago, queue profile had a plugin that accounted for allotting cloud provider resources to the user.

B

This section will also be added on the backend, not not exposed to the user. The obvious issue here is, if you were to allow the user to do it. It violates two things: one is now: we've exposed the complexity of kubernetes to the end-user and two they could effectively take over the cluster and a lot. You know all the all the GPUs or TP use out there.

B

So so the idea is that this would be inserted and it could be inserted by an admission controller or by Yadav's, and so this is an extension mechanism that we don't particularly have a particular say on, but we leave it open for the appropriate, tooling or the appropriate workflows so that, for example, you may have in the case of Rancher, for example, that pre allocates namespaces for users. You might have four or five namespaces one might specialize in earth.

B

The other might specialize in inception, and so you would want those namespaces optimized to be deployed on the right type of nodes, and this can be set up by DevOps. For example, you may hook it up to get up so that it ends up being a PR. If someone then has a particular request and then it could be approved and they're off in money.

B

So the idea is, you could either slice it up at a namespace level by model or you can slice it up by data scientists and what the what they're allocated we were agnostic as to how you would go about specifying that workflow. But this is a feature that we're going to push forward and hopefully get some input from from from the community.

B

So as part of the demo today, after coming shows you the the UI and the profile piece I thought we would run a tacked on pipeline and the pipeline is this going to run a model and the pipeline will specify actually for this demo? We don't have. It was originally in the original demo it's supposed to bind to a CPU of Cascade Lake, but we don't have that in our cluster today.

B

So it's we're not showing a CPU binding but, for example, in the CPU type, it's parameterised, so that you could specify a particular xeon class. So, for example, if you're running bert and you need- maybe you need because of the large emphasize a high-end eyes are on a processor like cascade lake or sky lake. You can make sure that when the job is deployed, it finds that pair a cascade Lake and a GPU or a sky lake and a GPU for example.

B

The other thing that this will show in turn through the overlays is that the fast patch and the past patch, if you see here, is asking for a Nvidia. So it wants to run on a GPU. And here the task is, as you can see, has no information about Nvidia in it, but via the overlay, the the video string under limits is inserted, and so what you have, then, is where these parameters are provided, as we do in other news cases in in queue flow via a config file.

B

So the config file has all the parameters you need in order for you to run that particular component successfully. In this case, you have, on the left, hand, side the accelerator and the count you want and, on the right hand, side, the CPU and the type of processor you want and I'll.

D

Hand it over to ya.

A

C

D

To know it's done with heart.

A

Os that started.

A

So today we are going to use to account. Why is Connecticut home this one has other main permissions for Boise can Carter. Another is over dark desire, has cast permissions on normal users or photo cancer, so.

C

A

To the couch, okay, non CI side, I'm logging, as the guys account, let's take a look at the control- you I was users. So here's a Koval you I have tracked apply cancers. If you deploy khufu any like a platform, follow the instructions in our document, you should do like the catalyst same exact same cursors, so.

A

Here it's all the current users and I think this. So as an amine I can see all the name series is available inside the car, sir I have a column in this owned by discount, have camps, names is owned by camp so and if I say, if I want to start a notebook, say it's a notebook.

A

In my own name, space I can connect with my own logo and just run some random, since anything can be wrong now, I, just print print out some random messages and no less switch identity and use a test account here for the same awesome. You are so the first time, for example, the first time a user login to the Coco's concerts.

A

If accosted it had this user down on any namespace, it work see also set up set up. Your own desires. Give it to me cause guests finish: the accident happens really fast. So after I set up on my own, it is my guest account should have access to the guy's name, please and on admin site, and they should notice that I knew it's been created, see here's a new name etudes. We just created owned by this Castle Combe and the caster can now, for example, create his own suggest one. My own notebook service.

A

So let's take a look at the CI, so.

A

By default, if I'm as a Gascon, if I want to fetch all the new materials, I will be denied because I don't have access to all the name species, but if I only want to cut resources, let's see within my own name sweets. It's doable yeah.

A

Yeah, so this is uh this: is a notebook I just created, it's been like initialized, so I can also check out other things like.

A

Let's see I try, let's see, I try to get a cat, a service counter from the mean name space I will be tonight because I cannot I, don't have permission to list the resources in the Denise Lindsay's. So now, forecaster I have my own notebook set up now. Let's, let's try the accessory, so imagine like admin decided like for some reason. I want to share my workspace with my guest account. Actually I wanna watch. My cats go to edit my notebook and then it's pretty easy. It's Connie place.

A

I, don't have the kind you know, so my name is sweet Andrew, so so like notice here. So if I requested it will be like Otto added all the UI, so I'm not going back to LA, Coste, so I know, I won't have access to the gas Nexus for the we were asking. Then I have access to the owner's name. It reads as well and I can access to the it will take a while. Let's see it should take up to 30 seconds to sink.

A

So, let's try now, let's try together the pod within the cast within the within the other. Miniatures game: okay, so now on see how I tied it's a thing up, I can have access to the resources in.

A

Admin names with not: let's try the UI side, okay,.

A

Okay, now I have access on both side of access to the admission images on post browser side and CSI. And finally, let's try out the the remove, remove access and I see I mean Josiah. Does the cooperation is done? I I want to revoke permissions from my gas account I, just click a disguise removed.

A

It should happen after. It should be think that within 10 seconds, but we can try now now the permission can be removed, I don't have access anymore and the progress I.

A

Early, it takes up to it's, not defined, still mean a windows. Let's see this is the cat's window.

C

A

Got out or also removed now, I only have access to my only which is now so about about demo is that about of the access Jerry and the revoke or the multi-tenancy experience. Apostasy is and browser controls. So next we are at a mobile access, accelerator plugins, so.

B

We're short on time on that I.

E

Think it's about time so I'm just wondering if I need question from aliens any questions. Thank.

F

You is there a way for normal contributors, to name it spaces between them, because I noticed that you had to access through the management.

F

A

Yes, let's, let's do it now, so this one is a cast counter. I'm logging in as Gascon now I want to add a. For example. I wanna invite her with me at home to my to my name. Trees I can do the same thing. It's just I can just type in the account here and and.

A

So me, encanta, now is invited. If you refresh from here, you can see the me account, is getter led to the guests name, sweets. So, for example, every every counter is the owner of his own images. By that I means the name is owner, can feel free to add and add and remove any as any at many user that they want.

B

Did you want the there's a queue color yeah.

A

If you want to use the cout hello, you can do it as well. Okay,.

E

Any other questions.

D

B

D

For the resources for a profile which is CPU and GPU, resources, which is short, I- think it's only possible with cube cutter right now and instead of wave we're, an admin could set defaults and change the default profile. Yes,.

A

Yes, so like as we our demo, commissioner, earlier we have, we define like a prospect or probe aspect for different kind of laws. For example I for the first time was his team to a lot of like mr training and that team do a lot of like a service like production service. So the training team might want a lot of GPUs and the 17. My want high memories, so we define different about different profile template for different for different profiles, so the training team can use the proper templates with the Hydra Buell's.

A

So all the all the pulse spawn in their pub in their work name, fleets will have a like that Jim peels are attached to the tulip all wild on a 17-0 get high memories. It's like a customizable, for example, of course, teams or cause a user. You can also type of workload one around so that.

B

That's an area that doesn't get to be specified by the user, it's added by their controller, otherwise it defeats the purpose. You know you could ask for 5 or 10 GPUs and block everyone out so and then it's up to you either like a get ops paradigm or a DevOps Jared I'm work. The flows would be inserting that and you can say well. This namespace is specified for a training for that requires cascade lake and TPU 180 yeah yeah.

A

We're running out of time to be for the demo, but you can try it out because it's all source. So if you want to try to find a hardware spec with event like a follower document to deploy the same concert and the trial of the child,.

E

Who can sing her? We are running out of time if we have any further question I think we can continue discussion across hallway and remember to submit your feedback is get downtown. San Fran joining the meeting.