Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2018 (Seattle), 16 Dec 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: How to Build Deep Learning Inference Through Knative Server... - Huamin Chen & Yehuda Sadeh-Weinraub

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

How to Build Deep Learning Inference Through Knative Serverless Framework - Huamin Chen & Yehuda Sadeh-Weinraub, Red Hat

Knative, a Kubernetes native serverless frame, was just made public. Although Knative is powerful and flexible, very little is known about how to integrate, use, and extend it. In this talk, we present a deep learning inference demo to walk through Knative end-to-end. Specially, we explain the following concepts and process: - How to extend Knative Eventing by showing a new Knative Eventing Source: Ceph Rados Gateway (RGW) PubSub. - Knative Serving concepts, such as route and configuration. We illustrate these concepts by providing a Knative route that is able to invoked by RGW PubSub event and trigger a deep learning inference task. - Knative build concepts, such as build and buildtemplate. We substantiate these concepts by a real-world source to image pipeline that builds the deep learning task.

To learn more: https://sched.co/GraI

A

Just brief introduction, so my name is Pam Incheon I work for dwell has city office. This talk was done by me and my coworker Yehuda, who was all workers, whereas with your office Yehuda, has some business to do today. So he cannot make the talk. So if you have any personal greetings, I can definitely pass along alright almost time.

A

So if you don't mind with your hands any of you who are about connectives already guess those trainings tutorials the past few days get your hands dirty with connectives right. So in case your hands are still cream. The idea is Caves, get your hands dirty today. If your hands already dirty, they will be dirtier when you leave okay, so we're not further ado. Let's get started, and so he native bathwater has been around for the conference for quite a while.

A

It's a journey for those of you for already participated in airport projects, so we have been working on the projects for almost half a year to me personally and a longer time from somebody else. I still commuters here, maintenance. Here, that's a good thing. It has been a journey. A lot of rap and dance buzzwords, announcements and applications all good things in my opinion, that's a very good finish these lines and lots of community involvement. We have weekly meetings on each of the projects and how to activities groups and the documentation working groups.

A

It has been a very exciting moment for me. It reminds me of kubernetes why it was between before the points 101 control release of your years ago. Just a few years ago, when it was Terrence is not, there were not even Cuba. It was announced as the office term open source' slam is in Portland the size of the tendency. Just like this. This reminded me a few years ago, when we were there is the community has been growing so far so explain. Actually this is a thousand attendees for conference.

A

So, if you are look at harder for few years, maybe we're going to have your own Canadian community, stronger and growing, strong and maybe different sessions, and even its own conference, so I am very excited with the projects. I have been with Cooper 90 for a while a long time, and this is going to be my own own, my own personal new journey.

A

So in this talk, I know most of you already heard about this creative and the three pillars of the event in build and serving, and also the pipeline, which is coming up next I'm not going to talk about about build area, focus or more or less on the event in and serving as well as the functions functions themselves and I will also take you out you how about a field trip to if you are on a mission, to create, to hook up your own event source to kinetic and we're going to have the field trip to do of.

A

You know relocates in real-world case study or how to make that happen in a quick and dirty and easy way, and very last I will show you a more kind of more of us develop our the deployments in my opinion, because if you are looking as if you are reading the documents or you examples creating ripples, many of the things make sense to you. But there are so many and I want to have a single one, Kirra applications just for you to take a chance to evaluate for your own personal departments, all right Kennedy.

A

This is going to be something you have already have been reading for past two days over and over again event in serving and build and event him. What is inventing is a mechanisms to catch your events from your source, your personal preference sources and deliveries to end points, which is your own service function and survey, is the mechanisms how the traffic's are being routed. How you are service functions in terms of containers because scaled up and down and from zero to many and from nine to zero?

A

As you know, server dysfunction is the mass schedule and build. How do you get your source images and how do you get your sauce from github, for example, a Norway of our piper down to the container images and into the registry again I'm only to talk about the embedding and a sort of vision, because you know there have been so many parts of you before. So you know this won't be reinforced in this second, so inventing, as you know, Canadian is really build on ideas of use extensively.

A

Leverage in the customer resource definitions, the ADIZ so inventing our CID is called a channel subscription and resources. So what is this channel channel? Is that if you are familiar with, if you have done these developments for the Mirabelle's, you know the events past. This channel is just like that. It's the price that you pull up your events from your different sources. For example, if you are using the crowdsource just Google, GCP pops up aw, ask us SMS. This could be of sources and that's what machineries forum? Where do you get your sources?

A

Casual events subscription which more or less is opaque? It's not like we are using in the traditional pops up subscription, it's more the kind of mechanisms that bind you of events channel with your serving consumer. So the idea, if I'm, not wrong with that, the idea is that if you just do one-to-one mapping from your event source to your service function, you probably don't think about a big in use case for the subscription.

A

But if you are thinking about fanouts deployments, if you have multiple subscriptions, if you have multiple downstream consumers of your events, then subscription is the place for you and you could create multiples, consumers for semi events and they will just be handled correctly and events controllers and the sources which is kind of new thing in my opinion, because it was created not too long ago. The sources is idea. That's you.

A

You know, because the inventing internally is created with the mechanisms in mind, as you have the controller, to create channels and subscreen subscriptions, but there's a special case. You want you don't want to create any provision as you don't you don't want to preview any consumers when you're done stream, because some possibly message handlers, you just want to pull out our sources and you directly programs the source to the consumer. So your sources are just for that purpose. So so far we have the container sauce I.

A

Think the methane is here in the second and a cron job and traumatic events and disappear, pops up and github and even more coming up. This is very good place. You can check out, you can go to the Canadian event in sources, a repo and find your favorite sources that you can make use of and create all your own example. And then you see ideas such as the api's. You have to have controllers. Remember that skroob Nettie's is all about definitions and resources and controllers right.

A

That's what the Canadian is beautiful as well, so the controllers is. The mechanism is to actually the execution engine that Springs your definition to the realization. So you declare your you go and then the controller's just creates them for you. So the most, if you are controller in pieces, all the internal. So if you are controller developers and the things come up to you is how do you provision your channel and how do you make the messages pass along and to the downstream consumer? So the channels here already in the revenge in ripples?

A

Are you memory? Basically, just do a relay from the source image, a source message to the time dimension and g3 pops up, there's something you have to do in the controller, because you can you have to create your own TCP pub/sub subscription, if you, if it's not there for you, it basically how to provision one. So that's what was the provision for and also there's Apache Kafka, not quite familiar with? So if you have good ideas, how it works with the manual which recently is, and mats and Matthews also C&C I've hosted projects.

A

So there's a controller for that as well, and then before we go on to the right, give you a case study how to address your own container source and have to briefly introduce the staff rgw. So for those of you who are already for me to be safe, ITW stands for video. Skateway is something if you are already used extensively. It's just like s3 on premise. So, as W can speak, two languages s3 as well as sweet, it's the object store and the rgw has multiple capabilities, one of which is multi sites.

A

Basically, you have the primary size. You have your secondary saw us what is primary sites and secondary sites, and if you want to make a replication periodically or consistently multi sites is the is the module that's for you, so you define your multi sites and create it and then once there's a buckets, get some objects. Objects will be replicated to the secondary sites and pub/sub is a new thing. Human masses?

A

Actually you don't see in any staff. Do this so far, but she's going to come up in the few months in the next staff release. Another natural if I pronounced, is correctly in the next staff release you're going to have it basically, a pub sub is a protein of the multi sites because the motor sites can do in replication for you right. So, instead of just your full all objects application, it just creates the events just directly the s-300s week, as if you're subscribed to a street events, you have multiple mechanisms you can use.

A

The + icon has use SMS ask us pops. Ups is just like this idea, so is creating events for you, and so anything you can define a topic. Basically, a topic is the created on the bucket level, so if you create shopping for that pockets, anything changes to the pockets. For example, if you have new objects uploaded, then the events we have just recreated for that there's a publisher. The publisher is the hi GW itself. The pub sub module will just publish the events to this topic and you creates a subscriber.

A

This subscribe actually subscription its terms confusing with the Canadian venting subscription. So you creates a subscription for this topic and then, from that topic, endpoints is basically REST. Api we go to the REST API endpoints and then you grab the events from there and you can catch all the informations like a which packets receive the objects and what's the name of the objects, so I suppose that it's not being widely tested so but I will come about our cover.

A

Does carrots and how, if you are really fall in love with this pops up how we are we're going to make that happen next year? That's too far, so if you're want to expand your this high-level overview, if you want to use that and expand your own personal invention source, this could be the way to get started on the right side of the picture. Is your invented source? So in this case it's table GW pops up and on the left side.

A

The first block on the left side is the container source I'm going to cover that and basically what's the container source that is the to a periodical Pole. You can start like five minutes time. House and powder go to your pops up, API, endpoints and you're, going to check if there's any inventor image and port events.

A

If so and then the events once you get the events, you can just pass along to the endpoints, which is the channel I'm, going to cover that it's basically endpoints and HTTP endpoints as a programmers view and but internally, that's how the actually be endpoint is resolved. The resolution goes to the events. Controller event in controller and a subscription, as I describes, is the basically the binding is a contract between the the channel and the swervy.

A

In the subscriber it either that could basically be either service the event and those confused with the chromatic service, either those service service or the rouse which is going to come up next and I, do not show the picture that you can have multiple subscriptions, I, think I, believe it works and but have math tested yet and if you have more ambitious deployments, please feel free to do so and break anything, and we are going to fix this one.

A

When I say we I mean the Canadian maintainers yeah I can be the helper right, so I'm specifically focused on the container source, and why stars you can have you can get your own container source and the if you are familiar with, if you are competitive at apparel and you are working on AP is you know the hassle right, you have to catch, your API object defined, you have have the generation function, works and you have to validate your API. Is that's a Pierre part, and then you rest, your own controllers.

A

That controller can handle everything correctly, creates updates based watch d-list inspectors and, after all, these things are done. York focus on business logic, how the container, how the events are handled and how they are delivered, and you know instructors so container sauce is one of those things are like very much and you don't have to advise your own ap. Is you don't have the handle? You all raise your own controllers. You just specifically focus on a business logic.

A

So if you wonder how, for example, in my case there are GW, container, sauce or I need to do is go to our develop your dress, APRN points and grab IPI from there and there's core HTTP POST, and that's it it's as easy as test.

A

So if you have a job, if you have a mission to create your own events bindings to handle your own firmness or just your precious event, source from your internal Enterprise and if you wanna catch this time, quick and dirty with mass victory is actually clean way and we went down the cleaner hours or days this. The way for you, if you have more ambitious, you won't have all eventsource feel free to do. Is it's going to take a days or weeks?

A

So if you are decided at The, Container source is right. This is the right way for you and clearest is basically the trick is the HTTP. How do you cast 82 endpoints and the cool thing is the good news is that is called a sting, and the sink is the way that's the HTTP endpoints is created is created automatically for you by the event in controller at the container source controller is either appended as CLI option or as if you already use the sync option in your program.

A

Unfortunately, and there's the second option is using this as an environmental variable. So Nicole is your vision in gold. You don't have to. You can rest any of your favorite language, nor GS, Python whatever, and so the read the quranic with the let's pretend you can read it: okay right. The to think in this case is a appended as a CRI option. So your program, you always expect that the options there and what it is is our endpoints no problem and there you just want to catch your event.

A

We just ask them to actually post and message is automatically sent to the servant. To reserve in controller to service. Has multiple controllers. How that happens? You can go to the details of my code and I. Have the link at the bottom of the page right so you know, does have your own control is. Have your container source? Is one thing: how do you? How do you guess is deployed?

A

And then, in this case we have to use the Y mo to, as you have been used incremental all through the years we create a Yama phase and the things that you had to take special attention. Special taking is use container sauce as you kind as I mentioned. If you are using somewhere.

A

If you have your own containers, if you have your own invention sauce, you use that so is you: have you define own API and useless API objects in the kind, but in this case you just container sauce I skip all details, but the next most significant things in the demo is the image right. You create your programs, you build a dark image and you deploy it. So we can reference this in the container source back the thing in this in this llamo.

A

The thing is part of the SPAC, but you don't really care how the house back turns into HTTP endpoint that was automatically automatically done by the container source is going to be resolved it because it says this is a API API objects of inventing is a channel, so that's going to be resolved. The channel will be resolved into activity and last name right. So what is a channel channel is that I covered before so channel is the place that you get your messages and messages are delivered to the other consumer.

A

So there's nothing magical here, except as you have to use the memory channel in this case, because there's just relay of what you have from the source to the invoice. There's nothing in between there's. No me where there's no cop car in between. So it's basically just a memory copy. It's not really. It's kind of copy memory copy from the source from your events and automated certain functions all right subscription. That's the the very last I believe the very last IP objects in the invention so subscription as I'm as email.

A

Was it's more like like a binder right. It binds the source of the channel and to the endpoints of the slabin in the soybean. There are some others. The ADIZ I use this pretty special attention to your service here, because we're going to swap service to something else later. But basically the idea here is just use. The new subscriber are you serving I haven't touched anything else yet, but it's could work, and so, if you are using a proprietary service, this may or may not work.

A

That's Jesse, but most like you have to use the service service because that's a simple way and if you were just do a 1 1 points. 2 points delivery, that's the better way to go. If you want to do one-to-many delivery, you what it really Ross or whichever we're going to do, is Ray at home. Alright.

A

So far for the seminar for event in serving that's where the magic happens and how these messages are delivered, how slowly function, container, spawn and quest, and this lower balance is Pingdom and that all happens in serving so the three ideas most extensively in the serving is route and, as the name suggests, it's basically the relative mechanisms. How do you get the message from the source and how do you deliver to the endpoints, the rap career?

A

You could have multiple endpoints, so the routing mechanisms have the magical happens in the roles and the configuration to fish and more or less in my opinion is like the part you accept as a there's a butte reference in the configuration, so you can either use a build.

A

You can roughly refer to a build, you know which is another pillar of the connective if I need refer to a build, or just in my case, just used containing image video, you don't really see, it very often is more or less the internal team, but just think about config configuration is each ripple and revision is like a Shia. Is the snapshot of what you are using, so you can, you could put situation, can have multiple revisions and you can as time progresses.

A

You upgrade your contain image and you can have another revision, but you don't really use it revision, your llamó, it's a more internal team and a service for years of use, because you don't really have you know, that's a quiz. You many use cases you just want to one-to-one delivery, one end-to-end delivery. You don't have multiplexing so so this is the easy way to go.

A

It's basically prevents the routes and configuration for you and you don't really worry about how they are delivers in they're handled internally, how they are created internally, which is a great way to get started. So this is a grossly simplified view of how Slovene looks like and all these theories, and once you get up soon request and the rouse the controllers- and we are just handle, decides how much traffic goes.

A

This way, how much traffic out that way, if you only have a service or traffic's 100% according to the one rules right, so the configuration is the price that we will define. Your part define your parameters to run with that part, and that's the placed in squares interesting, because eventually your function is going to going to be there and configuration either is a reference to a container image or is the reference to a build, and so, if you are, you know, I'm not having build here, but it's something you can fly by yourself all right.

A

The uses case, as I said, is the service and the service is the Browse flask configuration is everything handled internally, for you is a beautiful item as simple as you provide me, the container image or reference to the build so remember what we have used before in the previous event. In slide, we reference.

A

Okay, we reference the we reference service in which the ad the reference container we reference service in which channel or subscription.

A

Subscription, that's great, so your hands are really dirty now. Okay, and if you have a different function and which, in my ambiguous case missions cost is region, is a region service, cook or vision service? Simply a CD has changed the image, I points in a different image and that can be quickly switched.

A

Our examples are in my repo right. So if you have more so I'm, not sure if you have been to these files, one of the talks one of the few sparks actually because I'm also talk to people. One of the few talks I want in past few days is from a media which I'm a big fan of the company. It's about scale, 300 m2k of scales, actually scales, IRA our image difference right. So in their presentation they have the nice architectures.

A

They want the board balancing they have multiple GPUs and how they do the ingress and egress. The good news is we can the kinetics of ease or doing all these paths for you. So, as you see as if hi, so you just need to define the rules and define how much traffic space you need and internally everything's already handled, I'm, not sure if that's been considered already pronounced, but in case of fail or we don't really need to have a split. We would rather choose why if they land, then we find our source.

A

That's that's the difference from this. Please I'm not sure if that's being considered or just pronounced, but a lot of us to have to happen in serving eventually right. So if you have one have more advanced deployments, if you want your higher scales, you have Scout inference service and this the way to go.

A

You justify a route and in rouse you referred to how many configuration you won't have in my case, I have 50/50 splits between the google region and just not configuration, and so I don't have the configuration mo here, but you can go to my repo and I'll.

A

Take a look basically what's this llamo tells you is that as we, the traffic, if you have 100 requests, come in 100 events come in and they were going to be given its place between to choose container functions, which is defined by the configuration all right well, rusty and nos past one and functions: that's the core business right! You don't really care about. You are not really the consumer and you're, not really the developers of Kennedy's. You are what you use in Canadian for a big waste business.

A

So function is the way the entry point for you and what is function? Does it surprised, as you define how the from the? How is Cathy invoked, how you inject your business logic, so the function, entry points which is not really Dunston Mecca the function is more or less like a program. The entry point is HTTP service, so you define your HTTP service and you have to stand it up. First, is port is always 8080 I, don't know why, but just that just happens magically and then in your HTTP handler.

A

So if you read the code from the bottom to up so we have the entry points, HTTP service, 80-84, 8080, and then we define the HTTP handler. So if you work as a post message, the Hydra is going to be invoked in this quick and dirty case is just prints out the message right: that's your span. Employees. You want to see if they work tantrayana connections. The earlier events is positive, so that's the trivial function and let's go deeper.

A

So in an resinize function, which is an inference, is a my inference current and let's just build the code from bottom to up and then from the entry points you HTT the service in Defiance port 8080, and then you go to the event handler, which is where your beautiful logic of business logic is, and then you go practice image grabbed as a message. In my case, the message contains the information about the packets I.

A

Remember this is our gwe signature s3, so you grab the bucket and a key name, which is the objects name so from there. You know how to locate and how to download your objects, and then we encapsulated it's a skeleton, but everybody has a skeleton in the cupboard right all right, so in that so in function.

A

We pass this around to my beautiful clients, a retinal clients and to do the production, and this we are just grabbed me- was a museum internet forum for datasets and I used in retinitis, 50 I believe for a model, so that's just go handle ordering to the production serving from a servants part and then from there. It's just give me back the classes now.

A

What's the object is so if you are familiar with Imogen as an image not has 1,000 classes about which objects, the wish of the crisis there are objects falling into and just to reinforce what we have learned. Basically, the Torres case is the same. So if you start from bottom and the end point is the 8080 HTTP service and your business logic lives inside the handle, and you know you're the first panel, the image, the top function and you turn out the our table.

A

Objects first and then Spanish doorway to Google vision, service and then from there you cast annotations. This is different from the roughness which only use in the image nurse trustees. We actually class the descriptions about. In fact, you actually cut the labels of the image all right so far, so good for functions, and this is the big picture, how we put everything together and the first style and 1ft started the first tab. You pour out your event.

A

You put something to IDW it just exactly just put something into a GW and there is generates the events. Pops out. The charity, events and your container container source at the double container source port at end points every five seconds and then find our investor. Let's grab that events and posted in that encapsulate.

A

The events in Stratos rocked and poster objects into the HTTP endpoints, which Darren handled by your serving function- and you know it goes through the slobbing in the first place, but eventually our being as being accepted by the soviet functions and there's a 50/50 split is a basically scales. You know it's just inference: it's not free, but you can run out of york.

A

You know capacity budgets, you have to scale some way, one way or the other you to crowd or the coach add another inference server and then I understand your objects over the results back, so that imply English. If you speak like a native language and things becomes in this way. So the first thing you do is the first object to hit is the event in container sauce, obviously I'm just covering containing event sauce, and there are some other things you can try as well.

A

If you want to build your own controllers, your own containers yeah, that's the way to go and then the rgw event in sauce. We are just pour out everything and post a message to serving so the things the api's the objects being covered here is a channel subscription and then once everything goes through serving the objects being used here is either service or round and then eventually you're inventing functions. Your service functions started, it started at is apart and actually started at the deployment, and then traffic brought you to identify endpoints.

A

Your HTTP endpoint invokes the handler and your business logic get started, I want 40, and then we come to the deployments, which is really easy. Quick, so you create your event. Sauce I, don't have the event sauce here, the CMO. You can go to my people and then you start the event sauce. You creates a channel and it creates everything else. There's one thing I do not cover here, which, if you're a developer, if you are administrator you have to do this by yourself- is the service entry is Sto.

A

Api up takes is defined, the increase in egress and ingress. The policy starts your event. Sauce. Your service functions must comply so in Google regions, case suppose, Google region, service, region bass go go MPI, comm as a service entry, actually to be a sports as a service entry, so you're serving part. Your service function can go to the Google vision to grab them to do the annotation, and that is something because, as you know, kinetic is build exclusively on sto for traffic management. So that's something you have you take care of.

A

If you see your functions, gets blocked or your request gets denied because of the extra 500 or 400 I forgot the code and maybe 400 or whatever, because we, our account, is still gives you and that's there's a great chance. You do not have the service entry definition yet just created using the examples there. How in my repo- and that's we just creates everything.

A

The final has believe in us. I believe the the black has give you a gulag and I'd, so I create 10, copies of them and then I grab the cats. I create 10, copies I put them into my rgw pockets. I give the file special names, the cash numbers, 0 1, 2 files, or way to that night, and then hopefully, I can anticipate my event came here. All right, magically I, don't have screenshots I, don't have our videos I. Believe stickers speak louder than play.

A

The videos so I have all stickers here around just take a quick look, this time from the bottom to top from top to bottom. So so, if I look at the locks, which is the best way, in my opinion to debug your Soviets function is to see what is in the logs. So if you're on the labels is a serving the sky native past death and conviction across you resumes configuration remember we have. The routes refers to two of the configurations. Why is the ratna's?

A

The other one is google region, so you start label to check the locks and is has multiple, contains three containers. Actually there's a sto sidecar, there's a user container and of course the other one is queue or something. Oh yeah. Thank you. So the use the checker rocks of the user container and, in my case, I, see these wonderful messages. The cat number seven being captured is purely random and depends on lots of things. There's no deterministic orders.

A

Even you know the pop star could get hot father the resolution of the annotation as out of order things could happen and in my case you guess, crust is 286. I wish I look up as the image mass 286, it's basically the cat family. So that's more! That's close our x-rays. The right and there are mistaken function which magically shows me, the cat's number one being captured and we keep the wave of cats and a score for post 99.

A

That's great, alright, what's going to come on up nerds and when out of my time, but things will happen so this is just fever in terms of we only have the functions the function does matter, but sanitation. You are not really a care about. What's wrong house, you, you want to sound, do something more useful in your business logic, so things like auditing compliance.

A

We wanted impacts if there's copyrights in issues, if there's a sensitive information in your data and if there's anything you want to in, for example, you want to get rid of you know for just temporary free map since could happen. That way, and you think we're going to do next is to have the metadata tagging, which means once you have the labels. We want put some the labels on the object itself, so why do we restrict search which is a feature of asif, raja babu?

A

You can search by labels now, obviously we're going to make a GW available. You know easy to deploy and can use rook, and for those of you who already know Luke or just a silly root cultural rook pose here and they are friendly people over there who can help you navigate right, I think. That's all any questions raise your time.