Cloud Native Computing Foundation KubeCon + CloudNativeCon China 2019 (Shanghai), 5 Jul 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Open Standards for Machine Learning Deployment - Animesh Singh & Hou Gang, IBM

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Open Standards for Machine Learning Deployment - Animesh Singh & Hou Gang, IBM

Machine learning is typically viewed as simply training a model on data. However, the “last mile” of deploying models to production systems is often overlooked and yet is one of the most critical aspects of real-world machine learning systems. Despite this, currently there are few widely accepted, open and standard solutions available that cover deployment of end-to-end ML pipelines. In this talk, I explore the current state of ML deployment using open-source, standardized formats. The talk will cover the various available options, including PMML, PFA and ONNX, and how these fit in with the most popular and widely used ML libraries (including scikit-learn, Spark ML, TensorFlow, Keras and PyTorch).

https://sched.co/Nrvv

A

So let's get started.

B

A

Very happy to share.

B

A

Topic is higher than standard machine learning or deployments. First and I change gears myself.

C

A

From IBM I.

D

A

Been working in open source, related items like open, snack and also machine learning, know I focus on AI.

B

A

Two colleagues.

B

A

Cannot attend today's meeting because of visa issues.

D

A

Both of those two colleagues come from Center for open source data and AI technologies.

B

A

It aims to make AI solutions to enterprises and companies, and this involves machine learning.

D

A

Also, the machine learning workflows.

B

A

As the detailed treatment like data processing.

D

A

Also algorithm processing, as well as optimization, well and.

B

Customize the services so.

A

Today, we'll talk about one step of this whole workflow.

B

A

Is motor deployment.

B

A

Are the challenges and how could we solve it.

D

A

Have Spring ours today, I will introduce the background first and then I will introduce the open standards for machine learning deployment and then we'll have the summary.

D

A

Talk about what's your first impression when we talk about machine learning, maybe.

D

A

Will remind you of some bad robots who want to slaughter human beings.

B

A

No siree good guys good robots who will do good things to human society.

D

A

D

Shall learning.

A

It's like a double sward.

D

A

Learn from the data to make predictions.

D

A

Then you general generalize the rules.

A

To be specific,.

A

We learn from the historical data to make predictions.

C

D

A

Learn from historical data to make predictions about the future in order to make decisions.

D

A

The current machine learning, for example, for a I.

A

Don't think it involves a lot of like human elements.

B

A

You can see in the movies some machine they are friendly to human beings, they're, not friendly.

D

A

D

A

An intelligent.

A

It's more like data processing and they don't match.

B

A

Just a complicated match and correlation.

B

A

Using the current data to compute.

B

The result and then.

A

Give the feedback and then you further adjust.

B

B

A

Can be widely applied in smart city, smartphone, etc.

D

A

Like to talk about a whole workflow of machine learning,.

B

A

B

A

B

A

The clients data and then we viewed a model we make just we make predictions, and then we can sit there and wait for the results.

B

A

Wait for the results to come out, but.

B

A

The I mean the actual process is much more complicated and what do.

B

A

B

A

Involves not only a person or a team.

D

A

Involves multiple teams of different areas, it evolves their coordination and cooperation to.

B

A

This whole workflow.

A

First, you have quite a lot data.

B

A

In the information age.

D

A

To clear all those data ingest injects the part that you think is relevant to you.

B

A

I think, for this part, we use big data to us like Spock.

D

A

After you get the data.

A

Take this data to the lab, you write different kinds of models who make predictions I.

A

Think that's your direct impression of machine learning.

A

But it involves multiple steps and procedures.

B

A

To visualize the data.

D

A

To take out the features and transform.

B

A

B

A

Then we trend the model and the model might have different architecture or structure, and then you need to choose.

B

A

You might have multiple versions of the model.

B

A

You need to control the traffic to the a B test, etc.

B

A

Then, after the multiple life you need to think about how well it works you need to think about is usability involves multiple teams. Only one team.

B

A

It's quite complicated.

B

A

Graph is very famous coming from Google.

A

It reflects the true machine learning situation. You can see the black area.

A

That is machine learning code.

B

A

Then actually you spend most of your time. Other modules.

B

A

C

A

Or to pre-process after process brutal, monitor the system.

A

We'll talk about how to deploy the model completely, that's maybe the final step.

B

A

It's once that, sometimes you will knick knacks work when you're in the lab.

B

A

We deploy a model, we need to think about three questions.

B

A

Is what are you deploy.

B

A

You use test flow, other means, and then you need to think about. Where are you deployed whether is in cloud browser I.

A

Also injured think about whether it should be real-time or streaming, and the last question is: how are you deploying.

D

A

You find a certain engine.

B

A

How the model is presented.

B

A

Is a model it's like a document.

D

A

Have the document and then you can predict? Actually it's not.

B

B

A

Models to receive.

B

A

Sometimes, when you upload a picture actually, the model only receives some data, because the model needs to do some pre-processing and then it will output the data and then the data would be transformed into a visible and identifiable thing. For example, a picture.

B

For the user so.

A

Actually, we have the pipelines, no models.

D

A

Need to know what is involved for each pipeline.

D

A

We have some challenges.

D

B

A

Different departments they may use different languages.

A

Like they're, the scientists or researchers.

D

A

Care more about what kind of problems will be solved by this model. They care more about the water. As for the performance of the model, actually they don't really care, but what comes to, for example, production our department.

B

A

Will care more about performance.

B

A

Example, they might need C language, etc.

D

A

Also, different departments might use different frameworks.

D

A

To make sure they can cooperate with each other seamlessly, that's a challenge.

D

A

Also, why you deliver the model for IT department.

D

A

Hope you can have as less challenges as possible and also the changes could be under control, but for researchers or data scientists that they need to make sure that their model is the latest one for business department.

A

They don't really care whether it's based on container or other things they care more about whether this will create any profits, bring us the ability, a stable performance to the client.

B

A

Need to reduce the differences or differentiation as much as possible.

B

A

To make sure different departments can work. Similarly, without those gaps and.

B

A

B

A

So, for example, I like to use this flow, and others like use, touch.

B

A

I use passive flow.

B

A

Use PR touch and we don't have a unified standard.

B

A

Then you have different customized ways, then in the future you have a system that can support both of different languages.

B

E

Motto written by huge butt.

E

And also the Hornets love something she off-spinner vation.

E

B

E

Transformed from each other that.

B

Will be one leader what.

E

Are you doing so? We need this standard.

E

When this Congress models are transformed into a standard format,.

B

C

E

B

E

The Chinese.

E

Which are exclusive to our own culture or might not be applied to.

C

Other countries.

B

E

E

C

E

May be focusing on machine learning.

E

Process yeah does not support this format.

B

E

There exporters can do it.

E

E

Next, you dust mites GPM, also support their Manos, transforming into PMML.

E

E

Lot of metadata information in it.

B

E

Friendly to your users,.

B

E

You can see here right in how some of them don't one shortcoming the volume that is.

B

Normal standardized.

B

E

Might need other things this beyond a standardized aspect and which.

B

E

Be met whether P in my mouth, such yes, custom, plugins,.

C

E

Luckiest written.

C

E

It cannot be transformed. Yes, you, this email format, because.

C

It's not before this new.

B

E

There's the question of Licensing Mario walk walk so for companies who want a commercialize that they might have concerns.

B

E

Saw the short comes, my PM.

E

Supports mini customized.

B

E

Me it's different.

E

It's the point. Deductions.

C

E

C

A

E

So machine learning algorithm.

B

Is your body you.

E

Can transform IPF file.

E

E

So the cross language functions work very well.

B

C

E

E

B

E

And output are both very easy.

E

B

B

E

B

E

P mmm does not support.

E

Pfa also has its shortcomings.

C

E

Some limitations.

B

E

Was projected by open.

B

E

I cover scoring for yet they engage em and Python.

B

B

E

Advantages, that's to what Canyon they can do. Well are type systems you know, so the flexibility.

E

Machine learning right, it is not support deep learning. The well.

E

It is not support for generic tensors and also no awareness of GPU.

E

The tests are limited.

E

For IBM internal.

E

Leverage, if a in spark in though.

E

It's open source. We welcome everyone who is interested in.

E

The future we'll do some testing which to test if the kids suffer or deep learning.

B

E

B

E

Neutral Network.

E

Deep-Learning operations.

E

So many other platforms in the to give us lesser.

B

B

E

E

E

C

E

E

Here are also other supported girls. Maybe you want to focus on.

B

E

They're processing.

B

E

It only support static processing; it cannot be using it for further trainings, which we must improve in the future.

B

A

Very mature and.

B

A

It has good support for extender machine learning.

A

B

Relatively new compared.

A

With p.m. now,.

B

A

It's highly customized.

A

It can do cross sales and post-process own eggs. It's a relatively new.

A

It could be very good tip learning standard in the future.

B

A

B

A

Standards for civilization and employment I think that's necessary for the current market.

A

B

A

Dinners all have their pros and cons.

B

A

Cannot, for example, if single Oakland Center cannot recover all these items, solution, learning.

B

A

Is no open standard.

B

A

Cover all the problems.

A

We're thinking about how to apply this different vendors in two different scenarios or to use them as a portfolio to solve the problems to solve the clients, issues.

A

So ad, that's all of my presentation.