►
Description
All You Need to Know to Build Your GPU Machine Learning Cloud [B] - Ye Lu, Qunar
GPU is becoming the new common, but at the moment, GPU resources are still hard to find for people who wants to have a taste. So how to build your GPU machine learning cloud?
Resource management & App templating
Even if your company or organization have purchased some GPU devices. Environment and resource isolation is always a problem. Also at the beginning the cloud is more used as a playground, so another consideration is to improve usage rate of resources. How we use Kubernetes to solve this problems.
How to use a wizard to generate machine learning, you can choose using tensorflow or theano, how many GPUs you need, etc.
Make the “customized changes” in immutable container be played back.
The features of container is immutable, which is a double-edged sword, which can ensure the environment can be unique/portable. On the other side, any changes inside the running container can be lost after recreation. How the customed env is saved and reuse?
Managing persistence storage in Kubernetes
How to turn our RBD served as hosted s3, to save models, training data, and so on. So The data scientist can access their data both as a volume and s3 standard api.
Support the running machine learning app,like tensorflow to do online resize.
App model & permission control
We'll talk about the app center , design of appcode and permission control.
About Ye Lu
Devops Engineer @Qunar. Experienced in operating and managing OpenStack cloud, including Qunar OpenStack Cloud in 7 regions. Started constructing and using kubernetes Cloud since 2015. | OpenStack Ambassador.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
Okay,
thanks
for
coming
this
is
today.
Yes,
my
speech
is
my
speech.
Topic
is
all
you
need
to
know
to
build
your
own
GPU
machine
learning
clouds-
and
this
is
about
me.
I-
am
the
DevOps
engineer
for
China
and
I'm,
mainly
responsible
for
the
private
clouds,
building
operation
and
maintenance,
including
the
container
as
a
service
and
machine
learning
cloud
platform
and
the
private
of
stacked
house.
A
This
is
today's
agenda.
I
would
talk
about
the
rise
of
students
more
for
me.
Let's
see
sorry,
I
I
would
talk
about
the
rise
of
the
deep
learning
and
I
would
talk
about
some
applications
of
deep
learning
in
China
and
then
I
would
talk
about
the
GPU
cloud
solutions
and
then
some
tips
of
building
your
own
clouds
from
my
practice.
A
Firstly,
I
want
to
introduce
them.
I
want
to
talk
about
some
situations
we
face
now
the
rising
of
deep
learning
this
year,
the
hotness
in
the
deep
learning.
Maybe
the
Africa
has
been
cut
in
Oh
two
three
cozy
is
the
world
class
go
player
and
at
last
contest
he
cried
and
said:
I
couldn't
I
couldn't
find
any
chance
to
win,
because
the
alphago
is
too
perfect.
Another
news
is
that
Africa
zero
has
perfectly
beat
Africa
through
several
learning
and
without
human
intervention.
A
A
So
this
is
Hospice
he's
the
founder
of
deepmind,
as
well
as
known
as
the
father
of
alphago
and
africa
0
and
alpha
0,
so
his
confidence
smile.
We
can
tell
that
deep
learning
with
the
hopup
bridge
and
brighter
future
and
deep
learning
has
go
far
from
the
games
and
it
actually
have
more
practical
applications.
This
is
the
Google
driverless
car.
A
It
has
a
run
for
very
long
distance
like
I,
think
it's
two
hundred
thousand
miles
and
it's
not
without
no
accident
and
nobody
hurt
or
injured
in
in
the
task,
and
it
can't
be
reached
without
the
technology
without
the
technology
in
deep
learning.
So
all
those
big
companies
are
start
fighting
for
the
AI
euro
and
among
a
minder
all
the
resources,
the
talents
and
the
scientists,
maybe
the
one,
maybe
in
the
first
place,
as
maybe
some
people
know
that
Li
Fei
Fei
from
Stanford
has
joined
Google
and
Amazon
the
in
self.
A
A
from
Amazon
has
joined
Alibaba
as
the
vice-president
and
he's
best
known
projects
from
Amazon
is
the
Amazon
go.
Amazon.
Co
is
expected
to
subvert
the
traditional
retail
model
and
also
Google
has
changed
its
go
from
mobile
first
to
AI.
First,
ok:
let's
talk
about
some
history
on
the
top
of
the
picture
is
telnet.
A
Our
net
is
maybe
their
first,
a
successful
application
in
the
mouth
layer,
much
layer,
neural
network
and
the
it
was
used
for
the
handwriting
recognition
recognization,
mostly
for
the
numbers
and
the
zip
codes
and
another
huge
project
is
minused-
is
a
huge
handwritten.
T
digital
data
set
that
was
started
in
the
1980s.
A
Now
here
is
the
another
big
names
in
deep
learning
is
Hinton
and
young.
Actually
dip
learning
is
very
hot
now,
but
its
history
is
not
that
easy
back
to
the
1960s
you,
you
submit
a
paper
related
to
the
deep
learning.
Your
submission
would
be
rejected,
so
until
2006,
Hinton
and
young
proposed
the
idea
and
the
subject
of
deep
learning,
it's
just
another
name
of
the
previous
research.
A
Computer
resources
is
reduced,
a
lot
such
as
the
GPU
and
more
TPU
invented
by
Google
and
the
popularity
of
the
open-source
tools
as
such
as
since
Florence
some
like
Kyra's
or
pedal
pedal,
okay,
GPUs
up
to
be
learning
accelerators,
because
we
used
to
take
experiments
for
like
for
several
months
or
several
days,
and
then
they
we
finally
find
out
that
our
models
and
our
parameters
may
may
just
a
fault,
and
we
need
to
change
it.
And
maybe
another
months
passed
and
the
process
is
quite
pain.
You
can
tell
from
the
image
on
the
right.
A
A
Using
the
deep
learning
technology
we
can
turn
the
computer
to
smart.
The
computers
can
distinguish
the
good
or
bad
and
help
to
control
the
risks
and
to
meet
the
preference,
such
as
we
have
the
following
applications
in
Kunar,
like
the
hotel
recommendation,
based
on
your
order
or
browser
history,
and
we
can
calculate
the
different
hotel
room,
type
price
factor
and
we
have
the
smart
customer
service
the
thus
the
smart
cosmos
for
service
and
react
much
quick,
much
quicker
than
the
human.
A
When
you
die
a
spot
number
you
may,
which
for
quite
a
long
time,
but
with
the
smart
with
the
robots
they
can
response
in
seconds-
and
here
is
the
interesting
application.
It's
called
little
point
how
it
works.
You
can
look
at
just
you
can
just
upload
a
picture
and
it
can
wrote
a
write,
a
point
stir
it.
A
Let's
learn
something
about
in
the
old
days
in
our
company
in
China
and
the
risk
of
how
we
use
the
GPU.
We
used
to
share
the
GPU
resources
and
the
risk
of
sharing
GPU
resources
is
maybe
one
day
one
my
colleague
understand
out
and
say
who
killed
my
tasks
that
I
have
run
for
three
days.
Your
hard
work
may
be
interrupted
by
someone
easily.
A
Maybe
the
one
I
didn't
know.
What's
your
programs
running,
and
maybe
it
is
important,
but
they
don't
know
it
because
it's
not
it's
not
his
another
reason
is
that
the
purchase
cycle
could
be
quite
long.
It
could
be
days
our
man
says
in
some
enterprise
if
it
is
a
practical
problem
and
I
think
that
a
brilliant
idea
shouldn't
wait
that
long
or
you
can
buy
your
own
GPU
and
make
your
own
devices
they
used
to
face
the
challenge
to
the
you
still
face.
A
The
challenge
of
the
machine
broken
or
the
data
loss
and
I
have
tried
to
build
a
computer
like
this,
and
it
could
be
very
loud
if
you
run
some
applications
on
it
and
another
practical
problem
is
the
low
issues
virtualization.
As
you
can
see
from
the
monitor
at
some
point,
the
GPU
utilization
could
be
very
high.
It
may
be
up
to
100
percent,
but
at
most
of
the
time,
it's
zero,
so
on
the
average
the
utilization
is
low.
Okay,
to
sum
up,
there
are
some
problems
of
the
old
ways
to
using
GPU
resources.
A
A
If
you
change
your
machines
or
devices
the
you
need
to
reboot
or
your
environments
or
your
workstation
and
all
the
dependence
it's
quite
time
consuming.
So
what
should
we
do
now?
This
is
the
general
ghost
of
our
platform.
We
need
to
remove
all
the
obstacles
of
accessing
resources
and
to
the
second
one
is
to
improve
the
resource
utilization.
A
The
first
one
is
to
clarify
the
CPU
resources
so
to
clarify
you
can
easily
control
the
resources.
You
can
build
the
applications
in
seconds
and
you
can
disable
it.
You
can
release
it.
This
is
in
just
one
click
and
we
need
to
control
the
permissions
and
O's
management
in
Hoonah.
We
do
it
by
integrated
to
our
unified
application
control
center.
We
call
it
portal
I
believe
that
in
every
company,
maybe
we
have
something
like
that
and
the
third
one
is
the
environment
isolation,
and
then
we
need
to
ensure
the
data
availability
in
distributed
environments.
A
The
last
one
is
to
support
the
tents
float
for
toolchain,
like
the
tents
board
or
the
tents
preserving.
These
are
the
very
beginning.
Stage
goes
so
next
I
would
talk
about
some
choices
on
the
components.
The
first
one
is
white.
Tents,
blue.
Why
we
support
tens
flow
here
is
the
comparison
between
tens
flow
community
and
others.
A
You
can
see
that
the
star
the
number
of
star
fork
and
issues
and
the
PR
is
much
larger
than
other
community,
and
also
we
can
see
from
this.
This
is
a
simple
code,
four
minutes
it's
just
149
lines
and,
and
it
has
defined
two
hidden
lines
and
it's
defined
the
softmax
regression
model
and
it's
quite
clear,
and
so
people
can
maintain
maintaining
it
easily.
A
A
Okay.
After
we
pick
up
the
dip
learning
framework,
we
need
to
think
about
cloud
in
file
our
GPU
resources.
We
have
two
choices,
then
back
down
so
metals
and
other
use
communities,
so
why
we
don't
choose
mesos.
Actually,
muscles
is
good
at
in
job
scheduling,
but
back
to
then
communities
is
much
better
to
handle
the
GPU
resources
and
its
have
multiple
persistent
data,
integrate
integration
plug-in.
A
A
A
Also,
it
has
multiple
storage
back-end
to
integrate.
It
is
very
important
to
the
deep
learning
or
machine
learning,
okay,
I've,
to
pick
up
the
main
framework
to
clarify
our
triple
resources
we
need
to
think
about.
Where
is
our
data
stored?
As
we
know,
data
hub?
Could
be
very
important
why
we
doing
the
training
a
stateful
application
is
important
because
the
GPU
resources
is
precious
and
it
can
be
disabled
in
some
time.
A
A
Okay,
I
I
can
explain.
We
provide
two
ways
to
access
your
data,
and
this
is
based
on
staff.
The
first
way
is,
actually
we
are
not
using
the
s3
from
a
table
s,
we
are
using
it
from
staff
and
the
two
ways
we
provide
the
first
one
is
to
provide
the
data
volume
access
and
the
second
Monday
to
provide
the
s3
standard.
The
API.
A
As
you
can
see
from
this
picture
of
the
staff
monitoring
s,
it's
bigger
than
any
single
machines
capacity
and
for
one
advantage
from
staff
is
you
can
easily
resize
it
and
no
need
to
stop
your
application
and
no
need
to
transfer
your
data?
Why
you
change
your
bio
change,
your
devices
it
can,
just
it
can
be
done
just
with
one
click
and
then
your
space,
expanded.
A
A
Okay,
next
next
components,
which
is
the
is
Jupiter
so
chapter
notebook,
provide
a
web
version
of
code
writing
and
exported
by
the
Python
Python
I
colonel
you
can
choose
different
versions
of
Python
or
you
can
use
different
languages
like
the
language
R,
and
the
advantage
of
it
is
that
team
members
can
call
collaborate
to
read
codes
and
to
debug
it
also.
It
has
massive
extensions
you
can
which
can
improve
your
efficiency.
A
Ok
after
you
choosing
all
the
components
that,
let's
take,
a
look
at
the
architecture
here
is
the
glimpse
how
the
applications
deployed
in
our
account.
The
first
application
is
a
single
machine
test
flow.
Another
one
is
the
distributed
test
flow
across
two
servers
with
three
GPUs,
and
it
has
one
parameter
server
and
three
workers.
So
how
can
we
do
this?
How
can
we
build
a
application
like
this
first
I
didn't?
Have
the
screenshots
of
our
user
interface
I
I
can
just
say
about
it.
A
First,
a
wizard
would
lead
the
users
to
choose
what
they
want
for
their
applications
such
as
you
can
choose
the
framework
you
want
to
work
on.
Then
you
can
choose
how
many
GPUs
and
the
storage
size
you
won't
use.
Then
we
generate
the
results
defined
llamó,
then
the
community's
reader
llamo
and
then
deploy
it
according
to
the
fire
and
the
resource
type.
We
choose
these.
The
deployments
check
to
them
because
we
we
started
the
project
like
a
year
ago
and
back
to
then
the
stateful
applications
may
not
work
that.
A
A
Maybe
one
developer
came
up
with
some
idea
of
some
model.
Then
he
want
to
have
a
playground
to
test
it,
and
then
you
finally
figure
out
how
to
how
to
build
your
models
and
the
parameters
and
all
the
arguments
is
that
you
can
distribute
it
to
the
Brio
servers
and
our
platform
can
have
to
do
with
it.
And,
finally,
your
well
tuned
and
well
change
model.
You
want
to
put
it
online,
we
have
the
10th
floor
serving
and
it
can
easily
put
your
models
online.
A
A
The
first
one
is:
you
need
to
provide
the
consummation
of
the
workstations,
because
you
may
want
the
the
developers
one
maybe
want
to
choose
the
different
versions
of
the
machine
learning
framework
and
and
at
last
I
the
if
you
they
want
to
put
their
models
online,
you
need
to
provide
the
H
a
model
registry
and
serving
service,
and
then
the
tributed
parking
system.
Integration
is
always
wanted,
and
last
one
is
the
resource
billion
based
on
the
advanced
and,
as
we
know,
not
GPU
resources
is
relatively
limited.
A
A
We
have
the
self
APIs
has
a
storage,
back-end
and
the
HT
proxy
has
the
load
balancing
it's
just
been
online
for
like
two
months,
and
there
are
still
issues
to
fix,
such
as
the
trans
flows.
Deep
serving
demon
is
or
always
need
to
restart.
If
you
want
to
add
some
new
models,
if
you
want
to
add
new
list
of
models,
okay
here
is
some
tips.
If
you
want
to
build
your
own
platform,
the
first
one
is
the
network's
solution.
A
And
the
service
discovery
we
use
the
accordionist,
we
don't
use
the
original
the
qubit
in
us
because
coding
us
is,
we
can
easily
comma
spy
it
and
we
can.
It
have
so
many
api's
and
we
also
can
use
the
coding
as
together
with
the
traditional
k,
vm
Tina's
services,
and
another
thing
you
need
to
consider
is
the
upstream
and
downstream
data
pipeline.
A
Here's
more
tips
about.
If
you
want
to
built
your
own
chip,
you
Klaus,
you
can
choose
one
main
framework,
but
you
can't
limited
to
it,
because
people
always
want
the
freedom
to
customize
it.
They
may
want
the
Kira's
limit,
wants
another
framework
and
about
staff,
because
staff
is
quite
important
in
this
system.
We
are
all
based
on
it.
If
the
staff
is
broken,
then
the
whole
system
is
down
I.
Think
so.
A
A
And
another
problem
from
the:
if
arm
of
sharing
the
resources
is
the
environments
Lucian
you,
someone
may
install
the
different
versions
of
lips
on
your
workstation,
it
can
cause
crushes
in
your
apps
and
the
data
sharing
problems.
While
you
are
using
the
distributed
frameworks
and
also
the
data
size
is
limited
to
the
local
disk
size
and
after
we
have
this
cloud,
we
have
solve
the
environments,
isolation
problem
and
we
can
set
up
the
environment
and
disable
it
in
just
in
seconds,
and
also
we
have
the
multiple
access
to
our
data.
A
A
B
A
A
We
used
to
use
the
gems
to
build
the
the
document
is
we
we
have
the
red
race,
boat,
repo,
to
store
the
darker
fires
and
and
in
our
unified
application
control
center
and
every
people
have
app
code
and
they
can
only
submit
to
the
talk
fire
to
the
rip
to
to
the
TOC
file,
modify
modification,
I,
think
and
then
the
chickens
can
built
it
automatically.
Yeah.
A
Yeah,
we
always
have
that
problem
because
we
we
handle
that
by
the
Palin
system
yeah,
because
billing
is
important.
If
we,
if
we
don't
charge
them,
they
can
they
want
the
the
resource
all
the
time
yeah,
because
maybe
in
in
my
experience,
because
we
we
cannot
only
monitor
the
GPU
utilization,
because
people
made
preaching
their
teeth,
their
data
in
like
the
CPU
resources,
and
we
can't
cure
it
and
yeah.
A
Yeah,
we
use
our
body
to
just
ask
the
backend
of
the
persistence
volume
and
we
also
use
the
minion
the
application
of
minion
to
provide
the
s3
standard.
Api
yeah
I
I
have
just
mentioned
the
the
issue
that,
if
you
use
the
low
version
of
kernels,
it
can
bring
into
some
serious
problems
and
and
other
many
other
problems.
So
you
need
to
test
it
in
a
batch
market.
Before
you
put
in
on
the
production
environments,
I
think.