►
Description
Want to view more sessions and keep the conversations going? Join us for KubeCon + CloudNativeCon North America in Seattle, December 11 - 13, 2018 (http://bit.ly/KCCNCNA18) or in Shanghai, November 14-15 (http://bit.ly/kccncchina18)
Kubeflow Intro – Michał Jastrzębski & Ala Raddaoui, Intel (Any Skill Level)
The Kubeflow project is aimed at simplifying developing, deploying and using ML on Kubernetes. We will provide an introduction to Kubeflow that explains the benefits of using Kubeflow and walk through getting started. By the end of this session, attendees should be able to run and deploy an end-to-end ML model on their Kubernetes cluster.
About Michal
Kubeflow contributor and former OpenStack Kolla project technical lead. Open source fanatic and enthusiast of all things cloud, both compute and those on the sky. Recently explores new worlds of data science, machine learning and data engineering.
About Ala
Ala Raddaoui is a Cloud Software Engineer for the Artificial Intelligence Products Group at Intel. He is an open source advocate and very passionate about any cloud-native, smart, and scalable technologies. He is currently focusing on enabling and enhancing machine learning experience on top of cloud infrastructure tools.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
Hello,
everyone,
my
name
is:
we
have
a
stamp
scheme,
I
worked
in
Intel
and
I
was
supposed
to
be
here
with
my
good
colleague
a
lab,
but
due
to
unforeseen
circumstances,
circumstances
he
couldn't
I
get
here,
but
without
him
this
production
would
not
happen.
So
it's
his
presentation
as
much
as
it
is
mine
and
I
wanted
to
talk
to
you
about
cube
flow
today,
so
cue
flow.
A
Currently
today
we
have
a
big
problem
with
clouds
and
machine
learning,
because
pretty
much
we
expect
to
run
these
trainings
on
cloud.
Multiple
servers
right
now.
Let
the
scientists
need
to
really
understand,
what's
happening
under
the
hood,
to
be
able
to
do
it
effectively,
and
most
of
them
would
probably,
rather
do
you
know
new
models
rather
than
learning
about
networking
and
on
the
other
side
of
the
defense,
you
have
DevOps,
which
are
very
well
very
well
suited
to
reading
TCP
dump,
but
they
don't
really
want
to
mess
with.
A
A
We
yeah.
We
want
to
run
machine
learning,
kubernetes
pretty
much
that
so
what
it
really
is
right
now.
It's
it's
an
umbrella
of
a
collection
of
projects
and
keep
flow
itself
like
repository
cube
for
cube
flow,
is
a
set
of
cases
on
that
case
and
prototypes
to
deploy
all
of
that
and
more
things
besides
the
Jupiter
hub.
A
You
also
have
things
like
pachyderm
and
a
lot
a
lot
of
great
tools
for
machine
learning
and
all
of
those
besides
jupiter
hub
are
actually
part
of
cube
flow
umbrella,
and
so
things
like
total
tensile
operator
or
the
other
framework
operators,
a
cutup
which
is
out
on
an
automatic
hyper
parameter
cleaning.
So
it's
a
really
really
early
state
of
the
art
technique
and
kvc,
which
is
project
that
it's
meant
to
allow
you
to
handle
large
quantities
of
data
like
data
set,
much
easier
and
much
more
scalable,
which
is
quite
critical
in
this
space.
A
But,
most
importantly,
it's
a
community.
It's
people
who
are
both
DevOps
and
machine
learning,
people
together,
trying
to
bridge
the
guy
bridge
the
gap
that
and
make
this
thing
much
better
than
it
is
right
now.
So,
let's
talk
about
model
lifecycle
model
in
machine
learning
terms.
Is
this
this
magical
entity
thing
that
will
turn
your
question
to
the
the
computer,
much
to
the
the
inference
which
is
which
the
execution
of
the
machine
learning
machine
learning
script?
A
A
This
is
going
to
be
very
live
demo
heavy
talk,
and
this
is
the
infrastructure
that
I've
prepared
for
it.
We
have
a
couple
notes
of
Kilburn
at
ease
and
our
sort
main
storage
for
data
and
is
in
GCS.
We
also
use
rook
for
persistent
volumes
and
keep
low
our
latest
greatest
first
release
of
0.1,
which
is
yeah
they're
generally
just
a
great
couple
couple
weeks
ago.
A
So
let's
see
how
each
and
every
step
of
this,
what
I
described,
look
in
cue
flow
today
and
I
must
warn
you
it's
pretty
early
in
the
game.
So,
first
of
all,
we
want
to
install
cube
flow
and
right
here,
I
prepared
a
kubernetes
environment
which
is
reasonable
once
it
Internet
come
on
yeah
it.
So
what
I
think
yeah
there
we
go
well,
we
have
17
nodes
out
here.
I
asked
me
not
here,
and
we
want
to
deploy
cube
flow
on
people
on
this
environment
so
late
it.
A
Our
main
deployment
scenario
right
now
is
we
will
use
the
because
cue
flow
is
very
heavily
based
on
character
today,
and
we
will
want
people
to
have
to
be
able
to
use
all
of
the
casein.
That's
nice
niceties,
which
means,
instead
of
doing
like
a
drop.
We
want
to
deploy
our
tool
box.
That
will
then
allow
you
to
manage
your
any
lifecycle
of
your
typical
application,
and
it
should
be
done.
It
is
done
so.
I
will
right
now.
I
will
deploy
a
very
basic
flow,
but
I
would
suggest
you
to
I.
A
A
A
It
should
run
yep
it's
running,
so
this
is
the
Bourbon
cube
flow
deploy.
It
took
us
took
a
little
bit
more
than
the
five-second
that
was
on
the
keynote,
but
doc
much
more
so
it's
gonna
take
a
couple
of
seconds
to
do
to
spin
up.
Actually
it's
mostly
running
already
so,
although
that
our
main
composite
of
cube
flow,
so
I
wanted
to
talk
about
first
step
of
the
of
the
model
life
cycle
and
that's
model
development
with
cue
flow.
A
We,
as
you
can
see,
we
start
at
TF
hub,
or
rather
it's
a
took
their
hub
and
to
be
their
hub.
I,
don't
know
how
much
you
know
about
Jupiter,
but
it's
a
to
standard
in
the
industry
standard
of
the
web's
web
interface
for
data
science,
it's
extremely
powerful,
its
formerly
known
as
ipython
notebook
and
our
Jupiter
notebook
and.
A
A
Going
to
take
couple
of
seconds-
oh
it's
it's
there,
so
this
is
to
Peter.
If
you,
if
you
familiar
with
this
you,
if
you
pretty
much
the
data
scientist,
you
should
be
familiar
with
this
interface,
and
one
nice
thing
about
cube
flow
is
that
when
we
just
make
this
for
small
hug
here,
it
won't
work,
but
normally
it
would
in
use
before
that
it.
A
There
we
go
so
this
is
how
we
start
working
on
the
on
the
owner
model
and
one
of
the
first
steps
would
be
also
understanding,
data
and
pre-processing
the
data,
and
for
that
I
suggest
to
go
to
Daniels
presentation
about
pachyderm,
which
happens
tomorrow,
right
yeah,
so
pachyderm
is
a
data
pipelines
on
kubernetes,
and
it's
also,
it's
also
a
important
part
in
queue
flow
workflow.
So
once
we
put,
let's
say
we
prepared
our
data,
we
created
our
model,
smoke
tested
it
and
it
works.
A
A
Here,
so
what
our
training,
once
we
developed
our
model
in
our
Jupiter,
our
training
will
let's
go
to
our
our
favorite
editor,
because
at
the
end
of
the
day
we
want
Python
and
just
simple
Jupiter
will
not
handle
the
pretty
complex,
a
code
that
takes
that
proper
machine
learning
code
looks
like
and
we
can
just
we
just.
We
can
just
create
a
you
know:
tensorflow
code.
A
A
It's
an
T
of
job
CRD
that
was
deployed
by
tensor
flow
operator,
which
is
also
part
of
Q
flow,
and
it
allows
it's
the
biggest
gap
Bridger
right
now,
because
it
allows
data
scientists
to
don't
not
even
care
about
enduring
infrastructure.
They
just
say:
I
want
to
definitely
train
my
model
on
the
five
nodes
with
GPUs
and
that's
what
will
happen.
Let
me
show
you
the
model,
the
definition
of
the
TF
job,
which
is
the
actual
training,
training,
Fink,
okay.
A
Okay,
so,
as
you
can
see,
we
just
pretty
much
determine
which
what
how
many
replicas
of
the
workers
we
want.
We
will
use
namespace
cube,
converged
for
experiment,
and
these
are
our
image
and
all
the
other
you
know
stuff,
but
this
is
this
part
is
important
right.
It's
we
just
this
is
all
the
data
scientists
need
to
know
about
the
infrastructure
and
that
will
populate
the
environment,
variables
and
conflicts
within
the
within
the
in
the
container
and
tensorflow
will
understand
that
which
means
it
will
just
work.
Hopefully,
so,
let's
try
it.
Let's.
A
A
A
So
this
is
how,
after
that,
when
once
we
saw
once
we
so
first
of
all,
how
did
this
part
looked
without
Q
blow
before,
as
I
said,
die
the
scientists
need
to
know
how
to
Grundy
how
to
run
the
infrastructure,
because
they
would
need
to
specify
exactly
which
node
will
with
which
and
which
node
would
work
a
world
will
land
on
which
node
and
what
are
the
networks
there,
which
network
they
will
use?
Where
date
where
this
data
set
and
stuff
like
that,
it's
pretty
complicated,
especially
if
you're
not
in
server
space.
A
Then
right
after
all
that,
after
deploying
their
code
and
setting
up
the
orchestration
manually,
they
would
own,
then
it
will
be
able
to
reach
training
with
Q
flow.
You
bill
talk
image,
you
specify
the
TF
job
manifest
and
you're
done
so
after
we
trained
training
is
great
for
science
and
stuff
like
that,
but
the
real
usability,
the
dual
use
case
for
machine
learning
is
the
production
inference
it's
serving
them
and
I
was
hoping.
I
will
be
able
to
demo
it,
but
unfortunately,
it
broke.
A
However,
the
to
run
it
on
production
and
proper
production,
you
need
eight,
a
you
need,
resiliency,
which
means
you
include
balance.
You
need
stuff,
like
a
be
a
being
upgrades
unit,
rolling
upwards,
it's
very
much
DevOps
DevOps
problem,
which
we
DevOps
know
were
intimately,
and
machines
is
not
really
so.
This
is
how
it
would
look
like
before,
and
this
is
how
it
looks
like
with
the
cube
flow.
You
just
read.
The
you
just
point
to
the
model
and
Q
flow
will
deploy
it
will
deploy,
is
your
bindings
to
it.
A
The
issues
is
submitted,
so
hopefully
it
will
be
fixed
soon
and
if
you
want
to
want
to
learn
more
about
this
part
display
this
space
of
serving
the
model,
I
would
suggest
you
visit
the
demo
and
presentation
of
a
Selden
project
which
will
happen
in
this
Friday
and
which
is
also
I.
Think
it's
also
gonna
be
based
on
Kip
flow.
So
that's
serving,
and
just
one
more
just
one
more
thing.
So,
as
I
said,
we
need
contributors
I
want
it.
I
really
hope
to
see
you
guys.
A
Join
us
luck,
join
our
mailinglist
contribution
doesn't
mean,
doesn't
need
you
to
be
data,
scientist
or
DevOps
you
can.
There
are
many
ways
of
contributing
answering
questions.
Publishing
issues
publishing
documentation.
There
are
lots
of
ways
to
contribute
and
I
would
hope
that
you
would
join
us
to
this
new
and
quite
quite
interesting.
I
mean
fantastically
interesting
space
of
machine
learning
on
cloud.
A
A
It's
so
depends
on
how
you
build,
because
it's
all
gonna
be
it's
all
gonna,
be
based
on
your
labels
and
scheduling
and
kubernetes.
So
if
you're
DevOps
will
set
up
the
environment
correctly,
it's
going
to
be
pretty
much
as
simple
as
use
GPU.
Yes,
then
your
code
will
need
to
be
able
to
use
TPU
because
you
need
different,
different
tensorflow
built
for
for
GPU
and
stuff
like
that.
However,
that's
besides
our
product.
A
A
A
B
A
B
A
Mean
the
sensor
board,
so
the
way
to
do
it
right
now
would
be
either
put
it
on
as
free
or,
as
you
said,
file
shared
file
system
as
free
is
probably
easiest
to
handle
and
I
will
I'm
working
on
I
mean
working
on
a
easier
way
to
kind
of
handle
one
to
another.
But
right
now
it's
mostly
it's
in
code.
You
just
need
to
start
the
tensor
board
instance
the
points
to,
for
example,
as
free
anywhere.
You
will
save
your
logs,
but
that's
all
in
code,
there's
not
much.
B
A
Don't
I,
don't
think
so,
I
mean
quite
frankly,
I
don't
run
myself.
Gpus
I
run
xian's,
so
I'm
not
sure
about
that.
However.
I
become
because
the
because
cube
flow
is
using
kubernetes
primitives
cube
flow
itself,
probably
won't
have
dependencies,
but
your
particular
use
case
might
have
right
depends
on
the
infrastructure.
A
So
we
just
discussed
this
also
yesterday,
I
think
the
multi-tenancy
and
multiple
users
will
be
under
required
feature.
That
being
said,
it's
not
this.
We
did,
we
haven't
figured
out
yet
fully.
There
is
a
some
multi-tenancy.
We
didn't
it
within
the
Jupiter
Jupiter
hub
itself,
but
that
won't
translate
to
Q.
Burnett
is
right
now
my
answer
would
be
spawn
a
cluster
for
us
experiment
and
but
we'll
still
figuring
it
out,
but
yeah
the
answer
it's
well.
It
is
a
required
feature
for
us.
As
far
as
I
know,.
A
So
there
is
catapodis
hi
peppermint
evening,
which
kinda
touches
that
I
think
she
I
personally
think
CIC
D
of
the
models
itself
will
be
very
important
and
very
hard.
There
is
pachyderm
which,
which
handles
versioning
of
the
data
and
pre-processing
by
the
way,
but
the
actual
models
are
going
to
be
hard
to
handle.
I
would
love
to
see
product
like
that.
Quite
frankly,.
A
A
Well,
we
do
provide
some
regularity
of
serving
the
better
question
was
whether
or
not
we
besides
our
actual
serving.
We
provide
restful
and
endpoints
and
serving
itself
TF
serving
at
least
uses
gr
PC
by
default.
But
there
is,
we
also
have
wooden
cube
flow
HTTP
proxy.
That
will
pretty
much
create
some
I'm,
not
sure
if
it's
proper
rest,
but
it
will
be
HTTP
based
so
kind
of
and
other
questions.
A
It's
an
open
season,
I
would
say
like
this
is
this
are
free
that
are
being
worked
on
right
now.
I
know
that
there
were
there's
some
annex
net
as
well
and
not
sure
if
they
I
quite
frankly,
don't
know
if
there
is
any
cold
published
yet,
but
I
know
that
we
talked
about
it.
Tensorflow
is
by
far
most
certainly
most
major,
and
it's
not
that
much
major
at
this
time,
but
if
you
want,
my
answer
would
be.
A
So
kvc
uses
compare
nightly
storage
for
kubernetes.
Okay,
we
see
right
now
is
a
kind
of
caching
mechanism.
So
the
main
the
main
use
case
of
KB
Hevesy
is
pretty
early,
so
the
use
case
are
still
TBD,
but
the
one
that
we
that
we
are
using
it
today
is
that
when
you
have
a
you're
here,
data
so
known
as
free,
like
object,
store
somewhere
and
accessing
those
data
is
actually
a
quite
a
bottleneck.
A
It's
there's
the
other
head
of
networking
that
thing
that
when
you
want
to,
for
example,
ping
for
every
every
image
in
the
training
set,
which
can
be
hundreds
of
thousands
of
images,
it
makes
to
the
training
much
much
longer.
So
what
KBC
does
it
pulls
these
images
and
saves
it
on
a
local
volume
which
then
you
mount
to
a
to
TF
top?
So
it's
a
it
uses
this
the
storage,
but
it
kinda
abstracts
the
storage
back-end
and
allows
it
to
be
more
local
to
the
training
job.
A
A
Pipeline
itself
is
quite
a
pro
quite
a
big
thing,
because
when
you,
for
example,
though
we
have
the
image
you
want
to
resize
it
and
recolor
it
and
do
the
processing
of
the
data,
and
you
want
to
save
those
images.
That's
also
gonna,
be
pachyderm
/kv,
see
problem
so
yeah
we
are
using
the
storage
storage,
but
little
more
would
it.
A
Yeah
I
mean
that's
gonna,
be
a
problem,
but
so,
when
you
see
start
my
server,
you
can,
for
example,
we
put
the
we
put
the
notes
for
the
scheduler
that
will
allow
that
will
schedule
it
on
GPU
notes,
but
the
problem
is
that
one
server
and
that
that
won't
solve
it.
The
problem
that
you
mentioned
it's
like
you
need
to
prune
your
prone
or
maybe
run
some
cron
job
that
will
kill
the
running
kernels
stuff
like
that,
but.