►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Kubeflow is an end-to-end platform for Machine Learning on Kubernetes, with the goal of making deployments of machine learning workflows simple, portable and scalable. It is gaining significant traction among data scientists and ML engineers and has outstanding community and industry support.
In this webinar we will showcase a simplified data science experience with advanced data management on Kubeflow. You will learn how to:
* Get started with MiniKF, a production-ready, full-fledged, local Kubeflow deployment that installs in minutes
* Easily execute an end-to-end Tensorflow example with Kubeflow Pipelines locally
* Learn about data versioning and reproducibility during Pipeline runs*
Run a complete Kubeflow Pipelines workflow without K8s-specific knowledge.
A
I'd
like
to
thank
everyone,
who's
joining
us
up,
your
slides
popped
up,
mm-hmm
all
right,
I
like
to
thank
everyone
who
is
joining
us
today.
Welcome
to
today's
CN
CF,
webinar,
scalable
ml
workflows,
with
advanced
data
management
on
cue
flow,
I'm,
Jorge,
Castro,
community
manager
at
VMware,
and
a
cloud
native
ambassador
I'll
be
moderating
today's
webinar.
We
would
like
to
welcome
our
presenter
today,
Vangelis
cookus
CTO
and
founder
at
a
rick
to--,
a
few
housekeeping
items.
Before
we
get
started
during
the
webinar.
A
You
will
not
be
able
to
talk
as
an
attendee
there's
a
Q&A
box
at
the
bottom
of
your
screen
there
in
the
zumasys
Q&A.
You
can
click
on
that
and
submit
your
questions
feel
free
to
drop
your
questions
in
there
throughout
the
the
webinar
and
we'll
get
to
as
many
as
we
can.
At
the
end.
This
is
an
official
webinar
of
the
CN
CF
and
it's
such
a
subject
to
the
CN
CF
code
of
conduct.
Please
do
not
add
anything
to
the
chat
or
questions
that
will
be
in
violation
of
that
code
of
conduct.
A
B
You
George
thank
you,
these
presentations
about
scalable
and
mail
workflows
with
advanced
data
management
on
coop
flow.
What
does
this
mean?
What's
the
problem?
We're
gonna
be
talking
about
setting
up
an
all-stock,
a
full
and
no
pipeline
is
very
hard.
I'm
doing
it
in
production
is
even
harder
and
then,
if
you
wanna,
do
multi
cloud,
then
this
is
when
the
difficulty
skyrockets
and
by
multi-cloud,
we
also
means
working
on
your
laptop
working
on
a
local
deployment.
B
If
someone
is
working
on
tram
in
an
on-prem
kubernetes
cluster
and
on
the
cloud,
then
it's
a
multi
cloud
deployment,
so
it
seems
a
communist
conception.
Is
that
to
build
on
a
new
product,
you
need
to
focus
on
ml
code.
You
need
to
write
your
code,
write
your
model
trainings
and
then
you're
done
unless
there's
some
little
details
around
actually
writing
ml
code
that
someone
who
take
care
of,
but
the
reality
is
that
it's
a
DevOps
problem
configuration
data
collection,
verification,
managing
resources,
managing
processes
serving
introduction
monitoring.
B
These
are
much
more
important
and
take
up
a
much
bigger
percentage
of
time
compared
to
actually
writing
ml.
So
we
need
a
platform
for
ml
and
this
is
where
coop
flow
comes
in.
So
why
could
flow?
Coop
flow?
Is
an
open
source
project
started
by
Google.
It's
an
end-to-end
solution
for
building
ml,
our
platforms
and
now
workflows
on
kubernetes.
It
aims
to
containerize
workloads
and
allowed
users
to
experiment
and
explore
with
state-of-the-art
AI.
It
focuses
on
easy
onboarding.
B
We
have
contributed
our
code
and
the
packaging
for
coop
flow,
called
mini
coop
flume
in
kf2
is
the
process
of
onboarding
new
users.
I'm
gonna
be
talking
more
about
this.
In
this
presentation
and
coop
flu
itself
has
outstanding
community
and
industry
support.
It's
a
vibrant
community.
Lots
of
comments,
lots
of
new
features-
the
code
is
involved,
is
evolving
very
fast,
so
what
is
meaning
could
flow,
or
this
mini
KF.
It's
a
packaging
contributed
by
us
a
victim
to
the
coop
for
community
dot
packages.
B
Mini
cube,
could
flow
and
rock
our
data
management
platform
in
a
single
VM
image.
So
you
can
spin
it
up
very
quickly
on
a
single
node,
presumably
a
laptop
or
a
very
big
desktop
machine
you
may
have.
It
does
require
quite
a
lot
of
memory,
because
come
flow
has
lots
of
components.
It
features
the
latest
console
version,
0.6
points,
X,
currently,
zero
point,
six
point:
two:
it's
the
we
think
fastest
and
easiest
way
to
spin
up
a
full
coop
flow
deployment
and
start
playing
within
minutes.
B
So
this
presentation
is
going
to
be
about
deploying
mini
coop
flow
and
then
running
a
well
known
example,
on
top
of
it
with
advanced
data
management
that
comes
from
rock.
So
how
do
you
install
mini
coop
flow?
There's,
a
previous
webinar.
We
did
so.
You
can
see
the
recording,
there's,
an
installation,
video
there's
the
documentation,
but
the
too
long
didn't
read
version.
Is
you
use
vagrant
to
initialize
a
new
virtual
machine?
B
I'm
shown
the
mini
cook
flow
landing
page,
so
I
start
navigating
this
very
nice
script
except
you
and
what
happens
now
is
there's
a
short
provisioning
phase,
but
essentially
manages
the
virtual
machine
provisions
mini
cook,
as
your
kubernetes
substrate
all
could
flow
components
sets
the
map
rock
so
that
you
can
have
storage
and
manage
the
storage
alongside
coop
flow
and
eventually,
when
the
whole
platform
is
up,
it
gives
you
easy
to
use
links,
so
you
can
access
the
coop
flow
dashboard
and
rock.
This
will
take
a
few
minutes.
B
It
takes
in
the
order
of
five
to
ten
minutes,
depending
on
the
speed
of
your
machine
and
your
CPU
utilization,
so
switch
back
to
the
presentation
and
come
back
and
see
what
happened.
So
this
is
essentially
the
process
of
starting
mini
coop
flow.
You
initialize
a
virtual
machine.
Then
you
start
this.
Is
it
let's
focus
on
what
could
flow
does?
This
is
a
page
from
a
well-known
Google
paper
on
the
effects,
so
the
effects
focuses
on
building
and
a
mail
workflow.
B
We
in
steps,
Batum,
Alex's
transformation,
validation,
training,
the
model
tuning,
the
training
of
the
model,
with
hyper
parameters,
evaluating
the
model
and
then
should
be
serving
it.
So
coop
flow
comes
with
these
libraries.
He
containerized
components,
so
you
can
use
them
as
they
are
and
it
also
comes
with
a
hyper
parameter.
Tuner
containerized
that
can
run
multiple
trials.
Ok,
then,
you
need
an
integrated
front-end
for
job
management.
Monitoring,
debugging
data
scientists
like
notebooks
coop
flow,
comes
with
a
dashboard
and
a
notebook
based
UI
jupiter
notebooks.
B
B
We
run
the
pipeline
when
you
have
a
model,
that's
the
output
of
the
pipeline,
go
back
in
time
and
see
exactly
what
you
used
as
input
to
train
the
model.
To
do
this,
that's
where
we
come
in,
so
we
integrate
ROC
our
data
management
component
with
kubernetes
and
coop
flow,
to
give
you
an
end-to-end
platform
for
producible
machine
learning.
This
is
what
we'll
be
talking
about
today.
B
If
we
go
back
to
deployment,
we
deploying
the
database
waiting
for
it
to
come
up.
It's
progressing
so
moving
on
what
exactly
does
rock
do
when
you
run
on
a
male
workflow
different
steps
of
the
workflow
usually
happen
at
different
places.
This
is
the
ideal
scenario
data
scientists
like
to
experiment
on
their
laptops.
On-Prem
locally,
but
for
training
you
usually
need
to
move
to
a
bigger
deployment
with
GPUs,
for
example
the
Google
cloud,
or
you
can
have
spin
up,
GPI
enabled
instances
and
production
actually
serving
the
model
happens
elsewhere.
Sometimes
it
happens
at
multiple
locations.
B
It
happens
in
self-driving
cars,
it
happens
at
IOT
sensors.
It
happens
at
many
different
places.
So
how
do
you
move
the
process
from
experimentation
to
training,
to
production
and
keep
it
reproducible?
First
part.
We
contributed
code
to
coop
flow
to
make
it
data
we're
in
general.
This
means
we
extend
coop
flow
components,
so
they
use
persistent
volume
claims
provided
by
kubernetes,
so
they
store
the
data
in
volumes
they
vendor
agnostic
volumes
provided
ocular
notice.
B
So
when
could
flow
stores
its
data,
this
data
is
managed
by
rock
and
then
rock
forms
a
peer-to-peer
network
among
go
locations
and
allows
essentially
to
keep
your
to
start
from
your
laptop
keep
your
data
locally,
but
when
the
time
comes
for
training,
take
a
snapshot
of
this
data,
move
it
transparently
to
the
training
location,
run,
training,
training
would
produce
a
model,
take
the
model
and
synchronize
it
will
load
with
all
the
serving
locations
and
because
this
is
snapshot
based,
everything
is
immutable.
You
find
similarities
with
get
for
it.
There's
a
live
demo
later
on.
B
B
So,
what's
new
in
the
latest
mini
coop
flow
based
on
the
latest
couplet
version,
it
uses
coop
flow,
zero
point
six
point:
two:
it
enables
multi
user
authentication,
so
you
can
have
multiple
users
spinning
up
their
notebooks
in
an
isolated
way.
So
there's
authorization
for
notebooks
as
well.
If
it's
integrated
with
Roc,
then
you
can
do
nearest
and
10-years
restore
of
snapshots.
So
when
you
have
a
notebook
server,
you
can
snap
shut
it
shut
it
down.
B
Then
come
back
one
or
two
days
or
one
on
two
weeks
or
one
two
years
later
and
spin
it
up
almost
instantly.
We
have
significantly
improved
the
time
to
perform
the
snapshot
and
we
also
have
the
ability
to
snapshot
every
step
of
a
running
pipeline,
exactly
for
being
able
to
go
back
in
time
and
see
the
input
to
each
pipeline
step.
B
Why
do
that
and
let's
see
how
the
deployment
progresses
so
provisioning
I
switch
back
to
the
minikin
flow
desktop
provisioning
of
coop
flow
is
almost
there.
We
are
now
waiting
for
coops
of
resources
to
come
up.
Rock
is
already
up
when
coop
flow
is
up
and
running.
This
button
will
also
turn
green
and
then
I'll
be
able
to
connect.
So
what
I
did
is
I
started
a
virtual
machine,
initialize,
provisioning
and
I'm
waiting
for
the
30
time.
Pads
of
coops
all
to
come
up,
coop
flow
does
have
a
lot
of
components.
B
Why
did
we
build
mini
coop
flow,
because
people
need
an
easy
way
for
onboarding
and
people
like
to
experiment
locally
so
by
packaging
it
for
a
single
node
deployment
inside
the
virtual
machine?
We
give
a
very
easy
way
to
spin
up
good
flow
and
start
experimenting
with
it,
but
because
it's
still
kubernetes
and
it
still
coop.
So
you
have
the
exact
same
user
experience,
so
you
can
use
the
same
API
as
you
can
submit
the
same
kubernetes
resources.
B
B
So
our
goal
was
a
unified
user
experience.
If
you're
on
the
cloud,
if
you're
on
Prem,
if
you're
on
your
laptop,
you
run
kubernetes,
you
deploy
code
flow
on
top,
you
deploy
rock
for
managing
your
volumes,
your
storage,
and
this
gives
you
the
ability
to
use
the
same
API
and
find
your
data
where
you
need
them.
B
Please
try
it
out.
Let
us
know
how
it
goes.
Give
us
feedback
on
the
coop
flow
slack
initialize
the
machine
start
machine.
This
is
it.
The
requirements
are
kind
of
shifting.
It
means
at
least
till
gigabytes
of
RAM
to
get
the
virtual
machine
up
and
running
coop.
So
is
quite
resource
hungry.
You
need
a
bit
more
if
you
want
to
run
ml
training
depending
on
the
size
of
your
data
sets
and
because
it's
virtual
machine-based,
it
will
run
on
Linux,
Mac,
OS
or
Windows.
B
In
the
same
way,
it
uses
vagrant
for
managing
the
virtual
machine
and
VirtualBox
for
actually
running
the
virtual
machine.
So
let's
move
I
like
how
this
synchronized,
so
provisioning
is
complete.
Coop
flows
up
rock
is
up.
I
can
connect
to
it.
I'm
logging
into
coop
flow
using
a
standard
IP
address
accessible
only
by
the
local
host,
so
I
can
log
into
it.
This
is
the
authentication
part
I
mentioned,
and
this
is
the
crucial
dashboard,
so
I'm
logged
in
in
my
own
namespace
could
flow
user
I'm.
B
The
owner
of
this
namespace
Mina
cool
flow,
has
a
single
user
could
flow
in
general.
In
enterprise
deployment
supports
more
than
one
users.
I
can
access
my
pipelines,
my
notebook
servers,
chapter
notebooks,
hyper
parameter,
tuning
metadata
and
rock
as
the
snapshot
store.
So
what
are
we
going
to
use
mini
control
for
we're
going
to
use
it
to
run
a
live
demo
of
the
Chicago
taxicab
example?
This
is
a
tf-x
example
unplan
on
our
laptop
with
Mini
Cooper
flow.
B
What
exactly
is
the
Chicago
taxicab
example?
The
city
of
Chicago
produced
a
dataset
of
more
than
100
million
trips.
You
can
download
the
original
data
set
here.
This
data
set
recorded
a
few
quite
a
few
details
about
each
individual
trip,
including
how
much
money
the
trip
cost
when
it
started
where
it
started
where
it
ended.
B
How
long
it
was
what
means
of
payment
the
rider
used
and
whether
there
was
anything
and
the
result
of
this
example
is
a
trained
model
that
can
predict
whether
a
random
trip
will
result
in
a
tit,
that's
more
or
less
20%
of
the
fair.
So
we
get
this
data
set.
We
feed
it
to
a
model
and
build
a
classifier
that
can
decide
whether
a
random
trip
produces
a
tip,
that's
more
or
less
than
T
percent.
This
is
going
to
be
the
workflow
that
we'll
be
demoing.
B
Okay.
This
is
a
summary
of
the
input,
features
essentially
start
and
end
location
means
of
payment
and
the
amount
paid,
and
what
is
our
demo
going
to
be?
We
start
from
a
notebook
will
use
coop
flow
to
create
this
notebook
will
create
a
new
volume
as
storage
to
hold
our
data,
we'll
bring
in
our
data
set
from
an
external
data
source.
We
could
download
the
full
data
set
from
the
city
of
Chicago
side.
B
All
will
download
a
subset
of
this
data
set,
so
we
can
run
the
example
at
completion
from
github
once
we've
done
that
we
will
take.
This
is
ingestion
of
data,
we'll
take
a
snapshot
of
our
notebook
volume.
Why
do
that?
So?
We
have
a
data
community,
so
we
know
exactly
where
we
started
from,
so
we
can
rerun
the
pipeline
with
exactly
the
same
into
beta.
So
we
know
that
whatever
the
result
of
the
pipeline
is.
This
is
where
it
started
from.
At
this
point,
we
can
continue
working
with
this
volume.
B
We
can
continue
exploring
things
in
the
notebook
and
the
pipeline
is
not
going
to
be
affected
because
the
pipeline
will
run
from
this
snapshot
will
run
from
a
clone
of
the
snapshot.
This
is
going
to
be
the
starting
point
for
the
pipeline,
so
then
we
will
run.
The
pipeline
will
run
a
series
of
pre-processing
training,
analysis
and
eventually
serving
steps,
and
we
will
also
snapshot
a
step
in
the
pipeline,
the
final
step,
so
we
can
keep
it
and
explore
it
by
cloning,
it
sometime
in
the
future
into
a
new
notebook.
B
So
this
shows
that
we
can
take
any
individual
step
of
the
pipeline
snapshot
it
and
explore
it
again
using
notebooks
and
this
process
can
start
all
over
again
I
explore
the
data,
make
some
fixes.
Ingests
new
data
adjust
my
algorithm
adjust
parameters,
snapshot
again,
move
the
new
pipeline
start
over
I'm,
going
to
have
something
that
works
or
gonna
have
something
that
doesn't
work.
I
can
go
back
in
time
and
see
exactly
what
I
started
from
essentially
we're
using
rock
as
good
for
our
data.
B
B
B
And
launch
the
notebook
so
at
this
point,
I'm
launching
a
notebook,
a
notebook
server
using
Jupiter
notebooks
from
a
custom
image.
It
might
take
some
time.
Oh
okay.
This
was
fast
to
actually
download
the
custom
image.
The
notebook
is
up
it's
running
as
a
part
in
sizeable
matters.
If
I
just
could
cuddle
would
be
able
to
see
the
pod
running
and
I
can
connect
it.
B
B
B
B
It's
not
mandatory
to
take
a
snapshot
in
the
pipeline,
but
it's
best
to
take
a
snapshot
so
that
you
don't
have
the
notebook
operation
messing
up
what
the
pipeline
sees.
So
if
it
was
just
a
shared
directory
and
you
had
the
execution,
that's
a
very
good
question
by
the
way
and
had
the
execution
of
your
pipeline
actually
messing
up
the
same
volume,
the
same
storage
that
the
notebook
or
notebooks
were
seeing.
B
Then
you'd
have
your
work
or
your
colleagues
work
messing
up
the
result
of
the
pipeline,
so
you
can't
just
use
an
NFS
share
because
it's
like
working
the
same
get
directory.
If
you
have
more
than
one
person
right,
what
happens?
Is
you
can't
be
sure
about
the
result
of
the
execution?
The
result
will
be
essentially
be
random,
as
P+
are
changing
the
input
data
while
working
with
git,
you
can
say,
I
started
from
biscuit
commit
so
I
can
reproduce
the
bug.
I
can
reproduce
the
result,
no
matter
what?
B
Because
the
community
is
immutable,
it
never
changes.
It's
the
same
thing
with
a
snapshot
if
you've
started
from
a
snapshot,
no
matter
what
your
college
or
you
do
in
a
notebook,
it
won't
affect
the
execution
of
a
pipeline.
So
this
is
why
we
focus
on
working
with
snapshots
for
having
the
producing
one
now.
I
hope
this
answers
the
question.
So
the
pod
is
shown
as
running,
but
let's
not
believe
it.
Let's
actually
go
see
the
pods.
B
B
So
for
this
I'm
notebook
upon
years,
so
I
won't
have
to
type
two
main
commands,
so
this
is
a
notebook
and
what
it
does
is
it
has
the
commands
to
run
individual
steps
embedded
in
cells.
So
what
I've
done
so
far
is
I've
created
a
notebook
and
added
a
data
volume
I'll
now
ingest
data
from
an
external
source
so
ingesting
the
data
will
be
first
downloading,
the
actual
source
of
the
of
the
pipeline.
This
is
the
pipeline.
We
will
be
running
later
on
for
training.
B
So
I'm
cloning,
the
git
repository
and
then
copying
the
subset
of
the
data
I
need
into
my
data
directory.
This
is
essentially
simulating
ingest.
The
data
from
an
outside
data,
Lake
HDFS
snowflake
as
three
the
city
of
Chicago,
sighs.
Wherever
I
find
my
data
assembling
it
all
into
my
storage
and
then
I
have
a
reproducible
snapshot
of
this
storage.
So
I
can
start
changing
so
I
copied
the
data
into
my
data.
For
me
here,
I
can
actually
see
the
data,
that's
the
data
and
what
we'll
do
now
is
move
back.
The
presentation,
I.
B
Have
ingested
the
data
I
will
compile
the
Cupra
pipeline
I'll,
show
you
the
code
and
compile
it.
This
is
essentially
a
description
of
the
steps
that
are
going
to
take
place
and
then
I'm
going
to
snap
shut,
the
notebook
so
compiling
that
I
plan
create
a
parable,
we're
using
a
newer
version
of
coops
of
pipelines,
which
produces
a
few
warnings
and
let's
take
a
look
at
the
pipeline
itself.
So
this
is
the
pipeline
specification.
The
training
pipeline
that
we're
going
to
use
in
a
domain-specific
language
specified
by
coop
flow
pipe
lines.
B
So
this
defines
individual
steps
of
the
pipeline
as
components,
containerized
components,
mentioning
specific
container
images,
taking
arguments
and
producing
outputs
that
live
in
volume.
So
by
snapshot
in
individual
volumes,
we
can
snapshot
the
state
of
the
pipeline
and
weeks
that
at
each
step-
and
why
is
this
important,
because
the
most
difficult
thing
when
running
a
pipeline
is
essentially
having
inside
object
at
its
execution?
What
happens
at
the
pipeline
fails?
What
happens
if
a
step
fails
if
I'm
in
a
notebook
I
can
experiment
written
if
a
five-man
step
fails?
How
can
I
go
see?
B
What
happened
resist
ways,
take
a
snapshot
of
its
input,
clone
it
in
a
new
notebook.
I'm
gonna
see
exactly
what
happened.
I
can
actually
run
the
exact
command
that
he'll
be
used
to
produce
the
data
and
see
what
fails.
So
at
this
point,
I
have
my
input.
Data
here,
I
have
compiled
my
pipeline,
so
I
can
now
snapshot
my
notebook
for
this
I
need
to
rock
there's
a
bucket
here.
Bad
casing
is
where
you
store
your
files,
you
have
snapshots.
I
can
create
a
moon.
Let's
call
it.
B
B
This
lab
this
notebook
server,
I've
named
mine,
Mike,
made
initial,
commit
and
described
it
as
another
chest
and
jested
data
compiled
pipeline.
This
will
be
the
input
through
the
pipeline
run
under
snap
ship
it.
So
what
happens
is
a
new
a
synchronous
task
starts.
This
is
a
group
of
type
of
tasks.
Actually,
one
task
is
the
snapshot.
The
data.
The
other
task
is
the
snapshot,
the
workspace.
Why
do
we
snapshot
both
volumes,
both
disks,
so
we
can
be
reproducible.
B
You
have
seen
multiple
times
that
the
result
of
execution
is
not
just
a
function
linked
data.
It's
a
function
of
the
lab
edges
as
well.
First
birds
suddenly
appear
with
such
levels
as
behaviors,
desired
or
undesired
that
only
appearance
elaborate
so
to
be
reproducible
able
to
allow
our
colleague
to
see
exactly
what
you
did.
You
need
to
snapshot
both
your
home
folder,
your
workspace,
where
you
store
your
libraries,
but
you
pick
install
and
your
data
and
that's
what
we
do
here
in
sq
seconds.
So
I
now
have
a
snapshot
that
contains
snapshots
of
both
disks.
B
B
B
Snapshot
it
my
notebook
and
created
a
new
snapshot
at
this
point,
I'm
ready
to
create
a
new
pipeline.
So
what
do
I
do?
I
go
back
to
the
node
I'd
go
back
a
good
pipe
lines.
Actually,
this
is
central.
Dashboard
pipelines,
I
have
a
few
sample
pipelines.
I
need
to
upload
the
compiled
pipeline
created
here
to
upload
this
I
first
downloaded
to
my
box.
B
B
This
is
it
so
now
we'll
be
creating
a
run
of
this
pipeline
from
the
pipeline.
These
are
all
the
pipeline.
Steps
first
step
is
to
create
a
volume.
Why
do
we
need
to
create
a
volume,
so
we
can
have
a
clone
of
the
snapshot
we
start
with,
and
every
other
step
depends
on
this
volume,
because
every
other
step
manages
this
volume
and
creates
files
in
this
folder.
Then
we'll
validate
the
data.
Pre-Process
training
deploy,
predict
analyze
and
we
have
a
final
snap
chat
volume
step.
B
B
B
B
Massage
data,
for
example,
now
create
a
new
file.
This
new
file
will
not
be
part
of
the
pipeline.
The
fact
that
I'm
now
changing
the
data
does
not
have
an
impact
to
the
execution
on
the
execution
of
the
pipeline,
because
the
pipeline
runs
from
a
clone
of
this
data.
Now
this
answer
is
we
print
somebody's
question
exam?
So,
let's
touch
this
table.
This
will
be
not
up
here
at
the
end.
B
Okay,
where
were
we?
The
pipeline
is
running.
The
validation
step
is
completely
processing
run.
So,
let's
take
a
look
at
the
steps.
Eventually,
the
pipeline
will
snapshot
a
step
into
a
new
snapshot
and
we'll
clone
it.
If
we
have
the
time
so
the
steps
are
validating
the
data
using
TF
DV.
This
is
at
the
effects
library
this
step
has
already
completed.
It
runs
from
a
clone
of
the
notebook
data.
Then
we
transform
the
data
in
a
pre-processing
step.
B
We
transform
both
the
training
part
of
the
data
set
and
the
evaluation
part
of
the
data
set
the
input
that
we
have
kept.
So
we
can
then
evaluate
the
model,
then
comes
training
which
will
lead
to
a
train
model
which
will
also
be
stored
inside
the
volume
that
sees
this
progression.
The
processing
is
done.
We
now
run
training.
B
B
First,
we
created
the
jupiter
hub
based
spawners,
with
persistent
volumes
for
0.4.
We
replaced
it
with
an
eighth
dimension
of
the
native
node
manager
that
you
saw
in
0.5,
and
we
have
also
expanded
the
pipeline's
domain-specific
language,
so
it
can
access
persistent
problems
and
snapshots.
If
you
look
at
the
pipeline,
I
can
actually
show
this.
B
Every
step
is
losing
the
volume.
This
is
the
volume,
so
we
created
extensions
to
the
pipeline's
domain-specific
language,
so
we
can
create
volumes
as
clones
of
existing
snapshots
and
then
we
can
use
these
volumes
for
each
individual
step.
So
each
individual
step
uses
this
volume
and
eventually
will
also
produce
a
snapshot
to
define
this
as
well.
So
we
have
extended
the
DSL,
so
you
can
also
produce
snapshots
of
this
volume
with
dependencies.
So
when
the
final
steps
of
the
pipeline
are
done,
a
snapshot
of
this
volume
will
be
produced.
B
B
While
he's
rock
for
your
pipeline
step,
one
starts
from
a
notebook.
You
snapshot
it.
You
start
small
snapshot,
your
cloning.
You
make
some
changes
in
snapshot.
It
step.
Two
starts
from
a
clone
of
step.
One
make
some
changes:
it's
not
shut
it
again,
step
three
and
so
on.
If
this
breaks
you
can
take,
this
snapshot
examine
it
in
a
notebook
and
start
over
hybrid
pipelines.
B
You
have
a
pipeline
that
you
want
to
run
in
two
locations.
You
want
to
run
a
few
steps
in
location
one.
Then
you
want
to
run
serving
or
training
in
location,
two
step,
one
two
and
three
run
in
location,
one.
This
snapshot
step
three.
You
move
the
snapshot
using
rock
to
another
location,
and
at
this
point
you
can
run
the
second
half
of
the
pipeline
on
location
two.
So
this
is
a
hybrid
pipeline.
B
B
B
B
B
B
That's
a
good
question:
we
didn't
cover
it
in
this
presentation.
We've
made
it
as
easy
as
possible.
Using
rock
you
essentially
publish
a
link
to
a
published
bucket
to
another
service.
We
call
the
registry
and
then
any
other
user
can
subscribe
to
this
link
and
follow
your
changes
as
they're
happening.
What
we
mean
by
this
is
you.
B
Go
to
your
buckets,
you
publish
them.
The
dissention
gives
you
a
link,
you
can
share
with
others,
and
others
can
then
create
a
subscribed
bucket
and
follow
your
changes.
So
whenever
your
pipeline
runs,
engine
creates
a
new
snapshot
or
wherever
or
whenever
you
create
a
new
snapshot
of
your
pipeline
or
of
your
notebook
and
another
follower
sees
it.
So
this
is
how
we
share
snapshots
across
locations.
A
B
A
good
question
right
now
mini
coup
flow
does
not
support
Purvi.
We
are
thinking
about
using
it.
We
are
focusing
our
efforts
to
moving
coop
flow,
so
it
can
run
on
public
clouds
as
well.
So,
yes,
hyper-v
is
an
option,
but
we're
focusing
on
efforts
to
giving
mini
coop
flow
as
a
packaged
product
on
a
public
cloud.
So
you
can
run
welcome
to
worry
about
your
lamp
ram
or
a
few
limitations,
but
this
is
feasible.
Yes,
okay,.
B
That's
a
good
question:
it's
low
has
authentication
authorization
is
not
there,
yet
it
is
there
for
notebooks,
it's
not
there.
Yet
for
pipelines
for
hybrid
parameter
tuning
for
metadata
handling,
we
are
contributing
actively
to
multi-user
pipelines
right
now
and
we
hope
to
have
it
as
part
of
the
next
arkansas
version.
Please
come
in
touch
and
come
flow
slack
with
us,
and
we
can
talk
more
and
maybe
even
also
test
on
initial
implementation
that
we
presented
today
at
the
community
meeting.
That's
what
we
doing
right
now
to
be
enterprise,
ready
with
full
support
for
authorization.
A
B
A
good
question
Rock
manages
whatever
primary
storage
she
provided
with.
This
demo
was
using
local
disks,
so
there
is
no
cinder
or
another
API
underneath
we
just
gave
local
disks
on
each
individual
node
to
be
managed
by
rock
rock
cars,
individual
volumes
on
set
disks,
and
you
can
be
as
fast
as
possible
and
have
super
long
latency.
Because
of
this
you
just
work
over
local
disks.
You
can
work
over
local
nvme,
that's
what
we
are
targeting
and
you
can
still
big
stand,
node
failure,
because
you
can
be
snapshot
in
your
notebooks
once
every
five
minutes.
B
B
A
B
Not
sure
I
understand
these
questions,
I
think
you
mean
what
will
happen
if
I
mounted
my
object,
storage
as
a
volume
in
the
notebook.
That's
what
I
understand
that?
Maybe
I
could
use
a
user
space
mounting
mechanism
to
access
history,
for
example
a
mystery
bucket
as
a
volume
in
my
notebook.
This
has
two
main
issues:
one
is
you
lose
reproducibility
because,
while
you're,
it's
like
using
a
shared
NFS
share,
while
you're
making
changes
in
the
bucket,
you
don't
know
what
pipeline
executions
are
actually
using
the
bucket
right
now
to
read
data
from
it.
B
So
you
cannot.
If
you
want
to
perform
a
and
I,
don't
know
an
exploration
and
run
on
experimental
transform
on
the
data.
You
can't
do
it
because
you
don't
know
who
else
is
using
the
data
at
this
point
in
time
you
actually
have
to
copy
all
of
the
data
to
make
sure
you're
working
with
a
project
code,
but
then
this
becomes
too
expensive.
B
So
this
is
one
thing
and
the
other
thing
is:
if
you
compare
your
performance
and
latencies
when
working
with
a
local
volume
compared
to
working
with
s3
mounted
over
a
notebook
performance,
difference
is
abysmal,
the
latency
is
skywalker.
We
have
measured
in
the
order
of
800
megabytes,
a
second
bandwidth
and
microsecond
latencies,
because
we
work
with
local
MDM
ease
with
an
internal
data
path.
This
is
orders
of
magnitude
faster
than
going
to
an
object
store
every
time.
B
Hours
rock-rock
exposes
itself
as
a
storage
class
and
we
provision
volumes
using
de
storage
class
and
rock
underneath
carves
up
volumes.
As
you,
four
on
creme
Rock
is
deployed,
as
a
demon
set.
It
has
its
own
operator.
You
spin
up
a
custom
CR
a
custom.
My
resource
I
think
that's.
We've
done
well
Rock
cluster
yep.
B
B
A
B
A
B
Not
sure
I
understand
this
question.
We
we've
tested
this
with
a
few
hundred
gigabytes
of
data.
We
just
carve
up
local
disk
and
then
we've
made
lots
of
optimizations
to
make
sure
we
only
touch
the
changed
box.
So
when
you
have
a
very
big
data
set
and
only
change
a
few
parts
of
it,
you
can
still
snap
shut
it
very
efficiently.
A
B
Assume
it's
it
can
as
long
as
you
can
mount
a
persistent
volume
as
you
as
long
as
you
can
access
a
file
system
from
said
jobs
then
can
be
used
with
these
solutions,
but
please
come
in
touch
with
us
in
the
cook
flow
slack,
and
then
we
can
talk
more
about
it
because
we're
interested
in
various
cases.
Please
join
the
coop
flow
slack.
There's
a
link
to
join
the
coop,
so
slack
on
coop
flowed
org.
Please
follow
the
link.
B
B
A
Are
done
with
the
questions
that
and
that's
all
that
quite
that
questions
that
we
have
time
for
today,
sorry
and
thanks
for
joining
us.
The
webinar,
recording
and
slides
will
be
online
later
on
today
and
we're
looking
forward
to
seeing
you
at
a
future
CN
CF
webinar,
and
with
that.
Thank
you,
everybody
and
have
a
nice
day.