►
From YouTube: Tutorial: From Notebook to Kubeflow Pipel... Michelle Casbon, Stefano Fioravanzo & Ilias Katsakioris
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Tutorial: From Notebook to Kubeflow Pipelines: An End-to-End Data Science Workflow - Michelle Casbon, Google, Stefano Fioravanzo, Fondazione Bruno Kessler, & Ilias Katsakioris, Arrikto (Limited Available Seating; First-Come, First-Served Basis)
Please bring your laptop fully charged as we will have limited charging stations available in the room. This session targets data scientists and ML engineers who want to leverage Kubernetes to scale up their Machine Learning experiments. Attendees will learn a) the basics of Kubeflow, the ML toolkit for K8s, and b) how to build and deploy complex data science pipelines on-prem and on the Cloud with Kubeflow Pipelines. The tutorial will focus on two essential aspects: 1. Low barrier to entry: deploy a Jupyter Notebook to Kubeflow Pipelines on the cloud using a fully GUI-based approach. This workflow enables data scientists to exploit the scaling potential of K8s - no CLI commands, SDKs, or K8s knowledge required. 2. Reproducibility: automatic data versioning and volume snapshots will enable full reproducibility and collaborative development, as well as fine grained analysis and visualizations after pipeline executions. Setup: must bring own laptop. Qwiklab/GCP credits will be provided
https://sched.co/Uaeq
A
Thank
you
all
very
much
for
coming.
We
really
appreciate
you
sticking
around
and
coming
on
the
last
day
of
the
conference,
so
this
is
going
to
be
a
tutorial
from
going
about
how
you
use
coop
flow,
to
go
from
a
notebook
to
a
pipeline
to
do
end-to-end
data
science,
so
I'm
here.
Just
to
give
you
a
brief
overview,
really
pretty
sure
you
everyone
coming
out
to
do
this,
so
my
name
is
Jeremy
I'm.
One
of
the
tech
leads
for
coop
flow
at
Google.
I
will
let
my
co
speakers
introduce
themselves.
C
A
So
I'm
just
gonna,
kick
it
off
with
a
couple
quick
minutes,
providing
a
brief
overview
of
coop
flow
and
then
I'll
hand
it
back
to
Stefano
Ilyas
to
go
into
the
technologies
that
we're
gonna
be
using
today
and
then
go
through
the
code
lab.
So
what
is
coop
flow.
Coop
flow
is
a
open
source
project.
That's
dedicated
to
making
it
easy
to
deploy
machine
learning
on
kubernetes
right,
so
our
model
is
to
make
it
simple,
portable
and
scalable,
so
this
slide
is
meant
to
indicate
why
we
started
building
coop
flow.
A
So
what
we
saw
about
a
couple
years
ago
was
that
if
you
looked
out
in
the
world-
and
you
looked
at
what
were
the
companies
that
were
being
successful
with
machine
learning
all
of
the
companies
that
were
at
the
cutting
edge
of
machine
learning,
we're
really
talking
about
the
platform's,
they
were
building
to
do
machine
learning
and
how
critical
that
was
to
be
successful.
So
Google
was
one
of
the
first.
A
They
talked
about
their
internal
and
now
external
machine
learning
platform
called
tf-x,
but
other
companies
like
lyft
and
Bloomberg
were
often
talking
about
their
platforms
at
coop
con.
In
other
places,
because
they
were
built
on
kubernetes,
and
so
what
we
saw
with
a
lot
of
these
platforms,
particularly
from
like
lyft
and
Bloomberg,
is
that
they
were
building
on
top
of
kubernetes.
A
So
that's
what
became
coop
flow,
and
so
briefly,
the
motivation
is
that
you
know
ml
applications
are
distributed
systems.
So
a
lot
of
people
tend
to
think
about
ml
as
being
about
complex
algorithms
and
really
difficult
math.
But
if
you
actually
look
at
ML
systems
that
are
in
production,
you
have
all
of
these
services
and
systems
that
go
around
and
are
there
to
support
the
machine
learning
algorithm.
So
you
have
services
for
doing
data
collection,
you
have
serving
infrastructure,
resource
management,
monitoring,
etc,
and
most
of
your
time
is
actually
spent
in
these
surrounding
systems.
A
That
really
are
more
about
DevOps
and
less
about
machine
learning,
and
so
that's
really
the
challenge
of
getting
the
applications
into
production,
and
so
one
of
the
ways
that
we've
learned
in
the
industry
and
at
Google
to
deal
with
this
complexity
of
distributed
systems
is
that
continuous
integration
and
continuous
delivery
is
critical.
So
really
you
know
the
sort
of
the
motto
is
you
know?
A
If
you
push
on
green,
so
to
speak,
you
can
push
early,
you
can
push
more
often
and
what
you,
what
happens
is
that
you
actually
have
fewer
problems
and
those
problems
tend
to
be
less
severe.
I
put
some
a
couple
of
quick
citations
on
the
docs.
You
can
go
in
the
slides.
You
can
go.
Look
at
them.
There's
some
papers
to
actually
bub
sort
of
back
this
up
with
some
research
and
some
papers
from
Google
and
other
people,
and
so
the
key
idea
is
that
you
know
we
have
four
traditional
DevOps
and
application
deployment.
A
We
have
a
really
good
cycle
where
we
can
develop
push
build
test
and
deploy
our
applications,
and
so
we
really
know
how
to
iterate
quickly
and
push
software
into
production,
and
that's
really
been
critical
to
getting
dealing
with
the
complexity
of
dealing
with
distributed
systems.
So
we
want
to
do
the
exact
same
thing
for
CI
CD
for
ml,
but
we
have
a
couple.
A
And
then,
finally,
we
have
to
figure
out
when
we
have
a
mount
model
deployed.
How
do
we
go
about
actually
validating
that
model
and
seeing
whether
it's
actually
improving
the
predictions
or
quality
of
our
model
and
validating
that
we're
actually
solving
some
important
business
problem?
And
so
the
cuj
that
we're
going
to
go
through
today
so
see
you
G
stands
for
critical
user
journey
and
using
the
technologies
of
coop
flow
is
we're
gonna
try
to
close
this
loop,
so
we're
gonna
try
to
develop
a
model
in
a
notebook.
A
We're
gonna
then
use
kale,
which
is
a
project
Stefano
developed,
which
makes
it
easy
to
convert
a
notebook
into
a
pipeline.
We're
then
going
to
run
this
pipeline
using
couplet
pipelines
and
then
finally,
we're
going
to
take
advantage
of
rook
to
validate
our
model
and
investigate
how
well
it's
doing
so
that
we
can
figure
out
how
we
improve
our
model
to
actually
get
it
make
further
improvements
on
the
problem
we're
trying
to
solve
and
with
that
I'm
gonna
hand
it
off
to
Stefano.
B
Thank
You
Jeremy,
I'm,
gonna,
just
reiterate,
or
over
what
Jeremy
said.
So
since
this
is
a
data,
science
focused
tutorial,
we're
going
to
use
cue
flow
pipelines
and
I'm
not
going
to
details
of
exactly
what
exactly
is
queue
for
pipelines.
But
all
you
need
to
know
is
that
it
is
a
general
platform
for
building
composable
workflows
that
are
based
on
docker
containers
and
it
comes
both
with
a
UI
for
tracking
job's
experiments
runs
and
the
Python
SDK
for
building
components
and
pipelines.
B
Now,
first,
we
are
going
to
see
how
you
are
going
to
learn
how
you
can
deploy
a
single
jupiter
notebook
as
composable
pipelines
without
using
this
Python
SDK,
but
just
a
graphical
user
interface
and
then
you're
going
to
learn
how
to
have
complete
reproducible
pipelines
using
data
versioning
and
volume
snapshots
and
just
as
Jeremy
said,
the
first
point
is
tackle
using
kale
and
then
Ilyas
will
introduce
you
to.
The
second
part
is
in
our
rictus
data
management
management.
B
Software
block
I'm,
talking
about
deploying
it
Jupiter
notebook
has
that
composable
pipelines
because
having
pipelines
instead
of
say,
deploying
the
notebook
in
a
single
instance.
Vm
has
many
benefits
like
clearly
defining
the
workflow
and
every
step
so
that
every
step
is
clearly
isolated
and
you
can
like
or
analyze
different
steps
around
hyper
parameter
tuning
or
develop
more
collaboratively
on
different
steps
at
the
same
time,
and
also
it
becomes
easier
to
use
data
versioning
with
Rocco
with
any
other
software
and
having
a
composable
pipeline.
B
Let
you
run
single
steps
on
different
hardware
like
CPU,
VMs
or
GPU
VMs,
based
on
the
environment
requirements
now
without
using
kale
and
rock.
This
is
what
a
typical
workflow
for
developing
pipelines
on
cue
flow
might
look
like
so
first
you
write
your
usual
machine-learning
code.
Then
you
would
have
to
build
docker
images
for
it
either
one
for
every
step
or
multiple
docker
images,
based
on
the
on
the
requirements
of
the
components
that
compose
the
workflow.
B
After
that,
you
would
need
to
actually
define
the
workflow
using
the
Python
SDK,
then
compile
it
and
upload
it
to
keep
the
pipeline
amending
the
code
would
mean
to
go
back
to
the
creation
of
docker
images.
If
you
have
new
libraries,
new
requirements
so
build
new
docker
images,
push
them
recreate
the
workflow,
push
it
to
keep
the
pipelines
running,
etc.
With
scaling
rock.
It
becomes
very,
very
simple,
because
you
just
have
a
notebook.
You
write
your
code
there.
B
You
can
run
it
locally
and
then,
with
kale
UI
that
I'm
going
to
show
you
very
very
soon.
You
will
be
able
to
tag
the
notebook
cells
to
compose,
to
create
this
workflow
and
then,
with
the
click
of
a
button,
just
push
it
to
give
the
pipelines
whenever
you
need
to
amend
the
code.
Just
go
back
to
the
notebook
change
some
code
and
then
deployment
is
seamless
all
right,
so
we
can
start
with
the
handsome
part.
We
have
prepared
this
code
lab
with
all
the
instructions
you
will
need
to
follow.
B
C
C
We
will
be
deploying
immunity
F
on
GCP.
That
is
a
virtual
machine.
So
we
at
some
point
we
will
do
some
zones.
Please
choose
one
of
these
shown
in
the
presentation
and
just
pick
one
randomly
don't
own
one.
Don't
everyone
choose
the
first
one
and
everything
will
be
okay,
so
how's
everyone
joined.
Are
you
okay,.
C
Okay,
so
this
introduction-
we've
talked
about
cube
flow
and,
let's
see
the
goals,
we
will
install
mini
Kiev.
We
will
convert
the
tube
technology
flow
pipeline
without
any
CLI
or
getting
our
hands
dirty
using
SDK.
We
will
click
a
baton
and
run
a
keep
the
pipeline
and
then,
and
we
will
also
version
or
notebook
and
our
data
during
the
pipeline.
You
will
not
need
the
DCP
credentials
you
were.
You
should
find
in
front
of
you
in
your
seats
and
I'm,
going
to
the
next
step.
C
I'm
already
logged
in
so,
if
you
follow
the
link,
you
should
see
a
prompt
to
put
in
your
credentials
and
make
sure
you
once
you
are
logged
in
you
have
chosen
the
the
only
project
you
should
see
should
be
the
same.
If
you
haven't
selected
project,
just
click
here
and
choose
the
one
from
the
list,
it
should
look
like
this.
C
Let's
click
on
the
search
bar
and
type
minute
AF.
We
will
talk
about
minute
AF
later
on.
Let's
start
blowing
it
for
now
it
will
take
about
15
minutes.
So
if
you
search
for
min
KF
click
on
launch
on
compute
engine,
you
can
choose
your
deployment
name,
I'll,
call
it
Q,
Khan,
demo
and
I'll
choose
one
of
the
zones.
Let
me
remember
central
one
worst,
one
or
worse,
well,
central
one
and
leave
the
rest
with
their
default.
You
may
be
prompted
to
click
on
a
checkbox
here
to
agree
with
some
terms.
C
C
C
And
now
we
will
see
some
on-screen
instructions
just
to
watch
the
progress
of
Fulham
in
Kiev
deployment.
We
will
talk
details
about
min
Kiev
later
on,
so
what
I
should
do
just
type
the
command
min
Kiev
to
see
the
progress
of
the
provisional
script,
this
min
Kiev
the
procedure
of
deployment
is
to
pull.
Some
images
then
deploy
a
kubernetes
cluster
using
milky,
but
she
be
not
in
the
Nets
cluster.
On
top
of
it,
a
single
node
distribution
will
be
deployed
and
our
software
rock
for
data
management
and
data
versioning,
okay.
Well,
this
is
done.
C
C
C
This
is
a
figure
taken
from
Google's
defect
paper
to
high-level
overview
of
a
machine
line
platform.
There
are
several
stages:
the
several
components
which
are
the
storage
and
data
management
utilities.
Then
there
are
several
components:
performing
the
actual
machine
learning
less
than
the
desired
configuration
framework
and
the
job
Orchestrator
to
organize
these
components
and
a
front-end
is
required
to
manage
and
analyze
and
evaluate
the
result.
So
upin
has
open
sourced
attention
flow
components
that
perform
these
the
machine
learning
and
to
flow
encapsulate
these,
along
with
others
to
perform
area.
C
C
What
Rock?
Let's
say
you
have
multiple
distributions
across
the
world
to
flow
distributions.
So
you
do
your
experimentation
somewhere!
You
do
your
training,
some
other
infrastructure,
we'll
take
the
cloud,
and
then
you
serve
your
results
in
for
production
in
some
other
cluster,
you
have
data
of
libraries
of
BBC's
Rock.
C
Make
sure
that
your
environment
code,
libraries,
data
synchronized
across
or
across
all
locations,
and
let's
see
an
example,
so
that
this
is
your
data,
your
data,
you
want
to
run
with
some
learning
on
and
let's
say
that,
while
your
pipeline
starts,
you
decide
to
take
snapshots
so
one
validation
step
and
you
have
a
snapshot
of
the
validated
data.
Then
the
layers
of
the
pipeline
continues
again.
We
take
is
not
the
process
data,
then
the
training
happens,
but
you
know
failure
that
failure
could
be
anything
from
cold
problems
to
whatever.
C
C
C
C
C
C
C
C
C
C
Okay,
let's
proceed:
if
anyone
is
left
behind,
the
codelab
has
all
the
steps
we
will
follow.
So
in
this
page
we
can
see
a
link
for
the
mini
Kiev
dashboard
and
you
have
some
credentials
username
the
password,
please
don't
log
into
mine.
So,
okay,
let's
cope
with
the
password
and
make
sure
you
copy
exactly
the
password,
because
there
are
some
browser
issues
that
might
let
make
you
copy
some
extra
spaces
around
it.
Okay,
I,
have
it
copied.
I
will
go
to
that
URL.
C
C
This
is,
this
is
a
dashboard
from
here
you
can
get
to
any
tube
flow
component
pipelines
or
servers
and
there's
also
a
snap
suits
or,
let's
click
on
it.
It
will
take
us
to
Rock.
Ui
credentials
are
needed
there.
We
are
the
same
you
used
before
so
same
username
and
password
all
your
synapses
will
be
stored
here
in
budgets.
C
We
will
not
need
this
for
now,
just
we'll
come
back
later.
So
back
to
the
code
lab,
we
are
on
step
4
now,
we'll
create
a
notebook
server
to
do
that
we
will
head
to
farm
in
Kiev
and
notebook
servers
and
create
a
new
server.
I
will
give
the
name
I'll
call
to
come
demo
when
you
create
this,
make
sure
you
choose
the
second
image
from
the
list.
The
one
with
the
tip
flow
works
of
target
and
I
will
not
date.
C
C
C
B
That
some
of
you
are
already
going
through
the
the
notebook.
Some
others
are
a
little
bit
behind,
but
I'm
going
quickly,
I'm
going
to
go
quickly
through
the
next
session
of
the
code
lab.
So
if
you
want,
you
can
follow
me
here
and
then
you
can
replicate
it
at
your
own
pace.
So
I
will
first
download
the
code
that
we
are
going
to
use.
B
B
Guess
do
so
now
that
I
have
cloned
this
repository,
you
will
see
multiple
folders
and
each
one
of
this
folder
contains
a
specific
example
with
different
data,
a
different
machine
learning,
challenge
that
showcases
how
to
scale,
but
today
we're
going
to
use
the
Titanic
ml
dataset
challenge.
That
is
a
tangle
challenge
that
was
launched
a
couple
of
years
ago
and
basically,
here
we're
building
some
simple
machine
learning
models
to
try
to
predict
if
a
passenger
in
the
Titanic
shipwreck
survived
or
not,
based
on
the
feature,
the
features
provided
by
this
challenge.
B
Now,
let's
verify
that
this
environment
has
all
the
elaborate
libraries
we
need,
and
indeed
it
doesn't
so,
as
you
can
see,
I'm
missing
the
library
Seabourn.
So
in
order
to
execute
the
notebook
I
just
want
to
install
the
library
and
as
before,
you
can
copy
and
paste
the
command
required
from
the
code
lab,
make
sure
to
use
pip
3
installed
on
the
under
Python
3
and
using
the
flag
user.
B
B
B
And
instead
I'm
going
to
open
up
this
left
panel,
where,
with
the
cube
floor,
icon
and
enable
it-
and
you
will
see
some
magic
happen
so
now
there
is
a
bunch
of
information
on
the
left
and
a
bunch
of
colors
on
the
right,
but
for
now
I'm
going
to
explain
you
later
what
is
happening
but
for
now
I'm
just
going
to
click
the
compiled
in
one
button,
and
now
some
things
are
happening
in
the
black
background.
There's
a
taking
snapshot
dialog!
B
That's
basically
saying
that
in
the
background,
kale
is
interacting
with
rock
and
it
is
taking
a
snapshot
of
the
current
workspace
that
will
be
used
to
run
every
cell
in
the
pipeline.
That
will
result
from
this
notebook
and
then
kale
takes
this
notebook
and
converts
it
to
a
cube
flow
pipeline
workflow.
You
can
see
the
entire
workflow
by
clicking
on
the
link
in
uploading
pipeline,
and
this
is
the
structure
of
the
workflow
that
you
can
expect
this
before
running
the
workflow.
B
Every
cell
in
this
pipe
line,
corresponds
to
specific
cells
in
the
original
notebooks.
In
the
original
note,
Soquel
did
the
job
of
splitting
up
the
notebook,
resolving
the
dependencies
and
creating
a
new
pipeline
and
we'll
see
later
how
all
this
works.
And
then
you
can
click
on
the
running
pipeline
link
to
see
the
actual
pipeline
running.
B
So,
as
I
said,
kale
comes
both
with
the
Jupiter
at
lab
extension
that
provides
the
UI
that
you've
seen
there
and
a
Python
package
package
that
acts
as
a
back-end
to
that
UI
kale
is
able
to
take
as
input
an
annotated
notebook
and
splitting
it
up
and
composing
a
pipeline
without
the
need
of
using
the
cube
so
SDK,
actually
to
go
back
to
the
lab.
I
was
talking
about
undertaking
a
notebook.
So
what
what
the
the
jupiter
lab
extension
does
is
exactly
this.
B
Instead,
here
on
the
right,
all
these
colors
and
badges
refer
to
Tia
notations
that
we
had
already
made
on
the
notebook
itself.
So,
for
example,
here
the
load
beta
badge
is
referring
to
the
fact
that
this
cell
will
become
a
load
data
step
of
the
pipeline.
Here
on
the
right.
You
can
see
this
edit
button
and
this
dialogue
is
where
you
can
change.
The
notations
of
this
cell
I
will
skip
cell,
for
example,
with
I
skipped
from
the
workflow
and
I
want
to
show
you
here.
We
have
a
data
processing
hearing
in
this
case.
B
Data
processing
is
a
multi
cell
component
because
we
have
tagged
the
cell
as
data
processing
and
then
all
the
subsequent
cells
that
are
untagged
will
be
automatically
merged
to
the
first
one.
There
is
also
a
dot
that
indicates
the
dependency
of
data
processing
to
the
load
data
component,
and
these
dependencies
can
be
defined
very
easily
from
this
box.
As
you
can
see
here,
I
have
all
the.
B
Then
some
static
analysis
magic
will
resolve
all
the
data
dependencies
between
these
steps
and
K
will
inject
some
code
to
serialize
and
deserialize
data
when
a
step
is
completed
and
then,
when
an
afterword
step
needs
to
take
the
data
that
the
previous
step
generated.
After
all,
this
magic
happened.
K
will
generate
some
Python
code.
Actually
it
will
generate
a
single
executable
script
using
the
cube
flow
SDK
to
define
and
upload
the
pipeline.
B
B
So
when
you
are
interacting
with
with
this
with
this
extension,
what
you're
actually
doing
is
writing
metadata
in
the
JSON
file.
That
is
the
notebook.
So
if,
if
you
actually
want
to
see
this
metadata,
you
can
open
up
this
panel
here,
the
one
above
the
cue
flow
one-
and
this
is
the
notebook
tools
panel.
If
you
click
on
advanced
tools,
you
will
see
some
notebook
metadata
and
if
you
click
on
cells,
you
will
see
that
the
cells
metadata-
and
this
is
where
kale
is-
writing
this
information.
B
I
want
to
add
another
thing
that
I
think
it's
very
interesting.
So
when
I
click
compile
and
run,
you
saw
the
message
of
snapshot
taking
this
one.
So
this
is
very
important
because
if
you
remember,
we
have
installed
C
burn
at
first
and
when
we
did
that
the
notebook
run
correctly.
If
you
notice
the
pipeline
is
running
as
well,
so
you
might
wonder
on
what
darker
image
that
pipeline
is
running.
So
the
thing
is,
we
are
taking
the
same
image
that
this
notebook
servers
server
is
running
now,
so
you
have
the
base
environment.
B
B
Ok,
just
before
coming
over
I
wanted
to
add
that
I
wanted
just
to
show
you
this
slide
to
tell
you
that
kale
is
completely
open-source
and
if
you
want
to
contribute
open
issues,
you
can
head
over
to
this
goetaborg
and
also
there's
a
nice
medium
blog
post.
That
introduced
you
to
you
to
kale
and
how
it
works,
and
you
can
find
it
at
at
this
link.
C
C
C
C
That
way,
we
know
the
exact
input.
This
step
will
take
each
state
at
the
very
first
moment
of
its
execution.
Why
do
we
want
that?
We
have
tweaked
the
example.
We've
just
run
to
produce
weird
results,
so
I
will
now
show
you
I'm
going
back
to
the
code
lab
we've
done
all
these
on
step.
5
I
will
now
show
you
how
can
reproduce
the
state
of
a
pipeline
steps
execution.
C
C
We
need
to
see
what
did
the
models
do?
So,
let's
go
to
random
forests
and
explore
it.
We
are
going
to
explore
what
happened
right
in
the
beginning
of
random
forests,
so
clicking
on
random
forests
and
then
on
artifacts,
which
is
an
artifact
produced
by
kale.
That
has
a
link
to
the
snapshot
taken
the
very
beginning
and
all
the
instructions
required
to
reduces
takes
so
I'll
do
it.
He
do
it
right
afterwards.
I
click
on
the
link,
Robbie
I
comes
up,
and
this
is
a
snap,
so
that
was
just
taken.
C
So
this
is
our
Jupiter
lab
and
in
the
pipeline's
budget
we
store
the
auto
snapshots
during
the
pipeline.
This
snapshot
is
a
group.
It
has
two
members.
We
can
see
that
the
one
has
the
keyword
data
in
its
description.
We
can
imagine
the
data
volume
and
the
second
one
is
the
worse
with
volume.
So
we
had
two
volumes
on
our
pipeline
and
is
this:
if
we
click
on
some
versions
on
the
top,
then
we
will
see
all
the
snapshots
taken
during
the
pipeline.
Previously
we
were
just
seeing
the
latest
one.
C
Each
version
has
two
members,
the
exact
members
at
each
point.
So
let's
go
back
to
random
forests.
Here
is
where
the
link
you
click
on,
will
take
you
and
I'll
copy.
The
rock
URL
I'll
go
back
to
cube
flow
and
create
a
new
notebook.
Server
I
will
paste
the
link
rock
took
the
lab,
URL
and
click
on
auto
fill.
All
the
details
are
automatically
filled.
I
will
just
use
a
name.
I
will
call
it
explore
random
forest
I
can
see
that
I
have
the
same
image,
the
same
type
of
volumes.
C
Now
we
have
clones
and
some
environment
variables.
Some
extra
data
I
will
talk
to
you
about
later
on
and
I
click
on
launch
during
the
automatic
snapshots
rock
has
ingested.
Some
data
that
kale
will
grab
kale
will
recognize
and
reproduce
our
state
take
us
to
the
that
exact
point
in
a
notebook
in
the
notebook
and
the
specific
notebook
where
the
step
would
be,
and
it
would
have
pre-loaded
all
the
data,
all
its
inputs
and
every
import.
We
have
chosen
every
global
function,
we
have
defined,
etc.
C
C
C
C
C
Everything
is
in
the
column,
so
are
the
bug
fix.
I
will
copy
these
and
put
them
in
sell
and
run
again.
I've
dropped.
This
word
by
the
survived
column,
and
now
my
dad
Dallas,
it
looks
as
it
should.
I
would
run
again,
random
forest
just
to
be
sure
and
that
pretty
reasonable
accuracy,
and
since
it
looks
like
after
baguette
I'll
run
again,
I
hit,
compile
and
run
again.
C
A
So
how
many
people
got
cupola
deployed
using
mini
cab
all
right
great?
How
many
people
got
were
able
to
run
the
notebook
through
the
first
time?
Okay,
how
many
people
were
able
to
run
it
a
second
time
using
rope
and
fix
the
pipeline?
Okay,
fantastic,
so
I
think
we'll
just
wrap
it
up
here.
We're
just
gonna
wrap
it
up
by
providing
a
little
bit
of
information
about
the
latest
releases
of
kupo,
so
we
do
quarterly
releases
of
coop
flow.
We
just
released
0.7.
A
There
were
a
couple
of
new
things
in
coop
flow
with
0.7
that
we're
really
excited
about.
We
released
a
camp
serving
so
KF
serving
is
for
model
deployment
and
management.
It's
a
simple
custom,
coop
resource
kubernetes
custom
resource
that
makes
model
deployments
easy.
It
handles
complex
DevOps
tasks
like
auto
scaling,
as
well
as
complex
ml,
2ml.
Ops
tasks,
like
you
know,
inference
graphs
and
building
model
servers.
We've
also
created
much
simpler,
syntax
for
deploying
coop
flow.
So
it's
now
one
command,
as
opposed
to
three
we've,
significantly
improved
the
multi-user
support
to
better
support
enterprises.
A
And
finally,
we
had
a
bunch
of
hyper
parameter
tuning
improvements
and
then
for
pipelines
we
had
a
bunch
of
performance
improvements,
we've
improved
the
automatic
metadata
long
gate.
So
if
you
run
a
T
FX
pipeline
and
run
that
on
top
of
couplet
pipelines,
the
metadata
for
all
the
steps
will
automatically
belong.
And
finally,
we
have
new
looping
constructs
with
with
items
and
with
params.
So
you
can
easily
execute
individual
step
for
a
variety
of
different
inputs.
A
So
then,
what's
coming
next
after
0.7,
we
have
a
huge
a
for
notebook,
two
pipelines,
that's
what
we
did
demo
today,
so
we're
gonna
be
going
to
coop
flow
1.0
in
January
of
2020
and
one
of
the
things
that's
going
to
be
coming
after
couplet
1.0
is
polishing
up
the
cuj
that
you
saw
today
so
that
it's
very
production
ready
and
you
can
run
that
in
your
enterprise.
Coupe
flow
is
a
community
I'm,
so
we're
open
source.
So
over
30
companies
participating.