►
Description
Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Expanding Serverless to Scale-out Kubeflow Pipelines - Yaron Haviv, Iguazio
Serverless frameworks enable devops automation, resource allocation on demand and auto-scaling. However, so far they only addressed event-driven workloads and not distributed, batch oriented data science tasks like data preparation and ML training. Orit Nissan-Messing will introduce the recently announced Nuclio ML Functions which is tightly integrated with Kubeflow and extends the Nuclio open-source serverless framework to auto-scaling machine learning, training and data preparation tasks. Nuclio ML functions automatically tracks experiments and artifacts, is seamlessly integrated with Kubeflow pipelines and eliminates a significant amount of devops. Participants will learn how to work with Kubeflow and Nuclio to enable CI/CD for their machine learning workloads, enabling performance and horizontal scaling, while cutting infrastructure costs and tedious development tasks.
https://sched.co/ZepH
A
A
A
So
before
I
talk
about
kubeflow
and
and
serverless
and
let's
examine
what
are
the
real
challenges
that
we're
trying
to
tackle.
So
today,
what
you
see
in
most
organizations,
machine
learning
is
still
pretty
much
in
the
research
environment,
research,
environment,
people
take
a
few
csv
files
excel
spreadsheets.
You
know
they
upload.
Some
data
run
a
notebook
or
jupyter
or
something
like
that
run
some
analysis
on
the
data
and
based
on
that
they
run
some
training
and
generate
models
in
some
cases,
taking
those
models
and
put
them
behind
an
http
endpoint
and
serving
those.
A
The
challenge
is
really
when
you're
going
to
a
real
application.
Think
about
applications.
Do
fraud,
detection,
real-time
recommendations,
things
that
do
predictive
maintenance,
etc?
Those
are
real
pipelines
and
machine
learning
is
just
a
component
within
this
pipeline
and
also
needs
to
perform
fast
in
reasonable
performance
with
highly
high
availability
and
scalability
and
and
all
that
attributes
of
a
production
system.
A
A
Sometimes
terabytes
of
data
versus
few
gigabytes
on
an
in-memory
notebook
once
you've
done
that
you
need
an
automated
system
to
run
various
experiments,
training
with
different
sets
of
parameters
with
different
algorithms,
referring
to
it
as
server
automl
and
those
things
generate.
Machine
learning,
models
that
are
served
using
real-time,
interactive.
A
So
we
need
data
preparation
also
for
the
productization
part
for
the
serving
part,
and
once
we've
done
that,
we
also
need
a
monitoring,
essentially
a
feedback
loop
to
monitor
our
models,
detect
drift
in
our
model
behavior
and
sometimes
do
automated
retraining
or
formation
of
ensembles
changing
weights
and
and
other
things
to
mitigate
loss
of
accuracy
in
our
in
our
models.
Now
the
main
challenge
is
today
that
those
are
usually
two
different
departments
in
an
organization.
A
There's
one
group
that
builds
data
science,
products
or
models,
there's
a
different
group
that
tries
to
productize
all
of
that
and
that
in
many
cases,
involve
refactoring
the
entire
set
of
code
and
project
takes
forever.
You
know,
according
to
analysts,
more
than
eighty
percent
of
the
project
just
fail
flat
due
to
this
transition
of
essentially
rewriting
everything
recomposing
it.
You
know
taking
python
code,
changing
it
to
spark
and
scala,
sometimes
moving
code
to
c
and
go
to
make
it
higher
performance
and
using
different
paradigms.
A
Those
are
all
the
steps
that
it
needs
to
go
through
from
just
having
a
python
code
to
having
a
production
worthy
microservice,
that's
automated
in
production.
So,
for
example,
we
need
to
take
the
code
and
add
dependencies
and
packages
to
it
and
build
a
docker
container
out
of
our
code
with
run
scripts
and
parameters
and
arguments,
and
all
of
that
which
are
not
intuitive
for
too
many
data
scientists.
A
The
next
thing
which
is
very
challenging,
is
how
do
we
take
a
code
that
runs
on
a
single
process
single
container,
and
we
distribute
that
because
we
have
a
lot
more
load,
more
data
or
a
bigger
computation
problem
to
solve.
We
want
to
leverage
the
the
benefits
of
kubernetes
of
scaling
out
resources
to
essentially
break
that
piece
of
code
into
smaller
pieces
that
work
in
concert
to
generate
a
bigger
to
address
a
bigger
workload.
A
So,
in
many
cases
that
actually
require
refactoring
of
the
code
for
distributed
computing
now,
most
cases,
the
codex
design
in
research
is
not
fitted
into
production
environment
because
of
simply
performance
people,
don't
necessarily
think
about
parallelism
about
potentially
incorporating
caching
technologies.
So
they
don't
just
go
and
read
the
data
on
every
request.
A
Sometimes
gpus
make
a
lot
of
sense
and
we
need
to
incorporate
libraries
for
cuda
and
maybe
change
our
code
to
support
gpus
and
etc,
but
when
we
have
a
code
which
is
scalable
performance
package,
the
next
thing
we
have
to
start
thinking
about
is
observability
monitoring.
But
it's
not
just
monitoring
in
the
traditional
sense
of
kubernetes
and
microservices
of
you
know,
cpus
memory
etc.
We
need
to
monitor
and
log
everything.
A
We
need
to
monitor
the
experiments
that
we're
doing
the
data
that's
generated
with
the
experiment,
the
code
that
was
used
in
order
to
generate
the
experiment
et
cetera,
because
there
is
a
big
portion
of
liability
and
explainability
as
part
of
every
machine
learning
project.
We
run
a
project,
we
generate
some
results.
We
want
to
be
able
to
go
back
to
the
same
experiment
or
we
launched
something
to
production.
We
had
some
issues
with
how
we
predict
outcomes.
Someone
sues
us
in
a
court
of
law.
A
We
need
to
be
able
to
explain
what
we've
done
and
what
was
the,
what
were
the
models
and
how
we
train
the
models
and
make
sure
that
we
haven't
done
any
mistake
in
that.
So
the
monitoring
and
the
observability
part
in
machine
learning
is
much
much
more
extensive
than
what
it
is
when
you're
talking
about
standard
micro
services
and
the
other
aspect
is
security.
A
Of
course,
the
security
has
many
faces:
there's
data
security,
api
security,
role-based
access
control,
access
to
credentials
and
secrets
for
external
databases
and
services,
and
so
on,
and
the
final
piece
in
order
to
make
this
workflow
not
sort
of
a
single
or
one-time
flow,
but
something
that
can
reiterate
and
go
on
and
on
is
automation,
we're
all
familiar
with
ci
cd
in
the
notion
of
computing,
computation
and
coding.
We
have
to
apply
a
similar
approach
of
ci,
cd
and
githubs
for
machine
learning.
A
This
is
really
where
kubeflow
comes
into
play
and
we'll
talk
about
it,
and
obviously
we
need
to
think
about
continuous
deployment
ruling
upgrades
canary
deployments,
a
b
testing,
etc.
Usually
this
part
from
the
development
to
productization
is
the
biggest
spark
in
every
project.
So
there
are
some
companies
where
I
work
with
them
and
it
takes
them
like
12
to
18
months
with
the
original
project
to
deploy
into
production,
and
it's
very
essential
that
we
try
and
automate
those
processes.
A
One
of
the
technologies
that
we're
going
to
show
you
how
it's
pretty
significant
can
bring
significant
advantage
is
serverless,
because
this
is
essentially
what
service
technologies
in
general.
Do
they
allow
you
to
bring
code?
They
automatically
do
the
packaging,
they
have
inherent
scaling
out,
they
do
the
instrumentation
and
in
some
cases
the
automation
we
need
to
add
some
other
significant
parts
which
are
missing
from
traditional
serverless,
which
are
around
performance
and
parallelism
and
data
statefulness
and
and
also
more
specific.
A
A
So,
first,
what
is
amalops
before
we
dive
into
an
amalops
or
presentation?
We
all
know
what
devops
is.
Essentially,
this
is,
according
to
the
the
traditional
definition,
is
essentially
combining
development
and
operations
to
work
together
and
and
try
and
solve
the
main
business
challenges.
So
mlaps
is
pretty
much
the
same,
just
embedding
all
the
aspects
of
data
engineering
and
data
science
into
the
process.
So
it's
not
just
developers
and
operations.
It's
developers,
data
scientists,
data
engineers
et
cetera,
they
need
to
collaborate,
and
one
of
the
key
challenges
is
not.
A
A
What
we
need
to
do
is
essentially
ingest
all
of
that
data.
The
operational
data
coupled
with
the
streaming
and
transactional
data
and
throw
it
into
a
road
data
bucket
road
data
bucket
could
just
be
a
shared
file
system
with
parquet
files
or
csv
files
could
also
be
something
slightly
more
structured,
like
a
data
warehouse.
But
the
first
thing
we
need
to
do
is
place
everything
in
this
sort
of
road
data
bucket.
A
The
next
thing
we
want
to
do
is
essentially
start
taking
those
raw
features
and
convert
them
to
something
meaningful
that
we
can
run
training
on.
So
we
may
need
to
join
various
data
sets
to
extend
the
knowledge
we
may
need
to
aggregate
things
into
buckets
of
time,
like
hourly
average
number
of
events.
A
Here
we
have
a
essentially
a
single
denormalized
table
which
is
raw
going
in
through
a
training
logic,
but
even
before
going
into
a
training
logic,
we
need
to
sanitize
those
features
and
if
we
have
thousands
of
features,
maybe
we
don't
really
need
to
train
on
thousands
of
features.
Again,
it's
more
computational
power.
It's
going
to
have
skewing
in
the
model
if
you're
not
going
to
learn
based
on
the
right
features.
A
So
the
next
thing
is
to
go
and
filter
and
auto,
detect
what
are
the
meaningful
features
and
and
leverage
those
and
maybe
scale
those
to
the
right
degree.
Once
we
have
those
prepared
features,
we
may
need
to
go
and
run
training
and
the
training
will
generate
a
model.
This
model
will
be
later
used
for
for
serving,
but
also
before
we
we
use
that
model.
A
We
want
to
validate
it
using
a
different,
smaller
data
set
at
the
essentially
splitting
the
entire
data
to
a
training
set
in
a
validation
set,
and
then
we
would
generate
this
model.
We
take
this
model
and
validate
it
actually
performs,
and
what's
the
accuracy,
if
it
doesn't,
we
may
need
to
go
back
and
forth
and
tune
this
entire
pipeline.
A
Now,
once
we
have
a
model,
we're
not
done.
This
is
really
what
we've
just
covered
is
the
model
creation
part
which
is
covered
nicely
with
with
kubeflow,
with
the
extension
of
what
we've
discussed
essentially
moving
the
individual
steps
in
the
pipeline
to
a
reusable
high
performance.
Scalable
function,
we'll
we'll
see
the
architecture
in
a
minute.
A
The
next
piece
in
the
in
the
puzzle
is,
is
essentially
running
a
serving,
and
so
we
take
the
data
we
take
the
model.
We
can't
just
stay
with
them
all.
We
have
to
bring
data
because
remember
what
the
serving
logic
does
is
essentially
takes
a
set
of
parameters,
a
set
of
features
and
runs
some
mathematical
functions
and
produces
a
result
so,
and
the
request
coming
from
the
user
doesn't
contain
all
those
features.
A
Now,
once
we've
done
that,
it's
not
we're
not
done
yet.
We
need
to
monitor
the
behavior
of
our
model
through
some
sort
of
a
tracking
system.
How
do
we
know
that
our
model
keeps
on
performing
and
is
accurate,
or
we
want
to
be
able
to
monitor?
Maybe
there
is
misbehavior
in
the
performance
aspects,
maybe
glitches,
but
also
does
it
lose
accuracy,
or
do
we
see
different
data
than
what
we've
expected?
A
A
They
may
also
record
some
data
into
sort
of
batches
and
then
there
is
another
set
of
tools
that
may
compare
the
the
inferencing
logic
of
the
inferencing
data
with
real
reference
data
in
a
delay.
You
know,
let's
assume
we
we
predict
something,
but
in
a
day
from
now
we'll
actually
know
if
it
happened
or
not.
If
our
prediction
was
true,
so
we
can
compare
our
predictions
with
the
real
results
and
this
way
you
also
have
another
way
of
analyzing.
A
If
our
model
behaves
properly
and
if
it
doesn't,
we
can
trigger
something
to
fix
it,
retraining,
bringing
on
data,
changing
the
logic
of
our
serving
and
samples
etc.
So
this
is
how
usually
you'll
see
that
this
pipelines
work
again.
You
can
see
that
it's
not
trivial,
and
this
is
a
predictive
maintenance
pipeline.
It
will
be
the
same
problem
with
other
solutions
and,
if
you
add
things
like
deep
learning
and
neural
networks,
it
may
even
get
complicated.
A
Okay.
So
before
you,
you
tear
your
hairs
out.
The
most
important
thing
is
you
can
start.
You
see,
there
are
a
lot
of
different
moving
components
in
this
system
and
there
are
a
lot
of
different
software
projects
that
you
can
choose
from,
but
the
the
right
way
is
essentially
to
try
and
take
an
ecosystem
which
com
completes
the
most
of
the
building
block
in
the
puzzle.
A
For
example,
if
you
take
some
tools
that
were
designed
for
hadoop,
along
with
some
tools
that
were
designed
for
kubernetes
and
some
tools
in
a
cloud
provider
and
they
don't
work
with
each
other,
you
end
up
essentially
doing
a
lot
of
integration
by
yourself.
So
one
of
the
nice
things
here
is
that
kubeflow
is
a
mechanism
or
is
a
is
essentially
a
framework.
A
Okay,
so
essentially,
what
we
we
want
to
show
is
a
complete
pipeline
where
we
fit
in
data
that
we
do,
data
preparation,
we
accelerate
the
training.
We
deploy
an
application,
we
monitor
the
application
in
order
to
do
all
of
that
together,
we
have
to
also
have
some
a
baseline
of
data
that
moves
around
this
entire
fabric.
A
So
how
would
the
solution
look
like
if
we're
opening
that
sort
of
pipeline,
and
so
we
need
three
layers
in
the
solution?
The
first
layer
we
need
data
and
the
data
need
to
have
a
few
attributes.
First,
it
needs
to
be
relatively
fast
because
we're
not
talking
about
archival
data,
we're
talking
about
data
in
the
move,
we're
building
a
pipeline,
one
micro
service,
writes
data.
A
The
other
micro
service
reads
data
if
it
takes
too
much
time
we're
going
to
have
a
delay,
in
many
cases
we're
doing
distributed
computing
like
in
in
training
or
sometimes
in
in
analytics.
So
we
want
to
have
shared
data.
We
want
all
those
things
to
work
together
and
maybe
shuffle
some
data
around
between
those
micro
services.
A
We
need
sometimes
streaming
data.
We
need
some
key
value
stores
for
real
time
data.
We
may
need
time
series
data
in
order
to
log
all
sorts
of
matrix,
so
this
is
where
we
need
data
and
it
needs
to
be
fast,
secure
and
also
versioned.
So
if
we
run
multiple
experiments,
we
can
also
go
back
into
a
specific
experiment
and
see
and
recontract
that
from
the
data
we
have,
the
next
layer
is
micro,
services,
okay
and
those
micro
services.
This
is
a
simplistic
view
of
those
micro
services
we
need.
A
We
need
services
that
ingest
data
from
etl
from
streaming
from
scraping
from
http
apis.
We
need
data,
preparation,
so
functions
that
will
do
analytics
and
whether
it's
batch
or
real
time
or
stream
processing
to
join
aggregate
split
and
do
all
sorts
of
manipulations
on
data.
So
it's
going
to
be
ready
for
the
training.
A
Then
we
need
to
run
training
logic
again
with
usually
distributed
frameworks,
whether
it's
machine
learning
or
deep
learning-
and
here
we
may
also
want
to
incorporate
something
referred
to
as
hyper
parameter
tuning
so
run
the
same
experiment
on
many
different
permutations,
and
we
want
to
leverage
the
power
of
kubernetes
of
scheduling.
Lots
of
containers,
so
not
necessarily
all
the
permutations
run
on
one
container.
We
can
run
multiple
containers.
A
A
So
if
we
are
going
to
essentially
go
and
create
a
microservice
and
a
kubernetes
deployment
or
kubernetes
job
for
each
one
of
those
that
we're
going
to
run
out
of
steam
very
very
quickly,
so
the
general
idea
is
instead
of
going
and
running
and
creating
docker
files
and
yaml
files
and
and
kubernetes
crds,
and
all
of
that
for
each
individual
step.
We
want
to
have
more
of
a
serverless
mindset.
We
want
to
write
some
code
provided
minimal
description
and
we
want
something
else
to
build
those
services
for
us.
A
We
did
yamls
with
the
containers
with
the
docker
images,
with
the
dependent
packages,
with
the
considerations
around
scaling
and
and
all
the
aspects
of
logging
and
monitoring.
We
just
want
to
provide
some
code
and
let
someone
else
deal
with
that,
and
this
is
really
where
serverless
technology
will
come
in
very
handy.
A
A
So
a
typical
stack
would
look
like
something
like
that.
So
you
have
the
data
layer,
remember
we
we
mentioned
a
lot
of
times.
We
need
real
time
access
from
microservice
to
other
micro
services
from
my
micro
services
to
get
obtain,
features
that
are
more
depend.
You
know,
require
low
latency
access.
At
the
same
time,
we
need
access
to
big
data
in
in
the
form
of
files,
object,
store,
data,
warehouses,
etc.
So
we
need
to
have
a
layer
in
iguazi.
A
You
know
things
like
spark
and
tensorflow,
and
if
you're
doing
sequels
or
something
like
presto
and
for
machine
learning,
you
you
need
like
psychic,
learn
and
pi
torch
and
and
horvath
is
a
way
to
distribute
deep
learning.
Jobs
and
task
is
a
way
to
distribute
machine
learning,
et
cetera
and
jupiter
is
a
managed
notebook.
So
you
need
to
install
a
bunch
of
services
on
your
kubernetes
cluster
and
make
sure
they
all
work
in
our
solution.
Again,
it's
a
managed
offering,
so
you
don't
need
to
have
all
the
hassle.
A
It
just
works
out
of
the
box.
We
are
suffering
from
the
integration,
not
you,
and
then
you
have
a
set
of
tools
for
pipeline
orchestration.
Now
there
is
cube
flow
and
kubeflow
is
mainly
serve.
A
graph
execution
qflo
pipeline
is
a
graph
execution
entity.
It
is
missing
a
lot
of
other
aspects
that
we
need
like
feature
store
like
automl,
auto
hyperparameter
tuning
is
not
really
built
into
it
or
and
even
experiment
tracking.
There
are
some
challenges
around
how
you
do.
Experiment
tracking
you
want
to
do.
A
In
many
cases,
experiment
tracking,
not
only
within
a
workflow,
sometimes
just
run,
experiment
tracking
on
your
laptop
and
that's
something
you
cannot
do
with
kubeflow.
So
you
need
another
mechanism
that
can
work
with
qflo
for
tracking
experiments,
but
also
can
work
on
a
standalone
machine
or
just
running
individual
step
in
the
pipeline.
So,
and,
and
also
we'll
we'll
talk
about
the
gaps
and
what
we
need
to
do
so.
A
The
the
one
of
the
key
solutions
in
the
as
what
we
recommend
is
working
with
serverless
function,
because
we
mentioned
before
that.
One
of
the
biggest
challenge
is
moving
from
code
to
like
a
production
artifact
with
all
the
aspects
of
scaling
performance,
tuning,
instrumentation
logging,
monitoring
security
hardening.
A
So
one
of
the
ways
to
do
it
is
essentially
leverage
code
and
to
production
with
serverless
technologies,
and
for
that
there
are
several
technologies
that
we're
supporting
and
they're
part
of
the
open
source
and
nucleo
is,
is
one
of
the
highest
performance,
a
very
mature
serverless
framework
over
kubernetes,
more
than
3
000
stars
on
github
used
by
many
enterprises
and
there's
also
mlran,
which
is
more
for
batch
computation
and
I'll.
Show
you
how
it
works.
So
one
of
the
major
advantages
of
nucleo
that,
beyond
being
a
standard
serverless,
it's
very,
very
high
performance.
A
It's
using
a
real-time
kernel.
It
runs
about
100
times
faster
than
things
like
amazon
lambda.
It
runs
about
10
times
faster
than
most
other
serverless
frameworks
over
kubernetes,
including
kf,
serving
and
some
of
those
it's
stateful.
So
it
can
actually
work
with
data
and
gpus,
and
things
like
that.
In
the
function,
which
is
very
essential
for
machine
learning
and
also
one
of
the
nice
things
it's
glue,
it
has
glueless
integration
to
cube.
A
So
you
can
essentially
create
a
graph
of
computation
where
the
individual
steps
are
functions
essentially
serverless
function,
and
there
is
a
marketplace
where
you
actually
don't
need
to
write
most
of
the
functions
for
automl
for
serving
for
training,
for
data
preparation
for
feature
analysis
for
hooks
to
get
and
slack,
and
things
like
that.
So
you
just
can
put
a
bunch
of
components
on
a
canvas
and
get
a
pipeline
up
and
running
very
quickly.
A
Another
major
difference
in
jobs
and
and
data
preparation
and
training
is
instead
of
passing
json.
You
know
as
an
event.
We
are
actually
passing
data,
we're
creating
a
pipeline
with
parameters
and
data
that
feed
each
other.
You
know
we
have
data
set
output
of
one
step,
feeding
a
data
as
an
input
data
set
for
another
step
like
data
preparation,
creating
a
data
set
and
training
consumes
that
data
set.
A
So
we
need
to
have
a
new
serverless
approach,
which
has
those
attributes,
and
this
is
where
we've
created
a
new
framework,
essentially
just
for
that
so
part
of
what
we
refer
to
as
mlran,
which
is
open
source
again
relatively
new,
but
very,
very
powerful
solution.
It
has
several
layers.
The
serverless
engine
is
essentially
one
layer.
It
also
has
a
lot
of
higher
level
automation
layers,
and
so
the
general
idea
is
that
it
auto
scales.
It
allows
it
to
scalable
to
do
scalable
computing
over
multiple
frameworks.
A
You
can
create
a
serverless
function
of
spark
of
dusk
of
mpi
job.
You
know
a
lot
of
different
crds
are
orchestrated
by
ml
run,
so
you
can
essentially
create
distributed
computing
and
emerald
knows
how
to
break
the
the
work
and
control
the
distributed
job
etc.
Another
interesting
piece
is
that
it
has
auto
tracking
features.
A
It
says
she
knows
how
to
look
into
the
inputs
and
outputs
of
every
execution
and
auto
record
that
so
you
get
auto,
recording
of
all
your
experiments
and
runs
and
code
and
data
inputs
and
outputs
etc
as
I'll
show
in
a
minute
and
again
it
runs
over
multiple
frameworks.
Another
interesting
thing
is,
you
know:
there
are
other
frameworks
for
a
hyperparameter
in
kubernetes
like
khatib
and
others,
but
then
you
need
to
manually
go
and
integrate
them,
and
if
we
have
an
engine
here
that
knows
how
to
run
many
different
permutations
already
distributed.
A
The
only
missing
piece
is
having
something
that
knows
how
to
create
those
permutations
and
track
them.
This
is
actually
built
into
mlran,
so
you
don't
need
a
separate
project
for
hyper
parameters
and
you
all
have
automl
built
in
into
the
solution
so
which
is
new.
There's
no
automl
solution
on
top
of
kubernetes
beside
the
mrm.
A
Okay.
Now,
where,
where
is
does
it
fit?
So
kubeflow
is
something
that
couple
project
has
two
main
things
or
three
main
things.
One
is
a
tuple
pipeline,
which
is
this
orchestration
capability,
and
then
you
have
the
operators
like
mpi
job
or
distributed
tensorflow
and
others,
which
you
can
also
install
by
yourself
and
there's
also
managed
notebook
so,
but
the
most
interesting
piece
is
kilflow
pipeline,
which
is
based
on
argo
engine,
a
distributed
workflow
engine
and
it
allows
you
to
essentially
run
a
graph.
The
challenges
with
kubeflow
is
very
oriented
towards
devops.
A
They
want
to
make
things
very,
very
intuitive
and
simple
in
their
own
domain
of
expertise.
So
so
what
amaran
and
nucleo
provide
is
a
way
to
essentially
automate
the
code
to
creation
of
serverless
functions
from
code.
It
delivers
all
the
integration
around
data,
access
and
parallelism.
It
allows
you
to
do
distributed
training,
gpu
acceleration
and
other
features
that
are
not
built
in
and
it
does
end-to-end
tracking
of
everything,
code,
execution,
data,
versioning,
etc.
All
in
a
serving
a
graph
that
you
can
actually
just
play
around
and
move
around
between
the
different
aspects.
A
So
how
does
it
all
work
if
I'm
looking
into
it
from
a
flow
perspective?
Take
this
data
scientist?
What
he
wants
to
do
is
build
something
in
his
own
notebook
or
his
own
pycharm
or
vs
code.
Still,
it
doesn't
know
kubernetes
or
doesn't
care
about
kubernetes.
So
the
first
thing
we
want
to
make
sure
is
that
whatever
we're
building
can
work
in
a
standalone
and
that's
part
of
the
challenges
with
kubeflow.
A
If
you
want
to
run
kubeflow
stuff,
you
have
to
run
it
within
kubernetes
and
within
queue
flow.
You
cannot
run
a
instrument
and
do
the
tracking
and
all
of
that,
from
your
notebook
without
kubernetes
attached
to
it.
So
the
first
thing
that
we
we
want
to
do
is
essentially
just
import
an
sdk
and
allow
someone
to
run
from
within
his
notebook
and
still
get
all
the
experiment,
tracking
and
all
the
data
lineage
tracking
and
all
that
done
automatically
for
us
at
a
certain
point.
A
We've
debugged
our
code
and
we
feel
it's
ready
for
just
running
it
as
a
container
on
the
cluster.
So
we
don't
want
to
build
and
all
those
things
we
just
want
to
say
run.
But
this
time
on
the
cluster
so
essentially
just
a
symbol,
a
single
line
saying
not
locally,
but
in
the
cluster,
what
it
would
do
it
essentially
automatically
packages
your
code
throws
it
into
the
cluster
builds.
A
Your
containers
builds
the
crds
for
the
various
runtimes
that
are
supported,
like
you
know,
kubernetes,
job
or
spark
or
dusk,
or
any
of
the
other
ones
runs
it
and
you
still
get
the
full
tracking.
Even
while
you're
running
this
job
on
the
cluster,
now
note
that
at
this
stage
we
don't
really
need
a
pipeline.
We
just
want
to
run
a
single
step
of
let's
say,
training.
We
want
to
test
validate,
see
that
it
works
and
the
later
on
in
the
process
we
we
want
to
turn
it
into
a
pipeline.
A
So
this
once
we've
committed
our
code
into
a
repository
like
git,
then
we
want
to
take
those
pieces
of
code
and
just
put
them
in
a
layout
or
a
pipeline
and
in
this
way
run
the
entire
pipeline.
So
someone
else
or
the
same
guy
can
just
take
all
those
components,
build
a
pipeline
and
execute
that
through
kubeflow
or
mlrun
apis.
The
other
nice
thing
is
that
you
don't
really
necessarily
need
to
make
it
interactive.
You
can
make
it
as
part
of
an
automated
ci,
just
like
you
would
with
any
other
ci
product.
A
A
Amaran
goes
builds
the
entire
kubeflow
pipeline
for
it,
and
you
know
it
connects
all
the
dots,
the
data
and
and
everything
runs
the
keyful
pipeline
tracks.
All
the
information
about
the
the
pipeline
execution
with
the
pipeline
execution
terminates
records.
Everything
writes
it
back
into
the
pull
request
as
annotation
in
the
pull
request
and
throws
in
a
slack
notification
back
with
the
summary
of
your
execution.
A
A
We
convert
it
to
a
function
using
a
single
command.
We
test
it
on
the
cluster,
see
that
it
works.
Maybe
we're
missing
some
packages,
or
maybe
we
misconfigured
something.
We
go
back
fix
it
and
that's
we're
done
with
the
development.
The
next
thing
is,
we
need
to
think
about
integration,
we're
trying
to
pull
request
that
change
that
we've
made.
This
change
goes
into
some
inspection.
A
Once
we
finish
the
integration
we
want
to
go
and
run
the
continuous
deployment.
This
is
where
we
may
create
a
tag
of
our
repository,
deploy
the
models
and
the
code
that
goes
along
with
that
maybe
feature
engineering
and
some
other
aspects
and
deploy
that
in
a
canary
once,
it's
working
we've
examined
everything
as
well,
we're
going
to
promote
that
into
100
of
the
traffic
and
monitor,
keep
on
monitoring
the
service,
behavior
concept,
drift,
etc.
A
A
So
that's
with
that,
and
and
this
brings
a
lot
of
value,
so
people
can
build
pipelines
much
faster
and
the
pipelines
could
be
more
interactive
more
online.
One
of
the
examples
is
this
work
with
one
of
the
customers
that
we've
done
is
take
for
a
fraud
prevention
application.
The
original
solution
was
using
hadoop
where
you
have
a
database.
A
You
extract
everything
using
etl
into
a
data
warehouse.
You
run
some
queries
on
the
data
warehouse
and
then
have
r
servers
and
doing
prediction
using
an
r
model
and
writing
back.
You
know
identifying
the
fraud
accounts
and
blocking
those
product
accounts.
So
this
process
is
very
bad
oriented,
very
manual,
very
complicated
if
you've
worked
with
hadoop
and
all
of
those
frameworks,
and
instead
we
want
to
move
it
into
a
microservice
architecture
which
will
buy
us
two
things.
First,
faster
deployment
process
everything
instead
of
three
four
months
deployment
for
every
exchange.
A
It
could
actually
be
continuous
or
every
week
we
can
or
a
couple
weeks
we
can
drive
a
change.
The
second
is
we
don't
want
to
have
such
a
huge
delay,
because
everything
is
like
battery-oriented,
we
move.
I
want
to
move
to
something
more
online,
so
the
change
that
we've
applied
is
essentially
to
gather
the
information
from
the
server
the
database
server
directly
as
a
stream.
So
you
run
serverless
functions
using
nucleo
that
essentially
crunch
the
stream
in
real
time.
A
Nuclear
listens
on
about
13,
different
streaming
protocols,
kafka,
rabbit,
kinesis,
pops
up,
etc.
So
you
can
just
listen
on
kafka
or
rabbit
in
this
case.
Do
a
real-time
analysis,
because
it's
stateful
again,
so
it
can
do
things
like
real-time
analysis
of
of
data,
write
it
into
a
combined
online
and
offline
feature
store.
A
You
can
also
run
things
like
spark
to
do
batch
analytics
on
that
feature,
store
and
extend
the
features
even
further.
Looking
at
the
bigger
retrospective
or
bigger
time
window
and
and
then
periodically,
for
example,
on
a
daily
basis,
you
have
a
model
training
process
using
psychic,
learn
that
extract
a
feature
set,
a
training
set
of
a
bunch
of
features
run,
the
training
generates
a
model
file,
and
then
you
have
inferencing
function
which
in
this
case
they
have
to
work
in
a
stream,
because
there's
no
http
request.
A
A
Now,
okay,
so
what
I'm
opening
here
is
is
a
notebook.
This
notebook
is
going
to
show
me
how
I
can
build
a
full
pipeline,
so
this
pipeline,
it's
all
fully
documented,
but
the
general
idea
is:
I
have
a
function
in
this
case
again,
we
said
we
want
to
take
those
jumbo
notebooks
and
try
and
construct
them
in
a
way
using
certain
functions
and
not
just
like
straight
code.
So
I
have
some
function
which
is
generating
some
fake
data
set.
A
This
is
using
the
iris
data
set
and
once
it
finishes,
we
want
the
function
to
log
a
data
set
to
essentially
log
an
artifact
of
type
data
set,
essentially
a
data
frame,
and
this
logging
effect
will
essentially
record
everything
in
a
database
track,
lineage
track
statistical
behavior
of
the
data
set
and
all
those
things
behind
the
scene.
So
this
context
event
this
part
of
the
function
execution
can
carry
many
different
things.
It
can
give
us
access
to
secrets
and
and
other
things.
A
So,
at
a
certain
point,
we
also
may
want
to
package
all
those
functions
under
a
logical
project,
but
then
what
I
need
to
do
is
just
execute
this
function.
Now
I
can
just
go
and
execute
this
iris
generator
function
myself,
but
what
we're
doing
here
is
we're
running
it
under
a
wrapper
that
runs
the
execution,
it's
running
our
function
and
essentially
wrapping
it
up.
Why
do
we
want
to
wrap
the
execution?
A
Because
we
want
to
record
everything
that
goes
in
and
goes
out
into
the
function
into
a
database,
and
you
can
see
that
once
I'm
running
this
function,
I
also
see
some
nice
widget.
That
shows
me
the
trend,
the
name.
You
know
how
long
he
took
and
a
bunch
of
labels,
even
even
what
I
can
actually
see
the
data
that
was
generated
by
by
that
function.
Okay,
sorry,
so
I
can
just
run,
and
this
generates
a
data
set,
the
iris
data
set.
A
A
Essentially,
I
promised
you
a
single
line
of
code,
so
there
is
a
single
line
of
code
that
converts
my
code
into
a
function.
Essentially,
into
a
microservice
into
a
serverless
function,
now
I
have
a
function
object.
I
can
use,
for
example,
register
that
in
my
project,
or
do
other
things
with
that
now
I
generate
some
data
set
and
now
I
want
to
take
that
data
set
and
run
analysis
against
that
data
set.
A
So
I
may
write
the
code
for
analysis,
but
then
I
need
to
package
a
new
container
and
put
a
lot
of
libraries
in
it
and
build
yamls
and
docker
files.
Instead,
I
can
just
go
and
grab
a
function
from
a
marketplace
called
describe,
which
is
a
running
analysis
on
my
data
set
and
it's
showing
me
the
behavior
of
each
one
of
the
features
of
my
data
set,
because
I
don't
really
know
how
it
works.
A
A
So,
in
order
to
do
that,
I
need
to
run
this
function
called
describe,
because
I
wanted
to
analyze
data
from
my
notebook.
I
need
to
mount
it
over
a
shared
file
system,
so
the
notebook
and
the
job
will
see
the
same
data,
so
there
is
something
in
kubeflow
called
modifiers,
and
this
is
using
our
cluster
file
system,
but
you
can
also
use
nfs
or
pvc.
A
So
I
can
just
go
and
look
in
my
directory.
Remember
it's
a
shared
file
system,
so
everything
lands
back
into
my
own
directory
and
I
could
just
go
and
look
at
all
those
nice
charts
which
were
generated
by
this
analysis
job.
So
I
can
look
at
the
histograms
of
the
data.
I
can
look
at
you
know
the
feature
correlation
and
all
those
things
all
this
analysis
doesn't
have
to
burden
my
own
container.
A
The
resources
will
get
allocated
dynamically
and
if
I
I
need
gpus
for
those
services
and
I
register
as
part
of
the
requirements,
it
will
automatically
allocate
the
gpu
and
they
allocate
the
gpu
when
it's
done.
So
it's
very
important.
It
saves
huge
amount
of
cost.
Now,
after
running
this,
I
want
to
create
a
complete
pipeline,
a
pipeline
comprised
of
classification,
logic,
essentially
training
and
then
validation
logic.
A
Then
I
want
to
deploy
a
service,
the
endpoint,
the
serving
function,
and
once
I
deploy
the
string
function,
I
want
to
test
that
serving
function
to
see
that
it
has
the
right
performance,
latency,
etc.
So
I
can
just
go
in
and
pick
a
bunch
of
those
functions
already
from
a
library.
You
don't
need
to
write
them
by
the
way
it's
all
open
source.
It's
part
of
a
there
is
a
git
repo
hosting
all
those
functions
for
auto
ml
and
data,
engineering
and
others,
and
now
I
create
a
kubeflow
pipeline.
A
So
this
looks
like
a
q
probe
pipeline.
The
main
difference
is
that,
instead
of
having
a
kubeflow
pipeline
of
kubeflow
operators,
you
have
a
cupro
pipeline
of
serverless
functions
which
are
automatically
converted
to
like
a
kubflow
pipeline
operation.
So
I'm
just
going
to
take
those
individual
functions
that
I
I
have
I
can
have
build
functions.
I
can
do
an
analysis
function.
A
You
know
again.
Ingestion
function,
analysis
function,
a
a
training
function
with
auto
ml,
which
runs
all
those
very
variety
of
protocol,
algorithms,
and
then
I
need
to
have
a
validation
function
that
validates
the
output
model
and
the
testing
set
of
that
model.
And
finally,
I
want
to
deploy
a
serving
function.
So
it's
not
just
for
the
training.
A
I
can
actually
build
a
real-time
pipeline
using
this
engine
and
I
can
test
my
real-time
pipeline
with
another,
so
I
incorporated
data
engineering,
machine
learning,
pipeline
and
ci
pipeline
all
in
a
certain
in
a
single
workflow,
and
then
I
just
run
this
simple
python
code
here
locally
and
I
see
all
the
results
of
the
pipeline.
Once
it's
complete
brought
back
into
my
notebook,
I
actually
don't
have
to
go
into
kubeflow
and
examine
the
results,
although
I
can
just
go
and
see
all
those
results
in
cube
flow.
A
A
Also
for
the
data
analysis-
and
you
know
all
those
nice
widgets,
and
this
is
where
it
actually
deployed
the
nuclear
function
for
real-time
serving
and
then
we
use
the
tester
to
test
the
performance
of
that
serving
function.
So
we
can
actually
see
the
latency.
We
see
that
it's
around
11
milliseconds,
so
it's
really
decent
performance.
Obviously
we
haven't
tried
to
optimize
here
anything,
so
you
have
the
full
control
now
all
those
things
that
I
did
are
also
recorded
in
ml
run.
So
I
can
just
go
into
this.
A
Join
amarant
doesn't
care
where
you're
running
the
job,
whether
it's
running
in
cubeflow
as
a
separate
job
within
the
notebook.
It
sees
all
the
variety
of
jobs.
You
know
they
can
be
grouped
by
name
or
by
qflo
workflow,
and
then
I
could
just
go
and
look
into
each
one
of
those
jobs.
You
know
we
can
see
what's
running,
we
can
actually
see
the
code
commit
of
that
code
and
see
the
actual
code.
If
we
want,
we
can
look
at
all
the
artifacts
that
were
generated
in
each
one
of
those
steps.
A
A
If
the
training
functions
include
auto
ml
or
hyperparameter
tuning,
you
can
actually
see
all
the
individual
results
and
then
it
auto
selects
based
on
your
criteria,
the
best
result
to
move
into
the
next
step
of
the
pipeline,
so
the
automl
and
the
hyperparameter
tuning
is
built
into
this
framework.
You
don't
have
to
go
in
and
use
another
framework
just
to
get
this
thing,
and
also
you
have
a
way
to
manage
everything.
You
know
the
functions,
the
artifacts,
the
jobs,
there's
gonna,
be
a
new
ui
very
soon
and
again
everything
is
on
is
on
github.
A
A
You
know
everything
that
you
can
think
of
that
kubernetes
can
do
could
be
abstracted
using
nucleo
and
again
this
can.
This
function
was
actually
generated
from
a
notebook.
I
I
have
a
notebook
here.
This
serving
notebook
see
this
function,
which
I
can
write
in
a
notebook
at
just
when
I,
when
I
feel
I'm
ready
after
I
tested
this
notebook.
Have
one
thing
to
do
is
just
say:
function
deploy.
A
Another
cool
cool
thing
about
those
function
is
nucleo
also
supports
api
gateways
and
canaries
and
all
those
things.
So
you
can
go
and
create
a
gateway,
new
api
gateway
which
allows
you
to
you
know,
authenticate
an
endpoint
and
create
canaries.
You
know
assign
multiple
functions
and
give
different
percentage,
to
which
one
of
the
functions
and,
by
the
way
of
all
the
things
that
we
monitor,
are
also
written
to
prometheus
and
grafana.
All
the
statistics
of
functions,
jobs,
everything
so
think
about
how
much
work
we've
saved
you
with
those
serverless
frameworks.
A
So
all
the
logging
monitoring
all
the
auto
scaling,
regardless
of
how
you
want
to
scale
whether
it's
dusk
or
nucleo
or
spark
or
any
other
architecture,
all
the
tracking,
with
all
the
linkage
of
all
this
tracking
and
artifact
and
and
job
all
of
that
is
fully
automated.
So
that
means
that
you
can
turn
project
that
could
have
been
consuming
tons
of
devops
and
developers
and
take
months.
A
You
can
just
implement
the
same
project
within
weeks
and
everyone
will
be
happy
because
you
have
all
the
instrumentation,
the
best
performance
you
can
think
of,
etc,
and
so
with
that,
that's
my
session,
welcome
any
questions
or
or
comments.
You
can
also
a
you
know,
find
me
in
linkedin
or
twitter,
and
I'm
happy
to
talk
to
any
one
of
you
if
possible.