►
From YouTube: KubeFlow’s Serverless Component: 10x Faster, a 1/10 of the Effort - Orit Nissan-Messing, Iguazio
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
KubeFlow’s Serverless Component: 10x Faster, a 1/10 of the Effort - Orit Nissan-Messing, Iguazio
Serverless simplifies data science by automating the process of code to container and enables users to add instrumentation and auto-scaling with minimum overhead. However, serverless has many limitations involving performance, lack of concurrency, lack of GPU support, limited application patterns and limited debugging possibilities. Orit Nissan-Messing will introduce Nuclio, a KubeFlow open source component which is 10x faster when compared to alternatives at a 1/10 of the effort. She will explain how to use Nuclio to extend KubeFlow pipelines, accelerating and automating each step of the workflow. This includes parallel processing, automated code building/deployment, stream processing and artifact tracking. Orit will demonstrate how to achieve devops automation involving auto-scaling, automated logging and monitoring, security hardening, CI/CD and workload mobility.
https://sched.co/UaaL
B
A
So
usually,
when
one
thinks
about
the
data
scientist
works,
they
think
that
most
of
the
time
is
spent
on
defining
the
KPIs,
finding
the
right
algorithm
to
solve
the
problem
and
tune
it
surprisingly
or
not.
Most
of
the
time
is
being
spent
or
rather
said,
wasted
on
climbing
on
the
infrastructure
in
order
to
bring
this
model
into
production
to
bring
the
model
into
life.
This
is
a
slide
taking
from
Google,
but
we
held
last
month
ml
ops
conference
in
New
York,
and
we
heard
the
same
problem
coming
up
from
most
of
the
participant
in
attendees.
A
So
it
is
a
very
acute
issue
with
ml
ops
at
the
moment.
So
we
need
to
find
a
simple
solution
for
better
data
integration
and
automation.
So
when
I'm
saying
plumbing,
what
does
it
really
mean?
So
it
can
take
weeks
for
a
single
data
scientist
to
come
up
with
a
model,
but
it
will
take
months
of
several
people,
including
DevOps
and
data
engineers
and
the
data
scientists
to
really
bring
this
model
into
life
because
they
need
a
lot
of
packaging
dependencies
running
scripts
building
scale
out.
A
There
are
different
notions
of
scaling
out
depending
on
the
framework
that
was
chosen.
There
are
different
way
to
partition
the
data,
depending
on
the
frameworks
that
was
chosen.
There
is
the
I
promise
GPU
support
integration
would
GPU
parallelism
tuning
monitoring
all
those
kind
of
work
logging
taking
care
of
the
artifacts
security,
CI
CD,
that's
the
plumbing.
That
needs
to
be
done,
and
we
need
a
way
to
automate
all
those
different
steps
taking
reduce
the
time
that
it
takes
a
model
to
get
into
life
and
reduce
the
amount
of
models
that
address
dying
in
the
process.
A
So,
let's
look
take
a
look
at
a
very
simple
example
of
a
machine
learning
pipeline.
This
example
is
talking
about
a
customer
recommendation
based
on
location
like
if
I
move,
the
mall
and
I'm
usually
buying
debt
and
debt
product.
What
would
you
recommend
me
to
buy
now?
So
there
is
the
ingestion
part
there
is
tons
of
transactions
coming
in
ingested,
all
the
transaction
by
all
the
different
customers
in
swiping
their
visa.
There
is
the
location
where
the
customer
is
currently
located.
Those
kind
of
ingestion
to
restream
is
coming
in
ten
thousandth
events
per.
A
Second,
there
is
information
about
the
products
about
what
is
being
held
in
the
store
and
customer
tables
like
what
is
the
gender,
what
its
interest,
what
is
usually
buying?
What
is
interested
in
all
the
information
that
you
can
preserve
a
customer
depending
on
what
you
have
on
them,
and
you
can
also
scrape
information
from
the,
for
example,
environment.
If
it's
a
rainy
day,
you
would
like
to
propose
winter
product
to
that
customer.
So
all
this
information
is
being
ingested
and
being
saved
in
a
base
feature
store.
A
We
have
a
data
preparation,
labeling
stuff
like
that,
save
again
to
a
derived
features
store.
Then
we
do
the
training
finding
the
correlation
between
different
products
from
you
buying
this
product,
usually
I,
will
buy
that
and
that
product
as
well.
All
this
is
being
trained
into
a
model
and
to
a
learned
feature
store
which
will
then
be
used
by
the
model.
A
Then
I'll
deploy
the
model
and
when
now
I
have
information
about
the
location
of
a
specific
customer,
I
can
provide
a
customer
ID
and
its
location,
and
the
model
servant
will
provide
me
of
what
they
need
to
send
him
as
a
product
to
sell.
So
if
we
look
at
this
quite
simple,
a
machine-learning
pipeline
example,
we
see
as
usual.
A
There
are
four
stages
there
is
they
ingest
the
data
preparation,
the
data
training
in
the
Surfing
and
automation,
a
service
has
tackled
the
ingest
and
the
serve
and
we'll
see
in
a
moment
how
it's
being
tackled,
but
the
data
preparation
and
the
training
are
not
yet
to
be
handled
so
introducing
new
Clio
new
Clio
is
an
open
source
service
framework
which
was
developed
with
machine
learning
pipeline
in
mind.
It
can
handle
virtually
any
type
of
source
of
event
from
Kafka
for
Canisius
event,
a
paps
RabbitMQ,
cron
HTTP.
Of
course
it
has
integration
within
video
rapids.
A
A
Each
one
of
the
workers
when
there
is
data
worker
reviewed
at
work
a
decade
and
uses
GP
optimization
and,
of
course
it's
natively
integrated
with
cube
flow
and
with
jupiter
notebooks,
so
I
was
doing
just
being
used
so
for
in
this
example,
you
can
see
that
there
is
a
simple
Python
code
that
by
moving
that
code
and
invoking
it
using
a
nuclear
function,
you
get
almost
30%
more
throughput,
and
why
is
that?
The
reason
is
the
parallelism.
Nucleo.
You
have
multiple
workers
that
can
take
ingest
information
in
parallel.
A
Does
give
you
a
much
higher
throughput
in
a
low
latency
kind
of
a
session.
It's
a
very
simple
way
to
move
a
code
from
one
other
to
another,
and
it's
virtually
as
I
said
before,
can
get
any
type
of
event,
but
it
can
also
get
any
type
of
data
within
event,
so
it
doesn't
have
to
get
numbers.
It
can
get
images
it
can
get
URLs
and
stuff
like
that
model
serving
so
you
can
use
nuclear
to
serve
models.
A
It
really
uses.
Does
a
resource
a
utilization
and
you
can
see
in
this
example
the
same
kind
of
code
running
or
GPU
system
Whitney
clearly
get
four
times
the
performance
as
in
an
equivalent
system.
You
can
also
get
twice
the
performance
on
a
reduced
system
and
it's
a
very
simple
notion
to
take
a
notebook
and
we'll
see
that
in
a
moment,
Rupert
aerobic
and
convert
it
to
nuclear.
So
it's
very
easy
to
deploy.
It
gives
you
a
great
resource
utilization
within
the
same
pod
and,
of
course,
as
server
less
its
users
versus
on-demand.
A
It
means
that
it
can
scale
up
add
more
application
to
the
function
if
needed.
If
you
have
a
huge
burst,
for
example,
then
you
get
more
application
for
that
same
function.
It
can
reduce
even
to
zero.
If
there
is
a
downtime
or
not,
nothing
is
in
triggers
the
function,
so
it
gives
you
a
resource
on
demand.
A
So
service
is
such
a
great
notion.
It
solves
the
ingestion
of
the
serving.
Why
not
use
that
for
data
preparation
and
training.
So
if
we
take
a
look
at
C,
it's
several
characteristics
of
server
list
and
data
professional
training.
We
see
that
they
don't
fit.
If
we
look,
for
example,
it
the
tasks
lifespan
in
servlets
the
times
to
take
and
use
an
event,
is
millisecond
two
minutes
data
preparation
and
training
it
seconds
two
hours
too
many
hours
scaling
in
server
less
the
scaling
is
very
easy.
A
You
put
a
load
balancer
in
front
of
the
functions
you
get
a
trigger.
It
will
redirect
it
to
one
of
those
functions
in
that
operational
training.
The
scalability
is
much
more
difficult
and
there
are
so
many
diff
ways
to
partition
and
to
scale
the
infrastructure
in
depending
on
the
real-time
engine.
Underneath
it
you
can
do
partition
and
shuffle
is
being
done.
For
example,
in
Spock
you
can
do
hyper,
params
ring
or
reduce
has
been
done
in
hovered,
so
there
are
many
different
ways
to
scale
and
paralyze
your
workload
and
it's
very
dependent
on
the
engine.
A
A
So
what
we
did
is
we
took
we
wrapped
different
engine
such
as
spark
a
chance
of
horville
nucleo.
We
wrapped
them
around
such
it
will
abstract
and
each
one
of
those
different
runtime
will
handle
the
scalability
and
the
parallelism
within
it.
We
attached
a
fast
data
layer
underneath
so
it
they
can
preserve
the
state
and
the
differently
there
itself
and
attached
a
edit,
an
abstraction
layer
which
removes
the
operational
burden
from
you.
A
A
So
before
going
into
the
demo,
and
the
demo
will
show
you
ml
functions,
all
the
demos
can
be
found
in
d
dub
ml
1
itself
is
an
open
source
which
is
in
early
development
stages.
You
we
encourage
you
to
have
a
look
and
get
up
at
ml,
run
to
add
different
runtime
engines
to
it,
and
different
storage
is,
as
you
see
fit,.
A
A
A
Okay,
so
I'll
try
the
demo
and
may
the
gods
demo
will
be
with
me
and
the
network
as
well.
So
let's
give
it
a
try.
So
I'll
start
with
a
very
simple
demo,
which
is
the
lord
of
the
machine:
learning
it's
the
iris
model
and
we'll
take
a
look
at
that
and
from
there
we'll
move
to
a
more
complicated
kind
of
an
demo.
So,
first
of
all,
you
see
here
a
jupiter
magic
here,
it's
an
nucleo
notation,
which
will
tell
you
later.
A
A
So
we
do
all
this
beep
installs.
We
configure
some
base
image
for
the
nuclear
function,
but
most
of
it
mode
of
hole.
This
is
the
methods
that
holds
the
information
file
to
operate
that
training.
So,
first
of
all,
we
have
this
method,
which
is
IRA's
generator
which
take
the
data,
sets
embedded
and
save
it.
As
a
CVS
csv
story,
then
we
have
a
method
for
exhibit
rain.
We'll
take
this
file,
define
a
text
parameters
apparatus
that
will
provide
it
to
it
when
it's
being
called
and
creating
and
saving
the
model.
A
Okay,
so
what
I'm
doing
here
is
I'm,
creating
an
ml
function
and
running
it,
so
there
is
two
ways
to
provide
Iran
with
information
and
can
define
a
task
which
we'll
see
next
or
I
can
provide
it
explicitly
with
the
different
parameters
of
the
job
to
be
run.
So
in
that
case
we
chose
to
provide
it
within
the
run
itself.
So
we
have
a
job
name
here
is
Jan.
This
is
the
end
of
to
the
method
we've
just
saw,
and
the
parameters
is
where
to
locate
the
file.
A
A
Next,
we
want
to
train
the
model,
so
we're
taking
defining
parameters
and
now
that
we're
refining
it
in
a
task,
so
we
refining
the
task
the
different
parameter,
that
we
want
to
invoke.
Diapered
parameters
that
we
want
this
one
to
run
and
provided
they
run
the
task,
this
one
will
run
within
the
jupiter
notebook
itself.
I
didn't
provide
any
runtime
information,
so
the
local
one
is
being
chosen.
A
So
I'm
taking
the
same
code
converted
to
a
nuclear
function
now
providing
a
runtime
of
machine
nuclear
machine
learning,
attach
it
with
assured
a
data
volume
and
defining
that
the
HTTP
triggers
should
have
sixteen
workers,
and
I
deploy
this
function.
So
all
the
compiling
and
believe
believe
the
container
for
the
function
is
done
for
me.
I
didn't
do
anything.
A
A
So
this
is
the
UFO
nuclear-nuclear,
of
course,
is
open
source.
So
you
see
here
it's
the
code,
the
methods
that
we've
just
seen,
what
I
wanted
to
show
it's
the
configuration
of
this
function.
So
you
see
that
all
the
configuration
annotation
that
we
use
within
the
jupiter
notebooks
are
now
within
the
pod
itself.
We
have
the
base
image
the
different
people
stores
that
we
had
there,
the
video,
which
is
the
volume
we
attach
to
that
function.
We
can
also
see,
within
the
triggers
that
I
have
16
workers
for
HTTP
I.
A
Can
that
easily
add
another
trigger
to
a
function?
What
I
didn't
mention
before,
and
it's
very
important,
it's
nuclear,
it's
not
just
that
it
can
handle
different
many
different
triggers
it
can
handle
different
triggers
within
the
same
function.
So
I
can
add
another
like
kafka,
which
has
the
same
notion
of
different
event
and
added
another
trigger
to
that
function.
A
Moving
back
to
Jupiter,
so
I
have
I.
Now
have
this
function
being
deployed
it's
running
now,
I
want
it
to
invoke
the
Train,
so
I
need
to
provide
it
with
the
parameters
that
I
defined
earlier,
and
it's
just
that
easy.
I'm
taking
the
address,
this
is
the
address
of
the
function
and
we
are
using
that
to
do
a
Mel
function,
run,
providing
it
the
tasks
that
we
defined
earlier
and
use
to
run
on
the
Jupiter
itself.
A
Now
the
same
thing
this
training
is
run
is
ran
on
a
nucleo
and
parallels
like
using
the
60
workers
that
we
have
defined
and
we
have
the
job
results
for
this
training
as
well,
using
all
the
logging,
the
input
parameters
and
the
artifacts.
So
what
we've
seen
is
that
I
could
take
very
simply
a
same
code,
run
it
in
a
different
aspect,
different
runtimes
and
all
the
scalability,
and
all
this
notion
and
the
building
and
the
monitoring
and
the
logging
is
done
automatically
for
me.
A
So
let's
assume
that
I
did
all
this
kind
of
playing
with
the
Jupiter
now
very
happy
with
my
result
and
I
want
to
use
it
in
cube
flow,
so
I
want
to
define
the
different
steps
and
to
create
a
cube
flow
out
of
it.
So
it's
very
simple
to
do
that
with
ml
run,
I
have
here
a
within
a
pipeline,
a
four
steps.
The
first
one
is
the
gist.
It's
the
same
code
that
we
used
before
the
Indies
generator
and
its
output.
A
It's
the
data
set,
then
I
have
a
training,
step
training
and
we
see
that
it
gets
an
input.
The
data
set
that
was
created
as
an
output
of
the
IRA's
data
set
of
the
previous
step,
and
if
the
output
is
the
model
itself-
and
we
see
next-
that
the
plot
can
take
the
model
and
do
a
plot
in
of
iteration,
and
we
can
take
a
function
and
create
a
model
serving
for
that
using
that
model,
and
with
that
simple
definition
of
how
the
artifacts
should
move
from
one
step
to
another
step,
we
can
now
run.
A
This
and
create
a
pipeline,
and
we
can
see
this
pipeline
I
prepare
ahead
of
time.
So
we
see
this
is
the
pipeline
that
was
created.
We
have
the
iris
going
to
the
training,
and
then
we
have
two
polarized
stages:
the
plot
and
the
deployment
deployment
of
nuclear
function.
If
we
go
to
the
plot,
for
example,
we
can
see
the
artifact.
This
is
the
nice
sister
Graham
that
was
created
a
by
the
training.
We
can
see
the
logs.
A
We
can
see
all
the
other
information
that
was
created,
but
while
running
this
job,
so
we
saw
a
very
easy
way
to
take
a
code
convert,
run
it
on
different
runtimes
and
then
take
it
to
the
next
step
and
automate
it
with
cube
flow.
What
we
see
next
before
we
go
to
a
different
kind
of
job
is
how
we
can
take
nuclear
and
use
it.
We
care,
if
serving
so
ka,
serving
defines
methods
of
a
model.
How
would
should
it
be
called
so?
A
So
what
we'll
see
here
is
that
you
can
use
nucleon
as
a
model
serving
within
KF
serving,
but,
and
you
gain
the
serverless
notion
out
of
it.
So
when
you
use
it
with
K
serving,
you
still
need
to
compile
and
build,
and
do
all
the
notion
for
that
image
with
nucleo
the
birth.
This
burden
is
lifted
out
of
you,
so
you
can
and
again
the
performance
of
a
funicular
as
a
service.
So
we
see
here
I'll.
Do
it
very
quickly,
so
we'll
have
time
for
the
next
demo.
A
A
A
A
We
consider
the
function
its
again
with
the
prediction.
Everything
and
you
can
use
that
the
same
notion
is
you
with
KF
serving
so
I'll
move
to
the
next
demo.
This
one
is
taking
a
little
bit
more
complex
kind
of
a
job,
it's
utilizing
a
hoard
which
is
distributed
tensorflow,
so
it's
using
this
kind
of
an
engine,
so
briefly,
gold
stuff
that
we
already
talked
about.
But
we
see
here
that
there
steps
here
this
this
open
archive
method
which
take
the
images
out
of
s3
bringing
them
to
the
system.
A
So
now
we
again,
we
can
create
a
very
simple
task.
This
is
the
task
definition
it's
a
download.
It
has
the
different
parameters
and
we
can
create
them
ml
functions
using
that
task.
We
are
seeing
here
all
the
completion
and
everything
of
that
step.
We
want
me
to
worry
about
to
monitor
about
the
artifacts
and
stuff,
like
that.
The
next
stage
is
to
tag
the
images
again
it's
running
here
and
the
third
step
is
actually
do
the
training.
A
So
this
training
is
more
intensive
and
it's
using
a
horde
in
order
to
do
that,
so
we
are
defining
the
horford
file,
the
different
parameters.
There
is
the
different
data
path
and
all
those
file
as
an
input
and
simple
as
that
we
create
a
new.
A
machine
learning
function,
give
named
command
of
the
Opera
MPI
job
and
apply
different
a
configuration
for
that
job
such
as
are
many
replicas.
We
want
for
that
MPI
operator
for
means.
We
have,
will
have
four
walker
poles
that
does
a
dope
job
for
us.
We
can
easily.
A
A
Don't
see
if
you
can
see
that
never
mind,
and
now,
when
we
have
the
model
we
can
deploy
that,
but
before
we
going
into
the
model,
I
want
to
show
another
thing
which
is
ml,
run
UI
the
same
notion
of
drop
tracking
as
we
see
in
the
jupiter
notebook.
We
can
much
better
see
with
this
within
this
UI
and
it's
supposed
to
cube
flow.
It
monitors
every
different
job
that
you
have.
It's
not
just
the
cube
flow
jobs,
each
one
in
cube
flow.
A
A
Okay,
so
I
can
now
take
the
same
as
the
same
notion
as
before,
take
and
create
a
function
for
the
model
serving
using
the
model
that
we
have
just
been
created,
setting
its
specific
environment
that
we
need
and,
as
opposed
to
a
need
to
use
numpy
array,
we
can
use
with
clear
different
type
of
triggers,
so
you
can
invoke
that
function
using
a
URL
and
we
get
you
see
that
it's
a
URL.
We
get
it's
a
cat,
and
you
see
here
that
we
send
it's
in
the
image
you
see
in
the
context
that
we
send.
A
A
A
So
I'm
out
of
time.
So
if
you
have
question
that's
this
is
the
time,
but
before
that,
I
wouldn't
really
want
to
encourage
you
to
have
a
look
at
ml,
one
and
Emma
Wong
demos.
Everything
that
was
shown
here
is
we
can
get
up.
It's
an
open
source.
I
encourage
you
to
look
at
that,
play
with
that
and
contribute
to
ml
run
a
nucleus
also
an
open
source
which
you
can
contribute
to
that
as
well.
Thank
you.
B
C
A
So
the
scaling
in
nuclear
is
based
on
two
different
aspects:
it
can
either
look
at
the
CPU
see
and
you
can
define
if
it's
above
20%
CPU
50
then
scale
to
the
next
one,
and
it
can
also
look
at
the
time.
An
event
was
waiting
in
queue
until
it's
being
handled.
So
you
have
both
options
and
the
both
options
is
applicable
for
HTTP
triggers
and
for
the
streaming.
B
A
B
A
So
specifically,
Gua
Co
has
a
data
a
fabric
of
its
own,
but
it
can
also
integrate
with
different
data
layers
such
as
s3,
local
file
system
and
others.
Amber
one
is
specific,
is
not
a
coupled
to
any
type
of
data
storage,
it
can
work
with
file
system,
it
can
work
with
data
frames
and
it
can
a
other
data.
Elements
can
be
added
to
the
abstraction
layer
as
well.