►
From YouTube: DevOps in Data Science: What Works and What Doesn’t - Chase Christensen & Stefano Fioravanzo
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
DevOps in Data Science: What Works and What Doesn’t - Chase Christensen & Stefano Fioravanzo, Arrikto
We all want more models and better business buy-ins from our ML projects. Often before questioning quality, we begin spending far too much time engineering overly ambitious model development life cycles. When it comes to productizing a model, ML engineers want DevOps. Data Scientists want simplicity. This often leads to tension and technical debt.
Our goal was to leverage Kubeflow (the most widely used and mature OSS MLOps platform) to “shift left” by giving data scientists the power to leverage Kaniko to self-service build containers using Kubeflow Pipelines. ArgoCD was used to deliver the models. We failed and we needed a DevOps detox. The CI process we imagined was complex and didn’t serve the data scientists in a meaningful way. We will discuss why Kserve is a lightweight and production-ready solution that can improve the outcomes we initially sought with Kaniko and KFP and how we as engineers can improve the OSS MLOps community.
A
A
A
The
intro
was
really
great,
so
I'm
Chase,
Christensen
I'm,
a
Solutions
architect
at
aricto.
Really,
my
job
is
to
you
know,
work
with
mles
as
well
as
platform
engineers
and
data
scientists
and
kind
of
listen
to
their
pain
points
to
understand
how
we
can
help
them
with
open
source
tools
and
and
get
them
the
outcomes
they
want
been
through.
All
the
super,
the
the
kubernetes
certification,
which
means
I,
know
just
enough
to
know
that
I
know
nothing
about
kubernetes
right,
as
you
start
to
dive
into
those.
You
learn
about.
A
Just
the
ocean
of
kubernetes
knowledge,
there
is
out
there
and
I'm
really
really
excited
and
spiritually
aligned
with
this
kind
of
trend
of
platform
engineering,
which
is
you
know,
kind
of
devops,
with
a
purpose
or
with
an
outcome
of
building
a
platform
and
really
helping
people
get
the
most
out
of
their
work.
Yeah.
B
So
yeah
I've
been
a
software
engineer
for
quite
a
few
years,
and
I've
recently
moved
to
a
more
political
oriented
role,
so
I'm
driving
some
envelopes
initiatives
and
some
of
our
strategies,
so
I'm
I'm
really
aligned
with
the
needs
and
over
you
know,
end
users
being
data
scientists
or
male
engineers
and
having
been
on
the
software
engineering
side,
I
can
also
understand,
and
you
know,
engage
into
more
solutioning
and
architecture
discussion.
So
it's
real
fun.
A
A
A
A
Mlops
is
not
devops
and
we
kind
of
learned
this
the
hard
way-
and
we
actually
see
this
with
a
lot
of
end
users
and
we
think
it's
because
the
wires
got
crossed.
You
know
the
last
year
we've
been
hearing
about
Emma,
lops
and
saying
you
know
not
a
couple
years
right.
Envelopes
is
about
automating
and
testing
your
deployments
of
your
software
a
model
right.
A
model
is
just
some
application.
You
can
serve
data
scientists
out
there
training
them.
A
You
know
they're
doing
wizard
math
on
the
board,
they're
figuring
it
out
for
you
they're
getting
these
data
sets
with
the
hard
work
of
data
engineers,
and
then
you
just
want
to
iterate
faster
right.
We've
got
this
idea
of
hey.
If
we
iterate
faster,
our
models
be
great
and
then
we
say
well,
listen.
We
know
about
something
that
helps
us
iterate,
really
quickly,
right,
devops
and
of
course,
we
know.
Devops
is
different
hand
wavy
based
on
your
organization,
but
this
concept
of
hey.
A
If
we
collaborate
better,
if
we
understand
and
be
Mission
oriented,
we
can
drive
better
outcomes,
so
hey
High
Velocity.
We
both
align
with
this
models
for
applications.
So,
let's
just
start
doing
devops,
and
it's
not
our
fault
right.
When
we
do
things
with
ML
Ops
through
a
devops
lens,
we
start
to
make
these
assumptions.
That
may
not
always
be
true
right.
Data
scientists
should
write
their
own
code.
Data
scientists
should
promote
their
own
models
right.
They
might
be
make
a
model
push
to
prod
they're
good.
A
You
know
we
throw
a
party,
their
model's
perfect,
they
buy
all
the
right
houses
and
then
data
scientists
should
package
their
own
work
via
CI
CD,
right,
we're
starting
to
say
data
scientists.
We
know
that
you
work
really
hard
understanding
data
sets
and
math,
and
now
we're
going
to
teach
you
git
and
we're
going
to
teach
you
git
and
you're
going
to
like
it,
and
sometimes
that
can
be
really
difficult
right
and,
as
you
may
have
seen
in
person
these
these
assumptions
don't
really
work
for
data
scientists
right.
A
Their
work
could
take
months
and
weeks
and
years
not
days
right
if
something
goes
wrong
with
a
model
or
a
new
project
is
being
presented.
Looking
at
a
data
scientist
and
say
all
right,
you've
got
this
assumption.
You've
got
this
hypothesis.
Give
me
a
prototype
by
next
Tuesday
they're,
going
to
freak
out
right.
They're
gonna
be
like
what
are
you
talking
about?
I,
don't
even
know.
A
If
I
can
get
the
data
right,
there's
a
lot
of
work,
they
need
to
do
the
require
their
models
and
how
they
do
their
work
requires
a
lot
of
experimentation
right
domain
knowledge.
In
many
many
many
iterations
their
models
are
not
the
full
application,
a
lot
of
times
they're,
maybe
serving
a
model
or
building
into
a
model
of
an
application
that
then
using
that
logic
to
present
some
sort
of
business
outcome
and
many
above
these
problems
aren't
just
about
serving
the
model
right.
A
Just
because
you've
served
a
model
and
you
can
iterate
on
the
model
doesn't
mean
that
was
the
problem.
In
fact,
a
lot
of
the
talks
today,
we've
heard
about
listen,
there's
this
new
idea
of
maybe
it's
the
data.
Maybe
we
need
to
be
better
about
our
data
hygiene
understanding
how
to
Source
our
data,
because
the
model
could
be
you
know
an
open
source
model
or
whatever,
and
it
is
doing
our
job.
It's
just
your
understanding
of
the
domain
or
whatever
is
causing
quality
to
to
fall,
and
this
is
the
outcome
right.
A
We're
saying
you
need
to
understand
our
square
devops
and
then
they're
a
circle
and
you're
trying
to
Ram
it
through,
and
it
causes
a
lot
of
tension
and
here's
the
real
problem
that
we're
trying
to
solve
data
scientists
need
to
pull
data.
They
need
to
train
models,
they
need
to
validate
models
and
they
need
to
collaborate
on
model
development
right
they're,
not
just
they're,
not
the
entire
story.
Right,
they
want
better
ingredients,
they
want
better
models.
Data
scientists
also
want
to
use
common,
tooling
and
platforms
to
simplify
this
handoff
process.
A
This
work
with
this
model
and
they
want
to
trust
that
if
they
push
this
process
that
you
as
an
ml
engineer
or
a
data
engineer
have
given
them
that
their
model
is
then
going
to
be
put
into
production
in
a
way
that
provides
value
for
their
business
for
their
boss
for
owners
or
whatever
to
align
with
their
goals,
and
it
has
to
be
reliable
and
trusted
because
data
scientists
is
more
than
just
models
right.
Data
science
is
way
more
than
just
models,
and
sometimes
we're
only
just
a
small
piece
of
the
puzzle
here.
Right.
A
The
implementation
details
and
the
tooling
has
all
this
background
of
hey.
Can
we
even
solve
this
problem
with
data?
Do
we
have
the
you
know
the
right
design
and
order?
Is
there
an
investment
decision
and
then
hey?
Can
we
even
do
this
right
and
then,
of
course,
the
iteration
of?
Did
we
actually
do
it?
A
So
we
had
some
users
come
up
to
us
and
they
said
listen.
We
want
to
start
to
use
these
devop
workflows,
and
we
said
you
know
what,
if
we
built
this,
if
we
build
this
amazing
platform,
this
amazing
tool
set
with
all
these
cool,
open
source
tools
that
we've
been
hearing
about
at
kubecon
for
all
these
years
is
going
to
be
great.
The
data
scientists
are
going
to
love
us
and
they're,
going
to
put
us
on
our
shoulders
and
parade
us
around
town.
So
this
is
what
we
tried
to
build.
A
So
we
know
data
scientists
like
kubeflow
right
they
like
kubeflow,
they
like
to
be
able
to
build
pipelines
they
like
to
use
continuous
integration.
You
can
shift
left
into
the
data
scientist
World.
They
can
do
all
sorts
of
cool
things.
We
can
get
all
sorts
of
little
great
artifacts
that
we
can
check
into
kubernetes.
A
We
also
know
that
they
like
jupyter
notebooks
right,
which
is
why
we
start
to
look
at
kubeflow,
because
they
can
start
to
orchestrate
things
with
jupyter
notebooks
and
we
think
hey
if
we
have
jupyter
notebooks.
If
we
have
Kube
flow
and
we
have
something
else
to
to
manage
and
schedule
our
compute,
we
can
help
data
scientists
do
their
own
build
Pipelines,
which
is
why
I
use
kubernetes
right.
You
need
a
way
to
intelligently
schedule
things
once
you
have.
A
These
pipelines
need
to
be
able
to
create
these
jobs
and
move
them
wherever
and
you
want
the
data
science
to
be
self-service,
and
then
you
have
to
find
a
way
to
serve
the
model
right
once
you
have
this
model
artifact,
you
find
a
way
to
serve
this
model,
and
you
know
this
is
a
whole
other
decision
and
they
don't
really
want
to
dive
too
much
into
this.
They
just
want
their
model
to
be
served,
so
we
give
them
Canada
right.
We
give
them
a
Docker
file,
we
say:
here's
your
Docker
file.
A
Here's
kaniko
canoco
is
great.
It's
root
list.
It
can
run
in
the
cube
field
pipeline,
just
build
things
and
go
right,
and
then
we
use
Argo
or
Argo
for
Argo
workflows,
Argo
just
for
delivery,
because
you
can
just
update
a
manifest.
You
can
deploy
things
you
can
move
it
across
environments,
but
there's
a
lot
of
little
decisions
in
here
that
we
had
made
for
the
data
scientists.
We
asked
the
data
science
to
decide
right.
How
are
you
going
to
take
your
latest
model
right?
What
does
latest
mean?
A
How
do
we
version
that
and
then
how
do
we
bring
you
on
board
with
this,
and
it
worked
kind
of
right
and
I.
Think
we've
all
been
in
this
situation,
where
we
built
something
really
great
and
you
know,
hey
I've,
gotten
successful
container
builds.
You
know,
I've
done
a
sample,
light
GBM
model
and
served
it
with
a
container.
What's
wrong
with
you,
but
really
we
have
just
created
all
sorts
of
cognitive
loan
and
complexity,
giving
them
a
promise
that
hey
we're
just
giving
you
a
platform,
hey
we're
just
giving
you
a
build
process.
A
So
where
did
we
go
wrong?
What
we
went
wrong
is
actually
the
first,
our
first
reaction
of
data
scientists
doing
this.
This
was
not
bad.
This
was
actually
the
best
thing
that
could
have
happened
to
us
right.
The
data
scientists
looking
at
us
and
saying
what
you
built
does
not
work
for
me,
probably
saved
us,
because
we
want
this
to
be
able
to
scale
Beyond
us.
We
want
to
scale
the
all
sorts
of
orgs
doing
ml
Ops
right.
A
So
the
fact
that
we
gave
them
this
over
complex
workflow
and
they
pushed
back
was
actually
really
healthy
for
us,
because
what
we
heard,
what
we
heard
is
listen
the
way
we
designed
the
systems
and
the
way
we
have
the
interfaces
within
the
system
is
actually
designing
the
tension
and
the
ways
that
we
collaborate
within
each
other
and
with
our
organization
right.
This
is
like
a
loose
interpretation
of
Conway's
law
right
how
we
design
our
system
and
organize
around
ourselves
around.
A
It
is
how
the
the
interactions
that
we're
promoting
the
incentive
to
have
these
interactions
within
ourselves.
Our
systems
need
to
be
flexible
and
transparent
for
us
right.
They
can't
be
over
complex,
mles,
sri's
platform
teams.
They
need
to
understand
how
these
services
are
provided
and
they
be
simple
to
work
with,
and
it
also
needs
to
be
flexible
and
lean
and
transparent
right.
If
someone
wants
to
run
a
new
type
of
workload,
it
shouldn't
be
a
nightmare
for
them
to
do
it
and
we
should
be
able
to
be
pluggable
and
let
them
do
it.
A
We
also
learned
that
we
need
to
listen
to
our
data
scientists
way
more
right
when
you're
developing
things
in
a
vacuum
and
trying
to
build
a
platform
and
not
bringing
in
your
data
scientists,
you
can
run
into
a
trouble
and
your
first
prototype
might
be.
You
know,
kind
of
what
we
had,
which
was
the
dumpster
fire,
but
that's
not
to
say
that
there
isn't
a
place
for
devops
in
data
science
right
because,
as
you
just
heard,
we
learned
a
lot
of
lessons.
We
learned
that
hey.
A
We
built
some
tools
right,
which
took
devops
processes
right
Argo.
All
these
cool
things
had
really
smart
people
using
devops
processes
or
naming
devops
devops
processes
to
make
services
to
build,
really
interesting
solutions
to
help
data
scientists
do
what
they
want
to
do
in
an
easier
way,
we're
actually
helping
them
with
their
cognitive
load,
and
we
can
really
respond
to
this
quickly.
We
can
take
a
lot
of
different
attempts.
A
We
can
make
some
wrong
assumptions
and
we
can
continuously
improve
using
these
kind
of
devops
processes
that
we
learn
and
that's
how
we
work
together
to
build
a
platform,
and
we've
done
this.
We've
built
some
really
great
tools
that
have
actually
helped
use
these
devops
developer
skill
sets.
We
have
to
help
data
scientists
use
their
skill,
sets
best
and
Stefano's
going
to
dive
into
that.
B
Compile
no
protein
to
find
a
place,
how
many
sales
people
and
we've
seen
how
these
kind
of
approaches
are
very
well
received
by
the
community
and
our
customers.
So
at
the
model,
starting,
it's
clear.
How
to
service
pattern
really
helps
us
in
Building
Services
use
and
allows
us
not
to
regulate
the
wheel
and
not
do
feel.
B
B
But
you
know
it's
very
very
the
case
of
becoming
a
smaller
model
from
an
optic
store.
Certainly
a
survey
the
very
easy
to
write
young
file.
He
provides
demo
Frameworks
and
events
or
genetic
passing
by
tenants
and
then
there's
a
new
features
like
all
the
scaling,
intelligent
routing,
pretty
fast
being
it's
going
to
build
the
monitoring.
B
B
But
is
this
enough
right
so
yeah
we
have
a
great
software,
okay,
sir,
that
is
getting
done
with
a
bunch
of
young
Fighters
or
even
you
know
if
you
want
to
applying
code
so
organic
right
if
we
have
our
Health
platform
with
bundle,
caser
and
stuff.
Well,
not
angry,
because
at
the
end
of
the
day,
every
organization
has
already
rules,
accidental
policies,
integrated
objects,
Explorer
influence,
satellite
services
interact
with
one
another,
and
eventually
all
these
details
will
surface
up
to
the
end
users,
because
you
have
to
take
them
into
consideration.
B
So,
let's
take
a
couple
of
examples,
say:
storing
a
model
and
an
opportunity
publishing
it
to
a
an
object
store.
Well,
you
need
to
start
thinking
about
how
your
sterilize
the
model
hand
it
back
a
bit
so
that
you
are
sure
that
the
case
of
wrong
time,
U.S
platform
is
support.
You
know
they
are
from
Africa
and
then
how
do
you
manage
the
main
culture
or
versioning
how
to
make
sure
that
the
hierarchy
or
building.
B
B
B
B
B
So
we've
seen
that
we
can
easily
deploy
models
and
that's
pretty
much
the
standard
and
most
friendly
way
to
do
that
in
our
current
understanding.
What's
right,
but
it
is
so
now
you
know
we
want
to
sort
those
go
to
the
next
step
and
understand
where
we
can
build
next
and
discuss
it.
This
is
this.
Is
this?
B
What's
next
for
us,
as
a
community
of
developers
that
want
to
meet
with
an
end
user,
to
leave
attorneys
and
build
a
Synergy
of
skill,
sets
that
come
from
the
devopsy
world
and
from
the
data
science
board
and
be
together,
and
he
was
really
bad.
It
was
SRE
and
developed
principles
like
Granite
fields
and
reproducible
runtimes
that
we
all
bottom
line
for
us.
B
Well,
this
is
an
open
question,
and
this
is
something
that
we
like
to
discuss
about
and,
after
a
full
day
of
thoughts
about
analogs
and
we've
seen
how
much
this
way
of
taking
resonates.
Many
of
you
and
it's
very
hard
people
to
see
that.
So
we
want
to
open
the
discussion
and
understand
what
you
think
about
it
and
where
we
are
supposed
to
go
and
continue
to
better
support
our
tool
change.