►
From YouTube: Machine Learning on Kubernetes at Shell: A Kubeflow Journey - Alex Iankoulski & Vangelis Koukis
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2021 Virtual from May 4–7, 2021. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Machine Learning on Kubernetes at Shell: A Kubeflow Journey - Alex Iankoulski, Shell New Energies & Vangelis Koukis, Arrikto
In this session, Shell describes the lessons learned from working with multiple Machine Learning platforms and tools, the challenges of different systems, why we chose Kubeflow, and how we are now delivering successful models faster and at scale. Follow our journey as we learned how to deploy highly available, scalable, and secure Kubeflow clusters in the public cloud. We will describe the lessons learned and steps taken to improve our deployments including enterprise authentication and authorization, network integration, and data science workflows. We also discuss why we moved away from other platforms and chose Kubeflow, and how it has increased our Data Scientists’ productivity and reduced DevOps overhead. Today our teams are more self-sufficient, and iterate faster to produce production-ready models in a timely fashion. A zero to hero story made possible by Kubeflow and Kubernetes
https://sched.co/ekBt
A
A
A
In
today's
presentation
we
will
first
take
a
look
at
the
business
context
and
use
cases
we
aspire
to
solve.
Then
we'll
discuss
the
technical
challenges
associated
with
these
use
cases.
We
will
share
some
lessons
learned.
We
will
dive
deeper
into
a
few
of
the
technical
details
and,
of
course,
we'll
look
forward
to
showing
you
an
interactive
demo
of
an
end-to-end
machine,
learning,
workflow
and
last
but
not
least,
we'll
have
a
live
q
a
session.
A
A
In
2015
in
paris,
france
195
countries
signed
an
accord
to
reduce
carbon
emissions
in
order
to
limit
the
rise
of
global
temperatures
to
under
2
degrees
celsius
above
pre-industrial
levels.
To
achieve
this
goal,
humanity
must
drastically
reduce
greenhouse
gas
emissions,
reaching
a
point
of
net
zero
emissions
within
the
second
half
of
this
century.
A
Experts
agree
that
global
energy
demand
is
likely
to
double
by
2050
compared
to
the
demand
in
the
year
2000..
At
the
same
time,
greenhouse
gas
emissions
must
be
drastically
reduced
in
order
to
get
climate
change
under
control.
That
is
why
shell
has
set
itself
an
ambition
to
become
by
2050
or
sooner
a
net
zero
emissions,
energy
business.
A
The
world's
energy
system
is
changing.
Shell
is
investing
in
more
lower
carbon
technology.
This
includes
renewables
such
as
wind
and
solar,
new
mobility
options
such
as
electric
vehicle
charging
and
hydrogen
and
an
interconnected
power
grid.
Shell
is
investing
up
to
two
billion
dollars
a
year
in
its
new
energies,
business,
which
focuses
on
developing
more
and
cleaner
energy
solutions.
A
I
work
at
shell,
new
energies,
building
the
energy
platform
and
digital
foundation
for
the
business.
I
invite
you
to
follow
this
link
on
the
slide
here
for
a
short
introduction.
Video
in
the
context
of
xiaomi
energy's
digital
foundation,
there
are
several
areas
where
machine
learning
can
shine
when
applied
to
solving
business
challenges.
Some
of
these
areas
are
listed
here.
A
Infrastructure
needs
to
be
cloud
native,
but
also
cloud
agnostic,
so
we
don't
have
to
rewrite
code
if
our
platform
needs
to
run
on
a
different
cloud
or
even
across
clouds
at
the
sum.
At
the
same
time,
deployments
need
to
be
reproducible,
auditable
and
reversible,
because
we
want
to
know
exactly
what
runs
in
our
systems
and
how
it
got
there
scale
needs
to
cover
the
entire
spectrum
from
micro
to
hyperscale.
A
A
B
Shell
came
to
us
with
these
challenges,
and
here
are
the
solutions
we
came
up
with
and
have
implemented
together
for
infrastructure.
We
opted
for
auto
provisioned
kubernetes
clusters
and
used
the
kubernetes
api
everywhere.
The
kubernetes
api
is
the
common
language
we
use
to
orchestrate
workloads,
regardless
of
the
cloud
region
we're
speaking
to
then
we
build
on
coop
flow
as
the
de
facto
way
of
running
machine
learning,
workloads
on
top
of
kubernetes
and,
more
specifically,
the
ericto
coopflow
stack,
which
combines
cupflow
with
rock
our
data
management
software.
We'll
talk
more
about
drug
later
on.
B
How
do
we
deploy
goodflow
on
top
of
different
kubernetes
clusters?
We
follow
a
gitobs-based
methodology.
Everything
starts
from
a
git
repository.
We
have
opted
to
steer
away
from
the
kf
cattle
tool,
so
we
can
deploy
in
the
simplest,
kubernetes
native
way
of
applying
manifests,
and
hence
we
can
support
seamless
upgrades
with
rollbacks
on
scale.
Kubernetes
can
run
from
a
single
laptop.
For
example,
there's
micro,
kubernetes
arito
delivers
its
own
single
node
deployment
of
cookflow
called
mini
coopflow.
We
encourage
you
to
try
it
out
with
shell.
B
We
run
on
managed
kubernetes
services,
more
specifically
eks
on
aws,
where
we
can
auto
scale
seamlessly
either
up
or
down
based
on
customer
demand
on
tooling,
we
follow
devops
practices
and
try
to
shift
left
users
run
their
own
code
servers,
for
example,
visual
studio
code.
They
run
their
own
jupyter
labs.
They
manage
their
own
code
on
a
git
repository,
for
example,
provided
by
git
lab.
B
Eventually
they
run
their
own
workloads
on
kubeflow
and,
finally,
on
compute
being
able
to
run
with
reproducible
results
means
using
containers
based
on
docker
images
inside
kubernetes
spots,
but
this
is
not
the
end-to-end
story
and
reproducibility,
because
what
about
your
actual
data?
The
things
that
you
use
as
input
more
on
this
later
on,
so
more
challenges
alex.
A
Sure
storage
needs
to
be
fast
and
cost
efficient,
so
we
can
decouple
compute
and
storage
challenges.
Data
should
be
secure
at
all
times
yet
available
for
authorized
users
and
versioned.
So
we
know
what
changes
happen
over
time
and
can
scroll
back
in
time
as
needed.
Security
must
be
end-to-end,
enterprise-grade
and
integrated
with
our
corporate
identities.
A
B
So
for
storage,
we
moved
from
a
shared
file
system
over
nfs
that
is
efs
on
aws
to
local
super
fast
mounted
file
systems
provided
by
rock.
We
will
focus
more
on
this
in
the
next
slides,
but
the
basic
idea
is:
local
storage
is
super
flexible.
Your
code,
just
accesses
local
files
and
the
performance
is
orders
of
magnitude
faster.
But
what
about
data
management
so
rock
sits
on
the
side
of
this
local
storage
and
gives
you
thousands
of
point-in-time
snapshots.
Think
of
a
time
machine.
B
Let's
say
we
snapshot
once
every
10
minutes
for
your
notebook
servers,
so
you
can
go
back
in
time
and
reproduce
the
data
for
all
experiments
you
run
or
a
snapshot
happening
at
each
step
of
a
pipeline.
So
you
can
know
exactly
what
series
of
events
led
to
the
creation
of
a
specific
model
and
then
you
can
investigate
any
biases.
B
B
What
we
generally
do
is
we
create
a
private
namespace
per
user,
but
we
can
also
support
shared
namespaces,
where
different
users
share
access
to
the
same
set
of
resources,
for
example,
the
same
pipeline
runs
and
finally,
on
orchestration,
we
implement
ml
ops.
We
combine
the
power
of
git
ko
our
open
source
tool
which
converts
notebooks
into
production
pipelines
and
then
orchestrates,
hundreds
or
thousands
of
these
pipelines
for
hyperparameter
tuning
and
eventually
serving
the
glue
that
brings
all
these
together
is
the
metadata
we
record
at
each
phase
of
the
workflow.
B
What
is
githubs
here
is
a
deployer
on
our
left
hand,
side,
and
here
is
a
kubernetes
cluster
where
they
want
to
deploy
kubeflow
on
the
right
hand,
side
githubs
is
all
about
a
git
repository.
It
sits
in
the
middle,
the
deployer
commits
their
desired
state
of
the
cluster
as
yaml
manifests,
and
they
only
apply
committed
manifests
to
their
kubernetes
cluster.
B
They
can
use
standard,
coupe
cattle
apply
or
even
customize,
we'll
talk
more
about
customized
later
on.
Why
is
this
important
because
they
treat
their
infrastructure
as
code?
The
state
of
their
infrastructure
corresponds
to
a
commit
in
the
repository
and
their
infrastructure
goes
from
commit
to
commit,
but
most
of
the
time
the
actual
manifests
come
from
a
vendor.
In
this
case,
it's
us
ericto,
who
produced
the
erector
confluence
stack.
So
let's
use
my
sql
as
an
example.
B
Arikto
itself
implements
gitos,
we
publish
generic
vendor,
manifests
in
a
vendor
repository
the
deployer
clones,
this
repository
into
a
local
repository.
Then
the
deployer
creates
deployment
specific
commits.
We
call
them
customizations
with
okay,
because
we
use
customize
with
them.
On
top
of
the
vendor,
commits
eventually
the
deployer
uses
customize
to
combine
these
manifests
into
the
final
desired
state
which
they
apply
onto
the
kubernetes
cluster.
B
Using
githubs
makes
upgrades
a
breeze.
Let's
assume
the
vendor
is
at
version
v2
and
the
deployment
has
committed
deployment,
specific
configuration
as
commit
d1
on
top
of
v2.
At
some
point,
the
vendor
produces
v3,
the
deployer
pulls
and
rebases
their
changes,
so
they
now
sit
on
top
of
v3
as
a
new
commit
d1
prime.
They
have
essentially
upgraded
their
infrastructure
and
can
now
reapply
next.
Let's
talk
about
storage
and
data
management,
shell
used
to
run
over
efs
on
aws
a
managed
file
system
over
nfs.
B
B
B
Rox
solves
the
reproducibility
problem
by
giving
you
automated
thin
application
consistent
snapshots,
so
you
can
go
back
in
time
and
reproduce
your
results.
Every
snapshot
is
essentially
a
git
commit
for
your
data,
not
just
your
code
rock
archives.
These
snapshots
into
object
storage,
for
example,
s3
in
the
case
of
amazon
on
performance.
B
If
we
compare
a
standard,
m5d
dot
for
extra
large
instance,
the
difference
is
huge.
Local
nvme
gives
you
16
times
the
io
operations
per
second
for
reads
more
than
400
times
the
I
operations
per
second
for
writes.
Bandwidth
is
18
to
21
times
better
and
the
aggregate
numbers
scale
with
a
number
of
instances.
B
A
B
As
parts
these
parts
have
a
direct
path
path,
a
via
the
kernel
to
local
storage,
rock,
sits
on
the
side.
It
monitors
the
I
o
operations,
so
it
can
retrieve
the
change
data
and
produce
a
new
snapshot
which
it
archives
into
s3.
This
is
path
b.
At
a
later
time,
rock
can
restore
data
from
this
snapshot
path.
C.
On
the
slide.
B
This
is
rock
running
on
a
kubernetes
cluster,
on
multiple
nodes,
all
coordinating
access
to
an
object
store
within
the
same
region,
but
it
really
becomes
interesting
when
you
look
at
rock
running
on
multiple
regions
as
independent
clusters.
In
this
case,
each
one
of
the
kubernetes
clusters
accesses
a
local
independent
object
store.
B
So
let's
look
at
this
example
where
rock
runs
on
an
amazon
zone
top
left,
a
google
cloud
zone,
bottom
left
on-prem
top
right
even
on
a
laptop
bottom
right.
Our
architecture
brings
all
of
these
regions
together
via
the
rack
registry,
and
it
allows
them
to
synchronize
their
snapshots
in
a
peer-to-peer
fashion
over
the
blue
links.
B
So
why
would
you
need
to
synchronize
your
snapshots?
That
is
your
data
commits
because
generally
different
parts
of
a
data
science
workflow
run
on
different
locations,
for
example
on
one
location
locally.
In
this
case,
then
you
move
to
the
cloud,
for
example,
amazon,
to
train
at
scale.
Then
you
run
inference
in
production
and
this
happens
at
multiple
locations.
B
A
Thanks
enterprise
standards
require
that
all
services
and
applications
are
secured.
We
need
to
use
the
same
user
identity
throughout
all
secured
assets.
Therefore,
we've
implemented
single
sign-on.
One
lesson
we
learned
in
the
process
is
that
it
is
not
necessary
to
integrate
all
of
our
applications
with
the
enterprise
identity
provider.
Instead,
we
can
use
a
self-hosted
oidc
provider
which,
in
turn
federates
with
the
enterprise
idp.
A
We
use
gitlab,
but
other
oidc
providers
can
be
used
as
needed
in
the
case
of
kubeflow.
This
means
that
we're
able
to
provide
each
user
their
own
isolated
namespace,
as
well
as
a
shared
namespace,
where
users
can
collaborate
on
projects
to
have
end-to-end,
enterprise-grade
security.
The
journey
continues
a
step
further.
In
addition
to
single
sign-on,
we
have
implemented
a
single
log
out
functionality
which
allows
us
to
log
out
directly
from
our
application
screens.
The
last
van
gaal
is
to
explain
how
we're
able
to
do
this
within
tube
flow
and
then
dive
into
the
orchestration
details.
B
Thanks
alex,
so
we
have
extended
kubflow,
so
it
becomes
one
more
oidc
client
via
component.
We
call
the
oidc
of
service.
This
component
integrates
closely
with
istio
and,
more
specifically,
the
envoy
proxy
inside
istio.
I
encourage
you
to
learn
more
about
this
architecture
by
following
the
kubeflow
docs
and
the
number
of
blog
posts
we
have
made.
B
B
And
they
have
a
description
of
their
ml
pipeline
inside
the
jupiter
lab
notebook.
Each
one
of
the
cells
or
a
group
of
cells
corresponds
to
a
different
step
of
the
pipeline.
For
example,
data
pre-processing
model
building
model
training,
model
evaluation,
steps,
one
to
four,
our
open
source
tool
packages,
a
notebook
or
even
a
python
script,
which
is
what
we'll
demo
later
on
into
a
kubeflow
pipeline
here,
is
the
compiled
kubeflow
pipeline
kale
then
orchestrates
this
compiled
pipeline
for
hyper-parameter
tuning
it
spawns
hundreds
or
thousands
of
runs
to
find
the
best
combination
of
hyper
parameters.
B
B
A
Well,
the
reasons
to
choose
qplo
are
obvious:
it
runs
natively
on
kubernetes
in
a
scalable
and
reliable
way.
It
is
secure
and
integrates
with
external
identity
providers.
It
runs
both
on
your
laptop
and
in
the
cloud
you
can
use
the
open
source
version
or
purchase
a
commercial
version
with
enterprise
support
it
integrates
with
the
way
you
currently
do.
Your
work
streamlines
and
accelerates
your
ml
workflows.
One
example
would
like
to
share
here
is
a
case
when
we
needed
to
build
10
000
models.
A
Normally
the
time
required
to
do
this
work
manually
would
involve
about
two
weeks
of
coding
and
four
weeks
of
execution
with
kubeflow.
We
were
able
to
write
the
code
in
less
than
a
day
and
build
all
models
within
two
hours.
Seeing
is
believing
so
let
us
jump
into
my
favorite
part
of
this
presentation.
A
A
I've
heard
that
we
need
to
sit
together
and
iterate
on
building
a
container,
so
you
can
then
build
a
pipeline
then
use
the
pipeline
to
run
the
code
on
a
schedule
and
if
something
is
not
working,
we
need
to
go
back
to
step
one,
I'm
really
nervous,
because
this
sounds
like
it
will
take
a
while,
and
I
need
to
be
done
before
6
p.m.
Today,.
B
A
B
B
B
B
A
B
B
B
B
It
you're
now
running
against
google
pipelines
at
this
point.
Kale
compiles
your
code
into
a
kubflow
pipeline
and
it
pushes
it
to
kubeflow
pipelines.
It
uses
rock
to
take
a
snapshot
of
your
jupiter
lab
volumes
automatically,
so
your
pipeline
can
be
completely
reproducible
in
terms
of
both
your
code
and
your
data.
B
A
I
see
that
the
pipeline
was
generated
here
and
I
recognize
all
the
steps
from
my
workflow.
So
first
we
scrape
the
data
from
kaiser's
public
reports
on
energy
usage
and
then
we
pre-process
that
data.
We
split
the
data
set
into
training
and
testing
set.
Then
we
use
a
few
techniques
to
build
models
and
rank
them,
and
then
we
train
the
best
model
with
the
full
data
set
and
predict
30
years
into
the
future.
A
B
B
A
A
It
would
be
interesting
to
watch
and
see
how
this
prediction
changes
over
time
and
that's
why
I'll
schedule
this
pipeline
to
run
automatically
every
month
angelus.
I
can't
believe
how
easy
this
whole
process
was,
and
I'm
so
happy.
It
didn't
take
much
time
at
all
I'll,
be
orchestrating
all
my
ml
workflows.
Only
this
way
going
forward.
Thank.
A
A
And
this
is
the
kind
of
experience
our
teams
love.
Today
we
share
challenges
and
lessons
learned,
while
building
scalable
and
highly
available
mlaps
infrastructure
applied
to
real-world
use
cases.
We
discussed
how
kubeflow
and
the
erykto
stack
help
us
solve
these
challenges
and
demonstrated
a
data
science
workflow
from
xero
to
hero
made
possible
by
kubernetes
and
kubeflow.