►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Manage Multi-tenant ML Workloads Using Istio - Wencheng Lu & Limin Wang, Google
With rapid growth of machine learning workloads deployed on Kubernetes, it is becoming a popular demand to offer a multi-tenant pipeline to manage machine learning workloads that facilitates different data scientists to collect data, train and serve models on kubernetes. Come learn how Istio can be integrated into a multi-tenant machine learning pipeline like Kubeflow to provide isolation and protection of workloads deployed for different users through sufficient identity, access, and api management
https://sched.co/Nrvs
A
A
A
A
There
are
three
main
challenges
on
service
communications,
monitoring,
connectivity
and
security.
I
use
their
solve.
Also
the
problems
for
you
first
is
they
provide
you
uniform
visibility
into
your
service
communications?
You
get
latency
requests
patterns.
Error
is
all
sorts
of
telemetry.
You
want
you
to
know.
A
Second
is
operational
agility,
which
is
mostly
about
traffic
management,
so
easy
to
provide
you
advanced
load,
balancing
traffic,
shifting
traffic
splitting
circular
breaking
in
to
help
you
manage
your
charcoal
easily
and
the
last
thing
is
used
to
provide
the
security
you
needed
to
secure
service
communication
and
plus
declarative
policy,
which
you
can
easily
describe
your
intent.
Now
you
can
easily
turn
on
materials
to
enable
data
in
transit,
encryption,
class
of
sanitation,
authorization
and
audit
to
protect
your
service.
A
A
It's
the
integration
point
to
collect
and
report
metrics
who
or
different
backends
like
Prometheus
Gravina,
whatever
monitoring
system
or
login
system
tracing
system
you
like
to
use.
So,
though,
it's
a
key
to
manage
accuser
provision,
kisser
every
single
workload,
and
then
you
can
ascribe
humidity
as
to
see
to
secure
the
connection.
A
You
store
the
raw
data
in
some
persistent
storage
like
GCS,
bucket
Amazon
s3,
and
then
you
ingest
data
to
emotionally
workloads
that
analyze,
transform
and
split
data
to
the
format
you
can
can
be
consumed
by
the
second
stage
like
Peter
Collins.
The
second
stage
is
changer
models
that
include
selected
right
models,
hyper
parameter,
tuning
and
model
has
been
and
model
validation,
how
they
finish
the
machines
we
move
to
the
third
stage,
then
you
deploy
a
model
and
cell
model
as
prediction
service
in
a
multi-tenant
environment.
A
Each
tenant
may
want
have
its
own
pipeline
to
random
motion
on
your
clothes.
So
here
comes
the
challenge
of
my
dependency.
First
on
security,
you
want
also
the
isolations
each
tenant
may
want
the
Macmillan
provide
isolation
on
operators.
The
operator
can
only
manage
it
to
workloads
and
resource
for
your
own
tenants.
You
don't
want
operator
from
another
tenant
to
be
able
to
touch
your
workload
SEC.
Second,
in
isolation
of
communication,
you
only
want
communications
within
your
tenant.
Workload
can
talk
to
workloads
within
the
same
tenant,
not
across
tenants.
A
There
might
be
some
I
mean
data,
you
don't
want
to
expose
to
abstinence
and
the
last
ones
isolation
we
user
access.
So
you
don't
want
to
like
tenon
on
attendants
users
to
access
your
prediction
service
to
access
your
notebook,
the
second
event
operational
agility.
So
after
you
build
a
new
models,
you
can
help
you
easily
test
and
verify.
How
good
is.
The
matter
is
so
easy
to
provide
traffic
splitting
traffic
mirroring
after
verifying
it's
a
good
model.
Then
you
can
leverage
easier
to
roll
out
the
new
model
safely
and
easily.
A
Each
tenant
header
has
its
own
dedicated
namespace,
basically,
the
rongzi
workloads
on
their
namespace.
We
are
talking
more
about
how
q9n
space
can
be
leveraged
to
do
add,
resource
and
workload.
Isolation
then
Israel
order
inject
my
to
each
workload.
Along
with
the
ether
control
plane,
the
easel
provide
the
security
traffic
management
and
of
the
boot
observability.
You
needed
to
manage
this
machine
on
a
workload.
B
Yeah
so
I'm
going
to
do
a
case
study
come
follow
to
show
you
how
you
can
use
SEO
to
manage
multi-tenant
machine
learning.
Workloads
cope
flow
is
machine
learning
toolkit
for
communities.
A
control
pipeline
is
a
platform
for
building
deploying
and
managing
machine
learning
workflows.
A
typical
control
pipeline
included.
The
following
steps
first
unit
will
create
a
kubernetes
cluster.
Then
you
need
to
prepare
the
training
data
by
deploying
workloads
like
tensorflow
analyze,
tensorflow,
transform
to
analyze
clean
and
the
pre-processing.
B
This
training
data
and
the
next
step
is
for
change
of
measurement
Lenny
model
and
you
can
achieve
that
by
deploying
intensive
flow
training,
jobs
and
the
last
step
is
to
serve
your
machine
learning
model
and
by
deploying
tensorflow
serving
jobs.
So
I
just
used
the
tensorflow
as
a
one
example.
Cook
flow
path,
support
any
machine
learning
frameworks,
and
you
know
tensorflow
is
just
one
of
them.
B
In
multi-tenancy
scenario,
each
tenant
machine
and
workloads
are
managing
a
waiting
attack,
Ikeda
kubernetes
namespace,
for
example,
user
Alice
resource
should
always
be
deployed
in
a
namespace
called
cool
flow.
Ares
kubernetes
Arbor
can
be
used
to
control
the
operator
operations.
For
example,
it
can
control
who
can
deploy
the
machine,
any
workloads
who
can
run
the
machine
and
workloads
and
who
can
configure
the
machine.
A
new
workload,
for
example
like
a
set
clutter
and
set
access,
control
policy
or
such
traffic
policy.
B
B
Easter
provides
the
add
user
authentication
specifically
chosen.
Caching
for
any
open
ID
connect
provider,
for
example,
google,
austero,
Facebook
and
II
still
provides
the
identity
based
authorization
to
support
service
level
and
the
namespace
level
segmentation
at
both
the
HTTP
and
the
TCP
layer
and
Easter
support
service
to
a
service
and
end-user
to
a
service
authorization.
B
B
B
So
now,
let's
take
a
look
at
how
is
the
end
user
authentication
feature
can
be
used
for
secure
and
isolate
user
access,
machine
learning
workload.
Let's
consider
a
scenario
that
and
the
user
tries
to
access
machine
learning
workloads
like
a
serving
job.
The
end
user
request
arrives
at
Easter
ingress.
The
ingress
will
probably
request
to
a
survey.
Scada
is
the
earth
service.
B
If
there
is
no
end
user
credentials,
the
or
service
will
redirect
the
user
to
lobbying
with
an
identity
provider,
for
example
Google
login
github
logging.
If
the
logging
is
successful,
it
will
return
a
Josh
token,
which
we
call
request
access,
request,
context
token
or
as
a
token,
as
a
token,
contains
all
the
information
about
the
sauce,
for
example,
sauce,
IP
and
also
the
end-user
credential,
as
a
token,
is
the
forwardable
between
the
workloads
and
it
is
short-lived,
George
:
and
it's
very
the
only
insider
on
insta
mesh.
B
B
The
next
step
is
the
oscillation
Easter
provides
oscillation
picture
that
can
be
used
for
authorized
post,
channel
identity
and
the
NGOs
identity.
Some
typical
use
case
included.
It
can
authorize
a
developer
for
access,
a
Jupiter,
no
spoken.
It
can
also
authorize
them
and
the
user
to
access
a
model
serving
job
I
ensure
an
example
of
is
the
authorization
policy
here.
On
the
left
side,
we
define
a
row
called
a
least
serving
row.
It
allows
you
to
read
the
serving
workload
in
full
flow
areas.
Namespace.
B
B
You
can
use
easier
to
monitor
machine
learning
workloads
for
each
tenant.
He
still
allows
you
to
plug
in
any
money
to
infections
such
as
stackdriver
promises,
and
you
can
define
policies
to
configure
what
metrics,
what
stance
you
would
like
to
log
the
money
train
backhands
like
her
stackdriver,
provide
isolation
between
tenants.
For
example,
you
can
specify
the
access
control
policy
such
that
it
is
user.
Alice
only
sees
logs
and
metrics
in
coop
flow,
Alice,
namespace.
B
Is
your
traffic
management
feature
provides
a
lot
of
nice
functionalities
such
as
traffic
splitting
load,
balancing
retries
fire
burn,
for
example,
you
can
configure
policies
such
that
the
front-end
and
interface
send
the
95%
of
traffic
to
be
one
service,
and
the
remaining
5%
of
traffic
is
sent
to
be
twisted
little
workload,
and
you
can
also
traffic
splitting
based
on
the
request.
Content,
for
example,
requests
request
header,
you
can
say
if
the
request
comes
from
an
Android
device,
then
it's
12,
you
want.
B
If
you
want
workload
if
the
traffic
comes
from
Apple
device
same
is
true
with
whichever
video
workload.
So
by
doing
so,
you
can
achieve
some
functionalities
like
automatically
roll
out
to
new
versions.
Do
a/b
testing
for
different
machine,
any
models,
separate,
shame
of
a
staging
environment
and
the
production
environment
and
a
lot
more
all
these
can
be
configured
on
throw
traffic
policies
and
I.
Just
that
is
our
show
our
traffic
policy
here
there
is
no
application
changes
required
and
there's
no
traffic
disruption.
A
A
D
B
C
A
Yeah
so
depends
on
your
use
case
and
depends
on
the
performance
requirement
of
your
application
so
with
the
asam
performance
benchmark.
So
my
performance
II
actually
pretty
good,
it's
very
it's
equivalent
to
nginx,
which
is
like
a
high
performance
proxy
with
good
reputation.
The
latency
overhead,
as
wheat
has
the
it's
part
of
100
mark
ii
and
the
civil
cost
is
at
the
same
level.
So
if
your
applications
can
totally
the
100
microsecond
overhead
per
hop
I
think
that's
fine.
How
do
you
feel
application,
something
like
Cassandra
and
each
transaction
is
also
at
the
microsecond
level?