►
From YouTube: Understanding Isolation Levels in the Kubernetes Landscape - Jiaqi Liu, University of Chicago
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Understanding Isolation Levels in the Kubernetes Landscape - Jiaqi Liu, University of Chicago
In building open-source software with Kubernetes, it becomes important to understand the support and limitations for isolation and security at different levels. The ecosystem can be complex and it might be challenging to verify or fully understand the guarantees for isolation at each layer - from the cluster level to the container level. More importantly, how do you know what level of isolation you need at each level? Maybe if your cluster is secure, there is less of a concern for container level isolation? This talk will go over the impact and tradeoffs for optimizing for isolation at a given layer and help you understand what can be done at the cluster level, the namespace level, the pod, the container. As an example, this talk will present the case study of running Jupyter Notebooks within Kubernetes and supporting the requirement to provide isolation between each pod in order to create separate user spaces for each notebook launched in the cluster.
A
Cool
on
the
agenda
is
multi-tenancy
and
isolation
and
a
little
bit
of
introduction
to
that,
and
then
diving
deeper
into
the
components
and
isolation
layers
within
kubernetes,
that's
supported
by
kubernetes,
and
then
I'll
talk
a
little
bit
about
the
particular
project
that
I
worked
on,
that
focus
more
on
container
level
and
pod
level.
Isolation
going
straight
into
talking
about
multi-tenancy
and
isolation.
A
So
what
is
multi-tenancy
in
a
single
tenant
architecture,
each
tenant
or
user-
have
their
own
instance
of
their
own
cluster.
So
in
this
diagram,
you'll
see
that
user
one
has
their
own
cluster,
which
has
a
control,
plane
and
worker
nodes
user.
Two
is
completely
unaware
of
using
one's
cluster
and
has
their
own
cluster
to
work
in
as
well.
A
So,
in
the
diagram
on
the
right
in
a
multi-tenant
architecture,
user,
1
and
user
2
are
sort
of
both
using
the
same
cluster,
they
might
be
sharing
a
control
plane.
There
might
be
multiple
control
planes
and
they're,
sharing
the
the
worker
neural
resources
and
pulling
together
those
resources.
So
why
do
we
care
about?
You
know
multi-tenancy
what
you
know.
Why
do
we
want
it?
A
A
A
So,
ultimately,
the
the
trade-off
is
between
cluster
management,
complexity
and
isolation
in
the
the
far
right
example,
where
you
have
one
shared
cluster
where
access
is
is
very
liberal
to
or
everyone
has,
the
same
level
of
access-
that's
the
least
amount
of
cluster
management
complexity,
but,
on
the
other
hand,
there's
also
no
isolation
in
the
single
tenant.
You
know
use
case
where
everyone
sort
of
has
their
own
cluster.
A
It's
the
most
amount
of
isolation,
but
it's
a
lot
of
overhead
and
complexity.
To
maintain
and
along
the
lines,
you
might
decide
that
some
things
are
worth
sharing.
Some
resources
are
worth
sharing,
but
some
other
constraints
need
to
be
in
place,
so
there's
a
constant
trade-off
between
complexity
and
isolation
and
then
there's
no
perfect
isolation
scenario.
A
So
things
like
you
know
having
web
application
firewalls
for
your
external
apis,
restricting
access
for
for
internal
apis
to
a
given
network.
They
all
sort
of
provide
different
layers
of
protection
so
being
realistic
and
practical
about
what
your
security
policies
are
and
and
making
sure
that
you're,
creating
a
sustainable
development
environment,
also
sustainable
environment.
For
for
your
team
to
understand
and
manage
the
the
layers
is
really
important.
It's
not
always
about
you
know.
Everything
has
to
be
always.
Everything
has
to
be
locked
down,
but
understanding
the
big
picture
as
well.
A
So
that's
something
that
I
really
wanted
to
highlight,
but
the
majority
of
this
talk
is
going
to
focus
on
the
kubernetes
level
and
what
you
can
do
with
your
kubernetes
cluster,
okay.
So,
let's
jump
into
just
that,
I'm
going
to
introduce
these
topics
by
by
going
over
the
basic
atomic
primitives
within
kubernetes,
because
I
think
that
these
are
are
really
important
to
to
build
upon
to
talk
about
isolation.
A
A
Pods
are
then
deployed
to
nodes
which
are
at
the
virtual
machines
serving
as
the
workers
and
a
cluster
will
contain
multiple
nodes.
The
important
thing
to
know
here
is
that
you
can
run
multiple
containers
per
pod.
If
your
application,
logically,
you
know,
is
separated
into
different
multiple
containers.
A
So
what
isolation
layers
does
kubernetes
provide
you
at
the
top
layer
you
sort
of
have
the
the
workload
within
that
you
are
sharing
a
pool
of
nodes,
but
a
given
you
know,
example
of
isolation
might
be
that
different
tenants
might
have
priority
on
resources
or
the
number
of
nodes
allocated
to
them
or
how
scaling
out
works,
but
from
a
architecture
perspective,
it's
really
abstracted
away.
There's
a
shared
pool
of
nodes
that
anyone
can
theoretically
access.
A
A
lot
of
this
is
going
to
be
shared
across
the
tenants
and
one
of
the
benefits
of
having
a
multi-tenant
system
is
you
can
have
the
same
logging
same
monitoring?
You
can
share
these
platform
services.
You
can
upgrade
them
consistently
and
you
have
the
same
use
cases
across
the
different
users
on
your
platform.
A
Okay,
so
the
namespace
primitive
is
the
the
key
way
for
enforcing
and
implementing
isolation
at
each
layer,
kubernetes
doesn't
define
what
a
mult
tenant
is
in
a
multi-tenant
architecture.
So
it's
really
up
to
you
to
to
come
up
with
a
model
that
makes
sense.
For
example,
you
could
have
you
know,
namespaces
divided
by
a
user,
each
user
on
your
platform
has
their
own
namespace
or
maybe
your
tenant
is
a
team
within
your
organization,
or
maybe
your
tenant
is
actually
different
organizations
all
together.
A
Those
are
kind
of
user-based
ways
of
dividing
up
a
cluster
into
different
name
spaces
and
defining
what
a
tenant
means
to
you.
Other
examples
are,
you
could
also
have
different
name
spaces
for
for
different
environments
such
as
you
know,
a
qa
staging
or
production
environment.
You
could
have
different
name
spaces
for
different
applications.
If
you
have
you
know,
a
different
microservices
may
choose
to
live
in
in
their
own
namespace.
So
a
lot
of
different
ways
to
divide
up
what
attendant
means
and
kubernetes
is
not
prescriptive
about
it.
A
Okay,
so
the
namespace
allows
you
to
organize
your
cluster
and
define
what
attendant
means
and
then,
who
gets
to
share
this?
What
policies,
but
a
namespace
by
itself
doesn't
create
those
policies.
So
there's
no,
no
restrictions
when
you
create
a
namespace
on
whether
or
not
a
user
of
that
namespace
contributes
to
another
namespace.
A
You
know
resources
and
authorize
api
access,
so
at
a
basic
layer.
Maybe
you
know
in
namespace
a
these
are
these?
Are
the
you
know,
users
that
can
access
namespace
a
but
cannot
access,
namespace
b
and
creating
that
foundational
policy
using
things
like
resource
quotas
at
the
name,
space
level
to
constrain
cpu
memory
and
storage?
A
One
really
interesting
and
frequent
use
case
is,
you
might
need
to
tie
certain
nodes
to
a
particular
namespace,
like
I
have
done
in
this
diagram
that
doesn't
come
by
default,
but
you
can
definitely
use
the
pod
node
selector
to
implement
a
way
such
that
certain
pods
are
only
launched
on
certain
nodes
for
a
given
name
space.
So
those
are
examples
of
controls
that
could
be
potentially
useful
in
creating
isolation
at
the
name,
space
level
and
there's
plenty
of
different
things
that
you
can
do
and
configure
which
are
really
interesting
within
kubernetes.
A
Okay,
so
a
note
on
namespaces
in
most
kubernetes
deployments,
tenants
are
authorized
to
list
all
name
spaces
on
the
cluster,
so
this
is
something
that,
if
your
use
case
requires
tenants
to
not
be
aware
of,
who
else
is
on
the
cluster
name?
Spaces
might
not
be
the
isolation
level
that
works
for
you,
and
there
are
other
tools.
There
are
other
open
source
tools
available
that
you
might
need
to
look
into.
A
A
Okay,
I'm
going
to
talk
a
little
bit
more
at
the
container
level,
so
containers
overall
create
separation
between
workloads,
even
if
they're
hosted
on
the
the
same
node
within
kubernetes
containers
run
on
pause
and
each
pod
is
a
logical
unit
of
an
application
and
can
actually
contain
one
or
more
containers
and
a
pod
is
sort
of
deployed
onto
a
node,
okay
containers
by
nature
or
we
think
of
them
as
developers.
We
sort
of
think
of
them
as
vms
they're
isolated.
But
you
can
very
easily
break
container
isolation.
A
A
But
it's
genuinely
best
practice
to
to
run
as
a
non-root
user.
And
if
you
do
need
to
install
software
on
the
container,
you
can
install
the
software
first
and
then
actually
run
the
application
as
a
non-root
user.
So
that's
something
to
to
take
note
of
the
other
way
that
you
can
actually
break
container
isolation
is
with
any
kind
of
vulnerabilities
built
into
dependencies
or
or
code
that
you're
installing
onto
your
container.
A
A
So
the
idea
behind
this
is
you
want
to
treat
your
containers
as
immutable
objects
and
don't
download
code
on
runtime.
That
also
means
that
you
wouldn't
have
to
run
your
containers
as
root
on
runtime
and
be
able
to
store,
build
your
container
and
load
it
into
a
container
registry
where
you
can
just
scan
the
containers
in
the
container
registry
and
scan
for
vulnerabilities
there.
A
Instead
of
every
container
on
the
cluster
before
the
container
itself
is
deployed
to
kubernetes
or
to
your
cluster,
you
can
even
use
a
tool
like
open
policy
agent
which
can
actually
enforce
emission
control
based
on
the
scan
results
coming
out
of
your
vulnerability
scan
so
that
you're
not
deploying
compromised
containers
to
your
cluster
okay.
So
those
are
sort
of
my
you
know
two
really
important
things
to
highlight
about
containers
and
container
security.
There's
there's
quite
a
lot
more
and
I've
left
some
resources
at
the
end.
A
A
So
our
use
case
was
that
we
were
enabling
users
to
launch
their
own
jupyter
notebooks
into
in
a
browser
setting,
so
that
they
can
run
analysis
on
biomedical
data
and
the
flow
on
the
backend
kind
of
looked
like
this.
The
user
would
go
to
a
web
application
where
they
would
request
a
jupyter
notebook
and
they
would
be
told
what
the
cpu
and
memory
constraint
is
behind
the
scenes.
We
had
an
application
called
hatchery
that
would
call
the
kubernetes
api
and
launch
a
distinct
pod
for
that
user,
and
this
pod
is
assigned
to
to
that
user.
A
A
Hatchery
is
the
open
source
tool
that
will
deploy
a
pi
to
kubernetes
so
that
the
user
can
execute
their
own
code,
but
hatchery
in
that
process
denies
the
user
or
that
pod
access
to
the
kubernetes
api
and
internal
apis
through
network
policies,
and
it
also
imposes
a
constraint
on
cpu
and
memory
usage
per
pod,
so
that
in
case
there
is
a
really
heavy
workload
on
a
particular
jupiter
notebook
that
you
know.
Maybe
a
user
is
running
a
heavy
computation
on
on
data.
A
A
So
that's
sort
of
it.
For
my
talk,
I,
like
I
mentioned
before,
there's
a
lot
of
really
great
resources.
In
case
you
do
want
to
be
learn
more.
I'm
going
to
call
out
running
with
scissors,
which
is
a
talk
by
liz
price,
at
a
go
at
go
con
gophercon
where
she
did
talk
a
lot
about.
You
know,
containers
and
root
access,
and
I
also
want
to
call
it
hatchery
the
open
source
project
which
is
on
github,
okay.
A
A
But
it's
really
up
to
you
to
come
up
with
a
model
for
multi-tenancy
and
and
isolation
that
makes
sense
to
you
and
your
team
and
make
sense
for
the
broader
architecture
for
what
for
which
your
deployment,
your
kubernetes
deployment
is
in,
and
it's
really
important
to
decide
what
those
tradeoffs
are
and
understand
them,
because
you
don't
want
to
over
engineer
your
deployment.
You
don't
want
to
over
engineer
your
application,
but
I'm
hoping
that
at
least
in
this
talk
it
gave
you
a
sense
of
what
you
know:
basic
kubernetes
tools
and
container
level
tools.