►
From YouTube: Kubernetes Multitenancy Working Group 20210406
Description
Vigorous discussion about hostile multi-tenancy versus soft/hard concepts.
A
B
B
C
A
Okay,
so
adrian
you
just
reminded
me,
I
had
completely
forgotten
about
that.
So
where
this
came
from
was
someone
at
microsoft,
who's
very
senior
was
was
tweeting
and
blog
posting
about
hostile
multi-tenancy
versus
what
he
was
calling
enterprise
multi-tenancy,
and
I
just
thought
that
that
was
a
really
odd
way
to
describe
it,
because
what
we
talk
about
in
this
working
group,
the
language
that
we've
sort
of
coalesced
around
historically,
has
been
soft
versus
hard
multi-tenancy.
And
so
the
idea
is
that
soft
multi-tenancy
is
where
you
can
sort
of
trust
the
other
users.
A
So
I
thought
that
was
kind
of
weird
and
then
the
other
piece
is
you
can
accidentally
ddos
your
co-workers
and
it
still
hurts
just
as
much
as
if
you
intended
to
and
so
not
having
like
those
lower
level
controls
and
like
separation
and
like
trying
to
kind
of
control
the
firestorm,
when
you
have
like
more
than
say
two
people
working
in
any
any
environment
or
even
just
two
like
that,
was
like
a
little
odd.
So
I
think
I
went
on
this
rant
on
slack
and
then
adrian
was
like.
A
B
I
think
I
would
agree
it's
like
I
mean
I
have
my
own
problems
with
soft
and
hard,
but
as
long
as
we
understand
that
they're
on
a
spectrum
like
it's
not
one-dimensional,
it's
you
know
at
least
one
and
a
half
dimensional,
but
it's
like
as
long
as
we
understand
that
it's
on
a
spectrum.
I
think
that
that
it
becomes
a
lot
more
palatable,
at
least
for
me,
to
talk
about
the
degrees
to
which
you
trust
the
other
people
on
your
cluster
and
yeah.
So
it's
not
just
you
know
external
actors.
B
It
could
be
internal
threats
as
well.
It
could
be
accidental
or
malicious
and
and
like
simple
scale,
can,
can
drive
a
lot
of
that
as
well.
So
but
yeah
like
when.
I,
when
I
think
about
the
types
of
multi-tenancy
that
are
like
completely
distinct,
I
think
of
things
like
multi-team
tenancy,
where
everybody
who's
using
the
cluster
is
a
team
that
you
know
are
working
at
the
same
company.
B
So
that's
one
whole
like
attack
surface,
that
you
can
basically
just
shut
down,
but
they
you
can
assume
that
they
are
mutually
hostile
towards
each
other
like
actively
hostile,
perhaps
to
a
greater
extent
than
you
would
for
different
teams
in
a
small,
let's
say
like
wordpress
hosting
business,
and
so
in
that
case
you
need
a
lot.
You
might
not
need
like
control,
plane.
Protections
like
are
back
or
network.
Well,
maybe
network
policies,
but
you
don't
need
things
like
our
background
quota
as
much,
but
you
certainly
need
things
like.
B
B
D
The
one
that,
throughout
the
hypervisors
and
and
the
problem
with
g
visor
right
is
it's
a
virtualized
kernel,
but
you
still
don't
you're
not
guaranteed
like
cpu
level,
isolation,
yeah
and
so
that's,
that's
kind
of
where
the
where
that
came
from
is
like.
If
you
care
about
the
hardware
level
cpus
right
now,
they
literally
only
have
provisions
for
hypervisors
to
give
you
that
isolation,
and
so
that's
where
that
comes
in.
But
that
being
said,
both
intel
and
amd
are
working
on
container-based
isolation
at
the
cpu
and
memory
level.
So
that's
something.
E
F
C
A
E
A
But
because
of
the
way
you're
phrasing
it
you're
like.
Well,
you
don't
need
this
hostile
thing.
You
need
enterprise
multi-tenancy,
that's
kind
of
like
well,
not
really
sure
about
that,
but
yeah
I
don't.
I
don't
think
I
I
have
a
feeling
it
was
sort
of
like
a
random
post
like
I
don't.
I
would
never
presume
that
this
is
like
microsoft's
governing
strategy
as,
like
you
know,
with
regards
to
like
how
to
actually
achieve
secure
cloud,
but
I
actually
adrian
when
I
put
this
as
our
topic.
A
G
D
But-
and
I
mean
I
think,
kubernetes
in
general
has
made
it
clear
that
hard
multi-tenancy
is
guaranteed
to
be
impossible
right
now,
kubernetes,
like
the
only
form
of
of
all
the
tents
you
can
have
and
kubernetes
technically,
depending
on
you
know
your.
I
guess
the
attack
vectors
you're
worried
about,
but
it's
like
soft
multitasking
is
really
all
that
is
available
from
a
single
cluster
perspective
right,
so
you
yeah
like
you,
you
want
to
lock
it
down
as
much
as
you
can,
regardless,
because
you
know
it's
like
you've
got
one
cluster
right.
D
D
No,
so
they
they
had
a
bug
right
where
they
at
the
at
the
control
plane
level.
They
were
using
default
ports
and
they
were
properly
securing
it,
and
so
it
was
just
like
that.
Make
bitcoin
miners
were
just
literally
scraping
the
internet
looking
for
open
ports
and
they
were
injecting
bitcoin
containers
into
mesos
clusters
in
production.
It
happened
to
a
lot
of
people,
and-
and
you
know
that
that
sort
of
thing
is
possible,
and
so
you
need
to
treat
your
kubernetes
cluster
like.
D
C
A
A
What,
I
would
say
is
the
so
we
have
kind
of
different
people
trying
to
solve
different
problems,
who
are
all
part
of
this
group,
so
I
think
adrian,
the
set
of
problems
that
you're
trying
to
solve
might
align
the
most
with
kind
of
what
tommy's
thinking
about,
because
you're
basically
running
like
the
gke
service
right
and
so
like
that's
more
like
sas
where
people
are
coming.
Is
that
what
you're
thinking
tommy's
like
pass,
or
are
you
thinking,
like
I'm,
a
sas
provider
who
happens
to
use
kubernetes
to
deploy
containers.
D
D
So
anything
like
that,
where
you
can
potentially
have
you
know
access,
you
know
breaking
out
of
the
container
or
breaking
out
of
the
application
to
get
access
to
the
container
if
they
have
open
access
to
do
whatever
right
like
the
service,
the
service
account
can
list
all
namespaces
or
whatever.
Then,
eventually,
you
know
with
enough
effort
that
you
know
they
could
break
through
and
so
you're.
I
mean
you
really
at
that
level,
only
as
safe
as
the
application
you're
running
in
that
container,
and
that's
why
you
know.
G
Exactly
but
that
could
also
lead
to
access
to
the
control
plane
right,
because
if
you
can
get
out
of
the
container
you
can
access
secrets
in
the
namespace
or
let's
say
you
get
to
any
host
resources
that
gets
you
to
other
config.
If
you
happen
to,
you
know,
figure
out
where
the
control
plane
was
nodes
are
running,
you
can
even
get
to
fcd
from
that
right.
G
B
G
Yeah,
so
to
me,
I
don't
I
don't
really.
You
know,
I
feel
both
in
both
cases,
whether
even
if
the
end
consumer
is
using
the
application
as
a
sas,
the
tendency
models
are
still
applicable
and
still
seem
just
as
important,
because
if
you
don't
have
the
right
controls
underneath
any
any
exploit
in
the
or
any
vulnerability
in
the
container
which
is
exploited,
could
lead
to
bigger
challenges
and
issues
like
what
we're
describing
the
other.
The
other
kind
of
comment
on
that
is
even
if
the
end
consumer
is
using
the
application
as
a
sas.
G
Typically,
what
we've
seen
is
there
are
customer
success
teams
and
other
teams
assigned
to
different
customers,
and
they
may
need
access
to
just
those
name
spaces
to
manage
that
customer
right.
So,
ultimately,
you
still
need
to
figure
out
if
you're
doing
clusters
as
a
service
or
namespaces
as
a
service
each
you
know
each
kind
of
model
has
to
be
secured.
B
Correctly,
if
I
remember
there
was
this
great
talk,
I
think
at
cubecon,
north
america,
2019
or
so
called
like
what,
if
your
attacker
knows,
parkour
and
it's
like
what?
What
can
you,
what
like,
what
kind
of
mischief
can
you
get
up
to
if
you
can
escape
the
container
on
on
the
node?
Was
that
the
the
keynote
from.
D
D
F
B
Know
what
you're
talking
about
yeah
all
right?
It's
got
the
word
parkour
in
it.
It's
pretty
easy
to
search
on
youtube,
but
it
was
a
great
talk.
I
can't
remember
like
if
you
can
escape
that.
If
you
can
escape
your
container,
then
yeah.
What
do
you
have
access
to?
Do
you?
You
do
you
have
access
to
like
any
secret
and
any
service
account?
That's
ever
been
used
on
that
on
that
node.
D
So
that
all
comes
down
to
how
you've
actually
written
your
container
right,
that's
the
problem
right!
It's
like
a
container,
it's
just
a
c
group,
and
if
your
container
is
you
know,
uid
one
as
root
and
you
break
out
of
that
container.
That
uid
translates
over
right
to
that
node.
So
then
you
suddenly
are
user
one
or
you
zero.
Whatever,
like
your
root
on
the
node,
and
then
you
have
all
the
root
things.
D
So
that's
the
problem
with
container
breakout,
but
even
if
you,
like
you
break
out
of
the
application
into
the
container,
you
have
a
service
account
mounted
right,
and
so
you
that
service
account
whatever
it
can
do
and
talk
to
the
cluster.
That's
your
access!
So
you
can.
You
know
if
you
give
too
much
too
much
permissions
on
your
specific
containers
from
service
account
level,
because,
like
oh
yeah,
let's
let
this
container
list
all
the
name
spaces,
because
who
knows
what
they
need
right?
D
Suddenly,
you
know
they
have
everything
so
and
like,
and
the
main
thing
right
from
from
from
the
container
level
is
it's
what
tools
they
have
right.
So
if
you
just
have
the
binary
running
and
they
break
out
of
that
application
into
the
binary,
it's
pretty
impossible
to
do
a
container
breakout
right
from
that,
and
you
know
unless
you're
doing
some
pretty
crazy
things
in
the
application.
D
G
G
D
D
F
D
D
It
well,
it
depends
right,
you
can
still
technically
like
and
I'm
not
a
security
expert.
Clearly
right
like
I'm,
you
know,
but
like
it,
you
can
still
break
out
of
container
through,
like
cves
that
have
existed,
like
we've
totally
seen
them
in
kubernetes,
but
if
you
broke
out,
you're
still
only
have
the
levels
of
that
user.
You
are
that's
why
it's
like
it's
like.
If
you
just
give
a
uid
and
a
gid
that
doesn't
exist
on
that
node
or
like
a
high
one.
D
That's
why
a
lot
of
people
just
do
something
in
the
thousands,
because,
unlike
you,
have
you
know
users
into
that
level,
then
you
basically,
you
know
you
have
no
permissions
to
do
anything
right
like
and
so
then,
even
if
you
did
break
out,
you
have
some
sort
of
kind
of
prevention.
There
right
you
know,
and
you
might
have
read
access
some
places
and
you
can
kind
of
glean
some
in
for,
but
it's
not
as
much
but
like.
D
B
D
Yeah,
oh
well,
yeah
yeah
and
that's
the
that's
the
main
problem
with
the
original
bot
security
policies.
Right
is
their
cluster
level,
and
so
anyone
deploying
a
container
can
use
your
you
know
your
least
restrictive
psp.
So
if
you
have
the,
if
you
suddenly
got
the
capability
to
do
what
what
jim
was
saying
is
deploy
a
new
container
with,
and
you
had
a
pod
security
policy
that
said:
oh
yeah,
we
let
we
left
this
one
container
have
privilege
mode.
B
D
D
D
B
I
thought
the
big
problem
with
them
is
that
is
that
is
that
they're
either
on
or
off
at
a
cluster
level,
and
so
you
can't
like
incrementally
enable
them.
So
if
you
haven't
set
up
your
entire
cluster,
you
can't
like
gradually
turn
it
on
it's.
You
turn
them
on
and
like
every
pod
on,
your
system
will
suddenly
stop
running.
D
G
Yeah,
that
was
one
of
the
challenges.
The
other
challenges
was.
This
sort
of
you
know
somewhat
of
a
conflict
of
interest
right
where
you're
trying
to
you
can
assign
psps
based
on
roles.
But,
ultimately
you
know
it's
a
like
a
deployment
or
some
other
part
controller
running
that
pod,
not
not
a
role
right,
so
you
get
into
situations
like
that
with
trying
to
manage
psps
correctly
across
different
applications
or
workloads.
G
So
I
think
the
new
proposal
now
is
is
just
doing
this
based
on,
like
we
discussed
briefly
last
time
like
on
name
spaces,
just
to
make
that
boundary
clear
that
the
only
only
strong
isolation
boundary
is
a
name
space
and
anything
within
that
namespace.
If
you
can
run
a
part
in
that
namespace,
you
can
get
the
same
privileges
as
the
highest
privilege
pod
in
that
namespace.
B
D
E
D
D
D
G
G
G
G
And
yeah
on
the
vr,
so
I
think
yeah.
So
we
were
talk
just
talking
about
teams
versus
tenants
right
and
so
that's
one
one
thing
we
should
decide
on
on
the
terminology
and
I'm
fine
with
just
if
you
want
to
remove
multi
from
the
title,
we
can
kind
of
just
call
it
two
common
tenancy
models.
I
think
that
that
was
one
of
your
suggestions
to
adrian.
B
Sorry,
it
was
muted
yeah
that
I'm
I'm
gonna
back
down
on
calling
it
multi-team
tenancy
so
yeah.
Let's
just
take
that
off
the
table,
I
did
think
of
maybe
calling
it
just
tenancy
models
as
opposed
to
multi-tenancy
models,
just
because
I
think
most
people
consider
lots
of
different
clusters
to
not
really
be
multi-tenancy.
B
G
B
G
Yeah,
so
I'm
I'm
fine
with
that
too,
like
if
you
want
to
pull
out
virtual
clusters
from
clusters
as
a
service.
So
right
now
it
just
says
you
know
namespaces
as
a
service
and
clusters
and
the
thinking
there
was
the
end
user
may
not
care
whether
the
cluster
is
virtual
or
not.
But
of
course
there
are
some
differences,
so
yeah
any
other
thoughts
on
that.
Should
we
stay
with
two
or
or
what
would
we
call?
B
H
So
I
I'm
okay
with
currently
writing,
but
I
see
a
point,
so
you
want
to
make
a
distinct,
which
was
you
know,
dedicated
cluster
versus
dedicated
control
claim
right.
So
there
is
a
slight
difference
there
right
so
dedicated
cluster
means.
You
know,
as
you
said,
10
and
the
parts
cannot
arrive
in
the
same
node.
G
Yeah
so
one
way
of
titling
it-
and
I
think
I
had
that
at
one
point,
as
you
could
say
something
and
tasha-
and
you
would
use
these
terms
too,
like
shared
cluster,
dedicated
cluster
and
then
portal
clusters
right.
So
those
could
be
three,
but
then
it
moves
away
from
the
as
a
service.
So
I
guess
which
is
fine.
We
could
we
could.
H
F
That
was
exactly
what
I
was
going
to
say.
I
basically
refer
to
vc
as
control
plane
as
a
service
versus
cluster
as
a
service,
just
because
that's
really
that's
really
what
you're
getting
out
of
it.
You
shared
cluster
at
the
at
the
outset,
but
really
it's
just
a
dedicated
control
plane
that
you
get
to
muck
with.
H
B
B
G
D
G
G
Okay,
so
obviously
like
the
first
thing
once
we're
all
settled
on
any
changes
we
want
to
make,
I
can
go
back
to
them
and
see
yeah
check
with
them
and
others,
because
they'll
have
to
go
through
it
again,
and
you
know,
because
of
some
of
the
changes
and
once
they
sign
off
they'll
figure
out
who's.
The
last
reviewer.
G
Okay,
yeah
so
I'll
check
with
them
and
see
what's
going
on
and
just
mention
that
we
want,
you
know
if
you
can
get
this
published
before
kubecon
eu.
That
would
be
awesome,
and
so
I
don't
know
what
the
exact
release
timeline
is
but
yeah
once
based
on
when
that
goes
out.
We
could
time
it
before
or
after.