►
From YouTube: Deep Dive: Kubernetes Working Group for Multi-tenancy - Sanjeev Rampal & Adrian Ludwin
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Deep Dive: Kubernetes Working Group for Multi-tenancy - Sanjeev Rampal, Cisco & Adrian Ludwin, Google
This deep dive of the working group for Multi-tenancy will include an in-depth technical exploration of multi-tenancy in core Kubernetes and the tooling and services the multi-tenancy working group has been developing to mainstream how users of Kubernetes can achieve multi-tenancy.
https://sched.co/Uah1
A
B
A
From
Google
in
up
in
Waterloo
and
we've
got
several
folks
who
are
active
in
the
working
group
in
particular
Orion,
better
check,
Jimbo
Guardia
has
been
recently
getting
engaged
quite
a
bit
and
then
some
folks
aren't.
So
what
we
want
to
do
here
is
that,
although
this
is
a
deep
dive
we
want,
we
know
that
a
lot
of
people
may
not
be
familiar
with
this
working
group.
So
we
want
to
make
this
match
your
expectation.
A
So
we've
got
three
kinds
of
content
here:
we've
got
some
slides,
I'm,
going
to
put
those
slides
in
the
session
which
I
haven't
had
to
done
it
yet,
but
I'll
alive
those
slides.
So
we
were
thinking.
You
know
just
five,
ten
minutes
of
slides
just
to
catch
up
some
folks
that
might
be
family
not
familiar
with
what
are
the
projects
and
sort
of
a
little
bit
more
detail,
and
we've
got
a
couple
of
demos,
maybe
each
five
minutes
and
then
so
we
could
do
about
15
minutes
of
Q&A.
After
that,
does
that
make
sense?
A
A
Okay,
quite
a
few,
seem
to
want
that.
So
we'll
do
call
the
inaudible
space
we'll
do
that
for
we'll.
Do
it
short,
because
we
have
other
YouTube
recordings
in
the
material
right
and
then
we'll
how
many
we
want
to
see
demos.
We've
got
two
demos
of
five
minutes:
each
okay.
Everybody
wants
to
see
demos
as
well.
So
we'll
do
you
know
we'll
do
10
minutes
of
slides,
10
minutes
of
demos,
then
10
odd
minutes
of
Q&A.
Ok,
let's
go
with
that.
A
A
A
Adrian
right
here
is
driving
the
agency
work,
for
example,
Faye,
whose
I
think
not
here
today
Faye
if
you're
here,
please
raise
your
hand,
he
may
not
be
here
he's
driving
virtual
clusters
and
Tenant
controller
Jim
right
here
is
kicking
off
their
multi-tenancy
benchmarks.
Ryan
is
doing
everything
and
then
many
of
you
are
contributing.
So
we
really
appreciate
that
and
we
want
to
hope
to
continue
this
work
growing
this
working
group
right.
So
hopefully
we
can
tell
you
enough
about
it
that
will
get
you
excited
to
contribute
in
whatever
way
you
feel
like
you
can.
A
So
I'm
gonna
skip
over
what
Kuban
it
is
multi-tenancy
is
this,
hopefully
audience
knows
what
it
is
about.
I
just
mentioned
very
briefly,
since
we're
just
doing
ten
minutes
of
slides.
What
are
the
tracks
that
we're
trying
to
follow
and
how
we
plan
to
evolve
this
working
group
right
so
first
we
decided
to
say
that
this
is
a
nebulous
space.
People
haven't
quite
defined
it
right.
A
So
what
are
the
different
models
of
multi-tenancy
that
are
possible,
and
what
do
we
want
to
address
in
this
working
group
right
so
very,
very
briefly:
model
a
is
multiple
clusters,
multiple
multiple
kubernetes
clusters
on
an
is
model.
B
is
name
space
grouping,
creating
new
tenants,
er
DS,
that
own
communities,
resources
like
namespaces
model
C
is
virtual
clusters
and
modern
D
is
sort
of
a
theoretical
possibility.
Where
we
go
back
and
rewrite
core
Kuban,
it
is
to
make
tenancy
a
first-class
citizen
within
communities.
A
A
I'll
have
the
slides
on
on
the
website
for
you
to
look
at
in
more
detail.
Okay,
briefly
about
the
tenant
operated
model
right.
So
in
this
project
a
series
of
custom
resources
are
being
defined.
That'll
allow
someone
to
self-service,
create
tenants
in
a
kubernetes
cluster
and
define
a
set
of
namespaces
that
are
owned
by
that
can
enter.
Ok.
This
is
sort
of
the
high
level
architectural
model
from
PHA
and
you
can
see
that
there's
a
tenant
CR
that
creates
a
number
of
resources
in
what
is
called
a
tenant
admin
namespace.
A
So
there's
a
there's:
an
admin
namespace
per
tenant
instance
and
in
that
namespace
control
resources
specific
to
that
name
and
tenant,
are
maintained.
So
you
can
see
there's
some
things.
There's
another
row
of
CRS
they're
called
tenant.
Namespace
see
ours
right.
These
are
CRS
that
control
actual
user
level.
Namespaces,
ok
and
you
can
also
see
the
virtual
cluster
CR,
because
the
goal
there
is
to
have
the
concept
of
a
tenant
which
owns
X
number
of
namespaces
and
possibly
Y
number
of
virtual
clusters.
A
A
Ok,
let's
keep
moving
that'll
just
be
adequate
for
a
very
brief
introduction,
and
this
is
the
kind
of
Yambol
that
that
PHA
and
the
team
have
proposed.
Where
you
create
tenants.
Er's,
you
specify
the
tenant
admin
namespace.
The
required
namespace
prefix
is
true,
is
an
interesting
thing.
As
you
know,
in
a
single
color
kubernetes
cluster
namespace
have
to
be
globally
unique
right.
So
how
do
you
create
multi
tenancy?
Where
tenants
may
want
to
create
their
own
names
for
for
namespaces?
So
one
one
way
to
do
it
is
to
enforce
that
they
make
their
namespaces.
A
You
need
to
begin
with
and
you
reject
any
duplicate
namespaces
and
the
other
one
is
to
actually
prefix
a
tenant
name
in
front
of
the
namespace,
but
really
the
right
solution,
for
that
is
the
virtual
cluster
solution,
which
is
a
different,
which
is
the
other
thread
right.
So
this
is
the
model
B
for
tenant
operator
right,
that's
a
sample
of
it,
and
we
have
a
demo
for
this.
If,
hopefully,
we'll
be
able
to
do
a
quick
five-minute
demo
on
this.
The
next
thing
is
hierarchical,
namespace
controller,
so
agent.
B
The
idea
behind
hierarchical
namespaces
is
that
so
it's
not
nested
namespaces,
it's
not
a
way
of
getting
sort
of
more
unique
of
finding
spaces
for
free.
It's
really
about
a
couple
things:
it's
about
policy
enforcement,
it
is
about
self
service,
namespace
creation,
and
it's
about
some
kind
of
concept
of
standardizing,
an
idea
of
hierarchy
inside
kubernetes.
B
Our
back
policies
for
the
developers
and,
let's
say
some
secrets-
that
all
of
the
services
in
that
names
in
that
team
need
to
operate,
and
then
you
can
create
the
service
namespaces
for
additional
isolation,
underneath
that
and
all
the
objects
get
propagated
down.
One
of
the
nice
things
about
HNC
is
that
you
don't
need
to
have
cluster
level
permissions
to
create
these
namespaces.
A
And
this
is
a
brief
summary
again
on
behalf
of
Fae
of
the
direction
for
the
virtual
clusters,
the
solution,
so
in
so
the
difference
between
more
the
model
B,
so
I
mentioned
model
B
and
C
right.
Both
the
tenant
operator
and
the
hierarchical
namespace
solution
are
for
model
B,
where
you're
essentially
leveraging
namespaces
within
a
single
cluster
as
representing
a
tenant.
A
So
the
key
point
is
that
in
all
of
these,
in
both
of
these
solutions,
tenants
cannot
request
cluster
scope
resources
because
tenants
are
essentially
bound
by
a
set
of
namespaces
in
many
enterprises.
That's
seen
as
a
constraint,
because
some
tenants
want
to
define
their
own
CR
DS,
for
example,
or
other
cost,
plus
no
scoped
resources,
and
you
can't
really
have
two
different
tenants.
You
know
request
the
same
C
or
D
a
definition.
So
that's
where
model
C
comes
in
model.
A
A
Now,
which
we
will
be
putting
on
our
YouTube,
playlist
and
there'll,
be
a
lot
more
coming
on
that
project
and
I
sure
that
they
would
welcome
a
lot
of
input
and
contribution
from
the
community
on
that
one
in
particular,
and
the
other
thing
is
that
this
is
again
a
slide
from
Fei.
But
what
he's
saying
is
that
a
way
to
unify
all
these
threads
of
work
in
our
prop
in
our
working
group
is
to
look
at
the
concept
of
a
tenant
and,
as
we
said
earlier,
it
can
create
these.
A
It
can
instantiate
a
tenant
operator,
that's
the
logical
representation
of
a
tenant.
It
can
own
a
collection
of
namespaces.
Some
of
those
namespaces
could
actually
be
roots
of
namespace
hierarchies.
You
know
leveraging
the
HNC
work
that
a
DNN
team
are
doing
and
some
of
those
instances
could
be
virtual
clusters.
So
this
is
sort
of
a
unified
view
of
how
all
these
three
tracks
are.
Work
could
potentially
be
combined
into
a
single
architecture,
so
this
is
sort
of
a
long-term
view
right
now.
A
Okay,
another
thing
that
we
kicked
out
recently
is
okay,
coming
back
to
the
data
plane
for
multi-tenancy
and
in
particular
benchmarking
right,
so
so
so
Jim
and
a
team
here
have
recently
been
contributing
in
areas
related
to
benchmarking,
and
the
thought
there
is
that,
let's
define
somewhat
similar
to
CIS
benchmarks,
a
set
of
benchmark
tests,
which
you
can
run
on
a
cluster
and
get
a
quantitative
representation
of
whether
you
meet
the
criteria
that
the
community
decides
is
appropriate
for
for
defining
it
as
a
multi
multi-tenancy
ready
cluster.
So
there
are
different
dimensions
to
this.
C
Thanks
Sanjeev,
yes,
so,
like
Sanjeev
was
mentioning
you
know,
some
of
the
other
constructs
that
we've
been
working
on
talk
about
how
to
achieve
multi-tenancy
right,
but
with
the
benchmarks.
What
we're
doing
is
we
want
to
be
able
to
measure
the
desired
state
independently
of
how
multi-tenancy
is
configured,
so
you
may
even
manually
use
namespaces
other
constructs
and
you
know
configure
your
cluster,
which
is
fine,
but
how
do
you
prove
that
you've
achieved
some
level
of
multi-tenancy
within
the
cluster?
C
So
that's
where
we,
you
know
we
have
three
levels
defined,
which
were
on
the
previous
side
a
slide
a
level
one
would
be.
You
know,
independently
of
any
controller,
any
CRD
or
add-on
you
put
to
your
cluster
is
your
name
space
and
the
constructs
around
the
namespace
like
network
policies,
other
security,
contexts
of
policies,
things
like
that!
C
Are
there
securely
are
they
properly
configured
and
we
would
do
behavioral
as
well
as
configuration
tests
to
prove
that
they
are
properly
configured
much
like
if
you're
familiar
with
the
end-to-end
tests,
which
are
part
of
the
conformance
at
the
api
conformance
test
suite
as
well
right.
So
so
that's
the
idea
with
level
1
level
2
is
to
allow
self-service,
dev,
ops
2,
so
their
teams
should
be
able
to
create
their
own
namespaces
and,
along
with
their
namespaces,
maybe
even
request
like
different
quota
levels.
C
Thanks
to
that
nature,
so
that
would
be
level
2
is
what
we've
defined
and
level
three
would
be.
What
in
a
Sanjeev
mentioned
where
the
Alibaba
team
has
been
working
with
virtual
clusters
to
actually
virtualize
the
control
plane
to
get
your
own
API
server,
you
can
install
your
own
CR,
DS,
etc.
Right
so,
and
there
are,
these
are
still
proposals,
so
there's
things
to
be
reviewed
and
refined
there.
C
A
Thank
you
and
and
I've
got
some
more
information
on
the
slides.
So
let's
skip
over
all
of
this,
and
perhaps
we
should
move
onto
demos
now
so
hopefully
that
gave
you
all
who
might
not
be
familiar
with
the
working
group
completely
a
very,
very
quick
picture
of,
what's
going
on,
feel
free
to
reach
out
to
us,
either
during
the
Q&A
or
after
the
session
for
more
details.
So
now
it
didn't,
it
is
going
to
give
a
quick
demo
on
hierarchical.
Namespace
controller
is.
B
This
relatively
legible,
okay,
good,
so
I'm,
just
gonna
give
a
very
quick
demo
showing
exactly
what
I
what
I
was
describing
on
the
slide.
So
you
can
create
a
namespace
that
represents
your
organization
and
part
of
me.
Nobody
can
type
during
demo,
so
I'm,
cheating,
and
so
what
I've
done
here
is
I've
created
a
couple
of
namespaces
using
my
cluster
admin
level,
privileges
right
now
that
shows
all
of
the
the
different
aspects
of
the
organization
that
I
want
to
create.
B
It's
got
a
team
that
team
owns
a
service
and
I
can
create,
for
example,
roles
for
the
team,
asre
and
and
some
role
bindings
and
here
I'm
going
to
create
some
are
back
roles
for
the
organization
level,
sres
and
so
I've
now
created
these.
But
if
you
go
look
inside
the
service
itself,
you
will
see
oops
absolutely
nothing
because
they
haven't
been
hooked
up.
B
The
way
we
expected
it
to
be
and
well
before,
when
I
had
a
look
inside
of
service
one,
there
were
no
resources
or
sorry
no
roles.
If
we
look
now,
we
can
see
that,
as
expected
service,
one
has
org
a
sorry
which
claims
that
it
was
inherited
from
acne
org,
and
it
also
has
team
a
essary
which
was
inherited
from
team
a
and
in
this
demo.
By
the
way,
this
demo
is
public.
You
can
go
follow
along
yourself.
B
There
are
all
kinds
of
things
that
you
can
do
to
moderate,
modify
the
hierarchy,
and
you
will
see
that
these
objects
get
deleted
if
they
supposed
to
be
deleted.
So
if
you
move,
if
you
change
the
parent
of
one's
the
name
space
to
another
one,
if
you
update
them,
they
will
get
updated
and
it
doesn't
just
work
with
our
back
roles.
It
also
works
with
a
variety
of
other
objects.
Right
now
that
set
of
objects
is
hard-coded,
but
it
will
be
configurable
at
some
point.
B
This
is
using
version
0.1
of
the
HNC
can
of
the
agency
and
by
the
way,
do
not
attempt
to
use
this
in
prod.
This
is
an
initial
experimental
release
that
we
want
to
put
out
there
to
get
some
feedback.
The
other
thing
that
I
mentioned
was
the
ability
to
do
self-service,
namespaces,
and
so
here
what
I'm
gonna
say
is
that
Acme
org
our
top
level
and
our
top
level
namespace
has
a
required
child
of
team
B
now
I
didn't
create
the
namespace
called
team
B
and
that's
all
a
feature
of
the
controller
as
well.
B
You'll
see
that
it
has
the
org
a
Sri,
which
is
the
only
default
role
that
I've
defined,
and
so
that's,
basically
it
and
so
far
it
scales
relatively
I
mean
the
the
interface
I
should
say:
scales
relatively
well
I've
been
setting
up
some
fun
little
hierarchies
across
my
my
cluster.
Here
you
can
see
that
it's
not
really
it's.
There
is
no
single
route.
You
can
have
as
many
routes
as
you
like.
B
A
We're
tracking
just
about
right,
so
we
could
do
one
more
demo
or
we
could
do
straight
to
Q&A
any
preferences,
one
more
demo
hands
hands
for
one
more
demo.
Okay,
so
we'll
do
that
in
five
minutes
and
then
we'll
have
about
ten
ten
plus
minutes
for
Q&A.
So
this
demo
is
for
the
tenant
operator
work
right,
so
just
as
a
refresher,
a
set
of
CRS
that
you
use
to
represent
tenants
and
then
their
students,
own
customer
cluster
resources
like
namespaces,
so
n21,
mapping
of
namespaces
lieutenants.
A
A
Also,
we
take
a
look
at
the
CNI,
so
I'm
actually
having
a
little
bit
of
the
data
plane,
isolation
as
well
in
this
demo.
It
so
happens
we're
using
calico
as
a
CNI
right
so
we'll
show
of
if
we
have
time
we'll
show
an
illustration
of
combination
of
the
control
plane.
The
network
policy
and
the
port
security
policy
for
complete
isolation.
A
So
we
look
at
the
sample
yeah
Mel
for
creating
a
tenant,
very
baseline
version
here.
Okay,
so
we're
saying
here,
please
create
me
an
object
called
tenant
that
shall
be
called
tenant,
t1
that
shall
have
all
its
admin
stuff
in
this
namespace
called
t1
admin
space
and
whose
admins
will
be
this
service
account
right,
any
are
back.
Subject
can
be
admins.
The
reason
it
is
set
up
this
way
is
again
with
the
self-service
in
mind.
Anybody
belonging
to
an
are
back
role
called
tenant.
Creators
will
be
able
to
create
tenants
once
they
create
a
tenant.
A
A
Okay
and
we
go
ahead
and
create
that
tenant
and
it's
been
created
now.
We
also
notice
that
a
few
access
controls
have
been
set
up
for
us,
in
particular,
some
cluster
roles
for
this
tenant
admin
role,
I'm
not
going
to
go
into
the
details
of
that,
but
these
essentially
ensure
all
those
properties
I
mentioned
earlier.
The
appropriate
privileges
for
the
tenant
admin
insuring
tenant
admins
do
not
get
permission
in
other
tenant
spaces
and
so
on
right.
So
this
operator
takes
care
of
provisioning.
All
of
that
and
then
role
bindings
within
those
target
admin
namespaces.
A
So
this
is
the
only
admin
for
that
tenant
in
this
example.
Similarly,
creating
another
tenant
right
again,
anybody
belonging
to
the
group
tenant
creators
can
be
creating
self-service,
tenants,
okay,
similar
to
before
this
time,
somebody's
creating
a
kind
tenant
with
the
name,
tenant,
t2
and
he's
using
his
own
admin
space
and
his
own
admin.
A
A
A
Ok,
that's
been
created.
Similarly,
tenant
admin
in
action
grabbing
another
resource,
another
namespace,
and
confirming
that
this
is
what
tenant
admin
one
owns
for
now
and
we
and
the
cluster
admin
looks
at
the
global
set
of
namespaces.
He
notices
that
there
are
two
tenant
admin,
namespaces
and
then
tenant.
One
has
actually
created
two
user
namespaces.
A
Now,
as
the
as
the
identity
of
can
and
twos
admin,
I
try
to
see
whether
I
can
access
tenant
ones,
resources
right.
So
this
is
me
as
ten
and
two
Zeidman
trying
to
get
information
about
tenant,
one.
Okay,
that's
blocked,
so
those
automatic
are
back.
Rules
that
we
had
set
up
using
specific
resource
names
are
in
action
now
a
little
bit
of
a
networking
isolation
demo.
So
we
are
now
attaching
labels
to
make
sure
that
these
labels
indicate
which
this
will
be
useful
in
tracking
the
appropriate
network
policies
to
these
to
these
namespaces.
A
Okay-
and
this
is
an
example
of
a
calico
Network
policy.
This
is,
in
particular,
a
global
network
policy.
One
quick
note
here
is
that
one
thing
to
note
when
you
are
trying
to
make
a
cluster
multi
tenant
and
you
want
to
have
network
isolation.
You
probably
do
not
want
to
use
standard
kubernetes
network
policy
for
that
you
want
to
reuse,
standard,
kubernetes
network
policy
for
the
applicant,
the
users,
because
that
can
be
used
for
application
level,
microsecond
tation
right.
A
Instead,
you
use
global
network
policy
as
a
cluster
admin
stool.
So
the
cluster
admin
has
a
different
type
of
resource
for
network
policy,
which
he
uses
only
for
tenant
isolation
and
the
developers
have
a
different
type
of
network
policy
resource
which
they
use
only
for
application.
Microsegmentation
right
a
lot
of
the
current
ad
hoc
solutions.
A
They
tend
to
overload
my
and
Cuban
ideas
network
policies
to
do
both
tenant
isolation
and
application,
isolation,
which
you
really
shouldn't,
because
then
you
have
to
give
a
developer
some
cluster
admin
like
privileges
and
they
could
invade
invade
inadvertently,
break
cluster
rights
or
tenant
isolation.
Boundaries
not
sure
if
I
explained
that
enough,
given
that
we
are
rushing
through
this,
but
hopefully
you
get
the
point
that
you
need
to
think
about
not
just
leveraging
kubernetes
network
policy
for
both
tenant
isolation
and
application.
Micro
segmentation.
A
So
another
thing
is
that
this
is
got
a
part
security
policy,
so
it
will
not
accept
end
user
containers
that
that
require
privileges,
so
we
are
actually
running
a
curated
version
of
nginx.
So
just
as
you
perhaps
may
know,
may
know,
there
are
few
repositories
which
have
curated
versions
of
standard
containers
like
bitNami.
Nginx
is
curated
so
that
it
doesn't
actually
require
it
requires
minimal,
hosts
network
privileges.
So
you
want
to
be
using
those
containers
and
you
want
to
have
set
up
a
pod
security
policy
that
only
allows
those
containers
to
run
in
your
tenants.
A
B
D
A
So
we
will
yeah,
we
have.
We
definitely
need
to
be
preparing
more
documentation
on
how
these
things
turn
together
and
we
take
an
action
item
to
do
that.
We
also
do
not
want
to
be
too
opinionated
because
there
are
different
ways
of
doing
the
same
thing.
So
we
want
to
give
references
but
say
that
this
is
just
one
way
of
doing
it,
but
we
do
take
the
AI
to
prepare
more
I.
Would.
B
Add
that
not
all
the
primitives
are
in
place
yet,
and
so
that's
some
of
what
the
the
work
that
we're
doing
is
is
to
put
those
primitives
in
place.
Once
we
have
the
benchmarks
in
place
that
Jim
and
Co
are
putting
on,
then
we
will
be
able
to
more
easily
write
a
best
practices
guide,
which
will
basically
say
if
you
look
like
this,
do
that,
if
you
look
like
this
to
do
this,
I
mean.
A
A
Well,
you
know,
let
me
let
me
have
the
other
folks
as
well.
I'll
give
you
my
my
interpretation
and,
and
then
the
other
folks
have
an
opinion
as
well.
We
are
not
limiting
ourselves
by
timelines
at
this
point.
We
want
adequate
functionality
before
we
declare
something
as
usable,
so
I
think
that
we
are
looking
at
essentially
depending
on
the
level
of
multi-tenancy.
You
know,
Jim
talked
about
various
levels
right,
so
elementary
solutions
can
actually
be
run
today,
but
a
lot
of
these
projects.
C
You
know,
one
of
the
things
to
add
is
so
initially
we're
focused
on
level
one
and
the
idea
is
to
at
least
get
agreement
on
the
set
of
tests
that
we
want
to
do
and
then
we'll
go
about
automating
those.
So
there
is
a
document
where
we
have
been
collecting
different
use
cases.
If
you
go
to
the
multi-tenancy
repo,
you
will
see,
you
know.
Different
use
cases
and
suggestions.
C
We've
collected
so
certainly
contribute
your
use
cases,
because
that
will
influence
the
benchmarks
and
what
we
target
in
what
order,
and
you
can,
of
course
in
any
one
of
our
meetings.
If
you
come
in-
and
you
know
give
us
some
of
the
feedback
things
that
you're
seeing
and
how
you
would
like
multi-tenancy
to
be
measured,
that
would
be
helpful
as
well.
Yeah.
A
Well
right,
the
thing
is
that
you
can,
depending
on
your
operational
model,
you
can
allow
your
tenants
to
create
their
own
series,
even
in
the
other
models.
It's
just
that
there
will
be
more
coordination
right
you
in
a
namespace
grouping
model.
You
can
say
that
a
tenant
can
request
a
C
or
D
and
has
to
coordinate
out-of-band
with
the
cluster
admin
to
get
their
C
already
added,
as
at
the
global
scope
right.
So
that's
the
kind
of
quote-unquote
poor
man's
way
of
doing
it.
A
A
B
Of
the
issues
that
we've
had
with
multi-tenancy
is
because
so
few
of
these
primitives
have
been
in
place
up
until
now,
people
have
sort
of
taken
one
or
two
quick
looks
at
it
said:
oh
no,
harking
namespaces,
no
cell
service
creation,
note
tenant
no
per
tendency
or
die's
I'm
just
going
to
do
something
else.
The
more
of
these
barriers
we
break
down,
I
think
the
greater
a
demand
we
are
going
to
see
for
the
remaining
ones
to
be
broken
down
as
well.
B
A
E
A
Interactions
with
this
API
server
team
so
far
have
been
well
find
me,
20
developers
that
can
do
that
and
then
we'll
talk.
So
at
the
moment
that's
not
happening,
so
everything
that's
happening
is
on
top
of
Corcovado
days
now.
Hnc
is
an
example
of
a
project
that
actually
could
become
part
of
core
kubernetes
if
this
of
interaction,
but
at
this
point
everything
is
on
top
yeah.