►
From YouTube: Fine-Grained Permissions in K8s: What’s Missing, and How to Fix That - Vallery Lancey & Seth McCombs
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Fine-Grained Permissions in Kubernetes: What’s Missing, and How to Fix That - Vallery Lancey, Lyft & Seth McCombs, Triller
In this talk, we will walk through a number of common scenarios where Kubernetes lacks sufficient access control tools, or where access control is often not properly applied. For example, it is common for a team to own a subset of services in a namespace, yet RBAC permissions grant that team access to other pods within the namespace. We will demonstrate a number of solutions available for specific problems, such as pod network policies, the open policy agent, custom controllers that gate API functionality. We will also discuss problems with the namespace permission model, and possible alternatives. Namespaces create an arbitrary boundary around resources, which creates the need to then bridge those boundaries. We will demonstrate ideas for bridging namespace networks, and posix-style objection permissions within a namespace.
https://sched.co/MPdx
A
A
Personal
opinions
as
operators,
largely
working
with
small
companies
and
small
clusters,
and
we
are
generalists
with
kubernetes-
were
not
security
experts.
We
may
be
wrong
so
with
that
said,
in
Kerber,
Nettie's
users
interact
with
a
virtual
object
model,
so
some
of
these
objects
are
very
logically
corresponding,
like
pods
are
a
thing.
A
B
Right
so
before
we
dive
in
and
list
off
a
bunch
of
tools,
things
like
that
helps
to
think
about
the
fundamentals.
There's
three
broad
categories
of
access
control
in
kubernetes.
We
have
first
up
the
runtime
characteristics.
So
do
your
pods
have
access
to
the
disk
under
the
hood?
What
do
resource
limits?
Look
like
privileges
things
like
that.
Next
up,
we
have
the
network
side
of
things.
So
how
do
these
things
talk
to
each
other?
What
can
communicate
with
what
and
things
like
that?
B
Lastly,
is
the
kind
of
meta
aspect,
which
is:
what
has
access
to
our
control
plane
and
our
data
plane
and
that's
important,
because
that's
control
of
kubernetes
itself
and
with
access
to
those
things
you
can
probably
modify
the
first
two
all
right.
So
there's
a
bunch
of
different
approaches
and
tools,
like
Valerie
said
we're
not
experts,
but
we're
gonna
focus
on
a
small
but
useful
subset
of
them,
the
first
one
being
our
back
role
based
access
control.
A
lot
of
people
should
be
familiar
with
that
and
namespaces
that
go
along
with
it.
B
You
can
use
to
limit
your
scope
of
your
are
back
and
things
like
that.
Network
policies
again
helps
with
communication.
What
can
talk
to
what
things
like
that
custom
kubernetes
api
gateways
are
the
next
thing,
putting
something
in
front
of
the
kubernetes
api
and
limiting
what
has
been
exposed
next
admission
web
hooks,
validating
and
mutating?
A
Note
that
this,
since
we're
talking
about
a
subset
security,
there's
many
security
tools
beyond
this
we're
specifically
focusing
on
access
control
in
this
talk.
So,
let's
start
by
looking
at
the
built-in
authorization
system,
our
back
our
box
been
around
for
a
long
time.
It
went
GA
in
about
1.8.
Hopefully,
people
who
are
using
kubernetes
in
production
are
familiar
by
now,
but
as
a
recap,
our
back
provides
roles
which
have
several
components.
There's
two
kinds
role
or
cluster
role.
Role
applies
specifically
to
namespace
level
resources.
Cluster
role
applies
to
things
that
don't
fit
within
a
namespace.
A
A
So
next
up
is
the
role
binding.
The
role
binding
associates,
a
user
and
a
role
so
there's
similar
to
roles,
there's
a
role
binding
at
a
cluster
role,
binding
and
again
that's
at
the
printings
base
level
or
at
the
entire
cluster
scope.
We
give
it
some
users
and
service
accounts.
We
want
it
to
apply
to
you
and
then
we
give
it
the
roles
or
cluster
roles
we're
linking
to,
and
you
can
kind
of,
mix
and
match
a
lot
of
these.
A
So
it's
not
necessarily
one
role
to
one
user,
so
this
chain
of
things
can
be
easiest
to
visualize.
I
find
it's
very
easy
to
get
things
wrong,
because
you
have
a
very
wide
chain
of
connected
resources
between
the
thing
that
you're
running
and
the
actual
permissions
that
you've
assigned.
So
these
are
all
fairly
pluggable
things.
A
Lately,
I've
been
playing
around
with
ways
to
aggregate
resource
view,
and
so
in
cabinet
is
because
there's
often
a
concept
of
chains
where
one
thing
will
create
or
own
or
link
with
another.
So
this
is
the
kind
of
view
that
I
think
is
very
helpful
to
come
up
with,
especially
when
you're
debugging,
because
there's
many
ways
that
you
can
typo
or
forget
something
where
you
don't
necessarily
have
the
permissions
that
you
expect.
A
So,
aside
from
our
back
there's
a
few
other
forms
of
object,
access
control,
so
there
is
no
base
access
control,
which
is
what
couplets
use
internally.
Essentially,
it's
a
list
of
what
a
couplet
needs
to
run
inherently
and
kind
of
the
summer
permissions
that
are
necessary
for
the
workloads
on
it.
So
the
idea
is
that
couplets
can't
actually
access
everything.
They
can
only
do
what's
relevant
for
the
pods
actively
on
them
as
a
way
to
help
prevent
escalation
through
arbitrary
couplets
there's
a
Bock
which
is
the
deprecated
predecessor
to
arbok.
A
It
basically
jammed
permissions
directly
onto
users
with
a
really
a
concept
of
being
able
to
reuse
those
profiles
and
then
those
webhook
off
which
completely
outsource
of
the
decision
outside
of
the
cluster.
So
you
give
it
a
URL
it
posts
to
that
external
system
and
then
uses
the
response
to
decide
whether
or
not
to
allow
the
authorization.
A
So
our
box
main
limitation,
is
its
lack
of
granularity.
For
example,
our
bat
grants
permission
by
a
kind
of
resource
but
doesn't
allow
you
to
discriminate
on
that
within
a
namespace,
so
say
if
you
grant
permission
to
modify
pods
within
a
namespace.
There's
no
way
to
say,
but
only
the
Redis
pods
is
all
of
them,
and
so
similarly,
you
also
can't
narrow
scope
within
fields.
So
I
can't
say
you
can
only
edit
labels
or
you
can
only
edit
replicas
if
you
can
edit
something
you
can
edit
all
the
fields
in
all
its
properties.
A
So
let's
look
at
namespaces,
which
are
the
companion
to
our
back
is
for
a
scoping
things.
Namespaces
are
somewhat
analogous
to
containers
in
that
they
are
a
way
to
segment
to
the
control
plane
and
share
the
data
plan
in
a
logical
fashion,
so
they're
introduced
as
a
way
to
run
like
different
apps
different
systems
on
one
cluster,
without
worrying
about
the
cost
and
pain
that
comes
with
running
many
actual
clusters,
most
resources
fit
in
the
per
namespace
levels.
Some
resources
are
cluster
wide,
and
this
is
very
important
in
spaces.
A
Are
resource
boundary
they're,
not
a
security
boundary,
so
there's,
no,
not
necessarily
any
segmentation
of
resources.
So
if
someone
gets
onto
a
host
doesn't
really
matter
what
namespace?
What's
in
so
many
spaces,
let
a
scope
are
back.
This
means
we
can
just
wrap
everything
in
lots
of
tiny,
tiny
namespaces
and
give
it
exactly
the
permissions
we
want
right.
Not
quite
so,
if
we
get
very
granular
with
namespaces,
we
start
to
want
to
cross
these
namespaces
and
that
gets
tricky.
A
This
is
an
eternal
problem
when
it
comes
to
any
kind
of
resource
control
and
computing,
we
want
to
encapsulate
stuff,
wrap
it
up
and
then
suddenly
there
is
some
kind
of
scope
creep
between
things
and
it
gets
messy.
So,
first
off
we
have
many
clear
chains
of
resources
and
communities
like,
but
employment
who
owns
replicas
sets,
owns
pods
or
ingress
has
services
which
map
to
pods,
so
the
vertex.
We
need
to
keep
these
together
and
we're
probably
gonna
want
to
wrap
stuff
even
bigger
than
that.
A
B
All
right,
so
we're
gonna
look
at
a
few
ways
to
supplement
access
within
namespaces
what
we
can
do
to
control
things
within
that
namespace,
starting
with
the
most
manual
approach,
which
is
creating
a
custom,
API
gateway
or
a
deputy
that
sits
in
front
of
your
kubernetes
api.
So
what
if
we
don't
want
to
expose
direct
access
to
these
resources,
you
can
do
the
same
thing.
B
You
do
in
other
forms
of
software,
create
a
gateway
in
front
of
the
resource,
put
an
API
in
front
of
the
kubernetes
api
and
only
expose
things
to
downstream
systems,
either
other
systems
or
users.
The
majority
of
the
time.
This
is
not
something
you're
going
to
want
to
do
because
you're
solving
a
problem
in
a
complicated
way,
probably
something
that
could
be
solved
by
just
a
few
lines
of
config
in
some
other
tools.
B
So
in
this
example,
what
we've
done?
We've
got
our
kubernetes
api
and
a
deployment
and
we've
put
a
gateway
API
in
front
and
all
we're
exposing
is.
Let's
say
we
let
the
users
change
the
replica
count
on
this
deployment
and
that's
it.
So
this
is
nice,
because
consumers
of
this
API
only
have
to
understand
the
replicas
for
that
deployment.
Nothing
else.
B
They
understand
the
simple
API
rather
than
the
full
deployment
object,
which
I
don't
know
about
everybody
here,
but
was
really
confusing
when
I
first
saw
it
and
additional
logic
can
be
built
in
maybe
allowing
control
for
the
rate
of
change
that
you
can't
scale
this
up
or
scale
it
down
too
fast
and
you
can
get
as
complex
as
you
want.
So
this
is
neat,
but
it
has
some
obvious
drawbacks.
B
So
next
up
admission
web
hooks,
so
bringing
request
gating
to
the
API
itself
and
hopefully
eliminating
your
need
to
run
a
custom
service,
so
Mission
web
hooks
run
after
authentication
and
authorization
and
before
admission.
Hence
the
name
two
kinds
validating
and
mutating
validating
is:
does
the
object
I
want
to
create
comply
with
some
sort
of
policy,
a
replica
count
it
does
it
come
from
a
trusted
image.
Repository
do
I
have
resource
limits,
set
privileges.
Things
like
that.
B
Second,
is
the
mutating
web
hook
I'm
going
to
let
you
create
this
object
in
my
cluster,
but
I'm
going
to
change
it
in
some
way.
You'll
see
this
a
lot
with
sidecar
injection
with
a
lot
of
service
meshes
out
there
you
go
to
create
a
deployment,
and
it
just
dynamically
injects
that
sidecar,
if
it
doesn't
already
exist
things
like
that,
so
both
of
these
can
ensure
valid
things
are
launched
into
your
class.
Obviously
you
can
use
them
together
all
right,
so
you
can
do
a
lot
of
things
manually
with
admission
webhooks,
but
another
way.
B
B
But
we
want
to
have
policies
on
things
created
in
our
cluster.
What's
the
simpler
way
to
do
this,
the
open
policy
agent
is
I.
Think
one
of
their
tag
lines
is
a
lightweight
policy
engine.
What
does
that
mean?
It
takes
the
decisions
for
allow
deny
things
like
that:
the
policy
out
of
your
service.
So
when
you
go
to
create
something,
make
a
change
things
like
that,
OPA
knows
what's
running
in
your
cluster.
Has
some
policies
you've
created
and
will
check
against
those
and
say
hey?
B
Is
this
allowed
could
be
something
like
I'm
trying
to
pull
a
public
image
from
docker
hub,
but
I
only
am
allowed
to
pull
internal
images
for
my
company's
repository,
so
that
would
be
a
deny
and
then
would
not
be
allowed.
So
you
can
do
this.
You
know
context
specific
dynamic.
The
image
use
you've
used
what
ports
are
open.
What
protocols
are
they
serving?
Are
they
running
privileged
things
like
that,
and
this
can
work
together?
So
you
don't
have
to
really
speck
out
these
combinations.
B
B
Whatever
that
object
is,
is
it
a
service
with
an
incorrect
load,
balancer
name
things
like
that
and
then
can
return
to
you
why
you're
not
allowed
to
take
that
action
instead
of
just
giving
you
some
obscure
error
that
your
yeah
Milles
incorrect
or
something
like
that,
which
is
always
more
useful
than
just
getting
an
explicit
sorry
can't
do
this
another
nice
thing
is:
it
can
be
leveraged
to
return
a
patch
in
the
form
of
JSON.
That
tells
you
what
you
need
to
apply
to
your
object.
Hey!
You
didn't!
B
B
So
this
allows
you
to
have
this
kind
of
separation
of
duties
and
enhance
the
coverage
and
security.
So
you
don't
end
up
with
you
know,
let's
say
a
bunch
of
little
itty-bitty
security
holes
that
in
the
aggregate
can
be
a
bigger
issue
and
again
I
mentioned
gatekeeper.
Please
check
that
one
out
that
one's
a
great
way
to
get
started.
So,
lastly,
I've
not
seen
anyone
do
this
publicly
yet,
but
I've
heard
about
it
is
using
the
open
policy
agent
to
leverage
network
policies
again
controlling
that.
B
What
can
talk
to
what
and
a
lot
of
these
happen
it.
You
know
layer
for
this,
can't
talk
to
that.
But
what,
if
we
could
be
dynamic
and
do
that
at
little
layer,
7
can
I
post
to
widgets,
but
God
can
I
not
post
to
widgets
and
but
I
can
get
widgets
based
on
some
sort
of
labels
or
annotations
or
something
on
my
pods.
A
A
Kubernetes
by
default
has
a
flat
network,
and
this
is
one
of
those
things
that
I
was
shocked
when
I
learned
it.
So
every
single
pod
by
default
can
talk
to
every
single
other
pod.
This
includes
pause
without
services,
and
this
includes
cross
namespaces.
If
there's
a
port
open
on
a
container,
it
can
be
accessed
and
that's
pretty
scary.
A
An
attacker
can
use
that
to
traverse
your
network
or
escalate
or
do
anything
nasty,
and
on
top
of
that,
potentially
having
that
wide
network
behind
the
perimeter
means
you
can
start
building
up
some
accidental
service
dependencies
that
perhaps
aren't
properly
known
and
dealt
with.
So
that
can
be
a
big
surprise
one
day.
A
So,
with
any
service,
we
have
a
defined
set
of
calls
that
the
service
can
make
within
one
another.
So
you
can
get
this
if
your
service
is
small
and
simple
enough
from
just
putting
it
on
a
napkin,
you
might
use
a
service
mash,
but
either
way
you
can
create
a
graph
of
what
calls.
What
what
we'd
like
to
be
able
to
do
is
specifically
segment
everything
such
that
only
the
things
that
are
supposed
to
call
one
another
actually
can
access
one
another
to
prevent
some
nasty
surprises.
A
So
this
is
often
easy
to
think
about
service
the
service,
but
we
can
potentially
get
very
granular.
The
easiest
way
to
do
this
is
with
the
communities
network
policy.
So
the
network
policy
is
a
native
object,
although
somewhat
perplexingly,
there's
no
native
controller
for
it.
So
if
you
just
take
an
off-the-shelf
cluster
apply
some
Network
policy.
Amyl,
nothing
will
happen.
You
need
a
third
party
controller
such
as
calico,
so
the
policy
uses
selectors
like
namespaces
and
pod
selectors
to
apply
to
pods
when
it
applies
to
them.
Those
just
have
no
networking
allowed.
A
A
A
So,
there's
surprisingly
few
add-ons,
in
my
opinion
that
support
this
right
now
calico
is
one
of
the
main
ones
that
people
talk
about
as
far
as
executing
on
network
policies,
there's
also
a
lot
of
tools,
things
in
like
the
service
messy
space
that
don't
necessarily
work
with
network
policy,
but
offer
a
similar
functionality.
For
example,
this
teo
has
ways
to
restrict
inter-service
traffic,
so
some
of
the
issues
we've
identified
come
from
core
functionality
that
doesn't
exist,
for
example
the
flat
open
network
where
there's
no
controller.
A
Some
of
it
is
just
ways
where
namespace
is
in
the
our
back
are
too
wide
could
be
tough
to
manage.
So
we
see
kind
of
two
ways
that
someone
when
they're
architecting
out
their
cluster
can
address
this
problem.
One
is
getting
very
granular
within
namespaces
and
having
sub
namespace
permissioning.
The
other
is
having
lots
of
tiny,
namespaces
and
leaning
more
on
our
back.
So
let's
kind
of
examine
some
ideas
in
both
of
those
spaces.
A
First
thing
is:
we
had
a
kind
of
theoretical
approach
to
having
sub
names
based
granularity,
based
on
object
level
permission,
so,
instead
of
relying
on
tiny
namespaces,
you
can
grant
permissions
on
singular
objects,
however,
is
necessary
and
therefore
have
essentially
whatever
access
control.
You
want
with
a
real
protecting
the
cluster.
So
this
inspiration
comes
from
in
UNIX,
where
everything
is
a
file
somewhat
similar
to
an
kubernetes,
where
everything
is
an
object
that
has
some
yellow.
That
represents
it.
A
A
So,
since
everything
accumulate
is
the
structured
object,
we
have
our
API
version
kind
metadata,
spec
status.
Metadata
is
kind
of
the
easiest
thing
to
fuzz
around.
If
we
want
to
insert
something
with
a
custom
controller,
we
could
also
potentially
abuse
annotations,
it's
very
common
to
essentially
stringify
some
JSON
and
just
put
that
entirely
in
an
annotation.
So
that
can
be
a
good
way
if
you're
not
modifying
the
career
days,
core
itself.
A
So
hypothetically,
we
can
put
something
in
this
metadata
around
granting
ownership
on
that
object,
rather
than
try
to
apply
things
namespace
wide.
So
this
could
be
done
fairly
simply
by
giving
like
full
owners,
or
we
could
get
very,
very
granular
like
saying
certain
fields
that
these
can
edit
and
access.
A
However,
the
more
we
thought
about
it,
the
more
we
figured
out
that
this
was
not
necessarily
a
good
idea.
So
I
don't
know
about
many
of
you,
I've
administered,
some
pretty
large
and
messy
VMs
in
my
time
and
the
more
custom
and
bespoke
individual
objects
in
a
system
get
the
harder
and
more
unclear
they
are
to
manage
and
lack
of
clarity
and
difficulty
is
a
security
flaw
because
you're
more
likely
to
get
things
wrong.
So
this
isn't
a
good
idea.
A
B
Or
we
could
go
in
the
opposite
direction
and
keep
per
namespace
our
back
and
have
more
smaller
namespaces.
Think
of
it
like
the
genie
in
Aladdin,
phenomenal
cosmic
power,
a
DVD
living
space,
phenomenal
cluster
power,
anybody
name
space
so
as
it
stands,
objects
that
rely
on
one
another
need
to
be
housed
in
the
same
namespace.
Ingress
is
target
a
service
that
then
targets
pods
or
endpoints
deployments.
Control
replicas
sets
that
go
down
to
pods.
B
B
So,
to
keep
namespaces
small,
we
need
to
accept
cross
namespace
functionality
trying
to
get
rid
of.
It
leads
to
more
complexity
than
it's
worth,
so
you
can
restrict
access
between
namespaces
and
only
allow
explicitly
permitted
services
to
cross
those
namespaces,
and
you
could
even
try
and
restrict
things
further
within
that
namespace.
B
So
when
you
end
up
with
controllers
or
external
systems
that
need
to
target
things
in
multiple
namespaces,
you
can
either
consider
duplicating
that
component,
such
as
the
per
deploy
credentials
and
namespaces
or,
as
Valerie
mentioned
operating
it
at
cluster
level,
such
as,
like
a
cluster
wide
controller.
On
your
reticence,
tenses
or
things
like
that.
A
Firstly,
criminales
design
priority
is
deliberately
extensibility
over
out-of-the-box
usability,
so
this
is
why
we're
all
here.
This
is
why
coop
con
is
so
big
because
we
can
make
it
work
for
us,
but
the
problem
is,
we
do
need
to
make
it
work
for
us.
It
is
not
something
that
we
necessarily
just
install
and
go.
So
security
is
one
of
those,
many
considerations
where
it
may
work.
It
will
probably
not
work
well
without
some
fine-tuning.
B
So
you
can
group
resources
together
where
it
makes
sense.
I
can
do
namespaces
for
your
front
end
bits
namespace
for
the
back
end,
a
namespace
for
your
data
related
things.
There
can
be
some
sort
of
best
practices
around
this
something
along
those
lines,
but
it's
probably
going
to
look
different
for
everybody.
Based
on
your
individual
use
cases.
We
can't
tell
you
the
best
way
to
chop
up
the
namespaces
in
your
cluster.
I
could
try
but
I,
don't
know
your
environment
and
I'd
probably
be
wrong.
A
So
when
you
have
new
spaces
that
are
as
small
as
is
reasonable
to
make
them
use
our
back
to
make
access
as
reasonably
tight
as
you
need
it.
This
is
something
where
you
may
make
compromises
based
on
just
what
is
going
to
rabbit
hole
in
security
for
your
business
and
your
priorities
as
well
as
leaning
on
those
other
technologies
that
we
mentioned,
rather
than
trying
to
do
everything
in
pure
our
back.
B
You
can
move
onto
mutating
hooks
to
change
the
things
that
you're
launching
in
your
cluster
and
maybe
bring
them
into
line
just
a
little
bit.
If
they're
only
you
know
a
step
out
of
line,
you
can
use
tools
from
the
community
like
open
policy
agent
to
standardize
the
policy
for
things
like
this.
You
know
bring
some
uniform
management
to
your
policy
for
things
in
and
out
of
cluster.
A
B
C
Thanks
for
the
give
us
some
information,
we
actually
stumble
upon
a
lot
of
all
of
these
trying
to
build
a
multi-user
solution
for
cube
flow,
and
one
of
the
problems
were
having
is
we're
having
a
multi
user
scenario,
where
it's
user
has
a
separate
namespace,
which
I
think
is
pretty
common
right
and
the
thing
is
we
had
the
problem
with
the
scenario
where
we
want
its
user
to
be
able
to
delete
his
own
namespace,
but
this
doesn't
make
sense
in
our
buck.
I
mean.
Have
you
encountered
this
domain?
Do
you
maybe
have
a
workaround?
B
Never
encountered
anything
like
that.
Just
to
clarify
did
you
mention
that
every
user
has
their
own
namespace
exactly
oh
yeah
I've,
never
configured
anything
like
that.
That
would
be
an
interesting
problem
to
solve.
For
me,
I've
always
just
configured,
you
know,
grouping
like
projects
and
things
together.
Valerie
do
you
have
any
thoughts.
A
B
D
C
C
B
E
A
PA
Oh
PA
when
you
get
the
request,
you
have
a
user
info
on
the
request.
So
assuming
so
you
have
the
identity,
you
can
use
the
identity
in
OPA
yeah,
but
you
have
to
have
like
another
sidecar
container
that
extract
the
identities
from
LDAP
or
water.
What
is
your
identity
provider?
Because
you
only
have
the
object
ID,
so
you
have
to
have
to
inject
in
OPA
other
information,
but
it's
there
can
I
also
have
a
question
yeah.
E
E
So
one
issue
that
we're
having
is
that
we
went
with
the
route
to
use
OPA
with
being
more
granular
than
are
back
at
the
namespace
level.
So
we
allow
we
create
for
examples
and
default
Network
policies,
but
we
also
like
to
allow
the
user
to
create
or
delete
metal
policies
from
this
itself
himself
and.
F
A
G
B
I
know
there's
some
sections
of
the
sto
documentation.
I
was
in
Gareth's
unit
testing
kubernetes
with
OPA
before
this
I
haven't
douve
a
lot
into
using
OPA
and
sto
together,
but
I
do
know
it
is
one
project
that
is
bringing
it
in
and
it's
going
to
support
it
natively.
So
you
don't
have
to
be
super
happy
with
it.
Okay,.
G
D
I
have
questions
because
you're
also
mention
that
dividing
to
multiple
nine
space
to
get
more
control.
Also
for
your
experience,
what
is
based
on
your
experience,
the
approach
better,
that
one
big
cluster
for
multi-tenant
user,
or
do
you
have
some
limits
when
I
don't
know
you
reach
like
twenty
nine
space
and
you
create
a
separate
clusters
and
also
did
you
experiment
with
cross
cross
cross?
There
connections
I
think
so.
A
I'm
sure
there's
some
kind
of
point
at
which
sed
would
fall
over
with
too
many
namespaces,
but
certainly
wouldn't
be
xx.
Rule
of
thumb
is
more
likely
when
you
get
to
too
big
of
a
cluster
size
in
terms
of
nodes
that
workload
you'd.
Look
at
splitting
up
personally.
I
advise
against
splitting
workloads
between
clusters
unnecessarily
unless
they
either
have
very
different
SLA
is
required
very
different
clusters
themselves
or
have
like
entirely
different
security
classes
like
payment
processing
versus
a
website.
I.
B
E
F
Thanks
just
one
on
the
number
of
namespaces
yeah,
it's
from
a
different
story,
but
OpenShift
support
limit
is
on
the
order
of
ten
thousand
I.
Think
the
question
I
wanted
to
ask
was
about
so
you
looked
at
supplementing
objects,
resource
definitions
with
as
I
think
you
put
it
well
abusing
annotations,
but
have
you
looked
at
extending
the
role
and
the
role
binding
objects?
Would
that
be
like
something
like
a
label
selector.
A
Potentially,
the
concern
is
it's
way
easier
to
make
it
too
hard
and
easy
to
kind
of
foot
gun
yourself
versus
useful,
but
that's
definitely
another
approach
that
could
be
done.
Cuz,
that's
also
a
bit
more
unified
because
in
Kiruna
is
individual
objects,
are
a
bit
more
mass-produced
and
less
bespoke
compared
to
like
the
traditional
model
of
a
unix
file.