►
From YouTube: Kubernetes SIG API Machinery 20210602
Description
June 2nd
3 topics on platforms setting up clusters:
1) [soorena776] Should we webhook webhooks: https://github.com/kubernetes/kubernetes/pull/101815
2) [vivekbagade] Should we add a 2nd authz webhook?
Issue: link
Next Step: start a KEP and present into SIG Auth.
3) [carried over from last time] (Yuvaraj or Nabarun) - Advice on next steps on the CRD install problem (Discussion here)
-[Nabarun] The 2nd/multiple Authz webhook feature solves the problem of users modifying platform installed resources at runtime.
-[Nabarun] The 2nd webhook will allow us to have a “platform-admin” concept.
-[Nabarun] Open Question: Who initializes the objects? A new controller in controller-manager? An addon-manager like binary?
-How do we ensure HA?
A
This
is
the
sig
api
machinery
by
weekly
meeting
for
kubernetes
and
we
have
again
likely
a
happily
packed
agenda,
so
we
are
going
to
get
started
right
away.
We
did
a
little
bit
of
shuffling
on
the
topics,
because
I
know
that
people
that
participates
on
the
first
one
have
to
leave
around
11
30..
So,
let's
start
with
that.
B
Yeah,
maybe
I
can
introduce
the
first
three-
I
grouped
them
all
together,
because
they're
kind
of
variations
on
the
theme
of
platform
providers
setting
up
and
maintaining
clusters
yeah.
So
I
I
actually
think
the
the
second
one
on
this
list
is
getting
at
the
the
heart
of
the
problem,
which
is
that
there's
no
in
our
in
our
auth
system.
There's
no
space
for
a
platform
administrator
that
has
superpowers
that
a
cluster
administrator.
Doesn't
we
kind
of
do
that
by
putting
files
on
disk?
B
But
that
only
goes
so
far,
so
I
think
yeah,
that's
that's
my
current
feeling
on
it.
So
maybe
you'll
all
agree
after
this
part
of
the
first
part
of
this
is
is
or
the
one
I
put
first,
because
javed
has
to
go
at
11.
30.
Is
this
this
idea
of
sometimes
the
platform
administrator
wants
to
police
things
with
web
hooks,
but
you
can't
police
web
books
with
web
books
so
yeah
java?
Do
you
want
to
talk
more
about
this.
C
Yeah
sure,
hey
everyone,
thanks
for
the
opportunity,
so
this
story
is
a
little
background
of
this.
We
want
to
do
policy
enforcements
and
we
want
to
have
restricted
access
to
name,
space
and
resources,
name,
space
level
and
cluster
level
resources,
and
then
we
do
have
a
notion
of
privileged
identities.
Users
who
are
can
actually
bypass
these
resources.
C
C
But
what
can
do
they
do
is
like
they
can
create
a
mutating
webhooks,
and
then
these
meeting
webhooks
basically
captures
the
requests
which,
from
privileged
users,
which
are
kind
of
like
admitted
by
this
policy
enforcement
and
then
once
they
are
commonly
captured
by
these
end
users
using
web
hooks.
They
can
mutate
it
to
the
arbitrary
values
and
then
target
the
restricted
resources
that
we
have.
C
My
guess
is
it's
not
possible
because
we
wanted
to
avoid
cycles
and
also
want
to
make
sure
we're
not
getting
locked
out
of
well
folks,
in
terms
in
case
there's
a
bad
configuration,
but
then
we're
kind
of
like
saying.
Okay,
what
if
we
won't
do
clusters
coming
up,
we
become
like
a
specified
few
web
hooks
that
could
intercept
other
web
hooks
and
also
we
specify
what
we
call
maintainers,
which
are
exempt
from
these
bug
hooks.
C
D
D
Okay,
I
guess
I'm
trying
to
at
least
understand
what
this
is
telling
me.
So
I
think
what
this
is
telling
me
is
that
there
is
a
web
hook
that
is
being
created
in
the
example,
and
it
says
I
wish
to
allow
another
web
hook
to
intercept
me
is
that
what
that
web
hook
intercepting
web
hooks
with
identifiers
field
is
trying
to
tell
me.
B
B
Then
there's
a
second
layer
of
exception,
which
is
there
are
special
identities
which
may
make
changes
to
web
books
which
don't
go
through
those
special
web
hooks
and
the
reason
for
this
is
the
goal,
the
goal.
As
always,
is
we
don't
necessarily
care
if
you
break
the
cluster,
we
care
if
you
break
the
cluster
in
a
way
that
you
can't
recover
from
without
messing
with
scd.
D
D
Okay,
so
okay,
I'm
wrapping
my
head
around
this
here.
You're
talking
about
creating
a
web
hook,
configuration
that
allows
punching
through
some
users
to
allow
them
to
always
modify
something,
but
you
would
not
always
allow
the
system
master.
The
identity,
that's
actually
hard
coded
in
our
product
as
able
to
do
everything
you
would
not
allow
that
person.
B
D
B
C
B
E
Well,
I
was
gonna
say
I
agree
it's
novel,
but
this
is
similar
to
things
we
do
with
like
the
api
server
network
proxy.
I
mean
those
like
you
said
earlier
are
done
on
disk
daniel,
but
we
do
have
this.
You
know,
especially
in
the
managed
kubernetes
world.
We
have
this
concept
of
a
group
of
users
who
have
access
to
the
direct
access
to
the
control
plane
on
this
persistent
control
plane
on
disk
and
can
make
configuration
changes
that
cannot
normally
be
made
by
other
users.
B
B
B
E
F
B
D
Whether
whether
those
reservations
are
enough
to
say
like
no,
we
shouldn't
do
this,
I'm
not
sure
I
guess
I
do
see
it
as
large
enough
and
significant
enough
of
change
that
it
kept
seems
appropriate
right,
we're
changing
the
rules
about
I
mean
I
was
fairly
certain.
We
initially
wrote
it
said
web
hooks
cannot
intercept
other
web
hooks
flat
stop,
and
we
said
that
and
the
reason
why
is
because
these
you
can
end
up
in
cases
where
you
would
you
would
loop
right,
there's
also
a
consideration
like
if
you
say.
D
E
You
know
that
you
know
carrying
patches
is
really
expensive
and
I'm
not
saying
it
has
to
be
this
particular.
But
there
seems
to
be
a
fair
desire
amongst
the
managed
services
to
have
something
like
this,
where
they
can
have
a
protected
class
and
use
an
easily
added
protected
class
of
rules
or
policies
that
they
can
add.
D
D
I
I
I
have
a
comment
here
on
a
pr,
so
I
don't.
I
don't
know
what
that
intent
is.
There
is
some
behavior
of
of
like
what
does
this
api
do.
Is
it?
Is
it
an
intersection?
Someone
have
to
be
both
alice
and
bob.
At
the
same
time,
and
both
of
these
groups
does
it
have
to
be
alice
with
both
groups?
Is
it
any
combination
of
these
groups,
regardless
of
whether
you're
the
api
isn't
clear
here,
yeah.
B
D
That
is
an
interesting
concept
and
I'd
be
interested
in
knowing
what
the
boundaries
of
that
capability
are,
whether
they
actually
belong
as
part
of
the
surface
area
of
the
current
api
server
we
have
or
whether
we
want
to
create
a
different
set
of
apis
for
them
to
manage
right.
You
recall
that
sigoff
had
a
desire
to
configure
authentication
web
books
via
the
api.
B
D
B
D
B
I'm
not
planning,
so
I
I
in
terms
of
like
making
this
function
and
and
be
complete
feature
complete.
I
believe
that
the
only
changes
to
api
server
that
are
needed
are
the
web
hook
web
hooks
and
adding
the
second
auth
z
web
hook.
I
think
if
you've
got
that,
then
platforms
can
build
whatever
they
need
without
patches,
so
no
that
wouldn't
include
a
dynamically
adding
authen
web
hooks
right.
The
platforms
can
already
do
that.
If
they're
willing
to
mess
with
files
on
disk.
B
It
would
include
things
like
the
platform
admin
wants
to
ensure
that
a
certain
set
of
crds
are
available
and
that
users
can't
change
them
right,
because
you
can
already
do
that
with
our
api.
So
I'm
not
interested
in
adding
capabilities
that
aren't
already
in
our
api
right.
We're
in
this
weird
position
right
now,
where,
if
you
can
monkey
with
files
on
disk,
you
can
do
random,
powerful
things,
but
you
can't
you
can't
have
the
same
level
of
power
over
the
things
that
are
in
the
api,
so
that
that
that's
the.
B
D
The
concept
of
being
able
to
configure
it
exists
in
the
api
right.
Like
I
remember
you
arguing
strongly
against
having
I
guess
either
you
were
jordan,
I
don't
remember
which
it
was.
Are
you
against
having
authentication
web
hooks
present
in
the
api
for
similar
kinds
of
reasons?
Do
you
see
this
as
essentially
that,
like,
for
instance,
if
you
have
the
ability
to
simply
configure
a
hard-coded
admission
web
hook,
that
cannot
be
changed
in
this
admission?
D
D
D
B
D
B
So
that
gets
into
like,
like,
I
think,
there's
actually
a
better
class
of
extension
mechanisms
that
use
a
different
technology
rather
than
than
calling
out
to
a
web
hook.
I'm
talking
about
building
something
out
of
wasm
or
cell,
or
something
like
that
right
that
that
would
actually
be
a
significant
improvement
like
if
you
could
like
specify
at
various
points
in
the
in
the
code
base
like
call
out
to
this
opaque
binary
plugin.
That
would
be
pretty
cool.
B
D
I
mean
like
like
that
all
sounds
like
a
kept
update
to
me,
like
it
explicitly
says
what
it
does
and
this
changes
that,
and
it
says
that
reasoning
should
be
re-evaluated,
and
I
also
want
to
know
what
else
this
user,
what
else's
user
concept
should
be
able
to
do?
Will
it
have
a
consistent
name?
Is
it
addressable
is
a
platform
admin,
an
addressable
thing.
D
C
C
We
want
to
differentiate
between
cube
admin,
and
then
it
could
be
these
platform
provider
users
which
are
kind
of
like
trying
to
present
the
cluster
in
a
certain
way,
and
then
the
cube
admin
is
different
than
that,
because
q
batman
is
basically
the
end
user.
But
then
these
privileged
users
that
we're
talking
about
are
required
to
present
the
cluster
and
certain
configurations
that
we
want,
which
we
look
at,
is
different.
D
D
B
Yeah
I've
got
two
answers
for
that.
One
is
it's
just
less
work
if
we
make
everything
work
the
same
way
right,
we
only
have
to
build
one
one
set
of
mechanisms,
and
I
had
a
oh
and
the
other.
One
is
if
we,
if
we
do
build
two
mechanisms,
we
end
up
with
the
nice
api
for
users
who
use
it
all
the
time
and
the
not
so
nice
api
for
platform
administrators-
and
I
like.
Why
would
we
do
that
to
ourselves?
D
True-
and
I
think
we
may
have
gone
after
solving
this
problem
differently
right
where
you
can
go
after
this
problem
and
actually
put
the
things
in
your
platform
that
you
want
to
protect
and
maintain
in
a
place
that
a
user
can't
see
them
and
can't
touch
them
and
can't
modify
them.
And
all
those
properties
means
that
you
don't
end
up
trying
to
stop
users
from
touching
certain
pieces
of
special
objects
in
their
cluster.
D
D
B
B
Random
manifests
on
disk
and
api
server
would
ensure
their
presence
in
the
cluster,
and
I
thought
that
that
would
go
far
enough,
but
longer
the
longer
I
thought
about
it
and
I
think
vivec
or
javad,
or
both
here
have
written
up
increasingly
elaborate
designs
along
that
theme
and
and
like
we
just
it
doesn't
seem
to
really
solve
all
the
problems
I
mean
we
can.
I
guess
it's
the
next
thing
on
the
agenda,
so
we
can.
We
can
go
on
and
and
zoom
in
on
that,
but
the
next
one.
B
Seemed
more
of
a
slam
dunk
to
me:
oh
the
auth,
sorry
yeah,
the
authority
web
hook.
I
guess
we
haven't
actually
talked
about
that
yeah
yeah.
So
so
so
that's
it!
That's
interesting!
So
you're
not
against
giving
people
the
ability
to
make
a
super
user,
because
that's
that's
what
the
that's!
The
from
our
perspective.
That's
the
point
of
the
second
aussie
web
book
that
runs
first
is
well.
D
D
B
D
D
B
D
B
D
B
A
B
F
B
D
Sig
auth,
I
do,
and
I
expect
that
they
would
ask
you
for
a
cap.
A
lot
of
that
is
like
explaining
for
people
who
want
to
use
this
in
their
clusters
what
the
boundaries
are
on,
an
r.
Why
you
have
it
why
it
exists?
What
you
need
to
be
careful
of
right,
like
if
you
suddenly
start
denying
system
masters,
there's
gonna
be
a
lot
of
code
that
doesn't
expect
that
and
yeah
yeah.
I
mean,
I
think,
that's
a
reasonable
next
step,
but
basically
all
the
same
reasons.
B
B
B
Yeah
and
and
to
be
clear,
I
think
the
the
auth
z
web
hook
is
more
important.
There
are
more
possibilities
for
the
for
the
web
hook.
Web
hook
thing
right.
We
can.
We
can
imagine
that
actually
gets
into
the
entire
problem
of
like
there's,
there's
ways
you
can
break
your
cluster
with
adding
web
hooks
yeah,
which
we
definitely.
D
B
G
So
the
primary
goal
of
our
exploration
was
to
enable
kubernetes
contributors
and
optionally
administrators
to
specify
certain
crds
to
be
installed
at
cluster
startup
time,
which
they
can
use
as
soon
as
the
ebay
server
goes
delay.
So
our
exploration
was
not
from
the
point
of
view
of
like
adding
any
arbitrary
resource,
but
mostly
around
like
crts.
G
Also,
this
came
into
forefront
because
of
one
more
thing:
sick,
multicluster.
I
guess
they
proposed
something
like
a
cluster
property
and
wanted
some
reviews
from
cigars
and
that's
why,
like?
We
also
started
again
exploring
that
as
to
if
any
of
our
code
owners
want
to
introduce
any
crds,
but
they
want
to
ensure
that
the
crd
is
installed
as
soon
as
the
ap
server
goes
healthy.
How
do
we
do
it?
G
G
Now
there
there
were
some
caveats
or
open
questions
that
we
thought
number
one
was:
how
do
we
ensure
that
those
resources
stay
as
they
were
installed
during
the
run
time
of
the
cluster?
So
what
we
felt
was
through
your
response
and
reading
through
the
second
website
book
discussion
that
that
can
probably
prevent
it
now.
The
other
open
question
which
remains
is
the
cool
initialization.
G
B
Yeah,
the
upgrade
question
is
maybe
50
of
why
it's
not
a
good
idea
to
be
doing
this
in
api
server
post
start
hooks,
because
api
server
has
to
run
in
split
split
mode.
It's
much
easier
to
do
this
in
basically
anywhere
else.
Like
controller
managers
are
leader
elected
there's
only
one
of
them
running
so
like
at
least
you
know
if
you're
elected,
then
you
know
like
you're
responsible
for
setting
the
the
the
current
version
of
whatever
these
are
yeah.
B
B
D
D
Wasn't
clear
to
me
in
concept
how
that
would
work
in
the
case
of
cluster
upgrades,
where
the
thing
wasn't
present
when
the
cluster
was
available
before
and
during
an
aha
upgrade,
it
may
be
present,
it
may
not
be
present
and
there's
no
sense
of
like
it's
always
been
available.
So
what
does
it
mean
always
available
for
you.
G
G
So
we
were
also
like
stuck
on
that
question,
so
there
are,
there
are
cases
like
1.x
not
having
that
resource,
but
one
dot
express
on
introducing
anyone,
one
dot
x,
having
a
resource
one
would
express
on
updating
it
or
removing
it.
So
we
still
don't
have
like
answers
to
that
question
and
we
one
of
the
intents
of
that
email
was
to
like
try
to
assess
answers
to
that
question
like
thoughts
from
the
community,
so
so
that
we
can
reach
to
an
acceptable
solution.
B
Yeah
I
I
have
two
answers.
The
first
one
is
like
cluster
startup
is
a
different
case
than
a
cluster
upgrade,
and
it's
okay.
If
there's
some
different
meanings
there,
the
other
is,
you
know
if
there's
some
component,
that
is
responsible
for
updating
these
things
we
can
define
when
in
the
upgrade
process.
That
component
is
supposed
to
be
upgraded
right
like
right
now,
if
you're
not
upgrading
everything
at
once,
then
there
is
an
order
in
which
you
have
to
do
things
right.
You
have
to
do
web
books.
First
then,
api
server,
then
controllers.
B
B
It's
an
open
question
to
me
whether
we
want
to
do
this
in
open
source
or
not,
because
if
we,
if
we
have
this
second
aussie
webhook,
one
of
the
purposes
of
that
is
to
hold
non-privileged
traffic
to
the
cluster
until
this
initializer,
whatever
it
is,
initializes
things,
I
think,
that's
a
much
easier
sell
than
making
a
new
initializer
type
thing.
I
I
don't.
B
B
I
think
both
doing
the
add-on
manager
and
a
new
binary
place
constraints
on
cluster
providers
like
people
who
are
starting
clusters.
It's
like
oh
there's,
this
new
binary.
You
have
to
run
good
luck
figuring
out
how
to
do
that.
So
people
aren't
super
thrilled
with
that,
so
controller
manager
is
probably
the
the
path
of
least
resistance.
I
guess.
G
B
B
B
D
I
would
not
like
to
see
a
case
where
the
cube
api
server
readiness
is
informed
by
the
presence
or
absence
of
some
particular
set
of
resources,
and
I
see
difficulties
there
because
you
know
a
lot
of
times.
You'll
want
a
resource
that
actually
needs
something
else
to
take
an
action,
and
that
complicates
it
quite
a
bit.
So
that
means,
in
my
mind
that,
before
the
cluster
is
available,
isn't
a
concept
that
I
think
has
generic
meaning
across
all
cube
clusters.
I
think
this
is
just
a
after
cluster
is
running.
I
can
install
this.
B
That's
actually
a
great
point.
I
was
not
envisioning
this
as
affecting
the
like
health,
z,
endpoint
or
the
ready
z
endpoint.
I
was
envisioning.
The
auth
z
web
hook
itself
having
a
latch
right,
so
it
starts
up
and
it
blocks
traffic.
That's
like
not
system
masters
or
not
not
platform,
and
if,
if
you
have
that
concept,
then
this
installer
runs
and
then
it
directly
tells
the
authy
web
hook.
Hey
the
cluster
is
good.
D
To
the
actual
open
source
project
itself,
I
do
see
some
value
in
having
a
way
for
us
to
have
the
cube
controller
manager,
create
a
create
and
maintain
a
an
opinion
on
a
crd,
and
I
see
that
benefit,
because
I
think
that
is
what
stalled
a
couple.
Different
storage
efforts
was:
they
wanted
to
create
their
resources
as
crds
and
not
as
core
resources,
whether
that
was
you
know,
good
bad
or
otherwise.
That
was
their
goal
runtime
class,
I
think
yeah.
D
D
B
B
B
Universal
concept
right:
well,
it
is,
if
you
it
is,
if
we
maintain
and
encode
a
like
generation
on
the
on
the
set
of
objects,
right,
like
we
can
say,
we're
going
to
every
release,
we're
going
to
increment
this
number,
but
in
in
general,
it's
very
limiting,
because
people
do
occasionally
want
to
roll
back
the
cluster
version
and
in
theory
at
least
we
support
that
by
one
one
revision.
So
it's
a
little
weird
if
you
roll
back
and
then
your
crd
doesn't
go
back
with
you.
B
If
we're
going
to
run
like
a
web
hook
inside
it
or
something
like
that,
and
we
also
may
not
want
to
force
people
to
run
the
binary
in
the
cluster
right,
like
a
platform
provider
can
do
that
if
they
want
to,
but
we
in
open
source
can't
add
an
invariant
to
like
all
clusters
like
that.
I
don't
think
that
would
be
very
popular
at
all.
B
D
G
Fine,
so
one
of
the
things
yeah.
So
if
we
we
did
take
that
into
cognizance
that
whenever
we
install
a
crd,
it's
not
just
the
crd.
They
have
like
associated
by
books
and
controllers,
and
we
kind
of
assume
that
whoever
is
expecting
that
crd
to
be
there
they
will
install.
They
will
have
their
own
kind
of
processes
to
have
them,
which
also
is
kind
of
a
bit
finicky,
as
in
like
expecting
an
external
entity
to
externalizing
any
any
other
open
source
code
owner
to
plan
a
process
like
that.
G
B
Yeah,
I
think
that
the
at
least
at
first
the
plan
was
this-
is
just
left
up
to
the
to
the
platform
administrator
or
the
cluster
administrator.
If
it
happens
to
be
the
same
person,
it's
unclear
to
me
if
we
would
want
a
like
like
if
we
wanted
a
like
here's,
here's
a
thing
that
we
could
do
that.
I
don't
think
we're
going
to
want
to,
but
you
can
imagine
that
we
decide
like
we're
going
to
not
only
add
a
spot
for
a
a
new
auth
z
evaluation
to
happen.
B
We're
also
going
to
add
a
concept
that
is
describes
how
you
can
build
a
deny
rule
right.
That
would
be
an
entirely
larger
undertaking
and-
and
I
think,
we're
not
suggesting
that
we're
just
giving
way
for
this
to
happen.
If
the
platform
administrator
wants
it
to
happen,
and
I
think
we
could,
the
future
could
evolve
in
two
ways.
Right
like
there
could
be
a
a
platform
or
several
platforms
that
are
like
hey.
B
We
came
up
with
this
way
of
describing
deny
rules
and
we
think
it's
great
and
we
want
to
open
sources
so
that
so
that
all
clusters
can
work
together.
Blah
blah
like
that
could
happen,
or
maybe
it
wouldn't
happen
and
platforms
are
like
you
know.
This
deny
rule
is
just
for
us
and
and
we
don't
need
to
extend
it
and
give
it
to
users.
D
You
david
yeah,
I
mean
there
are
I've,
seen
a
couple
different
ways
where
people
try
to
describe
deny
rules
today.
The
one
that
I
like
best
is
the
one,
the
one
that
I
like
the
best
that
I've
seen
so
far
is
one
where,
like
there
is
some
aspect
of
a
token
review
that
says
the
information
on
your
token
says
you
should
be
restricted
this
way
and
we
built
a
way
to
pass
that
from
the
authenticator
to
the
authorizer
and
the
authorizer
uses
the
information.
D
B
G
D
D
We
were
able
to
solve
them
by
doing
this,
latching
concept
that
may
or
may
not
be
good,
but
even
if
we
get
them
in
the
cube
controller
manager,
it
won't
work
well
because
we
won't
be
able
to
have
the
web
hooks
handle
the
api
evolution.
So
in
the
end
we
thought
about.
It
came
up
with
several
different
ideas.
None
of
which
worked
super
well
and
the
person
comes
up
with
the
next
idea.
Please
let
us
know,
I
think,
that's
where
we
ended
up.
I
don't
know,
I
don't
know
if
you
daniel.
B
B
B
If,
if
api
server
like,
if
you
know
you
program
in
a
different
web,
hook,
configuration-
and
that
is
seen
by
an
api
server,
which
only
has
access
to
an
old
control
like
that,
that
gets
that
gets
complicated
again,
but
it
seems
possible
and
it
also
further,
it
seems
like
adding
a
new
controller
control.
Plane
component
doesn't
help
this
problem
right.
You
still
have
this
problem,
whether
you're
running
it
in
control.
D
B
I
think
there's
either
way
is
writing
it
down
yeah,
even
just
what
I
said
has
many
cases
right
and
I'm
sure
that
I
didn't
cover
all
the
bases
yet,
so
I
think
I
kept
discussing
a
some
sort
of
controller
and
controller
manager
that
adds
at
least
crds
and
maybe
a
few
other
resources
at
cluster
startup.
I
think
that
that
that
might
be
the
next
step.
I
can't
think
of
a
thing
that
would
be
more
useful
than
that.
G
G
B
Of
yeah,
I
I
feel
like
we
should
skip
just
punt
on
the
production
mechanism
at
first
right
at
first,
like
you
know,
if
you're
a
client
and
you
need
this
thing
like
just
keep
retrying
until
it
shows
up,
and
you
call
it
a
day-
that's
that's
that's
much
more
robust
than
like.
You
know,
shutting
down
the
cluster
until
the
thing
shows
up
anyway,
so
I
I
think
we
should
just
skip
the
protection
mechanism
to
begin
with,
and
if
platform
administrators
want
to
implement
something
like
great
go
for
it.