►
From YouTube: Kubernetes SIG API Machinery 20210825
Description
Aug 25th
-[Vijay Tripathi] Discuss https://github.com/kubernetes-sigs/controller-tools/pull/569
Remove the DangerousTypes
Continue discussion about pros and cons in the mailing list
-[jefftree] OpenAPI v3 https://github.com/kubernetes/enhancements/pull/2898
-[sttts,tkashem] inconsistent behaviour of kube-apiserver before readiness, and clients that care, like GC and namespace controllers
GC issue: https://github.com/kubernetes/kubernetes/issues/104342
before readiness (/readyz 200), i.e. without load-balancer
CRDs potentially unavailable (404)
aggregated resources potentially unavailable (404)
discovery incomplete
This one impacts the namespace lifecycle controller in an easy to see way. Possibly others.
OpenAPI incomplete (there is a PR at least fixing CRDs)
RBAC incomplete (403 where it shouldn't)
[deads2k] - this doesn’t sound so bad. Controllers retry and people F5.
Idea 1: --startup-send-retry-after-until-ready sending 429 with Retry-After header
Problem: problematic to unbrick self-hosted clusters
Idea 2: 429 instead of 404 for GC protection. But discovery?
Idea 3: 429 for GC/namespace-lifecycle-controller only
Idea 4: add request header that makes the request conditional on the apiserver being ready
Idea 5: add a reply header or content --- at least for discovery and 404 replies --- that indicates whether the server is ready
-[@mkimuram] Liens - https://github.com/kubernetes/enhancements/pull/2840
A
Hello
and
welcome
everybody
to
the
seek
api
machinery
by
weekly
meeting
today
is
august
25th
of
2021.
Can
you
believe
we
are
closing
august
already
time
is
flying.
We
have
a
great
agenda
today.
Hopefully
we
have
a
great
agenda
for
the
next
couple
of
meetings
too.
I
see
a
lot
of
topics
in
the
backlog
and
it's
great
to
have
these
good
discussions.
So
thank
you,
everybody
for
helping
us
to
populate
this.
Let's
jump
into
the
agenda
right
away.
I'm
sharing
my
screen
and,
let's
start
with
vj
hi.
B
Hey
everyone
good
morning
and
good
afternoon,
a
little
bit
of
background
about
the
topic
I
want
to
discuss.
So
I
work
on
ack
team,
which
is
aws
controllers
for
kubernetes,
similar
to
azure
service
operator,
and
what
we
do
is
we
consume
sdk
published
by
aws
and
try
to
convert
those
into
crds
and
build
controllers.
On
top
of
that,
so
we
ran
into
a
problem
where
aws
sdk
started
having
some
fields
which
were
map
of
map,
and
they
were
not
supported
by
the
controller
gen
library.
B
C
E
C
C
C
D
F
G
F
C
Yeah
david
asks
in
chad,
if
that's
a,
if
dangerous
type
is
a
cube
concept
or
a
controller
on
time
concept,
it
must
be
a
controller
runtime
concept,
because
I've
not
heard
of
it
before
we're
at
a
bit
of
a
disadvantage
here,
because
sally
has
switched
teams.
So
I
I
think
we're
we're
a
little
lacking
on
controller
on
expertise
here.
H
B
B
B
H
C
C
H
H
C
F
C
A
B
B
So
that's
what
I
like
intended
to
achieve
from
this
meeting,
like
maybe
somebody
who's
a
owner,
can
give
me
feedback
on
the
pr
and
give
me
steps.
I
need
to
do
to
get
it
merged
into
the
master
into
the
main,
and
that
was
my
intended,
but
I'm
hearing,
like
the
feedback
that
I
got,
was
at
least
remove
the
dangerous
type
first
from
the
pr.
C
And
then,
if
you
can,
if
there's
some
debate
over
whether
it's
good,
if
you
can
maybe
come
up
with
a
with
a
summary
of
the
debate
and
post
it
to
the
mailing
list,
like
pros
and
cons
or
or
or
something
like
that,
that
would
that
would
help.
I
think
I
think
it's
a
good.
This
is
a
better
debate
for
a
mailing
list
than
than
a
sig
call
and
and
like.
If
we
can't
converge
in
the
mailing
list,
then
we
can
bring
it
up
again.
B
I
I
I
I
One
additional
thing
for
v3
is
that
for
performance
reasons,
I
would
like
to
try
to
avoid
aggregating
the
entire
schema
and
only
publish
the
schemas
split
by
a
certain
category
right
now.
I
think
we're
thinking
about
splitting
everything
per
group
such
that
users
will
request
only
a
particular
group
instead
of
requesting
the
entire
schema,
and
the
benefit
of
this
is
that
our
aggregator
will
not
need
to
work
as
hard
to
aggregate
the
entire
open
api
schema
for
all
built-in
crds
api
services.
I
A
I
Cool,
so
just
as
a
brief
overview,
so
the
v2
endpoint
looks
at
the
open
api
v2
such
opening
psi
v2,
and
what
I
want
to
do
is
create
a
v3
endpoint
and
this
v3
endpoint
the
initial
v3
endpoint,
because
we
want
to
split
by
group.
This
will
just
serve
as
a
discovery.
Endpoint
such
that
we
have
a
list
of
all
the
groups.
I
This
may
or
may
not
be
necessary,
but
I
think
for
now
just
for
discovery
purposes.
It
makes
things
a
lot
easier
and
essentially,
given
this
v3
endpoint,
we
can
go
to
any
path
that
we
want
and
basically
only
the
the
resources
required
for
resources
and
path
required
for
a
particular
path.
So,
for
example,
in
the
apps,
this
is
probably
a
bit
hard
to
see,
but
this
contains
only
the
schemas
and
paths
available
for
the
paths
group
or
for
the
apps
group,
and
likewise
we
can
do
this
for
crds.
I
I
G
I
have
a
question
I
looked
briefly
through
the
cap.
Is
there
anything
about
client
go
the
main
reason
in
the
client
core?
Sorry
cube
cuddle
the
main
reason
for
the
stripping
we
do.
I
mean
there
are
some
fields
we
cannot
represent
in
v2,
but
the
main
reason
is
that
we
have
a
custom
belly
data
and,
of
course,
we
also
want
to
switch
to
go
open
api.
As
a
library
we
have
in
our
kubernetes
open
api
implementation
now
so,
basically
the
same
validator
we
use
in
the
server.
G
C
My
thought
on
that
is,
there's
I
I
think
we
heard
about
it
last
time,
but
there's
a
there's
a
another
effort
to
start
doing.
Server-Side
validation
so
that
cube
cuddle
doesn't
have
to
do
validation
anymore,
and
I
think
we
could
leave
the
v2
open
api
published
for
as
long
as
we
want
to
keep
the
existing
cube,
cuddle,
behavior,
working
and
then
possibly
keep
cuddle,
never
has
to
understand
open
api
v3
or
maybe
it
could.
I
don't
know.
H
J
H
I
H
G
C
The
argument
for
splitting
my
group
is
that
if
you,
if
you
assume
that
people
will
use
the
things
in
a
group
together,
then
you've
got
a
complete
set
of
it.
If
you
split
by
group
version
since
we're
letting
people
be
sparse
here,
you
may
need
to
consume
multiple
versions
to
have
a
complete
view
of
the
group.
C
I
don't
think
it's
a
particularly
big
deal
either
way
since
it's
going
to
be
computers
doing
this,
for
you,
that's
the
those
are
the
factors
I
can
think
of
other
than
you
know
just
just
being
consistent
and
having
a
if
you
do
it
by
group
version.
That's
that's
the
level
at
which
the
aggregator
splits
so
you've
got
a
very
simple.
H
C
G
G
C
G
Abu
and
myself,
I
will
start
maybe
so
this
came
up
internally
in
openshift,
but
mike
sprites
added
another
threat
in
our
sixth
channel
about
the
same
topic.
Colleagues
noticed
that
objects
go
away
and
garbage
production
seems
to
get
confused
when
the
server
is
restarted
and
we
quickly
answered.
Of
course
it
is
the
case
here
and
for
us
for
us.
Yes,
it's
of
course,
because
we
thought
about
that.
But
many
people
want
to
know
that
a
server
which
is
not
ready
yet
like
ready
set,
is
not
200.
G
As
a
return
code
has
some
some
issues
like
crds,
which
are
coming
from
informers,
might
be
unavailable
and
you
get
a
404
same
thing
for
api
services
from
aggregate
api
servers.
Discovery
for
the
same
reason
can
be
incomplete,
open
api
can
be
incomplete
because
it's
also
based
on
informers
and
arbuck
authorization.
The
same
case
most
of
the
things
are
ugly,
but
not
critical,
but
garbage
collection
and
the
name
space
controller
and
at
least
those
we
know
of
they
act
on
404s
or
on
incomplete
discovery.
G
So
this
was
a
problem
and
there
was
a
short
discussion
and
whether
we
want
to
change
that,
especially
the
out-of-the-box
solution,
because
of
course
you
can
have
an
open
answer
in
front
which
protects
the
client
from
non-ready
api
servers,
and
there
are
a
couple
of
ideas
how
to
do
that.
We
can
protect
ourselves
against
gc.
This
very
special
specific
issue
like
hiding
the
404
from
gc.
That's
the
idea
too.
At
the
bottom.
There.
J
D
G
J
H
G
C
D
C
D
C
C
C
D
C
I'll,
I
guess
I'll
I'll
state
my
objection
to
all
the
api
server
ones,
which
is
that
it
seems
incredibly
dangerous,
like
the
the
the
chance
of
getting
your
cluster
into
a
state
where
it
can't
actually
successfully
start
up,
because
you've
got
controllers
waiting
to
talk
to
api
server
to
make
some
necessary
condition.
True,
but
api
server
isn't
answering
requests
because
whatever
that
necessary
condition,
true
is
is
false
and
it's
unhealthy.
H
D
H
H
Then
you
end
up
in
states
where
you
actually
need
to
correct
configuration
by
talking
to
the
api
server,
and
if
you
have
self-hosted
clusters,
you
have
cases
where
components
on
the
cluster
need
to
start
and
make
requests
in
order
to
you
know,
sort
of
bootstrap.
You
can
imagine
needing
to
make
something
like
an
admission
web
book,
trying
to
make
a
delegated
official
call
right
and
so
you'd
have
to
find
all
of
those
categorize
them
and
and
allow
them
all
through.
So
I
think
it's
impractical
for
both
self-hosted
and
self-managed
clusters.
H
The
idea
of
of
returning
a
429
at
the
last
fall
through
point
that
we
have
for
for
the
qps
or
when
it's
built
up,
there's
like
a
final
handler
in
the
chain
that
returns
a
404
today,
the
idea
of
making
that
one
return
a
429
before
it's
ready
doesn't
solve
discovery,
but
it
would
resolve
the
immediate
gc
problem.
If
I
make
a
query
and
something
I
expected
to
be,
there
isn't,
instead
of
getting
a
404
I'll,
try
again
later.
D
H
C
G
G
H
C
J
D
C
J
H
D
C
H
B
D
C
D
C
J
C
And
while
old
clients
won't
be
setting
this
this
header-
yes,
we,
the
issue
is
like
400
errors
are
supposed
to
be
things
that
the
client
needs
to
do
to
to
address
the
problem
and,
like
the
the
precondition,
isn't,
isn't
a
client
problem.
It's
a
server
problem,
so
I'm
not
sure
it's
a
hundred
percent.
You
always.
C
C
L
C
A
M
D
M
M
H
C
D
H
What
I'm
saying
is
opt-in
right,
so
I
think
if
a
client
decides,
you
know
what
it's
better
for
me
to
get
a
failed
request
than
to
get
an
answer
from
an
outrage.
Server,
that's
sort
of
one
case
and
something
like
gc
and
namespace.
Lifecycle
controller
would
probably
use
that
on
their
discovery
goals,
but
but
the
rest
of
client,
the
rest
of
the
clients.
We
have
will
behave
as
they
do
today.
C
I
think
there's
a
large
number
of
clients
that
are
involved
in
getting
the
server
to
be
ready
in
many
installations,
and
I
think
we
should
assume
that
a
given
client
is
involved
in
the
becoming
ready
process.
Unless
the
client
tells
us
specifically
that
they
aren't
and
that
it's
okay
to
give
them
errors.
When
the
server's
not
ready.
D
D
I
guess
I'm
trying
to
make
the
distinction
here
right.
I
guess
I'm
trying
to
make
the
distinction
between
clients
that
don't
even
care
about
whether
they've
gotten
the
complete
result
or
not
right
and
clients
that
really
make
that
will
can
use
a
partial
result
and
will
use
it
differently
than
they
will
use
a
full
result.
C
D
Not
either
I'm
right,
my
question
is
you
know
it's.
I
can
think
of
a
user
right
as
a
human.
You
know
I
can
deal
with
that
kind
of
thing
and
it's
meaningful
to
me
and
potentially
you
know,
I'm
just
saying
you
know
if
we
do
it
with
the
the
precondition
we
kind
of
rule
this
out,
whereas
if
we
do
it
by
adding
an
info
bit
on
the
reply
and
we
we
allow
it.
H
C
If
you
get
up,
if
you
like
know,
you
get
a
partial
response,
because
that's
not
something
that
the
server
communicates
today
so
in
in
terms
of
helping
people
use
this
adding
a
adding
a
precondition
that
just
triggers
one
of
their
already
existing
responses
is
seems
very
safe.
I'm
not
saying
that
there
isn't
a
need
for
the
other
thing,
but
I
I
don't
know
if
I
see
it.
D
B
G
C
G
B
N
Yes,
could
you
go
to
summary
here:
escape
proposes
a
generic
feature
to
protect
objects
from
duration,
while
it
is
marked
as
in
use-
and
actually
this
is
discussed
in
issue
10179,
which
brian
grant
opened
as
generalized
prevention,
accidental
duration
mutation
very
long
ago,
and
in
this
issue
how
to
solve
this.
This
issue
has
already
been
discussed.
A
lot
and
concept
was
called
bn
and
therefore
I
borrowed
the
concept
and
continues
the
work
in
this
gap,
and
the
motivation
of
this
cap
is
currently
a
generic
mechanism.
D
C
Number,
I
can
look,
why
don't
you?
Let
me
answer
that
one!
That's
actually
a
reference
to
the
prior
iteration
of
this.
The
the
first
version
of
this
cap
had
a
mechanism
where
it
was
like
a
reverse
garbage
collector
that
kept
the
object
alive,
while
some
other
object
existed.
But
then
we
had
a
conversation,
and
I
see.
D
D
N
C
N
H
B
D
H
C
C
N
N
Multiple
resources
can
be
used
around
our
resource
can
be
used
by
multiple
resources.
So
I
would
like
to
make
begin
to
add
pro
reasons.
Could
you
go
down
a
little
bit
more
yes
like
this.
By
doing
so
controller
can
there
is
a
reading,
particularly
and
if
the
the
resource
is
not
used
the
target
resource?
N
C
C
C
We
are
running
out
of
time,
so
I'm
not
sure
I
don't
think
we
can
do
this
justice
in
one
minute.
I
I
I
would
say
I
think
that
this
is
what
people
usually
want
when
they
add
a
finalizer
except
for
particular
cases.
I
think
most
humans
would
prefer
that
it
stops
the
prevents
the
deletion
from
starting.
So
I
think
it
would
be
a
really
nice
feature
on
that.
I
think
there's
a
couple
open
questions.
C
There's
this
one,
that's
on
the
screen
right
now
and
there's
also
the
question
of
what
do
you
do
if
somebody
puts
this
on
and
then
somebody
on
some
object
and
somebody
deletes
the
whole
namespace,
should
that
block
the
namespace
deletion
or
should
the
namespace
just
collaborate
anyway,
when
it
gets
deleted?
What
happens.