►
From YouTube: SIG Instrumentation 20200430
Description
SIG Instrumentation Meeting - April 30th 2020
KEPs:
1) Mechanism for warning API clients about deprecated API use
2) Accurate Pod resource request/limit metric
3) Dynamic logs sanitization
A
B
A
A
B
It
doesn't
FYI
that
I
just
updated
it
today
because
there's
later
changes
but
yeah
the
release
cycle
was
published.
So
the
the
enhancement
freeze
is
pushed
two
weeks
to
May
19th
and
the
code
freeze
is
pushed
also
two
weeks
to
June,
25
and
yeah,
but
just
FYI
for
people
to
write
caps
that
they
have
additional
two
weeks
to
work
on
them.
That's
all.
C
D
C
C
A
D
C
Little
bit
a
little
bit
all
right,
so
this
this
can
has
a
few
aspects.
The
motivation
is
that
kubernetes
has
a
lot
of
things
going
on
and
one
of
those
things
is,
if
you
guys
graduate
from
beta
to
GA
and
we
deprecate
to
remove
the
pre-release
versions
over
a
fairly
long
period
of
time,
like
multiple
releases,
sometimes
a
year,
those
depredations
are
announced
in
release,
notes
and
in
API
Docs
and
sometimes
blog
posts,
but
there's
no
great
way
for
users
or
admins
to
have
visibility
to
the
fact
that
they're
actually
using
these
deprecated
API.
B
C
Release
notes
document
and
keep
track
of
the
timelines
and
everything
when
the
API
is
actually
go
away
or
oftens.
This
cat
is
proposing
a
way
to
surface
used
to
users
and
a
way
to
give
admins
visibility
to
deprecated
API
use
in
their
clusters.
So
this
is
a
demo
of
what
it
could
look
like
in
practice.
C
They
work
under
some
circumstances,
so
this
this
hook
stand
at
the
client
level,
which
means
that,
no
matter
what
operations
I
do
when
I
talk
to
that
endpoint.
If
I
get
a
warning
header
back
I
bet
that
header
gets
processed,
it
even
works
for
like
raw
raw
requests.
Where
we
point
Q
control
directly
at
endpoint
and
don't
do
any
other
sort
of
client-side
processing,
the
client
still
sees
that
morning.
So
from
a
user
perspective,
this
is
great
like
if
I,
if
I
am
using
these
deprecated
things,
I
get
told
frequently
like
this
is
deprecated.
C
This
is
what
it
was
deprecated.
This
is
when
it's
going
away.
This
is
what
you
should
use.
So
that's
that's
the
first
half
of
the
proposal.
What
that's,
perhaps
more
interesting
to
use
against
permutation
is
from
a
cluster
admin
perspective,
a
we
were
running
version
121
and
I
wanted
to
know
if
it
was
safe
to
upgrade
this
cluster
to
122,
where
this
deprecated
API
is
going
to
go
away.
C
C
If
API,
that
I
remove
in
that
version,
were
used
over
some
period
of
time,
the
last
day
last
week
last
month,
whatever
whatever
they
want
to
get
on,
and
then
if
there
is
usage,
the
third
part
that
the
the
capitalism's
is
adding
an
annotation
to
audit
events.
So
if
you
see
usage
of
these
deprecated
things,
you
could
then
go
query
your
audit
logs
to
track
down
like
which
specific
client
or
namespace
or
whatever
you
need
to
go
fix
and
then
reset
the
metric
or
watch
for
the
metric
no
longer
growing
over
time.
C
That's
my
demo!
That's
my
spiel
yeah
one
of
the
things
I
was
talking
with
Han
about
I
know.
We
had
cardinality
issues
with
some
of
the
other
metrics
around
request
requests
with
lots
of
dimensions,
and
so
we
tried
to
cut
this
down
to
just
version
resource
removed,
major
minor
version
and
verbs.
So
we
drop
some
of
the
really
problematic
things
like
the
component
and
the
content
type
and
the
dry
run
like
we
tried
to
drop
out
all
of
the
things
that
wouldn't
actually
make
sense
and
we
blow
up
our
cardinality
yeah.
C
A
A
A
F
I
think
it
reads
better
and
also
it's
more
consistent
with
the
way
that
we
began
expressing
versions
in
kubernetes
right
I
mean
we
dispatch
meant
that
version
string
and
we
use
some
very
library
through
the
parts
of
thing,
but
on
like
in
on
the
server
side.
I
want
metrics
and
gestures.
You
can
you
can
actually
just
you
know,
make
separated
virtually.
You
could
do
both
if
you
really
want
okay.
C
The
issue
is,
if
you,
if
you
use
those,
then
you
have
to
go
it's
on
the
admin,
basically
to
go
figure
out
which
API
is.
They
should
be
looking
at
for
a
given
release,
and
so
that's
the
thing
that
we
historically
had
a
lot
of
trouble
with.
Like
didn't.
Didn't
you
read
the
release
notes
nine
months
ago
it
was
on
page
17
that
we
said
this
random
thing
was
gonna,
be
removed
in
this
random
release,
and
sometimes
those
targets
even
change
right
like
what
we
said
in
the
release
notes.
C
C
G
C
F
G
G
D
G
Reporting
two
bits
of
data
that
are
the
same
data
in
two
different
series.
You
can
report
all
of
that
data
in
the
existing
series,
because
that
series
for
a
given
version
of
code
is
either
deprecated
or
not.
There's
there's
other
things
to
it
like
it
makes
that
metric
more
complex,
which
is
itself
an
argument.
But
like
would
we
come
up
with
other
links?
The
the
resource
tracking
one
actually
had
some
implications
of
this
is
like
or
the
scheduling
one.
So
it's
just
like
that's
a
good
thing
too.
A
Yeah
I
think
I
think
I
see
the
point
like
I
think.
Actually
it
would
almost
have
the
opposite
effect
of
by
introducing
these
these
metrics
right.
We
would
be
adding
more
metrics
nor
more
series,
whereas
what
Clayton
was
saying
is
that
we
could
actually
expose
the
same
information
with
the
same
amount
of
series.
Yeah.
C
C
G
Would
complicate
upgrades
a
little
bit,
but
upgrade
metrics
are
already.
You
already
have
to
take
that
into
account
for
failover
anyway,
because
you're
gonna
get
your
in
a
multiple
series
for
the
different
servers.
The
different
versions,
and
so
aggregate
queries
already
have
to
take
that
into
account.
It's
more
of
a
do.
G
We
decorate
is
there
actually
more
of
a
use
case
in
other
places,
for
us
adding
more
constant
labels
to
series
that
convey
operational
information
that
we
previously
did
not
consider
adding
like
aggregate,
api's
and
I'm
not
advocating
for
this
at
all,
but
like
as
a
theory,
the
aggregator
proxy
we
track.
All
of
those
API
request
calls
as
they
pass
through
the
API
server
when
those
are
two
groups
that
are
hosted
on
an
aggregated
API,
is
it
potentially
valuable
for
us
to
track
which
back-end
aerated
API
those
were
going
to
I'm
not
advocating
for
that?
G
B
F
G
C
A
F
So
it
is
yes,
this
is
the
counterintuitive
aspect
of
it
like
you
have
to
think
of
it
like
it's
the
database
table.
If
you
create
a
database
entry
with
a
field
that
doesn't
exist,
then,
basically
that's
going
to
the
air
out
right.
So
if
you
define
this
additional
dimension
and
that
dimension
doesn't
exist
in
your
bag
and
depending
on
your
back-end,
some
backends
are
more
resilient
to
it.
A
F
That's
true,
but
if
we're
going
to
have
this
on
the
request
counter
metric
I
would
prefer
that
we
create
a
good
one
like
a
really
good
one
without
all
of
this
stuff,
that's
in
it
right
now,
and
then
we
just
do
it
properly.
Haven't
we
already
got
that
twice
and
no
the
second
time.
The
second
time
was
basically
almost
exactly
the
same.
The
first
time
it.
G
Up,
which
was
a
press
count,
we
played
up
like
four
yard:
it
just
predated,
sig,
instrumentation
or
predated
their
race.
Yeah
like
this
one,
we
revved
on
a
bunch
like
both
cardinality
explosions
and
we've
changed
labels,
but
I
mean
we
have
a
process
now
for
having
stable
metric
safety
eyes.
So
we
should
practice
those
yeah.
D
D
G
To
get
consensus
so
just
a
real
high
level
I
had
spent
some
time
asking
a
bunch
of
fundamental
questions
like
can
I
actually
figure
out,
what's
scheduled
on
a
cluster.
What
the
capacity
usage
is:
we've
added
node
allocatable,
which
allows
us
to
understand
what
the
capacity
of
the
cluster
is,
and
we
have
a
few
metrics
and
cube
state
metrics
that
show
parts
of
the
life
cycle
I
actually
had
come
in
from
the
perspective
of
reviewing
how
SIG's
were
properly
reflecting.
G
That
found
a
couple
bugs
on
the
way
and
I
got
to
the
point
of
saying
you
know
this
is
actually
a
hard
with
metrics
we
have
today.
What
can
we
improve?
The
proposals?
Kind
of
a
unified
is
attempting
to
look
at
it
from
the
holistic
perspective
of
there's
a
set
of
standard
questions
that
most
admins
would
want
to
ask
about
capacity,
planning
capacity
usage
and
available
capacity
over
the
lifetime
of
their
cluster.
It's
a
little
specific
to
the
mindset
that
someone's
got
an
instrument.
G
Monitoring
solution
is
using
it
with
a
cluster,
which
I
think
is
a
fair
assumption
and
it's
effectively
proposing
trying
to
we've
kind
of
got
a
piecemeal.
Looking
at
it
from
the
proposing
at
least
a
metric,
we
haven't
mentioned
for
the
node
capacity
side
for
our
various
set
of
resources
and
proposing
the
equivalent
for
the
pod
side
in
a
way
that
captures
the
Civic
complexity
of
the
model.
Frederick
had
done
an
investigation
of
whether
we
could
add
more
metrics
to
cube
state
metrics.
That
would
allow
us
to
build
up
the
the
metrics
individually.
G
As
this
other
dimension,
there's
a
bunch
of
smaller
issues
prototyped.
It
basically
I
think
at
the
point
that,
if
there's
no
disagreement,
this
would
be
something
that
I
would
want
to
propose
as
a
cap
and
convert
it
from
the
Google
Doc.
But
I
wanted
to
get
feedback
from
this
group
and
I've
had
some
feedback
from
users
that
I've
tried
to
capture
the
the
thing
but
not
like
quotes
and
from
some
of
the
other
SIG's
that
were
not
in
the
form
of
like
we're.
Gonna
go
endorse
this
publicly.
G
It
was
just
like
hey,
like
we
really
like
this
too.
Let
us
know
if
there's
anything
we
can
do
to
help
some
people
were
interested,
but
I
think
the
feedback
phase
of
using
the
metrics
would
be
something
that
will
go
into
the
cap,
as
in
the
proposal
form.
How
do
we
get
people
to
try
these
and
use
it
and
offer
feedback
about
whether
this
actually
helps
them
understand
capacity,
planning
and
allocation?
D
G
G
G
We
had
to
cube
State
metrics,
that's
an
Oh,
pods
or
sometimes
Oh,
Forex,
pods
or
ever,
and
this
could
be
a
no
pods
that
for
most
people
most
of
the
time
most
admins
would
allow
you
to
get
pod,
namespace
and
node
level
constraints,
which
is
eighty
percent
of
the
questions.
People
are
gonna,
ask
and
would
leave
the
door
open
for
future
or
subdivision
if
we
needed
to
I
think.
G
A
Liked
about
this
compared
to
the
investigation
that
I
did
about
using
the
cubes,
eight
metrics,
metrics
or
potentially
adding
more
groups,
eight
metrics
metrics
to
do
these
calculations
is
actually
as
it
changes
within
the
scheduler.
This
can
be
reflected
in
those
calculations,
and
so
we
don't
actually
need
to
keep
up
with
also
writing
fairly
complex,
bronchiole
queries
or
actually,
even
if
you
have
a
effect,
if
you
have
a
separate,
doesn't
that's
not
prometheus,
it
would
not
work
with
those
right.
B
D
Have
one
more
cap
on
the
agenda
and
Pavel
has
been
waiting
very
patiently,
so
perhaps
let's
table
this
discussion
just
so
he
can
I
haven't
seen
any
links
or
any
proposed
caps.
So
maybe
can
you
speak
about
that
super
briefly,
with
the
three
minutes
that
we
have
left
and
we
can
come
back
to
this
discussion
clean,
it
might
be
good
to
like
actually
submit
that
as
a
Google
Doc
has
a
draft
cap.
So
more
folks
can
see
and
comment
on
it.
B
H
So
just
I
wanted
to
join
the
meeting.
Just
to
give
you
a
heads
up
when
it
comes
to
the
cap.
We
are
drafting
me
together
with
security
team,
and
the
problem
which
we
want
to
address
is
the
problem
of
some
incidents
when
we
had
sensitive
data
like
like
tokens,
keys
or
passwords
leaking
into
logs
of
control
of
manager
or
some
other
control
plane
component.
So
this
was
pointed
out
as
a
one
of
the
high
priority
things
to
be
fixed
in
kubernetes
security
audit,
which
was
done
like
a
year
ago,
or
so.