►
From YouTube: Kubernetes SIG Instrumentation 20190418
Description
Meeting notes: https://docs.google.com/document/d/17emKiwJeqfrCsv0NZ2FtyDbenXGtTNCsDEiLbPa7x7Y/edit#
A
B
So
basically,
the
idea
is
to
introduce
watch
ability
to
metric
api's,
I,
guess
all
three
of
them
so
resource
metrics
API
serve
biometric
server
and
custom
and
external
ApS
as
well.
The
latter
two
were
suggested
by
Marcos,
who
is
not
here,
but
I,
think
it
makes
sense
for
all
of
them
to
be
consistent
so
and
the
reason
why
we
want
to
do
this
is
basically
HPA,
so
in
HPA
is
scaling.
B
So
let
me
go
through
them.
One
is
the
existing
backends
for
custom
and
external
metrics.
Do
not
support
stirring
of
metrics,
so
it
had.
It
would
have
to
be
emulated
anyway,
on
the
whatever
is
serving
the
API
and
I.
Think
it's
a
this
wouldn't
be
the
case
for
research,
my
friends
because
per
metric
server,
we
can
just
notify
everyone.
Subs
like
as
soon
as
the
scraping
is
over.
B
We
can
just
notify
everyone,
so
that
would
be
wouldn't
be
a
problem
for
research
metrics,
but
it
would
be
a
problem
for
custom
and
experiment
ryx
and
for
those
I
think
we
could
just
move
the
Pauline
gloop
from
the
client,
the
server
so
that
the
client
could
just
subscribe
and
get
notifications.
The
server
would
do
polling
of
whatever
backends
it
is
supporting.
B
So
whatever
there
will
be
more
than
one
client,
it
would
be
like
simplified
for
multiple
metrics.
With
that
there
was
a
on
the
instrumentation
least.
There
was
another
problem
mention
that
whenever
there
are
different
metrics
that
require
different
polling
intervals,
we
probably
want
to
offer
them
individually,
so
that
single
watch
on
the.
B
The
metrics
good
result
in
a
multiple
watches
on
being
started
by
the
server,
but
I
think
this
could
be
mitigated
by
grouping
the
metrics
and
pulling
them
in
groups.
So
whatever,
like
all
the
metrics
dots,
are
refreshed
every
mean
it
would
be
pulled
together,
everything
that
is
poled
every
30,
second,
that
this
is
refreshed
every
30
seconds,
both
together
and
and
so
on
and
the
third,
that's.
The
lack
of
implementation,
part
problem,
but
I
think
one
more
argument
for
introducing
this
API.
Even
though
no
implementation
today
supports
it
would
be
it
would.
B
B
B
So
that
was
the
one
problem,
the
other
another
one
was
migration
happening,
because
there
is
a
number
of
implementation
that
exist
today
and
they
they
would
be
essentially
broken
if
we
just
added
that,
but
I
think
this
is.
Since
this
is
an
API
change,
it
would
require
changing
the
API
version
as
well,
so
those
so
the
all
those
implementations
in
order
to
migrate
to
a
new
API
version
would
have
to
implement
the
watch.
C
E
A
One
more
question
I
had:
have
you
done
an
analysis?
What
this
would
mean
for
the
HPA?
Would
there
need
to
be
modifications
to
the
HPI
other
than
supporting
the
watch
API?
Would
they
I'm?
Not
a
hundred
percent?
Sure
anymore,
is
the
the
API
version
is
not
specifiable
in
the
HPA
right,
so
currently,
they're
actually
bound
to
this
API.
That
would
mean
probably
mean
that
HPA
would
need
to
be
bumped
to
v3
right.
B
B
B
F
If
you
introduced
watch
at
the
front
layer,
then
now
it
becomes
possible
to
facilitate
a
different
mode
of
communication
between
you
know
the
metric
server
and
the
thing
that
it
is
getting
metrics
from
so
that
maybe
one
day
you
have
watch
based
thing
all
the
way
through
certain
would
be
a
necessary
requirement
of
that
I
mean
maybe
it
it
doesn't
get
you
there
in
one
step,
but
it
seems
like
if
we
ever
want
to
get
to
a
state
like
that,
and
this
would
have
to
be
done.
Yeah.
B
I
think
this
is
what
so
earlier
I
said
it
would
enable
implementations,
and
by
this
I
mean
further,
improvements
would
be
enabled,
so
subsequent
sub
seconds
would
be
like
subsequent
second
latency
would
be
eventually
enabled.
If
we
do
this,
and
there
are
other
changes
that
we
will
follow
in
the
implementation
of
those
metrics.
If.
F
A
A
A
C
B
C
A
Speak
for
the
driver,
yeah,
anyways,
I,
think
I.
Think
it'd
be
at
least
nice
to
have
an
analysis
of
what
like
the
watch
would
also
mean
in
terms
of
the
capacity
planning
because
for
metrics
server.
We
initially
did
some
very
specific
capacity
planning
and
I'd
like
to
see
what
this,
what
kind
of
impact
this
would
have
on
metric
server
in
terms
of
scalability.
D
B
A
Right,
the
there
was
a
proposal
to
change
metric
server
to
consume
the
Prometheus
format,
anyways,
so
yeah
yeah,
that's
the
one
I
was
referring
to
yeah,
okay,
yes,
absolutely
I
mean
and,
and
then
I
think
there's
so,
but
this
is
probably
further
out.
We
still
should
discuss.
We
probably
don't
have
time
for
this
today,
but
we
should
discuss
further
how
to
get
rid
of
see
advisor
in
the
long
run,
but
yeah
I
I,
don't
know
if
there
are
any
other
opinions
out
there.
A
A
F
A
F
F
So
the
reason
why
it
would
be
good
to
have
a
folder
that
say,
instrumentation
owned
is
because,
once
this
framework
is
in
place
and
we
have
static
analysis,
we
can
basically
do
the
same
thing
that
conformance
tests
do
and
the
thing
that
conformance
tests
do
is.
There
is
a
folder
which
cig
architecture
owns,
and
you
know
in
a
pre
commit
stage.
Static
analysis
is
run
against
things
which
are
tested
for
conformance
and
then
basically,
a
file
is
dipped
which
belongs
to
this.
F
So
what
this
would
do
would
it
would
enable
the
instrumentation,
instead
of
being,
you
know,
scattered
out
through
the
various
kubernetes
codebase?
It
would
actually
give
a
centralized
place
of
ownership
and
also
allow
us
to
automatically
be
tagged
for
PRS
I'm
sure
everybody
wants
this.
This
is
like
I'm
sure
metric
stuff
is
probably
been
driving
everyone
nuts,
because
it's
not
audited
so
yeah.
F
A
F
C
F
It's
perfect
so
so
then,
basically
so
for
now,
I
can
do
that.
I
will
add
an
owner's
file.
So
you
tell
us
metrics
I,
guess
we
can
try
to
figure
out.
If
there's
a
more
proper
place.
We
can
put
it,
but
does
anyone
have
any
objections
to
the
proposal?
I'm
sure
everyone,
it's
been
like
now
several
weeks
for
the
proposal.
Does
anyone
have
any
blocking
concerns
I.
A
Just
have
you
been
able
to
look
at
the
more
recent
changes,
but
what
I
reviewed
when
I
reviewed
looked
pretty
good
overall,
but
yeah
I
I,
don't
have
to
necessarily
block
us
if
there
are
enough
people
agreeing
with
this
I'm
happy
to
vote
to
move
forward.
Otherwise,
I'm
also
happy
to
review
once
born
yeah.
F
So
another
thing
is,
this
thing
will
likely
occur
in
a
number
of
stages,
so
obviously
we're
not
going
to
migrate
the
entire
code
base
right
away.
This
thing
will
be
isolated
to
our
newfound
metrics
directory
and
so
unit
tested
whatever,
and
then
we
can
get
various
components
in
place
like
the
static
analysis
thing
and
then
and
then
we
can
start
migrating
a
binaries
metrics
endpoints
one
at
a
time
uh-huh
and
this
will
mitigate
risk.
C
Just
from
a
like
implement
ability,
standpoint,
the
the
cap
doesn't
have
any
reviewers
listed
or
approvers
list,
there's
a
bunch
of
TDD
stuff,
basically
still
in
here
I'd
say
until
all
those
T
V
DS
get
filled
out.
It
can't
possibly
be
virtual,
so
that
would
be
the
big
blocker.
In
my
opinion,
there's
no.
F
Suit,
so
you
do
not
say
that
the
the
graduating
criteria-
it
almost
doesn't
have
any
graduating
criteria
because
it
will
not
actually
be
graduated
into
the
code
base
right.
It
will
live
in
an
orphan
state
until
we
have
a
migration
cap
until
we
have
static
analysis
cap,
but
I
mean
you
say
that
okay
yeah
I
can
write
that
I
can
write
that
I'm
sure.
F
F
F
A
A
F
F
F
Is
a
bit
coupled
I,
don't
think
that
they
are
Nessus.
At
least
the
person
I
talked
to
said
for
their
Java
client
library
that
the
code
was
quite
coupled
and
he
suspects
it's
the
same
way
for
the
going
labour
library.
So
it
would
take
some
wrangling
to
be
able
to
use
one
without
the
other
and
they
are
working
right
now
on.
F
Breaking
it
apart,
because
in
our
case
it
may
or
may
not
make
sense
to
bring
in
the
metrics
things
and
given
the
fact
that
we
are
using
the
Greens
client
so
anyway,
yeah
that's
just
I
had
I
just
started
a
line
of
communication
with
them.
If
anybody
has
any
questions
or
anything
I'm
happy
to,
they
said
literally
like
400
feet
away
from
me,
so
I
can
just
pop
by
and
ask
them
questions.
E
F
E
A
I
was
gonna,
say
last
time
we
last
time
we
talked
about
introducing
open
census
was
a
difficulty
with
that
is
at
least
for
tracing
and
I
guess
for
some
some
of
the
open
tracing
implementations.
This
is
the
same
same
case
that
there
is
some
agent
that
the
thing
need,
like
the
thing
that
actually
produces,
the
spans
needs
to
talk
to
right
and
that's
kind
of
a
difficult
situation
for
the
couplet.
For
example,
like
can
we?
How
can
we
like
we
significantly.
A
Complicate
the
setup
of
kubernetes:
if
we
force
everyone
to
have
this
agent,
there
I
think
I'm,
not
sure,
that's
a
that's
a
thing
that
we
can
do.
I,
I,
know
or
I've
heard
of
at
some
point
that
possibly
these
functionalities
could
be
embedded
into
the
application
itself.
I,
don't
know
how
feasible
that
X
of
a
thing
that
actually
is,
but
then
again
that
that
would
influence
what
Han
mentioned
about
binary
size.
A
I
think
there
are
various
various
things
that
we
may
need
to
think
through
a
little
bit
more
I'm
generally
super
I
think
I'm,
like
I'm,
a
huge
fan
of
especially
having
the
correlation
of
multiple
signals
like
being
able
to
make
use
of,
like
exemplars
and
metrics,
to
be
able
to
tell
a
trace
sample
from
a
histogram
bucket
or
something.
That's
absolutely
amazing.
I
totally
want
that.
Yeah
like
as
a
it's,
mostly
an
organizational
problem,
I
think
introducing
it
into
communities
and,
of
course,
not
breaking
everything
we
already
have
it.
E
F
E
Think
in
terms
of
like
where
you
would
actually.
Yes,
there
is
sort
of
inject
like
something
that
needs
to
an
agent
process.
It
can
be
run
inside
the
process
itself.
I
know
like
senses
these
babies
for
this,
so
you
don't
actually
connect
you
don't
have
to
run
anything
else,
they're
just
available
Italy,
some
point
I.
E
Think
you
would
be
able
to
I
think
it
seems
to
me
that
there
will
be
a
way
to
start
adding
this
through
just
the
tracing
side,
not
actually
bundling
anything
else,
and
then,
if
someone
and
then
provide
a
way
to
configure
where
they
get
sent
to,
and
then
you
have
sort
of
the
best
of
both
worlds.
People
that
wanted
to
opt
in
could
plug
in
you
know
a
Jaeger
or
for
stackdriver
or
whatever,
and
see
traces,
and
then
people
that
didn't
want
that.
Well,
if
you
don't
have
anything
there,
it
just
sits
there.
Yeah.
A
I
think,
like
the
reason
for
the
for
the
agent
is
so
that
all
this
logic
of
of
supporting,
Yeager
and
Zipkin
and
all
those
other
projects
is
so
that
it
doesn't
have
to
live
in
your
application
right,
but
then
pulling
that
then
revert
reverses
that
and
it's
kind
of
a
Divya
call
to
make
I
mean
there's
some.
Some
like
this,
essentially
open
census
does
standardize.
This
is
in
some
ayat
by
standardizing
the
communication
to
this
agent
right.
F
A
D
E
Yeah
and
it's
yeah,
the
faculty
would
run
the
agent
in
the
master
pod
or
whatever
you
would
configure
all
the
trade
like.
You
would
configure
all
the
tracers
to
report
to
that
pop.
To
that
you
would
make
it
available
at
you
know
whatever
name
and
then
I
think
the
only
things
would
be.
Maybe
a
little
weird
is
like.
How
would
you
there's
a
couple
issue?
The
agents
can
point
to
other
agents,
so
you
could
have
that
agent
and
then
you
could
have
a
user
actually
say
like
well.
E
I
want
to
direct
other
traffic,
this
agent,
where
I've
actually
put
in
my
exporters,
but
there's
still
keep
in
mind
like
some
of
these
terms,
might
change
and
like
right
now
we're
guaranteeing
support
current
open
sense
of
stuff
for
two
years.
There'll
be
a
bridge
between
that
and
the
new
project,
so
I
would
expect
that
there
will
be
some
changes
happening
to
the
agent
and
the
open
sense
of
certain
stuff
as
well.
Okay,.
A
F
E
Project
supported
yeah
sure,
but
if
you
so,
my
point
is,
if
you
said:
ok,
we'll
do
census
today,
then,
as
of
like
the
end
of
this
year,
I
think
we're
planning
to
sunset
in
November,
tracing
and
open
tracing
an
open
census.
There
will
be
2
years
from
that
date
like
we
will
support
the
bridges
and
to
the
new
project.
So
if,
if
this
isn't
something
that
we
need,
you
all
need
to
do
like
immediately,
then
it
might
be
good
to
give
it
a
couple
months
and
because
I
believe
we're
playing
work.
E
A
Sorry,
everyone
we're
already
5
minutes
over
I
think
this
is
a
very
interesting
discussion
and
I
think
we
should
continue
next
time.
Let's
put
this
one,
just
like
we
did
with
the
watch
API
and
last
time.
Let's
put
this
one
on
the
top
for
next
time,
and
then
we
can
continue
that
discussion.
Maybe
we'll
even
have
some
new
some
new
information
in
this
area.
Already:
okay,
thanks
everyone
for
attending
and
see
you
in
two
weeks
and
happy
locally
local
time,
all
right,
ow.