►
From YouTube: SIG - Performance and scale 2023-06-01
Description
Meeting Notes:
https://docs.google.com/document/d/1d_b2o05FfBG37VwlC2Z1ZArnT9-_AEJoQTe7iKaQZ6I/edit#heading=h.tybh
A
All
right
so
pretty
much
just
wanted
to
review
so
we've
got,
did
you
so
have
you.
A
B
Something
to
get
here
so
I
I
wanted
to
take
some
time
to
if
I've
not
uploaded
data
to
my
personal
website.
So
I
was
hoping
that
we
are
in
a
good
place
now
to
automate
all
of
that
in
this
repository.
So
just
wanted
to
review
that
and
spend
some
time
today
discussing
what
things
are
needed
in
order
to
get
that.
A
Yeah
agreed,
this
is
just
the
results
you
generated
manually
right.
B
A
A
B
So
I
generated
these
results
using
the
script
in
PR
2787,
that's
on
the
agent
Network,
okay,
so.
B
Yeah,
so
that
script
just
calls
the
the
perf
report
Creator
and
it
has
all
the
metrics
that
we
care
about
notice
online,
22
and
onwards.
The
graphing
commands
are
commented
out.
I've
done
that
purposefully,
because
it
looks
like
the
suggested
workflow
is
that
we
scrape
the
data
and
gather
it
into
weekly
report
and
push
it
into
the
Benchmark
repository
from
there.
We
will
have
a
post
submit
job
that
will
run
this
weekly
graph
and
add
things
like
add
the
graph
to
to
that
repository.
B
That's
the
job
to
scrape
data.
B
Yeah
we
can
I've
commented
on
this
VR
that
once
the
script
script
dot
sh
merges,
we
can
use
that
here.
A
Okay,
great
okay,
so
we
want
this
first,
okay,
so
let's
and
Brian
and
take
a
look
then
see
if
we
can
get
this
one
in
and
then
we'll
follow
up
and
we'll
change
this
here.
Good,
okay,.
C
A
Good
this
one
yeah,
this
one
I
mean
we're
just
going
to
take
so
anyway,
we
got
our
plan,
we
got
we're
gonna
go
here
and
then
we're
gonna
go
change
here
and
then
we
want
to
get
this
merch
and
then
I
think
we
have
our
as
then
do
we
have
our
job.
So
we
have
so
posting
a
job
with
plot
graph,
probably
about
data,
so
this
generates
the
job
and
automatically
so
this
gets
the
job
and
it
automatically
posts
the
data
in
our
repo
right.
So.
B
From
there
we
need
to
plot
it
so
plotting
we,
we
need
another.
C
B
In
the
new
repo
to
plot
the
data-
okay,
that
is
called
the
weekly
plot
sub
command,
and
that
will
be
so.
There
is
some
discussion
needed
on
this
right.
Currently,
the
weekly
plot,
sub
command
will
plot
for
all
the
directories
in
weekly
report
directory.
Right
so
let's
say
the
density
job
has
one
year
worth
of
data.
B
It
will
plot
all
of
those
one
years
worth
of
data,
ideally
like
we
need
to
decide
on
a
number,
let's
say
last
three
months
or
last
six
months,
or
something
like
that
to
make
sure
that
the
plot
is
within
reasonable
time
limit
and
the
the
convention
I
was
using
was
to
eight
weeks
in
in
my
personal
plots
in
my
personal
website.
So
just
wanted
to
have
that
discussion.
What
would
be
the
right
number
of
the?
What
would
be
the
right
timeline
of
the
data
we?
We
should
be
looking
at
three
months.
A
Yeah
that
sounds
immense
I
guess
I
mean
that
the
other
test
repo
has
like
does
like
a
year
or
two
or
something
it's
it's
way
longer,
but
we
can
I
mean
as
far
as
I'm
concerned
like
so
we
want
so
I.
Guess
we
I
guess
it
maybe
depends
how
we
want
to
look
at.
So
if
we
post
like
a
a
graph,
that's
in
the
readme
there
I
mean
I,
don't
I
mean
we
could
do
we
probably
want
to
do
cross
releases
right
so
six
months?
A
A
Okay,
wait
hold
on
my
masks
are
eight
months
because
we're
doing
three
releases
here
right.
So
it's
me
every
four
months
is
a
release.
A
B
Okay,
yeah,
okay,
so
that
I
will
take
that
as
an
action
item.
I
will
have
that
in
like
I'll
create
a
PR
for
that
soon
and
then
I
think
four,
two
seven,
eight
seven
you
so
the
pr
for
setting
up
owner
ref
owners
just
for
the
perf
report
directory
is
has
merged.
So
you
should
have
the
rights
to.
A
Yeah
I
forgot
where
that
was
so.
This
is
a
bit
right,
yeah!
A
A
A
You'll
need
to
call
in
the
calvary.
So
all
right,
I'll
take
a
look
at
this
and
we
can
all
right
and-
and
we
can't
get
this
merged
quickly
and
then
go
over
to
Daniel's,
PR
and
just
make
sure
everything's
good
there
and
make
the
change
to
support
the
scripts
Okay
cool.
So
we're
close
here.
This
is
good,
so
we
just
a
few
a
few
little
things
and
and
and
I
think
we're
gonna
be
yeah
like
the
next
week
or
two.
It
sounds
like
we're.
Gonna.
Have
these
plots
cool.
A
B
Okay,
so
one
more
thing:
I
wanted
to
I
have
an
accent
item,
but
I
have
not
gotten
a
chance
to
get
there
is
we
will
so
once
all
of
these
things
are
in
place?
The
next
thing
you'd
need
is
some
kind
of
static
website
to
you
know
render
this
automatically
either
we
can
do
that
in
CI
performance,
Benchmark
repository
or
in
a
separate,
separate,
GitHub
page
rendering
repository
I
just
need
to
look
at
the
logistics.
So
that's
another
action
item
for
me.
B
A
Yeah,
okay,
yeah
we'll
do
that.
One
last
I
think
that's!
This
is
a
good
one
to
have,
but
yeah
we
get
these
all
this
stuff
in
place.
Then
then
we
can
have
a
discussion.
A
Cool
okay
sounds
good
away
all
right!
This
is
all
good
progress.
Okay,
I
think,
then,
let's
go
to
okay,
sixth
scale
for
V1,
so
we
are
I
mean
we're
gonna
touch
on
some
of
the
some
of
the
similar
Concepts
here,
so
we
basically
want
to
have
right.
So
this
is,
we
just
went
over
the
mall,
so
we
we
have
okay.
This
is
done.
A
I,
think
right,
yeah,
I
think
this
is
done
yeah
we
set
up
our
honors
final
and
then
these
are
our
last
three
Okay,
so
we've
covered
it
already:
okay
and
then
exporting
the
artifacts
folder.
I
think
this
is
the
only
so
2773
we
spoke
about
yeah
this
one.
A
Okay,
this
one
is:
oh
I
should
be
able
to
merge
this
wait,
so
we
I
fixed
the
owner's
folder
right
I
should
be
able
to
merge
this
too.
Okay.
Let
me
take
a
look
at
this,
because
I
think
this
is
this
is
also
done.
We
just
need
to
I
just
didn't.
Have
the
approval
before
I
think
that's
what
it
was
right,
I,
don't
know
if
I
had
approval
on
it,
I
don't
know
if
I
did
no
I
didn't
but
I
think
I,
remember.
That
was
a
problem
somewhere
so
I.
This
looks
fine
to
me.
A
I
mean
we
already
discussed
this
at
length,
and
this
makes
sense
so
I'll
just
take
a
look
and
make
sure
this
I
hate,
everything's,
everything's
good,
with
the
approval
and
probably
the
owner's
file
and
I'll
put
an
approve
on
this,
and
it's
fine,
okay
good.
So
this
is:
where
did
my
tab?
Go?
Okay,
okay,
I
mean
that's
all
we
had
I
think
it's
just
those.
So
this
gives
us
this
gives
us
the
density
job
right
gets
in
the
artifacts
folder.
Okay.
B
Okay
I
mean
that
PR
will
just
make
sure
that
we
don't
have
to
work
on
regex
whatever
so.
B
You
should
be
able
to
deprecate
yeah
one
thing:
I
wanted
to
bring
up
that.
I
don't
have
on
the
agenda
is
so
in
some
of
the
downstream
scale.
Testing
I
I
ran
into
cases
where
keyword
creates.
Sorry
specifically,
word
Handler
creates
some
amount
of
watch
calls
to
Secrets
API
Handler,
yes,
okay,
the
watch
call
is
not
very
expensive
because
it
specifically
filters
for
the
owner
VMI,
so
the
events
will
be
limited
to
one
Secret,
but
I
wanted
to
raise
this
here.
B
I
am
not
sure
why
that
watch
call
for
a
specific
secret
is
required.
So,
for
example,
if
there
is
a
lot
of
churn
and
if,
if
users
are
creating
and
deleting
vmis
with
secrets
on
a
regular
interval,
then
you
will
see
a
regular
flurry
of
watch
calls
since
the
vertical
is
wired
to
fire
on
on
a
VMI
right.
B
So
I
was
wondering
if
we
can
identify
why
this
is
needed
and
somehow
remove
this
by
by
one
Global
watch.
So
let's
say
so
idea
is
that
if
word
Handler
can
have
a
watch
for
all
secrets
right
and
as
soon
as
a
new
secret
comes
up,
it
will
get
that
event.
B
A
I'm
wondering
I'm
wondering
if
we,
if
this
is
Upstream.
Actually
we
see
you
you
you
see
mentioned
this
at
the
time.
I
wonder
if
this
actually
exists.
Upstream,
because
there
is
a
I
know,
we
had
a
change
to
get
Downstream
where
we
have
where
we
are
watching.
The
secrets
for
to
make
changes
to
them
is
that
is
that
what.
C
It
is
I,
don't
think,
that's
Upstream,
that's
so
that
secret
was
added
by
me.
Two
years
ago,
Downstream
yeah.
A
C
Yeah,
that
was
just
the
the
downstream
requirement
where,
where
we
have
to
constantly
three
secrets,
so
that
so
yeah,
so
that's
for
them.
B
Got
it
okay,
interesting,
okay,
I
did
not
know
that.
Maybe
then,
and.
C
But
like
do
you
think,
do
you
think
like
even
for
Downstream
you're?
Do
you
think
your
suggestions
could
make
sense,
let's
say
having
a
global
secret
somewhere
else
instead
of
having
or
each
others
yeah.
C
We
because
the
reason
why
this
this
controller-
it's
been
very
Handler,
so
that
it's
it's
for
the
it's
for
the
word
Handler
to
issue
this
RPC
call
to
this
verb
to
this
word
launch
report.
So
that's.
B
Got
it
makes
sense
yeah,
let's
discuss
this
in
separate,
call
I
think
if
we
find
a
way
to
eliminate
n
secret
watches
with
one
watch
feeded
by
events,
then
it
might
be
more
efficient,
but
we
can
discuss
more.
A
A
Okay,
so
I
think
let's
go
to
this
next.
This
is,
let
me
just
put
the
link
in
there.
A
So
this
is
what
we
discussed
with
Upstream
kubernetes
six
scale.
I
just
took
a
crack
at
putting
together
a
little.
Our
few
thoughts
on
this
I
think
there's
a
there's.
A
There's
still
a
lot
to
do
here
and
I
think
sort
of
two
things
that
I
was
trying
to
get
out
of
this,
and
one
of
them
is
to
just
enumerate
the
number
of
the
the
different
metrics
that
are
existing
today
and
I,
and
you
can
see
like
with
it
there's
a
common
theme
for
a
lot
of
you,
so
they're
very
focused
on
they
get
end-to-end
latency
or
like
things
that
are
in
the
Aggregates.
A
We
look
at
them
and
we
see
like
a
lot
of
things,
are:
let's
see
what
else
duration
of
time
Singapore
first
time
to
the
Pod
starting
to
run
right.
So
this
is
another
another
aggregate.
It's
like
we
don't
have
that
that
breakdown
of
all
the
different
steps
that
go
into
it
right-
and
this
is
this-
is
really
the
point
that
we're
making
is
that
there's
value
in
actually
knowing
information
about
what
went
into
each
of
those
steps
and
and
in
the
sort
of
the
ideas.
A
There's
two
perspectives
here
and
the
way
that
the
kubernetes
Upstream
six
scale
like
it's.
My
understanding
is
that
they
want
to
look
at
this.
They
want
to
say,
okay,
this
is
what
they
care
about
right.
We're
gonna
We,
want
to
know
how
long
it
takes
the
Pod
to
run
right
and
we
want
to
quickly
tell
like
we
made
this
change.
It
got
faster,
got
slower,
makes
total
sense,
and-
and
so
our
perspective
is
that
okay,
we
also
want
to
see.
A
We
also
want
that
too,
but
we
also
want
to
see
the
the
individual
phase
transitions
right,
all
the
different
things
that
happen
underneath,
and
so
it's
just
a
more
granular
view
right.
We
could
look
at
a
code
and
say
a
piece
of
code
and
and
say:
okay,
this
change
happens
and
okay
got
faster
slower,
but
we
could
also
see
that
okay,
it
got
where
did
it
get
faster
or
slower?
A
And
so
there's
there's
value
in
that
and
that's
something
that
that
we,
you
can
see
just
by
the
existing
metrics,
that
that
there
there's
something
we
can
improve
on,
and
so
all
I
wanted
to
do
is
like
there's
a
lot
of
ways.
A
We
can
look
at
approach
to
actually
doing
this,
but
all
I
did
here
is
just
looked
at
this
in
terms
of
one
API,
and
this
is
just
with
volumes
or
two
apis,
PVS
and
PVCs,
but
specifically
related
to
volumes
and
went
down
a
few
thoughts
as
to
like
that's
how
we
could
do
this
and-
and
so
the
thing
that
was
interesting
when
I
was
thinking
about
this
is
phase
transitions.
When
we
do
tank
Hubert
right,
we
look
at
VM
vmis.
They
only
go
one
direction
right.
A
We
only
go
from
from
pending
all
the
way
to
to
running
and
eventually
to
succeed
right,
it's
a
it's.
It's
like
a
finite
State
machine.
We
go
from
one
end
to
another
with
things
like
EVS.
These
things
are
bi-directional
and
so
that
you
can
go
from
something
being
used
to
not
being
used
and
the
object
could
still
exist
and
then
that's
and
that's
that's
fine.
That's
that's
normal
way
of
things.
The
life
cycle
of
the
object,
and
so
this
is
a
challenge
and
and
to
sort
of
having
to
deal
with.
A
So
we
need
to
know
what
each
phase
has
to
do
with
whatever
it
is
that
we're
doing,
and
so
the
thought
was,
is
that
we
could
take
these
bi-directional
phase
transition
objects
and
we
can
tie
them
to
one
that's
similar
to
like
a
BMI
or
like
pause,
something
that's
unidirectional,
and
so
the
idea
was
that
we
could
take
something
like
we
could
take
something
like
PV
or
PVC,
and
we
could
even
tie
them.
A
We
type
PVS
to
PVCs,
we
could
type
PVS
and
PVCs
to
pods
and
the
way
we
do
that
is
is
maybe
we
have
like
a
metric.
The
way
we
post
the
metric
is
that
it
would
be
in
in
relation
to
a
pod
or
something
like
in
you
know,
a
specific
pod,
and
we
have
to
see
how
that
would
work
in
the
in
terms
of
like
how
the
metrics
are
report.
A
If
that
makes
sense,
but
in
my
mind
it
made
at
least
from
the
the
way
we'd
post
status
and
when
we
talk
about
posting
status
for
phase
transition
times.
It
was
kind
of
interesting
when
thinking
about
it.
This
way,
because
it
kind
of
comes
out
like
this.
This
picture
right
here,
where
this
looks
very
similar
today,
what
we
have
on
vmis,
but
we
have
for
for
PVCs
and
PVS,
and
we
just
take
you
know
whenever
these
different
PVS
or
PVCs
go
through
these
phases.
We
post
the
timestamp.
A
A
So
from
like
from
that
perspective,
it
made
a
lot
of
sense,
but
I
don't
know
we
need
to
look
at
this
in
in
terms
of
other.
You
know
how
this
works
in
terms
of
Prometheus.
If
this
makes
sense
how
well
that
metrics,
the
number
of
metrics
will
scale
and
and
even
just
like
the
data
that
we
Post
in
the
Pod
I
think
there's
there
could
be
other
Corner
cases
here
as
well.
So
that
was
just
my
thoughts
on
like
how
we
could
do
this,
and
maybe
it
could
be.
A
We
don't
do
with
pods
or
some
other
way,
but
those
are
just
the
initial
thoughts
and
an
approach.
C
Yeah
so
I
was
gonna
say
this
makes
sense
in
terms
of
troubleshooting
away.
So
that
means
that
we
can
just
look
at
a
part,
see
and
see
See
when,
like
when
the
volume
gets
ready,
but
I
think
from
a
benchmarking
perspective.
C
C
So
so
like
I
would
say
from
a
from
a
benchmark.
Perspective
like
Prometheus
metrics
would
be
like
would
be
preferred
options,
but
I
do
see
the
value
of
like
adding
these
in
the
plus
pack
as
well.
A
Yeah
I
agree
Shane
I,
so
my
perspective
on
this
was
that
we
would
do
both
like
I
I
guess.
What
I'm
showing
here
is
like
since
I
haven't
I,
haven't
thought
through
fully
the
Prometheus
side
of
things
and
sort
of
how
it
would
look
yet.
But
my
when
I'm
saying
like
when
you
know
when
posting
this
stuff
on
status,
my
expectation
is
the
equivalent
is
also
in
Prometheus.
A
The
only
thing
that
yeah,
like
I,
said
I
haven't
I
just
haven't
hosted
what
that
would
look
like
yet
I'm,
pretty
famous,
but
it
would
be
both
because
for
the
two
reasons
you
mentioned,
like
everyone
uses
Prometheus
right,
this
is
a
great
tool
and,
and
that
should
be
our
primary
target.
The
value
of
this
is
is
like
the
the
what's
this.
A
That's
that
part
you
just
mentioned
you
can
crack
open
upon
and
look
at
it
and
see,
and-
and
so
it
gets
all
those
Corner
cases
where,
like
oh
I,
don't
use
Prometheus,
which
I
mean
in
production
would
be
weird
if
you
don't.
But
if
you
talk
about
like
tooling
and
developer
tooling,
like
you
think
about
testing,
you
would
be
required
to
stand
up
Prometheus
in
order
to
do
your
testing.
In
some
cases,
people
don't
do
that
and
we
we
even
had
cases
where
we
don't
do
that.
A
So
it's
fine
like
if
you
have,
if
you
want
to
have
tooling
around
the
stuff,
so
it's
this
is
sort
of
meant
to
get
the
corner
cases,
but
the
reason
I'm
presenting
it
this
way
is
that
is
that
it
really
makes
a
lot
of
sense
when
I
sort
of
communicated
in
terms
of
PODS
and
and
so
that's
what
I
was
hoping
that
we
could
do
as
sort
of
our
our
our
our
data
packet
is
sort
of
is
communicate.
This
turns
in
terms
of
a
pod
in
Prometheus
and
on
the
object.
B
C
B
Yes,
that
part
makes
sense
Ryan,
but
do
you
see
you
see
that
it
will
be
like
a
little
bit
challenging
getting
this
API
change
approved
in
in
kubernetes?
The
reason
why
I
wonder
about
that
is
the
Upstream
API
convention.
Documentation
specifically
says
that
for
making
apis
evolve
nicely
over
time,
they
want
to
drive
away
from
phase
and
phase
related
things
in
status.
B
That
keeping
that
in
mind
and
then
tying
the
transition
times
of
volume
API
into
a
pod,
will
will
kind
of
bleed
some
of
the
implementation
issues
of
one
API
into
another.
B
So
because
of
those
two
reasons
I
feel
like
it
would
be
much
harder
to
except
like
it
will
be
much
harder
for
us
to
get
this
change
accepted
in
the
status
API.
So
I'm
curious.
What
your
thoughts
are
regarding
that.
A
Well,
I
I,
don't
know,
I
mean
I
I.
Think,
like
the
the
to
me,
like
my
my
thought,
is
that
I
I,
like
it
doesn't
matter
to
me
where
this
is
I,
it
doesn't
doesn't
make
really
a
difference
in,
in
my
mind,
I
the
kind
of
the
way
that
sort
of
the
thing
that
is
most
valuable
to
me
is
I.
A
Look
at
this
as
I
I
can
see
very
clearly
what
is
happening
with
you
know
my
attachments
to
my
workload
and
that,
like
that's,
where
that's
what's
really
interesting,
maybe
that
doesn't
make
sense
in
terms
of
the
way
that
the
apis
is
structured
in
the
way
the
community
approaches
it
I,
don't
know
and
and
like
well,
that's
kind
of
what
I
wanted
to
find
out.
It's
like
we're
at
the
idea
is
we're
adding
fields
and
kind
of
the
way.
A
I
was
thinking
about
this
about,
like
all
the
stuff
that
kubernetes
did
on
service
side
apply,
and
all
that,
like
all
the
data
that
was
added
to
to
objects
like
there's
still
stuff,
that
is
being
added
to
objects,
you
know
maybe
there's
a
way
we
can
do
the
same.
Maybe
maybe
it
doesn't
go
into
status?
Maybe
it
goes
somewhere
else,
I
I,
don't
know,
I
mean
I.
A
Think
like
I
mean
I'm
pretty
flexible
to
like
where
it
goes,
and
just
mostly
what
I,
at
least
from
my
perspective,
is
that
we
just
want
to
have
this
somewhere,
but
I,
don't
know
I
mean
it
could
be
out.
Maybe
this
ends
up
on
the
PVC.
Maybe
it
ends
up
in
annotations.
Maybe
it
ends
up
somewhere
else.
A
I
I,
don't
know,
I
think
what
at
least
my
thought
was
is
that
we
would
have
an
opinion
where
we
think
it
should
go,
and
then
you
know,
let's,
let's
talk
with
the
stakeholders
and
let's
I
mean
I
because
I
don't
know
all
the
reasons
like
I
don't
know
all
the
reasons
why
we
shouldn't
do
and
Status
so
I
mean
that's
fine
move
people
tell
us
and
then
we
can
find
you
know
we
can
work
around
that.
I.
Think
that
that's
at
least
what
my
plan
was
is
that
we
can.
A
B
Yeah
makes
sense
so
personally,
my
opinion
is
that
I
would
much
rather
see
it
in
Prometheus
than
in
in
the
status
object,
because
the
way
all
of
these
controllers
are
wired
up,
it's
hard
to
implement
this
in
in
a
consistent,
correct
way
across
all
CSI
implementations,
and
things
like
that
right,
so
I
was
more
thinking
of
putting
it
in
Prometheus,
but
I
have
not
spent
a
lot
of
time
thinking
it
through.
So
okay.
A
A
That
excuse
me
that
piece
of
code
now
we're
doing
it
and
basically
what
what
we
would
need
and
and
like
we
have
to
put
whatever
it's
easy
right.
We
can
just
have
a
right
after
we
find
that
that
we've
gone
from
you
know
gone
into
pending
or
something
right.
We
take
a
timestamp
and
we
post
it
to
Corinthians
right.
So
not
that's
not
too
hard
to
visualize.
A
The
part
that
gets
tricky
here,
which
is
what
you're
getting
at,
is
that
we
need
to
take
this
and
this
the
field
and
value,
and
we
need
to
pass
it
along
somewhere,
and
this
is
where
yeah
I
I
totally
understand
like
like
you
need
to
be.
Have
some
changes
in
the
CSI,
but
we're
we're
going
to
be
making
changes
anyway
to
add
the
metric,
so
The
Next
Step
would
be
it
would.
It
would
be
more
code
right.
A
We
have
to
have
some
sort
of
interface
here
where
we
can
fill
out
these
values,
and
maybe
there's
not.
Maybe
that's
not
an
appetite
for
doing
that
and
in
which
case
we
still
have
Prometheus,
and
that's
that's
fine,
but
all
I'm
saying
is
we're
going
to
be
we're
going
to
be
in
the
same
exact
spot.
A
Making
a
code,
change
and
sort
of
adding
this
piece
where
we
pass
it
along
somewhere
is,
would
be
in
the
exact
same
spot
would
be
some
a
few
additional
lines.
The
challenges
that
interface
right,
like
that
like
we
need
to
have
some
Library
I
guess
you
could
say
that
is
able
to
understand
the
the
data
structure
and
where
it
needs
to
go
that
that
would
be
the
the
challenge.
B
Yeah
makes
sense,
so
I
think
I
just
quickly
looked
at
the
PVC
metrics
and
there
is
just
counter
metrics
I
put
the
link
on
in
the
chat.
The
metrics
seem
to
be
limited
to
just
counters,
so
total
number
of
bound
and
Unbound
metrics.
B
There
is
no
metrics
of
when
first,
when
the
PVC
was
first
observed
to
be
in
pending
or
in
binding
or
bound
state.
So
yeah
I
think
that
that's
where
this
belongs
and.
A
So,
like
yeah
band
number
by
right,
so
here
we
go
so
bound
number
by
name.
Something
is
calling
so
P
yeah
PV.
Sorry!
So
does
this
look
at
collector
PVC?
Oh,
it
does
a
list.
Oh
so
someone
is
calling
PV
collected
and
it
actually
lists
them
out
and
then
interesting
so
they're,
not
even
counting
whenever
they
do
a
phase
change.
B
So
dpv
and
PVC
controllers
that
that
manage
these
things
seem
to
be
very
archaic
and,
like
they
cue
all
the
PVCs
in
the
work
queue
every
second
and
then
work
off
them
from
from
the
work
queue.
So
it's
not
like
a
normal
controller
which
would
like
wait
on
the
watch.
Events
and
you
know,
respond
and
reconcile
like
that,
so
that
that's
why
I
feel
like
it
would
be
little
bit
difficult
to
you
know,
make
this
work
in
a
very
crisp
man.
A
A
A
So
if
it's
clean
bound,
we
I
mean
you
could
do
it
right
here.
I
mean
I,
okay!
Well,
actually,
no,
you
can't,
because
this
is
not
I.
Guess
you
have
to
measure.
The
Delta
is
what
you
have
to
do:
yeah,
I
I,
don't
know
I
mean
I!
Think
like
see,
I
haven't
I
haven't
dug
into
this
yet,
but
so
look
looking
at
this,
like
yeah
I,
mean
seeing
that
their
listing
is
not
encouraging,
because
then
then
I'm
then
I'm
guessing
that
the
CSI
is
not
responsible
for
setting
the
phases.
B
If
you
search
for
mind
mentored,
there
is
it's
in
this
line,
let
me
say,
line
one
zero.
Seven,
eight.
B
A
C
A
So
this
would
be
where
so.
It
would
be
the
same
thing
where
actually
right
here.
If
there
was
no
error,
so
this
one,
so
this
line
would
be
send
out
timestamp
for
the
phase
transition
and
then
that
second
line
would
be
the
the
whatever
the
interface
would
be
for
getting
it
to
the
pod.
Assuming
upon
us,
where
we
want
to
put
this.
C
A
I
want
to
play
around
with
this.
It's
pretty
cool
I
didn't
I
had
not
dug
into
this
yet,
but
this
looks
this
looks.
This
looks
good.
Okay,
let's
have
to
see
how
this
goes
all
right,
cool
so
anyway,
I
I,
think
coming
back
to
this
I
like
I,
was
saying
I,
I'm,
really
flexible
on
how
we
want
to
do
this.
It's
Mo,
it's
mostly
about
it's
mostly
about
getting
these
time
stamps
to
the
places
that
people
can
consume
them.
So
I
I
mean
to
me
in
my
mind,
I
think
I
I.
A
A
We
need
to
figure
that
out,
and
so,
at
least
for
me
this
was
the
easiest
way.
I
could
explain
it
and
visualize
it,
but
it
would
make
sense
in
the
pot,
but
I
will
yeah
we'll
have
to
play
around
with
that
idea.
Okay,
go.
B
B
A
Yeah
makes
sense,
yeah,
yeah,
I,
think
I
haven't
gone
through
this.
That
exercise
yet
of
doing
the
because
I
I
don't
even
have
a
reason
for
like
why,
like
for
the
reason
for
adding
these
phase
transition
times,
I
haven't
done
that
yet
I
kind
of
skipped
ahead
to
write
thoughts
about
how
we
a
little
thoughts
about
how
we
can
do
this
so
I
agree.
We
need
to.
We
need
that
as
well
like
we
need
to
explain
like
this.
Is
here's
the
two
targeted
things
that
we
like
here's?
A
Why
that
we
think
they're
useful
for
users,
yeah
I
I,
don't
have
that
yeah?
We
I
can
add
that
okay
yep
yeah,
and
that
was
pretty
much
it
I
think.
The
last
thing
is
I've
got
some
PVC
phases.
These
are
just
the
things
right
that
we
care
about.
We
want
to
see
going
so
pv's
got
a
few
of
these
that
goes
through
and
and
then
I've
never
seen
lost,
I
didn't
even
know
that
existed,
but
the
yeah
I
mean
basically
for
this
one.
It's
good
like
we're.
A
A
Yeah,
okay,
well
I'll.
A
Something
that
I'm
hoping
we
can
talk
about
next
Thursday,
with
with
the
kubernetes
skill,
might
get
a
little
further
along
yeah
go
ahead
away.
What
else
did
you.
B
Yeah
one
more
question
I
had
is
so
I
think
we
all
agree
that,
regardless
of
the
status
discussion,
the
Improvement
on
the
existing
Prometheus
metric,
we
need
to
expose
more
metrics
on
these
timestamp.
Do
you
envision
that
independent
of
this
proposal,
or
do
you
want
to
handle
it
here
as
well.
B
So
the
so
I
mentioned
write.
The
metrics
right
now
is
only
a
counter
of
of
things,
and
we
just
had
a
discussion
that
we
should
have
it
in
both
places
in
Prometheus,
as
well
as
in
in
the
status,
so
the
other
part
part
two
of
having
these
metrics
reflected
in
Prometheus.
Do
you
envision
that
as
part
of
this
work,
or
is
that
going
to
be
a
separate
document?
B
A
So,
like
so
you're
talking
about
like
having
these
time
stamps
in
Prometheus
that
would
be
in
here,
please
yeah,
yeah,
that
no,
that
was
going
to
be
one
of
the
this
is
another
I
I.
Don't
have
I
have
not
mentioned
Prometheus
I,
don't
think
at
all,
but
that
is
actually
I.
I
would
think
the
way
I
look
at
this
is
Prometheus
is
like
90
8
of
the
the
use
cases
that
people
are
gonna
have.
A
This
is
the
the
last
two
percent
the
reason
the
reason
I
I,
just
like
I,
was
saying
before
the
reason
I
did
not
mention
it
was,
is
as
a
visual
I
like
not
mentioning
yet
I
will.
But
this
is
a
visual
because
of
this
sort
of
the
way
that
we
need
to
figure
out
because
of
this
problem
like
how
do
we
figure
out?
How
do
we
tie
these
together
like
how
we
can't
take
a
we
need
to
figure.
A
We
can't
take
a
bi-directional
phase
transition
object
and
we
need
to
tie
it
to
something,
and
so
this
is
just
an
easy
way
for
me
to
visualize
exactly
how
this
would
look,
whereas
if
we
were
just
if
I'm
just
gonna
list
out
the
face
resistance
of
the
objects,
I
don't
think
it
makes
sense.
This
is
just
sort
of
a
way
for
for
you,
as
the
user
to
like
consume
this
like
here,
like
you.
A
Can
when
I
tell
you
it's
the
status,
you
can
visual
doing
a
cube,
CDL
again
on
object
right
and
seeing
this
here
and
then
kind
of
you
can
see
how
like
okay,
here's
where
I
can
here's
what
this
means
right,
but
I
I,
totally
agree,
though
there
there's
going
to
be
Prometheus
component
to
this,
it's
going
to
the
majority
of
the
focus,
but
this
is
just
temporary.
This
is
just
sort
of
how
we're
doing
approaching.
B
No
I
think
now
now
this
makes
sense
so
by
by
bi-directional,
what
you
mean
is
that
The
Binding
happens
both
in
TV
as
well
as
PVC,
right
and
and
when,
when
this
happens
from
both
the
sides,
that's
that's
when
it
is
actually
bound
so
to
speak.
A
A
Like
well,
it's
that
we
can
go
is
that
these
phases,
for
PVS
can
go,
can
go
back
and
forth
like
a
PV
can
go
from
available,
can
go
to
bounds,
it
can
go
to
pending
and
it
can
go
back
like
we
can.
We
can
have
a
found
PV.
Then
we
can
have
an
available
PV.
We
can
have
it
released.
Pv
like
these
things,
go
in
in
different
directions.
A
That's
what
I
mean
I
see
like
you
can
so
like
look
at
this
way
like
how
how
would
I
track?
How
would
I
track
a
pv's
transition
times
if
it
could
go
from
I,
don't
know
what
the
first
phase
is
available
to
pending,
to
bound
back
to
available
like
what?
Why,
like
I,
can't
I.
B
C
Sorry
go
ahead.
Oh
sorry,
so
I
was
saying
that,
like
a
PE
can
be
reused
right
so
like
a
PV,
it's
not
like
so
therefore
I
think
it
may
like
it
may
like
it.
It
makes
more
sense
to
put
it
like
either
for
PVC
or
pod.
Pod
will
make
more
sense,
because,
because
I
think
we
can
still
use
PVC
like
I,
think
we
can
still
still
reuse,
PVC
right.
C
A
I'm
going
to
write
what
you
said:
Shane,
because
it
hit
it
right
in
the
head.
Pvs
can
be
reused,
that's
exactly
it
like,
because
vmis
pods
can't
be
reused
right.
They
go
once
you're
done,
they're
gone
right:
they
they
can.
They
can't
go
back
to
pending
phase
once
you've
gone
to
running
they.
Don't
they
don't
go
backwards
right!
That's
what
I
mean
by
bi-directional.
B
So
my
my
thoughts
on
this
are
that
PVS
are
the
are
not
the
thing
that
you
use
on
a
part
right
part
uses
a
PVC
and
PVC
can
be
bound
to
a
PV.
So,
regardless
of
how
the
PV
phase
transitions
I
think
if
we
can
capture
the
transition
of
PVC,
that
would
be
more
helpful
to
the
end-to-end
startup
time
of
pop.
C
So
that's
it
imagine
a
scenario
where
we
are
like:
there's:
no
there's
no
Dynamic
storage
class
and
we're
just
creating
a
static
PV
right.
So
how
so?
However,
you
would
do
it
is
that
you
would
create
a
bunch
of
TV
and
clusters
and
then
you
were
like,
and
then
you
have
created
PVCs
and
then
that
PVCs
points
to
that
to
like
to
a
manual
storage
class
and
then
your
pods,
you
are
basically
just
using
a
PVC.
C
So
what
happens
underneath
the
hood
I
believe
is
that
so
kubernetes
will
find
a
PV
for
this
PVC
and
once
this
PVC
is
attached
to
to
a
like
to
a
volume,
then
that
PVC
can
be
bound
to
the
pod,
but
I
think
I.
Think.
B
C
B
C
So
like
that
stuff,
sorry,
so
my
terminology
may
not
be
right,
but
like
let's
not
okay,
so
like
so
like,
instead
of
saying
you
know
bound
or
attached,
let's
use
a
let's
use
a
Nonpoint
return
like
write
it
right,
so
so
like
what
I'm
saying
is
that
a
TV
and
a
PVC,
the
like,
so
the
mapping
between
a
PV
and
PVC
doesn't
need
to
be
101.
That's
what
I'm
saying,
but,
however,
the
mapping
between
its
volume
and
the
pot
is
101..
So
that
means
that
a
part
cannot
dynamically
hot
plot.
B
I'm
not
sure
I'm,
not
sure
about
that,
but
yeah.
If
that's
the
case,
then
things
will
get
more
complicated.
A
A
C
A
C
A
A
Pvc
I
didn't
say
anything
right,
so
a
a
a
yeah,
so
a
a
single
PVC
can
only
be
used
by
a
single
pod.
A.
A
B
C
Or
not
during
them,
not
like
not
one
of
this
like
this
volume
is
in
use,
you
will
have
to
destroy
the
part
right.
Then,
if
you
like,
if
you
reattach
this
PVC
to
a
different
part,
then
the
PV
that's
associated
with
this
PVC
could
potentially
change
because
we
are
not
like
we
are
not
explicitly
control
like
controlling,
which,
which
PV
is
this
PVC
mapping
to
like
him?
B
B
C
B
C
C
C
I
see
so,
okay,
okay!
So
if
that's
the
case
then
I
guess
then
I
guess
we
can
put
then
I
guess
the
timestamp
in
the
PVC
can
reflect
the
status
of
the
PB
as
well
right
because
they
are
like
one
to
wine.
So
so
sorry
they're
like
one
to
one,
mapped
correct.
Yes,.
B
So
that's
what
I
was
suggesting
earlier,
that,
regardless
of
PVS
transition
time
as
far
as
the
Pod
is
concerned,
it
will
use
PVC
right
and
all
we
need
to
do
is
track
the
transition
times
of
a
PVC,
since
that
has
that
is
the
interface
via
which
the
part
will
use
the
volume
and
the
the
bounding
The
Binding
action
between
TV
and
PVC
has
to
be
Atomic,
so
both
the
timestamps
can
be
inferred
if
we
just
track
PVC
timestamps,
at
least
that
that
was
my
understanding
of
these
apis.
C
I
see
yeah.
B
C
B
Is
that
right?
No,
a
still
the
understanding
behind
the
concept
is
right,
but
I
don't
think
we
can
still
do
that,
the
reason
being
so,
let's
say
if
you
have
a
pvca
that
is
bound
to
a
pvcb.
Sorry
that
is
bound
to
a
persistent
volume.
B.
The
persistent
volume
B
can
have
a
retained
policy.
So
after
you
delete
pvca,
PV
B
can
still
exist
in
the
system,
because
the
retention
policy
is
retained
and
then
you
can
have
a
different
PVC
pvcc
that
can
bind
to
PVP.
A
A
Okay,
I
think,
but
the
the
shank's
point
is
interesting
because
it's
like
it's
like
what
we're
saying
the
PVC
controls
this
these
states
right
and
so
yeah.
The
strength
point
is
that
the
way
to
visualize
this
is
because
it
does
it's
trained.
The
the
transition
timestamp
should
be
on
the
PVC
right.
That's
the.
C
C
B
A
A
Is
there
like
another,
okay,
I
think
we
need
more
information,
then,
because
I
I
don't
yeah.
We
need
to
understand
that.
B
Yeah
I
think
Ryan,
where
you
are
getting
a
little
bit.
What
is
little
bit
confusing
is
that
some
PVCs
are
kind
have
this
policy,
where
they
wait
for
the
first
consumer
so
until
the
pod
doesn't
start
and
doesn't
use
that
PVC
they
go.
They
stay
in
pending
State
and
only
after
a
pod
is
explicitly
created.
That
is
you
using
that
PVC.
It
goes
to
a
bound
state,
but
that's
just
a
policy
on
the
PVC.
It's
not
specifically
a
controlling
Behavior
yeah.
A
Okay,
I
think
well,
I
I
kind
of
I
mean
I'm
glad
we
talked
about
this.
We
got
into
more
of
the
details,
but
yeah
I
mean
this
is
basically
what
I'm
gonna
need
to
spend
more
time
on
this
we
need
to.
We
need
to.
We
need
to
be
clear
of
those
phases
right
and
like
where
these
movies
go
open.
This
was
a
good
conversation
I'm
glad
we
kind
of
talked
about
it
in
in
this
way.
I
think,
let's
see,
I
can
do
some
more
research,
and
we
can.
A
We
can
talk
some
more
next
week,
see
if
this
is
ready
to
bring
to
the
kubernetes
scale.
If
we've
got
enough
here,
but
I
we're
at
time,
so
I
made
it
here,
but
we
can
we'll
follow
up
on
this
or
I'll
follow
up
with
everyone.
This.