►
From YouTube: SIG - Performance and scale 2023-05-04
Description
Meeting Notes:
https://docs.google.com/document/d/1d_b2o05FfBG37VwlC2Z1ZArnT9-_AEJoQTe7iKaQZ6I/edit#heading=h.tybh
B
A
But
here's
just
one
there's
just
one
output
in
this
particular
file.
Sorry
in
this
jump.
B
They
just
yeah,
so
it's
a
six
or
six
six
hundred
I,
don't
know
this
hundred
okay.
So
it's
a
hundred
yeah.
So
this
is
100
and
then
let's
see
what
else
I
think
it's
just
one
yeah
it
is
it's
just
one
but
hold
on.
That's
not
all
so
there's
this
one.
This
one
has
a
little
bit
different.
Let's
look
at
this
one
bullet
challenge.
Do
you
have
any
questions
about
this?
So
it's
just
one.
It's
just
101
output.
A
No,
what
I
was
wondering
is
over
my
tool
right
now.
It
scrapes
for
both
VMI
and
VM
together
and
then
so
the
regex
filtering
for
this
one
will
be
little
different.
However,
I
think
it
will
still
work,
because
we
have
recently
proposed
to
change
it
in
the
values,
so
it
will
find
the
correct
line
item
two
five
one,
one
so
I
think
that
will
work
here.
A
The
only
difference
is
in
the
code
path
where
it
should
not.
If
it
is
this
job,
it
should
not
look
for
the
second
one
and
fail
gracefully
okay.
If,
if
I
can
handle
that,
then
we
should
be
able
to
Crunch
these
jobs
as
well.
B
Okay,
all
right:
let's
look
at
the
second
one.
Now
this
one
I
don't
remember
if
it
does
two
runs,
but
it
should
have
more
than
100.
So
this
is,
let's
see
400.
B
B
A
Yeah
and
and
then
the
other
question
I
had
is
how
would
you
propose
to
format
these
output
so
right
now,
it's
very
simple
right
now.
All
we
have
is
a
weekly
BMI
and
a
weekly
VM
folder,
with
these
new
files
coming
in
I
think
we
should
put
it
in
like
density
tests,
folder
and
then
continue
to
segregate.
Well,
with
this
test,
segregation
at
VM
resource
level
does
not
matter
right.
Do
you
anticipate
in
future?
We
are
going
to
add
a
vmis
and
sorry
VMS
and
instance,
type
tests
for
the
density
test.
B
A
Okay
yeah,
then,
then
we
can
add
a
density
test
folder
and
then
within
that
100
400,
600
and
then
within
that
resources,
VMI
VM
and
instance
types
and
then,
with
the
doseable
host
host
the
charts
for
this.
B
B
A
No
just
wanted
to
give
you
a
heads
up
that
now
the
VMI
results
are
actually
VMI
results
and
I
was
hoping
we
can
do
one
or
two
tests
runs
to
you
know
verify.
This
is
indeed
as
expected.
A
A
I
wonder
why
such
a
big
spread
across
the
graph
to
look
at
most
other
charts.
It's
not
as
bad
as
this
one.
C
A
No
I,
actually
the
regex
thing
guard
took
more
time
than
anticipated.
So
I
don't
know
exactly,
but
this
is
something
you'll
have
to
check
out.
One
thing
I
can
I
can
think
about.
Is
that
if
the
patch
call
fails
because
of
the
older
resource
version,
that's
why
that
that
could
lead
to
more
batch
calls.
A
So,
like
all
the
bottom
ones
are
passing
in
the
first
try
and
then
the
top
ones
are
taking
more
than
one
try,
but
I
don't
know
if
that's
a
valid
Theory,
because
the
ones
at
the
bottom
they're
like
halfway,
for
example,
we
if
we
are
running
100
vmis,
then
40
or
50
particles,
don't
make
much
sense
like
there
is
no
Trend
with
that.
B
Yeah
yeah,
that's
I'm.
Thinking
too,
the
trend
is
bizarre.
There's
no
relationship
to
the
number
of
Creations.
We
did
like
this
one's
got
clearly
two
to
one
with
some
failures
right,
there's
like
never
less
than
200,
we
shouldn't
well,
it
is
one,
but
it
should
be
almost
never
less
than
200.
and
that's
two
to
one.
But
here
right,
like
you,
have
35
I,
don't
even
understand
that.
B
C
A
B
A
I
think
this
helps
us
find
the
uppercase
right,
so
there
is
nothing
Beyond,
one
more
than
one
call
per
VMI
so
that
that's
helpful
beyond
that
with
this
graph
is
noisy.
A
This
one
I
can
understand
because
it's
list
it
might
be
a
time-based
thing
or
something
sorry.
It's
list
of
nodes.
B
Yeah
this
was
the
one
where
we
were
kind
of
hoping.
If
we
figure
out
what
it
is,
we
actually
could
look,
and
here
you
can
see
what
we
get
for
the
trend
for
a
longer
running
one.
So
let's
say
this
is
list
node.
C
C
B
A
B
B
I,
don't
know
these
are
kind
of
low
up
too
so,
okay,
I
guess
we'll
see
when
we
get
the
data,
we'll
see
what
the
trend
line
shows.
I
guess.
So
our
expectation
is
that
we'd
expect
this
to
be
higher
than
than
these.
Like
400
emis
versus
100,
read
expect
to
be
a
little
higher,
yeah
I
guess
it's!
Maybe
it's
constant.
A
Yeah,
that's
I
mean
we
I,
don't
truly
know
I
think
we
should
not
be
listing
notes
at
all
honestly,
because
list
and
watch
call
should
only
start
at
the
beginning
of
the
whatever
controller
is
watching
for
it,
and
then
it
should
cache
all
the
entries
and
work
from
that
cache.
So
seeing
it
here
in
the
first
place
is
a
little
bit
surprising
for
me.
B
Yeah,
let's
see
what
the
trend
shows
because
I
wonder
so
the
dedicated
cluster
might
be
telling
us
that
we're
like
it
could
be
just
this
is
noise
from
the
background
and
another
job
sharing
the
cluster.
Maybe
this
is
from
you
know
the
dedicated
cluster.
B
B
C
A
Yeah
I
see
here
the
patch
call
beings
for
headache
again,
yeah
yeah
I
think
this
is
coming
from
the
underlying
oh
I
did
looking
at
the
in
link
by
any
chance.
Sorry,
the
link
in
the
top
bar
it
shows
VMI
should
be
looking
at
VM
right.
We
looked
at
the
vmis
earlier.
B
It's
gonna
be
okay,
okay,
so
here's
the
here's,
the
yeah
last.
What
else.
A
B
A
Yeah,
interestingly,
the
patch
calls
for
vmis
are
holding
up
nicely.
B
A
Oh,
you
know
what
actually
sorry,
but
what
I
was
thinking
is
that
it
includes
the
patches
from
the
VMI
controller
as
well
as
dbm
controller
and
the
variation
you
are
seeing
comes
from
the
VMI
controller,
the
older
chart.
We
were
looking
at
so
if
you,
theoretically,
if
the
patch
for
VMI
count
in
VM
controller,
remains
constant,
then
the
only
thing
going
up
and
down
is
the
previous
chart.
B
C
A
See
yeah
because
you
you
look
at
some
of
the
once
on
the
lower
end
of
the
graph.
It's
like
100
and.
C
B
A
I
compared
that
update
call
with
the
VMI
controller
and
the
VMI
is
at
800.,
pretty
constant,
so
I
think
my
understanding
is
that
this
call
is
split
into
two
eight
calls
by
DVM
I
controller
and
two
calls
by
the
EM
control
that
gets
us
to
10.
B
B
A
B
B
Yeah
yeah
I
mean
that
makes
sense,
because
I
mean
you
think
about
a
number
of
phase
changes.
You
have
to
do
that.
Okay,
so
like
in
terms
of
so
like
the
way
we
classify
this
in
terms
of
skill.
It's
like
we
get
two
more.
If
we
had
a
value
for
update
calls,
it
would
be
whatever
plus
two
times
the
number
of
VMS,
and
that
would
be
the
our
scale
number
versus
the
number
of
emis,
which
would
be
whatever
the
the
base
is,
which
is
eight
yeah
saber.
B
Okay,
that's
a
good
one
that
when
we
talk
to
the
kubernetes
six
scale
that
they
have
some
equations
to
model
a
few
things:
that's
where,
when
we
we
hoping,
we
can
get
some
punch
out
of
the
knowledge
share
where,
however,
they
have
their
calculations,
we
can
use
some
of
it
to
do
some
modeling
for
for
what
we
have,
and
that
should
be.
It
should
be
helpful
for
us
to
do
estimates.
B
A
C
B
So
you
need
this:
we
haven't
merged
this
guy
right,
yeah.
B
A
A
There
yeah
so
my
question
here
is
that
what
configuration
exactly
enables
that
artifact
directory
in
the
in
the
project?
A
So
if
we
look
at
the
other
tab,
the
density
cluster,
if
that
article
directory
is
missing
so
I
on
the
thread
in
keyword,
Dev
I
got
a
couple
of
data
points
as
to
you
can
we
can
populate
the
artifact
directory,
but
that
is
just
an
environment
variable
for
the
test.
So
I
still
don't
know
if
it
is
the
one
that
enables
this
artifact
directory.
D
I
have
linked
the
documentation
for
pro
where
the
environment
variables
are
explained.
They
are
basically
the
artifacts
is
the
one
you
are
looking
for.
Would
you
indicate
to
Pro
a
everything
which
is
written
to
this
path?
I
need
to
be
export,
I
need
to
export
thing.
We
studied
on
the
job,
but
I
would
need
to
check.
A
I
think
you
are
right,
I
think
we
set
it
up.
It
might
be
good
to
verify
that,
but
my
understanding
is
that
this
environment
variable
is
just
exposed
to
the
jobs
so
that
they
can
write
the
output
in
a
particular
directory,
so
my
that
that
plumbing
might
already
be
in
place.
The
the
question
is
that
does
this
environment
variable
also
enable
that
directory
being
populated
or
is
there
some
other
configuration.
A
D
Okay,
so
is
it
like,
is
it
all
the
time
or
does
it
happen
only
from.
D
C
D
D
We
sure
that
so
we
therefore
we
run
the
perf
scale
test.
Sh
right
are
we
sure
we
we
are
using
the
deluxe
artifacts
directory
there.
A
D
Interesting,
oh
yeah!
No!
No,
because
this
is
yeah,
but
the
path
was
not
not
the
same
right.
There
was
prefix
of
out
underscore
that
out.
D
Yeah,
so
that
will
be
it
I
mean
yeah,
not
sure
if
you,
if
we
use
to
be
dockerize
it
or
containerize
it
this
script,
probably
not
where
I
am
looking
at
it.
I
don't
see
anything
like
that.
You.
A
Know
I,
don't
think
so
so
I
think
the
the
place
the
environment
variable
for
this
output
file
is
the
audit.
Can
you
search
for
Capital
caps
or
audit
I
think
you'll
find.
D
Okay,
nice
I
can
see
it
right
now.
Let
me
actually
open
in
it
in
in
the
webs
browser,
so
I
can
share
with
you.
A
D
B
C
A
B
A
Yeah
I
will
try
it
out
I.
Another
thing
that
bugged
me
in
trying
this
test
is
that
I
have
not
actually
gotten
any
of
the
perf
skill
tests
to
run
in
my
environment.
My
Hardware
seems
to
be
the
limiting
factor.
Do
you
guys?
Are
you
guys
able
to
run
even
with
like
five
VMS
or
something
where
the
test
runs
end
to
end
in
in
the
local
environment?.
B
With
far
VMS
I
haven't
tried
in
a
while,
but
we
originally
did
it
yet,
but
you
have
to
you,
have
to
have
really
really
small
VMS,
like
you're
using
the
cluster
up
yeah
yeah.
You
had
to
get
like
really
tiny
ones.
I.
D
A
Yeah
I
think
my
Hardware
is
leafy
enough.
It's
just
that
I
I
am
using
the
default
configuration
I.
Think
I'll
have
to
change
that.
To
get.
B
B
Cool
okay
and
let's
do
it
all
right?
Let's
go
to
this
presentation,
so
this
is
what,
when
we
go
to
next
week
to
the
Upstream
sixth
scale.
This
is
what
the
slides
don't
want
to
talk
through.
So
does
everyone
have
an
invite
to
that
of
students?
Sixth
scale
calendar.
It's
like
it's
in
Thursday
afternoons,
it'll
be
next.
Thursday
is
every
other
week,
so
on
Thursday
afternoon,
well,
eastern
time,
I
don't
know,
especially
it
might
be.
B
It's
like
I
think
it's
I
think
it's
so
it's
let
me
see
so
it
would
be
like
like
2100
UTC
or
something
it's
later
than
this.
This
is
I.
Think
it's
like
four.
It
was
like
six
hours
after
this
time,
so,
if
you
don't
have
it
just
join,
grab
the
or
locate
the
invite.
If
you
want
to
attend
I,
don't
know
if
it's
gonna
you're
gonna
be
able
to
do
at
lubo
I,
think
it's
gonna
be
late
for
you
being
in
your
time
zone.
B
Yeah
I
think
it's
gonna,
be
it's
probably
late
for
you,
but
anyway,
so
I'll
just
give
you
the
what
the
slides
you
want
to
talk
about
with
with
them.
So
this
is
so
basically,
you
just
want
to
go
over
what
we
do
in
our
group
and
some
of
what
some
of
the
results
we've
seen.
So
this
is
our
mission,
so
we
maintain
our
own
API
server
scheduler.
B
We
have
workloads
that
are
independent
of
kubernetes,
and
so
we
should
have
our
own
performance
standards
tools
and
best
practices,
so
our
focus
with
measurement
is
to
leverage
existing
tooling
as
much
as
possible.
So
that
means
reethius.
We
have
we
focus
on
two
main
metrics
that
give
us
a
lot
of
our
data
right.
We've
got
phase
transition
times
and
we've
got
the
client
go.
Http
calls
to
a
kubernetes
API
server
and
the
two
things
we
focus
on
is
like.
B
And
what
this
does
is
allows
us
to
make
great
stuff
like
this,
which
is
valuable
for
from
seeing
how
virtual
machines
are
performing
and
in
our
clusters
and
create
a
bunch
of
expectations
around
those
runtimes.
In
those
different
phase
transitions.
B
And
then
same
with
the
client
go
HTTP
calls,
so
we
have,
we
should
be
able
to
catch
PR's
that
increase
the
hcp
calls
and
we
should
be
able
to
measure
it.
So
we
want
to
filter
for
these
PRS
and
and
catch
them
and
and
comment
on
them,
and
so,
in
this
case,
like
we've
got
the
ones
that
Olay
had
caught
earlier,
which
was
the
two
changes
we
had
to
finalize
with
the
finalizer
change
to
the
vmis
and
the
controller
location
which
affected
the
patch
counts,
that
we
have
for
VMS
that
are
managing
vmis.
B
And
so
our
future
goals
with
that
we
want
to
collaborate
with
that's
up
through
six
skill
groups.
We
want
is
that
we
have
a
virtual
machine
object.
It's
made
up
of
a
lot
of
things:
we've
got
pod,
pvc's
networks
and
other
things,
and
there's
just
too
many
variables
for
us
to
measure
and
isolate
performance
and
scale
for
just
the
virtual
machine
object.
It's
too
large,
there's
too
many
things.
So
how
do
we
isolate
these
things?
How
do
we
isolate
the
virtual
machine?
You
know?
B
Is
there
a
way
we
can
get
more
details,
our
detailed
phases
for
the
different
pieces
that
make
up
the
virtual
machine
that
kubernetes
is
primarily
responsible
for
so
things
like
PVC
attachment
times,
Network
attachment
time,
pod
phase,
Beyond
creation,
timestamp.
So
all
the
things
that
basically
applying
the
idea
that
we
have
the
phase
transition
timestamps
to
other
objects
in
the
kubernetes?
B
Maybe
we
don't
need
to
post
them
on
the
actual
object
themselves,
but
like
perhaps
we
can
have
a
way
to
post
this
in
Prometheus,
so
that
this
is
actually
actually
measurable
for
us.
So
more
than
just
that
creation
timestamp,
you
know
it
could
be
when
PVC
is
attached,
like
we
know
when
this
is
the
controllers
know
when
this
is,
we
just
need
a
Mark
when
it
happens
and
then
then
expose
it
so
same
with
any
other
things
that
could
make
up
possibly
a
virtual
machine.
We
want
it.
We
want
to
expose
those
things.
B
So
that's
the
ask
I
think
it's
the
last
that
I've
got
yeah.
So
that's
what
I
want
to
cover
with
the
with
the
group
and
then
hopefully
we
can
dive
into
discussion
on
some
of
these
things.
A
So-
and
this
looks
great
I
just
wanted
to
give
you
a
heads
up-
Upstream
kubernetes
has
they
have
deprecated
the
use
of
phase
in
in
things
like
broad
faces
and
stuff.
They
are
leaning
towards
conditions,
so
I,
don't
know
if
seeing
phases
here
will
trigger
that
discussion
points
in
people.
But
if
you
want
to
avoid
that
chance,
you
can
use
conditions
or
something
similar
of
that
nature.
Okay,.
D
B
B
A
No
I
think
so
the
way
this
will
the
way
I
see
this
happening
is
that
Upstream.
So,
let's
take
an
example
of
pod
right.
Pod
has
a
lot
of
different
agents
that
are
reconciling
pod
status
and
some
of
the
information
is
sent
out
in
conditions.
Some
is
sent
down
in
events
so,
for
example,
the
PVC
whether
it's
attached
or
not,
it's
it's
an
event.
Whether
network
is
attached
or
not
it's
an
event,
so
that
is
the
low
level
details.
B
A
B
That's
yeah,
that's
what
we're
looking
for.
So
if
it's
yeah,
if
it's
I
also
I
wanted
to
avoid,
is
that
someone
telling
us
that,
like
no
like
we're
we're?
Someone
else
is
working
on
this
because
then
it's
just
a
dead
Empress
like
it's
just
gonna,
be
so
they're.
Just
it's
gonna
be
unhelpful.
I
mean
you
know
what
I
mean,
because
they're
gonna,
basically
what
they're
doing
is
just
passing
this
off.
B
So
the
hope
is
that
what
whatever
this
is
like,
it's
just
to
convince
them
that
this
is
a
very
simple
idea,
like
it's,
the
equivalent
of
a
a
creation
timestamp
that
we
want
to
have
for
all
of
these
objects.
So
it's
at
least
I'm,
hoping
where
we
can
sort
of
take.
The
conversation
is
that
we
just
want
a
bunch
of
these
creation
timestamps,
but
we
want
them
to
be
about
creation
Thompson.
If
we
want
them
to
be
that
more
specific
yeah
things.
A
B
Okay,
yeah,
maybe
they're,
composed
as
sort
of
a
question
instead
of
like
just
a
well
I,
have
a
question
here,
but
maybe
I
can
make
it
as
a
little
bit
more
specific,
because
I
think
that's
sort
of
the
thing
is
we
want
our
goal
out
of
this
would
be
to
basically
to
have
like
a
cup
or
something
or
or
some
sort
of
PRS.
You
know
that
are
that
are
generated
at
the
end
of
this.
Not
you
know
that
that
would
be
the
goal
so
yeah.
B
If
we
can
get
the
conversation
that
direction
I
think
that's.
That
would
be
the
ideal
thing.
So
maybe
we
just
need
the
right
question
here
at
the
end,
okay,
all
right
sounds
good
guys.
I
think
this
is
so
yeah.
This
is
what
we'll
use
for
the
for
the
discussion,
and
hopefully
this
could
get
us
some
places.