►
From YouTube: Kubernetes SIG Node 20220712
Description
SIG Node weekly meeting. Agenda and notes: https://docs.google.com/document/d/1Ne57gvidMEWXR70OxxnRkYquAoMpt56o75oZtg-OeBg/edit#heading=h.adoto8roitwq
GMT20220712-170344_Recording_3440x1440
A
Good
morning,
everyone
today
is
the
July
12th
and
is
our
weekly
updates
and
a
few
of
us
cannot
join
I.
Think
because
it's
hot
is
the
vacation
time.
When
is
summer
time.
So,
let's
start
and
our
first
topic
Daniel,
do
you
want
to
talk
about
the
dynamic
resource?
Graduation?
B
Anyways
hi
all
I'm
Daniel
I
work
for
sap
I've
been
in
this
meeting
not
many
times
before,
and
today.
I
wanted
to
talk
about
a
a
enhancement
proposal
that
I
have
talked
about
already
a
couple
months
ago
and
I
can
always
show.
There's
also
some
comments
on
that
in
the
Google
sheet.
Let
me
try
to
share
now.
Okay
now
it
should
work
and
then.
C
B
The
last
time
I
talked
about
it
is
here.
There
were
also
some
comments
here
that
I
tried
to
address
over
the
the
time-
yeah
anyways,
just
to
recap,
because
it
was
a
a
while
ago.
The
the
main
idea
behind
it
is
targeting
cubes
Cuban
system
reserved
settings
on
in
the
cubelet
they're.
Currently
statically
set
and.
B
It
would
be
nice
to
to
be
able
to
do
that
at
runtime,
actually
so
the
the
and,
why,
as
the
motivation
behind
it,
we
have
seen
on
our
Landscapes,
we
are
we're
essentially
operating
a
kubernetes
as
a
service
internally
at
sap,
and
so
we
have
a
bunch
of
clusters
running,
and
we
have
seen
that
the
resource
reservations
that
we
are
setting.
B
We
have
a
similar
heuristic
based
on
on
Max
parts
and
these
kind
of
things
that
these
reservations
we
are
setting
statically,
don't
really
reflect
reality
and
for
one
it
depends
on
how
many
parts
you're
running,
obviously,
and
also
what
kind
of
workload
is
actually
running,
and
so
we
have
seen
for
the
same
amount
of
pots
deployed,
sometimes
also
Max
spots.
We
typically
do
in
like
200
or
250,
so
the
maximum
possible.
B
We
are
seeing
vastly
different
numbers
regards
to
utilization
and
also
in
regards
to
how
much
memory
and
CPU
actually
container
runtime
is
using
mostly
also
the
cubelet
but
the
cubelet.
We
have
seen
typically
scales
with
the
amount
of
Parts
deployed
or
not
scales,
but
the
model
memory
it
needs
and
CPU.
B
We
see
that
we
are
not
reserving
enough
and
then
under
some
circumstances,
that
can
we
have
seen
some
CPU
starvation
or
system
level
out
of
memory
issues,
because
the
the
C
group
limit
on
cue
Parts
that
is
set
from
system
Q
reserved
is
too
high.
Actually
that's
why
we
see
these
problems
and
we
without
restarting
the
cubelet.
We
have
no
time
have
no
option
to
adjust
it,
but
it's
not
a
really
a
good
option
to
restart
the
cubelet.
B
So
that's
would
be
one
of
those
reasons
why
it
would
be
nice
to
adjust
that
at
runtime,
maybe
just
a
second,
for
instance,
if
I,
let's
see
okay,
now
it's
just
burned,
reload
them
real,
quick
I
thought
it
would
be
nice
just
to
see
it
real,
quick.
What
I'm
even
talking
about.
B
Essentially,
I
just
wanted
to
show
you
real
quick
that
we
see
vastly
different
numbers.
This
is
just
a
random
cluster
here,
if
we
are,
if
you
see
this
here
on
the
right
this
column
here,
for
we
see
very
different
working
set
sizes,
for
instance,
for
system
slides.
B
We
see
different
numbers
ranging
from
two
to
eight
gigs
and
that's
already,
when
we're
have
100
requests
and
also
when
we
calculate
how
much
memory
we
should
Reserve,
then
it.
It
also
varies
from
like
here,
one
gig
to
sometimes
seven
gig
or
six
gigs.
So
it
really
fast.
It's
vastly
different
and
that's
the
main
main
motivation
so
yeah.
B
What
in
the
past
have
also
one
of
the
reasons
why
that
is,
for
instance,
for
CPU
that
I
have
checked,
is
the
container
logs
to
stand
it
out
right
and
then
go
through
a
pipe
and
then
the
container,
the
shim
decorates
those
locks
and
and
writes
it
to
the
cubelet,
the
barlock,
pods
and
so
on.
So
if
we
have
a
container
that
does
a
lot
of
blogging
that
influences
the
container
runtime
CQ
requirements.
B
That
is
one
of
those
reasons
why
that
could
happen
and
for
the
amount
of
Parts
is
because
the
container
Edition,
for
instance,
is
part
of
the
the
container
runtime
C
group
anyways.
B
That
is
the
main
motivation
why
that
would
make
sense.
From
our
point
of
view,
the
the
long-term
goal
would
be
to
for
one,
of
course
prevent
issues
in
our
stakeholders
clusters,
because
we
don't
control
these
clusters
to
be
like
a
like,
a
maybe
like
gke,
and
we
don't
control
clusters
from
our
stakeholders.
So
we
would
like
to
prevent
issues
in
those
clusters
and
we
also
don't
set
their
system
and
and
Q
preserve
settings.
We
we
just
do
it
with
the
static
formula,
but
certain
for
us.
B
For
instance,
in
one
occasion
someone
is
relying
on
Cuba
Division
and
they
basically
have
a
memory
leaking
container,
and
if
these,
if
the
the
secret
limits
are
not
properly
set,
no
eviction
will
take
place,
but
a
system
level
om,
and
that's
just
one
of
those
instances
where
we
don't
have
to
go
in
there
and
explain
and
try
to
find
the
right
settings
so
Cuban
system
reserved
and
it
is
causing
causing
effort
on
our
side
and
the
other
topic
is,
of
course,
utilization
just
to
be
able
to
safely
increase.
B
That
would
save
a
lot
of
money
yeah.
If
you
have
any
questions
or
something
like
that,
please
just
ask
if
I'm
not
making
sense
just
tell
me
otherwise.
I
would
go
to
two
concrete
proposals
and
that
would
also
be
included
in
this
Draft
cap.
D
B
D
Okay
but
see
if
it's
fixed,
usually
for
you,
it's
not
sufficient,
or
it
was
too
much.
B
For
the
majority
of
cases,
a
formula
like
gke
or
Azure
over
reserve
and
I
think
that's
the
main
purpose
behind
these
static
reservations
to
have
an
over
reservation,
but
we
also
see
that
in
certain
cases
we
do
not
Reserve
enough.
So
we
have
like
it's
never
really
correct
for
us.
That's
the
that's.
The
main
reason.
B
I've
actually
built
a
POC
for
this,
but
essentially
what
you?
What
I
think
you
need
is
you
need
to,
for
instance,
on
Linux
systems.
You
would
need
to
check
regularly,
like
a
Reconciliation
fashion
program
info
just
check
how
much
really
memory
you
have
available
on
a
system
plus
need
to
check
some
C
group
stats
from
cue
parts,
for
instance
to
overall
and
and
for
for
CPU,
for
instance,
measure
the
the
real
free
CPU
time.
I.
Think
the
advice
already
has
that.
B
B
Yeah
so
I
think
that
is
then
an
implementation
detail
right
I
mean,
but
of
course
it
matters
how
to
get
these
metrics
and
how
to
then
make
a
decision
based
on
those
definitely
matters
and
I.
Think
that
ties
well
into
those
two
proposals
here.
B
So
the
first
proposal
is
actually
to
to
run
some
external
recommended
process,
whatever
that
is,
it's
that
wouldn't
wouldn't
be
part
of
core
kubernetes,
and
this
one
calculates,
however,
that
they
won't
do
that,
for
it
could
be
obviously
different
for
Windows
and
for
Linux
a
recommendation
and
the
cubelet
would
have
a
means
of
telling
it
what
system
reserved
and
Cube
preserved
should
have
so
that
it
wouldn't
be
only
through
the
config
file
like
here
on
the
left,
but
you
could
do
that
at
runtime,
and
one
idea
would
be
to
expose
like
a
grpc
server
that
would
be
similar
to
where
is
it
here
to
the
Pod
resources?
B
Essentially,
the
the
cubelet
could
also
create
a
Unix
socket
and
call
that
I
don't
know
under
Dynamic
resource
reservations,
and
it
has
an
API
where
you
can
post
and
get
the
current
reservations
from
so
you
can.
You
can
have
an
ability
to
reconcile
it.
E
One
question
about
how
frequent
privilege
will
report
this
change
the
allocation
to
scheduler
on
the
Node
resource,
because
it's
going
to
impact
the
scheduling
decision
going
forward
right.
We
have
C
issues
like
a
static
apart
resource
consumed
by
Static,
Parts
being
reflected
at
a
scheduler
site.
Editor
node
object
side
with
the
latency
cause,
scheduling,
failure
or
couplet
out
of
resource
for
the
Pod
to
start
on
the
Node.
How
do
we
mitigate
that
with
the
changing
allocatable
resource
here.
B
Yeah
I
think
that's
one
of
the
biggest
difficulties
and
that's
also
one
of
the
things
that
should
probably
be
talked
about
when
in
this
cap,
because
yes,
you're
you're,
changing
that's
essentially
like
changing
the
config
right
on
on
the
Node,
making
a
different
Cuban
system
reserve.
What
today
is
already
possible,
but
you
just
do
it
a
lot
more
often
and
so
yeah.
It
is
really.
You
will
have
to
see
what
impact
it
has
on
on
the
scheduler.
Exactly
yes,.
F
And
I
think
one
thing
we'd
have
to
consider
is:
how
does
it
impact
the
running
workloads
as
well?
So
if
your
system
is
consuming
more
resources-
and
you
decide
to
allocate
it
more,
do
does
that
mean
that
we
have
to
start
evicting
the
existing
pods
that
are
running
on
the
cluster,
so
I
think
that
would
be
very
important
as
well.
B
I
actually
think
that
is
I
haven't
tested
this.
Yes,
it
was
on
my
list,
but
I
think
that
that's
how
what
the
human
would
do
so
for
for
some
reason.
The
system
reserves
the
like
your
container
runtime
needs
so
much
more
memory
than
it
could
happen
that
you
have
more
pots
right
than
more
reserved
on
your
pots.
Then
you
actually
have
allocatable
and
yes,
so
there
are
certain
things
we
have
to
take
care
of.
That
obviously
would
need
to
be
thoroughly
tested.
H
H
So
all
kind
of
problems
like
CPU,
hot
black
or
offline
or
memory,
hot
black
or
offline,
practically
required
to
restart
the
public.
So,
instead
of
just
the
reservation
part,
can
we
probably
think
about
what
kublet
resource
available
resource
discovery
of
the
dynamic?
Well,
like
the
whole,
the
whole
set
of
resources.
A
The
the
Alexandria-
this
is
the
fundamental
problems.
Scheduler
is
not
doing
the
usage
or
available
or
what
a
while
skydiving
right.
So
the
problem
it
is
when
we
report,
we
know
the
comeback
and
register,
and
we
report
back
see
here's
the
available
here.
It
is
the
machine
capacity.
Here's
the
available
resource
scheduler
just
take
that
one.
So
it's
not
really
consume
of,
so
they
don't
expect
those
changes
and
also
they
don't
expect
it.
There's
the
usage
you
take
into
consideration.
We
include
the
sky
learning
decision.
So
that's
why
we
end
up.
A
H
F
I
think
the
allocatable
is
the
value
that
is
capacity
minus
reserved.
It
is
not
taking
into
consideration
the
resources
occupied
by
the
pods
itself.
H
Yeah,
let's
find
somebody
what
I'm
saying
is
what
like,
instead
of
just
trying
to
make
one
parameter
like
resorbent
resources
change
it
like
well,
let's
have
all
the
capacities
to
be
adjustable
during
the
lifetime
of
a
Google.
A
A
How,
because
at
any
given
time
and
you
the
resource
usage,
just
think
about
the
simple
cases,
just
only
like
the
system,
usage,
kernel,
usage
and
also
those
system,
demons,
usage
and
again
it
has,
they
may
be
like
the
one
t0
the
usage
you
got
to
pick
and
the
T1
they
dropped
and
how
we
are,
through
those
kind
of
things,
normalize
those
things
right
so
that
it
is
needed
to
take
consideration
and
sometimes
those
usage
go
up
and
down
actually
rely
on
the
workload
right.
A
So
it's
not
working
on
the
working
of
the
also
like
it's
special
for
a
lot
of
the
first
of
all
the
usage.
It
is
sometimes
this
go
up
and
go
down
and
when
they
sometimes
when
they
go
up-
and
we
also
will
trigger
off
the
systems
right
kernel,
certain
colors
that
go
up
and
down
so
how
we
are
going
to
smooth
those
things.
A
That's
the
more
fundamental
problem,
and
so
this
is
why,
because
in
the
past,
we
didn't
have
that
as
like
average
like
to
normalize
the
data,
and
we
didn't
report
back
to
the
scheduler.
So
this
is
why
we
take
the
approach
in
on
the
Node
side,
make
that
over
reserved
a
little
bit
over
reserved
and
and
also
like
to
try
to
be
the
most
static.
That's
the
fundamental
problem,
but
in
the
book
in
the
past
we
did
in
the
node.
A
We
do
do
those
normalization
we
based
on
the
giving
time
and
the
moving
average
and
report
to
scheduler
Sky
do
not
take
that
into
consideration
and
so
that
we
can
be
more
dynamic.
Unfortunately,
we
we
think,
oh
that's,
really
over
complicated
when
the
initial
design,
especially
with
the
scheduling
design
even
today,
I
think
that's
still
think
about
this
over
complicated,
so
so.
This
is
why
we
love
our
really
doing
those
moving
average
to
report
back.
So
that's.
A
Why
end
up
on
the
notice
side,
because
we
have
to
protect
the
node
right,
you,
the
one
things
you
definitely
want
to
prevent.
It
is
out
of
disk
out
of
memory
right,
especially
out
of
those
things,
because
you
could
tear
down
the
entire
node
and
out
of
memory
even
include
the
cost
of
the
disk
corruption,
right
data,
corruption,
issues
and
hold
the
holder
like
the
node.
It
is
go
down,
so
that's
kind
of
things.
A
A
We
could
also
always
at
runtime
rejected
right.
So
so
we
don't
report
schedules
are
oversimplify,
and
but
we
don't
report,
but
when
the
scheduling
job
to
here
and
we
based
on
currently
used
it
based
on
current
allocatable
and
rejected,
but
that
could
cause
of
the
cascading
problem.
You
keep
just
reject
at
the
schedule,
keep
it
as
scheduling.
On
the
same
note,
so
that's
all
scheduling
to
try
to
avoid
that
kind
of
thing.
Then
we'll
do
another
level
of
the
Idaho
fixed.
A
So
this
is
why
I
kind
of
like
the
at
the
early
time,
because
for
the
simplified
scheduling
and
I
decided
to
make
this
also
is
over
simplified,
but
to
satisfy
customers
so
I
made
that
is
the
flag
or
configure
So
based
on
your
working
environment.
Let's
find
some
customers
say:
oh
I
have
the
large
node,
but
even
though
the
manager
node
I
only
have
the
one
part
how
much
management
resource
you
are
needed
right.
So
you
could
food
and
know
the
come
up
time
and
the
admin
next
Twitter
can
see
here.
A
So
this
node
and
I
I
type
the
node
label
the
node
and
then
make
some
changes.
Small,
like
the
reservation,
much
smaller,
so
that's
kind
of
flexibility
back
then
the
decision
I
made
it
yeah,
so
so
I
just
want
to
explain
some
background
and
the
context
where
this
is
complicated.
But
at
that
time
we
opened
you
if
scheduler
open,
to
take
some
consideration
to
take
some
usage
or
wires
scheduling
change.
B
Yeah,
thanks
for
the
background
and
I
think
that
that
my
question
would
be
currently
we
already
allowed
changing
the
other
cable
right.
Like
my
like
someone
else
already
said
before,
sorry,
I
I
missed
the
name,
so
it
actually
would
just
update
the
other
cable
like
today,
right
and
sure
maybe
I.
If
I
understood
it
correctly,
you
were
saying
that
if
that
would
happen,
while
the
scheduling
decision
happens,
then
that
could
lead
to
to
problems
during
the
scheduling
process.
If
I
understood
that
correctly.
B
The
thing
is
this:
how
often
that
happens
right?
How
often
you
update
they
are
locatable
is
same.
It
would
be
similar
to
how
often
do
you
restart
at
currently
the
cubital
process,
and
how
often
do
you
change
the
the
configuration
in
the
cubelet
so.
H
Well,
our
applicable,
for
example,
can
be
changed
if
device
plugins
goes
up
and
down.
That
leads
to
update
with
an
old
resource
status.
C
So,
on
the
basis
of
number
of
probes
you
have
and
how
frequently
they
run,
so
we
have
a
fixed
part
over
it
right
now
right
and
that
would
also
require
running
your
monitoring
process
inside
the
Pod
C
group.
So
in
cryo
we
have
an
option
where
you
can
either
run
your
your
shim
either
in
the
system,
slice
or
you
can
run
it
within
the
Pod
slice.
C
C
F
B
C
B
Yeah,
if
you
can
I'm
just
thinking,
if
you
can
make
sure
that
that's
the
only
thing
that
increases
cubelet
and
container
runtime
usage
right,
yeah.
C
Mainly
it's
that,
based
on
our
observations
like
at
stable
State,
when
all
your
pods
are
running
like
exec
probes
is
the
biggest
offender
and
then
okay,
now
often
you
run
them
rest
of
it
is
mostly
linear.
A
C
The
yeah,
the
cubelet
itself,
is
like
dynamically
calculating
the
Pod
over
it
on
the
basis
of
number
of
probes
and
how
frequently
they
are
run.
I
mean
it
won't.
It
won't
solve
this
entirely,
but
it
will
make
it
more.
I
mean
it'll
solve
part
of
it.
Right
I
mean
you
still
need
to
worry
about
pods
and
containers
that
are
using
them
that
have
a
memory,
leak
and
stuff.
I
C
So
I
mean
this
assumes
that
you're
charging
your
shim
processes
to
the
Pod
slice
right,
so
we
are
system
reserved
like
you
can.
Your
system
result
can
be
more
accurate.
It's
not
going
to
change
much
because
the
overhead
is
charged
to
the
part
itself,
but
right
now
that
pod
overhead
is
a
constant.
What.
I
I
I
Okay,
then,
does
it
really
solve
this,
like
over
configuration
of
the
system
reserved.
C
F
C
A
So,
basically,
how
the
overheader
we
introduce
it
is
the
one
to
more
accurate
to
charge
those
usage
right
prop
the
usage,
all
those
kind
of
things.
So
what
terminal
suggest
is
just
make
that
it
is
chatted
to
the
right
owner
right
usage
users
here.
B
G
B
C
A
I
know:
what
do
you
propose
actually
charge
to
the
power,
the
slice?
So
that's,
but
when
we
first
want
to
do
the
part
overhead,
we
do
think
about
the
move.
Those
kind
of
the
stride
into
the
per
policy
group,
or
so
like
the
charging
really
not
only
yeah.
C
A
J
I
I
took
a
quick
look
at
the
cap
and
I
think
I.
Damn
support
you
of
the
external
monitor
idea
there.
Various
Innovations
can
happen.
Ebpf
is
probably
a
good
way
to
do
it.
One
thing
is:
definitely
would
want
to
run
this
by
the
six
scheduling
folks
to
see
what
they
feel
about
modifying
this,
the
node
allocatable,
because
it's
definitely
going
to
affect
scheduling
decisions
have
had
to
deal
with,
explain
things
to
them,
just
for
the
Pod
resize
part
of
it.
I
This
is
you
know,
approaching
another
very
inter
introducing
another
variable
for
them,
and
then
we
want
to
take
safe,
Don's
thoughts
about
usage
as
well.
If
something
that
can
be,
that
can
be
used
to
make
this
better
yeah.
A
My
understanding
is
the
scheduler
is
a
really
optimized
based
on
assumption,
but
those
are
stable
nights
so,
like
the
like,
the
the
powders
vertical
scheduling,
those
kind
of
things
actually
I
think
is
the
only
effect
of
the
Single
part,
but
not
against
the
entire
of
the
exact
part
in
the
node.
So
they
are
really
Keen
about
that.
Yes,.
I
Yeah
this
this
would
introducer
is
the
resizing.
A
single
pod
has
a
race
condition
possibility
and
we
use
the
max
to
mitigate
you
know
to
kind
of
prioritize
existing
pod
resizing.
Now
this
is
like
resizing
a
node
as
far
as
the
scheduler
is
concerned,
and
that
would
affect
all
pods
that
are
being
you
know,
being
evaluated
for
a
particular
node
fit
and
definitely
would
run
this
by
the
six
scheduling
folks
and
see
what
they
feel.
B
Do
that
that
makes
a
lot
of
sense.
Actually,
that
was
also
one
of
the
that's
one
of
the
major
unknowns.
For
me.
It's
also
hard
to
test,
but
yeah
makes
a
lot
of
sense.
Yeah.
You
have
looked
at
the
draft
PR
right
and
the
draft
and
cap
that
I
posted
yeah.
B
So
I
think
that's
where
I
included
this
picture.
This
one
is
an
alternative
one
that
is
more
similar
to
the
BPA
and
how
that
that
one
works.
So
that
would
be
an
alternative,
essentially
that
that
I
just
wanted
to
quickly
show
you
so
instead
of
the
cubelet
having
a
grpc
server,
we
could
also
just
say:
Okay.
B
We
don't
have
any
of
that.
The
the
qubits
are
still
only
watches.
Node
resources
doesn't
have
a
grpc
server
and
we
we
would
introduce
like
in
the
specifications
like
you
preserved
and
system
reserved
that
field,
and
if
later
you
want
to
have
also
eviction
hard
or
something
like
this,
these
kind
of
settings.
B
You
could
also
introduce
that
in
the
spec
and
say:
okay,
instead
of
running
through
the
qubit
configuration,
you
just
can
specify
these
settings
on
the
Node
itself,
and
then
you
would
have
metrics
you
that
can
be
used
by
an
external
control
plane
that
does
it
for
all
nodes
right,
similar
to
the
vpa.
That
gets
it
through
the
core,
metrics
Pipeline
and
then
can
update
According
to
some
some
algorithm
or
specification,
and
that.
B
Yeah
so
that
I
I
would
say,
maybe,
if
I
open
that
draft
or
go
to
also
two
or
six
scheduling
that
I
would
also
then
show
different
proposals.
Maybe
these
two
I'm
currently
kind
of
preferring
this
alternative
implementation,
just
because
it
probably
wouldn't
even
require
running
like
a
demon
set
or
something
like
this,
because
typically
people
have
already
know.
Node
exporter
deployed
that
already
gets
like
metrics
from
proc
mem
info
and
then
maybe
through
C
advisor.
B
You
could
also
get
some
other
metrics
from
the
from
the
c
groups
that
would
be
required
to
calculate
the
reserved
resources
and
then
it
would
be
not
very
intrusive,
so
that
that.
I
B
Yeah,
currently,
the
cubelet
correct
me
if
I'm
wrong,
also
watches
notes
right
and
just
if
some,
if
somebody
nothing
would
change
essentially
just
still
watches
the
node
resource
and
as
soon
as
someone
like
our
reserved
resource
recommender
control
plan.
If
it's
deployed
changes
something
then
the
cubelet
would
adjust
the
C
group
limits
on
coupons
based
on
that,
so
it
the
the
main
difference
here.
It's
just
it
doesn't
have
a
grpc
server.
B
It
goes
through
the
kubernetes
API,
it's
a
little
bit
more
kubernetes
native
yeah,
but
then
you
have
a
little
bit
that
you
need
to
go
through
Prometheus
and
these
kind
of
things
like
DPA.
Does
it
no
okay?
That
was
wrong?
It
doesn't
go
through
promises
BPA,
but
yeah.
B
It's
just
a
little
bit
of
a
different
approach
and
I
think
these
the
details
can
be
discussed
also
in
when
I
open
the
draft
cap,
if
you
like,
if
it
goes
to
too
far
now
there
are
also
other
topics
and
I
wrote
down
some
dependencies,
for
instance
everywhere.
We
should
add
there,
of
course,
the
scheduling
dependencies
right.
What
happens
then?
What
happens
for
eviction
if
you
change
that
a
lot
and
also
in
the
qubit
code
itself?
What
happens?
B
Basically,
you
I
need
to
check
wherever
the
we,
we
assume
static,
resource
reservations
that
needs
to
be
looked
at,
for
instance,
and
as
far
as
I
remember
in
the
CPU
manager
in
some
places,
and
that's
the
case
and
also
with
this
memory.
Quality
of
service
feature
flag,
that's
currently
in
Alpha
that
and
and
the
memory
manager
as
well.
Yes,.
G
B
I
think
that
would
make
sense
when
implementing
that
to
make
sure
that
that
there's
nothing
going
wrong
or
unexpected.
B
Sure
I
think
that
makes
sense.
I
don't
have
a
lot
of
experience.
How
to
do
that
so
I
would
I
would
have
to
reach
out
to
some
of
you,
but
I'm
certainly
would
that
would
make
sense,
I
think
foreign.
B
Yeah,
so
these
are
all
good
points,
sure
make
sense
I'm
coming
to
the
end
here.
I
don't
want
to
take
too
much
of
your
time,
so
I
think
the
main
questions
for
me
would
be.
B
Is
that
something
that
you
guys
think
is
valuable
and
and
if
so,
how
would
be
the
process
and
going
ahead
with
this?
Would
I
open
a
a
draft
cap
and
then
more
discussion
takes
place
there
or
how
How
would?
How
would
that
go.
B
So
I'm
just
not
quite
sure
about
the
process
itself.
It
seems
to
well
from
what
I
read
is
you
need
to
have
some
traction
right?
That's
what
I
read
from
the
documentation,
I'm
so
I'm,
not
quite
sure
how
to
quantify
that
I
mean
the
issue
is
open
since
a
while
on
our
side,
we
still
think
it
makes
sense
to
implement
such
thing.
A
B
B
A
B
The
the
biggest
thing
for
us
is
actually
is
not
so
much
the
the
amount
of
pots
deployed,
because
we
typically
have
the
max
amount
of
Parts
deployed.
B
So
for
us,
it's
pretty
deterministic,
this
part
at
least,
but
what
is
not
really
deterministic
for
us
is
the
amount
of
workload
and
the
kind
of
workload
running.
So
that's
the
bigger
issue
from
from
our
side.
So
that's
why
it
would
be
good
to
have
that
pot
overhead.
If
you're
talking
about
this.
Yes,
we
have
that
if
that.
A
Yeah,
actually,
what
menu
suggests
is
not
doing
the
only
number
of
the
problem
number
of
the
color.
Also,
it
is
workload
specifically
because
a
certain
part,
the
quest
of
the
problem
and
the
requested
some
logging
and
other
things
that
also
consume
of
the
resources.
If
we
don't
properly
charge
those
to
the
powder
and
over
hide
and
then
you
will
charge
it
to
the
cube
assistant,
so
that's
kind
of
like
the
make
that
it
is
unstable
and
not
as
static,
not
accurate.
A
So
if
you
could
to
try
what
he
suggested
here
and
then
we
actually,
you
could
because
one
of
the
another
set
of
problems
you
see
that
you
don't
want,
like
the
gke
over
charged
right,
so
no
over
reserved
based
on
the
nursing
size
and
the
production,
because
you
are
not
controlled
about
the
cluster
as
gke.
So
so
you
want
to
give
the
more
flexibility,
so
you
want
to
reserve
even
more
smaller.
So
after
that,
the
overcharge
to
the
powder
slice,
the
cable
part.
A
So
then,
even
like
the
particular
powder,
you
could
estimate
down
about
your
Cuba
system
reserve.
Here,
that's
I
think
you
can
solve
the
Europe.
Your
original
code
there
so
there's
another
problem,
Alexandra
earlier
mentioned-
that
to
make
this
a
whole
thing
more
Dynamic
and
that's
the
even
more
complicated
problem.
This
is
why
I
mentioned
that
the
scheduling-
oh
wow,
that's
the
more
genomic
solution,
but
at
this
moment,
based
on
your
problem,
your
production
I
do
think
about
the
word.
A
Menu
suggested,
maybe
doesn't
solve
your
problem,
so
maybe
we
could
start
from
there
and
say:
do
we
need
to
do
this
implementation?
Do
this
change?
We
could
come
back
this
way,
make
that
even
more
generic,
but
I
do
think
about
that
to
solve
your
product
and
issue.
When
you
have
the
really
great
suggestion
there,
that's
how
initially
we
started
the
part
overhead.
L
Just
one
coming
once
you
end
up
doing
that.
You
probably
have
better
note
stability
in
that
these
overheads
will
be
constrained
because
they're
not
really
constrained
today
when
they
can
kind
of
fight.
So
there
is
kind
of
a
noisy
issue,
it's
constrained
just
to
the
Pod
and
not
to
the
other
workloads
running.
So
this
is
positive.
The
negative
side
is
that
now
you
are
going
to
expose
this
overhead
most
likely
to
the
end
user,
that
they
will
be
consumed
for
this.
So
in
my
experience,
this
is
the
drawback
challenge
of
pod
overhead.
L
That
will
have
to
be
understood
where
someone
says
I
started
a
50
ml,
CPU
pod
with
one
mag
of
memory,
and
you
have
this
overhead
in
place,
even
if
it
is
dynamic,
how
they
view
their
resource
quota
depending
how
sensitive
they
are
in
their
namespace.
B
I'm
currently
wondering
could
I
centrally
enforce
this
and
calculate
these
pod
overheads,
because
I
don't
have
I
assume
that's
possible
Right
to
do
that.
Currently,
you
cannot.
It's.
E
B
C
I
think
that's
that's
where
we
can
start
right
like
like
one
way
is
looking
at
the
exact
frequency
I
think
if,
if
we
some,
if
we
have
a
way
to
add
an
overhead
for
that
accurately,
then
that
will
solve
most
of
it.
So
that's
where
I
think
we
can
start
experimenting
in
this
area
like
how
can
cubelet
look
at
the
Pod
spec
and
adjust
the
Pod
over
it?
C
A
Exactly
that's
exactly
there's
the
two
things
that
I
want
to
comment:
why
it
is
we
are
using
that
to
to
estimate
and
the
feedback
loop
and
will
guide
the
customer
right
you
how
you
another
thing
is
the
customer,
because
that's
the
overhead
user
should
pay,
so
customer
didn't
estimate
that
the
correctly
initially
we
get
that's
a
static
one.
So
if
the
customer
wants
using
more
even
they
are
guaranteed
today,
so
they
may
be
cured
and
that
damage
constraint
by
that
part
level
that
participal
for
either
being
killed
or
whatever
slow
down.
A
So
then
they
they
could
make
that
burstable
right
first
to
make
that
versatile
and
did
the
measurement
again
even
without
the
the
verticals
Auto
skating,
and
this
still
could
make
that
burstable
and
didn't
measure
themselves
and
the
next
time
to
to
make
that
is
at
a
part
level
to
share.
So
they
still
have
the
way
to
do
these
things
to
move
forward,
but
I,
don't
think
about
that!
I
do
think
about
that
blind
charge
to
the
system.
That's
the
wrong!
We
did
that
initially,
because
the
we
that
time
is
too
earlier.
A
We
did
that
in
initially,
but
the
reason
we
introduced
the
Pod
overhead.
It
is
try
to
solve
this
problem,
especially
after
we
have
like
the
VM
workload
to
support
the
rest,
like
the
Kata
content
and
the
overhead.
It
is,
is
much
bigger
than
the
pulse
container,
and
so
that's
why
we
want
to
introduce
this
configurable
at
the
first
place.
Just.
B
On
top
of
my
head
trying
to
understand,
essentially,
then
the
cubelet
would
need
to
somehow
determine
how
much
CPU
and
memory.
If
we
just
talk
about
this
overhead,
it
causes
four
system,
slice
processes
right
so
I'm,
trying
to
understand
how
it
can
do
that
without
monitoring,
for
instance,
the
the
standard
out,
how
much
a
pot
is
actually
riding
at
what
frequency
to
to
stand
it
out.
How
would
it
be
able
to
to
practically
do
that?
That's
what
I'm
trying
to
understand.
B
Because
you
would
need
to
have
some
I,
don't
know
how
yeah
just
it
seems
to
be.
You
need
to
find
you
need
to
find
every
single
or
a
lot
of
the
single
reasons,
how
a
pop
a
container
process
can
influence
the
memory
and
CPU
usage,
so
you're
going
the
other
way
around
and
to
me
just
on
top
of
my
head.
That
sounds
difficult,
but
I'm
sure
you
have
a
lot
more
experience.
Doing
that
it
seems
to
me
more.
L
If
everything
is
in
the
appropriate
pod,
C
group
like
they
get
their
way
cryos
that's
up.
You
can
easily
always
know
what
the
overhead
is
in
that
it's,
this
sum
of
container
the
workload
has
their
own
unique,
C
group
as
a
child
in
there.
If
you
subtract
that
from
the
Pod
C
group
you'll
know
exactly
what
the
overhead
is,
the
problem
is:
what
do
you
do
with
this
information?
L
I
know
what
the
runtime
characteristics
are,
but
I
think
that
that's
a
whole
different,
challenging
problem,
and
should
that
be
done
at
the
node
or
the
scheduler,
you
know
today
with
it
being
static,
the
scheduler
knows
so
it
can
pick
a
node
appropriately,
knowing
that
the
summer
container
requests
and
the
overhead
is
what
it
needs
to
take
into
account.
If
we
don't
know
that
what
the
overhead
is.
L
A
A
G
A
I
Yeah
I
I
was
on
vacation
last
week,
so
not
not
nothing
much
to
say
about
it,
but
the
only
thing
I
had
to
do
was
rebase
it
and
get
to
the
latest
code.
The
outstanding
issues
that
are
they're
they're
tracked
in
my
Wiki
and
I
got
before
I
went
on
vacation
I.
Think
I
I
got
a
buy
off
from
six
scheduling.
I
They
were
okay
with
one
way
has
commented
on
the
cab
saying
the
outstanding
remaining
issues
can
come
in
a
separate
PR,
so
it's
we're
at
two
out
of
three
now
or
two
out
of
four
I,
think
Tim
is
lgtm
and
six
Credit
Union
is
LG.
Tm
I'm
wondering
what
all
is
left
to
get
node
and
the
CRI
lgtms.
Thank
you.
Everyone
all
for
working
on
the
just
getting
a
guidance
out.
I
One
of
the
questions
that's
still
outstanding
is
how
we
should
look
at
when
you
request
a
resize,
especially
this
is
especially
for
memory.
If
we
cache
it
and
then
do
it
later,
I
think
I
I'm
not
comfortable
with
that.
If
there
is
a
series
of
updates
it
could
you
know
we
we
order
them
from
smallest
to
biggest,
and
if
it's
not
really
changing
the
limit,
and
it's
telling
us
yes,
we
did
it
or
succeeding.
C
C
C
I
I
think
what
this
boils
down
to
is
I
need
to
make
another,
a
small
change
commit,
which
updates
the
guidelines.
A
few
lines.
I
would
say
on
the
update
container
resources
API
as
to
what
the
CRI
expects
a
lot
more
document
of
what
we
expect
from
the
runtime
and
a
lot
more
detail
in
the
documentation.
A
I
Okay,
so
hopefully,
when
director
gets
back
from
vacation,
we'll
we'll
do
one
final
look
and
get
this
boulder
up
to
the
top
of
the
hill.
K
Yeah
I
just
wanted
to
update
with
the
help
of
munal
and
and
Mike.
We
did
a
another
round
of
reviews
and
at
this
point,
I
think
everyone
is
happy
with
the
checkpoint
container
code,
pull
requests
and
I
think
now
we
only
need
approved
from
from
Derek
if
I
understand
and
if
I
understood
correctly,
foreign.
A
J
Can
everyone
hear
me
sorry,
yes,
yeah
working
good?
Thank
you.
I
I
was
not
able
to
add
that
item
to
the
agenda.
I'm,
really
sorry,
I
didn't
have
any
privileges
on
that
doc.
Just
wanted
to
follow
on
the
email
that
we
sent.
I
think
that
was
last
Friday
was
really
not
a
long
time
ago,
but
we're
looking
for
some
feedback
here
on
the
SRO
2K
MMO
transition
for
the
management
of
out
of
trick
and
modulus
and
kubernetes
I'm,
just
not
quite
sure
how
we
should
move
forward
with
this.
J
F
A
Have
any
contacts
I'll
miss
my
sorry,
I
didn't
get.
What
did
you
asked
requested
here?
Maybe
we
could
move
the
discussing
to
the
second
note:
Channel
slack
Channel
there
yeah
I'm,
going
to
join
yeah.
C
I
I
guess
Clinton:
let's,
let's
move
it
to
signaled
and
like
the
channel
and
then
follow
up
next
week.
J
A
Thanks
thanks
and
also
Alexandra,
we
I
will
I
will
reply
to
you
about
your
question
on
continuity.
Yeah
see
you.