►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
A Toolkit for Simulating Kubernetes Scheduling at Scale - Yuan Chen, JD.com
As Kubernetes becomes the de facto standard for container orchestration, new scheduling algorithms and systems are made for different scenarios and workloads. Unfortunately, it is very time and cost consuming to evaluate new schedulers or features in real K8S clusters at scale. We present a simulation toolkit, which can simulate large-scale K8S clusters and scheduling using a single machine plus a small number of containers. The simulator runs a real K8S master and schedules pods according to event traces generated from real K8S clusters. It provides a complete set of metrics, including resource utilization, detailed scheduling trace and performance metrics, enabling developers to evaluate the scheduling behavior and performance with a reasonable amount of confidence. We have used the toolkit extensively to optimize the scheduler for large scale K8S clusters (~10K nodes) at JD.com.
https://sched.co/UaVk
A
All
right,
it
is
for
25,
so
we're
gonna
go
ahead
and
get
started.
Welcome
everyone.
My
name
is
Robert
Bailey
I
work
at
Google
and
I'll
be
your
track
host
for
this
session.
Today,
I
have
the
pleasure
of
introducing
introducing
you
want
Chen
who's
gonna
talk
to
us
about
how
we
can
use
a
toolkit
to
do
some
performance
testing
of
large-scale
kubernetes
clusters
without
actually
having
to
spin
up
large-scale
coordinated
clusters.
So
please
everyone
give
a
warm
welcome
to
un
chen.
I'm
get
started.
B
Sang-Soo
Kimmy,
ok
good
afternoon,
welcome
to
this
session.
I
know
this
last
and
technical
session,
so
I
really
appreciate
and
your
attendance
so
I'm
you
enter
from
JD
calm.
This
is
a
joint
work
of
the
microRNA
cachito
and
Haiphong
and
Guang
and
Jen
gong.
So
let
me
first
stand
up
an
army,
and
probably
not
many
people
and
ya
know
that
your
ad
don't
come
and
so
I
just
would
like
to
spend
a
few
and
the
seconds
talk
about
it
so
JD's
and
largest
er
and
online,
and
off
my
retina
and
Internet
company.
B
Actually
by
revenue
you
probably
heard
Alibaba
and
other
company,
but
by
revenue
we're
not
just
one
in
addition
to
that,
JD
also
and
provide
analogist
the
logistic
and
a
fulfilled
network
and
nationwide
in
China,
and
we
build
and
very
good
and
a
strategic
partnership
with
Google
and
the
Tencent
and
all
that
so
I'm
part
of
this
in
the
JD
retail
and
infrastructure
team.
So
you
can
imagine
that
behind
the
base
and
the
business
and
we
are
technology
and
treatment
company.
B
So
we
are
one
of
the
earliest
lot
of
the
kubernetes
and
container
technology
and
right
after
Google
announced
and
open
source
kubernetes
in
and
later
2015,
and
we
start
and
yeah
looking
at
kubernetes
and
gradually
and
adopted
and
when
transitioned
our
business
and
application
to
the
kubernetes
in
a
baseless
system.
So
we
join
us
SF
and
as
and
remember,
and
also
wonder
and
user
award
last
year
at
Q,
County
Shanghai,
and
why
I
want
to
give
you
one
number
and
this
infrastructure
or
containerized
infrastructure
is
the
foundation,
for
example,
support
our
just
a
pastor.
B
We
call
it
a
single
day
and
major
in
the
Seri
vent.
The
single
day
is
November
11
right,
a
4
and
a
1,
and
so
you
can
see
this
and
the
Zeus
number
yeah
around.
Oh
yeah,
it's
the
30
and
pinioned
us
daughters.
So
this
is
huge.
Yes
and
QPS
is
about
an
acre
150
million
the
aquarius
per
seconds.
And
yesterday,
if
you
attended
and
keynote
talk
plenty
to
skew
in
a
city
of
sugu
mission,
elaboration
use
the
Vitesse
and
build
some
in
st
the
database
and
that's
the
number
we
have
so.
B
But
this
and
is
basically
and
hi,
never
architecture
and
of
entire
and
isn't
the
software
or
technologist
stack
so
I'd
like
to
the
top-down
that
give
your
ideas
when
I
can
mention
Ernie
JD.
We
have
some
online
and
of
course,
our
nice
yoking
services,
and
we
also
have
this
logistic
thing
and
also
have
a
separate
and
the
units
and
I
just
look
at
all
this
and
apply
and
advanced
AI
and
motion
any
technology
or
solve
the
problem.
B
We
have
the
AI
workload
and
overseas
and
we
have
the
site
and
in
Thailand
and
Indonesia
also
we
have
some
offline
retail
store.
We
quite
a
semi
fresh
for
this
24-hour
write
provided,
isn't
afresh
and
across
the
service.
So,
of
course
our
workload
is
very
diversity
and
we
have
this
and
online
micro
services,
and
we
also
and
Ronnie
or
not,
for
the
big
data
analytics
machine,
any
workload
and
a
I
walk
load
because
of
this
Center.
B
But
the
core
is
our
lease
and
the
JD
and
highly
customized
and
optimized
and
container
and
kubernetes
and
platform,
and
we
call
it
JD
dates
and
OS
is
basically
is
a
customized
version
of
the
kubernetes.
So
it's
support
horace
and
kind
of
the
different
and
the
workloads
and
all
of
the
JD
s--
business.
B
So
you
can
imagine
during
our
journey
right
and
we
started
using
the
very
early
version
kubernetes
and
we
start
from
a
few
hundreds
of
the
nodes
per
cluster
to
today
and
our
one
of
the
largest
cluster
kubernetes
cluster
we
ever
and
have
more
than
ten
thousands
and
the
physical
nodes
and
CSF
and
recognize-
and
we
are
ronnie
one
of
the
largest
network
that
is
cluster
and
in
production
in
a
world.
So
this
is
hard
and
we
we
try
to
and
optimize
something
we
add
a
new
feature.
B
We
develop
a
new
schedule,
a
how
to
evaluate
right,
Vani
data
defect
this.
So
this
always
a
very
challenging
a
problem,
and
so
but
the
beauty
thing
is
the
right
and
in
the
scientific
study,
research
and
dissemination
is
always
very
useful
in
the
tool.
So
here
we
do
catheters
can
we
average
and
take
advantage
and
by
using
some
kind
of
dissemination,
technical
to
help
us
to
evaluate
validate
and
our
software
and
the
new
features
and
the
scalability
performance.
B
So
if
we
do
catch
the
the
systems
and
the
cluster
management
rights
that
are
from
the
kubernetes
and
or
mesos
order
had
to
be
young
and
we
started
looking
around
and
we
did
some
in
the
study
and
we've
found
out.
Definitely
they
they're
on
the
numerous
of
the
to
our
software
and
available
and
do
some
kind
of
dissemination
or
emulation,
but
unfortunately
yeah.
If
we
really
want
to
and
the
two
very
and
realistic
or
complexed
and
the
simulation
scenario,
for
example,
you
can
replay
the
real
trees.
B
You
also
can
create
here
some
what,
if
analysis
kind
of
thing-
and
you
not
only
and
do
this
and
performance
and
evaluation
or
the
metrics.
You
also
want
to
look
at,
for
example,
your
schedule
and
the
placement
at
the
coordinate
he
and
the
finally
needed
to
provide
some
good,
visualization
or
dashboard
to
help
you
and
understand
right.
The
behavior
and
the
behind
it
as
curry
and
naan
them
actually
can
meet
all
this
requirement.
B
B
Given
the
more
detail
about
how
this
simulation,
the
architecture,
design
and
the
sum
and
implementation,
the
details
and
I
will
also
in
the
describe
present
a
few
and
the
case
studies
and
we
how
we
use
this
in
the
simulation
tools,
help
us
and
optimize
improve
our
kubernetes
and
the
largest
skill,
and
that
is,
and
a
massive
and
skill
and
kubernetes
cluster
and
the
efficiency
and
reduce
the
cost.
And
if
I
have
time
and
yeah
I'd
like
to
give
a
very
and
a
brief
demo
and
I
will
be
here.
B
B
Simulate
define
a
configuration
file,
that's
the
the
event
side.
Similarly,
and
we
provide
and
the
two
kind
of
the
modes.
The
first
one
is
just
the
reply:
correctable
events
from
your
Rio
and
clusters
and
replayed
the
events
and
traces,
the
second
one
and
is
you
can
create
here
in
the
sum
and
the
the
synthetic
and
the
workloads,
our
three
C's
and
then
play
this
and
the
three
C's
and
do
some
kind
of
what
he
Fernandez
is.
B
We
support
both
ways
and
online
and
off
mind
a
kind
of
dissemination
in
my
case,
study,
I,
wish
you
why
examples
to
the
offline
and
scheduler
and
performance
II
variation?
Otherwise,
basically,
we
we
use
this
simulator
and
to
the
Kentucky's
online
performance
and
order
decision
and
making
and
evaluation.
B
Finally,
is
our
lesson:
dissemination
platform
itself
is
managed
and
deployed
by
the
kubernetes,
see
it
yourself.
So
it's
be
easy
to
still
dance
Europe
operator.
So
this
is
the
architecture,
and
so,
as
you
can
see,
there
is
so
we
have
the
our
Rio
and
cluster
Romney.
So
there
are
two
things
we
cracked
from
now
real
cluster.
So
the
first
thing
is
always
and
the
interesting
events,
so
we
we,
we
cracked
it
and
receiving
a
database,
and
you
know
cases
the
influx
DB.
B
The
second
one
is
always
and
a
configuration
and
the
related
information,
and
this
will
be
used
to
populate
and
created
our
kubernetes
and
simulated
kubernetes
configuration.
So
the
one
component
interesting-
and
here
is
this
in
the
simulated
and
nodes,
and
we
use
this
and
more
clothes
and
also
each
more
clothes,
and
that
basically
is
simulated
and
kubernetes
and
a
node
but
multiple
materials
run
inside
a
container.
B
Another
thing
is
because
you
connect
the
data,
this
time,
step
and
other
thing
and
need
to
be
in
a
somehow
calibrated,
and
so
we
have
some
tools
and
make
sure
and
the
Odyssey
information.
They
are
consistent
view
because
you
could
collected
from
all
different
clusters
in
different
time
zone.
Other
thing
we
have
to
do
that
so
finally,
you
know
we
have
very
nice
and
the
dashboard
and
visualize
and
attract
all
this
information,
not
about
the
performance.
We
have
some
heat
map
and
the
visualization
about
the
ecology
and
a
later
and
yeah
I
wish.
B
You
know
some
more
details
about
that.
So
let
me
just
go
through
each
of
the
step
and
in
a
little
bit
and
more
details,
and
so
firstly
about
events.
So
the
events
and
like
I
mentioned
early
and
that,
firstly,
we
can
collect
and
yeah
very
deed
her
events
and
from
each
of
the
system.
Now
there
are
tons
of
tools,
could
help
you
and
do
that
and
just
I
think
two
o'clock
no
session
and
I
attended
a
session
talked
about.
The
events
are
not
interesting.
B
Use
cases
actually
listen
that
could
be
used
cases
as
well
correctly,
events
and
the
do
dissemination
so
so
far
we
implementation
and
we
collect
the
data
is
applied
and
the
agency
sent
a
telegraph
and
that
we
store
orders
in
the
data
in
the
influx
DB.
Of
course,
they'd
standard
mean
it's
the
past
and
implementation
and
that's
just
our
choice
at
their
time.
That's
the
best
one
available
for
us.
So
probably
it's
not
send
a
very
and
yeah
it's
readable
and
yeah,
but
the
slicer
is
available
and
yeah
online.
B
You
can
look
at
more
details
and
this
is
and
the
event
or
configuration
and
the
information
that
we
cracked
there.
The
second
one
is,
and
we
also
provide
a
like
a
synthetic
and
the
twist
generate.
You
can
define
the
different
kind
of
a
scenario
right.
You
can
define
the
cluster
size
or
the
scale
and
you
can
define
each
node
and
what's
the
capacity,
a
resource
and
a
CPU
core
and
memory
and
disk
space
and
etc,
and
then
for
the
event
generation.
B
So
that's
very
helpful
if
you
really
want
to
do
some
client
what
he
Fernandez
is
if
they
are
port
and
coming
and
require
and
I
could
once
all
then
the
contenders,
so
what
the
scheduler
and
at
the
behavior
we
look
like
if
there
are
five
kind
of
these
jobs
and
coming
and
what
the
cases
could
be.
That's
very
useful
for
us
for
the
our
de
single
day
in
a
major
Syria
right.
We
want
do
this
in
a
stress
testing.
Sometimes
we
create
some
kind
of
a
scenario.
B
We
want
to
evaluate
the
performance
scalability
and
identify
any
resource
portal
net
and
in
our
system,
so
that
no
dissimilar
issue
and
the
the
concept
under
here
is
also
very
simple,
and
so
we
we
basically
use
kubernetes
and
a
start.
A
bunch
of
you
can
specify
and
the
a
bunch
of
the
port
or
containers
so
for
each
container,
then,
depending
on
the
the
resource
capacity,
and
you
could
endure
start
and
Semin
eight
from
attend
to
more
than
100
of
the
kubernetes
and
nodes.
B
So
then,
this
kinase
and
kubernetes
and
the
basically
just
the
mimic,
the
real
behavior
and
of
the
mokuba
net
right.
It
will
interact
with
the
API
server
and
interesting.
Also
in
the
wachatis
events.
It
also
reports
and
anaconda
does
resource
usage.
Other
theme
so
from
the
api
server
or
a
master
controller
perspective
is
just
like
a
real
and
mokuba
net,
of
course,
and
this
in
the
simulated
and
a
fake
hoobnet,
and
that
does
not
the
real
and
educator
or
start
any
real
contenders.
B
B
We
provide
some
commander
9/2
and
also
you
can
import
import
and
appropriate
to
the
configuration
trees.
This
also,
you
can
export
and
origin
that
results
or
simulation
results,
as
well
as
your
your
configurations
so
the
visualization
later.
And
why
should
a
demo-
and
you
probably
can
get
a
better
idea
like
here-
probably
not
very
readable
but
anyway.
B
So
we
have
all
kinds
of
nemetrix
and
from
the
performance
like
the
API
service
performance
and
the
scheduling,
performance
and
also
scheduling
and
the
placement
of
the
corn
adhere
and
all
kinds
of
this
and
the
current
estates
have
a
cluster.
Also
with
reports
and
I
have
a
customized
and
the
range
definition
and
the
query
interface
and
you
can
search
and
a
Duke
a
Taurus,
maybe
I,
just
the
skill
to
router,
for
example,
here
and
the
folder
for
the
scheduler,
and
we
could
and
break
down
the
into
for
each
stage.
B
And
when
you
do
this
and
the
filtering
right
at
the
first
stage,
you
choose
the
feasible
nodes
and
we
we
have
this
in
the
heat
map
and
about
this
and
the
latency
distribution
about
Yoda's
and
the
predictor
and
yeah
and
performance,
and
also
the
second
stage,
is
about
an
ad
agency.
You
do
the
scheduler
duties
in
the
ranking
or
scoring,
and
each
of
the
nodes
right.
B
You
want
and
choose
the
pass
the
nodes,
and
we
have
very
detailed
and
the
the
distribution
information
about
that
or
the
overall
and
the
the
performance
of
latency
about
your
scheduler
and
also
the
I
think
this.
Why
is
yeah
the
binding
time
if
you
are
familiar
with
the
kubernetes,
the
scheduler
you
probably
know,
and
so
the
basically
there
are
two
stages
right
wise
make.
The
decision
the
second
and
is
an
abiding
is
basically
in
a
put
into
the
if
you
sue
and
at
CD
and
the
database.
B
B
Okay,
so
basically
the
in
a
quick
summary-
and
so
this
too,
and
we
enable
the
user
and
can
can
collect
and
replay
and
the
real
tree
sees
in
the
events
from
real
and
clusters,
also
in
a
customized
defiant
and
the
trees
or
the
configuration
and
of
the
cluster.
Now
we
can
run
the
replay.
This
tree
sees
and
collect
the
detail,
information
about
an
API
service
performance
and
as
well
as
the
some
scheduler
and
very
detailed
metrics,
help
us
understand
the
behavior
or
the
performance
and
behind
it
a
screen.
B
B
What's
the
main,
a
bottle
networking,
that's
the
API
server
or
always
and
related
and
masters
or
at
CD,
and
so
firstly,
in
this
to
help
us
understand
that,
because
which
one
isn't
real
and
at
CD
they
are
rights
and
even
hoobnet
itself
and
does
not
in
the
start
or
launch
container.
But
all
this
behavior
interact
with
this
and
master
node
is
real.
So
this
chaos
understand
so
for
this
experimenter,
for
example,
we
started
and
like
no
18,000
and
contenders
and
act
as
the
mock
servers.
B
So
then
each
the
smoke
servers
and
we
usually
to
simulate
and
one
hundreds
of
the
about
one
hundredth
of
the
real,
the
port's
in
all
simulated
the
environment.
So
this
inner
voice
and
a
basically
simulate
over
and
ten
thousands
of
Frisco
nodes
and
about
two
hundred
thousands
of
the
containers.
So
then
we
also
set
hopin
I'd,
like
the
six
and
the
API
server
for
some
dependency,
also
high
availability
and
stuff.
B
So
at
the
beginning,
now
of
course,
in
the
at
CD
and
has
done
a
lot
of
the
work
and
I
was
on
e
Baba
on
the
other,
yeah
I
think
a
players
and
to
try
to
optimize
they
to
perform
because
LCD
and
originally
to
realize
this
and
the
five
gigabytes
or
ten
gigabytes
and
the
storage
in
limited.
We
be
yeah
the
problem
for
large-scale
deployment
and
as
a
memory
education
here
with
some
other
things,
will
become
a
botnet.
B
So
one
improvement
organization
we
did
and
when
we
start
into
the
SKU
out
our
kubernetes
cluster,
we
just
use
multiple
and
AB
CD
clusters
and
to
store
different
resource
type
and
objects.
So,
for
example,
for
very
popular
like
this
poor
creation
and
upon
resources
objects,
we
use
a
separate
and
F
CD
and
class.
The
storage
other
we
use
or
separate
will
also
apply
some
compression
and
put
a
cache
layer
and
radius
and
in
front
of
the
LCD,
but
doing
always
and
we
have
to
validate
and
evaluate
and
validate
and
whether
or
not
it
works.
B
B
So
let's
give
our
sort
of
confidence
and
also
at
our
very
ending
our
very
early
work
and
we
do
get
a
scheduler
and
we
did
some
optimization
and
adjust
and
optimize
the
scoring
function
and
we
improve
our
schedule
QPS
from
the
two
and
a
to
the
forty,
and
we
also
use
this
and
the
simulation
to
help
us
to
eveni
to
to
evaluate
it.
So
the
second
and
the
study
is
what's
and
our
team
and
have
just
the
town
recent
any
and
so-
and
I
give
you
some
context
so
known
at
within
JD.
B
We,
like
a
mention
ernie
and
we
not
only
run
on
I
services
and
and
the
traditional
right
hander
stated
these
services
and
are
now
we
move
in
there,
because
the
improve
the
utilization
cause,
scheduling
co-locate
to
the
online
services
and
iranica
patch
jobs
and
big
data
and
workloads.
So
in
a
community-
and
there
are
not
one
GUI
work
at
a
very
active
like
cuba,
batch
and
listen
Betty
over-scheduling
and
like
the
spark.
What
was
wrong.
The
spark
under
the
the
Cuban
art
is
no
wonder
properly.
We
found
out
is
the
performance
skin
ability.
B
So
obviously,
sometimes
they
notice
in
the
smoke
jobs
and
it
requires
nakedness
and
to
thousands
or
several
thousands
of
the
small
contenders
and
wrong
a
batch.
Probably
we
lost
in
a
few
minutes,
but
the
existing
and
Kubb
kubernetes
and
the
schedule,
and
if
you
are
familiar
with
it,
it
conducted
scheduled
scheduling
and
a
poured
by
Porter,
so
we
find
only
through
pooed
and
a
Dayton
C
and
is
not
acceptable
if
we
want
to
support
this
and
yeah
high
throughput
and
a
kind
of
workload
compared
to
the
traditional
young.
B
So
we
did
some
kind
of
an
optimization
for
it
very
interesting.
We
can
talk
and
so
in
Q
Shanghai
cube
car
I
gave
a
talk
and
if
you
go
back
there
and
we
have
some
information
there,
basic
idea
is
we
just
do
this
and
the
scree
that
is
quality
for
the
performance.
So
we
don't
need
to
choose
the
best
one
right
we
can,
because
this
and
the
requirement
1000
or
2000
contender
there
requirement
a
very
similar.
So
we
coulda
just
do
a
group.
We
quite
a
group,
scheduling
and
group
ID.
Just
do
it
once
so.
B
We
can
approve
the
significantly.
So
during
our
this
and
optimization
and
demand
fermenter
and
the
the
face,
so
we
we
went
through
a
quite
a
few
phase.
So
how
do
we
evaluated
the
effectiveness
of
our
Arizona?
So
we
use
our
discrimination
tools
so
from
our
regionally
and
we
we
create
and
disseminate
and
cluster
just
use
a
few
or
tens
of
the
containers.
So
simulator,
like
thousands
of
nodes
and
Romney
like
tensor
close
to
ten
thousands
of
the
the
the
port
and
each
of
the
scheduler
to
over
then
require
and
five
hundreds
of
the
reports.
B
So
here
the
huge
number,
as
we
keep
in
the
improving
our
areas
on
how
we
significantly
improved
scheduler
performance.
So
once
we
reach
this
stage-
and
we
also
did
us
all
the
functionality,
scalability
stability
and
evaluation
useless
finding
in
already
deployed
a
sand,
yeah
optimal
schedule
into
our
system,
so
I'm
talking
in
a
working
delivered,
yeah,
the
cube
and
volcano
and
community
and
the
Seahawk.
We
can
actually
open
source
always
improved
and
schedule
and
to
make
it
available
to
the
community.
B
So
last
one
that
I
want
to
also
highlight
one
of
the
key
and
the
feature
or
interesting
scenario.
Is
we
we,
we
not
only
and
use
it
for
this
offline,
the
evaluation
and
when
we
yeah
have
a
new
areas
on
a
new
schedule
or
other
thing
hand.
So
we
also
in
the
incorporate
into
our
online
decision
in
particularly
for
you
know,
and
an
iconic
JD
I
mentioned.
B
We
have
very
large
skill
for
honest
and
online
services
and
like
a
willing,
Google
and
also
want
to
differentiate
different
kind
of
workloads
so
for
the
very
non
Romney
and
performance
critical
one.
We
come
forward
and
write
some
kind
of
this
and
the
long
wrongly
and
the
decision
to
justify
that
the
pastor
and
the
placement
and
make
sure
everything
in
the
working
in
a
file
so
but
it's
very
hard
to
come
up
with,
like
a
closed-form
kind
of
magic,
examine
the
fight.
B
That
is
been
packing
other
thing,
so
we
basically
and
when
we
do
this
and
decision-
and
we
use
this
in
the
simulator
and
we
evaluate
our
decisions
that
only
and
we
get
some
confidence.
This
work,
fine
and
then
we
you
actually
end
the
deployed
or
execute
and
yeah
this
and
a
decision
or
yeah
take
the
actions.
So,
for
example,
here
is
one
that
we
use.
It
predicts
some
of
the
memory
usage
and
then
based
on
that
we
consolidate
or
rescheduling
the
the
pause
and
the
contenders.
B
So
the
results
here,
and
is
we
only
without
a
consolidating
and
the
Nagre
Kooskia
Julie
patch
and
the
the
nice
service
just
the
wrong
the
online
services
necessary
latency
and
the
intensive
or
sensitive
wine
that
we
can
improve
this
and
the
memory
usage
you
average
you
want
to
61%
without
and
basically
we
are
mostly
eliminate
all
this
out
of
the
memory
and
yet
arrows
and
my
best
knowledge
in
the
yeah
compared
to
the
literature
and
reports
and
are
not
having
a
ton.
Less
is
a
yeah,
a
very
good
and
results
tandem.
B
B
So
now,
in
the
way,
under
working
and
very
closely
with
the
sync
scheduling
and
a
working
group
and
I'll
try
to
make
it
and
yeah
available
to
the
community
open
source
release
it
so,
since
yeah
I'm
running
out
of
time,
I
really
want
to
be
when
yeah
five
minutes
for
for
for
the
QA
and
I
have
a
demo
if
you're
interesting
and
if
you're,
interested
and
I
can
show
you
and
after
talk.
Okay
again,
thank
you
very
much
yeah.
So
this
is
my
contact.
Information
and
I'd
like
to
take
yeah,
maybe
a
couple
questions.
B
So
so
so
far
and
yeah-
that's
good
on
it,
because
our
real
and
the
API
server
and
the
master
ID,
which
one
the
real
one
so
so
we
were,
is
that
that's
easily
one
we
deploy
a
latest
one,
so
the
only
feeds
the
distant
accrued
net
and
all
assuming
it
hit
a
one,
but
to
our
best
our
experience
and
yeah
the
major
and
upgrade
who
new
release
and
the
goodness.
Because
for
us
and
what
we're
most
interesting
events
is
where
you
create
a
new
one.
B
You
T
need
a
new
events
and
all
this
new
and
the
features,
for
example,
in
the
upgrade
and
seems
done
knots
and
change
margin
for
our
lists
and
Morgan
silver,
but
they
are
needed.
Some
changes
that
require
us
and
are
just
out
this
mock
node
and
API
the
interaction
with
the
API
server.
So
the
short
answer
is
yes:
we
have
the
keeper
iron
on
the
upgrade
of
the
new
version.
So
so
far
we
are
not
always.
We
still
have
not
supported
latest
one.
So
far
we
supported
1.14
and
yeah.
So.
C
Mm-Hm,
yes,
I
I
have
a
question
here:
I'm
trying
to
understand
how
accurate
is
this
model
to
understand
how
the
scheduler
behaves
in
kubernetes
the
reason
that
I'm
trying
to
understand
this
is
because,
if
you
have
a
bare
metal
server
that
has
let's
say
hundred
cubelet
instances
as
long
as
one
single
cubelet
instance
is
running
an
application
that
is
consuming
80%
of
the
CPU.
The
other
cubelet
instances
obviously
are
gonna
see
80%
of
the
CPU,
so
your
scheduler
is
gonna.
C
Go
crazy,
saying
you
know
what
I
have
a
bunch
of
cubelet
instances
here
that
I've
seen
you
know
there
are
seen
80%
of
CPU,
because
the
process
is
not
it's
a
virtualized.
So
how
are
you
going
to
deal
with
that?
How
are
you
going
to
make
sure
that
your
scheduler
doesn't
react
in
those
scenarios?
Yeah.
B
And
that
that's
good
question,
so
so
that's
the
one
I
think
the
limitation
when
he
meets
I
shouldn't
mention
them.
So
as
you
can
yeah,
you
ask
a
question.
So
our
cube
net
because
we
lucubra
had
had
a
simulated
couplet
and
it
does
not
allow
any
container
and
it
does
not
run
and
his
real
workload.
So
we'd
reported
a
base.
It
adjuster
your
request
right
and
at
a
capacity
so
but
most
of
the
current
had
needs
the
existing
schedule
and
make
the
decision
mostly
is
based
on
that.
B
B
But
we
we,
we
think
about
that
and
we
think
in
a
one
way
and
we
couldn't
also
be
introduced
and
some
kind
of
distraction,
the
utilization,
the
into
our
the
simulated
environment
and
so
that
we
can
buy
it
and
also
can
treaty
there's
some
kind
of
assimilating
environment
you
some
kind
of
it
is
not
only
just
your
request
and
the
capacity
is
the
actual
resource,
utilization
or
usage
of
your
system.
So
but
the
Hana
fee
is
then
your
schedule
make
a
decision
and
how
it
impacts
yeah
and
the
usage.
B
So
that's
an
attorney
giving
you
could
use
some
kind
of
in
a
model
and
the
based
on
the
historical
data
you
created.
For
example,
I
know
you
actually
CPU
cycles,
usage
and
now
I
know
you
make
this
decision.
I
I
could
do
some
calculation,
but
how
accurate-
and
this
could
be-
and
yes,
the
open
question
so
so
so
so
far
and
yeah
for
this,
if
you
need
a
some
in
the
real
measurements
which,
based
on
the
real
workload,
the
behavior
yeah
so
fun
is
to
understeer
cannot
to
do
that.
Yes,
we.
B
Yeah,
so
that's
why
the
it's
probably
somehow
it's
an
emulation
had
or
something
yeah
so
far
we
just
because
this
and
the
traces
and
the
behavior
and
yeah
if
we
replay
the
real
tree,
says
yes
so
far.
We
replay
it
here
to
the
scene
pace
as
the
regional
one,
but
that's
a
very
good
point
and
yeah
for
some
scenario
in
particularly
for
this
and
watching
what-if
analysis.
I
think
it
depends
really,
depending
on
your
use
cases
and
the
cases
yeah.
For
some
cases.
It's
really
matters
and
yeah.
B
How
fast
and
I
guess
either
your
the
inter
arrival
and
the
workloads
are
coming,
but
there
might
be
some
cases
and
if
you
want
even
evaluate,
for
example,
some
stable
beneath
here
other
thing
yeah.
If
we
can
the
support
and
the
faster
pieces
and
that's
maybe
and
I,
think
the
very
nice
features
yeah
all.