►
From YouTube: Coordinate Workloads Colocation: QoS-Oriented Scheduling Enhancement on K8s - Zuowei Zhang & Tao Li
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Coordinate Workloads Colocation: QoS-Oriented Scheduling Enhancement on K8s - Zuowei Zhang & Tao Li, Alibaba Cloud
Kubernetes provides well-defined QoS Classes on pod as guaranteed, burstable, and best-effort. Users can colocate different QoS workloads to achieve resource overcommitment and improve cluster utilization. However, with scale increasing and workloads diversified, some limitations are becoming more: · Lower QoS will be easily throttled or killed once node runs out of resources · The noisy neighbor problem effects the performance of latency-sensitive application · Local hot spots affect the global We implements Koordinator based on Kubernetes with several add-ons to provide QoS-oriented scheduling enhancements: · Definition of sub-QoS classes for complex workloads in co-location scenarios and compatible with the Kubernetes existing QoS semantics · Using dynamic metrics of nodes and pod to provide a more reliable model for resource overcommitment, including resource usage profile and micro metrics such as CPU scheduling, memory allocate latency · Applying fine-grained resource orchestration and isolation mechanism on node to solve the noisy neighbor problem and improve the efficiency of latency-sensitive workloads and batch jobs
A
A
My
name
is
Trump
Huawei
and
me,
and
my
colleague,
guitar
are
both
worked
in
Alibaba
cloud
and
besides,
we
are
also
the
maintenance
of
the
open
source
project.
I
mean
called
coordinator.
It
is
our
it
is
the
project
we
are
going
to
share
today
and
first
I
will
introduce.
The
I
will
introduce
the
first
three
part
of
the
slide
and
my
colleague
will
introduce
the
next
following
parts
and
speaking
of
the
location.
Let's
go
through
the
background.
A
Some
researchers
says
that
in
2020
the
data
center
industry
consumed
around
196
to
400
Tera
Watt
hours.
It
is
quite
a
lot
so
improve
the
resource.
Efficiency
is
really
important,
which
can
save
the
TCL
Hardware
cost
and
the
maintenance.
It
can
also
reduce
the
carbon
image
emissions,
but
a
lot
of
data
centers
utilization
are
pretty
low,
which
means
that
the
resources
are
wasting.
A
So
we
need
to
improve
the
resource
efficiency
of
data
centers.
Actually,
the
applications
in
data
centers
can
be
separated
into
two
kinds.
First
is
on
a
Services.
The
other
is
batch
jobs
for
online
service
workloads.
Most
of
them
have
strict
essay
about
performance,
for
example
the
99
percentile
response
time
and
which
means
that
they
require
really
high
quality
resource,
and
if
we
take
a
detail,
look
at
the
resource
usage
of
them.
A
But
there
are
some
challenges
in
kubernetes.
First,
take
a
look
at
the
resource
model.
Actually,
the
resource
requirements
of
pod
can
express
as
a
requests
and
limits
the
request.
Resource
means
the
container
can
guaranteed
to
get
and
the
limits
determines
the
resource
uses
never
goes
about
the
the
threshold
and
the
different
value
of
requests
and
limits
determines
the
cost
of
Parts.
A
Actually,
there
is
a
classic
word
can
conclude.
This
problem
called
noisy
neighbors.
So
why
now
is
the
neighbor
happens?
There
are
some,
and
there
are
lots
of
reasons.
First
and
most
of
the
resources
are
sharing
node
by
different
containers,
not
only
the
CPU
memory,
but
also
the
CPU
cache,
Network
block,
IO
and
so
on,
and
but
the
the
there
are
only
six
isolation
on
CPU
and
the
memory
for
kubernetes
container.
A
There
are
some
key
features
in
coordinator:
first,
each
design
it
data
designed
the
priority
and
kills
for
batch
collocation
scenario,
and
it
also
provides
application
profiling
magazine
for
resource
over
commitment
on
the
Node
level.
It's
coordinator
provide
a
fine
grid
resource
organization,
institution
and
acceleration
magazine
mechanism
to
avoid
noise,
enable
problem,
and
it
also
provides
flexible
job
scheduling
mechanism
for
diverse
workloads,
so
that
workload
can
be
easily
used
and
the
purpose
of
the
community
is
to
achieve
High
resource
utilization
with
safety
equals
guarantees.
A
The
architecture
of
coordinator
can
be
split
into
two
parts.
First,
of
course
it's
the
control
plane
and
there
are
three
parts
of
the
control
plane
that
connector
works
on
First
and
it
provides
independent
component
called
code
manager.
There
are
three
models
inside
and
the
first
AIDS
SEO
controller.
It
is
responsible
for
both
for
resource
or
sale
and
queues,
strategy
management,
and
also
there
is
a
com
model
called
recommender.
A
It
is
used
for
resource
profiling
and
auto
scaling
and
code
manager.
Also
has
a
web
hook
called
collocation
profile.
It
is
used
for
for
code,
Pro
protection,
Auto
injections
by
self-defined
roads,
for
example.
We
can
set
the
batch
drops
as
low
priority
to
avoid
the
need,
for
instance,
anode
besides
cornet
also
provides
some
plugins
inside
could
be
scheduler,
and
this
scheduler
to
improve
the
the
the
performance
of
batch
jobs
and
also
for
online
services
and
under
node
level.
A
A
There
are
two
key
designs
in
coordinator:
the
first
is
resource
priority.
The
connector
provides
results
of
a
sales
strategy.
It
means
that
they
requested,
but
not
used.
Resource
can
be
resold
on
lower
level
so
that
the
node
provide
can
provide
several
levels
of
resources
as
product
meet,
pension
free
and
they
are
expressed
as
the
extended
resource
are
not
status
and
the
more
aggressive
overseer
strategy
across
the
more
resources
can
be
over
committed,
but
the
quality
will
be
worse.
Now
we
can
take
the
we
can
look
the
graph
as
an
example.
A
The
red
line
is
the
uses
of
product
level
and
the
dark.
Green
part
is
the
capacity
of
batch
resources
and
the
light
green
part
is
the
capacity
of
mid-level.
We
can
see
that,
although
the
mid
level
has
has
better
stability,
but
each
capacity
is
quite
low,
comparing
with
the
batch
level
and
the
batch
level
has
has
more
resources,
but
it
is
not
very
stable
if
the
part
want
to
specifies
the
risk
resource
priority,
it
just
needs
to
fill
the
kubernetes
priority
class
fields
and
it's
product
mid,
batch
or
free.
A
It
means
that
it
won't
request
the
the
priority
of
resource
and
we
also
provide
the
sub
priority
on
Port
for
queue
and
distribution
during
scheduling
and
since
the
resource
is
over,
committed
pause
with
lower
lower
level
will
be
zero
total
Shield
when
resource
condition
happened
on
node,
the
other
key
design
is
skills,
and
it
is
an
extended
definition
on
Port
level
rights
system
that
is
sensitive
and
best
effort,
and
the
latest
system
can
be
divided
into
more
detailed
queues.
According
to
the
performance
requirements
of
applications.
A
The
difference
skills
means
the
different
resources.
Isolation
parameters
are
containers
and
when
resource
competition
happened
on
node
part
with
hierarchy
will
be
set
price
first
and
speaking
of
the
queues
and
quality.
There
are
two
independent
properties
and
can
be
mixed
used
and,
of
course,
some.
A
B
As
well
as
some
technical
details
inside
the
coordinate
coordinator,
can
flexible
user
called
question
to
support
a
multiple
workloads
here
we
take
us
back
as
an
example
to
introduce
how
to
use
how
to
run
Sparkle
on
kubernetes
with
coordinator
first,
let's
briefly,
introduce
how
tobacca
runs
on
kubernetes
many
euros
currently
use
the
spark
on
kubernetes
operator
to
manage
the
sparkle
applications.
The
spark
operator
defines
our
crd
named
spark
application
that
describes
the
sparkle
application.
Specific
specification
spark
operator
manages
the
life
cycle
of
Sparkle
application
on
kubernetes
cluster.
B
B
Executive
Port,
the
executive
public,
completes
the
task.
Weight
deployed,
spark
application
and
are
the
latest
system
applications
on
the
same
base
of
nodes,
the
latency
system,
applications
use,
kubernetes
native
resources
and
the
spark
applications
use
coordinator-based
resources.
It
can
improve
those
resource
utilization
of
the
cluster
and
reduce
the
cost.
The
yml
on
the
left
describes
a
standard,
specific
level
powder
is
created,
it
will
be
updated
by
the
collocation
profile
webhook
of
the
coordinator
component
and
there's
a
coordinator,
cues
class
will
be
added
to
indicted.
B
Okay,
next
I
will
start
with
use
some
some
technique:
technical
details
inside
in
inside
the
coordinator,
how
the
coordinator
of
the
committee
resources
and
how
to
you
ensure
the
runtime
quality
of
the
power
when
users
use
kubernetes
it's
difficult
to
accurately
evaluate
the
resource
units
of
applications,
and
they
don't
know
how
to
exercise
the
request
and
limited
accounts.
Therefore,
in
order
to
ensure
the
stability
of
the
applications,
larger
Resources
by
say,
vacations.
A
B
B
B
You
know
how
to
how
spouts
can
be
greater
workloader
performance,
The,
Limited
system
applications
and
the
best
workloads
complicated
more
on
the
unloads
with
high
utilization,
which
affects
the
runtime
or
craft
quality.
To
optimize
this
problem
coordinator
scheduler
provides
a
plugin
to
come
to
the
utilization.
The
plugin
mainly
depends
depends
on
the
amount
of
the
Matrix
reporting.
The
back
coordinator
during
this
gradually,
not
with
a
lot
of
hair
than
a
certain
screw.
Holder
will
be
voted
out
to
prevent
the
poles
from
the
from
being
unable
to
obtain
resources
on
such
nodes
with
high
load.
B
It
can
also
prevent
about
the
ways
already
high
load
from
continued
deteriorator
now
deterioration,
then
also
with
the
lower
utilization
can
select
the
duration
phase.
The
plugin
to
Avail
about
the
plugin
will
avoid
the
overheating
called
the
node
after
a
period
of
time,
due
to
the
current
schedule
of
to
many
posts
to
the
colder
notes
based
on
the
time
window
and
the
estimation
mechanism.
B
Utilization
is
a
toy
right,
relatively
high
water
level.
The
results
are
complicated
between
the
distance,
the
applications
and
the
best
efforts
of
workloads
will
be
more
serious
and
the
more
refined
fingering.
The
resource
management
is
required
to
better
guarantee
the
runtime
quality
so
as
to
further
explore
utilize.
This
potential.
B
At
the
beginning
of
the
design
coordinator,
we
design
we
designed
the
q's
class
for
colonics
scenarios
based
on
the
kubernetes
native
or
closed
class
and
the
restricted
the
way
the
excuse
class
uses
the
CPU,
LIC
and
osr
representation
system
for
applications
with
high
performance
requirements
which
use
CPU
second
model.
The
CPU
allocated
by
iosce
will
be
will
not
be
decided
by
workloads
of
other
classes.
B
The
CPU
are
actually
the
by
our
star
can
be
served
by
Vice
Network
accused
class,
but
cannot
be
started
by
Ireland
but
other
than
Q's
class,
otherwise
adopt
the
CPU
star
model
star.
A
group
of
common
United
States
listed
since
the
group
of
common
basis
of
assisting
CPU
similar
to
our
staff.
These
CPUs
can
be
can
also
be
served
bypass.
B
In
order
to
ensure
the
runtime
quality
of
virus
because
of
the
caucuses,
so
their
scenarios
coordinator
provides
rates,
skills,
enhancement,
capabilities
on
the
roadside,
because
the
time
is
limited,
I
sell
a
few
representative
capabilities.
First
thing
introduce
CPU.
Surprise
is
missing
before
the
applications
often
have
unlocked
but
unused
resources.
These
resources
can
not
only
be
used
by
newly
created
based
environment
posts
through
the
overcoming
commitment
mechanism,
but
also
can
start
the
idle
CPU
resources
to
best
ever
poses
that
are
already
running
on
as
much
as
possible.
B
I
assume
in
the
finger
when
correlation
decided,
resource
utilization
of
the
linking
stations
of
applications
is
low.
According
to
the
Matrix
and
the
CPU
use
step
by
the
best
ever
approach
has
not
exceed
the
safe
tester
holder.
Then
the
idle
CPU
within
safety
throughout
the
throat
can
be
served
with
the
next
level
post
so
that
the
best
levels
can
execute
the
faster.
Therefore,
the
load
of
the
limiting
systems
of
applications
increases.
B
Which
helps
the
best
level
of
workability
opportunity
to
reschedule
and
distribute
them
to
loads
with
load
load.
Cpus
they
satisfy
consists,
is
the
ratio
two
of
the
total
amount
of
CPU
actually
allocated
to
the
total
amount
of
CPU
expected
to
the
to
be
elected
once
the
CPU
satisfaction
of
best
adverted
workload
is
lower
than
the
source
and
the
CPU
utilization
of
best
level
workloads
and
existing
Exchange
19
coordinator
will
effective
some
low
priority.
Best
effort
poses
and
release
some
resources
for
use
by.
B
Coordinator
provides
a
variety
of
adaptive
kernel
level.
Cubes
management
mechanisms
like
such
a
circus
CPU
burst,
allows
CPUs
rotor
data
to
rotate
the
containers
to
boost
their
CPU
utilization
for
hair
performance
and
the
low
lower
linkage.
This
filter
can
ensures
the
quality
of
cleaners
without
reducing
the
deployment
density
of
containers.
B
Correlated
adaptive
is
always
analysis,
OS
and
the
community
version
of
the
Linux
kernel
because
of
time,
Contents
I
will
not
give
a
detailed
detail.
The
expression,
if
you
are
interested
you
can
go
to
our
official
website
for
more
information,
also
call
the
scheduler
provides
a
variety
of
scheduling,
future
exercise,
a
house,
the
code
scheduling
it
can
it's
a
community
completed
with
a
cost
scheduling
and
the
corporate
group
of
CRT
and
the
support
multiple
product
group
to
complete
the
gun,
scheduling
as
a
group
quarter,
scheduling
comfortable
with
with
compensated
scheduling
and
the
elastic
contest.
B
Okay,
a
coordinator
has
released
the
seven
versus
implementing
most
of
capabilities
required
for
collocation
scenarios,
including
results
over
commitment,
CPU,
surprise,
five
grandestive
human
management
is
this
scheduling
and
the
elastic
contract
scheduling
and
all
we
have
a
private
provided,
a
lot
of
documents
of
the
official
website
and
GitHub
provided
and
the
best
products
is.
If
you
are
interested
you
can
try
your
coordinator.
You
are
warmly
welcome
to
hack
our
coordinator,
welcome
to
report
interest
to
improve
documentation,
documentation
fix
bugs
or
other
filters.
We
encourage
all
contributors
to
become
members.