►
Description
Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Network-aware Scheduling in Kubernetes - José Santos, Ghent University
Recent applications are latency-sensitive, demanding low latency between microservices in the application. Current scheduling algorithms in Kubernetes aim to reduce costs and increase resource efficiency, which is not enough for applications where end-to-end latency becomes a primary objective. Applications such as databases and multi-tier web services would benefit the most from network-aware scheduling policies that consider latency and bandwidth in addition to default resources (CPU and memory). We introduce a network-aware scheduling framework to tackle this challenge, including two controllers (AppGroup and NetworkTopology) and three scheduling plugins (TopologicalSort, NodeNetworkCostFit, and NetworkMinCost). The framework ensures bandwidth reservations and optimizes the end-to-end application latency since it schedules pods in an application with chained dependencies close to each other. We will show a demo highlighting the benefits of our framework.
A
A
Today,
I'll
talk
about
more
about
the
motivation.
Why
do
we
need
network
aware
scheduling
in
current
kubernetes
clusters
I'll
try
to
show
you
the
main
building
blocks
that
we
currently
have
in
our
network?
Aware
framework
I'll
try
to
explain
you,
some
of
the
implementation
details
and
some
of
the
considerations
that
we
have
in
our
framework
and
I'll
try
to
show
you
a
complete
example
on
how
you
can
use
our
framework
to
deploy
your
applications
in
a
kubernetes
cluster
trying
to
find
better
decisions
in
terms
of
network?
A
Aware
placement,
then,
if
the
demo
gods
and
the
wi-fi
help
me
I'll,
try
to
show
you
a
live
demo
by
deploying
the
online
boutique
application,
which
is
a
typical
application
used
in
microservice
research.
So
it's
a
web-based
application
with
several
workloads
and
by
deploying
this
application
with
the
default
scheduler
and
with
our
approach
our
network
aware
framework,
you
will
see
the
differences
that
you
get
by
deploying
this
application
in
kubernetes
cluster.
A
So
typically,
when
we
started
working
on
network
aware
scheduling,
we
noticed
that
the
kubernetes
scheduler
typically
mainly
focuses
on
resource
efficiency.
So,
typically
you
as
a
developer,
you
try
to
set
up
cpu
and
memory
requests
and
limitations
so
that
the
scheduler
can
find
better
decisions
for
your
pods
and
that's
particularly
important
when
you
are
focusing
on
energy
efficiency
but
for
certain
applications
like
latency,
sensitive
iot
applications
and
video
streaming
applications.
A
So
currently,
there
are
very
few
plugins
scheduling,
plugins
in
the
kubernetes
architecture
that
you
can
use
with
contextual
awareness,
so,
for
instance,
when
you
have
several
workloads
in
your
application,
typically,
the
scheduler
allocates
these
workloads
independent
of
each
other,
so
they
have
not
a
knowledge
of
which
workloads
I
will
contact
to,
or
I
will
communicate
to
and
also
there
is
no
knowledge
on
the
infrastructure,
topology
meaning
what
is
available
bandwidth,
let's
say
in
your
cluster
between
cluster
nodes.
What
is
the
communication,
latency
or
delay
that
you
have
on
your
network
links?
A
So
there
is
not
this
type
of
information
available
currently,
so
we
thought
about
how
can
we
do
better?
Well,
the
easy
answer
is
to
say
to
consider
some
sort
of
latency
or
network
bandwidth
metric
in
the
scheduling
process.
But
how
can
we
do
this?
There
is
a
lot
of
research
on
these,
which
typically
involve
a
theoretical
modeling,
where
you
can
handle
all
the
constraints,
all
the
complexity
concerning
tragedy
of
resources,
network
latency
among
cluster
nodes,
also
bandwidth,
but
typically
to
find
the
optimal
location
scheme.
A
It
takes
several
hours
to
find
it,
which
is
not
scalable
in
production
environment.
So
how
can
we
address
network
aware
scheduling
in
let's
say
operational
environment
like
kubernetes,
so
we
have
proposed
a
framework
where
we
are
essentially
considering
two
main
aspects:
application
characteristics,
because
we
believe
that
micro
service
dependencies
play
a
major
role.
So,
for
instance,
you
have
here
two
example:
applications
the
online
boutique
application,
the
redis
cluster
one,
and
you
can
clearly
see
that
you
have
several
workloads
here
and
they
communicate
to
each
other.
A
So
when
you
are
deploying
one
of
these
workloads
in
the
scheduler,
you
should
be
aware
of
that.
You
should
be
aware
of
the
dependencies
that
each
microservice
has
based
on
a
certain
application.
The
second
main
aspect
is
in
terms
of
infrastructure
topology,
so
we
want
to
establish
network
weights
among
cluster
nodes
based
on
different
metrics.
A
A
We
believe
that
all
kinds
of
topologies
will
benefit
from
this
framework,
but
of
course,
high
latency
is
a
major
concern
in
multi-region
clusters,
but
even
in
a
small
scale
cluster.
If
your
nodes
have
different
network
connections,
they
can
benefit
from
latency,
aware
decisions
and
as
an
example,
for
instance,
in
a
data
center,
you
have
certain
applications
like
financial
applications
that
if
you
deploy
dependent
workloads
far
away
from
each
other
in
different
zones
of
the
data
center,
you
may
have
high
latency
in
the
communication
between
these
two
workloads.
A
A
So
essentially
we
have
designed
our
framework,
based
on
their
repository,
based
on
their
current
framework
and
essentially
I'll,
try
to
explain
you
the
main
building
blocks
that
we
currently
have.
So
essentially,
we
have
two
custom
resources,
which
will
also
have
a
controller,
so
the
first
one
we
called
it
as
a
application
group
where,
essentially,
you
can
say
which
workloads
belong
to
a
certain
application
and
also
establish
dependencies
among
these
workloads,
and
I
will
go
more
on
these
later
on.
A
The
second
custom
resource
is
a
network
topology
where
you
can
establish
network
weights
among
different
zones
and
regions
in
the
cluster,
so
that
you
can
use
these
costs
later
on
on
our
scheduling
plugins,
we
have
three
scheduling
plugins,
essentially
in
the
communities,
six
scheduling
community.
They
have
opened
up
the
cube
scheduler
component
and
you
have
several
extension
points
where
you
can
develop
your
own
algorithms,
so
that
you
can
use
your
own
algorithms
to
schedule.
Pods,
let's
say
in
the
cluster.
A
We
also
have
a
net
perf
component
that
can
be
used
in
typically
in
small
scale
or
medium-sized
clusters,
where
you
run
net
proof
measurements
across
your
cluster
nodes,
and
then
we
save
the
measurements
in
a
config
map,
and
then
what
we
will
do
is
that
we
have
a
controller
that
will
access
this
config
map
and
they
will
update
the
costs
on
the
cr
based
on
the
net
perf
measurements.
It's
an
additional
component
that
we
are
also
proposing
and
then
another
important
part
is
bandwidth
in
terms
of
bandwidth.
A
We
also
use
the
bandwidth
cni
plugin
available
in
calico
to
limit
the
bandwidth
in
power
deployments.
It
currently
supports
ingress
and
egress
traffic
shaping.
So
with
this,
we
try
to
find
better
nodes
in
the
cluster
by
by
as
well
considering
bandwidth
as
a
resource,
so
going
to
a
complete
example,
trying
to
show
you
how
our
framework
works.
So
imagine
that
you
have
an
application
group
a1
composed
of
three
pods.
A
We
have
essentially
a
yaml
based
file
where
we
say
which
workloads
belong
to
this
application
group
and
we
establish
dependencies
among
the
workloads
in
terms
of
dependencies.
Currently,
we
support
the
minimum
bandwidth
requirement
and
the
maximum
network
cost.
So
essentially
we
tell
the
cluster
that
between
two
dependent
workloads
there
is
a
minimum
bandwidth
that
should
exist
between
these
two
so
that
they
can
communicate
properly
and
also
a
maximum
network
cost,
which
is
the
maximum
threshold,
let's
say,
or
that
in
terms
of
communication
latency.
That
should
exist
between
two
cluster
nodes.
A
When
they
are
allocating
these
two
workloads,
then
we
have
the
network
topology
cr
that
I
mentioned
before,
where
you
can
manually
input
costs
with,
based
on
the
knowledge
that
you
have
of
the
infrastructure.
Currently
we
are
focusing
on
latency,
so
we
want
to
find
placement
schemes
focusing
on
low
latency,
so
we
establish
costs
between
regions
and
zones
in
the
cluster.
As
you
can
see,
the
yaml-based
file
will
correspond
to
the
figure
in
the
slide
so
now
going
through
the
scheduling
plugins
that
we
are
also
proposing.
A
First,
one
of
the
things
that
we
thought
about
is
that
when
you
have
several
workloads
in
your
application
with,
how
do
we
select
the
first
spot
to
be
scheduled?
It's
because
the
order
from
which
you
will
allocate
all
these
pods
in
your
application
will
have
an
impact
on
the
final
end-to-end
response
time.
Let's
say
of
your
application.
So
how
do
we
solve
this?
Which,
which
part
should
be
we
consider
first?
So
we
have
a
potential
solution
for
this,
which
is
an
additional
qsort
plugin
that
sorts
spots
based
on
topological
sorting,
essentially
in
topological
sorting.
A
It
looks
for
the
dependencies
amongst
different
workloads
and
tries
to
find
the
preferred
order
based
on
topology
information,
and
currently
we
support
six
algorithms,
and
I
will
show
you
that
significant
differences
can
be
obtained
based
on
the
topology
algorithm
that
you
select
for
in
the
application
group.
So
for
now
we
are
supporting
six
algorithms
and
you
can
see
here
that,
depending
on
which
one
you
select
different
orders
are
are
obtained
so
currently
for
the
online
boutique
application.
A
We
achieved
lower
latency
with
three
versions
with
three
algorithms,
but
you
can
clearly
see
that,
depending
on
the
order,
it
matters
how?
How
much
can
we
optimize
the
latency
in
the
in
the
in
the
in
the
kubernetes
cluster?
So
essentially,
what
we
do
is
that
we
attribute
an
index
to
each
workload
in
our
application
group
and
based
on
that
index,
we
will
sort
the
pods
in
the
waiting
queue
to
be
scheduled.
A
Then
we
have
a
filtering
and
the
scoring
function.
The
filtering
essentially
tries
to
filter
out
nodes
that
will
produce
low
scores,
trying
to
filter
out
these
nodes
based
on
the
requirements
that
I've
already
mentioned
in
the
pods
application
group,
and
also
then
the
scoring
function
is
the
main
function
of
our
of
our
framework.
Let's
say
where
we
try
to
find
nodes
that
will
ensure
that
the
network
latency
is
minimized
for
dependent
workloads
based
on
the
network
costs
that
are
available
in
the
network,
topology
cr.
A
A
A
And
then
for
the
filtering
function,
let's
imagine
that
we
want
to
schedule
the
workload
p1
and
p2
is
a
dependency
so,
and
we
know
that
there
is
already
an
instance
of
p2
deployed
on
node
n1.
So
how
does
the
filtering
function
works?
So
essentially
it
tries
to
exclude
nodes
that
will
unmet
a
higher
number
of
dependencies
and
tries
to
reduce
the
number
of
nodes
being
scored.
So
in
this
example,
essentially,
it
will
check
the
maximum
network
cost
requirements
between
p1
and
p2
in
the
application
group
cr
and
the
nodes
that
will
produce
a
higher
cost.
A
A
We
normalize
the
accumulated
costs
for
all
the
nodes,
because
we
want
to
favor
nodes
with
lower
costs.
These
nodes
will
be
scored
higher.
So,
as
you
can
see
in
the
example,
the
nodes
that
will
be
scored
higher
are
n1
and
then
two
because
these
two
nodes,
based
on
our
network
topology,
will
produce
lower
latency.
In
the
end.
A
A
All
the
nodes
belong
to
the
same
region,
belgium
and
but
each
node
is
on
a
different
zone
and
I
have
emulated
different
network
connections
with
varying
delays
with
traffic
control.
So
you
can
clearly
see
the
differences
in
terms
of
latency
this
morning.
I've
run
the
additional
component
that
I
mentioned
the
netperf
component.
So
I
have
all
the
measurements
saved
in
a
config
map
and
I
will
try
to
deploy
our
network
topology
controller
that
will
access
these
measurements
and
will
update
the
network
topology
cr
accordingly.
A
A
A
A
A
A
And
the
idea
now
is
that
I
will
deploy
the
online
boutique
application
with
the
normal
ks
and
I
will
use
the
locus
load
tool
available
in
online
boutique
to
see
the
application's
response
time
and
then
I
will
deploy
the
online
boutique
based
on
our
framework.
With
this
different
version
of
the
schedule.
A
A
A
So
it's
running
so
here
you
can
see
that
typically
the
ks
when
deploys
the
online
boutique
application.
Typically,
we
have
higher
values
in
terms
of
latency.
So
here
you
can
see
that,
with
our
network
aware
framework
we
can
at
least
or
for
most
requests.
We
can
at
least
30
to
40
percent
reduce
the
latency
expected
in
the
cluster.
A
I'll
just
keep
it
running,
so
even
the
in
the
minimum
and
in
the
average
response
time
that
we
get
in
the
online
boutique
application.
With
our
framework,
we
can
typically
reduce
it
based
on
specifying
pod
dependencies
specifying
workload
dependencies
in
the
application
group
and
as
well
having
the
network
topology
in
the
network,
topology
cr.
We
can
typically
reduce
the
latency
in
a
kubernetes
cluster.
A
So
what
were
the
challenges
and
the
lessons
learned
that
we
learned
well?
Our
plugins
can
significantly
reduce
the
expected
latency
in
kubernetes
clusters,
especially
if
you
properly
define
the
application
group
based
on
the
workloads
of
your
application
and
establish
the
dependencies
that
you
will
have
in
these
in
your
application.
A
A
Another
aspect
that
I
did
not
mention
yet
is
that
our
plugins
do
not
add
significant
overhead
to
the
scheduler
process.
Currently,
if
we
want
to
schedule
a
pod
based
on
the
go
benchmark
tool,
we
are
well
below
one
second
for
a
cluster
with
ten
thousand
nodes.
So
we
are
not
adding
overhead
because
we
are
accessing
information
via
custom
resources,
and
my
main
message
today
is
that
we
are
currently
looking
for
contributors
and
engagement
from
the
community
that
are
interested
in
network.
A
Aware
scheduling,
because
currently
we
are
focusing
on
reducing
the
latency
by
specifying
workload
dependencies,
but
you
may
have
different
ideas,
different
concepts
that
you
would
like
as
well
to
add
to
our
current
implementation
and
we
are
open
to
it.
Currently,
we
are
actively
involved
in
the
kubernetes
scheduling
community
and
we
have
kubernetes
announcement
proposal
already
accepted
and
we
have
the
initial
pr
already
submitted
and
we
are
waiting
for
revision.
A
C
B
B
A
So
currently
we
have
the
additional
component
and
net
perf,
where
you
can
run
several
times
a
day.
Let's
say
the
component
so
that
you
have
updated
costs
in
a
config
map
and
our
controller
will
update
the
costs
in
the
cr
so
that
you
we
can
find
better
decisions
in
terms
of
scheduling
and
also
when
pods
are
evicted
or
rescheduled
all
the
workloads
that
are
already
deployed
in
the
system.
C
A
So
currently
we
are
waiting
on
the
revision
on
the
pr
that
we
submitted,
so
what
I've
shown
you
the
main
components
that
I've
shown
you
in
my
overview,
it's
they
are
running.
So
we
have
this
version
completely
running,
but
we
are
waiting
so
that
it
can
be
included
in
the
seek
scheduling
community,
but
we
expect
to
be
there
in
the
next
few
months.
Otherwise
you
can
access
our
current
implementation
via
the
fork
that
I
have
from
their
repo.
So
it's
available
in
one
of
my
branches,
so
you
can
testing
out
for
yourself
as
well.
B
E
A
One
of
the
things
that
we
have
planned
is
to
add
a
an
additional
plug-in
which
will
focus
on
monitoring.
Let's
say
the
bandwidth
that
the
incoming
traffic
on
a
given
node
and
based
on
that
we
will
develop
a
plugin
that
will
try
to
schedule
the
the
workloads
based
on
the
bandwidth
resources
requests
that
you
set
up
in
the
deployment
file
and
based
on
the
current
usage
of
the
node.
A
F
F
So
if
one
of
the
pods
evicted
or
getting
crash
lube
and
they
say
scaled
somewhere
else,
end
of
the
some
places.
So
what
does
it
do?
Can
you
put
some
light
on,
I
mean?
Is
it
going
to
regarding
to
network
benchmark?
Is
it
going
to
kill
every
pod
and
start
to
schedule
for
network
efficiency
or
is
going
to
live
as
it
is?
Like
I
say,.
A
I
would
say
that,
as
a
cluster
administrator,
you
need
to
make
decisions
or
decide
which
you
prefer
to
optimize
in
your
cluster.
So
currently
you
have
several
plugins
focused
on
resource
efficiency
and
we
are
proposing
one
which
is
more
focused
on
network,
aware
information
which,
based
on
workload,
dependencies
and
network
topology,
tries
to
find
placement
schemes
where
it
tries
to
reduce
the
latency.