►
From YouTube: Peloton - A Unified Scheduler for Web-scale Workloads on Mesos - Min Cai & Nitin Bahadur
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Peloton - A Unified Scheduler for Web-scale Workloads on Mesos & Kubernetes - Min Cai & Nitin Bahadur, Uber
Efficient use of cluster resources is important for web-scale companies like Uber. Those companies require large-scale clusters for stateless, stateful and batch jobs. Today, web-scale companies have built custom schedulers on top of Mesos due to lack of viable open-source solutions. Kubernetes has gained lots of momentum in recent years but lacks the scale and efficiency needed by web-scale companies. This talk introduces Peloton - A unified scheduler for mixed workloads that is horizontally scalable to 10K+ nodes and millions of containers. It has an extensible architecture and supports both Mesos and Kubernetes. Peloton manages compute resources more efficiently and guarantees hierarchical max-min fairness for different teams. It provides a seamless path for companies on Mesos to adopt Kubernetes. Peloton is also cloud agnostic and can be run on-prem or in any public Cloud.
To Learn More: https://sched.co/GrTx
A
A
Today,
along
with
me,
I
have
min
Kai
with
the
senior
staff
engineer
at
uber
and
who's,
one
of
the
co-inventors
of
the
technology
pallet
and
about
which
we'll
be
talking
about
in
the
audience
we
have
my
own
bunsen
who's,
also
one
of
the
co-inventors,
and
we
also
have
a
bunch
of
uber
developers
here,
who've
been
working
on
this
technology
in
a
few
weeks
from
now,
my
family
and
I
will
be
taking
a
vacation
in
Costa
Rica
either.
Looking
for
hotels,
I
came
across
this
hotel
and
it
intrigued
me
El
Faro
beach
hotel.
A
A
Little
bit
about
uber
we've
completed
about
over
10
billion
trips
and
we
are
doing
over
15
million
trips
every
day
we
are
in
over
65
countries
and
over
600
cities,
and
our
headcount
keeps
growing
day
by
day.
We
are
over
3,000
developers.
As
we
last
speak.
Our
business
growth
has
been
interesting
from
2010
to
2015.
A
We
grew
at
a
moderate
pace,
but
in
2015
we
hit
the
hakka
stir
hockey
stick
curve
where
we
were
doubling
our
users
and
our
rides
every
six
to
nine
months,
and
as
this
growth
was
happening,
the
infrastructure
teams
was
crumbling
to
make
sure
that
the
infrastructure
did
not
fall
through.
They
were
New
Year
Eve
night,
where
engineers
used
to
be
at
office,
not
parting
and
enjoying,
but
making
sure
things
didn't
fail
and
there
were
no
outages.
A
Fortunately,
we
are
well
beyond
that
and
we
are
pretty
stable
now
and
to
give
you
an
idea
of
where
we
have
come
since
then.
Today
we
are
running
thousands
of
micro
services
on
our
infrastructure.
We
have
over
100k
containers
running
four
clusters:
multiple
clusters,
all
over
the
globe.
We
have
separate
clusters
for
GPUs
with
thousands
of
GPUs
per
cluster
and
we
run
over
a
million
batch
containers
in
a
day.
A
I
know
this
is
cube
con,
but
in
2015
really
there
was
nothing
available
that
would
scale
in
2015
we
had
to
go
with
the
technology
that
was
known
to
work
and
we
went
with
Mae
sauce
and
in
the
middle
of
2016,
as
you
were
actually
deploying
all
this
stuff
out.
We
came
to
different
realizations.
We
realized
that
they
were
a
vast
majority
of
use,
cases
needed
scale,
scale
scale,
our
autonomous
vehicle
division
wanted
to
run
jobs
that
would
have
100k
tasks
and
they
wanted
to
run
millions
of
them
every
day.
A
Deep
learning
was
taking
a
long
time
to
complete
and
the
engineers
wanted
to
run
deep
learning
on
hundreds
of
GPUs
every
day
long,
and
there
was
no
really
no
scheduling
mechanics
out
there
and
as
we
deployed
these
clusters,
we
realized
hey,
there's
a
stateless
cluster.
There
there's
a
stateful
cluster
here
there
are
some
bad
jobs
running
their
diamond
jobs
running
and
all
these
clusters
are
really
not
well
utilized.
What
do
we
do?
Infrastructure
for
uber
and
for
us
one
of
the
important
thing
was:
how
do
we
reduce
the
cost?
A
So
we
can
pass
on
the
savings
to
our
drivers
and
riders,
so
we
went
to
look
for
help
in
industry
architectures,
and
we
saw
the
board
paper
and
one
of
the
things
that
the
board
paper
said
was.
If
you
can
co-locate
latency,
insensitive
and
latency
sensitive
workloads
on
the
same
cluster,
you
can
get
better
utilization.
We
thought
that
might
be
a
good
idea.
If
we
could
reduce
the
number
of
clusters,
we
might
be
able
to
buy
less
Hardware.
A
We
don't
have
to
provision
for
our
Peaks
because
we
can
start
mingling
bad
jobs
when
the
real
time
services
are
not
that
active
and
disaster
recovery
would
also
be
much
lesser
cost
because
less
Hardware,
but
we
had
to
be
careful
that
our
real-time
services
did
not
get
impacted.
Because
of
all
of
this,
we
didn't
want
a
rider
opening
his
phone
and
waiting
for
it
to
contact
kubarz
service.
A
So
we
looked
at
existing
cluster
management
solutions
to
see
what
could
solve
problems
get
a
scale,
get
us
efficiency,
but
there
was
no
silver
bullet.
We
looked
at
Google,
Borg,
yarn,
various
mesas
frameworks
and
kubernetes
as
well,
but
what
we
realized
was
Google
had
not
open
source
board
if
they
had
a
life
would
have
been
so
much
easier.
A
Yon
folks
were
not
focusing
on
stateless
and
stateful
jobs;
they
were
happy
with
batch
and
in
early
2016
communities
were
still
in
its
infancy.
We
were
talking
about
scale,
we
were
talking
about
elastic
resource
sharing,
we
were
talking
about
large-scale
GPU
scheduling
and
they
were
trying
to
figure
out
systems,
cables,
scalability
and
stability.
A
So
we
came
at
a
crossroads.
We
came
at
a
crossroads
of.
Do
we
keep
our
business
hostage
while
the
cluster
management
solutions
of
the
industry
sterilize,
or
do
we
act
and
keep
our
infrastructure
up
to
date
with
our
business?
And
that's
when
we
took
a
big
bet
and
to
talk
about
the
big
bed,
we
have
men.
B
Hi
everyone
so
to
address
the
challenges
between
just
mentioned.
We
introduced
peloton
back
in
2016,
so
basically
we
would
like
to
build
a
unified
scheduling
framework
which
is
could
be
running
on
missiles
or
cooperate,
so
we
can
support
a
colocation
of
mixed
workload,
including
state
service
services,
state
for
large
scale
batch
and
as
well
as
FEMA
sets
Parata
actually
integrates
with
different
workloads,
including
spark
tends
to
follow
young
api's
as
well.
Our
internal
micro
service
platform
call
you
deploy
and
it
can
be
run
on
on-premise
data
centers
or
in
the
cloud.
B
So
this
is
a
high-level
overview
of
Politan
on
the
container
execution
aspect.
We
can
either
use
in
Misurata
master
or
we're
actually
in
a
active
development
of
hooking
up
corporate
directly
to
a
peloton
and
then
on
top
of
peloton.
We
can
support
different
mixed
workload.
For
example,
a
homegrown
service
platform
called
you
deploy,
as
well
as
a
homegrown
stateful
framework
called
OD'ing,
which
we
can
run
Cassandra
the
elastic
search
radius
or
my
cycles.
B
So
now,
let's
take
a
look
at
how
parts
and
fitting,
through
the
comparison
knitting
just
showed
before.
Basically,
if
we
break
down
all
different
costs
management
systems
into
multiple
layers,
so
every
system
has
container
execution
engine
or
which
is
aging
running
on
each
machine.
Forgiving
poker
has
pocket.
B
No
yeah
have
known
the
manager
and
missile
missus
agent,
Cobras,
alkylate
and
Impala
to
actually
right
now
we're
using
missus
agent
as
a
can
excuse
avenging
in
production,
but
we
can
also
adapt
to
using
couplet
as
well,
and
then,
on
top
of
that,
we
have
like
different
layers
like
a
resource
allocation,
preemption
placement
and
a
task
and
a
job
life
cycles.
So
in
this
case,
parting
is
a
very
similar
to
companies.
We
implement
all
those
compunction
era.
B
T's,
then
application
workflows
can
be
extended
on
top
of
that
as
a
controller
jobs
or
as
like
say,
are
these
like
eating
Cuban
aids.
So
this
is
like
a
high-level
architecture
of
Politan.
We
provide
a
part
of
API
using
G
RPC,
so
basically,
you
can
implement
any
part
and
crunch
like
in
Java,
go
node.js
or
Python
and
we
have
operating
you
I,
CRI
and
different
extensions
right.
Now
we
already
have
more
than
ten
extensions
within
Google,
which
talks
to
the
party
API.
B
Then,
on
the
storage
side,
we
are
using
Cassandra
right
now
through
our
storage
gateway,
but
the
way
we
actually
abstract
out
of
that
layer,
so
we
can
potentially
plug
in
my
seekers
or
other
storage
solutions
right
now,
practice
they're
using
zookeeper
as
a
leader
election,
then,
on
the
actual
core
components
we
have
four
core
components:
the
first
one
is:
we
could
host
manager
right
now.
Host
manager
can
basically
either
talk
to
mrs.
master
or,
in
the
very
short,
very
near
future.
We
can
have
all
the
correlates.
B
I
talked
to
host
manager
as
well
I'm,
going
to
talk
about
more
tedious
down
the
road
and
they,
on
top
of
that,
we
have
a
component
color
resource
manager
which
is
managing
a
hierarchical
resource,
pool
trees
and
then
similar
to
cooperate
scheduler.
We
have
different
plugins
for
placement
engines.
Actually,
we
follow
the
same
idea
as
sparrow.
B
Basically,
we
can
have
multiple
patch
placement
genes
to
a
random
set
together,
like
a
big
scare
of
biggest
loophole
for
bachelor
clothes,
and
also
we
can
have
a
space
custom
or
special
placement
Jing
for
stateless
services
or
state
for
services
they.
On
top
of
that,
we
have
a
job
manager
which
improves
the
job
and
tasks
a
life
cycles.
B
B
So
the
idea
here
is:
we
can
have
different
organizations,
they
can
have
some
reserve
the
resource
and
they
have
some
limit,
but
how
much
they
can
use
is
very
elastic
and
dynamic
depends
on
the
amount
of
that
organization,
as
well
as
variable
resource
within
the
cluster.
So
they
were
killed.
Some
fair
shares
between
what
are
they
reserved
and
then
what's
the
maximum
it
is.
They
have
four
jobs
submitted
to
a
single
resource
pools.
They
will
have
local
priorities,
so
you
don't
need
to
introduce
any
global
priorities
across
the
company.
B
This
is,
like
example,
of
hierarchical
resource
pools,
party
hats.
So,
basically
you
have
a
tree
structure
and
you
can
break
down
all
the
resources
into
different
organizations
and
teams
and
then
for
each
resource
pool
you
were
defined.
What
is
the
minimal
reservation?
You
won't
give
to
this
organization
which,
where
we
get
guaranteed
resource
and
then
what's
the
maximum
and
also
what's
the
fair
share
and
how
much
resource
each
organization
can
use
is
the
record
in
tournament,
which
is
determined
dynamically
based
on
the
availability
of
the
cluster.
B
So
this
actually
issues
example
of
a
resource
elasticity
we
had
in
part
and
cluster.
So,
for
example,
at
the
beginning,
if
there's
no,
we
have
to
resource
pools
was
for
team
one
in
a
work
for
Team
two
initially,
if
there's
no,
the
team
tree
is
not
using
the
resource,
then
team
one
can
use
most
of
the
with
poorer
resource
from
team.
Two
was
team,
juice
has
a
moat
amount
and
then
we're
gonna
preamp
to
the
resource
from
team
one,
and
it
gave
back
to
team
two
and
also
part
an
imprimatur.
Quite
a
few.
B
B
Now
Partha
has
been
in
production
for
almost
more
than
one
year
now
we
are
running
various
part
and
were
close
at
uber.
For
example,
we
are
running
large
scale
batch
workload
in
pallet,
including
spark
digital
flows.
Large-Scale
map
were
close,
as
well
as
our
self-driving
car
units,
the
batch
workload
zero.
So
the
skill
we
have
right
now
is
like
we
have
more
than
8,000
hosts
in
different
part
in
clusters
and
the
more
than
2,500
GPUs
in
piloting,
managed
by
part
intially
and
while
running
millions
of
jobs,
mostly
and
more
than
milling
containers
every
day.
B
So
one
example
is,
we
are
running
spark
important.
Today.
Most
of
companies
only
have
two
ways
to
wrong
spark.
Actually,
including
now
have
three
ways.
The
first
one
is
the
one
in
spark
only
young
or
spark
on
missiles,
directly
as
individual
frameworks
or
use.
The
new
spark
driver
for
Cuban
AIDS,
which
is
still
in
development,
actually
actually
we're
running
spark
on
peloton.
B
We
had
our
spark
drivers,
which
imported
lots
of
features
which
only
young
has
like
a
dynamic
resizing
of
the
spark
clusters,
so
you
can
dynamically
add
and
remove
executors
the
next
big
workload
we
had
for
Palatine.
He
has
gotta,
distribute
deep
learning
and
there
are
lots
of
challenges
for
distributor
flows
like
elastic
a
GPU
resource
sharing
locality,
and
they
talked
a
where
placement
can
scheduling.
Actual,
consequently,
is
very
important
because
when
you
do
distribute
deep
learning
job,
you
want
to
launch
all
the
content
at
the
same
time.
B
Otherwise
you
would
have
a
data
logger
situation
if
you
don't
have
enough
to
be
using
the
cluster.
So
this
is
actually
a
high
level
diagram
how
we
are
running
to
shoot
the
offensive
Lausanne
peloton,
so
it's
very
similar
to
Spock
model,
but
instead
of
like
a
user
submitting
smug
jobs,
we
have
a
deep
machine
learning
platform
called
am
a
cannula
which,
using
the
party
API
through,
submit
deep
learning
jobs
to
peloton,
and
we
use
part
yourself
to
do
tasks
discovery
as
well
as
gain
scheduling
and
soul.
B
So
for
strictly
surveys,
this
is
actually
a
diagram
shoes
from
a
Jaeger.
We
had
a
talk
yesterday
and
tomorrow,
basically
witch-king.
We
can
map
all
the
connectivities
between
all
the
micro
services
within
Google.
So,
as
you
have
seen
before,
we
have
thousands
of
micro
services
right
now
they
are
still
running
on
Aurora
on
top
of
missiles
and
we
are
actively
moving
them
into
politics.
B
So
we
can
complete
the
story
of
collocating,
long
running
and
patched
clothes,
and
also
we
are
working
with
our
store
storage
team
to
onboard
to
start
POC
and
onboard
storage
workloads
state
for
services
to
peloton.
So
the
target
timeline
is
2020
h1,
because
this
and
to
the
19
we're
going
to
focus
on
get
all
the
micro
services
to
be
running
on
piloting
with
a
Hoover.
B
Firstly,
what
we
did
is
actually
we
were
able
to
hunker
grenades,
as
well
as
a
peloton
as
virtual
clusters
on
top
of
peloton.
So
you
can
imagine
with
me:
we
can
generalize
both
Cooper
masters
as
well,
cooperate
and
bring
up
a
kubernetes
cluster
on
top
of
a
physical
part
in
a
cluster.
So
this
way
we
can
run
large-scale
testing
with
a
limited
resource
like
in
terms.
B
For
example,
we
can
simulate
10,000
node
cluster
with
like
500
to
1000
physical
machines
and
we
basically
scale
the
cluster
size
from
1k
to
10
K
nodes
and
we
scale
the
batch
and
status
jobs
from
10k
200k
containers,
because
that's
actually
the
realistic
we
use
case
we
have
today
within
Google,
and
we
measure
the
true
data
points.
Y
is
the
total
time
to
completion
for
batch
jobs
as
well
to
the
time
to
rolling
update
for
stateless
jobs.
B
So
first,
let's
take
a
look
at
the
data
right
now
for
first
of
what
we
did
is
that
we
have
a
fixed
across
the
size
of
twenty
thousand
two
thousand
nodes,
and
then
we
launch
a
batch
job
of
10k
ports,
20k
pores
and
the
50k
ports
on
both
coordinates
using
batch,
API
or
palette
API.
So
we
can
see
here
how
actually
cookies
batch
the
batch
controller
today,
encumbrances,
obviously
not
where
optimized.
So
it
takes
more
than
5,000
mere
seconds
for
20,
so
20,000
pours
and
then
for
50,000
pause.
It's
actually
didn't
even
complete.
B
After
all,
so
we
pretty
much
give
it
up,
but
for
Politan
we
can
launch
all
those
within
close
to
the
second
minute.
Now.
The
second
experiment
we
did
is
we.
Basically,
we
scatter
the
number
of
nodes
from
1
k,
2
to
K,
to
5k
203,
and
then
we
compared
coburn
AIDS
with
the
peloton
using
10,
K
ports,
and
we
can
see
here
are
same
similarly
like
the
partner
has
a
way
better
performance,
then.
Similarly,
we
do
the
same
experiment
for
measure.
B
How
much
time
take
us
to
create
a
department
with
a
10k
ports,
20k
ports
or
50k
ports.
The
system
will
result
with
a
way
we
observe
the
similar
results.
Basically
Pilates
outperformed
companies
in
in
various
ways
and
the
same
for
rolling
upwards.
Now,
based
on
that,
we
would
like
to
say:
hey.
Can
we
get
at
the
best
of
both
world?
For
example,
we
can
get
at
the
ecosystem,
support
from
companies
and
the
open
source
effort
here
and
then,
on
the
other
hand,
we
can
we
get
the
performance
gains
and
scalability
in
peloton.
B
Another
good
incentive
is
also
we
have
to
always
like
running
business.
Today
we
cannot
really
shut
down
the
clusters,
anything
bring
everything
through
companies.
We
have
to
have
a
same
race,
migration
story
so
part
and
could
solve
that
problem
as
well.
If
we
have
everything
under
the
same
cluster,
so
true
possible
solutions,
we
are
talking.
We
have
come
up
wise.
B
We
can
take
the
kubernetes
feet
apart
and
features
and
plug
them
into
companies
as
a
custom
scheduler,
but
that
will
have
some
benefit
limitations
which,
like
the
etcd
and
api
server
port
america,
we
still
have
the
same
there.
In
the
second
solution
we
have
with
home
operators,
we
can
hook
up
all
the
corporate
in
two
parallel
host
manager
and
then
provide
a
community
api
bridge
which
we
have
supported
the
core
primitives
and
is
then,
potentially
you
can
have
all
the
CID
resources
to
build.
On
top
of
the
bridge.
A
So
in
summary,
pelican
has
been
running
in
production
for
over
a
year
now
and
we
are
solving
a
bunch
of
our
large-scale
business
use
cases.
It
was
designed
day
one
so
that
it
could
run
alongside
any
container
orchestration
system,
and
we
encourage
all
of
you
to
go
check
out
our
engineering
blog,
which
details
a
lot
about
what
this
technology
is,
how
its
progress
and
what
all
it
can
do,
we'll
open
it
up
for
questions
right
now.
Thank
you.
C
A
B
Over
that's
actually
a
very
good
question.
We
haven't
really
deep
type
analysis,
but
at
least
our
infrastructure
or
the
the
benchmark
can
set
a
step.
As
for
for
that,
to
answer
your
specific
question,
we
for
back
ok
for
batch,
we
definitely
reserves.
They
are
scheduling
limitation
because
it's
basically
ranch
all
those
containers
in
in
multiple
batches.
So
it's
not
like,
if
even
I
have
all
the
resources
available
in
the
cluster.
The
schedule
is
not
like
pushing
them
to
all
the
machines.
It's
basically
doing
like
hundreds
or
200
every
cycle.
So
it's
different.
B
Is
there
some
bottleneck
on
the
scheduling
aspect?
Maybe
I
don't
know,
maybe
a
scheduling,
algorithm
or
the
because
the
scheduler
is
watch
hdcd.
Every
time
you
get
like
a
punk,
a
chance
of
ports
right
so
also
incriminates
the
only
get
one
scheduler
important,
actually
quad
spray,
some
ing.
So
basically
every
time
you
only
have
one
place
the
managing
working,
but
in
part,
actually
we
have
two
or
three
which
we
do
the
same
time.
So
this
actually
follows:
if
you
guys
know
the
sparrow
paper,
so
we
you
don't,
the
the
placement
in
part
is
more
intelligent.
B
A
B
So
I
can
today's
question.
So
there
are
two
angles
why
I
got
from
uber
perspective
because
we
build
apart
info.
That's
why
we
already
have
very
fancy
service
discovery
mechanism.
Basically,
we
have
something
very
similar
to
the
programming
service.
Same
engineer,
develop
that
in
Google.
So
that's
more
scalable
than
Easter.
We
do,
but
we
are
different,
not
gonna,
change,
to
change
that.
So,
basically
the
pattern
we
are
still
support
of
the
uber
internal
service
discovery
system
to
serve
our
business
needs
externally.
B
B
B
All
right
now
we
have
like
a
different
with
a
goober.
We
have
like
a
very
large
big
data
system
right,
so
there's
like
a
spark
flying
high
for
those
kind
of
different
batch
systems
and
basically
we
gave
them
either
young-kyu
or
particle
resource
pools
and
they
were
submitters
their
jobs,
but
if
they
are
not
using
the
queue
and
it
because
of
the
elasticity
of
the
scheduler,
the
other
jobs
were
use.
Resource
yeah,
so
there's
no
static
partitioning
of
the
faster
it
ever
since
the
elastic.