►
Description
Intro: Autoscaling SIG - Solly Ross, Google & Michael Hausenblas, Red Hat
SIG Autoscaling develops and maintains the components related to automated scaling in Kubernetes: the Horizontal Pod Autoscaler, Vertical Pod Autoscaler, and Cluster Autoscaler. In this introduction, we'll explain common use cases for each of the three different components, and how they can be used together to build more complex autoscaling stories. Finally, we'll cover recent ongoing development work on each of the components, and discuss plans for the autoscaling components in the upcoming Kubernetes releases.
To learn more: https://sched.co/Grcx
Join us for KubeCon + CloudNativeCon in Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
Join us for KubeCon + CloudNativeCon in San Diego November 18 - 21. Learn more at https://bit.ly/2XTN3ho. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
B
So
in
kubernetes
we
have
basically
two
kind
of
categories
of
auto
scaling.
So
we
have
cluster
auto
scaling,
so
that's
creating
more
nodes.
When
you
run
out
of
space
to
schedule
your
pods
on
to
and
then
we
have
application
level
auto
scaling,
so
that's
making
sure
your
applications
themselves
have
enough
room
to
operate
in
and
that
there's
enough
instances
of
your
application
or
your
applications.
Big
enough.
We.
C
B
Few
few
of
you
right
so
so
for
those
of
you
who
may
not
be
familiar,
we
call
the
horizontal
pod
autoscaler,
so
that's
horizontal
application,
auto
scaling
making
more
replicas
of
your
application.
We
call
that
the
HPA
and
vertical
auto
scaling
that's
increasing
the
resource
limits
of
your
application.
We
call
that
the
VP
a.
B
B
There's
also
a
horizontal,
auto
scaling,
and
so
the
idea
behind
horizontal,
auto
scaling
is
that
we
declare
a
target
value
for
some
metrics.
So
for
CPU.
We
might
declare
that
we
constantly
want
our
application
to
kind
of
target
around
I
know
500
mil
of
cores
half
of
a
core,
or
we
might
say
that
we
generally
want
each
instance
of
our
application
to
be
serving
say,
50,
requests
per
second
and
then
at
a
regular
interval.
The
horizontal
pot
autoscaler,
we'll
look
at
are
the
actual
resource
usage
of
our
application,
so
maybe
our
actual
requests
per
second.
B
That
a
single
instance
is
serving
is
a
hundred
or
the
actual
CPU
is
one
core
and
we
say
that's
too
high
we're
getting
into
the
danger
zone.
If
maybe
we're
going
to
start
getting
performance
issues,
and
so
what
it
will
do
is
it
will
compare
those
that
will
form
a
ratio,
and
we
say
a
hundred
100
is
the
actual
usage
divided
by
50
is
what
we
have
desired.
So
we
scale
up
by
a
factor
of
two
and
simultaneously.
B
B
So
we
divide
the
number
of
replicas
by
2,
and
this
works
really
really
well,
when
you
have
an
application
that
can
be
scaled
horizontally,
it's
great
for
things
are
somewhat
stateless
things
like
web
applications,
not
so
great
for
things
like
databases
frequently,
unless
you
have
an
awesome
horizontally,
scalable
database,
there
are
a
few
of
those
and
so
for
something
like
a
database.
Oftentimes
vertical,
auto
scaling
can
be
better.
B
So
the
idea
behind
vertical,
auto
scaling
is
that
we
look
at
your
resource
usage
and
we
figure
out
if
you're
bumping
up
against
the
limits
that
you've
set
and
we
try
to
increase
them
so
right
now
that
process
actually
involves
restarting
a
pot,
but
in
the
future
we
hope
to
just
be
able
to
increase
in
place,
and
so
this
works
really
great.
If
you,
if
increasing
the
number
of
replicas,
won't
necessarily
solve
your
problem.
B
Alright,
so
in
the
past,
the
way
auto-scaling
worked
is,
is
we
only
really
in
the
past?
Had
horizontal,
auto
Skilling's
way
back
in
the
annals
of
kubernetes
history,
and
so
what
happened?
Was
we
had
this
tool
called
heap
stir
and
heap
screw,
would
query
all
of
the
notes
and
expose
an
API,
and
then
the
horizontal
pot
autoscaler
would
go
to
heap
stir
and
say,
give
me
all
of
the
metrics
for
the
containers
that
I
care
about
for
each
of
the
pods,
and
we
could
auto
scale.
B
We
only
auto
scaled
on
CPU
and
then
a
memory,
and
then
memory
went
away
because
it
was
hard
to
auto
scale
on
and
we
could
only
auto
scale
on
CPU,
and
that
was
the
way
it
was
for
a
long
time
and
eventually
people
said
I
would
like
to
scale
on
custom
metrics
and
we
had
a
few
false
starts
on
custom
metrics
and
we
eventually
settled
on
something
called
the
custom
metrics
API.
So
the
idea
behind
the
custom
metrics
API
is
that
people
have
very
strong
opinions
on
how
they
should
monitor
their
applications.
B
Some
people
love
Prometheus.
Other
people,
love
stackdriver.
Some
people
love
influx
what-have-you,
so
we
defined
an
API
that
didn't
have
an
implementation
kind
of
kind
of
the
same
way.
Ingress
doesn't
have
a
canonical
implementation.
The
custom
metrics
API
does
not
have
a
canonical
implementation.
Instead,
we
have
these
things
called
adapters
and
adapters
sit
as
a
layer
between
your
metrics
storage
and
the
Horace
horizontal
pot
autoscaler.
So
the
horizontal
pot
autoscaler
can
go
to
the
custom
metrics
API
and
say,
give
me
custom
metric
X.
B
Give
me
requests
per
second
on
this
pot
or
give
me
a
queue
length
on
this
object,
and
then
the
adapter
will
translate
that
into
a
query
to
Prometheus
a
call
to
stackdriver.
What
have
you-
and
this
allows
us
to?
Let
people
bring
their
own
monitoring
and
just
start
using
what
they
have
with
the
Horde
other
skill.
C
All
right
probably
should
have
mentioned
in
the
beginning
that
today
we're
gonna
give
the
introduction
overview,
and
tomorrow
yeah
we
will
have
a
guest
from
reg
Alex.
Who
will
talk
a
little
bit
more
about
predictive
analytics
around
that
topic?
They
have
an
offering
there
and
we
also
do
Q&A
today,
I,
don't
know
how
much
time
we
will
have
for
Q&A
but
yeah.
Let's
see
all
right
vertical
pot.
Auto-Scaling
s
already
mentioned.
There
are
certain
use
cases
where
the
HPA
stuff
doesn't
really
apply
or
doesn't
really
play
that
nicely
and
the
starting
point
really
is.
C
If
you
look
at
the
CMO
file
here,
essentially
the
and
the
resource
specification
where
you
essentially
define
requests
and
limits,
and
that
is
something
that
is
super
important
and
very
often
not
done,
because
it's
kind
of
like
how
should
I
know
how
much
memory
my
application
is
gonna
use.
So
a
bit
of
background
and
terminology
here
before
we
dive
deeper
into
that
the
process
of
scheduling
is
essentially
the
part
of
the
control
plane.
C
The
scheduler
there
that
year,
on
the
one
hand,
have
notes
the
work
notes
that
offer
resources
like
CPU
and
RAM,
and
then
you
have
parts
that
consume
resources
right
and
all
the
scheduler
really
does
is
trying
to
match
that
right,
saying,
can
I
find
a
node
or
nodes
where
the
requests
that
these
resources,
these
parts
come
along
with,
can
match
and
I
can
schedule
it
there.
The
rest
isn't
taking
care
of
my
the
cubelet
on
the
respective
node.
There
are
different
kinds
of
resources.
If
you
think
about
memory.
That
is
something
that
is
incompressible.
C
You
cannot
just
say:
okay,
I'm
gonna
randomly
take
some
pages
away
and
hope
that
the
whole
process
still
running,
whereas
CPU
is
compressible
right,
you
can
have
assigned
fewer
CPU
cycles
to
a
process
was
still
perfectly
run
just
slower.
So
you
have
to
look
at
your
at
your
the
type
of
research
that
you're
dealing
with
here
and,
most
importantly,
the
thinking
here.
The
underlying
thinking
here
is
the
so-called
quality
of
service.
C
So,
for
example,
if
you
take
resource
limits
and
resource
requests,
and
you
set
them
to
equal
values,
you
will
have
a
Q
s
of
guaranteed
right.
This
will
will
be
there.
You
have.
You
can
have
the
limits
setting
larger
than
having
a
higher
value
than
the
request,
CPU
and
memory
and
and
vast
0.
Then
it's
so-called
burst
of
all
Q
s,
and
if
you
don't
set
them
which
a
lot
of
people
are
fortunate
still
do,
then
it's
essentially
on
the
best
effort
basis.
C
C
Now
the
question
is:
where
does
that
come
from
right
and
I
just
pointed
out
to
blog
post
from
end
users,
who
essentially
complained
about
that
very
rightfully.
So,
if
you
look
at
the
timeline,
it's
a
bit,
you
know
to
two
years
or
whatever,
but
it's
still
very
much
true
that,
unfortunately
they
didn't
use
to
be
any
anything
there.
That
would
help
guide
users
to
actually
come
up
with
these
resource
limits
and
resource
requests
there.
C
So
the
goal
really
is
in
a
sense
that
we
can
automatically,
without
any
human
intervention,
figure
out
what
the
optimal
requests
and
limits
they
are
because
manually,
it's
it's
just
not
really
manageable,
and
with
that
we
have
a
quality
of
service
or
guaranteed
quality
of
service.
Otherwise
you
end
up
in
a
best-effort
case
the
other
hand
we
want
to
improve
the
utilization.
Obviously
the
more
we
can
pin
back
on
a
certain
note.
The
higher
utilization
gets,
of
course,
there's
always
a
certain.
You
want
to
leave
a
certain
Headroom.
C
You
don't
want
to
go
up
to
a
hundred
percent,
but
let's
say
80
90
percent
is
great
and
you
also
want
to
have
the
impact
on
the
other
function.
It
is
like
whom
handling
or
yeah,
enabling
it
still
I
think
not
really
optimizing
scheduler,
but
is
there
already
PRF
I
have
no
idea,
but
it
should
should
enable
that
as
well,
hopefully
at
some
point
in
time,
as
already
mentioned
by
solid
earlier
on
one
typical
use
case
where
it
is
any
kind
of
stateful
apps.
C
Think
of
database
like
my
cycle
or
or
Postgres,
or
anything
that
kind
of
packages
that,
like
WordPress,
that
you
know
under
the
hood,
also
uses
a
database.
I
will
point
out
at
this
point
in
time,
and
you
already
said
that,
given
that
we
have
currently
no
way
to
in
place
replace
the
part
essentially
means
a
new
part
with
the
new
effective
resource
limits
will
be
created.
This
first
statement
is
not
entirely
true.
C
It
is
only
true
if
you
can
actually
manage
the
state
in
a
way
that
you
either
make
sure
that
it
is
scheduled
on
the
same
node
or
that
the
data
goes
where
the
new
part
potentially
will
be
scheduled.
There
are
proposals,
or
this
one
proposal,
I'm
aware
of
that's
together
with
or
in
sick
node,
I
believe
where
it
essentially
boils
down
to.
C
And
definitely
if
someone
from
sick
node
is
round
and
dares
to
raise
their
hand,
okay,
so
this
is
really
something
that
at
least
for
me-
I
mean
we
when
we
initially
and
was
mainly
Google
in
implemented
an
oauth2
implementation
there.
It
was
clearly
called
out.
This
is
a
current
limitation,
no
limitation,
but
it's
better
than
having
nothing.
Even
the
recommendations
already
give
you
some
some
guidance
there
to
me
personally
at
the
current
point
in
time
with
the
current
implementation.
C
C
You
know
reaaargh
attacked
rate
or
whatever
reasons
you
just
take
an
existing
binary
or
whatever,
and
you
put
it
into
a
container,
and
that
is
something
where
in
this
case,
you
simply
cannot
use
a
horizontal
part
autoscaler,
because
there
is
nothing
to
you
know
you.
It
doesn't
make
sense
to
have
multiple
instances
or
replicas
running
in
that,
but
you
definitely
have,
but
with
growing
traffic
you
have
more
resource
needs
and
with
that
you
would
need
to
allocate
more
memory
or
End
or
CPU,
and
for
that
I
think
at
the
current
point
in
time.
C
This
is
certainly
something
if
you
have
that
case:
lift
and
shift
of
an
existing
one
of
the
where
you
can't
react
attacked
her.
You
probably
want
to
have
a
look
at
the
vp8
vertical
part,
autoscaler,
a
quick
look
at
how
this
actually
works
internally,
the
API
server,
the
control
plane,
most
importantly,
has
the
API
server
in
there.
Everything
goes
through
the
API
server.
Everyone
talks
through
the
API
server
and
that
are
on
a
principled
level
e
the
faces.
They
are
right.
You
get
a
HTTP
request
with
adjacent
payload.
C
In
there
there
is
some
authentication
authorization
verifying
who
that
entity
is,
and
if
that
entity
is
allowed
to
carry
out
the
operation,
like
you
know,
creating
a
part
or
creating
a
service,
and
then
this
step
the
yellow
or
little
step.
This
is
the
place
where
you
can.
Actually,
we
have
a
chance
in
that
flow
and
that
request
flow
to
change
the
actual
representation
of
that
respective
resource
or
object
in
this
case
before
it
actually
eventually
gets
get
stored
in
that
CD.
So
here
you
can,
for
example,
say:
oh,
the
user
didn't
set
any
request
limits.
C
C
Trick
here
that
we
are
leveraging
so
overall,
how
is
that
implemented?
Obviously,
we
in
order
to
come
up
with
a
recommendation
we
for
for
the
resource
limits
and
requests.
We
need
to
serve
what
is
that
application
actually
using,
and
that
is
yeah
done
and
and
come
comes
up
built
up
this
historic
profile?
C
That's
the
recommender
part
and
then
optionally
you
don't
have
to
you
can
apply
that
you
can
say
okay
now
in
force,
they've
now
set
the
actual
limits,
and
with
that
you
would,
at
the
current
point
in
time,
create
a
new
part,
and
that
is
on
a
very
high
level.
D
the
architecture
there.
As
you
can
see,
there
are
a
lot
of
moving
components.
That
is
no
doubt
I.
B
I
mean
I
can
talk
a
little
bit
more
about
it.
So
the
basic
idea
like
Michael
mentioned,
is
that
we
have
a
recommendation
engine
and
which
is
kind
of
the
heart,
and
so
the
recommendation
engine
periodic
will
will
fetch
and
periodically
update
a
recommendation
based
on
current
metrics,
and
so
it
will
look
at.
It
will
look
at
the
metrics
we
have
available
and
when
it
starts
up,
if
historical
storage
is
available
through
something
like
Prometheus,
it
will
attempt
to
pull
a
bunch
of
historical
metrics
and
via
Prometheus
and
then
as
it
runs.
B
It
will
also
periodically
attempt
to
pull
the
resource,
metrics
API
so
generally
provided
by
metric
server,
potentially
provided
by
Prometheus
as
well
and
update
its
recommendations.
Look
at
what
was
used
previously
and
figure
out
what
it
should
probably
be
using
now
based
on.
You
know
how
much
it's
using,
how
close
it
is
to
its
limits
and
then
there's
there's
two
steps,
so
the
recommendation
is
always
actually
applied
when
a
pod
comes
in.
So
a
pod
comes
in
for
creation.
B
E
C
C
C
Like
your
optimism,
so
it's
let's
agree.
It's
itself.
Okay,
cool
and
I.
Think
that
the
number
one
asked
here
is
especially
for
people
who
are
ready
either
looking
into
using
it
or
already
using
it.
Give
us
feedback
write
any
kind
of
data
points
that
we
can
get
to
see
in
what
direction,
because
obviously
the
the
current
efforts
are
on
stabilizing
that
and
once
the
API
is
stable.
Well,
you
can't
really
expect
changes
anymore
right.
So
now
it's
the
time
I
would
argue
to
you
know
be
loud
about
it
and
then
share
your
opinions.
B
If
that
use
cases,
I
need
in
place
updates.
We
are
aware
of
that
one
yeah
but
like
if
you
have
something
else,
that's
very
clearly
like
you're
like
oh,
it
looked
like
the
vertical
up
pot,
autoscaler
might
work
and
then
I
tried
to
apply
it,
and
there
was
just
some
complete
impedance
mismatch
with
my
use
case
or
everything
caught
on
fire
and
my
cluster
exploded
like
let
us
know,
because
we're
really
interested
in
those
so
that
we
can
update
our
design
change.
How
we
think
about
the
problem
before
we
go
to
beta
NGA
right,
I,.
C
Guess
the
majority
of
you
know
formal
informal
feedback
so
far
is
essentially
the
experience
that
Google
internally
has
with
autopilot
where
I
don't
know
60%
of
whatever
the
users
actually
use
that,
but
that
is
you
know.
You
can't
really
translate
that
one-to-one
here.
So
we
really
depend
on
you
know:
feedback
from
the
community
from
from
you
guys
out
there
to
know.
C
C
Okay,
do
you
want
to
voice
it
or
okay?
So
what
we
see
here
with
me,
what
we
see
here
on
the
left
hand
side.
It's
definitely
other
side
is
a
committees,
cluster
and
other
side,
the
the
command-line
interface
and
essentially,
we
have
something
prepared
there
that
we
want
to
use
or
apply
the
TVPA
on.
So,
let's
have
a
look
at
the
the
actual
resource,
the
statement
there,
the
manifest
there
and
if
you
look
at
you,
can't
really
see
it
nicely
there.
C
But
the
the
goal
here
would
be
the
hundred
milli,
CPU
and
50
megabytes
memory,
and
so
this
is
the
the
target,
deployment
and
WPA
in
there
and
the
manifest,
and
now
we
essentially
apply
or
create
that
in
the
first
step
and
as
we
would
expect,
the
this
example
hamster
app
comes
up
and
at
some
point
in
time
we
should
see
what
is
going
on
there
in
terms
of
what
what
it
actually
uses
and
after
some
more
time.
Okay,
there.
You
see
the
actual
hundred
Willy
course
and
fifty
megabytes
right
and.
C
C
Takes
a
bit
okay,
and
now
you
see
the
actual
change
there
right,
so
you
have
for
the
recommendation
the
lower
bound
the
target
and
the
upper
pound
pound.
That
is
the
recommendation.
We
got
out
of
the
BPA
there
that
you
can
choose
to
apply,
or
you
know
you
might
might
just
take
that
into
account
and
and
do
something
manner
with
it.
But
at
least
you
have
a
based
on
the
data,
a
clear
indication
on
what
your
application
actually
consumes.
I,
don't
know,
really
think,
there's
much
more
there
right.
B
B
We
try
to
always
respond
to
things
on
socata
scaling.
If
you
find
you
message
some
someone
and
they
have
an
or
ask
a
question,
no
one's
responded
feel
free
to
add
mention
us,
but
we
try
to
respond
and
we
have
pretty
decent
time
zone
coverage
between
some
of
the
folks
in
Poland
and
I'm
out
here
now.
So.
C
E
F
C
I
think
that's
pretty
much
it.
What
we
had
for
today,
so
I,
don't
know
how
much
time
we
have
five
minutes.
Does
anyone
know?
Is
it
five
minutes,
I
think
so?
I
actually
have
two
jet
sorry,
eight
minutes
I
take
the
eight
minutes.
So
then
we
can
actually
take
questions
already
today.
If
there
are
any
right
away
and
as
I
said
tomorrow
it
and
the
deep
dive
fizzing
in
the
afternoon
2:30
whatever
it,
there
is
some
all.
G
B
H
B
That's
a
common
question:
it
works
in
some
cases,
especially
if
you're
using
different
metrics.
If
you're
scaling
on
the
same
metrics,
then
you
run
into
problems
because
the
HPA
is
gonna,
try
to
optimize
on
one
access
and
then
the
vpa
is
also
trying
to
use
those
signals
to
optimize
on
a
different
axis.
But
I
think
there
have
been
a
couple
of
users
that
have
had
limited
success
in
in
scaling
on
difference,
so
they
might
scale
on
HPA
and
requests
per
second
and
vpa
scaling
on
memory
or
something
like
that.
I
Yeah
for
us,
by
the
way,
I'm
Emma
from
the
JD
drama,
we're
running
one
of
the
largest,
the
kubernetes
cluster
in
production.
We
have
a
half
media
continents
are
all
skinny
very
important,
so
so
why
use
cases
for
us
is
more
yeah
when
I
could
have
EPA
and
it's
simple
right,
easy
for
stateful
application,
but
sometimes
it's
limit
here
constraint
by
your
local
resource
right.
You
cannot
and
scre
it
unlimited
also,
so
we
could
do
in
the
wave
here
sometimes
and
but
up
to
a
certain
limit.
I
B
C
B
B
D
B
E
J
So
one
of
our
biggest
blockers
from
to
moving
to
kubernetes
is
scaling
down.
Our
pots
to
have
pots
are
performing
critical
operations
and
cannot
ever
be
sat
down
arbitrarily
I
noticed
a
similar
problem
with
the
game
server
guy.
What
they
had
to
was
interesting.
They
had
to
do
their
own
I,
think,
replication
controllers
and
their
own
see
artists
to
solve
a
simple
problem.
I
want
you
to
scale
down,
but
this
pot
in
this
pot
do
not
touch
please.
So
we
have
exactly
that.
B
B
B
B
Well,
but
you
won't
always
know
like
right,
like
if
you're
to
the
to
go
back
to
the
original
question
right.
You
have
some
number
of
workers
that
are
part
of
the
same
deployment
and
they're
pulling
items
off
of
the
queue
and
you
don't
want
to
kill
the
ones
that
are
currently
processing
things.
So
you
can't
just
dynamically
move
them
to
a
different.
G
B
Audit
you
create
an
individual
autoscaler
object,
targeting
a
particular
scaleable,
so
a
deployment
or
whatever.
So
you
can
always
just
not
create
an
auto
scaler
right.
But
in
the
case
where
you
don't
necessarily
opry
or
you
know
what
your
pods
are,
that
you
don't
want
to
touch
it's
a
little
bit
harder.
G
F
Right
actually,
I
just
wanted
to
point
out
I'm
the
auto-scaling
working
group
lead
for
cañedo,
and
so,
if
you
want
to
talk
about
scale,
zero
running
HBA
and
VP
a
at
the
same
time
or
we
actually,
we
deal
with
the
work
in
flight
problem
as
well.
So
we
have
like
hooks
so
that
existing
HTTP
requests
will
finish
before
we
kill
upon.