►
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2021 Virtual from May 4–7, 2021. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Introduction to Autoscaling - Guy Templeton, Skyscanner & Joe Burnett, Google
Come and see how to reduce the cost of your cluster and make your workloads more robust by dynamically adjusting them to their current load. During this talk members of SIG-Autoscaling will explain why you should be autoscaling both applications and clusters, and the tools Kubernetes provides to do this. You will learn the mechanics of Cluster, Horizontal Pod and Vertical Pod Autoscalers, their new and upcoming features as well as the best practices for applying them in production and things to watch out for. After this talk you will know where to look for information about the different autoscalers as well as which settings can be changed to adapt to different workloads.
A
A
Burnett,
I'm
here
with
guy
templeton,
who
is
the
co-lead
of
segato
scaling
so
he'll
take
over
in
the
second
half
and
run
you
through
some
some
practical
tips.
So
this
is
the
agenda.
I'm
also
going
to
talk
on
a
couple
upcoming
features,
so
you
can
get
a
preview
for
what's
coming
in
the
future,
we'll
save
about
10
minutes
for
questions
at
the
end.
B
A
So
the
same
concepts
of
vertical
and
horizontal
apply
to
cluster
out
of
scaling
as
well.
Cluster
autoscaler
scales,
a
node
pool
in
and
out
by
making
more
and
fewer
nodes,
and
it's
also
capable
of
scaling
vertically
by
selecting
different
node
pools.
So
if
you
have
node
pools
of
different
sizes
in
your
cluster,
it
can
make
intelligent
decisions
about
which
one
to
use,
but
to
kind
of
make
it
real
concrete
for
you,
I'm
going
to
walk
through
a
practical
example.
A
Two
workloads
deployed
here,
there's
the
blue
pods
and
there's
the
purple
pod
and
they're
deployed
on
three
notes.
Now
both
of
these
workloads
are
auto-scaled.
Let's
say
that
the
blue
and
purple
workloads
are
auto-scaled
horizontally
on
cpu
utilization
and
vertically
on
memory,
and
the
cluster
is
auto-scaled
as
well.
So
when
there
are
new
nodes,
new
nodes
will
be
created
when
new
nodes
are
needed,
so
suppose
that
blue
service
gets
some
more
traffic
and
the
hpa
will
notice
that
the
average
utilization
is
slightly
higher
than
the
target.
A
It
will
figure
out
how
many
pods
it
needs
to
create
in
order
to
bring
that
back
down
to
the
target
value,
and
it
will
cause
those
to
be
created
now.
The
hpa
doesn't
actually
make
the
pods
here.
What
it
does
is
it
is,
it
looks.
It
looks
at
the
scale
target
ref,
so
it
takes
a
scale
sub
resource.
It
looks
at
the
pods
that
are
selected
by
the
label,
gets
the
current
scale
and
when
it
has
a
recommendation,
when
it
thinks
we
need
more,
it
just
updates
that
scale.
A
And
if
this
is,
for
example,
pointed
at
a
deployment
that
deployment
will
turn
around
and
adjust
the
size
of
the
replica
set
and
the
replica
set
will
create
the
pots.
And
then
the
scheduler
will
see
these
pods
and
it
will
schedule
them
onto
a
node
and
the
node
will
see
the
pods
and
the
pods
will
be
started.
A
B
A
A
So
then
maybe
a
little
more
traffic
comes
in
for
the
purple
service.
Well
same
thing:
the
the
hpa
will
notice
that
we're
a
little
above
target
it'll
decide.
We
need
two
replicas
instead
of
one
and
it
will
adjust
the
deployment
and
a
pod
will
get
created.
But,
as
you
can
see,
there's
not
a
node
for
this
pod.
A
A
Simulation
and
it
says
if
I
was
able
to
if
I
create
another
node,
will
this
pod
be
able
to
be
scheduled?
And
if
the
answer
is
yes,
then
that's
the
thing
to
do,
and
it
can
do
this
across
multiple
node
pools
and
it
can
do
it.
You
know,
and
more
than
one
node
at
a
time,
so
it
actually
has
a
little
simulation
of
the
scheduler,
because
it's
ultimately
up
to
the
scheduler,
where
these
pots
go.
So
the
cluster
auto
scaler
says:
yes,
we
need
another
node.
This
sounds
good.
A
It
creates
the
new
node
scheduler
puts
the
pod
on
that
node
and
then
the
purple
service
has
scaled
out
so
over
time,
vpa,
which
is
paying
attention
to
the
actual
resource
usage
as
compared
to
the
request,
may
notice
that
the
purple
service
is
asking
for
way
more
memory
than
it
needs.
So
it's
going
to
draw
a
little
histogram
and
figure
out,
like
a
conservative
place,
to
draw
the
line
and
it's
going
to
recommend
a
smaller
pod
size.
A
So
the
recommender
makes
this
change
and
then
there's
an
updated
process
that
comes
through
and
we'll
carefully
delete
the
pods
one
by
one
and
cause
them
to
be
recreated.
So
the
updater
is
going
to
delete
this
pod
and
a
new
new
pod
will
be
recreated
because
the
replica
site
will
always
try
to
replace
the
pods
and
it
will
be
recreated
with
a
new,
smaller
size.
A
Then
vpa
updater
is
going
to
delete
the
next
pod
and
it
will
get
recreated
as
well
and
you'll
notice
that
actually
there's
an
empty
node
here
now
and
cluster
autoscaler
will
notice
this
too.
It's
able
to
delete
that
node
and
scale
the
cluster
in.
So
you
can
see
the
cluster
auto.
Scaler
goes
out
and
in
usually
these
nodes
are
what
you
pay
money
for,
so
it's
good
that
it
can
scale
down,
because
that's
what's
actually
going
to
cost.
That's!
A
A
A
It
may
water
down
your
your
metrics
and
so
take
a
look
at
this
cap.
Here,
the
it's
already
gone
through
api
review
and
it's
just
going
to
land
very
soon
and
it's
a
nice
way
to
focus
the
hp
on
specifically
which
containers
are
most
important
to
you
in
your
auto
scaling,
and
the
second
thing
that
we're
working
on
in
the
cigar
scaling
group
is
graduating.
The
v2
beta
2
api
to
stable
it's
been
around
for
quite
a
few
years
and
it
definitely
needs
to
get
graduated.
A
B
A
B
Thanks
so
just
for
a
bit
of
context
here,
I'm
a
senior
software
engineer
at
skyscanner,
and
so
a
lot
of
what
we
do
is
running
workloads
that
are
involved
in
scrape
either
getting
prices
from
online
travel
agents
or
airlines
and
then
doing
filtering
and
that
sort
of
thing
this
will.
This
will
come
into
play
in
a
couple
of
the
examples
I
get
for
things
that
we
should
consider.
B
So
I
think
us,
a
lot
of
people
watching
will
know
a
lot
of
the
hardest
problems
in
computing
on
technical
problems.
They're
people
problems-
and
this
is
one
of
those
spaces
where
the
complexity
of
all
the
options
and
all
the
the
tunable
parameters
and
knobs
that
kubernetes
gives
developers
and
cluster
admins
can
overwhelm
developers,
especially
when
they're
first
getting
started,
and
I
think
a
lot
of
the
time
developers
can
not
know
exactly
what
what
options
they
need
to
consider
when
looking
to
scale
workloads
safely
and
perform
in
a
performant
way.
B
B
At
the
same
time,
sending
in
a
sig
term,
so
you
can
either
set
up
your
application
to
handle
the
sick
term
gracefully
or
instead,
you
can
make
use
of
say
an
api
endpoint
that
can
be
hit
by
the
the
lifecycle
hook.
Use
that
to
gracefully
finish
any
work
and
refuse
new
work
potentially
and
then
allow
the
cubelet
to
shut
it
down,
and
so
these
pre-stock
hooks
they're
blocking
and
will
block
up
to
a
grace
period.
B
So
if
you
need
more
than
the
default
30
seconds
to
gracefully
terminate,
you'll
need
to
set
this
up
in
your
podspec,
but
by
default.
It'll
give
you
30
seconds
for
that
command
to
complete,
and
you
can
also
make
use
of
what
are
called
pulse
start
hooks
and
again.
These
can
be
hp
or
exec.
Central
pods
they're
not
guaranteed
to
run
before
the
container's
entry
point.
But
if
you
need
to
do
some
sort
of
environment
dependent
setup
work,
you
can
make
use
of
those,
and
the
next
thing
is
loudness
and
readiness
probes.
B
So
these
are.
These
are
fairly
key.
If
your
workloads,
for
instance,
on
startup,
need
to
do
some
sort
of
runtime
compilation
or
something
similar
to
ensure
that
you're
giving
your
users
the
best
possible
experience-
and
so
in
this
case
you
can
use
license
probes
to
mark
your
products
alive
and
healthy.
So
the
cube
that
won't
restart
the
pod.
B
However,
you
can
use
readiness
probes
to
state
that
the
pod
is
currently
not
ready
to
receive
traffic,
and
that
can
be
particularly
useful
in
combination
with
phk,
so
the
hpa
when
a
pod
is
not
marked
ready.
So
if
it's
doing
some
cpu
intensive
work
to
begin
with,
the
hpa
will
not
consider
those
that
pods
metrics
until
it
is
marked
ready,
so
that
that
ensures
that
the
hpa
is
not
seeing
new
pods
come
up,
use
a
lot
of
cpu
and
then
over
going.
B
Oh,
I
need
to
over
scale
again,
and
the
next
thing
is
pod
annotation.
So
in
the
case
that
you
might
have
say
a
machine
learning
workload
where,
if
you
interrupt
that
pod
via
scaling
or
the
cluster
autoscalers
trying
to
bin
pack,
you
might
end
up
with
the
workload
interrupted
and
effectively
losing
work
or
data.
B
And
in
this
case,
you
can
mark
pods
with
this
cluster,
auto
scaler
safe
to
evict
annotation,
and
if
you
mark
that,
as
safe
to
vect,
equals
false
effectively
when
the
cluster
auto
scaler
is
considering
bin
packing,
it
will
notice
that
that
node
has
a
pod
on
it,
which
is
not
safe
to
event
and
therefore
not
consider
it
as
a
candidate
for
scale
down.
And
so
theoretically,
you
can
spin
up
workloads
which
are
marked
as
safe
to
evict,
equals
false
and
then
either
update
that
annotation
or
remove
it
to
safety
vector
equals.
B
It's
important
to
note
that
this
doesn't
prevent
any
interruption,
so
hardware,
failure,
etc
or
manual
deletion
of
nodes
will
still
impact
that
pod,
but
the
cluster
autoscaler
won't
interrupt
it
and
vice
versa.
You
can
also
use
that
annotation
set
true
to
override
some
other
behaviors,
which
generally
the
cluster
autoscaler,
will
mark
pods
as
not
safe
to
vet,
and
so
things
like,
if
it's,
if
a
pods
using
local
storage,
the
cluster
auto
scaler
by
default,
will
consider
that
pod
as
being
unsafe
to
vect,
because
it
doesn't
know
what
data
it's
writing
to
the
local
disk.
B
It
doesn't
want
to
cause
data
loss.
Therefore,
it's
it
will
not
try
and
move
that
pod
around.
However,
if
you
can
tolerate
the
loss
of
that
data,
then
you
can
mark
it
if
yourself
as
safe
to
evict
in
the
cluster,
auto
scale
would
take
that
into
consideration
and
so
pod
disruption.
Budgets
are
probably
one
of
the
most
important
thing
so
going
back
to
joseph's
example
of
the
vpa,
where
it
was
recreating
pods
due
to
changing
the
resources
given
to
them,
the
cholesterol,
scaler
and
vpa
both
respect
the
pod
disruption
budget.
B
So
this
this
is
a
really
granular
way
of
service
owners
being
able
to
define
how
much
disruption
their
their
workload
can
tolerate.
So
if
a
given
service
can
tolerate,
say
20
of
its
pods
at
any
one
time
being
moved
around
and
being
unavailable
without
harming
the
user
experience,
the
the
pod
disruption
budget
is
the
way
of
defining
that,
and
then
the
claustrophobic
skill
and
vertical
pod,
auto
scaler
will
take
that
disruption
into
account
and
therefore
never
perform
a
disruptive
action
which
would
cause
that
budget
to
be
exceeded.
B
It's
important
to
note
that
this
is
this
is
the
way
to
define
this
so
that
other
components
respect
how
services
want
to
be
moved
around
deployment
rollout
strategies
which
are
a
sort
of
more
course
setting
that
people
may
be
more
familiar
with
those
aren't
respected
by
these
tools
because
they're,
that's
not
what
they're
set
up
to
look
at
and
finally,
for
this
slide
pod
priorities.
So
this
this
allows
you
to
decide
which
workloads
are
or
should
be
prioritized
when
your
cluster
currently
doesn't
have
enough
capacity
for
all
pods.
B
So
if
we
think
back
to
joseph's
example
again
where
one
of
the
services
was
scaled
up-
and
there
was
not
enough
room
in
the
cluster
for
all
of
those
pods,
if
instead
one
of
those
workloads
was
the
workload
that
had
been
scaled
up-
was
of
higher
priority
than
the
than
some
of
the
existing
pods
some
of
those
existing
pods.
If
they
could
have
been
evicted
and
higher
priority
pods
being
scheduled,
those
would
have
been
evicted.
B
So
you
can
effectively
say
if,
if
there's
not
currently
enough
room
on
the
cluster,
I
want
to
prioritize
this
user
impacting
workload
and
I'm
okay.
With
this
background
task
workload,
which
I'd
marked
as
lower
priority
being
evicted
for
a
period
of
time,
the
cluster
autoscaler
would
still
then
kick
in
once
those
lower
priority.
Pods
were
unscheduleable
and
scale
up
the
cluster
so
that
they
could
be
scheduled
again,
but
it
allows
you
to
get
the
higher
priority
workload
scheduled
faster.
B
So
horizontal
product
scaling-
and
so
there's
a
few
things
here,
especially
around
making
sure
that
you,
you
optimize
your
scaling
with
respect
to
cost.
So
in
potentially
you
could
have
a
service
owner
misconfigure
their
horizontal
product
to
scaling
rules,
maybe
a
metric.
That's
that's,
never
quite
reached
a
target
utilization
which
can
lead
to
runaway
scaling
one
one
way
of
handling.
This
is
name
space
level,
resource
quotas.
So
this
allows
you
to
effectively
say
in
a
number
of
dimensions,
so
pods,
cpu
and
memory.
B
What
is
the
maximum
resources
this
namespace
can
ever
make
use
of?
So
never
let
this
never
let
pods
in
this
namespace
take
up
more
than
100
cpu
cores,
say
and
that
that
will
the
horizontal
scale
will
still
work.
It
will
try
and
increase
the
desired
capacity
of
whatever
its
target
is,
but
the
resource
quota
will
then
prevent
new
pods,
which
would
cause
that
resource
quota
to
be
exceeded
from
being
created.
B
So
it's
really
important
for
preventing
any
runaway
scaling,
potentially
costing
you
hundreds
or
even
thousands
of
we
then
have
so
metrics
sources
for
non-resource
metric
based,
auto
scaling.
So
this
has
been
covered
in
great
detail
by
previous
psychotic
scaling
cube
contacts,
but
you
can
scale
as
well
as
on
resource
metrics
on
custom
or
external
metrics,
so
the
in
this
case
a
lot
of
cases.
There
will
be
a
better
metric
than
say
cpu
or
memory
utilization
for
scaling
on.
So
you
could.
B
B
If
that's
doing
a
lot
of
work,
you
can
end
up
causing
unintended
scale
up,
particularly
if
your
developers
aren't
aware
of
these
and
finally,
the
the
more
fine-grained
version
of
pod
affinities,
topology
spread
constraints,
and
so
this
is
only
in
beta
as
of
118.
So
I'm
guessing
a
lot
of
people
won't
have
had
a
chance
to
play
around
with
it.
B
Over
50
of
this
services,
pods
are
scheduled
on
just
two
instance:
types
increasing
our
potential
impact
of
a
loss
of
one
or
both
of
those
instance
types
due
to
spot
outage
or
something
similar,
and
this
is
without
using
the
spread
constraints.
We'd
really
like
to
make
use
of
them
to
obviously
limit
behavior
like
this
going
forward
vertical,
auto
scaling,
so
memory
based
vertical,
auto
scaling,
isn't
suitable
for
every
language,
I'm
being
a
bit
facetious
here
in
that
jvm
languages
can
now
be
tuned
through
jvm
parameters,
etc.
B
B
The
two
will
end
up
fighting
with
each
other
and
generally
one
or
both
will
end
up,
not
understand
understanding
what
has
gone
on
due
to
resources
changing
underneath
it,
and
so
generally
you
can.
You
can
combine
horizontal
and
vertical
auto
scaling
on
different
metrics,
but
don't
don't
try
doing
both
of
them
on
the
same
metric
at
the
same
time
and
finally
ensure
resource
policy
is
the
same,
and
so
this
is.
This
is
taking
up
the
same
role
as
the
resource
quarters.
We're
taking
with
the
cluster
auto
skate
with
the
horizontal
paradox
again.
B
Sorry,
and
so
this
is
preventing
runaway
scaling
by
saying
this
is
the
maximum
size.
I
ever
want
a
certain
container
within
this
target
to
scale
to
so
you
can
tell
make
sure
that
the
vertical
auto
scale
never
gets
into
a
situation
where
it's
runaway
scaling
up
and
a
given
container
or
a
given
pod
to
the
point
where
it's
you're
getting
ever
bigger
nodes
and
costing
yourself
money
again.
B
So
cluster
of
finally
cluster
auto
scaling,
and
so
the
the
number
one
tip
I
would
have
is
to
prioritize
node
startup
time.
So
this
not
only
has
the
benefit
of
your
cluster
scaling
up
faster,
pods
being
able
to
be
scheduled
from
unscheduleable
faster,
and
it
also
allows
you
to
tune
a
number
of
parameters
from
their
defaults
on
the
cluster
autoscaler,
so
you're,
more
tolerant
to
the
cluster
autoscaler,
realizing
that
nodes
have
developed
a
fault
whether
that's
due
to
cubelet
dying
or
a
hardware
failure
or
something
similar.
B
B
You
can
occasionally
see
instances
in
the
cloud
where,
for
instance,
the
memory
varies
slightly
from
node
to
node
and
potentially
the
cluster
autoscaler
will
think
that
it
can
schedule
a
new
pod
on
a
new
node
in
a
given
node
group,
but
that
doesn't
actually
match
when
the
new
node
is
brought
up
and
finally,
and
should
always
ensure
this
estimate.
You
have
enough
resources
on
the
nodes,
so
you
can
set
up
how
much
resources
are
reserved
for
the
cubelet
in
the
system
to
make
sure
that
they
are
operating.
B
That
was
pretty
much
it
in
terms
of
sort
of
best
practice
tips.
We've
got
a
couple
of
links
to
previous
auto
scaling
talk,
so
the
kubecon
eu
earlier
in
the
year
there's
some
good
stuff
there
on
custom,
metrics
and
other
bits,
and
we've
got
a
couple
of
links
as
well
to
some
nice
blog
posts
about
best
practices
for
autoscaling.
A
A
A
A
A
When
you
specify
a
target
for
external
metrics,
you
can
say
value
or
average
value,
and
these
two
behave
differently
when
you
say
I
would
like
to
target
this
value
for
this
external
metric,
the
hpa
by
default
every
15
seconds,
we'll
take
a
look
at
where
the
metric
is
now
and
where
you
want
to
be,
and
it
will
it
will
scale
out
or
scale
in
to
try
to
achieve
that,
and
it
will
keep
doing
that.
So
if
the
metric
doesn't
change,
it'll
keep
scaling
up
up
up
up
or
down
down
down.
A
A
B
I'm
just
just
having
a
quick
look.
I
think
there
was.
There
was
one
about
whether
there
were
recommendations
about
adding
cholesterol
to
scale
or
annotations
to
pods
the
way
I've
achieved
this
in
the
past
is
giving
the
the
pods
in
the
deployment
themselves
and
the
service
account
and
allowing
that
service
account
to
add
and
remove
annotations
to
those
those
pods
so
effectively,
they're
only
allowed
to
modify
themselves
and
that
way
they
can
implement
that.
B
But
I
think
that's
pretty
much
us
out
of
time
for
answering
questions
on
the
call.
However,
we're
going
to
continue
trying
to
answer
any
questions.
People
may
have
had,
or
we
haven't
managed
to
get
to
on
flat
channel
to
dash
kubecon
maintainer
so
and
if
you've
got
any
questions
that
we
didn't
answer,
feel
free
to
post
them
over
there
and
we'll
try
and
get
to
them.