►
From YouTube: Autoscaling in Kubernetes [I] - Marcin Wielgus, Google
Description
Autoscaling in Kubernetes [I] - Marcin Wielgus, Google
One of the nicest features of Kubernetes is its ability to automatically adjust the cluster size and the number of pod replicas to the current traffic and load. During this talk I will explain what is the current state of pod and node autoscaling in Kubernetes, how it exactly works, what metrics can be used to drive autoscaling, and what are the best practices to apply it in production.
About Marcin Wielgus
Marcin has been with Google since 2010, working on various projects ranging from Android applications to recommendation engines. Currently he is the part of the Kubernetes team and he is mainly focused on autoscaling and cluster federation.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
Let's
start
with
a
quite
common
of
historic,
a
dev,
a
person
meet
a
software
engineer
at
the
watercooler
and
the
first
one
asked
we
are
closing
the
budget
for
q2.
How
many
nodes
will
you
need
in
your
newest
project
and
this
software
engineer
scratches
his
head
and
says?
Well,
gosh
I,
don't
know
or
maybe,
if
he
or
she
around
some
artificial
tests
and
learned
it
for
every
500
QPS.
A
There
is
a
need
for
an
additional
web
server,
or
maybe
he
did
some
analysis
and
estimated
that
four
servers
would
probably
be
enough
even
on
spikes.
But
quite
probable
is
that
to
be
on
the
safe
side,
he
will
ask
for
five,
six,
maybe
even
seven
machines,
so
the
conversation
as
ends
with
I
need.
Seven.
The
software
engineer
is
happy
defended
of
the
question.
A
He
provided
an
estimate
which
is
quite
random
and
much
bigger
than
the
real
nice,
but
hell
no
one
likes
to
be
a
page
in
the
middle
of
the
night
and
responsible
for
the
downtime,
and
the
company
pays
couple
times
more
than
it's
actually
needed
before
this
presentation.
I
tried
to
Google
what
is
the
average
enterprise
data
utilization
I
saw
various
numbers
depending
on
who
and
how
I
was
counting
and
I
was
quite
surprised
by
the
number
rarely
very
rarely
got
to
the
level
of
15%.
A
There
are
many
reasons
why
people
over
provision.
Let
me
focus
on
a
couple
of
them.
First
of
all,
it's
hard
to
predict
what
the
tough
world
real
traffic
will
be.
It
changes
depending
on
time
of
day
day
of
week.
There
are
spikes
on
some
events
like
New,
Year
or
Superbowl.
It
can
be
hard
to
change
the
deployment
or
maybe
not
so
hard,
but
it
will
require
manual
intervention
and
nobody
likes
manual
intervention.
A
So,
in
the
absence
of
proper
automation,
we
over
provision,
but
there
is
another
way
auto-scaling
auto-scaling
is
an
ability
of
a
system
to
automatically
scale
to
the
current
needs.
It's
a
very
nice
things
which
we
are
in
comparing
incorporating
into
kubernetes
in
couple
places.
I
will
go
through
these
places.
So,
first
of
all,
we
have
horizontal
Patil
to
scale
which
controls
the
number
of
replicas
in
the
deployment.
A
Let's
start
with
replica
count
and
horizontal
cut
out
the
scaler,
what
files
into
scope
of
horizontal
scaling.
So
when
auto
scaling
replicas
count,
we
want
to
ensure
that
these
replicas
get
a
decent
load
that
it
that
each
pot
that
consumes
the
resources
in
the
cluster
has
a
reason
to
be
there.
So
it's
not
one
in
the
result
of
some
estimation
that
went
completely
off.
We
also
want
to
ensure
that
there
is
an
immediate
redundancy.
A
So
if
a
machine
or
a
pod
crashes,
the
service
will
be
still
up
and
running
and
we
also
want
to
operate
within
the
quadrant
budget.
What
does
it
exactly
mean
to
maintain
the
decent
load?
Here
we
have
an
example,
four
parts,
each
running
with
really
high
CPU
usage.
If
the
traffic
increases
there
will
be
no
capacity
within
the
post
to
handle
it.
A
A
Here
is
the
case.
Where
odds
are
not
so
utilized.
We
have
five
of
them.
They
are
running
on
40%.
One
could
say
that
60%
of
resources
is
wasted,
so
why
don't
we
remove
one
of
the
node
one
of
the
pots,
the
others
will
take
the
traffic
and
the
utilization
will
decrease
and
the
free
space
after
the
fifth
spot
can
be
used
by
someone
else.
A
A
A
We
are
talking
about
the
usage
and
what
exactly
the
use
it
is,
as
you
know,
each
part
can
declare
how
much
CPU
it
should
get
or
actually
its
container
should
get.
The
value
is
used
for
pot
scheduling.
A
pot
requesting
to
scipios
will
go
only
to
a
node
which
actually
has
this
two
CPUs
free.
Knowing
the
relation
between
the
current
CPU
consumption
and
the
declared
maximum
you
scan,
we
can
easily
tell
whether
there
is
some
capacity
left
in
the
pot
or
not.
A
Operating
on
the
ratio
instead
of
row
values
the
couples
HPA
configuration
from
pod
definition,
and
once
you
understand
it,
it's
quite
easy
to
handle.
Okay,
so
I've
showed
you
some
basic
formula.
The
reality,
unfortunately,
is
a
little
bit
more
complex.
Horizontal
pod
autoscaler
needs
to
take
care
about
a
couple
of
other
important
details.
First
of
all,
merchants
we
don't
want
to
go
from
three
replicas
to
foreign
applicants
back
and
back
every
ten
minutes,
just
because
the
traffic
changed
by
two
percent.
So
there
is
a
reasonable
margin
around
this
formula.
A
Some
pods
are
ready,
some
pods
are
starting
and
their
usage
is
not
representative
for
the
whole
population
and
they
should
not
be,
and
they
should
be
included
into
formula
in
a
slightly
different
way.
In
the
end,
some
metrics
may
be
broken
or
simply
missing,
and
we
should
also
have
a
special
handling
for
sudden
spikes.
All
in
all,
we
put
a
lot
of
logic
into
horizontal
path
autoscaler
in
order
to
make
it
work
really
really
well
for
the
users.
Okay,
how
to
turn
it
on.
A
It's
really
simple:
you
can
do
it
with
just
a
single
cube,
CTL
comment:
you
specify
the
minimum
and
the
maximum
size
of
your
deployment
and
the
target
CPU
use
it
and
your
deployment
is
of
the
scales
after
this.
This
is
comment
a
few
words
how
it
works
under
the
hood.
As
you
know,
pods
run
on
notes
and
on
note
there
is
an
agent
called
cubelet
each
cube.
A
Periodically
checks
do
node
and
adjust
the
replica
calls
on
the
controller
deployments
so
for
HP
I
it
is
critical
that
hipster
is
up
and
running
what
else
is
important,
so
I
cannot
put
less
emphasis
on
this
declare
request
for
pots.
It's
critical
not
only
for
HP
I,
but
without
request.
Hp
l
won't
work
at
all,
set
the
target
well
below
100%.
A
So
to
explain
it.
I
will
show
you
what
happens
when
you
set
the
target
to
70%,
so
this
green
pink
here
is
your
slug.
The
bigger
it
is,
the
more
of
sudden
traffic
you
can
handle
in
the
pod
before
the
new
HP
are
started,
pots
are
up
and
running,
so
please
pay
attention
to
it.
Also,
this
green
thing
is
what
scares
up
your
deployment.
You
can
look
at
it
as
the
amount
of
traffic
that
will
go
to
your
new
pots.
A
If
you
set
the
target
too
high,
you
might
have
no
ability
to
handle
the
extra
traffic
here.
You
remember
that
100%
is
basically
what
you
asked
in
your
pot
requests.
There
may
be
some
free
cycles
on
your
machine
and
you
may
go
above
100%,
but
you
should
not
count
of
it.
Also,
this
er
red
bar
may
be
too
small
to
trigger
a
decent
scale
up
or
any
scale
up
at
all
because
of
the
margins.
A
So
if
you
are
noticed
fully
packed
and
you've
said
this
type
of
usage,
you
may
have
real
real
troubles
also
keep
your
pots
and
not
healthy.
If
a
pot
is
blocks,
it
decreases
your
average
usage
and
it
may
actually
scale
you
down.
For
example,
if
one
pot
is
at
0%
and
the
other
one
is
fee
at
50
and
the
target
is
also
at
50,
then
the
deployment
may
scare
and
then
the
controller
might
scale
your
deployment
to
just
one
pot,
and
if
you
are
unlucky,
it
may
kill
the
healthy
part.
A
A
A
A
A
A
As
we
discussed
in
a
moment
ago,
there
is
horizontal
pot
autoscaler,
which
is
exactly
about
changing
the
number
of
pots.
Note
account
should
follow
HP
activity
if
there
is
a
need
for
new
pots
to
handle
the
traffic,
the
node
count
may
have
to
be
increased
so
that
the
HP
I
control
pots
have
the
place
to
run
so.
The
note
card
good
for
now
may
be
bad
for
tomorrow,
or
maybe
even
in
one
hour
on
10
minutes.
A
Note
are
expensive.
You
pay
for
them
on
your
cloud
providers.
Souls
being
spent
drift
is
bad.
On
the
other
hand,
pots
are
important;
they
earn
new
money,
they
run
your
business,
so
being
stingy
with
them
is
also
bad.
So,
as
you
can
see,
it's
a
complex
problem
and
to
handle
this
problem,
especially
at
a
larger
scale.
Proper
automation
is
needed,
but
what
should
it
look
like?
A
A
However,
if
all
of
the
nodes
are
kind
of
busy,
then
we
don't
have
a
place
to
put
the
green
pot,
so
a
new
node
should
be
provided
and
the
green
pot
will
go
there,
and
sometimes
there
is
too
many
resources
in
your
cluster.
For
example,
the
node
number
three
and
number
five
have
little
utilization.
They
have
only
this
small,
yellow
and
green
pots.
We
could
move
the
green
pot
for
not
number
five
to
node
number
three
and
then
the
last
node
will
be
empty
and
could
be
removed,
and
that's
exactly
what
cluster
autoscaler
does.
A
Let's
take
a
look
at
the
overall
architecture,
so
cluster
autoscaler
Randi
is
separate,
but
usually
on
the
master
node.
It
maintains
API
server,
watches
on
all
nodes
and
all
pots
in
the
cluster.
It
talks
to
your
cloud
provider.
You
are
running
clock
cluster
on
it
doesn't
use
any
matrix,
contrary
to
horizontal
path.
Autoscaler
like
I,
know,
currency
P,
lot
or
whatever.
It
just
focuses
on
the
API
objects.
Inside
of
the
API
server
like
pods
and
nodes,
and
their
current
state
nodes
in
cluster
autoscaler
are
evaluated
in
node
groups.
A
There
is
an
assumption
that
all
nodes
inside
of
a
single
node
group
are
similar
or
they
look
more
or
less
the
same
so
on
Google.
It
operates
on
Manatee,
instant
group
on
AWS,
on
auto
scaling
group
and
so
on.
I
will
explain
this
assumption
of
similar
nodes
in
in
a
moment.
So
how
does
it
works
inside
of
cluster
autoscaler?
There
is
a
loop
that
periodically
executes
is
a
series
of
checks
and
actions.
A
First
of
all,
it
checks
whether
the
cluster
is
in
good
overall
state.
If
half
of
the
nodes
are
broken,
then
obviously
something
is
terribly
wrong
with
the
cluster,
but,
for
example,
you
are
facing
a
network
partition
and
cluster
autoscaler
should
pause
not
to
make
the
situation
even
worse.
However,
if
there
is
just
one
and
ready
now,
then
it
should
be
able
to
continue
operation
in
Travis
version
of
Cooper
NetID.
It
wasn't
true.
A
The
cluster
autoscaler
was
stopping
once
there
was
a
single,
broken
node
with
one
point:
six
we
lifting
this
constraint
and
cluster
autoscaler
will
try
to
do
its
best,
even
if
there
are
already
nodes,
cluster
autoscaler
may
even
heal
and
remove
the
broken
notes
on
your
way.
So
it's
kind
of
like
a
popular
healing
solution.
A
A
If
there
are
on
schedule
reports,
then
cluster
autoscaler
runs
a
series
of
simulations
to
check
which
node
group
in
the
cluster,
if
expanded,
could
accommodate
some
of
the
pending
pots.
So
it
is
important
here
that
the
new
nodes
will
look
similar
to
denote
that
already
their
cluster
autoscaler
looks
at
the
existing
nodes
and
tries
to
guess
what
would
the
new
note
look
like,
and
it
assumes
that
it
will
be
similar
to
the
existing
one.
A
If
there
is
such
a
node
group
that
could
be
expanded,
it
increases
its
size
so
that
the
pending
pots
will
get
a
place
to
live
okay,
so
sometimes
we
are
not
scaling
up.
So
maybe
we
could
remove
some
nodes
and
still
have,
after
maybe
some
interruption,
all
pots
up
and
running,
so
we
are
checking
how
much
the
nodes
are
utilized
and
which
can
be
removed,
and
if
such
nodes
could
be
removed
for
long
enough,
then
we
removed
one
of
them
and
the
whole
loop
repeats.
A
Getting
rid
to
some
details,
especially
I
will
try
to
explain
what
the
unneeded
nodes
are.
So
what
does
exactly
mean
that
the
nodes
are
needed?
According
to
parent
Harris
heuristic,
a
node
can
be
considered
unneeded
if
its
utilization
is
below
50%,
both
in
terms
of
CPU
and
memory.
We
don't
want
to
harass
well
utilized
nodes,
just
get
a
little
bit
better
pot
packing
we
check
if
all
of
the
pots
running
on
the
note
can
be
moved
as
well.
A
A
And
when
we
kill
the
note
first
of
all,
we
make
sure
that
the
note
is
not
needed
for
at
least
10
minutes.
We
constantly
your
own
checks
and
do
the
simulation
whether
the
note
can
be
moved
and
where
would
the
pot
goes?
And
if
the
simulation
are
successful
for
10
minutes,
then
the
note
can
be
removed
and
we
remove
one
of
the
notes.
Of
course
we
do
it
and
if
we
haven't
any
scale
up
recently,
we
don't
want
to
go
up
and
down
constantly.
A
What
else
is
included
in
the
not
cleaning
process?
First
of
all,
of
disruption,
budgetary
issues
but
disruption
budget
were
introduced
in
1.5
and
get
a
got
a
little
bit
more
laughs
in
1.6.
So
they
are
an
effective
way
to
tell
cluster
autoscaler
not
to
touch
some
of
the
pots
or
to
remove
them
from
notes
slowly,
one
after
another.
A
A
A
A
Okay.
So
what
are
the
best
practices
of
running
cluster
autoscaler
in
your
cluster?
First
of
all,
do
not
modify
nodes
manually.
As
I
said,
a
cluster
autoscaler
assumes
that
all
nodes
in
an
oh
group
look
the
same.
They
have
the
same
CPU,
they
have
the
same
set
of
labels
and
so
on
and
so
on,
and
if
and
if
the
node
group
expanded,
we
expect
the
new
node
to
look
more
or
less
the
same
and
host
the
same
demo
set
and
manifest
run
pods
as
the
nodes
that
are
currently
in
the
cluster.
A
A
A
Cluster
of
the
scaler
can
work
with
notes
with
multiple
shapes
and
sizes.
However,
it
is
currently
in
beta
and
it
works
best
if
the
cluster
is
kind
of
homogeneous.
It
works
with
heterogeneous
cluster,
but
the
experience
is
better
with
homogeneous.
This
thing
is
likely
to
improve
in
the
future,
but
for
now
please
remember
it.
A
You
couple
it
is
0.6.
There
are
2
comments
that
may
help
you
to
tell
what
is
the
current
state
of
us
cluster
autoscaler
perséphone.
All
there
is
config
map
that
can
be
accessed
with
a
simple
cube,
CTL
comment
in
this
config
map.
We
start
in
information
about
the
current
state
of
cluster
altogether
and
information
about
what
actions
are
currently
being
used.
There
is
also
information
about
whether
there
are
some
nodes
that
could
be
possibly
removed
in
the
near
future.
A
Also,
cluster
autoscaler
published
events,
events
and
you
go
to
either
pods
or
not
if
they
are
directly
connected
to
them
or
to
this
particular
config
mode.
That
I
showed
to
you
if
the
event
is
more
about
the
whole
state
of
the
cluster,
so
please
check
them.
If
you
feel
that
something
wrong
is
going
on
around
the
number
of
nodes
in
your
cluster,
they
may
help.
A
A
So
what
is
missing
in
cluster
of
the
scalar,
so
one
of
the
thing
is
C+
autoscaler
friendly
scheduler
with
cluster
autoscaler.
We
try
to
pack
notes
as
tightly
as
possible
and
the
default
cluster
scheduler
tries
to
do
a
completely
opposite
thing,
so
it
tries
to
spread
the
load
across
the
cluster
and
put
the
pots
on
nose.
We
are
less
utilized,
so
it
completes
a
little
bit.
We
are
hoping
to
introduce
some
more
friendly
scheduler
settings
soon.
A
Then
we,
the
configuration
of
cluster
autoscaler,
depends
on
cloud
provider
on
gke.
It
is
a
single
G
cloud
comment
on
GCE.
It
is
based
on
Cuba
script
and
environment
variables.
There
is
also
a
support
for
nos
autoscaler,
in
k,
ops
for
AWS
and
so
on.
All
in
all,
it's
not
super
consistent
and
convenient
may
be
accepted
DK,
but
we
are
working
on
making
it
adequate
easier
and
smoother
for
you.
A
A
We
want
to
cover
it
all
in
the
continuous
integration
test
for
the
cluster
autoscaler,
so
it
is
rock-solid.
We
run
manual
tests,
but
they
don't
show
occasional
issues
like
the
one,
with
a
new
potent
networking
found
at
the
very
last
minute
before
one
plus
six
release,
and
we
first
ask
to
ask
you
guys
to
wait
until
one
point
6.1,
which
should
be
in
one
week
before
you
try
to
run
cluster
out
the
skeleton
it's
not
terribly
broken,
but
the
experience
may
be
a
little
bit
suboptimal.
A
We
don't
want
tore
this
to
happen
ever
again,
so
we
want
to
introduce
a
better
tester.
Also,
this
config
map
think
with
the
status
is
a
little
bit
of
a
hack.
We
are
hoping
that
in
next
release
there
will
be
something
called
component
status
and
the
status
of
cluster
autoscaler
will
be
great
there
and
in
the
end
we
want
to
hear
from
you.
Please
try
it
and
to
let
us
know
if
it
worked
for
you
or
not,
and
what
were
your
impressions
also,
we
invite
you
to
contribute
to
cluster
autoscaler.
A
A
The
reason
for
it
is
almost
completed
and
we
hope
to
have
some
proof
of
concept
alpha
version
around
the
June
anyway,
if
you
are
interested
in
VP
area
or
any
of
the
cluster
out
of
any
of
the
auto
scaling
efforts,
please
join
us
on
our
weekly
seek
out
the
scanning
meetings.
They
take
place
on
Thursday
5:30
p.m.
burning
time,
which
the
site
party
Pacific,
Standard,
Time,
okay,
and
we
have
I
believe
five
minutes
for
questions.