►
From YouTube: Music Matched to Different Ways to Scale in K8s, What Could Go Wrong? - Ria Bhatia, Microsoft
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Music Matched to Different Ways to Scale in K8s, What Could Go Wrong? - Ria Bhatia, Microsoft
The other week someone asked "how do I over provision resources so I have them immediately when I need them?" and I sat there in awe because the community has already come up with a couple options on how to scale on-demand, so why are Kubernetes users still asking for this? That question is the inspiration for this talk. From the operations perspective, I'll discuss what it takes to build up the cluster autoscaler and horizontal pod autoscaler. The talk will incorporate experiments and tests that were run to come up with conclusions on how to appropriately tweak metrics so scaling is cost-effective and efficient. The truth is, most scaling within Kubernetes is metrics based so I'll also take a look at the ecosystem to see if there are other possible methods of scaling. The goal is to teach operation folks of Kubernetes on what it takes to effectively scale with help from some pop music.
To learn more click here: https://sched.co/FuKG
A
A
But
what
I'm
here
to
talk
to
you
guys
about
is
scaling
and
kubernetes,
so
will
basically
go
through
the
basics
of
what
it
means
to
scale
in
kubernetes
today
and
then
I
will
give
you
kind
of
an
insight
into
like
how
us
is
a
sure
how
we're
thinking
about
scaling
and
then
kind
of
how
like
the
community
in
general,
is
thinking
about
scaling
for
the
future.
So,
let's
get
started
with
my
awful
graphics,
so
the
cluster
autoscaler
is
really
simple.
A
All
it
means
is,
if
you
have
a
bunch
of
nodes
in
your
cluster
in
kubernetes,
and
those
nodes
are
full
with
a
bunch
of
pods,
you
need
to
be
able
to
scale
those
nodes
up,
so
the
cluster
autoscaler
allows
you
to
scale
those
nodes
up
automatically.
It's
something
that
you
can
get,
for
example,
in
gke
automatically,
it's
something
that
Asscher
will
be
offering
in
the
future.
A
A
So
then,
when
we
get
into
more
of
the
specifics
of
it,
it's
supported
with
1,000
nodes,
I'm
running
30
pods,
each
that's
what
it's
been
tested
against
and
you
can
get
these
all
from
the
kubernetes
docks
too,
and
it
also
it
scales
with
pods
have
been
failed
to
run
the
cluster
due
to
insufficient
resources.
There
was
also
nodes
that
are
in
the
cluster
that
maybe
were
underutilized
for
an
extended
period
of
time,
and
then
those
pods
can
be
placed
on
other
existing
nodes
and
then
the
node
can
spin
down.
A
So
it
does
both
for
you
it'll
scale
up
and
it
will
scale
down.
So
this
is
another
example
or
awful
animation
of
mine,
of
how
you
would
get
those
waiting
pods
and
how
they
would
schedule.
So
you
have
a
bunch
of
pods,
maybe
they're
in
a
waiting
state
and
how
they
would
schedule
back
on
to
build
a
full
node.
It's
basically
the
premise,
all
of
those
pretend
all
those
waiting,
pods
weren't
able
to
get
scheduled
there,
and
then
they
all
automatically
moved
over
to
the
right.
A
This
is
the
weirdest
slide
that
you
will
see
the
day
the
virtua-pod
autoscaler.
So
this
is
an
interesting
one.
It's
still
in
beta,
but
we're
getting
closer
to
how
we
get
your
applications
in
your
pods
suited
for
the
right
amount
of
memory
or
CPU
that
you
need
people
are
just
bad
at
resizing
their
images.
A
We
don't
really
know
how
much
CPU
we
need
how
much
memory
we
need
until
we
experiment
a
little
bit
more
until
we
actually
get
into
the
details
of
what
my
applications
doing
like
when
I
have
a
lot
of
customer
traffic
or
when
I
don't
so.
This
comes
into
play
because
it'll
dynamically
resize
it
for
you,
so
you
don't
have
to
put
in
the
guesswork
for
this
is
the
amount
of
cores
I
need,
or
this
is
the
amount
of
memory
that
I
need,
but
also
there's
a
tidbit.
A
It
shouldn't
be
used
in
conjunction
to
the
HP,
a
baseline
CPU
or
memory
metrics
cuz.
What
will
end
up
happening
is
both
these
things
are
triggered
on
CPU
and
memory,
and
then
you'll
get
a
weird
thing
where
your
pods
will
start
to
resize,
but
then,
at
the
same
time
your
pods
will
also
start
to
replicate,
and
then
people
don't
know
what's
supposed
to
happen.
There
so
I
think
there's
a
lot
of
work
that
needs
to
go
into
figuring
out
that
model
and
flush
that
out
a
little
bit
more
and
then
the
horizontal
pilot
of
scaler.
A
A
All
of
these
things
are
just
basic
numbers
like
okay
I
want,
maybe
only
a
hundred
requests
or
ten,
but
then
you
actually
have
to
go
still
through
the
basics
of
is
my
pod
gonna
fail
if
I
have
150
requests
or
will
it
fail
on
100
and
that's
unfortunate
because
I
think
that's
where
the
community
will
go
to
figuring
out
in
the
future
and
here's
another
little
graphic
of
what
this
one
does.
It
literally
just
adds
pods
to
a
node.
A
A
So
there
you
go
and
now
I'm
gonna
get
into
something.
These
are
pretty
simple
concepts.
I
think
the
the
way
that
we
scale
today
you
scale
on
nose
and
paws
are
pretty
ingrained
in
people's
minds.
I
think
is
at
this
point.
What
I
want
to
start
talking
about
is
what,
if
you
wanted
to
when
you're
thinking
about
scaling.
The
reason
you
want
to
scale
is
because
you're
probably
spending
a
lot
of
money
on
infrastructure.
A
You
probably
have
over
provisioned
your
clusters,
so
you
have
a
bunch
of
nodes
and
a
bunch
of
underutilized
compute
and
that's
why
we
end
up
scaling.
That's
really
the
reason
why
we
want
to
do
those
things.
So
if
we're
really
talking
about
money
and
underutilized
resources,
that's
what
I
want
to
bring
in
another
concept
of
scaling
and
that's
just
scaling
with
pods
and
that's
what
I
need
to
talk
to
you
guys
about
ACI.
A
So
I've
done
a
similar
vision
of
this
talk
at
Linux,
con
China
and
it
went
pretty
well-
was
pretty
cool,
but
this
is
part
of
the
slides
from
it.
So
thankfully
they're
translated,
so
we
have
a
CI
as
your
container
instances.
It's
a
managed
service
by
Azure
by
Microsoft,
there's
also
other
services
that
Alibaba
has
that
Amazon
has
that
are
competitors
and
also
friendly
partners
of
ours.
So
it
starts
in
seconds
there's
no
VM
infrastructure
that
you
have
to
manage.
So
this
is
a
new
thing
you
just
shove.
A
It
give
it
like
any
kind
of
image,
give
it
the
amount
of
resources
you
need
CPU
and
memory
like
always,
and
then
it'll
start
up
for
you,
depending
on
your
image
size.
So
I
also
want
to
know
now
we're
getting
away
from
the
fact
that
you
need
to
scale
based
on
VMs,
we're
kind
of
starting
to
think
about.
Okay,
I
put
my
application.
A
Two
pods
can
I
only
think
about
pods
and
can
I
only
think
about
the
kinds
of
pods
I
need
to
run
in
the
kinds
of
compute
I
need
for
that
specific
pod,
rather
than
that
specific
node,
and
this
is
a
concept
that
will
continually
get
back
to
that
in
kubernetes.
Today,
pods
and
nodes
are
very
tightly
bound
together.
You
can't
have
a
pod
without
a
node,
and
this
is
something
that
we're
gonna
try
and
change
in
the
future.
A
And
now
we're
jump-started
into
kubernetes.
So
in
that
same
vein,
you
can
have
virtual
nodes,
which
is
a
new
feature
within
Asscher
itself,
but
basically-
and
this
is
something
that
you
can
run
in
any
cloud.
This
is
not
as
your
specific
I'm
using
Azure,
because
I'm
part
of
Asia
I
get
free
credits
there,
because
in
part
of
that
cloud,
but
you
can
run
the
same
kind
of
architecture
on
any
other
cloud
writer
or
by
yourself.
A
So
in
the
sense
we
have
a
bunch
of
VMs
and
we
have
wastes
in
here
because
I
mean
the
kubernetes
autoscaler
will
do
its
best
to
make
sure
that
all
of
your
PM's
are
fully
utilized.
But
maybe
you're
scaling
yourself
and
you
have
some
wasted
resources
in
there
in
general,
or
maybe
your
pods
were
just
overcompensating,
maybe
of
too
many
cores,
maybe
too
much
memory,
and
then
you
could
use
this
virtual
node
concept
as
a
spillover
area.
A
So
now
you
don't
have
to
have
hey
I'm
gonna
have
like
ten
nodes
in
my
puppy
in
my
cluster,
because
in
the
future
I
need
to
scale
up
to
ten
I
need
to
have
those
ten
nodes
for
the
future
for
rent
customers.
You
don't
have
to
think
like
that
anymore.
You
can
say:
hey,
maybe
I
just
want
one
or
two,
because
I
have
a
lot
of
stable
workloads,
but
then
in
the
future,
I
might
need
X
amount
of
nodes
more,
but
really
you're.
Talking
about
I
need
X
amount
of
deployments.
A
A
A
So
what
it's
not
is
multi-tenancy,
there's
another
thing
called
multi-tenancy
in
kubernetes
that
people
are
trying
to
figure
out
how
that
works.
These
are
kind
of
two
different
things
in
one
you're,
completely
abstracted
away
from
the
fact
that
containers
are
running
and
the
other
you
are
trying
to
make
that
just
abstraction
as
a
customer
to
your
customers,
possibly
and
that's
where
multi-tenancy
locally
will
be
important
to
you,
but
these
are
kind
of
two
different
things
in
the
way
that
I'm
talking
about
it.
So
how
does
this
work?
A
How
are
we
spilling
over
and
how
are
we
spilling
over
into
a
random
service
from
kubernetes?
And
how
does
the
user
actually
use
that
when
they're
trying
to
scale,
for
example,
it's
with
virtual
kulit,
as
I
mentioned
before,
and
basically
all
virtual
couplets
doing
is
talking
to
kubernetes
all
the
kubernetes
api
is
and
then
the
acia
pis
and
they're,
basically
connecting
and
figuring
out
what
it
means
to
create
a
pod
in
kubernetes,
but
the
definition
is
different.
So
what
we
said?
A
Okay,
if
you
go
ahead,
if
the
customer
goes
ahead
and
creates
a
pod
in
kubernetes,
what
will
happen?
Is
it's
actually
calling
our
api's
and
it's
saying
create
a
container
group
in
a
CI,
but
this
concept
can
be
used
in
many
different
ways,
so
there's
actually
an
IOT
edge
provider.
You
could
also
have
your
own
cluster
and
you
could
scale
out
into
different
regions
within
the
same
cluster.
So
these
act
as
nodes.
It
just
replicates
what
the
couplet
is
so
it'll
act
as
a
node
and
all
of
those
nodes
can
replicate
a
region,
for
example.
A
So
then,
in
one
cluster
you
can
have
one
place
to
manage:
pods
from
east
us
from
west
us
from
whatever
regions
you
want,
maybe
even
in
China,
which
is
pretty
cool.
The
networking
will
not
work
across
that,
but
these
are
just
interesting
ways,
so
you
can
start
to
think
about
how
do
I
scale
out
into
different
regions
and
different,
maybe
even
different
cloud
providers.
A
It's
just
the
same
thing,
so
alright,
so
before
I
demo.
This
and
I
also
want
to
talk
about
the
ways
that
you
specified,
how
you
scale
out
into
nodes
and
today
in
kubernetes.
This
is
another
reason
to
I'll,
say:
nodes
are
too
tightly
bound
to
pods
is
that
you
still
have
to
be
able
to
specify
with
what
kind
of
know
do
you
want
to
scale
out
to
to
get
those
resources?
A
So
if
you
want
a
GPU
node,
your
hat,
you
have
to
say
in
your
pods
back
that
hey
I
want
to
go
out
to
a
node
with
a
GPU
resource,
and
this
is
the
node
name
or
this
is
the
resource
and
that's
where
anti
affinity
and
infinities
come
in
I'm.
Basically,
what
this
one's
doing?
It's
saying
in
my
cluster
I
want
to
schedule
out
to
the
normal
nodes
in
my
cluster,
but
if
those
are
full
then
spill
out
to
virtual
couplet,
that's
all
it's
saying!
A
So
that's
where
you
can
use
anti
affinities
and
affinities
to
basically
say
I
want
to
schedule
out
to
this
kind
of
node
and
if
that's
full,
maybe
you
go
here
or
maybe
I
only
want
this
work
load
on
these
kinds
of
nodes,
or
it
only
wants
this
work
load
in
this
region
or
only
want
this
work
Road
in
GPU.
So
you
can
use
anti
affinities
and
Finney's
for
that.
A
You
can
also
use
toleration,
Zanda
mission
controllers,
so
this
is
important
if
you
want,
if
you
really
don't
want
your
developers
or
the
people
even
deploying
the
pods
to
kubernetes
to
really
they
have
to
figure
out
where
it
needs
to
go.
That's
where
an
admission
controller
would
come
into
play.
You
can
use
this
audit
on
automating
webhook
that
will
basically
edit
your
pod
spec
on
demand,
which
is
pretty
cool,
so
say.
If
you
want
in
the
default
namespace
in
kubernetes,
you
want
all
of
your
pods
to
get
scheduled
out
to
virtual
coolant.
A
For
example,
this
could
automatically
add
all
your
pot
specs
and
then
it
would
say:
hey
now,
you're
allowed
to
get
scheduled
to
virtual
couplet.
This
one
doesn't
say
that
you
have
to
get
scheduled
to
it.
It
just
says
now
that
I'm
allowed
to
get
scheduled
to
virtual
coolant.
So
that's
another
way
that
you
can
use
kubernetes
to
scale
out
and
to
kubernetes
and
different
nodes.
Things
like
that.
A
A
Okay,
so
I
had
to
go
in
incognito
mode,
so
azure
would
work
with
me,
and
so
what
I'm
gonna
do
is
I
have
a
simple
application.
It's
literally
this
funky-looking
thing
right
here
you
can
go
ahead
and
buy
a
surface
book
or
an
xbox
one.
Don't
actually
use
this
website,
it
does
not
hook
it
up
to
anything
in
the
backend,
but
it's
literally
just
a
simple
website.
So
you
could
think
of
this
is
I.
A
Have
a
bunch
of
WordPress
sites
or
I
have
a
bunch
of
sites
that
I
need
to
manage
or
I
just
have
in
general,
like
a
store
front
end
that
connects
to
a
lot
of
stuff
in
the
backend.
So
what
I'm
doing
is
I'm
going
to
start
man
in
car
genome
where'd
you
go
I'm,
so
I'm,
also
using
cloud
shell
in
Azure.
It's
a
fun
little
tool
that
just
plugs
in
right
into
your
portal
into
Doc's.
A
Whatever
you
want
and
it's
a
shell
on
demand,
can
you
even
use
it
on
your
phone
I'm,
so
I'm
using
this
tool
called
hey
and
basically
it's
just
gonna
ping,
this
front-end
as
many
times
as
I
said
and
it
currently
it's
doing
200
pings
on
in
a
second.
So
that's
a
lot
and
it's
running
on
kubernetes,
of
course,
and
now
we're
gonna
head
up
to
Griffin,
because
I
think
that's
probably
one
of
the
best
ways
to
show
you
guys.
A
What's
going
on
so
already,
I
can
see
a
spike
and
what
I'm
doing
is
I'm
calculating
the
amount
of
requests
per
second
per
pod.
So
anytime,
there's
over
ten
requests
per
pod,
I'm
saying
scale
out
and
I'm
using
the
horizontal
pot
at
a
scalar
to
scale
out.
So
that's
this
is
already
automatically
happening.
The
horizontal
pod
of
scalars
in
my
cluster
and
I'll
show
you
guys
that
in
a
second
I'm
also
I'm,
not
using
the
cluster
autoscaler
I'm,
not
scaling
out
of
nodes
I'm,
just
using
pods,
because
that's
just
easier
for
me.
A
The
up
is
also
way
easier
and
we
can
see
the
RPS
one
up
and
the
response
time
took
a
sharp
dip
down,
which
is
pretty
cool
and
then
the
RPS
per
pod
is
going
up.
So
we're
gonna
start
to
see
the
response.
Time
go,
go
up
and
down
and
up
and
down,
but
basically
it
should
scale
with
dark.
The
responses.
They're
requests
per
second
with
the
pods
and
we'll
also
start
to
see
that
the
running
pods
and
the
VMS
these
it's
only
gonna
hold
up
at
about
two
but
the
running
pods
in
ACI.
A
Those
are
gonna
jump
up
really
high,
because
I'm
just
scaling
out
in
ACI
and
I
only
have
one
or
two
nodes
in
my
cluster,
which
is
pretty
great
but
I'm
scaling
out
to
a
cluster
that
would
probably
need
1020
nodes
and
if
I
was
doing
that
with
an
azure,
it
would
take
me
a
little
bit
more
time
more
time
than
I
have
today
to
actually
scale
out
to
this
scale.
But
I'm
doing
it
right
in
front
of
you
in
a
couple
minutes.
A
So
you
can
start
to
think
how
powerful
this
technology
is
and
how
powerful,
like
node
affinities
are,
and
things
like
that
that
I'm,
using
to
just
automatically
fill
up
my
cluster
and
then
scale
out
and
the
greatest
thing
is-
is
right
after
this
cluster
is
done.
We're
after
this
demos
done
I'm
gonna,
walk
away
and
it's
gonna
clean
up
on
its
own.
The
HPA
will
do
that
for
me,
it'll
start
to
on
take
down
the
pods
just
pretty
great.
So
let's
go
out
into
Azure.
A
So
if
you
want
to
see
the
music
stuff
go
there,
but
basically
the
thing
was
scaling
should
be
like
a
calming
kind
of
just
I.
Don't
know
a
hypnotic
thing:
it
shouldn't
it's,
not
something
that
you
guys
should
have
to
touch
all
the
time
you
shouldn't
be
automatically
you
shouldn't
be
manually
scaling,
it
should
be
automatically
scaling,
and
it
should
be
that
easy.
It
should
be
customers
or
just
saying
I
need
all
of
this.
I
need
these
products.
A
I'm
gonna
hit
your
web
server
really
hard
and
your
infrastructure
should
automatically
and
dynamically
adhere
to
that
and
spin
it
by
itself.
So
let's
go
back
to
your
on
a
dashboard,
so
this
is
giving
me
our
PS
per
pod.
So
that's
why
you
see
all
the
colors
they
all
will
hit
up
at
about
ten.
We
have
twelve
running
pods
in
ACI
right
now
and
now
I'm
gonna
go
here.
A
A
Yeah,
it's
good
all
right,
so
those
are
just
some
examples
in
case
things
went
awry,
but
I
did
want
to
mention
one
thing
we
are
looking
into
the
fact
of.
Why
do
you
have
two
today
scheduled
out
directly
into
a
node
to
get
those
compute
resources?
We
are
looking
at
how
we
could
change
that
in
kubernetes
and
how
basically
people
could
bring
different
os's
to
the
playing
field.
So
today
your
couplet,
the
thing
that
we've
mimicked
in
virtual
couplet
has
to
conform
to
the
OS
that
you
have.
A
We
will
be
looking
into
ways
to
change
that
in
the
future,
so,
if
you're
interested
in
joining
us,
hopefully
this
talk
will
be
completely
different
in
a
year.
Hopefully
there's
different
concepts
on
how
we
scale,
but
please
reach
out.
So
thank
you
for
coming
in.
Thank
you
for
your
time,
reach
out.
If
you
have
any
questions.
Thank
you.