►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Scaling Your Cluster (Both Ways) - Scott Coulton & Patrick Chanezon, Microsoft
Kubernetes has many ways to scale your workloads, most of what we hear about is scaling our cluster up with either with vm sets or autoscaling groups. There is another way, in this talk we will look at virtual kubelet. Virual Kubelet will allow us to talk to a cloud providers container as a service platform like ACI, fargate or ECI. We will deep dive into how you can scale your applications across virtual kubelet. One issue is the kubernetes service type has is scaling to zero due to the way routing to the pod happens if there is no pod for the service to route too. Scaling our applications to zero is just as important and scaling up. We will look at projects that integrate with the horizontal pod autoscaler that fix this issue. Allowing us to not only scale our applications up but as easily down to make our cluster truly elastic.
https://sched.co/Uacu
A
Hi
everybody
thanks
for
coming
today
we're
going
to
talk
about
how
to
scale
your
your
cluster,
so
first
I'm
Patrick,
Chan,
assume
I'm
a
cloud
advocate
at
Microsoft
and
initially
we
were
supposed
to
do
a
geo
with
my
colleague,
Scott
Colton,
who
is
a
cloud
advocate
at
Microsoft
as
well,
but
he's
based
in
Australia.
There
are
some
fires
around
Sydney,
so
he
had
to
stay
home,
so
he
won't
be
there
well
show
you
his
code
and
his
demo
all
right.
So
we're
going
to
talk
about
several
aspect
of
scaling.
A
One
is
the
horizontal
part
of
the
scalar.
Then
we'll
talk
about
how
to
scale
compute
in
your
cluster
in
general
and
then
we'll
talk
about
scaling
to
zero,
which
is
super
useful
when
you
want
to
do
serverless
types
in
I/o
on
top
of
kubernetes
and
then
I'll
show
you
a
demo
that
wraps
it
all
up
together.
A
So
first,
the
horizontal
part
of
the
autoscaler
kubernetes
is
an
Orchestrator
for
containers.
When
you,
when
you
launch
containers,
you
want
sometimes
to
scale
your
app
in
an
automatic
way
and
the
horizontal
path
autoscaler
is
there
for
you.
It
helps
you
automatically
scale
the
number
of
pods
in
a
replication
controller
or
replicas
set
based
on
looking
at
CPU
utilization
or
different
metrics
out
there.
A
So
it's
a
component
of
kubernetes,
it's
looking
at
that
metric
server
and
based
on
the
metrics
that
are
coming
back
and
depending
on
how
you
program
it
you
can
do
it
will
manage
your
deployment
and
scale
the
pods
automatically
okay.
So
that's
the
horizontal
part
of
the
scaler.
Now
what
happens
when,
when
you
schedule
some
pods
on
your
cluster
and
and
the
pods
there's
no
more
computer
resources.
If
you
are
in
the
cloud,
there
are
several
several
ways
of
doing
that.
A
There
are
two
options:
one
of
them
is
through
virtual
machines,
adding
some
compute
nodes
to
your
kubernetes
cluster.
And
if
you
are
in
the
cloud,
you
don't
need
to
go
roll
a
new
physical
server
and
plug
it
and
install
all
that
stuff
and
then
add
it
to
the
cluster.
You
can
provision
VMs
in
a
pretty
automated
way.
The
other
option-
that's
getting
more
and
more
popular,
is
called
the
virtual
couplet
in
kubernetes.
A
The
component
that
is
scheduling
your
containers
is
typically
is
called
the
couplet
and
that's
that's
a
piece
of
software,
that's
running
on
the
physical
node
and
that's
leveraging
some
primitives
from
the
operating
system
in
order
to
create
containers
or
these
isolate
isolated
units
of
compute.
So
typically,
it's
doing
that
using
primitives
from
the
Linux
kernel,
Hey
good,
to
see
you
Ben
Bangor
is
here
so
primitives
from
the
Linux
kernel
or
from
the
Windows
kernel.
Sometimes
you
can
use
things
like
like
a
VMware
isolation,
primitives
to
create
these
these
containers.
A
What
the
virtual
couplet
is
is
an
implementation
of
the
couplet
that
responds
to
the
same
to
the
same
API,
but
creates
virtual
machines
in
different
cloud
providers
or
leverages
the
cloud
providers
api's
to
create
these
units
of
compute
so
to
scale
your
compute
with
virtual
machine.
This
is
for
many
cloud
providers.
There
are
some
implementations
of
what
is
called
the
cluster
autoscaler
when
you
schedule
your
pod,
if
there's
no
more
capacity,
you
can
configure
the
cluster
autoscaler
to
automatically
go.
A
Add
some
new
nodes
to
your
cluster,
for
example,
they,
like
media,
sure
implementation
and
then
once
the
the
node
has
been
added
to
the
cluster
automatically.
The
pods
can
be
scheduled
on
that
node.
So
that's
a
pretty
simple
way
of
doing
that.
It's
a
there's,
a
high
level
of
granularity
where
there
were
you
provisioned,
a
whole
new
machine,
potentially
for
just
one
more
container,
that's
coming
in,
and
so
the
the
other
option
of
using
the
virtual
Kubla
gives
you
more
granularity
there.
A
A
When
when
kubernetes
was
designed,
it
was
really
designed
to
create
containers
based
on
the
docker
run
time
that
was
leveraging
Linux
primitives
to
isolate
the
processes,
then
the
windows
runtime
was
was
was
a
Windows
support
was
added
in
darker,
and
so
kubernetes
is
still
using
that
to
schedule
schedule
containers
on
Windows
nodes
with
a
virtual
couplet.
You
just
get
rid
of
the
dependency
on
server
nodes
and
you're.
Just
leveraging
your
cloud
providers
api's
to
provision
virtual
machines
that
correspond
to
one
container,
and
so
it's
it's.
A
It
allows
for
it's
very
good
for
scenarios
where
you
have
burstable
workloads,
for
example,
when
you
have
workloads
that
depend
on
the
length
of
the
queue
you
can
leverage
your
cloud
providers
capacity
there.
Instead
of
having
to
do
capacity
planning
for
the
number
of
nodes
that
you
would
need
there.
So
the
way
the
virtual
couplet
works
is
it's
like
the
couplet,
but
it's
leveraging
API.
A
They
are
virtual,
cubelet
is
an
open
source
project.
There
are
many
other
implementations
for
Alibaba,
AWS
and
others,
and
it
supports
Linux
containers,
Windows
containers
and
nodes
with
GPUs.
So
you
specify
that
in
the
characteristics
of
your
deployment
for
your
for
your
pod,
so
in
order
to
deploy
an
application
to
the
virtual
cubelet,
once
you
have
one
here
once
you
have
configured
your
cluster
to
implement
the
virtual
couplet
and
configure
the
virtual
Kubla
to
be
able
to
to
leverage
that
it
looks
something
like
that
with
a
node
selector.
A
When
you
specify
the
US
and
the
type
will
be
virtual
cubelet,
and
then
you
can
add
some
toleration
x'
and
the
toleration
here
is
that
here,
when
I'm
scheduling
my
my
pod
I
want
to
make
sure
Ezard
come
/,
a
CI
provider
for
the
virtual
couplet
is
already
is
provisioned
on
the
cluster.
So
you
can
specify
these
with
durations
now
scaling
to
zero.
This
there's
a
project
by
Microsoft,
that's
an
open
source
project
and
there
are
a
lot
of
other
people
participating
in
it.
A
That's
called
kada
the
kubernetes
even
driven
auto
scale,
auto
scaling
and
so
kada
is
an
open
source
project
that
you
can
run
in
your
kubernetes
cluster.
It
can
run
in
the
cloud
or
at
the
edge,
and
it
integrates
it's
one
of
these
solutions
that
helps
you
build
your
own
server,
less
implementation
on
your
own
on
top
of
kubernetes.
So
there
are
lots
of
projects
up
there
doing
that.
One
of
the
characteristics
of
Kedah
is
that
it
is
very
I'd,
say
kubernetes
centric,
where
it's
leveraging.
A
First,
it's
an
agent
to
activate
and
deactivate
a
deployment
scale
it
from
from
off
to
0
when
there
are
no
events,
so
Kara
really
allows
you
to
implement
these
surveillance
architectures,
where
you
have
a
compute
that
is
triggered
by
events
and
that
can
scale
up
automatically
based
on
the
number
of
events
in
a
queue
or
scale
down
to
0,
which
is
very
good
from
a
cost
perspective.
So
Kara
is
the
component
that
is
activating
and
deactivating
your
deployment
based
on.
A
What's
in
that
queue,
and
then
the
second,
as
that
kaida
acts
as
a
metric
server
for
kubernetes.
So
it
exposes
some
of
these
events,
data
that
you
can
use
to
make
decisions
about
whether
you
want
to
to
schedule
more
containers,
things
like
queue,
length
or
stream
lag
and
then
you'll
be
able
to
decide
based
on
that,
and
that
is
driving
the
autoscaler
to
scale
up
and
down
your
your
deployments.
So
it's
Kida
is
leveraging
the
horrid
own
horizontal,
but
autoscaler
behind
the
scenes.
A
That's
I
would
say
one
of
the
big
difference
with
other
server
last
type
of
projects
who
are
implementing
their
own.
For
this.
This
is
using
the
native
kubernetes
primitives
that
are
in
there
and
then
so.
This
allows
a
rich
integration
with
the
events
themselves,
and
so
you
can
do
things
like
completing
or
abandoning
queue
messages,
and
so
that's
the
that's
okay
de
architecture.
It's
a
it's
a
it's
a
server
that
runs
in
a
pod
in
a
service
in
your
kubernetes
cluster.
A
So
there's
a
rich
library
of
events,
sources
that
kada
implements
out
of
the
box.
Things
like
Kafka,
RabbitMQ,
azure,
storage,
use,
Azure
service,
bus
queues,
AWS,
sqs
and
Prometheus,
and
there's
constantly
new
ones
who
are
added
in
the
open
source
project.
So
you
can
use
that
with
lots
of
different
type
of
event,
source.
A
So
we
have.
We
have
installed
RabbitMQ
in
that
cluster
and
then,
where
we're
going
to
deploy
we're
going
to
deploy
a
consumer
who's
going
to
consume
the
rabbitmq
events.
And
so
when
you
were
when
you
are
deploying
the
consumer
here,
you
can
see
that
I
have
the
node
selector.
That
tells
me
that
that
consumer
that
container,
that
I
deploy
as
a
RabbitMQ
consumer,
that's
going
to
consume
events
from
rabbit
I
want
that
to
be
deployed
with
with
a
CI.
A
And
so
that's
the
the
snippet
of
text
that
you
saw
in
the
mo
before
and
then
we
can
deploy
our
scaled
object,
and
so
here
you
can
see
that
Kara
defines
an
API,
KDOT,
kubernetes,
dot,
io
and
it
defines
a
new
kind.
That's
called
a
scale
object
and
in
there
we
can
specify
things
like
polling
interval,
cooldown
period
match
max
replica
count
and
we
can
pass
some
parameters
like
which
which
rabbit
server
should
I
get
my
events
from
and
then
once
I
have
deployed.
All
that
and
I
have
configured
all
this.
A
What
I
can
do
is
start
to
deploy
a
publisher.
So
that's
a
simple
pot:
that's
going
to
send
some
events
inside
of
rabbit
and
then
I'm
going
to
watch.
Kaida.
Look
at
that
rabbitmq
I'm,
going
to
watch
the
pod.
The
horizontal
part
autoscaler
create
all
these
all
these
ACI
instances
based
on
the
load
that
it's
receiving
from
the
RabbitMQ
filling
up
and
then
I,
just
I
just
kill
the
the
even
producer,
and
you
will
see
that
Kara
is
going
to
scale
down
the
number
of
sei
instances.
A
So
that's
that's
what
I'm
going
to
show
and
then
so.
There's
an
additional
and
a
fun
thing
that
you
can
do
with
this,
which
is
there's
lots
of
component
to
install
with
it.
Microsoft
and
doctor
had
created
a
spec
a
few
months
ago.
That's
advancing
a
lot
and
that's
getting
more
and
more
useful.
That's
called
see
NAB
the
cloud
native
application
bundle.
So
that's
a
specification.
There
are
two
implementations.
One
of
them
is
docker
apps,
so
dr.
implements
in
AB
for
docker
apps.
A
The
other
implementation
by
Microsoft
is
called
Porter
and
Scott
has
created
a
whole
like
that.
All
the
containers
involving
the
demo,
like
installing
kada
in
stunning
RabbitMQ
the
producer
of
the
consumer,
all
that
wrapped
together
inside
of
a
single
bundle-
and
there
are
some
instructions
there
for
how
to
do
that.
Okay,
so
let's,
let's
now
do
the
demo
so.
A
A
Just
to
show
you
that
I'm
using
kubernetes
cluster,
that's
in
I,
don't
remember
which
region
it
is
in,
but
it's
it's
an
aqueous
cluster
that
has
been
configured
with
Kara
in
it.
So
let's
take
a
look
at
what
Caden
looks
like
if
I
do
cook
card
or
get
namespaces,
let's
see
the
namespaces.
We
have
in
that
in
that
cluster,
so
you
can
see.
I
have
a
few
new
faces
in
there.
That
must
be
a
demo,
and
then
we
have
Kade
out
there.
A
Namespace
audition
yeah
yeah,
okay.
So
let's
take
a
look
at
what's
in
that.
What's
in
that
namespace,
so
here
you
see,
we
have
the
cadet
pod
that's
running
in
there
and
we
have
a
service,
and
this
is
where
the
quedagh
Takeda
matrix
server
is
going
to
be
exposed.
So
then
I'm
going
to
go
I'm
going
to
make
that
one
bigger.
A
A
So
here
this
is
the
so
that's
the
delete
command
so
here
I'm,
going
to
do
I'm
going
to
could
cut
all
apply
this
this
job
there
that
has
a
container.
That's
the
RabbitMQ
client
I'm,
going
to
give
it
to
give
to
it
the
the
address
of
the
RabbitMQ
service,
that
is
in
my
cluster
and
what
that
client
does.
Is
that
it's
it's
just
putting
a
bunch
of
events
in
the
RabbitMQ,
so
I'm
just
going
to
deploy
that.
A
A
A
So
here
I
have
a
bunch
of
RabbitMQ
consumers
boom,
so
lots
of
them
are
pending.
Some
of
them
are
creating.
So
you
can
see
that
there's
lots
of
consumers
that
are,
and
so
what
happens
behind
the
scenes
is
that
once
I
have
done
that,
once
I
have
launched
the
producer
events,
it's
loading
RabbitMQ
with
a
bunch
of
events.
A
Then
kada
is
listening
to
that
RabbitMQ
and
using
the
horizontal
potahto
scaler.
To
tell
it
hey.
We
need
more
of
this
of
this
deployment
and
that's
what
that's
what
the
positive
scalar
is
just
adding
more
these
pods
to
the
deployment
and
these
pods
as
you
remember,
they
were
specified
as
a
CI,
so
these
are
virtual
node,
a
CI
Linux
in
there
in
terms
of
types
I,
think
I
have
limited
it
to
30,
not
to
grow.
My
Asha
bill
too
much.
Let's
take
a
look
here.
A
We
can
see
some
of
the
metrics
that
are
coming
to
say:
hey.
We
have
one
replica.
We
have
four.
We
have
eight
and
boom.
We
reach
the
thirty
max
here,
and
here
you
have
things
like
the
number
of
average
time
that
it
takes
to
to
do
them.
So
now,
what
I'm
going
to
do
is
I'm
going
to
delete
the
deployment
in
order
to
in
order
to
show
that
Kara
is
going
to
scale
down
to
zero
by
itself.
So
I
don't
have
anything
to
do
there.
A
So
I'm,
just
I
delete
that
I.
Did
it
that
bad
and
in
there
I
should
start
seeing
some
interesting
things
in
the
in
the
HP,
a
where
the
horizontal
potahto
scalar
is
going
to
notice
that
so
Kira
is
going
to
tell
it
that
there
is
no
more
events
in
there
that
we,
we
know,
need
as
many
year
as
many
of
these
containers
and
you
can
see
they're
all
terminating
in
there
boom
boom
boom.
So
it's
scaling
down
my
my
deployment
back
to
zero
because
there's
no
more
events
to
consume
in
there.
A
Yeah
please
from
current
slide
yeah.
So
so
that's
the
demo.
You
can
find
the
coudn't
the
demo
either
at
this
URL
or
as
another
from
it's
part
of
that
demo
is
part
of
a
whole
set
of
learning
content
that
the
cloud
that
we
Casey
team
at
Microsoft,
that
we
build
for
a
series
of
events
that
we
are
doing
around
the
world
called
ignite.
A
A
Okay,
so
that's
so
that's
it
for
the
for
the
presentation
and
the
demo
I'll
tweet
the
link
to
the
demo.
I
went
when
that
when
that
presentation
is
finished
and
to
summarize
what
I
presented
today
is
different
ways
to
scale
your
workloads
with
kubernetes,
one
of
them
is
to
use
the
horizontal
potahto
scalar.
A
A
So
the
question
is:
how
far
can
you
scale
it
with
Keita
and
the
answer
is:
it
depends
on
your
provider,
so
here
I
have
setup
kada
to
go
between
0
and
30,
using
an
ECI
implementation.
So
there
are
some
limits
to
I.
Actually
don't
know
from
the
top
of
my
head.
What
the
limit
for
sei
is
in
a
specific
region,
but
the
limit
becomes
the
limit
of
your
of
your
provider
implementation.
A
A
So
the
question
as
I
understand
it
is:
can
they
be
a
kind
of
race
condition
where
both
HPA
and
Cade
are
going
to
try
to
scale
the
same
workloads?
I
guess
in
that
case
it
depends
on
how
you
have
configured
your
workloads.
You
can
deploy
it
by
asking
the
horizontal
part
autoscaler
to
take
care
of
it.
Then,
if
you're
using
Keita,
you're
configuring
you're
putting
some
data
in
there
to
say,
hey
I
want
this
deployment
to
be
managed
by
Kara.
A
So
that's
a
good
question:
is
there
work
to
merge
kaida
with
K
native
I?
Don't
know,
I,
don't
know
the
answer
to
that
I
guess
you
would
have
to
go.
Ask
the
Qaeda
team,
some
of
the
team
members
I,
think
are
at
the
Microsoft
booth
this
week,
or
else
just
ping
me
on
Twitter
with
a
question
and
I'll
try
to
find
the
answer
for
you,
one
of
the
things
that
I
see
there
is
that
K
native
has
a
lot
of
components
in
there.
A
I'm
pretty
sure
that
parts
of
what
kada
provides
must
be
implemented
in
a
different
way
in
there.
So
I
don't
know
what
Kira
is
a
much
smaller
system
that
is
very
integrated
with
primeras
and
doesn't
need
any
additional
things
like
ECU
or
you
can
use
it
with
any
service
mesh
that
you
want
there.
So
it's
I'd
say
it's
a
slightly
different
design
philosophy
than
K
native
and
I.
Think
yes,
one
more
question.
Maybe.
A
So
what
is
the
startup
time
for
these
workloads
for
the
virtual
cubelet?
So
what
is
the
start
time
for
the
four
containers
created
with
the
virtual
couplet?
The
answer
is,
it
depends.
It
depends
on
your
provider
so
for
a
CI
I
am
pretty
sure.
We
must
have
that
in
the
documentation.
I
I,
don't
know
it
from
the
top
of
my
from
the
top
of
my
mind.
That
said,
I
know.
If
you
saw
the
demo,
it's
so
yeah
or
you
can
try
it
at
home.