►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Knative: Scaling From 0 to Infinity - Joseph Burnett & Mark Chmarny, Google
Knative Serving builds on Kubernetes and Istio to provide request driven compute that can scale to 0. It supports a variety of pluggable scaling strategies and tools for managing concurrency and trade-offs between cost and latency. This presentation reviews the challenges of designing and building a system that can rapidly scale serverless workloads to 1000's of pods. And scaling to and from 0 when idle. It presents the scaling capabilities and extension points of Knative Serving and how to configure it to take advantage of both horizontal and vertical scaling. And the concurrency controls to configure Knative Serving as a single-threaded function framework or a multi-threaded application framework (or something in-between).
To learn more: https://sched.co/Gra9
A
B
A
Name
is
March
Marnie
I'm,
one
of
the
technical
program
managers
at
Google,
working
specifically
on
serve
lists
on
top
of
kubernetes,
so,
like
I
said,
this
is
the
fourth
day
we
already
had
I
think
11
sessions
depending
if
you
count
some
of
the
off
sites
or
not.
So
hopefully,
you
have
got
some
of
the
introductory
information
about
key
native.
A
If
you
haven't
we'll
capture
as
little
in
following
slides,
there
are
still
two
other
sessions
today,
so
this
is
total
or
thirteen,
which
is
in
itself
very
amazing
thing,
given
that
this
project
just
launched
four
months
ago.
So
if
you
haven't
attended
any
one
of
those
births
with
other
sessions,
please
do
just
a
quick
recap.
If
you
haven't
made
to
those
previous
sessions-
or
you
haven't
heard
about
Candide
before
we
try
to
shove
everything
in
what
a
slide.
A
So
there's
a
lot
of
goodness
there
at
the
core
there's
three
things.
We
would
like
you
to
walk
away,
knowing
about
Kay
native,
it's
simple
to
start
with,
there's
a
lot
of
emphasis
have
been
placed
on
kind
of
obfuscating,
the
complexity
of
cover
of
cornice
itself.
There's
a
lot
of
things
that
happen
for
you
automatically,
including
auto
scaling
which
we're
going
to
be
talking
about
today
and,
on
the
end
of
the
day,
there's
a
lot
of
different
platforms
that
you've
used
before
they
were
super
simple
for
the
hello
world
example.
A
A
So,
let's,
let's
go
into
this
I,
think
for
the
purposes
of
kind
of
level
setting
with
everybody
to
the
same
we're
going
to
capture
very
quickly
the
kind
of
principal
objects
that
are
implementing
key
native
there's
a
configuration
that
represents
instead
of
the
state
of
the
application,
both
config
the
code,
as
well
as
the
configuration
that
you
would
deploy.
We
you
follow
the
twelve
factor
principle
separating
the
code
and
the
application.
A
There
is
a
revision
that
represents
a
point
in
time,
snapshot
of
that
of
those
two
their
code
and
configuration
that
allow
you
to
kind
of
roll
back
to
and
from
if
you
wanted
to,
and
there
is
a
route
which
Maps
the
traffic
to
a
particular
version
of
the
of
the
revision.
There's
also
a
subset
of
these
objects
represented
by
service
or
K
service,
as
you
might
have
seen.
Some
of
this
referred
to
on
a
conference
here
that
allows
you
to
kind
of
manage
these
three
principal
objects
in
one
construct:
a
kind
of
high-level
object.
B
So
the
pod
autoscaler
is
another
custom
resource.
Just
like
these
other
boxes.
Here,
it's
a
entity
that
you
write
to
at
CD
has
its
own
controller,
the
pod
autoscaler
abstracts,
the
auto
scaling
requirements
and
the
auto
scaling
state
of
the
revision
so
for
each
revision,
there's
a
pot
on
a
scaler,
and
it
gives
us
some
really
nice,
extensibility
and
control
that
I'll
talk
a
little
bit
more
in
a
minute.
Awesome.
A
B
A
very
fast
auto
scaling
so
responds
within
seconds.
So
this
is.
This
is
pretty
powerful,
but
it
also
scales,
two
zeros,
there's
some
orchestration
that
you
get
out
of
the
box
in
R
and
a
scale
zero
when
you
don't
need
it
in
scale
back
I'm
going
to
show
you
some
of
that,
but
before
I
do
I
want
to
give
you
a
little
bit
of
orientation
to
the
screen
that
we're
gonna
be
looking
at.
This
is
an
intermediate
session,
so,
of
course
it
has
command
line.
A
B
Right,
more
contact
more
panels,
okay,
so
this
is
a
service.
This
is
what
what
Mark
was
talking
about,
that
top
level
box,
where
you
just
basically
give
it
a
container,
describe
how
you
want
to
run
it
and
how
you
want
to
roll
it
out.
I
have
another
panel
that
I'm
going
to
be
showing.
You
looks
like
this
on
the
upper
left-hand
corner.
There
are
all
the
pod
auto
scalars
that
are
deployed
in
this
cluster.
B
So
right
now,
there's
one
pot
of
a
scale
which
means
there
was
one
revision:
it's
currently
not
getting
any
traffic,
and
so,
as
you
can
see,
on
the
right
hand,
side,
there
are
no
pots.
Okay,
nothing's
wrong.
This
thing
is
ready
to
serve
traffic.
The
revisions
ready,
but
there's
no
pause
around
okay.
So
what
I'm
gonna
do
is
I'm
gonna.
Take
you
over
here
and
I
mean
we're
gonna
hit
this
application.
So
I've
got
this
wired
up
to
a
custom
domain.
You
can
see
how
to
do
this
in
many
of
the
Canada
documentation.
B
I'm
gonna
hit
it
with
a
request.
Oh,
what's
up
already,
okay,
I
guess
got
a
Google
fired
off
a
request
before
I
ever
hit,
so
there's
a
pod
now,
so
this
thing
scales
from
zero
and
it
will
go
away
again
if
I
stop
sending
a
traffic.
So
this
is
pretty
powerful.
Actually,
you
don't
have
to
worry
about
what
happens
after
you're
done
with
something
just
take
traffic
away
from
it
and
it'll
get
garbage
collected
on
its
own.
B
So
as
I
as
I
go
through
this
demo,
I
want
to
try
to
fulfill
promise,
which
is
the
KDF
comes
with
what
you
need
out
of
the
box
to
get
started,
but
Katie
it
is
auto-scaling
will
grow
with
you
as
your
service
gross.
That
means
that
you're
not
gonna
have
to
suddenly
have
a
point
where
you
reach,
or
everything
can
ativ.
B
A
lot
of
scaling
will
allow
you
to
address
your
new
and
evolving
requirements,
so
in
order
to
keep
it
real,
what
I'm
going
to
do
is
I'm
going
to
start
a
little
bit
a
load
against
this
revision,
I'm
going
to
send
it
traffic
through
the
whole
demo
as
I
evolve.
This
application,
through
the
various
auto
scale,
auto
scaling
configurations,
and
so
you
will
get
to
see
that
this
thing
actually
still
continues
to
work
and
serve
customers,
even
as
we
get
into
some
more
advanced,
auto
scaling
months,
so
I'm
going
to
say,
go,
run
load.
B
What
this
is
gonna
do
is
it's
gonna
start
16,
like
processes,
making
continuous
requests
against
this
revision.
I
happen
to
catch
it
right
when
it
was
trying
to
scale
down.
So
one
pot
is
going
down.
One
pod
coming
back
up
again,
you'll
see
that
the
last
response
here,
always
the
how
many
requests
have
been
made.
Each
one
consumes
a
little
bit
of
CPU
okay,
so
here's
my
load
generator
now
I'm
gonna
show
you
what
it
looks
like
when
you
roll
out
a
new
new
revision.
Okay,
so
here
is
my
service
again
and
I'm.
B
Gonna
I'm
gonna
change
a
few
things
about
this.
This
is
the
first
customization
that
I'm
gonna
do
suppose
that
my
service
has
a
whole
bunch
of
startup
latency
like
it
needs
to
do
some
pre-processing
on
the
first
starts
up
and
I
don't
want
my
I,
don't
actually
want
to
scale
to
zero.
I
can
actually
specify
min
scale
zero
and
tell
it
just
stick
around
and
I
can
adjust
the
auto
scaling
target
as
well.
So
let's
say
I
want
a
target
of
two
to
what
so
auto
scaling
like
a
native
auto
scaling
is
based
on
concurrency.
B
So
that's
the
number
of
requests
that
are
in
flight
per
pod,
so
the
cluster
that
I
deployed
here
has
a
default
of
ten.
So
for
each
pod,
it's
trying
to
keep
ten
requests
going
on
that
pod
at
any
given
time.
So
if
traffic
increases
you
get
more
pod,
if
latency
increases
you
get
more
pods,
but
that
seems
to
be
a
little
bit
too
heavy
for
my
application.
B
B
Oh
one
thing
I
forgot
to
tell
you
is
that
that
service
animal
I
have
a
little
script,
that
just
watches
it
and
applies
it
every
time
I
head
in
it,
so
you'll
see
me,
make
changes,
I'm
just
hitting
save
and
then
it
gets
applied
and
you
can
see
it
rolled
out
a
new
revision
here.
You
can
see
it's
serving
the
new
data
right.
It's
a
giving
me
cube
con.
The
old
one
is
still
there.
It
even
has
a
pod
running
still
because
it
hasn't
gone
away
entirely,
but
there's
more
pods.
B
A
So
Joe
has
showed
us
the
default
auto
scaling
strategy
in
key
native,
which
is
based
on
the
requests
per
second
rather
than
utilization.
It
showed
us
some
configuration
capabilities
that
are
available
in
a
manifest
for
service
itself
that
allow
us
to
define
a
minimum,
the
maximum,
the
explicit
target
and
other
capabilities
therein,
but
how
about
if
whatever
type
of
strategies
are
included
with
key
native,
that
I
can
easily
switch
to,
for
example,
can
I
go
back
to
the
standard
cpu-based
autoscaler
that
we
are
used
to
with
kubernetes
I'm.
A
B
You
can
so
the
pop
out
of
scheme
there
there
is
is
a
resource
that
actually
has
two
controllers.
Inkay
native,
and
if
you
want
to
change
the
strategy,
you
can
actually
select
a
different
controller.
If
you
want
to
go
back
to
each
CPU
based
auto
scaling,
maybe
you
just
want
to.
Maybe
your
your
service
doesn't
need
to
scale
on
request.
Maybe
it's
just
CPU
bound.
B
You
could
do
that
so
I'm,
going
to
show
you
how
okay
over
here,
I'm,
actually
going
to
change
the
metric
to
target
CPU
and
as
I
mentioned,
this
is
actually
implemented
by
a
different
controller
and
so
I
actually
specified
class.
Now
what
the
is
a
select
on
class
to
decide
which
pot
out
of
skills
they
should
operon
they
operate
on.
So
instead
of
the
default
kPa
or
K
native
autoscaler
I'm
gonna
go
with
an
HB
HB,
a
make
sure
I
spelled
that
right.
B
Okay,
so
I'm,
selecting
the
HPA
class
I'm
gonna
say
this:
is
our
CPU
based
one
and
I'm
gonna?
Show
you
one
more
cool
feature
about
the
service
animals?
You
can
roll
things
out
in
an
incremental
fashion,
so
you'll
this
going
from
the
currency
based
auto
scaling,
CPU
based
auto
scaling.
Maybe
it's
a
little
bit
hairy,
so
I
want
to
do
this
a
little
bit
at
a
time,
so
I'm
gonna
roll
it
out
to
50%
and
kind
of
make
sure
it's
ok
before
I
couldn't
continue
right.
B
This
is
this
is
how
we
actually
release
software
in
production.
Ok,
so
I'm
gonna
save
that
again
it
applied
it.
Let's
see
what
happens.
Oh,
we
got
a
new
revision.
That's
great!
You
can
see.
There's
one
pod!
You
can
see
it's
serving
about
50%,
which
the
traffic
here
ok
and
I'm
gonna
show
you
something
that
it
created.
B
So,
as
I
said,
this
just
uses
the
regular
kubernetes
horizontal
pod
autoscaler,
so
I'm
gonna
show
you
that
there's
actually
there's
an
HPA
that
it
turned
around
and
created
here
and
delegated
to
that,
so
that
this
is
using
the
normal
kubernetes,
auto
scaling
that
you,
you
know
and
love
gap.
So
this
looks
good
to
me
so
I'm
actually
gonna
go
ahead
and
roll
it
out
to
100%
delete
the
old
one
go
back
to
0.
B
A
So
Joe
has
showed
a
lot
of
flexibility
and
control
points
they're
already
exposed
to
any
developer
who's
owned
solely
through
the
manifest.
This
is
a
service
manifest
the
developer,
applied
in
a
llamo
file,
applied
and
pushed
it,
which
is
great
from
a
flexibility
standpoint,
but
I
could
see
the
ops
people
going.
So
what
are
some
of
the
operational
controls
that
can
ADA
provides
in
order
to
kind
of
have
a
little
more
of
a
boundary
or
control
points
that
I
can
introduce
to
the
deployment
sure.
B
So
it's
it's
true
like
if,
eventually
you
don't
really
want
somebody
just
going
and
modifying
production
and
rolling
out
arbitrary
strategies
now,
Canada
Canada
addresses
this
in
part
by
making
revisions
and
mutable.
So
revisions
are
immutable
snapshots
of
the
code
plus
the
configuration
so
every
time
you
change
something.
It
rolls
out
a
new
revision,
so
you
can
always
roll
back
right.
That's
it's
really
nice
that
way,
but
sometimes
you
don't
necessarily
want
to
roll
out
a
new
revision
like,
for
example,
maybe
it's
New
Year's
Eve
right.
B
You
have
a
very
popular
video
and
transcoding
app
and
you
are
running
at
80%
of
your
quota
and
you
realize
that
you
need
to
change
your
auto
scaling
parameters.
You
need
to
tweak
things
a
little
bit.
You
don't
want
to
roll
out
a
new
revision
right.
It's
10
minutes
before
a
New,
Year's
Eve
you
don't
have.
This
is
a
bad
idea,
so
maybe
you
would
like
to
just
reach
under
the
hood
and
tweak
some
of
the
other
scaling
parameters.
This
is
another
reason
why
it's
useful
to
have
the
auto
scaling
state
as
a
separate
resource.
B
The
pot
autoscaler
is
actually
mutable,
so
you
can
actually
Akal
this
down
to
your
sra,
org
or
whoever
you
think
should
have
that
power
and
you
can
change
the
parameters
in
place
and
they'll
take
effect
immediately
without
necessarily
rolling
out
a
new
version.
So
I'm
gonna
show
you
what
that
looks
like
and
come
back
over
here,
but
instead
of
changing
the
service,
I'm
gonna
go
modify
the
revisions
that
exist
already
so
I'm
going
to
just
use
Q
patrol
edit
and
I'm
going
to
modify
this
second
app.
B
Okay
and
I'm
also
going
to
modify
the
CPU
based,
autoscaler
and
I
realized
that,
even
though
we
said
infinity
infinity
is
not
a
good
idea.
So
I'm
gonna
go
with
max
scale
of
10,
okay
and
I'm
gonna
change
the
target
a
little
bit
so
target
you
know
means
something
different
depending
on
what
your
metric
is.
B
So
what
I'm
gonna
do
is
as
soon
as
I
save
this
you're
gonna
see
those
values
propagate
over
here
to
the
horizontal
pot
out
of
skimmer,
so
I'll
save
this
and
apply
it,
and
you
can
see
now
we're
targeting
50%
with
a
max
positive
10,
no,
no
revision,
just
kind
of
modifying
things
in
place,
and
this
works
the
same
way
for
the
the
Cape,
the
K
native
autoscaler
as
well.
So
yeah,
you
got
the
usual
kubernetes
goodness
for
managing
this
resource.
It's
good!
If
it's
a
more
powerful
resource
that
you
can
restrict
access
to
great.
A
So
we've
seen
the
automatic,
auto
scaling
that
basically
comes
out
of
the
box
source.
We've
been
calling
it
now
you've
seen
a
lot
of
control
points
where
you
can
impact
this
yourself.
We
also
have
seen
some
guard
guardrails.
You
can
define
Joe
talk
little
about
the
control,
organizational
controls
you
can
implement
on
the
end
of
the
day.
This
is
the
auto
scalars
implemented
as
like
CRD.
So
you
can
apply
our
back
rules
to
this.
That
will
be
organization
specific
within
your
team.
So
that's
all
great
there's
a
lot
of
flexibility.
A
B
Yeah
sometimes
sometimes
you
need
something
entirely
different
I,
don't
know,
maybe
something
we
haven't
thought
about.
That's
the
whole
point
right
is
to
put
these
extension
points
in
place
so
that
you
have
a
nice
escape
hatch.
So,
as
I
said,
we
have
two
controllers
that
we've
implemented
for
the
pod
autoscaler.
You
can
make
another
controller.
It
selects
based
on
class.
B
So
as
a
demo
of
how
easy
that
is
to
do
I've
created
my
own
controller
I,
call
it
the
Yolo
controller
right
and
what
this
thing
does
so
just
this
is
very
simple
for
the
purpose
of
this
demo.
If
you
actually
want
to
do
this
in
production,
there's
a
nice
example,
there's
a
sample
controller
here
that
you
can
copy
I
started
with
that
it
just
was.
It
wasn't
very
good
for
the
purpose
demo,
because
it
was.
You
know
too
many
lines
too.
B
Is
like
you
know,
50
lines
of
code
here,
alright,
so
what
it
does
is
it
establishes
a
watch
on
pause,
auto
spirits,
okay,
so
we
can
see
changes
to
them,
selects
just
those
that
are
annotated
with
Yolo.
Alright,
those
are
my
those
are
my
pod
Auto
skewers.
Rather
than
kPa
or
HPA
auto-scaling
negative
death.
It
runs
a
very
intelligent
algorithm
here
called
recommended
scale
which
does
something
okay.
Maybe
this
reads
for
database,
maybe
and
runs
a
machine
learning
algorithm.
B
You
know
this
is
a
controller
running
in
your
cluster,
so
you
can
give
it
permission
to
do
what
it
needs
to
do.
This
is
your
control
plan
code
right
then
you
just
update
the
scale
ref.
That's
part
of
the
pod
autoscaler
and
update
the
status.
So
the
one
other
signal
you
have
to
give
is,
you
have
to
say,
ready,
equals
true,
so
that
the
higher
level
controllers,
like
the
route
controller,
knows
that
it
can
send
you
traffic.
B
So,
for
example,
the
KPA
checks
to
make
sure
the
endpoints
are
there
make
sure
the
pot
actually
started,
and
then
it
says
okay,
I'm
ready
to
go
so
I'm
gonna
deploy
a
Yolo
revision.
So
don't
worry
about
the
metric,
we
don't
need
metrics,
okay,
this
is
gonna,
be
Yolo
because
you
live
only
once.
That's
right,
just
go
for
it
as
a
matter
of
fact.
We're
gonna
go
ahead
and
release
it
all
of
us,
because
that's
what
this
is
sorry
I
just
deployed
this
thing.
You
can
see
there's
a
new
revision
here
up
and
running.
B
My
controller
has
complete
control
over
the
pot
autoscaler
I'm,
the
one
that
decides
the
reason
for
the
status
right.
It
just
said:
I'm
ready,
I'm,
ready
to
go,
didn't
even
matter
and
even
check
the
pods
or
anything
and
it
scaled
too
right
and
there
it
is
so
now
I'm
serving
Yolo
and
just
to
kind
of
drive
the
point
home.
You
can
update
the
controller
too
so
I'm
gonna
come
back
over
here
and
I'm
gonna
I'm
gonna
make
my
algorithm
a
little
bit
smarter.
D
B
It's
mine,
that's
right,
I
mean
really.
We
know
the
answer
is
42,
but
I
don't
want
to
attempt
to
face
now.
I'm
gonna
build
my
controller
okay,
so
I'm
gonna
run
a
tool
called
Co
and
what
Co
does
is.
It
builds
go
binaries
in
user
space.
It
just
looks
in
your
yah
mph
or
something
that
looks
like
it's
probably
a
go.
Binary
builds
it
uploads
it
to
your.
Your
repository
gets
the
fully
qualified
hash
injects
it
back
in
and
then
queue
control
applies.
The
thing
just
basically
makes
it
easy
to
deploy.
Stuffs.
B
I
won't
have
words
with
you.
It's
awesome.
It's
one
of
these
random
things
that
came
out
of
Matt
Moore's
brain
isn't
really
cool.
If
you
happen
to
be
using,
go
and
you're
using
a
lot
of
yamo,
it
makes
things
a
little
smoother.
So
here
it
is
it
it's
built
to
deploy
it
uploaded
update
the
deployment
came
over
here.
Hailer,
our
algorithm
has
run
we've
calculated.
We
need
four
pods
notice
that
it
only
touched
the
yellow
revision
and
that's
it
I
mean
we're
now,
where
we're
using
my
controller.
B
A
A
You
can
actually
implement
strategies
that
would
be
based
on
other
variables
outside
of
the
cluster
Joe
mention
machine
learning
algorithm,
they
might
be
using
parameters
or
inputs
they're
from
some
gateway
upstream
or
even
some
use
cases
where
you
have
n
number
of
micro
services
that
implement
a
pipeline,
and
maybe
the
one
downstream
is
to
be
scaled
based
on
the
input
from
the
further
upstream.
So
a
lot
of
different
variables
and
customizations,
you
can
do
there
right
any
other.
No.
A
Hopefully,
we've
showed
you
at
least
a
few
ways.
You
can
kind
of
customize
the
auto
scaling
strategy
inside
of
K
native
we've,
put
up
a
number
of
different
links
and
documentation.
There
is
Joe
also
provided
every
single
sample
that
you've
seen
during
the
demo
today
on
his
repo.
We
have
ability
to
both
follow
the
instructions
for
installing
key
native
itself
by
yourself
on
a
kubernetes
cluster.
A
In
our
repository
we
also
have
a
gke
atom
that
it's
now
available
on
DCP
on
google
cloud
that
you
can
kind
of
subscribe
to
and
just
with
a
single
command
provide
a
fully
configured
and
validated
cluster.
We
also
have
octave
slack
channels,
including
one
dedicated
to
auto
scaling.
So
if
you
want
to
learn
more
about
it
and
ask
the
people
who
actually
work
on
this
from
the
broader
community,
this
is
a
good
place
to
ask
anything
else.
Just
ping
me
on
Twitter
and
we
love
to
love,
to
talk.
A
B
Yeah
sure
so
the
question
was:
is
this
following
the
pattern
of
ingress?
Yes,
it
is
ingress,
has
a
problem
which
is
it
has
an
explosion
of
annotations
I
think
this
could
have
the
same
problem.
I,
don't
know
if
I
have
an
answer
other
than
yeah.
We
need
to
make
sure
that
we
we
manage
that
we
have
fully
qualified
the
class.
C
A
Session
before
this
one
by
t-mobile
showed
I,
don't
have
any.
You
know,
intimate
knowledge
how
they
deploy
in
production
anything
but
showed
a
store,
locator
application
that
that's
definitely
some
portion
of
that
running
in
production.
I
think
the
IBM
team
has
something
out
there
too
I
think
in
a
channel.
There
was
a
couple
of
examples:
there
people
listed
too
I,
don't
I,
don't
have
a
good
of
a
knowledge
outside
of
Google,
but
yeah
not.
B
Of
the
one
of
the
reasons
why
I've
been
focusing
on
this
extensibility
is
because,
if
concurrency
based
auto-scaling
doesn't
pan
out,
I
don't
want
anyone
in
production
to
be
stuck
so
I
want
you
to
be
able
to
sort
of
tweak
the
necessary
parameters
to
make
it
work
for
you
so
yeah.
We
really
need
a
lot
more
production
experience.
I
encourage
you
to
try
it
tell
me
what
you
think,
give
some
Beach
feedback
and
if
there's
anything
missing,
tell
us
tell
us
about
your
use
cases
and.
A
This
might
be
a
great
point,
even
if
you're
not
developer,
and
you
are
working
on
a
specific
use
case
that
it's
not
addressable.
Currently
just
please
go
into
github
submit
a
issue.
It
doesn't
have
to
be
some
pulled
with
code
or
anything
like
this.
Just
tell
us
what's
missing
and
that's
the
only
way
Kennedy
was
gonna
get
better
other.
B
Latency
based
scaling,
I,
don't
have
examples
of
it.
This
the
concurrency
based
scaling,
does
take
latency
into
account
because,
of
course
it's
like
QPS
+,
latency
I,
have
it
like
I
mean
I.
Have
them
the
data
there,
like
one
of
the
reasons,
one
of
the
ways
that
I
was
able
to
achieve
fast
scaling
is
by
billing.
My
own
metrics
pipeline,
like
the
the
pods
kind
of
communicate
directly
with
the
autoscaler,
because
it
needs
to
be
fast,
I
have
latency.
I
was
I,
wanted
to
add
QPS
and
latency,
both
as
an
alternative
strategy.
B
So
cold
start
time,
like
from
the
very
from
zero
right
now
without
a
network
mash
it's
about
four
seconds
which
is
too
long.
It's
and
we
know
it's
too
long,
because
really,
if
you're
gonna
rely
on
server
list
to
be
like
fully
server
list,
you
want
to
know
that
you're
cuz,
it's
gonna,
come
back.
I.
Think
iBM
is
starting
to
spearhead
some
work
to
get
sub-second
scaling.
I
know
it's
possible,
it's
just
a
matter
of
sort
of
teasing
things
apart
in
a
way
that
it's
gonna
be
saying.
Yeah.
C
B
Is
there
a
limit
on
the
concurrency?
Well,
that's
a
good
question.
So
the
autoscaler
set
the
target
right
and
it
sort
of
tries
to
achieve
that,
but
it
might
not.
There
is
another
place
where
concurrency
is
referenced,
which
is
in
the
which
is
in
the
revision
spec
and
gets
propagated
to
the
pot
autoscaler
spec
that
says
container
concurrency,
that's
the
actual
limit
of
what
your
container
can
handle.
We
enforce
that.
So
you
can.
B
You
can't
enforce
a
concurrency
limit
and
if
you
don't
specify
a
target,
we
will
base
our
our
default
target
on
that
value
because
of
course
it
sounds
like
concurrency
is
important,
so
there
is
the
ability
of
limit
concurrency.
We
want
this
to
work,
isn't
like
work
for
apps,
but
also
for
like
single
threaded
functions
as
well.
Does
that
answer
your.
A
B
A
B
A
B
D
D
B
Scaling
down
so
one
thing:
I
didn't
talk
about
it
all,
was
scaling
up
and
down
nodes
and
that
actually
is
very
highly
dependent
on
your
cloud
provider.
So
right
now,
that's
way
that
those
signals
are
get
to
the
cluster.
Auto
scaling
is
just
by
scheduling
or
on
scheduling
or
deleting
pods.
So
candy
dip
doesn't
really
help
you
with
with
scaling
your
notes
on.
A
D
B
A
Yes,
the
answer
is
depends
but
technically
yes,
the
the
challenge
we
would
probably
see
is
the
versioning.
So
if
you're
deploying
K
native
with
all
the
dependency,
we
will
make
sure
that
the
right
version
of
East
you
is
configured
so
assuming
all
this
variables
were
met
in
a
tech
in
technically
yes,
but
to
be
honest,
I,
don't
think
we
have
done
a
lot
of
that
kind
of
validation.
Yet,
like.
A
B
A
Dedicate
right
now
is
there's
the
recommended
process.
Yes
deployed
cluster
pre
instance.
We
this
is
zero
to
version
within
the
gke
atom
we
are
trying
to,
because
we
have
a
lot
more
control.
We
know
how
it's
going
to
be
deployed
and
run
we're
running
we're
trying
to
address
those
kind
of
concerns.
How
do
you
upgrade
the
cluster
after
the
fact?
How
do
you
scale
up
and
down?
How
do
you
address
some
of
the
traffic
shaping
and
all
that?
But
that's
none
of
that.
It's
right
now
in
their.