►
From YouTube: Where's My Pod, Already? A Tale of Cold-Start Struggles on Kubernetes - Gregory Haynes, IBM
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Where's My Pod, Already? A Tale of Cold-Start Struggles on Kubernetes - Gregory Haynes, IBM
A core component of serverless platforms is scale-to-zero functionality which provides users with near 0 application cost when idle. With this comes the dreaded cold-start problem: When a user's application is not in use for an extended period an activating request can take significantly longer to complete. Knative is no different in this respect and until recently our cold-start time was measured in seconds due to the cost of Kubernetes deployments. Recently, a large effort was undertaken to improve this delay and the result is a system which scales Kubernetes deployments up from 0 in nearly a second. We will dive deep into what actually goes on inside Kubernetes when a deployment is scaled up, how we determined where time is spent, and then some of our novel techniques to drastically speed up this process.
https://sched.co/MPcb
A
Thanks
for
the
traduction,
so
as
mentioned,
my
name
is
Greg
Haynes,
some
soft
engineer
at
IBM
I
focused
primarily
on
open
source
and
cloud
software
development.
Recently
this
is
meant
focusing
on
K
native
and,
as
mentioned,
the
cold
start
performance
characteristics
of
K
native,
and
so
my
hope
here
today
is
to
give
you
a
little
bit
of
a
story
of
what
we've
worked
on
over
the
past
six
months
and
trying
to
improve
candidates.
Cold
start
performance.
A
A
So
how
we're
gonna
get
there?
I'm
gonna
start
with
a
bit
of
key
native
design
and
specifically
our
architecture
for
what
goes
on
during
scale
tip
from
zero.
Just
to
give
some
context
about
the
things
I'm
discussing
later
on.
There's
gonna
be
a
lot
of
discussion
about
investigative
techniques
and
how
specifically
how
we
can
get
tracing
spans
that
are
meaningful
and
detailed
analysis
about.
A
What's
going
on
when
we're
mutating
kubernetes
resources,
then,
through
that
process,
we're
also
going
to
look
at
how
we
can
given
this
information,
we
get
from
these
fans
how
we
can
use
that
information
to
find
some,
maybe
novel
techniques
or
some
clever
solutions
to
beat
the
performance
characteristics,
we're
seeing
and
last.
Lastly,
but
most
importantly,
we're
hoping
to
actually
get
a
result
out
of
this,
which
is
faster,
cold,
smarter
performance.
A
So
what
why
is
this
an
issue
for
us?
If
you
think
about
a
normal
application,
you
know
you're,
maybe
you're
running
a
blog
on
kubernetes
or
something
like
that.
The
s
ellos
of
the
cooperate,
control
plane
generally
are
not
affecting
your
application
right.
So
pod,
startup
time
usually
is
not
gonna,
hopefully,
is
not
going
to
affect
the
response.
A
Time
of
your
way,
application
on
kubernetes
if
it
does
through
something
terrifying,
probably
going
on
there
and
that's
great,
that's
it's
a
great
design
goal
and
it's
something
everyone
should
achieve,
and
again
it's
almost
certainly
recommended
and
to
be
avoided
at
all
cost
to
put
this
kubernetes
control
plane
within
your
applications,
data
plane
and
therefore
take
on
the
s
loz
of
the
kubernetes
control
plane.
It's
part
of
your
applications
performance,
but
here
in
K
native,
what
we're
trying
to
do
is
actually
extend
kubernetes.
A
So
our
application
to
an
extent
is
kubernetes
itself
and
therefore
it's
somewhat
unavoidable
that
the
kouhei's
control
planes
slows
our
dictate
RSL
applications,
s
ELO's.
We
are
essentially
the
platform,
and
so
we
can't
completely
avoid
this,
and
if
we
even
were
to
try
and
avoid
this,
this
would
essentially
mean
that
we're
reinventing
everything
below
us,
which
is
intentionally
not
a
design
goal.
So
these
SLO
characteristics
are
extremely
important
to
Kane
ADA's
performance
and
it's
somewhat
unavoidable.
A
Now
that
being
said,
not
everyone
is
building
a
kubernetes
and
I
understand
it's
not
always
generally
applicable,
but
many
of
the
things
we've
discovered.
Maybe
there
are
certain
areas
such
as
maybe
you're
running
tests,
infrastructure
it
where,
before
performance
of
your
test
deployments,
are
going
to
be
very
dictated
of
how
easy
your
test
infrastructure
infrastructure
is
to
use,
or
in
other
words
generally
performance,
is
something
that
people
can
always
enjoy.
Whether
or
not
it's
a
hard
frog
requirement
and
the
problem
to
state
it
again
is
that
cold
starts
in
a
native
are
costly.
A
To
give
you
some
context,
when
I
began
hacking
on
this
without
Sto
about
five
to
six
seconds,
what
this
means
to
give
someone
contacts
or
a
cold
start
we're
a
service
platform,
so
the
application
we
allow
it
to
scale
to
zero,
meaning
your
applications.
Getting.
No
work,
no
requests;
no,
no
one's
trying
to
use
it.
It's
great,
you
won't
be
you're,
not
paying
for
that
infrastructure
cost.
A
At
that
point,
we
stopped
running
it
entirely,
but
then,
when
you
get
another
request
later
on,
we
have
to
start
your
application,
so
we
have
to
pay
the
cost
associated
with
deploying
and
getting
your
application
up
and
running
and
kubernetes.
This
means
starting
a
pod
created,
creating
a
service
or
populating
the
services
endpoints,
so
that
networking
traffic
can
reach
that
pod,
and
this
is
somewhat
costly,
as
mentioned
in
the
previous
talk.
I
believe
the
target
SLO
for
pod
startup
in
kubernetes
is
99th.
Percentile
needs
to
be
below
5.
A
Second
response
time
for
pod
startup,
which
for
cold
start.
That
is
significant.
If
you
a
interesting
data
point,
you
know
github
web
books,
for
example,
generally
timeout
at
about
five
seconds.
So
if
you're
cold
start
is
above
that
that's
completely
limiting
that
use
case
for
us.
So
it's
important
that
there's
certain
targets
were
able
to
hit
to
to
provide
more
more
use
cases
and
even
better
than
that
if
we
can
get
below
one.
A
Second,
that's
when
many
applications
become
are
able
to
be
used
on
our
platform
and
present
cold
start
or
forward
in
their
interface,
and
then,
if
you
can
get
down
a
couple
hundred
milliseconds
at
that
point,
you're,
maybe
they
are
able
to
do
user
facing
applications
which
are
able
to
undergo
code
start.
That's
where
we
hope
to
get
to.
We
have
a
long
way
to
go
so
a
bit
about
our
architecture,
how
this
works.
Whenever
quest
comes
in
and
your
application
has
scaled
to
zero,
so
we're
no
longer
running
it.
A
We
have
this
pod
called
an
activator
and
it's
just
a
it's
a
deployment,
nothing
terribly
special
there
and
we
weave
out
all
networking
traffic
to
go
to
this
activator
service,
the
idea
being
the
applications
not
running
at
this
point.
Your
request
can't
really
just
float
out
in
the
ether
and
not
go
somewhere.
Something
has
to
handle
that
request
and
hold
on
to
it.
While
we
spin
up
a
pod
or
your
application
to
serve
it,
that's
the
activators
job
from
there
the
activator
knows
it
got
a
request
for
your
application.
A
A
The
autoscaler
does
this
by
talking
to
kubernetes,
and
it's
literally
sets
the
replicas
count
on
a
deployment
to
one
which
goes
and
creates
a
pod
which
goes
and
populates
an
endpoint.
Eventually,
at
the
end
of
this,
the
activator
notices
this
and
says:
ok,
I
can
now
forward
the
request
on
application
and
does
so
it's
about
a
four.
So
it's
a
roughly
four
step
process.
It's
not
terribly
complicated,
but
the
end
result
is
activator
sees
your
application
start.
A
The
first
question
is
well
where,
where
is
that
time
getting
spent?
Obviously
so
one
of
the
nice
things
were
able
to
do
in
kubernetes
is
build
off
of
ISTE.
Oh
the
idea
being
that,
then
we
get
its
observability
characteristics
for
free,
so
I've
highlighted
in
green
here.
The
request
path
of
a
request
coming
in
externally,
and
so
it
goes
as
I
mentioned
to
the
activator,
which
then
forwards
it
to
your
application.
Sto
should
be
giving
us
tracing
spans
for
those
things
immediately
and
we
can
see
if
anything
odd
is
up
and
that's
what
this
is.
A
You're
gonna
be
seeing
a
lot
of
these
chases
throughout
this
talk,
because
this
is
really
the
main
debugging
tool
we've
been
using
to
figure
out
where
time
is
spent
and
see
our
improvements.
If
you're
not
familiar
with
this
interface,
this
is
a
Zipkin
UI
and
essentially
these
are
just
spans
tracing
spans
around
different
operations
that
occurred.
What
I've
lit
the
top
span,
which
is
that
that
blue
line
you're
seeing
is
the
activator
wet.
A
So
when
the
request
comes
in
to
the
activator
that
span
begins
and
when
that
activator
responds
closes
the
span
and
I've
labeled
at
the
beginning,
that's
the
request
initially
be
coming
in
and
then
we
see
a
whole
lot
of
nothing
essentially
until
we
get
endpoint
ready
and
that's
when
our
activator
begins,
trying
to
send
a
request
to
the
user
application
and
fairly
shortly
afterwards.
You
see
that
the
request
was
served,
so
we
got
some
information
here,
but,
to
be
perfectly
honest,
you
know
it's
not
terribly
useful.
A
We
have
this
giant
block
in
the
middle
of
we're
waiting
on
something
before
our
endpoint
becomes
ready.
This
is
one
of
the
major
issues
grant
to
initially
even
using
Sto
there's
a
lot
of
context
missing
the
reason
being,
as
I
mentioned,
our
activator
talks
to
an
artist
scaler,
which
talks
to
kubernetes,
which
manages
a
set
of
resources,
which
then
creates
our
user
application.
Those
things
aren't
tied
with
the
request
lifecycle
right,
you
know
our
activator
talking
out.
Autoscaler
is
not
necessarily
one-to-one
with
the
request.
A
We
could
have
multiple
requests
come
in
and
what
to
create
want
to
scale
up
a
pod
to
one
initially
sim
with
kubernetes
resources.
There's
no
really
notion
of
a
pod
is
tied
to
this
initial
request.
That
came
in
there's
no
way.
Sto
can
know
about
that,
and
it's
a
bit
of
a
problem
because
there's
not
it's
not
a
great
solution
with
this
problem.
I
guess
I
would
say
well,
but
what
we've
done
for
now,
at
least,
is
we've
set
up
a
set
of
informers
inside
of
our
activator,
which
watches
these
kubernetes
resources.
A
So
we
sort
of
essentially
application-level
spans
that
watch
the
various
kubernetes
resources.
We
know
we're
gonna
get
created
as
part
of
this
cold
start.
It's
an
example
being
maybe
have
a
hello
world
application.
We
know
we're
gonna,
create
a
hello
world
pod
with
certain
labels
on
it.
So
we
set
up
informers
to
watch
for
those,
and
then
we
have
a
state
machine
watching.
You
know
here's
when
the
pod
started:
here's
when
it's
running
here's
when
it
became
ready.
A
Hopefully
this
you
know
gives
us
all
is
telling
us
a
story
about
what's
going
on
behind
the
scenes
when
this
activating
request
comes
in
and
as
mentioned
on
the
slide
here,
I
I
really
wish
I
had
a
better
solution
for
this
one
of
the
issues
being
as
I'm,
describing
it's
a
state
machine
that
we've
made.
That's
why
tracking
resources
and
trying
to
map
them
to
requests?
It's
not
a
one-to-one
mapping.
It's
fairly
fragile
and
not
crates
of
this
is
a
there's,
a
better
pattern
for
this
I
love.
A
So
doing
this,
though,
we're
able
to
get
a
little
bit
more
information,
most
notably
we
have
it.
We
have
a
new
span
now.
It
shows
our
pods
running
gives
us
basically
seven
seconds
of
our
pod,
we're
waiting
for
a
pod
from
when
it
starts
to
when
it
goes
ready
that
in
the
self
is
an
earth-shattering,
we
just
know
we're
waiting
on
a
pod,
but
it's
something.
Fortunately
we
did
a
little
bit
more
than
that.
A
Well
in
it
containers
it
does.
Let's
find
it
in
a
container
that
runs
some
iptables
programming
rules
and
it
containers
by
definition,
run
before
your
containers
of
your
pods
start.
So
just
off
the
bat.
This
means
they
have
to
happen
in
sequence
and
there's
always
a
base
startup
cost
with
an
operation
like
starting
container
it.
A
That
being
said,
it's
not
necessarily
an
is-2
your
problem,
but
you
this
needs
to
get
be
gotten
rid
of
if
we
can't
run
in
it
with
an
in
a
container
if
you're
looking
for
some
second
pod
startup
time,
so
fortunately,
Candida
doesn't
necessarily
hard
depend
on
Sto
at
least
everywhere.
We
can
just
simply
run
without
these
these
sidecar
on
users
applications.
It
still
use
it
elsewhere,
but
also
for
folks
who
are
very
fitting
using
it.
There's
some
alternatives,
specifically
there's
a
CNI
adapter.
A
You
start
to
see
now
it's
pod,
so
our
time
is
about
3.2
seconds
in
this
new
trace
after
we've
removed
this
in
a
container,
so
we've
halved
it
right
there,
that's
a
significant
low
hanging,
fruit
improvement
and
but
we
there's
also
an
additional
gap.
We're
starting
to
notice,
what's
going
on
here
is
the
overall
startup
time
has
decreased
significantly.
A
So
there
you
go
like
I
guess
the
the
key
thing
keep
coming
back
to
is
that
observer
ability
and
metrics
are
great
for
a
reason
that
mapping
out
of
the
way
those
two
low
hanging
pieces
of
fruit
being
fixed,
you
get
a
much
better.
I
we're
starting
to
look
a
little
lot
better
here.
We
see
now
pod
swipe
time
in
this
graph
is
two
seconds:
that's
because
there's
some
variance
in
it
due
to
several
components,
but
overall
startup
time
is
now
at
two
two
and
a
half
second
window.
A
So
it's
a
long
shot
from
our
Reba
game
and
more
than
that,
the
low-hanging
fruit
seems
to
mostly
be
gone
here.
What
we're
seeing
is
a
pod
most
of
the
things
that
we're
waiting
on
is.
Are
there
two
container
starting
up
pod
creation?
Time
takes
most
of
the
of
our
blocking
time,
and
our
request
time
at
the
end
takes
a
little
bit.
A
So
we're
gonna
have
to
dig
a
little
bit
further
to
figure
out
some
ways.
We
can
potentially
optimize
this
a
bit
more.
One
thing
you
don't
see
here
is
we
actually
added
additional
metadata
teeth,
each
of
these
traces
so
I've
described.
We
have
a
state
machine
recording
when
each
of
the
containers
and
a
pod
go
right,
starting
running
and
ready,
and
you
can
think
of
these
phases
as
started.
Is
when
essentially
the
CRI
SaneBox
create
call
happens.
So
that's
you
said,
create
a
pod.
A
It's
trying
to
create
the
environment
in
which
to
run
that
pod
running
is
essentially
when
the
CRI
container
run.
It's
called
so
it's
we've,
exact
your
program
and
we're
it's
starting
to
run
and
then
ready
is
dependent
on
Cooper
attendees
readiness
probes,
if
you
have
them
defined.
So
it's
whenever
your
application
reports
that
it's
healthy
and
you
can't
see
them
in
these
traces,
so
I've
labeled
them
for
you.
A
A
Taking
a
second
to
start
up
and
the
end
result
of
that
is
cout,
and
we
found
that
kubernetes
readiness
probes
have
a
minimum
granularity
of
about
a
second,
that's
just
the
value
used
to
the
various
probes
or
a
second.
You
can't
really
you
can't
or
second
base
you
can't
get
smaller
than
that,
so
it's
sort
of
a
hard
stop
and
we
have
to
get
clever
here.
A
Okay,
so
we
came
up
with
this
trick.
What
we're
doing
it's
long
pulling
of
these
readiness
probes?
Essentially,
so,
if
you
know
bit
about
how
can
it
works,
we
have
a
separate
container
and
everyone's
application
called
the
KU
proxy.
It's
just
a
it's
our
container,
where
we
can
do
kind
of
things
essentially,
and
what
we
do
is
we
define
a
readiness
probe
from
kubernetes
into
this
cue
proxy
and
we
don't
allow
users
to
define
readiness
probes
on
their
pod.
A
In
fact,
what
we
do
is
we
undefined
them
on
a
user's
pod
and
we
define
them
on
the
raid
and
perform
those
readiness
probes
from
our
own
cute
proxy.
The
idea
being
is
we
can
then
control
when
a
kubernetes
readiness
probe
hits
our
cue
proxy.
We're
gonna
leave
that
connection
open.
We're
not
actually
return
unhealthy
if
it's
unhealthy,
we're
just
gonna
wait
and
then
we're
gonna
pull
extremely
frequently
on
our
user
application,
so
hopefully,
eventually,
if
they
use
your
application,
returns
that
it's
healthy.
A
A
Unfortunately,
after
implementing
this,
we
still
are
seeing
this
one-second
delay.
That's
actually
that
trace
I
showed
you
before
is
from
a
code
from
a
version
of
our
code
where
we
implemented
this
long
polling,
and
what
we
found
out
is
that,
although,
although
our
solution
works
to
an
extent,
it's
really
only
setting
an
upper
bound
of
one
second
because
couplet
and
are
in
our
queue
proxy
code
can
race
starting
up.
So
essentially
our
queue
proxy
can
is
not
necessarily
started
by
the
time.
Kubla
sends
that
first
probe
in
and
there's
nothing.
We
can
do
about
that.
A
It's
gonna
get
a
TCP
reset
at
that
point
and
we
have
to
wait
the
full
second
till
the
next
probe
comes
in,
but
still
the
reason
I
mentioned.
This
is
because
for
most
applications,
sub-second
startup
isn't
a
thing,
but
maybe
you
do
want
to
have
more
accurate
readiness
probe
response
in
this
whole
concept
of
long
polling
of
readiness
probe
is
a
fair
neat
trick,
I
think
to
get
accurate
transition
times
for
when
your
pods
get
ready.
Similarly,
we
as
a
plan
to
mitigate
this,
you
can
do
over
to
an
exact
probe.
A
A
We
can
do
there
off
the
shelf,
but
let's
soak
on
the
side
here,
we
also
have
this
data
plane
delay
that's
going
on
and
if
you
notice
a
little
bit
or
if
you
notice
what's
happening,
is
it
we
have
to
send
our
traffic
to
our
pod
after
it's
become
ready
and
say
we're
talking
to
a
kubernetes
service,
for
example,
that
granny's
service,
like
the
cluster
IP,
isn't
able
to
route
traffic
to
its
backing
pods
until
the
pod
becomes
ready.
That's
the
only
time
when
an
endpoint
gets
populated.
A
So
even
if
we
did
some
clever
trick
to
hack
around
readiness,
we'd
still
be
blocked
on
this
right,
like
we're
not
be
able
to
talk
to
our
pods
directly
so
well,
we
might
have
to
talk
to
our
positively
if
you
want
to
get
around
that.
So
if
you
step
back
a
little
bit,
you
can
take
this
holistic
view
and
realize
that
there's
this
concept
of
not
ready,
it
addresses
that
pipettor
and
endpoints
so
to
clean
all
the
detail,
commands
endpoints
work
and
services.
A
You
have
a
service
which
you
know
generally
map
to
something
like
a
deployment
and
more
completely
a
set
of
pods
that
that
service
came
out
of
traffic
to
the
only
about
traffic
to
those
things
when
they're
ready.
That's
the
whole
point,
you
know
we're
not
going
to
send
traffic
to
pods
that
aren't
ready
yet
under
normal
conditions.
A
So
what
we
can
do
is
watch
this
because
in
activator,
as
I
mentioned
early
on
we're
watching
the
endpoint
object
to
know
when
a
pod
becomes
ready
already.
So
we
can
watch
that
and
say:
oh
if
there's
a
not
ready,
address
and
there's
no
ready
addresses
for
the
send
point,
go
ahead
and
send
traffic
do
those
not
ready,
addresses
or
at
least
attempt
to
perform
our
own
probing,
essentially
for
when
that
pod
becomes
ready
and
just
bypass
readiness,
all
together
from
a
community
standpoint,
says
the
nice
part
of
it.
A
Some
downsides
to
this
approach,
though,
end
up
being
that
we're
doing
our
own
load.
Balancing
at
that
point
right,
you
know
we're
an
activator
deciding
hey
here's,
a
set
of
IP
addresses.
We
can
talk
to
that
logic
ends
up
having
to
live
in
our
activator.
If
you
go
with
this
one
upside,
though,
what
or
way
to
mitigate
that
problem,
we're
only
watching
for
not
ready
addresses
as
soon
as
we
see
it,
a
service
become
ready.
A
First,
you
ends
up
being
a
pretty
neat
hack
if
you
can
get
this
working
and
in
addition
to
the
small
time
savings
what's
really
nice
here,
is
that
any
type
of
data
plane
delay
that
happens
at
scale
so
say
like
you're
running
our
largest
tier
clusters,
which
may
take
pod,
we
may
take
longer
to
configure
on
pod
startup.
That
is
bypassed
entirely
with
this
approach.
A
Roughly
this
is
where
we're
at
today,
the
most
of
the
low-hanging
fruit
has
gone
out
and
we're
at
it
we're
at
about
two
seconds
of
cold
start
time
and
there's
some
unmerged
code
and
there's
still
some
to
do.
This
is
especially
where
we've
hit
the
wall
of
now.
We
need
to
start
looking
to
more
difficult
things,
such
as
cupola
cupola
performance
enhancements,
which
we've
begun
doing
it's
a
slight
pitch
for
help
upstream
there.
A
But
that
being
said,
we
also
have
a
slightly
unrelated
thing:
that's
not
it's
not
going
to
affect
our
two-second
cold
start
performance
out
of
the
box,
but
if
you've
used
K
native
at
scale,
which
not
many
have
one
thing
you
end
up
noticing-
is
that
kubernetes
control,
plane
performance
ends
up
being
highly
variable
at
scale
ank.
This
was
mentioned
in
the
previous
talk
as
well
about
solos.
We
need
the
SL.
O's
are
dictated
about
a
cluster,
not
necessarily
being
under
high
load.
A
When,
when
your
crew
Bay's
cluster
becomes
that
our
high
load,
the
control
plane
before
latency
specifically
is
highly
variable,
and
what
I'm
showing
on
this
slide
over
here
is
on
the
left.
This
is
the
time
before
we
see
a
pod
start
creating.
So
what
I
did
here
is
I
just
doing
ten
cold
starts
at
parallel,
which
is
a
fairly
small
number,
and
we
already
see
about
a
second
with
the
cold
start.
A
Second
time
before
the
coup
based
control
plane
begins
starting
a
pod,
as
that
number
grams
up
that
becomes
significant,
so
I've
mentioned
the
the
crew
base,
control
plane,
performance
it
takes
out
is
highly
variable
and,
more
importantly,
many
of
the
components
such
as
the
replication
controller
deployment
controller
things
of
that
things
like
that
are
not
easy
to
really
scale
out.
You
know
you
can
do
H
a
and
things
like
that,
but
running
multi
master.
That
is
not
a
thing
that
they
were
really
designed
to
do,
and
it's
sort
of
a
inherent
design
flaw.
A
I
would
almost
say
of
this
reconciler
pattern
in
event.
It's
not
so
much
a
flaw
is
a
trade-off
that
would
decision
that
was
made
for
consistency
over
over
latency.
It
gets
you
reasonable
throughput.
We
were
things
like
batch
processing,
but
latency
isn't
one
of
the
upsides
of
this
design,
and
so
we're
looking
at
doing
next
and
we've
done
a
fair
bit
of
design
work
on
it's
actually
bypassing
this
humanities
control
plane,
the
idea
being.
We
can
run
the
the
activator
as
a
daemon
set
on
kubernetes.
A
So
when
requests
come
in
we'll
be
able
to
send
them
directly
to
activator
on
one
of
the
nodes
which
can
then
do
what's
do
local
scheduling.
This
is
done
because
if
you
know
how
scheduling
kubernetes
works,
there's
a
node
name
that
gets
written
into
a
pod
and
that's
what
tells
a
couplet
to
that.
I
should
start
looking
at
that
pod
definition
it's
and
run
it
locally.
Well,
as
a
user,
you
can
just
write
a
pod
definition
with
that
node
name
already
populated,
which
bypasses
scheduling
and
kubernetes
entirely,
so
we're
gonna.
A
Do
it
or
we're
hoping
to
do
is
run
activators
as
a
dataset
on
every
node
locally
scheduled
pause
so
that
same
node
when
requests
come
in,
which
bypasses
most
of
the
kubernetes
control
plan,
notably
this
doesn't
bypass
couplet.
So
things
like,
like
sidecar
injection
admission
controllers
at
several,
still
work
it
also,
but
the
reason
this
ends
up
being
okay
is
because
couplet,
unlike
most
of
the
kubernetes
control
plan,
is
chartered
and
scaled
out
by
node.
A
The
end
result
being
we
can
bypass
this
control,
plane
variability
for
the
most
part
and,
and
we
rely
on
that
they
put
parts
that
do
scale
out.
One
downside,
though,
is
if
you
notice
we're
sending
local
scheduling
to
whatever
requests
come
in.
That
means,
if
essentially,
a
load
balancer
is
responsible
for
making
scheduling
decisions
of
where
a
pause
get
placed,
and
so
we
have
some
design
that
we've
worked
out,
but
essentially
you
have
to
be
able
to
give
hints
to
your
load.
Balancer.
To
say
here
are
the
places
you
need
to.
A
You
should
schedule
requests
for
this
certain
application
if
they
come
in
ahead
of
time,
because
your
load
balancer
isn't
really
something
you
can
generally
put
a
whole
scheduler
into
and
hack
up
a
complex
scheduling
logic
for
so
you
do
ahead
of
time
placement
it's,
unfortunately,
I
don't
have
data
on
this.
We
did
a
little
bit
of
prototyping
a
while
ago,
I'm
just
a
very
simple.
A
A
And
then,
lastly,
I
just
want
to
give
a
pitch
for
coming
and
working
with
us
there's
a
whole
lot
of
work
to
be
done
and,
as
I
said,
I
was
hoping
to
kind
of
catalog
over
the
different
components
of
kn8
the
different
tricks
that
we've
found
over
the
past
six
months,
for
how
we
can
how
we
can
work
around
some.
These
base
performance
characteristics
of
kubernetes.
Hopefully
you
all
can
use
them
in
their
applications
that
they're
useful.
But
if
not,
please
come
help
us
out
in
this
website
up
here,
key
native.
A
B
A
What
are
the
points
of
improvements
in
couplet
we're
looking
for
so
one
of
the
things
that
I've
done
some
initial
analysis
on
what
we
can
do
with
it,
and
we've
realized
that
actually
contain
20,
RI
container
run
calls
are
made.
They
actually
happened
in
sequence,
not
in
parallel,
just
as
straight
so
like
every
one
of
those
calls
takes
about
0.4
seconds
depending
on
the
backend,
and
so
the
number
of
containers
you
have
actually
affects
how
long
startup
takes
so
that
and
that
was
sort
of
no-brainer.
A
When
does
the
V
that
actually
get
attached
to
the
into
the
that
that
C
group,
those
there's
no
back
reporting
interface,
you
just
say
hey,
they
must
create
in
your
CRI.
Java
goes
okay,
cool
I'm
gonna!
Do
that
at
some
point
then
I
think
couplet
is
some
polls
for
status
of
that,
so
some
type
of
edge
triggered
API.
There
would
be
very
helpful.
A
Yeah,
so
the
questions
about
whether
or
not
images
are
cached
from
the
node
already
when
you
go
to
store
an
application
we've
built
in
a
caching
sort
of
cop
out
into
a
native
where
it's
like
a
CRT
saying
like
oh
here's,
the
images
you
want
cache
and
then
we
assume
somebody
can
write
a
controller
that
can
say.
Oh
if
this
image
is
this
cache
CRD.
Has
this
image
populated
you'll
make
force
around
every
node?
A
The
questions
about
whether
like
essentially
don't
don't
cold,
start,
don't
scale
the
zero
right.
We
found
it's
really
user
specific.
Some
users
are
all
about
those
cost
savings.
Maybe
they
have
a
model
where
they
have
tons
of
applications
that
are
rarely
used,
and
some
people
are
definitely
like.
Yes,
give
me
that
switch
to
never
your
cold
start.
So
we
have
the
flag,
but
it's
definitely
not
a
thing.
We
can
just
say
you
don't
ever
do.
E
E
A
A
The
activator
is
watching
for
whether
there's
any
ready
addresses,
in
addition
to
not
ready
addresses-
and
this
is
one
of
those
it's
going
to
go
normally
through
the
cluster
IP,
because
that
should
work
and
that
kind
of
prevents
us
from
getting
this
case
of
actually
having
to
implement
of
real
load
balancer
in
the
autoscaler
or
in
the
activator.
Then
the
other
component
is
when
your
application
scales
up.
We
do
ask
your
ingress
controller
to
redirect
traffic
to
the
service
rather
than
the
activator
at
all.
F
A
So
we
we've
only
prototype
this.
The
design
we've
caught
so
far
is
when
you're,
essentially
deferring
the
load
bounced
or
introduced
scheduling,
which
is
a
horrible
scheduling
algorithm,
because,
but
all
it's
doing
is
saying
whatever
wherever
date,
whichever
node
the
request
comes
into,
that
is
where
the
pot
is
gonna
end
up,
printing
scheduled.
Our
plan
is
roughly
to
do
so.
A
It's
called
sub
setting,
so
we're
gonna
ahead
of
time
say:
look,
maybe
these
two
nodes
or
these
three
nodes
are
or
the
only
nodes
I,
should
serve
requests
for
this
application,
the
idea
being
then
you're
not
limit
you're,
limiting
the
path
of
likely.
If
comes
into
one
node,
then
the
second
request
comes
in
your
backup,
a
cold
start
every
single
time.
It's
just
ahead
of
time.
Placement
essentially.