►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Serverless Performance on a Budget - Erwin van Eyk, Platform9
Serverless promises on-demand, optimal performance for a fixed cost. Yet, we see that the current serverless platforms do not always hold up this promise in practice; serverless applications can suffer from cold starts, platform overhead and unreliable performance. In this talk we will investigate optimizations used in current FaaS platforms and research that helps to optimize the trade-off between cost and performance: function reuse, autoscaling, resource pooling, function locality, predictive scheduling, and autoscaling. Along the way we focus on how users can optimize the Fission FaaS platform to achieve specific goals around latency, throughput, resource utilization and cost. Following this overview we take a look at the horizon; where lie the opportunities and challenges to make FaaS even faster?
To learn more click here: https://sched.co/FuKT
A
Okay,
let's
start
yeah
so
today,
I'm
going
to
talk
to
you
about
two
of
the
key
aspects
that
have
been
popular
or
at
least
have
been
vital
to
me:
research
in
popularity
of
service
computing,
as
it
is
right
now
and
like
the
first
one
of
these
compacts
is
the
performance
model.
So,
as
you
might
know
like
with
service
applications,
the
idea
is
that
the
cloud
provider
manages
basically
everything
for
you.
So
what
it
especially
does
is
doing
do
the
out
scaling
for
you
and
you
can
rely
as
a
user.
A
To
the
point
that
you
can
already
see
that
there
are,
you
can
already
read
some
blogs
where
people
are
trying
out
service
applications
to
as
back-end
to
their
web
applications
or
other
user
facing
tasks,
and
the
other
aspect
is
still
is
the
cost
model.
So
you
may
be
the
most
popular
thing
about
service
service
computing
right
now.
Is
that
the
like?
You
only
pay
for
what
you
actually
use.
A
Not
very
it
doesn't
either
give
you
something
if
you
magically
tons
of
resources
for
free,
but
what
it
does
do
is
it
tries
to
maximize
the
performance
it
can
get
out
of
a
small
amount
of
we
or
a
fixed
amount
of
costs,
and
in
this
talk,
I'm
going
to
show
you
basically
what
techniques
I
use
in
their
platforms
to
to
optimize
this.
This
trade-off
between
performance
and
cost,
but
first
before
actually
dive
into
these
optimizations
I
would
like
to
give
you
a
short
introduction
of
what
a
function
is.
A
A
service
platform
actually
looks
like
so
that's
two
limits
to
this
talk:
I'm
not
going
to
talk
to
anything
about
other
than
function
of
service
in
within
service
computing,
so
not
about
service
containers
and
I'm
only
going
to
focus
on
the
runtime
of
a
function
of
service
platform,
so
I'm
gonna
focus
on
the
developer
workflow
or
how
you
create
or
monitor
your
functions.
So
to
start
off
with
the
data.
Basically,
the
data
model
of
a
function
service
platform.
A
You
guys
gave
two
conceptual
stores
of
data,
so
in
one
hand
you
have
the
function
metadata
store
where
you
store
all
the
all.
The
the
versioning
and
configuration
related
to
your
function
and
the
reason
why
this
is
this
is
there
is
because
of
the
med
date,
there's
a
pretty
small
amount
of
data
and
a
small
amount
of
data,
and
it's
free
its
exit,
frequently
and
expected
to
be
low
latency,
because
the
juice
for
lookups
queries
and
so
on,
the
other
energy
of
the
function
store
where
you
actually
store
the
actual
functions
that
you
you're
using.
A
So,
for
example,
we
can
have
small
Python
functions,
but
this
can
also
quickly
go
very
large,
for
your
functions
are
many
dependencies
and
many
static
assets.
So
then
you
can,
then
you
can
imagine
that
your
functions
start
become
almost
no
tens
or
hundreds
of
mb/s
and
how
we
actually
can
execute.
These
functions
is
basically
using
a
cup
of
components
which
comprise
the
runtime.
So
here
you
can
see.
A
We
have
a
router
which
whiskey
is
a
responsible
hand
for
getting
in
the
requests
for
function,
executions
with
a
deployer
which
is
responsible
for
actually
deciding
how
a
function
will
be
executed.
I'm,
basically
for
most
platforms.
You
this
barrier
actually
ends
up
the
actual
deployment
of
the
resource
to
a
resource
manager
and
since
we're
here
at
this,
of
course,
is
given
at
ease
and
what
the
resource,
what
in
the
end,
what
they
are
is
a
function
instance.
A
So
in
the
function,
this
is
basically
a
worker
which
can
accept
requests
and
execute
a
function
and
respond
to
give
you
a
bit
concrete
example:
I'm
working
on
a
platform
cultivation
and
if
you
strip
out
only
more
complex
components
and
optimizations,
you
see
the
same
model
in
our
platform
because
we
have
a
router.
We
have
a
thing
called
the
executor
which
is
similar
to
a
deployer.
A
We
use
custom
research,
definitions
as
our
function
method,
data
store
and
we
have
a
small
function
store
by
default
and
also
this
model
gives
Maya
media
opportunities
also
to
address
the
elephant
in
the
room,
which
is
the
cold
start
and
the
cold
start,
basically
the
worst
case,
behavior
of
your
functions.
So
basically
you
have
no
caching
involved
or
no
shortcut,
so
you
can
take
for
executing
your
function.
So
how
this
looks
like
is
to
give
you
some
basically
basically
a
bit
of
a
facial
example.
A
A
request
comes
into
the
router,
the
relative
checks
and
has
no
function
instance
available.
So
it
will
query
the
deployer
to
actually
deploy
a
function.
The
deployer
will
fetch
the
data
from
the
function
method,
datastore
to
decide
how
to
actually
deploy
this
function.
Well.
This
decision
is
then
handed
off
to
communities
and
community
does
the
actual
deployment
of
the
function
instance,
and
only
after
all
that,
then
you
can
actually
forward
the
request
from
the
router
to
the
function
instance.
A
The
function
instance
can
execute
a
request
and
respond,
and
the
router
can
port
this
back
to
whoever
is
waiting
for
us
and
now
contrast
this
with
the
of
the
the
best
case.
I,
don't
forget:
Sri
has
another
name
but
I'm,
just
gonna
call
it
a
worm
execution
and
here,
of
course
the
the
function
instance
is
already
available.
So
that's
really
a
do
because
then
you
can
simply
forward
the
request
immediately
to
the
function
instance
and
get
back
to
response
to
the
user
or
service
that's
waiting.
A
A
A
But
it's
more
challenging
to
actually
think
about
how
much
before.
Ok,
you
get
more
performance
for
the
same
amount
of
cost.
So
the
first,
the
first
optimization,
is
what
we
call
a
function.
Resource,
reusing
and
idea
is
basically
to
reduce.
Of
course
starts.
You
do
and
the
idea
behind
this
is
a
bit
of
a
bit
of
a
theoretical
one,
because
the
most
extremist
theoretical
point
of
view,
you
don't
want
your
function,
executions
to
be
able
to
influence
the
influence
each
other
at
all.
Sorry
I
do
invest
in
the
most
extreme
case.
A
You
want
to
have
a
function
instance
per
function
execution,
so
it
also
means
that
every
function,
execution
will
have
a
cold
start
and
of
course,
you
can
try
to
avoid
this
by
trying
to
share
function.
Instances
between
function,
executions
to
be
able
to
share
these
resources
between
the
two
and
that's
not
to
say
that
that
is
the
the
the
last
one
left.
The
most
example
is
extra.
It's
always
a
bit
silly,
because
there
are
many
there.
A
These
are
typically
groups
on
the
same
machine,
so
you,
your
your
function,
already
reusing
the
same
hardware
resources
still
these
functions
are
not
shared
themselves
because
they
are
still
virtualized
or
containment,
containerized,
basically
pretty
well
isolated
from
each
other,
but
the
function
executions.
These
are
actually
sharing
the
same
function
instances.
So
it's
certainly
using
certainly
used
in
practice
just
to
update
our
model
bits.
We
can
reflect
this
optimization
by
adding
a
cache
to
the
router.
So
whenever
a
request
comes
in
to
the
router
now
we
first
check
the
cache.
A
If
there's
an
active
function
instance,
and
if
so
we
forward
it
to
the
function
instance.
If
not,
we
have
to
go
through
the
entire
cold
start,
but
of
course
it's
also
involves
a
trade-off,
because
how
long
do
you
keep
your
function?
Is
this
is
alive?
You
can
keep
them
alive
for
a
very
long
time,
ensuring
that
you
that
any
subsequent
execution
will
be
able
to
reuse
these
function
instances.
However,
of
course
it's
not
that
doesn't
mean
it.
A
You
guarantee
that
actually
will
be
useful,
so
it
might
be
that
you're
just
keeping
around
idle
resources
for
a
long
time.
So
the
opposite,
of
course,
is
done
to
keep.
It's
very
keeps
life
very
shortly.
Reducing
your
your
cost
of
idle
resources,
but
less
functions
will
be
able
to
reuse
existing
function
instances.
A
So,
and
not
only
can
you
share
these
resources
between
and
after
your
function,
your
function
instances
are
running.
You
can
also
try
to
share
them
before
the
deployment
or
during
the
deployment
and
I.
Basically,
the
insight
here
is
basically
as
your
function
instance
as
a
as
a
component
exist
out
of
two
parts.
One
part
is
the
function
specific
code
which
basically
codes
the
here
as
a
user
provide.
A
Basically
it's
all
your
business
logic
and
the
other
part
is
the
runtime
they,
basically
all
the
operation.
Logic
that's
necessary
to
actually
make
this
function.
Instance,
executable,
monitor,
Bo
and
so
forth,
and
with
this
insight,
you
can
actually
divide
the
deployment
process
into
two
stages,
so
you
can
defy
it.
You
can
first
deploy
the
runtime
creating
a
generic
runtime
and
only
then
the
second
part
of
the
process.
You
actually,
you
deployed
the
function
on
this
front,
this
generic
function,
runtime
creating
the
actual
function
instance,
and
that
allows
you
to
do
something.
A
That's
maybe
with
maybe
that's
that's
very
common
in
many
domains
which
called
research
pooling,
so
you
might,
for
example,
know
in
multi-threaded
applications.
You
have
these
triples
where
you,
which
you
use
to
try
to
avoid
this.
This
cost
the
operation
of
creating
and
deleting
threads
and
surface.
This
works
very
similar.
So
what
you
can
do
is
keep
a
pool
of
so,
for
example,
free
function,
runtimes,
and
when
we
need
to
find
two
functions,
you
will
take
to
generate
runtimes
and
only
do
the
last
step
of
this
deployment
process
to
actually
create
these
function.
A
We
can
go
for
a
very
large
pool,
basically
meaning
that
we,
whenever,
regardless
of
how
many
workloads
we
get,
we
always
have
enough
generic
runtimes
in
this
pool
to
actually
give
to
these
functions,
to
be
deployed,
increasing
their
speed,
but
of
course,
especially
if
you're
not
actually
leaving.
So
many.
A
A
We
can
do
this
with
a
a
optimization
called
prefetching,
and
the
idea
of
this
prefetching
optimization
is
that
you
fetch
your
basically,
you
proactively
place
your
functions,
function,
sources
near
the
runtimes
actually
need
them,
and
this
might
actually
make
sense
to
you
if
you
have
a
small
cluster
with
a
few
nodes
or
with
a
few
functions,
but
once
you
start
to
get
into
a
more
larger
scale,
operation
with
many
functions
with
functions
that
are
far
larger
than
our
tens
or
hundreds
of
mb/s,
then
suddenly,
this
this
cost
of
having
to
transfer
your
function.
Data.
A
Sorry,
sorry!
Your
function,
sources
to
this
runtime
can
take
quite
some
time
so,
especially
for
use
case
very
in
the
Geo
distributed
world
and
in
the
educate,
the
educators
competing
world.
You
can
see
this
as
being
very
important,
so
I'm
I'm
fairly
sure
that
iw
as
lambda
at
the
edge
and
a
classifier
using
some
form
of
prefetching
and
the
you
can,
you
can
think
about
this
prefetching
as
being
a
hierarchy
of
of
layers,
so
I
think
most
at
the
top.
A
You
see
the
remote
storage,
where
you
actually
store
where
your
function
is
actually
stored
and
below
that
you
have
several
layers
of
possible
possibilities
to
crash
your
functions
before
they
actually
are
used.
So
we
attempt
you
can
catch
them
at
the
cluster
level
or
at
the
most
extreme.
You
can
catch
them
at
even
a
runtime
level.
A
If
you
play
that,
you
already
place
them
at
the
runtimes
in
those
those
resource
pools
ahead
of
time,
and
this,
of
course
again
is
a
trade-off
because
you
can,
if
you
play
the
go
for
the
most
extreme
way
of
basically
cashing
your
functions
everywhere.
You
you're
pretty
sure
if
your
performance
of
basically
your
you
sort
of
your
cost
of
transferring
this
function
to
the
runtime,
it's
very
it's
very
minimal.
A
But
of
course
you
are,
you
are
incurring
a
lot
of
storage
costs
and
yeah,
so
incurring
all
the
storage
cost,
because
you're
caching,
everything
everywhere
and,
of
course,
the
upset
also
again
the
opposite
of
the
trade-off.
Are
you
safe
on
the
caching,
the
caching
costs,
but
you
pay
for
it
with
higher
latency
in
some
cases,
so
just
to
keep
up
with
the
pattern
here.
We
can
now
add
this.
A
A
So
that
brings
to
the
last
optimization,
which
is
function
pre-warming,
and
the
idea
of
this
is
very
interesting,
which
is
basically
that
you
try
to
anticipate
function:
executions
ahead
of
time
by
deploying
these
functions,
predictively,
what
which
we
call
pre-warming
and
it
sounds
fancy,
but
this
optimization
help
new
at
all.
You
can,
you
might
know
it
from
other
domains,
such
as
the
branch
predictor
in
CPU
or
proactive
out
scaling,
but
maybe
it's
more
is
these
explained
with
a
visual.
So
here
you
can
see
in
the
register
amount.
A
The
function
instance
that
you
need
at
one
point
and
improved
amount
of
function,
instance
that
you
deployed
and
in
like
in
a
situation
where
you
have
no
pre-warming
at
some
point.
You'll
your
function,
your
sorry.
Your
platform
will
need
well
reckon
recognize
that
it
needs
a
function
instance
and
only
then
you
can
start
to
deploy
things
and
that's
regardless
of
how
much
you
optimize
your
quotes
or
process
you.
Always
all
of
this
code
start
later
see
that
you
actually
have
well.
A
It's
also
an
active
field
of
research
in
many
of
these
related
domains
of,
for
example,
proactive
out
scaling
is
still
a
trainee
subject
for
researchers,
but
there
are
approaches
for
the
to
solve.
This
problem
is
especially
to
other
two
types.
One
is
runtime
analysis
where
you,
where
there's
a
component
constantly
analyzing
your
your
functions,
behavior
and
tries
to
based
on,
like
some
mechanism,
form
from
simple,
rule-based
to
complex
machine
learning,
approaches
to
create
a
model
to
predict
the
function,
the
future
function.
Executions,
the
other
approach,
is
basically
to
use
more
static
information.
A
So
in
use
the
information
the
function
and
serve
a
function,
service
platform
actually
has
to
its
benefit.
So
if
you,
for
example,
already
know
that
function
B,
will
we
execute
off
the
function?
A
you
can
already
start:
pre
warming
and
or
if
you,
your
user,
already,
has
set
up
a
trigger
for
it
to
execute
your
function.
Everyday,
then
you
know
when
the
function
will
be
needed
and
of
course,
it's
also
again
involves
a
trade-off
because
I
like
she's,
never
certainty.
A
So
it's
item
when
you
start
to
believe
a
that
I
election
will
actually
resolve
in
a
pre-warm
so,
for
example,
keep
the
treasures
from
believing
these
predictions
very
low,
meaning
that
even
if
there's
a
slightest
hint
that
there
might
be
a
function,
execution
incoming
that
you
already
start
preforming
your
functions
and,
of
course
this
also
leads
to
a
misprediction
probability.
The
way
you
waste
a
lot
of
resources
on
function,
instances
that
are
not
actually
that
worried
about
the
function
execution
where
they'll
actually
arrive
as
predicts.
A
It
may
be
a
dismal
example:
they're
one
of
the
oldest
tricks
in
the
book
is
the
rescue,
but
I'm
not
sure.
If
there's
a
name
for
a
blind
calling
it
Bing
heck
is
where
people
when
a
double
yes
just
started,
they
set
up
a
timer
and
then
a
key
trigger
to
their
functions.
Every
five
minutes.
Just
to
keep
it
warm
and
this,
of
course,
very
simplistic
and
only
works
for
one
function
instance,
but
it's
already
a
most
extreme
form
of
optimistic
pre-warming
and
on
the
other
side,
you
can
be
very
pessimistic.
A
A
Executions
that
accident
actually
can
don't
don't
sorry
that
cannot
benefit
from
this
pre
barring
because
they
just
weren't
flicked
it
certainly
enough
and
to
give
you
an
example
of
this,
of
this
pessimistic
pre-warming
I
just
want
to
introduce
a
project
I'm
working
on
right
now,
which
is
called
fission,
workflows
and
function.
Operation.
Breakfast
is
a
approach
to
basically
to
compose
functions
together
into
more
complex
function
compositions.
So
this
allows
users
to
basically
reuse
their
function
instead
of
writing
them
from
scratch,
or
instead
of
having
to
write
all
this
low-level
function.
A
A
A
So
so
just
get
a
small
example
daresay
you
won't
go
further.
It
gets
into
a
workflow
where
we
have
we
first
fella
dated.
Then
we
branched
off
into
a
parallel
execution,
where
in
one
branch
we
actually
resize
the
image
and
the
other
branch,
we
actually
start
to
do
a
sequential
execution
where
you
first
do
an
image
recognition,
generating
some
tag
and
attack.
We
translate
and
then
we
have
a
final
function,
which
we
recombine
these
two
branch
into
a
resized
image
with
the
caption
and
so
in
the
basically
maybe
the
most
Mel.
A
A
So
there's
a
lot
of
possible,
but
we
implement
a
very
simple
way
to
pre-warm
with
functional,
efficient
workflows
and
the
idea
there
is
that
when
one
function
starts,
you
pre-warm
everything
on
the
right
and
what's
the
horizon
horizon
is
basically
here
in
this
case
all
the
functions
that
are
right
up
in
line
after
the
first
function.
That
I
started
so
in
this
case
the
next
three
will
be
pre-warmed
and
then
once
the
first
finishes,
then
the
all
the
ones
that
are
pre
runs
can
immediately
start
and
benefits
from
this
pre-warming.
A
We
can
add
this
model,
this
small
workflow
component,
to
this
image,
of
course,
a
where
fringe
is
much
more
complex
than
then
a
single
box
here,
but
just
for
sake
of
simplicity,
it's
a
single
box
and
because
the
important
thing
here
is
that
there's
a
Maui
arrow
from
a
work
for
engine
to
the
deployer,
which
means
by
security
deployer,
is
no
longer
solely
dependent
on
the
router
for
basically
deploying
functions
now.
Also,
something
else
can
deploy
the
functions
so
yeah
too.
Well.
A
To
summarize,
we've
seen
here,
four
techniques
that
are
used
to
improve
the
performance
of
function,
surf
platforms,
we've
seen
that
function,
resource
reusing
is
used
by
basically
all
of
the
function
and
surf
platforms
to
trade
in
performance,
isolation
for
better
performance.
We've
seen
that
you
can
basically
cut
in
half
the
deployment
process
by
having
these
pools
of
generic
runtimes,
and
we
we've
seen
that
the
this
prefetch
optimisation
allows
you
to
deal
better
with
large
large
functions
on
cross,
more
distributed
settings.
A
And,
finally,
we've
seen
that
you
can
try
to
avoid
this
entire
course.
Colton
search
process
entirely
by
using
pre-warming
to
predictively
start
deploy.
Your
function
uses
a
head
of
time,
and
then
we've
also
seen
that
every
one
of
these
optimizations
is
a
trade-off
between
performance
and
cost,
and
also
that
raises
an
interesting
distinction
between
the
close
source
oaks
or
the
close
to
our
service
platforms
in
the
open
source
platforms,
because,
as
of
today,
the
close
for
most
closed
source
platforms.