►
Description
Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Stateful Serverless and the Elephant in the Room - Stephan Ewen, Ververica
Kubernetes and FaaS have solved most of the challenges of dealing with stateless applications. But when it comes to handling state, it quickly becomes “someone else's problem”. Because of that, we have struggled with the same issues of data consistency and complex failure semantics for the past decade. For stateful applications, we are still far from the smooth development and operations experience associated with serverless. Stateful Functions is a new pattern to solve this problem. It uses containerized, event-driven functions with a stream processor (Apache Flink), not a database, to manage the state and handle the messaging between functions. This talk walks through the ideas behind Stateful Functions and shows how this simple framework solves the problem of consistency and failure semantics within and across functions, at the same time staying true to the serverless experience.
https://sched.co/ZelA
A
A
A
So
when
kubecon
was
originally
scheduled
as
a
physical
event,
and
I
was
I
was
starting
to
make
my
slides
I
had
I
had
some
introductory
motivation
slides,
but
then
I
came
actually
across
something
really
nice
of
you
a
few
days
ago
and
which
actually
captures
the
motivation
much
better
than
what
I
had
initially
laid
out.
So
I
just
decided
to
borrow
that
instead,
it's
a
it's
a
quote
from
from
kelsey
and
a
dilbert
comic
and
yeah.
A
What
what
they're
saying
here
is,
if,
if
we're
starting
out
with
the
with
the
non-scalable
application,
working
together
with
let's
say
legacy
database
and
so
on-
and
we
put
this
in
on
top
of
a
really
cool
infrastructure,
you
know
no
matter
how
fancy
features
of
that
infrastructure.
We
use
we're
we're
pretty
pretty
much
still
ending
up
with
a
non-scalable
application
and
yeah.
A
So
the
the
idea
in
this
and
this
discussion
that
that
kelsey
mentioned
there
is
we
should
we
should
probably
be
looking-
or
at
least
that's
how
I
understand
it.
We
should
probably
be
looking
for
for
for
something
in
the
application.
They
are
also
invest
in
a
new
application
frameworks
and
libraries
and
so
on
to
yeah
to
to
kind
of
have
an
have
a
way
to
to
make
applications
developed.
A
On
top
of
these
frameworks,
which
may
or
may
not
build
on
systems
like
kubernetes
yeah,
something
like
magically
scalable,
it's
probably
not
going
to
be
that
magical
in
the
end,
but
yeah
just
look
at
application
frameworks
and
libraries
that
help
build
these.
These
distributed
scalable
applications
and
and
scalability
really
is
only
one
aspect.
I
think
that's
that's
being
that's
being
mentioned
here.
There's
there's
a
lot
of
other
aspects
that
that
matter
as
well.
A
Of
course,
there's
the
scalability
to
computation
and
state
there's
consistency
of
the
application
security
observability
a
lot
of
things
that
that
currently,
despite
all
the
efforts
on
the
infrastructure
level,
we
still
have
to
worry
a
lot
about
doing
the
development
of
the
of
the
application
layer
and
and
that
that
is
very
much.
What
what
I
would
like
like
to
talk
about
today
and
like
an
idea
in
which
direction
these
this
distributed.
A
Application
frameworks
and
libraries
could
go
and
we're
we're
we're
building
a
project
that
falls
into
this
category
in
the
in
the
apache
flink
project
called
called
stateful
functions.
That's
looking
at
some
of
these
aspects,
not
all
of
them.
I
don't.
I
don't
think
it's
it's
yet
the
holy
grail,
but
it
has
some
some
very
interesting
ideas
or
interesting
directions
and
thoughts
that
I
would
like
to
share
and
yeah,
hopefully
also
get
some
some
comments
and
feedback
on
from
the
community
here.
A
The
other
part
of
the
title,
the
the
is
the
elephant
in
the
room.
So
what
do
I
yeah?
What
is
what
is
the
elephant
in
the
room
here
and
while
we
were
working
on
this
project,
one
one
thing
that
we
that
we
came
across
repeatedly
is
that
a
big
part
of
why
and
and
my
my
personal
belief,
we're
we're
still
struggling
with
with
aspects
like
scalable,
state
and
consistency
and
so
on.
A
All
together
is
that
we're
somewhat
still
relying
on
on
a
bit
of
an
an
age
old
model
with
the
traditional
request
response
databases.
It's
not
saying
databases
are
are
a
bad
thing
here,
they're,
actually
a
pretty
amazing
thing,
but
I
think
we
have
to.
We
have
to
rethink
a
few
aspects
of
of
the
database
to
to
make
it
a
like
a
really
fitting
player
for
for
this
effort
of
inherently
scalable
and
and
consistent
applications.
A
A
I
think
15
years
older
or
something
like
that,
but
it's
it
actually
is
it's
always
a
pretty
influential
papers
like
some
of
you
may
have
have
come,
come
across
it,
as
has
been
pretty
popular,
it's
a
while
all,
but
it's
a
bit
like
a
like
a
bob
dylan
song.
It's
like
it
has
some
some
timeless
truth
in
it,
and
I
hope
that
patelland,
wouldn't
wouldn't
kill
me
for
summing
it
up
in
in
such
a
in
such
a
simple
way,
but
it
it.
A
In
my
words,
it
advocates
for
for
building
applications
based
on
stateful
event-driven
entities
or
stateful
actors
and
in
some
sense
that's
really
the
core
of
what
we're.
What
we're
trying
to
do
so
to
explain
really
the
the
ideas
that
we're
trying
to
bring
into
this
stateful
surface
space.
Here,
let
me
introduce
you
to
the
to
the
staple
functions.
Project
first
describe
what
it
is
and
then
then
give
give
a
brief
example
of
how
how
you
can
use
this
to
build
applications
so
stable
functions.
A
The
subtitle
is
a
mouthful
polyglot
event,
driven
functions
for
distributed
stateful
applications
I'll
try
to
break
this
down
into
a
few
more
digestible
units
over
the
next
slides
yeah.
As
mentioned
in
the
introduction,
the
the
core
idea
is
to
to
try
and
build
like
an
api
framework
that
helps
with
building
distributed
stateful
applications
with
enhanced
scalability
and
consistency
and
and
very
much
like
the
the
motivation
in
that
in
that
paper,
that
I
that
I
mentioned
we're,
starting
out
with
a
with
a
building
block.
A
That's,
as
the
name
also
suggests
functions,
so
the
the
idea
is
to
to
model
the
model
applications
as
very
fine
grained
functions
that
that
represent
entities
and
really
represent
them
at
a
very
fine
grant
level.
Let's
say:
there's
one
function
for
each
user
that
that
is
currently
that
exists
in
your
system.
A
In
some
sense,
it
bears
some
similarities
to
if
you
wish,
also
actor
programming
with
a
similar,
fine,
grained
modeling
mechanism.
The
the
really
crucial
thing
here
is,
as
the
other
part
of
the
name
stateful
function
suggests.
This
is
the
stateful
nature
of
this
function,
so
each
function
contains
or
has
access
to
to
local
state.
You
can
almost
think
about
it
like
a
local
variable.
Each
function
can
access
a
local
variable
to
to
look
at
its
current
state
and
it
has
access
only
to
that
little
bit
of
state.
So
let's
say
the
function.
A
So
you
can,
if
you
want
to
think
of
this
whole
thing,
a
bit
like
a
a
transactionally,
consistent
virtual
stateful
actor
actor
system,
stateful
actors
in
the
sense
of
stable
functions
that
message
each
other
arbitrarily
virtual,
in
the
sense
that
you
don't
have
to
assume
that
they
occupy
resources
when
they're
not
active,
so
they're
not
like
extending
threads
or
even
you
know,
entries
in
memory
and
consistent
because
messaging
and
stator
are
like
transactionally
integrated.
You
don't
have
to
worry
about
messages,
getting
lost,
getting
duplicated
state.
A
A
The
way
we
the
way
we're
building
this
is,
it
might
be
a
bit
surprising
initially,
but
it
actually
doesn't
have
a
database
behind
it.
Instead,
it
it
actually,
rather
than
proxying
all
this
data
into
a
database.
It's
actually
working
based
on
a
distributed
snapshot
model
so
between
all
these
functions
that
are
that
are
there
with
all
their
with
all
the
state.
A
The
system
draws
a
snapshot
across
all
these
functions,
and
this
snapshot
is
consistently
taking
care
of
current
state
and
and
in-flight
messages,
and
so
on
to
make
sure
that
if
we
use
this
to
recover
from
a
failure,
we
we
don't
duplicate
or
lose
messages
or
or
anything
communication
with
the
outside
world
is
with
in
event,
ingress
and
egresses.
So
this
is
strictly
an
event-driven
system,
and
also
this
integration
of
the
outside
world
is
strictly
transactional.
A
How
does
it
look
like
if
you,
if
you
use
this
system
like
this,
what
what
what
components
does
it
consist
of?
If
you
deploy
it
and
it
it
consists
of
yeah?
It
consists
pretty
much
like
a
like
a
classical
disaggregated
architecture
with
you
know,
with
the
state
and
the
database
and
the
stateless
application
logic
of
these
two
parts.
Also,
we
have
the
stateful
part,
which
is
the
the
stafford
functions
cluster
I'm
running
with
running
running
on
top
of
the
apache
link
system.
A
Here,
instead
of
of
a
database
and
and
being
subscribed
to
the
ingresses
and
the
egress
streams,
and
then
we
have,
we
have
our
functions
which
we
can
deploy
in
in
pretty
much
in
any
way
that
you
know
that
we
that
we
can
that
we
can
deploy
state
this
piece
of
computation
with.
So
we
can.
We
can
put
this
in
a
container
within
with
a
request
handler
http,
request
handler
behind
a
service
and
then
deployed
on
kubernetes.
We
can.
A
We
can
take
these
functions
and
and
put
them
onto
something
like
a
function
as
a
service,
a
platform
like
aws
lambda,
or
we
can
encapsulate
them
in
in
a
different
way
behind
a
microservice
endpoint,
and
the
interesting
thing
to
notice
here
is
that
these
these
functions
really
are,
even
though
they
they
act
in
a
stateful
way.
They
are,
they
are
actually
completely
stateless.
A
All
state
that
they
work
with
is
always
supplied
to
them
on
demand
as
part
of
the
invocation
and
as
part
of
the
message.
So
state
and
messages
are
given
to
the
functions,
basically
together
as
one
unit
atomically,
and
that
makes
the
functions
themselves
really
really
pure
and
stable.
So
they're
they're
very
easy
to
scale
they're
very
easy
to
maintain
there
you
can,
you
can
really
operate
them
like
a
pure,
stateless
application.
A
This
there's
one
core
aspect
in
here
in
the
in
this
philosophy
that
that
I
want
to
highlight,
which
is
which
is
very
interesting,
and
it
goes
back
to
to
this
whole
elephant
in
the
room,
the
database.
We
have
to
rethink
a
few
of
these
aspects
there
and
I
tried
to
explain
it.
Maybe
based
on
on
this
slide,
there's
a
slight
type
on
the
left
hand
side.
The
second
stream
of
input
events
should
actually
be
the
result
or
output
events.
Sorry
about
that.
A
So
if
we
look
at
a
if
we
look
at
a
traditional
database
based
architecture,
you
know
computational
part
and
a
database
part
we.
We
typically
start
starting
the
application.
The
application
receives
either
the
event
or
it
receives
the
the
http
request
or
any
other
form
of
request
with
which
it
is
invoked,
and
then
it
it
talks
to
the
to
the
database
and
and
updates
the
stage,
gets
a
confirmation
or
a
failure,
deals
with
that
and
then
it
you
know.
A
If
you
wish,
the
database
is
the
part
that
receives
the
events
and
manages
the
state,
and
then
it
invokes
the
application
logic
with
the
state
and
the
events
together,
and
it
can
decide
when
to
involve
the
application
logic,
for
example,
when
nobody
else
is
currently
working
on
that
state,
so
to
make
sure
that
that
state
is
exclusively
used
so
in
some
sense
we're
we're
changing
the
role
of
the
of
the
driving
party
of
the
application.
From
from
the
from
the
application
logic
to
to
the
database
and
the
part
that
reacts
is
the
other
part.
A
The
function
would
decide
what
to
do
with
it.
Typically,
you
know
in
this
case.
Just
you
know,
add
this
item
to
the
card
and
then
respond
with
the
updated
state
and
the
set
of
messages
that
it
wants
to
send
out
which
goes
again
back
to
the
system.
So
the
system
will
then
update
the
state
and
send
the
event,
and
it
will
make
sure
that
this
is
a
transactional
unit.
So
there's
no
failure.
I
cannot
be
no
failure
in
between
once
the
function
responded
and
this
response
has
been
accepted
like
you're,
safe
and
yeah.
A
So
the
important
thing
here
would
be
that
yeah
everything
is
is
driven
by
is
driven
by
the
by
the
system,
that
in
one
unit,
manages
state
and
messaging
together
and
and
the
functions
really
become
kind
of
pure
and
even
though
they're
stayed
full.
You
can
pretend
they're
stateless.
They
they
are
only
stateful,
because
state
is
supplied
as
part
of
the
invocation
on
demand.
A
If
we
try
to
you
know
not
deploy
such
a
system,
how
what?
What
parts
would
we
what
we
need?
How
does
this
look
like
actually
using
it
on
on
on
infrastructure
like
kubernetes,
so
we
would
actually
need
the
the
logic
of
the
functions
which
we
would
put,
let's
say
just
on
a
deployment
and
create
a
service
in
front
of
them,
and
we
can
even
put
a
horizontal,
auto
scaler
around
them.
If
you
know,
if
we
expect
varying
load,
then
we
would
have
our
stateful
function
deployment.
That's
that's!
A
That's
an
apache
link
deployment
consisting
of
several
processes.
It's
actually
a
deployment
here,
not
a
stateful
set,
because
apache
flink
doesn't
really
work
with
with
like
stateful
volumes
or
anything,
but
instead
it
is
persistence
by
snapshots
through
something
to
something
like
s3
or
hdfs
or
so
so.
This
is
another
part
that
we
need,
but
it's
it's
an
interesting
architectural
choice.
A
I
think,
because
it
it
makes
some
some
things
actually
quite
quite
simple,
to
not
deal
with
the
with
these
persistent
volumes
and
unstateful
sets
the
the
apache
flink
clusters
also
subscribe
to
the
ingresses
and
egresses
in
order
to
receive
the
messages
and
send
the
messages
and
the
same
thing.
Actually,
you
know
maps
pretty
well
also
to
other
infrastructures
like
if
we,
for
example,
would
like
to
run
this
on
the
amazon
stacker.
So
we
could
just
put
our
functions
in
lambda.
We'd
have
an
api
gateway,
snapshot
to
s3
and
ingress
and
egress
with
kinesis.
A
A
It's
it's
a
very
simple
example
in
order
to
to
to
make
it
to
make
it
suitable
for
for
a
short
demo
here.
What
we're,
assuming
is
we're
we're
getting
events
about.
You
know
changes
to
the
subscription
status
of
the
user.
Let's
say
grabbed
through
change
data
capture
from
something
like
the
central
user
management
database.
A
We
would
actually
have
a
stateful
function
where
one
function
is
subscribed
to
this
set
of
events
being
a
user
function,
and
this
user
function
maintains.
You
know,
state
like:
what's
the
user
id
the
subscription
status
and
the
billing
interval
now
billing
is
something
that
doesn't
happen
all
the
time,
so
the
first
thing
we
actually
need
to
to
to
do
is:
when
do
we
want
to
do
the
building?
A
A
A
So
for
let's,
let's
see
this
in
action.
Actually,
so
we've
deployed
this
on
on
a
sample
kubernetes
cluster.
There
are
a
bunch
of
parts
running
here,
there's
the
therefore
functions
master
and
worker
processes,
which
are
the
equivalent
of
the
of
the
database
of
the
state
management
system.
Here
we
have
two
more
deployments
for
the
payment
workers
and
for
the
subscription
workers.
These
are
the
two
functions
that
I
just
explained
in
the
example
we
we
put
them
in
separate
deployments
here.
A
We
could
have
also
put
them
in
the
in
on
the
same
in
the
same
deployment,
they're
just
they're,
just
different
different
request:
handlers
invoked
through
http
and
through
different.
We
can,
you
know
through
different
your
eyes,
also
have
them
in
the
in
the
exact
same
deployment,
there's
a
there's,
a
log
that
actually
shows
the
output
of
the
payment
worker,
which
we'll
use
for
a
later
part
of
the
demo.
A
So
we
can't
see
a
lot
of
a
lot
happening
here,
but
there's
there's,
you
know
sample
data
flowing
around
in
the
background,
some
payments
being
processed
switching
to
the
apache
link
ui.
This
is
the
this
is
the
main
operator
from
fling's
perspective
that
handles
the
stateful
functions
and,
and
we
can
see
from
the
from
the
different
parallel
charts
a
few
hundred
events
per
second
going
through
it's
a
it's
a
very
it's,
a
very
tiny
deployment
just
for
for
the
for
the
sample
case
here
and
I
think
we're
we're
far
under
utilizing
it.
A
We
can
use
this.
You
know
you
can
use
all
of
of
links,
tools
to
diagnose
this
application.
For
example,
we
can
can
look
at
the
back
pressure
if
our
functions
cannot
keep
up
with
processing
the
events,
then
no
flink
will
eventually
back
pressure
the
whole
system
back
to
the
back
to
the
ingresses
and
not
pick
up
events
anymore
from
there
all
right.
So
one
really
important
thing
that
we
wanted
to
talk
about
is
the
consistency
and
for
for
payment
provider.
Consistency
is
really
important.
A
We
don't
want
actually
to
trigger
multiple
payments
for
the
same
user.
That's
really
embarrassing
thing,
then
we
have
to
you,
know
handle
it,
so
we
can
simulate.
Actually,
what
happens
if
a
duplicate
payment
happens,
and
we
can?
We
can
see
it
in
the
in
the
bottom
right
window.
We
inject
it
accidentally
a
duplicate
message
and
we
can
see
that
the
payment
functions
immediately.
Pick
up
this
consistency
deviation,
there's
you
know,
there's
there's
a
response
from
the
payment
provider
about
the
payment
we
didn't
actually
even
initially
initiate.
So
we
can.
A
So
usually
this
inconsistencies
can
happen
pretty
easily.
If
you,
you
know,
you
talk
to
the
outside
world,
let's
say:
send
some
messages
to
kafka
and
also
update
database
date.
So
if
you
actually
crash
everything
at
the
same
time,
you
usually
have
to
take
some
care
in
order
to
not
have
any
any
duplicates,
but
here
we
can
really
just
crash
all
parts
at
the
same
time,
get
them
restarted
and
we
can
actually
see
from
the
logs
that
there's
no.
A
We
cannot
detect
any
inconsistency
like
we
didn't,
really
get
our
internal
state
and
the
messages
that
we
send
to
the
outside
world
out
of
sync-
and
this
is
really
because
the
the
state
updates
the
message
sending
it's
all
inherently
transactionally
integrated,
so
you
don't
have
to
worry
about
duplicates
or
anything
there
yeah.
That's
that's!
Actually,
that's
actually
the
end
of
the
short
demo
and
that's
also
the
end
of
the
of
the
talk
pretty
much.
I
hope
you
enjoyed
it.
A
What
what
I
really
want
to
you
know
to
to
tell
you
today
is:
is
this
idea
of
the
stateful
functions
project
trying
to
help
build,
distributed,
scalable,
consistent
applications,
and
it
combines
these
two
two
ideas,
this
idea
of
a
programming
abstraction
based
on
stateful
entities
and
a
distributed
architecture
that
that
that
kind
of
you
know
rethinks
how
to
use
databases
and
in
this
case
doesn't
use
a
database.
It
uses
a
like
an
event-driven
system
like
apache
flink.
A
So
I
want
to
acknowledge
also
very
quickly
a
few
other
folks
that
work
on
this
project,
I'm
the
one
giving
the
talk,
but
this
is
really
developed
by
the
apache
flink
community
and
especially
those
for
folks
here
deserve
a
special
shout
out
for
for
for
being
the
ones
that
make
this
project
great.
So
thank
you
very
much,
and
that
is
it
I'm
I'm
very
happy
for
you
to
attend
this
talk
and
and
answer
some
interest
in
and
what
we're
doing
here.
A
A
The
the
first
question
that
came
up
is:
how
do
you
differentiate,
stateful
functions
from
from
cloud
state?
I
think
that's
a
very
good
question.
There
is
there's
a
lot
of,
I
think
similarities
between
the
two
projects
in
in
in
the
problem
that
they
want
to
solve.
They
both
they
both
try
to
give
you
this
stateful
serverless
abstraction
where
the
the
application
gets.
This
gets
this
consistent
state
by
by
notion
of
of
integrated
state
and
messaging.
A
A
A
So
so,
if
you
wish
here
I
mean
tldr,
I
think
the
differences
are
the
the
decomposition
between
state
and
computers.
I
think
different
in
in
stateful
functions
and
the
other
than
that.
The
the
stack
is
different.
You
know
in
this
case
it's
it's
apache
flink.
In
the
other
case,
it's
it's
akka,
plus
some
database
that
aggro
persistence
can
talk
to
like
cassandra.
For
example,
there's
another
question
is
stateful
functions
like
an
is
a
stateful
function
like
an
object
from
object-oriented
programming,
encapsulating
data
and
behavior.
A
So
the
the
notion
of
you
know,
processing
messages
or
you
know,
methods
of
the
function
being
invoked
only
only
one
at
a
time,
one
after
another,
at
least
per
you
know,
per
entity
key
is,
I
think,
different.
So
there's
there's
there's
no
there's
no
parallelism,
but
yeah
other
than
that.
I
think
that
the
differences
are
probably
the
same
as
what
you
would
say
that
are
the
differences
between
an
actor
and
an
object
from
object-oriented
programming.
A
A
A
I
think
pretty
well,
if
you
assume
that
the
let's
say
the
the
administrator
of
the
of
the
apache
flink
part
of
the
the
system
that
you
know
that
that
handles
the
messaging
in
the
state
that
subscribes
to
the
incoming
events,
that
this
is
like
a
trusted
party,
then
the
the
actual
function
code
can
come
from
different
parties
that
don't
don't
really
that
don't
really
trust
each
other,
because
they're
they're
running
a
separate
deployment
separate
containers
they
can.
They
cannot
really
that
they
cannot
really
interfere
with
with
each
other.
A
They
don't
have.
You
know
like
direct
access
to
state
in
the
system
and
so
on.
They
can
really
only
yeah.
They
can
really
only
communicate
by
sending
messages
and-
and
thus
they
they
cannot
really
interfere
with
the
with
the
other
functions
and
they're
running
in
separate
processes.
So
yeah,
it's
it's.
We
haven't
really
yet
spent
a
lot
of
time,
designing
it
into
a
strongly
multi-tenant
system.
A
So
there
are
probably
a
few-
you
know
like
a
few
quirks
you'll
run
into
in
in
let's
say,
set
up
configuration
aspects
that
aren't
that
aren't
absolutely
optimized
for
multi-tenancy,
but
I
think
the
architecture
as
a
whole
is
pretty
promising
for
multi-tenancy,
especially
because
the
computation
in
different
surfaces
is
strongly
isolated
against
each
other,
which
is,
which
is
a
very
big
part
that
that
makes,
for
example,
stream,
processing
or
or
other
systems.
Where
you
know,
code
from
different
parties
really
runs
in
the
same
system,
which
poses
a
big
challenge.
A
There
there's
a
question:
how
would
the
migration
to
stateful
functions
work
that
that
is
a
very
interesting
one
like?
How
do
you
actually
migrate
an
application
to
stateful
functions?
So
it's
it's
a
bit
of
a
different,
a
different
paradigm
right.
So
it's
not
as
easy
as
let's
say
it's
swapping
sequel
for
postgres
or
so
underneath
the
hood
and
keeping
everything
else
the
same.
If
you
do
want
an
an
application
to
to
migrate
to
stafford
function,
you
probably
do
have
to
to
rewrite
it.
A
First
of
all,
what
one
thing
that
we're
trying
to
to
add
to
the
project
is
tools
that
that
help
with
let's
say
migrating
the
state
from
another
another
application
into
state
for
functions,
meanings,
meaning
taking,
let's
say
a
dump
from
a
database
and
actually
loading
it
into
the
into
the
into
apache
link.
So
there's
there's
some
some
early
tooling.
For
this
already,
it's
called
state
bootstrapping,
where
you
can.
A
You
actually
have
to
write
an
apache
link
data
stream
application
for
that
at
the
moment
to
load
the
state
in
there
we're
working
on
making
this
easier
to
to
add
to
add
tools
like
you
know,
just
issue,
a
sql
query.
In
order
to
to
to
import
basically
state
into
interstate
4
functions,
stay
tuned!
There's
there's
going
to
be
more
releases
in
the
future
that
that
will
make
this
easier.
A
A
I
think
these
are
very
good
starting
points
for,
for,
like
a
detailed
comparison,
I
don't
know,
I
think,
if
somebody
is
looking
for
a
for
for
research
project,
some
someone
in
academia,
so
I
think
this
would
actually
be
a
very
interesting
one.
I
personally
don't
don't
know
that
something
like
this
already
exists.
A
A
I
think
you
can
you
can
kind
of
conf
comfortably
run
the
framework
with
something
like
half
a
gigabyte
or
so
so
the
let's
say
the
the
main
part
of
the
of
the
of
the
framework,
the
the
coordination
part.
I
think,
if
you
give
it
a
100,
100
megabytes,
it
should
be
actually
fine
and
then
there's
a
good
amount
of
memory
that
that
should
go
to
the
embedded
rocksdb
instance,
which
is
where
the
with
the
where
the
local
state
index
is
stored.
A
So
this
this
should
always
have,
I
would
say,
maybe
roughly
half
a
gigabyte
of
memory
for
for
decent
performance
and
so
on
so
yeah.
The
the
memory
footprint
of
link
is
pretty
much
that
jvm
with
embedded
rockstar.
You
can,
you
can
think
about
it
like
that,
so
half
a
gigabyte's,
a
good,
it's
a
good
start.
More
obviously
always
helps
with
with
performance.
A
A
Yeah,
how
how
do
you
deal
with
synchronous
communication
here
so
there's
I
think,
there's
two
two
two
answers
to
that.
Just
got
the
message.
This
is
our
last
question.
Sorry
about
that.
If
you,
if
you
actually
deploy
this
in
in
a
way
that
that
your
function
invocation
can
can
easily
can
easily
block
because
you
can
spawn
a
lot
more,
you
know
processes
that
handle
the
different
invocations
like.
A
If
you
put
it,
for
example,
on
on
a
function
as
a
service
platform,
then
you
know
just
do
synchronous
communication
in
the
the
function.
It
doesn't
doesn't
matter
if
it
blocks,
if
you
actually
are,
are
limited
to
the
number
of
processes
you
you
you
are
you
have,
then
you
you
have
some
some
facilities
in
stateful
functions
to
actually
to
actually
do
asynchronous
requests
and
asynchronously
handle
responses.
So
that's
that's
pretty.
A
I
would
say
pretty
much
how
in
many
reactive
systems
you
would
you
do
you
know
you
do
do
external
requests
in
terms
of
of
consistency,
how
do
you
do
synchronous
requests?
If
you,
if
you
were
worried
about
consistency,
you
wouldn't
all
external
communication
should
actually
go
through
egress
events.
If
you
do
that,
then
consistency
comes
out
of
the
box.
If
you
do
external
communication
that
that
affects
side
effects
from
within
the
function,
then
basically
it's
your
own
game
to
worry
about
consistency
again.
A
All
right
that
that
was
it.
Thank
you
very
much
for
for
listening
and
this,
like
this
very
interesting
questions,
very,
very
cool.
There's,
a
slack
channel,
kubecon
serverless.
If
you
want
to
ask
further
questions
there,
I'd
be
I'll
be
in
that
slack
channel
for
for
the
next
hour
or
so
so
feel
free
to
also
ping
me
there.
If
you
have
some
follow-up
questions,
thanks
a
lot
for
listening
and
yeah,
very
I'm
very
happy
for
for,
for
you
guys
attending
the
session
and
asking
these
questions
thanks
a
lot.