►
From YouTube: Compiling to Containers by Brendan Burns, Microsoft
Description
Compiling to Containers - Brendan Burns, Microsoft
"Containers have revolutionized the way that people build and manage distributed systems. More and more they are becoming the “assembly language” for modern, scalable distributed applications. But if containers are the “assembly language”, most people deploying distributed systems are actually still programming in this “assembly language.”
But what if, instead of writing in “assembly” we started writing our distributed systems, in higher level languages, that compiled to containers? Traditionally this has been the task of domain specific languages like Puppet, Chef, Salt or Ansible. In contrast, in this talk, I show how a general purpose programming language (in this case JavaScript) can be used to write programs that compile to a distributed system of containers that is then deployed onto Docker containers.
Using a general purpose programming language unlocks significant value for users. They now have a large array of development tools, from unit test frameworks, to code style conventions at their disposal. Further, the (optional) use of a single language that describes both the pieces of the distributed system as well as the manner in which these pieces are composed, radically simplifies building and maintaining such a system. This in turn increases application reliability and agility for developers."
About Brendan Burns
Brendan Burns is a software engineer at Microsoft Azure and co-founder of the Kubernetes project. Before Kubernetes he worked on search infrastructure at Google. Before Google he was a professor at Union College in Schenectady, NY. He received his PhD in Computer Science from the University of Massachusetts Amherst and his BA in Computer Science and Studio Art from Williams College.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
Yeah
I
think
we're
good
all
right.
Thank
you.
So
much
for
coming.
Thank
you
for
coming
to
cube
Khan,
it's
remarkable
honestly
to
see
a
thousand
people
here
when
there
were
three
of
us
I
think
three
years
ago,
so
I'm
still
sort
of
sinking
in
and
so
today,
I'm
gonna
talk
a
little
bit
actually
not
so
much
about
kubernetes
actually
and
a
little
bit
about
some
ideas
that
I've
had
floating
around
for
a
while.
A
I
took
some
time
off
in
late
May
and
early
June
of
this
year,
and
you
know
because
I
had
to
write
code
while
I
was
taking
time
off,
I
wrote
a
bunch
of
code
and
that
this
is
the
the
net
of
that.
It
was
kind
of
an
experiment
to
see
some
ideas
that
I
had
and
see
how
they
looked
so.
Hopefully
they
they
look
interesting,
and
hopefully
we
can
go
forward
from
here
and
that's
that's
the
basis
of
this
I.
B
A
Say
that
this
is
all,
although
I
you
know,
I'm
currently
employed
by
Microsoft.
This
is
all
my
own
ideas.
So
if
they're
terrible,
don't
blame
them
and
if
they're
good,
then
they
can
have
them
and
that's
great,
that's
just
the
way
it
works.
I
guess
we
used
to
say
what
is
it.
Success
has
a
thousand
fathers
and
failures
and
orphan
alright.
So
there's
a
brief
history
of
applications
and
I
think
when
we
start
think
about
applications
a
long
time
ago.
We
think
about
unicorns.
You
know
they
were
bespoke
software.
A
That
was
built
for
a
specific
machine
and
it
was
really
it
was
intended
to
run
on
one
machine
in
one
place
for
one
customer
right.
That
was,
you
know
in
the
1950s
on
mainframes,
that's
the
way
applications
were
built,
and
eventually
we
had
clones
right.
Eventually,
we
started
saying
like
oh
actually,
that
program
is
good
and
I
have
that
same
machine
in
four
different
places,
so
I'm
gonna
run
that
same
machine
in
four
different
places,
but
they're
still
disjoint,
they're
still
really
just
clones
of
each
other.
A
They
don't
know
that
they
don't
know
that
they
exist.
They
don't
really
know
anything
about
each
other
and
they're
really
not
connected
in
any
real
way.
So
then
we
move
on
and
then
eventually
we
get
to
a
world
of
client-server
and
and
these
things
are
still
kind
of
bespoke
clones
of
one
another.
It's
just
that
you
have
a
server
in
one
place
on
one
machine
and
you
have
a
bunch
of
clients
that
are
clones
of
each
other
and
they're.
A
Talking
to
that
one
machine,
but
I
think
what's
important
here
is
even
at
this
point:
there's
really
not
a
lot
of
configuration.
There's
really
not
a
lot
of
description
of
the
topology.
The
client
maybe
knows
where
to
go,
find
the
server,
but
that's
about
it,
but
eventually
we
started
saying
like
well.
Actually
we
don't
just
want
clones,
we
want
kind
of
parameterised
clones
and
we
started
adding
in
configuration
files.
I,
don't
know
what
everybody's
first
configuration
file
was.
A
But
there's
actually
this
other
configuration
that
describes
the
broader
topology
of
the
application
describes
how
you
want
to
deploy
this
application
and
cloud
only
made
this
worse,
because
suddenly
you
had
a
file
that
described
not
only
how
you
wanted
to
deploy
your
application,
but
actually
all
of
the
machines
and
everything
else
and
so
you're
furthering
and
furthering
this
divorce
of
the
application.
From
the
thing
that
describes
how
that
application
should
run
and
I
think
that
the
net
result
of
this
is
that
it's.
A
It
becomes
too
hard
to
reason
about
these
things
because
they're
so
far
separate
from
one
another
and
I
think
that
there
is
there's
an
experience.
That's
been
lost,
I
mean
in
most
applications.
It's
not
like.
You
can
take
an
application
that
was
designed
to
run
in
a
sharded
replicated
pattern
and
like
say
well,
hey
now,
you're
gonna
go
be
scatter
gather
right.
It
just
doesn't
work
like
the
application
knows
how
it's
supposed
to
be
configured.
It's
just
that
we've
decided
we're
going
to
write
the
configuration
somewhere
else.
A
You
know
parameterization
makes
sense
because
you're
going
to
reuse
it
in
different
contexts,
but
things
like
the
fact
that
the
topology
is
described
in
a
different
file
than
the
code
like.
Were
you
gonna
change
the
topology
right?
It
just
doesn't
make
that
much
sense
and
I
think
that
this
is
really
an
unnatural
separation
and
I'm
sort
of
hopeful
that
you
know,
and
some
of
the
things
I'm
going
to
talk
about
in
a
little
bit,
we'll
see
some
ideas
about
how
maybe
we
can
bring
these
things
back
together
and
I.
A
A
Yeah,
so
one
or
two
people
all
right:
what
does
this
do
all
right?
Everybody
knows
what
this
does
and
it's
the
same
thing
all
right.
This
is
just
the
dump
of
the
compiler
and
the
assembly
code
of
this
all
right
and
I
I'm
going
to
say,
unless
you
saw
the
string
hello
world
up
top,
you
didn't
actually
know
you
were
cheating.
B
A
I
think
that
this
is
this
is
part
of
the
point
right,
which
is
that
we
don't
and
we
deal
in
higher
language
level
languages.
Most
of
the
time
then
see
anyway
at
this
point,
but
we
don't
we've
decided
that
that
abstraction
is
not
worth
our
time
and
that's
not
that
we
can't
do
it.
It's
not
that
we're
not
smart
enough
to
figure
out
assembly
language,
it's
that
if
we
were
all
still
doing
assembly
language
I,
don't
think
any
of
us
would
be
in
this
room.
A
Kubernetes
literally,
would
not
exist
if
we
were
all
still
programming,
an
assembly,
language
and
yet
I
think
we're
writing
our
distributed
systems
in
assembly
right
I.
Think
that
we
are
in
talking
about.
You
know
deploying
this
PAH.
You
know
this
set
of
pods
and
this
service
and
these
pieces
the
building
blocks
that
we're
putting
together
they're,
not
concepts,
they're,
just
assembly,
language
instructions
right
and
we
need
to.
A
We
were
at
this
meetup
down
in
in
Seattle
earlier
yesterday,
and
I
think
that
the
thing
that
I'm
most
excited
about
and
the
thing
that
I'm
most
interested
in
is
democratizing
distributed
systems
all
right.
If
you
look
at
so
I
I
like
to
say
one
of
the
smartest
people
who
I've
ever
worked
with
what
he
did
before
we
worked
on
distributed
systems
together.
Is
he
built
serve
web
browsers
for
mobile
phones
like
the
Motorola
RAZR
right
that
kind
of
phone
and
he's
just
incredibly
smart
embedded
systems
just
knew
the
operating
system?
A
You
wanted
to
note
about
timing
and
the
operating
said.
He
knew
it
exactly
right
and
it's
because
he
had
to
be
that
good
in
order
to
build
a
web
browser
for
a
mobile
phone
right
and
at
this
point,
teenagers,
middle
schoolers,
my
daughter,
is
in
two
years
is
probably
gonna
be
riding
phone
applications
for
her
phone
we've
democratized
mobile
development,
we've
democratized
web
development.
We
haven't
democratized
distributed
system
development
and
it's
time
that
we
did
all
right.
So
the
rest
of
this
is
going
to
be
really
a
sketch
a
proof
of
concept.
A
If
you
will
some
ideas
to
get
the
discussion
started
about
how
we
might
actually
go
about
doing
this
all
right.
So
this
is,
and
it's
out
there
right
now,
I
called
it
meta.
Particle
tachyon
was
taken.
Sadly,
but
meta
particles
are
tachyons
and
that
go
faster
than
light
and
I
can't
remove
the
name
of
the
other
one
which
goes
slower
than
the
speed
of
light,
and
so
the
basic
idea
is
just
like.
A
We
had
that
compiler
before
for
take
the
source
code
and
turn
it
into
machine
instruction
or
turn
it
into
assembly
language
that
then
turns
into
machine
instructions.
We
have
the
same
kind
of
idea.
We
have
code,
we
turned
it
into
an
abstract,
syntax
tree.
We
feed
it
into
the
orchestrator
and
the
app
pops
out
the
distributed
app
pops
out
the
other
side,
okay,
and
just
to
give
you
the
the
smallest
little
example
of
this
here's
a
program
that
is
written
here
right
and
I'm
using
nodejs,
doesn't
really
matter
for
the
ideas
that
are
involved.
A
So
you
know
we
bring
in
our
library
we're
gonna
define
a
service.
This
is
just
gonna,
be
a
really
simple
service.
It's
gonna
have
a
request
in
a
response.
I'm
gonna
say
expose,
which
says:
I
want
to
show
it
to
the
world
and
I'm
going
to
say,
serve
now,
there's
actually
a
lot
of
things
that
happen
in
this
serve
line
at
the
very
end
right.
You
can
think
of
this
as
being
the
definition
of
our
program
that
thing
right
there,
it's
pretty
much
the
compiler
and
what
it's
going
to
do.
A
Is
it's
going
to
take
this
code.
It
packages
inside
of
a
container.
It
deploys
that
container
to
an
orchestration
system.
It
sets
up
a
service
to
met
to
share
down
to
that
container,
and
at
this
point
you
should
say
that's
an
awful
lot
of
work
to
say.
You
know
basically
hello
world
right
and
I.
Think
that
that's
true,
if
this
was
all
that
we
were
building
distributed,
systems
wouldn't
be
hard
and
we
wouldn't
need
to
talk
about
these
sorts
of
ideas,
but
there's
a
whole
lot
of
other
pieces
to
building
a
distributed
system.
A
A
I'm
gonna
hope
that
this
works,
because
yesterday
I
don't
know
it
was
anybody.
In
my
talk
yesterday
there
was
some
serious
demo
fail
there,
so
I'm,
hoping
that
all
of
the
blood
that
was
spilled
there
is
going
to
not
is
gonna
mean
that
I
don't
have
blood
spilled
here,
but
you
never
know
all
right,
and
so
here,
I'm
gonna
say
we're
gonna,
say
less
server
ones.
Yes,
there's
that
same
program
that
we
just
saw
up
on
the
screen
and
now
I
can
actually
say.
A
There's
my
simple
service
out
there
and
running
right
and
I
can
actually
say.
Node
client
is
simple
wall
service
oops.
Sorry
I
have
to
go
over
here.
Actually
write
service
find
the
service
IP
address
copy
that
yeah
ctrl
shift
C
copy
that
we
can
go
over
here.
We
can
say,
put
that
IP
address
in
here,
give
it
the
right
name
all
right
and
there's
that
same
like
here's,
what
the
request
was?
It's
just
a
simple
echo
server.
That's
that's,
echoing
the
request
back
to
me.
A
So
at
this
point
again,
I
hope
if
you
were
paying
attention
you're
all
saying.
Well,
you
know
that
seems
like
a
really
an
awful
lot
of
work
to
return
hello
world
so
and
now
we're
going
to
take
it
a
little
bit
further
further
forward
and
talk
about
like
well.
What
we're
can
we
actually
go
with
this
all
right,
and
so
this
is
the
next
example.
This
is
a
simple
replicated
service.
All
right.
A
We
have
a
happy
user
there
at
the
top
there
to
a
root
node
that
root
node
is
fanning
out
to
those
leaf
nodes
all
right.
So
let's
take
a
look
at
what
this
looks
like
in
meta
particle.
Here's
our
web,
sorry
I,
went
through
a
little
animation.
So
there's
you
know,
one
request
gets
replicated
over
it
or
it
gets
routed
over
to
leave
number
three.
Another
request
gets
routed
over
to
leaf
number
one.
It's
a
simple
round-robin
right.
A
So
here's
what
this
looks
like
it
looks
very
similar,
except
for
you
know
here
what
I'm
going
to
say
is
meta
particle
dot,
spread
and
so
in
saying,
spread.
I'm
saying
I
want
to
spread
my
requests
across
this
number
of
replicas.
Here's
the
function
that
I
want
to
run
at
the
leaf
again
I'm
going
to
say,
expose
the
service
run
this
service
all
right.
So
let's
drop
back
into
that
demo.
C
A
A
A
I,
like
Debian
a
lot,
but
it's
just
like
it's
very
manual.
It
reminds
you
of
the
good
old
days
all
right
so
now
we're
gonna
say
we
can
say,
there's
that
that
server
that
we
had
now,
we
can
actually
say
node
server.
Yes,
runner
it
closed
curing.
That
is
all
right.
Let's
do
that.
So
again,
it's
building
that
image.
It
deploying
it
and
now,
let's
take
a
look,
sweet
acute
control
get
pods.
A
Now
we
actually
have
three
replicas,
so
you
can
see
replicated
service
zero
one
two,
and
then
we
have
this
shard,
which
is
the
route
right.
So
we
declare
it
declared
three
of
our
leaf
nodes.
Are
one
route
and
container
is
creating.
Hopefully
it's
running
by
now,
all
right,
it's
up
and
running,
and
now
we
can
actually
also
say
we
can
get
services
grab
that
service
over
here
control
shift
copy
go
over
here
and
we
can
say
alright.
I'm
gonna
go
put
this
in
here.
A
Alright,
and
now
you
can
see
here
what
the
very
back
is:
it's
visible
right
up.
There
is
the
server
that
it's
serving
from
with
the
hash
that's
there
and
if
the
demo
gods
are
with
me,
when
I
do
another
request,
you
can
see
there
that
the
server
changed
right.
I
went
from
shard
number
one
to
shard
number,
two
still
in
shard
number,
two
back
to
number
one
back
to
zero
and
if
I
put
this
in
a
loop.
A
You
know
you
can
see
that
it's
round-robin
amongst
the
amongst
the
server's
right,
so
I
hope
that
this
is
beginning
to
show
you
I
hope
this
is
beginning
to
show
you
maybe
give
you
an
inkling
of
the
simplification
that
with
it
that's
possible
here
and
so
I
wanted
to
take
the
next
step
to
take
it
a
little
bit
further
and
talk
about
a
sharded
service,
so
who
here
knows
what
a
sharded
services,
it's?
Okay,
all
right
about,
half
about
half
the
audience
all
right
now,
I
think
there's
a
lot
of
concepts
and
distributed
systems.
A
Sharding
master
election
they're!
Really
not
that
hard.
They
they're
not
that
hard
to
understand
conceptually.
They
can
be
hard
to
implement
sometimes
but
they're
relatively
easy
to
understand.
Conceptually
and
I.
Think
one
of
the
keys
here
is:
we
want
to
make
it
possible
for
people
who
just
understand
the
concept,
but
don't
necessarily
want
to
understand
the
implementation
to
use
these
things
right
like
just
like
you
understand
how
you
can
use,
compare
and
swap
in
the
assembly
language
to
do
a
lock.
That
doesn't
mean
we
should
all
go
out
and
write
POSIX
locks
right.
A
There's
a
reason
that
everybody
uses
that
library
and
it's
because
it's
subtle
and
it's
hard
and
it's
honestly
a
waste
of
time
and
so
the
same
thing
so
that
the
sharded
service
here
or
request
comes
in
it
has
some
characteristic
like
what's
the
user
and
it
hashes
the
user
mods
up
by
the
number
of
replicas
that
you
have
and
and
your
requests
always
get
routed
to
that
particular
server.
But
you
can
scale
horizontally
because
you
know
my
request
gets
routed
to
one
server.
Someone
else's
request,
get
surrounded
to
another
server.
A
Someone
else's
request
gets
routed
to
another
server.
If
you
use
consistent
hashing
and
that's
one
of
the
things
that
people
get
wrong.
Sometimes,
then,
when
you
add
in
more
replicas,
you
actually
tend
not
to
have
much
churn,
and
so
you
can
do
things
like
cache
things
in
memory
for
performance
and
stuff
like
that
and
have
pretty
good
assurances
that
you're
going
to
you
know
that
that
cache
is
gonna,
be
relatively
hot,
all
right.
A
A
Well
here
we
go
I'm,
saying
to
say
MP
inside
where
I
said
spread
before
I'm
gonna,
say
MP
shard
and
then,
instead
of
and
I,
give
them
shards
and
instead
of
just
giving
a
service
function,
I'm
also
giving
us
a
sharding
function
right
and
so
now
that
root
that's
being
deployed,
it's
job
is
not
just
a
spread
load.
Its
job
is
actually
to
run.
A
A
I
can
spell
kubernetes
and
now
we're
building
that
image.
We're
deploying
that
image
we're
building
up
those
services.
C-Cute
control,
get
pods.
You
can
see
there,
there's
the
compute
the
leaf
function.
These
are
the
ones
that
I
just
deleted
from
before
here's
my
sharted
service,
they're,
all
the
leaf
functions
for
my
shot
at
service.
Here's
that
sharded
root,
that's
being
created,
they're
up
and
running.
My
other
previous
service
is
finished
dying.
If
I
get
services,
we
can
grab
this
service
shard
here
go
over
and
now,
if
I
go
over
here
paste
this
in.
A
A
It's
the
same
server
all
the
time,
but
if
I
go
in
here
and
I
actually
say
well
I'm
going
to
do
the
same
loop
but
I'm
gonna,
do
you
know
I'll
change
the
user
to
be
Brendan
with
some
degree
of
randomness
and
crossing
my
fingers
for
luck.
Yeah,
we're
gonna,
see
we're
gonna
target,
we're
gonna,
shard
and
target
to
a
different
server
right
so,
depending
on
the
input
that
I've
sent
in
in
the
request
I'm
getting
directed
to
one
particular
server
or
another
particular
server.
This
is
good
because
it
spreads
the
load.
A
This
is
good
because
if
one
of
those
servers
fails,
I
only
lose.
You
know
10%
of
my
users
as
opposed
to
100%
of
my
or
it's
better,
to
lose
10%
of
my
users,
a
hundred
percent
of
the
time,
instead
of
100%
of
my
users,
ten
percent
of
the
time,
it's
different
services
have
different
feelings
about
that.
But
that's
that's
the
reason.
You'll
choose
charting
all
right,
and
so
I
want
to
step
back
out
from
that
and
go
drum
drop
back
into
the
talk.
A
So
I'm
not
going
to
give
a
demo
this
one.
It
exists
in
the
github
repo
for
you
to
play
around
with.
This
is
the
next
sort
of
idea
behind
a
service
which
is
a
scatter
gather
service
and
so
in
a
scatter
gather
service.
What
happens
is
your
request
goes
out,
so
what
we
were
doing
before
it
was
really
just
request.
Routing
right
that
was
all
we
were
doing.
There
wasn't
really
a
lot
of
computation
in
a
scatter
gather
service.
Your
request
goes
into
a
root.
A
It's
scattered
out
to
a
bunch
of
leaves,
so
one
request
gets
multiplex
out
to
a
whole
bunch
of
leaves
all
of
those
responses
come
back
to
the
root
the
root
collates
them
in
some
way
and
returns
it
back
up
to
the
main,
to
the
main
user
right
and
so
there's
this
merge
function
that
happens
at
the
root.
So
you
have
a
leaf
function
that
returns
data,
a
merge
function
that
returns
roots
or
returns
results
at
the
root.
This
is
by
the
way
how
the
traditional
Google
search
index
was
implemented,
each
leaf
stores.
A
A
A
So
the
version
of
this
that
I
did
was
basically
a
Gaussian
distribution.
Each
leaf
returns
a
set
of
random
numbers.
The
root
returns,
a
histogram
of
those
numbers,
see
what
actually
a
time.
I
actually
think.
I
have
time
to
run
this
so
I.
Maybe
I
will
actually
run
this
for
you.
So
you
can
see
here.
This
is
the
leaf
function
right
here.
It's
just
pushing
a
Gaussian
random
number
between
25
and
100.
It's
going
to
return
the
result.
It's
going
to
return
a
hundred
of
those.
Then
we
go
into
the
merge
function.
A
The
merge
function
here
is
just
going
to
calculate
a
simple
histogram
right,
so
it's
going
to
take
all
those
random
numbers
that
it
got
back
bucket
them
up
in
and
calculate
the
histogram
right,
and
so,
if
we
can
go
back
over
here,
I
should
show
you
this
service
definition.
And
then,
if
you
look
at
the
service
definition
right
here,
all
we're
saying
is
we're
gonna
say:
well:
what's
the
name,
its
histogram
service,
I'm
gonna?
Do
a
scatter
I
want
ten
replicas
of
my
leaf
function?
Here's
the
function,
I
want
to
run
it
the
leaf.
A
I
don't
to
delete
it
yeah
thanks.
So
it's
built
again
building
that
image.
It's
deploying
the
image
and
again
like
the
good
thing
about
this,
is
if
you
edit
that
code
and
you
run
that
application
again,
it
redeploys
right,
the
it
rebuilds
the
image
it
redeploys
the
image
it
has,
that
sort
of
feel
of
edit
and
in
replace
on,
and
so
then,
if
we
go
over
here,
I'm
going
to
see
the
services
see
acute
control,
get
pods.
So
now
we're
gonna,
see.
There's
all
of
those
scatter
services
are
being
created.
A
The
observant
among
you
will
notice
that,
in
order
to
achieve
this
kind
of
sharding,
I've
actually
had
to
define
a
service
for
every
replica,
which
is
kind
of
sad,
but
I
could
also
have
implemented
it
via
endpoint
scraping,
but
this
was
easier.
Alright,
so
I
can
go
over
here
and
I
can
actually
say.
Alright.
A
And
the
input
is
actually
ignored
what
you
can
see
there
and
the
result
I,
maybe
I'll
clear
it
so
that
it's
easier
to
see
and
run
it
again.
What
you
can
see
there
in
the
result
is
that
histogram
right,
the
densities
in
the
middle,
it's
a
traditional
Gaussian
distribution.
It's
got
a
tail
on
either
end
distributions
in
the
middle
right.
So
this
is
scattered
out
collected
a
whole
bunch
of
random
numbers
gathered
back
in
run.
The
histogram
in
the
merge
function
return
the
result
back
to
the
user,
all
from.
A
Oops
cancel
all
from
the
code
that
was
expressed
right
here
right,
so
all
in
this
service
declaration,
alright,
so
I'll
step
back
out,
so
the
next
piece
is
well,
it's
great
to
be
able
to
define
your
topology.
It's
interesting,
it's
useful,
but
one
of
the
really
hard
problems
in
distributed
systems
is
storage.
Right
is,
is
in
the
fact
that
suddenly
have
all
of
these
different
processes
running
out.
All
these
different
machines
and
storage
becomes
very,
very
hard
and
so
I
think.
A
So
here
is
a
simple
server
where
we're
starting
out
to
a
replicated
replicated
leaf
over
there
over
here
we
wanted
to
requests
plus
plus
in
both
places,
and
we
really
would
like
it
if
requests
actually
was
incremented
not
on
the
machine
but
in
some
sort
of
global
notion
of
what
requests
is
right,
and
so
here's
what
that
looks
like
so
we
can
say
here,
is
I.
Have
this
request
coming
in
I'm
gonna
say
if
met
a
particle?
A
It
can
examine
the
scope
variables
before
it
executed
your
request
function
and
examine
the
scope
variables
after
it
executed
a
request
function
and
it
can
persist
those
changes
back
into
the
store
and
now,
if
those
of
you
who
have
done
distributed
systems
before
are
really
any
concurrent
programming
before
or
saying,
but
Brendan
what
about
the
read
versus
real
read:
update
right
race.
It's
like
one
of
your
favorite
races
right.
A
It's
one
of
my
favorite
I,
like
the
5k,
but
the
read,
update,
writes
race
is
a
close
second
and
so
what's
happening
here
is
we're
assuming
that
our
Storage
has
a
has
optimistic
concurrency
we're
assuming
two
things:
we're
assuming
at
our
storage
has
optimistic
concurrency
and
we're
assuming
that
in
your
request
function
here,
you
don't
have
side
effects
other
than
store
into
scope,
or
at
least
their
item
potent
side-effects
right.
You
can
make
web
requests,
but
they
better
not
be
data.
Specific
web
requests
right.
A
They
have
to
be
things
that
you
can
repeat
multiple
times,
and
these
are
the
assumptions
that
you
know
if
you
want
to
be
able
to
have
this
kind
of
simplification.
These
are
the
assumptions
that
you
have
to
bake
into
your
system
and
and
that's
where
the
compromise
happens,
and
this
is
probably
plate,
one
of
the
places
where
I'm
potentially
the
least
comfortable
right,
because
this
is
a
place
where
you're
imposing
a
programming
model
on
to
the
end
user
and
they
may
or
may
not
like
that.
A
A
Well,
you
know
what
that
particular
iteration
of
running
that
function
doesn't
count
and
we'll
drop
it
on
the
floor
and
we'll
sort
of
rewind
ourself
back
to
right
before
we
service
to
your
request,
we'll
reload,
all
of
the
data
from
the
storage
space
will
rerun
your
code.
That's
why
it
has
to
be
stateless
and
without
side
effects
other
than
in
the
scope.
A
A
And
now
you
say:
well,
you
know
what
I'm
going
to
say.
I'm
going
to
Redis
is
my
backing
story.
It's
this
is
implemented
in
a
pluggable
way.
You
can
plug
in
different
storage
layers
as
long
as
they
support
optimistic
concurrency
or
can
be
made
to
support
optimistic
concurrency.
We
can
support
any
kind
of
storage
layer
on
and
one
of
the
other
pieces
is
this.
This
is
really
gonna
feel
like
writing
a
program
and
running
a
program
on
your
local
machine.
Well,
you
need
to
take
your
environment
variables
on
that
local
machine
into
effect.
A
All
right,
if
you're
pretty
attention
earlier
when
I
listed
the
services
you'll,
see
that
there's
this
Redis
service
running
here
and
now
I'm
going
to
take
that
cluster,
IP
and
I'm
in
control
show
copy
it
and
I'm
gonna,
say:
export
Redis
host
equals
this
service
right.
This
is
a
local
environment,
variable
there's
an
environment
variable
in
my
shell
and
yet
what
I'm
gonna
do
when
I'm
doing
that
deployment
is
I'm
actually
going
to
introspect
the
environment
variables
across
my
shell.
A
I'm
gonna
push
those
environment
variables
into
the
container
right
because
I
want
the
experience
of
running
it
on
my
machine
to
look
and
feel
like
the
experience
of
running
it
in
that
container,
I'm
gonna
grab
those
things
push
them
up
and
so
by
knowing
that
this
is
the
Redis
host.
That
I
want
to
talk
to
it's
going
to
also
be
the
Redis
host
that
we
talk
to
inside
the
container,
and
so
then
I
can
say:
node
server,
6j,
yes,.
A
All
right
and
we'll
run
that
so
again,
it's
gonna
build
the
image
it's
going
to
deploy
that
out.
Now,
if
you
say
cube,
control
get
pods
it's
in
the
midst
of
creating
this
replicated
service
with
the
shard.
You
know
the
shard
front
end
at
the
beginning.
Eventually
we
will
get
there
all
right,
we're
up
and
running
on,
so
we
can
say:
get
services
you'll
grab
that
shard
over
here
we'll
go
over
here
and
my
final
act
of
faith
to
the
demo
gods.
I'll
see
if
this
demo
runs
mm.
A
All
right,
so
there
you
can
see,
request
is
14
the
and
obviously
practice
this
demo
a
little
bit
beforehand
and
the
server
that
it's
talking
to
you
know
it's
a
ey,
ooh
I
see
all
right.
So
we
can
run
this
again.
Gress
is
15
coming
out
of
the
different
server
and
just
to
give
it
a
really
little
bit
of
a
stress
test.
We
can
say
4x
in
0,
1,
2,
3,.
A
If
you
can
count,
we
have
18
19
20,
21
22,
all
like
we've
counted
all
of
those
requests
that
have
come
in
parallel
across
all
of
the
different
hosts
successfully
incrementing
the
counter
each
time
giving
the
user
the
illusion
that
everything
is
sort
of
running
in
this
one
program
on
this
one
machine
when
it's
fact
a
distributed
system
all
right.
So
that's
it!
Here's
where
to
go
from
here
it's
up
on
my
github!
This
is
really
the
only
beginning,
the
beginning,
I
hope
and
it's
a
proof
of
concept
at
that.
A
I
showed
it
to
Joe
while
ago,
and
he
said
like
you've
done
this
horrible
thing,
where
you're
combining
like
the
definition
of
you
needed.
You
need
a
stronger
abstract,
syntax
tree
in
there,
so
that
we
can
target,
go
programs
so
that
we
can
target
other
kinds
of
containers
and
that
sort
of
thing
and
I
think
that's
definitely
what
we
need.
A
We
need
to
build
a
library
of
these
standard
containers
so
that
I'm
not
actually
for
things
like
sharding
I'm,
not
necessarily
relying
on
this
JavaScript
container
that
I've
built,
but
actually
we
have
a
library
of
standard
containers,
and
so
this
is
just
an
invitation.
If
you
like
what
you
see-
and
you
think
it's
interesting,
please
come
hang
out
and
work
on
this
with
me.
If
it's
terrible
as
I
said,
please
don't
blame
my
employer
and
thank
you
so
much
for
coming
to
cube
Khan
and
for
being
interested
in
kubernetes.
B
E
A
I,
wouldn't
I,
don't
think
in
the
context
of
a
single
micro
service,
you're
going
to
combine
those
patterns
together,
except
for
this
sort
of
storage
and
sharding.
So
the
goal
in
exposing
them
all
as
kubernetes
services
is
that
that
is,
you
know,
that's
one
of
your
micro
services
and
and
I
think
I
definitely
believe
in
decoupling
micro
services.
So
I
don't
think
that
we
want
to
build
the
topology
of
like
this
is
how
all
my
micro
services
connect
to
each
other
and
so
I.
A
The
hope
is
that
that's
the
way
you
achieve
it
is
that
each
one
of
these
pieces
of
code
is
conceptually
one
application.
It's
conceptually
one
micro
service,
the
linking
of
the
micro
services
together,
is
something
that
the
program's
can
do
just
like
calling
any
other
web
service.
So
I
think
that's
that's
the
way
I
would
think
about
it.
Yeah.
F
A
That's
the
part
where
that's
the
part
where
you
can't
write.
That's
the
part
where
you
have
to
talk
to
I
mean
that's
why
we
have
scope.
The
scope
and
storage
like
scope
is
trying
to
give
you
enough
of
a
global
issue,
I,
looking
thing
that
you
can
that
people
can
adapt,
but
obviously
like
we
can't,
we
can't
go
from
that
world
to
distributed
systems
without
a
little
bit
of
you.
A
Don't
think
that
there
are
any
pieces,
I
guess
what
I
would
say
aside.
The
goal
is
that
there
aren't
any
pieces
of
the
language
that
are
excluded.
It's
just
that
you
say
like
hey.
You
know
that
thing
that
you
declared
is
var
up
here
in
the
top
level
of
your
thing,
you
need
to
say,
NP,
dot,
scope,
global
can.
D
Hi
really
interesting
one
question:
would
you
say
that
you
want
to
abstract
that
from
the
language
level
right
so
build
a
library
that
doesn't
only
understand
JavaScript
can
be
whatever
language?
So
actually,
when
you
do
that,
aren't
you
going
one
step
behind
separating
again
your
program
from
the
actual
topology
yeah.
A
I
think
you
can
you
I
mean
I,
think
you
can
view
it,
though,
and
I
think
that's
why
I
I
think
that's
why
I
ended
up
where
I
was
and
I
think.
This
is
that's
why
I
ended
up
with
the
system
that
I
built,
but
I
do
think
it's
important
to
realize
that
there
will
be
bits
and
pieces
that
you
take
from
behind
right,
like
that
you
take
from
you
have
stuff
that
you've
already
written.
A
You
have
code
that
you've
already
built,
or
you
have
code
that
someone
else
has
written
that
you
want
to
reuse,
I,
think
it's
important
that
whatever
we
build
in
this
world
be
extensible
in
that
way
it
may
not
be
the
happy
path.
It
may
not
be
the
the
best
path,
but
I
think
it's
important
that
it
be
possible.
So
that's
I,
guess
that's
that's
my
main
motivation.
A
And
so
then
yeah
that
would
be.
The
other
thing
is
like
yeah,
you
don't
I,
don't
I
think
it's
silly,
especially
in
the
replicated
service
world,
where
there
isn't
any
user
code.
That's
running
there.
It's
silly
that
it's
not
a
che
proxy
and
I!
Think
there's
also,
and
the
other
reason
to
say
you
want
to
abstract
it
from
the
language
is
I.
Think
that
we
instead
of
saying
like
hey,
you
need
to
define
your
function
in
JavaScript
and
then
I'll
call
your
sharding
function
in
JavaScript.
You
can
say
like
hey.
A
Your
sharding
function
needs
to
implement
this
webhook
and
I'll
have
a
shard
or
that
calls
that's
it's
a
sidecar
in
the
pod
with
you
that
calls
into
your
into
your
function.
Http
web
hook.
Style
gets
a
response
back
and
then
does
the
sharding,
and
I
think
that
that
is
a
more
sort
of
sustainable,
flexible
pattern
going
forward.
It's
a
little
bit
less
pretty,
but
yeah.
A
A
Here's
a
third
party
resource
definition
use
this
container,
as
my
sharding
function
use
this
container
as
my
lief
function
and
when
I
create
that
third
party
resource
it
poops
everything
into
existence,
I
could
imagine
doing
it
that
way,
it
becomes
more
declarative
and
I
think
it
depends
on
whether
you're
an
integrator
versus
a
developer
I
think
we
wear
those
different
hats
at
different
times.
Some
people,
you
know
sometimes-
and
some
people
are
just
kind
of
integrating
different
pieces
together
to
make
something
that
works
and
sometimes
you're.
Writing
your
program.
C
A
So
I'm
not
sure
that
I
mean
so
easily
you
could
easily.
With
that
global
scope,
you
could
maintain
a
state
machine
state
if
you
wanted
to
right
because
again
it's
like
it's
a
read,
update
right.
So
if
you
have,
if
you're
doing
a
distributed
advancing
of
a
state
machine,
it's
gonna
read
the
current
state.
It's
gonna
do
an
advance.
It's
gonna
try
and
write
the
state
back
if
there's
a
conflict
because
somebody
else
picked
it
up,
then
it'll
ignore
that
particular
computation
right
now.
A
It's
quite
possible
that
that
you
don't
want
that
kind
of
optimistic
concurrency
in
a
state
machine
right.
Everything
that
I've
done
here
is
entirely
about
optimistic
concurrency.
There's.
Definitely
some
distributed
system
patterns
where
you
want
to
have
a
clear
owner
right
where
you
want
to
say
this
person,
they're,
the
owner
and
I,
think
we
need
to
figure
out
in
that
same
kind
of
spirit.
How
do
you
build
that
into
this
right?
How
do
you
take
that
idea
of
an
owner,
a
master
elector,
maybe
or
something
like
that
and
make
it
work?