►
From YouTube: Slaying Monoliths with Docker and Node.js, a Netflix Original - Yunong Xiao, Netflix.com
Description
Slaying Monoliths with Docker and Node.js, a Netflix Original - Yunong Xiao, Netflix.com
"At Netflix, our data access platform is at the heart of nearly every request from our subscribers. It enables our innovative UIs to communicate efficiently with our bevvy of backend services while growing our subscriber base to 75 million members. As a result of this scale, this monolithic platform requires ever increasing resources to run and maintain, both regarding hardware (32 vCPUs per instance) and developer productivity (try running that on your laptop!).
As we continue to grow and expand our subscriber base globally (#netflixeverywhere), we need fundamentally to change the monolithic design of our platform. In this talk, I will discuss the new container-based data access platform that’s replacing the monolith. See how the architecture of this cross-cutting project allows us to build isolated microservices with Node.js and Docker. Examine the tools and infrastructure we’re building across our stack that enable engineers to effortlessly build, debug, test and their code on this platform anywhere -- whether it’s locally on your laptop, or remotely in the cloud -- all made possible thanks to Node and Docker."
A
Hello,
amsterdam,
how's
everyone
doing
nothing
come
on
beautiful
yeah.
That's
what
I
want
to
hear
beautiful
city
lovely
crowd.
My
name
is
yunnan,
I'm
from
netflix
and
really
excited
to
be
here
to
share
with
you
our
story
of
how
we
we're
slaying
monoliths
with
node
and
docker.
So
let's
get
started.
Clicker.
A
There
we
go
so
has
anyone
heard
of
us
anyone
nobody,
so
we
I
mean
we
started
with
this
really
simple
concept
right,
which
is,
you
know,
watch
your
content
wherever
you
want,
however,
you
want
any
device,
and
lately
you
know
we
really
made
this
push
into
originals
content
as
well.
So
here's
a
selection
of
five
of
our
really
exciting
thrilling
original
series.
A
Also,
you
know.
Over
the
last
year
we've
really
gone
global.
What
I
mean
global,
I
mean
everywhere.
So
all
the
countries
in
the
red
there
is
where
we're
at
where
we
are
so
netflix
is
literally
everywhere
in
the
netherlands,
south
korea
everywhere
in
the
world,
and
so
we've
got
this
explosion
in
content.
This
explosion
in
countries
and
also
we
have
this
explosion
in
subscriber
growth
right
so
from
2011.
A
When
we
had
a
little
over
20
million
subscribers
to
today
when
we're
almost
at
85
million
subscribers,
we
really
had
to
scale
our
stack
and
our
architecture
to
sort
of
handle
the
demand
of
intense
subscriber
growth,
all
the
original
series
and
going
global.
So
you
know,
let's
walk
through
sort
of
how
that
all
works
right.
So
you
want
to
watch
netflix.
It's
really
simple:
we're
like
one
of
the
biggest
poster
childs
of
moving
off
the
data
center
into
the
cloud.
So
that's
it
right.
We
move
to
the
cloud
our
engineers
are
done.
A
They
sit
at
work,
watch
netflix.
Actually,
that's
not
the
case
right
turns
out.
There
is
no
cloud,
it's
just
someone
else's
computers.
Oh,
so,
let's,
actually
you
know
look
at
how
we've
chosen
to
evolve
our
data
access
apis
along
with
all
of
our
explosive
growth
and
how
we
got
to
85
million
subscribers
and
how
that
all
works.
A
A
So
you
can
see
on
the
on
the
right
there
there's
a
bunch
of
backend
microservices
that
we
need
to
access,
and
so
each
microservice
team
will
provide
us
with
a
jar
which
is
a
client
that
we
could
use
to
access
all
of
our
data
and
there's
individual
clients
for
each
service
and
you
get
different
behavior
for
each
client
right.
So,
let's
play
a
game
called
spot.
The
monolith
right,
we're
here
to
slay
the
monolith.
A
Can
anyone
see
where
the
monolith
is
going
once
going
twice
right
right
right
there,
the
the
big
jvm
based
web
server
was,
is
the
monolith
and
the
reason
it's
a
monolith
is
that
everything
was
containing
one
process
right,
both
the
ui
and
the
rendering
and
the
data
access
all
that.
So,
if
you
wanted
to
push
an
update
to
the
ui,
let's
say
to
show
the
new
original
series
content,
you
have
to
push
the
process
and
push
the
server.
A
If
you
wanted
to
update
the
clients
up,
update
the
way
that
you
access
your
data,
maybe
to
make
it
more
efficient
or
improve.
Caching,
again,
you
have
to
push
the
service,
and
so
you
have
to
keep
pushing
this
process
when
you
have
to
make
every
little
change,
and
this
is
not
what
we
want
right
and
additionally,
you
know
we
got
more
devices
and
that
came
online
and
we
wanted
to
support
those
as
part
of
the
netflix
experience.
A
So
how
do
we
do
that
right?
Let's
look
at
the
next
step
in
our
api
evolution.
Well,
we
did
what
everyone
else
does
when
they
have
a
monolith.
They
take
the
java
web
server
and
they
turn
it
into
a
rest
api
right.
So
we
took
all
the
data
access
that
was
contained
within
the
one
single
process
and
we
made
a
rest
api,
and
this
is
what
you
know.
A
But
you
know
there
were
some
just
inefficiencies
as
well
right
with
a
rigid
rest
api.
It
meant
that
we
were
inflexible,
so
teams
are
often
waiting
weeks
between
api
changes.
So
if
you're
the
ios
team-
and
you
want
to
make
a
change
to
the
api-
you
go
tell
the
api
team
hey.
I
want
to
make
a
change,
and
then
you
know
some
weeks
later
that
gets
pushed
live
so
that
really
stifles
our
rate
of
innovation.
It
was
also
really
inefficient
if
you
think
about
it.
A
You
know
every
single
movie
that
you
see
on
your
netflix
app
is
a
rest
resource.
So
if
you're
returning
a
list
of
say,
100
movies
to
the
user,
you're
making
a
hundred
rest
calls
and
round
trips,
which
is
you
know,
very
slow
and
efficient,
and
over
time
as
we
added
additional
fields
and
changes
to
the
rest
api,
it
got
really
unwieldy
and
complex
and
was
hard
to
maintain.
A
A
This
also
means
you
know
more,
a
b
tests
and
more
more,
more
netflix
available
on
all
different
different
devices,
and
the
thing
that
we
really
want
is
for
literally
for
each
team,
then
to
be
able
to
write
their
own
customized
api
right
instead
of
having
to
use
a
rigid
non-changing
rest
api.
What
if
every
team
could
craft
their
own
api
and
change
change
the
way
they
access
their
data
and
suit
that
exactly
to
their
needs?
And,
lastly,
performance
matters
as
well
right.
A
We
don't
want
to
be
making
100
round
trips
to
load
a
single
page,
and
so
we
take
the
rest
api
and
we
turn
to
something
called
api.next.
And
so
what
we've
done
here
is
this
big
shift
in
the
way
that
we
access
data.
So
instead
of
having
this
rigid
rest
api.
Now
each
of
these
individual
teams,
whether
that's
tv
or
ios,
they
can
upload
in
groovy
scripts
to
a
jvm-based
server,
and
these
groovy
scripts
are
custom
apis
they
craft
that
handle
and
access
data,
exactly
the
way
that
they
would
want
from
the
back-end
microservices.
A
A
But
let's
play
the
game
again:
can
anyone
spot
the
monolith
here
right?
It's
it's
still.
We
brought
the
monolith
back
except
on
the
server,
and
so
what
do
we
do
right?
What
should
we
do?
Well,
let's
talk
about
our
scale
as
well.
This
model.
This
worked
really
well
for
a
few
years,
but
now,
as
we've
hit
some
of
these
incredible
scaling
limits,
for
example,
42.5
billion
hours
watched
in
2015.
A
billions
of
requests
per
day,
you
know
we
have
thousands
of
these.
What
we
call
scripts,
which
are
those
custom
apis,
that
teams
are
crafting
they're
running
prod,
and
then
we
have
tens
of
thousands
of
tests,
and
each
of
these
scripts
are
having
seen
lots
of
changes
and
we
also
have
hundreds
of
av
tabs
with
many
variants
per
test.
So
the
the
sheer
scale
of
this
system
is
pretty
overwhelming
and
there's
some
there's
some
there's
some
problems
right,
there's
vertical
scale,
for
example,
because
all
these
scripts
and
custom
apis
are
living
inside
the
same
physical
process.
A
You
as
an
individual
team,
is
intended
to
take
up
as
much
memory
and
disk
in
io
as
possible,
but
by
doing
that,
you're
sort
of
you
know
going
against
the
group
incentive,
which
is
the
leave
enough
space
for
the
rest
of
your
peers,
who
also
need
to
use
the
shared
resource.
Additionally,
you
know,
if
you're
they
say
some
one
team,
like
the
tv
team
or
the
ios
team,
and
you
deploy
a
bad
script
and
that
has
has
a
problem
or
a
memory
league
or
or
has
an
exception
that
takes
out
everyone
else
right.
A
So
all
this
made
us
realize
that
hey,
there's,
there's
also
an
additional
aspect
of
this,
which
is
developer
ergonomics
right,
which
is
that
if
I
split
the
api
server
right
down
the
middle,
you
see
that
on
the
left,
everything
there
really
belongs
to
ui
engineering
right.
We
have
the
ui
engineers
who
are
writing
javascript
on
the
server
on
the
client
for
all
the
clients
and
they're
right,
they're
writing
their
own
api
apis
in
groovy
and
then
systems
engineering,
which
is
the
back
end.
A
A
So
instead,
what
you
do
when
you're
testing
your
scripts
is
you're
uploading,
your
your
endpoint
scripts
to
a
test
server,
and
so
you
wait
for
the
upload
to
finish
and
then
you're
trying
to
test
it
by
hitting
the
endpoint
and
if
it
breaks
you
re-upload
the
script
with
like
a
console.log
statement
to
see
how
things
are
going
and
this
turnaround,
I
see
some
of
you
laughing
out.
There
takes
tens
of
minutes,
and
it's
extremely
painful,
so
developer
ergonomics
here
is
is
is
terrible
frankly,
for
our
developers
right.
A
A
Three
big
requirements.
Scalability
right.
We
don't
want
to
be
vertically
constrained
inside
one
process
availability.
We
want
everything
to
be
available.
We
don't
want
one
errand
script
to
take
down
the
process
and
developer
productivity.
Our
developers
never
want
to
have
to
wait
tens
of
minutes
between
turnarounds
when
they're
trying
to
update
or
test
their
apis.
A
So
let's
talk
about
the
first
couple
of
points:
runtime,
scalability
and
availability.
What
does
that
really
mean
right?
Well,
it
means
process.
Isolation,
g
wouldn't
be
nice
if
all
of
these
api
scripts
were
isolated
on
their
own
inside
their
own
processes,
instead
of
being
co-located
inside
one
monolith
java
java
process,
we
also
want
to
separate
data
access
scripts
and
the
api
servers
themselves,
so
we
can
horizontally
scale
them
individually.
A
This
will
help
us,
you
know,
reduce
infrastructure
costs
and
it
will
lead
to
faster
startup
times.
It
takes
like
almost
an
hour
right
now
to
start
an
api
server,
so
imagine
like.
Suddenly
we
have
a
huge
surge
in
customer
traffic.
It
would
take
us
an
hour
to
meet
that
demand.
That's
obviously
not
great
either,
and
we
want
these
scripts
to
be
immutable
so
that
they're
easy
to
debug
and
troubleshoot.
So
what
about
developer
productivity
right?
Well,
90
90
of
our
all
of
our
clients
are
written
in
javascript.
So
you
know.
A
A
Just
like
how
you're
developing
your
note
applications
today
or
your
ui
clients
today,
you
want
to
have
the
same
set
of
tool
sets
and
that
follows
with
you
know:
fast
incremental
builds
so
not
having
to
wait
hours
and
hours
for
a
new
server
build,
and
you
also
want
a
an
environment
that
as
closely
mirrors
production
as
possible.
So
maybe
I
can
take
a
production
image,
bring
it
down
test
it
locally
and
work
sources
now
really
help
with
the
developer,
ergonomics
and
productivity
of
it.
A
So
what
does
that
look
like
right?
Well,
we
take
the
monolith.
That's
in
the
old
api,
which
is
you
know,
which
is
everything
glued
in
together
in
the
one
process,
and
we
turn
into
something
like
this,
and
this
is
our
next
generation
data
access
api
that
we've
been
working
on
netflix
and
I'll
walk
through
this
in
some
detail
here.
So
what
we've
done
is
we
separated
those
data
access
scripts
and
just
into
the
node
api,
which
is
the
second
column,
and
then
we've
also
provided
a
common
data
access
api
called
the
remote
service
layer.
A
You
know
mutate,
however,
they
want,
according
to
a
b
test
and
they're
different
devices.
So
let's
look
at
the
evolutionary
traits
that
we
were
looking
for
out
of
this
new
platform
right.
I
think
it
consists
of
four
things,
which
is
the
runtime
platform,
the
actual
no
platform
that
you're
running
on
you
know.
What
do
we
need
from
that
platform?
How
about
application
management?
How
do
we
go
about
managing
now,
instead
of
one
big
monolithic
java
process,
all
these
different
discrete
and
disparate?
A
No,
no,
no
applications
that
all
these
teams
are
crafting
how
about
the
container
infrastructure
right.
We
have
to
now
run
all
of
this
stuff.
Inside
of
docker
with
containers,
so
how
do
we
manage
that
and,
lastly,
developer
tools?
How
do
we
make
it
easy
for
our
teams
and
our
developers
to
craft
these
apis
easily?
A
A
We
want
our
ui
engineers
to
concentrate
on
what
they're
really
great
at,
which
is,
you
know,
creating
world-class
client-side
applications
right,
and
so
our
aim
here
was
to
provide
a
paved
path
for
data
access
apps.
These
engineers-
and
you
know,
world
world-class
engineers-
writing
who
are
really
good
at
writing.
Uis.
They
don't
have
the
expertise
and
sort
of
don't
have.
You
know
the
passion
of
maintaining
servers
or
running
servers
in
production.
A
So
our
aim
was
to
provide
a
platform
as
a
service
model
for
these
folks,
so
that
they
can
just
write
the
javascript
which
accesses
their
data
and
not
have
to
worry
about
metrics
or
alerts
or
all
scaling
their
apps
up
and
down
or
low
balancing
or
discovery
or
analytics.
And
in
fact
the
whole
point
of
this,
of
the
no
runtime
is
that
it
integrates
all
the
netflix
ecosystem
right,
all
the
our
open
source
software,
which
makes
running
you
know,
production
apps
inside
of
the
netflix
ecosystem,
easy
and
reliable.
A
We
want
to
integrate
that
into
the
node
platform,
and
so
the
node
platform
is
really
a
production
ready
platform,
and
all
you
have
to
do
is
bring
your
javascript
business
logic
right.
So
everything
else
here,
like
I
said,
is
free,
so
you
don't
have
to
worry
about
logging
or
emitting
metrics,
or
you
know,
figuring
out
what
the
rpc
mechanisms
look
like.
All
that
is
provided
to
you
to
the
container
and
the
platform
as
modules.
A
So
let's
talk
about
the
next
step,
which
is
application
management
right?
How
do
we
manage
all
these
different
applications
that
these
teams
are
publishing
again?
The
aim
here
is
to
make
that
stuff
simple
right,
so
we
want
to
talk
about
how
we
version
these
applications,
how
we
deploy
them
and
how
we
get
once
they're
deployed
how
we
get
operational
insight
to
see
how
they're
running
so
you
know,
if
there's
problems,
we
can
debug
them.
A
So
versioning,
there's
a
bunch
of
current
problems
that
we
have
today,
which
you
know
we
talked
about
a
little
bit
earlier,
which
is
that
these
apis
change
all
the
time.
So
there's
literally
hundreds
of
thousands
of
different
versions
that
are
live
that
that's.
That
can
be
that's
inside
that
monolithic
process
and
thousands
of
these
are
live
in
production
at
any
one
time,
and
additionally,
there's
some
inconsistency
here
right
in
the
current
system.
In
the
monolith,
every
team
has
come
up
organically
with
their
own
version
version
scheme.
A
So,
for
example,
tv
ui
uses
a
build
time
stamp
and
the
web
team
uses
a
git
sha
and
the
ios
team
uses
the
version
of
their
app
and
android.
I
think,
uses
the
integer,
but
this
is
wildly
inconsistent
for
everyone
right
and
for
for
the
for
the
api
team
to
manage
this
is
actually
kind
of
crazy.
So
our
aim
here
was
to
make
sure
that
our
versions
and
our
builds
are
consistent
and
reproducible.
So
how
do
we
do
that?
Right?
A
So
here
you
can
see
this
is
our
our
node
api
index,
every
single
api
version
that
you
publish
not
only
does
it
get
built
as
a
docker
image,
so
that's
immutable.
It
also
gets
submitted
to
our
lo
to
our
local
node
api
index,
which
is
kind
of
like
npm,
but
for
your
applications
and
your
services,
which
is
really
helpful
because
if
you
want
to
later
on
revert
to
a
specific
version,
that
version
is
immutably
checked
into
a
central
repository.
A
A
The
problem
here
was
that
you
know
if,
if
I'm
the
ios
team
or
if
I'm,
if
I'm
any
team
that
publishes
to
an
app
store,
these
clients
live
forever
right
and
if
we're
embedding
the
versions
inside
the
app
itself,
it
means
that
I
can
never
update
the
service
endpoint
right,
because
if
I
did
and
I
rallied
to
a
new
endpoint,
then
the
old
and
I
turned
off
the
old
endpoint.
All
these
apps
would
break.
A
So
how
could
we
solve
this
in
a
new
world
where
all
these
versions
are
immutable
and
we
can't
deploy
over
the
top
of
them,
which
is
what
we
want.
We
want
immutable
deployment
artifacts,
because
it
means
that
we
can
reproducibly
reliably
reproduce
all
of
our
different
builds
again.
Our
solution
here
was
to
use
summer
routing,
which
is
now
instead
of
you,
know,
strictly
pinning
to
a
specific
version.
We
can
now
float
and
tell
all
these
clients
to
float
on
a
major
version.
A
So
if
I'm
the
ios
team,
I
push
out
a
client
here
with
you
know,
care
1.0.0,
which
will
make
it
float,
make
this
client
float
on
the
on
the
latest
major
version
of
the
1.0
service.
Then
I
can
push
out
different
versions
of
my
node.js
service.
In
the
back
end,
that's
running
set
containers
and
I'll
just
automatically
route
to
the
latest
one.
This
makes
the
updates
to
your
service
really
really
easy,
because
you
can
keep
all
the
old
versions
running
around.
A
A
This
really
makes
it
easy
for
them
to
do
that
and
operational
insights.
That's
our
sort
of
our
last
point,
which
is
that
you
know
we.
We
want
to
make
sure
that
we
make
it
easy
for
you
to
list
and
view
all
the
apps
and
different
routes.
So
this
is.
This
includes
capturing
all
the
deployment
histories.
A
All
the
metrics
and
providing
really
rich
analytics
and
because
we're
providing
a
platform
that
plugs
into
all
of
this
all
these
existing
systems
on
netflix
that
have
already
been
built.
This
makes
it
really
easy
for
for
any
app
that
you
write
to
get
all
this
stuff
for
free
right,
because
we're
already
publishing,
metrics
and
publishing
your
logs
and
all
that.
A
So
let's
talk
about
the
third
point
of
the
evolution,
entry,
which
is
our
container
infrastructure
right
and
we've
also
built
something
that
we
plan
on
open
source,
which
is
called
titus,
which
is
our
production
grade,
container
management
and
scheduling
service,
and
for
us
the
biggest
thing
is
that
you
know
we
want
to
be
able
to
make
sure
that
we
deploy
and
scale
containers
reliably
inside
of
our
cloud
ecosystem,
and
that
meant
creating
an
entire
team
to
handle
all
that
infrastructure.
And
again
this
is
its
own
talk.
A
My
colleagues
have
given
some
talks
on
this
if
you're
interested
just
definitely
google
and
check
it
out
so.
Lastly,
let's
talk
about
developer
tools
right.
Our
aim
here
is
for
developer
productivity.
Remember
that
we
said
hey,
all
these
developers
are
coming
up
to
us
and
saying
hey.
I
don't
really
ever
want
to
wait
10
minutes
between
between
iterations
when
I'm
building
my
api.
So
our
biggest
goal
here
is
to
run
and
debug.
These
scripts
locally
get
fast
incremental
builds
and
they
give
them
a
local
production
like
environment,
so
they
can.
A
They
can
easily
test
their
code,
but
there's
still
this
problem
right
with
local
development,
which
is
that
you
know,
even
if
I'm
using
docker
right.
If
I
write
my
javascript
endpoint
scripts,
I
commit
them
to
the
source
control
management.
I
still
have
to
build
the
dependencies
and
build
a
docker
image
and
bring
that
down
locally
to
be
able
to
test
it
out
and
that
step
of
building
and
building
the
dependencies
by
npm
install
and
then
the
docker
image
still
takes
tens
of
minutes.
So
how
do
we
solve
this
problem
right?
A
We're
back
to
tens
of
minutes?
Well,
we've
we've
actually
created
something
we
think
is
pretty
novel
which
unlocks
rapid
local
development.
In
essence,
we
deploy
a
docker
container
to
all
of
our
engineers,
laptops
wow
of
the
new
tooling
that
we
talked
about
and
inside
that
docker
container
will
actually
map.
You
know
onto
the
host
the
the
workspace
of
where
all
your
routes
live,
so
you
can
check
out.
You
know
your
application.
A
Make
changes
to
them
because
there's
a
mount
into
the
docker
container,
then
all
that
stuff
gets
automatically
picked
up,
and
so,
when
we
see
that
you
know
you've
made
changes
to
your
to
your
endpoints
or
your
your
package.json
or
whatever.
We
can
automatically
pick
that
up
and
just
bounce
the
service.
So
instead
of
taking
tens
of
minutes
just
now
literally
takes
seconds,
and
also
because
we
have
a
local
container
running,
you
get
access
to
all
of
your
logs.
You
can
attach
a
debugger
and
stuff
to
the
code,
so
you
literally
have
a
local
development
story.
A
So
a
quick
recap
on
what
we
talked
about
today
with
containers
which
containers
and
docker
really
gave
us
is
process
isolation
and
I'm
able
to
isolate
all
of
my
different
endpoint
scripts
from
each
other,
and
we
also
get
really
nice
layered
dependency
management.
So
it
makes
it
really
easy
for
me
to
manage
my
platform,
so
I
can
bring
a
specific
version
of
node
and
specific
version
of
the
operating
system,
et
cetera
and
there's
portability
across
all
these
different
environments
right,
so
I
can
take
a
production
image,
download
it
locally
and
test
it
to
see.
A
What
about
node
right?
Well,
we
chose
to
use
node
because
we
want
to
use
javascript
on
both
the
client
and
server,
and
this
really
unlocks.
You
know
the
potential
over
ui
engineers,
who
are
you
know,
experts
in
javascript
already
anyway,
but
additionally,
it's
super.
It's
super
fast
and
performant
right
and
it's
lightweight
and
efficient
to
run
locally.
So
we
can
run
all
this
stuff
locally
without
having
to
spend.
You
know,
give
everyone
100
100
gigs
under
on
my
laptop
or
anything
like
that.
A
A
So
a
quick
recap
on
the
no
platform
right,
the
things
that
we
want
to
hit
on
developer
productivity.
First
of
all,
which
is
that
hey,
you
know
I
never
ever
ever
want
to
wait,
10
minutes
again
between
between
iterations
and
we
sort
of
unlocked
that
by
letting
you
develop
locally
and
in
terms
of
scalability
and
availability,
we've
taken
the
monolith
right
and
we've
turned
broken
out
down
into
individual
microservices
for
each
of
these
teams
that
are
isolated
and
so
now
there's
better
availability.
If
I'm
a
team,
that's
pushed
that
pushes
the
bad
build.
A
I
only
take
down
my
service
and
not
everyone
else,
which
is
a
pro
as
a
plus,
and
I
can
horizontally
scale
this
stuff
out.
So
what
about
results
right?
We
actually
just
started
testing
in
production
by
shallow
traffic
recently.
So
now
we
have,
you
know
all
the
search
traffic
that's
coming
to
netflix,
it's
going
through
the
stack,
we're
not
returning
the
results,
because
it's
just
shallow
traffic,
but
we
we
plan
on
sort
of
going
living
production
over
the
next
couple
of
weeks.