►
From YouTube: Breaking Down the Monolith - Peter Marton, RisingStack
Description
Breaking Down the Monolith - Peter Marton, RisingStack
"The story of how we broke our Node.js monolith into 15+ services in just a couple of weeks. The talk will focus on what we have learnt during the journey and what technologies we use. This is the tale of how we did this rather than what microservices are in general.
Agenda:
1. monolith and microservices in nutshell
2. advantages of microservices from our view
- well focused service teams, happy customers
- fault tolerance
- distributed responsibility
- improving quality
3. disadvantages of microservices-based our experience
-growing complexity
-errors in transactions
-response time issue
4. how we did it
-service team principles
-co-operation between teams
-company architecture changes
-breaking down via proxy
-request signing
-circuit breaker
-event sourcing"
A
Okay,
so
hello,
everyone,
this
is
the
breaking
down
the
monolid.
So
it
seems
to
be
that
today
we
are
slaying,
Breaking
and
killing
the
monoliths,
but
this
why
I
really
like
this
kind
of
topics,
because
every
company
has
different
requirements
and
challenges,
so
these
companies
have
different
solutions
for
that.
So,
however,
my
talks
title
is
similar
to
the
previous
one.
I
am
going
to
talk
about
totally
different
things,
not
totally,
but
mainly
so.
Who
am
I,
I
am
Peter,
matin
I
am
a
co-founder
and
city
of
rising
stack.
A
What
is
trace
its
noisiest
performance
monitoring
tool
and
micro
service
debugging,
but
what's
really
important
for
us
today
that
this
is
built
with
no
GS
and
we
started
as
a
monolith,
but
later
we
broke
it
down
so
basically,
today
talk
will
be
able
to
or
experienced
with
it.
So
first
I'm
going
to
talk
about
in
nutshell,
what
is
immolate
and
the
micro
service
why
we
moved
to
micro
services
so
what
we
expected
and
our
expediency.
A
So
what
we
got
at
the
end
of
the
day
and
what
challenges
we
faced
during
this
journey
and
what
solutions
we
come
up
for
that
so
monolith
and
micro
service
is
really
in
a
nutshell.
We
call
something
to
model
it
when
it's
self-contained,
so
it
means
that
is,
it
can
work
independent
from
other
applications
and
it's
responsible
for
multiple
tasks.
A
So
one
process
is
doing
multiple
thing
in
this
application
and
instead
of
that,
we
call
it
micro
services,
when
it's
smaller
services,
usually
with
separate
databases,
and
we
split
the
code
more
by
features
instead
of
functionality.
It
usually
not
usually,
but
almost
every
time
it
comes
up
with
feature
teams,
so
teams
who
are
behind
these
features
and
these
teams
are
usually
cross-functional
teams.
A
So
one
team
is
containing
people
with
different
knowledge,
while
in
a
monolid
what
usually
happens
that
there
are
database
team
and
the
UI
team
in
the
micro
service,
the
cost
theme
is
more
popular
and
microservice
is
not
a
silver
bullet,
because
you
are
moving
the
complexity,
basically
from
in
memory.
Communication.
So
function
calls
to
the
network,
which
is
always
slower
and
I
like
to
mention
that
microservices
is
not
for
MEP.
So
if
you
are
building
a
Minimum
Viable
Product,
then
you
probably
don't
know
what
your
product
will
be.
A
So
you
also
don't
know
what
architecture
is
best
for
that
may
be
six
months
later.
Micro
service
is
the
best
for
you,
but
starting
from
scratch
with
micro
services
can
be
challenging,
so
why
we
move
to
micro
services
and
first
of
all,
our
engineering
team
was
growing
and
based
on
our
experiences,
we
know
that
larger
teams
with
lots
of
people
can
work
less
efficient,
so
we
wanted
to
improve
it.
A
We
also
wanted
to
have
separated
features
with
more
focused
people,
so
we
wanted
to
people
who
are
a
master
of
a
specific
field,
for
example
in
an
application.
We
have
different
features
like
memory
monitoring
or
disability
tracing,
and
if
everybody
has
to
know
everything,
then
it's
usually
a
less
focused
and
we
had
also
need
for
photons,
because
if
my
healing
goes
down
that,
who
knows
that
you
are
done
so
it's
usually.
We
should
be
that
the
guy
who
is
telling
that
you
are
done
and
what
we
got
when
we
moved
to
micro
services.
A
A
And
now
it's
working
really
well
I
like
to
mention
that
some
companies
choosing
micro
services,
not
because
technology
point
of
view,
but
because
they
have,
for
example,
distributed
teams
around
the
world
and
they
are
not
able
to
work
in
one
big
team.
So
they
want
to
break
it
down
for
smaller
teams
and
faster
innovation
with
smaller
services,
it's
easier
to
design
one
specific
service
easier
to
implement
and
ship
it,
which
also
means
that
it's
easier
to
replace
the
whole
service
or
specific
out
of
it.
A
So
we
can
try
with
new
technologies,
new
databases
and
it
really
has
the
innovation
in
the
company
easier
to
scale
specific
part.
We
have
separate
services,
separate
databases.
It's
really
easy
to
scale
only
that
part
which
is
as
high
or
traffic
for
example,
and
features
well
independently.
It's
not
easy
to
achieve,
but
we
could
achieve
it.
Some
of
the
features
which
means
that
by
maybe
only
the
host
system
is
specially
functioning,
but
some
function
is
able
to
serve
the
user
needs
while
another
one
is
done,
but
it's
not
easy
to
achieve.
A
A
Increasing
architectural
complexity,
it's
coming
by
design
for
micro
services,
so
more
services
communicating
each
other
in
the
network.
It's
it's
increasing
the
complexity,
especially
if
something
breaks,
then
it's
not
easy
to
figure
out
that
what's
happened
and
increasingly
spawns
types.
I
mentioned
that
the
communication
moved
to
the
network.
Usually
these
networks
are
slower
than
an
in-memory
communication
in
the
monolith.
A
So
if
you
put
more
services
in
a
chain
service
called
chain
after
each
other,
then
your
application
will
be
slower
and
harder
to
test
while
a
mullet
application
is
one
application
which
you
start
with
your
databases.
You
can
really
run
your
test
on
there,
but
with
micro
services,
where
you
need
multiple
other
services
to
test
it,
so
you
need
to
mock
those
services
or
find
a
way
to
test
them
together.
A
So
these
are
experiences
and
summarized,
and
now
let's
see
that
what
challenges
we
face
and
what
solutions
we
come
up
for
them.
So
photons
I
mentioned
it's
very
important
for
us,
and
services
were
separately.
I
mentioned
that
we
could
achieve
it
at
some
points,
but
it's
not
always
possible
because
you
have
some
critical
resources,
so
this
should
be
cached,
and
for
this
we
have
a
service
level.
Caching,
for
what
we
have
wrapper
library,
which
is
only
providing
extra
functionality
for
the
communication.
A
So
it's
as
the
key
of
the
service
is
the
same
library
in
every
service,
and
it's
just
providing
some
extra
functions
like
the
caching
is
providing
helpers
for
mocking
the
service,
some
extra
security
layers
as
well,
and
what
happens
if,
if
you
cannot
guess
something,
but
you
cannot
catch
it
for
in
a
long
time,
then
for
temporary
failures.
You
can
come
up
with
the
better
architecture
and
you
can
use
techniques
like
a
CQRS.
It's
a
common
query,
responsibility
segregation
and
let
me
show
an
example
for
this
architecture,
so
this
is
one
part
of
our
application.
A
What
you
can
see
on
the
screen
and
the
way
how
we
collect
some
data
from
our
customers
with
an
agent
and
after
that,
how
we
store
it
and
how
we
serve
it
for
the
user.
What
you
can
see
that,
when
a
customer's
collector
sending
data
to
a
cust
collector
API,
then
the
collected
API
doesn't
really
have
any
dependencies
except
the
queue.
So
it's
it
contains
release
in
validation,
layer
and
after
that
we
send
the
message
to
the
queue
immediately.
This
is
a
durable
de
I
book
you.
A
So
if
something
happens
here,
then
the
message
remains
in
this
queue,
even
after
they
starting
the
messaging
queue
system.
On
the
other
side
of
the
queue
a
worker
is
consuming
these
messages
and
calling
all
the
external
dependencies
and,
after
some
transformation
putting
the
data
into
the
database.
If
any
of
these
dependencies
breaks
the
database
or
any
other
service,
then
the
message
simply
goes
back
to
the
queue
with
some
retry
policies
and
and
some
maximum
they
try.
So
what
happens
is
by
the
user
receiving
the
database?
A
Maybe
the
data
is
not
there
yet,
because
we
face
some
issues,
but
after
we
solve
the
issue,
we
can
consume
all
the
messages
and
save
the
data,
so
we
don't
lose
any
data
and
that's
really
important
for
us.
You
can
see
an
example
here
what
happens
on
the
queue
side
in
this
case
when
the
worker
is
not
able
to
consume
the
messages,
then
a
couple
of
millions
messages,
even
four
hours
start
to
be
collected
there,
but
after
the
queue
size
goes
back
to
zero,
when
the
worker
comes
back
to
life.
A
Increasing
complexity,
I
mentioned
that
it's
coming
by
design
with
micro
services,
and
this
is,
for
example,
where
we
started
eight
months
ago.
This
was
or
manolito
so
normal.
It
was
never
really
one
protest,
but
what
you
can
see
here
that
this
application
is
split
mauled
by
functionality
instead
of
features,
so
we
had
a
worker,
we
had
a
front-end
server
and
we
have
a
collector
API,
and
this
is
how
the
trace
architecture
looks
today.
It's
you
can
see
much
more
services
and
databases
and
if
something
breaks,
then
it's
not
obvious
that
what
breaks
it.
A
But
what
you
can
do
with
that
and
if
you
have
some
pretty
good,
looks
then,
and
you
find
some
way
to
connect
them,
then
maybe
you
are
able
to
find
an
issue,
but
a
better
approach
is
if
you
are
moving
your
debugging
monitoring
to
something
which
is
called
disability
tracing.
This
is
some
trace
issue
in
the
trace
disability
testing
itself.
A
So
anytime,
when
one
service
is
calling
another
service,
you
need
some
kind
of
ID
to
being
able
later
to
connect
them
and
the
whole
concept
is
based
on
the
Google
dapper
white
paper
and
current
implementation
is,
for
example,
trace
or
Zipkin,
which
is
more
Java
based
and
response.
Time.
I
mentioned
that
if
your
network
is
slow-
and
you
put
your
services
after
each
other-
that
some
problem
can
happen,
for
example,
this
is
a
very
fast
service.
We
all
know
gos
develop
path,
so
you
know
what
does
it
mean?
A
It
has
below
2
milliseconds
medium
response
time.
What
happens
if
I
put
a
couple
of
those
fast
services
after
each
other
in
the
service
call
chain
in
a
synchronous
way,
but
my
network
is
slow
and
I
will
end
up
with
a
horrible
user
experience,
and
it's
happened
with
us
because
earlier
we
were
using
that
platform
as
a
service
provider,
but
we
didn't
have
control
over
the
network.
So
all
of
the
communication
between
our
services
went
out
to
the
internet,
reached
more
than
twenties
physical
devices
and
was
really
slow.
A
So
what
you
can
see
here,
it's
the
same
kind
of
distributing
tracing.
You
can
see
the
dark
grey
rectangle.
This
is
a
network
delay,
it's
multiple
of
the
processing
delay
and
because
of
that,
this
one's
time
for
the
user
is
slow,
so
natural
delay
is
always
evil
in
micro
services
and
what
you
can
do
about
that.
A
It's
a
pretty
easy
thing
that
if
you
keep
your
databases
and
services
in
the
same
data
center-
and
you
can
use
the
internal
network
for
the
communication
that
it's
much
better
foxie
approach,
most
of
the
talks
and
articles
when
we
are
talking
about
how
to
break
down
the
model.
It
is
starting
about
that.
Okay,
put
your
proxy
front
of
your
monolid
and
with
this
proxy
you
can
dilute
them
some
specific
requests
to
your
new
services.
A
It's
really
working
nice
because
it's
an
evolutionary
design,
so
you
don't
have
to
do
it,
one
step
which
would
be
really
risky,
and
you
can
even
put
more
logic
to
this
proxy
like
authentication,
but
what's
not
that
good?
If
you
only
do
direct
your
request,
then
all
of
your
services,
these
bones
and
protocol-
will
be
designed
for
the
browser.
So
your
service
interfaces
are
designed
with
browser
in
mind,
so
you
are
not
choosing
the
best
protocol
and
these
points
format
for
your
services.
You
are
choosing
one
which
is
required
by
your
client
application.
A
You
can
imagine
what
happens
when
you
have
a
mobile
application
and
the
browser
or
the
suddenly
turn
out
from
business.
Point
of
view
that
you
have
to
add
a
new
kind
of
platform,
so
it's
a
little
bit
similar
in
what
was
in
the
previous
talk
about
the
extra
layer
front
of
the
web
services,
and
this
approach
is
called
API
gateway
and
the
API
gateway.
You
can
simply
combine
these
ones
of
multiple
services.
A
You
can
even
sense
from
the
protocol,
for
example,
if
your
mobile
browser
needs
WebRTC
with
XML,
it's
a
stupid
example,
but
your
web
user
need
something
else.
Then
you
can
easily
use
the
same
services
with
the
same
interface
and
this
EP
I
get
where
you
can
transform
them.
So
can
you
can
move
application,
protocol
response
format
and
comedy
sourcing
to
api
gateways?
A
Security
I
mentioned
that
earlier
we
used
platform
as
a
service
provider.
So
basically
our
service
is
communicated
on
a
public
Internet.
So
to
prevent
the
unauthorized
request
between
services,
we
use
the
technique,
qualify,
signing
request,
signing
with
request
signing
you
have
a
private
and
the
public
key
and
the
service
which
is
creating
the
request.
It
grabs
specific
headers
and
the
body
and
creates
a
hash
with
the
private
key
from
them
and
send
it
in
a
header.
A
On
the
other
side,
the
service
which
receive
the
request
with
the
public
key,
can
check
that
really
it's
coming
from
the
service,
what
I
expected
and
for
this
we
are
using
the
node
HTTP
signature
by
giant,
which
is
luckily
built
in
the
request,
NPM
module
and
the
super
request
for
testing.
So
again,
the
community
is
really
great.
We
didn't
have
to
implement
anything
with
this
service
themes.
A
While
you
are
breaking
your
your
application
to
smaller
services
and
service
themes,
then
it
will
be
more
and
more
important
how
your
team
communicates
with
each
other
and
not
just
the
communication,
but
basically
your
services
are
products
for
their
customers
and
for
your
co-workers,
imaging
that
you
are
building
the
service,
which
is
depends
on
other
team
service.
If
that
service
has
a
lot
of
issues
or
slow
response,
then
it
will
affect
for
your
service
as
well
to
support
this
better
and
provide
some
guidelines
for
these
themes.
We
started
to
build
service
principles.
A
A
Every
time
when
we
add
a
new
feature
to
the
API,
if
it's
not
possible
to
keep
the
backward
compatibility,
then
we
simply
add
version
2
from
the
specific
API,
but
we
keep
the
old
one
until
we
until
any
other
service
using
it,
and
it's
also
true
for
services.
So
it's
totally
different
approach
to
the
previous
talk
that
we
don't
have
services
deployed
with
different
versions.
We
always
try
the
latest
code
to
be
deployed
on
production,
so
we
don't
give
the
code
for
a
long
time
on
branch
or
staging.
A
We
try
to
deploy
them
as
soon
as
possible.
We
do
this
because
we
don't
want
to
really
maintain
that
which
version
can
be
found
where
and
it's
much
more
easier
for
the
coworkers,
because
they
don't
have
to
ask
you
or
or
get
it
from
somewhere,
and
it
obviously
doesn't
means
that
we
ship
any
code
as
soon
as
possible
for
the
user.
It
would
be
crazy,
so
we
are
using
feature
toggles
and
flippers.
A
So
basically
we
disable
this
shipped
as
soon
as
possible,
but
the
feature
is
not
visible
for
the
user,
so
it's
called
the
dark
line
and
for
service
principles.
We
heavily
inspired
from
the
app
service
principles
you
can
find
it
on
github.
It's
lit
averse
to
read,
documented
ate
the
ice.
It's
again
that
you
don't
want
to
be
asked
all
the
time
that
what
grazing
parameters
should
I
use
when
I
using
your
service
so
enforce
documentation,
it
can
be
done
by
tools
or
simply
improving
the
habit.