►
From YouTube: The Secret Life of Monoliths - GitHub Satellite 2017
Description
Presented by:
Kir Shatrov
Developer Infrastructure Engineer, Shopify
Today, Shopify is the oldest actively developed Rails monolith. The codebase started in 2005, and it contains over a thousand models and 400 controllers, and remembers the very first Rails versions. Every day hundreds of developers are working on it and pushing new code into the single GitHub repo.
How do you scale, not in the number of requests served per minute, but from the perspective of developer experience? How can you automate code reviews and prevent developers from shooting themselves in the foot? We have built tools to make developers happy working with monolith, and I will be sharing our knowledge in this talk.
At Satellite 2017, attendees came to exchange cutting edge practices on how to bring together the people and tools you need to build great software. Watch the recordings to find out what GitHub's been working on, and hear how other teams use the GitHub ecosystem to customize the way they work.
We'll be getting together again in San Francisco on October 10th-12th for GitHub Universe. We hope you'll join us for three days filled with the creativity and curiosity of the largest software community in the world.
For more information, go to githubuniverse.com.
A
My
name
is
Joseph
and
my
talk
is
about
monoliths
big
applications,
and
how
can
you
scale
them
not
in
terms
of
requests
per
second
served
or
in
terms
of
the
load,
but
in
terms
of
developer
happiness
and
in
terms
of
how
many
developers
can
work
on
the
same
app
and
the
same
owners?
The
same
time
pushing
the
code
together
and
not
stepping
into
each
other
I
work
at
Shopify.
A
We
we
run
on
Ruby
on
Rails
and
the
company
started
at
the
same
time
when
the
framework
were
started,
so
it's
basically
the
12
years
old
codebase
that
have
never
been
a
written
which,
which
often
happens
in
in
terms
of
startups
when
they
choose
another
sect.
We
write
the
app
and
fixing
their
previous
mistakes.
A
Just
you
have
some
idea
how
how
large
is
our
model?
It's
hundreds
of
controllers,
hundreds
of
tables
in
the
database,
30%
more
tests
than
the
actual
code
and
have
almost
half
a
million
lines
of
code,
and
together
with
that,
we
have
hundreds
developers
working
on
Shopify
working
on
just
in
just
one
app
and
deploying
it
on
more
than
50
times
a
day
and
I.
A
So
we
have
a
team
that
works
exactly
that
works
on
internal
internal
tools,
developer
tools
to
make
our
developers
productive
from
from
the
very
first
day
when
they
clone
Shopify
locally
I,
start
developing
to
to
the
next
steps
when
they
push
code
on
github
when
the
the
CI
build
is
triggered
and
then
when
they
deploy
this
code,
we're
deeply
integrated
with
github
in
all
of
these
steps
and
I'll
go
through
through
these
steps.
Today,
in
my
talk
and
I
hope,
you'll
learn
some
cool
things
that
we
do
with
github.
A
We
think
about
this
as
a
foundation
with
a
ruby
of
a
language,
the
the
the
lowest
level
and
then
comes
Ruby
on
Rails,
the
framework
that
we
are
using
and
on
top
of
Ruby
on
Rails.
There
are
our
developer
tools,
starting
from
the
beginning,
the
very
first
day
you
just
started
it
you're
cloning,
the
repo
on
your
machine
and
what
you
would
usually
do
in
a
smaller
in
a
smaller
project.
You
would
clone
the
repo
and
you'll
see
I've,
read
me
describing
what
what
data
is?
You
need
locally
to
run
the
app?
A
Then
you
would
install
all
the
dependencies
with
the
commands
that
you
probably
call
it
base
it
from
the
readme
done
everything
manually,
and
hopefully
it
works.
Sometimes
you
maybe
you'll
have
some
libraries
missing
and
that
the
steps
from
Brittany
wouldn't
work,
and
you
would
be
frustrated
that
happened
with
our
developers
and
that
wouldn't
work
for
us,
because
we
had
ten
of
developers
starting
every
week
and
all
of
them
came
from
different
backgrounds.
A
So
we
thought
what,
if
we
put
all
development
environment
into
virtual
machine,
and
so
that
developers
would
have
this
virtual
machine
distributed
with
the
technology
like
vagrant
that
lets
you
that
lets
you
script,
the
the
local
virtual
machine
with
all
the
dependencies
that
you
need
without
messing
up
on
your
laptop
on
your
operating
system,
but
turned
out
that
vagrant
and
virtualization
as
what
allow
a
lot
of
an
overhead
and
when
developer,
bootstrap
the
virtual
machine.
It
took
them
an
hour
or
around
that
to
just
install
all
the
services
that
were
necessary
in
this
VM.
A
It
wasn't
very
stable
and
they
had
to
think
it
every
week
with
the
production
environment
to
have
the
same
version
of
my
sequel
locally
in
the
VM
as
in
production.
So
we
had
to
explore
and
rethink
it
from
the
scratch
to
make
our
developers
happier.
In
fact,
we
take
a
survey
of
our
developers
about
two
times
a
year
and
ask
them
what
are
the
tools
that
they
are
most
uncomfortable
with,
and
this
year
it
was
the
local
environment
that
they
wish
would
be
better.
A
We
discovered
some
technologies
like
it's
highway,
which
is
lightweight
in
native
virtualization,
for
my
class
that
is
now
used
by
docker
for
Mac,
compared
to
bigger
VM
solutions
that
we
tried
before
it
was
almost
zero
overhead.
We
had
to
write
a
create
our
own
wrapper
around
the
sky,
V
that
launched
all
all
smaller
virtual
machines
with
the
services
that
our
monolith
was
depending
like
my
sequel,
Rytas
and
all
the
databases
you
probably
know,
but
solving
that
we
still
have
the
problem.
Another
problem
to
solve.
A
It
takes
IMO
file
with
a
configuration
that
describes
what
the
project
needs
that
describes,
all
the
all
the
packages
of
the
databases
and
some
extra
steps
to
booster
up
the
project
and
after
it
we
created
that
with
our
models
in
mind.
But
at
the
same
time
we
also
have
hundreds
of
smaller
apps
at
the
company.
A
A
The
next
aspect
of
mana
is
that
that
often
suffers
from
when
the
code
base
growth
is
a
code
organization.
Imagine
you
want
to
add
at
one
of
those
hundreds
of
controllers
models
entities
and
you
edit
it
and
you
have
no
idea
who
is
the
right
person
or
the
right
team
to
two
being
and
discuss
the
solution.
Of
course
you
could
use
something
like
get
blame,
but
sometimes
it
happens
that
person
in
gateway
no
longer
works
here
or
they
are
they
move
to
another
team.
That's
not
no
longer
their
responsibility,
so
we
at
some
point.
A
We
had
to
organize
all
the
code
in
the
monolith
into
components,
which
is
something
not
very
usual
what
what
people
do
with
Ruby
on
Rails.
As
a
result,
we
had
this
this
structure
of
files,
where
every
team
owned
a
component
and
component
looked
almost
like
a
smaller
app.
At
the
same
time,
being
a
part
of
the
large
monolith,
the
large
code
base,
so
I
mean
when
you
now,
we
need
change
some
code
in
in
the
component.
A
A
We
have
this
bot
that
watches
a
set
of
files
according
to
the
rules,
and
in
this
case
someone
introduced
and
you
just
a
dependency
to
the
project
and
the
bot
automatically
being
the
group
of
people
who
are
interested
in
watching
your
dependencies,
because
you
know
monolith
already
has
so
many
dependencies
that
we
want
to
at
least
control
the
fact
of
growing
of
these
things,
and
this
board
helps
us
to
to
catch
some
fundamental
changes
that
draw
where's.
My
accidential
introduce.
A
In
fact,
we
had
70,000
tests
and
we
we
run
them
for
every
build.
That's
a
very
huge
number.
There
are
some
available
solutions,
open
source
solutions
to
paralyze
the
builds,
usually
they're
paralyzed.
The
test
runners
are
paralyzed
by
the
number,
of
course,
on
your
machine
and
the
Forex
or
a
tax
or
even
16x.
Program
in
improvements
are
are
very,
very
easy
to
get
to
and
the
big
step.
But
for
us
it's
no
longer
worked.
So
we
had
to
rethink
that
the
whole
approach,
how
we
do
CI
and
how
we
run
the
build.
A
It
works
as
a
as
a
big
queue
that
and
using
the
key
allows
us
to
distribute
the
test
very
very
evenly
between
the
workers.
So
we
push
70,000
test
in
the
queue
and
then
we
assign
a
couple
hundred
workers
here
built
these
workers
pulled
the
desk
from
the
queue
and
that
allows
us
to
distribute
the
test
in
the
very
even
way.
A
After
your,
your
build
path
is
damaged
to
ship
the
code
and
in
a
in
a
large
organizations
with
a
large
app.
It's
also
easy
to
do
to
create
a
mass
of
developers
trying
to
ship
the
code,
sometimes
sometimes
breaking
the
the
infrastructure
introducing
server
destructions,
because
people
wanted
to
ship
their
changes
concurrently.
A
So
we,
this
is
another
example
of
us
integrating,
maybe
even
hacking
into
github
in
a
way
that
you'll
see.
So
this
is
the
the
merge
button
on
github
that
you've
probably
seen
a
hundred
to
talk
of
times,
but
there
is
also
a
thing
and
the
screenshot
that
you
haven't
seen
yet
it's
the
it's
emerged
view
that
we
have
so
we
extend
the
the
github
merge
window
with
something
that's
called
merge
queue.
A
Another
cool
thing
about
the
Merc:
you
is
that
we
can
block
our
developers
from
merging
the
code.
For
instance,
if
there
is
some
incident
running
were
investigating
issues
and
we
don't
want
any
new
code
to
be
introduced
to
the
master
branch.
So
we
can
lock
the
merge
queue
with
some
message
to
ask
developers
to
to
hold
and
wait
for
for
another
hour
until
the
issues
are
resolved.
A
You
yoga
you'll,
also
get
a
notifications
from
the
from
the
chat
bot
that
asks
you
to
to
be
around
watch
the
deploy,
dashboard
watch
for
any
new
exceptions
that
can
pop
up
and
in
the
in
the
worst
case,
you'll
just
hit
the
rollback
button
that
works
in
seconds
and
another
message
when
the
code
is
deployed.
Many
of
these
tools
that
I
mentioned
are
are
open.
Source
and
I'll
have
links
to
to
all
these
tools
and
blog
posts
about
our
tooling
and
github
projects.
A
The
usual
upgrade
strategy
for
a
smaller
app
would
be
to
create
a
brain,
make
changes
see
if
it
works
and
merge
it,
and
it's
done
that
wouldn't
work
for
us
at
all,
because
upgrades
can
take
several
months
and
any
branch
would
out
date
during
this
time,
and
so
the
only
strategy
that
we
had
what
was
the
dual
strategy?
The
dual
strategy
means
that
the
code
in
master
well
during
the
upgrade
project
going
on
the
code
in
master,
should
be
able
to
run
on
both
versions
of
framework
on
the
old
one
and
on
the
new
one.
A
Of
course,
some
things
may
break
on
the
new
version
and
our
customers
wanted
that
path,
but
all
developers
that
work
with
the
company
at
the
time
of
the
upgrade
they
they
all
know
that
there
is
a
braid
going
on
and
the
the
new
code
that
they
introduced
has
to
be
compatible
with
with
both
versions
as
a
result
around
the
code.
We
have
this
conditionals.
If
it's
older
versions
do
this,
if
it's
the
new
version
do
slightly
different
things,
all
these
conditions
are
automatically
cleaned
up
as
the
as
the
upgrade
is
done.
A
A
The
last
upgrade
took
us
more
than
half
a
year,
and
this
is
the
the
github
issue
that
we
used
to
track
all
the
PRS
that
have
been
worst
and
discussed.
This
issues
really
long
I
think
it
takes.
It
have
like
30
seconds
just
to
render
the
that
page,
because
there
are
so
many
references
and
when,
when
you
that
the
venue
that
the
code
runs
correctly
on
both
versions
of
the
framework,
we
could
gradually
roll
out
the
new
version
to
some
part
of
our
customers.
A
A
We
love
collecting
feedback
from
our
developers
and
some
here
it
was
the
local
environment
that
they
felt
uncomfortable
with
and
later
it
was
the
the
old
CI
system
that
they
felt
unproductive
and
we
worked
on
that
and
we
improved
it.
We
also
try
to
instrument
the
the
whole
code
shipping
pipeline
to
instrument.