►
Description
Kellan Elliott-McCrea, former CTO of Etsy, describes lessons learned in his 5 year tenure leading an engineering organization. In this talk, he discusses how to enable a team to produce software of quality, given the constraints and difficulties inherent in the art.
About GitHub Universe:
Great software is more than code. GitHub Universe serves as a showcase for how people work together to solve the hard problems of developing software.
For more information on GitHub Universe, check the website:
http://githubuniverse.com
A
Hi,
like
Scott,
said
my
name
is
kal-el
McRae.
Until
a
few
weeks
ago,
I
was
the
CTO
at
sea,
I'd
been
there
for
five
years,
helped
grow
and
lead
that
team,
and
when
I
left
I
wrote
a
blog
post
about
the
things
I
learned
and
chris
kelly
from
github
emailed
me
and
said:
hey.
That
was
pretty
good.
Why
don't
you
come
talk
at
github
universe,
and
I
said
that
sounds
like
fun.
He
said
great.
We
want
you
to
come
and
tell
us
everything.
A
You
know
about
software
development,
engineering,
leadership
and
everything
you
learned
in
the
last
five
years.
You
have
20
minutes.
I
said
that
sounds
fine
will
be
plenty
of
time
for
questions
I'm
not
actually
going
to
take
you
through
the
whole
blog
post
or
quite
everything,
I've
learned
in
the
last
five
years.
I
want
to
focus
on
just
one
thing:
I
learned
and
it
underlies
all
of
the
work.
This
for
one
theme
that
runs
through
that
work.
A
We're
going
to
talk
about
the
one
true
way
to
build
software,
which
is
curiously
and
why
I
don't
actually
mean
sort
of
strangely
I
mean
with
an
open
mind.
I
want
you
to
bring
hypotheses
experiments
feces
to
be
tested.
Why?
Why
is
that
the
right
way
to
build
software,
because
we're
really
bad
at
it?
We're
really
bad
at
and
I,
don't
mean
I'm
bad
at
it
or
you're
bad
at
it,
or
the
people
in
this
room
are
bad
at
billing
software
and
someone
out
there
has
the
right
answer.
A
If
we
can
just
get
them
to
tell
us
will
be
enlightened,
I
mean
as
a
species.
We
are
bad
at
building
software.
This
occasionally
surprises
people,
because
some
of
us
are
good
at
writing
code
and
we
sent
him
to
think
it's
the
same
thing
and,
unfortunately
they're
not
actually
the
same
thing.
I
have
some
theories
about.
Why
we're
so
bad
at
building
software
I'm
not
going
to
linger
on
them
too
much.
A
A
Modern
software
development
is
a
complex,
large-scale,
fast-moving,
improvisational,
real-time
long
time,
scale,
creative,
medic,
endeavor,
it's
like
a
sports
team
or
a
band
or
a
dance
troupe,
had
to
get
up
every
day
and
instead
of
practice,
we
play
we
get
up
each
and
every
day
and
we're
on
stage
and
we're
performing
this
large
group
creative
endeavor,
and
we
do
it
for
months
or
years
at
a
time,
and
we
are
judged
by
the
entire
output
of
the
work
over
that
time
period.
It's
hard
and
we
don't
have
great
models
for
doing
it.
A
We've
tried
industrial
models,
we've
tried
military
models,
we've
tried
not
having
a
model,
but
it's
hard
and
the
other
contributing
factor
is
the
rate
of
change,
and
it's
almost
banal.
Let's
talk
about
how
fast
technology
changes
in
this
industry.
Almost
it
never
quite
gets
banal,
but
you
know
we.
We
don't
really
internalize
the
fact
that
the
rules
of
the
game
changed
two
to
three
times
a
decade,
if
not
more
quickly.
You
know
fundamental
leave
the
laws
of
physics
change
on
us
as
we
are
trying
to
build
software
several
times
a
decade.
A
You
think
about
architects
who
had
to
deal
with
new
laws
of
physics,
building
houses
several
times
a
decade.
They
would
not
be
as
good
at
building
buildings
as
they
are
and
we
haven't
really
internalized.
This
fact
we're
doing
something
incredibly
hard
and
the
rules
keep
changing,
which
brings
me
to
kill
and
second
law
of
software
development.
Your
answers
are
wrong,
or
they
will
be
soon
by
the
way.
The
first
law
of
software
development
is
never
built,
a
content
management
system,
no
good
can
come
of
it.
A
So
the
questions
stay
the
same.
The
answers,
change-
and
this
seems
like
it
may
or
may
not
seem
profound
to
you,
but
it
really
is
a
fundamental
shift
in
what
we
are
trying
to
do.
As
software
development
organizations,
we
are
no
longer
in
search
of
the
right
answer:
no
I'm
trying
to
find
the
right
product
or
the
right
technical
choice,
we're
now
a
team
of
people
creative
collaboratively.
A
So,
let's
talk
about
how
you
do
this
as
organizations
and
I'm
gonna
start
off
with
two
assumptions
that
I'm
naked,
which
may
or
may
not
apply
to
you
that
bill
didn't
work
right.
That's
fine.
First
is
you're
here
to
make
sustainable
long-term
change
in
the
world.
You
know
whether
or
not
that's
out
of
a
mission
or
just
you
think
the
world
could
work
better,
maybe
you're
trying
to
make
a
lot
of
money.
It
doesn't
really
matter.
A
You
know
you
are
trying
to
do
something
significant,
and
that
takes
time,
and
you
need
to
survive
to
get
to
that
significant
change,
you're
trying
to
make
in
the
world
sustainable
long-term
change,
and
you
care
about
the
thing
you're
building,
it's
important
to
you,
whether
it's
an
organization
accompany
a
product
and
open
source
project
the.
What
is
important
is
that
just
two
assumptions:
if
those
don't
fit
for
you,
if
you
you're,
sort
of
building
a
thing
quickly
to
like
flip
it
and
sell
it,
I'm
happy
for
you,
I
just
have
no
advice
for
you.
A
This
is
for
the
assumptions
that
I
bring
to
this
work,
so
we're
building
for
the
long
term
and
as
a
technology
organization
building
for
the
long
term,
it's
not
about
stone
and
steel
and
cement
in
rebar
and
mortar
and
sort
of
strength
and
depth
as
an
architect
might
think
about
it.
It's
about
getting
really
good
at
change.
Change
is
inevitable.
We
need
to
be
good
at
it,
and
this
is
this
is
what
long-term
means
for
us.
You
know
we're
still
stuck
in
sort
of
models.
Of
of
this
is
sort
of
a
long-term
unchanging.
A
In
the
face
of
time,
that's
irrelevant
to
the
work
that
we're
doing
your
culture
needs
to
be
a
learning
culture.
Your
software
development
organization
needs
to
be
a
culture
of
learning.
Great
boom.
Mic
drop
problem
done.
Enlightenment,
no,
of
course
not
you've
heard
this
before.
So,
how
do
we
do
it?
That's
the
really
interesting
question.
As
software
development
organizations,
how
do
we
become
learning
organizations?
A
Well.
The
first
question
is
what
keeps
us
from
being
learning
organizations.
You
know
there
are
some
behaviors
that
keep
us
from
being
wearing
organization's.
Culture
is
hard.
You
have
to
wake
up
every
morning
and
decide
you
want
to
promote
a
particular
culture.
It
doesn't
happen
on
the
side,
it
doesn't
happen
after
hours.
It
doesn't
happen
at
your
wednesday
lunch
and
learns
or
brown
bags.
Culture
is
an
ongoing
set
of
activities,
it
takes
a
ton
of
energy
and
it
needs
to
be
your
top
priority
if
you're
going
to
shape
your
culture.
A
So
first,
let's
talk
about
the
things
to
hold
us
back
from
being
run
on
cultures.
Some
of
these
may
sound
familiar
to
you
and
you're,
going
to
speculate,
firing,
people
for
making
mistakes
failing
to
hold
people
accountable
for
outcomes,
blaming
failure
on
human
air,
blaming
forces
outside
of
our
control.
No
one
could
have
foreseen
it.
Who
cares
thrash
frequent
focus
change
your
reorganizations
teams
that
don't
have
long-standing
trust
and
long-standing
goals?
Don't
learn
going
off
quietly
in
a
corner,
not
shipping.
I
put
my
headphones
on.
A
A
Also
under
investing
in
our
stack,
the
technology
will
take
care
of
itself
I'm
trying
to
build
a
business
here.
You
might
have
a
lot
of
free-floating
new
information
in
your
system,
but
you're
not
learning
if
you're
not
reinvesting
it
optimizing
locally,
if
you're,
not
thinking
about
the
entire
organization
in
space
you're,
not
learning,
autocratic
decision-making,
just
because
the
CEO
made
the
decision
or
the
best
engineer
or
the
loudest
voice
in
the
organization.
A
A
So
how
do
we
support
learning
and
a
technical
organization?
I?
Have
this
wonky
slide
that
I
use
for
internal
talks
that
actually
don't
think
I've
ever
used
it
externally?
So
thirteen
steps
for
learning
with
an
engine
of
TG
its
deployment
at
the
middle
of
it?
We
have
like
11
minutes
so
I'm
going
to
do
a
simplified
version
of
this.
There
are
four
key
quadrants
of
behavior.
This
is
still
a
cycle.
This
is
not
a
checklist.
A
You
know
I
figure,
1,
2,
3,
4
boom
done
something
you
do
over
and
over
again,
so
you
pick
a
quadrant
and
you
go
clockwise
in
your
behavior
and
you
do
it
as
you
get
better
and
better.
Let's
start
off
with
confidently
take
risks,
and
in
each
of
these
section
there's
going
to
be
the
one
key
question
you
need
to
be
thinking
about,
while
you're
in
that
quadrant.
A
So
in
this
one,
the
question
is:
what
is
the
largest
single
change
you
could
deploy
to
the
cyst
in
your
building
with
pert,
with
near
perfect
confidence
for
how
many
people
in
the
room
is
there
no
change?
You
could
deploy
with
your
perfect
confidence
handful
of
hands
for
how
many
people
are
you
confident
that
nothing
you
could
do
would
break
the
system
if
you
deployed
it
yet
no
hands.
So
everyone
else
is
on
the
spectrum
somewhere
there
that
wasn't
a
terribly
good,
interactive
behavior.
And
what
do
we
invite
confidence?
A
A
The
second
question
is:
what
gives
you
that
confidence
you
get
up
every
day
and
you
are
you
know
for
those
of
you
who
are
part
of
software
development
organizations
and
I
assume.
That
applies
to
most
of
us.
You
get
up
every
day
and
you
make
changes
to
your
system
and
something
keeps
you
from
running
screaming
through
from
the
room
and
those
are
the
things
that
give
you
confidence
and
in
a
modern
software
organization.
There
are
lots
of
answers
to
that
question.
A
There's
tests
this
training,
there's
QA,
there's
monitoring,
there's
metrics
this
code
review,
there's
ramp
ups,
there's
user
studies.
It
could
be
that
these
changes
are
trivial
or
the
routine
by
the
way,
those
two
words
or
anti-patterns-
don't
use
them.
And
but
you
know,
when
etsy
started
doing
continuous
deployment.
We
we
didn't
trust
our
test,
we'd,
actually
just
rmr
f
them,
because
they
were
flaky
and
we
didn't
trust
the
code
cuz.
A
We
had
inherited
it
from
a
bunch
of
people
who
were
brilliant,
but
insane
and
we
didn't
have
QA
and
we
didn't
like
our
monitoring,
but
you
don't
have
to
have
perfect
tools
to
get
started
at
the
point
of
the
story.
You
know
we
had
five
graphs,
we
built
a
little
tool
called
stats
d
and
we
used
it
to
graph
five
graphs
and
that
was
sort
of
how
many
people
were
logging
in
how
many
people
were
creating
new
items
for
sale.
A
How
many
items
were
selling
I'm,
forgetting
one
oh
yeah,
sign
ups
and
logins,
and
the
number
of
posts
in
our
bugs
forum
and
those
five
gaffes
gave
us
pretty
good
confidence
that
we
hadn't
totally
screwed
it
up
with
a
deploy
and
that
allowed
us
to
start
moving
forward.
So
that's
confidence,
you
know
and
you
make
risky
changes,
you
take
more
confidence.
So
there's
simple
change.
You'd
have
less
confidence,
but
you
have
confidence
and
you
take
risks
because
otherwise
nothing
else
happens.
A
So,
let's
move
on
to
the
second
quadrant,
we
honestly
reflect
so
we
made
a
change
and
we
had
confidence
in
it
and
surprise
happened.
Anyways.
This
is
inevitable.
Who
here
is
deployed
a
change
so
trivial?
It
couldn't
possibly
break
something
and
it
broke
everything
yeah
there
we
go
for
the
rest
of
you.
You
should
try
software
development,
it's
awesome,
you
know
at
SC
we
have
an
award
that
we
give
each
year
or
they
said
not
there
anymore.
It's
going
to
take
me
a
while
to
stop
saying
we.
A
It's
called
the
three
armed
sweater
and
person
who
most
spectacularly
broke
the
site
in
a
given
year
and
for
the
last
couple
years
we've
been
really
giving
it
to
the
people
who
most
spectacularly
broke
the
site
with
the
most
innocuous
seeming
change.
You
know
two
years
ago
an
engineer,
sort
of
I
think
I
sort
of
like
wrapping
up
at
the
end
of
day
the
day
pushed
a
one
line:
change
to
an
IE
eight
only
CSS
file
and
managed
to
ddos
our
entire
web
plus
there
with
a
sort
of
an
infinite
loop.
A
A
So
the
question
you're
thinking
about
in
this
quadrant-
and
this
is
the
hardest
question
to
answer
by
the
way-
is
what
is
it
about
the
environment
in
which
you
made
the
change
that
gave
you
false
confidence?
We
don't
think
about
this
question.
I
did
a
thing:
it
broke
I
up,
I'm,
human,
whatever
I'll
do
better
next
time,
not
useful.
That's
a
lazy,
an
unacceptable
answer.
There
was
something
about
the
environment
that
made.
A
You
think
that
the
change
you
were
pushing
was
going
to
work
and
it
was
wrong,
and
so
what
we
need
to
do
is
we
need
to
figure
out
what
about
the
environment
is
wrong,
so
that
we
can
change
it.
There
are
some
practices
that
help
with
this
question.
Five
wise
is
one
blameless.
Post-Mortems
is
another
we've
written
pretty
extensively
about
how
we
do
blameless
post-mortems
at
etsy.
A
These
are
documented
I
recommend,
checking
them
out
on
the
on
the
internet
and
actually
I
want
to
talk
about
one
particular
type
of
surprise
that
we
run
into
over
and
over
again,
which
is
problem.
The
surprises
around
detection.
This
is
an
interesting
subclass
of
surprise.
You
know
we're
often
surprised
by
how
long
a
bug
can
exist
in
our
system.
How
is
it
that
bug
has
been
out
there
for
24
hours
42
hours
two
years?
How
is
that
possible?
Are
we
amateurs
here?
Is
this
amateur
hour?
A
Nothing,
the
CEO
quite
going
like
finding
a
bug
that
had
been
out
there
for
six
months,
so
surprise
is
inevitable.
A
hundred
percent
prevention
is
impossible
and
actually
counterintuitively,
possibly
the
sort
of
asymptotic
approach
to
I
will
think
of
all
possible
ways.
Could
break
and
proactively
cut
around
them,
leads
to
more
complexity
in
your
system
and
more
breakage.
But
detection
is
the
things
really
worth
investing
in?
You
know
getting
good
at
detection,
getting
fast
detection.
Getting
to
the
point
that
you
are
very
quickly
surprised
and
you
can
respond
to.
A
It
is
the
engine
that
drives
a
lot
of
learning.
It
also
drives
for
modern
security
practices.
You
know
product
exploration,
getting
really
good
at
understanding
very
quickly,
the
Delta
between
what
you
thought
your
system
should
be
doing,
and
what
you
are
currently
your
system
is
currently
doing
is
sort
of
the
secret
sauce
from
an
engineering
standpoint.
A
So
we've
honestly
reflected
and
now
it's
time
to
take
those
reflections
and
learnings
and
reinvest.
You
know
the
question
for
this
quadrant.
What
are
the
changes
you
can
make
to
the
environment
you're
operating
in
to
increase
your
confidence
to
take
risks?
You
know
we
want
to
get
back
to
the
point
where
we
can
move
more
quickly,
do
higher
risky
things,
but
do
them
with
confidence.
Do
them
in
a
way
that
we're
not
just
thrown
over
the
wall.
A
You
know
we
are
not
Cowboys
here
unless
you
herd
cows,
in
which
case
I'm
happy
for
you,
but
for
the
rest
of
us.
You
know
we
are
trying
to
do
systematically
risky,
fast-moving
but
systematically
and
building
over
time.
So
you
took
a
risk.
You
got
surprised,
you've
reflected
honestly.
On
that
surprise,
you
now
understand,
and
you
are
a
unique
moment
in
time
right
now.
You
have
perfect
clarity
about
how
to
be
a
better
software
developer.
A
This
happens
to
you,
but
almost
no
point
in
your
career,
and
you
have
that
perfect
clarity
over
a
very
narrow
range
I
could,
if
I
hadn't
deployed
on
a
Friday
afternoon,
when
everybody
else
was
already
knocked
off
or
at
sort
of
beer
friday
I
might
have
gotten
better
feedback
on
how
to
do
this.
The
graph
was
misleading.
The
tests
are
flaky,
you
know,
CSS
actually
has
the
potential
to
cause
infinite
loops
in
our
precompiler.
You
have
perfect
clarity
for
a
moment,
and
this
is
really.
A
A
This
is
not
the
thing
that
we
need
to
do
to
survive,
but
if
you
want
to
survive
for
the
long
term,
if
you
want
to
build
sustainable
long-term
technological
impact,
you
need
to
not
get
crushed
under
the
weight
of
the
the
gear,
the
complexity
that
what
you're
dealing
with-
and
this
is
how
you
do
it-
you
prioritize
this
work
and
that's
through
the
rule
was
it
got
done
in
30
days.
If
we
got
surprised,
we
had
a
post-mortem,
we
learned
some
things.
A
Those
things
got
done
in
30
days,
come
hell
or
high
water,
Peter
SIBO,
who
used
to
work
at
sea
and
now
so
the
lead
engineer
on
Twitter's
engineering
efficiency
team.
So
the
new
team
that
created
gave
a
great
talk
last
week
at
at
scale
at
Facebook,
called
let
a
thousand
flowers
bloom
and
rip
out
99
of
them,
taking
all
about
how
over
time
investing
in
tools,
particularly
for
large
engineering
teams,
leads
to
this
massive
wins.
I
recommend
checking
out
the
talk.
Y'all's
did
a
nice
write-up
of
it,
so
we've
prioritized
investing.
A
This
is
the
final
quadrant.
As
we
teach
you
know,
the
reflection
phase
was
painful.
It's
always
painful
until
it
gets
fun,
the
investments
face
was
expensive.
We
were
building
a
bunch
of
tools
that
had
nothing
to
do
with
our
core
business,
and
so
the
question
becomes.
How
can
we
afford
to
keep
working
like
this?
How
can
we
afford
this
level
of
distraction?
How
can
we
afford
to
make
ourselves
better
software
engineers?
That's
not
what
we're
here
to
do
so
as
an
organization.
So
first
did
it
work
as
an
organization.
A
Can
we
move
faster
and
take
greater
risks
with
confidence?
Now,
if
no,
we
should
go
back
and
look
at
how
we
got
here,
but
generally
you
find
the
know
in
this
is
that
we
haven't
taught
it.
You
know.
I
was
surprised.
I
sat
down
with
my
small
team.
We
made
some
fixes
and
now
there
are
two
ways
of
doing
things.
This
is
classic
now
you
have
in
problems,
and
this
is
not
how
learning
happens.
You
know
in
organizations
you
need
to
be
glow,
it
optimally
globally.
A
Optimizing
you
know
the
the
more
and
more
solutions
to
the
same
problem
actually
undermines
your
confidence
in
your
system.
You
need
to
invest
the
time
in
teaching.
It's
the
only
way
you
can
afford
to
do
any
of
this
stuff.
If
each
of
you
have
your
own
sort
of
beautiful
microservice
flower
of
unique
snowflake
pneus,
you
are
not
a
learning
organization,
the
practices
and
documentation,
xand
tools,
you
build-
have
to
be
high
enough
quality
that
everybody
wants
to
use
them
and
you
have
to
invest
in
making
sure
that
happens.
A
The
question
we
asked
in
our
architectural
review
is
is
great,
so
we
now
have
two
solutions
for
this.
What
is
your
plan
for
making
sure
that
your
solution
kills
and
eats
the
other
solution?
Just
in
case
you
thought
we
were
all
hippies
here,
all
right
and
then
you're
going
to
step
one.
You
do
it
again
and
you
get
faster
and
you
get
better
and,
like
I
said
it
doesn't
actually
matter
where
you
start
in
the
cycle
pick
where
you
are
invest
over
time.
Take
the
time.
Take
the
energy
to
make
this
happen.
A
I've
got
one
and
a
half
minutes
left.
This
sounds
a
lot
like
agile
and
lean
to
some
people.
What's
different
about.
This
is
the
goal.
Our
goal
is
to
be
a
learning
organization.
If
your
goal
is
to
be
a
learning
organization,
that
is
your
number
one
goal.
You
can't
be
compromising
it
with
other
things.
Agile
is
a
set
of
practices
for
increasing
the
success
rate
of
software
development
lean
is
the
set
of
practices
for
increasing
the
chance
of
finding
a
proper
market,
fit
the
practices.