►
From YouTube: The code behind cars: How GitHub turned Ford into a software company - GitHub Universe 2019
Description
Presented by Tim Carmean, Central Software Process & Tools Supervisor, and Tom Erickson, Global Software Tools & Processes, Ford
Ford began writing software in the 1980s using assembly on a Motorola 8065 microcontroller to implement electronic fuel injection. Today, there are dozens of Electronic Control Units controlling the features and functions of modern automobiles. Writing code that powers vehicles carries unique environmental and regulatory compliance requirements, including functional safety, emissions, and warranty. Ford is transforming the way they think about software, thanks to GitHub.
This talk will discuss the history of software engineering at Ford, some of the unique requirements and challenges faced in their systems—and how GitHub is helping them deliver the products their customers want, with the quality they demand at the speed of the modern software industry.
About GitHub Universe:
GitHub Universe is a two-day conference dedicated to the creativity and curiosity of the largest software community in the world. Sessions cover topics from team culture to open source software across industries and technologies.
For more information on GitHub Universe, check the website:
https://githubuniverse.com
A
Henry
Ford's,
affordable
car
put
the
world
on
wheels
and
made
mobility
accessible
to
everyone
to
him.
Mobility
met
freedom,
freedom
to
choose
where
to
live
where
to
work
and
to
play
that
freedom
to
move
literally
change,
the
human
experience
and
elevated
the
quality
of
life
at
Ford
Mobility
is,
and
always
has
been
so
much
more
than
just
getting
from
point
A
to
point
B.
A
It's
about
progress,
human
progress
thinking
far
beyond
the
vehicle
today,
we're
still
driven
by
this
innovative
mindset,
one
that
pushes
us
to
challenge
traditional
ways
of
thinking
of
working
of
moving
it
sparks
our
imagination.
Lets
us
cultivate
fresh
ideas
and
new
approaches
to
find
transportation
solutions
for
a
constantly
changing
world
a
world.
That's
constantly
challenged
by
this
change.
This
mindset
is
also
the
beacon
that
guides
us
as
we
transform
from
a
traditional
vehicle
manufacturer
into
a
true
mobility
company.
That
offers
not
only
bold
products
but
exceptional
consumer
experiences
to
it.
A
Smart
vehicles
for
a
smart
world
to
move
people
safely
confidently
and
freely,
but
Ford
is
more
than
just
steel
and
lines
of
code.
Ford
is
powered
by
our
people
because
as
diverse
as
our
products
and
customers,
so
is
our
company.
We
not
only
celebrate
diversity,
we
embrace
it
because
it
makes
us
a
better.
A
stronger
and
a
more
robust
company.
Ford
is
driven
by
our
culture
and
a
set
of
simple
truths.
A
That
guide
us,
teach
us
and
remind
us
every
day
of
our
purpose,
to
take
them
and
be
empowered
to
ask
questions
to
challenge
customs
to
never
be
afraid
to
take
an
intelligent
risk
and
to
make
a
real
difference
and
like
it's
always
been
at
Ford,
it's
working
together
from
every
corner
of
the
globe
to
build
great
products
to
craft,
remarkable
solutions
to
give
back
to
the
communities
we
serve
and
to
collectively
achieve
something
really
meaningful.
It's
making
people's
lives
better
and
once
again
changing
the
way
the
world
moves.
B
B
C
C
C
C
So
I
started
at
Ford
in
1991
and
I
didn't
realize
that
was
a
pivotal
time
in
software
development
at
Ford,
Motor
Company.
We
didn't
really
use
the
term
software
engineer.
We
call
them
strategists,
so
the
software
was
a
big
part
of
our
control
systems.
So
when
I
first
started,
I
was
working
on
one
of
the
first
generations
of
electronic
engine
control
systems,
our
equation
of
software
systems,
but
we
focus
a
lot
more
on
the
controls
and
we
developed
a
custom
microprocessor
8061,
but
with
Intel.
So
it's
based
on
the
ad
886
architecture.
C
Now
a
lot
of
this
switch.
Why
did
we
need
electronic
energy
control?
Well,
emissions
was
until
in
getting
more
and
more
challenging,
and
so
we
needed
to
improve
the
performance
of
the
system
and
eliminate
some
of
the
mechanical
where
that
came
from
normal
mechanical
systems.
So
all
the
source
code
was
really
assembly
language.
We
had
a
some
custom
opcode
that
was
responsible
for
the
new
8061.
C
Our
software
architecture
might
look
a
little
simple
back.
Then
it
was
a
single
loop
with
interrupts
based
on
the
ignition
firing
order.
So
one
four
to
three
is
the
firing
order
of
a
four-cylinder
engine,
so
the
8061
didn't
include
an
on-chip
memory
at
that
point
in
time.
So
when
I
first
started
that
the
chip
on
the
upper
left,
the
32
kuv
EEPROM,
was
wife
and
was
my
friends
we
would
develop
with
that.
The
one
below
it
in
the
center
is
the
actual
production
chip.
That
is
the
a
squared
problem.
C
It
did
allow
us
to
reprogram
at
least
way
back
then.
Eventually
we
evolved
and
got
more
and
more
memory.
So
memory
became
a
big
part
of
our
lives.
All
of
our
code
would
just
package
into
the
small
tiny
memory,
so
we
have
sector
maps
created
for
the
assembly
code
and
then
the
calibration
data
was
separate
on
purpose.
We
have
built
our
code
embed
it
and
then
we
could
modify
the
calibration
data
on
the
fly
in
the
dynamometer
or
in
the
vehicle.
So
that
was
a
big
part.
C
We
had
custom
tools
developed
to
really
capture
the
calibration
data
itself,
so
at
Ford
in
the
70s
and
80s,
our
data
processing
group,
which
later
became
IT,
really
was
embedded
with
digital.
At
that
time
we
grabbed
some
of
their
servers
and
started
to
develop
our
own
tools
to
support
software.
So
you
can
see
that
we
had
to
cut
to
develop
a
custom
cross
assembler
for
that
new
opcode.
C
We
also
had
to
do
some
change
management
tools,
since
we
really
wasn't
a
lot
of
change
management
tools
in
the
market
at
that
time,
and
then
data
management
that
special
calibration
data
was
really
required.
So
we
had
a
whole
bunch
of
custom
tools.
All
the
code
was
was
managed
in
a
CMS
tool
from
digital,
which
maybe
you
guys
have
heard
about
that-
maybe
not,
and
then
we
had
developed
our
own
custom
Asics.
So
those
were
important
application.
C
We
had
an
opportunity
to
upgrade
the
8061
to
what
we
called
in
1865
and
then
we
included
a
lot
of
those
custom
Asics
in
the
chip,
but
we
still
didn't
have
on
on-chip
memory
and
we
still
had
assembler
code
and
we
call
this
our
generate
our
fifth
generation,
and
this
was
the
actual
model
that
fully
eliminated
carburetors
globally.
Prior
to
that,
the
ich
four
was
a
US
base,
and
so
we
extended
out
in
fact
in
91
when
I
first
started.
C
That's
when
I
was
a
part
of
the
International
powertrain
programs,
where
we
were
implementing
some
of
these
control
systems
for
other
markets
like
South,
America
and
Europe,
and
successfully
eliminated
carburetors,
so
assembly
code
not
necessarily
most
effective
for
us,
and
we
wanted
to
expand
and
and
leverage
some
of
the
newer
capabilities.
So
C
code
was
a
pretty
natural
migration
for
us
in
1992,
atria
launched,
rational
clearcase
and
so
by
94.
C
C
So
our
continued
to
changed.
We,
you
know
we
do
a
lot
all
the
software
in-house,
so
we
developed
our
own
real-time
operating
system
that
allowed
us
to
create
feature
code
that
now
expanded
the
content
well
beyond
the
hundred
or
so
features.
We
were
doing
at
that
time.
All
the
driver
code
for
all
the
hardware
interfaces
and
other
Asics
we're
also
embedded
there,
and
we
had
more
loops
that
we
could
leverage.
So
all
the
feature
engineers
could
actually
start
to
leverage
those
individual
loop
times
for
whatever
was
important
for
their
control
systems.
C
Now
we
were
already
global
Ford's
been
actually
global
since
the
beginning
of
the
company
in
the
1990.
No
three,
so
we
were
already
global,
but
our
feature
set
was
growing
beyond
200,
so
we
really
had
to
start
multi,
citing
our
clearcase
all
over
the
country,
all
over
the
world
and
so
I
had
served.
My
team
had
servers
in
Australia,
Brazil
and
Europe,
and
clearcase
really
started
to
have
a
lot
of
admin.
Overhead
now
I
had
custom
tools,
custom,
scripts,
multi-sided
code
everywhere.
C
It
was
a
very
big
challenge
and
at
the
same
time,
every
other
system
in
the
vehicle
started
really
developing
their
own
digital
control
systems
or
interfaces.
So,
as
those
teams
started
up,
they
would
choose
other
tools
so
I
had
we
had
lived
enclaves
of
different
toolsets
throughout
the
company,
so
that
really
drove
us
to
to
to
get
some
sort
of
commonality.
You
know
the
powertrain
was
already
pretty
agile.
We
were
running
even
in
the
90s.
We
could
do
build
software
on
a
weekly
basis.
All
those
other
teams
were
working
with
supplier
partners
and
so
oftentimes.
C
It
would
be
a
three
month
turnaround
or
old
waterfall
models
where
the
things
that
would
take
forever
to
come
out,
and
so
our
vehicle
teams
were
really
super
frustrated
with
you
know,
powertrains
there
all
the
time
and
then
Nevada
the
body
considered
module
or
restraints
module.
I
got
to
wait
three
months,
so
that
was
a
big
challenge
for
us.
C
So
what
are
our
key
challenges?
We
have
a
big
legacy
in
our
powertrain
systems
that
are
very
parochial.
I
run
my
own
data
center
right
now.
I
have
a
number
of
servers
that
are
really
kind
of
limited
and
therefore
I
can't
scale
this
for
all
those
other
teams
that
need
infrastructure
and
tools,
I'm,
not
that
old
data
center
I
just
got
a
message
overnight
that
it
was
over
temping,
so
I
need
to
get
out
of
that.
C
We
need
standard
methods
across
all
of
our
software
community
and
our
vehicles
are
starting
to
get
platforms
that
can
distribute
our
features
across
the
whole
network.
So
a
full
network
in
the
vehicle
is
our
future,
so
development
tool
stack
has
really
needed.
That
can
be
integrated
and
leveraged.
Let's
go,
and
this
is
where
I
step
in
all
right.
D
It
so
in
2014
I
was
working
in
our
research
and
advanced
infotainment
department,
and
at
that
time
our
mission
was
to
I,
develop
prototype
software
to
provide
new
infotainment
experiences
to
our
customers.
To
support
this
effort,
we
kind
of
needed
a
newer,
modern
tool
chain
that
would
support
like
a
fail,
fast
and
ship,
often
sort
of
mentality
and
having
an
open-source
background
from
college
I
gravitated
towards
the
tool
towards
the
toolset
in
that
space.
So
git
was
obviously
at
the
center
of
that
part
of
this.
D
This
newer
effort
was
to
be
partnering
with
universities
and
other
like
non-traditional
suppliers
in
the
automotive
space.
So
I
took
a
took
a
page,
took
a
note
from
my
buddy
Gotham.
He
teaches
at
Michigan
Tech,
University
and
part
of
his
coursework.
He
teaches
how
to
use
git
and
how
to
use
github
and
that's
how
that's
how
his
students
turn
in
homework.
D
So,
if
you're,
not
using,
if
you're,
not
version
controlling
your
stuff
and
get
and
you're
not
submitting
a
pull
request
in
github
you're,
not
getting
full
credit
on
your
assignment,
and
so
what
this
means
is
that
you
know
this
is
one
example
of
many
where
students
are
learning
these
newer,
modern
tools
in
University
and
Tom's.
Gonna
talk
a
little
bit
more
about
why
that's
important
in
a
little
later.
D
So
after
a
short
tools,
comparison,
we
landed
on
github
enterprise
and
our
purchasing
folks
negotiated
a
deal
where
we
could
get
a
small
price
break
if
we
bought
two
hundred
and
fifty
seats.
Instead
of
a
hundred
that
we
that
we
were
planning
for
and
so
with
some
and
with
some
help
from
some
friends
and
IT,
we
managed
to
get
github
enterprise
up
and
running
in
a
few
hours
once
all
that,
once
all
the
license
details
were
worked
out,
not.
D
So
it
took
about
a
month
for
word-of-mouth
to
spread
that
somebody
was
running
an
instance
at
github
in
the
in
the
company.
So
since
I
had
a
hundred
and
fifty
extra
seats,
I
kind
of
instituted,
a
policy,
you
know
if
you
want
to
come
and
kick
the
tires
on
this
thing.
You
know
I've
got
extra
seats,
but
you
got
to
understand
the
ground
rules
right
this
thing's
in
a
lab.
It's
not
backed
up
right
and
there's
no
uptime
guarantee
or
anything
like
that,
and
I
made
people
promise
like
hey.
D
D
So
what
I
didn't
realize,
as
I
was
handing
out
these
seats
to
people
that
I
didn't
know
inside
of
the
company,
the
people
that
I
was
handing
these
out
to
the
engineers
in
our
in
our
IT
department
that
were
architecting
our
next-generation
data
center.
So
this
is
by
this
time
we're
in,
like
2015
github
hadn't,
officially
launched.
Yet
it
was
already
a
critical
piece
of
infrastructure
in
the
company
and
I
think
you
know
one
of
the
coolest
examples
I
I've
learned
from
those
guys
that
were
that
were
architecting.
D
The
next-gen
data
center
is
how
they
manage
the
load
balancer.
So
it's
you
know
it's
a
modern
load
balancer.
It's
got
a
web
interface
and
everything
like
that
that
they
can
use
to
configure
it,
but
they
lock
that
interfacing
to
read-only.
If
you
want
to
make
a
change
on
the
load,
balancer
I,
you
have
to
open
up
a
pull
request
to
the
repo
and
that
kit
fires
off
a
web
hook
which
kicks
off
a
full
C
ICD
pipeline.
D
C
By
2017,
our
infotainment
teams
were
really
going
strong
delivery
more.
It
became
more
and
more
challenging,
so
we
really
needed
to
develop
something
new,
and
so
a
task
force
was
put
together,
Tim
and
I,
and
a
number
of
other
team
members
from
powertrain
infotainment
core
software
input
in
information
technology
really
wanted
to
create
a
ubiquitous
set
of
tools
that
we
can
really
consents
on
and
use
it
for
and
really
came
up
with.
C
This
mantra
tools
to
be
simple:
to
use
powerful,
easy,
easily
integrated
so
that
develop
teams
can
really
deliver
so
I
told
Tim,
I
said:
okay
I'm
in
clear
case,
I
I
can't
I
got
all
my
build
environment
embedded
in
this
customs
said:
oh
I
need
a
new
build
environment,
so
we
decided
that
my
team
is
I,
didn't
have
enough
team
members
or
people
I
couldn't
just
just
drop.
Stop
production
I
had
to
do
something,
so
we
discussed
it.
C
We
said
we
know
what
we
have
this
great
internship
programs
and
co-op
students
and
early
graduates
on
rotational
programs.
We
need
to
leverage
that
github
knowledge
that
they've
gotten
developed
in
college.
So
we
brought
those
team
members
on
board
and-
and
we
called
and
so
I'm
a
little
bit
of
a
history
buff.
So
automotive
history
is
one
of
my
favorite
things
and
one
of
the
big
parts
of
the
Ford
Motor
Company.
When
Henry
Ford
passed
away.
C
Henry
ford
ii
took
over,
he
was
28
years
old
when
he
took
over
Ford
Motor
Company,
and
so
he
didn't
know
how
to
run
the
company
very
well.
He
hired
a
whole
bunch
of
young
executives
and
that's
them
there
in
1946
and
they
were
called
the
whiz
kids.
So
I
I
immediately
started
calling
my
young
people
with
kids,
and
so
thanks
a
lot
to
those
guys
that
are
out
there
a
couple
of
one
of
them's
here
today
and
they
really
helped
us
to
accelerate
the
delivery
of
our
pipeline.
C
D
So
the
first
thing
that
we
did
was
we:
we
pulled
a
powertrain
application
out
of
clearcase
and
we
put
it
into
github
in
a
monolithic
repo
or
a
mono
repo.
We
knew
going
in
that
mono
repo
wasn't
going
to
be
up
to
the
task
of
handling
the
complexity
of
our
development
patterns,
but
our
goal
was
just
to
get
the
thing
to
build.
D
So
getting
getting
code
out
of
clearcase
and
into
github
is
a
non-trivial
task
and
it
challenged.
You
know
our
group
of
whiz
kids
to
write
some.
You
know
custom
scripts
and
some
automation
to
help
make
that
go
a
little
smoother,
so
they
spent
three
months
just
straight-up
hacking
the
make
files
to
unwind
all
the
hooks
that
clearcase
has
in
it
had
had
in
them.
From
the
you
know,
current
production
build
process
I'm
a
little
bit
different
kind
of
a
history.
Buff.
D
D
So
we
started
to
look
at
sub
modules,
but
we
decided
that
you
know
sub
modules,
kind
of
push
the
complexity
down
to
the
developer.
So
now
that's
one
extra
thing
for
them
to
manage
and
we
didn't
want
to
do.
That
is
that's
when
we
started
engaging
with
github
services.
So
in
early
2018
we
talked
to
get
up
services,
and
you
know
a
couple
months
later:
that's
when
that's
when
Jared
from
github
join
joined
our
team
to
help
us
out
with
who
you
know
with
our
DevOps
journey.
D
So
the
manifest
is
a
is
a
yeah
Mel
structure,
it's
readable
by
humans
machines
can
be
auto-generated.
It's
pretty
cool.
I
didn't
take
this
very
long
at
all
to
get
our
mono
repo
building
and
we
liked
it
because
we
could
you
know
cascade
this
this.
You
know
definition
file
to
various
layers
of
the
codebase,
so
it
was
extensible.
It
was
flexible.
It
could
operate
at
many
many
different
layers
of
the
architecture.
D
So
after
we
had
that
kind
of
figured
out,
we
took
a
look
at
the
components
themselves
or
the
smallest
unit
of
you
know
of
the
code
code
base
and
we're
looking
at
like
a
one-to-one
component
to
repo
sort
of
relationship.
So
we
have
about
250
of
those
in
our
in
our
current
context
and
the
build
process
for
those
is
more
or
less
the
same.
So
we
took,
we
took
advantage
of
Jenkins
class
libraries
to
abstract
a
lot
of
that
out
which
left
us
with
simple
one
to
two
line:
Jenkins
files.
D
We
really
like
this
pattern,
because,
if
we
needed
to
update
the
build
process
for
any
reason
we
all
we
have
to
do
is
submit
a
pull
request
to
one
repository
to
change.
The
the
class
library
and
all
250
components
automatically
consume
that
as
soon
as
the
PR
emerges,
so
the
next
part
was
how
to
spawn.
How
do
we?
How
do
we
kick
off
these
builds?
How
do
we
spawn
them
right
so,
given
again
that
all
these
250
components
are
more
or
less
the
same
from
a
build
perspective?
They
all
use
the
same
tools.
D
C
D
That's
right,
so
you
know
our
IT
department
runs
a
runs,
a
kubernetes
cluster
internally.
As
part
of
this,
you
know,
next-gen
data
center
effort
and
I
co
manage
what
we
call
the
software
development
ecosystem
product
and
our
mantra,
and
that
team
is
kind
of
like
no
matter
what
kind
of
software
you're
developing
there's
a
common
set
of
tools
that
that
you
kind
of
need
in
order
to
be
productive.
Two
of
those
tools,
in
our
opinion,
inside
of
our
company,
are
github
in
Jenkins,
and
so
as
part
of
our
Jenkins
work.
D
C
D
Needs
the
capacity
and,
what's
really
cool,
is
a
couple
of
weeks
ago
we
actually
ooh.
We
actually
took
this
whole
thing
and
dropped
it
into
Azure.
Just
to
see
if
we
could,
and
all
it
took
was
a
one-line
code
change
and
it
was
awesome.
So
if
you
think
about
it,
this
this
whole
concept
really
isn't
all
that
different
from
the
from
the
assembly
line.
D
D
When
we
started,
we
had
our
use
case
that
we
were
planning
for
was
a
hundred
devs
in
you
know
some
small
research
department
right
and
today
our
instance
is
pushing
about
10,000
users
out
of
our
own
datacenter
on
Prem
on
Prem
yep,
so
the
next,
the
next
thing
I
would
recommend
is
you
know.
Obviously,
github
is
the
largest
open
source
development
plan
form
on
the
planet.
You
know
we
learned
that
in
the
keynote
you.
D
Hit
a
hundred
million
developers
or
something
like
that
by
2025,
which
is
crazy
and
the
power
of
launching
something
like
it
of
enterprise.
Inside
of
your
network,
is
you
know
it
can
facilitate
an
inner
source
model
and
really
help
to
drive
a
culture
change
in
inside
of
your
company.
So
to
that
effect,
I
would
take
a
real
hard
look
about
organizations
and
how
what
what
kind
of
policies
you
want
to
put
in
place
for
allowing
people
to
create
organizations.
D
Then
you
gotta
have
a
good
reason.
All
right.
The
next
thing
I
would
recommend
is
rate
limits.
Yeah
take
a
since
github
enterprise
very
quickly.
You
know
kind
of
became
business
critical
for
us
a
you
know,
keeping
an
operational.
Is
you
know
it's
a
it's
a
big
deal
for
a
company
like
Ford.
If
you
know
Tom's
not
able
to
produce
software
and
brings
an
assembly
line
down
we're
talking,
you
know
what
million
dollars
dollars.
D
So
rate
limits
go
a
long
way
to
you
know,
helping
us
helping
us,
keep
our
users
in
check
and
like
ooh.
What
what
we've
been
finding
is
that
we've
got
a
lot
of
people
that
you
know
they
want
to
do
CI,
but
they
don't
necessarily
know
how
to
set
it
up.
They
don't
know
about
webhook
capability
and
stuff
like
that,
so
they
just
they
just
slam
the
API
hey
as
it
changed
as
it
changes
it
changed.
That's.
D
And,
and
so
you
know
putting
a
rate
limits
in
place
kind
of
you
know,
it
protects
your
infrastructure
from
from
going
down,
and
the
last
thing
that
we
do
is
you
know
we
we
pipe
all
of
our
logs
to
Splunk
and
we've
got
a
lot
of
custom
dashboards
inside
of
Splunk.
So
if
Tom
starts
banging
away
at
the
API
10,000
times
an
hour,
then
then
I'm
gonna
know
pretty
much
immediately
and
you
know
I'm
gonna
go
tap
them
on
the
shoulder.