►
Description
Developers love Git for its raw speed, powerful history traversal, distributed nature, and the fact that it was built by uber developer Linus Torvalds. What we don't love is the fact that, out of the box, Git has somewhat lacking support for tracking large binary files!
Fortunately, developers from GitHub, Atlassian, and Visual Studio Online have teamed up to work on an open source (MIT licensed) project to solve this problem: Git LFS (Large File Support).
In this session, Tim will cover the computer science behind Git LFS' internals & architecture, CLI usage and how to build an effective Git LFS workflow for a development team.
Tim has been an Atlassian developer for almost a decade spanning the JIRA, Bitbucket Server and Cloud, FishEye, and Crucible teams. He tweets (as @kannonboy), speaks and blogs about developer workflows, Java, JavaScript, Atlassian's developer tools and all manner of Git arcana.
About Git Merge:
Git Merge is the pre-eminent Git-focused conference: a full-day offering technical content, user case studies and training workshops for Git users of all levels.
Visit: http://git-merge.com/
A
Alright,
so
like
many
great
stories,
our
story
tonight
begins
in
Paris.
It
was
the
night
of
April
8
2015
the
day
before
the
annual
git
merge
conference.
So
just
like
this,
and
a
couple
of
atlans
we're
discussing
their
projects
that
they're
working
on
atlassian
with
a
couple
of
github
is
at
the
get
contributors
summit
that
happens
annually
before
they
get
merged
conference.
A
Now
the
conversation
got
kind
of
interesting
when
Nicola
pallucci
from
atlassian
said:
hey
we've
been
working
on
this
really
neat
solution
to
solve
the
problem
that
get
has
with
tracking
large
binary
content,
so
build
this
new
little
tool
in
go
that
operates
transparently
with
get
and
we're
planning
on
announcing
it
at
our
track
session
tomorrow
at
get
merge
and
rick
olson.
An
engineer
from
github
said:
hey,
that's
that's
kind
of
interesting
because
it
github
we've
also
been
working
on
a
solution
to
the
problem
that
get
has
with
tracking
large
content.
A
We've
also
built
a
new
little
tool
that
works
in
Ingo
and
we're
going
to
announce
it
at
our
track
session
at
get
merged
directly
after
yours.
So
we
were
like
what
that's
a
picture
of
James
Watt
by
the
way
and
Rick
was
like
totes.
Isn't
that
crazy?
What
do
you
call
it?
And
here's
where
it
gets
completely
insane,
because
Nick
said
what
we
call
our
project
yet
love
in
terms
of
get
long
objects
and
Rick
said:
we've
also
picked
a
three-letter
acronym
that
starts
with
L,
get
lfs
or
large
file
storage.
A
So,
after
a
bit
of
back
and
forth
between
the
two
companies,
atlassian
decided
to
archive
get
lob
and
we've
been
busy
porting
over
some
of
the
features
that
we
introduced
to
that
into
the
git
LFS
project.
So
that's
why
me
and
atlassian,
working
on
bitbucket
and
speaking
at
a
github
run
conference
about
get
LFS,
because
it's
a
combined
project
between
both
of
our
companies
to
solve
once
and
for
all
the
problem
that
get
has
with
tracking
Lodge
binary
content.
A
Now
this
is
really
exciting
because
it
opens
up
the
world
of
get
to
a
whole
bunch
of
software
development
teams
that
traditionally
couldn't
do
it
because
they
had
to
track
large
content.
So
these
are
teams
who
have
heavy
design
aesthetics
and
want
to
track
their
Photoshop
files
and
images
inside
they
get
repositories.
A
Gaming
teams
who
need
to
track
textures
and
audio
files
and
full
motion.
Video
can
now
start
to
use,
get
as
well
and
researchers
who
have
to
track
massive
datasets
in
both
textual
and
binary
formats
can
use
git
LFS
to
track
this
stuff.
Alongside
the
code
that
they're
writing
for
their
experiments
now,
today,
there's
four
different
areas
that
I
want
to
cover.
First
I
want
to
explain
quickly
why
it
has
such
a
hard
time
tracking
large
content.
Unfortunately,
Charles's
talk
covered
a
bit
of
that
material
already,
so
I
can
breeze
through
that
section.
A
Now,
as
John
mentioned
earlier
in
his
talk,
it
gets
call
you
have
this
directed
acyclic
graph
of
commits
and
other
objects
that
contain
all
of
your
repositories.
History
now
to
understand
where
the
problem
with
binary
content
comes,
you
need
to
look
a
little
bit
closer
at
these
commit
objects.
You
can
use
the
get
cat
file
command
with
the
dash
p4
pretty
flag
to
inspect
the
structure
of
one
of
these
commits.
When
you
look
at
it,
it's
pretty
simple
as
the
commit
ID
of
the
parent,
it's
a
directed
acyclic
graph.
A
After
all,
there
might
be
two
parents
there
or
more.
If
it's
a
merge
commit,
you
have
a
tree
object
which
we're
going
to
look
at
a
little
bit
Club
more
closely
in
a
second,
and
you
have
some
commits
metadata,
including
who
created
the
commits
who
authored
the
code
originally
and
then
a
commit
message.
Now
the
tree
object
is
analogous
to
the
root
directory
of
your
repository.
You
can
again
use
get
cat
file
p
to
inspect
it,
and
you
can
see
something
that
looks
a
little
bit
like
a
directory
on
your
file
system.
A
You
have
some
file
modes.
Are
the
types
of
the
objects
that
are
sitting
inside
this
directory,
which
are
either
blobs
the
actual
content
of
your
files
or
trees,
which
are
de
nested
directories
inside
your
repository.
Then
you
have
sha-1
hashes
of
all
of
these
because
get
is
a
content
addressable
file
store.
After
all,
so
these
are
just
the
well.
The
address
is
inside
you'll
get
objects
directory,
as
well
as
the
sha-1
hashes
of
those
objects.
Contents.
A
Now,
if
you
explode
this
out,
you
can
recur
stone
through
all
of
the
sub
directories
or
sub
trees
of
your
repository
and
see
all
of
these
blobs.
Now
the
problem
comes
when
you
start
looking
at
how
these
blobs
are
stored.
When
you
first
create
a
commit
all
of
these
blobs
and
trees
are
written
into
your
get
objects
directory
as
loose
objects.
A
The
reason
why
I
get
and
other
distributed
version
control
systems
have
such
a
hard
time
is
that
you
typically
end
up
copying
your
entire
repositories.
History
around.
So
when
I
push
my
new
elephant
images
up
into
my
get
hosting
server
and
the
rest
of
my
team
clones
and
they
have
to
retrieve
every
single
version
of
that
file,
even
if
they're
not
necessarily
interested
in
all
of
those
historical
changes,
as
will
my
continuous
integration
builds
and
my
continuous
deployment
jobs
so
get
LFS
is
a
really
exciting
project
that
solves
this
problem
in
a
really
elegant
way.
A
Now,
the
kind
of
high
level
picture
of
how
it
does
this
is,
instead
of
storing
your
binary
content
directly
inside
your
dag
as
blobs.
Instead,
you
store
these
lightweight
pointer
files
inside
your
repository
and
they
refer
to
these
large
objects
that
are
actually
sitting
in
a
separate
specialized
storage.
A
Typically,
this
is
the
tips
of
the
tip
of
the
branch
that
you've
just
checked
out.
You
can
configure
lfs
to
do
all
sorts
of
interesting
things
around
resolving
this
content
for
you,
which
we'll
look
at
in
just
a
second
now.
If
we
whip
out
a
kitkat
file,
HP
command
again,
we
can
inspect
the
structure
of
one
of
these
pointer
files,
and
it
really
is
just
a
few
bites.
You
have
you've
got
a
version,
schema
property.
You
have
an
object
ID,
which
is
a
shire
256
hash
of
the
objects
contents.
A
These
large
objects
are
also
stored
in
a
really
content-addressable
way,
which
is
nice,
and
then
you
have
the
size
of
the
object
to
so
getting
up
and
running
with
goodell
FS
is
very
simple.
You
can
install
it
using
your
favorite
package
manager.
There
are
binary
is
available,
I
believe
for
it's
written
in,
go
so
for
every
platform
that
get
is
available
on
and
once
you've
got
the
actual
binary
sitting
locally
on
your
path.
A
When
I'm
checking
out
or
adding
my
files
to
the
the
index,
the
diff
and
merge
filter
just
say:
hey
compute,
diffs
and
merges
using
the
lfs
filter
as
well,
and
dash
text
just
means.
Don't
try
to
create
textual
diffs
of
content
matching
this
pattern
because
that
typically
doesn't
work
out
very
well
for
binary
files.
A
A
In
the
get
/
lfs
/
objects
directory
instead
of
the
get
objects
directory,
then
it
writes
out
that
point
to
file
into
the
index
instead
of
writing
out
all
of
that
binary
content
and
then
that
the
object
of
that
pointer
file
gets
copied
into
our
get
objects
directory
instead,
so
good
objects
directory
only
increases
by
a
few
blue.
A
few
bites,
instead
of
the
traditional
50
megabytes
when
I
make
a
change
to
one
of
these
large
files,
but
gets
smudge
filter
operates
in
Reverse.
A
So
when
I
as
a
developer,
run
the
get
check
out
command
for
a
file
or
commit
our
branch,
then
it'll
read
that
point
of
file
back
out
of
my
get
objects,
directory
post
that
to
beget
lfs
smudge
filter,
and
then
it
will
relook
at
my
get.
Lfs
objects:
cash
for
the
file
matching
the
sha-256
hash
sitting
in
that
point
of
file.
Now,
if
it
can't
find
it
in
my
local
objects
directory.
A
This
is
the
point
where
it
actually
reads
through
to
the
backing
lfs
store,
which
could
be
sitting
on
github
or
s3
or
bukkit
server,
and
then
it'll
download
it
from
there.
It
actually
does
this
transparently
using
this
really
cool
earth
protocol.
That
I'll
talk
a
little
bit
more
about
in
a
second
anyway,
once
it
finds
that
object
right
sit
out
back
into
the
wac
tree
now.
This
is
really
nice
because
you'll
say
all
as
a
developer.
A
All
I've
been
doing
is
running,
get
ad
and
get
check
out
and
working
with
this
large
content
sitting
in
my
work
tree
I,
don't
have
to
know
anything
about
lfs
once
I've
got
it
installed.
I
can
just
use
my
regular
get
workflow
as
normal
and
get
lfs
handles
all
of
this
under
the
covers
for
me.
So
that's
how
it
works
without
working
with
your
local
repository
now,
I
want
to
talk
a
little
bit
about
how
you
transfer
those
large
objects
back
and
forth
between
your
lfs
store.
A
The
way
that
you
were
that
works
with
a
push
there's
a
special
pre
push
hook
that
gets
installed
either
as
part
of
that
get
lfs
install
command.
So
the
first
time
you
decide
to
start
tracking
repository
content
in
lfs.
You'll
need
to
actually
run
get
lfs
install
inside
your
repository
directory,
and
it
will
install
that
hook
for
you,
but
it
also
gets
installed
by
the
git
LFS
smudge
filter
automatically,
which,
when
I
first
came
across
it
I
was
like
man.
A
That
sounds
like
a
little
bit
of
a
hack,
but
actually
it's
really
neat,
because
if
I
clone
a
repository
like
if
I've
get
lfs
installed
and
then
I
go
off
in
clone
a
repository
that
has
some
lfs
content
in
it.
When
that
the
smudge
filter
gets
passed,
one
of
those
pointer
files,
it's
going
to
go
off
and
install
the
pre
push
hook
automatically
in
that
repo.
A
Similarly,
when
you
run
a
get
pull
you're
going
to
see
some
output
saying
that
it's
downloading
these
large
files-
and
this
is
basically
the
output
from
that
smudge
filter
command-
that's
fetching
through
to
the
remote
repository
back
to
populate
our
local
lfs
cash.
Now,
what's
really
cool
about
goodell
FS.
A
It
also
supports
the
case
where
you
have
a
completely
separate
external
lfs
store
that
potentially
the
host
application
can't
even
talk
to
so
maybe
completely
firewalled
off,
and
it
uses
the
client
to
broker
a
connection
now
the
way
that
it
does.
This
is
through
a
really
clever
use
of
hypermedia.
So
let's
take
the
simple
case
where
you
have
a
HTTP
repository
and
we're
doing
a
clone
or
a
pull,
and
we
need
to
resolve
some
of
those
large
objects.
What
the
smudge
filter
does
is.
It
adds
a
special
path
onto
your
HTTP
repository.
A
You
are
a
and
it
posts
adjacent
structure,
saying
hey:
these
are
the
objects
that
I
need
to
download
in
order
to
replace
this
pointer
on
disk,
then
the
server
well,
the
lfsr,
where
server
responds
with
a
very
similar
Jason
structure,
except
it
has
this
actions
hash
with
a
download
action
saying
where
that
large
object
can
be
found.
Now
this
URL
can
be
relative,
actually
I
think
it
has
to
be
absolute,
but
it
can
be
on
the
server
where
it's
being
served
from
or
it
could
be
sitting
somewhere
else
completely
separately.
A
A
You
can
well
that
you're,
like
the
bucket
service
case,
it
just
passes
down
a
JWT
token
and
says:
hey
pass
this
back
and
I'll
give
you
the
large
file,
but
you
know
that
could
potentially
be
basic
auth
for
s3
or
somewhat
oro
or
thought
or
really
whatever
you
want
to
use.
When
you
write
objects
to
the
server,
you
have
kind
of
a
similar
flow,
but
instead
of
a
download
action,
you
have
an
upload
action
and
you
also
have
this
special
verify
action
which
the
server
can
optionally
specify.
A
Now
the
verify
action
is
a
URL
that
the
client
must
call
once
it's
finished,
uploading
the
object.
So
this
is
where
we
get
that
flexibility
for
our
lfs
storage
to
sit
in
a
completely
separate
medium,
that's
divorced
from
our
get
host
because
it
we
can
rely
on
the
client
to
tell
the
host
once
it's
successfully
stored,
that
large
object
for
us.
So
that's
how
it
works
with
HTTP
with
ssh.
It's
a
really
similar
process,
but
a
get
lfs
aware.
A
Server
that
supports
the
SSH
command
that
supports
lfs
transfer
via
ssh
also
has
to
know
about
this
special
get
lfs
authenticate
command,
so
they
get
lfs
client
when
it's
attempting
to
do
a
push
or
a
pull
by
the
smudge
filter
will
post
this
special
JSON
structure
over
ssh
to
the
lfs.
It
will
then
take
a
command
and
then
it
will
respond
with
with
the
URL
for
your
the
HTTP
lfs
api.
A
So,
even
though
you
don't
know
where
the
repository
is
sitting
in
HTTP
land,
the
server
will
tell
you
for
a
tell,
tell
you
over
ssh
for
you
and
provide
again
credentials
for
authenticating
with
it.
So
it's
a
really
really
clever
use
of
hypermedia
I.
Think
with
lfs.
Now
the
sharp-eyed
among
you
may
have
noticed
that
the
API
for
lfs
is
a
batch
API.
However,
because
this
API
is
being
called
by
our
smudge
filter
every
time
it
comes
across
one
of
those
lfs
pointers.
A
Now
this
is
kind
of
inefficient,
because
it
means
you've
got
this
huge
overhead
of
calling
back
and
forth
with
your
get
host
every
time
you
need
to
download
an,
however
in
git
LFS
12,
which
is
going
to
be
released
any
day
now.
There's
a
new
command
called
git
LFS
clone
that
you
can
call
now
what
this
does.
Is
it
actually
disables
the
smudge
filter
for
you
and
then
batches
up
a
set
of
requests
to
that
object,
a
set
of
objects
that
it
needs
to
resolve
and
then
calls
that
batch
API
with
a
single
Jason
structure.
A
A
History.
Now
this
is
possibly
one
of
those
joke,
slides
that
Charles
mentioned,
but
as
a
as
someone
who's,
probably
familiar
with
get,
you
might
immediately
reach
for
the
filter
branch
command
to
go
back
and
rewrite
this
history
to
replace
it
with
those
pointers
you
might
start
out
by
just
attempting
to
delete
it,
which
obviously
isn't
great,
because
then
you
lose
that
history.
So
don't
do
this
now!
A
Now
it's
interesting
because
the
bfg
repo
cleaner
was
originally
built
to
kill
history,
you
can
use
it
to
delete
files
historically
throughout
your
repository
or
directories,
or
even
use
it
to
replace
individual
tokens
inside
files.
So
if
you
committed
a
password
or
other
sensitive
credentials
to
a
properties
file,
you
can
actually
use
the
Ripper
cleaner
to
go
back
and
replace
that,
and
it
is
significantly
faster
than
filter
branch
as
well.
A
If
you
look
at
the
repo
cleaners
homepage,
there's
a
set
of
projects
that
it's
been
run
over
with
some
example:
filter
branch
commands
that
do
the
equivalent
operations
and
it's
ten
to
seven
hundred
and
twenty
times
faster
and
as
of
I,
think
version
1,
dot,
12
dot.
Five
get
lfs
now
supports
the
the
what's
it
called
convert
to
get
lfs
command.
A
So
instead
of
this
gigantic
filter
branch
command,
you
can
just
say
the
FG
convert
to
get
all
FS
and
pass
it
the
patents
which
you
want
to
go
back
and
historically
replace
with
lfs
pointer
files.
Instead,
now
with
the
bfg,
you
will
need
to
pass
this
additional
flag,
the
no
blood
protection
flag.
All
this
does
is
it
tells
the
bfg
to
replace
the
tip
commit
of
your
repository
as
well.
By
default,
the
bfg
doesn't
modify
your
current
commit
for
safety
reasons,
but
this
is
totally
safe
to
override.
A
So
that's
how
you
can
convert
your
existing
repositories
over
to
get
lfs
now.
Would
it
give
you
some
tips
for
using
lfs
in
a
team
context?
Now
one
of
them
are
the
first
things
you
need
to
know
is
that
lfs
still
doesn't
have
any
concept
of
file
locking,
and
this
can
cause
problems,
particularly
when
you're
working
with
other
teammates.
You
might
want
to
modify
the
same
files
that
you're
working
on
now.
There
is
no
way
forget
to
automatically
reserve
resolve
binary
changes,
changes
to
binary
files
on
different
branches.
A
So
there's
you
can
get
into
these
really
depressing
situations
where
you'll
spend
all
day
working
on
one
large
binary
file.
Only
to
discover
that
someone
else
on
your
team
has
also
been
doing
the
same
thing
and
you
end
up
with
something
that
has
to
be
manually
resolved
or
potentially
you
have
to
throw
away
some
someone's
changes.
So
the
best
thing
you
can
do
at
the
moment
is
to
just
basically
tell
the
rest
of
your
team
that
you're
planning
on
making
extensive
changes
at
your
daily
stand-up
or
ping
them
on
whatever
you
use
for
instant
messaging.
A
Now
there
is
a
purpose
of
some
brief
proposal
notes
out
by
Steve
streeting
what,
if
the
contributors
to
get
other
fest
on
how
we
might
handle
file
looking
one
day,
but
there's
nothing.
Yet
that's
actually
sort
of
hit
the
code
phase
of
the
project
just
just
yet
so
I've
mainly
been
talking
about
how
you
can
individually
use
get
lfs
to
track
files,
but
sometimes
you
might
want
to
pull
down
large
files
that
other
members
of
your
team
are
working
on.
A
So
there's
a
command
called
git
LFS
fetch
that
would
you,
when
you
run
it
will
populate
the
replace
the
token
sorry,
the
file
pointers
that
you
have
locally
with
the
large,
with
a
corresponding
large
file
content.
But
you
can
pass
the
dash
dash
recent
flag
to
lfs
fetch
and
it
will
resolve
content
for
other
branches.
Are
that
you
have
locally
checked
out
dash
dash
reason
by
default
will
retrieve
content
for
any
branch
that
has
a
commit
on
it.
A
That's
newer
than
seven
days
old,
and
if
you
like
this
behavior,
you
can
actually
make
it
permanent
by
setting
the
lfs
fetch
recent
property,
you
can
also
configure
it
at
a
slightly
finer
grained
level
as
well,
so
you
can
change
that
sliding
window
of
recency
Arfa
commits
by
setting
the
factories
and
refs
days
comprar
purty.
You
can
also
configure
it
to
retrieve
LFS
content
for
commits
that
aren't
necessarily
the
tip
of
a
branch
by
using
the
fetch
recent
commits
days
property.
A
Now
this
is
kind
of
unusual
because
too-
and
that's
that's,
why
it's
set
to
zero
by
default,
but
I
could
conceivably
see
if
you're
planning
on
doing
some
rebasing
or
maybe
some
cherry
picking
from
commits
that
aren't
necessarily
a
branch
tip
and
you
weren't
gonna
have
some
internet
connectivity.
Then
you
might
use
this
to
pull
down
other
changes.
A
A
But
if
you
work
in
a
large
esteem
or
at
a
busy
road,
then
this
means
you're
going
to
be
pulling
down
lots
and
lots
of
data
when
you,
when
you
run
a
get
fetch,
the
git
LFS
prune
command
can
be
used
for
cleaning
up
this
local
data
or
data
from
your
LF
at
your
local
lfs
cash,
and
it
follows
fairly
similar
rules
with
respect
to
recency.
However,
you
can
change,
you
can
also
increase
the
period
which
get
lfs
prune,
considers
recent
by
setting
the
prune
offset
days,
which
is
by
default
3.
A
You
haven't
yet
pushed,
and
you
can
also
set
the
prune
verify
remote,
always
flag,
which
means
it'll
always
confirm
that
your
local,
that
the
local
object
that
it's
about
to
prune
has
actually
been
copied
to
the
server
and
still
exists
on
the
server.
So
it
actually
contacts
the
lfs
storage.
Api
I
usually
like
to
set
that
by
default,
because
I'm
kind
of
a
paranoid
person,
but
that
something
you
have
to
explicitly
do
so.
That's
how
you
can
fetch
more
content
or
more
large
content,
then
potentially
you're
going
to
need
just
to
fulfill
your
branch.
A
Get
your
branch
tips.
We
can
actually
configure
get
lfs
to
know
to
retrieve
less
content
as
well,
so
there's
a
there's,
a
flag
that
you
can
pass
to
get
lfs
fetch
called
exclude,
and
you
can
use
that
to
pare
down
which,
which
toke
it
was
sorry
which
are
which
file
point.
Is
it's
actually
going
to
replace
with
your
large
content?
A
In
that
case,
you
can
set,
you
can
block
all
of
that
assets
directory
by
specifying
a
pattern
using
exclude
there's
another
case
where
you're
you
might
be
a
specialist
on
your
team
instead,
and
so
you
might
only
want
to
work
with
a
subset
of
those
large
files.
In
that
case,
you
can
use
the
dash
dash
blued
flag
instead.
So,
for
example,
in
audio
engineer
might
want
to
pull
down
there,
wav
files
and
OGG
files
to
work
on
those.
A
Instead
of
grabbing
all
of
those
textures
and
full
motion,
video
and
again,
if
you
like
these,
you
can
make
them
permanent
defaults
by
setting
them
setting
them
in
your
global
or
repository.
Get
configuration
with
lfs
fetch
exclude
and
fetch
include.
So
that's
all
I've
got
today
in
terms
of
get
lfs.
A
If
you're
interested
in
learning
a
little
bit
more
about
the
project,
you
can
check
out
the
ducks
at
gale
lfs
github
com,
and
you
can
have
a
look
at
the
source
code
as
well
in
the
github
organization
under
get
lfs
and
it's
available
in
github
enterprise
and
bitbucket
server,
as
of
version
4
3.
Thank
you.
So
much.
A
A
A
Yeah
I
mean
you
can
push
to
alternate
remotes
as
well,
if
you,
like,
typically
I,
mean
depending
on
where
hosting
it
like.
There
is
a
single
point
of
failure,
but
at
the
end
of
the
day
I
mean
s3
will
have
backups
and
things
you
probably
not
going
to
lose
that
data.
But
you
are,
you
are
relying
on
your
upstream
hosts
to
have
sensible
disaster
recovery
and
failover
and
stuff
for
it
for
your
host
yeah.
A
Not
that
I'm,
aware
of
yet,
although
it
is
still
fairly
early
days
for
the
project,
so
I'm
sure
we'll
see
more
advanced
features
like
that
coming
out,
I,
definitely
yeah!
You
could
have
a
look
at
the
get
lfs
issue
tracker
potentially
and
see
if
there's
something
there,
because
I
definitely
very
open
to
taking
feedback
and
acting
on
that
noise.