►
From YouTube: How Gitaly fits into GitLab: Episode 2 – Git SSH
Description
A 1-hour training video for contributors new to GitLab and Gitaly.
What is in a gitaly-proto Repository message, legacy vs
hashed storage (repository directories), git clone via SSH,
gitlab-shell, authorized_keys and forced commands, what happens
during git push.
Recorded 2019-02-28
A
Cool
we're
recording,
so
the
topics
for
today
are
some
follow-up
from
last
time
on
how
occasionally
integrates
with
good
lepsy
and
then,
after
that,
we
want
to
look
at
how
get
pushed
and
get
full
ssh
works
and
hi
I'm
Jacob.
Maybe
you
can
see
in
the
recording
already
like
I'm
staff
back
in
engineering,
the
quickly
team
and
with
me
our
poll
and
Francisco
and
John
wolves
all
are
also
back-end
engineers
that
good
laugh
and
so
that
that's
the
audio
that's
was
in
the
call.
So
what
did
I
want
to
say
about?
A
A
A
A
A
This
is
one
area
and
then
there's
this
and
we
can
forget
about
reserved
stuff.
So
what
do
we
have
here?
Storage
name.
This
is
a
string
field
and
the
typical
example
would
be
the
string
defaults.
So
these
are
the
same
names
that
are
in
good
left's
config
file,
where
we
have
a
mapping
between
storage
names
and
grittily
addresses.
These
names
are
in
the
database.
Every.
B
A
Records
in
in
the
sequel
database,
the
gate
lepsy
sites
has
an
Associated
repository
storage
fields.
So
that's
what
would
go
into
this
field
into
the
storage
name
and
in
the
Gately
config
we
define
which
storages
this
kiddley
server
is
serving
so
yeah.
That's
this
field
and
then
there's
the
second
field,
relative
path,
which
sounds
a
bit
funny,
but
we
used
to
have
a
full
path
there
with
absolute
path,
which
was
in
retrospect,
makes
no
sense.
So
a
relative
path
really
means
is
so
in
your
Gridley
config.
Let
me
pull
up
forget
in
the
config
file.
A
Here's
the
thick
file-
you
will
have
something
like
this,
so
this
is
the
definition
of
a
storage
say.
The
default
storage
storage
name
default
will
be
served
by
this
Gately
server
and
will
be
at
this
path.
Then
every
request
that
comes
in
says
well:
I
want
to
every
request,
needs
to
reference.
One
of
these
things
so
I
would
say:
ok,
I,
want.
C
A
A
This
this
was
sort
of
our
starting
points.
Did
this?
This
is
still
how
it
works.
The
one
thing
I
wanted
to
touch
on
is
the
funny
fact
that
here
we
have
a
name
which
is
human
readable.
This
is
the
old
situation
and
we're
changing
to
a
new
situation
called
hash
storage.
This.
This
idea
of
passing
this
name
here
is
very
problematic,
because
this
name
can
be
changed
by
the
user.
A
The
user
can
just
come
in
and
rename
their
repository,
and
when
that
happens,
we
actually
have
to
make
a
grizzly
call
and
rename
the
directory
on
disk,
and
this
sort
of
thing
can
go
wrong,
so
it
can
happen
that
a
user
updated
that
is
sequel,
that
the
project
name
was
updated
to
another
path
and
then
the
rename
on
the
Grizzly
server
field,
and
then
your
repository
can
no
longer
be
found.
It's
just
gone
so
that
is
very
messy
and
problem
it.
A
A
So
now
we
have
some
hex,
that's
too
much,
let's
delete
that.
So
it
might
look
something
like
this,
and
these
two
things
are
mixed,
and
this
is
when
you're
working
on
getting.
It
is
good
to
be
aware
that
we're
in
a
transitional
stage
where
you
can
encounter
both
these
things
and
for
the
most
part
gisli,
is
completely
oblivious.
What
this
these
things
look
like.
A
D
A
No
I
it
looks
like
this
is
that
you
file
system
limitations
the
way.
So
if
you
have
a
lot
of
repositories
and
they're
all
under
this
thing,
if
you're
looking
you're
on
x2,
then
there's
a
limit
at
other
non
30,000
or
60,000
entries,
and
if
you
have
more
than
that,
then
you
get
a
hard
error
and
you
cannot
create
new
repositories.
So
you
you
want
to
have
some
sort
of
nesting
to
avoid
having
very
wide
directory
trees.
So
because
of
that
I
think.
A
A
A
A
A
C
A
A
And
to
copy
the
clone,
URL
I'm,
going
to
/tmp
and
we'll
get
clone
of
this
thing
and
what
happens?
Connection
refused.
I
know
what
that
is.
That
is
because
there
is
no
sh
t
on
port
two
two
two
two
and
this
is
one
of
the
GDK
defaults.
This
is
one
of
the
reasons
why
I
wanted
you
to
see
this
fail.
So
you
know
about
it.
What
you
need
to
do,
then,
is
stop
GDK
or
clear,
was
running
a
GDK
run
app
and
edit.
A
A
A
A
As
his
age
and
it
uses
this
config
file,
so
let's
look
at
that
contact
file
and
okay.
So
when
I
make
a
connection
on
the
server
side
is
using
this
authorized
keys
file
to
recognize
my
SSH
key
I.
So
this
is
the
basic
way
to
do.
Ssh
with
keys.
Is
that
on
the
server
you
have,
you
want
to
connect
your
user
account
and
then
in
the
user
home
directory,
you
have
an
authorized
keys
file
that
lists
the
keys
that
are
allowed
to
connect.
A
A
A
Let's
see
this
part,
is
just
my
public
key.
Let's
take
that
away
for
a
moment,
and
this
part
is
an
optional
thing.
You
can
do
in
authorized
keys
file,
which
is
to
set
extra
config
options
that
can
apply
to
any
session
that
connects
with
this
key,
and
the
most
important
config
option
here
is
that
the
commands
from
the
client
side
is
ignored
and
we
have
forced
commands
to
think
it's
literally
called
force
command
if
you
look
at
the
SSH
documentation.
A
A
A
This
is
what's
the
gift
client
is
trying
to
do,
and
this
is
a
remote
command.
It's
trying
to
start,
but
we
are
not
letting
it
because
we
were
overriding
the
remote
command
on
the
server
side
and
it's
going
to
launch
this
thing.
Instead
and
frankly,
all
these
options
are
just
there
for
security
to
say
you
can't
do
those
things,
so
we
can
ignore
those
for
moments
now
this
command
is
taking
us
into
Caleb.
Shell
and
the
other
important
thing
this
line
is
doing.
A
A
Yeah
so-
and
this
is
one
of
the
the
big
jobs
that
github
shell
has
in
the
big
picture-
is
that
it
needs
to
make
sure
that
this
association
happens
correctly
because
as
far
as
SSH
is
connected,
there's
concerns,
there's
just
one
user
and
on
a
production
server.
That
user
will
be
called
git,
but
we're
actually
multiplexing
that
one
user
into
millions
of
individual
users
and
and
that
mapping
is
one
of
the
core
jobs
of
of
collapse.
Shell.
A
So
this
takes
us
into
this
executable
and
I
think
we
can
just
click
that
executable
I
mean
it
is
a
binary,
we're
okay,
it
used
to
be
a
ruby
executable,
but
we're
trying
to
convert
it
to
go
gradually
and
now
there's
a
binary
shim.
So
we
can
look
at
it
directly,
but
what
I
can
talk
you
through?
What
this
does?
A
A
Yeah
this
one
is
executable,
but
this
one
is
the
scripts,
so
that's
the
one
we
want
I
happen.
Okay,
we
should
know
I'm
going
to
ignore
the
we're
gonna
take
Nord
excusable
for
now,
because
it
in
the
future.
All
that
work
will
happen
in
this.
This
goal
binary
but
I,
know
that
right
now
it
doesn't
do
much,
and
this
is
the
old
executable.
A
A
A
B
A
That's
okay,
let
me
demonstrate
with
force
command
us
a
bit
more
so
for
that
I'm
going
to
mangle
my
this
file
here,
even
though
it's
managed
by
Caleb
she'll
want
to
ignore
that
create
a
new
entry
and,
let's
start
with
just
a
plain
entry
where
there
is
nothing.
But
this
is
just
my
key
and
now,
if
I
would
connect
to
that
thing,
that's
to
to
to
to.
A
A
Now
I
have
a
shell
session.
This
would
be
very
bad
if
this
happens
on
your
clip
server,
because
then
you
have
shell
access
if
the
application
user
and
yeah
game
over.
So
this
is
never
what
you
want.
So
what
what
can
you
do
with
force
command?
So
I
can
just
say:
command
equals
echo
hello
worlds
and
then,
if
I
try
to
connect,
it
says
hello,
world
and
end
of
story.
A
A
If
you
just
run
it
this,
then
you
get
a
shell
and
you
can
do
stuff
like
print
working
directory.
The
other
way
to
run
SSH
is
to
give
it
a
remote
commands
if
I
give
another
our
command
to
say,
run
this
command.
For
me,
then,
on
the
other
end
on
the
server,
it
will
just
run
that
command
connect
and
like
Senate
input.
The
standard
output
gets
connected
across
the
network.
Command
runs
command.
Exits
session
is
over
if
I
run
this
I
get
this
so
for
security.
A
This
is
as
bad
right
if
I
kind
of
in
a
single
command
or
whether
I
get
a
full
shell
session,
and
if
you
get
this
you're
in
so
this
is
more
like
what
kid
is
doing,
except
kids,
isn't
around
PWD
and
runs
a
good
specific.
It
tries
to
ruin
a
kid
specific
server-side
command
that
generates
a
pack
file
with
all
the
data
it's
trying
to
clone.
A
So
now,
if
I
try
to
do
PWD
like
this,
it
was
show
here
that
that
was
the
original
command
that
got
preserved,
but
instead
it
run
the
other
commands
around.
The
second
part
echo
hello
world,
but
this
is
the
basic
principle.
A
good
client
comes
in
and
says
specifically:
I
want
to
run,
get
upload
pack
on
some
repository
and
that
repository
will
have
a
user
friendly
name.
A
Look
at
the
clone
command.
I
was
trying
to
run.
It
tries
to
run
something
like
this,
so
the
kids
client
thinks
that's
on
the
gates.
Gate
lab
server.
There
is
a
directory
called
kid
lab
work,
/
kid
lab,
tested
gate
and
right
there
when
you
login
with
SSH-
and
that
is
a
repository-
that's
what
the
client
thinks.
A
That's
not
true
right,
it's
not
like.
We
have
a
home
directory
up
there
and
all
the
repositories
are
boom
in
that
home
directory.
At
one
point
in
kid
loves
history.
It
was
almost
like
that,
but
it
was
never
100%
like
that.
So
there
is
this
translation
going
on
between
what
get
thinks
it's
running
on
the
server
and
that
comes
in
as
this
SSH
original
commands
and
then
it
gets
translated
into
well.
We
know
where
the
repository
actually
is,
and
then
we
run
get
up.
Look
back
in
that
directory.
A
That's
the
basic
idea
and
that's
how
things
used
to
work
before
giddily
and
then
with
the
introduction
of
Italy,
became
a
little
more
complex
because
the
server
where
sshd
picks
up
the
phone
is
not
where
the
gate
repositories
are
so
instead
of
saying.
Okay,
I'll
just
run
get
the
plug
back
in
this
directory.
It
needs
to
make
establish
acutely
session
and
connect
to
the
server
where
the
repository
actually
is,
and
then
a
proxy
data
back
and
forth
to
there.
A
So
that
is
the
yeah,
that
is
the
force
command
mechanism.
Oh,
it
is
very
important
that
this
file
is
in
order.
Something
I
should
say
now
is
that
the
idea
of
having
this
one
authorized
keys
file
is
something
that
didn't
scale
like
in
the
early
days
of
kids
lap.
You
would
have
an
authorized
keys
file
and
it
would
be
very
long.
It
would
be
as
long
it
would
have
one
line
per
key
known
to
get
lap
and
every
time
somebody
adds
a
key.
A
Some
people,
I
mean
I.
I
was
not
feeling
this
much
because
I
had
relatively
early
accounts,
so
my
key
was
relatively
high
in
the
authorized
keys
file,
so
I
could
I
would
have
a
session
much
quicker
than
somebody
who
signed
up
a
year
later,
so
this
whole
thing
didn't
scale
and
locally
around
the
time.
This
wasn't
scaling.
The
people
had
a
patch
to
sshd
that
allowed
you
to,
instead
of
having
one
big
authorized
keys
file
to
run
a
commands
to
spawn
a
commands
where
you
say:
here's
the
pop
key.
Is
this
ok
or
not?
A
A
Okay,
looking
at
these
coders
and
tell
you
much,
but
what
this
does
is
it
gets
the
arguments
from
sshd.
It
makes
an
API
call
with
just
the
public
key
of
the
user
who's
connecting
then
the
gitlab
rails
application
does
a
lookup
in
the
database
in
sequel,
where
we
have
an
index
on
the
column
that
contains
these
I
think
we
take
a
hash
of
these
and
then
we
look
it
up
here
that,
but
at
least
we
have
a
sequel
lookup,
not
a
linear,
scan
of
a
flat
file.
A
A
A
A
Against
prepended,
the
way
that
plugin
mechanism
in
sshd
works
is
instead
of
taking
100
megabytes,
authorized
key
file
and
scanning
through
its
ava
commands,
you
feed
its
the
pub
key
and
then
the
command
either
prints
out
nothing,
which
means
this
key
was
not
found
or
it
prints
out
a
line
on
standard
out
that
would
have
been
allowed
in
the
authorized
key
file.
So
it's
almost
as
if
we
are
generating
authorized
keys
on
the
fly
per
requests
with
at
most
one
line
in
its,
but
the
line
looks
exactly
the
same.
Okay.
A
A
A
Is
responsible
for
setting
up
the
initial
ssh
connection
to
the
Khitan
me
backends
and
ice
I?
Don't
like
that,
because
this
word
changes
means
nothing
in
that
context,
and
your
repository
is
not
known.
So
this
shouldn't
be
outgoing
data.
This
good
lab
shells
confusing
because
of
because
of
its
history,
but
anyway,
what
it
comes
back
with
is
that
would
be
in
this
exercise
class.
A
Will
roughly
build
up
the
arguments?
I
think
that's
happening
here.
So
this
yeah
repository
jewel
repository
project
path,
IDG
username.
So
what
I'm
going
for
is
this
thing
needs
to
plug
into
a
get
Li,
RPC
and
I
want
to
find
that
RPC.
What
I
did
I
put?
I
shouldn't
have
closed
Kinley
proto.
It's
a
peer.
A
Pack
requests:
it
needs
a
repository.
Their
standard
in
data
standard
in
data
is
part
of
gates
on
transport,
so
first
it's
a
bake
data
that
will
describe
copying
back
and
forth.
Then
there
are
conflicts,
options
and
a
good
protocol
version,
and
if
we
look
here,
what
do
we
have?
We
have
repository
geo
repository
project
path.
Geol
ID
idea
is
something
that
is
specific
to
push
we'll
look
at
that
later.
We,
okay,
that's
what
I
need
to
get
to
next.
How
does
this
work
with
push?
This
is
push
Pacific
and
this
yeah.
A
Looks
a
little
a
little
confusing
so
from
this.
This
goes
back
into
a
go
executable
because
we're
sort
of
stuck
in
the
middle
between
having
historically
this
is
Ruby,
but
because
the
GRP
seek
lines
and
go
is
much
nicer
than
in
Ruby.
When
we
were
doing
the
Gately
migration
for
the
scallop
shell
stuff,
we
wrote
a
Geo,
PC
client
in
go
that
does
the
heavy
lifting.
So
this
thing
makes
this
JSON
thing
and
then
it's
execs
into
another
executable
and
passes
these
arguments,
and
maybe
just
maybe
we
can
see
that
executable.
A
A
A
So
now
this
is
a
clone,
and
here
you
can
see
that
this
is
HD
accepted
my
public
key
and
start
the
decision,
and
then
it
was
done
so
that
worked.
But
what
I
want
to
do
now
is
see
if
I
can
grab
a
big
repository
so
that
things
are
slow
and
then
we
can
see
what's
going
on
better.
So
if
I
clunk
it
lepsy
II
and
if
I
look
up
the.
A
A
A
A
A
A
A
We
under
scape,
that's
okay,
now
it's
readable,
so
it's
running
this
commands
Gately
upload
back
then
it
passes.
The
address
is
the
second
arguments,
so
that
is
just
acutely
network
address,
and
then
this
is
a
big
JSON
blob
that
describes
that
gives
enough
information
to
establish
the
session.
So
there's
a
storage
named
default.
It
has
a
relative
path
and
you
can
see
here
that
it's
still
using
the
old
non
hashed
formats.
A
These
things
are
empty.
They
have
to
do
with
git
push,
and
this
is
for
it's
also
a
sleuth
kit
push
and
this
one
is
for
logging
because
in
the
case,
that's
the
real
path.
The
real
relative
path
is
some
hashed
hexadecimal
blah
blah
blah
string.
We
want
to
pass
along
this
nice
name
for
logging
purposes.
So
that's
why
that
is
in
there
yeah.
A
So
that
then
goes
into
a
go
executable,
which
establishes
an
RPC
session
which
is
I
guess
not
as
exciting
I.
Think
it's
more
interesting
to
look
at
to
go
into
git
push
little
bits
because
gate
pushes
more
special
and
why
is
get
push
special
gets,
pushes
special
because
a
pool
is
more
or
less
anonymous.
The
moment
you
are
downloading
data
there
on
the
server
side,
we
just
generate
it
back
you
something
comes
in
and
says:
I
want
these
objects
or
I
wants
the
I
I
see
that
you
have
these
objects.
A
A
So
good
lab
needs
to
know
that
some
data
was
just
pushed
and
that
notification
doesn't
doesn't
happen
at
the
start
of
the
as
of
the
push
session,
because
at
that
point
it
might
fail.
Also
we
don't
know
what
was
pushed.
You
only
know
that
at
the
ends
there
are
hooks
ingots
that
allow
you
to
the
way
you
can
get
a
notification
in
the
form
of
an
executable
when
a
push
happens.
So
that
thing
needs
to
report
back
to
get
up
and
say:
hey
this
user
pushed
stuff
to
this
repo
and
when
we're
in
Italy.
A
We
don't
know
what
the
repo
is
called
and
get
labs
database,
and
we
also
have
no
idea
what
the
user
is
accepts.
That's
we
have
these
keys
where
these
fields,
where
you
can
say
actually
it
was
the
user
with
SSH
key
ID
1
and
then
on
the
gate
website.
We
translate
that
back
and
say:
oh
Jacob
just
pushed
something
and
the
other
thing
we
need
when
we
make
that
call
back
from
the
guilty
server
back
into
the
rails.
App
is
to
know
what
is
the
sequel
idea
of
the
project
that
just
got
updated
because.
A
A
So
if
we
had
a
reverse
lookup
function,
we
should
shop
that
around
with
the
world's
intelligence
agencies.
So
we
don't
have
a
reverse
lookup.
So
we
need
to
pass
this
made
today
there.
So
that's
when
that
callback
happens,
we
can
say
a
push
just
happened
and
the
repository
you
said
that
the
idea
of
this
repository
is
this.
So
this
whole
back
and
forth
is
going
on
here
between
the
this
front-end
and
get
early
and
rails.
A
Does
the
story
makes
sense
so
far?
Okay
I
think
it's
important
to
understand
this.
So
let's
try
it
stop.
Okay,
I
hope
we
have
it
back
now,
yep
we
got
it
cool.
So
what
happens
during
get
push
over
SSH,
because
that
is
the
most
interesting
case.
So
the
client
connects
to
get
la
via
SSH
to
get
clients
as
it
HD
spawns,
get
up.
She'll
get
slap.
Shell
makes
API
call
well
parses.
A
All
back
information,
a
logical
username,
logical
project,
name
and
those
are
calls
for
historical
reasons,
just
when
it's
called
GL
ID
and
this
one
is
called
GL
repository
or
in
lowercase
with
a
but
the
same
idea.
And
what
else
do
we
get
I
think
that
is
roughly
as
IO
also
important
to
authentic
Asian
token,
because
the
the
kiddley
service
can
each
have
their
own
authentication
token.
The
Caleb
shell
client
does
not
know
a
what
Gately
servers
exists
or
B.
A
A
A
As
I
say,
it's
received
back
is
the
server-side
command
that
corresponds
to
kids
push
because
from
the
point
of
view
of
the
server
it
is
receiving
a
back
from
the
gate,
client
it
it's
just
what
it's
called
so
then
we
get
on
the
Gately
site.
So
literally
then
spawns
gets
receive
back
in
the
rights
repository
directory
with
environment
variables,.
A
Legal
ID
and
GL
repository,
and
maybe
a
few
others
then
get
receive
back,
receives
kids
transports
data
from
get
clients
all
the
way
up
here.
This
thing
is
now
pushing
data
into
its,
but
from
the
point
of
view
of
the
kid
client
there's
just
a
file
descriptor,
it's
just
it's
a
socket,
so
it's
just
pushing
data
into
that
socket
and
that
then
gets
encrypted
and
chopped
into
pieces
and
whenever
by
sshd
and
goes
across
the
network,
then
it
comes
out
of
sshd
and
it
goes
into
this
gait
lab
shell,
executable
and.
A
D
A
A
A
B
A
After
effect,
data
is
sent,
get
receive,
its
receive
back,
tries
to
update
refs
and
then
hooks
run
and
it
so
this
again,
these
hooks
are
custom,
get
lab
codes
and
hooks,
make
free,
receive
checks,
for
instance,
to
block
force
pushes
for
protected
branches
and
a
whole
bunch
of
features.
But
the
ever-growing
list
of
good
lab
features
is
about
validating
pushes
so
that
all
happens
during
oak,
and
that
is
an
HTTP
call
back
back
into
get
lab
rails,
because
all
that
application
logic
is
defined
in
the
rails
app.
A
So
thanks,
HTTP
callback,
request
to
Rails
and
that
uses
this
information
and
the
reason
that
information
is
available
is
that
it
is
an
environment
variable
on
get
received
back.
So
the
hook
inherits
these
hooks
inherit
the
environments
of
get
receive
back
and
that's
how
we
hand
down
information
to
the
hook
and
then
who
can
make
an
API
call
and
say?
Actually,
the
user
has
SSH
key
with
ID
1
and
in
this
project,
with
sequel,
ID
9
and
then
they
kill
up.
Api
knows
what
the
hell
we're
talking
about.
A
Gays
receive
back
prints
some
messages.
For
instance,
it
prints
messages,
saying
that
go
here
to
create
a
virtual
quest
from
your
push.
If
you
push
a
new
branch,
that
is
a
message
that
comes
from
the
gait
lab
internal
API,
get
specs,
sent
back
an
HTTP
response
to
a
hook
that
there
is
no
process
of
get
receive
back
and
it
ends
up
so
get
them,
puts
it
on
standard
error
or
standard
outs.