►
From YouTube: Demo of Gitaly Failover on 50K Reference Architecture
Description
Nick Westbury (Sr. SET for Gitaly) demonstrates Gitaly Failover on our 50K reference architecture.
A
Hi,
my
name
is
nick
westbury,
I'm
a
software
engineer
and
test
here
at
get
lab
today,
I'm
just
going
to
be
running
through
a
test
that
we
perform
for
italy
failover.
The
aim
of
this
test
is
just
to
show
how
italy
and
how
your
local,
git
commands
will
behave
during
a
failover
is
so
essentially
if
your
primary
gittake
server
goes
down
and
a
new
primary
needs
to
be
elected.
A
So
to
do
this,
we've
just
created
a
few
projects
in
our
gitlab
instance.
Here
we've
got
a
50k
architecture
and
we've
just
got
a
bunch
of
projects
here
which
are
pretty
much
empty.
A
They're
all
identical
in
that
they've
all
got
a
readme
which
just
has
the
title
and
they've
all
got
this
extra
file,
which
just
keeps
a
count
of
when
we're
doing
our
testing.
So
these
numbers
here
just
increment
and
shows
each
time
that
a
test
has
been
performed
and
we'll
see
that
happen
during
the
testing.
A
So
if
I
just
go
back
so
to
do
this,
all
we're
going
to
do
is
we're
going
to
send
load
to
this
system.
So
what
that
means
is
we'll
have
running
in
a
loop
we'll
be
checking
out
each
of
these
different
projects
all
at
the
same
time.
So
we'll
be
cloning.
Those
locally
we'll
be
adding
a
new
number
to
that
file
and
we'll
be
pushing
the
result
and
we'll
just
do
that
in
a
loop
throughout
the
entire
failover
process.
A
A
A
So
now
that
that's
running,
we
can
go
on
to
one
of
our
prefect
nodes
that
we've
got
here.
So
it's
just
any
of
our
preferred
nodes.
We
can
run
this
curl
command
and
this
will
show
us
who
our
guitar
leader
is
so
in
this
instance,
we've
got
gitly
one
so
to
perform
our
failover.
We
want
to
connect
to
our
githuli.
A
Node
and
in
this
instance
we'll
just
stop
the
node,
so
we
can
do
a
ctl
stop
before
I
do
that
I'll
just
show.
So
all
of
these
get
commands.
We
actually
are
printing
to
this
log
file
that
we
have
here.
So
each
of
these,
like
tests
that
we
run
just
prints
out
that
it
checked
out
this
repository
and
it
was
a
success.
A
So
you
can
see
that
if
we
go
to
the
bottom
of
the
file,
we've
got
start
time
here.
It
just
updated,
did
a
bunch
more
printed,
the
end
time,
and
it
just
keeps
going.
It's
just
continuously
going
in
the
loop.
So
now,
if
we
stop
the
italy
server.
A
Okay,
so
that's
going
down,
we
should
now
see
here.
What
actually
happens
is
that
this
starts
to
fail.
So
at
first
we
see
a
few
clones
fail
and
what
this
will
be
is
because
at
the
point
we
stopped,
these
were
trying
to
clone
from
that
repository
that
we
just
stopped
from
that
gated
server
that
we
just
stopped
so
they
fail.
But
then,
after
those
initial
clones
we
should
see
here
that
everything
it
says
clone
fail.
A
Everything
now
starts
to
say,
push
failed
and
that's
because
there's
no
primary
server
to
push
to
so
reads
can
happen
from
any
server.
So
those
those
keep
working.
We
keep
cloning
the
repository
without
issue,
but
pushes
all
now
fail
because
there's
no
primary,
but
what
happens
is
gilly
automatically
alexa
do
primary.
So
if
we
scroll
down
to
the
end
here,
we
can
now
see
that
everything
starts
passing
again
without
us
doing
anything
and
in
reality,
the
time
it
took
to
do
all
of
this
takes
from
10
27
49
to
10
27
59.
A
We
can
see
all
the
numbers
we
just
kept.
Incrementing
there's
no
gap
in
our
testing
because
it
while
it
can't
push
it
just
update
and
then
it
continues
to
push
again
and
we
end
up
and
now
we're
just
34.
34
refresh
it's
another
five,
three,
six,
eight
seven.
So
the
program
is
still
there.
It's
still
pushing
and
everything
is
updating
as
expected.
A
So
that's
gilly
failing
over
when
I
say
during
the
window
where
a
new
primary
is
being
elected
or
pushes
fail,
but
then,
as
soon
as
that
window
finishes,
everything
starts
to
pass
again.