►
From YouTube: GitLab Reference Architectures, Performance Testing, Cloud Native Hybrid and What is Gitaly
Description
Table of Contents:
00:00 - Start
00:52 - Ref Arch Performance Test Results
03:26 - Digging Into Reference Archectures
05:31 - GitLab Cloud Native Hybrid (Kubernetes)
06:20 - Cloud Platform as a Service
07:42 - Gitaly HA Cluster
11:08 - Wrap up and links
Resources:
- Reference Architectures (Including Cloud Native Hybrid): https://docs.gitlab.com/ee/administration/reference_architectures/
- Reference Architecture Performance Test Results: https://gitlab.com/gitlab-org/quality/performance/-/wikis/home
- Why Gitaly and Gitaly Cluster? https://gitlab.com/gitlab-org/gitaly/-/blob/master/doc/DESIGN.md
Play Lists:
- Learning to Provision the AWS Quick Start for GitLab on EKS (https://youtube.com/playlist?list=PL05JrBw4t0Koi8VBnoVhmj_MstnbJjGw5)
- GitLab and AWS (https://youtube.com/playlist?list=PL05JrBw4t0Ko30Bkf8bAvR-8E441Fy2G9)
- GitLab Alliances Tech (https://youtube.com/playlist?list=PL05JrBw4t0KqaWCrU4avIY6TQveKTDMso)
- GitLab Reference Architecture, Cloud Native, Performance Testing, Scaling and HA (https://youtube.com/playlist?list=PL05JrBw4t0KqgVugqCld2fUvrBulanchq)
- GitLab and Kubernetes (https://youtube.com/playlist?list=PL05JrBw4t0Kqp16k3nHTMdrWdATrcDZ5C)
- DarwinTheorizes (https://youtube.com/playlist?list=PL05JrBw4t0KoH74CcAtP0Ze7JXjX7zT6g)
A
A
Sometimes
when
we're
in
conversations
with
customers
or
partners,
it
becomes
evident
that
they're
not
aware
of
our
reference
architectures
or
they
might
not
be
aware
how
much
energy
and
actual
work
goes
into
maintaining
them,
as
well
as
maintaining
some
performance
testing
that
backs
them.
So
I
just
want
to
go
through
that
in
a
little
bit
of
detail
here.
So,
first
of
all,
reference
architectures
are
a
kind
of
a
cross-team
collaboration
at
gitlab.
A
They
got
really
strong
leadership
from
our
quality
and
distribution
teams
who
test
git
lab
constantly
and
try
to
understand
the
best
ways
to
put
it
together
and
to
configure
it
for
your
usage
at
different
user
sizes.
So,
on
your
screen
right
now,
you
can
see
that
we
have
the
reference
architectures
landing
page
up
and
what
I
want
to
show
you
quickly
is
that
these
are
actually
performance,
guaranteed
or
performance
tested.
A
So
I'm
going
to
open
this
gitlab
performance
tool,
repository
link
and
when
you
first
go
here,
you're,
like
oh
great
they've,
got
a
performance
tool.
Probably
only
someone
at
gitlab
can
do
that
and
that's
not
actually
true.
The
performance
tool
is
intended
for
anyone
to
set
up
and
use
against
a
gitlab
instance.
Now,
of
course,
if
gitlab
had
made
a
performance
tool
from
scratch,
we
might
be
tempted
to
make
the
performance
tests
come
out
in
our
favor,
but
we
actually
implement
the
open
source
tool
k6.
A
So
this
is
more
like
a
harness
that
goes
around
k6,
but
one
of
the
most
interesting
things
is:
if
we
go
down
here
to
the
wiki,
we
can
see
that
gitlab
maintains
published
results
for
each
of
these.
So
I'm
going
to
go
into
the
25k
test,
and
it
tells
us
information
here
about
what
kind
of
testing
is
being
done
so
on
this
option
section
here,
it's
telling
us
that
it's
driving
at
a
certain
requests
per
second.
A
So
this
is
500
requests
per
second,
and
we
can
go
down
here
and
look
at
each
of
the
rows
and
you
can
see
in
the
rows.
We
have
different
kinds
of
tests
that
are
considered
to
be
necessary
for
git
lab
to
do
well
in
your
environment.
We
have
how
many
rps
it's
being
driven
at
what
the
reported
results
are
and
then
ttfb
is
a
measure
of
how
much
time
the
first
bytes
or
first
activity
it
takes
to
come
through.
A
A
You
can
see
at
the
top
here
how
long
it
took
to
run
the
test
an
hour
and
24
minutes
in
this
case,
so
it
does
do
some
pretty
serious
exercising
when
the
performance
teams
do
these
tests.
They
make
sure
to
size
the
instance.
So
there's
no
scaling
going
on,
so
they
can
be
sure
of
what's
going
on,
here's
a
failed
test,
so
it's
not
absolutely
guaranteed
that
every
test
will
pass
it's
important
to
look
and
see
what
it
failed
by
and
exactly
what
it's
on.
A
This
is
a
typical
challenging
api
for
git
lab
to
be
performant
on
when
you
don't
have
elasticsearch
configured,
and
so
you
can
look
through
the
tests.
So
all
the
tests
are
here
for
the
various
sizes.
So
keep
that
in
mind
that
we
do
do
regular
performance
testing
of
the
various
reference
architectures
and
then
you
can
compare
your
results
if
you
configure
gitlab.
Similarly-
or
maybe
you
say
hey,
you
know,
we
want
to
trim
back
on
that
cpu
a
little
bit
because
we
don't
have
10
000
users.
A
We
have
7
000
users,
and
so
you
can
retest
and
make
sure
that
you're
going
to
get
reasonable
performance.
So
let's
take
a
look
at
the
individual
reference
architectures
and
what
they
consist
of.
First,
I'm
going
to
pick
up
the
1000
user
reference
architecture
for
1,
000
users
and
less.
We
recommend
a
single
machine
for
all
of
gitlab,
and
so
these
are
very
simple:
there's,
not
even
a
diagram.
A
Once
we
get
up
to
3000
users,
though
we
are
purposely
configuring
for
high
availability.
The
reference
architecture
is
by
default,
go
through
what
it
would
take
to
deploy
git
lab
on
individual
instances.
So
you
can
see
here
we
have
a
nodes
column
and
we
also
have
individual
server
roles.
Now,
as
we
go
back.
If
we
go
back
to
2000,
we'll
see,
there's
a
lot
less
rows
in
this
table,
because
a
lot
of
the
capabilities
are
either
consolidated
or
they
don't
need
to
be
broken
out
in
order
to
be
performance
managed
individually.
A
So
it
gets
more
complicated
with
more
of
these
services
as
you
as
you
scale,
out,
and
more
nodes
for
individual
services.
If
you've
worked
with
highly
scaled
applications,
you
know
that
the
different
parts
of
the
application
do
not
scale
evenly,
as
you
have
to
start
accommodating
more
and
more
traffic.
A
So
this
shows
you
that
the
details
there
and
then
as
we
go
down,
we
get
a
nice
diagram
that
shows
us
how
the
pieces
and
parts
work
together.
Now
you
should
be
aware
that
not
everything
needed
to
make
gitlab
work
is
in
a
reference
architecture.
So,
for
instance,
in
here
there's
no
mention
of
email
services.
You,
you
generally
want
an
outbound
email
service
of
some
type
or
an
email
server,
but
because
it
doesn't
affect
the
core
performance
of
the
instance.
It
is
not
necessarily
covered
by
reference
architectures.
A
We
can
go
down
to
the
25k
reference
architecture
and
see
that
it's
got
a
lot
more
broken
out.
I
also
noticed
that
the
node
types
for
google,
aws
and
azure
are
noted
here,
so
the
memory
in
terms
of
our
vcpu
and
memory
requirements
are
noted
here
and
then
the
instance
type
that
the
reference
architecture
uses
to
do.
Testing
is
noted
here.
A
At
the
top
here
we
see
a
section
called
cluster
topology
and
basically,
as
git
lab
tests
and
assures
that
certain
components
can
indeed
live
in
a
kubernetes
cluster.
They
are
added
to
this
table.
They
are
performance
tested,
the
pod
configurations
are
tuned
and
then
the
information
is
published
here,
so
everything
that
can
be
put
in
the
cluster
reliably
and
safely
in
a
tested
format
is
specified
here
then,
this
next
table
indicates
roles
that
cannot
go
in
the
cluster
notice.
These
footnote
notations.
A
So
generally,
these
footnote
notations
are
telling
you
what
services
can
be
run
on
cloud
platform
as
a
service
and
if
a
given
platform
as
a
service
has
been
tested
or
we
have
customers
who
are
working
with
a
significant
volume
on
a
particular
platform
as
a
service.
It's
noted
down
here,
so
you
can
see
that
for
postgres
sql,
this
role
console
postgresql,
sql
and
pg.
Bouncer
are
all
capable
of
being
done
by
a
platform
as
a
service.
Postgres
sql,
the
ones
that
we
know
to
work,
are
google
cloud
sql
and
aws
rds.
A
So
this
is
a
way
of
understanding
what
could
be
done
as
platform
as
a
service.
If
we
proceed
down
the
page,
we
can
also
see
the
updated
reference
architecture
diagram.
So
I'm
going
to
quickly
pop
this
out,
so
we
can
get
a
little
bit
more
of
a
zoom
in
and
we
can
see
here
what
can
go
into
the
kubernetes
cluster
at
the
top
of
the
chart.
And
then
we
see
database,
redis,
console
and
a
few
other
things
we
know
from
the
second
table.
The
database
and
redis
can
become
platform
as
a
service.
A
I
want
to
call
out
giddily
cluster,
because
gitly
is
a
capability
that
gitlab
has
built
specifically
to
help
with
some
challenges
of
git.
I'm
just
going
to
open
up
a
page
here
that
shows
us
out
on
the
gitly
website
our
project.
They
have
documented
some
of
the
things
that
are
the
reasons
why
gitlab
made
gitly.
So
basically,
git
itself
is
used
on
server
side
of
gitlab
and
other
git
collaborative
systems,
and
it
has
characteristics
that
really
mirror
an
acid
database
engine
depending
on
what
it's
trying
to
do
with
the
information.
A
That's
in
the
commit
or
the
pull
request,
it
might
want
a
lot
more
cpu
or
a
lot
more
memory
or
a
lot
faster
disk.
I
o
for
an
extended
period
of
time,
being
able
to
take
up
a
lot
of
cpu
or
a
lot
of
memory
or
a
lot
of
disk
io.
All
of
a
sudden
based
on
the
workload
is
something
we
we
generally
do
on
instance,
compute
or
vms,
or
traditional
hardware,
if
you
think
of
it
that
way,
and
so
because
of
this,
it
makes
get
very
hard
to
start
to
scale
horizontally.
A
A
So
when
you
do
a
well-managed
container,
cluster
you're,
usually
very
good
about
defining
limits
for
all
of
these
particular
resources,
so
that
we
know
when
to
scale
horizontally,
because
git
itself
does
not
a
good
implementation
of
the
12-factor
model.
It
does
not
easily
scale
in
this
manner.
When
we
have
a
well-run
kubernetes
cluster,
we
generally
define
these
limits
so
that
when
something
exceeds
them,
it
is
pushed
out
or
it
is
evicted
and
when
this
happens
during
a
stateful
transaction
such
as
saving
your
git
repository
data.
A
So
we
refer
to
this
as
the
gitlab
cloud
native
hybrid
architecture,
because
we
put
as
much
as
possible
into
kubernetes
and
then
all
stateful
services
have
to
reside
outside
of
it.
It
turns
out
that
the
stateful
services,
besides
getting
cluster,
are
generally
available
on
cloud
platform
as
a
service,
but
because
gitly
cluster
is
get
lab's
own
creation.
In
order
to
help
with
this
problem,
it
has
to
be
implemented,
on
instance
compute.