►
From YouTube: Easy Performance Testing an AWS Quick Start Provisioned GitLab Cloud Native Hybrid Instance
Description
Table of Contents:
00:00 - Easy Performance Testing for GitLab Cloud Native Hybrid That Was Prepared Using the AWS Quick Start AWS Quick Start for GitLab Cloud Native Hybrid on EKS
00:24 - Helpful Background
01:30 - Finding The Instructions
01:49 - Benefits of using SSM Accessed Bastion
02:43 - Manually Full Scaling the EKS Cluster
08:53 - Very Fast Cluster Scaling
10:14 - Scaling Complete, Create the Test Data
11:52 - Always Run GPT For Custom Configurations
17:08 - Run Performance Testing
22:19 - Testing Cluster Autoscaling Using GPT
28:40 - Cluster Scaling During Scale Test
29:41 - Scale Testing Results
30:23 - No Stateful Services In the Cluster
31:09 - Resources
31:46 - GitLab Customer Success
Helpful Background to Consume First:
- Explainer Video: Overview GitLab AWS Implementation Patterns (13mins): https://bit.ly/3lpGwpZ
- Explainer Video: Overview GitLab AWS Quick Start for GitLab Cloud Native Hybrid on EKS (9mins): https://bit.ly/3lrv6C0
- Explainer Video: Provisioning Ready-to-Run GitLab for 50,000 Users in 14 Clicks and a Long Lunch (20mins): https://bit.ly/3mbTqIr
Resources:
- GitLab AWS Implementation Patterns for GitLab Cloud Native Hybrid: https://bit.ly/3xhyJww
- AWS Quick Start for GitLab Cloud Native Hybrid on EKS: https://bit.ly/3lej9j4
- Easy Performance Testing for a GitLab CNH Instance That Was Prepared Using the AWS Quick Start: https://bit.ly/3CSFXen
Playlists:
- Learning to Provision the AWS Quick Start for GitLab on EKS (https://bit.ly/3xhyJww)
- GitLab and AWS (https://youtube.com/playlist?list=PL05JrBw4t0Ko30Bkf8bAvR-8E441Fy2G9)
- GitLab Alliances Tech (https://youtube.com/playlist?list=PL05JrBw4t0KqaWCrU4avIY6TQveKTDMso)
- GitLab Reference Architecture, Performance Testing, Scaling and HA (https://youtube.com/playlist?list=PL05JrBw4t0KqgVugqCld2fUvrBulanchq)
- GitLab and Kubernetes (https://youtube.com/playlist?list=PL05JrBw4t0Kqp16k3nHTMdrWdATrcDZ5C)
- DarwinTheorizes at GitLab (https://youtube.com/playlist?list=PL05JrBw4t0KoH74CcAtP0Ze7JXjX7zT6g)
A
Hey
everyone:
it's
darwin
cenoy,
with
git
lab,
I'm
a
staff
solutions
architect
in
gitlab's
alliances
group
just
wanted
to
take
you
through
easy
performance
testing
for
gitlab
cloud
native
hybrid
that
was
prepared
using
the
aws
quickstart
so
due
to
some
of
the
things
that
are
in
the
quick
start,
most
especially
the
bastion
host
it's
much
easier
to
set
up
performance
testing
when
using
this
infrastructure
as
code.
So
we're
going
to
take
a
look
at
this,
I
do
want
to
mention
that
there's
some
things
that
we
hope
that
you
will
review
beforehand.
A
So
these
different
videos
here,
ranging
between
9
and
20
minutes,
are
actually
lead.
Ups
to
what
we'll
talk
about
in
here.
So
it's
a
good
idea
to
potentially
even
pause
the
video
right
now
and
go
check
those
out.
So,
let's
head
into
it
right
now,
so
as
part
of
the
implementation
patterns
we
have
for
the
gitlab
cloud
native
hybrid
for
aws.
A
We
also
have
a
special
set
of
performance
testing
instructions
and
the
reason
why
they're
special
is
because
of
the
way
the
quick
start
is
set
up.
You
can
we
can
abbreviate
the
instructions,
so
you
can
use
the
bastion
host.
You
don't
have
to
set
up
a
special
host
in
order
to
do
the
testing
and
also
the
bastion
host
has
all
utilities
on
it.
So
we're
able
to
give
you
very
direct
instructions
on
how
to
do
the
the
fastest
path
to
performance
testing
a
git
lab
cloud-native
instance.
That's
been
set
up
in
this
way.
A
If
you
are
performance
testing,
something
else,
something
that
you've
set
up
on
your
own
or
omnibus,
then
you
will
need
to
dig
through
the
gitlab
performance
testing
project,
to
understand
what
steps
you
might
need
to
take
so
on
the
implementation
patterns.
I'm
going
to
simply
click
here.
Performance
testing
instance
prepared
using
aws
quickstart
for
get
lab
cloud
native
on
eks
and
we're
going
to
follow
through
these
instructions
in
the
aws
account.
A
Where
I
have
put
this
instance,
I'm
going
to
use
the
console
host
on
the
bastion
host,
so
one
of
the
cool
things
about
the
quick
start
that
I've
mentioned
before
is
that
it
has
this
concept
of
a
bastion
host.
And
so
I
don't
need
to
expose
my
endpoint
at
all
the
bastion
host
logs
things,
especially
it's
using
ssm,
which
also
you
can
control
who's
allowed
to
log
in
at
an
amazon
level
and
not
play
around
with
ssh
keys,
and
it
also
lets
you
do
full
logging
of
everything
that
happens
on
the
bastion
host.
A
So
because
this
bastion
host
is
a
standard
piece
of
hardware.
So
to
speak
in
the
eks
quickstart
which
the
gitlab
quickstart
depends
on
you
get
all
of
this
auditability
built
right
in.
So
I'm
going
to
do
a
connect
on
my
bastion
host
with
session
manager,
and
when
I
do
this
connect,
I'm
gonna
get
a
regular
console
and
I'm
gonna
be
switching
back
and
forth
with
the
performance
testings
and
in
here
it
tells
us
how
to
get
onto
the
console
host
like
we
just
did
and
that
those
instructions
are
also
in
the
quick
start
itself.
A
A
A
A
A
A
And
then
use
this
command
here
to
understand
what
chart
version
is
deployed,
so
in
the
future
the
instructions
are
kind
of
future
proof,
so
that
no
matter
what
you
used,
you
use
the
right
version
based
on
your
your
setup.
So
here
we
can
see
that
we
use
the
chart
version
412
3.
So
then,
when
I
run
this
command,
I'm
going
to
need
to
insert
that
and
actually
I'm
going
to
I'm
going
to
hold
off
on
this
for
just
a
minute.
A
We
have
the
eks
cluster,
auto
scaler
in
place,
and
if
I
throw
this
pod
manifest
at
it,
it
should
auto
scale
the
cluster.
But
when
you
do
performance
testing
that
matches
our
official
performance
testing,
we
don't
want
auto
scaling
to
be
in
play
because
it
may
not
perform
as
it's
tested
by
the
reference
architecture
team.
So
what
I'm
going
to
do
is
go
into
ec2
and
manually
scale
when
you
see
what
I'm
going
to
do,
you're,
probably
like.
A
Actually,
let's
set
all
of
them
to
28
just
so.
We
absolutely
sure
that
we
get
our
full
capacity
and
that's
going
to
start
deploying
the
individual
nodes
across
the
three
zones
that
we're
in
so
soon
here
we'll
see
it
start
to
say:
hey,
I'm
spinning
up
nodes.
Once
that's
done,
we
will
deploy
the
pod
manifest
in
theory,
there's
nothing!
A
Stopping
me
from
right
now
deploying
the
pod
manifest,
and
these
would
both
scale
together
and
it
would
probably
be
fine,
but
I
like
to
be
a
little
extra
paranoid
when
I'm
about
to
do
performance
testing
and
then
once
we
get
it
fully
scaled,
we'll
kind
of
close
out
this
video,
because
I
want
to
do
performance
testing
as
a
separate
video.
So
it's
consumable
separately
and
we'll
see
that
we
have
the
full
on
git
lab
at
50k
manually,
scaled
okay,
so
just
a
little
midway
check
in
the
asg
has
started
scaling
you
can
see
here.
A
A
Okay,
this
is
already
done
and
in
the
real
world
you
wouldn't
have
to
play
with
this.
This
is
the
performance
testing
world
and
getting
to
full
scale.
We
can
probably
figure
out
how
much
time
it
took
by
taking
a
look
here.
So
this
record
here
was
when
it's
scaled
from
zero
to
seven
and
then
this
is
the
first
record
when
it's
scaling
from
seven
to
twenty
eight.
So
the
start
time
in
this
record
is
eleven
forty,
nine
and
if
we
go
to
the
other
end,
we
have
eleven
50
11
51,
11
51.
A
A
Paste
it
and
then
I'm
going
to
grab
the
version
4.12.3
from
the
line
above
and
this
is
going
to
start
the
pod
scaling
for
us
and
then
we
can
use
this
command
to
check
on
where
it's
at
with
pod
scaling
to
kind
of
know.
When
it's
done
so
we'll
come
back
and
take
a
look
at
this
in
a
minute
or
two
okay,
our
manifest
was
applied
and
now
we'll
take
a
look
at
how
the
replica
sets
are
doing.
A
If
we
do
get
pods,
we
should
see
more
pods
okay,
so
we
got
a
lot
more
pods,
so
that
was
really
fast
scaling.
I've
done
some
stress
testing
and
some
auto
scaling
testing
on
git
lab
with
the
performance
tool
on
eks
and
eks
scales
very
smoothly
at
the
host
and
the
the
pod
level.
So
I
was
quite
impressed
with
that.
A
A
One
is
generating
performance
testing
data
into
the
instance
and
that's
what
these
next
two
stanzas
are
talking
about:
we're
going
to
use
the
get
lab
tools
as
containers
just
to
make
it
easier,
so
we
don't
have
to
install
them
as
well
and
that's
what
this
bit
is
about.
So
I'm
going
to
install
docker.
A
And
then
once
docker
is
installed,
we'll
make
sure
it's
started
and
then
we'll
be
able
to
use
docker
commands
both
to
populate
the
repository
with
one
utility
and
then
to
run
the
performance
testing.
So
this
all
is
very
easy
to
reproduce
and
redo
on.
You
know
whenever
you
set
up
an
instance
or
performance
testing,
the
goal
of
these
instructions
is
just
to
make
it
super
easy
for
you
to
do
this
because
it
especially
if
you're
tuning
anything,
let's
say
you're
like
hey,
we're
at
30k
and
there's
only
a
25
and
a
50k.
A
It's
pulling
the
image
down
got
to
make
a
few
more
gpt
folders
here
to
house
various
aspects
of
the
test.
I
mentioned
in
a
previous
video
that
we
actually
use
the
k6
utility,
so
this
is
really
a
harness
around
it,
so
we
don't
have
our
own
custom
utility.
That
is,
you
know,
being
special
and
making
gitlab
look
really
good.
We
use
open
source
third-party
tool,
and
so
here's
another
place
where
it
says,
update
your
ref
arc
size
as
a
reminder
so
right
there.
I
need
to
change
it
to
50k.
A
A
The
user
will
stay
root,
so
I'm
going
to
use
the
root
user
because
I
don't
have
to
set
up
a
separate
user,
then
and
log
in
as
them
and
all
of
that,
so
I'm
going
to
go
in
as
the
root
user
and
create
an
api
key
which
I
will
be
deleting
or
disabling
before
I
publish
this
video
so
we'll
do
an
access
token
called
gpt
just
for
fun,
so
we
know
what
it's
for.
We
want
to
turn
on
every
thing.
A
A
Oh
wait!
I
don't
use
a
token
here.
Quite
yet,
I'm
not
following
the
instructions
so
use
a
root.
Latency
five,
so
I'm
gonna
go
down
and
just
put
in
a
latency,
even
though
we're
on
a
ec2
instance.
It's
in
the
exact
same
region
as
the
as
our
gitlab
instance.
So
it
should
be
just
fine,
but
I
like
to
be
safe
and
then,
as
we
come
down
here,
it
says
make
sure
that
your
config
ended
up
in
the
right
place.
So
let's
do
that
because
if
it's
misplaced,
you
start
getting
weird
errors.
A
Also,
all
these
commands
have
been
worked
out
to
be
the
minimum
amount
of
fussing
around
for
you
altering
the
command
line,
so
there's
less
mistakes
to
be
made.
So
that's
why
I
always
call
the
target
files
like
this
test.json,
instead
of
giving
them
a
name
that
includes
the
size,
because
then
we
would
be
constantly
having
to
adapt
that
as
we
go
through
the
instructions.
So
now
I
need
the
token.
A
A
And
here's
where
it's
creating
all
of
our
test
assets
right
now.
This
is
going
to
take
nearly
an
hour
to
run,
at
least
it
does.
In
previous
experience.
I
have
this
first
time
I've
run
50k,
so
I'm
not
sure
if
it
generates
more
data,
I
don't
think
it
does,
but
so
we'll
wait
till
this
finishes
and
come
back
to
the
video
and
continue.
A
A
A
A
A
A
A
A
Let
me
take
a
look
here.
Okay,
so
I
think
it's
waiting
for
an
extension,
yep
and
so
we'll
cap
this
file.
This
is
the
easiest
on
the
console
to
view
and
we'll
take
a
look
at
what
we
did
so
just
briefly
how
to
read
this.
There's
some
information
up
here
on
the
top
that
tells
how
long
it
took
to
run
the
whole
test,
what
version
of
gpt
you're
running
and
then
down
in
here
it
tells
us
our
target
requests
per
second
and
what
our
performance
was.
A
So
you
can
see
here
that
we
are
running
really
close,
so
we've
got
a
hundred
percent
on
most
of
the
tests
and
99.99
on
some.
We
have
99.91
on
a
test
which
then
is
marked
as
failed
93
on
this
test
down
here,
and
what
we
can
do
from
here
is
tune
it.
So
if
you,
if
99.94,
is
not
enough
for
your
liking,
then
you
can
start
tuning
things.
Maybe
you
need
a
larger
redis
cache.
Maybe
you
need.
You
know,
database
to
be
a
little
bit.
A
Bigger
you'll
also
notice
that
even
on
some
of
our
official
tests,
there
will
be
some
failed
tests
because,
when
you're
within
a
certain
threshold,
doing
pretty
good
in
order
to
get
that
last
point
four
percent
of
performance
or
point
zero,
six
percent,
you
might
need
to
throw
quite
a
bit
of
resource
at
it.
So
this
might
be
acceptable
to
you.
So
that's
our
performance
test
results
for
the
10
or
the
50k
instance
size
I'll,
be
putting
these
performance
tests
up
on
the
same
place
where
the
instructions
were
for
driving
the
tests.
A
It's
kind
of
not
real
world,
because
we're
going
to
take
a
quarter
scaled
instance
and
then
just
jam
it
with
a
hundred
percent
of
the
performance
test.
But
I
have
seen
it
do
very
well
on
lower
sizes.
What
we'll
do
first
is
scale
back
our
pods,
so
we'll
reverse
one
of
the
changes
we
made
here
and
one
of
the
really
cool
things
is
that
the
kubernetes
auto
scaler
is
smart
enough
to
handle
pods
intelligently
when
it
starts
to
scale
down
the
cluster.
A
A
A
So
this
should
tell
the
auto
scaler
hey.
I
don't
need
any
more
than
this.
If
we're
not
under
load,
I
could
wait
for
it
to
scale
down
the
nodes,
but
it
can
be
kind
of
conservative,
so
I'm
going
to
actually
as
soon
as
it's
done
go
into
the
asg
and
force
it
to
scale
down
as
well
and
then
once
the
test
kicks
in
again,
it's
going
to
cause
it
to
start
to
scale.
Okay,
so
we've
told
it
what
we
want
to
do
there,
then
in
here,
go
to
our
asg.
A
A
A
Okay,
so
this
looks
like
the
entire
page.
So
when
we
kicked
off
the
performance
test,
we
got
a
increase
from
7
to
15
almost
immediately
and
then
it
looks
like
it's
been
scaling
a
little
bit
since
so
15
to
16
16
to
17,
and
we
saw
that
it's
up
to
18.
So
there
might
be
another
record
here:
yep
there
we
go
17
to
18..
A
A
So
I
seriously
doubt
it's
going
to
reach
28
instances
because
we
never
are
running
all
parts
of
the
test
at
once,
but
when
we
do
this
fixed
scaling,
we
scale
the
whole
cluster
as
though
all
those
things
are
going
on
at
once,
just
to
make
absolutely
sure
that
the
test
is
not
being
frustrated
by
any
scaling
activity
but
in
this
case
we're
trying
to
test
the
scaling
after
having
verified
the
other
testing.
So
we'll
have
to
see
how
our
performance
tests
work
out
at
the
end
of
this.
A
So
I
just
wanted
to
point
something
out
about
the
cluster
auto
scaling,
I'm
going
to
dig
into
the
scaling
here.
You
can
see
right
now
we're
at
13
instances,
even
though
our
max
is
28
for
this
size.
But
if
we
go
to
the
activity,
we
can
also
see
something
interesting.
It's
actually
been
scaling
up
and
down,
as
the
performance
test
has
been
running,
so
you
can
see
here
at
one
point.
A
It
was
all
the
way
up
to
23
instances
and
it
shrunk
down
to
22,
and
so
you
can
just
see
it's
shrinking,
shrinking,
shrinking
and
then
eventually
growing
a
little
bit
more,
and
so
the
auto
scaling
is
varying
it
in
response
to
the
actual
performance
test.
And
although
when
we
do
official
performance
testing,
for
you
know
what
our
max
capacity
is
capable
of,
we
wouldn't
want
auto
scaling
to
interfere
in
a
production
cluster.
You
generally
need
it
either
auto
scaling
or
some
sort
of
scheduled
scaling.
A
Okay,
our
testing
has
completed
it,
looks
like
like
last
time
we
failed
some
of
our
tests
where
we
got
the
final
test
right
here,
so
we
have
failed
quite
a
few,
although
I'm
not
sure.
I
understand
some
of
the
results,
because
it's
showing
that
we
have
a
hundred
percent,
but
we
still
somehow
failed.
You
know
what
it
is
likely.
If
I
look
through
the
logs
I'll
find
out
that
there
was
individual
failures
when
we
reached
out,
we
got
a
web
request.
Failure
and
those
kind
of
failures
are
recorded.
A
But
if
this
cluster
was
scaling
at
the
time,
then
occasionally
it
might
miss
a
web
request
or
something
like
that.
Also
it's
very
important
to
realize
that
we
have
no
stateful
services
in
the
cluster,
so
I
have
seen
customers
putting
prefect
or
gitly
into
their
cluster
and
the
problem
with
that
will
be.
A
So
I
hope
you've
enjoyed
seeing
this
performance
testing,
both
the
baseline
full-scale
testing,
as
well
as
the
auto-scale
testing,
and
it
might
give
you
a
basis
for
innovating
forward
and
finding
out
how
you
need
to
size
your
instance
and
what
kind
of
auto
scaling
you
may
or
may
not
want
to
do.
So.
Let's
take
a
minute
to
review
some
of
the
resources
available.
We
have
the
git
lab
aws
implementation
patterns
for
cloud
native,
hybrid
documentation
that
helps
you
really
plug
in
easy
to
use
values
to
get
a
cloud
native
hybrid
set
up
on
aws.
A
We
have
the
quick
start
for
gitlab
cloud
native
hybrid
on
eks,
and
this
is
the
actual
amazon
quick
start
document
that
we've
worked
with
them
to
help
create
and
then
finally
easy
performance
testing
for
gitlab
cloud
native
hybrid
instance.
That
was
prepared
using
aws
quick
start.
This
final
document
is
linked
in
the
previous
one
on
the
screen,
but
I
want
to
provide
an
independent
link
for
that.