►
From YouTube: Migrating Internet Archive to Kubernetes - Tracey Jaquith & David Van Duzer, Internet Archive
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Migrating Internet Archive to Kubernetes - Tracey Jaquith & David Van Duzer, Internet Archive
The Internet Archive has been on a mission to provide universal access to all knowledge for 22 years. While it continues to preserve the history of the web, the Archive must keep its infrastructure aligned with the present and future of the web. Tracey and David will review the Kubernetes migration: the 600,000 line PHP monolith powering core library operations, as well as a wide variety of auxiliary services (like web crawling) supported by many development teams. They will cover deploying Kubernetes at progressively larger scales in the Archive's own data centers, and integrating existing developer workflows with GitLab, Auto DevOps, Helm and Prometheus. Tracey Jaquith has architected much of Internet Archive since 1996 (starting with founder Brewster Kahle) and focuses on DevOps, TV, audio/video, and AR/VR. David Van Duzer manages web-wide crawl operations at Internet Archive.
To learn more: https://sched.co/GrSG
A
Okay,
thank
you
all
for
coming.
This
talk
is
about
migrating,
the
Internet
Archive
to
kubernetes.
It's
a
it's
a
love/hate,
long,
adventure
story.
It's
mostly
love.
It
takes
a
while
we're
here
to
share
tips
and
tricks
and
explain
a
little
bit
of
what
we
went
for
first
and
especially
explain
our
last
six
to
twelve
months.
Yeah,
oh
and
my
name
is
Tracy
I've
been
in
theater
archive
for
about
18
years
since
we
started,
although
I
left
for
four
years
and
then
came
back
this.
B
Is
a
noisy
stage,
my
name
is
David
Van
Duzer
I've
been
in
the
Internet
Archive
for
about
a
year
and
a
half
now
I
work
on
the
web
web
crawling
for
the
wayback
machine.
So
this
is
a
story
of
how
we
took
on
some
very
daunting
massive
technology.
The
Internet
Archive
has
been
around
for
22
since,
since
the
beginning
of
the
web,
the
backing
the
platform
that
we
have
is
in
and
of
itself
a
daunting
pile
of
technology
which
I
have
become
more
familiar
with
more
recently
and
we're
just
now
hitting
our
toes
into
kubernetes.
B
So
what
did
we
find
like?
It's?
Any
big
change
is
going
to
not
it's
not
necessarily
gonna
sit
well
with
all
of
your
colleagues
figuring
out
what
everyone's
pain
points
are
wrapping
all
of
that
of
answer
the
question:
what's
in
it?
For
us,
we
had
a
pretty
good
idea
of
you
know:
we'd
been
using
docker
for
a
little
while
we
knew
that
we
have
ways
of
packaging
our
PHP
monolith,
but
new
stuff
comes
up.
It's
nice
to
be
able
to
reuse
tools
if
you
can
but
new
build
chains,
new
things.
B
We
all
know
this
docker
container
is
a
ssin,
but
to
me
the
most
interesting
thing
was
that
it
enforced
a
constrained
model
of
how
to
deploy
things.
So
the
old
saw
constraint
reads:
creativity,
forces
us
force
me
to
learn
a
new
way
to
develop
something
that
will
be
ultimately
more
scalable.
So,
rather
than
get
into
the
weeds
of
like
the
technical
objections
and
different
things
that
various
people
might
have,
we
took
a
little
more
fundamental
look
at
well.
B
So
out
of
the
of
all
the
things
that
we
do,
what
what
was
most
obviously
immediately
kubernetes
ready,
so
we're
most
famous
for
the
wayback
machine
which,
which
is
which
is
the
thing
that
which
is
the
thing
that
shows
you
any
webpage
from
the
last
20
20
to
plus
years,
but
but
the
wayback
machine
itself
operates
on
this
platform
that
contains
books,
videos,
audio
and
whatnot.
Here
we
have
the
Apple
website
from
1897,
but
we
also,
we
also
scan
books.
B
B
So
we
wanted
to
look
a
little
deeper
to
our
storage
platform.
So
this
this
large
platform
keeps
50
petabytes
or
more.
It's
the
fourth
generation
of
the
system
that
we've
been
using
we're.
Not
the
important
thing
to
remember
is
that
we're
always
self-hosting
we're
not
going
into
the
cloud
because
then
they're,
the
library
that
cloud
providers-
library,
not
us,
so
we
want
to
meet
maintainer
of
Independence,
but
because
of
the
size
we've
gotten
to,
we
still
get
the
economies
of
scale,
but
we've
got
a
much
smaller
staff.
B
So
there
is
a
lot
of
skepticism
to
putting
50
petabytes
into
some
brand-new
distributed
storage
platform.
And
ultimately
that's
that's.
That's
too
much.
We
have
a
bias
toward
simplicity.
We
want
to
keep.
We
keep
the
metadata
next
to
the
files.
We
do
index
that
in
a
separate
database,
but
we
can
database
technology
may
come
and
go
if
we
always
have
that.
Just
in
a
simple
block,
storage
device,
we
don't
replicate
or
we
don't
replicate
our
failure
immediately.
B
B
Horizontal
scaling
is
what
kubernetes
is
supposed
to
do
best,
but
we
found
that,
like
even
that
was
a
little
bit
too
daunting
for
everyone,
so
we
narrowed
it
in
to
the
fact
that
we're
trying
to
expand
to
more
more
front-end
developers
bring,
you
know,
expand
the
fold.
We
have
volunteers
that
want
to
contribute
little
bits
to
that
front-end.
So
Tracy,
yes,.
A
So
we
have
all
this
content
that
we
sort
of
mentioned
in
starter
showed
some
samples,
and
we
have
all
these
people
coming
to
visit
us.
So
we've
got
a
huge
amount
of
content,
diversity
of
content,
diversity
of
bringing
in
diversity
of
serving
it
out,
and
what
we
wanted
to
do
was
explore.
How
could
docker
and
kubernetes
help
us
out
and
modernized
based
on
our
own
on-premise
cluster
hardware,
all
the
way
down
network
dark
fiber?
That's
us
we're
actually
in
a
church
in
San
Francisco.
A
If
you
ever
want
to
come
by
and
see
a
fun
converted
Church,
we
have
a
lot
of
power
coming
into
this
church,
a
lot
of
power.
So
in
late
2014
we
started
using
docker
just
for
audio
fingerprinting,
so
this
is
kind
of
like
Shazam.
So
you
can,
you
know,
match
different
audio
segments
or
figure
out
what
song
something
is.
We
were
doing
it
for
something
else,
but
it
actually
kind
of
came
from
the
top
down.
So
it
was
a
little
controversial.
A
I
personally
was
a
little
bit
frustrated
by
the
way
it
was
introduced
to
the
archive
so
I
little
grumpy
next
year,
I
went
to
MOSFETs
in
London,
hosted
by
Mozilla.
I
went
to
this
awesome,
open
news,
docker
talk
by
the
Texas
Tribune
and
they
just
stepped
us
through
running
docker,
on
your
laptop
kind
of
showed
it
to
me.
I
said:
I
won't
touch
the
live
layer
ice
at
all,
cross-armed
and
thought.
I
was
going
to
hate
it,
but
I
loved
it
and
I
just
blew
my
mind
and
all
I
wanted
to
do
was
somehow
use.
A
Docker
I
think
a
lot
of
us
love
docker,
so
fast
word
another
year.
So
when
we
talked
about
the
audio
and
video
and
the
different
diversity
of
stuff,
you
can
give
us
more
or
less
any
audio/video
file
or
book
format
and
we'll
make
a
PDF
or
an
mp3
or
mp4,
so
that
took
us
about
a
year
to
switch
the
processing
back-end
to
just
use
docker
for
that
processing,
and
that
gives
us
give
us
a
lot
of
safety
and
less
rude
exploits.
A
And
then
you
know
if
if
something
goes
wrong,
it
just
goes
away
and
and
off
off
it
goes
so
that
was
a
huge
win
for
us.
So
almost
another
year.
This
is
right
before
Dave
and
I
started
to
sort
of
pairing
up,
but
we
started
giving
some
talks
about.
You
know
we
should
probably
think
more.
We
should
think
bigger
than
docker
to
think
about
kubernetes.
It
was
starting
to
really
take
off
and
we
were
really
excited.
A
So
he
gave
a
few
talks
and
they
went
pretty
well,
but
you
know:
there's
some
skepticism
I'm
more
of
a
developer,
David's
more
of
an
ops
person,
but
we
meet
in
the
middle
and
we're
getting
a
little
bit
of
resistance.
I
think
that's
pretty
common,
so
fast
forward
to
July
of
this
year,
and
things
really
pick
up
for
caranx
for
us.
A
We
had
a
lively
discussion
at
the
end,
we're
open
so,
if
you're
interested
in
even
seeing
it
and
want
to
see
some
inside
baseball,
probably
the
end
of
that
talk
at
the
bottom.
There
might
might
even
be
interesting,
but
it
went
really
well
because
we
are
able
to
basically
say
we're
just
going
to
take
our
existing
gitlab
on
premise
and
we're
gonna
extend
that
pipeline.
So
things
really
started
picking
up
really
quickly.
A
We
a
lot
of
our
code
runs
from
a
monolithic
repo
at
archive.org
and
it
has
different
kind
of
roles
throughout
our
servers
and
we
had
a
homegrown
test
phase
that
we
needed
to
move
or
wanted
to
move
into.
Auto
dev
ops,
we
move
that
in
August
and
then
about
later.
We
kind
of
kicked
on
the
rest
of
the
pipeline
and
started
doing
what
they
call
review
apps.
A
So
we'll
talk
about
that
a
little
bit
more
too,
but
that's
basically
a
developer
or
a
contractor
or
a
volunteer
can
have
a
branch
of
your
tree
and
every
time
they
push
that
branch
it
automatically
makes
a
website.
It's
fully
testable
experienceable,
it's
great
and
I.
Guess
that
would
be
October.
We
had
a
developer
who
hit
me
up
on
the
weekend,
who
got
all
excited
and
was
about
to
deploy
some
more
web
apps
and
he
was
just
like
I
want
to
learn
kubernetes
like
can
we
just
you
know,
and
he
just
knocked
it
out.
A
B
A
Application
same
side
of
the
coin
and
then
last
last
week
alone
we
just
moved
the
d
web
archive
org.
So
that's
a
decentralized
version
of
the
archive,
so
the
archive
is
kind
of
centralized
or
getting
that
sense.
Since
we
host
our
own
clouds.
This
is
a
way
to
host
materials
all
around
the
world
using
fun
things
like
web
torrent,
ipfs
gun
web
sockets,
all
this
kind
of
stuff,
and
we
were
able
to
containerize
that
and
put
it
into
kubernetes.
A
They're,
like
oh,
two,
relatively
relatively
a
very
smart
person
like
agreeing
on
things
and
having
interesting
discussions,
and
we
were
foils
for
each
other
and
we
could
help
each
other.
So
when
one
of
us
wanted
to
smash
our
face,
your
laptop
one
of
us
was
like:
oh
just
do
this,
so
it
was
really
really
helpful
that
really
accelerated
filament
and
just
made
everything
great
and
you
just
feel
like
you
have
a
maid.
So
it's
kind
of
like
a
little
love
fest.
You
know
the
archive.
The
other
thing
was
this
kit
lab
Auto
DevOps.
A
It's
it
be
easy
to
not
emphasize
enough
how
much
that
helps
with
the
archive,
because
all
we
had
to
do
was
say
we're
already
using
it
lab
all
we're
gonna
do
is
extend
this
homegrown
pipeline
to
a
full
pipeline.
We've
already
got
keys
now,
we've
got
audited,
Z
so
and
so
push
this
branch
push
this
into
production.
It's
wonderful.
B
The
other
big
thing
that
get
lab
brought
us
we'd
been
using
a
docker
hub
or
whatever
the
basic
docker
hub,
poster
and
docker
hub
registry
was-
and
this
made
me
very
nervous
because
there
wasn't
built-in
audit
trails
out
of
authentication.
Goodlove
just
took
care
of
all
that
I
couldn't
know.
I
can
now
trust
that
whatever
image
I
was
pulling
down
came
from
some
specific
developers
build
rather
than
I
know
shared
server,
yeah.
A
B
A
A
We
made
this
little
repo,
it's
public
called
create,
and
all
you
need
is
one
VM
with
SSH
access
and
route,
we're
we're
aiming
for
a
bun
see
right
now,
but
it
would
probably
work
in
most
UNIX
environments,
and
this
allows
you
for
hoping
and
like
this
other
developer
did
a
co-worker
can
now
try
it
out
pretty
easily
and
after
or
you
know,
with
some
coffee
and
see
what
they
think
and
one
thing
that
we
found.
That
was
helpful.
I'm
just
gonna
try
to
show
this
now
not
in
a
semi
live
demo.
A
Is
we
built
this
little
simple
tool
called
collar
and
it's
just
cube
CTL
getting
some
of
the
most
interesting
get.
So
it's
really
helpful
in
us
understanding
like
what
does
kubernetes
doing
what
it's
a
replica
set?
What
why
is
there
a
deployment?
It
seems
like
there's
all
these
repeated
repetitive
pieces.
Some.
B
A
Yeah
yeah,
we
like
the
GUI
thing,
but
we
showed
it
to
our
network
guy
and
it's
like
this
is
great
I
want
that.
So
so
it's
just
in
a
little
watch
loop
I
can
control,
see
it
will
repeat
and
I'm
just
going
to
go
ahead
and
delete
this
this
this
top
pod.
So
this
is
just
a
little
web.
App,
pretty
simple,
I've
already
got
it
ready
to
go.
A
I
will
delete
that,
and
what
you
can
see
here
is
because
it's
using
watch
it
will
just
highlight
all
the
different
regions
pretty
nicely,
and
you
can
see
this
thing
is
now
terminating
those
control
seat
once
they
saturate.
So
you
can
control
see
again
and
then
you
can
see
it
starts
coming
back
and
what
that
enabled
us
to
do
is
as
we're
bringing
up
clusters
or
having
issues
you
can
see
like
hey.
Why
is
this
thing
in
a
crash
loop?
A
B
This
is
the
sort
of
thing
that
I
would
feel
like.
This
is
my
job
I
got
to
figure
out
how
to
set
this
stuff
up,
but
there
was
there.
Was
it
really
was
daunting
to
try
to
install
your
first
cluster
and
Tracy's
enthusiasm,
we're
like
I?
Don't
care,
I'm,
gonna,
just
plow
through
this
until
I
get
it
working
really
really
helped
me
too
yeah.
A
And
then
later,
when
we
needed
persistent
volumes,
his
enthusiasm
and
plowing
through
figuring
out
the
technology
cluster,
then
Stephan
rook
was
unbelievably
helpful.
So
again
you
know
you
wanna,
that
avoid
bad
blood
as
Taytay
would
would
say
you
want
to
make
sure
that
you
don't
push
your
devs
people,
your
ops,
people
too
much.
You
want
to
make
sure
that
that
they're
feeling
included
and
you
want
to
identify
common
pain
points
and
what
helps
both,
and
you
know,
honestly,
some
of
the
things
that
we
thought
we
were
gonna
do
for.
A
The
capital
D
dog
said
okay
anyway,
so
one
other
thing
we
thought
would
be
interesting
was
it
would
be
to
show
sort
of
before
and
after
we're
warning
both
of
these
right
now.
But
we
had
a
nice
way,
a
relatively
nice
way
to
have
quick
development
at
the
archives.
So
my
UNIX
name
is
stress
is
Tracy
and
we
have
home
directories
on
a
bastion
server
and
those
are
exported
and
shared
throughout
an
entire
cluster
over
NFS
as
home,
and
what
it
allows
you
to
do.
A
Is
you
can
edit
directly
on
that
Bastion
I
sort
of
started
out
that
way
using
Emacs
and
then
eventually
moved
to
vs
code
on
my
laptop,
so
you
can
do
both
or
either-
and
you
know,
kicking
on
these
SFTP
kind
of
things
that'll
sink
over
a
safe.
So
you
get
immediate
feedback.
So
if
you
want
to
change
a
title
or
something
or
other
or
a
typo,
it's
very,
very
quick
and
we
use
nginx
virtual
host
to
pull
out
this
minor
magic.
It's
not
really
that
exciting.
A
But
if
you
come
to
archive
org
and
you're
on
a
node,
that's
been
load
balanced
to
it's
going
to
go
to
slash
pedal
box.
Ww
know
something
rather
that's
where
your
route
will
go.
If
you
come
to
Doug
Doug,
your
name
you'll
go
to
home
your
name
pedda
box
tree,
and
then
you
can
get
instant
feedback
that
that
it's
kind
of
cute
and
it
works
out
nicely.
But
once
you
start
having
multiple
branches
or
you
have
some
complexity
or
trying
to
bring
in
developer
other
developers
or
volunteers,
it
starts
getting
a
little
heavy-handed
and
hacky.
A
It
wasn't
really.
It
didn't
really
feel
like
it
was.
It
was
starting
to
feel
a
little
outgrown,
also
there's
the
issue
of
NFS
home
mounted
on
servers
and
just
giving
out
key
access
and
SSH
keys.
So
what
kubernetes
the
auto
dev
ops
excuse
me
from
get
lab
with
the
kubernetes
tie-in
it
will
it?
Will
you
just
punch
in
your
your
kubernetes
credentials
and
it
ties
right
in
what
that
will
do.
Is
that
allows
you
to
have
developers
make
these
branches
not
have
SSH
keys,
not
log
in
anywhere?
A
A
Is
great-
and
we
also
made
in
that
same
repo,
the
something
called
branch
sync
edit,
which
is
kind
of
like
a
simple
finder
pod-
it
syncs
the
code
to
it.
So
you
can
do
a
quick
edit
if
you
don't
want
to
do
the
entire
pipeline
and
and
I'll
hand
the
mic
to
Dave
in
a
second.
But
if
you
haven't
seen
pipelines
before
or
maybe
get
lab,
auto
DevOps
pipelines,
this
is
fairly
standard.
It's
a
little
different.
We
put
tests
in
front
of
build
we'll,
go
over
that
and
extend
an
expand
of
the
test.
A
But
this
is
exactly
what
you
can
expect
to
see.
You
will
go
through
a
build
step,
a
test
step,
a
review
step
where
actually
pushes
it
out
to
that
review
server
and
then,
if
you
either
remove
or
merge
the
branch
or
you
want
to
just
manually-
stop
the
review
then
that
asset
will
go
away
and
by
the
way
they
pull
this
off
with
a
DNS
wildcard
trick,
which
makes
it
really
nice
that
they
can
come
up
with
any
name
the
audience.
So.
B
That
was
that
really
helped
out
our
front-end
developers
gave
them
like
a
bunch
of
obvious
benefits,
but
I
also
wanted
to
find
out
what
wanted
to
test
this
theory
that
this
could
make
other
kinds
of
application
development
a
lot
easier.
So
I
took
our
existing
crawling
I'll
just
quickly
review
like
what
crawling
does
it's
pretty
simple?
You
start
with
a
bunch
of
a
list
of
websites
called
seeds,
you
go
out
and
you
download
a
bunch
of
websites.
You
extract
links
from
that
and
then
what
do
you
have
now?
B
You
have
another
list
of
websites
that
you
keep
calling
so
you're
just
going
to
keep
doing
this
over
and
over
again
and
so
back
in
2009
Internet
Archive
developed
its
own
Java
open
source
Karl
are
called
Hara
tricks,
but
it's
designed
for
it's
a
jape.
It
you
know,
runs
on
the
JVM.
It
really
is
designed
for
big
machine
and
once
the
web
grew
you
know
billions
and
trillions
of
URLs.
Now
one
machine
wasn't
good
enough.
B
B
The
initial
design
of
this
was
pretty
straightforward
that
you
get
in
that
loop.
You
aggregate
all
of
the
URLs.
You
have
some
kind
of
store
that
says
have
I
seen
this
URL
in
this
crawl
cycle,
yes
or
no,
because
we
don't
want
a
thousand
copies.
I
didn't
want
to
fetch
a
thousand
copies
of
the
Google
logo.
Every
time
we
crawl,
and
so
they
just
sits
in
the
middle
of
scroll
HQ
tool.
So
I
I
took
that
on
as
well.
Let's,
let's
figure
out
how
we
can
deploy
that
in
kubernetes.
B
So
it's
real
easy
to
replace
38,000
lines
of
in-house
code
with
a
bunch
of
other
people's
code,
but
Kafka
and
foundation
to
be
really
fit
the
bill
for
for
the
volume
of
work
that
we're
that
we
were
working
with.
So
the
the
discovered
URLs
just
go
into
a
Kafka
Kafka
topic,
then
there's
a
little
worker
that
has
to
say
well.
Have
we
seen
this
yet
and
it
uses
foundation
DB
for
that
store
and
I
could
have
run
like
I
could
have
run
this
like
a
bunch
of
processes
on
a
few
different
VMs.
A
And
it's
actually,
we
have
petabytes.
The
code
is
gigabytes,
but
we
just
thought
that
would
be
fun
to
show
or
that's
actually
one
of
our
real
storage
racks.
So
you
know
we
could
break
up
things.
We
could
kind
of
come
up
with
a
schedule
and
spend
a
lot
of
time,
but
ultimately
because
I
like
to
smash
my
head
and
things
I
just
thought
like
well
what
if
we
can
actually
Jack
a
huge
monolith
of
code
in
and
make
it
work
somehow.
So
it's
just
a
little
background.
A
What
our
developers
would
get
feedback
within
seconds-
or
you
know,
maybe
a
minute
or
two,
as
opposed
to
waiting
for
the
entire,
build,
we're
able
to
pull
that
off,
because
we
start
with
a
baseline,
a
docker
image
with
a
shallow
clone,
and
that
helps
with
being
able
to
jump
forward.
We're
going
to
talk
about
that,
because
what
we
found
is
out
of
the
can.
The
the
initial
way
that
get
lab,
auto
develops,
which
might
change
in
the
future,
is
doing
things
is
it's
doing
a
git
clone
parfaits,
and
that
was
just
a
non-starter
for
us.
A
We
were
waiting
for
minutes
parfaits,
you
know
times
10
or
20.
You
know
sometimes
in
parallel,
but
it
was
just
way
too
too
much
so
so.
Give
you
just
a
little
bit
of
more
background.
Our
code
base
is
peda
boxtree
and
we
make
something
we
call
a
pet
of
base
image.
We
don't
have
to
go
through
all
this,
but
it's
basically
a
baseline
docker
image.
A
Some
of
the
other
images
we
were
making
again.
We
won't
have
to
go
in
all
this
detail,
but
so
when
I
was
talking
about
the
mp3s
and
mp4
making
PDFs
before
that's
this
left
side
here,
so
that's
will
make
the
work
base,
which
has
additional
packaging
and
things
like
that.
That's
third
party
and
then
we'll
make
worker
on
top
of
it.
But
all
of
these
endpoints,
like
for
the
website
kubernetes
candidate,
for
testing
on
your
laptop
or
for
miscellaneous
nodes.
A
It's
all
using
the
same
trick
start
with
Petta
base
fast
for
it
in
yeah,
and
so
when
you
get
shallow
clone
for
us
at
least
we
found
we
could
drop
our
tree
size
by
1/3.
So
that
was
significant
enough
to
really
speed
up
our
times.
Mm-Hmm
it
and
anything.
That's
going
to
go
ahead
and
use,
it
is
going
to
ask
for
a
specific,
commit
and
fast
forward
right
to
that
and
we'll
go
over
that
in
just
a
second
and
then
to
speed
up
our
build
phase.
A
We
found
using
the
prior
build
phase,
which
was
healthfully
right
in
the
gate
lab
doctor
registry,
just
pull
from
that
bootstrap,
because
we're
mostly
just
doing
code
changes
and
go
from
there
that
really
sped
things
up
all
right,
so
you
might
be
on
your
phone.
That's
great!
It's
it's
getting
towards
the
end
of
the
conference!
If
you
are,
you
might
want
to
take
we're
gonna
hold
up
your
phone
and
take
a
picture.
The
next
slide,
because
how
do
you
fast
for
these
get
shallow
clones?
A
It's
surprisingly
difficult,
surprisingly
difficult
I
was
working
on
this
for
weeks,
I
was
not
crying
exactly,
but
I
was
getting
Tomatoes
hurled
at
me.
The
devs
were
really
upset
okay.
So
what
this
does
in
a
nutshell,
is
it
takes
that
get
shallow
repo?
It
does
a
check
out
detach
that
just
kind
of
keeps
it
out
of
the
way
in
case
there's
anything
locally
modified
or
don't
keep
you
out
of
trouble
and
the
magic
here
really.
A
Is
this
get
fetched
depth
one
using
the
origin
of
the
branch
from
the
origin,
the
branch
or
the
commit
you
want.
So
what
that
does?
Is
that
keeps
your
your
repo
shallow?
Because
if
you
just
did
a
get
fetch
it
would,
it
would
basically
fill
out
the
entire
repo
and
you
now
you've
got
to
pay
four
minutes
of
penalty,
and
then
you
can
just
check
out
immediately
to
it.
A
Now
it's
worth
pointing
out
this
sort
of
fell
off
screen,
but
there's
a
git
lab
on
Prem
config
setting
that
will
allow
you
to
get
to
any
commit
because
I
think
there's
a
concept
of
you
know
you
might
want
to.
You
might
decide
a
commit
had
some
sensitive
information.
You
want
to
remove
it,
but
it's
still
kind
of
there.
So
there's
things
like
that.
There's
how
much
time
you
can
get
back
if
you
want
to
jump
six
months
forward
or
something
like
that.
So
if
you
set
this
one
little
setting,
you
can
see
the
PDF.
A
If
you
want
to
see
the
full
thing
that
that's
the
other
magic-
and
this
is
this-
is
really
because
we
have
this
guy,
Mark
Sullivan,
so
huge
shout-out
to
him
he's
a
huge
get
pro.
If
you
need
to
get
pro,
maybe
talk
to
us
because
he's
he's
amazing,
some
other
tap
top
tips.
Sort
of
wrapping
up
here
are.
If
you
want
to
try
like
a
kubernetes
with
one
or
two
nodes,
you
can
have
a
single
node
and
taint
the
master
and
say
I
want
to
run
pods
just
keep
it
simple.
A
You
can
use
local
storage
and
that's
a
huge
time
savings,
because
you
don't
have
to
figure
out
how
to
use
something
else.
If
you
want
to
just
run
it
on
your
laptop
or
whether
it's
it's
a
nice
little
win
get
lab.
Lotto
DevOps
comes
in
with
something
called
Postgres.
It
comes
in
with
a
an
empty
Postgres
database
trying
to
be
very
helpful,
but
that
makes
persistent
volume
claims
that
you
don't
need.
If
you're
not
going
to
use
it
and
by
default,
you
probably
won't
use
it
so
recommend.
A
Turning
that
off
in
this
little
CI,
CD
CD
very
variable,
it's
right
in
the
GUI
and
then
the
get
lab
API
just
a
shout-out
to
it.
Like
everything
you
see
on
get
labs
website,
you
can
pretty
much
get
to
you
through
the
API,
so
we
could
do
pretty
advanced
things
like
saying
where's,
the
most
recent
master
branch
has
it
succeeded
through
test
or
failed,
and
our
devs
can
instantly
find
that
on
a
page
and
know.
A
Is
it
ok
to
push
out
periodically
you
might
want
to
remove
some
of
your
old
replica
sets
on
the
kubernetes
side
we
found
when
you
host
it
I'm
sure
the
cloud
providers
do
some
of
this
for
you
and
we
found
that
when
you
want
to
probably
trim
your
registry
periodically,
because
it
just
sort
of
builds
up
one
one
very
last
thing
this
is
really
in
the
weeds
but
4d
web.
We
needed
to
expose
all
sorts
of
fun
and
interesting
ports.
A
We
had
a
hard
time
with
the
with
any
of
the
ingress
'iz
and
it
was
baffling
I
spend
almost
a
week
on
that
and
David
jumped
in
and
tried
to
help
that
did
help,
and
basically
you
can't
do
it
at
least
I
couldn't
find
any
way
to
do
it.
So
you
just
make
a
separate
load
balancer
and
it's
just
two
lines
of
code.
It
figures
out
what
the
services
figures
out,
what
the
exports
are,
and
it
just
handles
all
those
ports
for
you.
A
So
that
was
a
really
nice
thing
not
allowed
the
the
T
web
to
go
out
after
some
delays.
A
little
tip
of
the
hat
tip
of
the
hand
we're
going
to
try
to
hopefully
knock
on
wood,
move
the
archive
or
web
production
website
to
kubernetes,
because
right
now
we
have
a
static
list
of
the
ends
with
H
a
proxy
and
we'd
like
to
be
smarter
err
and
move
to
something
like
kubernetes
with
elastic
ceiling.