►
From YouTube: TGI Kubernetes 138: Harbor
Description
Come hang out with Tiffany Jernigan and Paul Czarkowski as they explore Harbor! As usually we'll talk about what's going on in the cloud native space, then transition to Harbor..
00:00:00 - Welcome to TGIK!
00:03:40 - Week in Review
00:21:00 - Installing Harbor
00:38:51 - Harbor Achitecture
00:44:29 - Vulnerability Scanning
00:48:30 - Creating a new Project / Repository
00:50:23 - Create robot account & kubernetes pull secret
01:07:20 - Uploading and Sharing Helm Charts
01:23:43 - Proxying through to dockerhub
01:35:55 - Replicating from another Harbor
01:50:00 - Wrapup
tanzu/tgik/tree/master/episodes/138
A
Hopefully,
you
can
hear
me
and
tiffany
hi
hello.
How
are
you.
B
Hi
there
doing
pretty
good
it's
a
nice
sunny
friday
in
seattle,
no.
A
Surprise,
great
I'm
glad
so
welcome
to
this
is
your
first
time
on
tgik
right.
A
You've
been
a
you've
been
a
long
time,
troll
first
time,
caller
yeah,
basically
exactly
that,
basically
all
right
well,
this
is
only
my
second
time
hosting,
so
I'm
not
exactly
a
veteran
myself
but
welcome
everybody.
We
are
so
we're
both
pretty
new
to
this
and
we
are
trying
to
do
a
dual
stream.
So
if
we
have
weird
technical
difficulties,
please
bear
with
us,
but
I'm
already
seeing
people
saying
they
can
hear
us.
That
is
a
that
is
a
good
start.
A
So,
martin
from
the
netherlands,
hi
how's
it
going
jonas
in
boston,
knoll
in
india,
eric
in
seattle,
the
tac.
You
know.
B
A
I
kind
of
miss
referring
to
places
by
their
airport
codes.
Now
I'm
not
really
traveling.
I
don't
really
do
that
anymore.
Riv
from
boston,
hey
how's,
it
going
michael,
michael,
the
very
famous
michael
michael.
He
says
that
he
can
hear
us.
So
that's
always
good
walid,
hey
how's!
It
going
you're
a
regular
as
well
juka
from
helsinki,
wow
and
there's
a
tasha
eisenberg.
Do
we
know
it
tasha
eisenberg
tiffany.
A
And
then,
of
course,
josh
russo
is
here
long
term
long
time
host
of
the
show,
marcus
and.
B
A
Great
graceful
enough
to
step
aside
and
let
a
couple
of
amateurs
run
the
place.
So
thank
you
and
we
apologize
for
we'll.
A
Alex
from
sunny
northern
california
tim,
hey,
how's
it
going
eric
dfw
nice
we're
almost
neighbors.
We
just
have
a
long
stretch
of
I-35
between
us
all
right,
so
oh
wences
is
from
dallas
as
well,
and
we
have
ismail
from
turkey,
nice
and
bogdan
from
bucharest.
A
So
we've
got
a
as
usual
a
pretty
international
audience,
thanks
to
all
the
folks
in
far
off
time,
zones
that
are
either
up
late
at
night
or
very
early
in
the
morning.
We're
very
grateful
took
the
time
to
join
us
tiffany
and
I
are
going
to
be
doing
an
episode
on
harbor.
It
has
been
a
long
long
time
requested
sitting
in
the
backlog,
and
I
guess
maybe
because
it's
kind
of
a
project
that's
been
out
of
vmware.
A
We
sort
of
have
taken
a
little
bit
for
granted
in
tgik.
So
I'm
glad
we're
actually
getting
up
in
front
of
folks
and
we're
gonna
go
through
it.
Tiffany
you're
sharing
your
screen,
so
you
are
in
control
of
where
we're
at,
but
we're
at
the
notes
page.
So
maybe,
let's
have
a
look
at
our
our
news
and
review
our
week
in
review.
I
should
say.
B
Yep
so
first,
let's
take
a
look
at
the
things
for
core
kubernetes.
So,
as
some
of
you
may
know
or
may
not,
kubernetes
is
now
moving
to
three
releases
per
year
and
you
can
actually
add
some
your
input
on
that.
If
my
internet
would
go
there,
we
go
so
basically
as
just
a
open
issue
and
describes
everything,
that's
kind
of
happening
there,
lots
of
votes,
and
if
you
want
to
comment,
you
can
be
one
of
the
people
here
at
your
comments
or
ad,
whether
thumbs,
ups
etc.
A
By
the
way,
folks,
if
you
like
to
play
the
tgik
drinking
game,
we
no
longer
get
to
drink
every
time,
duffy
jangles
his
keys
or
coins,
or
whatever
it
was,
but
you
can
definitely
take
a
drink
every
time.
Tiffany
complains
about
the
slow
speed
of
her
internet
or
her
computer
and
you'll
probably
get
quite
drunk
by
the
end
of
the
year.
B
B
Yeah,
maybe
I
should
everyone's
gonna
be
drunk
but
yeah.
Maybe
I
should
have
restarted
my
internet
before
this.
We
should
test
this
out.
I
will
let
you
comment
to
this
one
paul.
If
you
know
much
about
it,.
A
A
I
actually
had
to
look
it
up
on
the
the
wikipedia
earlier
when
I
saw
this
because
it
looked
familiar,
but
I
couldn't
remember
it
and
at
first
I
thought
we
were
getting
sftp
support
and
I
was
very
concerned
about
what
we
were
doing.
I
thought
maybe
we're
gonna
get
like
a
storage
storage
class
or
something
based
on
ftp,
and
I
don't
think
I
would
have
liked
that
nearly
as
much
but
yeah
so
sctp.
I
think
we've
got
a
link
to
say
what
scp
sctp
is.
A
I
can't
even
I
can't
even
say
the
initials,
so
good
luck
actually
knowing
what
it
is
but
we're
bringing
this
in,
and
so
I
think
this
will
allow
like
load
balance
load
balances
in
kubernetes
to
support
doing
the
load
balancing
for
the
networking
yeah.
So
there
is
that
what
else
have
we
got
on
there?
Tiffany.
B
B
Okay,
so
then,
now
like
outside
of
just
the
core
part
of
kubernetes,
so
we
have
the
pvc
auto
resizer.
So.
A
So
that
that's
actually
pretty
exciting
for
me,
I
will
often
make
a
really
bad
guess
at
how
much
space
I
need
on
a
pvc,
and
then
I
freak
out
because
I'm
stuck
with
it
actually
a
good
example
is
using
something
like
harbor
right
you're
like
oh.
A
I
only
need
a
hundred
gig
for
my
images
and
then
like
a
week
later,
you're
like
oh,
no,
I
needed
more
than
that,
and
so,
instead
of
trying
to
like
back
it
up,
create
a
new
one
and
restore
it,
I
guess
we
can
use
a
resizer.
A
C
A
Csi
drivers
will
support
volume
expansion
because
that's
something
we've
been
doing
with
our
our
sands
and
our
nasa's
for
a
long
time.
So
I
would
assume
that
all
the
major
clouds
will
provide
it
and
most
of
the
like
in
data
center
providers
will
support
it.
So
that's
that's
a
good
thing.
I
wonder.
Is
it
a
oh?
It's
an
auto
resizer.
So
I
wonder
if
you
set
like
a
maximum
and
minimum,
does
it
have
anything
about
the
spec
in
the.
C
A
How
much
is
being
used
and
maybe
even
the
like
speed
at
which
it's
being
built
so
and
then
we
have
the
resize
thresholds.
C
A
Looks
very
much
like
the
like
the
horizontal
pod,
auto
scaler,
except
for
storage.
As
far
as
how
it.
A
So,
that's,
that's,
that's
pretty
cool,
I
wonder,
can
you
can
you
just
edit
a
pvc
and
resize
it?
I
don't
think
I've
ever
tried.
A
A
A
All
right,
so
you
can
size
up,
you
can't
size
down
which
makes
sense.
Yeah
it'd
be
pretty
bad.
If
you
could
say,
if
we
get
excited
and
we
get
through
the
harbor
stuff,
we
can
just
try
resizing
one
of
the
harbour
volumes
and
see
what
happens.
C
A
A
Well,
let's
see
what
else
we
got
going
on
here
so
that
is
auto
resizing
for
pvcs.
A
So
isn't
this
waypoint
is
what
josh
did
on
last
week's
tgik
right.
A
So
I
guess
this
is
just
a.
I
guess.
A
C
A
Good
to
see
more
activity
around
waypoint
folks
trying
to
figure
out
where
and
how
it
might
fit
into
their
tooling
ecosystem.
Obviously
everyone
loves
them.
Some
hashi
products
going
all
the
way
back
to
like
vagrant.
C
A
You
know
what
do
you,
what
what
are
your
thoughts
on
the
fact
that
they
have
their
own
markup
language
is
it?
Is
it
hcl?
B
A
Gotcha
yeah.
So
while
it
said
that
he's
a
big
fan
of
hcl,
because
you
can
put
comments
in
it
and
I
agree
and
in
fact
it
feels
very
much
like
a
like
a
slightly
easier
to
read
json
that
lets.
You
also
put
comments
in
there
and
yeah.
A
I
don't
know
where
I
was
going,
I,
but
let's,
let's
move
on
so
let's
have
a
look
back
at
the
episode
and
see
what
else
we
have
to
talk
about.
A
Oh
yeah,
so
there's
been
a
lot
of
activity
in
the
helm
community
recently
trying
to
figure
out
like
these
we
announced
we
were
going
to
decorate
use.
My
use
my
words
we're
going
to
deprecate
the
community
repos
in
about
a
year
ago.
So
it's
been
about
12
months
and
the
main
reason
was
that
the
drain
and
burnout
on
the
chart
maintainers
trying
to
keep
up
with
the
the
charts
and
updates
of
the
charts.
A
You
know
there
was
three
or
400
charts,
some
of
which
were
very
active
and
some
of
which
were
very
large,
and
it
just
meant
that,
like
the
chart,
maintainers
were
just
constantly
triaging
the
the
charts
and
didn't
really
have
a
great
time,
and
so
we
sort
of
made
the
decision.
We
were
going
to
decentralize
it
and
we
also
got
the
artifact
hub
recently,
which
made
it
a
lot
easier
for
us
to
decentralize.
A
So
what
we
did
is
we
sort
of
announced
we
were
going
to
deprecate
the
charts,
but
we
sort
of
found
that
a
lot
of
folks
were
still
using
them
and
so,
rather
than
just
the
original
plan
of
deleting
them
come
the
end
of
november,
which
was
kind
of
not
necessarily
the
community's
choice,
but
more
of
a
someone
had
to
pay
for
it
choice,
and
so
we
ended
up
with
at
the
last
minute.
A
Github
came
to
the
party
and
said
that
we'll
help
you
host
a
read-only
replica
of
of
the
chart,
repos
and
so
we've
built
that
out
in
in
github,
just
in
github
pages,
actually
and
basically
come
the
13th
we'll
be
switching
across
to
those
and
if
you're,
using
an
older
client
you'll
just
need
to
follow
the
instructions
we
have,
and
I
think
I
I
link
to
it
on
how
to
switch
to
use
the
reader
only
repositories.
A
But,
of
course,
if
you
are
using
the
helm
chart
repositories,
your
best
bet
is
to
go
to
the
artifact
hub
and
look
for
who
now
owns
the
non-deprecated
version
of
that
chart.
So
a
lot
of
charts
have
gone
to
whoever
owns
it
and
then,
of
course,
we
have
bitnami
and
other
folks
that
are
hosting
some
really
great
sets
of
curated
charts
and
so
before
you
just
switch
to
using
the
read-only
replicas.
I
highly
recommend
seeing
if
you
can
switch
to
like
nginx,
is
now
hosting
their
own
charts,
etc.
A
So
yeah,
I
think,
that's
probably
all
of
the
helm
stuff
pretty
important,
because
it
is
widely
used
at
a
lot
of
companies
and
it's
been
hard
to
really
get
the
word
out
about
the
upcoming
deprecation
and
we're
trying
the
helm
community
is
trying
their
best
to
have
it
not
be
a
huge
surprise,
so
yeah
that
is,
that
is
our
helm
stuff.
What
else
have
we
got
on
that
list?.
A
A
A
B
And
if
anyone's
unfamiliar
eks
is
amazon's
managed
kubernetes
offering.
A
Amazon
resources:
I
wonder
if
this
wraps
the
operator
under
cluster
api
or,
if
they're,
using,
if
not
they're,
probably
using
the
same,
like
libraries
and
sdks.
B
A
I've
used
the
like
the
google
operator
a
fair
bit
and
it
is
kind
of
cool
to
be
able
to
do
like
hey
kubernetes,
give
me
a
gke
cluster
and
now
that's
not
using
cluster
api.
But
I
would
imagine
that
it
must
be
on
its
way
if
it's
not
there
already
right,
because
if
it's.
A
A
Right
now,
I
actually
had
a
an
architectural
diagram
to
stick.
A
I
did
and
then
I
forgot
about
it
yeah
there.
It
is
right
there.
How
do
I
get
it
to
you?
I
can
probably.
B
A
With
that
sort
of
stuff,
because
they
have
loki-
which
is
this
sort
of
a
similar
thing
for
logging,
that
sort
of
gives
you
an
alternative
to
the
elk
stack,
which
I
find
really
appealing,
because
it
can
be
just
kind
of
a
thin
shim
around
your,
like
object,
storage
plus
a
database
to
store
your
indexes
in
which
is
a
lot
easier
to
tackle
than
building
out
a
massive
elastic
search,
cluster
and
keeping
the
replicas
working
etc.
A
So
I
always
enjoy
seeing
the
tools
that
the
grafana
folks
come
out
with.
So
I
just
pasted
into
the
notes
the
architectural
overview,
so
when
we
get
to
that,
hopefully
we
can
pop
through
that.
So
this
isn't
the
current
version.
This
is
2.0
the
calibration.
B
A
But
there's
not
a
ton
of
differences,
so
we
should
be.
We
should
be
good
and
I
just
thought
we
could
have
this
here
in
case
we
needed
to
look
at
the
actual
like
how
all
the
components
work
together,
but
tiffany
do
you
want
to
just.
I
guess
maybe
take
us
through
like
a
basic
introduction.
What
harbor
is
what
problem
it's
trying
to
solve.
B
Sure,
well,
first
off,
if
you
haven't
actually
like
looked
at
it,
you
can
just
go
to
hard
to
read
from
here.
I
don't
I
don't
know.
C
A
Text
on
the
screens
are
hard
to
read,
please
let
us
know
and
we'll
try
to
zoom
in
okay,
cool.
B
Okay,
that
was
the
architecture
map,
so
we'll
worry
about
that
later
yeah.
So,
basically,
hardware
is
created
as
like
being
an
open
source
registry.
You
can
have
like
images
in
there.
You
can
have
helm
charts
that
you
created
or
pulled
from
somewhere
in
there.
B
You
can
also
have
like
your
private
and
public
registries,
there's
the
idea
of
how
you
can
have
different
projects
within
it
and
kind
of
separate
things
out.
A
bit
more
versus
like
if
with
docker,
there's
one
single
place
that
you're
pushing
everything
into,
is
there
anything
else
that
you
think
should
be
added?
To
that
I
mean.
A
I
think
that
gets
that
gets
a
bunch
of
it.
I
I
guess
I
would
probably
add
that
it
does
kind
of
target
the
enterprise
use
case
and
therefore
it
does
support
things
like
replication
and
actually,
while
it
just
mentioned
it,
things
like
replication,
proxying
and
mirroring,
I
also
is-
has
really
strong
support
for
like
external
authentication.
A
So
if
you
want
to
back
your
authentication
to
your
like
ldap
or
active
directory,
you
can
do
that
if
you
want
to
back
it
against
your
like
an
an
oidc
slash,
oauth
server
like
you
could
back
it
against
like
github
auth
or
google
auth.
If
you
wanted,
and
so
that's
that's
super
appealing
and
I've
used
that
before,
and
it
works
really
well
and
actually
marcos
brings
up
a
really
good
point,
and
actually
it's
really
timely.
A
We're
doing
harbor
right
now,
because
docker
is
starting
to
enforce
limits
this
weekend
and
as
marcos
said,
it
is
the
perfect
time
to
be
thinking
of
harbor.
A
A
Nice
and
ansel
says
he's
using
harbor
as
a
proxy
in
his
home
lab.
That's
a
great
idea.
You
know:
saving
saving
your
internet
data,
if
you
don't
especially,
if
you
don't
have
high
speed
internet
or
you
have
meted
internet
and
some
isp
is
charging
you,
you
know
extra
extra
gigabits
or
gigabytes
or
giga
dollars,
maybe
for
all
these
docker
pools,
you're
doing.
A
Immanuel
asks:
is
there
any
plan
to
enable
the
pull
cash
proxy
service
without
running
two
instances
of
harbor,
I'm
not
actually
sure,
but
I
do
know
we
have
some
folks
from
the
harbour
project
in
chat.
So
hopefully
one
of
them
can
speak
up
and
answer
some
of
the
more
specific,
deeper
harbour
questions
and
alex
indeed
has
said.
We
only
need
one
instance
and
michael
michael's
replying
with
the
same
thing
cool.
Does
it
support
arm?
I
I
don't
know
I
I
guess
it
would.
If
the
images
have
been
built
for
arm.
A
I
I'm
gonna,
take
a
guess
and
say
that
they
haven't,
but
I
don't
that's
not
a
definite
and
ben.
It
is.
B
A
It
is
absolutely
possible
to
make
harbor
h
a
and
we'll
talk
about
that
a
little
bit
and,
in
fact,
tiffany.
Why
don't
you
click
through
to
the
installation
and
configuration
guide,
because
I
believe
it
mentions
doing
h.a
there.
A
A
Few
times
on
the
harbor,
but
there
we
go.
You
also
use
helm
to
install
harbor
on
a
kubernetes
cluster
to
make
harbor
highly
available.
So
why
don't
you
click
through
on
that
deploying
harbor
with
high
availability
link.
C
B
A
Yeah
helm
two
is
is
not
something
you
should
be
using
in
2020.,
so
ingress
control
the
so
the
ingress
makes
a
harbor
sort
of
h.a
from
a
web
web
front
end
point
of
view,
and
then
you
can
tell
helm
to
use
an
external
postgres
database
and
an
external
redis,
and
so
that
lets
you
have
h
a
on
your
databases.
A
It
doesn't
make
a
lot
of
sense
for
harbor
to
try
and
run
h.a
postgres
and
h.a
redis,
because
you'd
end
up
with
like
very
complex
helm,
charts
and
then,
of
course,
using
pvcs
for
storage
for
any
of
the
harbour
bits
using
object.
Storage
for
the
actual
artifacts
is
all
going
to
help
you
get
to
and
then
once
you've
done
that,
like
the
various
components
like
you
can
run
multiple
clear
pods,
you
can
run
multiple
of
the
core,
which
is
like
the
api.
A
You
can
run
multiple
of
the
registry
server
and
to
do
that
in
your
helm,
values
file,
you're,
literally
just
changing
how
many
replicas
you
want.
So
it
is
pretty
easy
to
do
so.
So
as
long
as
you
can
get
access
to
database
and
redis,
which
I
think
most
clouds
will
have
some
kind
of
service
you
can
tie
into
and
then
alex
says
the
is
leveraging
the
natives
native.
C
A
A
So
let's
have
a
look
at
their
instructions,
so
download
chart
and
then
configure
it,
and
these
are
kind
of
the
the
bare
minimum
things
you
want
to
do
right.
So
you
need
to
set
the
hosts.
So
your
core,
which
is
kind
of
the
main
entry
point
to
helm
and
notary,
have
different
entry
points,
and
so
you
need
to
set
those
and
then
tell
it
where
your
database
is
and
where
redis
is
and
set
up
your
credentials
etc
and
that's
kind
of
it.
C
A
A
Across
to
okay,
let's
have
a
quick
look
at
the
install
skip,
pass,
helm
2
because
we're
not
doing
helm2,
so
helm3
right,
really
simple:
helm,
install
my
release
dot.
Now
what
they're
doing
in
these
instructions
is
they're
telling
you
to
pull
down
the
entire
chart
and
modify
the
values
file
directly?
The
alternative
is,
of
course,
is.
A
You
could
create
a
separate
values
file
and
do
a
helm
install
from
the
helm
repo
and
that's
the
steps
that
I
took
when
we
installed
helm
on
the
cluster
that
we're
going
to
be
using
when
we
sold
harbor
on
the
class
we're
going
to
be
using
so
tiffany.
Do
you
want
to
bring
up
your
vs
code
and
we'll
take
a
look
at
how
we
got
our
infrastructure
running.
B
A
A
I
I
have
this
running
on
a
gke
cluster,
and
so
you
can
see
I
installed
contour
as
an
ingress
controller,
and
then
I
installed
cert
manager.
So
I
grabbed
the
crds
created
some
name
spaces
and
then
created
some
secrets
with
google
credentials
that
only
have
access
to
dns
to
modify
my
dns
settings
and
then
I
add
the
jet
stack
and
bitnami
helm,
charts
and
install
cert
manager
and
install
external
dns
with
a
typo.
A
A
So
you
can
see-
that's
really
simple.
Dns
external
dns
just
needs
to
know
about
my
google
credentials
so
that
it
can
manage
my
dns
and
then,
if
you
click
on
cluster
issuer.
A
So
here
I
created,
I
think
three
cluster
issuers,
let's
encrypt
and
let's
encrypt
staging
and
did
I
do
a
self-signed
one
as
well.
A
Yeah
and
I
did
a
self-signed
one
as
well
yeah
and
that
way,
as
I
was
deploying
it,
I
wasn't
deploying
with
real
cert
manager
certs,
because
I
didn't
want
to
hit
the
let's
encrypt
limits
on
their
real.
So
I
got
it
all
working
with
self-signed
and
then
I
switched
to
let's
encrypt
prod,
and
that
way
I
get
real
certificates
and
that's
pretty
important
for
your
registry,
because
the
last
thing
you
want
to
do
is
have
to
fight
your
google
class.
A
Google,
your
kubernetes
cluster,
to
get
your
cas
up
into
your,
like
your
data,
plane
nodes,
your
worker
nodes.
So
they
trust
your
registry
right
because
you
need
a
valid.
You
know
you
either
need
a
valid
cert
or
you
need
to
inject
your
ca
up
into
your
data
plane
and,
if
you're
running
your
own
cube
cluster,
that's
not
too
hard
to
do
because
you're
installing
kubernetes,
but
if
you're
on
gke,
if
you're
on
aks,
you
don't
necessarily
have
easy
access
to
to
do.
C
A
A
The
cluster
to
get
the
the
certs
up
there,
so
that
was
kind
of
my
infrastructure,
so
there's
not
too
much
needed,
but
those
things
definitely
help
and
then
so,
if
you
click
on
the
values
harbor,
we
can
have
a
look
at
how
I
set
up
my
helm.
Chart
values
for
harbor.
Now
I
left
the
admin
password
in
there,
which
I
don't
mind
you
you're,
seeing
because
nobody's
gonna
do
any
horrible,
goose
activities
and
hackers,
and
also
I'm
going
to
destroy
this
infrastructure
as
soon
as
we're
finished
here.
A
So
we're
on
the
honor
system.
Don't
hack
us,
but
you
can
see.
These
are
all
value
files
that
get
rendered
into
the
home
templates.
So
I'm
basically
saying
I
want
ingress
enabled
and
I
will
use
tls
and
I'm
going
to
provide
my
own
certs
and
then
you
can
see.
As
the
docs
mentioned,
I
supply
the
ingress
hosts,
so
harbour
dot
and
notary
dot,
and
then
I
pass
in
some
annotations,
and
so
I
always
want
to
make
sure
that
people
use
https,
and
so
I
pass
in
the
force
ssl
redirect.
A
So,
if
especially
with
nginx,
if
you
leave
the
defaults
on
the
proxy
body
size
and
a
few
other
settings,
you
end
up
not
really
being
able
to
do
much
because
it
breaks
through
like
after
a
couple
like
after,
like
20,
meg
or
so
of
image,
size,
and
so
you've
got
to
be
very
careful
that
whatever
ingress
you're
using
you
configure
to
allow
it
to
pass
through
larger
images,
then,
because
I'm
using
contour,
I'm
passing
in
my
ingress
class
in
case
there's
another
ingress
controller,
and
then
I'm
setting
my
cert
manager
cluster
issuer
to
use
it
let's
encrypt.
A
So
this
is
using
the
the
real
one
and
then
the
rest
of
it
is
sitting
setting
some
persistence
so
taking
some
guesses
at
the
sizes.
I
need
for
the
database
and
redis
as
well
as
setting
up
google
cloud
storage,
as
my
as
my
backing
service
for
artifacts,
and
that
way,
if
I
need
to
destroy
this
infrastructure
and
spin
it
up
again,
at
least
my
artifacts
are
still
sitting
there
in
the
back
end,
so
that's
kind
of
how
we
got
the
install
going
and
then
tiffany.
A
So,
like
home,
repo
ad,
create
the
namespace
and
then
helm,
install
and
pass
in
that
values
file.
So
pretty
simple.
It
took
like
five
or
ten
minutes
to
to
get
installed
and
running
because
obviously
it
takes
a
while
for,
like
a
postgres
database
to
start
in
the
redis
database
to
start
and
so
there's
some
cascading
dependencies
that
sort
of
need
to
resolve
themselves
on
the
cluster
to
get
everything
working.
But
when
it
is
all
working,
we
then
end
up
with
a
do.
B
A
Just
yeah
our
t-mac
session,
so
we
can
see
there.
A
Yeah,
it's
all
up
and
running
here.
Let
me
let
me
let
me
do
some
some
activities,
so
we
do
get
harbor.
Oh.
B
So
if
you
don't
know
what
octane
is
basically
it's
a
really
cool
ui
that
you
use
with
kubernetes
and
you
can
see
like
all
the
different
things
that
you
have
running
per
like
what,
depending
on
like
what
context
you're
using
and
then
the
different
name
spaces
it's
real
time.
So
it's
kind
of
like
when
you
do
a
watch.
You
can
actually
see
things
as
they
are
coming
up
or
if
they
fail
and
then
you
can
like
dig
into
things
like
look
into
a
pods,
you
can
see
you
can
actually
get
into
a
pod.
B
B
Okay,
it's
a
little
harder
to
like
funky
looking,
so
I
might
actually
just
make
it
harder
to
read,
because
that
way
you
can
kind
of
see
all
the
different
things.
Let's
see
there
we
go.
We
can
kind
of
scooch
that
over
a
bit,
so
basically
it
it
ends
up
creating
a
bunch
of
different
deployments
for
different
things
I
needed
like.
So
there
was
a
stuff
with
notary
that
was
mentioned.
There's
the
claire,
which
is
used
for
the
image
scanning.
B
A
Stuff
deployed,
so
we
kind
of
have
like
core
is
the
main,
the
main
sort
of
entry
point
to
harbor
and
it
then
redirects
work
to
say,
chart
museum
to
host
helm
charts
to
the
registry
to
do
the
registry
work,
there's
notary
and
notary
signer,
so
that,
if
you're
signing
your
actual
images,
you
can
ensure
that
they're
signed
correctly
job
services
there
to
like
schedule,
jobs
like
I
guess,
if
it's
scheduling
like
daily
scans,
using
the
vulnerability
scanner,
it's
using
job
service
and
it
has
claire
and
trivi-
are
both
installed
and
they're.
A
A
Vulnerability
scanners
and
I
think
that's
it
oh
and
then
there's
portal,
which
is
an
nginx
server
that
runs
kind
of
in
front
of
core,
and
I
think
it's
optional,
and
so
I'm
not
sure
if
it's
really
doing
too
much
there
apart
from
kind
of,
I
guess,
maybe
adding
a
little
little
bit
of
protections,
you're,
not
exposing
your
raw
go
app
to
the
internet,
so
that's
kind
of
the
infrastructure.
A
C
A
And
so
you
can
see
core
is
doing
a
ton
of
stuff
right
and
then
it's
reaching
into
the
various
bits
as
as
needed,
like
the
the
registry,
server,
etc.
A
Yeah,
actually
I
didn't-
I
didn't
know
about
engine
and
and
do
sex,
so
it
comes
with
claire
and
trivi,
but
I
guess
you
can
plug
in
engine
and
do
sec
as
well
for
scanning
and
there's
a
ton
of
support
for
the
replicated
registries.
So
that's
cool,
so
yeah
and
then,
of
course,
a
little
side,
loading
of
your
identity
providers,
if
you
have
active
directory
or
oauth
or
whatever.
A
So
there's
a
there's
a
lot
going
on
inside
of
harbor
if
you've
ever
used
like
if
you
ever
run
the
docker
registry,
it's
kind
of
just
one
binary
and
it's
pretty
simplistic,
but
it
doesn't
really
do
anything.
A
B
Yeah,
it
kind
of
gives
people
the
choice
of
what
they
want
to
do.
So
I
guess
yeah
you
can
pair
with
any
third-party
commercial
or
open
source
scanner.
A
Yeah,
so
why
don't?
We
actually
have
a
look
at
the
the
scanning
side
of
things
as
a
first
spot.
So
let's
bring
up
our
harbour
cluster
itself
now
we
actually
have
two
harbour
clusters
up.
So
hopefully
we
remember
to
use
the
right
one.
So
this
is
the
new
one
that
I
spun
up
and
then
we
have
one.
That's
the
slightly
older
version
in
case.
We
want
to
try
and
set
up
some
replication
or
something.
A
B
Uh-Oh,
I
totally
put
it
as
remember
me.
Let's
see
one
moment,
while
I
figure
look
at
what
the
password
is
again.
A
It's
just
my
admin
password
alex.
We
actually
set
it
to
something
other
than
the
default
just
for.
B
A
To
access
it.
B
Yeah
so
like
by
default,
the
password
is
harbor
one,
two,
three
four
five
and
you
have
the
option
to
change
it.
If
you're
doing
things
are
fun,
maybe
it
doesn't
really
matter,
but
if
you
don't
want
people
to
screw
with
it,
you
ideally
give
a
different
password
and
you
don't
show
people.
A
A
A
B
A
Yeah,
so
I
guess
maybe
switch
to
projects,
and
I
think
I
just
threw
a
alpine
image
in
there
just
as
something
to
have,
and
so
hopefully
so
in
there
we'll
have
a
scan
result.
Next
to
the
image.
B
Yeah,
so
by
default,
there's
a
project
called
library.
You
can
choose,
have
everything
that
you
want
into
this
project,
but
you
can
also
create
other
projects
as
well,
which
we
can
show
after
I
poke
into
library
and
see
if
alpine
is
correctly
there.
So
here
we
can
see,
there's
alpine,
you
can
see
that
it's
broken
down
to
repositories,
there's
the
helm,
charts
and
then
a
bunch
of
other
things
that
we
can
look
into
later.
So
if
we
click
on
alpine,
you
can
see
that
initially
it
is
not
scanned.
B
B
A
I'm
just
curious
anyone
that's
out
there,
so
I
noticed
that
this
project
is
listed
as
public.
So
if
someone
wants
to
go
ahead
and
try
pulling
library
alpine
from
this
repository,
that
would
be
interesting
to
see
if
it
just
works
for
you
yeah.
So
the
full
registry
is.
B
B
A
C
B
B
A
Admin
yep,
I
feel
like
creating
users,
is
pretty
boring,
like
everyone
knows
how
to
create
a
username
and
password.
But
what
I
did
see
up
there
was
something
about
robot
accounts,
and
so
maybe
we
can
go
ahead
and
create
a
robot
account
and
then
see
if
we
can
get
kubernetes
to
pull
an
image
from
this
repository
once
it's
once.
We
have
one
there.
B
Sounds
good
what
what
what
should
we
call
the
robot.
A
C
B
B
Is
it's
harbor
dot,
tgik,
dot,
demo,
dot,
paul
c
z,
a
r
dot,
w
t
f?
And
then
it
would
be
slash
library,
slash
alpine.
A
Yeah
and
it's
just
a
https,
so
it's
the
regular
ports.
A
So,
let's,
why
don't
we
just
throw
that
in
a
a
variable
in
your
command
line
in
that
way,.
B
A
So
that
is
that
done,
I
guess
let's
push
an
image
into
that
registry,
so
we
could
just
use
your
like
just
the
admin
credits
we've
already
logged.
B
A
So
yeah
nginx
works
great,
so
go
ahead
and
maybe
pull
down
engine.
B
B
A
B
A
Vulnerabilities
are
discovered,
it
can
oh
well.
This
is
about
par
for
the
course
for
when
you
pull
an
image
from
the
docker
registry
right.
I.
A
B
A
B
A
So
if
we
had
that
setting
set,
it
would
probably
stop
us
from
being
able
to
pull
this
image
down,
but
we
don't
have
that
set.
So
we
can
definitely
we
can
definitely
still
pull
from
it.
So
what
do
we
need
to
do
to
authenticate
kubernetes
to
our
cluster?
Because
this
is
a
private
image?
We
need
that
robot
user
and
I
think
we
create
a
secret
using
that
robot.
C
A
Right
now,
of
course,.
B
A
So,
as
you
evaluate
the
risk
of
a
particular
cve,
you
can
then
decide
on
whether
or
not
you're
willing
to
accept
that
risk.
So
it's
not
just
a
blanket
ban.
It
actually
probably
allows
you
to
build
up
a
good
graph
of
what
cves
you
have
deemed
to
be
worth
the
risk
and
what
you
haven't.
So
that's
that's
super
cool.
A
B
A
A
No,
you
don't
need
to
docker
assumes
https.
If
you
don't
put
anything
in.
A
B
B
Oh
yeah,
let's
verify
oh.
C
B
Start
all
right
now.
A
So
now,
let's
does
it
give
us
a
cheat
pod,
manifest.
A
B
There
we
go
so
that
crazy
thing
that
we
copy
paste
it.
A
A
A
A
A
A
A
C
C
A
Right
so
I
guess
you
could
do
a
a
port
forward
and
you
can
do
a
port
forward
inside
of
octane
right.
B
B
A
C
A
A
I
think
I
think
that's
it
and
then
you
do
8080
colon
80.
and
if
you
do
a
ampersand
at
the
end
it
will
background
it.
So
you
can
keep
typing.
A
Yeah,
it
did
so
now
you
can
do
just
just
hit
enter
to
clear
the
text.
That's
there,
you
you're
oh
you're,
doing
it
from
the
tmx
machine.
So
just
do
a
curl
local
host,
8080.
B
A
All
right,
so
that's
cool,
so
that
works.
So
now
we
have
an
incredibly
insecure,
vulnerable
image
running
on
our
cluster,
which
is
what
everyone
wants
inside
of
a
kubernetes
cluster.
A
Yeah,
so
do
we
want
to
what
do
we
want
to
try
now?
Do
we
want
to
try
to
do
some
replication
or
set
it
up
as
a
proxy?
Let's,
let's
set
up
a
what
a
public
repo
as
a
proxy
to
like
docker
hub
and
see
if
that
works.
B
Okay
and
then
we
should
also
show
adding
a
helm
chart.
A
B
A
Well,
especially
now
that
the
public
helm
chart
repos
are
deprecated,
it's
a
great
time
to
spin
up
harbor
to
run
your
helm
charts
for
you.
B
Cool,
so
there
might
be
some
other
ways
to
do
this.
I'm
gonna
you
can
paul.
You
can
tell
me
if
I
should
be
doing
it
some
other
way,
but
I'm
just
gonna
go
off
of
getting
like
bitnami
engine
x,
because
I
don't
know
what
the,
if
is
the
other
one
nginx
flash
engine
x
like
I
don't
actually
know,
but
you
already.
A
B
Okay,
so
we
he
already
did
he
already
added
the
repo
for
bitnami,
so
I
don't
have
to
do
that.
So,
basically,
I'm
just
going.
I
thought
you
did.
A
Oh,
I
thought
I
had
two.
Oh
I
know
I
I
did
that
I
did
the
install
locally
not
on
there.
So
what
is
the
if
you
look
at
my
installed
at
sh?
It's
in.
C
C
B
B
So
if
we
look
here
what
happens,
it
ends
up
pulling
down
the
like
the
tar
file
for
all
that
stuff,
which
is
like
what
paul
is
mentioning
earlier
for
the
other
one
where
you
could
like
open
up,
look
at
the
values,
etc.
That's
the
same
kind
of
thing
here
so
basically
now
what
we
would
need
to
do
is
add
paul's
chart
repo.
B
So
do
I
need
the
is
there
something
special
that
I
need
to
do
with
this
one
being
private
with
no,
I'm
just
admin
all
right,
nevermind,
okay,
so
I'm
just
gonna,
do
home
repo,
add
and
then
let's
just
call
it
ggik,
and
then
we
give
it
the
whole
path
here.
B
Yeah,
I
was
thinking
at
the
very
beginning,
and
then
I
just
forgot
and
didn't:
do
it?
Okay,
I'll
do
that
after
this
okay?
So
then
the
part
like
if
we
look
over
here
in
the
helm,
charts
right
now,
we
can
see
there's
nothing
there
so
like
when
you're
doing
stuff,
with
just
with
the
images
there
is
like
the
flash
library
or
whatever
it
is
for
this
one.
You
end
up
first,
having
slash
chart
repo
and
then
you
give
it
which
one
it
is
so
tgik.
A
A
In
there,
so
why
don't
you
do
the
library
and
we'll
stick
it
in
library,
because
that's
public.
B
I
was
thinking
that
might
happen
all
right,
so
if
we
do
okay,
so
I
added
that
and
then,
if
I
do
just
like
a
helm,
repo
update
and
then
it's
pulling
from
things
from
tgik.
B
B
Oh
wait:
I'm
a
dum-dum
all
right
so
afterwards.
C
A
Have
to
upload
a
chat
before
you
download
them.
B
Yeah
yeah
yeah
yeah
yeah,
okay,
so
basically
I
don't
know
if
there's
a
command
line
way
of
doing
it,
but
in
here
you
click
upload,
you
browse
for
the
chart
file
except
for
wait.
How
do
I
do
this
when
the
chart
file
is
over
there.
A
Oh,
let's!
Let's
check
that
out.
I
think
you
would
do
we'll
probably
need
to
google
for
the
plugin
itself,
but
I
think
you
just
to
do
a
helm
chart
plug-in
you
do
like
helm,
plug-in,
install
and
give
it
the
git
url.
C
B
Okey-Dokey,
let's
see
we.
B
Already
have
that
yeah
well,
do
we
need
to
add?
B
Well,
we
installed
the
plug-in,
but
do
we
need
to
just
do
this.
C
B
C
B
A
A
B
A
B
A
C
B
A
B
Yeah
so
like
tjik
and
then
this
giant.
B
A
Yeah
engine
and
you
might
need
to
do
nginx,
dot,
tgz
or
tar
gz
or
whatever
it
downloaded.
C
A
Because
I
think
it
was
saying
that
you
do
the
like
it's
expecting
to
be
in
the
chat,
repo
directory,
sorry,
the
the
chart
directory
and
we
have
the
chart
tabled.
So
you
could.
C
A
So
probably
do
this
and
then
it
was.
A
A
A
A
A
Of
course
we
need
to
auth
because
it's
a
public,
it's
a
public.
B
A
A
Where
do
I
have
that?
Oh
boy?
A
Oh,
because
I'm
looking
at
your
screen,
not
mine,
do
you
want
to
grab
the
password
out
of
the
pretty.
B
C
A
Was
way
more
complicated
thanks
vivian
for
your
help
in
chat
yeah
thanks
a
couple
of
amateurs,
smashing
keyboards
and
hoping
that
the
correct
command
comes
out.
B
A
Oh
yeah,
that's
right
for
the
cnabby
kind
of
yeah,
so
helm
has
some
like
alpha
support
for
cnab.
That
lets
you
do
that,
but,
as
you
said,
it
does
it
as
like
a
cnab
image,
not
the
helm,
chart
itself.
So
I
think
what
we
did
was
probably
the
mo
the
more
common
way
that
people
would
want
to
do
it.
So
that
is
a
great
plug-in
to
to
remember
the
I
may.
A
That
plugin,
because
I've
always
done
it
like
an
idiot
through
the
through
the
gui,
so
can
we
replicate
now.
A
So
I
think
if
we
get
some
replication
and
or
proxying
working,
it
would
be
a
great
way
to
end
in
things,
because
we're
probably
going
to
start
losing
people's
attention
spans
pretty
soon.
B
If
we
haven't
already
with
the
struggle
busting
with
helm,
all
right
so
which
one
do
you
want
to
do?
First.
A
Let's,
let's
set
up
a
proxy,
I
feel
like
that,
should
be
nice
and
easy
and
quick,
and
then,
when
that
works
we
can
then
do
the
next
one.
So
I
guess
we
create
a
new
project
right.
C
A
Yeah
and
I
guess
we
can
call
it
proxy
and
then
click
the
button,
make
it
public.
B
B
B
B
I
guess
this
is
the
time
where
heavy
in
white
mode
on
my
screen
would
make
more
sense.
A
A
B
A
A
B
B
B
C
C
B
B
C
C
A
B
C
A
Doing
this,
maybe,
instead
of
apache
just
do
nginx
or
alpine,
because
the
apache
image
is
probably
called
hdp
or
something.
A
C
B
C
C
C
B
C
A
Right
well
and
alex
is
saying,
maybe
check
the
logs.
A
A
Oh
see
the
event
log
2
with
a
towards
the
top
right.
What
does
that
say.
B
B
A
Maybe
try
try
pulling
something
a
little
bit
bigger,
maybe
do
open
jdk.
B
C
B
A
So
that's
pretty
big
and
then
we
can
do
a
a
docker,
rm,
open,
jdk
and
then
do
a
docker
pull
again
and
we'll
see
if
it
looks
if
it
seems
quicker.
C
A
A
C
A
C
C
A
I
I
guess
it's
one
of
those
like
eventual
consistency,
things
because
you
know
the
images.
C
A
B
Yeah
and
the
thing
is,
I
pulled
nginx
and
I
don't
think
nginx
is
there
yet.
A
A
B
And
for
the,
why
it's
showing
one
for
open
jdk
only
is
because
the
first
time
we
pulled
it,
it
was
pulling
it
like
getting
it
from
docker
hub,
and
then
it
ended
up
putting
it
up
into
our
repository
here
on
harbor,
and
then
we
pulled
it
again.
So
the
only
time
I
pulled
directly
from
harbor
was
the
second
time
we
did.
It.
C
A
Did
that
initial
proxy
pull,
that
seems
to
make
sense
we're
nearly
there.
C
B
A
A
Let's
do
let's
do
push
push
based
so
we'll
push
into
here
and
then
we'll
see
if
it
shows
up
in
yours.
B
Cool,
do
we
need
any
of
this
stuff
right
now.
A
B
A
Leave
them
empty
and
then
destination
namespace.
I
guess
we'll
want
to
just
use
library
as
well.
B
B
B
A
B
C
B
C
B
A
A
That
replicate
button,
no
in
your
hover.
C
B
Nice
all
right,
so,
let's
see
I
I
I
know
what
we
should
have
done
is
checked
what
I
had
already
in
there,
but.
A
B
C
A
And
works
out
what
it
needs
to
do
conrad
is
asking:
do
you
need
to
explicitly
pull
official
images
from
docker
hub
and
push
into
harbor
or
say
way
to
automatically
do
it,
so
you
could
automatically
do
it?
The
way
we
just
did
with
the
proxying
or
replication,
oh
and
alex
says
the
new
robot
will
allow.
B
B
A
A
Yeah,
so
I
guess
there's
like
a
bunch
of
ways
you
can
do
it.
What
we
did
with
the
docker
tag.
You
could
do
what
we
just
did
with
replication
but
reversed,
or
there
are
some
tools
that
let
you
move
images
from
one
registry
to
another.
I
think
they're
part
of
like
the
cnabby
kind
of
things.
A
I've
seen
a
couple
of
tools
that
do
that,
like
it
gives
you
like
an
image
relocation
but
effectively
it's
pulling
it
down
and
then
pushing
it
up
again,
but
I
feel
like
the
replication
method,
might
be
the
good
way
to
do
it,
especially
on
the
manual
replication.
So
you
can
kind
of
decide
when
you
want
to
replicate
newer
versions
and
stuff
yeah
conrad's
talking
about
official
images
that
you
don't
own
so
yeah
I
mean
you
would
pull
them
tag
them
push
them
or.
A
A
Cool,
so
I
think
I
think
we've
done
pretty
well,
I
mean
there's
like
quotas
and
labels
and
stuff
like
that,
but
I
don't
feel
like
we
need
to
like
everyone's
seen
those
things
before.
B
Yeah
so
just
like
to
reiterate
like
there,
you
can
create
users
and
give
them
permissions.
There's
the
registries,
and
these
are
the
ones
that
we
just
added
for
docker
hub
and
harbor.
There's
the
replication
that
we
just
created
and
you
can
do
like
there's
the
distributions
you
can
create
labels.
You
can
set
the
their
quota
or
check
out
like
if
you
set
a
quota
in
the
beginning.
When
you
create
the
project
you
can
see
how
far
along
like.
Are
you
almost
full
of
what?
B
Whatever
quota
you
had
again
integration
services
are
using
these
scanners?
You
can
create
a
new
scanner.
You
can
delete
one
you
can.
There
is
a
you
can.
Also,
let's
see,
I
don't
remember
exactly
where
it
is,
but
there's
a
way
to
also
oh
yeah
there.
It
is
I'm
done
so.
You
can
also
set
which
one
you
want
as
default
and
then
you
can
actually
choose
like
what,
when
you
want
to
scan
everything
that
you
have
so
you
can
put
custom
and
you
can
check
it
every
hour.
A
A
So
if
you
feel
pretty
handy
on
the
old
curl
x
post,
you
can
start
sort
of
automating
your
way
through
doing
that,
and
then
the
other
thing
that
I
think
is
pretty
useful
is,
if
you
click
on
one
of
the
projects,
so
go
back
to
project
and
just
pick
any
of
the
projects.
C
B
A
A
So
I
don't
know
if
it'll
show
it
because
we're
we're
using
a
real
certificate,
but
if
you
have
a
self-signed
certificate
for
your
harbor
there'll
actually
be
a
button
at
your
repository
to
let
you
download
the
ca.
That
was
that
that
is
used
to
sign
your
certificate
and
that
way
it
makes
it
a
little
bit
easier
to
like
set
up
your
docker
connection,
because
in
docker
there
is,
if
I
bring
up
my
command
prompt
in
like
let's
jump
into
root,
because
we
can
so
in.
A
Like
etsy,
docker
search
dot
d,
you
could
do
like
harbor
dot,
tgik
dot,
demo,
dot,
pulsar.wtf,
slash,
ca.sir,
and
so
that
isn't
actually
there.
But
if
you
grabbed
your
ca
from
harbor,
you
could
stick
it
in
here
and
then
docker
on
this
machine
would
trust
harbor,
because
it's
it
knows
that
it.
Now
it
has
a
ca
and
a
ca
is
valid
or
that
you're
choosing
to
trust
that
ca.
A
So
that's
pretty
useful,
except
it's
still
not
super
easy
to
do.
For
your
actual
kubernetes
nodes.
It's
made
even
more
difficult
by
the
fact
that
in
kubernetes
you
might
have
docker
and
you
might
have
container
d
and
they
both
handle
ca,
certs
differently
and
container
d.
Does
it
differently
through
a
few
different
versions
of
container
d,
as
well
so
trying
to
deal
with
these
self-signed
certs
when
you're,
actually
wanting
to
pull
and
push
from
kubernetes
is,
is
not
a
lot
of
fun.
A
Certificate
in
front
of
it
using
cert
manager
and
then
also
we
didn't
really
show
it
off,
but
harbor
does
support
using
tls
for
all
of
the
like
intra
harbor
communication
through
all
the
different
pods
it
has
there.
So
you
can
be
fairly
confident
that
the.
A
But
anyway,
I
think
it's
fine.
So
let's
let's
call
it
here:
we
we
did
a
bunch.
You
gave
a
pretty
good
summary
of
what
we
did
any
any
last
words.
A
B
Let's
see,
I
wonder
when
we
paste
links
are
those
not
showing
up
yeah
links.
I
don't
know
whether
it's
the
settings
that
vmware
has
on
this
or
if
it's
youtube
but
yeah.
Basically,
if
you
paste
anything
that
has
a
link,
it
will
not
go
through.
A
A
A
I
just
don't
know
the
exact
mechanism
for
configuring
how
to
make
it
do
it
at
the
times
you
prefer,
and
you
may
need
to
figure
out
how
to
do
it
through
a
proxy
if
you're
inside
of
a
corporate
cluster
and
inside
of
a
corporate
data
center
or
whatever,
but
I'm
sure
I'm
sure
you
could
figure
that
out
cool
all
right.
B
Yeah-
and
we
can
also
just
to
also
mention
like
if
you
click
on
logs
here,
you
can
see
all
the
random
things
that
we
were
doing
like
whether
it
was
proxy
like
this
stuff
that
I
was
trying
to
shove
into
my
like
into
my
harbor
again
or
like
what
we
did
as
an
admin
et
cetera.
So
you
can
kind
of
see
like
all
the
different
things
that
we've
that
you've
been
doing
like
there's
the
robot
et
cetera.
A
B
A
A
Oh
yeah
alex
michael
michael,
thank
you
so
much
for
helping
us
out.
It
was
really
great
to
have
you
on
here
and
to
help
steer
us
in
the
roughly
correct
direction,
as
we
kind
of
just
poke
around
and
thanks
for
other
folks
that
jumped
in
and
helped
with,
like
the
helm,
plug-in
and
stuff
like
that,
you
made
our
jobs
a
lot
easier
and
stopped
us
from
doing
even
more
fumbling
around.
So
thank
you.
Everyone
and.