►
From YouTube: 2020-09-24 GitLab.com k8s migration APAC
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
B
A
Should
we
start
going
through
blockers.
B
Yeah,
we
can
do
that.
I
I
just
went
through
the
list
before
this
meeting
myself,
oh
okay,
but
but.
B
Cool
so
graham
just
joined,
let's
get
started.
B
Going
through
blockers
first,
I
went
through
the
full
issue
list,
just
to
make
sure
the
lines
with
the
highlights
and
the
things
that
we
should
be
tracking.
It
looks
good
I
closed
a
couple
issues
that
are
no
longer
actually
both
either
closed
or
unlabeled
issues
that
are
no
longer
blockers.
B
This
list
is
pretty
much
all
we
have
right
now,
which
is
the
build
logs
mapping
services,
proxy
request,
buffering,
which
is
sort
of
related
to
mapping
services,
since
we
need
to
change
that
per
service
and
then
the
cross
az
network
traffic.
The
focus
right
now
is
on
cross
hazy.
Network
traffic,
the
mapping
services
is
a
problem,
but
it's
not
an
immediate
problem.
B
We're
gonna
do
get
ssh
after
get
https,
it's
not
a
blocker
for
that,
but
as
soon
as
we
finish
get
ssh,
then
we
should
move
on
to
doing
web
and
api.
There
are
no
known
blockers
for
web
and
api
other
than
buildblocks,
but
that's
in
progress
so.
A
How
would
you
feel
about
mapping
services
to
for
me
to
raise
priority
because.
B
B
That's
cool
yeah,
I'm
yeah,
I'm
not
really
yeah,
I'm
curious
what
direction
we're
gonna
go
with
that?
I
think
I
think
right
now.
My
my
current
thinking
is
that
it
it
is
important
for
us
to
split
out
at
least
get
https
from
https
traffic,
given
how
different
those
workloads
are.
A
B
B
Yeah,
I
think
the
alternative
would
be
all
https
traffic
goes
to
web
service
pods
and
you
would
which
would
mix.
I
mean
web
and
api.
Sure.
That's
like
not
the
they're
very
similar
workloads
but
get
https
is
very
different.
A
I
I
I
remember
the
problems
we
used
to
have
back
in
the
day
before
the
split
I
just
I
don't.
I
don't
think
this
is
acceptable
for
gitlab.com
scale
to
mix
those
things
together.
I
just
think
we'll
encounter
so
many
odd
problems
just
because
of
yeah,
I
no
I'll
I
raise
it
to
p2.
I
think
we
need
to
figure
out
what
what
we
are
going
to
do
there
and
if
the
charts
is
not
going
to
provide
us
that
we
all
need
to
find
a
different
way,
including
forking
the
chart,
if
necessary,.
A
Many,
how
much
do
we
need
to
raise
the
complexity
of
what
we
run?
You
know
around
a
product
deficiency
like
I,
you
know
I
don't.
I
don't
feel
comfortable
with
that.
Given
the
complexity
of
what
we
are
running
here.
Right
like
we
are
running
hybrid
infrastructure
with
now
like
multiple
clusters,
with
layer
on
top
of
layer
on
top
of
layer
like
it
just
feels
a
bit
for
me,
it
feels
overwhelming,
and
it
could
be
that
because
I
don't
know
much,
but
I
don't
know.
B
Okay,
well,
that's
pretty
much
it
for
blockers.
A
So
the
proxy
request
buffering
that
one
you
said
it
really
depends
on
mapping
services.
Can
we
tights
tie
them
together.
B
Yeah
I
was
going
to
move
it
underneath.
I
think
we
can
move
it
underneath
and
and
to
be
honest,
like
it's
not
100
clear
whether
this
is
an
issue
because
we
disable
it
globally
and
we
already
have
a
cdn
in
front
and
we
think
that
might
be
sufficient
and
we
can
just
leave
it
disabled
globally.
So
I'll
I'll
move
it
up
under
mapping
services.
B
Okay
projects
pages:
are
we
still
looking
at
68
months.
A
I
raised
that
this
monday,
as
a
concern
and
update
from
the
from
the
working
group
multi-large,
is
that
we
set
ourselves
a
deadline
which
is
being
able
to
stand
up
individual
instances
using
a
tool
for
july
next
year.
I
I
know
it's.
A
I
know
I
know
I
know,
but
it
it
will
be
a
forcing
function
already,
even
if
it's
only
july,
given
that
the
projections
that
the
teams
made
specifically
for
pages
was
that
they'll
be
able
to
roll
everything
out
and
finish
in
may
that
ain't
gonna
work
right
like
that.
A
Work
so
they'll
need
to
pull
in
the
timeline
a
bit.
So
I
think
this
week
next
week
and
the
week
after
you'll
see
more
yeah
discussions,
first
like
how
what
and
how
and
then
I
think,
priority
will
have
to
change
so
I
think
it'll
be
pulled
in
it's
not
going
to
be
that
long.
A
I
hope
people
are
aware
that
it's
it's
too
much
or
too
little
rather
for
for
what
we
need
to
achieve.
C
A
Couple
of
things,
graham
so
one
thing
first,
is
that
we
have
been
blocked
with
the
migration
of
on
gitlab.com
right,
like
pages,
is
a
crucial
service,
and
you
know
the
story
on
gitlab.com.
A
That's
one
number
two
is
that
we
have
self-managed
customers
that
are
refusing
to
use
omnibus
because
they
spent
a
lot
of
time
migrating
their
services
to
kubernetes.
They
don't
want
to
have
a
one-off
tool
in
vms
and
pages
is
not
supported
in
health
charts
at
all.
It
can't
be
right
because
of
those
weird
dependencies
and
then
number
three
is
we
are
working
on
the
multi-large,
which
means
like
multiple
github.com,
like
instances
set
up
like
day
two
operations.
Basically,
and
in
order
to
do
that
effectively
and
efficiently,
we
can't
replicate
gitlab.com.
A
We
can't
we
can't
run
hybrid
infra
easily
right
so,
and
the
idea
is
well
if
we
untangle
pages
and
ensure
that
you
can
use
kubernetes
properly,
and
you
ensure
that
we
have
tooling
around
it
to
to
set
you
up
for
success.
A
C
A
Yeah,
it
has
the
nfs
dependency
across
multiple
services,
so
there
was
a
suggestion
for
us
to
introduce
nfs
into
kubernetes,
and
I
yeah
I
well
jared
know
knows
how
I
reacted.
Jar
was
much
more
calm
and
said.
Well,
these
are
the
technical
reasons
I
just
went
out
and
said
like.
This
is
not
something
that
we
should
be
doing
at
all
or
thinking
about,
even
like
not
entertaining
the
dots,
so
yeah
mixing
30
year
old
technology
with
five-year-old
technology
feels.
C
A
B
Cool
on
to
the
demo,
I
wanted
to
give
a
quick
tour
of
where
we
are
and
where
we're
going
for
the
multi-cluster
kind
of
shape.
First
of
all,
the
thing
that
came
up
the
last
time
we
talked
was
the
naming
and
trying
to
keep
the
names
like
not
crazy.
B
So
what
I
settled
on
was
we
have
the
regional
cluster,
which
is
going
to
be
environment,
gitlab
gke
we're
just
keeping
the
same
name,
because
we
don't
want
to
change
the
name,
because
that
requires
changing
like
rebuilding
a
cluster
and
the
zonal
cluster
will
just
be
environment
plus
the
zone.
So
for
pre-prod.
It
looks
like
this.
Of
course,
this
isn't
set
in
stone.
We
haven't
done
anything
like.
We
can't
revert
so
doing
the
poc
on
pre-prod.
B
It
looks
like
pre,
gitlab
gke
is
gonna,
be
the
regional
cluster,
and
then
we
have
pre
one
b,
c
and
d,
and
that
keeps
it
nice
and
short.
It
keeps
our
names
short
that
derives
from
the
cluster
name.
So-
and
I-
and
I
think
I
did
one-
I
wanted
to
include
the
full
zone
because
including
the
region,
because
we
could
eventually
add
additional
regions
here
and
that'll
be
important
for
the
zonal
configuration
I
did
a
bit
of
refactoring.
B
Instead
of
having
everything
tucked
into
main.tf,
we
now
have
gke.tf,
which
has
the
regional
cluster,
and
then
we
have
the
gke
zonal
tf,
which
just
has
the
zonal
clusters.
The
gke
zone
ltf,
which
is
what
I'm
showing
here,
is
where
we
have
some
repeated
module
calls.
Basically,
we
have
two
modules.
We
have
the
module
for
all
the
ip
reservations,
so
that
kubernetes
can
as
a
so.
We
pre-allocate
the
internal
ip
addresses
for
kubernetes,
and
then
we
have
the
gk
module
itself,
which
creates
the
cluster.
B
The
maintenance
policy
here
will
be
different
for
each
cluster.
I
haven't
made
that
determination
yet,
but
we'll
eventually
probably
have
like
alternating
maintenance
times
for
the
different
clusters,
and
you
can
see
like
basically
usc's
1b
usc
1c,
your
c7d.
It's
not
great.
It
would
be
nice
with
like
the
next
version
of
terraform.
We
might
be
able
to
do
this
in
a
loop,
but
keep
in
mind
also
that
each
of
these,
like
I
don't
know
like
they,
each
have
different
sub
networks.
B
B
The
the
reservations
module
is
only
doing
ip
and
dns
reservations
that
are
gke
cluster
specific
we
thought
initially,
I
was
thinking
like
we'll
just
have
a
generic
module,
that's
not
gpe
specific,
but
if
you
think
about
it,
this
is
fairly
unique.
B
You
know
it's
a
unique
thing,
because
we're
actually
calling
this
module
three
times
so
to
use
it
outside
of
gke
doesn't
really
make
sense.
So
it's
that's
why
it's
namespace
to
gke
and
it
just
it
just
creates
a
bunch
of
ip's.
That's
all
it
does
right
now
that
are
needed
for
monitoring
the
ingresses
forget
you
know
and
registry,
and
all
this
yep.
C
So
I,
like
that's
fine.
I
think
this
is
just
all
good
one
of
the
things
actually
now
we
have
external
dns
deployed
and
available
in
all
the
clusters.
We
can
actually
get
rid
of
reserved
ips
and
just
use
external
dns
to
create
internal
dns,
entries
and
stuff
and
use
that,
like
we
don't
actually,
especially
in
monitoring.
For
example,
we
actually
don't
need
fixed
ips
at
all,
because
it's
all
that
it's
exposed
through
external
dns
and
that's
the
entries
we
use,
so
we
may
even
actually
be
able
to
simplify
that
model.
Quite
a
lot.
B
Yeah
registry
and
and
git
because
those
are
to
ingress
so
yeah.
That's
a
good
point.
I
didn't
move
over
the
external
dns
because
I
wasn't
actually
sure
how
to
do
that.
Yet
currently,
this
is
defined
in
main.tf.
C
B
Yeah
well
it's
in
here
somewhere,
but
there
is
like
configuration
for
the
external
dns
and
I
need
to
make
that
compatible
with
the
zonal
clusters,
still
cool.
That's
it
for
terraform
and
then.
A
I
have
a
question
for
you:
jeff,
it's
more
stylistic
thing.
Is
it
possible
to
be
explicit
for
the
terraform
files
right
now
you
have
gk,
zonal
and
gke.
Can
we
call
that
gk,
zonal
and
gk
original?
I
think
like
we
need
to
be
explicit
here.
It's
going
to
be
a
better
time
that
makes.
B
B
C
C
There's
two
there's:
I've
looked
at
this
a
lot
quite
recently,
so
I
can.
I
can
very
quickly
run
you
through.
So
when
we
define
the
maintenance
period,
two
things
are
going
to
happen.
Master
upgrades
will
always
happen
first,
in
which
case
we
won't
be
able
to
do
a
deploy
during
that
period.
However,
I've
got
a
little
bit
of
automation
now
that
adds
certain
annotations
to
afar
and
that
could
be
extended
for
anything.
If
you
want
to
set
a
flag
to
stop
deploys
or
whatever
we
want
to
do.
C
We
now
do
have
control
over
when
those
we
we,
we
have
no
an
entry
point
for
understanding
when
those
master
upgrades
happen
and
it's
going
to
be
in
the
matter
of
minutes.
They
spin
up
new
masters
in
the
background
and
then
when
they
actually
send
us
the
alert
the
upgrades
happening
they
just
swapped
them.
I
I
actually
thought
they
took
down
the
masters
and
what
have
you
so?
The
control
plane
outage
is
like
it's
only
a
couple
of
minutes.
C
What
I've
seen
like
two
minutes
or
something
the
node
upgrades,
is
obviously
they
like
cycle
through
all
the
nodes,
and
that
is
more
disruptive.
However,
that
problem
is
it
it's
it's
more
impactful
to
potentially
to
our
workloads,
but
we
should
be
able.
We
really
should
be
handling
it
and
I
think
we
can
handle
it.
We
do
handle
it.
We
could
make
it
faster
by
doing
better
pod
disruption
policies,
but
every
upgrade
I've
done
so
far
and
even
watching
it
doing
auto
upgrades,
it's
been
very
safe.
It
just
takes
a
long
time.
C
We
can
still
stagger
the
upgrade
windows
and
I
actually
have
a
big
discussion
about
how
we
want
to
stagger
it
and
stuff,
because
the
choices,
if
we
because
the
outage
windows
have
google
mandate,
that
they've
got
to
be
so
big
like
eight
hours
or
something
the
more.
We
stagger
them.
It's
like
we're,
basically,
the
whole
week.
C
We're
just
got
one
cluster
constantly
in
an
upgrade
window,
or
we
just
box
them
all
together
during
us
sunday
evening,
australia,
monday
morning,
where
I've
got
it
at
the
moment,
which
is
the
quietest
period
of
the
week,
and
we
just
try
and
crack
them
through
all
together
and
in
theory,
we
we
are
doing
multiple
at
once,
but
each
of
those
node
pools
should
be
rotating
independently
and
the
pod
disruption
budget
should
be
making
for
making
sure
we
don't
have
major
problems.
So
I'm
I'm
happy
to
do
it
whatever
we
want.
C
A
Yeah,
okay.
That
makes
sense
thanks
for
explaining
that
okay
and
then
my
my
my
remark
is
invalids,
then
it
doesn't
matter
then
cool.
Where
is
that
issue
grim?
Could
you
link
me
that?
Because
I
might
have
missed.
A
C
C
We
may
even
do
one
cluster
at
some
point
and
then
every
other
cluster
all
overlapped
at
the
same
time,
so
we
have
one
kind
of
canary
in
case
the
upgrades
go
wrong,
but
yeah
the
google
have
very
specific
requirements.
They're
not
like
you
can
only
do
one
hour.
You've
got
to
do
like
four
to
eight
hour
blocks.
Wait.
They
can
do
it
at
any
point.
So
it's
really
tricky.
C
A
C
We
still
have
some
tweaking
we
can
do
on
pod
disruption.
Budgets
I
feel
like
I
feel
we
can
make
like.
The
upgrades
at
the
moment
are
a
little
painful
because
it's
like
it
kills
a
hundred
pods
and
every
pod
takes
like
two
minutes
to
kill.
So
it's
like
just
sitting
there
grinding
over
it,
so
I
feel
like
we
can.
C
C
C
A
It's
not
only
cloud
native.
I
think
that
question
has
been
asked
before
for
for
our
regular
setup,
which
is
that
we
never
finished
answering
it.
I
think
I
think
I
asked
you
about
it
like
I
couldn't
find
it
like.
I
was
trying
to
find.
Where
was
that
discussed,
but
I
remember
there
was
a
discussion
of
what
is
reasonable.
A
What
is
a
reasonable
timeout
and
you
were
explaining
to
me
how?
Because
of
the
way
we
are
rolling,
vms
and
so
on,
we
we
gave
like
a
large
channel.
B
B
Yeah,
just
by
just
by
the
fact
that
the
omnibus
takes
a
few
minutes
to
install,
we
wait
much
longer.
I
kind
of
wonder
how
much
of
this
is
also
a
monolith
problem.
Where
you
know
we
don't
real
realistically,
like
get
hdps,
has
very
little
dependency
on
rails
right,
it's
just
using
it
for
authorization.
B
A
I
think
that's
a
question
we
need
to
ask,
so
I
would.
I
would
ask
you
to
to
raise
an
issue
for
us
to
discuss
this
like
what
is
the
actual
limit
right,
like
what
kind
of
expectations
do
we
need
to
set
to
our
users
explicitly
now
right,
like
not
just
what
we
are.
B
Doing
now
by
accident.
A
B
Yeah,
I
think
the
first
thing
we
need
to
do
is
somehow
figure
out
how
long
these
connections
are
on
average
for
git
https,
and
I'm
not
sure
if
we
can
do
that
with
metrics.
I
need
to
check,
but
that
would
be
interesting
to
see.
C
I
think
registry
is
just
the
bigger
one
like
if
I
pull
a
docker
image
from
get
labs
registry
of
my
home
connection,
which
to
be
fair,
I'm
halfway
around
the
world.
My
connection
sucks,
you
know
it
could
take
me
20
minutes.
If
I
get
18
minutes
into
that,
you
roll
a
pod,
it
would
just
chop
the
whole
thing
and
then
fail
and
that's
kind
of
like
I.
I
don't
know
whether
we
have
to
see
like
does
docker
support,
http
retried
or
we
need
to
set
some
headers
or
something,
but.
C
B
C
B
We
we're
independently
deploying
registry
separate
from
the
monolith,
so
it's
it's
not
as
much
of
an
issue
because
we
aren't.
We
aren't
rolling
those
pods
as
frequently.
C
B
Only
for
config
updates
that
affect
registry,
so
it's
this
is.
I
think
this
this
helps
a
lot
and
it
would
help
for
get
https
if
we
did
the
same,
but
divorcing
get
https
from
the
monolith
is
going
to
be
a
uphill
battle,
so
we'll
probably
need
to
come
with
a
lot
of.
B
Evidence
cool
while
mary's
typing
that.
B
You
can
take
a
look,
I
don't
know.
Graham,
I
saw
you
commented
on
the
hum
files,
mr
and
you're
you're.
Okay,
with
the
current
approach
like
by
having
the
separate
environments
for.
C
I
think
that
makes
sense.
It's
yeah,
I
I
you
would
know
better
than
me.
If
they're
literally
going
to
be
identical,
we
could
probably
get
away
without
separate
environments,
but
I
suspect
that
there's
going
to
be
some
point
where
they're
not
going
to
be
identical,
in
which
case
helm
file
environments
is
the
only
kind
of
mechanism
we
have
to
cleanly
make
the
differentiation.
B
So
yeah
so
so
now.
C
We
have
necessarily
a
bad
thing
too,
because
I
think
with
environments
we
can
also
actually
force
it
to
certain
cube
contexts,
which
is
another
thing.
We
should
actually
probably
look
at
doing
as
well
like
if
you
do
like
helm
file,
environment,
usc
1b.
It
will
automatically
always
use
the
context
for
that
cluster.
So
to
avoid.
B
Yeah
exactly
yeah,
I
I
decided
you
probably
saw
this.
I
saw
I
decided
to
put
the
cluster
and
region
in
the
environment
file
itself,
and
just
put
it
explicitly,
one
thing
that
where
this
is
repeated
is
that
we
also
set
cluster
and
region
as
environment
variables
in
ci,
and
this
is
how
hem
file
this
is
how
heb
file
knows
like
connects
to
the
right
cluster
right,
because
this
happens
before
it
reads.
Environments.Ml.
B
I
I
think
setting
this
in
two
places
is
the
right
thing
to
do
for
now,
and
until
we
change
we
can
change
ci
so
that
it
it
reads
this
file,
and
maybe
it
gets
the
cluster
and
region
from
it.
Instead,.
C
B
Yeah,
maybe
that's
an
option
yeah,
but.
B
One
thing
I
one
thing
I
learned:
what
I
initially
was
thinking
of
doing
was
that
if
you
go
to
kit
lab
values
canary,
you
can
see
here
that
we
we
inherit
canary
from
gprd
this.
This
doesn't
really
work,
though,
if
gprd.yaml.comtemplate
is
an
invalid
yml
file
and
it
becomes
an
invalid
yml
file.
When
you
have
these
variable
definitions,
do
you
know
what
I'm
saying
or
not.
C
B
Yeah,
for
example,
values.yaml
that
go
template
is
a
valid,
go
template,
but
it's
not
a
valid
yml
file
this.
So
so,
if
you
inherit
something
that
has
like
variable
definitions,
this
will
fail
because
it's
doing
from
yemen.
C
B
I
was
looking,
I
couldn't
figure
it
out,
but
but
anyway
this
was
my
first
thought
was
that
we
could
just
have
all
the
zono
ones.
Do
a
read
file
on
the
base
regional
pmo,
but
that
didn't
really
work.
So
instead.
B
So,
instead,
what
we're
doing
is
this
like
in,
in
the
help.
B
File.Dml
not
for
thanos
here
you
can
see
that
it
actually
first
it
loads.
This
end
prefix,
which
is
a
value
that's
set
for
environment,
so
for
for
all
of
these
zono
clusters
and
prefix
is
set
to
just
pre.
So
first
it
loads,
pre,
dot,
yaml.co,
template
right
and
then
it
loads.
The
environment
name.dml
by
go
template
it's
kind
of
like
because
if
the
environment
name
is
pre,
what
happens?
Is
it
loads,
pre
and
then
pre
again?
And
it's
fine?
It's
the
same
file,
but
it's
a
little
bit
janky
yeah
yeah!
B
There's
probably
we
could
probably
make
this
a
bit
nicer
by
being
more
explicit.
Maybe
like
maybe
make
the
environment
name
for
the
you
know,
regional
cluster
be
like
pre-regional
or
something
and
then
have
a
base
environment
named
pre.
But
the
thing
is:
is
that,
like
I
don't
want
to
overload
environments
too
much
and
I.
C
C
B
Exactly
yeah,
that's
exactly
what
we're
doing,
but
it
works
okay
for
now
so
yeah.
C
B
Yeah
you
can't
make
help
about
yaml
can't
be
a
go
template
right.
It
has
to
be
because
then
you
can
get
rid
of
these
like
really
this
janky
syntax
here
where
you're,
because
you
could
put
you,
could
make
a
variable
definition
instead
of
embedding
it,
because
what
I
found
was
that
if
I,
if
I
did
that
right
home
file
right
now,
it
needs
to
be
valid
ammo.
So
you
can't
put
variable
yeah
exactly.
C
You
can't
put
variable
declarations,
yes
and
that's
why
we
have
to
like
yeah,
you
gotta
quote
it
that
way:
yeah.
B
A
There
is
one
thing,
but
I
only
have
two
minutes
and
okay.
Let
me
because
there's
some:
what
is
it
confidential
information?