►
From YouTube: 2022-06-15 GitLab.com k8s migration EMEA/AMER
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
B
D
C
C
What
ends
up
happening
is
that
we
end
up
triggering
an
alert
which
is
kind
of
the
opposite
effect,
so.
C
I
think
our
only
way
to
accomplish
you
know
trying
to
figure
out
what
alerts
that
we're
going
to
run
into
is
to
practice
this
in
staging.
So
I've
already
got
an
issue
to
you
know
perform
the
practicing
procedure.
What
I
need
to
do
is
add
the
fact
that
we
need
to
keep
track
of
what
alerts,
trigger
and
figure
out
a
good
silence
mechanism
that
is
broad
enough
to
cover
a
cluster
being
down,
but
not
so
broad
that
it
ends
up
triggering
the
alert
and
for
historical
context.
C
C
No,
we
should
not.
In
fact,
the
only
alerts
that
I'm
expecting
to
trigger
is
anything
related
to
our
monitoring
stack
that
might
be
installed
into
a
cluster,
because
those
components
will
effectively
be
able
will
either
not
be
able
to
scrape,
or
they
will
not
be
reporting
themselves
upward
to
like
thanos,
where
all
those
metrics
get
gathered
and
stored.
C
When
I
went
through
trying
to
identify
all
the
saturation
levels,
you
know
we've
done
a
pretty
good
job
with
our
hpas,
such
that
they'll
scale
up
normally
and
I'll
touch
a
little
bit
on
this
a
little
bit.
But
you
know
we
should
be
okay
with
adding
additional
load,
so
I'll
touch
back
I'll
touch
back
on
that
a
little
bit
more
in
a
second.
A
C
A
Sorry
about
question
john
yeah
did
we
have
any
issue
in
the
past
where
something
seemed
to
happen,
and
we
had
like
a
overload
of
alerts.
C
C
Thank
you.
The
other
thing
that
I
had
identified
was
that
our
get
http
node
pool
that
handles
all
our
web
service
get
traffic
that
was
dangerously
close
to
saturation
today
so
like
we
were
running
at
near
100
saturation
during
our
peak
workflow
times
today
so
well.
Prior
to
today,
I
started
working
on
this
and
I
knocked
out
this
issue,
so
the
issue
I
fired
up
to
address
this
is
not
closed.
Our
saturation
level
was,
I
think,
at
86
this
morning.
It's
now
down
to
12,
so
I'm
super
excited
about
that.
C
So
with
12
saturation
across
all
three
clusters,
we
should
be
close
to
if
I'm
doing
the
rough
math,
my
head
correctly
about
25
saturation
across
two
clusters
when
the
cluster
gets
when
they
get
hdp
pool.
C
C
C
The
way
that
our
ancient
proxies
are
configured,
we
set
the
weight
values
on
two
clusters:
the
cluster
that
is
inside
of
the
same
zone
that
that
he
proxy
node
is
in
and
then
canary,
which
is
a
regional
cluster.
So
our
production
clusters
are
normally
set
to
a
weight
of
100,
whereas
canary
set
to
a
weight
of
five.
This
provides.
Roughly
4.76
percent
of
the
traffic
is
going
to
land
on
the
canary
cluster.
C
If
we
mark
one
of
those
clusters
as
a
maintenance
period
or
min
in
sort
of
maintenance
or
drain
state
canary
will
start
to
take
100
of
that
traffic
for
any
h,
a
proxy
node.
That's
in
that
same
zone,
that's
a
very
dangerous
situation
and
while
for
the
most
part,
we
should
be
able
to
accept
that
amount
of
trafficking
canary,
I
don't
want
to
risk
it
because
canary
serves
a
very
specific
purpose.
C
C
That
just
doesn't
make
sense
that
it's
introducing
in
a
very
awkward
consistency
situation
that
is
going
to
make
for
an
interesting
headache
that
I
don't
want
to
have
so
I've
got
some
ideas,
I'll
start
sharing
my
screen
again,
because
I
created
an
issue.
I
just
haven't
really
fleshed
everything
out
just
yet.
C
I
want
to
introduce
a
set
of
tooling
improvements.
This
is
just
a
kind
of
proposal,
so
one
of
these
is
that
we
have
a
set
server
state
script
that
allows
us
to
set
the
state
of
back
ends
in
h.a
proxy.
This
was
used
more
when
we
were
not
in
kubernetes,
but
the
tool
still
exists.
D
C
One
would
be
to
let's
try
to
shut
down
back
ends
in
a
specific
zone,
so
instead
of
asking
h
a
proxy
to
shut
down
cluster
c
across
all
h,
a
proxies
target,
only
the
h
e
proxies
that
live
in
zone
z
and
shut
down
the
cluster
for
zone
c.
C
What
that
should
enable
us
to
do
is
preventing
the
situation
where
I
guess
I'm
going
out
of
order,
but
that
should
prevent
a
situation
where
all
the
traffic
ends
up
in
canary,
because
we've
shut
down
the
traffic
for
just
that
zone.
C
The
other
improvement
I
would
like
to
try
to
introduce
is
to
make
these
changes
slowly
right
now.
This
script
operates
against
pretty
much
all
the
fe
nodes
all
at
the
same
time,
and
that's
going
to
be
kind
of
dangerous
because
with
one
third
of
all
traffic
or
you
know
in
certain
like
web
requests,
we're
getting
93
million
requests
per
15
minutes.
C
Okay
right
now,
if
I
recall
correctly-
and
this
again
is
something
I
need
to
research,
if
I
recall
correctly,
aj
proxy
has
stood
up
in
a
way
where
it's
checking
for
the
status
of
our
servers
and
if
it's
in
an
upstate,
auto
deploy
is
happy.
The
prepared
job
is
happy
and
allows
the
deploy
to
continue
if
a
service
is
in
down
state
autodeploy
would
be
like
I
something's
wrong.
C
Look
at
this
I'm
going
to
fail
that
way.
The
deploy
does
not
continue
if
we
put
them
in
maintenance
state.
If
I
remember
correctly,
the
prepared
job
is
okay
with
that,
but
that's
something
I
need
to
recheck,
because
I
can't
remember
off
the
top
of
my
head.
What
that
looks
like
I.
C
Okay,
so
alongside
that,
the
other
thing
I
want
to
make
sure
is
that
the
low
balancing
works
okay
in
that
situation,
because
of
our
weighting
and
our
backup
configurations
instead
of
aj
proxy
meaning,
because
we
have
say
I'm
operating
in
zone
c
again,
zones
b
and
d
will
be
marked
as
backup
servers,
backup
endpoints.
So
they
don't
see
traffic
for
the
most
part
ever
from
a
zone.
Ch
epoxy.
C
I
want
to
make
sure
that
low
balancing
still
operates
as
we
expect
it
to
when
we
start
tearing
things
down
for
a
given
cluster,
given
the
state
that
they're
in-
and
this
is
just
a
quirk
inside
of
h-
a
proxy-
I
want
to
validate
that
we're
not
going
to
see
anything.
That's
surprising!
C
That's
all
outside
of
that
outside
of
these
tooling
improvements
that
I'm
proposing
the
next
step
as
far
in
or
in
regards
to
this
particular
epic
is
actually
starting
the
testing
and
staging
now,
let's
tear
down
the
cluster
and
staging
and
rebuild
and
see
where
we
are.
C
I
think
the
only
other
things
I
have
listed
under
this
epic
are
the
tooling
updating.
I
just
proposed
performing
the
testing
and
staging
and
then
some
future
items
so,
like
you
know,
trying
to
figure
out
what
to
do
with
network
spaces
and
kubernetes
overall,
which
I
don't
feel
like
we
need
to
address
now
it's
going
to
be
something
we
need
to
do
when
we're
ready
to
execute
in
production
that
we
can
rebuild
clusters
as
necessary.
C
I
think
that's
a
precursor
to
actually
formalizing
what
that
angle
looks
like
and
then
also
creating
saturation
alerts
for
the
networking
space
which
I
don't
think
we
have
tamland
is
doing
that,
but
we
don't
have
alerting
for
it
and
maybe
tamline
is
sufficient.
I
don't
know
because
those
are
something
that
gets
evaluated
over
time
and
we
get
like
a.
A
A
C
Because
if
I
recall
correctly,
it
runs
on
a
tuesday
but
like
just
to
showcase
the
work
so
far
like
this
was
prior
to
the
get
nodes
yeah.
You
know
this
is
back
from
last
week
but,
like
you
know,
we
were
on
an
upper
trend
and
if
I
recall
we
were
going
to
hit
70
saturation
by
july-
and
I
didn't
market
here
but
like
we
were
on
target
for
being
fully
saturated,
come
august
september-ish,
but
now
that
we've
done
all
the
node
pools
except
the
one
I
just
fixed
today.
A
C
Okay,
so
like
there
is
no
forecast
at
hitting,
neither
are
100
or
80
percent,
which
is
fantastic,
and
you
see
where
the
drop
the
dots
are
now
down
to
like
40-ish
upwards
of
50
during
the
load
time.
So
that's
fantastic,
and
this
will
come
down
even
further
next
week,
when
tim
lane
runs
now
that
they
get
no
pulls
run.
A
A
very
good
good
result.
Yes,
thank
you,
john.
Do
you
think
we
can
have
a
little
demo
in
the
next
next
week
to
see,
which
is
our
saturation
level
after
tamland
runs
yeah.
C
D
C
Good
question
so
here's
my
thought
on
that
and
here's
what
I
plan
to
do
to
address
this
with
the
he
proxy
tooling
improvements.
My
goal
with
this
is
that
canary
is
still
usable
and
autodeploy
is
smart
enough
to
skip
deploying
to
a
cluster
that
might
be
undergoing
maintenance,
which
that
is
a
separate
issue
that
I
need
to
still
create,
which
that's
a
good
reminder,
because
I
need
to
figure
out
what
to
do
with
with
all
the
deploys
during
cluster
outages.
D
It
right
like
so
we
have
the
the
allow
it's
causing
like
kate
to
allow
failure
flag
that
will
that
will
not
will
not
attempt
to
deploy
to
that
cluster.
C
C
D
Yeah
we
so
we
talked
about
it
quite
a
lot
because
it
was
like.
Oh,
it
sounds
largely
bad,
but
except
for
the
case
where
you
have
a
deliberate
time
like
this,
where
you
actually
don't
want
to
attempt
to
to
deploy
to
a
known
cluster,
so
we
do
have
a
way
of
skipping
out,
but
what
I
was
going
to
actually
ask
as
a
follow-up
is
assuming
we
have
a
way
to
keep
deployments
running.
D
C
C
Yep
so
part
of
my
procedure,
when
we
do
this
in
staging
is
going
to
add
a
step
that
indicates
how
to
perform
a.
I
guess,
a
true
up
after
the
fact.
So
you
know
autodeploy
does
this
automatically,
but
if
a
cluster
gets
skipped
or
if
we
rebuild
the
cluster,
it's
not
going
to
have
gitlab
deployed
to
at
all,
or
it's
going
to
have
their
own
version.
C
We
have
the
ability
to
do
this
locally.
Our
tooling
operates
or
can
talk
to
our
clusters
outside
of
ci,
so
the
esta,
the
steps
would
be
you
know,
do
whatever
is
necessary
to
rebuild
the
cluster
after
it's
up
healthy,
ready,
ready
to
go,
perform
a
step
which
is
just
running
the
k,
control
command
like
we
do
in.
A
C
B
Thank
you
so
come
proxies
on
production
already
out
the
door
and
we
are
not
accepting
any
traffic.
Yet
I
was
checking
today
the
change
request
to
accept
traffic,
and
there
is
a
point
we
can
basically
discuss
at
this
point.
It
says
fairly
like
like.
B
B
B
I
don't
know
like
monday,
because
I
thinking
actually,
if
we
are
going
to
execute
this,
to
execute
before
my
duty
as
a
release
manager,
which
is
coming
after
next
week.
B
C
A
C
A
C
B
D
C
Effectively
we're
turning
the
cluster
valid
or
the
cache
mechanism
is
being
revoked
because
the
the
infrastructure
is
currently
in
virtual
machines.
B
We
are
basically
gonna
update
one
version
like
it's
currently,
six
on
the
one
of
the
column
on
the
database,
the
markdown-
I
don't
know
what,
what
what
column
is
it
but
like
from
six,
it's
gonna
be
like
updated
to
seven,
which
is
gonna,
invalidate
all
of
the
markdown
rows
on
the
database.
So
I
think
this
is
why
potentially
it's
going
to
have
like
some
impact.
D
D
Don't
do
it
on
monday
because
I
think
we're
doing
this
database
stuff
at
the
weekend
and
we
will
give
someone
a
heart
attack
if
we
throw
up
a
database
problem
on
a
monday
morning.
But
perhaps
like.
D
B
D
You
think
you're
going
ahead,
oh
good
day,
I'll,
take
you
next
week,
but
yeah
keep
getting
sick.
I
guess,
if
I
guess
it's
so
the
question
is:
if
we
have
no
way
of
testing
this
to
know
how
long
the
impact
is
we're
going
to
have
to
do
a
best
guess,
there's
no
rollback
so
we'd
have
to
just
wait
it
out.
I
guess
like
in
the
event,
this
is
horrible
and
it's
much
worse
than
we
expect.
Is
there
anything
we'd
be
able
to
do.
C
C
C
D
Let's
keep
you,
let's
do
it
on
a
day
you're
around
just
in
case
something
comes
up
later
on
or
it
drags
out
and
you
get
caught
up.
So
so
don't
do
friday.
A
C
C
C
So
let's
go
ahead
and
schedule
this
work
for
friday,
meaning
you
know
this
week.
I
can
try
to
accomplish
it
and,
in
the
meantime,
leading
up
to
this
I'll
try
to
see.
If
there's
any
way,
we
could
try
to
measure
what
that
potential
impact
will
be
I'll.
Do
some
research
and
see
if
I
can't
figure
anything
out
and
if
I
do
discover
something,
either
I'll
delay
or
reach
out
and
try
to
meet
up
so
that
everyone
understands
what
that
is.
C
C
D
D
If
we
don't
manage
to
go
on
friday,
like
I
would
suggest
like
maybe
next
thursday
could
be
another
opportunity
in
your
time,
I
thought,
graham
should
be
around
with
the
post
release,
so
that
might
be
another
good
day.
Sure.
C
D
An
approval
but
like
we'll
approve
that
right
like,
but
I
think,
if,
in
terms
of
it,
giving
people
a
suitable
potential
heads
up
makes
sense.
A
One
thing
also:
I'm
gonna
update
the
some
things
a
bit
related
to
the
template
for
the
change
request
since
c1
and
c2.
The
report
is
a
gitlab
production
calendar,
so
I'm
gonna
add
an
extra
check
there.
I'm
gonna
submit
an
mr
in
a
few
minutes.
A
D
D
B
Thank
you
scarback,
actually,
for
taking
this
either
you
or
me
like
gonna,
execute
it
so
finger
crossed.
D
D
What
I'm
actually
wondering
is
if
we
even
need
to
do
something
with
the
denialist,
because
it's
never
been
used,
and
I
I
wonder
if
there
is
any
sre
on
call
who
knows
it
exists,
because
if
they
don't
know
it
exists,
they're
never
going
to
use
it
in
which
case
probably
doesn't
matter
if
it
if
it
works,
so
it
might
be
worth
as
actually
like
checking
in
with
you
know,
java,
or
you
know,
reliability
basically,
and
actually
is
this
something.
D
For
you,
scarborough
I've
added
on
one
eye,
this
issue,
two
two,
five:
three,
where
we
talked
quite
extensively
about
this
flag
and
do
we
pull
it
out
or
do
we
keep
it,
but
the
reason
we
kept?
It
is
almost
exactly
this
scenario
you're
going
to
need
of.
I
possibly
want
to
keep
deploying,
but
I
know
I
don't
have
the
full
set
of
clusters.
C
Yeah,
so
this
flag
is
for
the
entire
pipeline
specific
to
all
of
the
gitlab
com
deployment,
and
I
really
don't
want
to
impact
that,
because
I
want
to
make
sure
that
if
a
failure
does
occur
in
another
cluster,
like
maybe
it's
overloaded
and
so
a
deploy
is
like
struggling
on
the
cluster.
I
want
to
make
that
well
aware
and
if
we
hide
this
with
this
flag,
we're
preventing
the
release
manager
from
discovering
some
sort
of
failure.
C
C
But
I'll
get
an
issue
created
because
I
should
have
created
one
by
now
to
figure
out
what
to
do
with
that
and
this
I'll
link
to
this,
as
this
is
a
certainly
an
option,
but
I
think
it's
too
broad
to
be
a
sane
option.
In
my
opinion,.
C
Cool
michaela
is
there
anything
that
we
want
to
discuss
related
to
gitlab
sshd
we're
kind
of
on
hold
at
the
moment.
So
I
didn't
know
if
there's
anything
that
we
wanted
to
discuss
in
here
at
all.
A
I
can
give
an
update,
so
the
rollout
of
the
proxy
functionality
is
on
old
because
of
the
blocker
that
identified
the
blocker
was
that
we
would
introduce
a
regression
in
the
case.
We
are
going
to
release
that.
So
there
is
a
new
feature
coming
in
to
avoid
that
a
subset
of
customers
are
gonna,
have
this
regression
in
and
they
actually,
they
change,
requests
that
they
ping.
A
You
in
the
channel
today
is
to
fill
the
data
and
after
the
merge
request,
is
it's
in
production
and
the
migrations
are
run
to
fill
the
data
in
this
new
in
this
new
attribute
coming
in
so
after
that,
I
guess
we
will
have
a
new
updated
date
to
run
the
change
request
for
production
enablement
of
the
proxy
functionality,
but
so
far
that
one
is
an
old
asked
yeah.
I
asked
the
team
resourceful
team
also
to.
A
A
I
got
a
ping
right
now
from
sean
as
soon
as
I
finish.
This
call.
I'm
gonna
check
the
ping.
I
guess
is
the
script
and
I'm
gonna,
I'm
gonna,
let
you
know.