►
From YouTube: 2021-09-08 AMA about GitLab releases
Description
Monthly AMA with the Delivery team
A
Okay,
so
welcome
everyone.
This
is
the
monthly
ama
with
the
delivery
team
who
are
most
of
us
are
here.
So
thanks
everyone
for
joining
this
is
a
chance
really
for
anyone
to
ask
questions
about
deployment
releases
for
gitlab.com
releases
to
our
self-managed
customers
or
anything
else
that
we
are
working
on
as
well.
So
would
anybody
like
to
kick
us
off
with
a
question.
B
B
A
That
is
a
super
good
question.
So
yes,
as
you
as
you
state,
we
have
added
in
a
stage
to
our
deployment
pipelines,
which
is
a
staging
canary.
So
this
will
get
us.
It's
not
an
exact
replica
yet,
but
this
gets
as
much
closer
to
our
production
deployment
setup
where
we
have
a
production
canary
and
then
we
allow
some
time
for
it
to
bake
before
we
move
to
the
production
main
stage
so
on
staging,
we
will
have
a
similar
setup
now,
an
end-to-end
time.
This
will
slow
down
on
mttp
like
changes.
A
A
So
overall
that
should
speed
things
up
following
this,
though,
once
we
see
the
impact
of
having
this
extra
station,
we
can
consider
like
we
have
an
extra
stage.
We
get
a
bit
of
extra
protection
from
staging,
so
we
can
consider.
Does
that
affect
the
hour
of
baking
time?
Maybe
we
readjust
that?
Maybe
we
re-juggle
how
we
run
tests
against
baking
time
to
bring
things
down,
so
it
does
add
an
extra
stage,
but
hopefully
it
adds
in
an
extra
safety
stage
that
we
can.
C
I
have
one
and
I'll
put
in
the
doc.
I
don't
know
if
it's
the
right
area,
but
what
are
we
doing
to
communicate
better?
The
changes
that
we're
making
to
customers
workflows
in
regards
to
like
feature
flags
that
we
push
out
a
lot
of
the
feedback
I've
been
receiving
from
customers?
Is,
I
don't
know
where
to
find
any
of
these
issues
like
do
I
just
go
search
for
feature
flags
being
released
that
will
break
my
stuff.
C
A
That's
a
super
good
question:
yes,
because
every
release
does
have
so
many
changes
going
in.
So
all
of
these
things
should
be
going
into
the
release
post.
So
for
a
patch
release,
we
generate
those
within
delivery
with
the
the
changes
that
we
include
for
the
bigger
ones
for
the
monthly
release.
The
product
managers
are
sort
of
incorporating
changes
into
the
the
release
post.
So
that's
the
main
way
that
people
should
be
able
to
find
out
about
things.
I
think,
if
that's
not
it's
either
the
formatting
or
the
content.
C
Okay,
how
much
lead
time
are
we
thinking
to
give
customers?
The
reason
why
I
ask
is,
I
ask
them
what
they
would
expect
and
for
some
customers,
their
goal
of
their
platform
is
to
be
100
automation.
So
as
their
automation
platform,
what
they're
saying
is,
you
know
we
can
log
into
gitlab
and
it's
up,
but
for
us
reliability
and
stability
for
gitlab.com
is.
C
Can
I
get
in
and
log
in
and
do
my
workflow
of
the
day
so
when
they
logged
in
last
week,
a
lot
of
them
had
broken
pipelines
everywhere
they
got
four
or
four
errors
and
they
were
just
wondering
what
was
going
on.
So
what
he
was
asking
was
you
know,
when
are
you
going
to
make
that
change
like?
What
are
you
going
to
announce
it
and
then
how
can
you
let
us
know,
because
what
they
were
asking
for
is
about
two
weeks.
That's
what
he
would
expect.
C
I
said
I
don't
know
if
we
can
meet
that,
but
at
least
we
have
you
know
a
requirement
from
you
and
then
we
can
go
from
there
and
say
yes,
we
can
deliver
that
or
no
you
know
I
know
we're
not
going
to
release
something
in
one
day.
Right
like
I'll,
make
an
issue,
and
then
you
say:
let's
do
it
right
now.
A
I'm
not
totally
sure
on
which
situation
we're
talking
about
here,
but
so
I
mean
it
might
be
worth
like
if
you've
got
a
link
or
something
we
can
follow
up
on
that
one.
But
I
think
generally
breaking
changes
should
be
on
the
major
milestones
and
that's
kind
of
the
intention
is
that
monthly
releases
are
new
things
coming
in
that
people
can
switch
over
to.
A
If
we're
doing
things
that
are
removing
things,
then
my
understanding
was,
though,
to
be
limited
to
the
like
the
14.0
or
15.0
14.0
was
out
in
june,
and
for
that
month
there
are
lots
of
breaking
changes
and
they
should
be
all
communicated
like
in
the
months
leading
up
to
that,
so
that
people
do
have
sufficient
time
to
switch
over.
But
it
sounds
like
this
one
might
be
a
slightly
different
case,
so
maybe
we
should
dig
into
actually
what
happened
there.
D
A
D
Awesome,
thank
you.
The
second
one
is
a
rollback
and
I
I
was
excited
to
see
that
the
rollback
is
so
much
faster
than
it
has
ever
been.
37
minutes
is
a
record
time.
I'm
sure
you
guys
are
gonna
get
that
down
even
more,
but
my
question
really
is
like:
should
this?
This
would
be
something
we
should
be
communicating
to
people
that
we
should
not
hesitate
to
do
this
anymore,
because
in
the
past
I've
been
a
little
hesitant
because
well,
I'm
not
sure
if
it's
going
to
break
something
or
cause
more
problems.
D
And
yesterday
we
didn't
fully
understand
the
problem.
The
incident,
so
we
weren't
sure,
like
what
version
rolling
back
to
would
have
been
productive,
but
if
it
was
the
kind
of
thing
where,
if
it
took
10
minutes
to
roll
back-
and
we
had
to
do
it
again-
maybe
that's
not
a
big
deal,
it's
worth
the
cost.
So
is
that
something
we
should
be
discussing,
or
at
least
you
know
talking
about
to
more
to
the
wider
community
here.
E
I
can
take
this
one
amy,
okay,
so
the
biggest
problem
we
have
right
now
with
rollbacks
are
post
deployment
migration.
So
what
we
have
right
now
is
that
if
there
is
a
positive
requirement,
migration
between
what
we
are
running
and
the
previous
package,
we
cannot
roll
back.
So
all
the
tooling
that
we
built
around
this
are
just
limiting
us
to
comparing
what
is
running
now
with
what
was
running
before.
So
we
just
do
one
step,
roll
back,
there's
no
real
limit.
E
The
only
the
only
reason
why
we
have
this
limit,
because
it
makes
it
easier
for
us
to
figure
out
if
there
are
positive
alignment,
migration
or
not.
So
our
what
we
think
has
a
team,
I
think
I
can
speak
for
everyone
here
is
that
we
believe
that
rollback
are
possible,
but
we
we
have
only
the
specific
case,
which
is
common
because
we
often
ship
bus
deployment
migration.
E
So
our
goal
in
this
direction
is
more
about
trying
to
solve
the
rollbacks
problem
so
that
we
can
pack
them
together
and
do
them
on
weekly
basis
or
in
control
of
the
environment,
so
that,
when
a
release
manager
is
around
the
side.
This
is
the
point
in
time
where
we
cannot
go
back,
so
we
run
the
positive
migration
and
then
we
know
that
everything
else
can
be
rolled
back
because
long
term.
Our
vision
on
this
is
more
about
going
toward
also
something
like
automated
rollback.
D
E
So
we
are
building
on
a
metrics
on
this
because
we
don't
know
for
sure.
So
when
we
test
so
when
we
were
testing
the
rollback
the
beginning,
sometimes
we
are
rebuckable,
sometimes
not
so
I
would
say
no,
I
I
can.
I
can't
ballpark
a
number
here
so
but
at
least
one
for
a
week,
I'm
quite
sure
we
have
this
one
for
a
week,
but
probably
also
one
for
day
we
had
one.
Today
at
least
I
can
say.
A
If
the
answer
to
that
is
yes,
and
we
have
a
rollback
check
that
gives
us
that
answer,
then
absolutely
we
should
be
evangelizing
that
that
is
like
so
far
we're
in
the
early
stages
of
of
rolling
these
out
and
testing
on
production,
but
all
of
our
staging
tests
have
been
very
successful.
We've
had
two
recent
incidents
on
production
that
we've
managed
to
mitigate
with
rollbacks.
A
So,
yes,
I
think
we're
at
the
stage
where,
if
a
package
is
suitable
for
rollback,
we
suspect
the
issue
is
related
to
the
deployment
we
we
probably
should
just
go
ahead
and
roll
back
and
see
if
that
improves
things
like
a
rollback
should
be
safer
than
a
deployment,
because
it's
a
previously
installed
package,
so
there's
no
great
risk
to
doing
the
rollback.
In
that
sense,
we
just
need
to
run
more
of
them
to
build
the
confidence.
F
I
was
actually
wondering
about
that
on
the
incident
yesterday
stan
because
I
feel
like
if
we
had
rolled
back
as
soon
as
we
were
like
something's
wrong.
Let's
just
roll
back,
then
it
would
have
been
much
harder
to
actually
isolate
what
the
problem
was.
I'm
not
sure
if
there's
what
the
balance
is
there,
how
we
debug
it,
but
also
just
fix
it
right
away.