►
From YouTube: 2021-02-23 Delivery team weekly rollbacks demo
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
B
B
A
B
A
B
B
B
B
B
B
B
D
C
C
E
Just
before
we
go,
are
you
going
to
go
ahead
and
do
a
full
roll
back.
C
E
This
is
big,
sir
and
zoom.
I
read,
I
read
all
of
reddit
the
other
day,
so
yeah
quite
a
lot
of
it.
Apparently
there's
a
setting
inside
video
sharing
where
you
can
set
the
thing
that
will
hopefully
fix
this,
which
is.
E
C
E
It
doesn't
make
a
difference,
I
don't
think
there's
a
thing
inside
your
settings
screen
share
which
apparently,
if
you
click
use,
tcp
connection
for
screen
sharing,
is
meant
to
fix
this.
I
haven't
tested
it
yet,
though,.
E
B
C
Okay,
so
I
think
there's
still
some
annoying
work
there.
We
can
automate
to
make
things
easier.
B
Yeah,
there's
also
another
thing
that
I
put
as
a
note
in
the
in
the
round
book
itself,
which
is
that
it
marks
every
type
of
migrations
that
are
potentially
unsafe.
B
So
I
mean
obviously
it
is,
but
it
is
potentially
unsafe,
but
we
should
not
put
the
red
mark
on
when
there
is
on
when
there
are
only
regular
migrations,
because
it
is
exactly
the
same
situation
in
which
cannery
and
production
are
running,
because
we
know
this.
We
keep
repeating
this
every
week
and
but
yeah
by.
Maybe
we
know
that
this
is
safe,
but
I
would
like
the
tool
to
be
more
clear
in
this
series.
B
B
B
B
B
We
forgot
to
write
them
down,
so
I
just
want
to
make
sure
that
we
we
we
keep
track
of
what
to
decide
what
we
decide
to
do,
because
I'm
quite
sure
we
decided
to
do
also
other
things,
but
I
haven't.
I
don't
remember
what.
E
On
the
on
the
board
and
that
they
were
the
main
things
we
decided
last
week,
so
that's
still
true.
I
think
we
had
a
lot
of
release
fun
last
week.
Instead,
what
I
kind
of
related
to
those
issues,
the
deadline,
their.
E
Actions
so
I've
lost
english,
which
is
a
bad
situation.
So,
on
the
main,
let
me
actually
see
if
I
can
share
this.
E
So,
within
our
epic
we
have
this
we're
doing
the
pipeline
for
code
rollback
piece.
What
I
did
last
week
as
part
of
other
epic
updates,
was
to
set
a
due
date
of
march
the
5th
which
is
next
friday.
No,
yes,
next
friday
does
that
feel
like
it's
a
vaguely
reasonable
time
we
have
we're
pretty
close,
so
we've
got
rollback
documentation.
Improvements
is
basically
just
the
things
we
decide
here.
E
The
main
we've
got
two
big
betas
right.
One
is
adjusting
the
pipeline
for
getalian,
prefect
and
standardizing
process
and
investigating
the
diff
between
the
auto
deploy
packages,
and
then
we've
got,
I
think,
alessia
you
might
have
already
done
investigating
the
rollback
feature.
We
need
to
write
some
notes
there.
So
does
that
march
5th
sound
like
a
reasonable
date
to
aim
for,
and
if
so,
do
we
need
to
take
some
actions
to
actually
get
these
items
complete.
B
B
I
think
this
is
enough,
because
then
we
discussed
the
thing
that
on
a
real
pipeline
once
we
want
to
automate
this,
we
want
to
double
check
the
status
on
the
database,
but
I
will
not
count
this
as
part
of
of
this
epic,
because
it's
it's
another,
it's
not
a
topic,
so
I
don't
know
if
others
have
are
aware
of
something
that
we
are
actually
missing
here,
because
otherwise
I
think
this
is
done.
E
One
piece
I
wasn't
sure
on
it
doesn't
have
to
be
part
of
this,
but
I
know
we
touched
on
it
a
little
bit
at
times,
but
I
don't
know
maybe
what
we've
whether
we've
actually
made
a
decision
is:
what
do
we
do
with
in
progress
deployments.
B
It's
a
good
question,
so
I
don't
believe
we
have
enough
automation
and
data
points
right
now
to
figure
this
out,
because
it
really
depends
on
where
it
broke,
and
why.
E
B
So
I
will
consider
the
documentation
yeah.
I
would
consider
this
as
a
kind
of
a
documentation
thing
to
put
in
the
run
book
with
some
ideas,
but
I
mean
it's
more
about
if,
if
it
broke
in
the
middle
think
very
carefully
about
what
you're
doing
and
then
we
can
provide
some
data
points
like
before.
Post
deployment
migrations
this
or
it
really
depends
on
what
broke.
E
E
B
Yeah,
I
think,
there's
a
bit
of
a
tribal
knowledge
around
this,
and
probably
scarborough
and
henry
are
the
best
one
to
try
to
figure
out
this,
because
the
my
so
I'm
just
trying
to
remember
the
discussion
that
I
had
with
jarv.
So
there
is
one
point
about
load
balancer.
So
the
thing
is
that
at
the
beginning
of
each
batch
of
deployment,
we
remove
machines
from
the
load
balancer
so
depending
on
where
how
they
broke.
B
It
may
require
some
manual
action
before
we
can
actually
even
roll
back,
because
when
we
will
deploy
again
the
old
version
on
that
part
of
the
fleet
the
machines
may
be
in
in
the
wrong
state,
so
the
pre-check
may
not
allow
us
to
deploy
so
I'll
yeah.
I
think
that
we
should
defer
this
to
our
sres
so
that
they
can
they
better
knows
what's
happening
there.
A
Yeah,
so,
as
the
lasso
noted,
we
remove
a
node
from
a
low
bouncer.
Well,
so
let
me
back
up.
We
check
for
the
stat
status
of
all
our
nodes
before
pipeline
begins.
If
they're
in
a
maintenance
mode,
things
are
fine.
What
we'll
do
is
we'll
leave
the
node
as
is
deploy
to
it,
and
we
won't
put
the
node
back
into
the
low
bouncer
on
purpose.
A
A
Ansible
will
will
stop
at
that
moment
in
time,
because
the
health
checks
are
failing,
it's
expecting
the
node
to
be
up
inside
a
low
bouncer,
but
it's
not
so
ansible
will
fail
at
some
point
in
time.
If
it's
the
first
node,
we
know
that
something
is
probably
very
bad
with
the
package
we
just
installed
if
it
happened
in
the
middle,
which
has
occurred
sometime
in
the
last
few
weeks
because
of
a
bug
in
the
ruby
garbage
compaction
process
when
we
upgraded
rails,
that
was
a
sporadic
situation.
A
In
that
particular
case,
we
had
to
go
mainly
remediate
that
node,
I
think
in
those
types
of
situations,
it's
better
for
us
to
figure
out
what
the
failure
scenario
is
and
determine
if
a
rollback
is
legitimately
worth
pursuing,
because
if
it's
like
a
ruby
compaction
problem,
you're,
probably
going
to
want
to
roll
back.
We
did
not
do
that
in
the
last
previous
cases,
but
it
may
have
been
the
wise
solution
that
case,
but
we
would
need
to
get
the
node
back
online.
A
Obviously,
there's
a
flag.
We
could
there's
a
flag
you
get
set
that
ignores
that,
but
I
forget
the
name
of
the
flag,
something
with
the
word
aj
proxy
in
it,
but
like
it's
very
dependent
on
the
failure
what's
involved
in
that
failure
and
where
we
are
in
the
deployment
process.
I
don't
know
how
we
can
make
a
run
book
that
if
this
situation
occurs
do
this
because
it
might
be
different,
so
we'd
have
another
if
clause
and
that
may
be
different.
We'd
have
another.
B
Let's
start
by
just
documenting
the
gating
conditions,
so
at
the
beginning
of
the
client
we
expect
this.
We
do
this
and
at
the
end
we
do
that.
If
something
happens
in
between
then
we
are
in
a
mixed
state
and
later
on,
we
will
figure
out
what
to
do,
and
I
was
also
thinking
about
another
flag
because
you
mentioned
flag.
There
is
also
the
omnibus
skip
chat,
throw
or
some
variation
of
these
terms.
B
So
what
I'm
thinking
is
if
we
yeah.
So
if
something
bad
happens
during
a
deployment,
the
rollback
will
fail,
because
at
chef
because
of
I
think,
because
it's
the
same
thing
that
happened
when
staging
deployment
failed,
then
the
new
package
is
ready
and
the
staging
deployment
starts
and
the
pre-check
fails
because
they
there's
a
mismatch
in.
I
don't
know
you
know
better
than
me,
but
does
it.
A
Yeah,
so
if
they
deploy
to
staging
it
gets
all
the
way
through,
but
like
say,
qa
fails.
For
example,
we
miss
the
need
to
swap
a
flag
in
omnibus,
so
the
next
time
the
deployer
comes
around.
What
it
wants
to
do
is
to
validate
that
it's
not
stepping
on
top
of
an
existing
deploy.
We
use
a
chef
role
to
set
a
flag
to
prevent
it
as
a
lockpicking
mechanism
prevent
two
deploys
from
here.
A
B
Yeah,
but
I
mean
I
suppose
in
a
rollback
situation
where
we
know
that
this
may
happen
and
we
are
really
in
control
of
what
is
happening.
Probably
we
want
to
have
all
the
information
up
front.
So,
let's
just
say
just
I
I
say
drawbacks
to
roll
back,
and
the
other
thing
is
that
this
is
a
question
and
is
I'm
aware
that
these
things
happen
in
canary
and
staging?
But
do
we
have
the
same
thing
for
production
because
we
don't
run
qa
at
the
end?
A
C
A
A
A
I
guess
I
could
share
my
screens
just
so.
You
guys
see
what
I'm
talking
about
share,
so
this
is
just
looking
at
the
latest
staging
deployment.
So
in
this
case,
gstg
version
is
the
name
of
the
job
that
runs
and
we
do
a
set
version
and
we
set
it
to
the
name
of
that
item,
and
then
we
do
a
set
ominous
updates,
which
sets
a
very
simple
flag
to
true
and
that's
just
modifying
our
gstg
omnibus
version
role
that
sets
a
flag
to
enable
that
thereby
removes
the
lock
that
this
pipeline
had
that
way.
A
A
B
So
I
think
that,
in
terms
of
understanding
what
we
need
to
do
so,
if
even
just
starting
with
something
simple,
so
if
something
broke
during
a
deployment
and
was
deemed
that
rollback
is
a
good
option.
That's
something
that
we
want
to
do
because
otherwise
we're
just
wasting
extra
time,
because
the
check
will
start
and
will
fail.
And
it
takes
a
lot
of
time
to
fail,
because
unless
you're
checking
it
and
see
the
failures
coming,
it
just
retry
for
a
very
long
time.
E
A
A
E
E
B
E
Yeah,
what
do
we
like?
Is
it
worth
us
like?
I
guess
once
we've
run
it
on
stage
right
and
we're
happy
with
it,
like.
Maybe
it's
a
case
of
us
then
starting
to
run
the
check
for
post-deployment
migration
thing
when
we
see
incidents,
because
if
we
have
an
incident,
we
might
be
able
to
find
an
opportunity
to
do
a
rollback.
E
Because
a
good
thing
about
an
incident
is
we're
starting
from
a
something's
broken
which
no
or
me
I
guess
we
could
just
roll
back
something
right.
There's
no
reason
why
we
couldn't.
B
We're
gonna
have
a
fake,
I
mean
we
can
schedule
something
so
that
we
do
this
in
okay
in
safe
environment
right.
So
we
we
coordinate
with
development.
We
do
change,
which
is,
I
don't
know
something
in
the
help
page
something
really
trivial,
but
you
can
visually
see.
B
B
A
To
our
documentation,
so
theoretically
we
should
be
okay
to
go.
What
I
don't
want
to
do
is
interfere
with
any
potential
upcoming
releases,
specific
security
release,
and
that
kind
of
thing,
if
we
can
avoid
it,
I
think
it'll
be
easier
to
schedule
in
that
particular
case.
B
So
what
about
we
do
this,
but
we
ask
release
manager.
To
I
mean
we
can
do
it
also
as
a
team
doesn't
matter,
but
then
we
roll
we
roll
forward
again
right
after
because
as
long
as
we
play
on
staging,
who
cares
but
productions
may
there
can.
Maybe
customers
see
something
and
new
features
whatever
and
then
it
disappear.
Maybe
there's
some
engineer
that
is
testing
something
say:
yeah
finally
landed
in
production,
but
it
is
no
longer
there,
so
maybe,
as.
A
Long
as
we
communicate
up
front,
we
we
advertise
externally
as
well.
So
maybe
we
should
get
the
cmoc
involved
for
part
of
this
exercise,
because
that
that
would
be
kind
of
disruptive
for
some
people
depending
on
the
change.
A
B
B
Yeah
exactly
so
march
4th
means
march
9.
Could
it
be
no
yeah
nine.
B
Doesn't
make
sense,
I
mean
again
we
check
the
status
of
everything
when
we
run
when
we
start,
maybe
isn't
we
can't
do
it?
Maybe
there's
an
incident
whatever
right,
but
I
will
yeah.
Maybe
we
can
say
that
starting
from
march
9
up
or
maybe
yeah,
nine
or
16,
because
then
there's
a
release
day,
there's
release
another
time
so
the
first
occasion
we
can
run
it.
We
run
it.
E
E
I'll
begin
sharing
comms
on
that
around
infrastructure,
so
that,
if
anyone's
got
any
concerns
on
that
we
can,
we
can
start
addressing
them.
But
I
think
from
what
we've
seen
in
staging
it
looks
pretty
like
it
either
doesn't
work
or
it's
a
pipeline
and
it
rolls
through
so
nice.
E
That's
a
great
idea:
yeah,
so
should
we
find
like
it's
going
to
be
like
two
hours
right,
roll
back
prod!
Is
that
right,
I'm
assuming
it's
roughly
the
same
as
a
deployment
like
a
little
shorter,
but
not
a
huge
map
right.
D
E
B
D
Maybe
we
should
also
make
a
note
on
the
production
calendar,
because
I
mean
we
do
something
in
production
right
and
I'm
sure
the
sre
teams
and
brent
would
like
to
have
some
notification
for
a
front,
and
you
know
be
in
the
picture,
I'm
not
sure
about
what
announcements
we
should
do,
but
I
mean
we
constantly
deploy
something
without
announcements.
So
I
guess
it's
fine,
but
at
least
in
the
production
calendar.
Maybe
it's
a
good
thing
to
have.
E
Yeah,
it's
great,
oh
yeah.
Let
me
put
an
issue
together
because
I
think
it
would
be
good
for
us
to
have
a
bit
of
a
checklist
of
all
the
pieces
that
we.
C
E
To
make
sure
we
do
before
that
and
then,
like
I
doubt,
it'll
be
the
last
time
we
do
a
test
like
this,
so
we
can.
We
can
copy
it
later
if
needed,.
B
E
Later
speak,
the
same
cool,
so
was
there
anything
else
people
wanted
to
cover
today.
E
C
C
A
Gitlab
doesn't
have
a
feature
to
accept
web
hooks
from
someplace.
A
C
C
D
We
had
this
issue
today
that
a
token
was
removed
right
or
even
a
slack
application
which
broke
the
sre
on
call
and
the
release
manager.
Notifications,
I
mean
there's
the
place
where
we
have
those
kind
of
web
services
hooked
up.
I
think
serverless
functions
in
nodejs
or
something
and
if
you
could
replace
it
with
something,
that's
just
easier
to
understand,
and
that
would
be
already
an
improvement,
and
I
know
that
craig
furman
is
working
for
the
incident
management.
D
E
Yeah
for
sure
nice,
okay!
Well,
let's,
let's
keep
that
in
mind
like
if,
if,
if
someone
wants
to
like
think
through
that,
then
go
for
it
or
say
we
can
keep
it
and
keep
in
mind
for
a
future
future
iteration
awesome
is
there
anything
else
that
anyone
would
like
to
cover
for
rollbacks.
E
No
okay
awesome
like
I
love
the
fact
that
this
has
become
like
quite
routine
quite
rapidly
in
just
a
few
short
weeks,
so
that's
pretty
exciting,
so
yeah
onwards
towards
production,
cool
all
right,
enjoy
the
rest.
Your
day
speak
to
you
soon,.