►
From YouTube: Ops Cross-Stage ThinkBIG for September 2020
Description
A conversation for the strategy team members in the Ops Section devoted to looking at and exploring Feature Flag Issues
A
Hi
everyone-
this
is
our
cross
stage-
think
big
for
the
ops
section
nadia's
not
here
today
she
had
a
conflict,
so
I'm
going
to
be
kicking
us
off.
Today
we
have
a
really
exciting
conversation,
but
before
we
dive
into
that,
I
wanted
to
talk
a
little
bit
about
a
change
in
the
way
that
we're
going
to
structure
the
meeting,
we're
trying
to
figure
out
a
way
to
make
this
more
collaborative
and
allow
for
ideas
to
kind
of
come
together
a
little
more
smoothly.
A
So
today,
what
we're
going
to
do
is
two
different
things.
We
are
going
to
utilize
a
mural
board
for
those
of
you
who
haven't
used
it
it's
kind
of
like
a
digital,
whiteboard
or
mural
wall
that
we
can
all
interact
on.
There's
a
link
in
the
agenda
as
well
as
I've
added
a
link
in
the
chat
for
everyone
to
jump
in,
on
the
other
thing,
we're
going
to
try
a
little
bit
differently
and
we've.
I
think
we've
done
this
before,
where
we're
going
to
split
the
session
into
two
different
pieces.
A
The
first
half
of
our
session
will
really
focus
on
thinking
big
divergent
thinking.
What
are
different
ways
we
can
solve
or
interpret
the
problem,
and
then
the
second
half
will
be
on
think
small,
which
is
much
more
in
line
with
the
gitlab
idea
of.
What's
the
smallest
changes,
we
could
make
in
the
next
milestone
to
start
working
towards
these
big
ideas
that
we've
been
talking
about.
B
So
this
all
originated
from
the
fact
that
I
did
some
competitor
analysis
was
launched,
darkly
and
I
found
a
bunch
of
missing
functionality
and
when
they
started
to
open
up
all
the
missing
issues
that
that
we
have
to
implement,
I'm
gonna
give
two.
For
example,
I
was
like
this
sounds
like
an
issue,
so
one
of
them
was
the
fact
that
they
wanted
their
in
launch
darkly.
B
You
can
assign
feature
flags
to
someone
that
you
can
have
ownership
of
a
flag,
and
why
is
that
needed
because,
let's
say
on
sre
in
charge
of
production,
but
I'm
not
the
person
who
actually
created
the
flag.
I
need
to
know
like
who
I
need
to
contact
if
something
goes
wrong,
and
so
you
can
assign
someone
to
the
flag.
So
there
you
have
a
nice
assignee
and
another
thing
that
they
had
was
the
ability
to
add
tags
to
flags
that
you
can
easily
filter
and
sort
them
and
find
them.
B
So
let's
say
you
can
filter
all
the
flags
you
have
in
production
or
all
the
flags
that
are
associated
to
specific
user
management
or
something
whatever
you
can
do,
whatever
you
want,
which
really.
When
I
started
opening
this
up
sounded
like
labels.
I
was
like
hey
wait,
a
minute
everything
that
I'm
opening
up
for
now
sounds
like
an
issue
and
it
sounds
like
maybe
feature
plugs
needs
to
be
an
issue.
B
Yes?
Okay,
so
let's
see
how
we
zoom,
okay,
so
from
the
think
big
issue.
We
have
with
the
problem
that
we
want
to
solve.
So
there's
a
bunch
of
them.
I
kind
of
posted
a
few
before
the
meeting,
but
if
you
have
more
ideas,
feel
free
to
jump
in,
so
one
of
them
is
it's
hard
to
find
future
information
that
is
related
to
the
future
flag
and
basically,
we
already
started.
We
added
a
new
feature.
B
I
think
it
was
in
13.1,
where
you
can
add
context
of
featureflex
to
issues,
so
you
can
now
know
which
issue
is
related
to
a
feature
flag.
So
you
can
know
if
it's
titled
on
or
off
and
it's
specific
to
feature
and
then
later
on.
You
can
see
it
in
the
release,
which
is
really
really
really
great
information
and
in
the
future
we
want
to
add
additional
functionality,
like
mrs,
so
what's
the
merge
request
that
actually
created
this
toggle
and
epics
and
requirements,
and
maybe
james
one
day
in
the
future
tests.
B
So
we
want
to
have
a
lot
of
context
around
the
feature
flags.
I
even
have
a
an
issue
now
that
I'm
working
with
the
source
code
team
to
actually
point
it
back
to
the
code
itself
that
connects
the
feature
flag.
So
the
idea
is:
how
can
we
utilize
git
lab
as
a
single
source
as
a
single
tool
to
get
as
much
information
as
I
can
in
my
client?
B
Another
thing
that
we
have
a
problem
with
is
that
we
don't
have
a
way
to
find
to
filter
and
search
feature
flags,
and
we
have
a
feature
flag
list
which
is
great,
but
once
you
have
over
an
x
amount
of
flags,
it
kind
of
becomes
really
really
hard
to
find.
The
flags
are
currently
sorted
by
their
create
date
or
by
numbers,
and
they
don't
actually
have
to
be
a
sequential.
B
They
can
be
on
the
instance
level.
So,
even
if
I
created
two
flags
one
after
the
other,
if
another
customer
did
it,
it
would
have
another
number.
So
it's
really
hard
to
follow
it's
hard
to
find
and
imagine
you
have
a
hundred
flags
in
your
project
and
now
you
need
to
find
specific
ones.
So
it's
it's
hard.
So
we
want.
We
want
to
make
it
easy.
B
There's
no
ability
to
collaborate
or
have
discussions
around
feature
flags
so
again
feature
flags
can
be
a
team
effort
that
can
be
used
even
from
people
without
outside
the
team
and
maybe
a
feature
flag
that
I
created
has
an
effect
on
another
team.
So
it's
really
important
to
have
a
place
that
we
can
discuss,
discuss
and
collaborate
and
bring
ideas.
B
If
you
even
think
about
the
concept
of
feature
flags,
the
idea
is
to
control
the
audience.
That
gets
a
new
feature.
So
if
I
want
to
now
expose
more
more
users,
I
may
need
an
approval
or
some
kind
of
discussion
or
maybe
just
give
an
fyi
to
people.
Hey,
I'm
increasing
the
exposure
to
70
of
the
users
now,
just
so
people
know.
So
it's
really
convenient
to
have
the
discussion
area.
B
We
talked
about
assigning
a
feature
flag,
so
I'm
not
going
to
repeat
that
no
ability
to
close
feature
flags.
So
today
you
can
create
future
flies.
You
can
edit
them
and
you
can
delete
them,
but
basically
there's
no
history
once
you
deleted
them.
So
if
we
utilize
something
similar
to
close
an
issue,
you
can
basically
archive
the
flag
and
then
always
bring
it
back
to
life
if
needed,
which
is
a
really
really
nice
advantage
and
labels.
We
discussed
so
that's
kind
of
around
the
problems
that
we
want
to
solve.
A
I
actually
did
have
a
question
for
you.
You
mentioned
that
you
think
the
feature
of
flag
could
be
really
comparable
to
an
issue.
As
you
kind
of
talked
to
customers
and
gathered
information
is
a
feature
flag,
an
issue
level
entity
and
so
there's
kind
of
one
initiative,
and
it
may
have
many,
mrs
or
do
you
think
there
are
organizations
that
may
bring
it
up
to
an
epic
level
where
they
have
many
issues
that
all
tie
into
one
feature
and
they
need
to
use
the
feature
flag
to
kind
of
release
that
one
big
feature.
B
It
kind
of
goes
both
ways.
It
really
depends
also.
It
depends
how
small
a
team
is
working.
Usually
people
want
flags
to
be
on
the
smallest
possible
thing,
but
it
could
be
also
on
an
epic
level.
I
don't
think
that
the
flag
itself
is
an
epic,
but
I
may
be
wrong.
It
seems
to
me,
like
the
use
case,
was
more
issue.
B
But
I
think
in
the
level
of
get
lab
issues
is,
is
the
right
form,
and
we
do
want
to
use
this-
to
also
create,
like
a
first-class
citizen
experience
for
future
flags,
because
it's
really
really
kind
of
hard
to
find
them
today
and
use
them
and
there's
a
lot
of
missing
data
in
them,
like
even
very
basic
data
like
the
create
date
or
in
the
creator
of
feature
flags.
That's
something
that
we
don't
have,
which
we
would
get
like
for
free
if
we
created
it
as
an
issue
type.
B
Cool,
so
let's
go
ahead
to
the
jobs
to
be
done.
Unless
there's
more
more
comments,
sorry,
okay,
so
jobs
to
be
done.
We
have
one
for
when
working
with
feature
flags,
I
want
to
be
able
to
have
full
access
to
the
future
information
that
is
belonging
to
it
to
properly
manage
the
feature
flag.
So
this
is
all
that
data
that
that
we
discussed
previously.
B
I
can
connect
it
to
an
issue
at
the
end
of
the
day
when
you
have
an
issue
you
already
will
know,
if
there's
a
feature
flag
associated
to
it
and
you'll
you
get
to
see
all
the
information
on
which
environment
is
it
enabled
and
what
strategy
is
linked
to
it
like.
Even
this
is
enabled
for
only
50
of
the
users.
You
get
all
that
data
just
from
one
from
looking
for
one
place
or
from
any
place,
which
is,
you
know,
what's
really
powerful
about
using
gitlab,
because
you
can
see
it
from
anywhere.
B
So
that's
really
really
important.
It
gives
you
a
lot
of
power
and,
and
also
like
again,
if
we're
thinking
about
the
gitlab
use
case,
where
we
have
sres
that
are
in
charge
of
production
that
have
nothing
to
do
with
the
flags,
it
gives
them
full
visibility
of
what
what
team
did
this?
What
feature
this
is
connected
to
how
many
people
are
exposed
to
it?
A
lot
of
data.
B
Next
one
is
when
working
with
future
flags,
I
want
to
be
able
to
quickly
find
the
one
that
I
need
in
order
to
properly
manage
that,
and
then
I
kind
of
talk
about
that
in
the
problem.
Validation
in
the
problem
section
and
the
next
one
is
when
working
with
feature
flags.
I
want
to
be
able
to
assign
owners
to
it
so
that
I
can
know
who
to
contact
in
case
of
questions
or
assistance.
B
I
want
to
be
able
to
have
a
discussion
with
my
team
around
it
in
order
to
manage
collaboration
around
it
and
that's
what
we
talked
about
the
discussion
area
and
the
personas
that
we're
targeting
at
the
moment
is
sasha
and
devon.
I
see
someone
wrote
here:
release
manager,
absolutely
and
when
we
expand
this
to
a
b
testing,
it's
even
more
personas,
which
is
really
exciting
because
we're
going
to
bring
a
lot
more
personas
into
the
release
stage.
So
we're
talking
about
ux
designers,
product
managers
and
a
bunch
more.
B
I
don't
know
it's
maybe
a
little
bit
premature
to
talk
about
av
testing,
but
I
think
in
terms
of
the
think
big.
It
definitely
is
the
place
to
talk
about
a
bunch
more
personas
who
are
less
technical
than
the
developers
and
the
devops
engineers
that
need
the
space
to
know.
What's
going
on
with
their
production,
it
can
even
be
support
engineers,
so
customer
support
that
enable
or
disable
a
feature
for
a
specific
customer
is
also
super
important,
so
more
personas,
which
is
really
really
cool
for
us.
C
Have
you
talked
about
tying
a
feature
flag
or
being
able
to
trigger
like
a
b
tests
or
rollbacks
based
on
performance?
I
mean
so
many
times
we
talk
about
an
a
b
test
in
terms
of
business
outcomes,
but
we
run
into
or
I've
run
into
in
the
past
cases
where
an
a
b
test
it
gets
turned
off
because
of
impact
to
customer
experience,
because,
like
page
loads
are
so
much
slower
or
back-end
processing
is
taking
so
much
longer.
B
B
It's
usually
not
those
new
personas
that
I
mentioned,
it's
usually
still
the
developers
or
the
devops
engineers,
and
we
already
have
a
feature
that
we're
working
on
right
now
called
post-deployment
monitoring
and
when
you
think
about
it,
it's
actually
the
same
concept,
post-deployment
monitoring,
kind
of
looks
at
alerts
and
incidents
that
come
up
post-deployment
and
they
let
you
know
when
something's
going
wrong.
So
it
could
be
a
bunch
of
500s.
It
could
be,
you
know
other
stuff
and
it
could
be
based
on
an
experiment.
So
we
already
have
that
mechanism
in
place.
B
B
B
B
This
is
a
kind
of
tricky
one,
because
we
kind
of
started
with
the
solution,
which
was,
let's
make
future
flags
an
issue
type,
but
I'm
open
to
suggestions
for
anything
else
or
and
like
there's.
If
you
take
a
look
at
the
issue,
that's
linked
on
the
top
there's,
not
even
a
proposal
like
it's
just
a
general
idea.
So
I'm
really
really
interested
to
hear
any
thoughts
on
this.
A
I
was
kind
of
curious
when
you
started
talking
about
making
an
issue
type
that
is
specifically
for
feature
flags.
Does
that
mean
from
kind
of
an
experience
perspective
and
thinking
about
that
product
manager,
persona
where
there
is
a
dedicated
issue
in
the
backlog
that
needs
to
be
organized
and
and
that
kind
of
feel
to
it?
Or
do
you
think
that
the
feature
flag
kind
of
needs
to
live
in
its
own
section
of
the
ui?
That's
devoted
entirely
to
just
feature
flags.
B
So
I
think
it
needs
to
be
entirely
different
and
if
you
look
now,
when
you
create
a
new
issue
on
gitlab,
you
have
a
type
and
you
can
choose
issue
incident.
I
imagine
this
to
be
like
another
type,
which
is
feature
flags.
We
can
also
discuss
the
name
because
experimentation
experiments
would
also
be
under
the
same
category.
I
wouldn't
create
two
different
ones:
they're
very
similar
and
jackie.
I
also
mentioned
release
run
books
as
is,
should
I
say,
there'll
be
a
few
here.
E
C
B
Yeah,
so
absolutely
what
I'm
missing
here
is
the
right
hand
right
panel,
where
I
still
see
the
signings
and
the
labels
and
so
on.
But
yes,
this
is
the
idea,
and-
and
here
you
could
still
have
your
linked
issues-
ethics
requirements,
mrs
and
flags,
because
some
flags
may
be
dependent
on
other
flags.
So
that's
what
I'm
imagining
here
and
here's
your
discussion
section
as
we
mentioned.
B
D
When
you're
thinking
about
the
full
life
cycle
management
of
a
feature
flag,
will
you
ever
reuse
a
feature
flag
or
reopen
it
or
reopen?
Yes,
that's
what
I
mentioned
that
one
of
the
I'm
sorry
I
was
late
to
this.
I
don't
mean
to
rehash
something
that
you've
already
talked
about.
I
apologize.
That's
fine!
If.
B
B
You
know
it
needs
to
actually
be
facilitated
until
it's
ready
and
then
it's
on
and
then
on
which
I'm
not
sure
how
to
translate
that
yet.
But
on
is
fluid
because
it
can
be
on
based
on
the
strategy.
It
can
be
based
on
user
ids
or
it
can
be
based
on
percentage
or
others.
So
I
don't
know
if
there's
a
really
good
definition
of
what
a
done
flag
is
because
done
could
be.
I
reached
100
or
it
could
be.
I
reached
50
because
that
was
my
my
intention
now
feature.
B
B
I
want
to
see
if
it
works
well
and
if
it
does,
I'm
going
to
slowly
expose
more
users,
and
so
I'm
super
happy
with
with
the
results
and
then
I'm
going
to
leave
it
on
forever
experiments,
they're,
usually
on
for
a
finite
amount
of
time,
let's
say
30
days,
and
then
I'm
going
to
select
one
to
be
a
permanent
behavior
of
the
code
based
on
performance
or
business
metrics
or
whatever
metric.
I
wanted
and
then
there's
permanent
flag.
So
perimeter
flags
is
really
interesting
because
people
can
decide
to
enable
features
to
specific
segments.
B
C
Like
I've
always
thought
about
a
feature
flag
as
a
temporary
situation
like
in
the
context
of
how
we
talk
about
feature
flags
at
gitlab
as
part
of
our
development
process
and
more
broadly,
I've
talked
about
configuration
being
more
of
a
permanent
thing.
This
feature
is
configured
to
go
to
this
license
type
or
to
mobile
experience
or
just
windows
users,
but
not
as
a
feature
flag
feature.
Flags
have
always
thought
about
as
more
temporary
in
nature,
and
so
they
kind
of
have
that
you
have
an
end
date
for
it.
C
D
C
D
A
very
interesting
perspective,
I
have
a
very
different
experience
with
future
flags,
where
we've
always
had
them
instrumented
as
a
part
of
our
as
a
part
of
our
software
and
once
we've
added
more
features
like
incrementally.
We
may
have
needed
to
completely
turn
off
that
node,
for
example,
yeah.
So
it
would
always
live
in
perpetuity,
and
sometimes
we
would
remove
them.
You
know
like
if
we
found
that
this
was
a
stable
future
set
or
we're
done
incrementally
rolling
out
to
that
server
or
you
know
whatever.
C
I
think
about
a
feature
flag
as,
like
you,
all
of
a
sudden,
create
all
of
these
potential
divergent
paths
within
your
application,
especially
like
when
you're
talking
about
b2c
and
now,
like
you,
can
have
a
thousand
user
paths,
so
there's
no
way
to
test
all
of
them
as
you're
doing
regression
testing,
or
something
like
that.
So
you
could
find
and
create
really
funky
experiences
when
you
introduce
a
new
flag
upstream
or
a
new.
A
b
test
is
really
what
I'm
talking
about
and
then
all
of
a
sudden.
B
B
So,
for
example,
if
you're
creating
this
really
cool
application
for
playlists
of
songs,
and
you
decide
that
you
want
to
give
premium
users
as
anyone
who
has
more
than
100
songs
on
their
playlist.
You
can
make
that
rule
because
it's
based
on
your
application,
so
people
just
love
it.
Because
of
that.
B
What
do
we
say?
Can
we
move
on
to
the
yellow,
or
do
we
want
to
stay
here
on
the
ideal
solution,
yellow,
yellow
yeah,
yellow
just
to
mention
that
we
cannot
discuss
online
solution,
anything
about
dashboards
because,
I
think
that's
a
whole
different
topic,
but
yes,
managing
feature.
Flags
is
like
a
whole
new
complexity
and
another
thing,
a
big
section.
So
that's
just
like
the
teaser.
B
So
let's
go
to
yellow
okay.
So
how
do
we
differentiate
with
competition
and
that's
a
great
question-
and
this
is
following
a
conversation
that
I
had
with
jackie
feature
flags.
There's
like
a
bunch
of
companies?
That's
that's
all!
They
do
look
at
that.
It's
just
part
of
a
small
group
that
does
feature
effects
so
differentiating
is
is
difficult,
but
where
can
we
differentiate
so
I
think
anything
that
utilizes
our
gitlab
as
a
single
tool
will
give
us
differentiation.
B
B
Contextual
connection
to
source
code
is
super
powerful,
so
launch
darkly
has
a
feature
that
you
get
snippets
of
the
code
launch
dorothy
is
built
on
top
of
it's
integrated
with
other
tools
like
microsoft,
team,
tfs
or
with
alessi
and
jira,
and
you
they
they
bring
you
the
snippets
of
the
code,
so
you
can
see
like
if
there's
a
flag,
hey
you
see.
This
is
the
snippet
of
the
code
that
turned
on
the
flag,
but
it's
not
managed
by
launch
directly.
B
So
we
as
gitlab
since
we
already
hold
the
source
code,
we
can
show
the
context
to
the
code.
I
was
thinking
in
a
manner
like
you
can
see
the
code
on.
Mr,
so
you
can
see
the
entire
section
you
can
like
change
it
directly
from
there.
You
can
do
remarks,
suggestions
whatever
code
reviews
directly
through
there
and
get
really
good
compatibility
in
context
with
from
the
feature
flag.
B
So
I
was
brought
in
on
the
call
from
on
a
customer
call
from
a
large
enterprise
bank
that
are
migrating
from
another
tool
to
gitlab
and
they're.
Buying
gitlab
ultimate
and
one
of
the
things
they
were
really
excited
about
was
that
they
saw
that
our
feature
flag,
that
we
have
feature
flags
inside
that
offering
and
when
I
talk
to
them,
they're
using
launch
darkly,
which
is
an
awesome
tool.
It's
really
really
great
tool,
but
they
charge
per
api
request
and
they
have
a
million
users.
This
specific
customer
so
think
about
it.
B
If
you
get
charged
for
api
requests
and
you
have
a
million
users
using
it
on
a
daily
basis,
that
just
translates
to
a
lot
of
bucks.
So
if
we
make
this
compatible
enough
for
for
customers
to
think
about
transitioning
to
future
flags,
we
can
save
them
a
bunch
of
money
while
still
bringing
you
know,
capabilities
into
feature
flag.
So
so
I
think
that's
another
differentiation.
B
A
A
So
I
always
feel
like
that's
whenever
there's
a
new
idea,
and
we
want
to
talk
about
that
competitive
differentiator,
that's
the
area.
I
always
like
to
focus
on
it's
like
how,
since
we
already
have
all
the
other
stuff,
how
can
we
make
that
transparent,
even
if
it
is
just
a
moment
in
that
future
flag
issue?
That's,
like
click
on
this,
to
go,
see
the
code
or
see
the
mr
that
started
all
of
this
or
whatever
it
is.
Customers
love
that.
B
B
Yes,
absolutely
they
do
do
that
with
different
integrations,
but
I
think
that
we
can
have
like
really
natural
flow.
That
will
give
a
really
nice
user
experience
to
customers
another.
Like
nice
cool
feature.
I
don't
know
if
this
would
be
a
differentiator,
but
it's
nice
feature
is
the
ability
to
toggle
flags
by
quick
actions
or
markdown.
A
A
B
Possibly
the
only
thing
that
I'm
a
little
bit
hesitant
to
say
yes
is
because
the
pipelines
are
usually
building
the
deployment
out
and
it's
not
post,
and
this
usually
happens
after
something
is
deployed,
so
future
flights
usually
change
after
so
I
think
it
gives
us
a
lot
of
insight,
not
so
much
on
the
pipeline,
visible
visualization.
But
yes,
in
terms
of
metrics,
that
we
collect
incidents
alerts.
B
You
know
if
you
turn
the
flag
on
and
then
all
of
a
sudden
you
have
this
spike
in
the
number
of
500
that
you're
get
you're
getting
and
then
you
go
and
drill
down,
and
you
see
that
that
this
started
when
the
feature
flags
turned
on,
then
you
know
you
know
the
root
clause.
It's
really
easy
to
turn
it
off.
D
One
opportunity
for
differentiation,
I
also
see,
is
remediation
with
future
flags
so
like,
if
there's
ways
to
automate,
you
know
kind
of
going
back
to
our
previous
conversation
rollbacks,
but
also
building
quick
commands
inside
of
our
run
books.
So
if
you
have
a
quick
command
turning
on
and
enabling
a
feature
flag
building
in
those
scripts
from
your
run
book,
so
that
if
a
user
has
a
threshold
met
within
their
monitoring,
they
just
turn
on
that
feature
flag.
B
Yeah,
so
I
think
that's
really
really
awesome.
I
don't
know
like
larger
issues.
I
kind
of
feel
like
this
has
just
become
a
feature
flag
conversation,
but
I'm
going
to
flow
I'm
going
to
go
with
it,
not
specifically
to
this
feature,
flag
issue
type,
but
we
have
like
a
lot
of
ideas
of
how
to
utilize
like
make
review.
Apps
feature
plans.
B
A
All
right,
this
is
so
cool.
The
next
part
of
our
conversation
is
the
think
small
aspect,
so
we've
had
quite
a
bit
of
conversation
around
how
awesome
future
flags
are
and
how
much
they
do
way
more
than
I
thought
they
did
so
when
thinking
about
creating
this
issue
type
that
has
the
archive
of
information
and
a
single
source
of
truth
with
labels
and
authors
and
owners
and
stuff
like
that.
A
B
Yeah,
so
that's
a
really
really
good
question
and
one
that
I
one
of
the
reasons
why
I
wanted
to
talk
about
it
in
this
form,
because
I
know
that
we're
all
well,
not
all,
but
some
of
us
are
working
towards
this-
create
this
new
issue
type.
So
I
think
we
need
to
figure
out
the
first
steps
anyway,
regardless
of
what
the
issue
type
is.
B
D
Ones-
and
that
is
the
paradigm
that
monitor,
has
implemented
with
incident.
So
if
we
go
down
that
path,
it's
a
pretty
nice
iteration
path.
It's
easily
chunked
into
multiple
milestones
already
following
that
paradigm,
there
might
be
some
other
lighter
weight,
instrumentation
that
we
could
also
discover
as
well
when
it
comes
to.
How
do
we
think
about
settings
inside
of
feature
flags
to
make
it
easier
to
configure
and
set
up
future
flags
for
users
so
that
their
experience
is
more
inside
their
existing
flow?
D
If
we
think
about
all
the
personas
and
the
entry
points
for
future
flags,
it's
something
really
broad.
You
know
like
we
could
be
thinking
about
an
operations
team
member,
a
release
manager,
a
developer,
so
we
need
to
consider
settings
not
being
locked
behind
the
maintainer
progression
set,
but
then
also
allowing
potentially
organizations
to
configure
elevated
permissions
for
certain
environments
behind
that
lock
and
key,
so
making
sure
that
we
have
these
tiers
of
configuration
is
important,
but
I
think,
from
a
think
small
perspective.
That's
been
paved
pretty
pretty
well
from
an
issue
type
view.
B
D
Oops,
I
think,
unfortunately,
this
is
so
solution
to
specific
right
now
that
it
makes
it
kind
of
hard
to
think
small
on
it
and
because
we
already
have
like
a
good
example
of
how
to
iterate
from
monitor
with
creating
an
issue
type.
So
maybe
we
could
explore
if
there's
another
opportunity
to
create
feature
flag
workflows.
D
So
if
we
were
to
look
at
a
different
entry
point,
james,
going
back
to
your
a
b
testing
perspective,
how
do
you
foresee
feature
flags
playing
into
your
workflows
and
how
could
we
introduce
future
flags
into
a
micro
experience
today?
I
think
we
lost
james,
oh
geez
dang.
He
was
the
only
other
person
that
was
engaged.
D
F
B
Okay,
let's,
let's
walk
you
through
an
experiment,
let's
say:
you're
you're
thinking
about
designing
a
whole
new
issue,
type
called
feature
flag
and
you
want
to
create
an
ap
test
to
see
which
one
is
easier
for
the
users
to
to
see
and
now
you're.
You
have
like
a
few
different
designs
and,
and
you
want
them
exposed
to
different
audiences.
B
B
Okay,
so
let
me
take
you
through
another
workflow
you're
doing
pipeline
visualization,
and
you
have
a
few
ideas
of
what
this
should
look
like
in
the
for
the
users.
Now
you
need
to
create
an
issue
to
describe
this
and
you
have
like
three
or
four
different
variants
of
mockups
that
you
want
to
try
out.
How
would
you
open
that
up?
B
B
F
F
So,
let's
say:
if
I
talk
about
that
visualization,
what
we
are
doing
is
there
was
one
mvc
that
was
rolled
out,
but
we
are
bringing
in
the
same
capabilities
to
our
static
visualization
as
well
and
what's
happening
is
our
static
visualization,
along
with
telling
the
dependencies
to
the
users
between
jobs,
it's
going
to
also
communicate
to
them
all
the
other
kinds
of
information
which
the
other
dedicated
visualization
would
not,
and
how
would
we
present
both
of
them
to
users?
Can
someone
please
help
me
out
here.
B
So
you
have
two
different
mock-ups.
How
do
you
decide
who
gets?
What
is
there
does
it
matter
if,
like
if
it's
arbitrary,
then
you
just
say:
50
get
this
and
50
get
this.
If
it's
important
to
you
like,
let's
say
based
on
persona,
then
you
give
a
specific
persona,
a
specific
mockup
like
how
would
you
target
it.
F
B
So
it's
a
little
complex
and
also
what
I'm
scared
about-
and
maybe
this
is
just
me
being
cautious-
is
that
people
start
using
something
that
I
decide
to
remove
in
like
two
milestones-
and
I
don't
know-
maybe
we
can
get
away
with
it
by
saying
it's
an
alpha
or
something
like
that,
but
but
that's
what
I'm
a
little
bit
hesitant
about.
I
mean
like
there's
some
things
and
issues
that
make
no
sense
to
to
have
as
a
feature
flag.
In
my
mind,.
D
Yeah,
so
here's
how
I
would
approach
it.
I
would
do
all
of
these
iterations
behind
a
future
flag
and
only
make
it
available
on
the
front
end.
Once
I
have
the
actual
issue
type:
that's
attached
to
feature
flag
strategies
that
has
the
ability
to
attach
it
to
an
environment,
because,
like
your
your
job
to
be
done
for
future
flag
is
specific
to
an
environment.
So
an
issue
type
is
not
specific
to
an
environment.
D
So
in
order
for
a
user
to
be
able
to
turn
off
and
on
a
feature
flag
for
an
environment,
they
would
need
to
be
able
to
do
that
first.
So
if
we
approach
it
from
like
a
job
to
be
done
perspective,
I
think
that
the
the
mvc
for
this
can
be
not
usable
and
have
all
the
stuff
for
an
issue
type,
and
you
don't
have
to
ship
that
to
production
in
a
viewable
state.
F
Yeah
that
makes
sense
there's
another
idea
that
I
just
had
that
how
about
tying
in
some
telemetry
tools
to
this,
so
that
we
are
able
to
trace
the
action
flow.
I
mean
we
can
follow
the
clicks
like
what's
the
if
they
are
entering
a
feature,
are
they
staying
there
and
accomplish
accomplishing
what
they
have
to
and
then
just
going
away
to
perform
the
other
tasks,
or
are
they
switching
the
views
to
find
and
alternate
and
what
they
need
is
not
provided
in
the
new
feature.
F
So
I
I
briefly
used
this
telemetry
tool,
which
allowed
me
to
track
the
actions
that
users
are
performing
on
a
particular
product
and
it,
let
me
understand,
like
what
was
the
flow
of
their
actions
as
in
if
they
are
entering
from
this
particular
page,
which
page
are
they
navigating
to?
Are
they
going
back
and
forth?
Are
they
staying
there
for
some
time
or
are
they
looking
for
some
alternative?
Are
they
feeling
lost,
or
are
they
just
leaving
it?
There.
B
Yeah,
so
absolutely
that
connects
again
to
a
b
testing
and
an
a
b
test
has
to
be
connected
to
some
kind
of
metric
system,
we're
thinking
about
starting
with
snow
cloud,
because
we
can
use
it
internally
in
gitlab
and
then
based
on
the
results
of
whatever
the
metrics
are.
That's
the
result
of
the
experiment
and
that's
when
you
know
like
what
variant
to
choose.
B
So,
if
we're
talking
about,
let's
say
two
different
mockups
that
we
have
for
some
feature:
let's
just
use
pipeline
visualization
for
for
the
example
and
you
just
roll
it
out
to
50
of
the
users,
and
you
see
that
one
of
them
has
just
so
much
more
engagement
on
it,
because
there's
more
people
clicking
or
more
people,
I
don't
know
completing
the
stage
or
whatever.
Then
that's
the
one
you
choose.
So
in
the
concept.
Is
you
define
the
experiment?
F
B
A
All
right,
we
are
at
time
at
get
lab
time
that
10
minutes
fast.
Does
anybody
have
any
final
thoughts
of
quick,
iterations
or
ways
that
we
could
kick
this
off
the
ground?
I
did
want
to
call
out.
It
would
be
kind
of
interesting
similar
to
jackie's
idea
of
kind
of
building
them
building
your
feature,
flag,
idea
behind
a
feature
flag
and
then
experimenting
internally.