►
From YouTube: Testing Group Think Big #6
Description
This is a follow-up to our Think Big about Custom MR widgets; https://youtu.be/mFkPiVb-9PE.
Issues created/found from the sessions:
- Blog/video about how to use existing functionality to get data into an MR: https://gitlab.com/gitlab-org/gitlab/-/issues/233543
- Don't show Testing MR Widgets if green - https://gitlab.com/gitlab-org/gitlab/-/issues/233540
- Minimal Custom MR Widget - https://gitlab.com/gitlab-org/gitlab/-/issues/26533
A
All
right,
this
is
the
think
big
for
the
verify
testing
team
for
august
4th.
This
is
actually
a
think
small
session
today.
So
this
is
follow
up
from
last
week's
think
big,
around
user
defined.
Mr
widgets,
I'm
going
to
do
a
quick
recap
which
is
mostly
reading
from
the
dock
and
then
we'll
jump
into
the
think
small
part.
A
A
That's
the
problem
that
we
have
and
that
there
are
different
personas
who
might
build
an
mr
widget
and
might
use
an
mr
widget.
The
best
example
that
I
read
through
the
notes
is
probably
the
security
one
where
a
security
researcher
or
security,
someone
on
the
infosec
team
or
security
team,
rather
is
gonna,
create
that
widget
for
developers
to
see.
A
So
as
part
of
that,
we
thought
a
great
outcome,
or
a
really
amazing
outcome
would
be
that
there's
a
wysiwyg
editor
that
admins
can
use
to
build
these
widgets
decide
what
information
appears
in
the
widget
and
decide
a
threshold
for
hey.
When
will
this
fail?
The
pipeline
block
emerge
from
the
data,
so
that's
a
high
level
recap
of
what
I
just
read
in
the
notes
10
minutes
ago.
A
B
This
might
be
thinking
too
small,
but
there's
no
such
thing
drew
has
kind
of
found
that
problem
that
we
identified
with
the
murder
quest.
Was
it
comparing
reports
to
maybe
not
super
correct
things,
and
that's
why
some
of
the
results
in
the
widget
aren't
accurate?
I
don't
think
we
can
move
on
any
of
our
grand
ideas
without
kind
of
solidifying
and
making
what
we
have
right
now
useful.
A
All
right,
so
that
would
be
a
good
way
of
increasing
confidence
in
the
mr
widget.
One
of
the
other
things
I
didn't
call
out
in
the
recap
was
part
of
the
part
of
the
problem
is
also
just
the
amount
of
data
that
shows
up
on
the
mr,
mr
page
today
is
there
something
that
we
could
test
around
reducing
that
amount
of
data.
B
I
was
thinking
about
essentially
demoting,
some
of
the
widgets,
so
there's
this
there's
this
this
fun
table
in
the
agenda
that
ricky
pasted
in
there.
Thank
you
and
I
sort
of
I
got
the
idea
that
I
so
I
was
thinking
about
widgets
in
comparison
to
the
danger
bot
and
the
danger
bot
is
very
the
danger.
Bot
is
nearly
an
automatic
process
for
us
already.
B
B
That
is,
I
think,
by
nature,
a
more
suggestive
sort
of
piece
of
content
and
we
have
some
widgets
that
are
you
know,
theoretically
things
that
are
blocking
it's
like.
Oh
it's
red
x,
on
your
merge
request
and
in
practice
we
read
those
like
suggestions
and
we're
sort
of
like.
Oh,
is
there
a
code
quality
issue?
Is
there
a
security
problem?
I
might
go
check
that
out
again
before
I
merge
this.
B
So
if
I
think
by
removing
widget
removing
widgets
that
we're
not
very
confident
in
to
the
level
of
we
can
automate
review
on
these,
it
would
increase
the
trust
of
the
ones
that
remain.
You
know
it.
It
give
a
status
to
the
widgets
that
you
know
this
really
is
a
blocker,
so
you
know
we
could
we
could
remove.
B
We
could
make
security
violations
and
code
quality
comments
and
then
make
dangerbot
a
widget.
You
know
a
dangerbot
does
a
lot
of
sort
of
checks
already,
but
essentially
increasing
trust
in
the
widgets
by
removing
like
taking
an
axe
to
the
ones
that
aren't
trustworthy
and
demoting
them
to
comments
and
be
like
these
are
remarks
or
suggestions
that
you
should
read,
but
you
can
merge
without
them.
B
A
So
we
introduced
a
threshold
in
the
web
performance
widget,
where
it
does
not
display,
unless
I
don't
remember
which
data
point
we
use,
but
we
kind
of
introduced.
That
is
there
a
way
to
look
at
data
from
users
who
are
using
it
to
see
hey.
Is
this
a
good
idea
to
add
more
thresholds
so
that
widgets
that
actually
appear
are
actionable
and
people
trust
that
they
do
need
to
do
something,
whether
it's
blocking
or
not?.
B
Yeah
so
there's
two
kinds
of
thresholds
in
there
there's
like
the
threshold
of
severity
right.
If
we
of
you
know
the
the
widget
says,
this
is
a
warning
or
the
widget
says
this
is
an
error
and
there's
also
a
threshold
of
how
confident
are
we
that
a
warning
is
really
a
warning
and
an
error
is
really
an
error
and
I
don't
know
how
to
measure
that
second
one.
B
It's
going
to
take
a
while
to
get
us
to
the
point
where
we
are
able
to
have
that
much
confidence
in
the
widgets,
and
I
think
we
got
to
come
up
with
a
plan
to
get
us
there.
That's
going
to
be
a
little
bit
more
long
term
and,
like
I
said
the
first
and
like
you,
also
acknowledge
the
first
step
in
building
up
that
confidence
is
to
fix
some
of
the
issues
with
the
way
the
reports
are
being
compared
and
make
sure
that
there's
a
high
signal
to
noise
ratio
in
those
mr
widgets
from
there.
B
I
think
we
talked
about
last
time.
We
were
talking
about
the
checks
api
or
like
how
can
we?
How
can
we
codify
and
allow
for
a
configuration
that
allows
people
who
use
gitlab's,
ui
to
say,
hey?
This
needs
to
block
merging
and
this
shouldn't
block
merging
and
having
that
kind
of
like
okay,
this
block's
merging,
if
it
says
this
this
box
merging
if
it
says
this
and
have
that
be
configurable
and
customizable,
and
that
way
after
we
start
ironing
out
our
widgets
and
we
know
okay.
B
Actually,
if
the
code
quality
widget
says
something
wrong,
something
is
wrong
and
then
we
can
toggle
that
setting
that
basically
says
okay
block
merges
when
the
code
quality
widget
is
red.
You
know
what
I
mean
like
having
that
flexibility.
You
also
don't
want
to.
Maybe
you
probably
don't
want
to
mandate
that
across
the
app
entirely
like
you,
don't
want
to
say:
okay,
cool
quality
widget
now
blocks
merges
for
everybody,
because
different
organizations
are
gonna,
have
different
needs.
A
B
B
So
the
more
rigid
you
can
make
a
user-defined,
widget
or
piece
of
information
on
a
page,
a
better
chance
that
you're
going
to
have
a
high
signal
interaction,
whereas
the
more
flexible
you
make
something
so
wysiwyg
or
whatever
the
the
more
possibility
that
you're
gonna
have
low
signal
to
noise
ratio.
You're
gonna
have
more
stuff
in
there.
So
the
way
that
I
was
looking
at
that
is
kind
of
like
well.
A
A
C
D
D
A
Point
first
of
there's
data
you
want
to
get
into
an
mr
that
expose
as
for
the
artifact,
is
it
enough?
You
want
more
context
actually
into
the
mr
itself.
Here
is
a
way
to
do
that
and
then,
if
the
follow-on
to
that
is
great,
but
I
also
want
to
block
merge
or
I
need
to
you
know-
trigger
some
other
action,
then
that
would
be
learning
from
a
first
step.
There
right.
B
So
I
think
the
reason
another
reason
why
I
think
that,
having
kind
of
that
rigid
structure
with
high
signal
would
be
like
a
good
like
80
20
trade
off
is
because,
like
actually
making
user
defined.
Mr
widget,
that
is
super
rigid,
is
going
to
be
a
lot
easier
on
the
engineering
side
than
oh
yeah
for
sure
jamming.
Another
wysiwyg
into
the
mr
thing,
because
that's
going
to
be
a
whole
can
of
worms.
B
A
Is
there
an
even
easier
iteration?
Is
there
something
lower
that
is
there
like
documentation?
We
can
write?
Is
there
a
sample
project
we
can
set
up?
Is
there
a
survey
we
could
run?
That
would
help
us
answer
the
question.
How
many
customers
want
more
data
into
the
mr
that
they
can't
that
they're
not
getting
today.
B
A
A
All
right,
I'm
gonna
put
a
pause
on.
What
is
the
next
thing
we
could
build?
I
just
want
to
flush
out
a
little
bit
of
the
hardest
part.
What
I'm
taking
away
from
this
conversation
I
know,
but
I
want
to
see
what
what
is
the
hardest
part
or
what
does
the
team
think
is
the
hardest
part
of
this
problem
to
solve
around
giving
a
user
letting
users
put
information
into.
B
Nmr
that
letting
people
insert
whatever
they
want
in
a
very
open
way
decreases
the
strength
of
the
signal
in
the
merge
request,
the
more
the
more
freedom
we
give
people,
the
less
we
know
about
what
they've
actually
done
with
it,
so
we
can't
make
assumptions
and
just
say,
something's
red.
This
is
not
mergeable.
B
A
A
B
A
Yeah,
so
what
I
was
interpreting
as
the
hardest
problem
to
solve
is
making
sure
that
the
data
is
of
high
enough
quality,
that
it
means
something
to
the
user.
But
I
don't
know
that
that's
our
problem
to
solve,
if
you
as
an
engineer
at
acme
anvil,
need
information
about
the
tensile
strength
of
the
steel
of
the
anvil
as
part
of
your
build
pipeline.
A
Great
we're
going
to
give
you
a
way
to
do
that,
and
it's
kind
of
up
to
you
to
understand
your
own
context.
Now
we're
going
to
do
that
with
our
own,
mr
widgets,
like
code
quality
and
web
performance,
and
things
like
that.
We
need
to
understand
those,
but
I
don't
know
that
we
can
build
a
interface
that
allows
for
you
have
enough
context
here
in
this,
mr
widget,
to
make
it
actionable
to
your
own
developers
or
not.
B
C
B
When
I
was
thinking
about
that
question
about
what
the
hardest
thing
to
solve,
is
that
my
mind
went
in
a
completely
separate
direction?
Oh
okay,
yeah!
For
for
me,
it
seems
like
the
mr
widget
space
is:
has
no
owner,
there's
no
dri,
there's
no
person,
that's
concretely
responsible
for
mr
widgets.
I
think
the
biggest
challenge
is
going
to
be
getting
cross-departmental
buy-in
for
revamping
the
mr
widget.
D
D
B
A
D
B
It
depends
on
how
we
decide
to
do
it
like
there's,
probably
going
to
be
a
limited
set
of
mr
widgets
that
are
built
directly
into
the
application.
If
we
allow
the
users
to
define
their
own,
mr
widgets,
then
it'll
just
be
as
simple
as
displaying
what
they
gave
to
us.
It
wouldn't
necessarily
slow
down
the
page
if
we're
storing
that
information
in
a
clever
way-
and
you
know
expiring
it
and
not
having
it
be
calculated
every
time.
B
I
think
the
the
challenge
is
that
a
lot
of
the
mr
widgets
that
we
do
build
into
the
application
we're
actually
calculating
on
the
fly
in
order
to
display
the
results,
particularly
the
ones
where
we
have
two
reports
that
we're
diffing
and
then
putting
output
of
that
diff
into
the
widget.
That's
a
computationally,
expensive.
D
Process
yeah,
that's
what
I
was
thinking
of
if
we're
just
taking
text.
You
know
that
they
have
generated
in
some
other
place,
but
it
still
had
to
generate
it
somewhere
right.
So
I
was
thinking
of
yeah,
so
whatever
pipeline
time
it
takes.
For
that
I
mean
I
don't
know,
but
but
from
a
page
perspective
you're,
I
think
you're
right
that
shouldn't
affect
it.
D
B
A
Accommodation
of
the
metrics
widget,
if
you're
at
a
premium
using
expose
as
to
show
artifacts
that
you've
generated
that
we're
not
parsing
for
you
and
potentially
like
and
here's
how
you
would
add
comments
into
your.
Mr,
like
here's
three
approaches
to
getting
other
information
in
your
pipeline
and
then
looking
at
engagement
with
the
post
or
the
unfiltered
video
that
might
be
a
good
way
of
measuring
hey.
Is
this
a
problem
that
people
have
and
then
we
could
explore
a
different
solution
space
for
solving
it.
A
A
A
D
One
quick
question:
so
the
metrics,
the
mr
widget
for
the
custom
metrics,
it's
literally
just
a
dumping
ground
for
everything
you
want
to
put
into
it.
So
it
takes
every
possible
thing.
You
know,
if
we're
going
to
build
on
that
future
wise
to
become
a
potential
blocker
to
be
able
to
allow
somebody
to
configure
it
come
up.
Blocker
we're
gonna
have
to
have,
I
think,
probably
multiple
custom
metric
widgets
show
up.
B
A
B
Think
I
think
that
simple
user-defined
widget
is
like
the
next
iteration
after
we
check
on
the
metrics
widget.
Okay,
can
the
metrics
widget
solve
some
of
these
problems
and
what's
the
next
iteration?
Okay,
we
need
this
simple
line
of
text
link
checkbox.
Maybe
you
have
five
of
those
if
you're
testing
various
anvil,
hardnesses.
A
A
Thank
you
team.
This
has
been
another
great
think,
big
thing
small.
I
think
I'll
write
up
some
issues
and
drop
them
into
the
channel
and
link
to
them
in
the
unfiltered
video
if
you're
watching
later
so
you
can
click
on
those
below.
Thank
you,
parker
and
ricky,
and
probably
joanna.
I
saw
you
jump
in
late,
but
I
assume
that
you
also
took
notes,
because
you
always
do.
Thank
you,
everyone
for
helping,
take
notes
and
keeping
us
on
track.