►
From YouTube: CI/CD UX meeting - March 29th, 2023
Description
The CI/CD UX team got together for our bi-weekly meeting. Emily Bauman (Senior Product Designer) presents a Kubernetes analogy. The team goes through each group and provides updates. Erika Feldman (Senior UX Researcher) shares Verify UX Benchmarking study results.
B
I
did
a
lot
of
reading
about
kubernetes.
A
lot
of
it
was
in
a
very
technical
jargon
and
nothing
was
sticking
until
I
came
across
the
post
office
metaphor,
where
it
kind
of
subbed
in
kubernetes,
as
like
a
post
office
management,
and
everything
here
then
started
to
make
sense
with
me.
So
this
is
just
like
a
cute
little
story
about
that.
B
B
To
help
achieve
this,
there's
a
team
of
postal
workers
responsible
for
processing
and
delivering
packages.
However,
with
packages
there's
many
challenges
such
as
sometimes
there's
an
influx
of
packages
coming
in
there's
incorrect
addresses
that
need
to
be
fixed,
and
this
is
where
the
orchestration
portion
comes
into
play.
B
So
when
you
think
about
kubernetes
in
the
same
frame
as
a
post
office,
you
can
think
about
containers
which
is
a
package
of
code
as
like
the
mail
packages,
and
when
you
get
a
group
of
containers
together
into
pods,
you
can
think
of
this
as
just
a
group
of
Mail
Packages
that
need
to
be
delivered
to
the
same
place.
So
you
have
now
that
group
of
Mail
Packages
needing
to
get
somewhere
and
then
there
was
no
emoji
for
a
postal
worker.
So
just
pretend
he
is
a
postal
worker.
B
It
also
makes
sure
workers
have
the
resources
necessary
to
handle
each
pod,
such
as
enough
room
in
their
delivery
truck,
which
is
the
CPU
or
memory,
and
then
kubernetes
also
makes
sure
that
each
worker
is
not
overwhelmed
by
packages
by
doing
the
load,
balancing
or
adjust
the
number
of
workers
based
on
how
many
packages
are
coming
in,
which
is
the
automatic
scaling,
and
if
a
worker
calls
in
sick
or
a
node
becomes
unavailable,
kubernetes
can
help
get
the
new
worker
on
the
job.
So
the
delivery
goes
uninterrupted.
B
So
yeah,
you
can
think
of
kubernetes
as
an
operating
system
that
manages
the
processing
and
delivery
of
packages
in
a
post
office
by
ensuring
they're
all
assigned
to
a
worker
and
that
the
resources
are
allocated
efficiently,
so
workers
can
do
their
job.
So
this
is
just
a
very,
very
tiny
story.
I
put
together,
but
when
doing
my
onboarding,
this
is
the
one
thing
that
helped
me
really
really
understand
what
kubernetes
was
and
I
thought
it
was
cool
to
share
out
that
with
everyone
else.
A
That
was
so
great
I
love
that
and
it
makes
it
like
so
much
more
obvious,
I
think
as
to
what
kubernetes
means
and
I
have
one
question
for
you.
Yes,
which
is:
do
we
the
the
information
that
you're
saying
like
CPU
and
memory
that
is
there
and
if
a
node
goes
down
and
stuff,
do
we
get
that
visibility
into
git
lab,
or
do
you
only
get
that
when
you're,
using
whatever
kubernetes
platform
you're
using.
B
D
Yeah
I
just
wanted
to
point
out
I
believe,
like
I
love
the
presentation,
it's
super
simple
with
like
the
Emoji
theme,
I
love
it
and
then
what
I
would
feel
like
honest
feeling,
I
needed
this
like
three
years
ago,
when
I
started
to
work
on
the
distribution
team
I
with
a
lot
of
peers.
We
had
a
lot
of
time
watching
the
YouTube
developers
talking
about
kubernetes.
It
wasn't
easy.
Ali
is
laughing
because
he
knows
I
know
that
Ali
was
reading
a
book
about
kubernetes.
So
this
is
what
I
needed.
D
So
thank
you
for
that
and
I
I
also
really
like
the
point
that
Gina
just
mentioned
that
he,
this
concept
could
be
work
like
load
balancing
and
then
it's
kind
of
make
things
more
scalable
and
then
I
think
it
could
be
also
applied
to
Runner
and
many
other
places
and
I
think
that's
the
reason
why
kubernetes
is
getting
more
popular,
so
two
thumbs
up
for
your
presentation.
Thank
you.
B
B
E
Think
here
next
and
yes,
thank
you,
Emily
I
wanted
to
share
like
the
origin
story
which
I
don't
know.
If
you
know
this,
but
this
really
helped
me
in
as
much
as
I
understand
kubernetes,
which
I
don't
think
anyone
ever
will
so
there's
that,
but
so
it
started.
This
is
like
an
Erica
version,
so
factual,
don't
just
we're
going
with
the
story.
Origin
story,
okay,
so
Pokemon
go.
E
Was
this
big
app
right
where
everyone
was
trying
to
catch
them
all,
and
it
was
this
huge
craze
and
it
was
a
Google
startup
that
was
in
charge
of
it
and
spinning
it
up.
So
what
happened?
Was
they
didn't
anticipate
that
so
many
people
would
be
using
it
so
quickly
and
within
the
first
three
days
everything
crashed
because
they
couldn't
serve
those
users
and
then
they
were
like
well.
F
F
It
was
a
great
presentation,
thanks
Emily
and
it
actually
like
it.
It
kind
of
explained
in
such
a
simple
way.
The
whole
like
the
environment.
When
I
talk
about
everything
related
to
communities
like
load,
balancers
workers
I
mean
what's
the
individual
rule.
I
have
been
through
some
pretty
complex,
comic
books
in
the
past,
but
they
did
not
do
this
job
so
yeah.
B
Well,
I
I
I
took
the
inspiration
from
one
of
the
courses.
I
did
actually,
it
might
have
been
the
documentary
we'll
just
mentioned,
but
someone
said
this
and
then
that's
how
that
was
really
my
my
light
bulb
moment.
So
all
the
thanks
has
to
go
to
I
believe
it
actually
was.
The
documentary
I
made
a
link,
the
wrong
thing
in
my
slides,
but
this
documentary
will
kind
of
linked
here,
as
was
great
in
helping
me
understand,
kubernetes.
G
Like
briefly
touched
on
what
you
covered
in
your
slides,
like
maybe
the
first
two
or
three
slides
that
didn't,
go
like
very
deep
with
that
metaphor,
so
it
was
nice
to
like
have
a
better
understanding
of
it,
and
then
it
also
I
think
it's
like
a
two-part
documentary,
because
the
whole
thing's
posted
to
YouTube
I
think
it
was
a
section
of
it.
That
also
touched
on
what
Erica
mentioned
with
like
Pokemon
go
is
kind
of
that
example.
C
B
C
C
G
C
D
D
C
A
C
A
C
A
I
see
this
sort
of
reminds
me
of
well.
Actually,
no,
it
doesn't
I
was
going
to
say
we
deal
with
tokens
that
are
only
displayed
for
a
certain
amount
of
time
in
the
UI
for
Runner,
and
we
just
tell
them
like
this
is
going
to
be
displayed
only
for
a
short
period
of
time
so
copy.
We
give
them
the
chance
to
copy
it
so
that
they
don't
lose
it,
but
then,
if
they
lose
it
like
it's
on
them,
so
that's
kind
of
tell
them
right
now.
C
A
A
All
right
mine,
the
first
one,
is
just
read
only
and
then
I'm
starting
to
validate
runner,
Fleet
dashboard.
You
might
have
seen
I
shared
it
in
our
York's
co-working,
Channel
and
I'll
just
share
the
issue
that
I'm
doing
this
through,
because
we
have
a
very
hard
time
finding
Enterprise
customers
to
meet
with
and
there's
this
process
in
the
handbook
that
you
can
lean
on
customer
success
managers
for
that
so
yeah.
If
you
want
to
take
a
look
at
that
issue,
if
that
applies
to
you
in
the
future,
it
might
be
helpful.
D
A
B
Oh
so
I
just
had
one
open
question
about
larger
designs
that
are
put
in
through
a
lot
of
small
Mrs
that
are
kept
behind
a
feature
flag.
I
was
getting
this
question
from
some
of
the
engineers
as
if
they
get
a
ux
review
for
each
small
Mr
or
a
larger
view.
At
the
end,
mostly
being
they
were
doing
small
reviews,
personal
Mrs,
but
we
were
getting
people
wondering
why
the
experience
wasn't
complete,
like
you,
couldn't
go
through
everything,
because
it
was
just
one
piece
of
VMR
and
then
the
rest
seemed
broken.
A
B
Well,
thanks,
yeah
I
think
the
problem
was
we're
not
like
clear
enough
where
it
is
in
the
experience,
because
the
issue
it
links
to
links
to
the
entire
feature,
not
just
the
little
portion,
so
I
guess
just
being
a
lot
more
clear
about
what
the
Mr
is
changing
and
what
is
out
of
scope
for
that
particular
Mr.
Yes,.
A
F
Vitica
did
you
wanna
yeah?
It
was
very
simple
to
what
Gina
said
so
this
I
experienced
when
we
started
to
work
on
the
CI
job
token
I'm,
trying
to
find
the
Mr,
but
like
that
proposal
it
had
to
be
broken
down
to
like
a
few
parts
like
some
preparation
by
the
front
end
some
preparation
by
the
back
end
and
then
eventually
they
would
like
Club
all
of
that
to
form
the
final
to
work
on
the
final
development
and
implementation.
F
So
for
that,
what
we
the
process
we
followed,
was
Engineers
used
to
invite
me
first
to
do
a
ux
review
and
to
make
things
easier
for
them,
like
once,
I
reviewed,
I
left
a
very
detailed
sort
of
summary
for
the
next
designer
who
will
be
assigned
by
the
reviewer
to
look
at
so
I
explained
to
them
that
this
is
the
bigger
proposal.
This
Mr
only
takes
care
of
like
this
small
part
of
that
solution,
and
eventually
this
will
lead
to
something
else,
so
yeah
that
that's
how
we
did.
B
It
no
thanks
for
sharing
that'll
help
us
a
lot
with
there's
two
big
initiatives
going
on
on
my
team
right
now
and
I.
Think
we'll
have
to
do
this
for
both
of
them.
So,
okay,
yeah.
My
final
thing
is
I'm
finalizing
the
group
level
environment
solution,
validation
study
this
week,
so
hopefully
this
will
make
some
progress
after
being
paused
during
kind
of
like
the
team
switch
time.
F
Thanks
I
just
have
this
small
update,
like,
besides
all
the
state
troop
work
that
I
anyway
keep
sharing
and
co-working
and
other
channels
that,
in
preparation
for
the
ux
team
workshop
that
I
thought
I
would
be
conducting
next
month.
I
figure
that
we
don't
have
a
vision
documented
for
pipeline
security.
F
So
I
created
sort
of
this
homegrown
template
to
have
that
conversation
that
discussion
so
that,
eventually
like
we
make
sure
that
whatever
Vision
we
arrive
at.
It
is
done
like,
after
being
informed
of
everything
that
else
that
we
have
in
front
of
us.
So
the
first
mural
is
where
we
started
from
I
just
made
a
copy
when
I
created
that
so
that
was
the
very
first
date.
F
The
plan
was
that
we
will
create
clusters
like
we'll,
be
bringing
together
item
that
could
form
one
theme
and
we
ended
up
with
four
groups
of
information
or
of
like
the
work
that
you
want
to
do,
and
then
we
just
voted
on
them.
One
of
the
options
that
we
had
was
to
maybe
take
it
forward
and
instead
do
a
like
come
up
with
the
rise
score,
but
we
refrain
from
doing
that
since
we
are
going
to
be
doing
the
ux
team
workshop
anyway,
like
in
future.
F
So
this
we
just
kept
this
very
simple
and
once
we
identified
the
larger
themes,
the
next
step
we
took
was
put
them
like
use.
This
business
model
innovation
framework
to
bring
together
like
who
are
we
making?
Who
are
we
creating
the
solution
for
how
we
are
planning
to
do
that
and
to
achieve
that?
How
what
is
it
that
we
would
be
working
on
in
terms
of
features
in
terms
of
changes,
then?
F
Why
do
we
think
like?
Why
are
we
confident
that
these
set
of
features
are
going
to
like
solve
the
problem,
and
the
good
part
with
this
Innovation
framework
is
like
Hue
change?
If
you
make
slightest
of
change
in
one
part
of
this
diagram,
you
will
have
to
like
make
adjustments
all
throughout.
So
it's
always
balanced.
It's
never
imbalanced
and
it
worked
out
pretty
well
for
us.
F
D
So
for
the
pipeline
offering
we're
focusing
on
the
solution,
validation
for
the
placement
like
where
we
place
this
catalog
feature.
Is
it
the
left
side,
navigation
or
is
it
more
embedded
version
on
the
pipeline
editor
and
there
were
another
option
in
the
navigation
but
inside
the
Explorer
tab,
which
is
the
same
level
with
the
organization,
the
new
tab,
but
it's
slightly
behind
it.
So
it's
the
second
step,
so
we
just
try
to
work
on
another
round
of
research,
so
thanks
Erica
to
jumping
in
so
we
just
started
this
conversation.
D
How
can
we
pursue
with
the
new
test
and
then
once
we
got
the
result?
Probably
we
can
more
confidently
say
okay,
so
we
should
play
this
feature
to
where
so
I'm
also
working
on
that
I
will
work
on
that
also
the
rest
of
the
week
and
there's
another
very
critical
discussions,
which
is
mostly
about
like
how
we
architecture,
how
we
create
architecture,
folder
back-end,
which
will
eventually
impact
the
ux
work.
D
So
I'm
really
glad
that
I'm
participating
in
this
discussion
and
just
trying
to
follow
up
what
they're
saying
and
then,
if
we
make
this
decision,
then
we,
our
MVC
scope,
might
be
slightly
changing
in
terms
of
technical
perspective,
but
not
a
lot
from
the
ux
perspective.
So
this
is
still
ongoing.
Just
wanted
to
share
with
you
all
and
yeah.
That
was
my
agenda.
Do
you
have
any
questions
feedback.
E
Verbalize
your
comments,
yes
yeah,
so
I
think
that
this
has
been
like
really
good
work
and
we're
trying
to
figure
out
like
the
way
to
approach
this
solution.
Validation.
But
what
we're
going
to
end
up
doing
is
having
an
approach
that
other
teams
can
use
and
basically
like
what
we've
arrived
at,
is
having
two
groups,
one
that
has
the
side
nav
and
the
Prototype
one
that
doesn't
have
a
side
nav
and
a
prototype
same
tasks.
E
Different
entry
points,
performance,
comparison,
maybe
a
satisfaction
question
in
there
and
I
think
that's
what
we're
gonna
need
to
make
an
argument
for
the
side
nav,
so
just
wanted
to
like,
say
that
this
was
a
hard
problem
and
to
give
props
to
say
good
job
being
patient
and
getting
through
it
and
then
I
think
like
we're.
Gonna
have
a
model
so
that
next
time
it
won't
will
require
so
much
thinking
as
much
thinking
to
get
to
how
we
approach
it.
D
Yeah
thanks
for
sharing
that
Erica
because
it
wasn't
easy,
but
the
good
thing
is
they
pulled
the
first
round
of
validation.
We
could
eliminate
like
at
least
one
option,
and
now
we
have
two.
So
that's
good
and
then
I
think
since
it's
the
new
process
like
there
are
some
challenges,
it's
just
not
our
team.
There
are
some
other
team
also
going
through
a
similar
process
at
the
moment
and
then
I
think
once
we
design
nicely
and
then
come
up
with
the
results.
E
Yeah
and
exactly
I've
been
calling
it
The,
Bagel
problem
where
I'm
like
we
could
ask
them
to
search
for
bagels
in
the
product,
and
if
we
had
Bagels
on
the
side
map,
they
would
always
just
go
to
the
side,
nav,
so
yeah.
So
that's
why
I
think
having
the
two
groups
there,
where
one
they
just
don't
even
have
that
option.
So
it's
not
like
Bagels
on
the
side
nav.
Then
we
can
see
if
they
can
still
perform
or
not,
and
that's
why
it's
with
it.
That's
why
it's
two
different
groups.
E
E
Level,
so
that's
why
I
think
it's
like
if
performance
is
the
same
with
those
two
groups,
we
don't
have
evidence
to
add
it
to
this.
Okay,
it's
only
that
if
they
are
failing
without
that
side
nav
option,
then
we
can
say:
hey
it's
important,
okay,
okay,
and
we
don't.
We
don't
know
like
with
the
other
approach.
D
So,
first
of
all,
I
I
think
they're
excited
that
they're
validating
this,
because
if
their
input
is
coming
from
the
user,
then
like
we
could
really
be
more
confident
to
say
like
we're
placing
this
menu
at
this
place
because
of
the
this
outcome.
So
I
think
they
like
that
from
that
perspective,
but
on
the
other
hand,
like
I,
think
they
are
still
having
this
on
Mr
behind
the
future
flag
and
I'm
also
sure
they
I
don't
think
they
haven't,
they
have
merged
it.
D
G
G
So
we've
started
that
in
an
issue
and
now
we're
moving
over
to
a
Google
doc.
To
put
that
in
there
I've
also
drafted
an
issue
based
on
an
initial
team,
call
that
we
had
with
switchboard
that
that
group
as
I
was
writing
it
up
after
the
meeting.
I
was
a
little
bit
confused
about
the
request,
so
I'm
gonna
need
some
context
on
how
this
you
know
fits
into
the
foundational
research
plan
that
hayana
has
drafted
so
I've
Ellie
I've
tagged
you
and
I
had
offer
for
some
additional
feedback.
E
G
B
E
E
But
one
thing
I
can
just
quickly
make
note
of
is
with
the
the
personas.
We
know
that
there's
this
like
shift
left
with
security,
so
even
developers
are.
That
was
like
a
big
thing
that
we
keep
finding
is
even
developers
are
getting
their
hands
on
security
and
compliance
stuff,
and
we
did
in
the
sequence,
features
prioritization
survey.
We
asked
I'll
bring
up
the
finding,
but
we
asked
them
if
they
knew
what
compliance
requirements
they
were
working
under
and
surprisingly,
they
had
a
sense
of
it,
even
the
developers.
C
G
E
This
is
all
detailed
and
in
a
way
like
we
issue
a
ties,
this
report,
but
basically
there
we're
in
good
we're
doing
good,
but
with
each
of
our
workflows
that
we
tested
back
to
me,
Mouse
there's
like
one
sort
of
really
painful
task
in
each
of
the
workflows
that
we
need
to
address.
So
it's
use
the
include
syntax,
identifying
the
failure,
the
reason
for
a
failure
and
understand
and
adjust
unit
tests
and
then
the
first
of
the
find
and
fix
pipeline
error,
workflows
and
I.
E
So
this
table
here
kind
of
gives
you
a
sense
for
each
of
those
workflows.
This
column
gives
you
two
to
three
two
to
three
word:
summary
of
each
of
them.
So
for
the
author
of
pipeline
workflow,
we
found
a
barrier
to
entry
with
the
include
syntax,
which
is
important
because
that's
the
entry
point
for
our
TI
components,
catalog
and
as
a
companion
analysis.
E
So
we
really
want
to
focus
here
on
making
that
whole
yaml
related
workflow
easier,
and
then
it
occurred
to
me
that
we
now
actually
have
all
of
these
people
who
gave
us
these
responses
about
the
yaml
experience.
So
we
can
actually
that's
great
because
they're
hard
to
recruit
right.
It's
hard
for
us
to
find
good
matches.
So
here
we
have
this
lovely
sample
of
43
participants
who
gave
us
feedback
on
the
yaml
experience.
E
There's
like
all
these
different
categories
where
they
made
points,
but
basically
it
bubbles
up
to
having
a
hard
time
with
the
animal
experience
so
will
is.
Thank
you
will
is
going
to
give
us
the
email
contacts
for
those
and
what
we
can
do,
because
we
can't
really
have.
Although
people
would
probably
try,
we
can't
really
schedule
43
calls,
but
what
we
can
do
is
run
them
through
that
CI
Alpha
components,
program
and
those
assignments,
and
then
we'll
get
a
nice
read
on
the
satisfaction
scores
and
how
those
are
increasing.
E
Right
now
we
have
like
a
dedicated,
really
well-rounded
sample,
where
we
can
do
Deep
dive
interviews,
but
actually
they're,
pretty
satisfied
with
the
current
template
experience
when
we
ask
them
so
it
means
that
it's
harder
for
us
to
raise
that
bar
in
terms
of
their
feedback
and
getting
them
into
a
really
satisfied
place.
So
these
folks,
who
gave
us
this
us
feedback
and
were
negative,
we
should
get
a
good
a
good
read
on.
E
If
we
can
move
them
up
to
positive,
then
we're
really
doing
a
good
job,
so
that
was
what
will
was
talking
about
and
that's
the
author
of
pipeline
workflow
and
then
understand
and
adjust
unit
tests.
Two
words
for
that
is
his
hidden
treasure.
So
it
was
amazing
I.
Just
so
one
of
the
things
we
did
at
the
end
was
we
gave
them
this
job
to
be
done.
Rating
question,
where
we
had
them
agree
or
disagree
on
a
scale
of
one
to
seven
where
or
one
anyway.
E
So
we
had
them
look
at
their
agreement
and
they
gave
us
really
high
scores
here
and
they
were
just
they
were
delighted.
They
were
delighted
once
they
under
they
didn't
understand
what
was
quite
happening
with
the
unit
tests
at
first,
so
it
wasn't
discoverable
for
them
because
they
went
into
the
logs
and
they
weren't
looking
at
the
UI,
and
they
did
this
like
circular
thing
where
they
were
like.
E
Oh
I've
arrived
at
this
artifact,
don't
know
what
that
means,
and
then
they
would
like
arrive
at
the
artifact
in
a
different
way
in
a
different
way
and
and
then
like
when
they
then
we
kind
of
like
pointed
out
hey
the
artifact
is
going
to
be
helpful
for
this
task,
and
then
they
were
like
what
is
this
artifact?
How
does
it
relate
to
this
error,
but
once
we
did
the
reveal
they
were
like
Thanking
us
at
the
end
of
the
session
like
they
were
like,
and
so
this
rating
here.
E
This
6.1
is
very
high
because
they
were
just
so
pleased
about
it.
So
I
think
that
there
is
some
small
I
mean
I
I'm,
not
the
designer,
but
if
we
can
make
it
more
discoverable
and
a
little
bit
more
obvious
to
them
how
those
things
are
related.
Like
an
example
is
so
in
this
in
the
unit
test,
it
showed
them
a
screenshot
of
what
the
website
would
look
like
if
it
displayed
as
per
the
code
and
they
weren't
sure
if
it
was
what
the
website
should
look
like
or
it
does
look
like.
E
So
we
just
need
to
like
connect.
Those
dots
for
them
and
then
they
will
make
them
so
happy,
and
here
we
saw
only
three
percent
alignment
with
the
sus
verbatim,
but
that
actually
like
tracks
with
this
hidden
treasure
idea
right
where
they're
not
quite
under
that's,
not
discoverable
for
them
and
then
for
the
find
and
fix
pipeline
errors.
E
Nine
percent
they're
waiting
too
long.
Another
way
to
tell
the
story
there,
though,
is
that
they
learned,
which
is
really
cool.
So
if
you
look
at
this,
so
we
had
find
and
fix
pipeline
errors
and
build,
and
then
one
in
test
and
one
in
deploy,
and
so
you
can
see
here
that
they
kind
of
bombed
that
first
one,
but
then
on
the
second
on
the
next
one.
E
They
were
in
green
again
and
basically,
what
it
was
is
if
they
could
step
back
and
look
at
the
patterns
in
the
yaml
file,
they
could
use
that
to
quickly
come
up
with
the
solution,
as
opposed
to
like
digging
into
the
logs.
So
if
we
can
figure
out
how
to
help
them,
take
that
step
back-
maybe
it's
AI
I,
don't
know.
But
to
like
look
at
the
discrepancies
in
the
different
stages
of
the
yaml
file,
we
could
really
help
them
succeed
there.
E
So
that's
a
theme
that
came
out
here
and
it's
actually
that
specific
theme
is
small
in
terms
of
the
percentage
of
stuff
verbatim
I.
Think
that's
because
we
logically
have
a
smaller
percentage
of
Enterprise
respondents
and
that's
a
particular
Enterprise
problem
that
we
were
getting
feedback
on.
So
another
thing
that
will
and
I
yeah
like
are
working
on
is
to
track
business
size
in
the
assess
responses
now
and
so
I
think
that
will
be
helpful
but
yeah.
E
So
this
frustration
with
running
the
entire
pipeline
for
every
fix
I've
heard
that
before,
but
it
was
really
resounding
and
we
see
them
bringing
down
our
our
satisfaction,
scores
or
agreement
with
the
job
to
be
done.
Ability-
and
that's
because
the
user
behavior
in
terms
of
fixing
a
pipeline
is
just
to
like
throw
a
fix
at
it
and
see
if
that
worked
and
throw
a
fix
at
it
and
see
if
that
worked,
they're
not
like
now.
E
Would
this
work
like
they're,
not
doing
that
kind
of
thinking,
and
so
then
it
becomes
really
frustrating
they're,
like
I
just
wanted
to
try
that
thing
and
now
I
have
to
sit
here
and
wait
for.
However,
many
minutes
and
I
asked
so
I
did
my
like
due
diligence.
Follow-Up,
like
tell
me
about
your
tell
me
about
your
workflow
and
and
like.
Is
this
really
a
big
thing
and
they're
like
yes?
Yes,
yes,
this!
This
gets
me
out
of
my
flow.
It
gets
me
out
of
context.
E
A
E
E
Later
on
and
like
Jackie's
question
was
like:
is
that
good,
it's
like
so
so
yeah?
So
we
wanted
to
be
able
to
characterize
it
in
those
ways
and
so
just
being
transparent
about
how
we're
doing
that,
but
I
think
it's
legitimate,
because
if
we
look
at
like
kind
of
the
other
findings,
33
of
them
were
not
categorizable
right.
So
like
something
about
inconsistent
behavior
and
cicd
pipelines
like
I.
E
Don't
know
how
you'd
map
that
to
much,
but
so-
and
you
might
think
like
that's
a
lot
is,
should
we
be
concerned,
but
actually
like
it's
pretty
to
me,
it
legitimizes
it
because
it
means
that
there's
a
bucket
of
things
you
can't
Force
fit
into
these
Benchmark
pain,
points
and
themes
and
so
like
having
the
a
third
of
it
be
a
round
number
is
good
yeah.
To
answer
your
point.
We
just
came
up
with
these
kind
of
categories.
E
That's
just
a
way
of
labeling
it
cool
thanks,
yeah,
but
they're
mutually
exclusive.
So
it
has
to
fit
into
one
of
the
categories
and
only
one
of
the
categories
and
to
do
that.
I
ended
up
having
to
parse
out
a
lot
of
the
verbatims,
because
they'll
talk
about
all
these
things
and
so
to
get
the
one-to-one
scoring
and
I
think
it
was
317
verbatims
that
we
had
wow.
Okay,.
G
E
E
E
A
Erica
I
just
wanted
to
say
really
that
this
was
really
great
work,
maybe
because
it
related
to
things
that
I
was
working
on
so
I
was
like
more
interested
in
it,
but
yeah
this
was.
This
was
awesome
and
I
love
that
we're
seeing
feedback
across
like
the
whole
pipeline
experience.
It's
not
just
one
section.
F
Yeah,
especially
when
it
touches
upon
the
places
that
we
have
always
like
talked
about
and
there's
never
been
like
enough
evidence
that
we
should
really
focus
on
that
work
and
improve
those
experiences.
So
I'm
very
hopeful
that
the
results
of
this
benchmarking
would
like
help
us
prioritize
things
which
are
really
going
to
make
big
changes.
E
Isn't
linking
yeah
I
put
I
put
a
deck
together
after
kind
of
now
having
this
idea
that
we
are
focusing
on
environments,
I
was
like,
oh
I,
can
think
of
a
key
problem
that
I've
heard
so
like
across
these
four
studies.
We
keep
hearing
this
problem
related
to
coordinating
variables
across
environments,
so
I
just
pulled
out
slides
from
each
of
those
reports,
because
I
would
say
that
that's
a
big
problem
like
we're
not
even
asking
about
that.