►
From YouTube: Release Evidence JTBD
Description
Welcome to our discussion on Release Evidence at GitLab
https://about.gitlab.com/direction/release/release_evidence/
A
Hello,
hey
Shawn,
so
yesterday,
I'm
at
our
release,
evidence
jobs
to
be
done.
I
actually
mapped
everything
and
stack
ranked
it
across
our
entire
release
management
product
area,
so
just
to
kind
of
go
through
the
structure.
So
we
know
what
we're
looking
at.
We
have
the
category
in
the
first
column,
there's
a
UX
issue.
That's
usually.
A
B
A
Of
years
of
stories,
but
I
love
seeing
so
many
of
them,
you
know
I
tried
to
be
as
comprehensive
as
possible.
Oh
we're
all
good
okay,
so
the
first
column
is
the
category.
Second
column
is
the
UX
issue.
If
we
had
a
scorecards
that
we
ran
against
a
user
story
or
a
type
of
action
that
we're
expecting
a
user
to
conduct,
we're
gonna
end
up
inputting
the
grade
from
that
user
story.
I've
indicated.
If
this
has
a
user
story,
that's
required
for
a
release
management
maturity
path.
A
So
what
I
mean
by
that
is
the
user
base
inside
that
category
has
to
be
able
to
successfully
execute
this
job
to
be
done
in
order
to
to
be
ranked
as
a
viable,
complete
or
loveable
maturity.
I've
also
added
a
priority,
which
is
just
a
numerical
value
and
we'll
notice
that
there's
a
couple
of
items
that
are
crossed
off
here.
The
crossed
off
items
are
things
that
we've
already
delivered.
Okay,.
B
A
And
then
we
have
the
actual
job
to
be
done
in
these
columns.
I've
also
added
the
personas
that
those
jobs
to
be
done
are
for
where
this
touches
the
gait
lab
application
and
the
current
status
of
that
user
story
in
columns.
Oh
through
our,
we
have
the
various
issues
that
contribute
to
that
user
story.
Okay,
so
I'm
gonna
go
ahead
and
filter
out
all
of
the
other
ones,
except
for
release
evidence.
So
when
we
look
at
release
evidence,
we've
delivered
to
user
stories
already
for
it,
which
are
centered
around
I'm.
A
Looking
at
changes
to
a
production
environment
I
want
to
be
able
to
see
who
made
the
changes
when
and
why
the
changes
were
made
so
that
I
can
adequately
respond
to
an
audit.
This
is
for
a
release,
manager
and
a
developer
team
lead.
We
had
the
collect
release
evidence
at
release
end
as
what
people
could
use
to
complete
that
job.
A
The
second
user
story
that
we've
delivered
is
the
things
that
you've
recently
just
added
and
shipped,
which
is
being
able
to
easily
and
naturally
provide
a
production
source
of
change
so
that
they
can
respond
to
not
it
so.
These
on-demand
release,
evidences
somebody
can
go
in
generate
release,
evidence
and
provide
those
two
JSON
files
as
a
premium
customer
to
an
auditor
as
an
indication
of
this
is
how
a
release
has
changed
over
time
or
as
a
starter,
customer
or
a
core
customer.
They
could
sample.
A
A
The
next
priority
for
release
evidence
which,
if
we
were
to
go
ahead
and
plug
everything
in
here
and
sort
by
priority
again,
we
would
notice
that
our
next
item
next
items
are
in
release,
orchestration
secrets,
management
and
then
finally
release
evidence.
So
it
would
be
my
expectation
that
we
would
deliver
these
things
first
prior
to
delivering
these
items
just
based
off
of
where
we
need
to
move
our
direction,
but
going
back
to
the
release,
evidence
kind
of
job
to
be
done
and
what
we
need
to
accomplish
to
make
it
complete.
A
When
I
was
speaking
with
a
couple
of
customers
that
were
in
financial
services
in
the
insurance
industry
and
banking,
which
are
pretty
much
the
target
market
for
a
release
evidence,
they
cited
that,
when
a
compliance
team
surfaces
to
a
development
team
lead
or
tow
release
manager
that
they
want
to
confirm
the
records
of
changes
made
to
production,
they
want
to
be
able
to
minimize
the
disruption
to
the
development
team.
So
they
would
rather
have
like
the
compliance
or
auditing
team
self-serve
the
release
changes
without
having
access
to
the
source
code.
A
So
this
will
then
enable
them
to
adequately
respond
to
non-it
without
disrupting
the
developers.
The
features
that
we
are
attributing
to
this
are
hardening
the
release.
Evidence,
which
is
this
feature
set
back
about
permissioning
for
the
auditor,
so
that
they
can
go
in
and
as
an
auditor
and
self-serve
that
evidence
and
then
the
other
one
is
improving
the
performance.
So
why
I
included
this?
A
Is
we
wanted
to
make
sure
that
the
IDs
and
the
merge
request
IDs
are
in
the
JSON
file
so
that
we
have
that
continuity
of
the
change
that
we're
requesting
is
the
issue?
And
now
this
is
the
merge
request
that
made
that
issue
change
and
I
think
this
is
more
representative
of
changes
in
production
and
the
issue
ID
anyways
and
then
the
last
issue
I
have
linked
here
is
any
bugs
that
are
associated
with
release.
Evidence
I
need
to
check
that
link.
A
A
So
the
next
kind
of
category
move
that
we
would
be
trying
to
make
would
be
taking
release
evidence
from
viable
to
complete
and
a
big
piece
of
that
story
is
satisfying
auditor
rules
so,
for
example,
they
may
they
being.
The
auditor
may
want
to
see
that
there's
a
separations
of
duty
in
the
project
and
what
that
means
is
they
want
to
be
able
to
see
that
who's
ever
made
a
change
to
production
is
different
than
the
person
who
approved
it.
A
So
the
merger
quest
submitter
it's
different
than
the
person
who
actually
promoted
that
change
to
production
and
the
items
that
I've
attributed
here
are
like
hardening
the
release
evidence
from
the
first
category.
That
would
also
satisfy
important
a
portion
of
this
and
then
highlighting
the
evidence
differences.
B
A
A
So
what
I
have
here
is
this
issue
that
says:
use
release
evidence
to
triage
rule
violations,
so
here
what
we'd
want
to
see
its
released
evidence
having
some
sort
of
way
to
show
that
this,
mr,
doesn't
follow
a
rule
that
they
set
or
a
rule
that
we
have
set
and
get
lab.
So
let's
say
that
we
add
a
configuration
item
at
the
project
that
says
the
deployer
has
to
be
different
than
the
submitter
on
the
mr
and
the
release.
A
Evidence
would
then
show
a
comment
or
a
flag,
some
way
that
this,
mr,
has
the
same
improver,
a
submitter,
and
we
need
to
go
through
and
kind
of
review
that
and
make
sure
that
that
change
actually
is
acceptable.
This
is
just
about
visibility,
so
we
wouldn't
be
able
to
like.
You
know
how,
in
the
UI
today
when
you
use
web
ID,
you
can
apply
a
suggestion.
There
wouldn't
be
any
of
that
functionality
we
wouldn't
be
able
to
like
actually
fix
the
problem.
From
that
view,
this
would
just
be.
A
We
can
see
that
this
is
a
problem.
Another
kind
of
thing
that
people
might
be
interested
in
is
if
code
coverage
for
that,
mr,
is
of
a
certain
threshold,
so
being
able
to
indicate
that
this
is
not
meeting
the
code
coverage
requirements
and
then
having
that
as
hey.
This
is
existing
in
production.
Today,
you
may
want
to
go
check
out
that
mr
and
get
that
address,
and
then
security
scans
are
also
something
that
we
may
want
to
include
and
to
me
release.
A
Evidence
should
be
this
artifact
that
allows
and
empowers
users
to
make
sure
that
they
are
compliant,
because
that's
really
the
story
here.
This
again
is
like
lovable.
We
obviously
haven't
sized
it.
It's
pretty
far
down
the
priority
list
when
we
pull
up
all
the
rest
of
the
other
initial
that
we
have
and,
of
course
it's
not
scheduled,
but
we
need
to
deliver
several
things
and
secrets
management
and
released
orchestration
before
we
deliver
this
feature
set,
but
it
could
be
could
be
very
compelling
for
our
financial
services
and
compliance
iterations.
A
A
B
Fact
you
know
so:
I
had
I
had
worked
a
bit,
you
know
in
compliance
and
and
not
not
in
order
putting
you
in
compliance.
Basically
and
and
in
fact
my
ex-wife
was
compliance
officer
at
Citigroup,
and
so
so
I'm
quite
familiar
with
these.
You
know
what
we're
trying
to
achieve
here
and
in
fact,
what
we're
building
is.
You
know
it's
I
can
see
getting
quite
complex
like
it's.
You
know,
there's
a
lot
of
amazing
features
here
and
you
know,
even
at
last
one
Nicki
pointer,
which
I
know
is
quite
far
down
the
track.
Yeah.
A
B
Mean
I
was
just
thinking,
then
we
could
even
have
something
that
would
be
a
bit
like
a
ruder
cop
for
compliance.
You
know.
Where
would
you
know
we
could
be
completely?
You
know,
we'd
be
so
far
out
from
the
competition
with
that
and
and
so
they're
the
two
thoughts
that
I
had.
So
it
all
looks
great
and
really
I
really
appreciate
you
taking
the
time
to
do
this
and
produce
this
spreadsheet.
Like
this,
it's
good
to
see
it
all
kind
of
you
know
as
a
dashboard,
so
the
two
thoughts
I
had
were
like.
B
Maybe
should
we
talk
to
whoever
used
compliance
or
ordering
that
it'll
air
to
start
with,
as
well
as
obviously
customers
and
I'm
just
wondering
you
know,
maybe
the
reason,
but
maybe
there
is
some
open
source
product
that
does
some
part.
What
we're
trying
to
do
I
mean
it
is
pretty
tied
in
with
what
we're
the
structure
of
gear
lab
itself.
So
might
there
might
not
be,
but
you
know
if
there
is
a
standard
out
there
here.
B
Maybe
we
can
either
learn
from
it
or
try
and
integrate
it
even
or
commercial
product
learn
from
them.
They're
just
they're
the
only
tools
but
I
think
otherwise
you
know
I,
think
everything's,
great
and
I
think
we're.
You
know
when
I
first
heard
a
bit
evidence,
I
thought
it
was
going
to
be
little,
but
in
fact
it's
going
to
be
a
major
feature.
I
think
yeah.
A
B
A
This
will
be
a
differentiator
for
us
and
the
fact
that
we
could
potentially
have
released
be
used
by
people
that
aren't
using
CD
I.
Think
is
part
of
this
angle
too,
so
like
if
people
are
using
just
source
code
management
and
merge
request
in
CI,
they
could
just
use
this
feature
and
create
releases,
and
this
will
be
a
big
part
of
their
compliance
story
and
it'll
be
very
complementary
to
our
compliance
team's
efforts
that
they're
doing
too
so
this
last
issue,
the
compliance
team,
is
looking
at
building
out
policies
inside
of
get
lab.
A
So
we
can
then
have
the
release.
Evidence
look
at
the
policies
that
the
person
has
configured
and
then
fly
their
violating
their
own
policies.
We
removed
three
or
four
tools
in
their
tool
chain
for
the
customers,
so
I
agree
that,
yes,
we
should
definitely
talk
to
our
internal
compliance
team,
about
these
user
stories
and
look
at.
Are
there
smaller,
maybe
MBC's
or
iterations
that
were
not.
B
A
B
No,
no
you're
right,
because
in
fact
you
know
the
comparison
comparison
that
we've
been
talking
about.
Okay,
that
won't
be
so
bad,
but
the
once
we
start
to
you
know
think
about
complex
comparisons.
It
it'll
be
it'll,
be
a
lot
to
build.
Of
course
we
can
build
it,
but
at
the
same
time
you
know
in
order
them
I
just
want
to
take
it
out
of
the
system
anyway,
and.
B
It
was
like
this
and
she's
going.
What's
this,
you
know
they
said
well,
this
is
how
we're
gonna
give
them
to
you.
You
know
anyway,
and
so
she
picks
up
the
bondo.
She
opens
the
last
page,
and
just
you
know
just
like
just
out
of
intuition
and
the
the
log
head
arrowed
it
actually
and
I
printed
of
the
whole
thing
out,
and
no
one
ever
looked
at
the
printout
and
here's
this
error
and
said
you
know,
incomplete
or
something
right
and
and
I
think
that's
the
sort
of
thing
you
know
we're.
B
You
know
if
we,
if
we
talk
to
people,
are
in
the
field,
tell
stories
all
that
and
be
like
that's
you
know
straightaway.
We
need
to
highlight
flag
those
things
right.
Likewise,
these
voices
test
always
earring
or
wires.
You
know
why
is
that?
Why
are
these
warnings
always
being
ignored
in
a
particular
trace
file,
because
we
only
look
at
errors
actually
like
warnings?
We
just
basically
ignore
you
know
things
like
that
and
maybe
maybe
maybe
some
warnings
dharak
ignore,
but
maybe
for
a
particular
company.
A
It's
interesting
that
we
that
we
say
this
because
I
could
see
a
really
interesting
use
case
where
we
look
at
monitoring
customers
and
we
tie
like
monitoring
event
to
release
evident
it's
kind
of
like
a
defect.
Leakage
metric
and
we
show
like
what's
the
last
release
that
caused
a
monitoring
event
and
displaying
like
the
history.
B
Of
those,
because
all
those
things
it's
like
there's
so
much
and
we're
not
they're
not
connected
back
rad
like
a
production
outage
is
not
never
is
not.
We
don't
have
that
retrospective.
You
know
it
came
out
of
this
release
and
and
in
fact
it
we've
had
two
similar
problems
in
the
last
two
releases.
You
know
it's
just
like
all.
He
is
a
critical
issue
who
fix
it.
You
know.
Do
a
patch.
A
Given
that
story
of
defect
leakage,
I
think,
is
something
that
I
could
really
see,
release
evidence
accomplishing
and
that
would
be
immensely
valuable
for
people
who
are
using
third-party
tools
to
currently
monitor
that
right.
Even
if
it's
just
a
check,
you
know
if
they
are
using
issues
and
get
labs,
then
that
is
a
whole
all-in-one,
but
most
of
our
companies,
they're
consuming
and
interested
in
release.
Evidence
are
using
six
or
seven
tools
to
accomplish
what
we're
hoping
to
build
in
this
one
feature.
B
A
Because
that's
going
to
be
what
what
drives
like
the
usability
of
this
future
is:
how
can
we?
How
can
we
support
these
different
personas
that
end
up
being
on
a
receiving
end
of
a
request,
because
it
the
job
to
be
done,
ends
up
being
more
global
like
I
need
to
complete
an
on
it?
And
then
these
are
the
individual
breakouts
as
felt
by
these
different
people,
but
you're,
absolutely
right.
A
That
I
started
considering
third-party
tools
for
this
and
then
I
realized
that
we
could
use
something
like
open
policies
like
OPA
for
it,
but
then
we're
tying
our
users
to
have
to
use
OPA
to
declare
their
policies,
and
they
may
already
have
like
another
tool
that
they're
using
for
that
and
the
more
that
we
integrate
with
another
tool
that
isn't
like
we
build
it
in
to
get
labs.
So
it's
unknown
to
that.
It's
like
it's!
It's
just
a
feature:
that's
unnatural!.
B
A
A
So
I
think
some
of
my
next
steps,
I'm
going
to
pull
from
this,
is
I.
Since
this
was
helpful,
we
should
probably
reach
out
to
our
compliance
internal
teams
and
see
if
this
is
like
something
that
they
would
want
to
add
to
get
lab.
Dot-Org
was
something
they
used,
and
then
they
can
start
dogfooding.
These
features
and
help
kind
of
illustrate
what
would
be
more
importantly
for
us,
because
I
think
we
have
a
viable
product
now
that
it
makes
sense
to
try
to
start
dogfooding.
It.
B
And
you
mentioned
earlier
banks,
because
you
know
they've
always
got
compliance
issues
right
and
well,
not
issues
but
they're.
You
know,
they're
always
got
the
high
focus
on
compliance
exactly
and-
and
you
know,
I'm
wondering
what
tier
we
can
bring
some
of
these
features
in
that
and
and
the
way
to
ask
that
might
be
to
ask
what
do
you
need?
What
would
you
pay
for?
You
know
I
think.
A
B
A
A
B
A
Think
the
the
biggest
problem
with
finance
institutions
jumping
to
ultimate
is
that
they
don't
see
enough
value
for
the
individual
out
of
ultimate
features.
That
a
lot
of
ultimate
is
for
reporting
for
security
scanning,
for,
like
very
large
initiatives.
So
bridging
that
experience
showing
that,
like
yeah,
your
individual,
gets
the
most
value
at
premium
and
if
there
was
more
compelling
feature
user
sets
that
reduce
the
time
spent
for
the
developer,
then
they
would
feel
more
compelled
to
invest
in
it.
So
I
think
this
could
be
one
of
the
areas
where
we
show
that
yeah.
B
Absolutely
there's
a
big
I
think
one
of
the
barriers
to
ultimate
is
is
the
big
jump
in
premiums
right,
but
you
know,
but
if
it's
something
that's
really
actually
going
into
the
business,
this
is
really
going.
This
is
somehow
leaving
a
bit
technical
I'm,
a
no
obviously
they're
auditing,
the
technology.
But
you
know
it's
really
it's
from
the
perspective
of
the
business,
not
from
a
perspective
of
the
development
team
and
the
business
of
the
budget.
B
You
know
the
development
teams
are
a
work
in
a
way
andis
basement
somewhere,
and
you
know
where,
as
where
is
the
business
teams
or
particularly
the
risk
management
teams?
You
know
they've
got
the
budget
to
make
sure
it
doesn't
go
Sarah
on
them
and
with
an
online
banking
we've
got,
you
may
have
all
apps,
there's
a
lot
to
think
about
actually
in
the
finance
area,
I
think
exactly.