►
Description
Some customers have complained that they don't fully trust the data being displayed by Value Stream Analytics (VSA). Larissa has found the problem, and here she explains it to the group. Specifically, the date range should apply to end events, not start events. Based on that, we decide that we probably also need to change how the overview and individual stages behave. Then Adam describes some of the new performance issues this will raise, and how we might address them.
B
All
right,
and
so
over
the
past
month
or
two
I've
heard
two
different
customers
say
that
they
don't
trust
the
data
they're
seeing
in
bsa
and
have
stopped
using
it
and
gabe
who
who
dog,
feed
dog
foods.
Bsa
also
said
the
same
thing,
and
so
I
started
diving
in
to
try
and
understand
why
people
were
feeling
that
they
weren't
seeing
the
data
they
expected
and
that's
what
I
want
to
walk
through
as
a
team
today,
just
to
try
and
understand
that
a
little
bit
better
and
dan.
B
B
B
B
Let's
say
I'm
a
director
of
engineering
and
my
engineering
leadership
just
wants
to
know
based
on
everything
we
shipped
in
the
most
recent
release
on
average.
How
long
is
it
taking
us
to
get
value
out
to
customers
for
the?
How
long
is
the
entire
workflow
taking
all
right
dan?
Do
you
want
to
share
your
screen
with
the
essay
group
level
open.
B
C
B
B
B
B
C
B
A
Just
not
I'm
not
hundred
percent
sure
if
we
always
add
the
verification
label,
though
so
if
we
go
straight
to
production
since
we
realized
this,
this
is
actually
already
working
on
production.
I
think
we
sometimes
skip
the
verification
label
and
then
it
would
be
not
tracked.
That's
my
that's
my
immediate
thought,
but
not
sure.
B
B
B
D
C
D
A
B
C
C
B
D
B
B
B
C
B
A
A
D
A
B
B
But
then
there
is
this
other
work
use
case
that
then
gets
difficult
to
support,
which
is
the
use
case
of
I'm
a
director.
My
leadership
has
said
has
asked
me
how
long
it
takes
from
end
to
end
to
deliver
value,
and
so
I
want
to
go
in
and
see
what
I
delivered
in
1312
and
how
long
it
took
me
from
either
the
time
that
the
issue
was
created
or
the
time
that
we
started
development
work
on
it
until
we
actually
got
into
production
and
so
right.
B
Now,
if
you
filter,
if
you
set
a
date,
range
you're
filtering
by
create
date.
In
for
that
use
case,
I
think
that
we
could
probably
filter
by
closed
issue
closed
date,
so
that
you
can
see
everything
you
closed
in
the
last
month
and
then
show
the
cycle
time
or
lead
time
based
on
that,
but
how
we
fit
that
view
into
a
view
of
our
our
stage
times
where
we
filter
by
when
the
issue
went
through
the
stage
time
they're
like
two
totally
different
use
cases
and
two
totally
different
views.
C
C
The
overview
is
a
summary
or
sort
of
a
summation
of
each
of
these
individual
stages,
and
I
think
that
probably
needs
to
change
as
well.
I'm
less
certain
on
this
one,
but
I
think
so
because
I
think
like,
for
example,
when
somebody
clicks
on
this
verification
stage,
they're
expecting
to
see
anything
that
was
verified
in
this
date
range.
C
I
think
I
think
that's
what
users
are
expecting
to
see
like
that's
what
I'm
expecting
to
see,
but
if
that's
what
users
are
expecting
to
see,
then
that
means
that
the
overview
would
no
longer
be
a
summation
of
these
things,
because
these
things
would
no
longer
like
sort
of
sum
up
to
the
overview.
They
are
a
different
perspective
on
the
same
frame
of
time,
rather
than
being
like
a
summation
of
a
series
of
stages
across
time.
If
that
makes
sense,
so
I
think
we
need
to
make
that
change
too.
B
A
I
think
it
should
still
be
part
of
the
flow
because
well
we
we
want
to
click
on
it
right,
so
it
should
be
part
of
the
navigation
somehow,
but
maybe
we
should
visualize
like.
Maybe
we
need
a
visual
distinction
to
say.
Okay,
this
is
you
know.
This
is
not
really
part
of
the
flow.
It's
it's.
It's
also
not
a
sum
of
the
flow.
It's
just
an
overview.
B
B
B
D
E
Yeah
yeah,
so
I
totally
agree
that
the
current
setup
does
not
make
sense.
There
is
probably
some
historical
stuff
going
on
why
it's
behaving
like
that.
I
don't
know
why.
Actually
the
future
predates
me
so
if
in
case
you
want
to
filter
items
by
the
start
or
end
events
or
basically,
both
events
should
happen
within
the
same
time
range
we,
we
will
hit
basically
an
optimization
problem.
So
currently
we
are
optimizing
for
one
particular
column,
which
is
the
creation
at
column,
the
creation
date
for
an
issue
or
an
mr.
E
This
is
quite
simple
to
address.
We
use
this
column.
Quite
often
we
query
many
parts
in
many
parts
of
the
application,
so
it's
it's
quite
easy
to
make
it
performant.
Let's
see
so
vsa
supports
several
different
events
closed
date
merged
at
bit
when
the
label
was
added,
and
things
like
that
and
some
of
these
type
stamp
columns
are
quite
difficult
to
query
without
going
through
other
issues
within
your
group.
E
So
worst
case
scenario
we
would
have
to
the
database
would
have
to
iterate
over
all
the
issues
in
your
group
and
filter
out
the
items,
and
this
is
currently
about
50
000
issues
in
gitlab.org
and
it
takes
about
10
seconds
kind
of
and
this
grows
as
we
move
forward
in
time,
because
the
issue
count
is
not
going
to
get
smaller,
it
will
just
increase
and
we
have
significantly
larger
groups.
So
github
is,
I
think,
the
sixth
biggest
group
on
on
on.com.
So
we
have
groups
10
times
bigger
than
that
so
yeah,
it's
it's
problematic.
E
I
don't
say
it's
not
possible
to
do
it,
but
it's
a
big
investment,
because
we
will
have
to
look
into
each
start
event
and
see.
How
do
we
make
this
performant
a
simple
in
some
cases?
It's
quite
simple:
we
need
to
introduce
a
new
database
index.
This
is,
I
don't
know
it's
about
the
weight
of
three.
So
it's
a
relatively
simple
change
database
migration
and
we
have
to
verify
if
the
database
queries
are
performed.
So
that's
that's
that
the.
E
So
if
it's
not
about
creation,
then
we
might
need
to
denormalize
these
tables
to
expose
the
project
id,
because
in
some
cases
the
project
id
is
not
available
on
this
database
table
and
that's
more
complex
work.
It
requires
data
migration
where
we
have
to
backfill
create
a
new
column
in
the
database.
We
need
to
backfill
it
and
we
need
to
make
sure
that
it's
up
to
date
so.
B
E
So
we
have
an
issue
stable.
If
you
are
not
familiar
with
databases,
it's
a
giant,
you
can
imagine
it
as
an
excel
sheet
and
you
have
about
90
million
issues
in
it.
So
that's
gitlab.com
and
we
have
a
column
called
project
id
which
specifies
which
project
are
you?
Is
this
issue
created
right?
So
that's
kind
of
makes
90
million
issues
to
50
000
right,
it's
a
slice
of
that
table,
and
what
do
we
do
on
the
group
level?
E
So
we
look
at
your
group
hierarchy
and
find
all
the
projects
in
your
group.
So
in
github
we
have
about
a
thousand
projects
in
the
gitlab
work
group
and
for
each
project.
We
look
up
the
issues
right,
so
we
are
looking
at
that
huge
table.
Huge
access
sheet
that
find
this
is
a
project.
Give
me
the
issues.
This
is
another
project,
give
me
the
issues
and
only
take
out
the
records
that
are
matching
the
created
at
timestamp
condition
and
we
have
additional
tables.
Let's
say
you
have
the
issue
table
that
huge
table.
E
So
it's
like
you
have
to
do
a
double
jump
to
actually
figure
out
a
small
set
of
data
and
in
the
middle
there
is
that
issues
table
which
is
huge,
and
you
need
to
actually
filter
that
wall
table
down
in
order
to
figure
out.
What's
in
the
second
table
and
the
solution,
is
you
look
at
the
issue
matrix
table?
You
add
the
project
id
there
as
well,
so
you
can
skip
looking
into
the
issues
table,
so
you
go
directly
there.
That's
it's
called
a
normalization.
E
B
E
E
E
So
if
we
do
this
for
the
other
tables
in
vsa,
then
we
can.
We
can
have
similar
speed
than
we
have
today
could
be
even
faster
in
some
cases
because
carrying
the
issue.
Stable
is
a
bit
it's
a
big
table
and
it's
a
quite
high.
It
gets
high
traffic.
So
there
is
a
bit
of
contention
congestion
very
quickly,
so
yeah.
B
E
The
database
migration
to
create
an
index.
The
simple
case
is
about
the
weight
of
three
the
normalization.
I
think
we
have
like
two
tables
to
then
denormalize.
It's
about
us
five
or
six.
It's
actually
a
two
release
process
because
first,
you
have
to
create
the
database
column
backfill
the
data
in
a
background
process.
So
that's
one
mile
one
milestone
and
in
the
second
milestone
you
can
start
actually
using
that
column.
B
E
E
F
D
C
D
F
F
D
D
F
Actual,
I
guess
state
of
the
code
base
we're,
I
think,
we're
in
a
much
better
position
to
make
quicker
changes
to
the
front
end
definitely
than
we
were
six
months
ago
and
then
the
other
aspect
is
it's
getting
close
to
the
point
where
anything
we
do
on
the
group
level
will
start
being
reflected
on
the
project
level
as
well.
So
we
probably
just
need
to
bear
that
in
mind
as
well.
D
F
A
B
And
we
have
a
couple
of
things
I
think
in
140
to
have
a
pop
over
and
add
a
sentence
to
the
pop
over
that's
next
to
the
date
selector
to
explain
that
it's
using
the
issue
created
date
depending
on
when
we
decide
to
move
forward.
If
and
when
we
decide
to
move
forward,
we
may
or
may
not
want
to
do
that
issue.
B
B
The
amount
of
effort
that
you
think
is
involved
so
you've
got
an
issue
in
14-0
to
allocate
some
time
to
that,
and
then
I
want
to
go
and
run
this
by
some
more
customers.
Just
to
make
sure
that
I
I
feel
pretty
confident
based
on
the
conversations
I've
had
so
far,
but
I
would
just
like
to
do
a
bit
more
validation
to
make
sure
this
is
the
right
change.
C
F
Good,
I
was
just
gonna
jump
in,
I
suppose.
The
only
area
I'd
probably
want
to
get
a
little
bit
more
validation
before
we
jumped
ahead,
would
be
making
changes
to
the
overview
stage,
taking
out
the
numbers.
Fine,
but
yeah.
I
think
if
we
to
start
playing
around
with
the
navigation
again,
we
may
then
need
another
feature
flag
which
yeah
we've
all
been
down
that
path.
So.
B
A
I
was
just
going
to
add,
I
think,
the
overview
I
think
we
should
not.
Personally,
I
think
we
should
not
make
changes
to
the
way
the
navigation
works.
I
think
it's
just
it's
just
a
matter
of
what
we
want
to
show
on
the
overview
page,
but
it
should
still
be
part
of
the
flow
right
because
it's
it's
part
of
the
clickable
tiles
and
at
some
point
you
want
to
get
an
overview.
A
B
G
B
Fix
what
if
customers
come
back
and
say
something
like
okay,
but
we
also
yeah.
I
want
to
see,
I
want
to
be
able
to
select
a
date
range
and
see
what
passed
through
the
stage
in
that
date
range.
But
I
also
want
to
only
look
at
the
items
that
were
closed
in
a
certain
date
range
or
opened
in
a
certain
date
range.
I
don't
know
if
that's
gonna
be
a
thing,
but
if
we
get
requests
like
that,
is
that
just.
E
D
E
Will
find
a
lots
of
additional
filtering
options
and
theoretically
we
could
support
those.
So
actually
it's
much
more
than
we
support
in
vsa,
like
you
can
filter
by.
I
don't
know
this
emoji
what
you
can
put
on
the
issues
and
things
like
that
and
if,
if
that's
like
extra
time
stamp
feature
is
available,
then
we
could
probably
do
that.
It
wouldn't
add
too
much
too
much
processing
on
the
backhand
side,
because
the
the
initial
date
range
filter
that
we
build
or
planning
to
build.
E
We
have
reduced
the
number
of
rows
significantly,
so
an
additional
feed
that
doesn't
really
make
too
much
difference,
but,
of
course
it
needs
to
be
measured
properly
and,
let's
see
height
height
behaves.
I
cannot
tell
for
sure
if
it
will
work
something
else
if
the
customer
needs
like
a
special
reporting
or
additional
special
filters
based
on
something
completely
as
apis.
So
if
vsa
would
expose
data
via
the
apis,
your
issues,
the
related
issues,
are
merge.
Requests.
The
customer
can
build
automation
on
top
of
that
to
further
filter.
F
E
E
Several
times
aggregation
data
aggregation
came
up
and
my
question
is:
how
can
we
validate-
or
maybe
you
can
tell
me
if-
let's
say
you
configure
your
stages
and
we
start
aggregating
the
data
after
you
kind
of
finalize
your
your
value
stream
configuration
instead
of
having
data
available
right
away,
you
would
silently,
in
the
background,
build
up
the
the
necessary
data
that
belongs
to
the
stages.
So
it's
like
not
today,
like
you,
can
just
start
querying
and
filtering
stuff,
but
it
needs
some
time
to
get
the
data
into
a
shape.
B
E
D
E
E
B
E
B
E
E
E
B
E
B
G
The
the
freshness
of
the
data
also
is
dependent
on
the
particular
use
case
that
you're
looking
at
as
well
like
this,
the
scope
of
the
the
data
that
you're
looking
at
so,
for
example,
like
the
historical
view
where
we
are
interested
in
seeing
how
our
stage
time
has
has
changed
over
time.
That
really
doesn't
need
to
be
real
time.
G
I
don't
think
it's
too
bad
but
like
there
is,
if
we
can
find
the
right
balance
of
providing
enough
real-time
data
for
that
work-in-progress
view,
whilst
balancing
it
with
the
historical
stuff
that
we
can
load
over
a
longer
period
of
time.
I
think
that
would
be
ideal
if
that,
if
that's
valuable
at
all,
I'm
just
going
to
follow
along.
E
Yeah
that
makes
sense
to
me
to
be
honest.
We
within
github,
we
didn't
really
do
these
kinds
of
aggregation
tasks,
jobs.
We
just
start
exploring
these
ideas
in
in
order
to
to
improve
the
performance
of
the
application-
and
I
I
thought
for
for
for
volunteering
analytics-
it
could
be
also
a
way
to
go,
but
I
don't
have
the
specifics.
E
E
Obviously,
this
is
not
going
to
be
available
enough
past
storage
or
we
won't
be
able
to
access
it
quickly
because
we
cannot
aggregate
it
for
all
groups
for
all
projects
is
this
data,
because
that
would
be
quite
significant
load
on
the
database
and
it
would
cost
disk
space
so
for
default
stuff
that
the
user
are
not
really
changing,
cannot
really
change
and,
let's
say,
the
the
open
source
version.
Where
you
have
these
default
stages,
we
would
still
rely
on
the
current
way
of
calculating
things,
looking
into
tables
and
do
allegations.
G
Another
concept,
which
is
really
useful
in
value
stream
is,
is
like
using
different
item
types,
so
there's
a
distribution
of
different
types
of
items
which
come
through
your
value
stream.
You
have
bugs
features
security,
vulnerabilities,
typically
of
like
five
or
six
in
total,
and
these
are
mutually
exclusive
things.
G
So
is
there
some
way
to
optimize
around
pulling
from
a
smaller
data
set
once
you've
defined
these
workflow
items
so
say
so,
rather
than
going
from
like
the
entire
pool
of
issues,
once
we
have
issue
types
implemented
and
an
issue
type
is
defined
as
this
and
within
your
value
stream,
you
say:
I'm
only
interested
in
seeing
these
types
of
issue
types.
Would
that
reduce
the
data
set
and
somehow
help
with
performance
at
all.
E
So
yeah,
so
there
are
two
options:
one
you
define
this
extra
flag
in
your
stage
definition
when
you
set
up
the
stage
at
this,
this
particular
stage
only
look
at
incidents
or
or
bugs.
Then,
when
we
aggregate
the
data,
we
only
aggregate
a
really
small
set
of
data
which
makes
everything
everything
fast.
If,
if
you
don't
do
this,
what
we
can
do
is
on
the
ui.
You
would
have
maybe
an
extra
filter
in
the
in
the
drop
down
that
you
say
this
is
my
volume
stream.
E
If
we
want
to
do
that
on
that
special
table,
where
we
collect
this
data,
we
will
have
to
record
the
issue
type,
for
example.
So
this
is
like
a
post
filter.
We
can
call
this
like
that.
This
can
also
speed
up
things,
but
it
don't.
It
won't
reduce
the
number
of
data
vr.
The
amount
of
data
we
are
storing
is
just
a
fast
access
to
a
slice
of
data.
We
get
something
like
that.
E
A
limiter,
so
if
it's,
if
it's
like,
really
a
state
column
like
a
state
or
a
type
on
the
on
the
issue
or
the
mr
tables,
that's
fine,
but
as
soon
as
you
say
to
me,
like
I
had
the
backlib,
I'm
only
interested
in
the
bug
labels.
Then
it's
a
bit
tricky
because
then
I
will
have
to
look
into
the
labels.
I
will
have
to
find
the
issues
with
the
with
the
bug,
labels
and
aggregate
those.
So
that's
an
extra
extra
step.
B
E
E
C
E
A
E
Yeah
yeah,
so
that's
the
other
thing
so,
a
year
ago,
one
and
a
half
year
ago,
when
this
feature
the
new
backend,
was
initially
merged
and
and
and
went
through
review.
Our
timings
were
at
least
four
times
less
than
the
current
and,
as
the
group
increased,
the
number
of
issues
and
projects
things
got
slower.
So
that's
this
trend
will
likely
to
continue.
D
E
Yeah
yeah,
that's
that
might
work
in
some
cases,
but
we
also
have
aggregations
like
median
or
average,
and
that
requires
the
whole
data
set.
That
is
matching
the
the
filters
to
be
loaded,
sorted
and
data
midpoint
out
of
it.
So
that's
a
tricky
one.
Honestly,
I
haven't
looked
at
the
queries
lately
how
they
are
performing
there
could
be
that
you
know
somebody
thought
that
okay,
I
don't
need
this
database
index
or
I
change
something
deep
in
the
in
the
core,
very
logical,
but
we
also
using
vsa
and
things
got
slower.
C
So
that
issue
is
the
the
most
recent
link
in
the
chat
here,
and
I
will
also
add
it
to
the
agenda.
By
the
way
I've
been
I've
been
keeping
some
notes
in
our
like
optimize
meetings
doc.
So
we
have
some
notes
there
as
well.
We
have
like
maybe
30
seconds
left
on
this
call.
Is
there
anything
anybody
wanted
to
wrap
up
with
real,
quick.