►
From YouTube: A Tale of Metrics Faux Pas: Answers Without Questions | Brian Proffitt | CHAOSSconEU19
Description
Slides: https://chaoss.github.io/website/CHAOSScon/2019EU/slides/Tale-of-Metrics-Faux-Pas.pdf
Session: A Tale of Metrics Faux Pas: Answers Without Questions
We all recognize that metrics are key to measuring community health. And that quantitative data is a key to these metrics. But, as Brian Proffitt will describe in this talk, all the data in the world won't help you find answers if you don't know what the questions are. Brian will walk attendees through what happens when pretty data can distract from the real purpose of metrics.
A
Have
a
new
title:
I
haven't
memorized
it
yet
seeing
your
principle
community
architect,
which
is
a
big
wrong
way
of
saying
I,
think
I,
know
stuff
about
community
I
work
on
the
open
source
and
standards
team
at
Red,
Hat
our
job
within
Red,
Hat
and
also
with
outside
of
Red
Hat,
is
to
try
to
foster
community
health
development
and
growth,
primarily
around
the
open
source
upstream
versions
of
our
commercial
offerings.
But
we
also
do
a
lot
of
collaborate,
collaborative
work
with
community
other
communities
inside
of
Red
Hat
and
also
outside
of
Red
Hat.
A
So
we're
trying
to
be
as
helpful
and
with
our
expertise
as
possible.
I
work
at
a
fantastic
team
of
people,
some
of
whom
are
here
today
or
have
been
on
our
team,
and
they
are
amazingly
smart
people,
especially
since
they
conned
me
into
doing
this.
Talk
so
we'll
see
all
right.
So
I
want
to
talk
a
little
bit,
and
this
may
be
a
recurring
theme
for
us
today.
I've
heard
this
coming
up
a
couple
of
times,
but
I
want
to
kind
of
talk
about
how
beautiful
things
are,
but
also
how
harmful
they
can
be.
A
Now
this
kind
of
relates
to
what
we
do
around
metrics
and
metrics
analysis,
but
I
wanted
to
kind
of
walk
through
a
couple
of
analogies
that
we
find
in
nature
so
hello
to
the
blue
dragon.
This
is
a
delightful
animal.
Very
beautiful
I
actually
think
the
picture
doesn't
even
do
it
justice.
This
is
a
three
centimeter
sea
slug,
also
known
as
a
snow
angel,
it's
beautiful
it's
so
pretty
so
yeah
a
bit
except
it
eats
Portuguese,
man-of-war
jellyfish,
and
then
it
stores
the
poison
in
its
spine.
A
A
This
is
the
mantis
shrimp,
which
is
a
pretty
large
15
to
30
centimeter,
shrimp,
which
you
know
it
can
see
more
colors
than
any
other
animal
human
beings
have
three
color
cones
in
their
eyes
as
it
Roger
cones
yeah,
okay,
that,
but
we
can
see
three
colors
RGB
and
then
you
kind
of
can
find
bad.
Sixteen
different
ones,
oh
yeah,
but
the
other
thing
about
this
thing
is
those
two
front
costs
and
accelerate
them
at
speeds,
greater
than
a
rifle
gunshot
which
can
deliver
1,500
newtons
of
force.
A
They
can't
keep
these
things
in
aquariums
because
they
will
break
the
glass
just
with
the
force.
They
don't
even
touch
the
glass.
The
force
of
the
water
hitting
the
glass
can
break
the
glass
not
to
mention
they
also
kill
every
other
animal
in
the
in
the
tank
with
them
and
for
more
information
and
a
delightfully
funny
way.
If
you've
not
read
the
oatmeal
comic,
it
is
certainly
there
and
the
link
will
be
on
the
slides
out
in
the
I.
A
Don't
know
what
Kaos
site
and,
as
we
all
know,
because
I
sense
you
or
like
me,
we
are
all
data
nerds.
Here
we
like
data
data,
is
pretty
just
like
the
mantis
shrimp
yeah,
my
wife.
She
didn't
know
what
to
do
with
me
at
all
so
yeah
I'm,
the
guy
that
likes
the
spreadsheet
kind
of
things
and
well,
let's
make
a
chart,
go
that's
what
we're
here
for
we're
here
to
look
at
data
and
we
find
patterns
and
data.
A
Yeah
the
boys
not
dumb
so
sometimes
I,
know
okay,
but
let's
talk
about
beautiful
data
because
we
see
it
all
the
time
and
and
and
very
much.
Alas,
this
isn't
going
to
really
show
very
well.
It
might
show
a
little
bit
better
on
the
on
the
slides
that
you
see
and
and
definitely
find
a
link
to
this.
This
is
some
really
great
data
out
here.
A
This
one
I
love,
because
this
is
an
informational
data
showing
like
how
much
earlier
the
East,
so
Middle
Eastern
cultures
in
and
around
Persia
and
Iran
and
Saudi
Arabia
were
discovering
things
like.
Oh
that
the
Sun
was
the
center
of
the
solar
system.
They
did
that
in
611
and
unfortunately,
this
got
cut
off,
but
I
think
that
was,
it
was
fifteen
hundred
and
something
before
the
Europeans
figured
that
out.
You
know
this
is
telling
a
story.
This
is
basically
saying
that
anytime,
you
believe
one
culture
is
superior
and
advanced
to
another.
A
A
This
is
a
nice
big
informational
thing
about
all
the
data
breaches
that
have
happened
in
the
last
few
years
and
how
many
user
accounts
have
been
thrown
out
into
the
wild
and
again
you
know,
there's
a
lot
more
information
at
the
actual
URL,
so
I
encourage
you
to
go.
Look
at
that.
Then
that
just
makes
me
sad
and
nervous
and
the
introvert
part
of
me
is
like
yes,
I
am
right
to
stay
home
and
not
talk
to
anybody.
You
know
because
my
I
don't
want
my
data
out
there.
So
again
it's
telling
a
story.
A
Is
it
the
right
story?
Because
all
kidding
aside,
you
know
yeah,
that's
a
lot
of
hats
and
that's
a
lot
of
data
floating
around
in
there,
but
you
can
take
two
things
with
this.
You
can
say
well,
I
just
have
to
be
more
careful
with
my
data
or
another
path
might
be:
oh,
my
god,
the
sky
is
falling.
You
know,
I
want
to
get
myself
off
the
internet
in
all.
You
know
possible
ways
and
any
other
path
in
between.
So
it's
a
lot
more
than
two.
A
Actually,
it's
telling
the
story,
you
can
draw
your
own
conclusions,
okay
and
that's
what
all
data
really
does
you
present
data
you're
trying
to
tell
a
story?
Well,
your
audience
accept
that
story,
you
hope
and
that
maybe
not
I'm
hoping
you
accept
the
story
that
I'm
telling
you
now
when
we
get
to
Q&A
at
the
end
of
this,
you
might
be
like.
Oh,
you
know,
you're
an
idiot
and
then
I've
got
to
kind
of
tell
a
new
story.
Oh,
we
have
to
work
out
a
new
story
together.
A
That's
part
of
dialogue,
that's
part
of
how
this
works,
but
I
want
to
kind
of
back
that
up
a
little
bit
and
let's
talk
about
community
and
metrics,
because
it's
not
always
helpful.
You
can
have
all
the
data
in
the
world,
but
sometimes
it
just
isn't
very
going
to
be
very
helpful
to
you
and
it
can
actually
be
in
some
respects
harmful.
Like
I
said
before
this
could
be
helpful.
This
could
help
you
figure
out
a
strategy
for
managing
your
online
presence.
A
It
could
be
harmful
because
it
could
drive
people
to
panic
and
worry
about
things
and
create
a
lot
of
friction
around
commerce
and
e-commerce
when
really,
what
we
need
to
be
doing
is
finding
solutions.
So
this
is
a
clear
example
of
how
data
can
be
helpful
and
harmful
depending
on
what
story
is
getting
told,
so
we're
all
familiar
with
mature
gian,
again,
my
graphics
for
this
with
kooky
nettie,
but
we've
seen
enough
pretoria
dashboards
already
today.
We
know
what
this
looks
like.
A
This
is
actually
an
active
one
that
we
have
in
Red
Hat,
not
specifically
within
my
OSS
team.
This
is
what
the
open
shift
project
uses.
Openshift
is
a
kubernetes
based
distribution
that
manages
containers
and
does
a
lot
of
platform-as-a-service
kinds
of
work
and
I
really
shortened
that
down,
but
they're,
not
here
so
I
can
get
away
with
that.
A
They
actually
use
this
still
on
a
daily
basis
and-
and
we
know
how
this
works-
and
we
know
if
you're
not
familiar
but
turgid,
does
a
very
good
job
of
presenting
data
in
a
very
informative
and
clean
way.
Ray
had
excellent
examples
of
how
he
could
do.
You
know
surveys
on
that
and
then
and
figure
out
what
was
going
on
in
this
community.
But
let
me
tell
you
a
story.
A
We
have
approximately
well
when
the
story
started.
We
had
approximately
ten
active
communities
that
we
were
managing
and
open
source
and
standards.
We
really
wanted
to
get
some
kind
of
look
about
how
healthy
our
communities
were
not
just
from
the
pure,
purely
theoretical
aspect
of
yes,
we
need
to
make
sure
communities
are
healthy,
but
there's
also
a
business
side
of
this
too.
You
know,
Red
Hat
is
a
business.
We
have
to
make
things
work
well
and
efficiently.
A
Okay,
we're
not
robots,
but
by
the
same
token,
we
just
can't
spend
money
willy-nilly
and
hope
that
you
know
solutions
are
presented.
Okay,
so
metrics
were
seen
as
a
way
of
figuring
out
how
our
communities
were
and
what
their
status
were,
and
if
there
were
problems
we
can
find
them
and
actually
right
before
I
came
on
board
with
Red
Hat.
This
could
we,
my
team
had
started
discussions
with
Pretoria
and
then,
when
I
came
on
board,
my
manager
was
leading
those
discussions.
A
I
would
tagged
along
with
meetings
with
him
and
and
eventually,
when
he
moved
to
another
part
of
the
company,
I
sort
of
took
over
the
the
contact
person.
True
and
we
had
a
great
relationship
with
Peter
gia
and
they
built
us
beautiful
dashboards,
they
were
gorgeous.
I
was
working
on
one
project,
then
called
overt
and
overt.
A
A
This
is
the
more
in
the
problem,
because
as
much
of
a
data,
geek
I
am
and
basically
I
would
really
describe
myself
as
a
very
talented
amateur
or
with
data
I'm,
not
data
scientist
in
any
way
shape
or
form.
I
was
having
trouble
finding
actionable
items
out
of
this
data
and
we
used
to
have
conversations
and
I
know
and
I'm.
Sorry
by
the
way
this
is
my
public
apology.
I
know:
I
was
driving
these
people
crazy,
because
I
was
like
they're
like
well.
A
What's
wrong
and
I'm
like
I,
don't
know,
but
you
know
I'm,
not
really
getting
a
lot
of
value
out
of
this
and
he's
like
was
something
broken
and
yeah
hey-zeus
was
I,
think
he's
about
ready,
throw
a
shoe
at
me
and
and
I'm,
probably
too
close
now.
So
it's
all
good,
but
it's
one
of
those
things
it's
like
I,
couldn't
think
about
what
was
worthy.
A
What
was
not
working
about
this
and
made
it
worse,
my
colleagues
and
again
I'm
not
trying
to
throw
anybody
under
the
bus,
okay,
they
weren't
really
getting
this
either,
and
these
are
talented,
smart
people
and
and
and
they're
very
good
at
what
they
do,
but
they
weren't
understanding
like
you
know
they
well
know.
Let
me
back
that
up.
They
weren't
interested
in
using
the
patter
giad
dashboard
to
glean
out
information
about
their
community
health
and
we
used
to
have
meetings
about
the
console.
You
well
what's
wrong
right.
A
Well,
it's
hard
to
find
what
I'm
looking
for
in
here.
Well,
how
can
we
help
you
find
what
you're
looking
for?
Well,
I?
Don't
have
enough
time
to
do
this.
You
know
I've
got
other
things.
I
have
to
do.
I've
got
daily.
Things
like
I
have
to
manage
events.
I
have
to
manage
social
media
accounts.
I
have
to
you
know:
I
have
to
get
contributors
in
line
because
they're
having
a
fight
over
on
this
mailing
list.
A
You
know
there
are
a
lot
of
different
reasons
and
Gailey
life
seem
to
be
pushing
this
out,
and
so
you
know
we
have
two
ways
of
doing
this
one.
We
could
just
blame
for
turgid
and
say
no,
the
sucks
were
done.
We
did
not
okay,
said
I'm
not
doing
it
now
either
or
we
can
be
honest
with
ourselves
and
say:
the
problem
was
really
as
simple
as
the
old
Douglas
Adams
adage
from
Hitchhiker's
Guide
to
the
galaxy.
What
is
the
answer
to
life,
the
universe
and
everything?
A
Exactly
so,
I'm
not
really
giving
anything
new
here,
but
that
was
the
problem.
We
didn't
know
what
questions
we
needed
to
ask.
We
had
a
vague
idea
like
mailing
lists
growing
in
traffic
and
that
graph
is
moving
up
and
to
the
right.
That
was
good,
but
what
did
it
mean?
It
could
mean
that
you're
getting
more
users
and
you've
got
a
lot
more
questions
and
answers.
A
So
with
regret,
we
kind
of
eased
up
a
little
bit.
We
said:
look
we
can't.
We
don't
really
want
to.
You
know.
We
can't
really
justify
working
with
you
guys
when
we
don't
know
what
we're
doing
we
haven't
figured
this
out
and
and
the
the
information
is
not
going
to
be
used.
So
we
use
up
in
our
relationship
with
Peter
jet
lease
on
a
commercial
level.
A
A
Oh,
no,
don't
don't
put
that
away!
I
want
that,
because
that
community
manager,
whose
name
is
daya
Muir
if
you're
familiar
with
her
diane
muir,
knew
what
to
do
with
this.
She
found
the
use
in
the
dashboard.
She
was
getting
the
story
she
needs
and
I
were
taught
to
Diane
a
lot
and
say
what
are
you
doing?
How
can
I
get
that
to
translate
and
some
of
it
was
translatable
and
then
some
of
it
wasn't?
A
A
What
kind
of
saved
us
was
really
project,
chaos
and
I'm,
not
going
to
repeat
it.
What
project
chaos
is
here
because
I
think
we
all
know
what
project
chaos
is
here,
but
that
saved
us,
because
more
than
anything
else
that
started
laying
the
front
the
groundwork
for
questions
to
be
asked
and
now
we're,
starting
to
as
a
team
revisit
the
notion
of
metrics.
With
the
idea
of
we
have
certain
questions
that
we
want
to
be
answered.
A
You
have
to
be
ready
to
ask
the
questions.
We
have
to
be
scientists.
It's
it's
not
really
a
question
of
running
with
your
gut
anymore,
which
is
what
community
managers
have
been
doing
for
quite
some
time.
We
have
to
be
able
to
define
questions
test.
Those
questions
as
hypotheses
and
see,
if
that's
the
that's
the
conclusion,
and
if
it's
not
the
conclusion,
don't
reject
it
work
with
it.
A
This
is
what
we
should
be
doing,
and
this
is
why
I
believe,
if
you
get
two
things
out
of
this
one
is
that
you
know
questions
are
going
to
be
critical.
You
have
to
understand,
you
know
what
it
is.
You
know
what
it
is
that
marks
a
healthy
community
and
that's
where
project
chaos
can
come
in
and
help
you.
You
have
to
be
able
to
codify
your
questions
instead
of
saying
you
know,
why
is
my
mailing
list,
you
know
growing
in
size.
Maybe
you
should
narrow
that
down.
You
know.
A
Are
there
maybe
what
what's
the
content
of
your
traffic?
Maybe
if
you're
lucky
you
can
do
some
semantic
analysis,
you
know
run
some
run.
Some
of
that
are
there
loaded
words
in
there?
Are
there
people
using
language
that
probably
wouldn't
be
considered
civil?
You
know
that
might
mean
the
aggressive
state
of
your
mailing
list
is
a
little
high
and
maybe
you
do
have
a
troll
problem.
A
A
You
know
and
and
kubernetes
is
cool,
but
they're
big.
You
know
so
you've
got
to
be
able
to
kind
of
figure
out
what
might
be
happening
with
your
community
and
and
don't
be
afraid
to
ask
dumb
questions.
That's
another
thing
that
people
do
well,
my
community's
fine
we've
been
around
for
two.
You
know
X
amount
of
years
and
everything's
going
great.
There's
really,
you
know,
I,
don't
have
any
problems
in
this
area.
No,
no!
No!
No!
No!
No!
No!
Okay!
A
Probably
it's
really
the
case
more
power
to
you,
but
don't
be
afraid
to
ask
the
obvious
questions.
You
know.
Is
the
sky?
You
know
no
bad
metaphor
we're
moving
on
so
don't
know.
That's
that's
me
trying
to
make
a
joke
and
right.
No
we're
done.
That's
not
gonna
be
funny,
we'll
pull
in
the
back,
so
questions
are
critical
and
the
answers
have
to
tell
stories
now.
I
want
to
be
very
careful
here,
because
this
is
my
journalism
background
coming
a
journalism
we
tell
stories
by.
A
There
is
no
such
thing
as
an
unbiased
journalist,
okay,
because
I'm
a
human
being
rocked
in
my
shell
I'm,
going
to
tell
a
story
the
way
I'm
telling
it
I'm,
basing
it
on
my
own
experiences,
my
own
language,
my
own
metaphors,
my
own
cultural,
our
shared
cultural
values
and
and
and
language.
So
to
speak,
that's
how
stories
are
told.
Okay,
you
have
to
be
ready,
but
it's
okay
tell
the
story
anyway.
It
people
won't,
may
not
agree
with
you.
A
You
know,
but
if
you
tell
if
the
data
that
you're
looking
at
will
tell
you
a
story
and
you
can
put
that
story
out
there
now
it
becomes
open
for
collaboration.
Now
it
becomes
open
for
comment
and
query
and
more
questions,
which
is
the
magical
magical
thing.
Okay,
because
anytime
a
story
raises
more
questions
than
any
answers
to
me.
That's
that
you've
done
a
good
job.
A
There
is
a
little
bit
of
this
here.
You
should
know
your
audience
I'm
telling
you
stories
about
data
I'm,
not
in
here
telling
you
stories
about
football
or
anything
like
that,
because
maybe
you
don't
care
because
I
know
my
audience.
If
you
do
come,
see
me
later,
but
I
know
my
audience
and
that
doesn't
mean
placate
your
audience.
I
want
to
be
very
careful
there.
I,
don't
want
you
to
tell
a
story
that
makes
you
always
look
good.
A
A
Well,
you
know
afraid
of
you
know
being
being
affected
by
impostor
syndrome,
but
by
the
same
token
we
have
to
be
honest
with
ourselves
and
say:
these
are
the
things
that
I
know
I
can
do,
and
these
are
the
things
I
know
I
can't
and
where
I
get
help.
So
answers
should
tell
stories
and
if
you
keep
that
in
mind,
I
think
your
experience
with
metrics
will
be
much
better.
So
with
that,
thank
you.
That
is
me
on
online
when
I'm
not
afraid
of
data
breaches.
So
are
there
any
questions?
We
got
five
minutes.
B
B
B
I
hate
it
agree
with
you,
so
the
prime
is
finding
out
how
the
data
can
be
useful
and
now
I
move
into
my
dream.
This
afternoon
we
are
going
to
talk
it
there
in
the
working
group
for
the
empty.
Exactly
with
that,
because
we
are
trying
to
figure
out
how
to
make
matrix
useful
for
people,
and
we
have
to
make
two
different
things
in
part.
Learning
from
your
experience,
one
is
use
cases.
So
what
is
people
using
matrix
for
whatever
I
think
said?
B
A
Very
welcome
what
he
said.
So
that's
good
yeah
and
what
you
would
call
you
use
cases,
that's
kind
of
what
I'm
calling
stories.
Basically
the
same
thing:
we're
constructing
a
model.
You
know
a
use
case,
a
story,
something
that
displays
what
we're
trying
to
convey
so
same
thing,
any
other
questions
or
comments.
A
And
I,
thank
you
for
reminding
me
about
that.
I
did
not
get
a
chance
to
attend
that
talk,
but
I
will
tell
you
that
all
the
tots
at
that
conference
you
mentioned
dev
comp,
dot,
C
Z.
If
you
go
there,
they
are
putting
the
recordings
up.
Eventually,
the
recording
for
Diane
Diane
sand
and
Daniel
stock
will
be
online,
so
I
invite
you
to
go
check
that
out
and
you
will
learn
more
about
how
they're
getting
good
stories
out
of
open
ship.
Because
again,
this
is
not
me
trying
to
slam
a
dashboard
I'm.