►
Description
Metrics are more than the sum of their parts
There was a time in human history when it was believed that all of human behavior could be explained with math. That widely acclaimed and highly respected science was disproved by an “incompleteness theorem” that showed math cannot explain the world on its own. In another time in human history, psychologists believed all human behavior could be reduced to cause and effect. This gave rise to Gestalt theory that shows perception is not always mapped cleanly or with certainty. We are in a state of Community building that maps success to metrics like lines of code and GitHub stars. I would like us to explore where these metrics fall short on capturing the most significant value of a community, and how we can use other metrics to begin to show the much larger impact Community organizations like Developer Relations (DevRel) can have on a business. This improved process begins with mapping value to business need, pairing it with a community need, and back to the business opportunities. If you enjoy a little science, a lot of community metrics, and are looking for new ways to show a business the value of your community efforts, this talk is for you. There will be practical advice on how to pitch and position Community whether you’re a Community Manager, Developer Advocate, or other Developer Relations professional.
Slides: https://chaoss.github.io/website/CHAOSScon/2020EU/slides/MattBroberg.pdf
-----
Matt Broberg
Technical Advocate & Editor - Opensource.com
@mbbroberg
Matt is an advocate for open source software and currently the Technical Editor of Opensource.com. He specializes in designing technology communities that develop products and content in a way that tells a powerful story.
Matt was an EMC storage expert, VMware vExpert, and former fan of other proprietary technologies. He now focuses on open source and DevRel adoption. He is a serial podcaster, best known for the Geek Whisperers podcast, co-built the DevRel Collective, and often shares on Twitter and GitHub @mbbroberg. He’s also a fan of tattoos and cats, though he remains unsure of Schrödinger’s.
-----
About CHAOSScon
Learn about open source project health metrics and tools used by open source projects, communities, and engineering teams to track and analyze their community work. This conference will provide a venue for discussing open source project health, CHAOSS updates, use cases, and hands-on workshops for developers, community managers, project managers, and anyone interested in measuring open source project health. We will also share insights from the CHAOSS working groups on Diversity and Inclusion, Evolution, Risk, Value, and Common Metrics.
https://chaoss.community/chaosscon-2020-eu/
A
A
This
is
a
very
industry
focused
talk,
as
opposed
to
some
of
the
academic
once
and
you'll
see
that
I'll
hold
a
lot
of
opinions
that
don't
necessarily
have
statistical
models
behind
them
and
we'll
talk
about
that
after,
but
I've,
measured
everything
from
page
views
to
things,
I'll
never
quite
understand
we're
expected
of
me
to
daily
active
users,
monthly,
active
users,
everything
you
need
to
try
to
make
money
with
open
source.
This
is
also
probably
gonna,
be
a
little
american
in
the
framing
of
open
source.
A
A
It's
all
the
Hagar's
are
result
of
that
experience
that
we're
participating
in
and
there
are
two
classic
models
you
can
think
about
a
mouse,
pushing
a
lever
and
giving
a
pellet
and
doing
that
over
and
over
again,
you
can
think
about
ringing
a
bell
and
a
dog
drooling,
the
Pavlovian
model,
but
they're,
both
ideas
of
which
creatures
interact
humans
as
well
there.
It's
a
really
strong,
statistical,
a
statistically
backed
way
of
understanding
human
behavior
and
creature
behavior.
It's
it's
helpful
because
it's
observable
it's
easily
measured
it.
A
You
can
see
modifications
in
behavior
and
then
be
able
to
modify
them
again.
So
there's
a
lot
of
scientific
validity
to
it,
but
it
also
holds
its
weaknesses.
It's
unable
to
really
understand
the
the
human
aspect
of
things,
the
feelings,
the
moods,
explaining
all
the
reasons
why
so
in
a
similar
time
there
is
this
counter
argument
coming
from
the
gist
alt
community
and
just
alt
says
that
the
whole
is
more
than
just
the
sum
of
its
parts.
We
one
of
the
classic
ways
in
which
you
see
this,
you
may
see
a
duck.
A
I
might
see
a
rabbit.
We
are
both
in
fact
seeing
the
same
object
and
they're
both
in
fact
true,
and
that
is
a
way
in
which
we
are
being
there's
an
allusion
to
this,
but
there's
also
something
that's
clearly
very
human
about
that.
We
are
fooled,
but
not
fooled.
At
the
same
time,
they're
both
we
can
hold
both
truths
simultaneously
and
there
are
a
number
of
other
laws
that
were
discovered
and
tested
methodically.
A
So
there
there's
a
lot
of
metrics
and
science
and
really
interesting
research
papers
behind
that
that
show
that
by
the
law
of
proximity,
despite
this
being
just
an
incoherent
number
of
blobs,
we
see
a
donation
or
the
fact
that
most
to
all
of
us
will
see
a
triangle
here,
even
though
no
triangle
actually
exists.
These
are
ways
in
which
are
we
are
mentally,
modelled
to
see
the
world
in
in
patterns,
and
it
tends
to
contradict
the
idea
of
behaviorism
if
it
was
a
simple
input
receiving
output.
How
could
we
see
things
that
don't
exist?
A
The
statistics
that
we
use
around
communities
so
I've
always
worked
for
enterprise
businesses
and
businesses
love
business
models.
We
are
surrounded
by
them,
there's
the
classic
one
that
is
effectively
as
useful
as
this
diagram,
which
is
not
useful
at
all.
This
idea
that
we
are
all
all
companies
are
engines
that,
if
you
put
the
right
gears
together
and
you
put
the
right
information
in
the
top
of
the
funnel
you'll
poop
out
some
sales
on
the
back
of
the
side
of
this,
this
is
something
that
we
all
kind
of
live
in
that
existence.
A
Where
that
this
is
something
that
people
are
funding
billions
of
dollars
based
on
this
belief
system,
the
sort
of
model
you
know,
maybe
a
more
structured
one
and
more
one
that
will
help
you
get.
Some
VC
funding
comes
from
the
Harvard
Business
Review,
a
sort
of
business
case
model
where
you
understand
your
revenue
streams
and
your
cost
structure
and
all
these
different
ways
that,
if
you
fill
in
these
boxes,
you
will
get
funding.
A
Then
there's
the
one,
many
of
us
if
anyone's
worked
in
a
corporation
or
even
nonprofits,
they
have
org
charts,
which
is
another
model
where
we
have
all
these
print
lines
of
business.
That
are
important
by
some
definition
that
I
haven't
looked
up,
but
somebody
believed
it's
important
and
we
keep
pushing
it
as
important
and
we
have
all
the
classic
ones:
sales,
engineering,
marketing,
support
and
wit
community,
maybe
not
community,
because
sales
makes
money
for
an
organization
which
I
hate
to
admit
is
actually
what
companies
need
to
do.
A
And
then
you
have
engineering
which
is
producing
a
product.
That's
one
step
removed
for
the
money
go
on.
Marketing
is
about
leads,
getting
people
in
loyalty
and
support
and
then
boom
community
we
produce
hugs
and
then
what
do
hugs
produce?
We
we
hug
each
other
more
like
we're
all
about
that
sense
of
belonging,
that
sense
of
purpose
that
we've
collectively
built,
but
it
is
a
far
cry
from
the
monetization
strategy
that,
whether
we
like
it
or
not,
we
are
seeped
in
if
we're
in
a
corporate
environment.
So
what
is
our
model?
A
What
is
our
community
model
if
you're
in
a
community
team,
if
you're
a
developer
relations,
team,
I'm,
gonna
kind
of
meld
those
into
two,
because
I
have
some
commonality
and
we'll
talk
about?
Why
we're
measuring
things
so
that
we
can
maybe
get
to
a
model
what
we're
going
to
measure
and
who
we
should
be
focusing
on
as
we
measure
so
one
of
my
favorite
models
on
how
to
think
about
why
we're
measuring
is
to
think
about
where
we
are
in
this
axis.
A
Thank
you
to
Matthew
from
Hoopoe,
which
does
some
great
consulting
on
Deverell,
but
he
is.
This
idea
of
small
companies
have
different
goals
than
large
companies
and
technology.
First
companies
like
if
you
sell
a
server
or
if
you
sell
a
service,
your
technology
company,
where's
this
technologies
and
neighbors-
think
about
like
all
the
car
industries
or
the
bank
industries,
adopting
a
bunch
of
technology
in
this
digital
transformation
and
using
software
to
do
something
else.
You
have
different
goals
on
each
of
them.
A
I'm
gonna,
stick
to
what
I
know,
which
is
above
I've
only
ever
worked
at
tech.
First
companies
and
I
found
this
pattern.
This
generalization
this
model
to
be
true
in
my
career
that,
when
I'm
at
a
small
company
were
always
desperate
for
revenue
that,
if
you're
not
making
money
whatever
you're
doing
in
community,
we'll
always
end
up
evaporating
away
and
when
you're
at
a
big
company
community
is
this
function
of
reputation,
you're
trying
to
standout
you're
trying
to
be
important
to
something
or
someone?
And
that's
where
your
funding
is
being
derived
from.
A
So
thinking
about
that
thinking?
Where
that,
as
this
kind
of
dangling
appendage
on
the
org
chart,
we
don't
actually
get
to
live
as
our
own
appendage.
We
end
up
rolling
that
something
else
that
already
has
inherent
value
to
somebody
who's
funding
us.
So,
on
the
product
side,
you
end
up
being
user
centric
or
you
end
up
helping
to
derive
some
sort
of
code,
value
or
user
experience
value
and
it
might
be
in
product
management
or
whatever
you
want
to
call
it
or
chart
wise.
A
Well,
on
the
product
side,
you're
building
products,
you
have
to
care
about
the
release
cycles.
You
have
a
lot
of
time
to
get
hands
on,
but
you
might
be
a
little
bit
more
distance
from
that
budget
that
you
think
you
need
to
do
your
job
or
you
might
be
this
kind
of
unicorn
magic
person
in
the
middle
where
you
get
to
be
community
first,
you
may
be
you:
are
your
own
org
chart
in
a
small
startup,
but
what
I
found
in
my
career
experience
is
that
you're?
A
Actually
here
and
you
just
don't-
know
it
you're
in
a
corporate
purgatory,
where
you
have
multiple
org
charts
that
matter
to
you
and
either
one
to
defund
you
in
some
way
that
will
end
up
harming
what
you've
built
the
people
that
you've
gathered
together
in
your
community
and
that's
really
what
I'm
always
trying
to
avoid.
So
when
I
say
why
we
measure
you
have
to
understand
why
you
keep
your
job
and
by
keeping
our
jobs
by
keeping
that
sort
of
funding
stream
and
a
corporation.
A
You
understand
the
value
you're
being
paid
to
drive
for
the
company,
and
that
means
you
get
to
continue
to
build
that
community
and
sustain
it
for
the
long
run,
without
a
comprehension
of
that
I
find
these.
These
programs
that
I
build
floating
away
and
you
get
moved
into
some
sort
of
the
new
hotness,
that's
happening
in
the
technology
community.
So
moving
into
what
we
try
to
measure,
we
have
our
raw
data
that
feels
like
it's.
You
know
just
kind
of
glossing.
It's
it's
not
important
in
a
really
deep
sense.
A
It's
the
the
github
stars
that
everyone's
chasing
the
pageviews
that
we
all
talk
about
the
talks
that
we
give
you
have
a
little
tally
mark
for
today.
But
if
you
try
to
remember
why
we're
measuring
these
it's
for
popularity
or
attention
or
adoption
or
monetization,
depending
where
you
are
on
that
first
axis.
So,
if
we
think
about
what
we're
measuring
I
mean,
I'd
have
to
give
a
plug
where
it's
due,
like
it's
easier
than
ever,
to
measure
the
right
things.
It's
built
into
chaos,
it's
supported
by
bits.
Orgia.
A
You
have
incredible
support,
incredible
tools
that
it's
easier
than
ever
to
get
it
all
in
one
place
and
aggregate
it
and
see
that
additional
value,
so
the
what
is
the
easier
part
than
ever,
but
though
the
metric
isn't
necessarily
in
that
what
I
want
to
draw
a
distinction
between
raw
data,
the
data
that
is
the
input
to
something
and
the
metric,
the
thing
you're
going
to
use
to
to
grow
and
add
value
to
something
and
we
get
to
choose
our
metric.
But
we
don't
get
to
choose
our
data.
A
Necessarily
the
data
is
either
there
or
not.
We
can
gather
it
or
not,
but
there
are
distinct,
there's
distinctness
and
there's
power
and
understanding
what
metric
you're
choosing.
So
when
we
go
back
to
this
idea
of
pageviews
pageviews,
yes,
they're
boring.
Yes,
it's
a
tally
mark,
but
if
you
measure
them
as
a
comparison
to
something
like
how
much
you're
spending
in
Adwords
and
if
I
am
bringing
in
a
number
of
pageviews
for
keywords
that
are
very
expensive
on
Google
AdWords
I
can
be
telling
you
I'm
saving
the
company
millions
of
dollars.
A
What
time
that's
actually
a
way
in
which
I've
gotten
funded
in
the
past
give
hub
stars
as
a
point
of
comparison,
vlogs
written
per
week
as
a
way
of
getting
like
mid-fall
content
for
your
sales
organization.
So
it's
about
like
what
function
and
motivation
you're,
providing
your
organization
to
keep
funding
what
you
are
doing.
So,
if
you
get
one
thing
out
of
the
sock,
I
hope
it's
this!
A
It's
that
there
when
we're
thinking
about
the
metrics
that
we're
gathering
I,
think
we
think
about
it
as
one
big
blob
of
the
things
we're
gathering
but
I
find
it
very
important
to
have
a
distinct
Mis
between
the
sort
of
measurements
that
we're
taking
to
improve
what
we're
building
the
communities
that
we
run
or
ourselves
and
those
I
call
internal
metrics,
they're
ones
for
self-reflection,
so
that
you
can
understand
what
have
I
built
so
far.
How
long
does
it
take
to
build
it's?
How
good
is
it?
A
A
It
get
easier
for
new
contributors,
have
people
on
a
contributor
ladder
like
those
are
internal
to
us,
but
what
we
often
forget
to
do,
or
I've
forgotten
to
do
most
of
my
career
and
I
advocate
very
loudly
for
now
is
understand
how
we
have
external
metrics
ones,
where
we're
marketing
ourselves
effectively,
where
we're
saying
why
people
should
care
about
our
community
while
we're
talking
to
our
organization
and
managing
up
effectively,
which
is
a
term
for
managing
up
your
org
chart.
So
why
does
your
manager
know
that
your
community
is
valuable?
A
What
is
his
or
her
manager
know,
or
their
manager,
and
so
on
up
to
the
CEO?
If
you
can
have
the
corporate
sponsor
that
high,
but
you
need
to
understand
what
the
value
you're
deriving
for
the
organization
so
that
you
can
continue
to
do
that,
so
internal
and
external
are
great
value.
So
what
do
we
measure?
It's
always
going
to
end
up
being
a
comparison
to
something
that
your
organization
already
cares
about.
I
find
like
the
more
I
try
to
bring
new
metrics
in
and
like
do
a
little
song
and
dance
and
convince
people.
A
It's
valuable,
usually
barking
up
a
tree
that
I
don't
quite
understand.
Sorry
for
the
mixed
metaphor,
but
the
idea
is
like
you
need
to
understand
why
people
care
and
then
you
talk
to
them
in
a
way
that
drives
whether
its
financial
or
reputation.
You
need
some
sort
of
function
that
already
exists
in
another
team
and
to
use
that
as
a
point.
So
here's
an
example
of
a
time
where
I
tried
to
make
an
argument
that
I
thought
was
really
witty.
So
let's
talk
about
the
ROI
of
stickers,
so
stickers,
we've
got
a
table
of
them.
A
They
cost
money,
they're,
not
free.
If
we
think
about
the
amount
of
money
it
costs
of
per
day
to
produce
content
that
gives
us
page
views,
those
paid
views
are
a
form
of
reputation
and
visibility.
So
if
I
was
going
to
fund
people
to
make
stickers
a
number
of
employees,
a
certain
amount
of
time,
I
could
derive
a
number
that's
about
five
cents,
seven
cents
per
page
view.
A
If
that
those
articles
are
read
often
enough,
if
I
think
about
sicker
is
the
same
way
and
I
make
some
smart
assumptions
now,
let's
say
like
only
one
and
ten
are
actually
used,
and
one
and
a
hundred
of
the
people
that
take
a
sticker
as
a
speaker
at
some
conference
and
you'll
see
them
on
the
back
of
their
computer.
Like
so,
and
those
the
stocks
are
recorded
like
this
one
and
is
posted
to
YouTube,
and
it
gets
millions
of
views.
A
You
know
this
is
CEO
right
or
talks
given
that
can
be
about
the
customer
references
you're
getting
to
you
or
community
advocates
you're
building
something
to
connect
you
to
something
of
value
that
already
exists.
Another
way,
if
you're
looking
for
a
newer
metric
and
something
interesting,
that's
as
diverse
as
our
community
I
have
to
give
a
shout
out
to
Mary
thing
of
all
she's
author
of
the
business
value
of
developer
relations.
A
She's
this
idea
of
community
or
Deborah
qualified
leads
it's
this
idea
of
any
time
a
developer
relations
professional
can
bring
somebody
into
the
organization,
that's
valuable
to
another
part
of
the
organization.
So
if
you
talk
to
somebody
here
and
they
end
up
being
a
great
sales
lead,
that's
a
Deborah
qualified
lead
if
you
get
some
product
feedback
and
get
that
to
the
right
product
manager.
If
you
end
up
recruiting
somebody
that
gets
hired
by
the
company,
these
are
all
ways
in
which
communities
are
wildly
valuable,
but
we've
never
had
an
aggregate
metric.
A
That
is
meaningful
enough
to
get
it
up
to
somebody
and
they
understand
why
so
their
advantages,
I
think
it's
really
interesting
is
a
powerful
metric.
It's
a
unifying
metric
for
all
the
diverse
work
that
professionals
like
us
do,
but
it
also
has
its
weaknesses,
its
alliance,
to
kind
of
a
sales
terminology.
Maybe
you
want
to
avoid
that
correlation,
but
in
the
end
it's
a
little
untested
right
now,
I'm,
really
looking
forward
to
research
on
how
that
goes
in
practice
and
see
if
it
works
in
certain
environments.
A
Another
kind
of
interesting
model
that
you
can
apply
is
the
most
valuable
user.
I
read
it
in
a
bit:
sérgio
blog
posts
that
I
really
enjoyed
this
idea
that
you
have
these
preferred
or
important
users
that
you
care
about.
They
can
be
you
know,
maybe
early
in
the
sales
cycle.
Pre-Qualified
leads,
they
can
be
open
source
contributors
that
you're
reaching
out
to.
They
can
be
just
a
list
of
influencers
that
a
certain
team
cares
to
reach.
A
If
you
give,
if
you
scope
a
project
around
them,
and
you
talk
about
them,
I
think
it
has
incredible
advantages
when
it
comes
to
the
story
that
you're
gonna
end
up
producing
kind
of
packages
up
the
diverse
work
that
we
do,
that
doesn't
necessarily
make
sense
to
everybody
and
explains
it
in
a
way
we're
like.
Ok,
I
targeted
this
group.
We
did
this
stuff,
here's
what
happened
on
the
back
end,
so
maybe
it's
not
one
or
the
other.
A
Maybe
it's
both
that
the
dql
concept
could
be
that
core
metric
that
drives
a
lot
of
community
work
together
and
the
MBU
concept
can
be
part
of
our
model.
The
community
model
of
how
you
are
successful
as
a
community
leader
inside
an
enterprise
tech
company,
so
if
I
were
to
offer
a
community
model.
That
would
be
this.
A
It's
to
understand
why
you're
funded
to
work
through
what
plan
you
can
have
to
measure
what
you're
doing
understand
who
you're
doing
this
for
and
who
you're
talking
to
you
standardize
on
some
sort
of
comparison,
that's
already
valued
in
your
organization
and
then
communicate
those
results
really
strongly
understanding
that
you
need
to
manage
up.
You
need
to
market
your
work
effectively.
A
A
40
metrics
on
somebody's
plate
is
just
as
useless
as
giving
them
zero
it's
better
to,
and
it's
better
to
not
argue
ROI,
but
to
understand
like
what
value
you're
driving
for
who
and
then
package
up
your
work
in
a
way,
that's
effective
and
ultimately
yeah
report
on
it
regularly
to
get
your
point
across.
So
the
bringing
sorts
of
conclusion.
Yeah
all
models
are
wrong.
Some
are
more
helpful
than
others.
Metrics
are
not
the
end
goal.