►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Community Data is Not Community Metrics - Brian Proffitt, Red Hat
As Project CHAOSS highlights, metrics are key to measuring community health. And that quantitative data is a key to these metrics. But, as Brian Proffitt will describe in this talk, all the data in the world won't help you find answers if you don't know what the questions are. Brian will walk attendees through what happens when pretty data can distract from the real value of metrics, and what the right questions can be.
https://sched.co/Nrt7
A
Good
afternoon
my
name
is
Brian,
prophet
I
am
a
community
architect
with
the
Red
Hat,
open
source
program
office
and
I'd
like
to
thank
you
today
for
joining
my
talk
about
community
data
are
not
community
metrics.
So
we'll
start
the
talk
with
a
little
bit
of
a
story
about
how
data
can
be
very
good,
but
not
necessarily
useful.
A
Okay,
so
there
are
a
lot
of
things
in
nature,
which
can
be
very
beautiful,
but
also
rather
harmful,
and
data
is
a
part
of
that
story.
So
let
me
tell
you
about
the
blue
dragon.
The
blue
dragon
is
a
three
centimeter
long
sea
slug,
also
known
as
a
sea
angel
and
in
each
Portuguese
man-of-war
jellyfish.
So
it's
very
beautiful,
lovely
to
look
but
because
it
eats
poisonous
jellyfish,
it
can
basically
kill
you
with
the
little
things
at
the
end
of
its
appendages.
A
A
It
can
also
shoot
its
front
legs
out
to
the
velocity
of
a
gunshot
from
a
rifle
and
that's
basically
how
it
kills
anything
that
comes
near
it
and
it
it's
hungry
so
and
that
will
strike
any
prey
with
1,500
newtons
of
force.
I'm,
sorry
I,
don't
know
the
metric
conversion
for
your
news,
but
that's
basically
a
sledge
hammer
of
course
dropped
on
a
tiny
little
fish
near
it.
So
again,
these
are
things
that
are
very
but
they're.
Also
they
can
tell
they
can
deceive
you.
A
A
A
A
So
we
have
to
have
our
communities
be
strong
and
efficient
and
vibrant.
Ok,
we
meet
and
we
needed
data
to
show
that,
but
the
problem
was
that
last
slide
we
had
I'll
back
it
up.
We
have
a
lot
of
data
here,
but
we
did
not
know
what
to
do
with
it.
It
was
not
telling
an
efficient
story,
and
this
is
why
this
is
the
point
of
my
talk
today.
We
have
to
come
up
with
better.
A
So
we're
going
to
talk
a
little
bit
about
you
know
how
what
makes
communities
healthy?
How
do
they
work
and
and
how
you
know,
what
are
the
factors
that
make
communities
strong
in
the
past
you
might
say:
well,
my
community
has
a
million
downloads
of
our
software
every
month.
Okay,
that's
great!
That
means
a
lot
of
people
like
your
software
and
they
like
to
use
it.
Does
that
make
you
a
healthy
community,
though?
A
Maybe
not
because
maybe
these
while
people
are
downloading
your
software,
maybe
you're
having
a
huge
fight
inside
of
your
community
about
the
best
way
to
develop
and
the
best
tools
to
use
to
release
your
software
or
where
are
you
going
to?
You
know?
Have
your
next
conference,
you
don't
know
these
things
where
an
outsider
would
not
know
any
of
this.
Okay,
they
just
know
your
software
is
going
smoothly
and
but
eventually,
if
your
community
is
unhealthy,
the
software
that
you
create
will
not
be
as
good.
A
A
So
I'm
going
to
talk
about
the
four
things
that
we
at
Red
Hat
and
also
we're
working
with
a
project
in
the
linux
foundation,
known
as
project
chaos.
Chaos
is
community.
Health
analytics
for
open
source
software
and
chaos
has
four
different
areas
and
Red
Hat.
It's
very
invested
in
these
areas
that
talk
about
aspects
of
community
health,
the
first
one
being
evolution
evolution
is.
We
look
at
the
products
or
projects
entire
lifecycle,
so
when
a
project
is
growing
and
new,
there
are
certain
things
about
it.
A
That
will
be
say
it
is
healthy
or
not
healthy
when
a
product
is
mature,
you're
looking
at
other
metrics
that
determine
whether
or
not
it's
healthy.
So
if
you
have,
for
instance,
as
an
example,
a
new
project
and
there's
only
one
company
like
say
red
hat,
that
is
usually
are
like
building
that
project:
that's,
okay,
it's
brand
new!
It's
a
baby
project.
A
It
has
to
be
nurtured
and
grown,
and
eventually
more
people
will
come
and
work
on
it
if
the
project
is
healthy,
if
the
project
has
only
one
company
or
organization
working
on
it,
when
it's
been
around
for
a
long
time
and
is
mature,
that
is
a
problem,
because
now
no
one
else
is
coming
in
to
help
you
and
collaborate
with
your
project.
Your
company
is
the
only
one
using
it.
A
Eventually,
that's
going
to
hurt
you
because
you
will
lose
creativity,
you
will
lose
innovation
and
probably
you
will
lose
users,
because
no
one
will
want
to
use
your
software,
even
if
it's
really
good,
because
they
won't
want
to
participate
and
see
it
develop
and
grow.
And
then
somebody
else
will
come
along
with
something
that
is
better
and
people
will
go
to
that.
So
it
is
important
to
have
as
many
organizations
and.
A
A
The
Apache
Software
Foundation
is
a
good
example
of
this.
The
Apache
Software
Foundation
retires
their
projects
into
what's
known
as
an
attic
and
and
an
attic
is
where
you
store
things
in
your
home
and
you're,
not
going
to
see
them
for
a
while
and
the
Apache
Software
Foundation.
That's
that's
a
key
thing
because
no
one's
working
on
the
code,
but
it's
still
there.
If
somebody
else
wants
to
come
later
and
maybe
use
it
again,
they
can.
The
software
is
preserved
to
be
used
later.
A
Another
area
within
a
community
health
that
is
very
important
is
diversity,
and
when
we
talk
about
diversity,
we're
talking
about
lots
of
different
things,
we're
talking
about
gender.
So
are
there
too
many
men,
not
enough?
Women
are
we're
talking
about
race.
Are
there
too
many
of
one
race
and
none
of
another?
Oh
and
then
the
corporations,
as
I
said
before,
are
there?
Is
there
too
many
people
from
one
company
involved
in
this
project
and
not
not
enough
of
others?
So
diversity
is
a
lot
of
different
things,
but
it
is
important.
A
Mark
of
health,
for
a
community
value
value
is
something
that
is
important
for
community
health,
especially
in
when
we're
talking
about
corporations.
You
know
what
is
the
business
value
of
an
open
source
project?
How
do
we
determine
that?
How
much
money
do
I
save?
If
a
lot
of
my
software
testing
and
chewie
is
being
done
in
the
community
and
I
don't
have
to
pay
for
people?
You
know
in
my
company
to
do
a
lot
of
QE
for
the
commercial
version
of
that
project.
A
Okay,
that
is
one
example
of
value,
so
that
is
something
that
cannot
be
ignored,
use
me
and
then
finally,
we
have
risk.
Risk
is
important
to
note,
because
risk
is
something
that
we
we
look
at
every
day.
So
there's
there's
lots
of
different
kinds
of
risk.
Some
people
will
say
that
licensing
is
risk
if
I
lie
since
my
software
a
certain
way
there
is
a
you
know.
People
have
a
fear
they're
uncertain
about
whether
or
not
someone
will
take
their
code
and
you
know,
run
off
with
it.
A
That's
usually
not
a
problem,
but
it
can
be,
and
people
do
assign
risks
to
that.
Another
example
of
risk
is
something
there's
an
English
jargon
for
this
known
as
the
bus
factor
where,
if
your
project
has
one
person
who
is
very
very
important
to
the
project
and
that
person
may
be-
and
you
know
is
hit
by
a
bus
which
is
bad,
what
will
that
do
to
your
project?
A
Okay,
a
non
violent
version
of
this
by
the
way
is
the
we
call
it
the
lottery
factor
if
someone
wins
the
lottery
and
they
win
millions
of
dollars
or
millions
of
euros
or
one,
and
they
say
my
you
know,
because
they're
going
to
an
island
in
the
Pacific
somewhere.
What
will
happen
to
your
project
when
that
person
wins
the
lottery?
Okay,
and
there
are
many,
many
very
important
open-source
software
projects
that
have
a
bus
factor
of
one.
A
That's
bad
because
a
lot
of
projects
will
use
their
software
and
if
that
person
retires
or
wins
the
lottery
or
for
whatever
reason
leaves
that
project
that
project
will
languish
and
die
and
other
projects
that
use
that
code
are
going
to
be
hurt
as
well.
So
risk
can
be
very,
very
important
and
and
and
also
really
tricky
to
us.
A
So
these
are
the
four
areas
that
we
look
at.
These
are
where
our
questions
are
going
to
have
to
come
from.
As
we
talk
about
community
health
and
community
strengths,
and
the
answers
have
to
tell
stories,
we
have
to
tell
stories
with
our
data
as
we
go
go
through
these
communities.
All
the
metrics
that
I
talked
about
are.
A
I
already
did
it
in
my
first
example,
when
I
was
talking
about
corporate
diversity
in
the
evolution
model
of
growth,
maturity
and
decline,
I've
combined
divert
the
the
metrics
around
diversity
to
come
up
with
a
new
question
and
a
new
story
about
where
my
project
health
is
when
I'm
growing
and
when
it's
mature.
These
are
these
are
things
we
have
to
think
about
and
what
the
chaos
project
is
doing
and
and
red
hat
we
are
adopting
this
within
our
own
open
source
program
office.
A
Is
that
we're
going
to
find
metrics
on
a
very
low
level
that
can
be
combined
and
and
form
new
questions?
How
many
here
are
familiar
when
I
talk
about
Lego
bricks,
yeah,
so
Lego
bricks
are
the
little
tiny
toys
that
you
you
put
together
in
different
ways
and
I
can
make
you
know
a
building
or
I
can
make
a
spaceship
or
something
like
that,
but
they're
the
same
bricks.
A
Okay,
that's
what
we're
trying
to
do
with
project
Cass,
make
small
atomic
measurable
metrics
that
you
can
combine
to
build
the
stories
that
you
need
to
tell
about
your
community
and
and
ask
the
right
questions
about
the
data
that
you're
delivering.
So
a
very
easy
example
of
this
is
I'm.
Looking
at
mailing
lists
in
my
community
and
every
month
my
mailing
list
traffic
is
about,
let's
say
a
thousand
messages
a
month.
Maybe
you
know,
and
it's
the
same
number
a
little
bit
higher
a
little
bit
lower.
It's
about
a
thousand.
A
Every
month
and
then
one
month,
I
have
1500
messages
and
the
next
month
I
have
2,000.
So
my
traffic
is
going
up,
as
we
say
in
yeah
and
data
up
and
to
the
right.
That's
good
right,
maybe
well,
it
could
be
good.
It
could
mean
that
more
people
are
participating
in
your
community.
It
could
also
mean
that
there's
somebody,
you
know
causing
problems
on
your
mailing
lists
and
there
are
fights
going
on
in
your
community
on
your
mailing
list.
A
A
So
our
metrics
try
to
set
scientific,
quantitative
reasoning
and
not
just
feelings,
an
intuition
around
analyzing
a
community,
and
once
you
do
that,
then
you
can
tell
the
stories
about
your
community
that
are
strong
and
accurate,
and
that
is
a
conclusion
of
my
talk.
But
I
will
be
happy
to
take
questions
if
there
are
any
from
the
group.
A
A
A
So
I
will
I
will
give
you
the
answer.
I
was
just
repeating
the
question
for
the
recording,
so
a
good
example
is
one
that
I've
already
mentioned
before
so
downloads
and
and
and
how
that
is
historically,
that
has
been
used
as
a
metric
to
determine
community's
strength
a
better
way
to
turn
that
around
would
be
instead
of
looking
how
many
people
are
consuming
and
using
your
software,
you
might
want
to
turn
that
around
and
say
how.
How
fast
is
your
software
being
produced?
A
Maybe
when
your
software,
your
project,
was
first
starting,
you
were
putting
out
a
release
every
six
months,
and
that
was
regular,
but
now
your
project
is
releasing
every
eight
months
or
nine
months,
and
and
it's
getting
longer
and
longer,
and
you
need
to
look
at
that
and
determine-
is
there
a
problem
in
your
community?
Is
the
software
so
complicated
that
you
can't
do
it
in
six
months?
That
would
be.
That
would
be
a
good
reason,
but
maybe
there's
a
problem
with
your.
A
You
know
your
testing
process
or
your
you
know
your
release
manager
is
not
doing
their
job
effectively.
So
you
need
to
look
at
you
know
not
whose
can
in
your
software,
but
how
is
it
being
produced?
That
would
be
an
example
of
turning
that
number
around
and
seeing
if
using
that
to
determine
if
your
community
is
healthy.
C
Right
for
the
metrics
right
for
the
community
metrics,
there
are
some
fields
which
you
directly
have
such
as,
like
you
set
the
number
of
downloads
or
the
traffic
on
the
mailing
list.
But
do
we
take
into
consideration
some
derived
metrics
things
like
relation
between
the
increase
in
download
and
the
number
of
increase
in
the
full
requests
or
get
of
activity?
Could
we
have
some
sort
of
a
relation
from
that
and
we
can
identify
that?
Yes,
the
increase
in
downloads
is
actually
a
good
sign,
because
there
has
been
a
lot
of
activity
on
the
product.
A
Yes/No
answer
yes,
because
if,
if
you
can
so
you,
we
need
to
start
applying
the
scientific
method,
so
you
have
a
hypothesis
that
you
just
said:
okay
is
the
increase
in
downloads
and
pull
requests?
Is
there
direct
correlation?
So
you
need
to
what
I
would
do
in
that
situation
is
test
it?
Okay,
you
you,
if
you
do
the
scientific
method
and
it's
an
experiment,
and
this
is
what
we
all
have
to
do
when
we're
talking
about
these
data
stories.
We
need
to
make
sure
that
they
are
testable
and
repeatable.
A
If
my
download
number
you
know
if
my
pull
requests,
numbers
change
in
a
certain
way
did
that
affect
our
downloads.
That
would
probably
mean
that
the
quality
of
my
software
is
getting
better
right.
There
are
fewer
problems
or
people
are
more
responsive
to
the
changes.
More
pull
requests.
People
are
happier
with
the
software
downloads.
Go
are
yeah.
C
A
Downloads
go
up
right,
that's
a
story
you're
looking
at
right,
so,
but
you
need
to
test
that
and
make
sure
that
that's
consistent
right.
You
know
if
my
poll
requests,
if
I
have
200
one
month
now,
let's
go
up
300
the
next
month.
Doubt
you
know
down,
let's
go
up
even
more,
but
then
I
have
like
a
hundred
poll
requests
on
month.
Three
and
downloads
went
up
still,
maybe
there's
no
relation
right
right,
so
you
have
to
test
everything
and
that's
and
that's
something
I
probably
should
have
mentioned
in
the
top.
A
One
thing
that
I
wanted
to
share
with
you
to
what
another
side
point
to
all
of
this
is
a
lot
of
people
say
well,
my
community
is
different
from
their
community
okay,
and
this
is
why
testing
is
very
important
of
your
data.
One
thing
to
note:
when
we're
talking
about
community
health,
it
doesn't
matter
what
size
your
community
is.
Okay,
so
Shanghai
is
a
very
large
city.
Okay
and
there
are,
and
compared
to
like
a
village
in
Mexico,
it's
there
they're
very
different.
A
Okay,
their
sizes
are
extremely
different
and
their
needs
are,
but
guess
what
both
communities
have
similar
needs.
There
needs
to
be
a
way
to
get
food
to
people
who
live
there,
they're
all
going
to
have
streets
in
Shanghai,
it's
a
multi-lane
highway
or
multi-lane
Road,
okay
and
the
village
in
Mexico.
It's
a
dirt
path.
Okay,
but
it's
still
the
same
thing.
It's
a
it's
a
tool
that
people
use
to
get
from
one
place
to
another.
A
You
know
in
the
village
in
Mexico
they
need
water,
maybe
they're
getting
it
from
a
well
okay
or
multiple
Wells
in
Shanghai.
It's
from
you
know
plumbing
and
pipes
and
a
huge
water
facilitation
facility.
Okay,
that's
it's
the
same
thing.
Okay,
you
just
are
using
different
tools
to
do
the
same.
The
same
processes,
okay,
both
communities
are
healthy.
The
village
in
Mexico
is
doing
great
Shanghai
doing
great
okay,
even
though
they're
you
know
wildly
different
in
terms
of
what
they
are
delivering
to
are
using
to
deliver
to
their
communities.