►
From YouTube: Regulations, Ethics And Messy Data: Tales From An Open Source Analyst - Sophia Vargas, Google
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Regulations, Ethics And Messy Data: Tales From An Open Source Analyst - Sophia Vargas, Google
Metrics can be a valuable tool for communities to monitor project health, sustainability, operational efficiency and identify deficiencies. For individuals, projects and companies seeking to establish or mature open source metrics programs, this talk will explore opportunities and challenges in and around open source software data and analytics, from identifying relevant metrics to working with unreliable datasets while navigating the ethical challenges of data collection in and around open source communities.
A
A
In
that
role,
I
served
as
a
consultant
for
builders,
buyers,
sellers
of
data
center
infrastructure
solutions
and
then
in
about
2018,
I
moved
to
google,
where
I
still
continue
to
be
a
market
research
and
industry
analyst,
but
focusing
on
supporting
our
products
and
then
last
year
I
moved
over
to
the
open
source
programs,
team
or
programs
office
where
my
title
is
officially
a
program
manager,
but
in
this
role
my
focus
is
on
research
and
analysis.
I
get
to
look
at
a
lot
of
our
contribution
data.
A
You
might
also
know
me
from
the
kubecon
registration
page
thanks
to
the
cancellation
of
all
in-person
events.
My
face
has
been
stuck
on
the
website
since
march
of
2020,
so
I'm
personally
looking
forward
to
the
return
of
in-person
events
and
getting
some
new
faces
out
there
and
maybe
eventually
being
able
to
retire
myself
from
this
page.
A
So
the
general
theme
of
this
talk
is
that
I
gave
you
a
little
bit
of
background
about
myself.
I've
been
working
with
all
kinds
of
data
for
the
last
10
years,
whether
or
not
it's
survey,
data
business
data,
public
data,
financial
data
and
then
in
the
last
year
now
working
with
data
around
open
source
and
open
source
projects
and
communities,
and
there
was
a
fair
amount
of
things
that
I
didn't
really
realize
or
know.
A
Well,
even
though
I
thought
I
knew
the
space
and
so
that
kind
of
comprises
the
theme
of
the
talk
in
terms
of
things
that
I
wish
I
knew
before
I
started
doing
this
job.
I
gave
a
similar
talk
at
open
source
summit
a
few
weeks
ago
for
this
one.
Today,
I'm
focusing
a
bit
more
on
the
implementation
details
in
itself.
A
So
one
of
the
first
things
that
I
had
to
come
to
terms
with,
and
this
shouldn't
be
groundbreaking
for
anyone
is
that
there
is
no
single
definition
of
an
open
source
project.
We
could
be
talking
about
a
code
base,
a
data
set
a
framework,
a
series
of
designs
if
you're
working
with
open
source
hardware.
I
also
am
an
active
member
of
the
chaos
community,
where
we
talk
about
metrics
and
building
common
language
to
discuss
things
like
community
and
project
health.
There
are
some
sticklers
that
would
say
that
open
source
is
defined
by
them.
A
You'll
see
even
more
variation
in
all
the
different
ways
that
you
can
work
on
and
contribute
to
the
project
from
interacting
with
the
code
base
to
community
management
and
event
planning
to
documentation,
localization,
translation,
marketing,
communication
content,
development,
legal
financial.
We
could
go
on,
and
this
is
an
active
area
of
conversation
in
multiple
areas
in
the
community,
where
we're
all
trying
to
come
together
to
better
understand
how
to
contextualize
contributions
outside
of
just
code,
because
a
lot
of
these
things
aren't
in
readily
available
systems
that
we
can
collect
data
from.
A
So
we
need
to
artificially
create
templates
and
ways
to
recognize
these
other
kinds
of
contributions.
If
this
is
a
topic
you're
interested
in,
I
put
in
a
couple
of
resources
here.
The
first
is
a
piece
of
research
coming
out
of
the
university
of
vermont
that
looked
at
various
ways
that
you
could
start
to
categorize
and
model
various
types
of
contribution
outside
of
just
code,
and
the
second
is
within
the
chaos
community.
We're
actively
talking
about
various
types
of
metrics
that
you
could
look
at
or
consider
to
measure
all
these
different
types
of
contribution.
A
A
I
wanted
to
bring
this
up
too,
because
we
start
talking
about
building
out
metrics
programs
or
analysis
programs
and
when
you're
thinking
about
all
these
different
ways
that
we
can
work
and
interact
with
the
project.
This
is
sort
of
leading
us
into
the
conversation.
Okay.
What
kinds
of
data
sources
are
we
going
to
be
working
with?
Are
we
predominantly
focusing
on
git
or
issue
streams?
Maybe
if
we're
thinking
about
troubleshooting
and
support,
we
want
to
better
understand
how
users
are
interacting
with
the
project.
We
could
be
looking
at
forums
like
stack
overflow.
A
If
you
are
thinking
more
about
community
engagement
or
want
feedback
on
your
event
like
today,
or
maybe
feedback
on
tutorial
or
documentation,
maybe
you'll
run
a
survey
and
then
maybe,
if
you're,
focusing
on
more
marketing
and
promotion
of
your
materials,
you
might
set
up
something
like
site
analytics
so
there.
These
are
just
a
couple
examples.
There
are
so
many
different
ways
and
platforms
that
you
can
collect
data
from
to
better
understand
your
community
composition
and
your
project.
A
I
did
a
quick
exercise
while
I
was
putting
this
talk
together
to
say:
okay,
can
I
actively
name
all
the
repositories
that
are
part
of
the
kubernetes
project,
anyone
in
the
room?
How
no
my
many
repositories
are
currently
active
today,
okay!
Well,
when
I
looked
six
months
ago,
it's
about
220,
something
I
want
to
say
today's
a
little
bit
over
300,
but
what
I'm
unclear
on
is
how
many
of
them
are
still
active.
I
know
some
have
been
actively
archived,
so
I
reached
out
to
bob
killen
who's
works
in
within
the
kubernetes
community.
A
We've
seen
this
in
the
cncf,
where
projects
merge
and
become
one
like
open,
telemetry
or
they
break
off
and
become
more.
We
keep
on
adding
more
things
to
the
pie,
so
I
really
wanted
to
expand
the
universe
a
bit
to
acknowledge
the
fact
that
any
time
you're
building
a
metrics
program
you're
most
likely
only
looking
at
a
subset
of
this
information.
So
it's
not
the
full
picture.
It's
a
very
focused
piece
of
the
puzzle
and
it's
completely
fine.
A
That's
how
we
can
draw
boundaries
to
create
real
analysis
and
things
that
we
can
actively
measure
versus
just
conjecture,
but
it's
never
going
to
be
the
full
picture.
So
one
thing
I
like
to
note
here
is
there's
another
area
of
research
coming
out
of
the
university
of
vermont,
which
is
looking
at
trying
to
actually
put
a
number
on
how
much
of
open
source
is
represented
in
github,
because
there
is
a
lot
of
open
source
and
there's
still
a
lot
of
open
source
that
exists
outside
of
it.
A
So
even
there
are
copious
amounts
of
research
efforts
that
are
making
use
of
the
wealth
of
information
that
is
happening
on
github
and
they're,
trying
to
create
conjectures
around,
what's
happening
in
open
source
that
isn't
necessarily
even
the
full
picture
there.
So
this
piece
of
research
takes
a
look
at
how
to
contextualize
that
as
well
as
starting
to
investigate,
maybe
alternate
trajectories
or
things
that
we
can
learn
about
things
that
are
happening
in
github
and
other
styles
of
work
that
are
happening
outside
of
it.
A
The
next
area
of
conversation
I
want
to
bring
up
is
about
the
data
itself.
So
when
I
started
working
with
this
kind
of
information,
there's
sort
of
the
assumption
that
okay,
it's
it's
public,
you
put
information
on
github
about
yourself.
You
perform
activities,
anyone
can
go
to
your
page.
They
can
look
at
what
you've
done.
They
can
look
at
the
information
you
volunteered,
so
it's
just
public.
What's
the
problem
right?
Well,
I
I'm
going
to
put
out
a
proposition
here
that
even
this
kind
of
public
data
deserves
classification.
A
Maybe
I'm
coming
from
the
background
of
working
for
a
large
company
that
thinks
a
lot
about
privacy
and
data
regulation
and
every
company
has
their
own
way
to
associate
risk
categorization
around
data.
They
collect
whether
or
not
it's
their
employee
data,
their
customer
data,
product
usage,
data
infrastructure,
monitoring
data
everything
has
its
own
classification
and
risk
associated
with
either
exposure
risk
to
the
individual
risks
of
the
company.
A
There
are
so
many
different
avenues
that
you
can
do
this
and
it's
very
contextually
driven
by
individual
parties,
and
then
there
are
the
regulating
bodies
that
help
to
influence
and
guide
the
policies
and
things
that
are
implemented
at
the
company
side.
And
then
you
have
the
project
communities
where
something
that
I
love
about
open
source
is
that
every
individual
project
gets
to
create
its
own
governance
structure,
decide
how
decisions
are
made,
decide
how
they're
going
to
guide
the
evolution
of
that
project.
A
A
In
addition
to
that
say,
I've
also
volunteered
that
I
work
at
google.
I
live
in
new
york.
I
have
a
twitter
handle.
You
can
find
me
there,
and
maybe
you
can
see
my
github
page.
It's
very
sad.
I
haven't
actually
done
a
lot
there,
but
this
is
enough
information
now
to
find
me
on
another
social
media
platform
like
linkedin,
where
now
maybe
you've
confirmed
this
information.
A
You've
also
learned
where
I
went
to
college
and
where
I
worked
before
I
worked
at
google,
so
these
things
can
quickly
snowball,
and
so,
if
you're
thinking
about
it
from
a
data
aggregation
standpoint,
you're
adding
more
and
more
information
about
someone.
So
even
though
they've
opted
into
a
specific
platform
around
the
level
of
data
collection,
that's
happening
around
that
person.
A
The
more
that
you
aggregate
as
part
of
your
analysis
program,
the
more
you
know,
which
is
the
suggestion
for
maybe
thinking
about
how
you
might
want
to
classify
and
treat
this
information
and
not
to
say
that
there
aren't
valid
use
cases
for
collecting
pi
irm
projects.
I
know
many
projects
use
a
contribution,
information
to
inform,
who
should
be
going
up
for
election
on
any
given
year,
or
maybe
you
want
to
better
understand
the
composition
of
your
community
to
know
how
you
can
better
support
underrepresented
groups.
A
If
you're
like
me-
and
you
also
work
for
company-
and
you
are
also
working
on
projects
that
are
trying
to
collect
contextualize
employees
contributing
in
and
around
open
source,
then
you
might
also
bring
in
that
data
and
add
it
to
the
pile
to
better
understand,
say
who
within
your
organization,
is
doing
what?
What
projects
are
they
working
on?
What's
their
background
and
for
me
working
in
an
open
source
programs
office,
we
use
this
information
to
know.
Are
we
serving
our
population?
Do
we
know
who
our
population
is?
A
Are
we
priding
the
right,
tooling,
the
right
documentation
to
ensure
that
everyone
has
it
they
need
in
order
to
feel
that
they
can
go
out
and
work
in
these
sorts
of
forums
and
as
a
data
analyst,
I
will
definitely
state
that
it's
so
much
better
to
work
with
huge,
comprehensive
tables
where
everything
is
in
the
same
place.
It
makes
it
easier
to
write
queries.
It
makes
it
easier
to
design
filters
and
explore
the
information.
A
There
might
be
reasons
why
you
want
to
share
some
of
this
back
with
the
community
directly,
and
I
don't
think
this
is
a
problem,
but
when
you
start
to
marry
all
these
various
sources
and
increase
the
risk
levels
of
that
information,
then
you
want
to
be
sensitive
about
how
much
of
it
you
expose
at
a
given
time.
So
the
general
suggestion
is
here
if
you're
comfortable
with
it
tokenize
compartmentalize
and
then
be
able
to
individually
authorize
access
to
tables
and
then
join
it.
A
A
The
stack
overflow
api
and
all
the
activity
and
information
shared
in
those
platforms
are
contained
in
the
platforms
and
the
users
and
individuals
interact
to
it,
understand
those
parameters,
but
in
direct
collection
you,
as
the
person
who
is
going
up
to
someone
and
asking
where
did
you
come
from
what
groups
you
identify
with
your
just
like
that?
Interaction
can
potentially
impact
what
people
say,
because
sometimes
it
can
be
uncomfortable.
Maybe
people
aren't
as
comfortable
sharing
with
information
with
people.
They
don't
know
their
involves
high
level
of
trust
here
or
maybe
they're
just
shy.
A
There
are
many
reasons
why
these
sorts
of
forms
can
potentially
impact
what
you
can
learn
about
people,
so
the
general
recommendation
is
try
to
provide
as
much
anonymity
as
possible,
especially
when
you're
asking
about
some
of
these
more
sensitive
or
personal
areas
and
demographics
even
feeling
satisfaction,
opinions
and
feedback
are
all
better
served
in
completely
anonymous
formats,
and
to
do
that,
you
can
preserve
either
do
not
collect
any
email
addresses
in
these
forms.
I
suggest
doing
that
in
a
separate
venue
if
you
need
something
like
follow-ups
or
meetups
or
contact
forms.
A
There
are
other
ways
to
do
this.
Just
don't
mix
them,
because
this
can
make
people
feel
uncomfortable
and
less
likely
to
share
information
with
you
and
also
provide
opt-outs.
People
don't
have
to
ask
answer
every
question
if
they
don't
feel
comfortable
with
it.
This
is
not
something
that
you
have
to
force
people
into,
there's
also
the
caveat
here
with,
if
you've
ever
thought
about
size
of
data
and
being
able
to
identify
individuals.
A
If
your
community
is
very
tiny,
then,
even
though,
if
everything's
completely
anonymous,
you
might
still
be
able
to
figure
out
who
people
are
based
on
what
they
write.
So
you,
as
the
manager
of
that
survey
of
the
data
collected
around
the
survey,
use
your
best
judgment
here
to
understand
whether
or
not
things
are
appropriate
to
share
back
or
if
they
will
have
some
level
of
risk
and
exposure.
A
The
general
recommendation
across
all
of
these
things
is
it's
all
about
being
transparent,
especially
in
open
source,
where
we're
trying
to
actively
build
trust
and
work
with
each
other.
So
if
there
are
reasons
for
you
to
collect
this
information,
be
upfront
about
it.
What's
your
motivation
here,
I
want
to
better
understand
the
community,
so
I
know
what
to
invest
in
to
encourage
more
participation
to
encourage
more
representation.
A
How
are
you
going
to
collect
this
information?
How
are
you
going
to
store
it?
How
are
you
going
to
secure
it
and
how
are
you
going
to
maintain
accountability
of
it
again?
Another
example.
I
work
in
the
chaos
community
and
we've
recently
been
working
on
diversity
equity,
including
badging
programs,
for
events
to
basically
help
events,
assess
the
diversity
and
the
representation
of
their
speakers
and
other
participants
throughout.
But
in
order
to
do
that,
we're
asking
some
pretty
sensitive
questions,
and
so
we
want.
A
I'm
noting
here
another
another
piece
of
research
at
the
bottom.
The
university
of
uc
davis
recently
investigated
the
they're
calling
sustainability
forecasting.
There's
some
debate
about
the
use
of
sustainability
there.
But
looking
specifically
at
the
graduation
rate
of
projects
in
the
apache
incubator
program
to
see,
are
there
any
discernible
metrics
that
are
better
indicators
of
success
than
others?
A
And
the
overwhelming
observation
with
this
is
that
it's
communication,
just
communication
levels,
not
specific
kinds
of
communication
and
there's
basically
opened
up
a
whole
new
slew
of
research
opportunities
to
better
understand,
maybe
some
of
the
more
dynamics
here
but
tangentially
related
to
what
I'm
talking
about.
But
I
in
my
gut
feel,
like
it's
related
the
better
that
we
can
can
talk
to
each
other
and
communicate
about
what
we're
doing
the
more
collaboration
and
trust
that
we
can
build
within
our
extended
community.
A
The
last
point
I
want
to
make
here
is
a
bit
of
an
odd
one,
because
I'm
not
really
sure
what
the
best
thing
to
do
is
here.
But
if
you're
working
with
a
lot
of
say,
get
activity
logs,
it
is
filled
with
bots
and
automation,
they
could
be
actively
call
bots
and
they
could
be
automated
scripts.
Coming
from
personal
based
accounts
for
a
sense
of
scale,
I
wrote
a
very
basic
query
by
identifying
bots
that
contain
the
letters
vot
and
their
actor
login.
A
So
if
anyone
in
the
room
has
bot
in
their
github
handle,
I'm
counting
you.
So
this
is
not
perfect.
Thus,
I
say
that
we
estimate
that
and
that
I
rounded
the
number
severely
down
over
120
million
bots
did
some
sort
of
activity
on
github,
as
logged
by
the
event
stream
in
2020,
for
a
sense
of
who
these
bots
were
when
you
say
who
we
accept,
that,
with
a
great
result,
depend
bot
being
the
largest
one,
so
no
show
knowing
noting
that
they
are
showing
activities
in
millions.
A
So
this
is
quite
substantial.
This
actually
piqued
my
interest,
and
I
went
further
digging
and
found
that
say
bot
related
accounts
were
responsible
for
44
of
push
events
in
2020
up
from
25
in
2019..
So
this
is
a
significant
portion
of
activity.
That's
now
mixed
in
with
human
driven
activity.
So,
depending
on
what
you're
trying
to
show
in
your
program,
you're
going
to
have
to
deal
with
this
at
some
point,
and
so
the
current
best
practice
or
good
practice
here,
is
to
create
some
kind
of
referenceable
list.
A
So
in
my
case
working
for
a
company
where
we
want
to
understand
activities
that
our
employees
are
generating,
I
have
a
known
list
of
employees
and
we
can
pull
them
in
from
there.
On
the
flip
side,
you
can
have
a
known
list
of
bots
and
you
can
filter
out
from
there.
So
that's
what
the
devstat
program
is
doing
if
you're
familiar
with
devstats,
they
build
all
of
the
wonderful
metrics
and
analytics
around
cncf
projects.
A
The
next
point
I
want
to
make
is
on
the
measurement
component
itself.
One
you
measure
influences
the
thing
that
you're
measuring,
and
this
seems
kind
of
straightforward
and
boring.
I
was
thinking
about
this
talk
this
morning
and
I
looked
at
my
fitbit
and
I
had
600
steps
at
8
o'clock.
I
hadn't
left
my
hotel
room
yet
and
now
I
have
over
4
000.
So
it's
now
been
a
few
hours.
A
I've
moved
around
a
bit
clearly
when
I
took
this
measurement
impacted
the
measurement
itself,
but
what
I
really
mean
here
is
that
git
activity
logs
do
not
stay
consistent
over
time.
In
fact,
history
can
change,
which
was
kind
of
mind-blowing.
For
me,
as
an
analyst
who
worked
with
a
ton
of
business
data
reported
data
financial
data,
the
idea
that
historical
information
can
change.
A
So,
depending
on
when
you
pulled
the
data
and
you
wrote,
queries
around
a
specific
time
period
and
you
rerun
that
say
in
a
few
years
it
can
change.
It
can
look
different
because
the
composition
of
the
project
has
changed
over
time,
so
your
history,
historical
and
your
current
state
don't
always
necessarily
match.
There's
also
all
of
the
inconsistency
challenges
that
other
data
sets
have
specifically
in
this
context.
Github
apis
have
rate
limits,
so
there
are
ways
to
get
around.
A
A
A
I
ran
a
very
basic
query
that
just
looked
at
the
total
activity
count
for
the
same
time
period
2020
as
it
was
shown
in
the
yearly
table,
the
monthly
table
and
the
daily
tables
in
the
same
time
period,
and
you
can
see
that
these
numbers
do
not
agree
they're
in
the
same
order
of
magnitude,
but
there's
a
fair
amount
of
variation,
so
something
is
probably
missing
somewhere.
Do
we
know
exactly
what
is
something
being
overcounted
somewhere
potentially
again
thinking
about?
Are
we
actually
certain
about
this
information,
or
is
this
just
an
estimate?
A
A
Publishing
metrics
also
has
the
potential
impact
of
influencing
the
behavior
that
it's
actually
reporting
on.
So
this
I've
seen
come
up
in
things
like
leaderboards
here,
I'm
showing
the
kubernetes
kubernetes
leaderboard
around
total
contributions
that
I'll
come
back
to
in
a
second,
because
this
one.
Actually
I
actually
like
this
one,
but
there
are
other
examples
of
where
leaderboard
can
actually
potentially
influence
things.
A
You
don't
want,
say
if
you're
reporting
on
just
commits,
then
maybe
you're
encouraging
people
to
just
submit
a
lot
of
commits
and
not
do
other
things
that
can
help
support
the
projects
like
addressing
issues
or
supporting
event
management.
So
they're
just
kind
of
be
wary
of
what
you're
incentivizing,
by
showing
any
kind
of
ranked
list
like
this,
and
that,
just
depending
on
where
you
are
in
the
rank,
it
doesn't
actually
translate
to
value
or
quality
of
those
individual
contributions.
A
So
if
you
do
want
to
share
something
like
this,
this
is
important
to
include
context
with
it.
So
here
I'm
showing
the
combined
contribution
metrics,
which
I
like,
because
it
combines
a
few
things
across
push
events,
issues,
comments
and
reviews
to
ensure
that
we're
looking
more
well-rounded
at
just
code
contributions,
we're
not
actually
looking
beyond
code
in
this
sense
picking
a
time
period,
that's
relevant
to
you.
A
So
not
to
say
that
you
shouldn't
publish
metrics.
I
think
they
can
be
a
very
effective
tool
to
influence
behavior,
it's
more
about
doing
it
purposefully.
So
what
outcomes
are
you
trying
to
achieve
by
showing
some
of
these
metrics
in
the
same
project?
I
really
liked
the
focus
on
looking
at
rate
to
first
review
to
as
a
way
to
demonstrate
productivity
in
the
project,
but
also
responsiveness
in
the
project.
A
A
So,
just
as
a
general
recommendation,
it's
always
good
to
do
a
little
bit
of
homework
to
understand.
Are
there
any
specific
terms,
policies
or
regulation
that
this
data
is
subject
to
whether
or
not
it's
coming
from
the
data
set,
the
project,
the
tooling
any
specific
privacy
or
policy
documentation
from
a
platform,
a
company
or
an
event,
and
then
there's
also
local
regulations
or
regional
regulations
that
can
create
issues
around
how
and
where
you
can
collect
and
store
information
about
individuals?
A
The
platforms
themselves
have
to
dictate
their
own
policies
and
how
those
policies
are
subject
to
regulations.
Then
there
might
be
another
layer
we're
using
a
pipeline
tool
and
aggregation
tool
that
could
have
its
own
policies,
but
then
it's
also
inheriting
everything
that
came
before
it.
So
just
because
the
policies
are
sitting
on,
say,
github
doesn't
mean
they
don't
apply
to
you.
A
If
you've
now
pulled
the
data
from
github
you're
you're,
inheriting
everything
that
they've
stated
within
their
own
privacy
and
policies,
there
is
the
aggregating
entity
that
is
now
actively
storing
the
information
aggregating
and
amassing
that
information,
and
they
could
be
subject
to
their
own
regulations
and
requirements.
They
could
be
coming
from
per
company
policies
or
they
could
also
be
dictated
by
the
project
or
the
community
leadership.
A
Now
you
could
go
through
that
entire
exercise
and
find
a
lot
of
missing
pieces
and
that
not
everything
is
going
to
be
clearly
defined,
and
so,
in
the
absence
of
defined
terms
of
use,
then
potentially
it
might
be
time
for
you
to
think
about
designing
privacy
documentation
again,
I
brought
the
chaos
example
because
we
had
hit
a
point
where
we
were
starting
to
collect
information
beyond
what
people
knew
they
were
sharing
with
us
say
by
opting
into
using
github.
So
we
wanted
to
openly
state
that
this
is
what
we're
going
to
do
with
this.
A
This
is
how
long
we're
going
to
retain
it.
This
is
how
we're
going
to
hold
ourselves
accountable
to
it
and
essentially
assuming
that
position
for
yourself
as
the
individual
or
corporation
or
entity.
That's
amassing
the
information,
if
you
have
any
doubt,
and
you
have
access
to
lawyers,
involve
them
and,
as
a
recommendation
of
the
users
read
the
fine
print
or
else
you
will
end
up
like
me,
stuck
on
a
website
and
going
back
and
checking
my
barcelona
registration
to
say.
A
A
This
is
how
people
will
contextualize
what
they're
looking
at
sources
include
the
method,
the
assumption
any
sort
of
inbaked
bias
or
base
or
boundary
that
relates
to
this
particular
subset
of
information.
So
remember
that
the
how
and
the
when
impact,
what
we're
actually
looking
at
and
can
impact
the
information
itself,
maybe
also
openly
state
what
kind
of
or
type
of
contribution
or
contributor
does
this
represent
and
then,
as
the
last
call
out,
there's
a
lot
of
funkiness
in
a
lot
of
this
information.
A
So
is
this
a
certain
fact,
or
is
this
an
estimation
if
you've
read
academic
papers,
there's
a
fairly
rigorous
template,
as
it
relates
to
reporting
on
the
methodology
around
how
the
data
was
composed
and
analyzed
in
free
form
content?
It's
kind
of
up
to
you
as
the
writer,
so
I
for
an
example
I
want
to
share.
I
wrote
a
blog
a
few
months
ago
and
included
in
about
this
data
section
so
that
you
could
see
an
example
of
the
information
I
thought
was
relevant
to
convey
when
sharing
this
information.
A
If
you're
looking
at
building
out
a
metrics
program,
I
did
want
to
call
out.
There
are
some
tools
available
in
the
community.
I've
called
out
devstats
a
few
times
that,
if
you're
curious
is
a
is
based
on
the
github
archive
project,
which
is
again
that
comprehensive,
github
event
stream
uploaded
into
a
postgres
database
with
grafana
as
the
visualization
layer
for
chaos,
the
project
we
have
a
couple
of
things
like
gremora,
labs
and
auger
that
are
specifically
designed
to
automate
the
collection
and
visualization
of
contribution
metrics.
A
If
you
are
collecting
more
sensitive
information,
allow
people
to
opt
out
not
have
to
share
everything.
If
it's
not
relevant
for
your
case
and
then
just
a
really
question:
do
you
need
pii
or
will
anonymized
version
be
sufficient
for
your
needs?
There's
a
fair
amount
of
research.
That's
just
looking
at
behavioral
trends.
We
probably
don't
need
to
see
pii
where
another
piece
of
research
that
I
read
recently
looked
at
building
social
network
models
around
projects
to
better
understand
how
some
of
those
relationships
could
impact
productivity.
A
A
A
You
can,
if
you're
part
of
say,
like
the
cncf
foundation,
you
can
go
up
through
any
of
the
steering
or
advisory
groups.
I
know
they
do
have
some
sort
of
legal
pieces
to
it.
So,
if
you're
under
a
foundation
that
could
potentially
be
a
support
model,
I
know,
for
example,
the
the
cncf
steering
committee
was
fairly
influential
in
designing
the
community
survey
for
kubernetes
around
some
of
these
more
sensitive
issues,
so
they
were
definitely
imparting
their
own
guidance
to
encourage
better
or
less
risky
data
practices.
A
But
that's
one
starting
point:
it
is:
they
are
pretty
expensive.
I
actually
have
a
friend
who's,
starting
at
a
startup
that
they're
working
on
a
basically
a
legal
consulting
group
that
can
be
shared
and
distributed
for
a
lot
of
smaller
parties,
but
that
typically
is
geared
toward
companies
versus
projects.
So
I'd
say
it's.
This
is
an
area
where
I
I
hope
there
are
more
lawyers
that
are
interested
in
volunteering
their
time
because
they
are
very
expensive,
but
where
possible,
look
for
existing
guidance
and
again
leaning
on
your
your
foundation.
Boards.
A
Yeah
well
yeah,
it's
a
consideration.
So
that's
one
of
the
things
where
I
want
to
ensure
that
anything
that
I
collect
is
also
compliant
with
gdpr,
even
though
it's
technically
not
business
data
or
customer
data.
So
because
of
that
we're
treating
any
data
that
we
have
master
on
open
source
as
user
data,
even
though
they're,
not
technical
users,
but
that
puts
it
in
a
higher
level
of
consideration
and
management
and
also
limits
the
exposure
of
it
so
we're
assuming
a
higher
level
of
responsibility,
even
though
it's
mostly
public
information.
A
Additionally
to
that,
you
can
look
at,
say,
github
does
explicitly
list
how
their
data
collection
practices
available
through
api
should
be
considered
in
the
realm
of
gdpr.
So
I
think
they've
done
a
really
good
job
of
sort
of
outlining
how
how
to
enforce
all
the
specific
elements
and
policy
elements
that
are
required
in
gdpr.
If
you're
pulling
from
that
platform,
not
every
platform
has
done
this.