►
Description
It is critical to measure the right activities and investments to evolve the Open Source Program Office (OSPO) at Comcast. It is also important to reward the right behaviors and culture changes. I will talk about the metrics project at the Comcast OSPO and how it has helped us shape a more effective OSPO.
Slides: https://chaoss.github.io/website/CHAOSScon/2019EU/slides/Value-Metrics-Drive-OSPO-Plans.pdf
A
Jake's
from
an
open-source
program
office
perspective,
and
let
me
go
through
a
couple
of
things:
introducing
myself
I'm
Nydia,
Roth
I
work
at
Comcast.
How
many
of
you
know
who
Comcast
is
quite
a
few?
That's
good,
because
I
was
curious
from
a
European
perspective.
You've
typically
been
a
very
North
American
centric
company
that
provides
media
and
Internet
services
throughout
the
United
States,
but
with
the
integration
of
Sky
TV
in
the
UK,
we've
had
more
of
a
footprint
now
in
Europe
as
well.
A
The
other
parts
of
Comcast
include
things
like
DreamWorks,
NBC,
Universal,
Fandango
and
content
creation
type
of
companies.
So
you
can
imagine
it's
been
a
very
traditional
company
and
but
it's
embraced
software
in
a
big
way
over
the
last
10
15
years,
and
but
the
opening
of
the
Osmo
I'll
talk
more
about
how
we
are
measuring
ourselves.
A
Introducing
myself
again
I'm
a
mother
of
two
daughters,
I'm
one
is
on
her
way
to
medical
school
this
year,
so
I'm
very
proud
of
that.
The
other
one
is
a
human
rights
journalist
in
New,
York,
City,
so
very
different
careers,
but
incredibly
proud
of
those
two
young
women
I.
Consider
myself
more
of
a
pragmatist,
so
I
am
NOT
idealistic
about
any
one
side.
I
tend
to
kind
of
fall
in
the
middle
and
and
see
what
serve
the
purpose
of
the
business
and
what
serves
the
purpose
of
people.
A
I'm,
passionate
about
open
source
and
innovation
I've
been
in
open
source
since
1998
when
I
started,
work
at
Silicon
Graphics
and
worked
with
the
likes
of
Jeremy,
Allison
and
Dave
McCallister
to
try
to
see
how
companies
should
work
with
open
source
they
that
was
the
wave
of
the
early
companies
that
were
adopting
open
source
in
those
days.
I
work
a
lot
in
diversity
and
inclusion,
especially
for
women
in
tech,
because
I
am
a
woman
in
tech
and
I've
been
in
this
business
for
about
for
30
years.
A
I,
understand
that
and
I've
been
working
on
inclusion
across
tech
for
all
people,
women
of
color.
She
her
that's,
that's
how
I
go
with
my
pronouns
and
it's
important
that
I
add
those
pronouns.
Someone
came
up
to
me
at
the
open
source
summit
last
year
and
said:
hey.
We
need
to
normalize
the
use
of
pronouns
because
people
don't
use
them
and
they
think
only
you
know
some
people
use
them.
So
all
of
us
should
really
be
using
pronouns.
A
So
I
thought
I'd
go
through
a
little
bit
about
the
history
of
Comcast
in
open
source
and
really
lead
up
to
how
we
started
the
open
source
program
office
and
why
we
think
open
source
is
such
an
important
part
of
our
journey
as
a
company,
especially
in
serving
our
customers
and
being
an
innovative
technology
company.
And
then
you
know,
I'll
talk
about
a
little
bit
about
how
we
cast
ourselves
as
a
na
SPO.
Every
ausco
is
different
right
it
it.
A
It
serves
the
business
and
each
business
has
different
needs,
so
our
SPO
was
really
custom-made
for
what
Comcast
needs.
Why
measure
at
all
right
as
a
na
SPO
and
then
what
to
measure
and
then
how
we
went
about
doing
metrics
inside
the
company
and
some
takeaways,
so
I'm
gonna
build
it
out
because
it'll
take
a
while
to
kind
of
get
this
all
set
up.
It
was
actually
back
in
early
2000s.
A
So
it
was
around
the
year
2000
that
we
started
moving
from
just
buying
products
to
actually
creating
products
inside
the
company
through
a
lot
of
consumption
of
open-source
I,
think
most
companies
start
out
that
way
starting
to
consume
and
then
around
2010
and
and
yes,
it
took
us
a
little
while
to
start
realizing
that
another
aspect
of
open-source
is
to
give
back.
Both
bug,
fixes
and
also
innovation,
that
we
do
our
customization.
We
do
and
my
boss,
John
Moore
who's.
A
The
chief
Software
Architect
was
one
of
the
first
ones
to
give
back
and
he
started
contributing
to
a
patchy,
HTTP
client
and
then
some
some
milestones
I'll
talk
about.
It
was
around
2013.
We
really
had
a
good
flow
of
contributions
back
so
we
started
the
open
source
advisory
council
and
its
role
was
to
review
and
approve
contributions
back
into
open
source.
One
of
the
big
projects
that
we
contributed
was
a
content
distribution
network.
A
So
all
of
the
set-top
boxes
and
routers
that
sit
in
customer
homes
is
based
on
Linux
and
based
on
Yocto
project,
from
a
build
and
tool
perspective,
and
is
an
open
source
stack,
that's
available
through
Apache
to
dot
or
a
licensing.
So
it
was
around
2017
that
I
joined
Comcast
to
start
the
open
source
program
office.
Initially
it
was
just
me
today.
We
are
about
four
of
us
and
I'll
explain
a
little
more
about
that,
and
we
are
now
doing
almost
a
hundred
and
sixty-four
contributions.
A
Last
year
we
made
many
of
them
are
brand
new
individual
works
that
we
created.
Some
of
them
are
contributions
to
existing
projects,
and
then
we
don't
even
measure
the
amount
of
sandbox
contributions
that
we
do,
which
is
bug
fixes
you
know,
documentation,
etc.
The
so
that's
kind
of
a
little
bit
about
the
history.
When
we
set
up
the
program
office,
I'll
talk
more
about
the
functions
of
the
office.
We
have
about
four
members
in
the
office.
A
It
just
so
happens
we're
all
women
so
for
women
office,
but
we
now
have
a
little
bit
of
growth
happening
this
year,
we'll
be
hiring
two
more
people.
So
that's
about
6,000
people
in
the
office
serving
about
7,000
developers
in
the
company
rough
ratio
of
about
1,100
people
to
write
1,100
developers
to
one
it's
also
a
little
more
complicated,
because
we
also
advising
and
help
our
brothers
and
sisters,
I
suppose
in
NBCUniversal,
as
well
as
DreamWorks
and
Fandango.
A
They
have
their
own
open-source
program
offices,
but
we
kind
of
guide
them,
because
we
are
one
of
the
largest
law
spas
in
in
the
company.
We
have
about
200
repos
on
github.com,
which
is
our
contributions
and
external
projects.
When
we
set
up
the
office,
our
job
was
to
say
our
job
is
to
communicate,
educate
inside
the
company
and
outside
the
company.
About
who
open
Comcast
is
what
we're
doing
an
open
source,
it's
to
guide
teams
in
consumption
and
and
really
to
guide
them
or
to
approve
and
to
accelerate
the
process
of
contribution.
A
We
highly
highly
recommend
and
advocate
that
teams
upstream
contribute
give
back
as
much
as
they
can.
In
fact,
this
is
quite
unusual
because
it's
a
company
that
there
are
also
values,
patterns
and
values.
You
know
building
a
patent
portfolio
right.
So
it's
it's.
It's
a
nice
positive
tension
there
and
then
we,
our
team
in
our
office,
works
on
memberships,
advises
teams
on
opportunities
to
collaborate
with
other
companies
like
at
Kaos
a
project
and
at
open
chain
and
projects
of
such
sort,
and
we
really
feel
via
cultural
ambassadors.
A
We
are
transforming
the
culture
of
the
company,
especially
our
developers,
to
be
more
open
to
be
more
collaborative
to
use
open
source
collaboration.
So
we
do
a
lot
of
inner
source
or
there's
a
lot
of
interest
in
in
a
source
in
the
company,
and
then
compliance
I
was
saying
to
my
brothers
and
sisters
at
the
to
do
group
that
compliance
is
huge.
It's
it's
a
huge
part
of
what
we
do,
because
we
want
to
really
respect
the
licenses
and
we
want
to
make
sure
that
we
comply
with
the
licenses.
A
A
So
why
do
we
need
to
quantify
the
work,
especially
in
in
a
business?
It
becomes
extremely
important
to
quantify
the
impact
of
an
AA
SPO.
It's
often
a
new
function
in
the
company.
People
don't
understand
the
function,
they
don't
understand.
You
know
what
value
we
bring
to
the
company.
Are
we
just
a
cost
center?
Do
we
actually
make
an
impact,
so
the
business
impact
of
Oslo
and
open-source
is
one
of
the
big
elements
that
we
need
to
measure
and
I'll
tell
you
it's
not
easy
and
we
are
not
there
yet
in
terms
of
showing.
A
How
do
we
help
the
company
from
a
business
impact
perspective?
The
second
aspect
would
be:
is
the
Oslo
itself
being
effective
is?
Is
it
delivering
value
to
its
developers,
avi
doing
processes
more
efficiently?
Are
we
automating
things
better
and
then
third
would
be?
How
are
our
projects
which
we
open-source
are
they
healthy?
Are
we
maintaining
them?
Are
we
creating
value
for
people
who
use
our
projects,
and
then
we
also
measure
how
we
engage
with
communities
inside
and
outside
the
company?
A
How
many
you
know
talks,
we
do
how
many
meetings
we
do,
how
many
ambassadors
we
have
how
many
events
we
have
sponsored
those
kinds
of
things.
So
it's
really
around
encouraging
the
culture,
change,
encouraging
business
impact
and
hence
the
staffing
and
budgeting
of
the
office,
making
sure
that
we
add
value
from
an
effectiveness
perspective,
making
sure
that
the
projects
that
we
open
are
healthy.
A
How
much
time
did
we
take
to
help
them
get
their
project
ready,
go
through
security
scans
and
actually
publish
the
project?
And,
as
you
can
imagine,
we
do
well
on
the
bug,
fixes
and
documentation,
because
those
are
approved
almost
instantaneously
online.
But
we
do
really
badly
on
the
open
sourcing
of
whole
projects,
so
we
are
trying
to
this
year
reduce
our
end-to-end
by
10
to
15%,
and
some
of
it
is
out
of
our
control.
A
So
navigating
communities.
We
get
a
lot
of
questions
on
hey
I
want
to
join
this
project.
You
know
what
what
should
I
do?
How
should
I
conduct
myself
in
the
project?
What
what
are
the
rules
and
rules
of
engagement
can
Samar
review
this
membership
agreement?
That
sort
of
thing
and
we
haven't
started
measuring
some
of
the
softer
requests.
We
measure
mainly
the
open
source
advisory
council
requests.
A
Project
effectiveness
is
the
other
area
we
measure
and
we
published
some
metrics
on
our
github
dot
IO
in
terms
of
what
are
our
top
projects
based
on
stars
and
you
know,
downloads
etc.
But
we
don't
publish
nearly
the
number
of
metrics
that
Remi
and
Twitter.
You
know
publish
envious,
I'd
like
to
get
there
from
an
external
publication
perspective
right,
but
we
do
publish
a
lot
of
these
and
then
we
work
with
developers
and
maintainer
x'.
A
So
one
of
the
questions
we
ask
in
the
open
source
advisory
council
is:
have
you
allocated
sufficient
time
to
maintain
this
project?
Do
you
have
your
managers
approval
to
spend
time
to
maintain
this
project
and
we
really
impress
upon
them
that
this
is
not
a
dump
and
run
you've
got
to
maintain
the
project.
A
You've
got
to
answer
questions
you've
got
to
provide
documentation,
you've
got
to
create
a
good
community,
and
so
when
we,
when
we
see
projects,
are
thriving,
we
try
to
make
a
success
story
out
of
it
in
a
case
study
inside
the
company,
so
that
other
teams
can
follow
suit.
We
have
one
project
called
trickster,
which
is
really
doing
a
fantastic
job.
They
speed
up
Prometheus
dashboards.
They
have
a
very
diverse
set
of
contributors
worldwide,
from
lots
of
different
companies
highly
responsive
to
requests.
They
have
a
twitter
account,
they
have
a
slack
channel.
A
The
other
aspect
is.
We
do
indeed
use
a
lot
of
open
source
software,
as
you
can
imagine,
every
single
thing
that
you
can
imagine
we
use
because
we're
a
very
diverse
company
and
and
to
this
point
to
say
about
a
year
or
so
ago
we
gave
a
lot
of
autonomy
to
developers.
We
basically
said
to
them:
you
can
do
whatever
you
need
to
do
to
get
to
market
quickly,
so
we
didn't
enforce
standards
or
we
didn't
try
to
converge
on.
You
know
certain
CI,
CD
pipeline
and
code
or
certain
project.
A
You
know
certain
code
bases
or
Stax,
but
starting
couple
of
years
ago,
the
office
that
I
work
in
which
is
like
the
CTO
office
at
Comcast.
It's
called
a
softness
strategy
and
transformation
group.
We
are
trying
to
do
more
architectural
gills
and
more
working
groups
to
converge
on
common
Stax
because
it
just
gets
out
of
control.
If
you
have
too
many,
you
know
stacks
in
the
company,
so
we
help
with
things
like
running
a
census
in
the
company
to
understand
what
software
we
use.
A
We
try
to
understand
what
top
projects
we
contribute
to,
what
top
communities
we
work
in
and
that's
how
the
Apache
foundation,
for
example,
we
do
a
ton
of
consumption
of
Apache,
so
we
felt
it
was
very
important
to
support
the
Apache
foundation,
both
from
a
monetary
support
from
a
contribution,
support
perspective
and
evangelizing
the
project,
and
then
we
do
support
for
OpenStack.
We
do
a
lot
of
support
for
CN,
CF,
open
chain
and
a
number
of
other
projects
that
we
consume.
A
So
some
these
are
some
of
the
newest
projects
on
Comcast
github
do
and
that's
where
you'll
find
three
things.
You'll
find
all
the
projects
we've
open
sourced
and
then
you'll
find
open
source
jobs
at
Comcast.
You'll
also
find
that
we
do
innovation,
funding,
meaning
we
actually
give
grants
and
funds
to
projects
and
universities
that
need
money
to
maintain
the
projects,
and
so
we
have
done
that
a
number
of
times
and
you
can
actually
apply
for
funding.
A
How
many
minutes
do
I?
Have
oh
gosh,
okay
project
held
project
held
it
also
right
project
health
also
tells
us
which
projects
are
sick,
need
to
be
revived
or
killed
or
archived,
or
you
need
to
kind
of
intervene
and
tell
the
developers
you
you
either.
You
know,
take
care
of
this
or
else
so
we
we
do
a
ton
of
that.
One
of
the
big
areas
that's
become.
A
part
of
our
job
is
to
work
with
talent
acquisition.
A
You
know
progressive
open-source
policies
that
we
have
lab
week.
We
have
other
things
we
do,
which
are
highly
effective
for
developers
to
come
work
in,
and
then
I
mentioned
that
we
also
measure
how
we
engage
with
external
companies
and
communities
through
blogs.
Through
events,
we
attend
through
membership
sponsorships.
This
is
highly
manual
in
terms
of
how
we
capture
these
metrics
I.
Don't
have
an
automated
way
to
capture
these.
How
did
we
create
these
metrics?
We
really
cross
collaborated.
A
We
have
lab
week
three
times
a
week
three
times
a
year
and
we
basically
stood
up
a
project
and
said
we
want
to
do
metrics
for
ospa
in
the
company.
Is
there
anyone
who
wants
to
come
work
with
us
and
help
us
a
bunch
of
developers
joined
us
and
we
kicked
off
our
metrics
project
and
and
the
team
used
things
like
kubernetes
and
fluent
D,
as
well
as
I
stood
it
up
on
OpenStack
and
use
dev
stats,
grow,
fauna
and
Postgres,
and
this
they
used
I.
A
My
takeaways
is
it's
very
hard
to
measure
the
business
impact
and
most
of
our
companies
really
are
business,
driven
as
they
should
be.
The
purpose
of
a
company
is
to
make
money
is
to
grow.
Revenue
is
to
make
profits,
and
it's
hard
for
me,
it's
a
very
long
association
to
show
how
cost
reductions
happened
or
how
we
were
able
to
attract
more
people
into
the
company
or
how
we
allowed.
A
You
know
product
teams
to
get
to
market
faster
or
to
create
more
innovative
features
or,
to
you
know,
do
things
in
a
more
standard
way
and
create
larger
ecosystems.
So
measuring
of
business
impact
is
still
something
that
I'm
struggling
with.
If
you
guys
have
any
ideas
and
how
to
measure
business
impact
from
your
perspective,
I'd
be
very
interested
in
in
learning
them
rewarding
the
right,
behaviors
is
still
very,
very
dicey,
should
be
award
people
for
the
number
of
contributions
they
make.
Should
we
award
them
for
the
rank
they
acquire
in
a
in
a
project.
A
So
if
they
go
from
contributor
to
commit
her
to
maintainer,
should
they
you
know
get
some
I
guess
recognition
in
the
technical
ladder
right
so,
for
instance,
people
get
awarded
for
patents,
they
get
monies,
they
get
to
be
on
the
track
for
fellow
and
distinguished
engineer,
etc.
So
we
are
trying
to
create
a
similar
currency
if
you
will
for
open
source
contributions
and
open
source
knowledge,
and
it's
not
easy,
because
it's
a
softer
thing,
patents
are
very
measurable,
open-source
sometimes
may
not
be,
and
then
measuring
diversity
is
important.
A
It's
extremely
important
to
measure
diversity
of
projects,
not
just
gender,
and
you
know
where
they
come
from,
but
different
companies
different
parts
of
the
world.
So
we
are
trying
to
make
sure
that
that's
a
measure
of
the
health
of
a
project
and
then
qualitative
measurement
is
just
as
important
as
quantitative
in
my
mind,
so
we
measure
some
sentiment
analysis
on
Twitter.
We
also
look
at
coats.
We
look.
We
do
a
survey
a
couple
of
times
a
year
in
terms
of
and
capture
some
of
the
verbal
comments
that
people
make.
A
We
also
have
a
couple
of
recognition
systems
in
the
company.
One
is
called
reflective,
so
we
get
a
ton
of
comments
about
the
open-source
program
office
they're,
also
on
our
slack
channel.
So
we
capture
some
of
that
sentiment
and
and
demonstrate
that
you
know
people
really
like
what
we're
doing
like
working
in
open-source,
and
so
we
kind
of
combine
all
of
that
together.
So
I
know
it
was
not
a
ton
of
actual
metrics
that
we
measure,
but
I
hope
you
got
a
sense
of
our
history.