►
From YouTube: Securing Critical Projects (January 14, 2021)
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
B
B
Let's
wait
actually
one
more
just
well,
we
can
get
started
whatever
abhishek's
here,
I'm
gonna
paste
the
meeting
notes
in
the
chat
and
welcome
anyone.
That's
that's
new.
This
is
our
working
group
talking
about
how
we
secure
critical
projects.
B
Today
we
have
two
people
that
are
going
to
chat
with
us.
I
think
the
first
step
in
securing
critical
projects
is
defining
what
projects
are
critical,
so
we're
going
to
talk
about
that
today.
Avishak
is
going
to
kick
it
off
and
talk
about
the
criticality
score
project
and
talk
about
some
of
the
feedback
on
it
and
and
then
we
can
have
a
discussion
around
that
and
then
jenny
from
harvard's
innovation
lab
is
going
to
follow
up
with
discussion
about.
B
You
know
how
we
can
how
we
can
do
more
here
and
in
getting
that
criticality
list
and
then,
if
there
are
any
other
quick
topics
that
we
have
time
feel
free
to
add
something
quickly
to
the
agenda
or
we
can
table
it
for
our
next
meeting,
so
cool
abhishek.
Do
you
wanna?
Take
it
away
or
introduce
yourself.
C
Both
hello
folks,
I'm
abhishek,
I
lead
the
fuzzing
efforts
in
google
and
recently
started
contributing
to
various
oyster
security
efforts
and
collaborate
very
closely
with
kim
and
dan
here.
So,
let's
start,
let
me
present
one
screen.
C
Can
you
guys
see
my
screen,
yeah
cool,
so
ask
him,
sir,
like?
Why
do
we
really
want
a
criticality
score?
So
let
me
go
over
the
problem
first,
so
you
might
not
know
this,
but
we
have
like
at
least
on
github
itself,
like
around
200
million
repositories
and
if
we
decide
to
make
it
a
purpose
to
improve
the
security
of
them,
all.
That's
not
feasible,
and
most
of
these
projects
are
not
even
relevant
for
the
open
source
community
like
they
can
be
these
fun
toy
projects
or
just
forks
that
nobody
uses.
C
So
the
problems
boils
down
to.
We
have
to
find
the
really
critical
ones
that
are
important
for,
like
the
security
of
the
internet,
that
critical
infrastructure
depend
on,
but
the
problem
is:
how
do
we
find
it
and
right
now
there
isn't
a
way
you
might
can
think
of
like
interesting
projects
like
open,
ssl
kubernetes.
But
what
goes
beyond
that?
So
we
want
to
answer
that
question
and
one
of
the
things
that
comes
to
mind
is
hey.
Why
not
just
look
at
like
the
github
stars?
C
That's
also
there
in
let's
say
gitlab,
but
it's
not
really
a
quantifiable
metric
and
the
reason
being
it's
kind
of
similar
to
like
this.
Facebook
like
button,
like
any
user,
can
click
it,
and
it
might
be
not
the
case
that
most
users
depend
on
this.
So
to
give
you
an
example,
this
is
a
repository
talking
about
the
harms
of
like
working
nine
to
nine
six
days
a
week
in
china,
and
this
repository
has
like
250k
stars.
So
it's
not
really
relevant
for
the
user
security.
C
C
So,
to
give
you
an
example,
if
you
have,
let's
say
a
repositories
with
500
contributors,
is
it
less
important
than
a
repository
with
5
000
contributors?
Maybe
maybe
not
so.
Log
scale
helps
to
shrink
down
that
those
outliers
and
at
a
very
high
level,
it
uses
these
three
parameters,
s
I,
which
is
the
parameter
value
itself
alpha
I,
which
is
the
weight
for
that
parameter
and,
as
I
was
saying,
we
want
to
shrink
down
the
outliers,
so
we
can
have
a
max
threshold.
C
C
Things
like
when
was
this
project
last
updated
and
so
on.
Second
set
of
parameters
are
also
try
to
get
user
activity
on
the
project,
so
things
like,
let's
say
users
are
filing
a
lot
of
issues
in
there
or
commenting
on
the
issue.
C
So
we
started
with
a
pretty
hacky
version
of
this,
which
is
how
do
we
know
the
dependencies
of
a
project
in
a
language
agnostic
manner,
and
this
is
a
pretty
hard
problem,
because
you
can
get
this
information
for
languages
that
have
package
managers,
but
not
really
for
let's
say
cc,
plus
plus,
where
most
of
the
security
vulnerabilities
are.
C
C
Yes,
so
we
so
that's
the
dependency
count
parameter
and
we
have
plans
to
improve
it.
C
So
we
got
quite
a
bit
of
community
feedback
after
the
launch
last
month,
both
positive
and
negative,
and
I
think
this
was
kind
of
expected
because,
as
you
know,
any
ranking
algorithm
can
be
pretty
subjective,
but
we
have
to
start
somewhere-
and
this
is
very
well
said
by
the
first
feedback
from
let's
say
marcel
here,
which
is
if
let's
say
you
have
x
million
dollars,
you
have
to
start
somewhere
and
you
want
to
then
fund
those
projects,
and
so,
let's
all
work
together
towards
that
goal.
C
Some
of
the
negative
feedback
we
got
around
was
people
wanted
to
see
how
we
are
differentiating
between
critical
projects
and
critical
dependencies.
So,
for
example,
there
can
be
projects
like,
let's
say
bash,
which
had
like
little
to
no
activity
in
the
last
few
years
and
are
just
maintained
by
one
one
developer.
So
should
we
be
funding
those
instead
of
these
other
popular
projects?
So
these
are
interesting
discussions
to
do.
C
We
got
quite
a
bit
of
bugs
filed
in
the
repo
as
well,
which
we
started
fixing.
So,
to
give
you
an
example
of
these,
is
there
was
easily
in
confusion
about
like
when
a
repository
was
created.
C
So,
as
you
know,
many
of
the
repositories
got
ported
over
to
github,
so
the
real
value
is
actually
looking
at
the
commits
itself.
So
the
first
commit
was
the
right
value,
so
we
fixed
those
bugs
another
large
set
of
bugs.
We
got
was
around
how
we
did
the
sampling
from
the
initial
set.
C
So
what
we
did
here
was
we
started
with
a
smaller
set
based
on
stars,
like
by
descending
order
of
stars
and
then
picked
like
the
top
200
repos,
and
this
didn't
cover
quite
a
lot
of
the
important
repo.
So
we
have
now
expanded
the
sample
set
to
5000
and
are
planning
to
increase
it
more,
but
this
initially
kind
of
fixed,
most
of
the
bugs
reported,
the
other
set
of
bugs
we
got-
was
around
supporting
at
least
the
gitlab
repo.
So
we
added
support
for
that.
C
However,
there
are
currently
problems
with
the
gitlab
apis
as
well
like
they
are
not
as
powerful
as
the
github
ones.
To
give
you
an
example
on
git
lab
it's,
there
is
no
way
to
know
like
the
top
projects
or
top
popular
projects
on
gitlab
by
stars,
and
we
have
filed
a
bug
on
this
issue.
C
Also
gitlab
provides
or
allows
hosting
on
your
own
custom
domains.
So
this
makes
it
pretty
infeasible
to
query,
because
you
need
different
access.
Tokens
for
non-git
lab
repo
for
non-gitlab.com
repo.
C
The
next
set
of
bugs
are
pretty
simple
to
fix.
We
were
just
missing
to
specify
some
languages
so,
for
example,
typescript
and
coffeescript
are
kind
of
variants
of
javascript,
so
they
are
now
included
in
the
list
and
there
is
a
last
set
of
bugs
which
we
couldn't
fix,
and
we
kind
of
know
about
this
issue,
which
is,
we
cannot
use
any
parameters
that
are
not
applicable
universally
or
their
data
is
not
available
universally.
C
So,
to
give
an
example
like
download
count
for
a
particular
project
are
not
available
for
most
cases,
even
with
the
api.
C
So
the
final
next
steps,
one
of
the
big
things
we
want
to
fix,
is
adding
better
dependency
information
inside
the
criticality
score
and,
as
I
said,
this
is
currently
not
available
for
cc
plus,
so
we
are
thinking
of
or
we
are
looking
for,
your
feedback
on
how
we
can
solve
this.
C
The
next
question
comes
down
to:
where
should
our
funding
be?
Where
should
our
funding
go?
Should
it
be
restricted
to
critical
projects,
or
should
we
be
focusing
on
these
esoteric
dependencies?
That,
like
everyone,
depends
on
and
may
be
just
maintained
by
one
developer?
C
So,
to
give
you
an
example,
lib
jpeg
turbo
is
another
one
that
comes
to
mind.
It's
pretty
popular
project
maintained
by
one
developer,
for
at
least
the
last
10
years.
C
So
there
is
some
work
being
planned
here,
like
I
know,
at
least
folks
at
harvard
are
trying
to
collect
this
usage
data
anonymously
and
if
that
works
out,
that
can
very
well
plug
into
the
criticality
score
and
make
it
really
powerful,
and
we
are
also
looking
for
more
ideas
on
this.
A
I
did
so
there's
prior
work,
so
a
colleague
of
mine,
nuthin
manaya,
was
co-author
on
one
of
these
papers.
I
I
don't
honestly
know
the
details
of
the
work
they
did,
but
they
had.
They
were
doing
similar
work
to
identify
projects
of
interest
in
github
right.
So,
if
you're
doing
bulk
analytics
and
software
repositories,
you
want
to
select
out
all
the
noise
right,
and
so
I
there's
this
tool
called
reaper
that
they
developed.
To
do
that,
I
linked
a
presentation
to
it.
I
have
a
pre-draft
copy
of
the
paper.
A
I
think,
if
you
go
to
peer
j,
if
you
just
google
or
duckduckgo,
I
used.
If
you
search
for
that,
you
know
you
can
find
find
the
copy
of
the
paper,
and
I
believe
that
they
have
similar
metrics
to
the
ones
that
you
talked
about
in
there
and
they
were
developing
a
similar
system
for
pulling
out
important
projects,
or
at
least
I
believe
they
called
them
engineered
projects
right.
B
A
A
My
the
thought
I
had
here
was
just.
I
don't
know
that
this
is
the
question
I'd
ask
like
like,
like
whether
you
support
critical
projects
or
critical
dependencies,
I'd
I'd
frame
it
more
from
the
standpoint
of
what
does
each
project
need,
so
the
metrics
might
indicate
that
there
is
a
need
for
stronger
unit
testing,
for
example
right,
and
so
maybe
you
know
you
want
to
tailor
the
types
of
support
you
give
based
on
what
the
metrics
are
telling
you.
A
We
you
know
need
to
be
clear
about
what
goals
it
is
that
we're
trying
to
do
to
improve
security
right
and
security
is
a
large,
multi-faceted
thing.
So
it
might
look
like
testing.
It
might
look
like
code
reviews.
I
I
don't
know
what
it
looks
like
you
know.
Maybe
it
helps
with
management
structures
and
teaching
people
how
to
or
working
backlogs
with
bugs.
I
you
know,
there's
a
lot
of
different
things
that
could
be
wrong
from
a
software
standpoint.
C
C
So
we
operate
this
service
called
oss
first,
which
currently
fuzzes
like
top
400
critical
projects,
I
would
say,
or
some
good
critical
projects
like
even
open,
ssl,
kubernetes
and
stuff,
and
we
want
to
really
increase
the
scope
of
that
and
we
couldn't
come
up
with
this
list
by
just
thinking
about
the
projects
anymore
like
so
that's
where
this
criticality
score
work
can
help
and,
as
you
were
saying
yeah,
we
need
to
have
those
metrics
like
code.
C
Ql
comes
to
mind,
like
all
those
reports
should
have
at
least
code
ql
checks
running
there
or
better
unit
testing.
So
yeah,
please
give
us
ideas
on
what
could
be
those
set
of
checks
and
what
should
be
the
priority
of
things.
A
Yeah,
I
don't
have
good
evidence
for
what
should
be
the
priority.
I
think
you
know
you
we
could
look
to
be
sim
or
other
things
like
that
for
at
least
super
sets
of
things
to
cherry
pick
from
right
and
use
our
professional
judgment.
I
don't
think
there's
a
lot
of
evidence
that
says
static
analysis.
Warnings
are,
you
know,
yield
86
percent,
better
findings
than
code
manual
code
review
or
some
other
kind
of
thing
like
that.
But
I
think
that
you
know
we
could
get
close
with
just
with
a
manual
list.
Basically.
C
Yes,
we
have
another
question
from
derek
derek:
you
want
to
ask
it
or
I
can
just
read
it
for
you.
E
Sure
my
immediate
concern
is
that
the
way
this
algorithm
works,
it
may
weight
older
projects
that
are
likely
to
be
replaced
by
newer
for
lack
of
a
better
word
better
technology
down
the
road.
There's.
Definitely
some
disruptive
things
in
the
pipes.
Now,
where
you
know,
projects
like
open,
ssl
openvpn
are
still
industry
standards,
but
they
are
rapidly
being
replaced
by
alternatives.
C
Yes,
so
I
would
say
that
created
sense
parameter
was
just
one
of
the
things
that
contribute
to
the
weight.
So
if
there
are
other
parameters
that
let's
say,
user
activity
is
much
more,
those
are
also
counted
in
the
weight.
So
that
was
the
point
of
the
scoring
metrics,
which
is
not
just
have
one
metric
indicate.
C
F
Yes,
I
am
mentored
or
we
can
open
source
cardboard
and
it's
translated
to
about
60
languages,
the
request
now
and
then
four
new
languages
and
it's
used
in
most
countries
of
the
world.
F
The
snap
version
is
about
at
eight
thousand
servers
worldwide
and-
and
I
don't
know,
stats
for
other
platforms,
but
there
are
very
many
platforms
and
we
can
also
have
a
lot
of
dependencies
for
different
kind
of
kanban
features
and
it's
quite
full
featured
and
the
contributors
are
the
full
request
daily,
and
I
do
also
release
recurrent
soon
about
once
a
day
and
there's
new
features
and
fixes.
F
C
Yes,
sorry,
I
didn't
get
the
question
correctly,
like
it
looked
like
definitely
a
very
popular
project.
But
what
was
your
question
again
like.
F
Yes
about
in
this
checking
for
security,
vulnerable
abilities
and
getting
it
more
up-to-date
and
so
on.
With
this
security.
C
Project
does
anyone
else
understand
the
question
correctly?
Sorry,
I'm
not
getting
your
question
right.
C
B
B
Yeah
so
there's
one
thing
that
I
that
you
know
that's
related
to
like
this
working
group
and
this
stuff,
open
ssf,
is
doing.
That
was
an
interesting
use
case
that
I
saw
the
envoy
project,
which
is
a
large
project.
They
started
using.
B
Our
scorecards
project,
which
is
which
is
part
of
openssf
now
and
and
the
scorecard
is,
is
doing
a
bunch
of
security
checks
and
the
you
know
each
check
either
gets
a
pass
fail
value,
and
so
what
the
envoy
project
has
done
is
created
a
new
dependency
policy
for
their
project
in
general
and
they're
using
the
scorecards
as
part
of
it.
B
So
you
know
if
anyone
tries
to
introduce
a
new
dependency
they're
pro,
I'm
not
exactly
sure
what
they're
doing,
but
probably
flagging
it
in
some
way
to
take
like
a
further
look
at
what
that
project
is,
and
then
the
other.
The
flip
side
of
that
is
it's
it's
forcing
them
to
to
look
at
those
dependencies
and
see
if
they
can
get
the
score
improved.
B
So
there's
a
lot
of
good
conversation,
that's
happening
like
hey,
so
and
so
can
you
you
know
enable
or
do
your
code
review
processes
or
something
set
up.
I
was
trying
I
wrote
a
quick
blog
post
on
it
and
I
was
trying
to
dig
it
up,
but
I
can't
find
the
link
but
I'll
paste
it
I'll
paste
it
in
the
chat
here.
B
But
I
think
you
know,
one
thing
we
did
earlier
in
this
working
group,
too,
is
just
start
talking
through
like
different
ways
that
we
could
help,
because,
like
chris
was
saying
it's
not
thank
you
chris
was
saying
not
just
throwing
money
at
a
project
is
not
exactly
gonna
solve
the
problem,
so
we
brainstorm
like
a
bunch
of
ideas
for
how
we
could
actually
help.
You
know
what
projects
actually
need.
B
So
I
think
it'd
be
interesting
if
you
sort
of,
if
you
were
able
to
write
up
like
a
quick
summary
of
like
how
you
know,
you
think
this
group
could
help
the
wekin.
Is
that
how
you
pronounce
it?
What
can
project
could
be
super
interesting
and
then
we
could
all
sort
of
look
at
that
and
come
up
with
ideas
and
brainstorm,
and
it
would
be
a
good
use
case
for
us
in
general,
if
you're
willing
to
willing
to
do
that,
and
you
could
go
from
there.
F
B
B
I
think
so
and
then
to
oh
sorry:
jenny
were
you
gonna
us,
you're,
muted,.
B
Dan's
been
going
on
a
dependency
hunt
which
has
been
pretty
entertaining
to
read
as
well
trying
to
trying
to
track
back
everything
that
happens
when
you
do
a
a
helm
chart
install
for
a
repo
he's,
got
a
blog
post,
that
you
need
a
cup
of
coffee
and
maybe
a
snack
to
get
all
the
way
through
it.
But
it's
a
it's
a
good
read.
I
don't
know
if
dan
wants
to
find
some
blog
post
in
the
in
the
chat,
but
if
anyone's
interested
should
check
that
one
out,
it's.
A
Drop
in
some
links,
while
we're
waiting,
also
newton,
just
joined,
so
he
was
one
of
the
authors
on
that
github
screening
curation
task,
so
abhishek
he's
he's
in
my
top
right.
You
can't
point
to
anybody
online.
B
The
work
that
you're
doing
I
haven't
looked
at
it
yet,
but
it
looks
like
there
might
be
some
interesting
things
that
we
can
collaborate
on
as
we
talk
about
criticality
of
open
source
projects.
So
we
can
follow
up
in
another
meeting
or
or
take
a
look
and
see
what
we
can
do
together
be
cool.
B
G
G
G
Yeah,
I
guess,
while
we
wait,
I
was
going
to
give
a
really
quick
update
on
the
first
phase
of
osif's
work
with
reviewing
the
linux
kernel.
So
I
mentioned
that
in
presentations.
G
I
did
to
the
group
back
in,
I
believe
in
november
or
october,
maybe
even
now,
but
yeah,
so
that
is
just
about
wrapped
up
and
we're
going
to
be
making
a
announcement
and
publishing
the
actual
report.
So
it's
and
that's
going
to
be
very
soon.
So
as
soon
as
that's
ready,
I
think
I'll
send
it
out
to
our
workgroup
email
just
so
everyone
can
get
that
directly
and
if
any
questions
come
up,
we're
anticipating
it
to
generate
a
lot
of
discussion.
G
So
if
any
questions
come
up
or
thoughts,
you
know,
please
feel
free
to
contact
myself
or
derrick
directly,
but
yeah
we're
really
excited
about
releasing
those
results
with
y'all
and
kind
of
show.
Some
of
the
work
we've
been
doing
to
secure
something
critical
like
the
linux
kernel,
so
hopefully
more
on
that
really
soon
cool.
B
Yeah
it'd
be
awesome
if
you
wanted
to
give
like
a
give
a
little
deep
dive
on
that.
One
of
these
meetings
too.
D
Can
you
hear
me
now?
Yes,
oh
thank
you
guys
for
your
patience,
for
whatever
reason
bouncing
between
zoom
and
google
meets
has
just
proven
beyond
my
capabilities
these
days
anyway,
I
apologize
if
one
of
the
things
I
wanted
to
say,
and
I'm
not
sure
if
it
was
said,
while
I
was
trying
to
you,
know
fix
the
situation
was
at
least
for
the
weekend
project.
There
was
there's
the
chaos
metrics
is
a
is
a
source
where
people
can
kind
of
do
a
quick.
D
You
know
check
of
the
health
of
their
own
their
own
project
and
making
sure
that
that,
on
you
know
several
different
metrics
and
I'm
going
to
go
into
this
in
a
little
bit.
But
that
was
just
something
I
wanted
to
promote.
It's
something
that
the
has
come.
D
I've
learned
about
through
my
work
with
the
linux
foundation
has
helped
us
when
looking
into
some
of
the
health
metrics
of
different
projects,
and
it
can
also
be
a
good
way
to
kind
of
have
a
starting
point
for
a
self-assessment
of
a
project
when
we're
creating
that
kind
of
that
use
case
to
see.
Okay,
here's
where
we
are
in
different
metrics.
It
looks
into
the
elephant
factor
or
the
truck
factor,
I
guess
and
and
and
getting
a
better
sense
of
where,
where
help
might
be,
might
be
best
targeted.
D
I
wanted
to
thank
abhishek
for
really
getting
the
conversation
started
on
something
that
we
wanted
to
bring
together.
So
I'm
going
to
go
ahead
and
start
sharing
a
document.
It's
unfortunately
it's
not
as
as
interesting
as
the
slides,
but
it
was
kind
of
the
best
way
that
I
could
bring
bring
together
a
lot
of
the
ideas
that
we,
the
lynx
foundation.
D
You
know
kim
and
dan
and
then
us
here
at
the
laboratory
for
innovation
science
at
harvard,
but
we've
come
together
to
try
to
kind
of
circle
strife
for
those
who
don't
mind
the
military
industry
imagery
circumstances,
problem
of
criticality
and
figuring
out.
You
know
how
best
to
target
our
our
our
efforts.
So
let
me
go
ahead
and
present
now.
Hopefully
I
won't
break
the
internet
again
and
then
the
screen.
D
A
D
No,
I'm
I'm
on
windows
and,
for
whatever
reason
it's
yeah
and
today
is
just
not
my
my
technological
day,
all
right.
So
this
was
the.
This
is
the
link
that
was
shared
in
the
agenda
for
for
everybody
for
for
today's
meeting
and
it's
taking
a
look
at
and
kim's
going
to
be
so
kind
of
to
open
this
up.
D
For
me,
taking
a
look
at
how
we
are
going
to
expand
the
efforts
that
linux
foundation
and
and
the
laboratory
for
innovation
science
at
harvard
that
we've
we've
been
working
on
over
the
past
a
couple
of
years
and
are
now
shifting
over
into
the
open
ssf
which
brought
us
to
you
guys
and
it's
looking
at
how
we
can
answer
some
of
the
questions
that
that
abishek
brought
up
which
is
there
are,
you
know,
hundreds
of
thousands
of
different
projects.
How
do
we?
D
We
can't
use
stars
it's
hard
to
it's
hard
to
really
see
from
downloads
because
you
might
download
it
but
not
use
it,
and
so
our
first
effort
had
been
really
going
through
and
analyzing
source
code
analysis
or
software
composition,
analysis,
data
that
looked
at
language
level,
libraries
and
and
which
ones
were
the
most
commonly
found,
either
as
as
either
directly
found
or
as
a
as
a
dependency,
and
so
that's
good.
D
But
that
really
only
gives
us
one
segment
of
one
segment
of
of
of
the
whole
picture
right
and
another
piece
that
we
wanted
to
look
at.
If
you
can
see
the
a
little
bit
of
a
diagram
below
so
right
now
we
we're
only
looking
at
libraries,
but
you
know
downstream
of
that.
D
You
have
applications
upstream
of
that
you
have
to
look
at
the
firmware
and
the
operating
systems,
and
even
even
before
any
of
that,
looking
at
the
build
tools
and
the
compilers
that
are
used
and
really
seeing
which
which
of
any
of
these
five
segments
are
really
going
to
be
what
we
should
target
well
at
the
very
least,
we
need
to
know
more
about
what's
being
used
in
these
kind
of
segments
of
of
the
the
ecosystem.
D
What
is
most
commonly
found
out
there
within
the
private
within
the
private
sector,
so
there's
some
of
the
data
that
we're
not
always
able
to
see
beyond
that
in
order
to
be
able
to
see
some
of
some
of
the
the
the
more
foundational
segments
of
the
ecosystem
like
the
compilers
and
the
development
tools,
as
well
as
the
the
kind
of
the
higher
layer
of
which
end
user
applications
are
most
heavily
used.
D
We
want
to
get
a
better
sense
of
what
people
are
using
and
that's
not
something
you
can
always
get
immediately
out
of
automated
data.
There
was
you
know,
one
suggestion
had
been
for
willing
companies
to
offer
up
code
that
can
do
a
scan,
a
scan
of
their
their
networks
and
their
environments
to
see
what's
being
used,
but,
as
I'm
sure
anybody
that's
in
currently
in
a
private
company
or
has
worked
with
private
companies,
not
always
something
they're
excited
or
willing
to
do
introducing
outside
code.
D
That,
supposedly,
is
supposed
to
do
one
thing
again:
that's
something
that
I
think
people
are
going
to
be
a
little
more
reticent
to
do
so.
The
best
way
we've
we've
found
to
to
try
to
get
more
information
about
what
is
at
least
the
most
prevalent
software.
That's
out
there
is
to
do
this
via
a
self-reporting
survey
from
different
organizations.
D
So
in
our
objectives
we
want
to
in
our
objectives
we're
trying
to
also
figure
out
this
criticality,
this
criticality
factor,
and
so
for
us,
a
lot
of
it
can
come
down
to
one
of
three
types
of
criticality
right,
there's
the
function
of
the
software.
If
this
is
software
that
helps
run
a
nuclear
power
plant
or
another
part
of
critical
infrastructure,
the
function
of
that
particular
project
is
going
to
be.
What
makes
it
critical
another
way
it
can
be.
Thought
of
is
just
having
a
critical
mass
of
the
software.
D
Is
this
something
that's
so
ubiquitous
that
should
something
happen
to
it
should
should
it
suddenly
disappear?
You
know
what
is
that?
What
impact
is
that
going
to
have,
and
sometimes
it's
not
necessarily
this?
I
think
this
criticality
is
more
a
criticality
of
of
avoiding
inconvenience.
I
know
that
it
sounds
a
little
flippant.
I
don't
mean
it
that
way,
but
in
the
sense
that
you
know
it
might
not
have
dire
dire
consequences,
but
at
the
same
time
the
disentanglement
or
exit
costs
of
of
you
know.
D
If
this
project
were
to
disappear
are
quite
high
and
then
the
third
way
we've
thought
about
it
is
really
looking
at
a
critical
reliance
upon
the
software.
So
you
know
what
how
many
other
pieces
of
software
call
upon
this
this
component
or
this
application,
or
you
know
what?
D
How
is
it
interconnected
with
the
rest
of
with
the
rest
of
of
the
ecosystem
and
so
looking
looking
from
the
critical
reliance
of
the
software
kind
of
more
at
the
high
level
we're
seeing
you
know
if
there's
a
one
of
the
severity
of
consequences,
if
it's
exploited,
what
are
the
severity
of
consequences
if
there's
malicious
code
inserted
and
then
what
are
the
severe
consequences?
If,
if
for
whatever
reason
it
were
to
you
know
it's
not
able
to
sustain
itself
either,
so
there
were
a
couple
of
instances.
D
Forgive
me
if
I'm
wrong
on
that
one,
but
one
where
the
primary
maintainer
was
facing
possible
jail
time
and,
as
a
result,
there
was
a
a
bit
of
a
crisis
of
succession
and
people
not
really
sure
about
how
the
governance
would
continue.
There's
things
like
that,
there's
there's
issues
where
projects
have
been
abandoned,
but
they're
still
relied
upon
in
these
dependency
networks,
and
so
those
are
kind
of
the
three
main
buckets
of
how
we're
looking
at
criticality
and
how
how
we're
trying
to
figure
out
okay?
How
does
the
usage
of
each
of
these?
D
How
do
the
usage
of
each
of
these
projects?
How?
How
do
they
kind
of
which
of
the
buckets
of
criticality,
do
they
fit
into?
D
And
then
once
we
figure
out
how,
once
we
figure
out
how
to
identify
and
then
weigh
and
then
identify
and
then
and
then
better
understand
how
critical
each
of
these
projects
that
we're
seeing
appear
in
usage
data
once
we
figure
that
out,
we
need
to
figure
out
how
to
weigh
that
across
across
others.
So
if
it's
something
that's,
if
it's
a,
if
it's
a
project,
that's
heavily
used
in
one
sector,
then
that
could
be
that
could
be
maybe
about
this
level
of
criticality.
D
But
if
we're
looking
at
that
same
a
different
project
and
it's
used
across
three
or
four
different
sectors
of
the
economy,
that's
something
that
we
would
maybe
want
to
end
up
weighing
more
heavily
and
again.
These
are
these
are
not.
I
appreciate
that
average
tech
was
able
to
to
to
try
to
quantify
some
of
these
very
difficult
weight
weights
within
his
algorithm,
but
I
just
this
is
for
our
from
our
perspective.
This
is
kind
of
our
high
level
view.
D
Ten
thousand
foot
view
that
we
want
to
bring
to
you
guys
and
see
if
there
are
either
existing
tools
or
ways
that
we
together
can
can
take
into
account
these
different
aspects
of
criticality
and
weighting
criticality
and
and
better
understand
it,
as
I
had
mentioned
before.
D
If
you
were
looking
under
the
reliance
criticality
there's
the
chaos
metrics
that
I
linked
to
some
of
them
and
they
look
at
at
least
a
few
ways
to
assess
business
risk
or
the
quality
of
code
or
different
security
issues
that
might
come
up,
and
so
that's
one
one
tool,
that's
kind
of
already
in
our
tool,
but
that
we're
hoping
to
build
off
of
so.
D
How
are
we
hoping
to
conduct
a
survey?
One
of
the
things
that
we
thought
might
be
the
best
way
is
to
is
to
start
with
a
survey
of
public
and
private
organizations
within
the
within
the
critical.
D
What
has
been
identified
as
the
the
critical
infrastructure
sectors
by
the
us
government,
and
so
within
that
we
would
you
mean?
Yes,
sorry,.
A
D
Yes,
so
the
so
yes,
so
we
identified
the
16
critical
sectors
from
dhs
and
we're
trying
to
match
that,
with
with
other,
with
other
pools
of
contacts
that
we
might
be
able
to
to
kind
of
make
overlap,
but
I'll
get
into
that
a
little
bit
a
little
bit
down
the
road.
D
D
What's
your
com,
you
know
what
open
software
is
on
your
custom,
your
company's
allow
list
or
deny
list
to
give
it
give
us
a
better
sense
of
what
people
are,
at
least
in
theory,
hoping
to
what
what
they're,
trusting
or
depending
upon
more
or
what
their
go-to's
are,
as
well
as
what
they've
already
kind
of
shunned
aside
or
said.
You
know
we're
not
going
to
be
using
that.
But
beyond
that,
we
also
want
to
know
what
does
the
reality
on
the
ground
actually
look
like.
D
D
You
know
what
is
actually
happening
on
the
ground,
so
just
because
oops,
just
because
you
know
something
is
on
the
allowance,
isn't
doesn't
mean
that
it's
necessarily
the
first
go-to
for
most
people
or
just
because
it's
on
the
deny
list,
if
there's
possibilities
that
I
could
be
still
sneaking
in
understanding
what
each
organization's
most
important
products
and
what
is
contained
within
those
products,
I
think,
is
another
way
that
will
help
people
to
to
kind
of
dig
a
little
deeper
into
what
open
source
they
are
actually
using.
D
Even
if
it's
not
not
something
that
would
come
to
mind
immediately.
We
want
to
how
these
principal
questions
is
really
trying
to
get
the
contacts
within
each
of
these
organizations.
D
Thinking
about
what
open
source
they're
using
in
a
lot
of
different
ways
and
from
a
lot
of
different
angles,
instead
of
just
saying
hey,
what
open
source
do
you
use
because,
as
I'm
sure
we
all
know,
you
know
it's
not
always
going
to
immediately
come
to
mind,
you
got
to
kind
of
take
it
back
layer
by
layer.
Another
piece
that
we
wanted
to
look
at,
but
we
thought
wouldn't
necessarily
be
as
as
important
to
to
really
go
into
as
much
detail
was
compilers.
D
The
reason
we're
not
as
worried
about
this
is
just
it's
a
smaller
pool
of
what's
out
there
and
we
kind
of
already
know
some
of
the
big
actors
in
this
particular
segment
of
the
you
know
of
the
ecosys
of
the
ecosystem
and
then
also
looking
at
the
build
development
tools
and
what
they're
planning
to
use
in
the
future,
and
most
of
these
principal
questions
would
be
directed
towards
an
organization's
either
cto
or
pcert
team.
D
But
additionally,
we
thought
there
could
be
other
data
requests
that
different
key
respondent
groups
could
provide
a
little
more
information
about,
and
so
we
would
look
into
asking
development
teams
to
get
a
better
sense
of
the
package
manager.
D
You
know
slice
of
the
pie
looking
into
distros
and
distro
release
managers
asking
them
more
specifically
about
about
that
level
as
well,
and
then
also
looking
specifically
and
asking
more
more
targeted
questions
of
cloud
service
providers
or
container
creators,
and
then
obviously
this
is,
you
know,
to
survey
private
organizations
and
even
public
ones
as
well.
We
kind
of
have
the
same
issue
that
abhishek
had,
which
is
there
are
so
many.
Where
do
you
even
begin?
D
And
so
this
is
where
we
thought
what
we,
what
we
could
do
was
using
the
cia
sa's
definition
of
which
were
the
critical
infrastructures
we
kind
of
marry
those
up
with
some
of
the
the
linux
foundation's
initiatives.
D
So
they
have
a
lot
of
industry,
specific
initiatives
that
we
can
pull
from
and
and
kind
of
use
that
as
a
starting
point
within
each
of
these
very
critical,
very
critical
sectors
I
haven't
gotten
through
and-
and
you
know,
married
up
each
of
these
major
16
major
sectors
with
an
lf
initiative,
but
there
are
quite
a
few
out
there
and
I'm
going
to
go
ahead
and
link
those
as
I
as
I
can
sorry
so
kim.
D
This
is
down
on
the
third
page,
yeah
perfect,
thank
you
and
thank
you
again
for
driving.
I
appreciate
it
and
then
so,
and
also
you
know
if,
if
anyone
in
this
within
this
group
knows
oh
okay,
well,
you
know
my
group
is
already
currently
working
with.
D
I
don't
know
if
anybody's
you
know
really
big
in
the
dams
sector,
that's
one
that
I
think
we're
going
to
end
up
having
you
know
holes
in,
but
or
even
if
there
are
other
kind
of
working
groups
that
are
already
out
there,
that
are
combining
industry
as
well
as
people
kind
of
on
the
research
side
in
any
of
these
particular
sectors.
D
I
welcome
people
to
add,
add
it
in
and
and
link
it
so
it'll
be
a
great
place
to
kind
of
have
a
starting
point
to
start
looking
at
some
of
the
leaders
in
each
of
these
sectors
and
then,
hopefully,
from
there
we'd,
be
able
to
create
a
good
distribution
list
to
at
least
get
to
at
least
get
kind
of
the
heavy
hitters
in
each
of
these
sectors,
and
and
that's
going
to
be
a
good
a
good.
You
know
jumping
off
point.
D
A
I
put
a
link
there.
There
are
organized
groups,
they
predominantly
do
threat
sharing,
but
okay
they're
organized
by
sector.
So
you
have,
you
know
the
big.
A
Finance
dib
defense
and
I
think
energy
is
pretty
big,
but
that
there's
a
list
there
and
those
have
member
organizations
and
they
have
they.
What
they
have
is
the
distribution
lists
that
you
need.
D
Great
great,
thank
you.
Thank
you
very
much.
Yes,
I
will
include
that
as
well
and
and
yes,
as
I
see,
someone
named
anonymous,
bat
is
going
in
and
adding
additional
pieces
in
our
energy
sector.
Thank
you
anonymous
bath
in
my
google
doc,
then
the
next
piece
that
we
want
to
look
at
is
this
is
obviously
going
to
be
a
big
undertaking,
and
it's
something
that
we
wanted
to.
You
know
draw
upon
the
broad
network
of
contacts
that
open,
sfss,
open.
Ssf
has.
D
Obviously
you
know.
We
have
a
lot
of
great
a
lot
of
deep
knowledge
here
within
this
working
group,
and
so
we
wanted
to
see
you
know
as
we
pull
this
together.
D
D
If
you
know,
if
there
are
people
who
are
already,
you
know
who
already
have
an
in
with
particular
sectors
or
companies,
that's
going
to
be
great,
getting
people's
feedback
on
analyzing
or
sorry
getting
people's
feedback
on
the
design
and
creation
of
the
survey
prior
to
it,
going
out
making
sure
we're
asking
the
right
questions
and
really
trying
to
cover
all
our
bases.
D
But
beyond
that
as
well.
There
was
a
thought
that
there
there
could
be
an
option
to,
as
as
companies
respond
to
add
in
what
they've
already
listed
as
options
for
people
to
select
within
a
you
know
as
they're
answering
the
survey.
D
So
if
I
came
in-
and
I
said,
I'm
using
project
a
b
and
c
and
then
amir
comes
in
and
he
says
I'm
using
projects
a
b
and
d
and
then
kim
comes
in
and
says:
oh
I'm
using
you
know,
j,
a
and
c
that
the
fourth
person
that
would
come
in
would
be
able
to
see
all
of
the
different
options
that
have
been
added
and
then
also
add
their
own,
so
that
there's
kind
of
a
real
time
updating
of
potential
options
which
can
help
trigger
people.
D
D
We
want
to
continue
to
do
that,
but
then
we
want
to
use
this
survey
as
a
a
complementary
project
that
would
better
inform,
like
I
said,
all
of
the
the
different
segments
of
this
the
ecosystem
and
get
better
usage
on
on,
what's
being
used
in
open
source
across
these
different
segments
from
compiler
to
end
user
application.
D
Well,
I
hope
that
I
mean.
I
know
that
is
the
10
000
foot
view.
I
hope
that's
not
too
dizzying
for
people
here.
I
know
I've
tried
to
cover
a
lot
of
ground
like
fairly
quickly,
but
this
is
something
that
we're
we're
hoping
to
delve
into
more
in
the
coming
in
the
coming
weeks
and
are
definitely
open
to
anyone
reaching
out.
D
Let
me
go
ahead
and
put
my
email
directly
in
the
chat
and
that
way,
if
anybody
has
any
questions
also,
I
want
to
double
check
all
right.
So
we
do
have
these
and
thank
you
again
for
directing
me
to
the
ifca.
B
This
is
awesome
jenny.
We
only
have
a
couple
minutes
left.
Is
there
anything
immediately
like
that?
You
would
want
this
group
to
do
just
review
this
doc,
maybe
share
it
or.
D
D
Do
you
think
there
are
tools
out
there
that
could
that
already
exist,
that
might
that
might
answer
some
of
the
outstanding
questions
we
still
have
so
really
just
take
a
look
through
this
doc.
Add
to
it
any
good
resources
that
you
can
see
if
there
are
particular
tasks
within
the
division
of
labor
section
that
you
think,
oh,
I
would
be
really
stoked
about
trying
to
figure
out
how
to
target
the
distribution
of
this
survey,
or
I
know
ways
to
design
a
survey
where
it's
really
helpful
to
incr.
D
You
know
where
it
already
knows
how
to
incorporate
some
of
the
answers.
Other
respondents
have
given
making
it
easier
for
people
to
to
just
start
like
dragging
and
dropping
what
you
know
what
what
they're
using,
if
it's
already
been
given
as
an
answer
by
someone
else
but
yeah
those
are.
Those
are
the
two
big
asses
help
us
make
sure
that
we're
we're
catching
what
we
need
to
and
we're
really
understanding
the
problem
correctly,
offering
up
any
any
ways
to
to
identify
the
big
players
in
each
of
these
16
sectors.
D
And
then,
if
there
are
any
parts
of
this
project
that
that
you,
your
you
or
your
company
or
other
people,
that
you
know
either
in
academia
or
elsewhere,
would
be
really
excited
about
or
could
could
provide
a
little
bit
more
insight
into
yeah?
That
would
be
great.
Those
are
my
that's
my
big
three.
I
think
it's
still
a
lot
to
ask,
so
I
won't
ask
too
much.
B
Yeah
how
how
widely
do
you
want
this
circulated
like
if
it
was
in
the
working
group?
I
wonder
if
we
could
share
it
with
the
attack
of
the
foundation
or,
if
you
have.
D
Me
get
back
to
you
on
that.
Definitely
within
this
working
group
go
nuts,
I
would
say
if
we,
if
it's
shared
with
others,
that
it's
that
it's
known,
that
this
is
still
very
much
a
rough
draft
and
not
anything
that's
to
be
published
publicly.
That
would
be
fine.
B
D
For
sure
and
honestly,
if
you,
if
you're
not
sure,
go
shoot
me
an
email
and
we'll
we'll
we'll
make
a
decision
together,
sound
good.
B
D
Yeah
excellent
well,
thank
you,
everybody!
So
much.
I
really
appreciate
it
and
forgive
me
I
do
have
to
drop
a
little
early.
I
gotta
get
prepped
and
hopefully
avoid
all
of
the
technical
difficulties
I
had
here
for
my
next
call,
but
thank
you
again
for
your
time
and
your
help
yeah.
Looking
forward
to
this
thanks,
everybody
much
appreciated
cool.
B
And
thank
you
abhishek
for
presenting
we're
about
out
of
time,
so
we
don't
have
anything
on
the
agenda
for
next
time.
If
there
are
topics
you
want
to
discuss
or
if
you
want
to
dive
into
some
of
that
other
research
that
microsoft
has
done,
that
would
be
awesome
but
feel
free
to
add
it
to
the
agenda
and
have
a
great
day
everybody
good,
seeing
you
all.