►
C
Way
over
there,
I
don't
know
why.
I
have
a
quick
question:
are
you
the
the
only
one
working
on
criticality
score
or
you've
got
some
folks
there
in
sydney
with
you.
D
So
so
the
answer
is
yes,
but
to
both
of
those.
So
at
the
moment,
I'm
currently
the
only
person,
but
we
had
someone
join
this
week
last
last
week,
he'll
also
be
contributing
to
the
package
analysis
project
in
criticality
school,
so
cool,
yes,
not
for
much
longer.
All
right,
that's
good!
That's
good!.
D
In
like
part
of
the
the
work
that
I'm
trying
to
do
is
to
make
sure
that
it
facilitates
other
people's
use
of
it.
So
as
well
as
productionizing,
it
there's
an
interest
in
making
sure
the
documentation
is
like
easy
makes
it
easy
to
use
and
clear
how
you
can
actually
run
it
yourself.
So
yeah.
A
A
All
right,
well
we're
a
few
minutes
after
so
we'll
go
ahead
and
get
started,
looks
like
we
have
more
more
people
joining.
That's
great
welcome
to
the
securing
critical
projects,
biweekly
working
group
meeting
in
our
shiny
new
apac,
friendly
time
that
we'll
be
alternating
on
from
here
until
until
daylight,
savings
which
isn't
the
same
everywhere
so
we'll
try
to
figure
out
how
how
we
do
the
transition
from
from
this
time
to
the
new
time
after
daylight.
Savings
switches,
that'll
be
fun,
but
yeah.
A
A
And
then
we'll
move
on
to
new
faces
is
anybody
here,
first
time
joining
or
hasn't
been
in
a
while
and
like
to
introduce
yourself.
B
It's
my
first
time.
Oh,
I
guess
I
should
hi.
I
forgot
the
camera
it's
my
first
time
I
am
katherine.
I'm
an
open
source
evangelist
at
intel.
I've
been
dropping
into
a
few
of
these
meetings,
just
to
kind
of
get
an
idea
of
what
the
different
working
groups
are
working
on
yeah.
So
I'm
here
and
I
have
some
colleagues
that
I
think
attend
these
and
I'm
interested
in
what's
going
on
and
then
at
some
point,
I'd
like
to
figure
out
how
I
might
be
able
to
contribute.
A
Excellent
welcome
yeah
free
to
also
take
a
look
at
our.
You
know
the
readme
page
on
our
github
repo
covers
kind
of
like
the
top
high
level.
The
projects
working
on
that
also
reminds
me.
We
mir-
and
I
did
a
quick
update
to
the
attack
on
on
tuesday
attack
likes
to
get
updates
from
the
working
groups
on
regular
on
regular,
regular
intervals
now
so
I'll
find
a
link
to
that
youtube
and
a
timestamp
and
also
post
that
in
the
chat
as
soon
as
I'm
done
talking.
A
Okay,
great
so
we
will
move
on
and
also
agenda
looks
a
little
light.
So
if
anybody
has
any
topics,
don't
hesitate
to
add
those
and
we'll
move
on
to
caleb
who
has
an
update
on
the
criticality
score
project.
D
Hello-
and
I
haven't
been
in
a
while-
so
I
was
I'm
caleb
from
google-
I've
been
working
on
criticality
score.
The
project
and
package
analysis
stuff,
there's
work
going
on
with
package
analysis,
but
I'm
not
going
to
cover
it
at
this
time.
I'll,
probably
wait
till
the
next
sync
that
we
have
in
terms
of
criticality
score
in
june.
I
I
managed
to
get
some
of
the
new
code.
No,
I
got
the
new
code
that
I
wrote
running
and
was
able
to
score
or
collect
signals
for
about
900
000
github
repos.
D
I've
put
the
link
in
there
and
for
anybody
to
look
at
I've
yet
to
compare
that
to
the
previous
all
dot
csv
that
was
posted
to
the
project
and
to
see
what's
changed
or
how
it
differs.
But
I
intend
to
do
that
in
the
next
couple
of
weeks,
so
yeah
feel
free
to
take
a
look.
D
I've
done
some
like
early
analysis
on
it
and
I'm
basically
intending
to
clear
house
on
the
signals
that
we
use,
because
most
of
them
aren't
particularly
useful,
in
particular
a
lot
of
them
oversee
over.
What
do
you
call
it
over
index
on
popularity
and
activity
as
opposed
to
actual
like
how
many
people
depend
on
this,
or
is
this
actually
secure
or
not?
Oh
sorry,
yeah
the
security
posture
and
things
like
that.
D
So,
for
example,
and
some
of
the
data
is
just
not
very
useful,
so,
for
example,
org
count
or
contributor
count
tends
to
follow
forks
and
so
a
bunch
of
linux
forks
rank
far
too
high,
so
things
like
that
are
interesting
in
terms
of
how
they
affect
the
data
set.
I
also
did
some
analysis
on
because
I've
now
got
a
big
set
of
signals.
I
can
do
some
statistical
analysis
and
looking
at
the
actual
distribution
versus
the
is
it
actually
zipfian
distribution
and
turns
out.
D
Most
of
them
are,
but
some
of
them
aren't
so
yeah,
I'm
hoping
to
kind
of
report
this
more
formally
as
well
so
but
out
of
that,
there's
also
work
going
on
productionizing,
the
entire
infrastructure,
so
it's
kind
of
expensive
to
run
this
locally
yourself.
D
It
took
about
10
days
to
generate
the
data
for
900
000
repositories,
and
that
was
with
about
eight
github
personal
access
tokens
and
me
nursing.
The
job
the
whole
way,
so
that's
not
particularly
easy
for
anybody
to
repeat
so
the
idea
of
productionizing
it
is
that
it
is
very
easy
for
well,
no
one
has
to
do
any
work.
It
just
starts
producing
signals
and
if
we're
trying
to
improve
signals,
then
it's
really
important
to
be
able
to
rerun
that
job
over
and
over
again
and
so
being
able
to
have.
D
This
productionized
makes
it
easy
to
start
iterating
on
adding
new
signals
and
like
yeah,
trying
to
improve
the
actual
sort
of
sources
of
data
that
we
have
in
terms
of
how
this
works,
the
other
part
of
productionizing.
It
also
means
that
the
data
is
current
and
we're
not
waiting
a
year
for
someone
to
actually
regenerate
the
data
itself.
D
So
hopefully
we
can
start
to
gain
some
momentum
and
interest
around
the
data
it's
producing
as
well
so
yeah
and
hopefully
like
research,
researchers
and
other
people
can
start
to
use
it
for
their
own
interests
or
extend
it
and
add
signals
that
they're
interested
in
as
well.
So
that
is
the
state
of
the
work,
so
I've
got
two
links
in
here.
One
is
to
a
pull
request.
I
just
chucked
up
with
the
current
state
of
the
design
dock
that
I'm
working
on.
It
refers
to
some
skalk
it.
D
It
kind
of
is
going
to
use
a
lot
of
the
scorecard
infrastructure
pieces,
but
the
document
that
I
refer
to
isn't
actually
public.
So
I'm
I
don't
know
exactly
how
I'll
adjust
the
dock
to
kind
of
make
that
easy
for
people
to
follow,
but
basically
it's
going
to
leverage
a
lot
of
the
scorecard
infrastructure
to
try
and
make
it
to
get
this
up
and
running
quickly,
so
that
that's
basically
it
does
anyone
have
any
questions
or
comments.
D
I
know
that
some
of
this
has
been
put
up
very
recently,
so
yeah
I'm
interested
in
any
feedback,
and
I'm,
if
you
don't
have
it
now,
please
reach
out
to
me
on
slack
or
email,
whatever
I'm
happy
to
chat
at
any
time
as
long
as
it's
in
apac
friendly
time.
B
C
Yeah
I
had
I
had
one
one
comment.
I
guess,
which
is
that,
if,
if
you
haven't
already
try
to
make
it
a
github
app
because
they
hand
out
a
much
more
generous
allowance
of
rate
limits,
I
have
also
been
scraping
recently
and
the
5000
limit
doesn't
get
you
very
far.
D
No,
it
doesn't
so
I
I'm
hoping
to
leverage
the
scorecard
experience
in
running
and
I
think
they
have
some.
They
already
have
app
access,
so
it
may
be
that
I
don't
have
to
do
anything,
but
I'll.
Certainly
keep
that
in
mind.
I
may
have
to
do
something
specifically
for
criticality
school.
A
I
believe
the
scorecard
stuff
uses
paths,
but
for
for
reading
do
you
have
to
you
for
the
github
app?
You
still
have
to
have
it
installed
on
the
packages
you
want
to
read
or
the
github
repositories
right.
B
C
You
know
like
the
permissions
system
for
github,
I
might
have
done
it
differently,
not
knowing
the
technical
constraints
that
were
in
in
play.
But
the
main
thing
is
that
you
get
your
your
I'm
forgetting
the
magic
word
for
scope.
You
get
your
scopes
and
then
that
scope
is
is
sort
of
like
joined
with
what
you
can
see
in
your
account
if
you're
logged
in
and
that's
what
you
can
do.
C
So
if
you
take
the
reaper
permission
and
you're
logged
in
then
you
can
see
repos
that
you
are
able
to
see
that
are
private,
that
that
are
visible
to
you,
but
not
ones
that
are
not.
And
of
course
you
can
see
any
public
repo
and
read
it.
A
Okay,
yeah.
I
have
to
look
at
the
app
because,
like
on
the
apps,
actually
you
know
what
I
haven't
done
is
is
apps
can
like
ask
people
to
you,
know,
authenticate
and
like
the
app
can
act
as
a
person,
but
I
think
also
the
app
can
just
act
as
an
app
and
at
that
point
I
think
it
only
can
look
at
the
stuff
that
it's
been
installed
on,
but
yeah,
maybe
there's
another
another
way
to
get
higher
higher
quota
for
the
public,
publicly
readable,
repos.
So
yeah.
Let's
take
a
look
at
that.
C
We
could
we
could
also
ask
our
friends
who
we
know
through
npm
whether
they
can
escalate
this
question,
because
I'm
I'm
sure
that
there
are
under
the
table.
Sweetheart
deals
for
for
the
right
people.
D
C
So
microsoft
will
give
us
millions
of
dollars,
but
not
more
than
5000
requests
per
hour.
I
mean
fair
enough.
D
I
think
the
the
the
other
part
of
this
is
there's
projects
like
gh
archive
and
gh
torrent,
and
they
are
potentially
useful
sources
of
gathering
this
data.
But
there
are
some
questions
I
have
around
the
reliability
and
I
don't
necessarily
want
to
be
reading
from
a
data
source
that
then
starts
to
have
gaps
or
misses
stuff.
So
that's
partly
why
I
focused
on
reading
from
the
origin
as
opposed
to
a
mirror
of
the
data.
D
So
in
time
that
may
change
and
I'd
like
it
to,
but
at
this
stage
it's
kind
of
as
it
is
yeah.
A
B
A
For
that
all
right,
any
other
updates.
Are
you
done
caleb.
D
D
So
hopefully,
some
of
the
stuff
here
starts
to
accelerate
as
well
in
terms
yeah,
so
I'm
hoping
that
that
he
can
join
at
some
point
as
well
these
meetings
but
yeah,
I
think
that's
about
it
yeah
and
I'm
also
yeah
a
whole
lot
of
the
way
that
the
project
has
been
built
is
to
try
and
encourage
or
facilitate
other
people
making
use
of
this.
So
yes,.
E
And
one
thing
I'll
just
throw
in
quickly:
I
do
apologize
for
the
background
noise,
but
if
you'd
like
to
have
a
brainstorming
session
with
the
work
group
on
signals
or
any
way
that
we
could
brainstorm
together
to
help
your
work,
we
can
certainly
do
that
at
a
future
meeting.
D
Definitely
yes,
and
I'm
also
interested
in
people
collecting
papers
and
stuff
where
others
have
done
research
and
figured
out
whether
things
are
useful
or
not.
For
example,
I
read
one
recently
that
said:
popularity
isn't
a
very
good
indicator
of
security,
so
things
like
that
are
useful,
so
any
papers
that
may
also
give
indication
around
what
we
should
be
collecting
would
be
really
helpful
as
well.
A
Great,
so
I
think
I'll
move
on
to
our
next
topic
I'll
go
ahead
and
present
the
the
charter.
A
Okay,
so
yeah
so
with
feedback
on
the
charter
and
questions
on.
Should
we
be
voting
on
this
thanks
again
to
jay
for
for
setting
us
up.
E
Yes,
absolutely
thank
you
jay
and
yeah.
I
think
it
says
somewhere
in
the
charter
that
we
have
to
essentially
formally
vote
on
it
as
a
working
group.
So
maybe
because
this
was
our
first
alternate
time
session.
Maybe
we
didn't
have
as
many
folks
as
we
normally
do.
We
can
at
least
do
some
basic
discussion
today
and
then
we
could
let
everyone
know
that
maybe
at
the
next
working
group
meeting
we'll
take
a
maybe
a
formal
vote
or
a
semi-formal
vote
to
essentially
codify
this,
so
to
speak.
C
One
thing
if
if
this
is
following,
like
the
template
design,
one
thing
that
stood
out
when
we
did
this
for
securing
software
repos
was
that
there's
nothing
in
there
that
defines
the
initial
contributors.
I
think
it
is
no
maintainers.
I'm
sorry
maintainers
are
like
the
special
special
class
of
folks,
so
we
wound
up
adding
a
section
at
the
bottom,
where
we
listed
the
initial
maintainers
to
bootstrap.
It.
A
Yeah,
I
I
it
does
say
like
the
maintainers
here
will
document
it
in
the
repository.
I
was
hoping
that
that
would
be
essentially
the
people
that
were
the
the
leads
from
from
our
project
is
my
proposal.
There.
Any
thoughts
on
that.
C
I
have
no
objection
just
bearing
bearing
in
mind
that
we
should
we
should
am.
I
came
back
yeah.
No
sorry.
F
F
Maintainers
aren't
defined.
C
The
the
way
it's
defined
this
this
is
my
recollection.
It's
been
a
little
while,
since
I
read
read
the
boilerplate,
but
the
way
it
was
defined
was
that
maintainers
are
quite
powerful
or
are
essentially
the
final
deciders
in
in
disputes,
but
there's
nothing
in
it.
That
defines
who
is
a
maintainer.
F
Yeah,
that
might
be
that
might
be
solved
by
just
on
that
on
the
initial
page
there
giving
them
a
title
at
the
top
saying
you
know
whatever
and
maintainers,
and
then
that
and
then,
and
then
in
the
I
mean,
I'm
not
sure,
that's
a
that's!
My
quick
fix,
that's
a
band-aid
shrug,
but
if,
on
the
page
the
page
you
were
at
before
jeff
yeah
here,
you
gotta
hope.
But
you
have
individuals
here
at
the
top
here
that
they
can
leads
or
the
contributors
or
underneath
maintainers
right.
F
That
jack.
C
F
Because
I
almost
want
to
say
that
in
in
the
whole
hell
I'm
I'm
going
to
check
right
now,
because
when
I,
when
I
did
this
I'll
be
honest,
all
I
did
was
I
pulled
from
one
that's
been
adopted,
so
so
I
pulled
for
one
that's
been
adopted
and
then
I
looked
up
another
one
that's
been
adopted.
They
both
look
the
same.
I
said
this
is
what
we're
riding
with
as
a
template.
C
C
Yes,
I
I
don't
disagree,
we
could.
We
could
have
a
section
that
just
says
for
the
purposes
of
this
document
the
maintainers
of
the
people
listed
in
the
readme.
You
still
need
the
maintainers
to
vote
people
into
the
maintainership
or
to
vote
them
out.
F
E
Yes,
I'm
in
total
agreement
too.
I
would
want
to
really
be
thorough
before
you
know
formalizing
this,
because
I
think
changing
it
will
be
kind
of
a
hassle,
so
not
that
it's
it
shouldn't
be
ever
changed
ever,
but
I
I'm
in
agreement
that
we
should
really
make
sure
this
sounds
good
and
and
review
pretty
thoroughly
before
codifying
it
so
to
speak.
A
Yeah,
I
just
want
to
say
I
agree.
I
agree
with
you.
Jay
like
I
like.
I
like
that
the
the
charter
will
be
essentially
saying
the
maintainers
are
documented,
but
we
do
need
to
have
that
documented,
and
I
would
I
would
propose
just
doing
it
on
the
readme,
where
we
do
say
that
the
maintainers
of
this
this
working
group
are
the
people
that
are
the
leads
of
any
of
the
the
sigs
projects.
F
F
A
Yeah
I
mean
yeah
there's
a
lot
in
here.
That's
like
the
maintainers
will
determine
the
process
for
selecting
future
maintainers
like
we're.
Not
we
don't
have
that
determined
yet
right,
but
I
think
it's
okay
to
adopt
the
charter,
even
if
all
these
processes
aren't
determined.
E
C
So
the
way
I
saw
it
when,
when
we
talked
about
this
in
securing
software
repos,
the
way
we
saw
it
is
that
the
tsc
is
essentially
dead
letter
until
there's
a
dispute.
So
it's
it's,
not
the
working
group.
It's
it's
a
different,
a
different
body
that
is
formed
out
of
the
maintainers
who
attend
or
who
are
involved
with
the
working
group,
and
they
see
it
as
not
necessary
to
spin
it
up.
C
Unless
you
have
a
dispute
that
requires
a
vote
and
there's
always
the
backstop
that
even
if
that
doesn't
settle
matters,
you
can
fall
back
to
the
attack
to
make
the
decision
and
and
settle
matters.
C
If,
if
we
need
to
spin
up
the
tsc
elect
a
chair,
take
a
vote,
then
it's
probably
a
unhealthy
situation
anyway,
but
mostly
electing
a
chair.
Just
comes
down
to
you,
you
put
the
meeting
together,
you
call
you
call
for
candidates,
you
call
for
votes
and
then
whoever
has
the
most
votes
is
the
chair,
and
then
they
act
like
a
chair
from
there.
C
The
one
thing
that
I
took
out
when
I
did
this
and
I
I
lodged
a
bug
in
the
the
templates
repo,
which
is
you-
can
see
that
on
2d
there's
a
little
snippet
of
text
which
sort
of
like
came
writing
shotgun.
The
tsa
chair,
any
other
tfc
members,
so
designated
blah
blah
blah
serves
as
the
technical
initiatives
voting
representative
from
the
technical
advisory
council.
There's
no
such
thing
the
attack
doesn't
accept
outside
voting
members.
It
is
its
own
thing
so
that
that
simple
text
is
is
a.
I.
F
C
A
So
I
mean
I
might
be
getting
ahead
of
myself,
but
what
do
we?
You
know
what?
What
do
we
think
of
the
essentially
the
you
know
we're
supposed
to
define
like
who's
a
maintainer
who's,
a
contributor
who's,
a
collaborator?
A
What
would
we
think
of
the
people
under
leads
being
maintainers
people
under
contributors
being
either
contributor
or
collaborator
and
like
in
in
the
idea?
Also
that
like
if
we
want
to
essentially
accept
a
new
project
to
the
working
group,
the
the
lead
on
that
project
would
be
a
maintainer.
Is
that
does
that
seem
right,
or
does
that
seem
like
it
should
be
somebody
that's
more
or
that
you
know?
That's,
not
a
news
like,
let's
say
a
new
project
coming
in.
E
A
Yeah
so
yeah,
because
there's
gonna
be
like
incubating
and
then
you
know,
there's
gonna
be
different
levels
of
projects,
so
maybe
it's
once
it
reaches
a
certain
level.
Then
the
lead
would
be
a
maintainer,
maybe
okay,
I'll
I'll.
A
Take
a
look
at
a
you
know.
I'll
propose
some
text
for
next
meeting
for
like
who
would
be.
You
know,
defining
some
of
the
things
that
this
says
will
be
defined
like
who's,
a
maintainer
who's
gonna.
How
do
we
add
maintainers
and
yeah
and
then
who's
a
collaborator
or
whatever
yeah,
and
then
that
we
can
put
that
in
the
readme,
because
that
won't
be
part
of
the
charter?
But
I
think
the
charter
looks
good.
E
Very
cool
but
yeah,
I
think,
but
but
again,
initial
reaction
is
the
way
that
those
definitions
are
the
way
you
suggested.
Jeff
makes
sense
with
collaborators
being
you
know,
anyone
in
the
work
group
who
shows
up
who
participates
and
then
the
contributors
can
be
the
leads
of
the
kind
of
the
individual,
the
collaborators
that
could
be
the
leads,
as
kind
of
like
the
representatives
responsible
for
those
efforts.
A
Awesome
so
yeah,
like
you,
said
amir:
let's
wait
until
next
meeting
to
do
about.
E
Gonna
update
the
notes
to
reflect
that
right
now,.
A
E
Oops
yeah
I'd
love
to
hear
from
anyone
in
the
work
group
if
they
had
a
chance
to
review
it
if
they
had
thoughts,
any
any
feedback
or
discussion
around
that.
C
I
am
still
getting
the
access
firewall,
I'm
afraid,
still
getting
there.
You
need
access
darn,
I'm.
E
Okay,
then,
then,
maybe
till
I
get
that
sorted
out,
we
can
move
on
to
something
else.
If
someone
has
another
topic.
E
Okay,
can
can
everyone
see
the
screen.
E
Oh
okay,
okay,
wonderful
yeah,
so
so
just
to
provide
a
little
bit
of
context.
So
the
the
objective
of
this
form
is
really
just
to
capture
feedback
from
openssf
and
the
greater
open
source
community
on
helping
provide
some
direction
as
to
which
projects
we
should
be
considering
it's
not
the
best
name,
but
for
the
timing,
it's
called
the
ingestion
form
for
securing
critical
projects.
E
It
currently
has
just
a
couple
of
very
basic
questions.
So,
first
one
being
pretty
straightforward,
you
know
what
project
or
projects
do
you
nominate
is
critical.
This
is
currently
short
answer,
so
I
guess
I
can't
make
edits
right
now.
Oh
I'm
just
adding
questions,
okay,
yeah.
So
this
would
just
be
a
short
answer,
question
and
then
taking
in
some
feedback
that
we
got
at
the
last
session,
trying
really
hard
to
make
the
answers
in
a
way
that
we
can
automate
this
a
little
bit
easier
or
or
reduce
the
risk
of
error
or
inconsistencies.
E
So
first
follow-up
question
is
a
metadata
or
package
link,
in
which
I
think
we
should
put
a
recommended
format
so
so
that
we
can
give
an
example
of
what
we
want
it
to
look
like
and
then
the
next
question
being
a
pretty
simple.
You
know
why
question?
E
What
are
what
criteria
is
that
based
on
this
is
also,
I
think,
I'm
working
on,
maybe
having
a
couple
of
pretty
common
answers
that
we
can
use,
but
then
also
giving
folks
open
ability
to
specify,
because
we
won't
be
able
to
capture
everything
I
don't
think
and
as
drop
down
or
as
as
designated
answers
and
then,
lastly,
just
again
for
automation
or
for
simplification,
just
like
a
link
to
a
project
page.
E
You
know
github
link
or
you
know
what
have
you,
but,
as
you
can
see
it's
it's
very
simple
so
far
and
and
and
going
back
to
kind
of
what
our
objective
is
for
for
even
doing
this,
I
always
defer
back
to
our
documentation
of
of
really
what
our
objective
is,
which
is
to
just
to
provide
a
best
effort
analysis
of
research
to
generate
a
curated
set
of
projects.
E
So
with
that,
I'm
going
to
try
and
mute
myself
because
I'm
guessing
the
background
noise
is
unbearable
for
you
all,
and
I
do
apologize
for
that.
But
I'd
love
to
hear
any
initial
feedback,
any
thoughts,
anything
that
people
have.
We
greatly
appreciated.
So
thank
you
and
love
to
hear
some
feedback.
C
I
was
curious
about
the
the
github
link.
Gitlab
link
type
thing
is:
is
there
a
reason
we
wouldn't
just
have
a
text
field
and
just
ask
them
to
paste
the
link
to
the
to
the
source
or
the
page.
E
Yeah
that
probably
makes
sense.
I
was
just
wondering
for
kind
of
automation
purposes,
or
you
know
if
this
is
going
to
feed
into
like
a
larger
list
or
some
kind
of
a
feed
but
yeah,
I
think
it.
I
might
have
just
been
overthinking
this
one
and
it
could
just
be
as
simple
as
you
know,
please
provide
the
link
to
the
project,
and
maybe
we
could
put
examples
so
that
people
do
it
in
a
specific
format.
C
C
Yeah
I'm
thumbs
upping
and
feeling
feeling
like
don't
like
you're,
not
seeing
me
as
I
see
it,
yeah
yeah.
E
D
So
a
couple
of
I
think,
first
of
all,
I'll
follow
on
with
that.
I
think,
having
a
link
to
the
source
repository
and
any
other
alternative
links
as
well
may
be
useful,
because
sometimes
the
canonical
like
source
repository
can
be
a
bit
confusing
because
they'll
have
a
mirror
on
github
or
something
else
also.
Actually,
the
main
point
I
had,
which
is
the
criteria,
may
not
want
to
be
like
one
option
pick
out
of
a
list.
D
It
might
be,
I'm
basing
this
on
more
than
one
option,
so
there
may
be
a
reason
why
they
choose
to
have
like
it's
based
on
criticality
score
and
some
other
criteria.
That
they've
like
decided
based
on.
D
E
E
So
that's
that's.
That's
good
to
know
I'll
be
sure
to
to
have
something
like
you
know.
Please
select
all
the
answers
that
something
like.
G
Yeah
hey,
so
I
had
a
quick
question
about
this.
So
are
you?
Are
we
expecting
that?
Whoever
is
using
this
form
to
input
a
data?
They
are
also
like
providing
the
the
input
to
justify
their
their
input,
as
in
like
do,
they
have
to
actually
provide
the
criticality
score,
or
let's
say
you
have
github
fork
like
do
they
have
to
give
you
the
quantitative
number?
So
are
we
also
attaching
like
a
text
input
with
each
of
these,
or
this
is
just
like
we
just
or
somebody
just
selects
like.
E
Yeah,
that's
that's
a
great
question.
I
think.
Generally,
we
want
to
make
this
as
easy
as
possible,
so
the
more
we're
asking
people
to
do,
the
more
likely
we
are
to
lose
them,
but
I
do
think
it's
important
to
have
you
know
some
type
of
justification
or
selection
criteria,
so
maybe
not
requiring
them
to
you
know,
write
an
essay
as
to
why
this
was
selected
but
having
at
least
some
reasoning
for
why
a
project
is
selected.
E
I
think,
is
going
to
be
very
important
because
you
know
the
the
the
first
thing
people
are
going
to
ask
when
or
or
see
when
they
see
that
something
like
this
is
well.
Why
this?
Why
that?
Why
was
this
project
chosen?
Why
was
that
project
chosen?
So
I
think,
having
at
least
something
to
attribute
to
why
to
the?
Why?
Even
if
it's
just
you
know
it
has
a
high
criticality
score,
it
was
identified
in
you
know
this
research
paper
and
it's
one
of
the
top.
You
know
by
download
count
in
this
platform.
E
G
And
pardon
my
naivete,
but
I'm
also
like
curious
about
like
what
is
the
intent
of
using
this
form.
So
because
I
mean
I,
I
see
the
the
list
of
projects
in
the
csv
forms
that
has
already
been
created.
Caleb
was
sharing
that
and
I
mean
so
for,
for
example,
for
python
it
has
like
130
000..
Do
we
expect
somebody
to
suddenly
break
into
like
top
100,
and
that's
that's
very
unlikely,
so
why?
What?
G
E
Sure
yeah
I'd
love
to
share
at
least
kind
of
what
the
initial
thought
process
was.
So
if
you
see
on
my
screen
here,
this
was
kind
of
our
first
iteration
of
this
exercise.
Basically,
and
we
kind
of
had
multiple
kind
of
layers
right.
So
first
we
had
what
were
called
candidate
projects,
so
this
so
the
idea
being
that
you
know
we
could
take
you
know
from
the
community
at
large
ingest.
E
These
nominations
to
have
kind
of
an
initial
we'll
call
it
a
set
and
then
to
be
able
to
kind
of
chop
away
at
it
using
kind
of
further
analysis
which
what
we
did
here
was
when
we
had
candidate
projects
we
actually,
if
it
was
warranted,
discussed
with
the
work
group
to
you,
know
either
give
it
a
yes
to
either
give
it
a
no
and
to
essentially
to
do
that
curation
step
to
get
to
kind
of
a
final
list,
or
I
should
stop
using
the
word
list
to
to
set
on
a
final
set
of
projects
that
you
know
have
gone
through
this
kind
of
filtering
or
this
kind
of
curation,
because
that
I
think
is
is
is
the
value
that
we
are
trying
to
provide.
E
Is
you
know
we're
not
just
one,
because
because
you're
absolutely
right,
there
have
been
many
other
efforts.
I
mean
one
that
comes
to
mind
is
like
harvard
census,
2
and
many
other
different
research.
I'm
trying
to
to
answer
this
question.
You
know
what
are
what
what
what
are
projects
out
there
that
are
critical,
so
I
think,
having
a
community
driven
curation
is
what's
gonna
is,
what's
gonna
really
provide
the
value
because
because
because
automation
and
lists-
and
what
have
you
are-
are
great
but
they're
always
gonna
be
imperfect.
E
So
I
think
doing
a
little
bit
of
both
where
you
know
we
have
this
quantitative
process
or
quantitative
reasoning,
but
then
this
qualitative
kind
of
community-driven
curation
to
help
kind
of
shine.
The
light
on
you
know
which
projects
to
focus
on
is
the
intention.
So
I'd
love
to
hear
your
initial
thoughts
on.
E
If
you
think
you
know
we're
on
the
right
track
to
to
do
that,
or
if
there
are
other
things
we
should
be
considering,
because
at
the
end
of
the
day
yeah
we
just
want
to,
we
want
to
do
what's
best,
for
you
know
the
open
source,
community
and
open
source
ecosystem.
So
that's
why
another
reason
we're
going
through
this
process
of
getting
a
lot
of
feedback
and
and
trying
to
make
it
as
as
complete
as
we
can
be
before
it
starts
kind
of
being
used
on
a
bigger
scale.
G
Yeah
so
we
actually
consume
the
data
like
that,
like,
for
example,
the
excel
sheet
that
was
shared.
I
mean
I
just
like
emailed
it
to
my
group
and
then
basically
I
was
telling
them
to
to
use
the
data,
so
you
are
aware,
but
I
mean
just
for
everyone
else.
We
have
this
project
that
we
are
working
on
there
of
scanning
the
pipi
project,
python
projects
that
are
in
the
pipeline
repository
to
understand
or
to
detect
vulnerability.
G
So
we
have
this
pro
like
website
pipi.openrefactory.com,
where
we
are
working
with
alpha
omega
like
microscope
and
others
to
scan
the
ipi
projects
and
list
the
vulnerabilities
that
are
being
identified
and
with
very
high
precision
triaging,
so
that
whatever
is
listed
there
is,
is
useful
and
one
way
that
I
mean
I
I
found
it
very
interesting,
especially
the
csv
list
that
was
shared
in
the
document
that
I
think
kelly
was
pointing.
To
is
that
I
mean
we
can
just
identify
our
like.
G
We
can
have
that
list
on
the
background
we
just
say
like:
let's
run
on
projects
100
and
like
201
to
250
or
whatever,
and
then
we
our
tool
or
our
thing
that
is
running
or
driving
the
website.
You
can
automatically
do
that
scan
and
and
generate
the
results.
So
that
gives
us
a
a
good
way
of
of
like
having
a
long
list
of
stuff,
and
then
we
can
pick
and
choose
our
our
targets.
G
I
do
had
one
question
also
regarding
to
that,
as
in
how
often
have
you
seen
that
the
ranks
or
these
lists
change,
I'm
just
curious
as
an
academic
interest
like
from
time
to
time,
do
you
see
like
things,
wake
into
things
get
out
and
like
does
it
happen
at
the
top
level?
Does
it
really
happen
at
the
other
end,
where
we
don't
necessarily
care?
It's
just
something.
That's
of
academic
interest
with
somebody.
A
I
mean
to
answer
the
last
question
and
we've
only
done
this
exercise
once
so.
It
hasn't
changed,
yeah
and
we're
looking
at,
like
you
know
the
hundreds
of
packages,
not
in
the
thousands
that
we
want
to
automate
like
things
with,
but
like
in
the
hundreds
where
we're
trying
to
do
things
that
can
be
useful,
like
at
a
manual
level.
A
I
hope
that,
but
I
also.
H
A
A
C
I
was
going
to
respond
to
many
one,
but
I
think
randall's
had
his
hand
up
for
a
while
randall
is
it
okay?
If
I
just
quickly
make
a
comment
and
then
bounce
out
abs.
C
Cool,
so
at
the
risk
of
self-promotion,
something
I'm
slightly
notorious
for
I'm
gonna
point
to
a
talk.
C
I
did
at
the
last
open
source
summit
and
there's
a
youtube
link
there
and
a
lot
of
the
things
we're
asking
are
sort
of
issues
we
grapple
with,
and
I
I
had
a
particular
subset
of
the
approach
which
was
talking
about
you
know,
expert
opinions
about
things
in,
in
addition
to
two
data-driven
approaches
like
the
criticality
score,
because
my
view
is
that
will
probably
wind
up
needing
both
in
the
long
run,
but
a
lot
of
the
issues
you
discuss
like
for
example.
What
order
do
you
assess
things?
C
There's
a
bit
of
a
turtles,
all
the
way
down
problem,
which
is?
If
you
knew
how
to
assess
things,
then
you
wouldn't
need
experts
to
give
opinions,
so
there's
there's
sort
of
a
circularity
there.
So
I
think
hopefully,
some
of
that
talk
will
help.
C
Thank
you
all
right.
Well,
good.
I've
saved
some
time
randall!
Thank
you.
Thanks
for
waiting,
it's
all
yours.
H
No
problem,
I've
also
seen
the
talking
it's
pretty
good,
but
I
was
gonna
say
so
amir
and
I
had
talked
about
turning
this
into
a
github
repo
in
the
past
and
we,
I
did
do
some
work
on
that
this
weekend
and
I
found
out
that
criticality
score
had
changed
from
originally
how
I
thought
it
would
or
we
could
set
it
up,
but
I
did
notice
that
there's
a
lot
of
similarities
between
what's
going
on
now
and
the
former
python
tool
as
to
what
we're
trying
to
do
here
as
well.
E
Yeah
and
and
if
I
could
jump
in
real
quick,
I
could
one
reason
I
could
see
github
working
is
is
because,
even
with
our
first
iteration
of
this
I'm
going
to
share
my
screen
again
here.
This
is
the
work
groups,
repo
and
when
we
did
our
first
iteration,
a
decent
amount
of
folks
submitted,
opened
issues.
Saying
hey:
have
you
considered
this
project?
Have
you
considered
that
project?
E
So
that's
one
of
the
reasons.
I
guess
I
think
I
could
see
github
being
a
useful
tool,
because
you
know
it's
a
public
place
where
people
can
see
all
of
this.
So
you
know
if,
in
theory
someone
were
to
say,
you
know,
consider
adding
these
projects,
and
this
is
the
reasoning
we'll
be
able
to
really
trick
to
to
trace
all
of
that.
E
I
think
in
a
very
effective
way
where
we
can
see
how
this,
how
this,
how
this
list,
I'm
sorry
how
this
set
grows
and
and
and
evolves,
and
and
to
go
back
to
to
to
menards
to
minora's
question.
I
do
think
over
time.
E
I
know,
come
if
you
were
to
look
at
its
let's
say
it's
criticality
score
or
a
lot
of
these
other
different
things
out
there,
so
so
having
a
curated
kind
of
living
thing
that
the
community
drives
is
where
I
could
really
see
the
value
of
it
coming,
because
I
think
it
will
change
over
time
and
being
able
to
consider
you
know
emerging
technologies,
maybe
things
that
fly
under
the
radar.
You
know
part
of
this
whole.
You
know
we
use
that
that
that
graphic
a
lot.
E
You
know
the
lib,
the
lebraska,
but
you
know
this
could
very
well
be
a
way
to
help
identify
those
projects
that
are
just
you
know,
one
solo,
maintainer
or
even
are
even
not
maintained,
anymore,
hardly
maintained.
E
You
know,
I
think
it's
really
important,
that
we
have
some
kind
of
a
mechanism
that
can
shine
light
on
these
different
types
of
situations
in
open
source,
so
so,
based
on
all
that
sorry,
I
know
that
was
quite
long-winded.
Randall
does
this
kind
of
fit
into
what
you've
been
working
on
in
terms
of,
like
you
know,
using
github
as
a
way
to
curate
this
and
just
kind
of
track,
all
the
things
that
go
on.
H
So
it
is
very
similar
in
that
regard.
What
I've
chosen
to
do
right
now,
just
to
like,
have
like
a
dummy
test,
is
I'm
using
mpm,
so
it's
actually
quite
easy
to
pull
out
specs
out
of
npm,
so
if
we
did
want
to
have
the
tool
kind
of
completely
not
like
with
no
like
hard
score,
if
that
makes
sense,
but
there's
more
like
hey
this
down,
or
this
package
got
x
amount
of
downloads
and
was
last
updated.
This
date
like
stuff,
like
that,
we
can
also
do
that
for
this
list.
H
H
D
H
And
right
now
we
just
tried
pulling
things
like
the
amount
of
times
the
package
was
downloaded
last
time
it
was
updated
and
a
few
other
things,
I'm
not
the
mpm
api,
it's
a
guy
that
we
know,
john.
H
I
don't
know
if
you
guys
know
john
neal,
but
he's
been
helping
out
with
the
or
the
mpm
side
of
things,
but
yeah
we're
just
pulling
out
basically
basic
information
that
npm
has
available
just
to
put
that
up,
because
john
thinks
that
it's
a
really
terrible
idea
we're
going
to
make
this
like
non-competitive
to
start
attaching
hard
scores.
So
this
is
about
like
just
having
an
opinion
of
some
of
whether
or
not
something
is
critical,
then
that
shouldn't
necessarily
go
attached
to
a
hard
score.
H
D
So
so,
first
of
all,
like
it
sounds
like
some
of
the
work
you're
doing
is
similar
to
like
depth.dev
and
the
open
source
insights
team
at
google.
They
they
are
scanning,
like
npm
and
pipey,
and
a
bunch
of
other
repositories
to
collect
data.
I
don't
think
they
collect
data
specifically
for
generating
a
criticality
score,
but
the
criticality
score
project
that
I'm
working
on
does
use
it
as
a
data
source.
They
have
a
bigquery
data
set.
You
may
be
interested
in
and
find
it
easier
to
query
than
npm
directly.
D
You
can
yeah
in
terms
of
like
github,
as
well
as
a
data
source,
I'm
very
interested
in
not
being
just
github
centric.
It's
just
that.
Github
is
a
big
one.
It
has
a
lot
of
the
data
already
and
and
to
its
api.
Apis
are
fairly
straightforward
to
query
against
in
the
future,
so
I
have
an
enumeration
tool
that
enumerates
github,
but
I'm
interested
in
like
how
can
we
expand
that
to
other
other?
Well.
D
Yeah,
so
so
I'm
actually,
I
think
my
next
numerator
is
actually
going
to
enumerate
the
depth.dev
open
source
insights
data
set
in
bigquery
and
and
like
spit
out
all
of
the
urls
for
the
repos.
It
knows
about
gitlab
and
github
ones,
because
gitlab
doesn't
actually
have
something
like
github
search
for
creating
that
yeah.
H
Yeah
all
right.
Well,
I
know
I
know
that
we've
had
call
in
previous
calls
that
has
come
up
about
this
list
about
how
would
we
handle
non-github
related
things
so
yeah?
But,
as
I
said,
I
know
that
this
list
is
very
focused
on
not
necessarily
turning
this
into
a
competition,
because
I
was
one
of
my
things
because
I
do
work
in
a
pretty
big
js
project.
It's
one
of
my
many
day
jobs
and
I
know
I
know
how
some
people
can
get
real
competitive
in
js.
D
I
I'm
very
keen
like,
in
fact,
if
you're
generating
data,
that's
useful,
it
may
be
something
that
I
end
up,
choosing
to.
We
end
up
choosing
to
source
for
the
criticality
school
project
as
well.
So
it's
it.
I
think
it's
like
a
thousand
flowers,
blue
or
whatever.
It
is
like.
It's
also
helpful
to
have
some
some
competition
in
the
space,
because
people
have
different
ideas.
Well,.
H
F
H
D
Yeah
and
I'm
certainly
interested
in
the
the
approaching
this
from
the
the
artifact
side
of
things,
as
opposed
to
the
source,
repo
source
repository
sort
of
thing,
so
yeah
very
interested
in
how
that.
H
F
B
D
A
H
D
Yeah
I'll
I'll,
try
and
update
that
readme
over
the
next
week,
just
to
make
it
clear.
I
appreciate.
A
E
H
I
was
gonna,
wait
until
it's
a
little
bit
like
I
had
like
a
full
demo,
which
I
think
I
was
supposed
to
have
this
at
next
time,
but
I
I
yeah
I've.
I've
been
really
scatterbrained
as
of
the
last
week,
but
I
I
think
I
do
have
like
I
am
working
it,
I'm
working
with
it
up
with
peter
who
was
in
last
meeting.
He
was
here.
B
H
A
Awesome
well
looks
like
we're
out
of
time
any
last
things
before
we
go.
I
don't
see
anything
else
in
the
agenda.
A
Oops
all
right.
Well,
thank
you.
Everyone
for
joining
thanks
for
joining
us
at
the
new
time
and
hope
to
see
you
all
in
slack
or
on
github
issues
or
at
the
next
meeting.