►
A
C
Hi
everybody
sorry
I
was
having
a
bit
of
a
technical
difficulties.
There
just
gonna
get
our
notes
up
and
well.
We
can
go
ahead
and
get
started.
C
C
Thank
you,
Justin
Caleb,
for
getting
those
notes.
Updated
typically,
we'd
like
to
start
with
any
new
faces.
I
see
a
lot
of
familiar
faces,
but
if
anyone
has
never
given
a
proper
intro
before
or
would
like
to
give
a
update,
now
would
be
a
great
time.
We'd
love
to
hear
from
you.
If
anyone
has
anything
awesome
Chuck,
you
have
your
hand
up.
D
More
of
an
outro,
unfortunately,
due
to
changing
circumstances
at
at
my
end,
in
Shopify
I'm,
going
to
have
to
drop
out
of
this
group
kind
of
sucks,
but
I
thought
I'd
be
upfraid
about
it
rather
than
ghosting.
All
of
you.
C
You
I
definitely
appreciate
yeah
the
the
heads
up
and
yeah
to
Echo
adjusted
so
I
didn't
mean
to
cut
you
off
there.
Jeff,
but
yeah
I
will
definitely
miss
you
and
you're
your
insights
so
but.
C
That's
totally
understandable,
though,
definitely
seems
like
a
lot
of
changes
are
kind
of
on
the
horizon
for
some
folks
and
companies
are
gearing
up
for
who
knows
what
so
but
yeah
again,
thank
you
for
everything
and
yeah
definitely
hope
we
can.
At
least
you
know,
be
able
to
keep
in
touch
in
some
way.
If.
E
D
C
That's
totally
understandable.
Well
again,
we
really
appreciate
it.
So
thank
you,
Jacques
cool.
Do
we
have
any
other
folks
want
to
say
anything
for
us.
C
E
C
Okay,
so
it
looks
like
our
first
agenda
item
Caleb.
If
you
want
to
take
it
away,
tell
us
more
about
how
criticality
score
has
gone.
B
Yeah
so
I'm,
looking
at
the
date
since
I
last
spoke
yeah,
so
we
have
criticality
score,
is
now
running
in
production
continuously.
So
basically
that
means
that
every
week
it
kicks
off
and
starts
generating
new
data.
Weekly
fields
are
like
really
frequent
for
something
that
doesn't
change
heaps,
but
I
figured.
It
was
easier
to
run
it
more
frequently
and
then
don't
dial
it
back
than
it
was
to
go
the
other
way.
You
get
used
to
things
running
a
certain
way
and
then
it's
hard
to
change
in
a
better
Direction.
B
So
I
decided
you
know,
let's
start
at
one
week
yeah,
so
that's
going
well.
I
just
saw
that
it
got
stuck
on
something
so
I'll
fix
that,
but
anyway
yeah
it's
so
yeah.
It's
working
pretty
reliably.
B
There's
a
deploy
process
in
place,
I'm
working
on
a
blog
post
at
the
moment
as
well
to
kind
of
announce
this,
but
the
whole
point
of
being
in
production
is
that
it
actually
takes
it
from
being
something:
that's
really
hard
to
run
at
scale
to
verify
improvements
and-
and
it
needs
to
run
at
scale
basically
to
be
able
to
have
an
impact
and
to
be
able
to
find
critical
projects.
So,
for
those
two
reasons,
that's
why
we've
been
working
on
getting
it
running
in
production.
So
that's
pretty
cool.
B
So
now,
if
someone
wants
to
introduce
a
change
or
try
and
understand
whether
a
new
metric
helps
or
doesn't
they
can
do
that,
we
also
have
the
data.
Is
it
in
bigquery
as
well
as
CSV,
so
people
who
want
to
actually
query
the
data
and
try
different
scores
that
that
way
or
explore
it
in
that
approach
using
bigquery?
They
can
do
that.
They
can
also
just
download
the
CSV
and
use
it
offline.
So
yeah,
that's
really
where
we're
at
with
that,
which
is
really
good.
C
Awesome
awesome
and
then
there's
a
particular
URL
domain
for
criticality
score
now
correct.
C
Thought
so
am
I
misremembering
that
let
me.
B
Yeah
yeah,
so
the
criticality
score
as
well
at
the
moment
is
very
popularity
and
activity
oriented
and
we
want
to
move
that
towards
being
more
around
impact
and
that
sort
of
thing
so
yeah,
hopefully
like
when
you
measure,
publicity
and
stuff
like
that.
It
actually
kind
of
makes
sense.
But
certainly
we
could
do
that.
B
It'd
be
interesting
to
see
as
well
like
now
that
it's
running
at
scale
regularly.
But
we
can.
You
can
start
to
do
change
like
analysis
on
how
things
change
over
time
as
well,
which
would
could
be
interesting
for
people
yeah.
C
And
that's
that's
fantastic
and
actually
building
off
David's
Point.
Here
from
the
chat,
you
bring
up
a
really
good
point
about
publicity,
because
we're
gearing
up
to
do
some
some
announcements
soon
too,
and
we
worked
on
some
pretty
high
criticality
projects.
So
I
think
we
could
easily
incorporate
criticality
score,
even
if
we
just
literally
State
the
Project's
criticality
score
as
part
of
the
results
and
would
be
a
very
easy,
quick
way
that
we
can
get
more
get
more
eyeballs
on
it.
F
F
In
both
directions,
I
mean
I'm
being
so
much
real.
That
I'm
sure
that
you
know,
like
you,
know,
we're
gonna,
you
know
now
this
is
live
and
it's
tracking
it's
a
horse.
Races
like
oh,
my
God
yo.
You
know
when
you
turn
it
into
a
game.
People
are
going
to
gamify.
This
I
mean
it's
like.
Oh
my
gosh
I
need
to
get
my
criticality
score
up.
You
know
this
is
going
to
become.
You
know
a
game
for
people,
but
also
that
just
you
know
it'll
be
interesting.
Just
from
a
psychological
point
of
view.
It's
like.
F
Okay,
all
of
a
sudden,
it's
on
you
know
Hacker
News,
all
of
a
sudden,
it's
on
Phonics,
all
of
a
sudden.
It's
on
like
what
does
that
do
to
the
credit.
You
know
how
house
you
know
steady.
Is
this
versus?
How
much
is
this
reactive
to
you
know?
Oh
my
gosh.
All
of
a
sudden,
you
know
that
you
know
this.
That
came
up
or
you
know
the
register.
You
know
had
a
a
good
article
about
a
bad
article,
but
you
know
yeah.
C
That's
a
great
Point
yeah
people
will
definitely
gamify
it
and
you
know:
try
and
yeah
they'll.
Just.
B
The
way
to
become
critical
is
to
actually
like
have
your
dependencies
in
places.
So
it's
going
to
be
interesting
as
well,
because
I
know
internally
at
Google,
the
people
who
run
depths.dale
are
looking
at
using
exposing
that
score
through
their
website
as
well.
So
it's
yeah.
Definitely
the
ability
to
race,
like
turn
it
into
a
public
school
board,
is
a
is
a
pro
like
it's
a
potential
problem,
or
maybe
it's
not
yeah
and.
F
I
mean
maybe
we've
got
great
sorry.
We've
got
really
bright,
Engineers,
who
have
still
reverse
engineer
this
and
find
a
way
to
to
boost
this,
even
if
it's
not
intended
to
people
it's
like,
but
but
that's
sort
of
good
in
the
sense
that
they
can
find
problems
with
the
score
that
it's.
That
is
not
actually,
you
know
a
consistent,
reliable
and
that
there
there
is
this.
C
Excellent,
do
we
have
any
other
questions
for
Caleb
or
any
other
discussion
points
regarding
the
criticality
score.
D
I
had
a
question
which
is
I
might
have
asked
this
before
and
then
forgotten
the
answer,
because
that's
just
how
I
roll
but
is,
is
there
sort
of
like
versioning
or
auditioning
for
the
criticality
score.
So
if
I
see
a
criticality
school
just
naked
in
the
world,
I
don't
know
which
version
of
the
criticality
score.
So
what
I
see
criticality
B1,
V2,
criticality,
October
Edition.
B
So
that
is
really
like:
how
do
you
present
the
criticality
score
and,
like
I,
haven't
considered
how
these
get
presented
out
of
context?
In
the
data
itself,
it
is
capturing
the
data
was
collected
or
the
basically
roughly
when
it
was
collected,
and
it
also
includes
a
commute
ID
but
they're,
not
particularly
usable
information,
so
yeah
I
haven't
really
thought
about
how
this
gets
presented
over
time.
B
The
the
score
itself
will
evolve
and,
and
the
reason
the
schema
is
called
v0
is
because
I
literally
just
took
what
was
coming
out
of
the
tool
and
put
it
into
bigquery.
B
There's
some
thought
around
the
the
and
the
other
reason
is
like
the
evolution
of
the
scoring
column
is
something
that
I
want
to
put
some
more
thought
into
because
of
this
problem
like
how
do
you
capture
the
change
in
the
score
over
time
and
what
that
might
reflect
to
people
who
are
consuming
it
so
yeah?
This
is
some
yeah
haven't
really
thought
about
how
it's
presented
certainly
like.
B
If
you
were
looking
at
it
and
then
one
day
the
score
changes
dramatically,
then
you
may
be
wondering
why
certainly
yeah
an
interesting
question
and
and
I
mean
if
we
had
a
website
that
you
could
link
back
to
or
some
other
thing
like
understanding
the
score
might
be
useful
or
that
it
could
deliver
some
more
context
around
how
it
was
calculated
yeah.
Those
sorts
of
things
would
be
useful,
but
like
have
not
thought
about
that
much
at
all.
At
this
stage,
cool.
C
Is
there
still
going
to
be
like
a
somewhat
public
like
CSV
file
of
the
of
what
the
output
of
the
criticality
score
generates
and
I
wonder
if
there's
that
could
be
I
mean
if
we
just
took
it
one
step
further
and
and
instead
of
just
obviously
I
think
it'd,
be
cool
to
have
the
link
to
download
this
to
the
CSV
file,
to
do
your
own
analysis,
but
if
it
just
basically
read
out
even
in
a
markdown
file,
like
you
know,
top
100
top
100
Java
top,
you
know
kind
of
like
how
it's
been
so
can
I
I'm
sure
it's
in
the
notes,
I
think
you
you've
put
the
link
to
the
outputs
of
criticality
score
before
and
yeah
I'm,
just
wondering.
B
C
Okay,
yeah!
No!
This
is
great,
though,
and
I
just
if,
if
it's
part
of
the
plan
or
or
what
have
you
I'm
sure
this,
this
data
could
feed
into
you
know
something
that
could
easily
be.
You
know,
presented
or
consumed
going
back
to
the
you
know
the
the
presentation
that.
E
Awesome
I'll
just
post
a
link
to
it
here.
C
Okay,
great,
do
we
have
Jeff?
Do
you
want
to
give
us
an
update
on
just
some
of
the
next
meeting
times
and
we'll
expect
this.
A
C
A
Thanks
for
everyone
for
making
it
thanks
for
getting
up
early
Caleb
yeah,
so
this
is
our
new
post
DST
time
next
on
the
15th
will
be
Amia
friendly,
the
regular
time
29th
is
canceled
and
then
12th
will
be
the
same
time
that
we
are
at
right
now
and
then
we'll
continue
to
alternate.
A
The
the
opennesses
of
calendar
should
be
updated
through
the
next
DST
change,
which
then
we'll
probably
figure
out.
What
we'll
do
then.
C
Awesome,
thank
you,
Jeff.
Okay,
if
there
are
no
other
agenda
items,
I
see
Randall
is
here
today.
Do
you
think
we'll
do
you
have
any
updates
for
us,
and
would
you
like
to
spend
some
time
picking
up
the
conversation
from
two
weeks
ago,.
H
Yeah
actually
I
do
have
updates
and
I
I
actually
need
to
come
back
to
this,
because
I
was
kind
of
pulled
off
by
open
ssf
Powers,
because
I
had
to
get
some
other
plans
situated
that
are
finalized
this
week.
But
I
should
I
like
I.
Pretty
much
have
a
working
demo
at
this
point,
so
I
just
I'm
at
the
DMV,
so
I'll
send
it
on
Slack
a
little
later,
but
yeah,
but
I
do
have
a
working
demo
where
I
am
posting
the
results
in
a
GitHub
comment.
H
C
Awesome
awesome
well,
thank
you
so
much
for
joining
from
the
DMV,
probably
the
the
last
place
I'd
be
able
to
to
do
anything
productive.
So
I
really
appreciate
that
and
cool
well
yeah
yeah.
Please
send
it
over
and
then
maybe
what
we
can
do
then
one
of
our
next
meetings.
We
can
do
a
more
kind
of
thorough
walk
through
and
and
talk
about
it
so
yep.
H
Adjust
David
for
the
record
the
way
that
we
have
the
GitHub
repo.
It's
it's
not
numbered
we're
trying
to
actively
not
get
people
caught
in
the
numeration.
So
I
I
made
a
specific
point
that
I
try
to
keep
it
as
neutral
as
possible
because
we
tried
we
originally
discussed
about
turning
it
into
a
set,
as
opposed
to
a
list
like
an
ordered
list.
So
yeah.
C
D
One
of
the
the
ironies
of
APAC
time
thinking
about
Caleb
being
up
at
six
o'clock.
If
I
was
in
Western
Australia,
this
would
be
3am.
It's.
D
D
C
Okay,
wonderful
I
was
just
looking
at
some
of
our
some
of
the
things
under
future
topic
ideas.
The
one
thing
that
we
kind
of
keep
on
going
back
to
when
I
was
being
this
an
11-7
meeting
for
feedback
and
objections.
C
Okay,
so
I
guess
what
we
can
do,
then,
regarding
the
charter
Jeff,
since
we're
recording
this
session
and
and
all
the
other
sessions
we'll
just
for
the
next
two
to
three
sessions,
so
that
we
get
a
good
representative
sample
of
everybody,
we'll
just
do
a
open
call
for
any
feedback
or
objections
to
the
Charter
that
is
currently
under
review,
basically
by
the
working
group
and
after
I
would
say,
probably
two
to
three
meetings
of
giving
the
work
group
and
everybody
a
chance
to
come
back
with
any
feedback
or
objections.
C
C
A
C
C
Definitely
have
been
doing
it
kind
of
unofficially
but
I.
Think
because
we've
been
doing
it.
Unofficially,
it's
kind
of
dragged
a
little
bit
so
so
I'm
gonna
go
ahead
and
bring
up
the.
C
I
was
going
to
bring
up
the
link
to
it
to
provide,
and
then
I
was
going
to
ask
everybody
give
everyone
a
chance
to
speak
up
if
they
had
any
feedback,
so
that'll
give
me
that'll
take
one
second.
If.
E
C
Has
any
other
topics,
while
I
pull
up
that
link
check
out
the
link?
Oh,
you
do
awesome.
Thank
you,
Jeff
excellent!
Yes,
yes,
the
pulse
58..
So,
regarding
this
pull
request,
number
58,
we
had
a
charter
template
suggested
by
Jay
White,
and
so
we're
make
doing
an
open
call
over
the
next
couple.
Work
group
meeting
working
group
meetings
to
provide
the
working
group
the
opportunity
to
speak
up
if
they
have
objections
or
thoughts
or
are
okay
with
the
charter.
C
So
we
have
now
until
the
end
of
the
meeting,
I
would
say
for
any
thoughts
or
feedback
on
that
it
looks
like
Randall.
You
have
your
hand
up.
H
I
think
it
would
be
a
good
opportunity
on
the
charter
to
clarify
what
we're
doing,
because
I
feel
like
a
lot
of
different
working
groups,
because,
as
you
know,
I
kind
of
cover
around
a
lot
of
different
working
groups,
there's
kind
of
somewhat
of
a
misconception
of
exactly
what
we
do
here.
C
H
H
A
little
bit
of
a
different
understanding,
depending
on
who,
you
ask,
for
example,
whether
we
have
a
list
or
it's
a
set
or
how
can
that
be
used
because
there
are
other
working
groups
and
users
comes
to
mind
and
also
Jay,
whites
and
Adrian's
working
group
I
forget
what
they
call
it,
but
I
I
believe
there.
There
was
talks
about
the
list
that
we're
going
to
be
producing
or
the
output
of
our
efforts
and
how
they
wanted
to
go
about
using
that
and
I
know.
H
In
the
past
there
have
been
different
people,
including,
for
example.
There
is
a
conversation,
that's
happening
with
a
rune
from
Intel,
because
he
had
an
idea
of
kind
of
surveying
everybody's,
like
companies
coming
together
and
doing
kind
of
a
collaborative
survey
of
like
what
dependencies
you
use
and
he
wanted
to
know.
If
that
was
useful,
I
recommended
him
to
approach
you
Amir
or
Jeff,
to
come
talk
to
this
working
group,
but
I
don't
know
if
they're
ready
to
do
that
yet,
but
I
know
that
that's,
for
example,
their
understanding
and
I
yeah.
G
H
Yeah
and
also
maybe
like
what
our
intended
output
is
and
what
the
intended
use
of
that
output
is
because,
as
I
said,
there
is
a
slight
confusion
over
like
what
we've
talked
about
as
a
group
and
the
progress
we've
had
as
a
group
and
like
putting
together
this
set
of
critical
packages
and
kind
of
everybody's
ideology,
because
there
there
is
kind
of
this
big
misconception
that
to
a
certain
degree
we
are
a
group.
H
That's
going
to
sit
here
and
nitpick
who's
going
to
be
on
the
top
ten
and
yeah,
like
I
I've,
had
to
explain
numerous
times.
That's
not
really
what
we're
going
for
mm-hmm
so
and
for
to
some
people,
they're,
really
surprised
and
they're
like.
But
why
aren't
you
and
then
it's
like?
Well,
you
would
have
to
come
to
the
working
group
to
understand
why
there's
certain
things
we're
trying
to
avoid
so
yeah
I
I.
C
H
C
A
Yeah
thanks
so
yeah
great
Point,
Randall,
I
I
do
think
there
is
a
differentiation
between
the
goals
and
scope
of
our
working
group
and
the
goals,
and
you
know,
stated
scope
of
projects
or
sigs
within
the
working
group.
So
I,
you
know
for
the
current
project,
we're
working
on
the
set
of
critical
critical
projects.
I
I
think
all
those
things
should
be
defined
and
and
written
down,
and
that
should
be.
A
You
know
explicitly
stated
as
this
is
what
this
project
is
about,
but
I
don't
know
if,
like
it
makes
sense
to
limit
the
working
group
scope
to
that,
because
it
can
include
other
things,
but
I
think
that
that
is
a
upper
discussion,
as
always
like
with
the
group's
goals-
and
you
know
the
Top
Line
things,
so
you
know
right
now
we
have
we've
talked
about
a
couple
times.
We
can.
We
can
always
it's
been
a
it's
been
a
long
time
since
we
talked
about
it,
but
we
can
always
revisit
it.
A
As
you
really
noticed,
only
have
the
two
two
goals
identify
critical
open
source
projects
and
secure
them,
which
are
which
are
copied
from
the
readme
into
the
charter,
is
kind
of
like
the
main
edit
we
had
to
do
to
the
Charter
but
yeah.
If
we
want
to
get
a
little
bit
more
specific,
there
I
think
that
that
would
be
okay,
but
we
do
kind
of
have
those
two
places
where
we
can
state
things
and
it
doesn't
always
have
to
be
the
the
charter.
A
G
Agreed
sorry
for
coming
in
late
to
this
and
and
possibly
already
covered
this,
but
one
of
the
challenges
is
that
different
people
will
Define
criticality
differently
and
we
should
not
ignore
that
or
pretend
we
have
one
choice.
Oh
no
put
it
this
way.
One
choice
is,
you
could
say,
we're
a
highly
opinionated
Bunch.
We
could,
you
know
a
solid
in
our
convictions
with
a
lot
of
science
behind
it,
and
here
is
the
algorithm
for
determining
your
criticality
score
to
four
decimal
points.
G
You
could
also
say
here
are
the
various
factors
one
may
consider
relating
to
that.
Here's,
a
spreadsheet
in
which
you
can
plug
in
your
weights
for
your
different
variables
and
come
up
with
your
own
criticality
scores
or
here's
the
three
different
groups
with
valid
methodologies
and
they
come
to
different
scores
and
here's
the
overlap
and
here's
the
Divergence
between
them
all
right,
but
I,
think
I,
think
my
personal
take
is
recognizing
the
diversity
of
different
opinions
out
there,
how
to
do
it
and
and
where
the
real
data
is
behind.
E
G
You
know,
and
you
need
the
asterisks
next
to
quantifying
that,
but
the
billions
are
tens
of
billions
of
dollars
in
Lost
productivity
from
the
log
for
Shell
vulnerability
could
have
been
prevented
if,
but
for
a
hundred
thousand
dollar
again
Asterix
investment
in
a
secure
code
review
of
that
code
and
that
a
graceful
remediation
process
it
would
have
ever
had
been
found
and
think
about
the
leverage
of
that
now.
G
It's
kind
of
like
Drake's
equation
for
figuring
out
how
many
civilizations,
conscious
civilizations,
are
out
there
on
the
planet
right
the
devil's
in
the
details,
but
if
I
could
say
here's
the
200
or
500
to
10
000.
You
know
that
matter
most
and
have
real
credibility
behind
that
rigor
and
and
people
who
say
point
to
that
and
say
yeah
that
makes
sense
or
that
angle
of
looking
at
it
makes
sense.
Then
then,
that's
that's
really
essential
to
making
that
claim.
E
C
C
Excellent,
oh,
it
looks
like
we
have
some
hands
up
Randall.
Please.
H
If
you
don't
want,
jocks
can
go
first,
because
mine
might
be
my
it's
it's
a
there.
There
is
a
point
of
that.
I
want
to
make
and
bright
kind
of
touched.
D
On
it
so
sure
I'll
I'll,
say
two
things:
I
guess
the
first
one
is
I'm
still
pretty
keen
on
the
idea
of
SIA
that
I
floated
a
while
back
if
you're
not
familiar
Brian.
That
was
an
effort
to
build
a
a
program
and
also
a
program
so
to
speak,
of
collecting
direct
estimates
of
impact
and
probability
from
experts
on
particular
projects
as
an
adjunct
to
the
criticality
score
effort,
just
recognizing
those
things
that
will
never
be
able
to
factor
into
a
purely
data
driven
operation.
D
So
that's
one
thing
so
I'll
put
that
to
the
side.
The
other
thing
that
I've
I've
started
off
very
recently
and
very
recently,
I
mean
as
of
this
morning.
Thinking
of
us,
like
the
reference
case
for
an
open
source
disaster,
is
what
happens
if
the
water
supply
of
a
major
city
shuts
off.
D
If
there's
no
water
supply
for
three
days
five
days
two
weeks,
it
would
be
completely
catastrophic
if
that
happened
in
New.
York,
like
the
global
economy,
would
take
a
measurable
hit
if
it
happened
in
London.
If
it
happened
in
Paris,
if
it
happened
in
Sydney,
if
it
happens
in
a
major
city,
if
it
happens
in
New
York
Washington
San
Francisco
Los
Angeles
Chicago,
there
will
be
a
lot
of
Regulation
introduced
and
we
won't
like
all
of
it
right.
We
would.
We
would
have
a
post-9
11
situation.
D
That
would
be
that
kind
of
massive
shift
in
the
gravity
of
the
universe
around
the
legislation.
We
have
to
take
our
shoes
off
every
time
we
fly
for
the
last
20
years,
because
some
put
some
and
you
know
tiny
amount
of
explosive
in
his
shoe
and
we're
gonna
have
to
do
that
forever
and
something
quite
as
spectacular
could
happen
to
us.
So
that's
that's
now
my
my
reference
case
for
what
does
a
catastrophe?
Look
like.
H
But
what
I
wanted
to
say,
Amir
and
Brian
and
Jeff,
because
this
kind
of
ties
in
a
lot
of
points.
So
in
the
past
there
have
been
things
that
have
been
shown
to
me
that
basically
kind
of
stipulate
that
this
group
or
somewhere
in
open
ssf,
there's
a
group
that
is
ranking
packages
or
like
specifically
choosing
packages.
So
when
I
explain
that
we've
decided
to
make
it
a
set,
because
the
way
or
what
is
critical
to
one
person
may
not
be
critical
to
another.
H
So
we
really
don't
want
to
get
into
what
is
like
critical
critical.
So
we
kind
of
made
a
set.
There
has
been
kind
of
a
little
bit
of
not
great
approval
if
I,
to
put
it
lightly
to
that
idea,
because
they
think
that
we're
actually
producing
a
list.
So
maybe
going
to
what
Jeff
said.
Maybe
at
some
point
we
will
produce
a
list.
H
I'm,
not
I,
don't
know,
but
I
think
that
there
are
plans
that,
like
I
know,
John
Meadows
has
one
and
I
said
Adrian's
plan
where
to
some
degree
they
were
expecting
that
we
were
going
to
produce
something
of
A
ranked
list
because
in
certain
open
ssf
publication
you
can
almost
read
that
at
some
point
it
was
thought
of
that
there
was
going
to
be
project
selected
as
critical,
so
I
I
I've
only
been
here
about
a
year
a
little
over
a
year.
So
I
missed
that
beginning
part.
So,
like
yeah.
H
D
So
this
is
my
recollection
of
how
we
got
to
the
point
of
talking
about
a
set,
which
is
that
we
want
to
get
to
the
point
where
we
can
rank
on
some
agreed
measure
or
some
agreed
technique,
so
I
obviously
had
a
horse
in
this
race
and
I
felt
that
it
was
always
possible
to
arrive
at
a
ranking,
but
that
that
ranking
would
be
particularly
dense
around
certain
parts
like
once
you
got
past,
say
the
top
ten.
D
But
but
my
understanding
is
that
we
we
went
down
the
road
of
a
set
because
we
wanted
to
be
like
tactical
before
we
got
to
the
point
where
we
said.
Yes,
we
stand
behind
this
as
an
audit
list
right,
because
if
we
say
we
stand
behind,
this
is
not
a
list.
The
very
fair
question
is:
why
do
you
stand
behind
it
in
this
order?
D
What
is
what
is
your
criteria,
disorder
and
at
the
moment,
it's
basically
like
we
want
to
push
things
that
are
both
threshold
into
people's
visibility
as
soon
as
we
possibly
can,
and
that
argues
for
a
for
a
set
rather
than
going
to
a
list
right
now.
But
the
the
list
is
not
out
of
the
question
forever.
A
Yeah
and
it'll
just
say,
like
the
you
know,
kind
of
the
answer
to
that
is
like
we
should
we
should
kind
of
keep
our
project
documentation
up
to
date,
like
I,
just
pasted
a
link
to
kind
of
where
we
have
maybe
kind
of
the
previous
version
of
this
project,
but
we
maybe
we
should
have
the
the
current
version
or
and
also
the
you
know
in
the
readme.
A
We
call
out,
let's
see,
like
our
current
projects
here,
paste
that
in
the
chat,
but
we
could
have
a
like
a
short
list
here
and
essentially
you
know
if
anybody
thinks
that,
like
you
know,
has
a
misconception,
we
can
point
them
to
here
and
say
please.
You
know,
please
join
the
working
group.
If
you
disagree
right.
A
The
other
thing
is,
you
know
if
you've
heard
people
thinking
are
we
saying
a
project?
Is
yes
or
no
critical
yeah?
We
were
doing
that
like
it's
either
in
or
out
of
the
set.
So
that
is
exactly
what
we're
doing,
but
we're
not
we're
yeah
we're
just
not
doing
the
ranking,
or
at
least
with
this
iteration.
H
A
Right
yeah,
so
we're
we're
trying
to
pick
a
number.
That's
helpful
to
the
most
people
since
we're
not
like.
If
we
do
the
top
200
We're
not
gonna,
be
able
to
tell
you
what
the
top
100
is
from
that
right.
So
we're
just
trying
to
pick
a
number
that's
going
to
be
useful
or
if
they
take
the
whole
list.
C
We
did
yes,
we
we
definitely
started
writing
some
of
this
down
and
I
and
I
totally
agree
with
what
everyone's
saying
and
that
yeah.
It
definitely
makes
sense
to
write
down
in
our
methodology
how
we
got
to
this
and
then
so
those
are
being
uploaded
to
the
repo,
but
they
are
all
available
and
in
our
notes,
kind
of
at
the
top
and
under
those
key
documents.
C
But
yes,
totally
agreed
and
I
I
think
maybe
then
and
then
two
other
things
kind
of
explaining
our
methodology
I
like
how
Vicky
kind
of
conceptualized
it
that
what
we're
kind
of
doing
first
is
kind
of
determining
you
know
like
100,
100
greatest
hits
of
all
time,
and
then
you
know
once
we
start
to
prioritize
analyze
breakdown,
we
can
determine
you
know
the
hundred
greatest
rock
hits
of
all
time.
C
100
years
pop
hits
of
all
time
and
kind
of
I
think
kind
of
similar
to
how
criticality
score
does
it
as
well.
So
that's
kind
of
I
think
the
justification
for
or
or
a
conceptualization
of
kind
of
what
our
approach
is.
We
want
to
first
kind
of
shine
a
spotlight
on
you
know
this
set,
and
then
you
know
from
there
be
able
to
you
know,
prioritize
and
rank
it,
and
what
have
you
and
then
going
back
to
some
other
thoughts?
I
mean
I.
C
I
really
love
the
idea
of
of
if
we
can
make
it
a
process.
That
is
a
kind
of
like
a
living
breathing
thing
where
we're
considering
a
lot
of
different
inputs.
C
I
think
we
definitely
could
incorporate
things
like
you
know
serving
you
know
what
are
Intel's
top
dependencies.
What
are
Google's
top
dependencies
Etc
in
the
past
I
like
I,
can't
remember,
who
mentioned
arun's
idea
about
that.
C
But
it's
I
think
it's
a
great
idea
and
concept
I
think
it
just
the
execution
of
it
is
going
to
be
tough,
because
you
know
there's,
you
know,
don't
necessarily
want
to
reveal
everything,
that's
in
your
stack
and
etc,
etc,
but
I
like
the
idea
of
if
we
have
kind
of
a
living
breathing
set
where
you
know
because,
like
we
were
saying
you
know
things
change
over
time,
Technologies
change
over
time.
The
landscape
changes
over
time.
C
So
if
we
can
create
the
process
in
a
way
where
it
evolves
with
it,
you
know
people
will
be
able
to
continue
using
it
instead
of
just
you
know,
on
December
1st
2022.
This
is
what
we
thought
you
know
were
the
most
critical.
C
You
know,
projects
on
the
ecosystem
and
then
I'm
going
to
Brian's
point
about
you
know
the
different
kind
of
views
or
definitions
of
criticality
I
think
it
would
certainly
make
sense
if
we
were
able
to
at
least
agree
on
maybe
like
a
high
level
definition
of
it
where,
whereas
you
know
we
can
explain
the
reasoning
behind
it,
but
then,
of
course,
you
know
with
that
idea
that
you
know
this
is
all
open
to
interpretation
depending
on
what
you're
using
you
know,
this
might
not
apply
to
you
as
much
but
I
feel
like
we
could
get
to
a
place
of
certain
projects
where
you
know
it
doesn't
matter
what
company
you
work
for
or
where
you
live
in
the
world
or
what
have
you
like?
C
These
are
critical
to
everybody,
but
I.
Don't
know
if
that's
too
I
don't
know
if
that's
too
high
level
or
even
possible
so
but
definitely
I,
think
yeah,
maybe
if
we
spent
some
time
specifically
working
on
this
kind
of
like
almost
like
beginning
with
the
end
in
mind.
So
if
we
start
with
like
what
would
we
output
to
open
ssf
as
a
whole,
you
know
like
a
summary
document
explaining
our,
like.
C
You
said
our
intent
and
what
we're
trying
to
do
what
the
output
is,
make
some
basic
definitions
and
basically
show
how
we
arrive
to
where
we
did
kind
of
show
our
methodology
and
then
say
you
know,
have
this
output
of
of
projects
I
think
we'd
be
in
a
good
place,
because
you
know
I
definitely
want
to
balance.
C
You
know,
making
it
as
good
as
it
possibly
can
be
and
as
accurate
as
it
possibly
can
be
versus
you
know
letting
perfect,
be
the
enemy
of
good
and
getting
something
out
that
people
can
use
and
give
us
feedback
on.
So
I
want
to
definitely
try
and
walk
that
fine
line
between
the
two
so
yeah.
These
are.
C
These
are
all
very
good
points,
very
good
ideas,
so
so
Brian
do
you
have
any
immediate
thoughts
as
to
kind
of
how
we
should
approach
this
in
terms
of
timelines
and
maybe
setting
some
soft
deadlines
for
us.
You
know
to
kind
of
help
us
keep
us
on
the
target
and
so
forth.
If
you
have
any
thoughts
there,
I.
G
Can't
think
of
any
external
triggers
for
pressures
for
coming
up
with
with
answers
for
these
I
think
it's
just
you
know,
there's
a
perennial
ask
for
what
are
the
most
critical
projects
or
what
is
your
philosophy
towards
towards
developing
them?
So
you
know
I
I,
nor
do
I
know
of
like
okay.
Here's
a
big
budget
item.
G
That's
going
to
ask
for
the
10
most
critical
projects
to
go,
do
security
audits
of
all
of
them
or
anything
like
that,
but
I,
I,
I,
think
yeah,
I
I,
don't
have
any
external
forcing
function
for
that.
You
know
we
have
our
end
of
year
year
kind
of
reporting
right
now,
I
believe
you
all
submitted
a
report
from
the
working
group
for
the
end
of
your
report
for
openssf.
That's
the
only
deadline
I
can
think
of
relevant
to
this
working
group,
but
I
I,
no
I
I.
Just
think
this
is
stuff.
G
That's
highly
useful
for
us!
I!
Don't
know!
If
we're
gonna
do
a
follow-up
to
the
census.
2
work
with
Frank
I
haven't
Frank
Nagle
at
Harvard.
We
haven't,
as
far
as
I'm
aware
touched
upon
that,
but
Hillary
might
Hillary
Carter,
who
runs
all
of
research,
might
be
running
that
no
I,
don't
think
of
any
other
dates
that
that
force
or
suggest
a
certain
certain
Pace.
Okay.
E
C
Then
I
would
then
what
I
would
recommend
we
do
is
maybe
just
make
a
list
of
all
the
things
that
we
should
be
doing
and
considering
and
then
just
prioritize
them
and
go
through
them
one
by
one.
So
I
really
like
the
I
think,
first
and
foremost,
we
should
probably
start
with
like
a
high
level
document
or
I
mean
we
already
have
one
so
maybe
build
on
that
work
on
that
together
as
a
group
to
get
it
to
a
good
place
so
that
you
know
everything
that
we
build
off.
C
Of
that
we
have
that
high
level
document
as
a
reference
point
and
I
think
that'll
help
us
a
lot
too,
with
kind
of
what
we
were
talking
about.
You
know
defining
things
listing
or
explaining
explicitly.
You
know
what
our
intent.
That
of
this
is
and
clarify
what
we're
doing
and
what
the
outputs,
what
folks
can
essentially
guide
folks
who
are
ingesting
this
as
well?
You
know
how
to
how
to
ingest
this
and
and
understand
it.
C
So
I
think
that
would
make
the
most
sense,
starting
with
a
high
level
document
and
then
maybe
in
concurrence
with
that,
then
we
would
start
working
on
a
on
your
demo
on
Randall's
demo
and
and
look
for
ways
to
to
build
off
of
the
the
MVP
that
we
currently
have
feed
that
into
the
high
level
document,
and
vice
versa
and
and
and
then
we
can
go
from
there.
Oh,
we
have
a
hand
up
random.
H
I
was
gonna,
say
kind
of
what
you
said
that
maybe
also
thinking
about
what
our
first
deliverable
of
that
repo
ecosystem
is
going
to
look
like
everybody's,
going
to
check
in
a
markdown
list
with
the
packages
we
had
before
and
start
with
that.
So
like
first
pull
requests
would
just
add
package
101
or
whatever.
It
might
be
a
thought.
C
Okay,
that
sounds
good,
so
then,
what
we'll
do
then
is
for
next
meeting.
We
will
set
dedicated
time
to
work
on
the
high
level
document.
A
Yeah
I
just
have
an
idea
in
preparation,
for
that
should
we
should
we
like
version
our
lists
or
sets
like,
should
we
take
what
we
have
and
call
it
V1
yes
and
then
make
into
our
GitHub
repo
two
directories
under
that
you
know
take
over,
we
have
put
it
in
a
V1
directory,
create
a
V2
directory.
D
D
A
To
stay,
I
was
so
so
the
thing
is
is
like.
Amir
was
proposing
last
meeting
that
maybe
we
do
like
a
quick
Refresh
on
our
old
list,
so
we
would
do
that
today,
but
we
would
call
that
V
1.1
or
something
yeah
but
I
agree
like
I,
don't
know
yeah.
It
needs
to
make
sense.
H
I
I
know
that
it's
a
request
when
of
all
these
things
that
eventually
we
have
like
a
Json
or
a
yaml
file
for
other
uses.
So
what
I
you
was
gonna
do
with
like
the
glossary
was
I
was
just
going
to
generate
on
GitHub
releases
out
of
the
markdown
and
then
generate
version
numbers
that
way.
A
Yeah,
but
all
of
this
GitHub
you
know
generated
with
is
this
is
a
new
process
so
like
we
have
two
like
completely
orthogonal
processes
for
creating
lists
or
sets
that
we
need
to
differentiate
between
the
old
way
in
the
new
way
so
yeah
for
the
new,
like
you
know,
for
for
everything
based
on
our
automation,
ingestion
engine,
we
can
have
releases
of
that,
we
can
have
dated
releases.
Maybe
that
makes
more
sense,
but
I
think
we
need
like
a
major
version
or
major
process
way
to
differentiate
between
the
two.
C
Yeah
I
I
think
that
makes
sense,
and
that
might
actually
even
be
a
good
exercise
too,
where
you
know
if
we're
gonna,
if
we
want
to
create
and
essentially
come
to
an
agreement
on
a
like
a
high
level
document,
you
know,
maybe
we
could
start
with
documenting.
You
know
what
we
did
for
D1
as
you're
referring
to,
and
then
you
know
include
that
with
the
directory
of
that
version,
that
would
give
I
think
like
like
basically
like
a
readme.
You
know,
so
that
would
give
a
lot
of
context
to
folks.
C
Excellent
okay,
yeah
yeah.
This
looks
really
good
and
I
I
mean
I'm
I'm,
more
partial
to
us
using
work
group
time
to
kind
of
do
this
stuff,
as
a
group.
Does
that
in
general
work
for
everybody
or
do
you
think
it
makes
sense
to
try
and
do
some
of
this
stuff
asynchronously?
H
C
Yes,
yes,
great
thinking,
okay,
cool
cool,
so
then
so
yeah.
So
we'll
definitely
set
dedicated
time.
Then,
for
our
next
meeting
to
to
start
with
yeah
like
a
high
level
document
iterate
on
the
the
process,
kind
of
to
find
some
more
key
things
that
we're
working
on,
write
it
down
because
that'll
help
as
well
with
feeding
into
the
charter
and
then
we'll
also
do
some
as
part
of
that
some
GitHub
updates
and
refresh
and
as
well
as
the
start.
Looking
at
the
MVP
for
the
ingestion
engine.
C
Does
that
sound
good
to
everyone?
Okay,
wonderful!
So
we've
got
about
eight
minutes
left
I
thought.
If
anyone
has
any,
we
could
use
this
as
an
open,
Forum
time
discuss
anything
on
people's
minds
or
any
updates
or
just
general
discussion
related
to
the
work
group.
B
Yeah
I
have
one
thing:
someone
mentioned
sorry
sorry,
you're
about
to
get
a
minute
back.
B
I
think
Brian
was
talking
about
engaging
with
census
to
people
they
may
or
may
not
be
interested
in
the
fact
that
we've
got
criticality
School
productionized,
like
as
in
a
research
engine,
a
vehicle
for
researchers
to
attach
stuff
to
to
be
able
to
do
broad
scale
analysis
across
at
the
moment,
GitHub,
unfortunately,
but
hopefully
in
the
future
more
than
that
yeah.
B
So
that
was
that
was
one
idea
where
I've
been
tossing
around
internally,
that
this
is
a
an
Avenue
for
researchers
as
well
be
good
to
be
able
to
engage
with
them.
C
Absolutely
yeah
and
now
that
you
bring
up
census
too
there's
one
thing:
that's
been
burning
on
my
mind
as
well
is
the
report
I
think
specifically
calls
out
or
mentions
that
I'm
gonna
totally
butcher
the
the
statistics?
So
don't
quote
me
on
it,
but
it
was
something
like
80
of
development
in
open
source
is
done
by
like
136
maintainers,
or
something
like
that.
C
It's
it's
one
of
those
statistics
that
show
you
know
the
80,
20,
Rule
and
and
I
wonder
like
they
must
I
guess
know
who
those
maintainers
are
if
they
were
able
to
have
that
statistic
and
I
know
like
a
lot
of
working
groups
have
been
trying
to
engage
with
maintainers
and
like
help
maintainers
of
critical
projects,
and
you
know
even
our
working
group.
You
know
we've
talked
about.
How
can
you
know
how
can
we
help
secure
these
projects?
C
So
I
I,
wonder
yeah
I,
wonder
if
there's
there's
a
way
to
to
iterate
on
or
get
access
to
some
of
the
data
from
that
off
of
census
2
because
it
was
a.
It
was
a
great
effort
so
yeah.
If
there
are
any
updates
on
that
Brian.
Please
keep
us
updated,
we'd
love
to
know
how
things
are
going
there.
G
C
B
A
related
thing,
I've
I've,
been
interested
in
the
demographics
of
Open,
Source
maintainers
because
and
I.
It
just
got
me
thinking
big
and
this.
This
plays
into
the
criticality
side
of
things
as
well.
We
kind
of
imagine
there's
these
like
people
like
how
much
of
our
open
source
security
is.
B
Teenagers
and
uni
students
writing
code
versus
people
with
lots
of
Industry
experience.
Writing
things
like
this
is
kind
of
a
question
that
I
had
in
my
head,
like
we
have
a
main
someone
who's
contributing
to
test
unit
tests
and
criticality
score,
who
looks
to
be
someone
who's,
a
teenager
which
is
great,
I'm,
really
happy
that
someone's
contributing,
but
it
got
me
wondering
like
how
how
many
like
how
what
is
the
spread
of
experience
across
open
source
maintainers,
and
how
does
that
factor
into
like
securing
projects
anyway?
C
G
Was
a
survey
of
containers?
What
would
you
things
like?
What
would
you
really
want,
and
the
overwhelming
answer
was
more
time
for
my
employer
to
work
on
open
source
code?
The
second
one
I
think
was
tools
that
helped
make
my
code
be
more
secure
by
default,
so
validation
we're
on
kind
of
the
right
path,
but
but
it
was
more
more
narrative
than.
B
Well,
so
a
self-selected
survey
is
like
take
with
a
grain
of
salt,
I
suppose
as
well.
That's.
C
Well,
thank
you,
everyone
for
the
great
discussion
today
the
insights
and
feedback.
Thank
you
again,
Jacques
for
everything
and
we
look
forward
to
seeing
what
great
things
you're
up
to
and
if
you're
ever
got.
If
you
ever
have
some
free
time,
you
know
you're
always
welcome
to
pop
in
and
yeah
I
hope
you
all
have
a
great
rest
of
your
day
rest
your
week
and
we'll
see
you
soon.