►
From YouTube: Securing Critical Projects WG Bi-Weekly (June 30, 2022)
A
Indeed,
on
this
beautiful
last
day
of
jan
of
june,
I
almost
said
january
wow.
A
Is
this
your
first
workbook
meeting
yeah,
it
is
cool
cool,
well
welcome
when
we
officially
kick
off
we'd
love
to
have
you
do
a
quick
intro
sounds
good.
B
D
All
right,
I
am
entering
my
name
and
such
into
the
document,
which
I
think
is
the
same
one,
we're
all
using
I'm
gonna
post
the
link
to
the
url
of
the
thing
I
think
we're
using
for
our
meeting
notes
for
the
day.
F
E
F
F
E
D
A
All
right,
if
you
do,
we
can
give
it
another
minute.
Five
minutes
past
the
hour
to
officially
kick
off.
I
don't
know
if
I
have
ability
to
record.
D
If
you
don't
it's,
it's
already
recording.
Oh,
oh.
A
Okay,
well,
it's
just
about
five
minutes
past,
so
hello
welcome
everybody.
Thank
you
for
joining
on
this
last
day
of
june.
Very
excited
to
have
you
all
here
today,
jeff,
I
couldn't
make
it
today
so
I'll
be
I'll,
be
facilitating.
We've
got
some
couple
things
on
the
agenda.
A
As
always,
we
like
to
start
with
new
faces,
and
I
know
we
have
at
least
one
and
we'd
love
to
hear
from
folks
who,
if
this
is
either
your
first
time
or
one
of
your
first
meetings,
to
introduce
yourself
kind
of
what
interested
you
about
this
work
group
and
kind
of
yeah
anything
anything
of
that
nature,
so
I'll
I'll
start
off
with
chapman.
Who
is?
It
is
his
first
meeting
here
today.
C
I
work
at
bloomberg,
I'm
on
the
foundational
infrastructure
team
providing
infrastructure
for
securing
like
s-bombs
and
cve
scanning
in
my
free
time,
I'm
a
pretty
active
contributor
to
encores
products.
So,
like
sift
and
gripe
yeah,
I'm
just
trying
to
be
a
fly
on
the
wall.
I'm
gonna
attend
all
of
the
working
group
meetings
just
to
get
a
feel
for
opens.
The
stuff
and
kind
of
you
know
where
I
could
fit
in
bus,
but
just
hoping
to
be
here
and
learn.
A
I
had
the
pleasure
of
meeting
some
of
your
your
bloomberg
colleagues
in
austin
and
so
yeah,
there's
they're
great
folks.
You
got
over
there.
Very
lively
group.
Yes
welcome!
Do
we
have
anybody
else
that
would
like
to
to
go.
A
Okay,
yeah,
it
looks
like
everyone
here
looks
familiar
so
yes
again.
Thank
you
again
for
joining
first
thing
I
just
threw
on
the
agenda
is
just
a
quick,
open,
ssf
day
recap
from
last
week.
I
thought
it
went
really
well
julia
caleb
and
I
had
the
opportunity
to
speak
a
little
bit
about
kind
of
the
the
hard
problem
we're
trying
to
solve
here
in
the
in
the
working
group,
and
I
gotta
say
throughout
the
week
I
did
get
a
lot
of
positive
feedback
on
the
on
the
panel
discussion.
A
So
thank
you,
everyone
who
was
a
part
of
that
and
helped
out
with
that
and
yeah.
I
thought
the
rest
of
open
ssf
day
went
well
too.
We
heard
from
michael
scaveta
talking
about
alpha
omega
and
a
lot
of
other
great
talks
too
on
that
day
and
throughout
the
week
in
austin.
So
if
you
didn't
get
a
chance
to
be
there,
I
believe
most
of
the
sessions
are
going
up
in
a
couple
of
weeks.
Does
that
sound
right.
D
Actually,
this
is
david
wheeler.
I
just
had
a
conversation.
We're
we're
gonna,
try
to
accelerate
getting
the
videos
up.
D
It
takes
a
little
while
because
they
record
them
in
big
blocks,
and
then
they
want
to
put
out
the
videos
for
one
for
each
speaker,
but
we're
going
to
try
to
accelerate
that,
because
so
many
people
really
wanted
to
hear
some
of
the
things
that
were
said
so
we'll
both
and
we,
I
know
for
at
least
open
ssf
day
we're
going
to
try
to
get
it
sooner.
D
I
don't
know
if
we're
going
to
be
able
to
get
all
the
other
talks
quite
that
quickly
and
I
certainly
want
to
get
as
soon
as
jacques
videos
up.
I
would
love
to
link
to
it,
but
and
doc
gave
a
great
talk
about
selecting
critical
projects
and
so
on.
So
it
was
not
just
the
how
to
do,
but
you
know
the
alternatives.
D
A
That
was
a
nice
talk
and
I'm
I'm
looking
forward
to
seeing
julia's
talk
as
well.
I
heard
some
good
feedback
on
that
one
and
sadly,
sadly,
was
at
the
exact
same
time
that
I
was
talking
so
could
not
be
in
two
places
at
once,
but
that's
the
beauty
of
recordings
right
so
hopefully
we'll
see
those
soon
and
yeah
again
great
job
to
everybody
and
thank
you
to
everyone
who
participated
and
yeah,
hopefully
by
the
next
meeting
or
two.
A
Those
links
will
be
up
so
all
right
so
and
then
we
have,
as
we
typically
do,
kind
of
our
projects
and
sig
and
related
updates.
A
If
we'd
likes,
we
could
jump
right
into
into
jacques
topic
as
kind
of
like
the
meat
of
today's
meeting
and
then
and
then
we
can
go
over
all
of
the
related
updates
and
whatever
else
we
need
to
do
towards
the
end.
If
that
sounds
good
with
you,
jacques
and
everyone
else
in
the
work
group,
I
mean
I'm
I'm
happy
to
go
through
in
the
order.
But
you
know
I'll
just
hear
the
group.
F
F
So
I
guess
that
means
I'm
jumping
up
good
times.
Yes,
so
I
linked
to
a
proposal
in
the
slack
channel
a
few
days
ago,
there's
been
some
really
great
feedback
and
discussion.
Thank
you,
david
marcus.
I
think
his
name
is
julia
dave,
great
feedback
as
well.
Thank
you.
F
So
there's
been
like
a
bunch
of
editing
that
I've
done
to
sort
of
reflect
some
of
that
feedback,
but
the
gist
of
it
is
coming
off
the
back
of
the
talk,
which
is
that
it
would
be
nice
to
have
some
way
of
eliciting
expert
opinions
directly
directly
eliciting
probabilities,
basically
not
bad
things
happening
and
how
bad
they
are
when
they
happen
and
using
that
to
formal
ranking
system
or
a
ranking
overall
of
software
project
risk
to
do
that,
there
would
need
to
be
some
effort
applied
to
developing
a
tool
to
support
that,
and
also
to
the
operational
aspects,
how
to
select
experts,
how
to
onboard
them,
how
to
train
them,
how
to
calibrate
them,
which
is
a
particular
kind
of
exercise
and
so
forth.
F
So
I
explored
a
lot
of
that.
The
thing
that's
still
like
I'm
uncertain
about,
but
I
know
that
lots
of
people
disagreed
with
me-
is
whether
a
to
reveal
identity
of
estimators,
I'm
kind
of
on
the
fence
about
that
and
the
other
one
is
whether
to
reveal
the
ultimate
ranking
to
folks
outside
of
the
open
ssf,
so
whether
whether
to
hold
the
results
of
the
ranking
exercise
closely
or
whether
to
make
them
public.
F
There
was
a
related
question
of
sharing
raw
data
with
researchers.
I
think
that's
fine.
As
long
as
that,
there
are.
You
know
a
bunch
of
guardrails
put
around
it
to
prevent
people
from
associating
estimates
with
individual
experts
in
public,
because
we
want
them
to
be
open
and
honest
with
their
opinions
and
not
think
about
what
somebody
will
think
about
what
they
think
about.
D
All
right
so
so
I
I
think
those
are
excellent
questions
to
ask.
So
let
me
let
me
briefly
pitch.
At
the
very
least,
I
think
the
results
need
to
be
public
and,
although
I'm
kind
of
on
the
fence,
I
would,
I
think
it
actually
makes
sense
to
also
have
the
list
of
experts
public
or
at
least
at
least
by
pseudonym,
if
not
by
by
real
name
for
the
louder.
The
for
the
real
issues,
though,
let
final
results.
D
D
It's
going
to
be
really
really
hard,
as
a
public
group
to
hide
what
we're
focusing
on
and
frankly,
I
think
it's
important
for
anybody
else
to
know
where
we're
focusing
on,
because
I've
already
talked
to
like
u.s
government,
and
I
might
I'm
expecting
the
same
to
be
holding
for
other
governments
around
the
world
and
for
other
large
organizations,
which
is,
we
are
never
going
to
be
able
to
focus
on
everything.
D
We
can
focus
on
some
things,
but
individual
organizations
are
going
to
need
to
figure
out
for
themselves,
what's
critical
for
them
and
I'm
expecting
they're
going
to
especially
want
to
focus
on
things
that
no
one
else
is
focusing
on,
and
they
can't
do
that
if
they
can't
do
the
set
difference
and
know
what
we're
already
focusing
on.
So
I
I
I
particularly
would
like
to
recommend
at
least
the
final
results
need
to
be
public,
and
I
see
I'm
sorry
julia.
My
eyes
are
dilated,
so
things
julia's
is,
has
a
hand
up.
E
I
always
like
to
to
do
that
just
so
that
it's
clear,
I'm
actually
trying
to
say
something.
I
agree,
I
I
agree
with
you
david.
I
think
that
you
know
trent
erring
on
the
side
of
transparency
is
good
and
allowing
people
to
filter
and
slice
and
dice
the
data
in
ways
that
make
sense
to
them
is
is
good
de-identifying.
E
The
expert,
I
think,
is
really
really
important
because
in
in
this,
in
this
day
of
the
internet,
harassment
is
a
thing,
and
so,
let's
not
paint
targets
on
anyone's
back
for
saying
that
a
beloved
project
is
yeah.
E
A
Yes,
just
to
echo,
and
then
we
can
go
to
michael,
I
do
agree
with
the
trans
being
transparent
as
well,
because
we've
run
into
that
before,
where
you
know
a
a
company
will
come
to
us
and
say
you
know:
we'd
love
to
do
security
audits
and
we're
like
okay.
Well,
what
do
you
want
to
audit
and
they're
like?
A
We
don't
want
to
tell
you-
and
it's
like
you
know
being
because
not
essentially
revealing
you
know
what
you're
dependent
on
or
or
what
your
critical
infrastructure
relies
on,
but
I
think
it
can
be
assumed
that
a
lot
of
these
projects-
you
know
they
are
critical,
so
it
doesn't
matter
which
organization
you're
with
it
does
apply
to
you,
and
it
is
critical
to
you
and
being
transparent
about
that.
You
know
would
be
just
an
another
way
to
justify
choosing
which
projects
to
do
and
which
ones
to
prioritize.
H
So
michael
yeah,
just
to
agree
to
both
of
you,
I
think,
to
state
a
different
way.
The
the
list
of
experts
should
be
well.
Perhaps
the
list
of
experts
should
be
public,
but
how
each
individual
expert
votes
should
not
right.
Okay,.
D
E
Well,
so
here's
the
thing,
I
think
we
have
to
think
about
the
unintended
consequences
of
publishing
that
list.
There
are
good
consequences
like
people
can
evaluate
and
point
out
weaknesses
and
representation
in
areas
and
industries.
F
F
One
would
be
to
make
it
opt-in
whether
your
name
is
public,
so
you
could
just
refuse
to
have
your
details
published
and
that
opt-in,
yeah,
opt-in,
the
other
one,
was
to
make
everybody
pseudonymous
like
assign
a
pseudonym
to
every
expert,
perhaps
reveal
some
basic
biographical
data,
but
we
would
have
to
be
very
careful
not
to
allow
the
identification
from
those
details.
D
Why
not
just
do
opt-in
and
then
those
who
put
their
name
or
early
pseudonyms
in
you
can
look
that
up
and
those
who
don't
we
you
know
my
guess
is
that
most
people
will
put
their
their
names
in,
but
you
don't
think
so.
Julia.
F
Yeah,
I
think
I
think
it
would
be
some
mix
right
like
to
julia's
point.
Some
people
already
feel
harassed
and
attacked
and
they
want
to
give
people
another
excuse.
I
think
one
possibility
is
that
we
make
the
names
opt-in.
F
In
that
case,
I
would
not
suggest
publishing
biographical
data,
but
instead
we
would
collect
that
data
and
publish
applications.
You
know,
20
of
our
experts,
come
from
finance,
10,
identifies
underrepresented
minorities,
etc.
That's
that's
information.
We
do
want
to
publish,
because
we
do
want
people
to
call
us
up
to
the
carpet
to
say
you
are
underrepresenting
x
or
your.
Your
pool
of
experts
doesn't
cover.
Why
does
that
make
sense?.
F
Well,
the
limit
is
probably
going
to
be
administrative
overhead.
Okay,
like
the
difficulty
of
administering
individuals
and
account
resets
and
people
saying,
did
something
wrong.
H
That
sort
of
thing,
because
I
I
guess
where
I'm
going,
is
the
larger
the
pool
set
size,
the
less
important
anyone
in
particular's
opinion
is,
and
therefore
it's
kind
of
diluted
over
the
average,
which
I
still
get.
You
know
if
for
folks
that
want
to
opt
out
or
not
opt-in,
great
like
it
should
be
fine,
like,
obviously,
we
need
to
know
who
they
are
because.
H
To
be,
it
can't
be
completely
anonymous,
yes,
so
yeah,
so
I
think.
F
One
of
the
things
to
bear
in
mind
is
that
the
the
goal
is
to,
to
a
limited
extent,
tell
that
sounds
really
really
hard,
but
it's
kind
of
true,
which
is
to
to
select
the
next
elicitation
for
the
experts.
So
while
we
would
allow
them
the
freedom
to
search
for
projects
that
they
want
to,
you
know
give
an
opinion
for
because
they're,
like
I
know
about
this
php
library-
and
I
want
to
give
give
an
opinion
about
it
when
they
run
out
of
things
that
they
want
to
do
themselves.
F
We
need
some
scheduling
mechanism,
which
basically
says
all
right.
I'm
going
to
reach
onto
the
shelf
pick
a
project
that
needs
an
expert
opinion
and
put
it
in
front
of
an
expert,
possibly
several.
So
it
wouldn't
be
so
much
that
you'd
have
a
thousand
experts,
all
the
pining
on
one
project.
You
would
be
looking
to
spread
those
thousand
so
that
you
might
have
like
two
or
three
per
project
over
time
and
one
of
the
things.
A
I
And
again,
you
know
from
the
discussion
we
had
the
the
last
time.
I
would
have
thought
that
what
we're
looking
at
is
for
something
that
seems
to
be
an
outlier
that
comes
up
statistically
based
on
scorecard
or
whatever
the
tool
right
and
then
an
expert
says,
or
you
know
somebody
who's
very
familiar
with
the
project
or
you
know
somebody
that
comes
in
takes
a
look
and
says
this
doesn't
seem.
This
doesn't
feel
right
to
me.
F
F
The
problem
is
that
there's
a
turtle's
other
way
down
problem,
which
is
that,
if
we
knew
which
things
we
should
look
at
first,
then
we
would
already
have
a
ranking
so
we're
relying
on
secondary
signals
to
say
things
like,
for
example,
the
score
given
by
criticality
is
very
high,
and
therefore
we
want
to
get
that
looked
at
quickly
to
see.
If
that
holds
true
yep.
F
I
I
I
would
have
thought
that
it
would
be
much
more
of
a
potentially
narrative
kind
of
a
thing
and
yeah
whether
or
not
my
preference
would
be
to
have
it
be
non-anonymized,
because
I
think
sometimes
you
factor
in
the
credibility
of
the
the
person
who's
making
the
comment
into
whether
or
not
you
you
think
it's
justified
or
carries
any
merit
any
weight
now,
but
I
would
have
thought
that
it
would
be
much
more
like
you
know.
F
In
in
the
prototype,
now
it
wasn't
shown
in
the
demonstration
at
the
conference.
There
is
like
a
description
field
or
a
justification
field
for
that
kind
of
narrative
information,
because
because
there
will
be
a
lot
of
it
and
it'll
be
useful
together,
but
I
still
think
we
need
some
transformation
into
a
numerical
ranking,
which
is
where
it
comes
back.
To
saying
like
we
need
to
elicit
experts,
and
then
I
went
through
all
the
options
like
markets,
voting,
etc,
and
that's
that's
how
I
landed
on
what
I
landed
on.
D
I
will
observe,
for
example,
that
in
census
2,
the
linux
kernel
is
completely
not
critical,
because
it's
not
loaded
by
npm
or
by
ipi
or
these
sorts
of
things,
and
I
think
part
of
the
challenge
is
that
there's
a
lot
of
information.
That's
not
you
know,
that's
not
visible
in
these
strictly
data-driven
approaches.
E
A
that's
a
great
point,
david,
and-
and
this
is
something
that
I
I
also
discovered
during
my
early
investigations-
into
critical
projects.
One
of
the
other
areas
that
that
people
tended
to
ignore
were
the
tools
that
people
use
to
develop.
D
E
So
like
when
I
did
my
analysis,
I
think
that
vim
and
emax
came
up
multiple
times
like
a
ton
because
it
was
installed.
They
were
installed
on
every
single
workstation
right,
whether
or
not
that
they
were
used.
So
there's
there's
a
lot
that
cannot
be
captured
through
dependency
graph.
F
I
wanted
to
bring
up
a
point
that
are
you
made
in
the
chat,
which
is
that
nothing
will
be
able
to
compare
linux
kernel
to
google
driver.
I
disagree
this.
This
is
broadly.
This
is
broadly.
The
purpose
of
of
the
approach
that's
suggested
is
to
create
that
commensurability
between
the
different
projects.
F
Ultimately,
at
the
end
of
the
day,
things
have
like
bad
things
will
happen
with
some
frequency.
That's
true
for
both
and
when
the
bad
thing
is
realized,
it
will
have
a
certain
amount
of
badness.
F
They
use
differently
different
audiences,
different
kinds
of
development,
style,
etc,
and
so,
if
you
just
lined
up
all
the
metrics
and
the
settings
side
by
side,
you
just
like
I
don't
know
so.
This
is
this
is
why
the
push
is
to
directly
elicit
the
two
values
of
like
what
is
the
frequency
that
you
think
something
bad
will
happen
to
this
project
and
when
it's
something
bad
happens,
what
kind
of
dollar
value
do
you
attach
to
that
like?
What
is
the
level
of
loss
and
destruction
that
is
attached
to
that
event?
J
I'm
not
sure
my
initial
point
was
that
you
won't
find
an
expert
who
can
compare
linux
kernel
to
google
guava,
because
right
you'll
find
two
experts
they're
important
in
each
of
their
own
space,
and
it.
F
But
that's
kind
of
the
advantage
right
like
it
doesn't
require
this.
This
is
true
of
markets
as
well,
but
it's
much
less
true
at
voting
is
that
you
don't
need
coordination
between
experts
to
come
up
with
the
ranking.
Like
I
don't
need.
I
don't
need
to
have
the
single
expert
who
can
do
both
as
I
do
with,
for
example
voting.
F
I
need
each
expert
to
be
able
to
do
pairwise
comparisons
between
projects
that
they
might
not
feel
that
they
can
do,
but
I
can
say
if
you're
familiar
with
this
or
if
you
want
to
give
an
estimate
for
this,
then
you
can
and
that's
directly
commensable
with
estimates
given
on
other
projects
by
their
experts.
D
J
I
I
guess
the
bit
I'm
a
bit,
I'm
struggling
on
it
and
like
using
google.
I
guess
using
was
it
log
kit,
maybe
like
whatever's
the
top
item
in
the
harvard
item.
J
It
feels
like
we're
going
to
end
up
with
a
whole
bunch
of
buckets
and
we
have
a
bunch
of
top
items
in
each
bucket,
but
the
the
ability
to
pay
to
put
a
dollar
value
on
the
the
impact
of
a
linux
kernel,
security
issue
and
the
impact
of
a
log4j
security
issue
both
feel
that
they
become
the
industry,
big
and
loss,
and
any
of
those
kind
of
estimates
just
become
an
exercise
in
inventing
a
big
number
based
on.
You
know
not
good
enough
data,
because
you
can't
really
see
everything
and
yes.
F
This
is
a
problem
we
have.
If
we
can
access
to
those
leading
indicators
of
loss
and
destruction,
then
then
we
could
just
use
them
right.
We
just
created
a
chronology
or
machine
learning
model
yeah.
So
that's
that's
kind
of
the
the
circle
we
keep
going
in
is
like
we
don't
we
don't
have
all
the
data
that
we
wish
we
had.
So
we
need
expert
opinion.
J
And-
and
I
guess
the
bit
I'm
getting
trying
to
get
to
and
I'm
I'm
in
morning
mode
right
like
brain,
is
still
speeding
up.
I
don't
think
experts
like
we're
never
going
to
solve
that
one
at
those
high
levels
we're
totally
going
to
get
an
ability
to
understand
within
the
php
space.
These
are
bigger
than
these
ones
and
our
automated
our
automated
findings
discovered
some
odd
things
because
of
the
one
of
them
moved
to
a
different
repo.
J
So
actually
it's
in
two
separate
repos
and
therefore
didn't
get
added
together
properly
in
the
analysis,
so
we'll
get
that
and
then
we'll.
Therefore,
obviously,
I
think
have
a
good
feel
between
obscure
php
library
versus
really
popular
perl
library
and
they'll
be
a
good
feeling.
Yes,
we've
suitably
calibrated
the
two
and
we
can
compare
across,
but
I
think
it's
gonna
be
tricky
to
compare
across
when
you're
at
the
upper
end
or
even
the
lower
end
of
each
one
right.
We're
gonna
get
a
magnitude
within
a
bucket.
B
B
Yeah
I
mean,
I
think,
that
it's
possible
to
say
with
fungibility
of
let's
say
money
that
we
can
say
this
is
important
or
that's
important
and
we
can
say
so.
Maybe
if
you're
saying
in
using
this
other
intermediate
representation
of
you
know,
I
mean
money
in
terms
of
business
impact
right
now
how
much
of
the
world
is
gonna,
you
know
explode
if
this
happens
and
okay,
we
can
say
with
that,
but
I,
but
maybe
I'm
I
mean
I
guess
this
is
sort
of
a
you
know,
pedantic.
Maybe
this
is
what
henry's
talking
about
that.
F
Yeah,
that's
that's!
That's
true
like
I
don't
think
that
there
is
a
I'll
finish.
This
comment,
then
I'll,
let
you
go.
I
don't
think
that
there's
like
this
is
sort
of
like
an
economic
question
like
the
concept
of
of
inherent
value
versus
subjective
value.
I
don't
think
that
there
is
so
to
speak,
an
inherent
value
that
can
be
sorted.
F
That's
why
we're
forced
back
to
the
subjective
value.
What
would
somebody
pay
for
it,
or
what
do
we
estimate
somebody
would
pay
for
it
or
cost
in
dollar
times,
because
those
those
are
commensable
across
projects
versus
other
things
we
can
come
up
with,
which
would
not
be
because
there's
just
so
much
variation
in
open
source.
A
Yes,
I
was
what
I
was
gonna
say
is
in
interest
of
kind
of
my
objective
of
kind
of
getting
a
an
mvp
of
of
kind
of
this
process
going.
I
if,
if
the
work
group
would
like
to
keep
talking
about
this
topic,
please
let's
do
it.
It's
a
very
important
topic
in
the
context
of
generating
a
list
of
projects.
A
We
started
kind
of
working
on
this
process
for
doing
that,
and
I,
after
looking
at
this
the
document,
let
me
share
this
document
again
with
everybody
I
would
like
to
if
we
can,
if,
if
we
can
set
some
time
for
it,
spend
some
time
working
on
the
first
phase
of
kind
of
getting
a
list
of
projects
which
I
think
is
going
to
be
essentially
having
an
ingestion
engine
or
something
where
we
can
generate
a
list
and
have
a
list
of
projects
that
we
at
least
identify
and
then
prioritize
from
that
list
kind
of
using
this
expert
seeking
expert
opinion
and
prioritization.
A
C
A
Work
on
so
I
will
I'll
I'll
I'll
offer
the
floor
to
julia
and
david
and
chris
who
had
their
hands
up
after
me
in
in
regards
to
this
discussion,
but
yeah
I'd
love
to
to
move
forward
with
kind
of
that
process
for
getting
stuff
on
paper
that
we
can
chop
down
and
prioritize.
But
with
that
I'd,
I'd
love
to
hear
from
from
julia
david
and
chris.
F
I
think
I
think,
for
the
moment.
Yes,
we
need
to
stick
with
the
ad
hocratic
methods
that
we've
used
up
to
this
point,
simply
because
it
will
take
time
to
develop
the
tooling
and
to
develop
the
administrative
muscle-
and
I
admit
that,
like
we
need
stuff
sooner
than
that,
so
you
know
I'm
perfectly
comfortable
with
that.
I
don't
see
them
as
possibly
exclusive.
F
E
Yeah
and
and
to
to
that
point
like,
let's
drive
multiple
work
streams
right,
we
don't
have
to
do
one
after
the
other.
So
I
think
that,
like
to
to
jacques
point,
we
can
we
can
still
achieve
phase
one
while
developing
a
system
to
to
seek
expert
expert
opinions.
E
The
the
thing
that,
like
I,
I
just
wanted
to
emphasize
in
terms
of
comparing
projects
is
that
I'm
really
trying
to
avoid
an
analogy
that
I
don't
want
to
make
right
now,
but
there's
the
reason
that
you
seek
out
expert
opinion
on
things
is
because
people
have
knowledge.
They
cannot
be
captured
in
data.
E
E
Figuring
out
how
to
capture
that,
in
a
way
that
can
be
incorporated
into
a
data-driven
system
is,
is
the
challenge
and
the
seer
proposal
is
one
one
approach.
But
I
also
think
that
there
that
we
should
explore
doing
a
a
kind
of.
E
The
categorized
approach
as
well
so
that
we
can
do
we
can
compare
like
like-ish
to
like-ish
but
still
arrive
at
one
list
at
the
end,
and
I
think
you
it
that's
that's
a
framework
problem.
It's
not
a
tooling
problem.
D
I
I
think
I'm
next
yeah,
so
I'm
I'm
wondering
if
a
lot
of
this
is
the
is
conflating
value
and
costs
I
mean.
Maybe
this
is
not
it's
a
problem.
I
often
say
I'm
wondering
you
know
the
you
know.
People
have
great
value,
but
I
only
I'm
going
to
pay
them
a
certain
amount
for
a
salary
you
know.
So
is
the
linux
kernel
valuable?
Yes,
is
google
guava
valuable?
Yes,
but
in
the
end
we
have
limited
resources,
so
I
think
we
need
to
find.
A
Thank
you
david
chris.
I
Yeah,
so
I
always
say
this:
I
I.
G
I
I
said
in
the
chat
and
I'll
say
it
again:
I
think
context
matters
and
it
and
and
having
there
there
is
no
getting
to
a
single
list
without
taking
context
into
consideration
again
within
the
context
of
node.
I
And,
conversely,
you
know,
why
are
people
who
are
focusing
on
go
or
rust,
gonna
focus
on
something
wrong
with
love?
You
know
so
so
my
point
is,
I
think
a
list
is
going
to
be
relevant
in
the
eye
of
the
beholder
or
in
the
context
of
where
am
I
going
to
put
my
money
now
david?
I
Maybe
this
is
where
I
don't
really
understand
how
the
the
the
the
the
funds
are
supposed
to
be
operated,
but
it
would
seem
to
me
that
you
know
if,
if
you
know,
if,
if
I
were
a
company
that
invested
very
heavily
in
sort
of
a
node
or
a
java
or
a
you
know,
go
or
a
rust
or
whatever
sort
of
ecosystem
for
the
software
that
I
produce
and
consume
that
that's
where
I
would
want
my
money
to
go,
and
so
you
know
that's
that's
why
I
think
I
said
you
know
way
way
back
when
I'm
not
sure
that
you
know
we're
ever
going
to
get
to
a
point
where
we
can
all
agree
on
where
the
money
should
go.
I
But
the
crowd-sourced
approach
to
things
would
be
a
little
bit
easier
to
address
where
people
could
then
go
through
and
say
here
are
some
of
the
things
that
I
care
about
now.
Show
me
a
list,
that's
reflective
of
my
interests,
my
concerns,
and
that
would
be
a
singular
list.
But
it's
in,
in
my
perspective
of
that,
as
opposed
to
you
know,
trying
to
come
up
with
the
global
god
view
of
what
you
know
what
the
priorities
are,
because
I
don't
think
we're
ever
get
to
that.
B
B
They
really
want
to
say
you
know,
you
know
I
mean
you
can
go
to
some
people
and
say
you
know
we're
just
doing
you
know
good
things
for
the
world
and
just
give
us
general,
but
some
you
know
victoria
say
I
want
to
I
care
about
this
and
I
want
to
you
know,
do
about.
You
know
this
thing
in
africa
or
this
thing
in
there
and
you
know,
or
you
know
this
thing
in
you
know
I
you
know
healthcare
here
this
I
mean
so
I
think
we
need
to
take
both
approaches.
B
So
I
mean
I
think
it's
a
matter
of.
We
can
definitely
go
with
the
open,
ssf
and
say:
okay
are
some
general
things
going
on
and
you
know
you
know
motherhood
apple
pie.
This
is
good.
We
need
to
do
these
things.
I
mean,
for
you,
know
amir,
and
you
know
to
sort
of
get
this
list,
which
I
completely
agree
with.
I'm
just
suggesting
to
to
have
an
open
mind
about
this,
which
will
you
know
take
this.
B
You
know
broader
view
of
both
of
these
that
okay,
we
have
general
funds
in
the
lf
and
the
opennesses
have
to
apply
to
projects
and
a
whole.
We
think
are
good
for
the
community,
and
you
know
big
donors
want
to
work
on
that,
but
there
also
will
be
and
we
can
sort
of
trade
off.
In
other
words,
if
we
have
a
donor
who
says
well,
I
really
want
to
work
on
this.
I
mean
this
is
what
I've
seen
with
boundaries
versus
many
other
things
that
I've
been
working
on
as
well.
B
Is
that
you
can
you
can
play
this
game
internal
versus
external
funds
at
ibm?
So
in
other
words,
you
could
go
to
these.
Have
this
big
pool
of
funds
and
say:
okay,
here's
our
list,
but
you
can
do,
is
well.
I
really
only
care
about
you
know
java
stuff,
or
I
only
really
care
about
node.js,
okay.
Well
now
we
can
take
your
money.
B
I
mean
it's
a
little
bit
more
complicated,
it's
balancing
but
say
we're
going
to
invest
your
money
in
the
java
stuff,
but
that
then
allows
us
to
free
up
the
general
funds
that
we
we're
going
to
have
to
spread
everywhere,
and
we
can
now
put
that
somewhere
else
that
we
think
is
important.
So
I
mean
you
know,
balancing
out
the
way,
we're
doing
the
fundraising
with
this.
This
priority
that
to
be
able
to
you,
know,
accommodate
donors
who
want
to
be
able
to.
B
D
So
I
I
had
a,
I
think,
I'm
next,
I
had
a
point
which
I
think
is
at
least
consonant
with
david
he's
a
slightly
different
view,
but
I
think
we
end
up
in
the
in
the
same
place,
when
particular
governments
or
large
organizations
have
asked
me
what's
what's
critical,
the
I
mean
the
obvious
answer
to
me.
Is
you
have
to
figure
out
what's
critical
for
you,
however,
as
you
get
larger
and
larger,
first
of
all,
organizations,
bigger
organizations
and
governments,
and
particularly
if
governments
are
looking
not
just
for
themselves
but
for
their
constituents,
their
society.
D
Good
luck!
You
know,
you
know
you,
the
I'll
use
the
us
government
as
an
example,
because
I'm
more
familiar
with
them.
The
us
government
really
has
no
idea
what's
in
a
whole
lot
of
those
critical
infrastructures
and
certainly
not
what
everyone
is
implementing
on
every
website
in
the
entire
united
states
or
even
more
what
what
u.s
citizens
depend
on,
which
is,
frankly,
global
they're,
never
going
to
get
that
either.
D
So
I
and
what's
more,
I
think
that
if
we
unioned
a
lot
of
the
different
organizations,
you
know
clearly
shopify
is
going
to
have
a
different
set
of
interests.
You
know
than
microsoft.
If
you
look
and
say
hey,
are
they
identical,
but
I
think
we're
also
expecting
as
soon
as
you
look
a
lot
of
lists,
there's
going
to
be
a
lot
of
overlap.
D
Linux
kernels,
going
to
show
up
in
a
lot
of
places,
lots
of
other
things
are
going
to
show
up
in
a
lot
of
places,
and
so,
if
we
can
focus
on
the
things
that
show
up
a
lot
of
places,
we
make
it
a
lot
easier
for
those
folks
who
are
going
to
also
identify
the
specific
items
by
by
helping
get
rid
of
broad
challenges.
A
Okay,
so
some
very
good
points
came
up
just
now
and
I
think
it
would
make
sense
to
tackle
it
as
a
as
a
group
and
do
a
consensus
on
this,
so
in
that
in
that
document,
for
you
know
the
process
for
identifying
and
creating
a
list,
I
added
a
new
section
reasoning
for
list
or
I
want
us
to
capture
like
you
said
what
is
the
who
is
the
audience?
What
is
the
goal
of
this?
F
Yeah,
please
feel
free
to
suggest
text.
I
mean
it's.
It's
set
to
suggestion
at
the
moment
because
it's
in
the
shopify
g
drive
because
we
couldn't
find
a
bucket.
F
D
D
The
idea
is
to
to
improve
and
apply
the
process
for
identical
productivity,
critical
projects.
I
would
call
that
a
list
number
two,
because
we
already
have
a
list
and
then
the
idea
would
be
maybe
use
this
seer
approach
for
list
number
three
or
version
three
I
mean
they
should
call
them
rev
three.
D
F
It
yeah
it's
like
you're,
three
or
four:
it's
it's,
it's
a
generation
or
two
down
the
road
simply
because
we
do
need
some
input
for
our
omega
and
others
sooner
than
it
would
be
possible
to
stand
this
up
as
a
production
system
and,
as
I
said,
make
sure
that
we
understand
what
administrative
overhead
may
be
involved,
because
that'll
affect
lf
staff
most
likely
and
we've
got
to
be
cognizant
of
the
lf,
the
open,
ssf
staff
workload.
F
I
don't
know
that
that
makes
it
easier
for
people
to
to
reason
about
who
are
consuming
like
they
don't
have
to
deal
with
the
list
changing
constantly.
A
Yeah,
I
was
thinking
something
very
similar
where
we
have
a
process
where,
basically
it's
like,
we
throw
a
bunch
of
stuff
up
on
the
wall
for
a
bunch
of
projects,
because,
as
we
were
talking
about
capturing
those
outliers
and
some
of
those
other
projects
is
going
to
require
that
we
get
a
diversity,
diverse
opinions.
A
So
we
get
a
bunch
of
stuff
up
on
the
wall
and
then
there's
a
process
for
categorizing
and
prioritizing
that
which
would,
I
believe,
include
your
this
year
this
year
process,
as
part
of
that
so
does.
Does
that
does?
Does
that
sound
like
we're
in
general,
consensus
kind
of
on
that
kind
of
strategy,
strategy
or
strategic
plan,
and
part
of
that
will
be.
A
I
will
create
some
some
like
a
folder
that
everyone
who's,
a
part
of
the
work
group,
can
easily
access
because
that's
been
helpful,
I
would
say
I
mean
like
for
the
document
I
created
for
us
for
the
process.
A
I
just
added
the
work,
the
work
group
to
the
document
to
be
able
to
edit
it
so
so
that
way
we
can
be
a
little
more
consistent,
but
in
general,
do
we
feel
good
about
kind
of
where
we're
headed
with
this
and
and
kind
of
what
needs
to
be
done
in
order
to
to
to
have
something
by
end
of
summer.
F
During
the
summer,
I
think
emma
ad
hoc
approach
will
be
necessary
just
because
sia
would
not
be
ready
in
that
time.
A
F
Yeah,
I
guess
one
thing
I
want
to
take
back
with
me
is:
would
we
be
like
committing
at
least
to
taking
seat
attack
say
this
is
something
we
would
like
to
get
a
blessing
for
as
a
project
like
we,
we
as
a
working
group
ourselves,
can
spin
up
projects
in
my
understanding
of
the
draft
governance
framework,
but
it's
still
only
a
draft.
So
de
facto
we
have
to
sort
of
go
to
tech
and
say
hey
we're
doing
this
thing.
F
We
hope
you
like
it
the
reason
for
that
being
in
turn
that
people
will
need
to
work
on
this
right.
All
of
us
have
things
to
do
and
for
me
to
go
back
to
my
management
and
say
I
want
us
to
work
on
this,
but
nobody
has
done
anything
on
paper
that
says
that
they'll
accept
it
is
difficult.
F
F
Discuss
it
go
through
the
questions
that
we
went
through
today,
that
sort
of
thing,
but
I
want
at
least
I
guess,
a
vibe
that
people
are
not
in
favor
of
pursuing
this
approach
so
that
I
can
at
least
you
know,
tear
up
the
conversations
with
my
management
that,
like
hey,
I'm
going
to
want
to
try
and
chisel
some
people
off
the
head
count
to
work
on
this.
D
Can
you
can
you
create
an
estimate
of
what
would
what
it
would
cost?
I
mean
if
you're
asking
if
you're
asking
for
funding,
then
the
process
is
pretty
straightforward,
create
a
proposal
hopefully
make
some
sense.
Gotta
have
a
dollar
figure
raise
it
to
the
attack
and
in
the
end,
it's
the
governing
board,
but
the
governing
board
always
wants
to
hear
the
tax
opinion.
First.
F
F
F
I
would
think
of
this
in
terms
of
myself,
plus
another
pair
and
I'll,
be
saying
like
myself
being
part-time,
because
I
got
a
lot
of
stuff
going
on
saying
like
can
I
have
them
for
six
months
or
can
I
have
them
for
a
year
and
see
how
far
we
get?
And
hopefully
you
know
start
to
attract
contributors
from
other
participants
in
the
open
ssf?
So
it's
not
just
not
just
shopify,
so
we
don't
have
a
risk
of
over
reliance
on
one
one
participant.
D
F
Kind
of
what
I
mean,
I
don't
think
it
would
take
like
super
long
right.
It's
just
that
the
devil
is
in
the
details,
as
we've
discovered
today.
There's
there's
lots
of
there's
like
the
big
feature
which
I've
already
gotten
the
prototype,
which
is
like
doing
the
monte
carlo
and
that's
the
the
shiny,
exciting
thing
that
you
talk,
talk
at
conferences
about,
but
then
there's
all
the
tiny
details
of
like
the
permissions
have
to
be
right.
So
who
can
see
what
at
what
times?
And
how
does
the
ranking
get
published?
F
D
I
worry
about
got
it,
so
I
it
sounds
like
you're
looking
for
more
of
a
commitment
that,
yes,
we
think
this
is
the
right
way
to
go,
and
I
don't
think
everybody
here
has
had
a
review
time
to
review
it.
Yet
I'm
so
sure.
F
F
D
A
As
we
always
do,
because
we
have
great
discussions,
I
will
say
I
think,
for
the
next
meeting,
which
is
on
the
14th
of
july.
A
We
can
prioritize
taking
a
a
stab
at
that
ingestion.
Engine
kind
of
list,
identified
project
list
generation,
basically,
and
talk
about
that
a
little
bit
in
more
detail
since
we
didn't
get
to
it
today,
okay,
and
then
I
understand,
we
do
still
need
to
make
some
movement
on
doing
a
either
every
other,
or
maybe
you
know
once
every
couple
of
meetings
to
be
apac
friendly
time
zones.
A
So
we
are
still
working
on
that
because
it
because
thinking
about
I
don't
know
if
it
makes
sense
to
alternate
where
you
know
every
other
meeting,
we
would
have
it
at
the
apac
friendly
time
zone.
That
might
be
easier
in
terms
of
consistency,
but
I
know,
for
example,
this
hour
is
blocked
off
for
me
now,
because
I've
been
meetings
for
for
some
time.
So
I
don't
know
if
changing
that
will
will
affect
attendance
if
having
like
regularly
different
meetings.
A
Does
anyone
have
any
a
particular
way
about
that
or
have
any
thoughts
on
what
might
work
best
for
accommodating
different
time
zones.
E
E
Oh,
I
was
just
gonna
say.
The
only
reason
I
was
able
to
make
this
meeting
is
because
my
other
meeting
was
cancelled.
So
generally,
I
would
say
alternating
mornings
and
after
late
afternoon,
it
is,
is
a
good
way
to
to
set
it
up
regularly
and
is
what
I've
done
for
other
other
working
groups.
E
It
will
impact
attendance
like
that's
just
something
that
we're
gonna
have
to
be
be
okay
with,
but
it
also
means
we're
going
to
get
people
that
we
wouldn't
otherwise.
So.
A
Yeah
and
it
looks
like
I'm
looking
at
kind
of
the
communal
calendar,
it
looks
like
on
on
certain
thursdays.
There
is
a
3
p.m,
scorecards
all-star
bi-weekly
meeting,
so
I
would
just
want
to
make
sure
we
don't
overlap
with
that
one
and
maybe
just
go
after
4
p.m,
which
which
I
think
would
work
for
the
apac
folks
and
would
not
wouldn't
overlap
the
scorecards
4pm.
A
It
just
kind
of
would
that
work
in
general
for
4
p.m.
Central
time
for
alternating
that
would
be
they'd
be
a
little
late
for
the
east
coast.
Folks,
that'd
be
5
p.m
and
then
2
p.m.
For
the
west
coast.
Folks.
D
A
Yeah,
okay,
okay,
so
then
I
think
that
kind
of
that
solves
that
so
I'll
reach
out
to
jory
and
hopefully
in
one
of
our
upcoming
meetings.
I
don't
know
if
it'll,
maybe
starting
after
the
14th,
we
will
alternate
to
to
apac
friendly
time,
sounds
good
awesome,
I'm
glad
we
were
able
to
solve
that
yeah.
Sadly,
there
are
only
two
minutes
left
if
there
are
any
kind
of
last
minute
thoughts.
A
D
G
A
A
Smile
on
a
laugh,
thank
you
so
much.
Thank
you
so
much.
Everyone
have
a
great
fourth
for
everyone
celebrating
and
see
you
all
soon.
All.