►
From YouTube: Securing Critical Projects WG Bi Weekly (June 2, 2022)
A
D
C
C
C
E
C
Okay,
well
moving
on.
I
wanted
to
give
an
update
about
the
meeting
time,
so
we
had
a
discussion.
So
so,
if
you
don't
remember
two
weeks
ago,
we
had
a
later
meeting
to
support
aipac
region
time,
and
we
discussed
the
meeting
time
then,
which
is
kind
of
unfair
to
the
people
here,
because
only
the
people
that
could
make
that
meeting
were
there.
C
So
a
couple
other
at
least
one
other
working
groups
have
alternating
meeting
times
every
other
meeting.
One
to
support
me.
I
want
to
support
apac
in
general.
We
got
thumbs
up
that
we'd
like
to
do
that,
although
we
didn't
set
the
time
exactly
to
that
time
that
we
used
last
time,
we
thought
we
would
have
a
poll
out
for
actually
for
both
times,
but
mostly
for
the
for
the
apac
time,
which
we
haven't
done
yet,
but
just
wanted
to
to
bring
that
up.
C
C
You
know
the
process
and
the
action
of
creating
the
list
and
make
it
a
more
formal
project
within
the
working
group
with
you
know,
identified
people
as
as
leaders
on
that
so
mirror
is
taking
a
stab
at
the
the
document
on
the
first
part,
which
is
essentially
just
kind
of
covering
the
the
process.
Here.
C
A
Hey
I'm
caught
off,
there
goes
I'm
caught
a
little
off
guard.
I've
been
working
on
the
like
risk,
elicitation
approach,
since
when
did
I
present
like
february
or
march,
something
like
that
yeah
and
I'm
even
doing
talk
about
it
in
in
at
the
conference.
In
a
couple
of
weeks,
I
submitted
the
video
yesterday,
some
sort
of
curious
weather
or
how
that
still
fits
in
okay.
I
just
think
you
know
I
wasn't
able
to
go
to
the
meeting
last
week
or
I
would
have
asked
about
it.
There.
C
Yeah,
I
agree
so
what
I
think.
What
I
want
to
have
happen
is:
have
you
also
be
like
a
maybe
leader
or
whatever?
I
don't
know
what
the
terms
we
have
in
this
in
this
effort
and
build
this
collaboratively
with
somebody
like
amir
or
or
whoever
else
is,
is
willing
to
do
that.
So
I
think
I
think
amir
wasn't
feeling
comfortable
with
figuring
out
how
that's
gonna
work
work
in
yeah.
A
A
C
Process
right
so
maybe
it's
like
we
decide
to
do
something
in
the
first
phase
and
that
you
know
and
then
that's
a
sooner
thing
and
then
that
comes
in
later
cool.
But
I'm
kind
of
my
takeaway
is
that
we've
got.
We've
had
a
lot
of
people
presenting
different
ideas
and
I
think
they're
they're
all
have
you
know
very
good
parts
to
them,
but
we're
not
really
having
somebody.
F
Oh
michael's,
mate
cool.
I
I
think
that
this
kind
of
all
sounds
great
in
terms
of
like
let's
experiment
and
do
something
and
then
iterate,
and
I
I
do
think
that
there's
I'm
not
sure
how
voting
on
something
like
a
hundred
projects
out.
You
know
if
there's
a
pool
of
500,
you
have
to
choose
your
favorite,
your
the
ones
that
you
would
categorize,
as
as
your
as
the
100
most
critical
to
you.
F
I
don't
know
that
humans
are
good
at
like
that
kind
of
work,
but
what
like,
whatever
it's
fine,
even
if
it
doesn't
need
to
be
perfect.
The
other
thing,
though,
that
I
that
I'd
recommend
is
thinking
about
different
verticals,
so
the
most
critical
open
source
project
to
some
subset
of
the
the
population
out
there
may
be
completely
uninteresting
to
another,
really
large
subset
so
like
if
you
look
like
automotive
or
financial
services
or
consumer,
or
things
like
that,
those
those
lists
may
be
very
different.
F
So
when
we
collect
data,
it
would
be
good
to
include
the
persona
that
that
data
kind
of
comes
from,
or
is
expected
to
to
align
with
something
that
we're
thinking
about
for.
On
the
the
alpha
omega
side
is
to
specifically
have
like
different
verticals,
where
the
the
target
engagements
or
the
target
analysis
for
one
sector
may
be
a
different
list
than
for
another
sector.
We
haven't
ironed
it
out.
A
A
F
F
But
if
you
imagine,
if
you
had,
if
you
had
good
like,
let's
just
say,
usage
data
from
a
representative
set
of
like
the
automotive
industry,
then
you
could
say
well,
these
are
the
you
know,
top
50
packages
that
run
everything
these
are.
These
are
critical.
These
are,
these
are
used
in
you
know.
Whatever
the
thing.
Oh.
D
F
G
G
I
think
one
of
the
challenges
we
we
did
a
we
did
synchronously
earlier
and
you
know
when
you're
in
a
rush
and
we
and
so
on.
There
are
reasons
we
did
it
that
way
that
time,
but
if
we
can
do
it
asynchronously,
I
think
life
would
be
much
better.
You
know
people
have
different
time
zones
on
all
the
rest
of
that
and
there
are
rank
voting
systems.
G
I'm
sure
jacques
can
tell
us
the
pros
and
cons
of
of
a
thousand
of
them,
but
if
we
have
to
compare
the
difference
between
different
counter
save
voting
systems,
you
know
I
I
I
think
we're
we're
in
the
noise
level
we're
you
know,
you
know
pick
a
plausible
ranked
system
and
move
on
and
there
are
systems
that
can
do
this
before
voting.
I
think
one
thing
is
not
clearly
captured
here.
I
don't
think
it's
inconsistent,
but
it's
not
done
here
not
clearly
spelled
out
is.
G
I
think
we
need
to
capture
not
just
from
the
original
submitter,
but
from
anybody
else
who
has
data
on
why
this
should
or
should
not
be
considered
a
critical
component,
and
then
let
people
do
the
voting
you
know
so,
hey
include
it
because
it's
in
harvard
list,
it's
number
10,
look
at
all
these
systems
that
use
it
and
somebody
else
says
yeah,
but
it's
on
its
way
out.
That's
kind
of
misleading
because
of
point
abc
and
and
then
let
folks
vote
with
a
much
more
informed
set
and
again.
H
Yeah,
I
guess
mine
is
a
combination
of
both
voting
and
and
the
comment
on
verticals,
just
obviously
big,
plus
one
on
the
verticals,
but
just
thinking
to
what
david
just
said
about
you
know.
This
is
on
its
way
out,
or
it's
a
you
know,
essentially
a
transitive
or
indirect
dependency,
and
that
can
really
be
a
key
across
verticals
and
other
things
as
to
what
the
criticality
of
it
is
because,
let's
say
version
2.7
of
something
is
secure.
H
You
know
their
governance
plans
may
not
be
as
good
as
they
should
be.
So,
how
much
of
a
factor
is
that
in
defining
what
is
the
top
100
versus
the
you
know,
it's
an
issue
and
it
affects
you
know
whatever
the
threshold
is
to
determine
it's
it's
valuable
or
it's
not
so
is
that
you
know
10,
000
or
less
potential
company,
I'm
just
throwing
numbers
out
there,
but
you
know
some.
Some
threshold
of
companies
have
to
be
affected
by
this
before
it's
considered
critical
or
what
is
that
method
yeah?
H
H
I
C
So
I
think
kind
of
that's,
maybe
not
in
this
dock,
but
in
in
our
previous
areas
around
the
read
me
and
like
howard,
critical
projects
selected,
we
kind
of-
I
don't
think
it's.
I
don't
think
it's
exactly
defined.
That's
a
good
question,
but
it's
more
implied
by
the
the
inputs
we're
using
right.
So
if
we
say
we're
looking
at
the
criticality
score
results,
then
that's
like
a
busyness
input.
If
we
say
we're
looking
at
the
harvard
census
results
towards
the
how
many
other
packages
are
are
depending
on
that.
C
I
I
I've
run
into
this
with
trying
to
define,
say,
like
safety
and
well-being
impact
in
other
vulnerability.
Prioritization
things
the
uscdc
has
stuff
about
well-being
impact.
You
know,
there's
the
iec.
I
forget
what
the
number
is
now
on.
Safety
impact,
the
international
standards
and
the
different
levels,
but
in
general,
loss
of
life
is
probably
not
commensurate
to
busyness.
I
Right,
like
loss
of
money,
is
probably
not
commensurate
with
either
of
those
two
things.
I
think
it
would
be
risky
to
ask
people
to
vote
without
defining
what
it
is
they're
voting
on,
because
if
one
person
is
like,
I
have
this
use
case
story
where
this
package
can
cause
a
loss
of
life.
If
something
goes
wrong
and
another
person
is
using,
I
have
this
business
criteria
where
this
ranks
the
highest.
Those
are
not,
they
both
might
be
right,
and
it's
not
clear
how
to
adjudicate
between
those
and
that's.
My
comment.
C
J
J
It's
unclear
to
me
that
there's
ever
going
to
be
a
singular
100
projects
that
are
most
critical.
A
lot
of
it
has
to
do
with
point
in
time.
Right
I
mean
you
know,
we
could
have
said
you
know
in
october
or
november
of
last
year.
Hey
log4j
is
fine,
and
then
somebody
oops,
you
know
we
find
a
criticality
and
then
we
realize
holy
crap.
You
know,
so
I
think
there's
a
certain
aspect
of
our
our
task
here.
J
J
You
know
number
one,
and
yet
you
know
over
the
the
holiday
week
a
couple
of
weeks
there
it
was
number
one
for
anybody
who
was,
you
know
heavily
invested
in
java
because
it
was
pervasive.
It's
everything
you
know
about
your
logging.
So
and
and
again
you
know
the
the
other
comment
that
struck
me
was
the
maturity.
You
know
the
oh.
This
one
is
sort
of
going
out
of
style
or
whatever,
but
that
was
the
case
with
log4j1.x
right
it's.
J
You
know
it
was
deprecated
five
years
ago
right
and
yet
still
people
are
using
it.
So
you
know
that
we
have.
You
know.
I
think
we
have
a
combination
of
things
that
you
know
these.
You
know
the
leading
edge
of
something
you
know
may
become
critical
at
some
point,
but
we
also
have
to
be
factoring
in
that
you
know
the
reality
of
the
software
industry.
Is
that
way
too
much
technical
debt
continues
to
agree
at
an
ever-increasing
pace
and
that's
where
most
of
the
risk
is
right
and
how
do
you
measure
that
and
is
there?
J
Is
that
a
point
in
time
thing
or
you
know,
does
that
become
when
a
when
a
project
decides
you
know
what
we're
not
going
to
be
supporting?
You
know
version
1.x
of
something
anymore,
we're
focusing
on
the
you
know
the
head
of
the
you
know
the
head
of
development,
and
you
know
you're
on
your
own,
all
of
a
sudden
that
puts
that
thing
a
little
bit
higher
than
it
was
before.
J
So
I
don't
know
that
we're
going
to
get
anything
out
of
voting
as
much
as
if
we
can
sort
of
get
down
to
here
at
the
top.
I
don't
know
whatever
and
put
enough
information
around
these
things
so
that
people
can
look
at
it
and
and
sort
of
crowdsource
the
criticality
of
you
know
to
them
right.
I
mean
I
mentioned
this
before
from
an
ibm
perspective,
I'm
much
interested
much
more
interested
in
understanding
which
of
the
projects
that
I
have
incorporated
into
the
ibm
products
and
services
offerings
are
the
ones
that
have
the
most
risk.
J
Those
are
the
ones
that
I
care
about.
Yeah
I
mean
yes,
we
do
care
about
the
other
ones.
Obviously,
but
you
know
very
selfishly.
We
also
care
much
more.
You
know
about
the
things
that
we
can
get
caught
up
on
right.
So
again,
I
I
don't
know
that
there's
ever
going
to
be
a
universal
here's,
a
hundred
things
that
everybody
has
to
focus
on.
J
I
don't
know
that
it's
worthwhile
even
trying
to
get
to
that
100,
but
we
could
come
up
with
a
system
that
allows
people
to
understand
yeah,
I'm
very
heavily
invested
in
java.
You
know
so
on
and
so
forth
and
giving
them
the
tools
that
they
can
leverage
to
understand
their
own,
their
own
particular
risk
and
then,
from
a
alpha
omega
perspective.
J
K
I
believe
matt
had
you
had
your
hand
if
I
may
so
you
know
chris.
I
had
no
idea
what
chris
is
going
to
say
and
did
not
coordinate
this,
but
the
more
chris
talked,
maybe
the
less
I
have
to
say,
but
I
want
to
you
know.
I
think
that
you
know
this.
I
want
to
build
a
bridge
between
what
michael
I
see
michael
saying,
and
I
and
I
don't
always
attend,
admittedly,
the
alpha
omega
project.
K
I
did
this
week
and
I
think,
through
the
discovery
project
of
trying
to
get
alpha
mega,
going
we're
finding
that
we're
finding
communities
who
know
they
have
security
problems
and
they're.
Just
looking
for
people
to
line
up
the
correct
set
of
tools
for
them
and
the
correct
give
them
the
a
way
to
now
analyze
what
they
have
much
the
right
metadata,
the
right
transit
dependencies
to
help
them.
They
don't
want
to
pull
the
ocean
and
we
can't
build
ocean.
That's
what
I'm
summarizing.
K
What
chris
is
saying
so
give
them
the
right
set
of
tools
and
tell
them
how
to
run
that
and
tell
them
what
we,
how
you
analyze
the
data
and
what
we're
giving
you
to
analyze
and
if
there's
something
we
group,
we
work
on
the
tools
they
work
on,
how
to
do
it,
the
methodology,
the
process
and
then
maybe
we
get.
We
can
then
work
through
alpha
mega
and
work
cooperatively
to
find
these
groups
to
say
here
are
the
tools
to
help
you
figure
out.
What
are
the
most
critical
things?
K
If
you
don't
have
that
and
then
maybe
we
can
get
more
interest.
We
can
like
pick
like
like
a
use
case
or
a
or
a
group
that
we
want
to
go
after,
like
kubernetes,
community
or
java
community,
and
then
we
can
get
some
interest
there
and
they
can
tell
us
what
their
profile
looks
like
what
their
you
know,
what
they
found,
what
they
think
the
most
is
critical
and
we
get
interest
there.
K
You
know
we
can't
solve
it
for
everybody,
but
let's
get
some
different,
dedicated
groups
for
communities
who
are
willing
who
know
who
want
to
do
these
things
enabled,
and
I
will
just
build
on
what
chris
said
as
well.
I
mean
I
talked
to
device
manufacturers
who
use
java
and
they
tell
me
my
problem
is
my
device
is
in
the
field
for
10
20
years.
It's
got
log4j
in
it.
You
know,
so
it's
still
important
to
them
and
it's
a
big
deal
when
they've
got
to
update
devices
because
they
don't
they
lose
track
of
them.
K
G
I
do
have
another
point
yeah,
so
you
know
with
with
al
all
respect
to
chris
ferris.
I
do
think
there's
value
in
making
the
shorter
list
because
of
resource
constraints.
The
reality
is,
we
cannot
spend
you
know,
money
time,
deeply,
investing
in
every
single
open
source
project
on
the
planet.
I
mean
we
should
do
things
that
help
broadly,
but
we
should
also
focus
efforts
on
the
things
that
matter,
and
so
we
need
to
have
some
idea
of
what
matters.
G
G
I
think
it
at
the
moment.
It
might
be
easier
to
separate
the
two
michael,
but
obviously
that's
worth
discussion.
You
know
back
when
I
did
the
census
one.
We
combined
those,
but
you
know
simply
because
there's
so
many
projects
out
there
trying
to
come
up
with
the
hey,
what's
important
and
then
look
delving
a
little
further
and
trying
to
estimate
and
what's
the
risk
here.
So
I
mean
the
the
linux
kernel.
For
example,
the
negative
is
negatives,
include,
hey
it's
in
c,
we've
got
all
it's
unsafe.
G
It
directly
interacts
with
the
volt
with
the
hardware,
it's
much
easier
for
a
volt
for
a
mistake
to
become
a
vulnerability.
On
the
other
hand,
there
are
a
lot
of
people
involved,
they're
doing
analysis,
lots
of
tools,
whereas
here's
this
other
project,
it
has
all
sorts
of
opportunities
from
problems
and
there's
nobody
at
the
store.
G
G
J
So
so
let
me
just
respond
very
briefly:
I'm
I
don't
disagree
with
what
you're
saying
david.
I
don't.
I
don't
think,
and
I
did
mention-
maybe
a
smaller
list.
I
do
think
there's
some
value
in
producing
a
list
based
on
some
objective
criteria.
I
don't
see
the
purpose
of
voting
on
stuff
is
my
point
number
one
and
number
two.
I
again
it
and-
and
I
know
I
think
said
it
well
in
the
chat-
it's
very.
It's
rather
subjective
right.
J
You
know
whether
it's
a
point
in
time
thing
or
whatever
you
know
something
could
be
more
important
than
others.
If
the
purpose
of
this
exercise
is
to
come
up
with
a
list
of
stuff
that
the
people
in
alpha
omega
could
be
focusing
on.
Okay,
that's
that's
fine
and
let
them
vote
and
decide
where
to
put
the
dollars
out
of
alpha
omega
rather
than
us
to
sort
of
objective.
You
know,
rather,
you
know
subjectively
trying
to
sort
of
vote
on
things.
J
The
second
point
that
I
want
to
make
you
know
addresses
your
risk
versus
criticality.
I
don't
think
they're
different.
I
think.
They're
the
same,
I
think
you
have
to
factor
as
the
the
thing
that
you
want
to
focus
your
attention
on
is
the
thing
that
carries
with
it
the
most
risk
and
the
thing
that
carries
with
the
most
risk-
and
you
describe
this,
I
think
fairly
well,
linux.
J
Yes,
there's
risk
it's
everywhere,
it's
pervasive
and
so
forth,
but
to
your
point,
there's
a
lot
of
eyeballs
looking
at
it.
There's
a
lot
of
people
focusing
on
security.
There's
people
ready
to
land
patches
on
a
moment's
notice,
and
we
have
in
process
a
place
for
updating
you
know,
releases
and
so
forth.
J
That's
pretty
that
that
that
is
pretty
well
taken
care
of
the
thing
where
something
like
log4j
version:
one.x,
that's
embedded
in
things
like
kafka,
that's
embedded
in
other
java,
based
project
solutions
and
pro
and
products
is
that
going
from
log4j1.x
to
log4j2.x
is
a
significant
amount
of
effort,
and
so
therefore
the
risk
of
carrying
1.x
baggage
is
much
higher
than
if
something
were
to
have
been
found
in
linux.
That
we
could.
You
know
patch
fairly
quickly,
and
and
you
know
we
have
the
processes
in
place
to
do
that.
J
We
did
not
have
the
processing
in
place
to
deal
with
log4j.
We
just
didn't
right,
we
didn't
you
know
many
people
didn't
even
know
where
it
was
right,
and
so
you
had
to
have
this.
You
know
spelunking
exercise
of
people
digging
into
the
software.
They
had
to
figure
out
if
they
have
log
for
j1.xml
right
so
anyway.
I
just
wanted
to
sort
of
tie
those
things
together,
because
I
think
it's
critical
that
we
focus
on
the
things
that
are
carrying
the
largest
amount
of
risk.
D
D
I
totally
see
the
value
of
having
a
single
list,
but
then
also
when
talking
to
folks
in
my
company
and
with
folks
from
the
same
industry,
I
think
it
would
be
valuable
having
these
verticals,
because
it
would
potentially
help
us
scaling
the
resource
issue
that
david
and
others
mentioned
beforehand.
So
certainly,
and
very
focused
effort
like
of
omega
needs
a
list,
but
then
potential
is
the
automotive
and
it
makes
it
much
easier
also
within
those
companies
to
argue
for
investment
in
something
like
open,
ssf
right
to
to
bring
more
resources.
C
C
F
Yeah
one
more
thing
just
go
back
to
criticality
versus
risk,
and-
and
this
is
a
great
like
conversation
like
this-
is
a
good
night
conversation
because
I
don't
think
like
it's
all
opinions,
but
the
way
I
look
at
it
is
from
a
business
perspective.
I
think
a
company
is
going
to
want
to
know
are
the
critical
open
source
projects
that
I'm
using
covered
in
some
way,
and
they,
knowing
that
they
all
are
low
risk,
because
of
some
activities
have
taken
place,
is
good
enough.
It
checks
a
checkbox
for
them.
F
F
You
know
graph,
so
so,
if,
if
critic
separating
our
criticality
and
risk
means
that
they
can
get
a
a
warm
feeling,
because
the
most
well
the
most
important
to
them,
projects
have
some
something
has
been
done
to
them
or
for
them
without
getting
into
their
like
specific
risk
scenarios
and
less
or
more.
I
don't
know
if
that
made
sense,
it
made
more
sense
in
my
head,
as
I
was
thinking
about
it,
but.
C
A
G
G
I
think
the
notion
here
is
well
and
then
there
therein
lies
the
question.
I
think
for
many
folks
by
criticality
they
mean
the
the
what
are
the
most
impactful
open
source
projects,
one
of
the
ones
where
things
go
wrong,
it's
the
most
harm
can
occur
now,
whether
or
not
the
likelihood
of
of
those
actually
going
wrong.
That's
a
different
question.
G
G
C
G
C
And
then
it's
mentioned
mention
again,
I
mentioned
a
couple
times
in
the
chat.
I
just
wanted
to
reply
again
that
I
think
we've
also
firmly
like
understood.
That
likelihood
is
also
a
product
of
the
complexity
of
the
product,
the
project
and
the
you
know
posture
of
the
project
like
how
many
eyes
are
on
it.
G
I
think
that's
one
challenge
is
that
there's
a
huge,
huge
number
of
ways
I
mean
if
I
was
gonna
measure
likelihood,
I
would
ask
things
like
how
many
maintainers
when's
the
last
commit
date.
You
know
do
they
know.
Is
there
evidence
of
security
practices,
things
like
badging
and
tools,
and
so
on
I
mean
there's
there's
many
many
ways
to
measure
likelihood
and
those
are
all,
of
course
estimates.
But
still
that's
those
are
ways
to
do.
It
likelihood.
B
B
Or
processing
user
supplied
input
like
you
know,
but
so
many
things
and
unintentionally,
processing
user
users,
flight
input
that
were
never
designed
for
that
like
they
were
designed
as
one
off
like
you
know,
pdf
generators
that
you
know
like
you
know
ooh.
Let's
do
that
and
then
you're
running
user
and
supplied
input
and
then
it
gets
sorry
ssr
for
people.
K
Matt
I
mean
I
I
just
want
to
say
that
criticality
is
is
relative.
If
anything's
been
hammered
home
during
this
call,
I
mean-
I
don't
know
if
anybody
here
is
directly
representing
or
indirectly
representing
groups,
but
I
have
a
friend
and
coincidentally,
we
were
talking
last
night.
He
works
for
a
defense
contractor
and
he
was
talking
their
reinforced
onboard,
something
called
milk
cloud
that
any
all
these
companies
will
never
be
represented
in
open
source
groups
directly.
They
usually
don't
they're
using
tons
of
open
source
and
so
to
them.
You
know
you'd.
K
If
you
knew
what
this
software
is
building
and
going
into
airplanes
and
missile
systems
and
stuff
like
that
would
be
critical.
I
don't
even
think
that's
across
their
mind.
So
again
we
need
to
be
mindful
of
groups
who
won't
be
represented
here,
won't
give
us
money
but
are
but
are
critical
to
our
planet
and
our
lives.
You
know
so
I'm
just
saying
we
have
to
be.
We
have
to
have
a
way
of
disseminating
these
things
in
a
way
that
it's
not
it's.
K
It's
clearly
portrayed
as
something
a
set
of
tools
and
things
that
you
can
use
whatever
definitions.
We
create
whatever
methodology
or
process
we
create,
can
be
used
by
anybody's
and
the
more
examples
we
create
of
use
cases
that
slice
across
different
industries
or
things
like
that.
Then
people
like
the
milk
cloud
people
will
pick
those
up
and
use
those
things
they
will.
A
Yeah
I
had
a
couple
of
things:
yeah
I'll
do
one
first
and
then
the
next
one,
so
the
first
one
quickly
was
just
quickly
on
sort
of
like
what
you
might
think
of
as
the
intelligent
risk.
I
usually
say
frequency
of
magnitude,
but
you
know
people
say
likelihood
and
impact
that
they're
more
or
less
interchangeable.
A
I
want
to
distinguish
that
from
the
predictive
variables
that
affect
either
of
those
two.
I
think
we
tend
to
muddy
the
waters
on
that
when
we
sort
of
say
well.
On
the
one
hand,
they
have
lots
of
contributors,
but
on
the
other
hand,
it's
very
high
impact.
It's
like
well
there's
a
different
voya.
You
know
different
variables,
pointing
to
different
two
things:
it's
important
not
to
mix
them
up
and
also
that
there
are
probably
dozens
of
variables
that
predict
either
or
affect
the
outcome
of
either
the
frequency
of
the
magnitude.
A
The
second
one
is,
I
wanted
to
sort
of
respond
quickly
to
what
matt
was
talking
about.
One
of
the
things
that
I
talk
about
in
my
talk.
Email
me.
If
you
want
to
see
a
copy
is
essentially
that
we
still
need
expert
judgment
for
exactly
that
kind
of
scenario
that
that's
literally,
like
the
nightmare
case,
that
an
expert
somewhere
knows
something
that
the
data
did
not
reveal
for
whatever
reason.
A
So
you
know
we
still
need
to
rely
on
the
data
just
because
of
the
scale
of
the
problem,
just
the
vastness
of
of
the
amount
of
stuff
we
have
to
survey,
but
we
want
expert
opinion
where
available
and
where
relevant,
where
it
changes
things
like
what
we
want
to
do
is
maximize
surprise
right.
We
want
to
be
surprised
by
things
that
were
not
in
the
data.
That's
that's
the
most
valuable
information
to
us,
something
that
confirms
what
the
data
already
tells
us,
not
very
useful
right.
That's
just
wasted
effort,
so
I
yeah
I'm
gonna!
L
You
know
I
was
part
of
the
so-called
great
mfa
distribution
project
that
some
of
us
participated
in
which
basically
con
you
know,
was
about
getting
tokens
to
what
we
thought
were.
You
know
critical
maintainers
of
open
source
projects
out
there
to
improve.
You
know,
security
overall
and,
and
one
of
the
projects
I
was
assigned
to
was
some
some
npm
package,
and
I
contacted
the
author
and
I
said,
hey
we're
doing
this
project
and
your
project
was
selected.
We
are
we'd
like
to
offer
you
a
token.
L
This
is
our
hardware
tokens
right
and
the
guy
says
why
this
project
I
maintain
like
dozens
of
them
and
this
by
no
means
the
most
popular
project
that
I
actually
maintain-
and
I
couldn't
have
an
answer
I
mean
just
to
be
clear.
We
based
the
this
effort
based
on
the
list
that
was
produced
initially
with
a
hundred
top
level
projects
right,
most
critical,
and
so
even
for
the
author
of
the
the
package,
it
wasn't
clear
why
this
one,
rather
than
some
others,
that
he
had
that
were
more
popular
in
his
opinion.
L
So
I
think
that
illustrates
very
well
the
difficulty
that
we're
talking
about.
You
know
that
we're
facing
here
it's
really
hard
to
define
some
universal
criteria,
and-
and
that's
why
I
was
saying
in
this
chat
and
along
with
what
it
was
saying
earlier,
I
think
it's
very
important
to
have
the
right
disclaimer
with
the
list
and
and
explain
how
we
come
up
with
that
list
so
that
at
least
people
have
an
idea.
You
know
this
is
how
that
it's
not
necessarily
universal
point
of
view.
F
Oh
michael,
I
I
think
that's
actually
a
great
point.
I
wonder
how
many
open
source
authors
would
call
themselves
critical
when
they
are
not
or
or
vice
versa,
just
because
they
don't
by
almost
by
definition,
know
where
their
product,
where
their
project's
being
used
and
providing
that
insight
back
to
them,
might
be
a
value
be
like
you
know,
you
are
actually
being
used
on
the
mars
rover.
Like
did
you
know
that
you
know
I
don't
know,
what
can
we
do
differently,
but.
M
It's
it's
it's
nice
to
think
it's
nice
to
think
we
could.
But
my
own
experience
with
a
couple
of
key
projects
is:
they
know
that
they're
being
used
in
scada
applications
and
aerospace,
applications
and
transport
applications
and
national
security
applications
and
their
customers
are
already
extremely
frustrating
just
giving
them
bug.
Reports
won't
give
them
full
context.
M
M
K
Say
given
tool
or
if
you
give
them
tools
that
are
meant
to
secure
their
supply
chain
or
evaluate
scorecard
or
look
for
for
scanning
tools,
if
you
give
them
an
a
an
opinionated
set
of
tools
that
are
vetted
by
you
know
general
security
people,
and
they
know
that
they're
available,
and
they
can
trust
them.
They
will
employ
them
in
their
private
clouds
or
in
their
their
respective.
You
know,
industries,
they.
M
They're
they're,
I
don't
I
I
don't
think,
there's
actually
a
simple
ordinal
criticality,
because
it's
it's
in
using
the
term,
it's
a
multi-dimensional
thing,
depending
on
which
particular
supply
chain
which
and
who
their
particular
customers
are,
and
eventually
every
single
project.
Every
project
turns
into
the
nail
of
the
want
of
a
nail.
F
I
I
don't
know
I
mean
from
from
all
of
your
companies
here.
Do
you
feel
like
well,
maybe
because
you're
here
you're
not
represented
about
who
I'm
going
after
but
like
do
you
think
the
average
company
knows
what
the
most
critical
open
source
they
use
is
because
I
would
say
that
of
every
company
I've
worked
for,
like
none
of
them
do.
K
It
it's
the
same
thing.
We
need
to
know
where
we
invest
in
open
source
as
well.
We
need
to
know
you
know
we're
getting
to
a
point
we're
trying
to
figure
out
what
we
use
with
s-bond
since
normalization
we
have
like.
I
said
we
have
2000
products,
you
know
so
getting
a
point
assessment
and
knowing
what's
critical
at
giving
point
in
time
and
given
all
the
different
factors
of
our
own
internal
factors,
how
prolific
are
the
tools?
Are
they
to
use?
They
stole
service
lines,
that's
relative
to
a
company.
K
I
can
use
the
same
set
of
tools
for
a
company
if
you
do
it
for
industry,
we're
not
trying
to
it's
not,
but
it's
all
based
on
open
source.
It's
we're
not
trying
to
if
they
came
across
that
where
any
of
this
was
or
at
least
my
perception
of
it
or
how
I
interpret
it,
is
we're
just
trying
to
solve
it
in
a
create
a
normalized
way,
a
an
opinionated
way
and
a
prescriptive
set
of
tools
and
help
people
along
the
way,
regardless
of
their
company
using
open
source
or
an
industry
using
open
source.
K
It
doesn't
matter.
I
don't
care
who
uses
it
make
it
better.
You
know
I'm
going
to.
I
can't.
I
can't
help.
If
I'm
going
to
use
microsoft,
software
from
ibm
or
apple
software,
I'm
going
to
use
it,
I
don't
care
to
get
the
job
done.
I
want
it
wherever
I
get.
The
software
from
a
corporation
or
an
industry
or
group
make
it
better,
and
this
is
what
this
group
is
being
paid.
You
know
paid
to
do
by
its
members.
You
know
figure
out
a
way
to
make
this
better,
because
open
source
is
the
problem.
K
C
Okay,
so
I'm
going
to
have
to
I'm
going
to
cut
it.
Sorry,
david
due
to
time,
but
I
want
to
make
a
really
important
point.
Is
that
so
again
you
know
we
have.
We
have
discussions
kind
of
the
similar
discussions,
often
in
the
meeting
about
the
same
topic,
if,
if
be
sure,
to
check
the
meeting
notes,
if
you
see
that
your
comment
or
your
concern
was
missed
or
you
know
misinterpreted,
please
try
to
correct
it
there.
C
C
You
know
not
not
just
I
mean,
and
I
don't
want
to
downplay
like
the
discussion
and
the
the
you
know
coming
up
and
and
providing
input
when
we
ask
for
it,
but
we
need
to
have
people
like
leading
the
effort
here
and
and
making
things
making
things
happen,
and
we
asked
at
the
previous
meeting
you
know
so
I
have
I
have
in
the
meeting
notes
current
working
group
project
sig,
there's
a
link
to
the
current
projects.
The
very
first
project,
most
important
project
for
this
group
is
creating
the
list
of
critical
projects.
C
We
need
to
have
people
say,
I'm
a
maintainer
or
a
leader
of
this
initiative
in
the
previous
meeting.
Amir
caleb
and
david
raised
their
hand,
many
of
you
weren't
there.
So
I'm
going
to
ask
now
who
wants
to
be
involved
either
in
a
contributor
or
you
know,
to
a
contributor
for
for
working
sessions,
type
of
thing
or
a
leader,
and
I
see
jacques
raiser
hand.
Anybody
else
would
like
to
speak
up.
C
C
It
exists
like
most
working
group
initiatives.
People
go
off
on
the
side,
they
do
work,
they
create
documents,
they
maybe
have
working
meetings.
They
come
back
to
this
group
for
feedback
right,
so
your
feedback
is
is
very
well.
You
know
needed
and
received
when
we
come
back
with
documents
like
this,
which
is
an
early
document
we
want
to.
We
want
to
hear
that
feedback.
C
We
want
to
get
that
in
the
notes,
so
just
people
that
are
going
to
be
you
know,
I'm
not
saying
if
you
don't
raise
your
hand
now,
you're
not
going
to
be
come
back
for
the
feedback
that
this
is
a
an
initiative
of
this
group
feedback
will
be.
You
know,
brought
this
the
the
project
we
brought
to
this
group
and
for
feedback,
but
if
you
want
to
do
the
kind
of
do
that,
you
know
feed
on
the
ground,
work
raise
your
hand.
C
All
right
we're
at
the
end
of
time.
Thank
you,
everybody
for
the
discussion.
There
was
one
last
thing
that
we're
also
going
to
be
working
on
status
reports
for
the
attack.
That's
not
a
big
deal.
We
can
talk
about
it
again
as
well
in
the
next
one.
So
thank
you
very
much.
I
really
appreciate
it
and
see
you
in
the
next
meeting.