►
Description
Weekly meeting of the Identifying Security Threats working group. Our GitHub page is located at https://github.com/ossf/wg-identifying-security-threats and from there you can find meeting minutes and other information.
A
There
we
go
pick
up
projects,
do
it
and
then
learn
from
it
and
iterate,
and
just
just
do
that
for
for
a
couple
months
and
get
more
people
trying
it
and
you
know,
don't
try
to
come
out
with
a
standard.
That's
going
to
work
for
anybody
just
come
out
with
something
that
might
work
for
you.
B
A
I
mean
even
though,
like
what
we
need
is
a
standard,
because
you
need
to
know
that,
like
distribution
points,
dot
package,
name
package,
dash
name
means
the
place
that
you'd
go
and
look
for
it
on
npm.
If
somebody
calls
it
like
just
package
well,
now
it's
kind
of
no
better
than
just
a
loose
file.
You
know
just.
B
A
C
So
sorry,
I
had
a
couple
questions
I
I
know
I
haven't
been
that
involved
in
this
kind
of
effort,
thus
far,
so
I
was
just
kind
of
curious.
So,
first
of
all,
I
think
the
idea
of
yaml
is
like
really
cool.
I
guess
to
just
piggyback
just
because,
for
the
reasons
already
stated,
it's
easy
to
parse
easy
to
read.
C
All
of
that,
and
my
general
understanding
of
what's
kind
of
been
what's
going
on
here
is-
is
to
get
some
kind
of
more
kind
of
widely
accepted
kind
of,
because
the
security
md
it
just
sounds
like
there's
not
really
much
too
much
format
and
all
that,
I'm
sorry,
I'm
reiterating
some
obvious
things.
I
just
want
to
make
sure
I'm
like
understanding
kind
of
the
idea
behind
everything,
so
to
get
a
more
kind
of
organized
layout
to
all
this,
which
I
think
is
really
cool.
C
So
I
guess
the
other
thing
then
that
comes
to
mind
immediately
is
just
that
like
how
much
is
just
like
what
people
are
saying
what
they
have
right,
like,
for
example,
versus
like?
What's
actually,
you
know
like
in
our
data
in
the
scorecard?
One
idea
I
was
thinking
like
I
don't
know
how
useful
this
would
be
it,
but
like
what,
if,
like
you
know,
people
like
organizations
can
list
whatever
they
have
of
this
information
in
their
repos?
C
Is
there
any
way
that
we
can
like
display
what
data
is
like
verified
like
on
the
scorecard,
like
you
know
how,
like
it's
kind
of
like
an
out
there
concept,
but
you
know
on
like
social
media
accounts,
people
have
like.
Oh,
I
say
I'm
this
person,
but
this
is
a
verified
account
like
what,
if
some
of
this
information
had
a
little
like
verified
check,
mark
from
like
oh,
like
scorecard,
is
verified
that
they
do
have
this
here
here
and
here,
because
it
kind
of
reminds
me
of
that
conversation
we
had
earlier.
C
I
can't
remember
who
was
complaining
that
we
said
I
said
that
we
we
had
some
security
flag
for
them
on
scorecard
and
they're
like
no.
We
actually
do
have
that
here.
So
that
would
kind
of
show
that
you
know
it's.
A
combination
of
most
of
their
stuff
is
verified,
but
oh
by
the
way
we
also
manage
our
pr's,
even
though
it
says
it's
you
know
doesn't
in
scorecard.
We
list
it
here.
So
there's
like
a
level
of
trust.
A
A
It's
not
going
to
find
it.
That
doesn't
mean
I
don't
have
a
security
policy.
So
it's
so
it's
wrong,
but
it's
kind
of
that's
how
it
works.
So
if
I
had
my
a
link
to
my
security
policy
here
that
it's
slash
security.text,
then
then
scorecard
could
use
that
and
then
say.
Yes,
he
does
have
a
security
policy
because
I've
looked
it
up
in
the
metadata,
so
it
kind
of
makes
scorecard's
job
easier
if
it
doesn't
have
to
guess.
C
That's
a
great
yeah.
I
love
that
totally
could
go
both
ways.
It
could
score
a
card,
could
parse
this
and
say.
Well,
I
don't
know
if
I
don't
know,
if
scorecard
should
just
change
that
data,
necessarily
because
it
is
just
he
said
she
said
kind
of
stuff,
but
maybe
it
can
say
like
it's.
There
could
be
like
a
slightly
different
metric
where
it's
like.
They
say
they
do
this.
If
that
makes
sense-
or
I
don't
know,
I
guess
it
could
be
like
officially
count
or
whatever,
but.
C
A
But
I
think
for
the
most
part:
let's
if
we,
if
we
just
if
we
assume
positive
intent
and
and
just
kind
of
go
with
the
you
know
like
if,
if,
if,
if
you,
if,
if
you're
an
attacker,
this
is
so
not
the
avenue
that
you're
gonna
yeah,
I
was
gonna
say
this
is
like.
A
C
No
yeah
you're
right.
I
I
yeah
that
that
that
makes
sense,
but
yeah
there.
A
A
B
C
B
B
Cross-Referencing
between
tools
could
be
a
way
to
justify
or
verify
data
too,
where,
if
scorecard,
let's
say
you
know,
also
came
up
with.
Yes,
you
have
security
policy
or
whatever,
and
this
structured
security
also
came
up
with
the
same
value.
It
could
be
like
a
verification.
C
By
the
way,
there's
a
smaller
note
but
scorecard
I
I
asked
them
about
the
just
some
small,
like
less
related
stuff,
I
asked
them
about
adding
some
stuff
for,
like
watchers
and
like
stuff
like
that,
for
the
dashboard
I've
been
trying
to
get
for
a
while
and
basically
they
said
all
right.
So
the
guy
who
responded
said
cool
because
I
added
it
and
I
merged
it
to
the
repo.
So
I'm
like.
C
Oh,
can
you
regenerate
the
excel
spreadsheet
they're
like
okay,
cool
I'll,
merge
this,
but
can
you
regenerate
all
of
our
scorecard
data?
So
basically,
that's
like
a
lot
of.
I
think
it
takes
like
several
days
running
on
cpu
to
regenerate
it.
So
before
I
started,
I
guess
with
microsoft.
I
was
kind
of
an
issue
because
I
needed
my
mac
for
just
things
every
day,
but
I
guess
I
can.
I
don't
know
just
use
it
on
this
device
or
is
there
some
resourcing
scorecard
is
asking
you
to
generate
yeah
he's.
A
C
A
Because
I
I
know
the
scorecard
is
having
a
hard
time
running
against
throttling
limits
and
things
like
that
and
with
the
50
000
projects.
You
know
it's
it's
many
days.
It's
basic,
I
think
they
said
it's.
It's
practically
an
entire
week.
Yeah.
A
To
to
regenerate
it
all
but
like
we
could
wait
until
next
week
or
the
week
after
or
next
month
or
whenever,
like
so
if
it's
merged
I
it
should
eventually
be
in
the
distributed
thing.
Yeah.
C
It
sounds
like
I
mean
eventually
they're
gonna.
Do
it
so,
like
I
don't
know
when,
like
you
said,
if
it's
once
a
week
or
a
month
or
I
don't
know,
I
think
you're
right,
they
do
update
it,
and
basically
all
I
did
was
I
added
watchers
and
then
stars.
I
guess
we're
going
to
replace
it
with
stars
anyway,
but
stars
is
no
longer
there
when
I
regenerated
it-
and
I
asked
him
why,
like,
why
is
he's
like?
Oh,
that
was
old
code.
C
A
I
I
still
don't
regenerate
the
dashboard
automatically.
A
It
again
later
this
week,
hopefully
that'll
that'll
that'll
get
right.
C
Yeah,
I'm
sorry,
I
could
not
figure
out
exactly
what
was
go.
I
tried.
I
tried
looking
through
that
the
script
that
what
was
that
daily.
I
forgot
what
it's
called
and
I.
A
It's
all
good
yeah
anyway,
sorry
cool
okay,
so
I
mean
I
guess,
the
the
takeaway
for
that.
So
what
we
were
planning
to
do
from
the
from
the
previous
meeting
on
security
yama
was
to
kind
of
take
today
and
kind
of
continue
iterating
on
this,
but
without
matt
here,
I'm
a
little
reticent
to
kind
of
go
too
far
down,
because
I
want
him
to
be
kind
of.
I
I
like
him
just
to
kind
of
take
over
this
project
and
kind
of
run
with
it.
A
A
So
for
hiring,
so
we
got
approval
for
40k
for
a
software
engineer
to
work
on
the
dashboard
project.
The
main
tasks
are
in
a.
I
think
I
posted
this
somewhere,
but.
A
I'm
kind
of
over
here
and
then
there's
a
linkedin
job
post
and
basically
it's
like
rebuild
the
infrastructure
like
doing
it
the
right
way
and
not
like
my
way,
so
you
know,
but
I
would
actually
have
like
it'd,
be
different
pieces
and
containers
and
like
running
separately
and
not
like
all
in
the
same
vm.
A
It
shouldn't
be
a
tremendous
amount
of
work
for
someone
that
has
kind
of
done
this
multiple
times
update
the
the
front
end
with
you
know
we
have
some
some
things
like
you
know,
improving
the
api
and
some
bug
fixes
certainly
getting
the
chrome
stuff
working,
making
sure
that
postgres
is
is
kind
of
correctly
configured
and
optimized
and
whatnot
and
then
basically
just
do
feature
work.
So
I
think
we
have
enough
time.
A
We
have
enough
money
for
somewhere
around
two
months
of
full-time
work
or
because
I
think
we
have
to
be
done
by
the
end
of
december
up
to
four
months
of
part-time
work,
based
off
of
some
of
the
rates
that
I've
seen.
So
we
have
about
20
or
so
candidates
that
have
applied
through
the
through
linkedin
and
then
I
have
you
team
and
top
tal
who
are
kind
of
freelancing.
A
Well,
one
of
the
freelancing
firm
and
the
other
one
is
like
a
referral
escrow
service.
I
guess
where
you
know
the
relationship
is
with
some
other
company
that
they
have
a
relationship
with
either
way,
though
we
should
kind
of
get
on
this
get
on
this
quick.
So
we
can
get
somebody
on
the
ground
doing
this.
I
do
have
a
question
out
to
david
on,
like
okay,
so
if
we
have
somebody
now
like
they
need
to
do
this
somewhere
within
azure.
A
That
is
not
associated
with,
like
my
work
teams
like
azure
subscription,
so
we
need
to
get
this
out
into
something
that's
open,
ssf
owned,
so
I
just
need
need
to
know
how
they
want
to
do
that
and
whatnot.
A
So
I'll
do
that.
But
if
anybody
is
any
of
y'all
are
passionate
about
hiring
and
want
to
kind
of
you
know,
you
know
help
like
review
resumes
and-
and
you
know,
interview
and
stuff
like
that,
it
shouldn't
be
too
many.
I
mean,
I
think
I
think
if
I,
if
I
don't
go
with
you
team
or
top
towel,
then
yeah
it's
about
20
right
now
and
from
what
I've
seen
most
of
them
are.
A
Much
more
on
the
they're
all
devops,
but
they're
more
on
the
ops
than
the
dev.
So
it's
a
lot
of
kind
of
cis
admin
in
the
cloud
work
which
is
great,
but
I
really
need
somebody
that
can
also
do
the
dev
work.
B
A
B
So
I
could
provide
a
little
bit
of
insight
here
about
how
lf
would
want
to
handle
this.
I
actually
just
threw
in
a
dock
in
the
meeting
notes,
but
basically
it
goes
through
the
process
of
essentially
how
to
get
paid
through
lf,
so
there,
where
it
says,
cdoc
and
there's
like
a
google
doc
right
there
oh
got
it
yep
yeah,
so
that
basically
outlines
the
process
but
yeah.
B
If
things
have
been
approved
by
the
governing
board,
which
I
believe
they
have
been,
then
they
would
basically
just
go
through
this
process
of
basically
doing
a
monthly
invoice
to
lf
and
basically
see
seeing
david
and
possibly
you
or
anyone
else,
who's
kind
of
reviewing
this
with,
like,
basically
a
like
a
summary
of
like
what
work
the
work
and
stuff,
then
that's
typically
how
they're
gonna
go
about
paying
paying
out
people
so
now
that
this
contractor
has
been
identified,
I
could
certainly
help
with
it.
B
If,
if,
if
there's
going
to
be
some
kind
of
like
a
basic
agreement
written
up,
I
think
that
would
be
a
good
idea
and
then
part
of
that
would
be.
You
know
invoicing
linux
foundation
for
like
their
monthly
salary
or
whatever
their
monthly
hours.
Yeah.
A
Perfect
that
I
think
I
think
that
works
great,
that
certainly
leans
toward
just
making
this
a
freelance
thing,
rather
than
trying
to
go
through
an
agency
because
the
agencies
are
gonna
want.
You
know,
I'm
gonna
want
money
in
escrow
and
and
pre-pay,
and
I
feel
like
it's
gonna
get
complicated
with
contracts
and
legal
stuff.
B
A
So,
okay,
so
I'll
be
I'll
I'll,
advise
this
towards
just
finding
a
freelancer
that
convec
is
capable
and
interested
in
doing
this
work
for
a
couple
months.
Cool
awesome,
awesome
cool,
so
the
other
thing
that
I
wanted
to
chat
about
was
a
proposal
that
I
made
well.
Let's
say
made.
I
very
briefly
described
to
the
governing
you
know
the
planning
committee
on
monday,
so
I
wanted
to
open
this
up
and
get
lots
more
opinions
on
it.
A
So
here's
the
pitch
at
a
high
level,
a
lot
of
the
work
that
open,
ssf
does
and
a
lot
of
other
organizations
in
this
space
do
is
kind
of
this
meta
security
work
where
it's
not
it's
not
an
anything.
I
mean
you're,
actually
not
included
in
this
because,
like
the
work
that
you're
doing
is
like
direct
like
look
at
the
thing
is
the
thing
safe
this
project
and
I
hate
that
project.
A
I
just
needed
a
name
to
call
it,
so
we
can
call
something
better
is
let's
run
high
quality
tools
across
all
the
open
source?
We
can
possibly
find
like
do
this
at
like
mega
mega
scale,
dump
all
the
results
in
a
database.
A
Basically
and
then
build
a
triage
experience
on
top
of
that,
so
that
folks
can
go
in
and
do
things
like
show
me
the
most
critical
issue
across
all
projects,
according
to
the
criticality
of
the
vulnerability
itself,
the
number
of
times
the
project's
used
whatever
so
some
other
magic
formula
that
that
that
prioritizes
well,
that
hasn't
already
been
triaged.
I
triage
it,
meaning
I
look
at
the
vulnerability.
I
look
at
the
code.
I
have
whatever
I
need
in
this
in
this
space.
A
I
confirm
that.
Yes,
it
really
is
a
vulnerability,
and
then
I
have
a
choice.
I
can
create
a
patch
for
it.
I
can,
like
you,
know,
privately,
identify
and
and
disclose
to
the
maintainer.
I
can
comment
on
it.
A
I
can
collaborate
with
my
you
know,
fellow,
you
know
security
researchers
and
the
idea
is
that
you
know
the
the
output
of
this
is
a
steady
stream
of
new
confirmed
security
vulnerabilities
that
are
that
get
fixed
and
either
they
get
fixed
within
the
triage
portal
or
they
get
fixed
by
the
maintainer
and
basically
just
rev
this
up
and
do
this
at
scale.
A
So
this
part
of
it
is
like
they're
running
the
tool,
there's
the
building
the
portal
and
then
there's
the
process
of
like
the
folks
that
are
doing
this
triage,
which
is
the
expensive
like
hard
part,
and
for
that
we
just
we
hire
security
analysts
to
do
it.
So
what
I
was
thinking
for
the
mvp,
it's
just
like
a
half
a
person
just
to
just
to
test
out
the
the
concept,
but
then
we
scale
up
to
three
and
then
we
scale
up
to
maybe
15.
A
Well
so
yeah
so
metrics
are
like
you
know
what
we
analyzed.
What
have
we
confirmed?
What
have
we
reported
fixed?
The
the
big
difference
here
that
I've
seen
a
lot
of
these.
A
I
haven't
seen
a
lot
of
things
like
this:
the
ones
that
I
have
they've
universally
failed
because
they
produced
garbage
results,
meaning
the
false
positive
rate
was
far
too
high
and
there
was
no
investment
in
reducing
that
false
positive
rate
or
was
just
deemed
too
hard
or
not
my
problem
or
we
outsource
this
to
some
other
vendor
and
the
vendors.
You
know
we'll
have
to
wait
till
next
quarter
to
get
a
to
get
an
update.
A
It
doesn't
like
just
to
get
that
kind
of
nonsense,
so
instead
we
over
pivot
on
engineering
work,
I'm
wondering
by
engineering
work
is
every
false.
Positive
is
a
bug,
and
you
know
you
fix
the
bug
so
that
no
one
ever
experiences
that
falls
positive
again,
basically
and
that
that's
part
of
this
kind
of
somewhere
between
the
automated
tooling
and
the
triage
portal.
A
So
so
imagine
the
case
where
you
know
you
find
a
you're
looking
through
like
sql
injection
things,
and
you
realize
that
the
tool
is
you
know,
doesn't
understand
that
these
actually
aren't
like
sql
statements.
It's
something
else.
You
know
you
you'd
be
able
to
provide
immediate
feedback
in
there
to
either.
You
know
flag
all
similar
findings
as
probably
false,
positive
or
open
up
a
bug
for
the
engineering
team
to
like
fix,
and
you
kind
of
have
a
really
tight
feedback
cycle
there.
A
I
mentioned
here
and
I've
gotten
I've
gotten
pushed
back
against
the
bug,
bounty
patch
bounty,
so
I
I'm
I've
taken
that
all
to
heart
and
I
think,
will
make
bug
bounty
patch
bounty
like
a
phase
two
of
this.
If
anything,
but
the
thinking
was,
you
can't
make
this
entire
database
public,
because
it's
just
a
pile
of
zero
days
and
there
are
a
lot
more
security
researchers
out
there
than
we
could
hire.
A
And
if
we
could
incent
those
security
researchers
to
join
and
do
the
triage
themselves,
then
we
can
do
more
stuff
and
if
they
can
get
bug
bounty
money
out
of
it
from
somebody
else
great
if
they
get
cves
and
if
that's
what
they're
motivated
by
great,
but
at
the
end
of
the
day
like
we
still
want
to
pay
them
cash
because
they
need
to
eat
two
and
would
a
pack
like
if
we
were
to
pay.
I
don't
know
a
thousand
dollars,
five
thousand
dollars
for
a
patch
to
a
critical
vulnerability.
A
Matching
this
criteria,
like
whatever
we
pay
for
that
is
probably
very
worth
the
worth
the
price.
However,
the
overhead
of
administering
that
program
is
is
probably
in
excess
of
the
rest
of
the
cost
of
the
program.
So
we
don't
want
this
just
to
be
about
that
plus.
A
I
think
that
that
maybe
orgs
like
hacker
hacker,
one
or
bug
crowd
or
maybe
tide
lift,
might
be
another
way
to
kind
of
achieve
the
same
ends
with
like
payouts,
and
things
like
that
without
having
to
get
involved
in,
like
you
know,
writing
checks
so
that
that's
the
thought,
I'd
love
to
get
your
thoughts
on
on
this
hi
ryan
on.
If
this,
if
this
makes
sense.
D
D
Yeah,
I
love
this
idea
personally,
I'm
really
happy
to
see
it
kind
of
come
together.
One
thing
I
was
going
to
put
in
in
this
document
was
just
sort
of
like
if
we
do
all
this
work,
what's
sort
of
like
our
long-term
plan
for
being
able
to
like
continue
maintaining
it.
That's
that
was
kind
of
my
biggest
concern
is
like
we
go
in.
We
find
all
these
vulnerabilities.
We
do
these
fixes.
We
do
this
stuff,
great
cool
and
then
we
get
in
we
get
out
now.
D
A
So
I
look
at
this
kind
of
like
this
is
the
we.
We
are
the
internet's
janitor
with,
and
I
don't
I
just
can't
think
of
a
better
analogy,
but
like
we're
constantly
like
going
around
and
we're
mopping
up
the
floor,
where
there
are
spills
or
whatever,
so
I
I
do
see
this
being
a
long-term
investment.
A
So
it's
so
it's
not!
It's
not
a
a
campaign
to
like
eliminate
critical
vulnerabilities
in
the
top
hundred.
It's
a
long-term
program
to.
A
Period
like
for
you
know,
ad
infinitum,
right,
yeah
that
makes
sense
so
so
yeah
I
mean
I
I
I
mean
it
would
be
great
if
at
some
point
we're
like
you
know
what
we're
out
of
we're
out
of
alms
like
we're
good,
then
we
just
you,
know
high
five,
each
other
and
and
go
home
right.
A
But
the
reality
is:
there's
new
open
source
being
created.
You
know
all
the
time
unless
we
can
go
significantly
over
yeah.
I
think
it's
like
a
thousand
a
day
or
1500
new
projects
a
day
published
so
like,
unless
our
study
stream
is
way
over
that
we're.
Never
we're
never
going
to
catch
up.
You
know,
but
I
think
by
sorting
it
by
this.
A
We
we
just
naturally
like
while,
yes,
we
will
we'll
be
scanning,
you
know,
I
don't
know
20
million
projects
or
20
million
like
art,
like
things
you
know
so
we're
somewhere
between
two
and
twenty
there's
gonna,
be
a
tip
of
the
spear
or
a
tip
of
the
the
priority
list
and
that's
kind
of
where
we
go
and
if
other
people
are
interested
in
like
so.
If
people
are
interested
in
looking
down,
they
should
be
able
to.
A
You
know
kind
of
look
at
anything
project
maintainers
on
github
or
npm
or
wherever
should
be
able
to
view
their
own
results,
and-
and
you
could
argue
that
well
isn't
that
really
the
same
as
like
get
up
code
scanning
and
like
like?
Are
we
just
like
duplicating
stuff
and
yeah,
but
we're
built
if
we're
gonna
build
it
anyway
like?
Why
not
give
them
access
to
it?
Like
it's
kind
of
you
know,
plus
we
can
integrate
things
that
aren't
in
well.
We
can
handle
projects
that
aren't
github.
D
A
Yeah
yeah,
I
mean
I
guess
the
only
I
mean
the
the
some
of
the
risks,
like
obviously
like
disclosure
of
the
database
would
be
really
bad.
A
D
Yeah,
like
that,
typical
risk
that
I
think
large
corporations
always
have
in
open
source
projects
right.
There's
things
like.
Oh,
why
is
google?
Why
is
microsoft?
Why
is
facebook
doing
this?
What
do
we
know?
What's
their
angle
right,
there's
always
this
healthy
dose
of
skepticism?
That
comes,
I
think,
anytime.
We
do
things
like
that
right,
which.
A
Is
which
is
why
I
think
we
need
to
show
up
with
patch,
or
at
least
our
best
effort
at
a
patch?
I
I
don't
expect
you
know
to
become
deep,
deep
experts
in
every
open
source
project
out
there
so
that
we
can
create
a
you
know.
A
I
I
think
we
need
to
like
what
I
don't
want
is
just
to
be
like
you
know,
sending
you
know
like
like
somebody
just
like
posts
like
fortify
results
in
a
github
issue
and
says:
hey
my
security
team
sent
me
this
go
fix,
so
we
need
to.
We
need
to
be
careful
on
how
we
how
we
approach
that,
but
I
think
I
think
we
can
do
that
by
just
being
smart
about
it.
Yep
definitely
agree
other
thoughts.
A
Correct
yeah,
so
so
so
the
the
database
in
the
triage
portal
and
all
this
stuff
would
not
be
publicly
available.
It
would
be
available
to
the
analysts
that
we
are
that
that
are
essentially
employed
that
are
are
our
employees.
You
know
doing
this
work
for
this.
I
I
think
opening
it
up
to
vetted
security.
A
Researchers
would
make
sense
to
if
you
know,
if
there's
interest
and
if
we
can
find
a
way
to
to
you,
know
incent
them
or,
if
they're,
just
intrinsically
incented,
that's
great
too,
but
but
the
database
itself
would
not
be
public
and
we
would
need
to
make
sure
that
that
database
remain
remain
protected
at
the
same
time
and
be
clear
like
we're
only
going
to
be
using
public,
open
source
or
publicly
available
to
do
this
work
and
all
of
our
the
the
the
triage
portal
and
the
analyzer
orchestration
part
and
all
that
stuff
would
be
open
source
too.
A
That
said,
we
don't
obviously
don't
want
to
make
it
super
easy
for
them
to
do,
but
it's
not
like
we
have.
A
bunch
of
you
know,
licensed
proprietary
findings
that,
like
attacker,
wouldn't
have
access
to
any
way
if
they
really
wanted.
C
So
quick
thought
so
like
yeah.
That's
that's.
Definitely
a
really
good
point.
I
think
this
definitely
like
the
the
idea
of
the
database.
It
definitely
compiles
everything
and
will
make
it
significantly
easier
than
like
individually
kind
of
using
all
those
open
source
tools
for
an
attacker,
but
at
the
same
time
I
that's
not
to
say
I
disagree
like
I
think
you
know
as
long
as
it's
it's
like
done
in
a
really
careful
way.
I
think
that
I
don't
know
my
personal
opinion
is
that
it
should
be
totally
great.
I'm.
C
Like
what
kind
of
tools
I'm
kind
of
curious,
what
kind
of
tools
and
how
automated
the
process
is
of
like
generating
this
database,
like
what
I
don't
know
like
you,
mentioned
they're
all
going
to
be
like
publicly
available
open
source
tools,
but
could
be,
did
you
have
any?
Do
you
have
any
thoughts
on?
Maybe
what
what
kind
of
things
would
be
utilized
for
that?
So
so.
A
The
specific
tool
I
mean
like
I,
I
I
have
ideas
on
which
tools
would
be
more
or
less
appropriate
here
some
of
them
are
per
language.
Some
of
them
are
just
kind
of
the
the
big
diff
like
code
ql,
I
think,
would
be
kind
of
silly
not
to
not
to
use
that
the
hard
part
is
the
is
the
false
positive.
So
I
I
think
you
know
like.
A
Running
you
know
20
or
30
tools.
In
you
know,
kind
of
a
batch
against
each
project
is
super
easy
and
great
now
we
have
like
you
know
a
thousand
results
per
project
which
of
those
are
actually
critical,
and
so
what
I
would
suggest
is
we
start
out
with.
A
You
know,
pick
pick
one
or
two
tools
and
adjust
the
rules
that
we
think
are
very
high
confidence
and
very
high
severity,
and
we
get
that
working
and
then
expand
over
time
and
as
we
expand
and
add
a
new
rule
or
a
new
tool,
we
come
up
with
a
process
of.
When
do
we
accept
those
results
as
meeting
our
quality
bar
and
then
when
they're,
when
they
don't?
That
is
an
engineering
bug
to
go
fix
right.
A
C
I
think
it'd
be
like
yeah.
I
totally
agree.
I
think
it
would
be
far
more
valuable
to
have
a
fraction
of
the
findings
in
the
database
with
higher
confidence
than
have
significantly
more
findings,
but
you
know
they're,
all
hidden,
gems
and
and
that
I
can
see
that
kind
of
being
a
big.
A
You
know
95
percent
confidence
and
up
and
critical,
you
know
or
cbss
9-0
and
up
and
then,
if
somebody
wants
to
turn
the
dial,
that's
fine
and
then
it
can
see
all
things
like
you
know,
like
I,
don't
know
the
results
of
like
a
process
trace
when
you
do
an
install
or
like
things
that
you
know,
I
guess,
as
a
security
researcher,
they
might
be
really
interesting,
especially
to
find
new
attack
patterns,
but
not
necessarily
something
that's
actionable.
A
A
E
A
A
This
was,
let's
include
all
the
tools
and
we
didn't
put
enough
time
into
minimizing
the
false
positives,
so
we
have
way
more
bugs
than
we
can
look
at,
and
you
know
so
for
for
this
external
thing,
I
I
want
to
make
sure
that
we
don't
we
don't
fall
into
that
trap.
There
are
definitely
critical
volumes
in
that
in
that
giant.
A
You
know
set
of
findings.
I
don't
have
a
good
good
grasp
on.
Like
you
know,
one
out
of
a
hundred
projects
has
a
critical
vulnerability
that
we
would
that
we
would
do
or
one
out
of
a
thousand
or
one
out
of
a
hundred
thousand.
A
I
think
part
of
the
mvp
here
would
be
running
this
on
enough
projects,
and
maybe
I
think
down
here
I
say
500,
but
maybe
that's
not
enough
to
come
up
with
that,
that
that
data,
but
as
as
part
of
like
the
initial
phase
of
this,
is
being
able
to
understand
if
we
turn
the
lever
up
to
here.
What
should
we
get
out
of
the
machine
here
and
being
able
to
predict
that
a
little
bit
better.
A
The
the
other
sorry,
the
the
other
thing
where
I
I
didn't,
really
include
this
in
the
paper,
but
I
I
think
what
we
also
need
is
a
way
to
create
new
rules
and
have
that
have
that
be
the
the
other.
So
the
engineering
work
is
like
reduce
false
positive
increase,
true
positive,
so
the
increase
true
positive
is,
like
you
know,
hey
this
this
you
know
critical
vulnerability
was
found
in
this.
A
You
know
one
that
came
out
in
like
libsy
or
glibc
like
yesterday.
It
was
like
a.
I
think.
It
was
a
null
pointer
access,
but
like
okay,
great
did
any
of
our
tools
find
that
no,
why
not?
A
C
As
an
idea
to
reduce
false
spots,
so
I
get
that
reducing
false
positives.
Increasing
true
positive,
like
you
said,
especially
after
those
big
volumes,
are
super
important
one
idea,
maybe
for
reducing
false
positives.
Since
you
mentioned
code
ql
as
a
source,
you
can
maybe
just
start
out
by
just
running
all
the
like
high
precision
queries.
If
you
want,
I
mean
that's
just
like
a
flag
in
the
in
the
queries
as
well
as
yeah.
Obviously
you
change.
C
You
do
all
sorts
of
severity,
but
I
think
maybe,
starting
with
the
higher
precision
queries
will
get
will
at
least
get
initial
stuff
in
there
to
be,
mostly,
you
know,
ripped
like
rid
of
false
positives.
Hopefully.
A
Yeah,
yeah
and-
and
you
know,
we'll
we'll
kind
of
learn
as
we
go
on
that,
but
yeah
yeah
right.
So
so
what
I?
What
I
was
thinking
was,
you
know,
I
mean
so
high
precision
is
kind
of
a
coq
construct,
but
like
run
it
all,
but
then,
when
you're
viewing
the
results,
the
default
filter
is
high
precision
and
up
and
security
severity
of
critical
and
up,
and
so
you
only
see
this
tiny
portion
of
the
list.
C
Right,
but
not
all
tools
like
cochlear
happens
to
have
that
like
precision
filter,
but
not
all
tools,
so
I'm
saying
like
for
code
ql
for
that
data
source.
That
might
that
could
be
a
great
idea
to
utilize.
You
know
coming
in
by
just
I'm
not
so
sure
about
other
ones,
but
but
yeah.
Hopefully
you
can
find
ideas
of
how
to
you
know,
get
that
precision
filter
out
for
other
ones,
as
it
goes.
B
I
would
just
put
my
two
cents.
I
would
definitely
say
that
a
huge
focus
would
be
on
getting
the
best
quality,
basically
reports
and
because
getting
a
ton
of
false
positives
will
just
run
into
that
issue.
That
bug
bounties
run
into
and
not
having
a
lot
of
actionable
data.
But
if
the
tool
can
and
can
be
engineered
as
best
possible
to
find
to
find
the
good
stuff
basically
yeah,
then
I
think
everything
else
that
comes
after
that
will
be
valuable
but
finding
those
finding.
That
would
be
probably
a
huge
focus
of
this
tool.
A
I
mean
you
kind
of
argue
that,
like
the
like,
the
automated
tooling
is
kind
of
like
github
code
scanning
or
whenever,
like.
A
Pick
a
static
analysis
vendor
goes
in
and
like
scans,
the
ecosystem
and
like
find
a
whole
bunch
of
stuff
and
write
to
paper
on
it
and
that's
great.
The
triage
portal
is
kind
of
like
I
haven't
seen
anything
like
that.
Publicly
private
sharing
is
all
ad
hoc
but
like
if
this
has
already
been
done
like
maybe
we
can
learn
from
that
or
just
kind
of
tail
onto
that,
but
I'm
I'm
not
aware
of
anything
that
kind
of
puts
this
all
together.
A
B
I'm
not
familiar
with
any
tools
specifically
like
this
that
have
been
built
specifically
the
triage
portal.
C
A
Yeah,
I
meant
more
like
even
like,
because
because
I
would
think
we
would
still
like
market
the
fact
that
we
are
doing
this
even
though,
like
most
people
would
never
have
access
to
the
actual
content.
Similarly,
I
I
haven't
heard
about
any
projects
that
that
are
doing
this
and.
C
C
A
Cool
we
have
the
five
minutes
left,
so
I
I
guess
the
the
next
I'm
gonna,
I'm
gonna
continue
to
kind
of
pitch
this
and
and
refine
it
a
bit.
I
think
ultimately
this
would
I
I
would
want
to
make
this
a
formal
pitch
to
the
probably
tac
first
and
then
governing
board.
If,
if
I
understand
the
right
process
and
then
to
see
if
openssf.
A
D
A
A
Yeah
and
that's
the
the
I
also
want
to
head
off-
well
not
head
off,
but
some
of
the
other
conversations
that
are
that
are
starting
to
go
on
about
direct
funding,
and
things
like
that.
I
I
I
don't
want
to.
I
don't
want
to
be
late
to
that
party
and
find
that
all
the
money
has
dried
up
for
those
kinds
of
things.
So.
A
You
know
it's
not
it's
not
a
silly
amount
of
money
to
do
this,
so
we
should
be
able
to
make
it
work
if,
if
either
open,
ssf
or
one
of
the
large
organizations
that
remember
open,
ssf
just
want
to
say
go,
do
we
can
make
that
happen.
A
Cool
awesome,
wonderful,
any
anything
else.
Anybody
would
like
to
chat
about
in
the
last
couple
of
minutes.
A
Cool
well,
thank
you
all
very
much
for
your
time
and
attention
and
all
that
and
see
everybody
again
in
about
two
weeks.