►
From YouTube: Security Tooling Working Group (May 24, 2022)
Description
C
C
C
I
mean
there
will
all
about
my
point
on
my
on
that
issue.
Is
there's
always
going
to
be
stuff,
you
want
to
know
there
isn't
data,
you
you've
got
to
just
use
expert
opinion
and
that's
okay.
You
know,
but
but
when
you
have
actual
real
information,
you
need
to
start
replacing
it
or
use
the
experts
to
explain
why.
In
fact,
what
seems
to
be
true
is
totally
false,
which
is
I
mean?
D
C
D
D
C
D
C
A
E
You
I
am
gonna
share.
I
have
a
couple
of
slides,
so,
let's
see
so,
if
you
are
seeing
my
spice
right,
is
that
correct?
Do
you
have
my
slice
shirt
yep?
We
see
yep
amazing
amazing,
so
I
have
a
couple
of
slides
here
showing
you
know
I
just
wanted
to
share
the
status
of
you
know
fuzz
introspector
with
you.
E
E
Basically,
it's
a
tool
to
get
overall
understanding
of
how
the
fuzzer
is
doing
and
how,
to
you
know,
improve
the
coverage
in
terms
of
you
know
performance,
so
it
monitors
the
fuzzer
performance
and
it
activates
different
kinds
of
data
like
it's
like
dynamically
collected
coverage,
data
like
hit
frequency
and
etc.
Also,
it
provides
understanding
of
you
know.
Digest
of
you
know,
static
data
right
yeah.
What
are
the
good
entry
points
to
write?
E
In
terms
of
fast
blockers,
we
may
identify
two
kind
of
blockers.
One
is
coverage
blockers,
which
are
the
places
of
the
code
that
the
fuzzer
fuzz
engine
basically
is
having
hard
time
to
bypass
and
go
beyond
to
reach
more
coverage.
Also,
it
may
have
you
know
performance.
You
know
blockers
like
shallow
failures
that
prevents
the
fuzzing
from
going
forward,
something
like
misconfiguration
in
terms
of
like
not
enough
resources
allocated
to
fuzzing,
etc.
E
Also,
like
uncapped
memory
allocations
that
are
causing
you
know
a
lot
of
memory,
so
all
of
these
can
be
identified
and
helpful
to
to
improve
the
function
so,
along
with
that,
introspector
also
provides
a
recommendation
in
terms
of
what
are
the
good.
What
are
interesting
entry
points
to
write,
fast
targets
for
to
reach
more
coverage
and
reach
more
complex
code.
It
also
these
are
important
to
to
have
in
terms
of
having
recommendation,
for
what
functions,
to
focus
if
the
current
fuzzing
is
basically
blocked
out
somewhere.
E
So
currently
we
are
pretty
close
to
finishing
the
integration
with
os
as
far
as
it
is
supporting
all
cnc
plus
plus
projects,
and
we
already
is-
we
already
are
seeing
its
effectiveness
in
terms
of
recommending
new,
fast
circus
and
pinpointing
where
the
coverage
plotters
are
and
as
a
result,
by
incorporating
these
you
know,
recommendation
and
information.
We
were
able
to
increase
coverage
for
a
couple
of
projects
that
I
will
provide
a
couple
of
examples
here
before
going
into
examples.
I
just
quickly
show
how
the
interface
looks
like
at
the
moment.
E
E
Different
summary
of
the
code
that,
like
what
functions,
are
the
most
complex
one.
What
are
they
being
hit,
which
one
are
not
being
hit
specifically
for
each
fast
target?
We
are
showing
how
it
is
reaching
its
aesthetically
reachable
potentials
in
terms
of
coverage.
If
they
are
not
reached
the
portion
is
marked
green
yeah.
This
will
a
table
is
very
important
in
terms
of
if
the
current
fuzz
fast
target
is
is
not
doing
good
where
the
functions
are
that
are
blocking
our
fuzzing
genes.
E
D
E
So
yeah,
that's
that's!
That's
basically
can
be
one
of
the
cases.
Another
case
it
can
be
just
a
simple
branch
that
your
fuzzer
is
not
successfully
providing
valid
or
meaningful
input
to
explore
all
its
subsequent.
You
know
sites.
For
example,
you
have
a
branch
that
say
if
the
value
x
is
like
bigger
than
five,
your
fuzzer
is
not
providing
over
five,
then
the
one
side
of
branch
is
not
taken,
so
that
kind
of
you
know
situation
can
be
a
blocker
for
a
fuzzer
as
well.
E
I
have
an
actual
real
world
example
in
my
in
my
next
slides.
Actually
so
another
thing
that
I
like
to
highlight
is
where
we
suggest
basically
new
entry
points
and
and
showing
that
if
we
write
false
targets
for
for
for
these
functions,
then
we
can
increase
the
the
the
coverage
of
the
functions
in
the
code
so
moving
forward
here
I
have,
I
have
two
examples.
One
is
for
xpdf.
E
Xpdf
was
interesting
to
us
because,
basically,
there
were
a
blog
post
by
project
zero,
deep
diving
into
the
nso
hack
into
I
imessage
users,
and
basically
the
the
vulnerability
was
in
xpdf
library
and
specifically
in
a
function
which,
even
though
the
xpdf
was
integrated
in
or
oss
was,
we
were
not
able
to
basically
trigger
that
part
by
our
already
infrastructure
code.
And
if
the
reason
was
that
the
oss
was
not
basically
covering
the
vulnerable
function.
E
So
we
looked
into
fuzz
intersector
report
for
the
xpdf
and
found
out
that
it
is
actually
reporting
a
new
entry
new
first
target
to
write
a
target
for
which
is
jbc2
stream
reset,
which
potentially
can
reach
to
the
vulnerable
function,
which
voice
was
not
covering.
So
we
moved
forward
and
wrote.
The
first
target
and
integrated
into
oss
was-
and
here
we
are
on.
The
left-
is
the
vulnerable
function
which
was
not
being
hit
at
all
and
on.
E
E
Another
project
is
where
like
jsonnet,
which
the
it
has
already
won
a
fast
circuit
integrated,
but
most
of
it's
called
code
potential
was
not
reached
and
it
was,
I
identified
that
one
function
is
called,
is
being
blocked
in
terms
of
coverage,
for
example
here
here
this
switch
case,
which
is
checking
on
the
variable
crime.
Always
one
side
of
the
case
is
taken
and
the
rest
of
the
cases
case
entries
are
not
taken.
So
basically,
this
is
one
example
of
you
know:
fuzz
blocker
josh,
referring
to
your
question
here.
E
D
E
I'm
sorry
these
are
yes,
I'm
sorry
about
that.
I
thought
I
I
had
to
give
more
provide
you
know.
Information,
so
is
the
number
by
the
lines
are
basically
number
of
times
the
fuzzer
hits
this
line
of
the
code.
If
it
is
zero
means
that
the
fuzzy
is
not
reaching
if
it
is
blue
and
some
numbers,
it
means
that
this
entry,
this
line,
is
basically
hit
that
many
times,
for
example,
for
xpdf
on
the
left.
You
can
see
that
the
the
vulnerable
function
is
not
hit
at
all
none
of
its
line.
E
No,
we
have
the
you
know,
branch
coverage
and
the
reason
that
I
I'm
showing
this
is
because
this
is
easy
to
visualize
in
terms
of
line
coverage.
So
so
I'm
just
using
the
you
know
a
standard.
You
know
line
coverage
before
so
yes,
this
is
actually
you
know,
populated,
based
on
the
current,
from
the
actual.
You
know,
fuzzer
that
we
are
getting
so
yeah.
So
this
is
an
example
that
the
blockers
are
identified
and
we
were
able
to.
E
You
know
basically
unblock
the
fuzzer
in
terms
of
you
know,
making
it
more
green
green
means
that
we
are
hitting
more
lines
of
the
code
or
more
branches
in
the
code.
So
so,
as
I
said,
we
are
integrating
the
introspector
into
oss
files
and
for
the
projects
that
are
c
and
c
plus
plus
right
now.
The
project
maintainers
can
see
introspective
report
on
their
portal
and
just
use
it
to
do
their
good
and
improve
their
fuzzing.
E
F
E
So
assembly
level,
assembly
level
yeah,
so
you
know
right
now
we
are
working
basically
from
for
the
oss
bus
and
for
osu's
project.
We
have
the
source
code
basically
and
and
the
the
two
main
source
of
information
that
inspector
relies
on
is
one
the
coverage
report.
So
as
long
as
you
have
the
coverage
report,
this
is
good.
Another
one
is
basically
if
we
use
an
eleven
plugin
to
instrument
the
builds
and
identify
the
you
know
call
graph
at
the
at
the
at
the
code
level,
so
aesthetically.
E
So
if
that
source
of
information
can
be
provided,
let's
like
another
way
so
right
now,
we
are
not
supporting.
You
know
binary,
that's
the
short
answer
to
your
question,
but
I'm
saying
that
if
we
can
provide
so
coverage
report
is
there
and
probably
if
you
can
extract
the
call
graph,
because
the
call
graph
is
important
for
recommending
new,
fast
targets
showing
is
saying
that
hey
these
couple
of
functions
has
potential
to
reach
a
bunch
of
you
know
code
that
is
not
already
covered.
E
F
E
Sure
definitely
we
can
follow
up
and
see
as
long
as
you
have
those
two
sources
of
information
coverage
and
you
know
call
graph,
then
we
can
fuse
in
the
the
post,
processing
of
fuzz
introspective
and
have
some
sort
of
you
know
report
for
for
your
campaign
for
your
fuzzing
etc
as
well.
So,
yes,
I
see.
C
Spectre
has
been
part
of
this
working
group
for
a
while,
but
I
don't
think
a
lot
of
people
know
about
it.
I
mean
I've
even
put
it
in
the
open,
ssf
decks
as
one
of
the
many
many
many
projects,
that's
in
open
ssf,
but
in
le,
but
you
know,
unless
you
read
the
fine
print
people,
don't
don't
notice,
it
looks
like
there's
enough
new
functionality,
that's
worth
kind
of
making
a
little
post
about
it.
Do
you
think
it's
time.
E
A
E
C
So
I
I
have
a
couple
requests:
if
you're
going
there,
okay,
one
one
is:
if
google
posts
google
can
post
whatever
it
wants.
Okay,
but
if,
if
there's
going
to
be
an
open,
ssf
post,
I
think
we
want
to
make
sure
that
people
take
a
look
and
in
particular
I
think
we
want
to
make
sure
the
attack
isn't
surprised
by
by
mystery
posts,
especially
since
you
know,
we've
got
new
tech
members,
they're
not
familiar.
A
lot
of
people
frankly
aren't
familiar
with
fuzz
introspector.
C
C
A
C
G
C
A
C
You
know
so,
basically,
I
think
we
need
to
give
a
little
more
hand-holding
so
that
they
understand
what
this
tool
is
for
and
how
one
might
use
it,
and
maybe
even
just
beyond
a
demo
of
what
you've
shown
in
the
hey.
I
found
this
here's,
the
here's,
a
specific
fuzz
target.
I
added
here's
the
code,
I
added
here's,
what
I
did
with
it.
We
don't
need.
A
C
E
Yeah,
I
see
so
that
you
know
that
that's
in
the
direction
of
you
know
having
documentation
and
we
spent
some.
You
know
efforts
in
terms
of
you
know,
documentation,
and
here
is
an
in-depth.
You
know
walk-through
for
the
cases,
for
example,
specifically
to
your
reference
that
you
said
that
hey,
we
wrote
this
exact
first
target
here.
It
is
basically.
E
B
E
C
E
H
Yes,
hi.
Thank
you.
I
mean
I,
you
know
full
disclaimer,
I'm
a
complete
newbie
in
this
kind
of
technology.
So
you
know
if
this
is
a
stupid
question
and
you
just
want
to
tell
me
to
go
rtfm,
that's
totally
acceptable
to
me,
but
you
you
know
to
me.
I
I
was
very
interested
in
what
you
showed,
but
it
seems
like
you're
skipping,
a
piece
that
for
me
I'm
like
this
is
magic.
It's
like
okay,
you
inform
me
on
what
my
further
doesn't
hit.
That's
fine!
But
how
do
I
actually
go?
H
E
Yeah,
exactly
very
good
question,
actually
very
good
question
right
to
the
point.
Basically,
that's
the
purpose
of
us
introspecting
in
terms
of
showing
you
that
hey
these
are.
You
know
places
that
your
father
is
not
doing
good
and
what
are
the
you
know,
immediate
steps
that
you
can
do
for
you
know
moving
forward.
As
I
said,
referring
to
the
examples
for
xpdf,
we
used
one
suggestion
from
fuzz
introspective
fast
interest.
E
Victor
provides
some
suggestions
for
the
functions
that
if
you
write
fast
targets,
then
you
get
this
many
like
this
percentage
of
you
know,
coverage
increase,
so
that's
one
one
approach
to
go.
Another
approach
is
basically
using
the
blocker
identified.
So
if
you
dive
in
into
the
exact
location
that
the
code
is
not
going
going
beyond,
then
you
can
understand
that
hey.
This
is
the
thing.
The
next
thing
that
I
can
do
like
this.
E
So
what
I
did
here
was
that
I
I
jumped
in
into
the
function
that
the
introspector
was
pointing
me
to
that
fuzzer
is
having
hard
time
to
bypass
and
identify
this
switch
case
to
be
imbalanced.
Basically,
like
some,
some
entry
is
being
hit,
but
the
rest
is
not
being
hit
so
then
finding
the
root
cause
is
the
next
step
to
see
why
that's
happening.
So
that's
so.
Basically,
that's
the
direction
that
you
know.
Fuzz
introspective
gives
you
or
or
sheds
the
lights
on,
and
then
you
can
move
forward.
C
C
Called
but,
of
course,
the
problem
with
that
is
that
it's
not
being
called
in
the
same
context
necessarily
another
is
you
know,
hey,
you
know,
I'm
never
calling
b
a
calls
b
at
least
statically.
Why
is
a
never
actually
calling
it
and
basically
finding
a
way
to
convince
you
to
provide
the
data
necessary
to
trigger
the
a
calling
b
path,
if
that's
the
only
bat?
So,
basically,
you
have
to
go
backwards
to
see
what
calls
it
and
find
a
pat
a
way
to
enable
that
path
within
your
fuzz
processing
process
did
capture
that.
A
C
E
C
E
Right
right
so
yeah
we
are,
you
know,
doing
efforts
to
make
it.
You
know
even
further
in
terms
of
you
know,
going
beyond
this
just
pure
suggestions
and
taking
actions,
for
example,
for
lip
puzzle.
We
have
this
focus
function
right,
so
we
have
identified
some
function,
which
is
not
you
know,
covered
very
good.
Then
we
pass
it
in
the
we
have
a
feedback
from
introspector
into
lip
faster,
saying
that
hey
focus
on
this
function
and
see
how
it
goes
so.
H
C
E
E
E
Why
is
that
I?
I
did
not
provided
a
lot
of
you
know
implementation
detail
here,
but
the
reason
is
that
we
have
a
compiler
plugin
for
for
for
for
a
static
part
of
it
and
in
terms
of
extracting
you
know,
call
graph
and
etc,
and
the
compiler
plugin
is
lvm
pass
at
the
moment
and
it
supports
cnc
plus
plus,
and
we
lift
you
know
the
code
into
llvm
ir
and
do
the
you
know
rest
of
analysis.
E
So
the
next
step
for
us
for
adding
you
know
a
new
language
like
python
ancestry
is
that
we
we
basically
shift
that
in
a
static
analysis,
part
to
support
new
languages.
So
that's
the
reason
at
the
moment
we
are
supporting,
cnc,
plus
plus,
and
basically
our
our
focus
where,
on
you
know,
memory
unsafe
languages.
That
was
you
know
that
made
sense
to
focus
as
the
first.
You
know,
step
towards
what
the
introspective
is
aimed.
D
And-
and
I
have
one
last
question
I
mean
so
sure
you're
pretty
like-
can
you
click
on
slide
11
as
long
as
you're
sharing
your
screen?
Oh
yeah,
yeah!
Oh
here,
are
you
publishing
these
pretty
pictures
anywhere
because
I
know
like
oss
fuzz
runs
across
a
huge
amount
of
open
source.
Do
we
know
like
what
those
look
like
for
a
bunch
of
packages.
E
E
D
C
D
E
D
E
D
C
C
I
I
I
will
say,
and,
and
I
I
just
had
a
conversation
post
on
the
attack
josh-
I
I
think
every
working
group
should
at
least
periodically
have
the
projects
within
it.
You
know
report
on
all
the
cool
stuff
they're
doing,
because
I
I
think
we
often
miss
nuggets
like
this.
So
this
is
this.
Is
this
is
perfect.
Thank
you.
Naveed.
D
C
Right
now,
the
main
problem
is
that
I
was
overwhelmed
with
that
washington
dc
meeting.
So
I
haven't
I
I
had
to
focus
on
that.
So
that's
so
we
haven't
made
as
much
progress.
Probably
the
big
one
of
the
bigger
questions
here
is:
there's
some
scoping
questions
you
know.
Do
we
want
to
include
all
linters?
Are
we
really
more
focused
on
you
know,
security
related
things.
I
I
I'm
not.
C
F
I
think
my
point
is
that
sometimes
the
line
between
security
related
to
all
and
the
generic
linter
is
blurry,
since
sometimes
some
linters
start
bringing
a
security
related
rules,
so
I
don't
see
a
reason
for
distinguishing
or
for
limiting
this
standard
for
security.
Only
tools
I
think
we
can
make
we
can
make
good
by,
including
or
but
not
limiting,.
C
F
F
C
Right,
I
think
if
so,
what
we'll
need
to
be
careful
of
is
if
we,
if
you
disable
up
a
false
positive,
it
needs
to
be
once
you
see
some
code
that
then
needs
to
be
soon
quickly,
re-enabled.
Otherwise,
if
it
keeps
hunting
for
the
next
thing
to
dis,
suppress
that
maybe
many
many
lines
down,
you
may
suppress
the
wrong
thing.
D
I
mean
I
I've
got
no
skin
in
this
game
david,
but
yeah.
Okay,
that's
fine!
Any
any
time.
We
do
things
that
are
security
only.
I
feel
like
they
often
end
up
falling
off
the
cart,
because
people
get
annoyed
by
them
or
they're,
not
providing
enough
value
to
certain
groups,
whereas
when
you
combine
abilities
or
features,
I
think
it
often
becomes
an
easier
sell.
C
Sometimes,
but
you
know
what
I
don't
think
that
we
need
to
have
that
fight.
If
we
cover,
if
we,
if
we
cover
it,
then
then
we
cover
it
so,
and
I
think
the
main
concern
I
have
is,
if,
if
you
say,
suppress
the
next
warning
and
the
next
warning
is
50
lines
down
the
weeks
you
just
you
know,
ignore
the
you
know,
a
suppression
should
be
with
the
line
immediately
related
so
that
we
don't
accidentally
suppress
the
wrong
thing.
That's.
C
Yeah
well,
what
I'm
thinking
of
is,
when
you're
implementing
the
code
to
say,
suppress
the
next
message,
an
easy
way
to
code.
This
is,
I
notice
the
suppression
message,
the
next
time.
There's
a
warning
I'll
suppress
it,
but
there
are
no
warnings
and
300
lines
down.
You
finally
get
a
warning:
it's
the
next
warning.
It
got
suppressed.
C
Well
that
wasn't
what
was
meant
so
I
think
we'll
just
have
to
make
sure
and
that
becomes
way
more
common
when
you
have
such
different
tools,
so
I
think
we
just
have
to
be
more
cautious
and
warn
if
you
implement
suppression.
You
know
it
needs
to
not
do
that,
so
I
I
think
we
can
word
it.
We
just
have
to
make
clarify
that.
D
Perfect
all
right,
so
the
last
thing,
then,
is
a
document
I
wrote,
let
me
open
it
and
I'll
put
a
link
to
the
chat.
So
the
reason
I
took
this
group
over
for
anyone
who
wasn't
here
for
the
early
meetings
was
there
is
a
report
resilient
research
did
which
actually
it's
somewhere
in
the
notes.
If
you
scroll
down.
D
I,
of
course,
can't
find
it
now
all
right.
Whatever
it
doesn't
matter.
Oh
there,
it
is
all
right
all
right,
so
what
I
would
really
like
to
see
us
do-
and
this
is
very
related
to
what
we
just
saw
from
navin
about
the
fuzzing
work,
where
we
have
a
lot
of
tools
today
and-
and
I
know
one
of
the
conversations
in
this
group
is
always
oh,
what
should
the
group
do
and
there's?
D
Oh,
we
should
create
lists
of
tools
like
there's
millions
of
listed
tools
right
whatever,
but
resilient
did
this
blog
post
and
it
they
measured
tools
to
detect
log
for
show
and
the
powerful
part
about.
I
think
what
they
did
is
a
bunch
of
organizations,
the
one
I
worked
for
ancore
included.
We
looked
at
this
list
and
we
said
we
can
detect
more
of
this
stuff,
and
so
we
did-
and
I
would
love
to
see,
efforts
done
where
similar,
I
guess
tests.
I
don't
know
exactly
what
to
call.
D
It
are
created
for
a
variety
of
tools,
in
my
case,
I'm
most
interested
in
like
s-bomb
today,
because
that's
just
what
I'm
working
on
most
of
my
time
and
having
the
ability
to
basically
say
like
here's,
a
bunch
of
s-bomb
tools,
here's
what
they
detect
and
here's
what
they
don't
detect
and
it's
not
meant
to
be
like
this-
is
the
best
spam
tool.
That's
not
at
all
what
I
want
to
do
with
this.
What
I
want
it
to
be
is
like
here's,
a
bunch
of
stuff
s
bomb
tool,
here's
what
you
detect!
D
D
So
there
there's
different
ways
to
kind
of
look
at
this,
but
anyway
I
I
finally
found
a
few
minutes
to
sit
down
and
start
just
kind
of
writing
some
of
this
down-
and
I
realized
like
there's
more
going
on
like
the
fuzzing-
is
a
great
example
right.
We
have
the
fuzzing
introspection
project
like
we
should
keep
track
of
that
and
understand
what
is
detecting
and
what
it's
not
detecting
and-
and
I
think
you're
doing
a
fine
job
of
that
today.
D
So
I,
I
obviously
don't
want
to
like
be
a
pest,
but
you
kind
of
get
the
idea,
and
so
I
just
started
laying
some
things
out
and
I
thought
if
anyone
has
ideas
or
thoughts
and
comments,
I
would
love
david.
Is
there
a
way
we
can
move
this
document
like
into
the
this
project?
This
is
currently
just
living
in
my
google
docs,
which
is
stupid
right
because
that
that's
not
sustainable,
but
that's
we
can.
We
can
figure
that
out
later
too.
C
C
A
I
C
Agree
so
so
yeah
you
can
ask
jory,
but
really,
I
think
right
now.
You
know
start
you
know
start
whatever
you
want
to
do
and
we
will
fix
fix
the
ship
as
it
goes,
but
I'd
much
more
just
if
you
think
this
is
something
that's
worth
doing
now
I
will
say
this
has
been
a
little
bit
of
a
challenge.
Measuring
all
tools
is
kind
of
a
big,
that's
a
big
thing.
You
might
want
to
start
with
one.
D
Absolutely,
and
that's
ex-
I
even
mentioned
that
somewhere
down
below
of
like
not
everything
and
and
additionally,
we
need
to
interact
with
groups
that
we
want
to
work
with,
because
you
can't
just
show
up
and
be
like
hey.
I
measured
your
tool,
look
what
I
did
they're
going
to
be
like
who
the
hell
are
you
and
what
are
you
doing?
You
know
it's
that
never
ends.
Well.
I've.
I've
been
open-sourced
long
enough
to
know
that
surprising
projects
does
not
make
friends.
D
So
it's
it.
This
is
a
big,
ambitious
thing.
I
I
get
that,
but
it's
it's
it's.
It
is
the
reason
I
wanted
this
group
to
survive,
because
there
was
a
lot
of
question
about
like
what
should
this
group
do?
What's
its
future
and
like
this
is
what
I
I
want
to
do
now.
That
doesn't
mean
it's
the
only
thing
we
have
to
do
of
course,
but
it
I
I
don't
have
a
lot
else
to
say.
I
think
you
know
read
it.
I
I
value
thoughts.
D
If
there's
other
things,
people
want
to
work
on,
but
like
fundamentally,
what
I
want
to
see
is.
I
want
to
see
this
group
not
have
meetings
every
other
week
and
like
that's
it
right.
I
want
to
see
us
doing
work
off
to
the
side
and
and
creating
progress,
and
then
this
meeting
is
where
we
just
talk
about
the
cool
stuff.
We
just
did
not,
unlike
the
recent
presentation
we
had
and
I'm
causing
introspection
so.
C
And
also-
and
frankly,
I
would
add-
also
answering
questions
about
hey.
I
have
a
big
question
about
the
direction
you
know
like
the
should
this
include
lenters,
I
mean,
I
think,
the
the
scope
of
project
before
you
delve
in
I
think
is
a
good
time
to
have
a
larger
gathering
but
you're
right
most,
the
actual
work
needs
to
be
somewhere
else.
C
And
you
know
what
we
already
we
already
do.
We
actually
already
have
a
project
specifically
within
this
working
group.
That
does
that.
But
it's
been
kind
of
moribund,
but
basically
some
other
folks
had
identified
a
whole
bunch
of
existing
cves
and
then
you
know
before
and
after
and
the
question
is:
can
your
tool,
here's
a
known
vulnerability?
C
Can
your
sas
tool
find
it
and
frankly,
I
think
you
could
probably
make
the
same
question
of
dast
as
well,
but
you
know
the
fuzzers
and
if
it's
a
web
app
for
web
application
scanners,
I
have
I'm
careful
about
das
because
different
p,
everyone
agrees,
that's
a
topic
and
nobody
agrees.
What
the
definition
is
so.
D
C
C
D
Absolutely
and
and
now
on
that
note,
this
is
the
second
part
of
what
I
wanted
to
well.
I
guess
the
second
problem,
I'm
concerned
with
is
are
there
measuring
frameworks
that
exist
because,
like
I
don't
want
to
write
a
measuring
framework,
I
mean
there's
already
one
here.
I
guess
in
the
cbe
benchmarking
project
and
maybe
just
trying
to
build
off
that
to
measure
other
things
is
the
right
way,
but
I
I
don't
want
us
to
have
10
projects
each
with
their
own
measuring
framework
they
build
because
that's
dumb
right
like
no
one.
D
C
C
Here,
I'll
I'll
even
throw
the
link.
This
is
the
wrong
part
in
the
document,
but
hopefully
you
can
fix
it
and
put
it
in
the
right
place:
josh,
okay,
so
the
summate
folks
issue
a
whole
bunch
of
sample
code
of
you
know:
here's
a
vulnerability.
Here's
vulnerable
code,
there's
not
vulnerable
code
number
cases
they
have
pairs
like
this.
Here's,
the
not
vulnerable
most
of
it,
is
what
they
called
synthetic.
C
In
other
words,
it's
not
real
programs.
It's
you
know.
Somebody
wrote
a
program
to
show
the
difference
between
vulnerable,
not
vulnerable.
That's
really
different
than
the
cde
benchmarking
that,
but,
but
I
think
the
two,
the
two
actually
pair
pretty
well
the
synthetic
stuff.
The
advantage
is
they're
really
simple
and
short,
so
it's
much
easier
to
do
quick
analysis
and
assign
them
in
as
homework
projects
the
cve
benchmarking
they're
real
programs,
much
bigger
more
real.
C
So
I
I
think
those
and-
and
I
think
summate
has
some
of
the
non-synthetics
too,
but
I
think
the
two
together
are
very
useful,
at
least
for
sas.
I
don't
know
how
I
don't
know
if
anybody's
used
them
seriously
for
the
dynamic
stuff
I
mean
I
could
call
up
paul
black
at
nist,
and
he
could
tell
me
more
so
I
actually
have
the
context
there.
If
you
want
to
have
them,
give
us
a
presentation.
C
A
C
I
C
C
Right
and
for
the
summate
folks
actually
have
reached
out
to
open
source
tools
and
run
some
of
them
against
their
their
tool
set,
and
you
know
hey,
I,
I
ran
one
of
the
tools
and
my
tool
didn't
didn't,
find
a
lot
of
stuff
either,
and
I
didn't
expect
it
to
so,
but
yeah
I
I
think
you
know
transparency
and
what
things
act.
How
things
actually
are
is
a
good
thing.
C
D
Be
shocked
if
they
do,
I
mean
I
think
this
is
one
of
the
challenges
we
get
in
this
space
also
is
like
a
lot
of
people,
get
a
cool
idea
and
they
kind
of
work
on
it,
and
it's
really
hard
to
just
discover
what
anyone's
even
doing-
and
I
think
that's
one
of
the
values.
I
think
that
open
ssf-
and
I
think
this
group
specifically
should
strive
to
do
is-
is
sometimes
we'll
get
involved.
D
C
G
Just
a
housekeeping
issue-
I
don't
know
if
I
know
that
some
of
the
groups
were
updating
their
charter,
updating
their
mission
statement.
That
type
of
thing
do
we
need
to
do
any
housekeeping
like
that
as
part
of
this
and
david.
How
do
you
envision
trying
to
now
kind
of
reconcile
the
former
scope
of
the
open
ssf
to
the
scope
host
the
meetings
associated
with
the
white
house
meeting
just
a
week
back.
C
Go
okay!
Let
me
attempt
to
tackle
the
second
one
which
was
cut
through
me
for
a
loop,
because
there
is
no
planned.
The
the
scope
of
the
open
ssf
is
anything
rel
involving
open
source
software
security
that
hasn't
changed.
Okay,
so
you're,
saying
the
hey,
the
white
house
changed
the
scope.
No,
it
didn't
change
the
scope
at
all.
C
More
resources
and
more
things
we
can
do
and
more
things
that
people
want
the
openness
of
to
do
within
that
big,
broad
scope.
But
that's
that's
a
different
statement.
So
let
me
let
me
just
say
that
I
don't
think
the
meeting
was
about
hey
we're
going
to
eliminate
the
open
ssf
or
it's
the
it's
wrong.
It's
I
think
it's
enabling
the
openness
to
have
to
do
more
of
the
things
it's
been.
G
Is
that
if,
if
you
say
our
scope
of
the
open
ssf
is
open
source
security,
it's
like
saying
we're
going
to
solve
world
hunger
right,
so
you've
got
to
basically
say
and-
and
this
is
my
conception
is
that
when
they
redid
the
governing
board
back
in
the
fall
and
then
they
have
recently
like
reassigned
re
sort
of
elected,
a
new
attack,
there's
an
opportunity
to
look
at
the
existing
working
groups
and
this
increased
resource
from
the
recent
white
house
meeting
and
say
how
does
that
affect
each
working
group?
Now,
maybe
some
working.
C
C
Right,
okay,
so
it
so
it
sounds
like
really
the
proc
we're
asking
is:
okay,
we
had
a
meeting
with
the
white
with
the
white
house.
You
know
how
does
that
affect
the
various
working
groups
and
parts
of
the
open
ssf,
and
I
I
think
the
the
the
shorter
answer
is,
that
is
in
the
process
of
being
worked.
I
mean
as
soon
as
you
talk
about
hey:
where
does
the
open
ssf
put
money?
Well,
first
step,
it
depends
on
who,
wherever.
C
I
think
a
couple
pieces
of
that
are
probably
more
government
roles,
in
which
case
governments
governments
have
roles
to
play,
and
you
know,
presumably
that's
what
the
governments
will
do,
but
for
the
for
a
lot
of
it,
I
think
the
expectation
is
that
the
open
ssf
is
going
to
do
a
whole
lot
of
it
and
we're
still
in
process
of
working.
That
out,
you
had
two
questions
and
that
second
one
was
so
big
and
huge.
I
was
struggling
to
figure
out
how
to
answer
it.
D
I
I
looked
over
what
we've
got
we're
actually
in
better
shape
than
a
lot
of
the
working
groups,
and
I
think
what
we're
doing
still
technically
falls
under
the
charter.
We
should
definitely
review
it
at
some
point,
but
I
don't
want
to
yet
because
I
think
we're
still
working
out
a
lot
of
kind
of
intent
and
purpose,
but
once
we
have
that
yes,
we
will
absolutely
update
the
charter
to
reflect
what
we
want
this
group
to
do
and
how
we
want
it
to
work.
C
I
it
seems
to
me
that
I
mean
in
in
the
longer
term,
there
probably
needs
to
be
something
more
written
down,
but
a
whole
lot
of
working
groups
work
the
same
way,
so
I'm
hoping
that
the
tac
can
kind
of
work
out
a
more
general
one
and
then
say:
hey
working
groups.
If
you
like
this
go
otherwise,
we
need
to
somehow
move
between
too
much
process
and
no
process.
We
can't
figure
out
how
to
get
anything
done
so
yeah.
H
So
if
I
may,
I
mean
taking
another
crack
at
what
I
think
jeff
was
touching
on,
but
in
a
more
narrow
you
know
more
narrow
fashion.
It's
like
you
know.
I
think
it
is
a
good
good
exercise
for
the
working
group
to
look
into
how
what
we
do
here
gets
positions
with
regard
to
the
work
streams
that
are
in
the
plan
that
was
presented
in
washington,
and
so
that's
a
bit
different
than
I
mean
david
and
I
were
well.
H
H
C
C
H
C
Yeah
so
yeah-
I
guess
maybe
we're
out
of
time,
but
maybe
that's
one
of
the
agenda
items
we
can
talk
about
next.
Next
time
is
hey.
Looking
at
that,
are
there
any
pieces
that
this
particular
working
group
thinks
are
really
within
its
remit
and
should
say
hey,
let's
we'd
like
to
pick
up
this
part
of
the
puzzle.