►
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
Welcome
to
the
identifying
security
threats
working
group
for
february
18th,
so
we've
got
some
stuff
coming
up
next
week.
There's
a
open,
ssf
town
hall.
I
hope
everybody
has
registered
for
that,
and
plans
to
attend
will
be
slides
from
each
of
the
different
working
groups
and
kind
of
all
up
on
our
accomplishments.
Over
the
past
quarter.
I
think
quarter
a
little
bit
more
than
a
quarter
so
looking
forward
to
to
kind
of
seeing
that
we
also
have
john
who's
new.
B
Yeah,
so
I'm
john,
I
work
for
intel.
I
did
some
similar
work
to
this
and
I
presented
that
at
b-sides,
2019
or
besides,
pdx
2019
I'll
just
put
a
link
in
the
chat,
so
you
guys
can
go
look
at
that.
But
basically
you
know
dependency
review
internally
at
intel
and
we
had
this.
B
You
know
review
process
and
and
then
we
figured
out
we
automated
it
using
some
machine
learning
and
stuff,
and-
and
so
I
thought-
hey,
you
know,
and
then
we
expanded
on
that
a
little
bit
and
we
wrote
something:
that's
a
little
more.
Like
the.
I
think,
the
code
that
was
associated
with
this
repo
and
so
I'm
curious
to
see
if
we
can
get
some.
You
know
some
sharing
of
code.
There.
A
Awesome
yeah
that
that
that
sounds,
terrific,
we'll
save
some
time
to
kind
of
talk
more
about
that
a
little
bit
a
little
bit
later
today,
we
do,
let's
see
so
k,
is
not
on
the
call
okay,
so
we're
gonna
move
this
one
to
to
next
time
there
have
been
some
discussions
on
if
we're
going,
to
use
kind
of-
let's
see,
let's
just
say
outside
to
the
working
group-
help
to
build
out
the
security
metrics
project,
so
you
know
to
kind
of
take
it
beyond
the
current
proof
of
concept.
A
I
think
now
is
a
perfect
time
john,
to
have
a
conversation
on
like
what
makes
sense
all
up
like
is
there
you
know?
Does
it
make
sense
to
continue
the
proof
of
concept
in
the
current
form,
or
is
that
just
proof
of
concept
and
we
need
to
like
you
know,
either
build
something
else
or
leverage
something
else
or
or
whatever?
A
And
you
know
that
that
will
to
continue
to
have
these
conversations,
so
I
don't
have
too
much
new
on
the
metric
stuff,
although
I
I
guess
just
for
for
for
john-
maybe
I'll.
Do
a
super
super
short
demo
of
of
what
that
dashboard
currently
looks
like
cool.
A
Yeah,
so
so
so
kubernetes
is
has
so
the
the
openness
practices
badge
program
which
david
runs
is
it
has
data
that
we've
collected
so
that
that's
this
kind
of
section
down
here
the
openness
of
scorecard
metric,
which
is
the
scorecard
project
under
openssf,
collects
data
which
this
shows
this
stuff
here
in
the
middle
is
data
that
we
collect
directly
from
github.
A
So
it's
you
know,
project
releases
over
time,
and
you
know
it's
what
the
it's,
what
the
words
say
and
then
up
here
is
the
description
I
believe
from
the
open,
soft
best
practices
area.
So
all
this
stuff,
what
the
repo
does?
The
repo
just
kind
of
slurps
the
stuff
in
throws
it
into
basically
one
flat
database
table
and
then
each
of
these
queries
again.
A
So
I
can't
really
show
you
at
the
moment,
but
this
query
is,
like
you
know:
does
this
key
this
project
name
and
this
key
have
a
value
of
one
or
true
or
something,
and
then
that
chose
a
check
mark.
So
it's
super
super
simple,
but
it
was
designed
to
kind
of
get
the
conversation.
C
So
so,
since
I
haven't
written
my
amazing
email,
I
wanted
to
comment
that
you
know
the
best
practices
badge
and
the
scorecard
metrics
great,
the
little
graphs
cool,
but
I
kind
of
question
the
value
of
some
of
these
other
metrics
in
the
middle.
I
think
it'd
be
better
just
to
drop
them
at
least
for
now
replacing
them
with
something
more
useful.
If
so,
but
you
know
like
the
the
months,
I
I
think
something
about
hey.
C
C
I
think
the
number
of
contribute
contributors
over
a
year
is
probably
good,
but
I
don't
know
about
the
90
days
and
I
don't
I
I
don't
believe
the
issues
open
and
issue
duration
has
really
any
value.
I
will
say
for
the
best
practices
badge
we
close
issues
when
either
we
resolved
it
or
we've
decided
not
to
do
it,
but
we
have
issues
that
are
old,
that
you
know
they're
low
priority,
we'll
get
to
them,
maybe
but
yeah.
You
know
I'm
not
sure
that
those
numbers
will
be
helpful.
B
C
There's
no
convention
for
say
I
you
know
well
anyway,
I
I
I
think
we
want
to
emphasize
right
now,
just
the
metrics
that
are
more
likely
to
be
yeah.
I
kind
of
agree
that.
D
C
A
C
I
got
I
don't
I
don't
want
to
kill
the
exp.
I
I
get
the
ability
to
to
vary,
but
I
I
do
think
that
at
least
for
an
initial
showing,
if
they're
too
dubious
people
will
focus
on
the
dubiousness
instead
of
the
oh.
I
like
this
yep.
A
C
Okay,
yeah,
but
I
I
love.
I
love
this
overall
idea
now,
of
course,
as
you
know,
what
I
want
to
do
is
I
want
to
see
this
for
all
open
source
projects
period,
so
you
know,
go
to
the
website
and
you
type
in
the
project.
Names
and
it'll
show
you
all
the
matching
and
you
click
on
the
what
you
meant
and
there
it
is.
I
realize
that
might
not
be
today,
but.
A
A
Caching
right,
you
know,
so
there's
no
reason
why
we
can't
do
that
and
there's
also
no
reason
why
this
couldn't
be
available
as
a
tool
like
scorecard,
where
it
really
is
something
that
you
run
yourself
on
your
own
things
or
things
that
you
care
about
so
experiment
with
lots
of
different
designs.
C
Right,
I
think
the
challenge
here
would
be.
I
I
think
that
running
on
your
own
is
great,
but
a
lot
of
people.
It
is
a
big
difference
being
able
to
click
on
a
website
versus
download
install.
Oh,
it
doesn't
install
correctly
or
oh
my
network.
Doesn't
let
that
happen
or
whatever
you
know
having
a
url
is
a
tremendous
advantage.
Even
if
people
eventually
decide
to
install
it
locally.
A
Yeah,
I
figure
if
we
have
the
it
if
it's
simple
enough
to
run
as
a
command
line
tool,
we
can
just
glue
it
to
the
website
and
then,
like
you
know,
that'll
be
fine
cool
okay,
so
I
won't
spend
most
of
today
so
we'll
defer
this.
A
I
want
to
talk
about
security
reviews
because
this
is
imminent
and
and
public
and
and
all
that
we
do
have
some
some
work
and
some
decisions
to
do
so.
For
those
of
you
who
don't
don't
know,
we
went
live
last
week
with
the
secure
reviews
project
that
we've
been
talking
about.
A
So
it
is,
we
changed
the
the
url
to
the
lowercase
security
dash
reviews,
so
it
could
be
more
easily
well,
so
it
just
looks
nicer,
so
we've
got
a
couple
different
reviews
in
in
there
now.
Thank
you
dylan
for
doing
some
of
the
mpm
ones.
Thank
you.
Sorry.
The
the
630
area
code
number
that
just
held
in
hello
hi
there
that's
amir
calling,
oh
hey
amir,
great
hello,
dear
hey.
A
We
just
want
to
thank
you
for
the
for
the
linux
linux
kernel
review
that
you
throw
in.
C
A
F
By
the
way
like
there's,
I
don't
know
if
you
saw-
and
I
can
do
this
with
other
stuff
too-
I
don't
know
if
we
decided
how
much
we
want
to
look
at
these
like
public
widely
available
ones,
but
I
there's,
I
think,
400
something
npm
modules.
I
have
a
separate
repo
because
I
want
to
add
a
pull
request
with
like
500
things,
for
you
to
like
look
at
probably
seem
confusing,
but
but
yeah
we
can
do
that
and
then,
if
you
think
that's
good,
I
can
do
that
with
other
kind
of
popular.
G
A
And
against
it
for
different
reasons,
so
so
yeah,
so
so
we
have
that
and
in
particular
oh
we
got
another
pull
request.
Thank
you,
yeah.
Thank
you
luigi.
So
I
added.
H
Yes,
I
we
need
to
edit
the
configuration
in
our
repo,
of
course,
but
in
my
I
have
tested
my
github.
My
personal
github
account.
Of
course,
the
redirecting
my
personal
github
account.
Don't
work
well,
because
I
have
connected
another
domain
to
my
github
domain,
but
the
redirects
run,
and
I
it
should
be
right.
I
suppose
I
hope
we
can
try,
maybe
in
if
it
is
don't
work.
A
Perfect
perfect,
so
so
what
do
I
do?
I
I
commit
this
to
the
like
gh
pages
branch
and
then
it
shows
up
or
is
that
how
you
know
it
it
doesn't
matter.
We
can
chat
later
about
the
specifics.
I'm
just
curious
as
to
I
should
know
this
already,
but
how
github
pages
actually
like
knows
what
to
publish.
C
Yeah
it's
within
the
branch
and
then
it
it
knows
how
to
generate
in
certain
cases.
But
in
other
cases
you
just
need
you
need
to
generate
the
html
got
so.
A
Okay,
oh
I
got
it
okay,
so
I
do
a
build
from
at
various
points
in
time.
Push
the
results
to
the
gh
pages
branch
and
then
google
github
does
the
does
the
magic.
C
Right
github
does
the
matching
now,
if
you
want
it
to
show
up
with
a
particular
domain
you
if,
if
you
just
leave
it
be,
it
shows
up
as
the
name
of
the
name
of
user,
which
is
in
this
case
is
ossf
and
the
name
of
project
we
can
also
make
it
show
up
as
a
website
on
its
own.
That
does
require
some
additional
dns
setups,
which
I
have
done
actually
so.
A
Cool
we
can,
we
can
do
whatever
cool
and
then
last
night
I
added
a
couple
more
reviews,
and
one
of
them
I
mean
so
some
of
them
are
are
kind
of
not
very
interesting.
I
mean
one
on
city,
hash
and
clap,
but
this
one
is
actually
the
more
interesting
one.
So
I
want
to
give
you
guys
come
on,
not
a
diff.
I
just
want
to
see
the
file.
C
A
So
you
guys
are
familiar
with
the
dependency
confusion,
blog
post
that
went
out.
A
Was
last
week,
so
we've
been
finding
a
lot
of
attempted
attacks
here.
So
what
this
review
here
is
is
all
of
those
packages.
Well,
packages
and
versions
were
created
by
a
let's
say,
security
researcher,
whether
well
it
looks
like
he's
looking
for
bug
bounty
money.
A
He
or
she
is
looking
for
bug
bounty
money,
but
you
know
I
wrote
this
up
after
the
packages
have
been
removed
from
npm,
so
there's
no
risk
to
describing
the
attack,
but
I
thought
that
that
this
would
at
least
because
right
now,
there's
no
public
discussion
of
this
because
they
are
just
evaporated
from
the
npm
repository.
A
So
I
thought
so
one
of
the
questions
is:
do
you
guys
think
that
that
these
types
of
reviews
are
relevant
because
they're
not
I
mean
they
are
kind
of
against
the
package?
But
it's
usually
like
a
group
of
packages.
B
D
B
It
seems
to
to
branch
into
sort
of
a
you
know:
an
exploit
space
that
might
be
a
different
sort
of
scope.
C
A
A
I'm
sorry,
then,
I'm
kind
of
just
rewriting
alex's
blog
post,
because
that
was
that
was
the
hey.
This
is
an
endemic
problem
in
how
enterprises
consume
from.
B
A
C
C
I
would
hate
to
lose
this.
I
agree.
It's
not
it's
not
really
what
the
others
are
about
right,
but
yeah.
Why
don't
we?
I
like
the
idea
of
an
uncategorized
and
we'll
try
to
figure
out
and
who
knows,
maybe
maybe
what
we
really
need
is
a
separate
group
that
works
on.
You
know
countering
supply
chain
attacks
and
we
move
that
document
over
there.
A
C
A
Yep
totally
great,
which
is
actually
why
the
the
one
that
the
post
I
thought
was
was
was
great.
Let
me
go
there.
A
B
So
so
there's
no
there's
no
standardized
methodology
across
these
reviews.
B
That's
probably
gonna,
be
I
I
quickly
foresee
that
becoming
a
problem
to
compare
sort
of
apples
to
apples
and
different
reviewers
and
stuff,
because
this
is
a
problem
we
suffered
from
it
until
was
even
when
you
have
a
standard
methodology,
you
have
a
lot
of
subjectivity
sometimes,
and
it
becomes
very
hard
to.
It
becomes
very
hard
to
to
weigh
things
very
quickly.
C
Think
the
problem
there
is
in
order
to
have
a
standard
methodology,
we
would
have
to
have
broad
industry
consensus
on
what
that
methodology
is.
I
think,
right
now.
The
problem
that
mike
is
trying
to
counter
is
there's
generally
no
information,
it's
incredibly
hard.
You
know
if
you
just
use
google
you're
very
unlikely
to
find
a
particular
security
review
and
so
having.
B
C
C
Maybe
yeah
I
I
used
to
work
with
the
common
criteria
and
trying
to
agree
on
a
methodology
for
doing
security
evaluations.
I
think,
is
way
harder.
E
Option
to
categorize
the
type
of
review
to,
I
think,
will
help
a
ton
and,
given
the
the
kind
of
the
main
idea
is
to
have
a
repository
to
inform,
I
think
it'll
certainly
do
the
job
and
the
template.
The
template
that
you
came
up
with
michael
on
the
github,
I
think,
is
solid
as
well
just
to
kind
of
keep
it
consistent
in
terms
of
the
theme
and
where
to
find
the
information
and
what
it
looks
like
yeah,
yeah.
C
C
B
A
I
think
oh
okay,
like
the
the
the
devil,
is
kind
of
literally
in
the
details
you
know
is,
is:
is
this
kind
of
review
differentiable
from.
A
You
know
this
kind
of
review
and,
and
does
it
need
to
be
at
a
programmatic
level
or
is
really
the
person
like
is
effectively
folks
workflow
going
to
be,
I
use
zealand.
Do
you
know
anything
about
zlib?
Oh
yeah,
you
can
look.
Oh
wow
there's
a
little
trail
of
bits.
That's
super!
What
does
it
say?
Oh
it's
like
I
like
and
kind
of
diving
in
that
way.
A
Rather
than
a
I
mean,
there's
enough
metadata
here
that
you
could
say
that
okay,
zlib
128
has
non-severe
issues
identified
and
of
the
1000
components
that
I
use
dude.
Maybe
that
makes
z
lib
bubbles
slightly
higher
than
ones
that
have
nothing
but
not
as
high
as
ones
that
have
severe
ones.
So.
A
A
Which
is
the
other
reason
why
it's
standardized
on
package
urls
so
that
it's
crystal
clear
exactly
what
this
is
yeah,
okay,
cool,
following
on
from
the
from
the
pr
thing,
though
I
I
feel
super
uncomfortable
approving
my
own
prs,
especially
for
things
where
there's
a
security
view
involved.
A
So
we
need
a
pool
of
people
that
are,
you,
know,
willing
and
able
to
review
and
approve
prs
and
we'll
just
we
can
do
you
know.
One
reviewer
is
is
enough
for
now,
and
we
can,
you
know,
fix
things
in
the
future
if
we
need
to,
but.
C
How's
this
and
this
is,
I
don't,
I
think
I
know
the
answer
in
general,
but
the
only
problem
with
with
approval
is
what
exactly
are.
Are
we
approving?
I
mean,
presumably
we're
not
re-redo
re-reviewing
the
work
in
detail,
so
basically
it's
much
more
of
a
sanity
check.
Does
this
review
scene
seem
to
does
this
review
seem
to
be
credible
or
something
like
that?
I
think
yeah.
I
think
we
ought
to
write
down
just
quickly
a
couple
words
if
you're
reviewing
if
you're
approving.
C
What
does
that
mean-
and
I
think
it
just
means
it
appears
to
be
from
a
credible
source.
It
appears
to
be
to
be
active.
It
appears
to
be
accurate
for
what
it
says,
we're
not
claiming
that
we
re-reviewed
it,
and
if
this
is
not
a
peer
review
or
a
redo
of
work,.
E
Yeah,
I
would
be
happy
to-
I
might
need
a
little
bit
of
guidance
with
github's
functionality,
but
I'm
I'm
happy
to
volunteer
cool.
C
F
A
So
I
did
actually
add
that
as
a
wiki
page,
so
we
can
make
it
or
whatever,
but-
and
I
don't
like
the
word-
vulnerability
disclosure,
because
we
start
out
by
saying
this
is
not
a
vulnerability
disclosure
mechanism,
but
this
was
kind
of
I
tried
to
to
distill
the
conversation
we
had
on
one
of
the
other
issues
on
like
when
is
it
okay
to
talk
about
a
thing
here?
I
think
this
makes
sense
if
you
guys
want
to
just
give
this
give
us.
C
A
It's
supposed
to
be
because
either
sorry
I
I
should
walk
through
either
the
vulnerability
is
public
and
it's
been
fixed,
which
I
think
is
of
course.
Yes,
we
should
do
it
or
it's
fit
it.
The
vulnerability
report
is
public,
it
hasn't
been
fixed,
but
the
report
was
made
over
90
days
ago.
C
B
F
C
Yeah
and
the
way
you
just
verbally
explained
it
is
actually,
I
think,
what
you
what
you
should
do
it
you
know.
Basically,
the
way
you
verbally
explain
it
is
way
better
than
this
text
by
the
way,
the
other
we
just
talked
about
the.
What
am
I
approving
don't
tell
tell
me
that
it's
accurate
appears
to
be
accurate,
because
that's
probably
too
much
to
ask,
because
you
know,
I
think
the
appears
reasonable
and
and
is
self-consistent,
I
think,
is
what
we're
looking
for.
E
B
A
A
C
H
Yes,
I
have
a
question:
yep
publicity
disclosure,
but
if
it
is
on
twitter
or
if
it
is
on
vulnerability
database
is
different.
I
mean
if
it
if
a
tweet
is
popular.
I
think
we
need
to.
H
I
don't
know
if
we
can
wait
for
90
days
also,
it
is
not
fixed,
because
if
a
tweet
is
popular
I
we
can.
We
can
think
that
an
attacker
know
it,
and
similarly,
if
a
vulnerability
is
on
a
database
if
it
is
on
a
database,
but
it
is
not
90
days
holder,
but
it
is
on
a
database.
The
attacker
can
know
it.
Also
it
is
not.
It
is
not
fixed.
So,
but
that's.
A
Yeah
but
yeah
so
so,
where
I'm
trying
to
go
go
with
the
90
days
is
yes,
so
so,
if
I
drop
it,
if
I,
if
I
drop
a
no
day
on
twitter
because
I'm
you
know
whatever
mean
or
whatever,
I
don't
think
that
we
that
this
project
should
amplify
that
by
including
a
review
immediately,
because
I
think
what
that
what
that
person
did
was
almost
certainly
wrong
and
that
the
proper
vehicle
is
for
the
vote
for
the
vulnerability
disclosure
process.
A
You
know
generic
process
all
up
to
inform
the
author
of
the
problem
and
give
them
time
to
fix
it,
and
if
they
do
once
it's
fixed,
then
it's
totally
fair
game.
Or
if
the
author
has
been
told
about
it,
but
doesn't
you
know
whatever
reason?
Doesn't
it
doesn't
acknowledge
it
or
doesn't
fix
it
or
whatever?
Then
I
think
after
a
90-day
period,
it's
in
the
best
interest
of
users
to
know
about
the
problem.
I
also
don't
want
to
get
into
the
game
of
now.
A
C
I
just
added
some
new
text
underneath
there
you
know.
Basically,
this
is
not
intended
to
be
a
vulnerability
disclosure
process,
but
wait.
You
know,
there's
other
things
that
are
vulnerability,
disclosure
processes
and
you
should
use
those.
Yes,
I
I
think
I
think
that
helps
a
lot
actually,
if
you
just
make
it
clear
right
from
the
beginning.
A
Yeah
so
try
to
I
mean
people
aren't
going
to
read
words,
but
so
so,
if
I
have
this
this
in
there
and
read
me
up
top
yeah,
you
know
you
should
public.
A
You
should
disclose
in
another
way
a
new
thing
and
then
also
here
I
don't
know
if
you
know
this
tiny
checkbox
is
good,
but
it
won't
let
you
submit
unless
you
check
the
checkbox,
and
I
don't
know
that
this
is
exactly
the
right
words
either,
but
actually
that
comma
is
wrong
because
it's
it
can
be
a
significant
vulnerability
as
long
as
it's
disclosed
or
patched
so
we'll
fix
that.
A
Cool
we
are
making
progress.
Okay
next
one
number
35
publication
state.
A
This
is
a
a
good
comment
right
now,
so
so,
right
now,
the
publication
state
has
three
options.
I
was
really
just
more
trying
to
build
in
some.
A
Maneuverability
room
in
the
future,
because
it's
just
active,
then
we
don't
need
the
line
at
all,
but
I
think
we
should
so
for
right
now.
I
can't
imagine
why
we
would
want.
I
mean
perhaps
well,
okay,
I
don't
know
why
we
would
ever
have
draft
there,
so
maybe
we
get
rid
of
draft
but
then
active
or
remove.
The
question
is:
if
something
is
active
and
then
somebody
points
out
like
hey,
you
got
this
totally
wrong
blah
get
rid
of
it.
A
Should
we
delete
the
review
from
the
repo
or
should
we
mark
it
as
removed,
and
I
could
go
either
way?
I
I
think
that
if
you,
if
you
delete
it
from
the
repo
it'll
disappear
from
search
it'll,
effectively
be
really
deleted
from
history,
even
though
it's
not
technically
deleted
from
history.
C
A
Yeah,
no,
no
sorry
I
meant
like
if
it's
not
discoverable
by
anyone,
then
it's
like
the
the
bar
to
go
like
sleuthing
through
get
history
is
super
super
high
for
like
a
normal
person.
If
your
market
is
removed,
then
it'll
still
be
there,
but
it
may
be
confusing
for
someone
to
see
it
and
then
like
start
reading
it
and
be
like.
Oh,
my
god,
this
is
terrible
only
to
be
told.
A
Like
yeah,
you
didn't
look
on
the
first
line
like
gotcha,
you
wasted
your
time,
so
I'm
I'm
up
for
what
the
group
thinks.
C
If
it's
really
awful,
I
think
just
removing
it
it's
in
the
version
control
it
can
always
get
restored.
If
it's
just
a
matter
of
it,
doesn't
tell
you
ever
it's
it's
a
you
know,
it
doesn't
go
in
very
deep,
but
it
makes
it
clear
it
doesn't.
Well
I
mean
it's,
it
is
what
it
says
on
the
tin
and
it
may
have
been
more
than
I
had
before.
C
A
Cool,
oh,
we
already
talked
about
this
one,
so
we're
good
on
this.
A
That's
all
good
okay,
so
we
got
a
couple
things
that
we
we
can
do
some
of
these
these
offline,
but
I
would
appreciate
everybody
taking
a
look
at
particularly
the
readme.
A
And
the
the
readme
is
probably
the
most
important,
certainly
the
the
the
template,
the
quick
start,
some
of
the
reviews,
any
feedback
now
is
now
is
a
great
time.
A
I
would
like
to
formally
announce
this
stuff
next
week
at
the
town
hall
or
maybe
the
morning
of
the
town
hall,
or
something
like
that,
and
what
what
I
really
mean
by
that
it
was.
It
would
be
great
to
have
a
blog
article,
go
out
under
open
ssf,
so
I'll
talk
about
that
in
a
sec,
but
I
guess
one
of
the
the
questions
that
I
have
is.
A
I
would
really
like
some
other
organizations
to
commit
to
doing
a
couple
security
reviews,
even
if
it's
just
one
at
this
point,
I
would
like
to
be
able
to
say
down
here
in
the
in
the
blog
article
that
you
know
we
would
like
to
thank,
like
you
know,
x,
y
and
z,
for
contributing
you
know,
content
to
that,
and
it
could
be
super
unspecific
in
terms
of
quantity,
but
I
think,
having
a
list
of
four
or
five
organizations
would
kind
of
be
good
for
everything.
A
So
I
agree
so
yeah.
So
I'm
looking
for
you
know
folks
for
that,
so
I'm
going
to
come
back
to
this,
so
we'll
give
you
guys
a
chance
to
think
we
do
have
a
couple
other
things
that
we
do
need
like
specific
owners
for
so
luigi.
Thank
you
for
for
jumping
on
the
the
quick
start,
publishing
to
github
pages.
I
think
that's
important
for
next
week
also,
so
that
the
water
we
can
change
the
links,
so
people
can
actually
like
do
it.
You
know
without
having.
H
To
know
if
it
is
working
because
we
need
to
set
some,
we
need
to
set
some
setting
in
the
the
github
account.
So
we
need
to
do
that.
Do
you
know
what
that
setting
is
because.
A
H
And
for
this
reason
I
can
test
it,
but
when
I
have
launched
the
apr
on
my
github
account,
I
can
see
the
redirect
to
quick
start
pages,
so
I
suppose
it
run
but
got
it
okay.
So.
H
I
don't
know
if
we
can
set
the
we
can
change
the
setting
only
for
a
branch
because
get
a
pager
I've
have
a
setting
in
the
in
the
account
panel.
So
if
I
just
do
so
with
the
gh
pages,
is
that
the
I
can
wait
like?
Maybe
I
can
share
how
how
we
need
to
do
to
test
it.
Yeah
wait.
A
Okay,
so
so
do
you
want
to
do
that?
Let
me
keep
keep
moving,
but
we'll
try
back
in
when
you're
ready
and
we'll
we'll
just
we'll
just
do
it.
Okay,
so
so
second
thing,
so
I
so
again
this
was
super
late
last
night,
so
I
know
this
sounds
awful,
but
I
wanted
to
get
the
the
point
across
of
what
I
thought
the
blog
article.
A
It
should
be
what
I'm
looking
for
is
someone
who
would
like
to
review
and
severely
edit
and
improve
this
blog
article
so
that
we
can
send
it
over
to
the
folks
that
open
this
stuff
and
try
to
get
them
to
publish
it,
whether
it
is
tuesday
or
tuesday.
E
E
I
had
to
call
in
I'll,
but
I'll
definitely
check
the
notes.
As
soon
as
I
get
back
to
the
office.
A
No
no
worries
at
all.
Thank
you
and
yeah.
Now
also
you
separate
email
with
this
to
connect
you
and
and
kind
of
leave
it
in
your
capable
hands.
Okay,
sounds
good
awesome
and
finally,
oh,
I
should
have
talked
with
this
when
dave
was
here,
so
I'm
sure
he
has
an
opinion
here,
but
we
do
need
to
formalize
our
dispute
policy
or
dispute
process.
A
A
This
one
said
that
the
severe
issue
in
one
in
1.0
point-
let's
say
somebody
else,
comes
by
and
says
you
know
what
absolutely
not
this
is.
This
review
is
totally
wrong
and
you
know
I
want
you
to
take
it
down
and
when
dylan
you
come
back
and
you're
like
no.
This
is
totally
legit.
How
do
we
as
the
project
arbitrate
that
and
make
a
decision,
and
how
do
we
do
that,
particularly
if
there's
a
perception
that
you
may
be
biased
in
it
true
or
false?
It
doesn't
doesn't
really
matter,
but
you
know.
A
You
know
something
that
they
should
not
have
submitted,
so,
let's
just
say
a
zero
day
right
now.
It's
I
mean
at
that
point:
it's
public
or
worse.
We
accidentally
merge
it
now,
it's
in
our
rep
can't
really
get
it
out.
What
is
the?
What
is
the
process
for
the
submitter?
What
is
the
process
for
those
affected?
What
is
that
process
for
like?
Well,
you
know
what
what
is
our
playbook
there,
and
there
was
some.
A
Yeah
what
this
was
about,
you
know
we
remove
an
advisory
and
it's
like
we
can't.
Okay,
you
know
we
should
make
it
clear
that
you
shouldn't
be
doing
this.
I
think
we
did
a
good
job
with
that.
But
like
do
we
want
it?
That
was
my
just
resolution
process.
A
Yeah
does
it
make
sense
to
have
basically
a
private
email
address,
probably
at
open
ssf
that
will
come
to
maintainers
outside
of
the
github
public
universe
where
those?
A
So
so
what
you
don't
want,
if
you
want
somebody
contributing
a
review,
it
looks
fine
and
then
they're
like,
oh
my
god.
No
that's
really
like
this
is
their
day,
but
they
don't
want
to
put
it
in
the
issue
like
hey
everybody.
I
I
just
posted
this
awful
zero
day.
You
want
them
to
be
able
to
privately
contact
us.
I
think
that
has
value.
I
hope
it's
not
used
often,
but
I
think
having
something
like
that
is
is
at
least
something
we
should
consider.
A
You
know
so
somebody
says,
or
just
somebody
says,
hey
you,
you
wrote
a
review
of
my
thing.
I
object
to
that.
Take
it
down,
do
we
have
any
any
obligation
to
do
so,
but
we
should
at
least
have
a
vehicle
to
have
those
conversations
or
delegated
completely
to
the
attack,
which
is
the
other
way
of
doing
it,
and
that
way
we're
not
even
involved
in
that.
E
My
my
gut
reaction
says:
if
it
really
came
down
to
it
in
the
spirit
of
open
source
and
open
collaboration,
if
I
mean,
if
it
really
came
down
to
it
would
would
it
be
added
to
a
meeting
agenda
just
like
a
10
minute.
You
know,
let's
resolve
this,
we
take
it
to
a
vote
of
you
know
the
attendance
and
you
know
just
resolve
it
kind
of
in
a
public
way,
because
you
know
all
the
meetings
are
public
in
a
way
so
yeah
they
are
referenceable.
So.
A
Oh,
no,
no
sorry,
I
I
thought
I
I
sorry
where
I
was
going
was
if
the
person
that
was
going
to
post
the
issue
publicly
would
prefer
to
post
the
issue
privately
right.
Now
they
don't
have
a
vehicle.
B
D
F
We
can
make
it
clear
too
that
just
if
they
think
that,
like
I'm
biased,
for
example,
in
that
particular
example
or
I
shouldn't
have
done-
create
a
particular
review,
for
example,
that
maybe
publicly
in
the
readme
or
something
that
you
know
all
like
disputes
or
whatever
will
be
handled
by.
You
know
our
kind
of
organization
as
a
whatever
vote
consensus,
something
something
to
kind
of
show
that
there's
no
kind
of
direct
kind
of
level
of
bias
or
anything
that
we
handle
things
appropriately
like
maybe
staying,
maybe
somewhere,
would
be
a
good
idea.
A
Yeah,
I
I
I
agree-
we
do
need
to
be
public
about
that.
Okay,
so
resolving.
Does
anybody
think
that
having
a
having
an
email
address,
that
is
just
it's
the
if
the
stop
gap
for
if
we
missed
something
or
if,
for
whatever
reason,
somebody
wants
to
contact
us
without
opening
up
a
public
issue?
Does
anybody
think
that
would
be
a
good
idea.
E
Yeah,
I
think
we
could
have
lindsay
or
the
new
person
that's
helping
out
they
can.
They
can
probably
create
that
for
us.
A
Wonderful,
okay,
so
the
yeah
that's
kind
of
what
I
have
so
so
I'll
send
this
just
full
access.
I
will
copy
this
and
send
this
to
a
mirror
separately,
so
you
can
work
off
of
that
version.
But
if
you
guys
have
any
comments,
if
you
guys
want
to
mark
this
up
up,
feel
free
feel
free
to
do
so
right
in
the
right
in
the
dock.
Here.
D
One
comment
on
the:
if
somebody
has
a
dispute,
is
there
gonna
be
a
way
for
to
fight
get
that
it
is
in?
Somebody
knows
it
comes
to
it
that
it
is
in
dispute
that
we're
going
to
review
it
at
this
meeting.
A
I
think
the
process
would
be
so
somebody
contacts
us
through
whatever
way
we
say,
yup
acknowledge
our
next
meeting
is
on
whatever
next
tuesday
we'll
talk
about
it
then
you're
welcome
to
join
us.
I
think,
should
be
the
perfect,
the
normal
workflow
there.
If
it's
an
emergency,
we
can
talk
on
slack
or
you
know
you
know
we'll.
We
can
figure
that
out.
If
it's
I
mean,
let's
plan
for
the
worst
case
scenario,
but
if
it's
not
able
to
deal
with
it,
okay,
cool
luigi.
H
Sorry,
but
if
you
want
to
test
the
gita
page
at
the
end
of
the
meeting
we
can
do
because
I
think
it
will
be
really
two
minutes
to
test
it.
You
you
need
to
go
to
setting
yeah
control
f,
so
we
can
search
a
gita
page
at
the
end
of
the
page.
I
think
you
can
use
control
f
to
search
in
the
setting
yeah
yeah.
So
let's
jump
here,
okay
get
a
page
change
to,
because
there
is
a
branch,
because
it's
the
new
branch
right.
H
Sorry,
I
need
to
create
a
new
branch
right.
No,
I
think.
No
sorry,
it
is
only
a
request
from
my
from
my
repo,
I
mean
I
recreate
a
branch.
It
is
called
the
quick
start
website.
I
think,
by
this
one
yeah
right.
A
You're
merging
this
into
maine,
and
actually
how
about
we
do
this,
I'm
going
to
create
a
new?
Is
it
it's
it's
gh
dash
pages
is
the
is
the
convention.
A
Whatever
great
branch
pages
now,
gh
pages
looks
like
that
which
doesn't
really
matter.
I
don't
think.
H
And
select
where
I
have
put
them,
I
will
create
a
new
folder
called
docs,
because
it
is
a
sort
of
a
standard,
but
we
need
it
docks
and
save
and
merge
pull
request.
I
suppose
so
we
can
try.
It.
A
H
I
can
create
a
new
branch
I
can.
We
can
cross
the
pr.
I
can
recreate
a
new
one
in
a
new
branch.
I
where,
yes,
I
think
I
don't
know
I
have
a
privilege.
Usually
I
fork
the
repo,
but
I
think
so.
H
H
H
Okay,
because
it
is
in
my
in
my
request,
the
folder.
Well
you
I
have
choose
docs,
but
we
can
choose
if
we
want
to
put
the
website
in
the
main
folder.
I
don't
like
in
the
main
folder,
because
it
is
quite
noisy
because
we
are
the
main
files
in
the
main
folder.
So
I
prefer
put
in
one
subfolder
like
docs
like
like
website.
There
is
not
a
standard
about
this
yeah
and
but
you
can
use
this.
You
can
use
gita
pay
to
see
the
all
the
html
files
and
it's
called
my
opinion,
cool.
H
Usually,
I'm
not
sure
but
usually
github
search
the
index
file
and
you
can
create
an
index
file
to
redirect
to
other
files
like
the
like
every
web
server
by
default
or
wordpress
or
every
yes,
nice
I
to
redirect.
I
have
put
only
quickstart
in
the
docs
folder
and
to
redirect
to
this
page.
I
have
created
an
index
page
that
use
javascript
to
redirect
easy.
A
H
I
can
but
white,
I
am
stupid.
I
mean
okay,
I
need
to
edit
the
pr,
but
don't
worry
because
of
course
I
have
wronged
the
redirect.
It
is
a
it
points
to
quickstart
dot
html,
but
it
should
be
point
to
security
reviews,
slash
quick
start,
but
okay,
correct
yes,
because
if
you
now
you
go
on
the
rapper,
you
you
it
redirect
to
a
quick
start,
and
but
this
is
not
the
the
right
path.
H
A
I'm
I
yeah,
I
mean
that
that
would
be
great
or
I
can.
H
H
Usually
I
use
the
entire
domain
I
use
opens,
for
example,
open
ssf,
dot,
github
dot,
io,
slash
name
of
the
repo
slash
html,
but
because
usually
it
is
easy.
A
Okay,
cool,
we'll
we'll
figure
this
out.
We
don't
need
to
have
it
ready
cool,
so
I
think
we,
I
think,
we're
in
good
shape.
So
I'm
thank
you
amir
for
the
volunteer
on
the
on
the
blog
thing.
Thank
you
for
luigi
on
the
github
pages.
I
think
we're,
I
think,
we're
good
for
everything.
Is
there
anything
else
that
anybody
would
like
to
talk
about.
F
A
Absolutely
yes
do
that
right
now,
so
the
question
is
so
npm
issues,
security,
advisories,
they
publish
them
on
github,
so
they're
actually
quite
easy
to
find.
So
this
is
an
example
of
one
of
them.
It's
just
json,
so
it's
so
structured
data.
It
goes
against.
I
mean
it's
a
little,
the
the
the
way
that
they
do
vulnerable
and
patched
we'll
have
to
we'll
have
to
work
out
how
to
how
to
figure
how
to
translate
that
to
package
urls.
A
But
the
question
is:
would
basically
syncing
the
github
security
advisories
to
our
repo
provide
value?
A
The
argument
against
including
this
is
that
it
it's
you
know
it's
it's
a
vulnerability.
It's
a
vulnerability
notification.
It's
like
a
cve
database,
we're
not
going
to
suck
in
all
the
cves
and-
and
you
know,
display
them.
I
don't
know
thoughts.
E
If
I
could
jump
in
a
mirror
here,
I
would
say
it's:
it's
a
good
thing!
Absolutely
because
you
said
it
yourself.
I
mean
there
really
isn't
a
single
place
to
go
to
see
what
kind
of
security,
research
or
audits
or
security
reviews
being
done,
and
I
think
in
general
security
reviews
have
a
big
pr
problem
and
an
awareness
problem
so
having
this
as
a
repo.
E
As
for
you
know,
the
public
to
access
kind
of
in
one
place,
I
think,
is
going
to
provide
a
lot
of
value
even
alone
in
awareness
just
because,
typically
you
know
as
more
organizations
and
more
folks
are
aware
of
this
kind
of
work
being
done.
Hopefully
that
will
promote
more
organizations
and
folks
to
to
get
on
board
so
to
speak.
E
So
I
think
it's
a
fantastic
idea
and
I
think
yeah
I
I
there's
a
lot
of
resources
in
the
bug,
bounty
space
and
then
a
lot
of
different
kind
of
offshoots
of
this,
but
I
think
security
reviews
in
particular
they
they
could
use
more
resources,
and
I
think
a
repo
like
this
is
definitely
a
great
place
to
start
cool.
Thank
you.
B
So
is
this:
is
this
mirroring
on
the
mirroring
on
the
as
a
service
site?
Is
where
you're
talking
about
the
the
the
one
that
you
displayed,
that
collects
metrics
or
where,
where
are
you.
B
B
Repo,
you
know
it.
It
seems
like,
if
there's
already
a
gate,
repo
that
that
holds
that
information.
You
could
just
reference
the
git
repo
and
then
once
you
have
the
you
know
the
dashboard.
You
know
what
you
want,
then
you
can
you
can
proxy
through
that
right
and
that
way
you
have
one
source
of
authoritative
truth.
A
B
A
A
So
a
sub
module
might
but
a
sub
module
might
do
sub
modules
show
up
in
the
github
like
yeah.
B
D
B
Data
and
and
if
you
don't
plan
on
editing
it,
it
doesn't
make
sense
to
replicate
the
whole
structure
right
and
and
if
you
did,
then
I
mean
you,
you
would
you
obviously
you're
tracking
tracking
version
changes
right,
but
but
yeah
there's
no
reason
to
track
any
changes
if
you're
not
going
to
edit
it
and
yeah.
A
Well
so
I
mean
the
so
so
from
a
consumer's
perspective
if
they
wanted
to.
So
if
we
didn't
have
anything
to
say
about
closure,
you
till,
but
like
nobody
like
no
one
wrote
a
review
like
for
us
for
closer
util,
but
this
one
exists
here.
B
B
Because
then
your
proxy
right,
you
choose.
Okay,
basically
you
know.
If
I
have
no
review,
then
I
display
this
right.
This
is
the
information
that
I
have
right,
but
otherwise,
then
you
you
end
up
with
an
alternate
structure
for
your
review
documents
right
if,
depending
on
whether
they're
they're,
you
know,
npm,
based
or
node,
based
or
or
you
know,
have
been
contributed.
A
That
make
sense,
so
so
perhaps
okay,
so
so
the
the
guess,
the
web
front
end,
could
have
a
you
know,
type
in
your
type
in
your
typing
closure
util,
and
it
says
security
reviews,
none,
npm,
advisories,
one
cv
whatever
in
this
case
I
don't
know
none
or
one
or
whatever
yeah.
A
But
that
would
require
folks
to
use
the
front
end.
Oh,
I
guess
we
could
do
a
web
service
too.
That
does
the
same
thing.
Sub
module
would
also
would
also
work,
but
the
sub
module
at
that
point
is
really
just.
We
might
as
well
put
a
link
to
like
friends,
yeah
yeah,
which
wouldn't
be
bad
either
because,
like
like
off
the
top
of
your
head,
do
you
know
the
one
for
ruby
like?
I
don't
I'm.
B
Yeah,
you
would
just
probably
want
to
replicate
the
so
if
you,
if
you
want
to
you,
know
sort
of
mirror
this
in
you
want
to
make
sure
that
the
directory
structure
is
going
to
be
the
same
right.
So
if
you,
if
you
have
a
review,
then
you
need
to
sort
of
map.
You
need
to
have
some
kind
of
automation
that
grabs
this
stuff
and
puts
it.
You
know
where
it
should
be
in
your
repo
commits
it
if
there's
no
review
right.
F
Just
just
to
jump
in
here
the
the
the
whole
like
singing
issue
and
finding
things
like
once
you
just
like
script,
that
up
and
and
kind
of
you
know
fit
that
to
each
kind
of
public
advice
like
npm
or
ruby
or
whatever
it
is.
Then
it's
I
mean
that
they
don't
change
it's.
I
don't
see
like
why
it
would
require
a
whole
ton
of
like
manual
fixes
once
it's.
A
A
Just
it's
part
of
the
part
of
the
github
action
build
on
github,
it's
separately,
does
a
does
a
clone
of
you
know
for
for
this
one
particular
this
for
for
npm
security,
which
which
could
just
be
one
of
many
it
just
has
a
clone
import.
I,
I
guess
the
gig
of
action
underneath
the
ability
to
commit
back.
E
B
To
proxy
it
through
the
web
front
end
or
whether
you
want
to
duplicate
the
information
on
the
keyboard,
though
yeah,
let's
see
you
do,
may
need
no,
because
the
the
one
thing
that
can
happen
with
this
is
when
you
think
about
upstream
repos,
right
and
and
like
what
happened
with
left,
pad
and
stuff
right,
where
you
get
end
up
with
a
new
repo
of
the
same
name
or
something
right.
I
don't
know
if
they,
I
think
they
fixed
that
in
npm
or
something
right.
I
can't
remember
what
yeah
they
do.
Security.
B
Yeah
but
sort
of
other
package
managers
and
stuff
right.
You
know,
I
don't
know
you
know
you
never
know
what
everybody
else
has
done
so
yeah,
so
that
can
be
yeah.
I
don't
know
it's
yeah.
I
think
the
github
actions
thing
makes
a
lot
of
sense
right.
You,
you
really
want
to
go
with
with
I
mean,
because
that
that
that
gives
you
that
gives
you
an
automated
way
to
do
this
right.
You
want
some
kind
of
automated
way
to
do
this
and
you
want
to
maintain
the
structure
in
the
same
format,
yep
and.
A
I
think
we
add
with
two:
we
wouldn't
overwrite,
because
there
could
be
instances
where
I'm
reviewing
elasticsearch
from
a
operational
security
perspective
and
dylan
zoom,
one
from
a
crypto
perspective
and
whatever
so
I
think
they
and
they
in
theory
they
could
conflict
with
each
other.
And
I
don't
know
that
we
want
to.
B
A
Yeah
yeah
sorry
secure
it
is
we
we
so
the
way
that
it's
organized
now
it's
just
it's
similar
it.
So
it's
npm
project
name
and
then.
A
Literature
is
unique,
yep
and
yeah,
so
they
should
just
kind
of
fit,
and
this.
B
Okay,
yeah,
that's
part
of
where
my
my
proxy
web
web
service
suggestion
was
coming
from,
was
the
unique
per
so
yeah
cool,
all
right
yeah.
That
sounds.
A
Okay,
so
how
about
this?
Let's,
what
I
propose
is:
let's
have
one
more
discussion
on
this
in
two
weeks
when
you
get,
we
have
a
couple
more
people
on
to
make
a
decision.
So,
let's,
let's
let's
hold
on
doing
that,
because
I
think
once
you
do,
the
giant
merge
it'll
be
hard
to
undo
that
if
we
decide
that
was
the
wrong
thing.
So,
let's
go
slowly
here.
A
Yeah,
okay,
cool
yeah,
anything
else.
Anybody.
A
Awesome
well,
thank
you
very
much
for
your
time.
I
hope
I'll
see
everybody
at
the
town
hall
next
week
and
hopefully
we'll
have
a
blog
article
out
shortly
before
there
shortly
before
then
on
announcing
this
project,
and
then
you
know,
certainly
you
know
amplify
and
contribute
wait.
Sorry
before
everybody
leaves
forgot,
one
more
thing
promise
to
come
back
to
this
yeah.
This
guy
right
here
so
now
would
be
a
great
time
to
con
to
open
up
a
pr
with
at
least
one
of
these.
A
So
what
I'll
do
is
I'll
take
all
the
pr's
that
I
have
I'll
make
sure
that
they're
all
merged
before
or
actioned
before
before
the
town
hall
and
we'll
just
use
that
as
the
the
basis
for
you
know
the
kind
of
the
you
know,
content
contributors,
obviously
there's
value
in
doing
other
things
other
than
just
content.
Thank
everybody
for
that.
But
I'm
gonna
see
if
I
can
call
in
some
favors
and
have
have
other
folks
contribute
here
too.
So.
B
Do
you
guys
want
to
have
a
little
technical
discussion
on
on
the
on
the
similar
work
that
that
I
had
done
now,
or
do
you
want
to
do
that
some
other
time
or.
A
B
All
right
cool
yeah-
I
just
want
to
give
you
guys
some
brief
overview.
So
so
we
can,
you
know
sure
sure
learnings
be
on
the
same
page
here.
Okay,
so
let
me
let
me
I
think
I
can
present
here.
B
B
We're
actually
doing
some
sort
of
similar
stuff
internally
now
as
well.
Okay,
here
we
go
so
anyway,
so
we
gave
this.
I
gave
this
presentation
at
b
sites,
pdx
and
and
back
in
2019,
and
we
did
some
dependency
review
process.
It's
pretty
much
a
similar
effort
to
this
internally
at
intel
and
what
we
ended
up
with
was
was
we
standardized
sort
of
on?
B
We
had
a
set
of
reviewers
on
the
open
source
security
team
and
we
were
responsible
for
being
reviewing
all
of
the
open
source
packages
at
intel,
they're
that
that
development
teams
that
intel
were
using
from
open
source
and
the
way
that
we
did
this
was
we
essentially
we
built
this
review,
or
we
had
this
review
form
it's
similar
to
this.
B
This
is
sort
of
a
legal
approved
one
I'm
going
to
see
if
I
can
get
legal
to
to,
let
us
share
a
little
more
and
so
basically
we'd
go
through,
and
the
reviewers
would
look
at
the
the
the
upstream
url
that
was
submitted
by
a
project
team
that
wants
to
use
this
dependency
and
they
would
go
through
this
form
and
they'd
say:
okay,
you
know,
like
you,
know,
I'd
answer
these
questions
as
a
reviewer,
and
that
would
help
me
this.
This
form
helps
guide
my
decision.
B
Whether
this
is
basically
I
should
use
this
or
I
should
not
use
this
project
so
a
bit
of
a
different
different
use
case
here,
based
on
you,
know,
issue
severity
and
stuff,
but
this
is
essentially
a
thumbs
up
thumbs
down
on
on
whether
you
should
use
this
thing
or
not
at
a
high
level,
and
it's
based,
you
know,
on
maintenance
and
security
practices,
so,
okay,
well,
most
of
this
presentation
is
about
automation.
Let's
see,
let's
skip
down
to
sort
of
the
thing
that
it's
more
interesting
here
from
this
perspective.
B
Okay,
so
this
is
basically
it's
a
python
project,
and
so
we
sort
of
took
this
a
little
bit
farther
and
and
made
this
tool
called.
Should
I,
and,
and
should
I
is-
is
essentially
this
idea
of
okay?
Well,
you
we
run
pip
install
to
install
something
with
javascript.
Well,
should
I
install
it
so
instead
should
I
install
it
in
the
package
name,
so
we've
got
this
sort
of
data
flow
execution
engine.
It's
this
directed
graph
execution
thing.
It's
not
super
important
to
this
conversation,
but
let's
see
okay
yeah.
B
So
I
guess
this
is
good,
but
essentially
what
we.
This
is
sort
of
a
metastatic
analysis
tool
for
python
and
we're
ending
up.
You
know
extending
it
to
be
a
medic.
The
idea
is
to
extend
it
to
be
a
metastatic
analysis
tool
for
anything
and
what
I
mean
by
a
metastatic
analysis
tool
is
that
it's
going
to
run
other
static
analysis
tools,
so
you
know
arbitrary
tools
and
collect
all
of
their
results.
B
So
this
is
very
similar
to
the
the
project
that
other
project.
What
is
that
one?
The
one?
That's
the
the
checklist
project
right.
B
It's
similar
to
that
is
the
idea
right
sort
of
create
this
extensible
interface,
so
that
you
can
keep
adding
metrics
that
you
collect
about
a
project
right
and,
and
then
on
top
of
that,
so
this
is
for
python
and
then
you
know
what
we
wanted
to
do.
On
top
of,
that
is
create
something
that
works
for
you
know
other
languages
and
so
we've
sort
of
expanded.
Let's
see
what
do
we
have
now,
I
think
we-
I
don't
know
if
we
have
all
of
these.
B
I
think
we
have
go
and
rust
and
python
right
now
and
just
kind
of
you
know
some
some
tools
here
right
and
so
it's
yeah.
So
that's
the
idea
right
and
this
is
went
with
the
approach
of
you
know
the
I
can
run
this
client
on
my
somewhere.
You
know
right
like
I
could
run
this
as
a
local
developer
and
we
also
have
it
exposed
itself
as
a
web
service
as
well.
So
it's
a
sort
of
same
same
same
thing,
different
interface.
B
You
can
run
it
as
a
command
line
tool
or
you
can
run
it
over
the
webinar
or
http,
but
yeah.
So
this
is
sort
of
we
built
this
framework
so
that
we
can
collect
arbitrary
metrics,
and
this
is-
and
we've
got
some
metric
collection
things
and
then
also.
The
idea
is
that
the
way
that
it's
written
allows
people
to
write
small
metric
gatherers
without
having
to
worry
so
much
about
all
the
code
that
other
people
have
written
and.
D
B
Nice,
no,
I
sent
the
link
in
the
chat
I
can
maybe
yeah.
Maybe
we
could
put
it
in
the
meeting
minutes
or
something.
If
somebody
wants
to
go,
you
know
poke
around
more
at
it,
but
that's
basically
what
we've
been
working
on.
It's
sort
of
an
ongoing
project.
We
haven't
done
much
work
in
this
space,
but
I
have
some
I
base.
B
I
have
some
students
that
are
involved
and
through
google
summer
of
code
they've
been
working
mostly
on
sort
of
the
machine
learning
aspect
of
this
project,
but
the
security
aspect
of
the
project.
I
have
one
student
who's
interested
in
doing
this
and
he's
pretty
good,
and
so
we
could
have
some.
We
have
some.
We
have
some
work.
People
who
can
do
work
on
this.
If
we
want-
or
you
know
and
and
kind
of
part
of
this
is
right-
you
can
expose
it
as
a
web
service
right.
B
You
guys
have
a
web
service
right
now.
You
know
we
can
stand
up
a
few
of
these
things
in
containers
written
in
whatever
languages
we
have,
and
we
can
just
start
thinking
about
what
metrics
should
we
collect
right
and
get
some
some?
You
know
some
muscle
behind
writing
some
stuff
to
go,
collect
those.
A
I
like
it,
I
like
it
a
lot
cool
thanks,
yeah
it.
It
reminds
me
so
I
mean
I
think,
a
lot
of
security
teams
go
down
the
path
of
like
you
know,
look
let's
collect
a
whole
bunch
of
tools
and
put
them
in
one
tool,
and
then
you
know
kind
of
abstract
that
stuff
away,
I'm
particularly
interested
in
the
in
the
machine
learning
part,
because
the
the
challenge
that
I
have
frankly
is
as
we
collect
too.
B
Yeah,
I
can
talk
a
little
bit
about
that
so
yeah,
I
kind
of
glossed
over
that,
but
you
that's
the
so
this
is
sort
of
the
first
step
right
is
gather
all
the
metrics
and
and
I'll
flip
back
to
my
slideshow.
I
shouldn't
have
lost
over
that.
So
too
many
windows,
okay,
okay,
come
on
give
me
that
tab
all
right
there
we
go
all
right.
So
basically
what
we
ended
up
doing
is.
B
Is
we
had
this
review
process
right,
like
I
showed
you
this
form
and
I'm
gonna
see
so
so
this
has
been
a
big
fight
to
get
even
this
form,
which
is
not
the
real
form
out
based
on
legal
stuff,
but
I'm
gonna,
I'm
I'm
talking.
I
sent
some
emails
while
we
were
talking
here
and
I'm
going
to
see
if
we
can
get
some
of
our
reviews
publicized.
B
At
all-
but
you
know,
hopefully,
we've
already
got
reviews
right.
We
ideally
we
could
publish
them
right
and
that
would
give
some
some.
You
know
insight
into
the
methodology
that
we
use
as
well.
So
basically
we
had
these
review
forms.
The
other
part
of
this
was
that
most
of
these,
a
lot
of
these
review
forms
were
not
filled
out.
B
So
so,
if
we
do
contribute
reviews,
we're
gonna
go
make
sure
that
they're
ones
where
the
form
was
filled
out
well
and-
and
so
this
was
sort
of
the
crux
of
our
problem
here.
So
we
went
to
go,
apply
machine
learning
to
this,
because
we
basically
had
this.
B
You
know
giant
data
set
of
good,
good
or
bad
in
the
review
form
data,
and
so
this
is,
you
know
you
should
be
able
to
train
a
classification
model
on
this
right,
so
we're
just
going
to
train
different
models
until
we
get
one
with
the
right
hyper
parameters
and
the
right,
you
know,
model
architecture
to
give
us
good
accuracy
right
and
then
that
that
that
then
we
then
we
can
just
go,
throw
it
into
production,
but
obviously
you
know
we.
B
We
had
some
bad
data
here,
so
if
we
contribute
once
we're
going
to
be
contributing
ones
that
that
we
make
sure
are
have
the
form
filled
out
right
and
follow
the
proper
methodology
because
part
of
the
problem
that
happened
was
basically,
we
have
other
full-time
jobs,
and
this
is
something
that
was
sort
of
thrown
on
us
and
the
automation
was
how
we
got
it
off
our
backs.
But
you
know:
we'd
go
and
review
things
and
by
the
time
we're
almost
done
reviewing
it
then
we're
like
okay.
B
B
You
know,
let's
automate
the
collection
of
it
since
we
already
have
the
classifications
and
then
we
can
go
and
essentially
just
keep
this
we'll
repeat
this
process
until
we
get
good
accuracy,
so
collect
generate
a
data
set,
do
sort
of
feature
engineering
on
that
data
set,
decide
what
features
matter
right
and
that's
what
I
talked
about
earlier
when
david
was
talking
about
you
know
what
things
matter.
We
have
some
ways
of
there's
some
ways
that
you
can
figure
out
sort
of
what
things
matter
in
determining
a
good.
B
You
know
getting
a
good
accuracy,
so
we
created
this
sort
of
plug-in
architecture
to
collect
arbitrary,
different
things,
and
then
we
went-
and
we
basically
did
this
loop
here.
You
know
this
train
evaluate
pick
new
features
thing
until
we
got
good
accuracy
right
and
so
that's,
why
part
of
why?
Having
sort
of
standard
methodology
and
standard
and
standard
you
know
standard
methodology
and
then
agreeing
on
what
figuring
out
what
data
really
matters
can
help
us.
B
You
know
come
up
with
an
automatable
approach
here
and
I
think
that
I
think
that
that
yeah,
that's
part
of
why
the
methodology
matters
and
also
why
you
know
it's
important
to
figure
out,
maybe
figure
out
whether
we're
doing
a
classification
based
thing
or
something
you
know
like
what
is
the?
What
is
what
is
one
of
the
goals
of
the
project
and,
I
think,
figuring
out
how
are
people
going
to
use
this
project
right?
B
The
security
you
know
the
the
identifying
security
project,
the
security
threats
right
so,
like
you
said,
are
people
going
to
come
and
they're
going
to
look
at
the
report
right
or
are
they
going
to
go
and
you
know,
are
they
going
to
automate
this
and
basically
take
their
bomb
and
scan
it
against
the
security
threats,
project
and
say?
Okay,
any
of
these?
You
know
who
don't
who
are
severe
right
and
or
basically
say,
okay,
the
ones
in
these
categories.
I
know
we
need
manual
review
on
the
ones
where
they
aren't.
I
need
to
go.
B
You
know,
I
just
accept
that
right
and
figuring
out
so
how
people
are
gonna
use.
This
can
help
us
determine
you
know
what
we
collect
and
how
we
automate
it.
I
think
so
yeah,
I'm
really
interested
in
working
with
you
guys
on
this.
I
think
that,
obviously
this
is
this
is
cool
stuff
and,
and
so
so
let
me
know,
let
me
know
how
I
can
help
you
know
and
I'll.
I
intend
to
keep
coming
soon.
A
No,
that's.
That's.
That's
awesome,
happy
to
have
you
here.
I
did
post
a
link
in
the
chat.
We
did
some
work
last
year
and
a
half
something
whatever
sometime
in
the
business.
A
Yeah,
so
that
should
be,
and
obviously
if
that
is
not
providing
you
the
signal
that
you're
looking
for,
let
us
know
because
it's.
B
Yeah
yeah:
this
is
one
of
those
things
yeah.
I
wanted
to
integrate
this,
because
this
is
cool
right
this.
This
helps,
you
figure
out,
you
know
also
sort
of
what
kind
of
this
is.
Definitely
it's
in
the
same
vein,
right
and
and
sort
of
this
idea
of
you
sort
of
just
spider
out
and
try
to
find
more
more
information
right,
yeah
like
this.
A
Cool
and
then
the
the
other
one
that
I
would
be
remiss
if
I
didn't,
if
I
didn't
plug,
was
this
guy
alyssa's
gadget,
which
was
our
attempt
at
like
we
do
a
lot
of
things
with
open
source
which
is
like
you
know.
How
do
you
find
the?
How
do
you
find
the
you
know?
A
How
do
you
actually
download
the
bits
for
the
package
given
you
know,
and
how
do
you
calculate
risk
or
health
or
we're
doing
one
for
typo
squatting,
which
is
in
progress
and
and
things
like
that,
so,
oh,
this.
B
Oh
fantastic,
you
know
one
of
the
other
things
that
I'm
really.
This
is
one
of
the
one
of
the
the
the
harder
problems
in
this
that
I'm
curious
about
solving
is
is
how
do
you
compile
an
arbitrary
package,
because
when
you
get
into
c
and
c,
plus
plus
it's
very
tricky
and.
B
A
So
so,
obviously
we're
kind
of
a
little
bit
biased
towards
code
ql,
but
coql
does
have
auto
builders.
And
I
my
understanding
is
it's:
it's
a
like.
You
know
70
or
80
different
like
it
could
you
know,
configure
and
make
a
cma.
A
There's
also
the
vc
package,
where
I'm
starting
to
look
at
that,
because
I
believe
that
has
some
information
on
how
to
com,
because
the
whole
holiday
buying
vc
package
is
you
know
it's
the
package
installer
for
cnc,
plus
plus
stuff
and
there's
a
couple
thousand
packages
already
kind
of
defined
there,
and
I
think,
there's
enough.
I
think
it's
like
a
cmake
wrapper
for
a
lot
of
these,
so
that
might
help
and
then
for
the
javascript
stuff.
A
We
use
there's
a
package
called
trust
but
verify
which
I
think
it
just
does
a
it
probably
also
has
like
five
or
six
different
like
builders
and
if
the
bits
that
pop
out
are
the
same
as
the
bits
that
you
get
from
npm,
then
it
says,
reproducible.
Otherwise
it
says
fail.