►
From YouTube: OpenSSF Identifying Security Threats WG (April 30, 2021)
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
No,
I
want
to
move
over
to
to
meet.
I
just
got
access
to
the
to
the
thing
yeah.
It's
on
my
my
laundry
list.
B
A
C
E
Hi,
my
name
is
john
and
later
I
am
working
at
gradle
as
a
security
software
engineer,
I
am
an
open
source
security
researcher
yeah,
so
open
source
security,
researcher,
popularly
known
for
a
big
vulnerability
and
zoom,
but
also
I
do
a
bunch
of
stuff
with
codeql
and
finding
security
vulnerabilities
in
open
source
and
part
of
the
github
security
lab
stuff.
No,
no.
I
don't
work
for
get
out,
but
I
do
I
do
stuff
under
their
their
bug
money
program.
So
that's.
F
G
Welcome
g'day.
Thank
you
michael.
My
name
is
adrian
wood.
I
work
for
wells
fargo,
I'm
the
tech
lead
of
their
application
incident
response
research
team.
It's
like
a
function
of
their
red
team.
I've
been
doing
that
for
too
long
now
nice
to
be
here.
Thank
you
for
having
me
particular
interest.
In
obviously,
you
know
open
source
libraries
supply
chain
attacks
really
looking
forward
to
collaborating
with
you
all.
Thank
you.
Terrific.
B
Dylan,
hey
everyone.
My
name
is
dylan
no
fancy
titles.
Yet
I
suppose
I'm
I'm
a
senior
studying
electrical
engineering
and
computer
science
at
uc,
berkeley
and
yeah.
I
got
interested
in
the
open
source
security
spice
space
interning
with
mike
last
summer,
and
I'm
very
excited
to
be
here.
A
Awesome
laurent.
I
G
J
Luigi
hi,
I
am
luigi,
I
am
security
engineer
at
small
pdf.
It
is
a
startup
and
well-
and
previously
I
have
worked
in
arduino,
so
an
open
source
based
company
in
italy.
H
F
Okay,
yeah,
okay,
sorry,
I'm
driving,
I'm
ryan
and
I
work
for
microsoft.
Security
response
center
focused
primarily
on
open
source
security,
and
that's
it
cool.
A
Awesome
welcome
buddy,
so
yeah,
so
so
I
have
a
well
kind
of
a
good
agenda,
but
I
think
we'll
be
able
to
get
through
it
all,
because
I'm
optimistic
right
now,
okay,
so
the
first
thing
I
wanted
to
chat
about
was
so
so
the
high
level
things
are
the
budget
for
the
rest
of
2021
and
2022,
just
final
sync
on
the
metric
dashboard
and
the
security
reviews
site
and
then
remaining
time.
A
I
wanted
to
float
an
idea-
and
this
is
like
definitely
not
fully
baked
but
wanted
to
to
kind
of
get
thoughts
on
whether
whether
this
kind
of
thing
made
sense
so
I'll
start
with
the
budget,
because
we
we
need
to
give
what
we
already
gave
numbers,
but
I
told
them
that
we
finalized
them
today.
A
So
what
they're
asking
for
what?
What
the
governing
board
is
asking
for,
or
what
kind
of
what
we
think
we'll
spend
on
projects
that
we're
associated
with
for
the
rest
of
2021
and
all
of
2022,
obviously
22
the
end
of
2022
is
so
far
out.
I
have
no
idea
so
I'm
trying
to
take
a
guess
project
hosting
for
security
metrics,
I
think,
will
run
around
200
a
month
at
at
most
that's
without
being
kind
of
super
careful
and
and
trying
to
keep
it
cheap.
A
I
think
right
now
it's
running
like
140,
but
I
know
I'm
getting
some
sort
of
an
internal
discount
on
on
azure.
So
I
think
that's
that
that's
a
reasonable,
a
reasonable
guess
for,
but
one
of
the
things
that
we've
been
we've
talked
about
is
sulfur
dev
time
to
do
things
like
work
on
the
dashboard
and
make
updates
and
more
collectors,
and-
and
actually
you
know,
someone
who's
dedicated
to
this
and
can
put
you
know
40
hours
a
week
into
it.
A
A
You
know
if
we
talk
to
chaos
and
we
decide
that
there's
some
integration
there
that
we
want
to
do
you
know
kind
of
implementing
that
improving
the
api
and
kind
of
turning
this
into
a
real
into
a
real
thing.
I
think
this
is
kind
of
it's
not
super
conservative
like
we
might
run
out
of
that,
but
I
also
don't
want
to
ask
for,
like
you,
know,
500k
and
then
not
be
able
to
spend.
You
know
a
quarter
of
it.
So
does
that
sound
reasonable
at
a
super
high
level
for
security
metrics?
H
A
H
A
couple
questions
actually
michael,
so
I
don't
know
how
you
figured
out
the
project
hosting.
You
probably
ought
to
point
to.
Here's
where
it
is
now.
Is
that
just
for
hosting
or
is
that
also
to
run
tools
to
like
run
a
bunch
of
static
analyzers
to
collect
data,
or
something
like
that.
A
This
is
the
cost
for
one
reasonably
sized
vm
and
the
storage
to
back
that.
So
it's
it!
It's
a
reasonably.
I
mean
it's
not
a
high-end
machine
by
any
means,
but
it's
it
should
be
enough
to
to
do
whatever
we
need
to
do
aside
from
if
we
wanted
to
mass
run
static
analysis
on
the
universe,
in
which
case
you
know,
we
need
multiple,
more
zeros
there.
Okay,.
A
Because
well
part
of
the
120k
here
would
be
to
yeah
if
we
can
migrate
to
a
container
and
it's
cheaper.
That
would
be
great
too
right
now
it
is
dockerized,
but
I
have
the
the
compose
thing
running
in
a
vm.
So
if
we
can
save
200
a
month,
that's
great
too.
I
don't
know
you
know
it's.
It's
a
rounding
error
relative
to
the
other
one
side.
H
A
And
then
yeah
and
then
120k.
What
I
was
thinking
is,
you
know,
that's
probably
a
half
time.
You
know
20
hours
a
week
for
the
duration
give
or
take.
You
know.
I
think
that's
probably
like
I
don't
know
that
we
actually
need
40
hours
a
week
anytime
soon,
but
I
mean.
H
D
A
A
Let's
explore
this
it'll
cost
some
money
and
I
don't
want
to
have
to
like
go
back
and
try
to
argue
and
and
and
get
money
from
the
pot,
so
I'd
like
to
kind
of
have
a
pre
not
allocated,
but
a
pre
kind
of
semi-allocated
pool
that
we
can
use
for
either
experiments
or
or
or
things
like
this
and
and
amir.
I
know
that,
obviously,
like
you
play
exactly
in
this
space
and
I'm
not
trying
to
like.
A
This
was
really
off
the
top
of
my
mind.
I
was
like
well
what
what
would
be
a
fun
experiment
being
like?
Could
we
do
like
you
know,
really
cheap
distributed
security
review
bounties
like
how
can
we
increase
the
number
of
security
reviews
that
we
have
done?
I
don't
know
if
this
makes
sense
or
something
else,
but
the
point
is
to
put
about
75k
extra
in
a
pot
that
we
can,
I
would
say,
use
at
our
discretion,
because
that
sounds
like
we're.
A
Gonna,
you
know,
buy
a
car,
but
you
know
allocate
towards
interesting
projects
that
we
don't
know
right
now
what
they
would
be.
H
The
way
you
describe
it
looks
a
whole
lot
like
the
kind
of
work
that
I'm
expecting
the
critical
projects
work
to
do.
You
know,
folks,
like
austin
and
emir's,
on
the
call,
which
is
good
so
amir,
I'm
assuming
that,
whatever
reviews
that
you
do
for
any
of
the
other
things
will
end
up
in
some
sort
of
report
that
we
could
then
slap
into
security
reviews,
or
you
know
at
least
a
link
to
it.
Is
that
your
assumption
as
well.
C
That's
correct,
yeah.
We
specifically
bake
that
into
the
pr
and
media
plan
to
include
every
review
in
the
security
reviews
project
repo,
just
to
show
that
that
acknowledgement
of
both
the
supporters
and
acknowledgement
of
the
work
being
done
and
to
populate
the
repo
and
show
that
you
know
this
is
work
being
done.
That
people
can
point
to,
and
I
think
michael's
proposal
seems
reasonable.
You
definitely
want
to
have
a
pool
of
funds
for
projects
for
experiments,
and
I
mean
I
know
for
a
fact.
We
could
put
that
to
good
use.
C
If
you
know
if
nothing
is
seeming
like
a
good
option,
but
I'm
sure
you
know
having
a
pool
of
fun
would
certainly
help,
and
you
know
we
need
all
we
can
to
drive
the
needle
forward
and
get
more
people
excited
about
security
reviews
and
more
organizations
more
organizations
on
board
with
them.
So
I
I
think
it's
a
reasonable
proposal.
H
Yeah
I
see
that
michael's,
just
switching
it
over
to
you,
know,
yeah
know
money
to
be
used
as
as
determined
it's
harder
to
object
to
that
for
sure.
Yeah
yeah,
whether
or
not
they
say
yeah,
I
don't
know
but
yeah.
A
H
I
I
would
love
to
see
more
available
for
doing
analysis,
but
I
think
that
you
know
I'm
sorry.
H
H
And
go
but
not
practically,
but
I
I
don't
know
how
to
estimate
that
yeah.
A
So
so
so
so
the
problem,
so
my
experience
so
here
is
that
running
the
tools
is
actually
really
cheap.
Even
so
we
we
built
a
containerized
version
of
codeql
and
and
a
bunch
of
other
tools,
so
we
run
these
things
across
right,
right,
vast
swaths
and
I
think
we're
paying
it's
like
300
bucks
a
month,
300
bucks
a
month
and
we
get
about
500
scans
a
day.
A
Out
of
that.
So
it's
a
reasonably
good
clip.
We
could
expand.
You
know,
increase
it.
What
whatnot
the
problem
is
you
get
all
those
results?
You
dump
them
in
a
pot,
and
now
you
have
a
pot
of
maybe
bugs
and
the
analysis
of
the
triage
actually
costs
people
time
and
that's
I
I
you
know
the
best
you
can
do
is
tweak
the
rules
so
that
they're
less
noisy
or
they're
more
actionable,
but
you
still
have
to
look
at
them
with
your
eyes.
A
H
In
fact,
over
time,
if
you
work
on
those
I
mean
I'm
not
the
first
to
observe
this.
If
you
work
at
stuff
over
time
pretty
much
after
a
while,
all
the
reports
are
false,
positives.
A
Yeah
yay,
so
so
as
a
as
a
potential
experiment.
What
if
we
you
know
gave
you
know
a
batch
of,
I
don't
know:
100
static
analysis
results
for
a
particular
project
to
somebody
and
they
triaged
them
all,
and
they
said
you
know
these
60
are
false,
positive,
low
value.
These
three
are
really
really
interesting
and
we
gave
them
you
know
and
that
took
them.
You
know
x
amount
of
time
and
we
gave
them
y
amount
of
dollars
for
that.
So
it's
kind
of
like
a
triage
bounty.
H
Well,
you
can
mention
right
here
we're
discussing
a
triage
boundary,
I'm
not
sure
exactly
how
it
would
work,
but
why
don't
you
just
mention
as
a
you
know,
maybe
a
triage
bounty,
and
then
we
can
keep
talking
about
that.
I'm
not
sure
how
that
would
work,
but
let's
talk
yeah,
it
might
might
be
really
cool
if
we
could
figure
out
how
to
make
that
work.
I
mean.
E
From
experience,
I,
like
I've,
sat
down
with
codeql's
results
and
like
just
scrolled
through
pages,
and
it
took
me
two
hours
of
scrolling
through
like
an
hour
and
a
half
worth
of
scrolling,
and
I
found
one
remote
code
execution
vulnerability
in
a
pretty
you
know,
pretty
critical
quote.
Excuse
me
vulnerability
and
piece
of
software
called
express,
but,
like
it
took,
you
know
from
experience.
There's
a
you
know:
there's
a
lot
of
small
projects
out
there,
so
also
knowing
which
projects
to
look
at
to
begin
with
is
also
really
valuable.
Correct
right.
A
A
You
look
at
the
other
way
and
you
say
it
only
took
you
two
hours
to
find
a
critical,
remote
code,
execution
vulnerability
in
module.
Express
that's,
not
bad!
That's
fair!
That's
fair!
You
just
regard
a
lot
of
scrolling
well,
yeah,
good,
auto
scroll.
H
Yeah
I've
I
I've
gotten
used
to
it.
I
I
michael,
I
would
suggest
we
move
on
for
ty.
We've
got
a
lot
to
cover
today,
yep,
okay,.
A
So
we've
been
live
for
about
a
week
and
a
half,
maybe
maybe
two
weeks.
This
is
the.
This
is
the
opening
page.
There
are
a
few
major
projects
like
curl,
kubernetes
and
node
that
are
always
here,
the
other
ones
kind
of
change
randomly,
but
each
of
these
are
projects
that
we've
cataloged.
So
if
I
want
to
search
for,
I
left
these
are
all
projects
so
the
project,
the
links
here
go
to
the
dashboard.
A
The
links
here
go
to
the
api,
so
if
you
so
hold
holman
left,
if
you
click
that
this
is
the
raw
data
that
we've
collected.
A
If
you
go
here
instead,
you
see
the
dashboard
dashboard
still
has
a
couple
bugs
which
I
will
get
fixed
before
the
demo.
On
monday.
A
H
A
Oh
the
point,
so
what
we
have
here
is
data
from
the
pr
from
the
criticality
score
best
practices,
badge
scorecard,
secure
reviews
and
then
high-level
stuff.
If,
if
available
and
the
urls,
that
we've
pulled
out
of
things,
so
it
all
goes
here,
you
can
also
just
search
you
know
through
there,
if
that,
if
you'd
rather
do
that
right
now
we
have
collected.
A
A
hundred
and
six
thousand
projects
two
and
a
2.6
million
metrics,
but
you
know
you
can
see
most
of
that
is
the
criticality
project,
indexed
a
ton
of
stuff,
so
that
that's
where
a
lot
of
the
data
is
coming
from
like
this.
H
A
I
don't
know
it's
a
good,
let
me
so,
if
you're,
if
you're
on
this
page,
I
I
want
to
say
that
I
put
it
over
here.
Yeah.
B
A
Which
we
could
do
I
mean
yeah,
I
blog
you
know,
I'm
not
gonna
touch
it
now,
because
it's
friday
good
play
good
play,
but
but
yeah
we'll
do
something
here
where
it's
like
106,
000
and
and.
G
A
So
yeah,
so
that's
that's
that
so
what
I?
What
I
would
ask
everybody
to
do
is
play
with
it
for
15
minutes
be
kind,
don't
don't
try
to
make
it
explode,
but
you
know,
obviously,
if
you
find
any
bugs
or
anything,
please
just
open
up
an
issue
and
if
it's
the
kind
of
thing
that
I
can
fix
before
monday,
I
will
there
was
a
blog
post
going
out
on
monday.
A
A
B
Mike
sorry,
I
just
had
a
quick
question.
I
was
wondering
about
the
I'm
not
sure
exactly
how
grafana,
like
versioning
control
kind
of
usually
works.
But
do
you
think
there's
any
way,
because
I
I
actually
I
went
through,
I
fixed
a
lot
of
the
color
coding
like
a
lot
of
just
the
more
kind
of
aesthetic
bugs,
because
I
knew.
H
B
Presented
soon
like
is
there
any
way
we
can
like
toss
that
jason
api
in,
like
somewhere
in
version
control
and
get
because
then
I
think
it'd
be
easier
for
me
or
for
everyone
to
be
able
to
help
out
with
these,
like
just
messing
with
the
dashboard,
so
you
don't
have
to
do
it
all
yourself
or
anything.
Yeah.
A
A
A
This
is
the
full
there's,
nothing
well
other
than
the
data
source
connection,
there's
nothing
else.
So,
okay,.
A
In
in
the
repo,
maybe
create
a
folder
or
whatever
dump
it
in,
and
we
can
do
that.
At
the
same
time,
though,
or.
D
Versions,
every
version
is
saved,
so
you
can
always
go
back.
B
Oh,
that's
awesome
so
yeah.
I
always.
B
Little
less
stressful
in
case,
something
like
nuclear
happens
by
accident.
H
So,
michael
sometimes
we
want
not
before
march
may
3.
not,
but
sometimes
we
ought
to
sit
down
and
kind
of
walk
through
this
dashboard
of
what
to
add
what
to
remove.
Yes,
I
have
no
trouble
with
having
an
early
alpha.
Go.
A
H
E
H
Yeah,
I
I
have
to
admit
I
was
I'm
not
sure
how
to
apply
apply
that
here,
because
you
basically
have
to
list
out
things.
Their
proposed
approaches
identify
the
problems
of
your
organization,
probabilities.
You
know
low
and
high
costs
and
for
an
individual
project.
I
don't
know
how
you
would
do
that.
E
Yeah,
the
big,
the
bigger
thing
that
I
wanna
I
wanna
capture
is
here
is
like
great.
We
have
all
this
data
and
we
like
kind
of
know
what
the
what
the
risks
are
in
these
things,
like
does
do
the
numbers
that
we
compute
actually
match
the
reality
of
like
the
projects
that
have
been
popped
in
the
past
and
caused
damage
right
like
would,
if
we
were
to
retroactively,
apply
this
data,
would
that
have
actually
identified
the
things
that
were
risky,
that
caused
issues
to
organizations
and
companies,
and
you
know
all
that
stuff.
H
J
H
Does
that
make
sense
so,
instead
of
trying
to
now
the
only
the
risk
with
that
and
the
slight
difference?
Is
that
you're
not
really
comparing
against
all
things?
So
you
know?
Oh
my
gosh.
Somebody
didn't
review
every
line
of
code.
Well,
how
many
projects
review
every
line
of
code
well
very
very
few.
Well,
then,
that's
not
really
distinctive!
Is
it
but,
but
nevertheless,
if
we
could
look
and
say
hey
this
project,
you
know
if
you
could
find
at
least
correlations
they're,
not
causations,
but
they
still
make
you
suspicious
up.
I
see
a
hand
up.
G
Hey
yeah,
just
on
that
note,
I
noticed
you've
been
talking.
You've
got
some
notes
in
there
about
chaos.
I've
been
using
chaos
in
a
related
project
to
analyze
post
incidents,
to
see
if
there
was
any
of
those
kinds
of
trends,
and
one
of
the
things
that
I
noticed
on
that
front
is
often
when
a
package
is
taken
over
by
an
attacker
like
if
they
get
publishing
rights
when
they
submit
their.
You
know
nefarious
code.
G
So
there's
like
there's
a
lot
of
data
there
that
I've
noticed
as
giving
that
as
one
example
where
you
can
start
to
see
some
sort
of
like
trends
that
an
incident
might
be
taking
place.
I
hope
that's.
G
Yeah
that
data
is
coming
from
the
git
history,
I
believe,
and
basically,
if
you're
tracking,
on,
if
you're
tracking
a
project's
health.
One
of
the
data
points
that
you're
collecting
is
how
long
they
take
to
review
commits
yeah
and
the
average.
The
average
seems
to
be
for
the
projects
that
I've
looked
at
might
be,
let's
just
call
it
24
hours,
but
on
the
commit
that
contains
nefarious
code,
it's
abnormally
short
like
15
minutes
or
less
because
they.
H
Okay,
all
right,
there's
a
paper
called
the
backstabbers
knife
collection.
If
you're
looking
for
supply
chain
attacks
on
open
source,
it's
one
of
the
best
survey
papers.
It's
really
the
best
survey
paper
I
know
of
on
the
topic
and
they
identify
some
some
some
things.
We
could
look
for
so
yeah.
C
A
Cool
okay,
luigi
you
had
a
question
on:
will
the
project
be
free
for
all?
I
would
say
yes,
I
I
I'm.
I
can't
imagine
a
scenario
where
we
would
want
to
charge
for
access
to
to
the
metric.
I
assume
you're
referring
to
the
metric
dashboard
right
yeah.
I
can't
I
can't
imagine
an
area
where
we
would
charge
for
it.
I
don't
want
to
be
a
business
in
in
this
regard.
A
If
the
costs
are
out
of
hand
or
like,
I
think
we
would
have
to
explore
other
options
on
what
this
thing
would
turn
into
so
yeah,
I
would
say
I
would
say
for
now:
let's
just
assume
it'll
be
free
for
all
forever
and
open
source.
So
if
somebody
wants
to
run
it
themselves,
so
the
nice
thing
about
it
right
now
is
there's
nothing.
A
A
You
can
run
it
yourself
and
that's
fine
too
we're
just
hosting
it
because
it's
convenient.
I
think
once
we
add
different
collectors
that
might
be
different,
we
might
have
some
something
that,
like
we
keep
history
about,
that
you
wouldn't
necessarily
have,
but
I
think
it'll
be
fine.
Okay,
thank
you,
michael
yeah,
security
reviews.
I
do
wanna
think
about
how
we're
gonna
increase
this
number.
You
know
the.
A
I
would
like
to
get
the
blog
out
on
monday,
so
we'll
we'll
just
push
and
we'll
we'll
get
that
all
all
going.
You
know
that
this
was
kind
of
the
the
driver
behind
like
hey.
Could
we
just
do
like
a
crowdsource
bug?
Bounty
thing
just
like
increase
the
numbers?
A
I
also
think
I
mean
some
of
this
is
actually
on
me,
so
I
would
like
to
get
make
sure
that
other
people
have
maintainer
access
to
the
repo
where
they
can
do
merges
and-
and
things
like
that,
because
I
feel
like
more
and
more
of
a
bottleneck
these
days.
So
so
we'll
do
that
and
but
yeah
we'll
just
try
to
get
get
more,
get
more
reviews
out
there.
A
C
Yeah
and
just
to
just
to
reiterate
and
we're
we're
really
hoping
that
if
we
get
the
manage
audit
program
off
the
ground,
that's
going
to
be
a
direct
pipeline
of
you
know.
The
full
program
will
be
around
40
plus
research
papers,
so
reviews.
So
that's
the
goal
so
hopefully
we'll
we'll
be
getting
that
number
up.
Cool.
A
Cool
yeah-
and
I
have
I
have
a
a
little
bit
of
a
backlog
that
I
want
to
publish
out.
So
I'm
I'm
gonna
try
to
get
that
out
today-
probably
wouldn't
be
merged
by
monday,
but
at
least
it'll
be.
C
Okay,
yeah
yeah.
If
you
could,
if
you
don't
mind,
merging
my
pull
request,
because
I
cleaned
my
reviews
up
a
little
bit
just
so
that
it
looks
cleaner,
and
so
I
figured
that
would
look
good.
Let.
C
A
Okay,
so
yeah,
so
on
monday
hope
everybody's
gonna
make
the
town
hall
we've
got
a
bunch
of
stuff,
including,
as
I'll
do
I'll
talk
a
little
bit
about
the
metric,
dashboard
and
scooter
reviews,
so
it
will
be
officially
public
monday
at
one
of
the
blog
goes
out.
So
I
guess
like
10
or
11..
I
know
dave
is
talking
a
bunch
about
recent
events
like
code
cove
and
dependency,
confusion
and
the
university
of
minnesota.
A
No
minnesota,
michigan,
sorry
minnesota
stuff.
So
be
a
good
conversation.
Adrian
you
see
your
hands
still
up,
but
I
think
it's
just
up
from
before,
but
if
not.
A
A
A
And
really
what
I'm?
What
I'm
looking
for
here
is.
This
is
a
terrible
idea.
This
is
an
idea
that
somebody
else
is
already
like
there's
so
many
dead
bodies
on
the
hill.
From
trying
to
do
this,
don't
even
try
it
or
it's
a
great
idea.
A
No
one
else
has
thought
of
this
or
somewhere
in
the
middle,
I'm
not
going
to
feel
bad
if
this,
if
this
is
a
terrible
idea,
so
one
of
the
main
challenges
that
I've
seen
expressed
is
that
when
you
get
a
package
from
npm
or
nougat
or
pipe
I
or
any
place
else
there's
you
have
no
guarantee
that
the
package
has
any
semblance
or
any
resemblance
to
the
source
code
that
it
claims
to
be
from
and
often
the
package
itself
is
less
it's
harder
to
reason
about
the
security
of
the
package
than
it
is
to
reason
about
the
security
of
source
code.
A
You
could
make
the
reverse
argument
as
well.
I've
had
people
do
that
as
well,
but
my
feeling
is
that
source
code
is
easier
to
analyze
than
minified,
obfuscated
or
binary
bits.
A
And
there
aren't
today
prevalent
publishing
pipelines
that
say:
okay,
I
give
this
to
the
trusted
party
that
takes
my
source
code
checks
out.
This
tag
does
a
build
and
then
publishes
for
me,
and
you
can
set
this
up
yourself,
but
most
packages
don't
do
that.
So,
if
you
just
given
a
random
npm
package,
you
have
to
trust
that
it
came
from
the
source
that
was
associated
with
it,
and
I
was
thinking
well.
Wouldn't
it
be
interesting
to
test
that
out.
A
Take
a
random
npm
package,
find
the
source
code
and
try
to
reproduce
the
package
from
the
source
code,
and
it
doesn't
have
to
be
like
bit
for
bit
exact
match,
but
at
least
show
the
diff
and
point
out
where
the
diff
is
interesting.
Are
there
like
different
commands
in
the
package
that
aren't
anywhere
in
the
source,
or
is
it
just
like
version
number
increments?
Because
the
author
forgot
to
to
do
a
thing
you
know:
are
there
additional
files
in
the
package
that
aren't
in
the
source
code?
Are
there?
Are
there
you
know?
A
Certainly,
there
should
be
files
in
source
code
that
aren't
in
the
package
like
documentation
and
stuff
like
that,
but
like
kind
of
drawing
that
out,
so
what
I
was
imagining
was
a
website.
You
don't
reproducible.openssf.org.
A
You
type
npm
left
pad
and
you
see
perfect
matches
exactly
and
then
you
type
npm
or
you
know,
pi
pi.
You
know
foo
and
you
see
this
thing
only
has
like
a
47.
You
know
correlation,
there's
all
sorts
of
other
stuff
here
we
we
can't
build
it
or
you
do
zlib
and
you're
like.
I
can't
even
find
what
compiler
zlib
used
here.
So
I
can't
I
can't
even
attempt
this
or
you
know
we
tried
12
different
compilers
and
we
found
the
right
version
of
the
compiler
to
produce
the
same
binary
output.
A
A
Would
this
provide
any
value
like?
Would
people
use
this?
Would
this
be
helpful
because
the
natural
argument
that
I
would
have
against
this
is
yes,
but
you
still
don't
know
if
the
source
code
is
any
good,
so
all
you've
determined
is
that
you
don't
have
the
case
where
the
source
code
is
benign,
but
the
package
is
evil.
That's.
H
H
A
fairly
it's
not
I
mean
the
most
common
attack
is
typo
squatting,
but
once
you
once
you
eliminate
type
of
squatting,
this
is
a
already
a
legit
serious
problem.
So
let
me
first
of
all
give
you
let
me
answer
your
question
two
thumbs
up
for
me.
Okay,
I
think
this
is
very,
very
sensible.
I
I
quickly
typed
in
some
comments
on
the
notes
you
can
buy
it
or
not,
but
I
think
what
we
ought
to
do.
H
H
If
you
click
on
that
link,
you
will
see
that
a
large
number
of
linux
distros
are
already
working
on
this
with
very
it's
very
hard
to
get
a
hundred
percent
of
all
packages,
there's
always
a
couple,
but
especially
on
big
ones
like
debian,
where
you
know
it's
just
kind
of
it's
hard
to
fix
that
last
one
percent,
but
the
vas
I
mean
they're
over
some
of
these
folks
are
over
99
reproducible
for
their
packages
given
or
at
least
reproducible
in
a
in
a
variant
and
they're
working
on
getting
the
variant
into
their
mainstream.
H
But
I
don't
know
about
the
language
level
repos.
I
don't
think
most
of
them
were
doing
this
at
all.
Does
anyone
can
anyone
contradict
me?
I'd
be
delighted
to
be
contradicted.
C
I
know
that
again,
I
second
reproducible
builds
as
well.
There's
been
a
lot
of
talk
about
it.
We've
talked
about
it
in
securing
critical
projects
as
well
and
with
our
contact
chris
lamb,
who,
I
believe
is
doing
that
currently.
C
Yes,
so
I
don't
know
a
ton
about
it
in
terms
of
the
details
of
it,
but
I
know
that
a
lot
of
people
are
advocating
for
it
and
and
from
what
I've.
From
my
experience,
it
can
also
lead
to
fixing
security
bugs
and
just
by
making
something
reproducible.
You
find
a
lot
of
problems
and
things
that
have
been
around
for
a
long
time.
A
A
Great
to
get
you
know
I'll
actually
do
you
know
chris
personally
or.
C
I
do
yeah
I
can.
We
can
try
and
get
him
in
to
speak
at
one
of
our
meetings.
A
Move
it
around
for
him
yeah,
because
so
I
guess
the
main
question
then
is,
would
reproducible
well,
the
fundamental
question
is:
does
reproducible
builds
require
each
project
to
do
something,
in
which
case
it
might
be
really
hard
to
get
the
open
source
ecosystem
to
as
a
whole?
To
do
that,
or
is
it.
H
I
can
tell
you
for
sure
that
for
the
compiled
languages
think
c
in
particular.
Typically,
yes,
now
I
don't
know
about
python
javascript,
I
mean
it's
not
I'm
no
stranger
to
these
languages,
but
I've
not
tried
to
do
a
lot
of
reproducible
builds
there.
Typically,
there
are
several
problems,
though.
You've
got
to
force
orders
in
some
places
where
the
tools
don't
force
an
order
and
yeah.
J
H
A
But
but
I
would
say
even
there
the
if
the
differences
are
like
you,
you
could
have
a
fuzzy,
compare
that
like
doesn't
care
about
differences
in
dates,
or
I
guess
even
orderings.
I
don't
know
how
hard
that
would
be
to
actually
do
but
hit
it's
hard.
H
Been
there
tried
that
yeah
it
turns
out
as
soon
as
you
leave
the
nice
pleasant
world
or
a
fit
for
bit.
Equality
life
can
get
hard
date.
Time
stamps
actually
aren't
too
bad,
but
a
lot
of
that
other
stuff
can
get
hard
in
a
hurry.
But
you
know
what
I
would
say:
don't
kill
the
idea
anyway,
because
once
you
have
a
tool
suite
that
automatically
does
the
checking.
H
Even
if
today,
a
minority
of
packages
do
it
actually
succeed.
First
you'll
know
which
ones
are
succeeding
and
now
you're
giving
visibility
into
which
ones
aren't
now
I'll
I'll
tell
you
a
way
that
you
can
make
your
life
simpler.
The
reproducible
builds
folks
have
already
built
a
debug
tool.
H
A
debug
tool
is
probably
the
wrong
term.
It's
called
diffisco,
oh
yeah,.
H
Yep
so
basically-
and
I
added
this
to
the
bottom
of
the
notes,
when
you're
done
with
the
reproducible
builds
one
thing
you
could
do,
maybe
as
a
separate
run
or
something
is,
if
they're
not
equal,
run
diffiscope
and
make
it
easy
for
somebody
to
see
what
the
differences
were.
Because
then,
even
if
you
can't
completely
automate
the
hey,
it's
equal,
it's
it's
really
the
equivalent
same
thing.
It's
just
not
bid
for
bid.
H
A
H
I
mean
he's
already
spoken
once,
but
I
suspect
he
said:
hey
different
working
group
yeah.
B
H
We're
all
talking
about
this
guy,
I
don't
think
he'll
mind
too
much
talking
about
reproducible
builds,
so
you
know
having
him
back
for
another
round.
I
don't
think
we'll
kill.
A
A
H
But
you
know
I:
I
don't
think
that
these
are
in
conflict
by
the
way,
because
what
the
reproducible
builds
folks
is
they
point
off
to
different
projects
which
are
trying
to
reproduce.
I
I
showed
the
link
there.
If
reproducible
open,
ssf
org
reproduce
some
packages
for
some
language
ecosystems,
I
suspect
that
they
would
go
and
link
to
it.
A
H
Yeah
I
mean
they,
I
think
they
do
run
some,
but
I
mean
yeah.
We
I
mean
we
could.
We
could
certainly
talk
to
him
and
see
hey.
What
can
we
done?
Yeah
they
list
there
by
the
way,
just
fyi
my
phd
dissertation,
although
it
wasn't
on
reproducible,
builds,
was
on
something
related.
So
I
have
way
too
much
experience
in
trying
to
get
things
reproduced.
A
Very
cool:
that's
the
end
of
what
I
have
to
talk
about
anything
else.
Anybody.
H
No,
but
let
me
re-rain
back
may
third
is
the
town
hall
I
mean
you
know:
we've
had
several
town
halls
and
I
so
I
don't
know
how
many
people
will
come
to
the
next
one,
but
the
the
in
the
upcoming
down
hall
really
the
the
feature.
The
feature
event.
The
cool
thing
really
is
the
stuff
from
this
working
group
so
and
I
notice
and
I'm
grateful
that
you
had
slides
that
did
screen
captures
from
the
oh.
A
H
All
right,
yeah-
and
you
know
I
I
put
in
the
chat
note
about-
we-
have
talked
about
creating
a
new
reproducible,
builds
working
group,
but
that
hasn't
happened.
So
so
I
will
add
that
to
the
notes.
A
Cool,
and,
and
actually
so,
particularly
for
david,
you
and
ryan,
there's
a
chat
that
we
had
at
the
very
beginning
about
the
package
manager
summit
that
that
we
did.
A
I
don't
know
somewhere
in
the
before
times
that
I
think
everybody
thought
was
very
useful
and
important
and
getting
spinning
up
an
open,
ssf
working
group
for
the
package
managers
themselves.
So
having
npm
github,
pi,
pi,
gradle,
maven,
everybody
there
and
kind
of
sharing
and
talking
would
be
very,
very
high
roi
for
the
community.
A
I'm
not
sure
what
it
would
like.
I
think
it's
probably
more
of
a
governing
board
or
tac
decision
and
like
program
to
go
out
and
like
make
happen.
But
I've
heard
it
a
couple
times
now
and
I
think
it
would
be
super
important
to
explore.
H
Yeah,
I'm
right
now,
looking
through
their.
Unfortunately,
I
don't
know
which
date
the
presentation
was
on.
F
H
C
Oh
yeah
for
for
the
original
reproducible,
builds
yeah.
H
That
just
I
think
it
was
recording,
it
was
recorded
all
right,
so
those
of
you
who
aren't
familiar
with
reproducible
builds
before
we
even
reach
out
to
chris.
We
could
take
a
peek
at
that
video
and
we
could.
We
could
still
invite
him.
I'm
sure
he'd
be
happy
to
talk,
but
at
that
point
we
can
say
we
already
watched
your
video.
We
have
questions
yes,.
A
Cool
so
sorry
working
group,
the
the
the
concept
of
a
working
group
for
the
package
managers
themselves.
Do
you
think
that
that's
something
that
the
critical
projects
working
group
should
drive
or
governing
board
or
attack
like
who?
Who
do
you
think
I
should
who
do
you
think
that
we
should
talk
to
in
order
to
advocate?
For
that
conversation
to
happen.
F
F
Security
and
critical
projects
is
really
the
right
group
to
drive
it
per
se,
because
they've
got
a
few
other
topics
that
they're
dealing
with
but
yeah
I
mean,
I
think
it's
definitely
useful
and
I
don't
know
if
it's
really
even
a
working
group
or
just
you
know,
a
forum
that
you
know
we
sort
of
host
to
to
have
people.
You
know
come
in
and
and
have
these
conversations,
but
yeah
I'll
definitely
add
it
to
the
next
okay
yeah.
I.
A
Mean
so
so
just
be
clear.
What
I
was
really
thinking
is
just
a
I
mean.
If
I
get
a
forum
is
probably
the
right
is
probably
the
right
word
for
it,
but
something
where
npm
can
learn
from
pi,
pi
and
gradle
can
share
knowledge
with
you
know,
nougat,
and
so,
when
they're,
like
god,
what
are
you
seeing
like?
Oh
yeah,
we
keep
getting
hit
by
this
same
ip
address
and
wherever
they'd
be
like.
Oh
great.
A
Thanks
for
sharing,
we
can
all
like
protect
the
the
overall
ecosystem
together
and
it
doesn't
even
need
to
like
it
can.
It
could
be
insular
from
the
rest
of
open
ssf,
because
I
would
want
them
to
be
able
to
share
things
privately
amir
you're
on
mute.
A
Yeah,
so
I
think
that
would
be,
I
think
I
think
it's
important
to
have
that
forum
for
the
everybody's
sake.
F
Like
I
said
I'll,
add
some
medium,
we'll
figure
out.
You
know
the
right
way
to
drive
that
but
perfect.
A
A
Wonderful,
thank
you
all
very
much.
I
appreciate
your
time
have
a
wonderful
weekend.
I
will
see
you
guys
on
on
monday.
Hopefully
thanks
so
much
bye.