►
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
A
A
B
We
we
could
call
out
to
him,
or
he
can
just
call
in
direct
on
on
his
cell
phone
if
he
just
wants
audio
that
should
bypass
if
there's
any
weirdness
going
on
with
teams.
A
Access
to
the
duck-
yes,
I
yeah,
I
think
so.
We
might
need
michael.
B
If
you
can't,
I
will
I
just
I
just
got
home,
so
I'm
good
now.
A
A
E
I
don't
know
it's
the
one
on
my
calendar,
but
I
tried
the
one
on
the
meeting
notes
and
that
did
work.
A
Okay,
so
we've
we've
got
a
good
group
here.
Finally,
with
some
you
know
the
usual
video
conference
kerfuffles
at
the
beginning
of
the
meeting.
Michael,
are
you
do
you
want
to
take
over.
B
No,
that
you're
back
home
or
do
you
want
me
to
yes,
I
mean
I
think
yeah.
I
could
just
kick
it
off
real,
quick
but
sure,
as
you
said,
welcome
everybody.
Okay
have
a
nice
weekend,
happy
monday
and
good
morning
or
good
evening.
So
do
we
it's
on
the
court.
B
Welcome
back
everybody
I
think,
for
agenda,
for
today.
Anybody
has
anything
else
that
they
would
like
to
talk
about.
Just
add
it
into
the
at
the
end
of
the
agenda
on
the
on
the
notes
for
I'm
going
to
check
my
email
and
give
everybody
edit
access,
who
doesn't
have
it
so
that'll?
B
Oh,
it's
actually
a
bunch
of
you
guys
right
now,
so,
if
you
guys
are
if
for
for
dicks
and
dan
maya,
nixon
dan
and
maya,
you
guys
should
have
access
and
like
60
seconds.
B
And
yeah,
so
I
think
the
first
first
order
for
today
is
going
through
the
requirements
document,
so
okay,
I'll
I'll
kind
of
hand
it
off
to
you.
If
you
want
to
drive.
A
All
right,
so
in
our
meeting
two
weeks
ago,
or
a
little
less
than
two
weeks
ago,
we
we
walked
through
the
goal.
Scenarios
and
requirements
for
three
projects
that
are
part
of
currently
part
of
this
group
are
related
to
this
group.
One
is
the
cii
best
practices
badge,
the
other
is
the
security
metrics
project,
which
is
definitely
this.
A
That's
what
this
group
has
been
working
on
for
a
couple
of
months
now
and
then
also
a
security
scorecards
initiative
that
the
the
google
team
has
been
kicking
off
and
that
is
they're
looking
at
that
to
feed
into
their
into
some
of
the
work
that
will
go
on
in
the
securing
critical
projects
working
group.
A
So
this
week
you
know,
and
as
we
talked
through
those
we
identified
places
where
there
are
commonalities,
common
goals
and
scenarios
and
some
places
where
there
are
some
differences
and
what
I
have
done
since
then
is
I
made
an
effort
to
create
the
section
that
I
call
common,
where
I
was
trying
to
capture
the
things
that
are
common
across
them
and
and
yeah.
So
I
guess
I
we
can.
We
can
start
with
that.
A
I
did
another
thing
as
I
was
going
through
this,
which
was
to
you
know,
try
to
frame
some
of
the
requirements
that
were
listed
in
the
top
ones
in
more
general
ways,
so
that
you
know
so
that
it
might
be
that
you
know
we
can
think
about
them
all
as
having
similar
goals,
scenarios
and
requirements.
So,
but
let's
just
walk
through
this
and
see
what
people
think
and
what
what
comments
we
might
have
so
for
common,
let's
see
from
so
for
common
goals.
A
When
I
tried
to
summarize
across
all
of
them,
a
goal
that
I
came
up
with
was
collect,
curate
and
communicate
metrics
that
reflect
the
security
posture
of
open
source
components.
A
So
why
don't
I?
Why
don't
I
go
through
all
of
this
first
quickly
and
then
we
can
come
back
and
dig
in
where
there
might
be
questions
or
we
want
to
try
to
clarify
some
wordings.
A
According
for
scenarios,
I
had
maintainers
and
open
source
contributors
can
understand
what
concrete
steps
they
can
take
to
improve
the
security
posture
of
their
projects.
A
Another
one
is
organizations
can
understand
the
security
posture
of
open
source
projects
they
depend
on.
Another.
Is
organizations
can
automate
internal
checks
or
policies
based
on
security
data
and
then
the
last
one
I
have
is
security
reaches
researchers
can
understand
the
security
posture
of
critical
at-risk
components
and
share
those
learnings
with
others
in
the
community
and
potentially
collaboratively.
A
So
then
for
requirements-
and
this
is
rough
and
these
will
need
some
required.
Some
refinement
I
had
security
criteria
can
be
as
specific
as
possible
and
not
limited
to
general
recommendations,
for
example,
for
a
specific
programming
language.
We
might
have
criteria
that
recommends
or
requires
language,
specific
winters
and
analyzers
to
be
run
on
the
code
base.
A
A
second
one
is
consumers
of
the
data
can
filter
data
to
meet
specific
requirements.
So,
for
example,
a
customer
may
require
that
the
data
received
contained
only
automatable,
objective
data
or
consumers
may
require
that
the
data
must
be
actionable
such
that
it
can
be
used
for
helping
a
project
to
check
all
the
boxes.
E
Can
do
you
want
comments
as
we're
going
along
or
yeah
go
ahead?
Okay,
yeah!
I
I
specific
recommendations
is
good.
I
do
worry
about
work
and
recommending
specific
lenders
and
analyzers
good
idea,
especially
if
you
make
it
easy
to
click
a
button
requiring
specific
lenders
or
analyzers.
I
worry
about
because
you
know
stuff
gets
changed,
staffer
gets
abandoned
and
I
would
much
rather
require
you
know
here.
Are
the
checks
not
use
tool
x.
E
G
E
And
then
telling
here's
a
tool
that'll
help
you
do
that,
but
but
if
a
better
tool
comes
along,
I
want
us
to
be
able
to
move
to
the
bender
tool.
A
G
A
Yeah
dan
had
a
conflict
for
the
first
part
of
the
meeting,
so
so
he'll
be
back
with
us,
but
I
know
where
they're
coming
from
so
for
the
securing
critical
projects
they,
I
think
they
are
trying
to.
You,
know
they're
trying
to
create
a
framework
where
they
will
require
specific.
They
want
to
require
specific,
linters
or
analyzers
to
be
run,
and
that's
so
that
they
can
have.
A
You
know
so.
They're
looking
at
and
others
are
looking
at,
providing
some
resources,
some
funding,
so
we
can
have
contractors
that
go
actually
do
a
bunch
of
work,
and
so
some
to
you
know
to
to
analyze
code
bases,
and
they
want
to
be
able
to
do
that
work.
And
then
you
know
get
a
check
when
when
that
work
is
done.
So
I
think
we
might
want
something
that
recommends,
but
maybe
also
captures
what
tool
was
used
to
meet
the
recommendation
or
something
like
that.
B
A
B
I
I
think,
having
that,
as
I
mean,
maybe
I
think
it
might
make
sense
to
kind
of
separate
out
the
what
what
the
secure
piece
will
be:
the
characteristics
of
a
secure
piece
of
software,
meaning
it
doesn't
have
this
type
of
issue
and
the
ways
that
like,
in
other
words
in
order
to
get
the
whatever
you
want
to
call
it,
the
the
the
the
the
critical
project
stamp
certification
of
goodness,
that
might
require
a
specific
tool
to
to
pass,
but
from
a
for
other
parts
of
this
kind
of
document.
B
If
we're,
if
we're
giving
security,
metrics
and
recommendations
on
how
to
validate
those,
I
think
I
agree
with
david,
mostly
that
we
should
be
careful
about
going
going
like
over
indexing
on
a
particular
tool.
We
can
certainly
do
that
on
the
on
the
certification
side.
I
think
because
the
certification
is
that
you
pass
that
that
the
tool
says
you
passed
right.
A
A
Right,
yeah,
okay,
I
I
see
good
and
thanks
for
capturing
that
david
in
a
comment,
and
I
can
take
a
pass
at
updating
that
text
a
little
later.
What
we
want
is
there,
what
we
want
is
the
the
security
requirements
and
the
tool
can
status,
be
used
to
satisfy
that
so
or
just
document
the
that
has
been
satisfied.
A
Okay,
all
right!
So
next
one,
let's
see
I
did
I
read
through
this
one
already,
but
we
can
stop.
One
has
comments
on
that.
Consumers
of
the
data
can
filter
it
to
meet
their
specific
requirements
so
anyway,
so
there's
granular
querying
on
the
data.
I
think
it's
the
way.
I
want
to
summarize
that
requirement.
A
Okay
data
may
be
populated
for
any
open
source
project
without
any
work
or
interaction
from
maintainers.
G
C
B
The
first
any
before
any
open
source.
E
I'm
sorry
right
right,
but
but
but
for
example,
if
there
is
a
repo
that's
accessible
for
any,
I
would
say
any
open
any
open
source
software.
You
know
as
long
as
a
public
repo.
B
B
Right,
but
I
think
we're
just
going
to
include
them,
because
I
so
it's
actually
a
good
point
like
like
what
is
the
target
of
like
what
is
our
kind
of
first
class
object
in
this
whole
thing?
Is
it
a
repo?
Is
it
a
distribution
of
a
piece
of
open
of?
Is
it?
Is
it
a?
Is
it
a
binary
archive
that
you
get
from
nougat
or
npm
or
pi
pi?
Is
it
a
repo?
Is
it
the
the
the
spiritual
essence
of
z,
lib
or
left
pad,
and
then.
E
But
but
you
know
what
I
I,
although
I
think
that
I
think
folks
who
implement
the
tools
should
have
flexibility
in
what
they
want
to
do.
I
think
if
there's
no
public
repo,
maybe
we
don't
worry
about
it.
So
much
because
that's
already
a
massive
risk,
you
know,
maybe
you
don't
want
to
deal
with
that
project
at
all,
and
even
if
you
do
you
don't
have
a
reasonable
way
to
interact.
Now,
I
don't
it
doesn't
have
to
be
on
github,
it
could
be
get
lab,
it
could
be
savannah.
E
You
know
somebody's
special
site,
some
people,
some
larger
projects,
have
their
own
sites,
but
I
mean
if
somebody
wants
to
do
something
else
where
you
know
any
anything
posted
on
npm
I
mean
go
do,
but
I
I
think
that
would
be
a.
I
think.
That's
excessively
hard.
E
A
E
A
A
A
And
are
you
thinking
of
data
like
machine
learning,
data
or.
H
Any
data
that
I
might
use
as
part
of
my
application
eventually
so
my
training
data
set,
for
example.
If
I
could
verify
the
same
things
for
my
training
data,
that
would
be
pretty
cool.
E
Yeah,
if
you're
gonna
do
this,
please
put
repo
somewhere
in
this
bullet,
because
yeah
projects
repos
before.
E
C
Before
yeah.
I
E
C
E
J
Can
see
it
just
to
add
to
maya's
points
around
binaries?
I
think
it's
nice
to
display
this
data
for
binaries,
for
which
we
don't
have
the
type
of
information
we'd
have
from
source
code
as
well
to
help
underline
and
illustrate
the
risk
associated
with,
with
just
picking
up
a
binary
off
the
internet
and
also
like,
I
guess,
there's
this
whole
question
of
provenance
and
we
can
start
to
highlight
in
a
really
consistent
way
across
the
different
forms
that
software
can
take
within
this.
J
This
metrics
dashboard
like
what
those
kind
of
meta
best
practices
are
like
don't
run.
You
know
inside
random,
binaries
that
you
find
on
the
internet.
So
I
think
it's
it's
helpful
to
illustrate
those
things.
I
guess
to
the
point
about
data.
I
struggle
with
it
a
little
bit
just
because,
like
you,
can
have
something
look
like
a
data
set,
that's
actually
super
malicious
and
I
don't
know
if
we
will
in
any
near-term
time
have
tools
that
will
help
us
validate
that.
J
So
I
would
worry
a
little
bit
about
us
giving
like
false
promises
on
a
data
set
just
because
thinking
about
how
to
do
the
tooling
to
analyze
it
sufficiently.
I'm
not
sure
if
that
would
be
something
we
could
build
in.
H
Yeah,
that's
totally
fair.
I
don't
think
that
data
is
something
that
at
least
I'm
not
attempting
to
tackle
anytime
soon,
but
with
something
much
less
so
maybe
maybe
you're
right,
maybe
remove
it
from
here.
Yeah.
J
It
would
be
amazing,
like
I
think,
about
like
the
extensibility
and
if
we
build
for
extensibility,
that
would
be
really
cool
to
do
it
with
data,
but
I
think
about
like
just
as
a
threat
actor
like
what
you
can
do
with
a
with
a
malicious
data
set,
and
I
I
would
just
worry
that
we
have
a
big
problem
to
solve
before
we
could
do
it
with
any
level
of
assurance.
J
B
Sorry,
sorry,
I'm
assuming,
and
I
I
think
we
should
do
this
but
open
source,
meaning
if
it's
not
under
an
open
source
license.
It
doesn't
exist
to
us
and
I
think
that,
being
under
the
umbrella
of
the
open
ssf,
I
think
that's
a
reasonable
way
to
split
the
universe
in
between
kind
of
like
the
the
freeware
or
like
under
some
proprietary
or
weird
license
versus
an
osi
approved
license,
in
which
case
we
only
care
about
the
osi
approved
stuff,
because
we're
doing
this
for
the
good
of
humanity
and.
E
I
think
there's
a
distinction
between
what
a
tool
can
do
and
what
the
requirements
are.
I
mean
you
know
a
tool
may
be
able
to
analyze
an
arbitrary
repo,
but
you
know
we're
you
know
we're
focusing
on
the
open
source
ones
and
then,
if
the
tool
can
be
used
for
other
additional
purposes.
Well,
that's
great
right
and
I
suspect
a
lot
of
these
can
be
reused
for
private
repos,
but
but
yeah.
We,
I
think,
the
at
least
I'm
I'm
looking
at
that
word
requirement.
A
Yeah,
well,
we
can,
I
think,
of
this,
as
you
know,
we're
now
we're
defining
the
sort
of
the
the
big
scope
or
like
the
extensible
scope
or
defining
our
extensibility
and
then
we'll
narrow
back
down
to
you
know
what
what's
our
first
step.
What's
our
what's
our
next
step
it
sometimes
this
is
called
the
north
star.
So
what
you
know
it's
our
defining
our
north
star
and
then
we'll
define
how
we
the
steps
we
take
to
get
to
the
north
star.
A
B
It's
like
it's
kind
of
a
meta
principle
like
that
that
if
we
come
across
problems
that
are
more
applicable
to
non-open
source,
not
so
much
open
source,
then
those
will
be
not
focused.
On
I
mean
I
don't
pay
ourselves
into
a
corner
where
we
can't
do
something
that
we
want
to
do,
because
we,
you
know,
wrote
something
down
but
but
but
I
think
I
I
mean
at
least
for
me-
I
I
am
frequently
asked
you
know
hey
what
about
this.
Freeware
thing
is
this
you
know,
and
just
in
my
day
roll
you
know.
B
E
A
Okay,
there
might
be
a
better
way
to
say
this,
which
is
that
I
don't
know
I'll
I'll
think
more
about
this
first
first
bullet,
because
I
do
think
there
might,
you
know,
there's
an
opportunity
that
we're
building
something
that
that
we
use
for
open
source,
but
a
lot
of
companies
would
be
interested
in
doing
it
for
their.
You
know
the
software
inside
of
their
own.
You
know
their
own
proprietary
use.
So,
okay,.
A
A
I
think
that's
still
fair,
so
even
if
it's
you
know,
even
if
it
is
an
npm
or
something
if
it's
available
from
a
public
repo,
in
this
case
the
package
repo
there's
some
data
that
we
can
gather
about.
That.
E
Yeah,
I
think
you
know
it
should
support
common
repos
like
github
and
gitlab,
and
so
on.
I
mean
you
know,
there's
a
limit
to
how
far
you
can
go
that
but.
B
D
E
Yeah,
to
be
honest,
if
you're,
if
you're
encountering
something
like
you
know
something
from
cpan
or
npm,
I
think
the
first
step
would
be
okay,
yank,
the
url
of
the
repo
go
there.
B
Well,
I
mean,
but
but
to
to
be
fair,
you
know.
One
of
the
challenges
we
have
is
that
in
npm
for,
for
example-
and
this
is
true
for
any
package
manager-
it's
not
trivial-
to
determine
that
the
source
code
in
let's
say
the
github
repository
that
it
reports
to
come
from
actually
derived
actually
be.
You
know,
was
turned
into
the
thing
that
you
found
on
cpan
or
npm,
because
it
is
usually
a
build
step
and
the
build
is
not
always
reproducible
and
you
know
so.
B
The
thing
that
people
actually
install
is
the
thing
from
npm,
not
the
thing
from
github.
So
right.
E
I
I
have
a
position
on
that.
I
I'd
like
to
just
kind
of
pitch
it
and
I'll
rep,
I'm
sure
I'll
re-pitch
it
again,
which
is,
I
think,
in
the
long
term.
I
mean
that
is
a
serious
problem
and
we've
seen
a
number
of
malicious
code
examples
exploiting
this.
I
think
there
is
a
simple
solution,
which
is
the
package.
You
know
when
somebody
submits
to
a
package
repo,
a
a
package,
repo
npm
and
so
on,
step
one.
E
E
Initially,
that's
a
warning
and
later
that's
an
error
and
basically
eliminate
that
problem
entirely.
There's
really
no
reason
that
that
distinction
needs
to
exist
other
than
historical
and
it
is
creating
just
a
bonanza
for
the
attackers
who
are
you
know,
so
I
I
think
I
agree
with
you
that
that's
a
problem.
E
I
think
that
we
should
do
is
work
with
the
language
repos,
so
that
this
be
stops
being
a
problem
and
there
are
known
solutions.
B
E
Well
and
many
folks
use
circle,
ci
or
travis
or
something,
but
but
there's
going
to
be
some
pipeline,
there's
going
to
be
some
set
of
of
code
that
executes
the
pipeline,
and
you
know
if
a
if
a
language
level,
repo
reruns
the
pipeline
and
gets
a
different
answer.
E
G
A
I
believe
that
that's
dan,
that's
dan
from
google,
is
that
right,
dan
lawrence,
yep,
okay,
good
okay,
so
I'm
gonna
do
a
quick
recap,
because
I'd
like
to
get
your
your
feedback
on
this.
So
so,
what
we're
doing
now
is
we've
got
a
common
section
to
this
requirements
document
and
the
requirements
are
across
the
best
practice
badge
the
security
metrics
project
and
also
the
security
score
cards,
and
so
what
I've
done
is
taken
a
stab
at
trying
to
define
this
in
a
way
that
does
cross
all
of
those
and
for
goals.
A
And
then
for
scenarios
I
talked
about
maintainers
and
contributors
can
understand
what
they
can
do
to
improve
the
security
posture
of
their
projects.
Organizations
can
understand
the
posture
of
the
projects
they
depend
on.
They
can
automate
internal
checks
based
on
security
data
and
then
security
researchers
can
understand.
You
know
kind
of
more
broadly
the
posture
of
security
projects
and
share
the
learnings
with
others
in
the
community.
A
So
those
are
the
the
top
level
scenarios.
Let
me
stop
on
those
and
see.
Do
you
think
I've
captured
the
security
scorecard
in
there.
A
Okay
and
now
we're
starting
to
work
through
a
set
of
requirements,
so
I'll
quickly
recap
those
as
well
so
the
solution
we
come
up
with
may
work
for
more
than
open
source,
but
our
first
focus
or
our
primary
focus
is
on
analyzing,
open
source
software
and
the
idea
there
is
that
you
know
we're
we're
creating
something
we're
we're
doing
it
primarily
for
open
source,
but
we
don't
want
to
restrict
it
to,
if
possible,
to
be
something
that
organizations
couldn't
use
in-house,
for
example,
or
for
scenarios
other
than
open
source.
A
We
want
the
solution
to
be
extensible
so
that
security
criteria
can
be
collected
for
not
only
for
open
source
projects,
but
repos
packages
binaries,
you
know,
maybe
in
the
future
data
like
training
data,
we
want
to
be
able
to
cover
cover
all
of
those
okay.
So
far,.
A
Okay,
security
criteria
can
be
as
specific
as
possible
and
not
limited
to
general
recommendations,
so
this
came
out
of
the
security
court
car
example.
So
for
specific
programming
language,
the
criteria
might
recognize
winter's
analyzers
to
be
run
on
that
code
base.
Actually
we
had
a
conversation
here
where
we
said
what
we
want
is
that
the
criteria
would.
A
Recommend,
maybe
it's
just
recommend
specific
winter's
analyzers
to
be
run
on
the
code
base.
The
thing
that
we
didn't
want
to
do
is
have
the
criteria
be
tied
to
a
specific
tool,
a
linter
or
an
analyzer,
because
every
time
the
tool
might
change
the
tool
version
might
change.
So
what
we
want
is
a
criteria
and
then
the
criteria
could
be
met
by
running
a
certain.
A
You
know
certain
tools
or
classes
well,
a
certain
tool
or
a
version
of
a
certain
tool
that
does
that
make
sense,
and
david
tell
me
if
I,
if
I
captured
your
your
comment
correctly.
A
E
A
E
B
Actually,
as
you're
saying
that,
I
think
I
think
there's
actually
two
different
avenues
here.
One
is
we're
telling
people
you
know
when
you,
when
you
develop
code
in
python,
you
should
use
bandit
because
bandits
awesome
or
whatever
and
that's
that's
great,
but
that's
a
should
or
recommend
not
not
really
requirement
on
the
other
side
is
the
what
tools
do
we
use
to
determine
these
metrics
and
calculate
things?
And
and
for
that
we
need
to
choose
like
a
concrete
tool.
We
can
always
change
that
later.
B
The
what
we
display
in
in
let's
say
the
ui,
I
think,
should
be
kind
of
tool
agnostic.
So
we
might
talk
about
right,
susceptibility
to
certain
conditions
or
characteristics
or
whatever,
but
we're
not
saying
like
you
know,
bandit
output
is
here,
or
at
least
we
should
try
to
minimize
that
it's
not
possible.
E
E
E
Yeah
so
pep
8's,
the
pep
8,
is
here:
do
this
and
and
here's
a
tool
that
will
check
that
for
you
for
those
who
don't
believe
pep
baits,
the
usual
python
require
guidelines
for
coding,
standard
recommended
coding
standard.
If
I
remember
the
number
correctly.
K
K
I'm
not
trying
to.
I
think
it
is
an
important
distinction
here
like
for
something
like
pep
8.
We
can
say
your
code
should
be
pep
8
compatible
or
for
go.
We
can
say
your
code
should
be
formatted
compatibly
with
the
go
format
tool,
but
I
think
other
cases
it's
like
there's
a
go
security
scanner
that
most
people
use,
there's
bandit
which
most
people
use
for
python.
I'm
not
quite
sure
there
is
a
simple
standard
way
to
say
that
your
code
meets
the
output
of
those
scanners.
K
A
K
E
A
Okay,
all
right,
so
our
next
one,
consumers
of
the
data
can
filter
data
to
meet
specific
requirements.
A
couple
of
examples-
a
customer
require
that
the
data
receive
contain
only
automatable,
objective
data
and
then
the
second
one
is
customers
may
require
that
data
must
be
actionable.
This
is,
I
think
these
are
similar
things.
These
first
two
examples
and,
and
also
data
can
be
queried
at
a
granular
level.
A
I
think
you're,
I
suspect,
you're
fine
with
that
dan
yep.
Okay,
all
right,
then
we
were
at
data,
may
be
populated
for
any
open
source
project
with
public
information,
eg
source
code,
repo
or
package
repo,
without
any
worker
interaction
from
maintainers,
and
then
david
was
busy
putting
a
problem
in
here.
E
Yeah
we
okay
and
my
apologies
for
those
who
have
heard
this
and
are
well
aware
of
this.
You
know
what
goes
into
a
a
software.
You
know
we're
calling
a
like
a
package
repo.
You
know
what
goes
into
package
repos
like
npm
is
not
necessarily
matching
the
source
code
repo
and
we
need
another
name
for
these
distinctions,
but
this
will
do
for
now,
and
attackers
love
this
because
it
means
they
can
insert
some
licious
code
upload.
E
You
know,
grab
somebody's,
you
know
password
or
whatever
upload
an
update
and
nobody's
checking
that
the
that
the
source
code
people
are
reviewing
and
the
packages
that
everyone
is
actually
installing
match
in
any
way.
The
in
my
mind,
the
solution
is
pretty
straightforward
over
over
a
long
time.
E
I
really
want
to
see
npm
pi
pi,
everybody
else,
redoing
those
builds
when
they're
submitted
to
make
sure
that
they
match
the
source
if
they
don't
match
at
least
warn
both
the
submitters
and
users
and
eventually
make
that
an
error
and
over
time,
and
once
a
package
reproduces,
then
don't
let
a
non-reproducible
one
show
up
and
so
over
time.
E
That
would
eliminate
an
entire
attack
vector,
and
that
would
also
mean
that
tools
that
we're
talking
about
for
analyzing
a
source
code
repo,
would
be
justifiably
related
to
the
packages
that
people
install
because
analyzing
what
nobody
installs
doesn't
really
help.
Anybody.
E
That's
right,
that's
right,
but
the
the
problem
with
reproducible,
a
lot
of
the
folks
working
on
reproducible
builds
are
really
at
the
at
the
system.
Repo
levels-
and
this
is
at
the
also
at
the
language
level,
which
in
many
ways
is
easier
frankly,.
K
Yeah
there
there's
another
take
to
this
too,
though,
which
maya
wrote
a
super
awesome
paper
on
right
before
she
left
google.
I
can
try
to
dig
up
the
link.
Maybe
maya
still
has
it
handy,
which
is
verifiable,
builds
slightly
different
concept
from
reproducible
builds,
but
it
kind
of
attacks
the
same
thing,
I'm
not
familiar
with
this
term.
It
sounds
like
I
need
to
be.
D
H
A
All
right,
but
this
is
a
side,
a
side
topic,
so
you
know
for
the
purposes
of
of
the
you
know
the
security
metrics
system
that
we're
thinking
of
we.
You
know
the
point
is
just
that
we
want
to
populate
data
from
any
public
location.
A
I
am
going
to
stop
for
a
second
and
do
a
time
check.
It's
9
48,
so
we
still
have
12
minutes.
Is
there?
Do
we
want
to
keep
going
on?
This
is
there's
michael.
Is
there
something
else
that
that
we
want
to
get
to?
We
could
go
through
these
last
three
items.
We
should
talk
about
what
our
next
steps
are.
B
Let's
keep
going
unless
anybody
has
anything
that
they
really
need
to
talk
about,
although
I
did
want
to
get
just
give
a
shout
out
to
to
luigi
on
the
converting
the
the
dock,
the
the
threat
dock
to
markdown,
which
looks
terrific
for
those
of
you
who
haven't
seen
it.
Please,
please
take
a
look
I'll
wait
until
maybe
the
end
of
the
week
to
to
merge
it
and
then
we're.
B
L
Well,
I
have
tried
to
add
an
alt
text
to
images.
I
don't
know
if
this
is
a
good.
I
hope
so
and
I
have
put
the
reference
I
have
used
to
try
to
understand
how
how
right
these
alternative
texts.
I
think
it
is
important.
I
think
the
markdown
document
is
really
important
to
make
this
document
more
accessible
to
others.
B
B
I'll
post
a
link
to
that
in
the
in
the
chat
here
for
you
cool.
A
B
B
A
bad
actor
here,
but
the
maintainer
is
not
the
one,
that's
incurring
the
risk,
and
while
you
do
have
to
trust
the
maintainer
to
a
certain
extent,
maintainer
saying
you
know,
nope,
don't
care
you
shouldn't
be
using
it.
You
you,
none
of
these
vulnerability
should
apply
because
you
should
never
run
it
in
in
a
cloud.
Environment
doesn't
actually
make
the
risk
go
away.
So
yeah,
correct,
slash
comment
is
fine.
Maybe.
J
To
support
michael's
point
just
because
I
do
a
ton
of
vulnerability
disclosures.
I've
had
a
troubling
number
of
vulnerability
disclosures
where
maintainers
will
come
back
and
say:
that's
not
a
security
vulnerability.
That's
you
know
just
a
regular
bug
and
it'll
like
a
cvs
score
on
it
would
be
like
eight
or
higher.
Sometimes
I'm
hearing
back,
that's
not
a
security
bug,
so
just
to
support
that
point
that
there
needs
to
be,
I
think,
some
level
of
checking
when
the
when
the
maintainers
respond.
J
E
E
J
Yeah,
I
think
the
logic
on
it
is
not
there.
So
I
it's
been
more
of
like
oh
you're,
viewing
this
as
a
security
bug,
but
it's
actually
of
another
type
of
bug
and
we're
like
well,
it's
exploitable.
So
I
mean
so
you
know,
and
this
is
not
what
happens.
Obviously,
the
majority
of
the
time,
but
just
it's
happened
enough
times
in
the
last
year
or
so
to
me
that
I'm
like
wow.
J
This
is
still
a
contentious
issue
at
times,
so
I
just
bring
it
up
because
I
think
it
it's
to
michael's
point
that,
like
taking
things
at
their
word
ignores
the
fact
that
we
will
sometimes
have
debate
around
the
security
relevance
of
something.
B
Do
do
we
so
so
I
think
this
is
important
appeal
to
whom
so
one
way
would
just
be
where
everybody
can
kind
of
comment
in
a
public.
You
know
comment
publicly
and
then
the
reader
can
make
what
they
want
of
it,
and
maybe
we
do
up
down
votes
or
stack
overflow
style
kind
of
reputation
points,
and
then
you
kind
of
get
into.
Is
this
like
a
crowdsourced
thing,
or
is
there
like
a
council
of
security
elders
that
you
know
are
like
the
the
moderators
of
this
entire
system
and
then
how
does
that
scale?
B
So
I
think
there's
it'll
it'll
be
really
easy
when
there
are
like
10
projects,
but
that
won't
work.
When
there
are.
You
know,
20
thousand,
all.
A
Right:
okay,
I'm
going
to
just
leave
that
as
a
comment
right
now
to
think
about
later
next
item
down
any
solution
to
collect
and
curate
information
should
be
usable
by
the
greater
open
source
community
to
monitor
upstream
security.
B
For
this,
do
you
mean
that
if
I
am
the
maintainer
of
a
piece
of
open
source
that
the
dependencies
that
I
take
influence
my
metrics
or
is
it
kind
of
like
a
drill
in
where
you
can
see?
Oh
you're
using
webpack,
therefore
you're
using
crypto
browserify,
and
I
can
drill
into
that
and
like
go
upstream.
A
So
this
came
from
the
scorecards
requirements
and
I
think
what
they
were
getting
at
is
that
any
data
that's
created
from
that
should
be.
I
think
it's
more
should
be
made
available
to
the
upstream
community.
So
it's
just
that
it's
more
of
a!
I
think
the
requirement
is
more
about
transparency,
so
yeah
cool.
E
A
A
So
I
I
put
a
note
here:
it
might
be
a
little
bit
outside
of
the
it's
something
that
it
feels
like,
not
a
not
a
requirement
for
the
gathering
of
criteria,
gathering
or
reporting
of
criteria,
but
something
that
we
can
do
once.
We've
identified
criteria
to
help
it
help
make
it
easier
for
people
to
to.
A
All
right,
we
have
just
four
minutes
left,
and
you
know
that
last
one
we
could
flush
out
more,
but
I
I
think
what
I'd
like
to
do
in
these
couple
of
minutes
is
think
about
what
our
next
steps
are.
A
A
You
know
what
the
work
is:
that's
currently
underway
for
security,
metrics
and
then
also
look
at
the
the
work
that
dan
and
kim
have
been
doing
for
the
security
badge
and
see
if
there
isn't
some
way
that
these
things
all
fit
together
like,
for
example,
the
the
security
scorecard
work
could
be
used
to
populate
data
that
sits
inside
of
the
best
practices
in
the
security,
best
practices,
infrastructure
and
maybe
likewise,
maybe
somehow,
security
best
practices
becomes
the
storage
layer
and
then
metrics
becomes
a
a
user
experience
on
top
of
that
and
the
security
scorecard
work
you
know
feeds
into
the
storage
that
underlies
that.
A
E
Can
you
hear
me
yes
yeah,
I
mean
dan
and
I
have
had
a
brief
conversation
about
this.
You
know
the
very
short
answer
is
you
know
I've
made
sure
that
he
can
yank
the
ci
best
practices
data
into
his
the
best
practices
badge
does
have
some
automated
mechanisms
and
we've
actually
been
briefly
talking
about.
You
know:
hey,
can
we
you
know,
can
you
can
there
be
additional
merging
and
so
on?
Certainly,
I
could
see
in
the
long
term
the
ci
best
practices
badge
has
has
some
automation.
E
Maybe
that
could
be
moved
into
the
scorecard
system
and
then
basically
the
badge
calls
on
the
scorecard
to
get
some
of
the
auto
some
automated
data
and
then
points
off
to
another
place.
To
get
more
details.
I
think
there's
opportunities
for
lots
of
ways
for
these
things
to
work
together
with
with
different
pros
and
cons,
and
that's
fine.
A
So
what
should
our
next
steps
be
like,
but
before
our
our
next
meeting,
which
is
about
two
weeks
from
now
david,
do
you
you
know,
I
feel
like
it's
kind
of
between
david
and
maybe
either
michael
orion
and
dan,
to
maybe
put
together
some
options
that
that
we
can
do.
Let's.
B
E
E
Metrics,
I
don't
know
if
you've
got
another
tool
in
mind
and
then
we
could
try
to
list
you
know,
given
that
better
understanding,
you
know
what
could
be
a
longer
term
strategy,
because
the
the
badge
captures
things
that
can't
be
easily
automated
and
I
think
that's
an
advantage
of
it,
but
there's
things
where
clearly
there's
serious
advantages
to
measuring
things
that
it
doesn't
support,
and
so
I
think
that
you
know
combining
those
just
makes
sense.
G
It
out,
I
agree,
I
think,
if
we
can
get
together
and
just
put
together
even
like
a
really
high
level,
architectural
style
diagram
to
just
show
how
these
different
pieces
interact
and
what
influences
what
that
would
really
clear
up
a
lot
of
the
confusion
about
where
these
things
should
sit,
whether
it's
this
group
or
the
other
group
and
who
does
what
and
that
type
of
thing
in
my
mind,
it's
very
clear
but
clearly
there's
some
there's
still
some
confusion
about
it.
So
maybe
a
picture
will
help.