►
From YouTube: OpenSSF Town Hall #3 (May 3, 2021)
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
All
right,
good
morning,
everyone
or
good
evening,
it's
great
to
see
you
all
and
we'll
go
ahead
and
get
started,
we're
a
couple
of
minutes
after
the
hour.
A
So
let
me
start
off
with
a
couple
of
housekeeping
things
and
first
as
anti-trust
policy
notice,
we
do
have
people
in
this
meeting
from
a
number
of
different
organizations
who
are
competitors
in
the
business
sense,
and
because
of
that,
we
need
to
make
sure
that
our
everything
in
this
meeting
follows
with
antitrust
and
competition
laws.
A
The
second
thing
is
that
we
do
have
a
code
of
conduct
for
these
meetings,
and
so
we
will
be.
You
know
we
want
to
guarantee
a
harassment,
free
experience
for
everyone,
and
so
we
ask
everyone
to
behave
in
accordance
with
professional
standards.
We
have
not
had
any
problems
with
this
before
and
I
doubt
we
will
again,
but
it's
just
a
reminder
for
us
and
the
last
bit
of
housekeeping.
A
A
Again,
I'm
super
excited
to
be
here
today
with
you
all
and
to
share
what's
some
of
the
things
that
are
going
on
in
the
open
source
security
foundation,
our
agenda
for
today
we
have
a
section
where
we'll
be
talking
about
some
of
the
recent
supply
chain,
attacks
that
are
in
the
news
and
for
those
who
will
be
talking
about
things
that
you
can
do
to
help
secure
yourself
against
those
and
also
things
that
open
ssf
is
doing
and
can
be
doing
in
the
future.
A
We'll
also
talk
about
an
upcoming
exec
white
house
executive
order
on
supply
chain
security,
software
supply
chain
security,
so
that'll
be
a
fun
good
chunk
of
our
time
again,
where
you
know
it's
interactive,
so
we'll
be
looking
for
and
hoping
for
questions
then
we'll
have
an
update
on
things
that
are
happening
in
the
open
ssf,
followed
by
two
announcements
from
michael
scaveta,
who
leads
our
identifying
security
threats
group.
A
So
they've
they've
been
hard
at
work
in
that
group
and
they'll
share
with
you,
some
of
the
results
from
that,
and
then
we've
got
one
slide
at
the
end,
where
we
just
talk
about.
What's
next
kind
of
what
we
see
on
the
horizon
for
open,
ssf
and
we'll
leave
open
for
q
a
before
we
jump
into
our
agenda
and
talking
about
in
the
news
I
wanted
to
update
you
on
some
of
the
momentum
in
the
open
source
open
ssf.
A
A
Our
website
visits
are
up
from
57
to
74
so
and
that's
just
monthly
visits,
and
then
our
edx
course,
which
we
announced
early
on
that
was
one
of
our
first
announcements
in
our
first
town
hall
meeting
has
grown
now
and
we
have
over
30
close
to
3
200
participants
in
that.
So
it's
exciting
to
see
the
interest
in
open,
ssf,
and
you
know
the
engagement
and
and
see
the
pickup
of
some
of
the
initiatives
that
we've
sponsored.
A
So
so
with
that
I
will
turn
over
and
we'll
let
david
wheeler
talk
us
through.
What's
in
the
news,
go
david
you're
on
mute
david.
B
Muting
myself,
when
I'm
not
speaking
and
so
okay,
you
I'll
just
tell
you
next
when
we
go
to
the
next
slide,
so
I'm
going
to
talk
about
just
four
different
kind
of
events
that
have
been
happening
in
the
news
and
some
connections
possible
connections,
at
least
to
open
ssf.
So
the
first
thing
I
wanted
to
talk
about
today
about
things
that
have
been
happening
in
the
news
is
the
so
called
dependency.
Confusion
attacks.
B
First
of
all,
it's
a
kind
of
attack,
dependency,
confusion
and
it
exploits
the
ambiguity
about
the
source
of
some
dependency.
So
you
know,
most
programs
today
bring
in
a
large
number
of
libraries
from
other
places.
It's
a
much
more
efficient
way
to
work.
But
let's
say
that
you
depend
on
some
library
x
and
you
say:
hey
bring
it
in,
but
you
actually
have
identified
multiple
sources
for
packages
that
you
bring
things
in
well,
what
happens
if
some
package
x
that
you
are
trying
to
bring
in
could
come
from
multiple
places?
B
Well,
somebody
investigated
this,
namely
alex
per
sand,
and
he
found
out
that,
in
fact,
an
attack
shows
up
and
here's
how
the
attack
works.
The
victim
uses
some
pr
private
package
registry
contains
a
number
of
packages,
including
something
I'll
call
internal
thing.
C
B
The
victim
says:
hey
I'm
going
to
add
my
list
of
dependencies
internal
thing
and
the
victim
is
assuming
that,
since
internal
thing
only
exists
in
their
private
repository
registry,
it'll
obviously
only
come
from
there,
but
an
attacker
can
either
learn
or
guess
that
they
there's
an
internal
repository
with
that
name.
B
Maybe
they
learn,
maybe
they
guess,
and
so
they
create
something
with
that
name
in
a
public
registry.
The
public
registry
has
no
reason
to
deny
it.
It
doesn't
have
anything
of
that
name.
Somebody
wants
that
name,
awesome
and
the
attacker
will
often
create
it
with
a
really
big
version
number
you
know
100.0.0.
B
So
obviously
it's
the
very,
very
latest
version
of
this.
The
next
time
the
victim
uses
their
build
infrastructure,
they're
going
to
select
and
download
internal
thing
from
the
public
registry
instead
and
chaos
ensues.
So
what
can
you
do
about
this?
Well,
there
are
a
couple
quick,
immediate
solutions
you
can
do.
One
is,
of
course,
when
you
have
identified
a
set
of
libraries
you're,
bringing
in
make
sure
you
reference
only
one
feed
only
one
repository.
B
If
there's
no
only
one,
then
there's
no
possibility
of
ambiguity.
We're
done
for
a
number
of
organizations
that
doesn't
really
work,
so
another
thing
you
can
do
is
make
sure
you
control
your
scopes.
When
you
say
I
want
package
internal
thing,
you
make
sure
that
for
every
package
you
clearly
identify
where
it's
coming
from.
So
yes,
there's
more
than
one
source,
but
there's
no
ambiguity
for
any
particular
package.
B
B
We
can
educate
people,
but
in
the
longer
term,
one
thing
that
the
open
ssf
could
do
is
work
with
package
managers
to
ensure
that
they're
secure
by
default,
because
I
think
in
many
many
cases,
and
certainly
in
this
one
instead
of
trying
to
depend
on
humans
to
always
every
turn
always
do
the
right,
not
not
easy
thing.
It's
far
better
to
make
the
default
the
easy
thing,
the
obvious
thing,
the
secure
thing
so
that
in
general
we
want
software,
including
package
managers
to
be
secure
by
default.
Next.
B
By
the
way,
okay,
codecov
so
codecov
is
a
very,
very
widely
used
service
to
measure
test
coverage.
Obviously
it's
important
to
test
software.
B
If
you
care
whether
or
not
it
works
correctly-
and
there
are
various
ways
to
measure
to
see
to
at
least
get
some
sense
of
how
good
your
test
coverage
is
so,
for
example,
if
you
write
tests
and
you
find
out
that
only
five
percent
of
the
lines
of
code
in
your
in
your
library
that
you
wrote
are
actually
being
tested,
that's
a
pretty
good
indication
that
your
tests
are
terrible.
You
know
that
means.
B
95
of
your
code
is
no
test
at
all,
and
you
know
there
are
various
kinds
of
ways
to
measure
test
coverage.
But
the
important
thing
is
that
code
cover
codecov
provides
a
widely
used
service
to
make
it
easy
to
do
one
of
the
mechanisms
they
they
provide
for
uploading
data
to
them,
which
they
need
is
something
called
the
codecov
bash
update,
uploader,
and
it
makes
it
very
very
easy
to
to
provide
updates,
but
they
the
way
they
make
it
super
easy,
is
using
a
dangerous
construct,
and
this
is
actually
straight
up.
The
construct.
B
This
kind
of
code
is
a
construct
called
a
pipe
to
shell
you're
grabbing
some
data
from
the
internet
and
sending
it
directly
to
a
shelf
for
execution.
This
is
widely
considered,
dangerous
and
insecure
practice.
There's
no
there's
little
really,
there's
no
opportunity
to
check
anything.
You
just
are
downloading
it
and,
as
you
download
it,
you
immediately
detect
immediately
execute
it.
In
fact,
there
are
known
attacks
where
this
is
detectable,
where
it
detects
that
you're
doing
this
and
it
will
and
the
attacker
could
give
you
malicious
code
instead
of
the
normal
code.
B
This
is
not
generally
considered
a
good
practice.
In
january
2021
an
attacker
subverted
the
distributed
version
of
bash
uploader.
Now
what
was
in
the
code
cov
file
that
was
being
downloaded
was
not
the
same
as
the
github
code,
but
it
didn't
matter
because
no
one
was
seeing
it.
They
were
piping
the
code
directly
to
the
shell,
so
there's
no
opportunity
to
notice
that
what
was
being
downloaded
was
not
the
correct
version.
It
was
being.
It
was
in
fact
a
subverted
version.
B
That's
not
good!
So
what
can
you
do?
Well,
obviously,
one
thing
you
can
do
is
apply
best
practices,
so
in
general,
trying
to
avoid
pipe
to
shell
is
a
good
thing
and
in
fact
the
open,
ssf's
secure
software
development
fund
models
course,
which
we
mentioned
earlier.
Okay,
specifically
in
the
section
on
downloading
and
installing
reusable
software
specifically
discusses.
Why
pipe
to
shell
is
dangerous
and
why,
in
most
cases
it
should
be
avoided.
B
B
In
the
longer
term,
openssf
could
even
help
some
projects
replace
pipe
to
shell,
where
they
are
recommending
it
with
more
secure
approaches
as
their
default
and
so
that
again,
so
that
people
are
less
likely
to
use
more
dangerous
practices
and
more
likely
use
practices
that
are
easier
to
defend
by
the
way.
I
can't
mention
this
earlier,
but
I'd
love
to
re-remind
questions
comments.
Suggestions
are
absolutely
welcome,
as
we
go
along
happy
to
try
to
answer
some
things.
B
B
B
So
the
university
of
minnesota
wanted
to
do
some
experiments,
fine,
great
research,
good
and
what
they
want
to
do
is
they
want
to
try
to
measure
their
linux
kernel
processes,
resistance
to
intentionally
vulnerable
proposals,
now
nothing
wrong
with
that
goal,
but
in
practice
what
they
did
is
they
experimented
and
by
sending
commits
to
well
sending
proposals
to
humans
and
basically,
what
this
meant
was
they're
performing
experiments
on
humans
without
their
prior
consent.
B
This
is
a
huge
no-no.
Anybody
who
is
curious,
you
can
do
you.
Can
you
can
do
some
searches
and
find
out
why,
since
the
1950s,
it's
been
incredibly
important
in
the
research
community
to
check
on
and
get
consent
even
for
simple
things
like
surveys.
Typically,
you
have
to
go
through
things
like
institutional
review
board
and
other
processes.
Unfortunately,
experimenters
didn't
do
any
of
that.
They
just
experimented
on
people.
B
So
the
linux
kernel
has
basically
soft
banned
the
entire
university
of
minnesota
they're
in
process
of
reviewing
all
commits
from
the
university
of
minnesota,
because,
frankly,
as
soon
as
this
was
rev
as
soon
as
this
became
clear,
the
lynx
curl
is
too
important
to
have
any
questions
about
whether
or
not
there's
malicious
code
in
there,
and
when
this
was
revealed,
it
was
not
clear
exactly
what
was
done
that
was
potentially
intentionally
vulnerable,
so
the
lynx,
colonel
the
lynx
foundation,
did
send
a
letter
to
the
university
of
minnesota.
B
You
can
actually
zd
nets
actually
published
it
now.
So,
if
you're
curious,
you
can
see
what
the
requirements
were.
We
weren't
working
looking
for
anybody's
heads.
We
just
wanted
to
heal
the
problems.
B
You
know
make
make
sure
that
what
was
known
that
we
want
to
make
sure
we
knew
exactly
what
had
been
submitted
that
was
potentially
vulnerable
and
known
to
be
vulnerable
and
other
things
like
you
know.
How
can
we
make
sure
that
this
kind
of
that
doesn't
happen
again?
The
paper
was
formally
withdrawn.
We
we
actually
are
okay
with
things
being
on
the
internet,
but
we
just
don't
think
that
things
that
do
unethical
research
should
get
formal
credit,
but
we
didn't
want
it.
B
You
know
at
this
point:
it's
done
it's
done.
So
that's
all!
That's
the
one
other
thing
that
we
asked
for
the
university
of
maryland
minnesota
actually
has
recently
posted
a
list
of
proposals
that
were
intentionally
vulnerable.
People
are
kind
of
going
through
and
just
making
sure,
because
again
we
want
to
make
absolutely
certain
to
the
best
that
anybody
can
that
no
vulnerabilities
were
inserted
into
the
linux
kernel
or
any
other,
because
at
the
time
we
didn't
even
we
weren't
even
sure
if
just
the
linux
kernel
had
been
targeted.
B
I
will
note
that
there
is
humor
here
one
of
their
proposals
that
was
supposed
to
have
a
hidden
vulnerability
in
fact
was
had
a
bug
and
as
because
it
was
a
had
a
bug,
it
was
actually
correct
and
so
that
one
that
one
was
accepted,
and
it
appears
that
all
because
it
actually
did
fix
something
and
everything
else
appears
to
have
been
rejected
that
were
intentionally
vulnerable,
there's
still
some
double
checks
and
triple
checks,
but
so
what
can
you
do?
Well?
First
of
all,
research
is
great.
B
The
villains
foundation
itself,
funds,
research
research
is
great
stuff.
We
love
research,
please
be
ethical
in
general
humans,
experiments
on
humans
need
consent.
If
you
have
questions,
you
can
go
back
and
look
the
history
of
this
and
why,
since
throughout
the
1950s,
this
has
been.
I
don't
know
the
exact
dates
on
some
of
the
stuff,
but
you
know
it's
very,
very
important
that
experiments
on
humans
need
consent
and
be
ethical.
All
right
use
tools
to
look
for
likely
vulnerabilities.
B
The
open,
ssf's,
tooling
working
group
is,
of
course,
working
to
make
this
easier
to
do.
Consider
tools
to
look
for
suspicious
patterns
having
code
conventions
that
reduce
the
likelihood
of
certain
kinds
of
attacks
is
a
good
thing
and,
of
course,
in
general,
having
eyeballs
on
code
bases
reviews
extremely
important.
B
Okay,
let's
see
here
chaos,
and
I
I
think
actually
that
will
be
best
asked
answered
later.
So
I
see
a
question
in
the
chat.
Let's,
let's
talk
about
that
a
little
bit
later,
where
I
think
it'll
be
in
better
context.
B
Next,
okay
white
house
executive
orders
I've.
There
are
actually
again
whether
this
involves
the
u.s
federal
government.
There
are
actually
two
executive
orders
that
are
that
are
probably
especially
related
to
this
group.
One
is
already
out.
If
it
came
out
in
february,
it's
called
executive
order
on
america's
supply
chains
in
general.
They
it
wants
reviews
at
various
dates
and
times
of
various
supply
chains,
to
try
to
identify
supply
chain
vulnerabilities
and
in
general,
to
increase
resiliency
of
the
country.
B
It's
not
limited
to
software
at
all.
It's
just
in
general
supply
chains
of
certain
especially
important
areas,
which
you
probably
could
guess
without
even
reading
the
letter,
but
the
letter,
the
executive
order
is
public.
Next
is
it's
known
that
there
is
a
cyber
executive
order
to
come?
B
It's
not
public,
so
its
contents
to
me
at
least
are
unclear.
However,
several
folks
are
apparently
involved
and
somehow
there
have
been
some
leaks
about
what
it
appears
what
these
drafts
have
contained.
So,
while
we
don't
know
what's
going
to
contain,
we
do
have
some
indication
of
what
it's
likely
to
contain
and
so
in
a
spirit
of
trying
to
get
prepared.
Let
me
just
quickly
summarize
first
of
all,
it's
quite
clear
that
this
order
was
spurred
in
part
by
a
really
devastating
subversion
by
solar
winds.
B
Orion,
the
the
us
government
is
extremely
spun
up
from
this.
I
saw
one
report
that
estimates
the
damage
caused
by
the
solar
solar
winds.
Orion
was
about
a
hundred
billion
dollars.
That
doesn't
mean
that
suddenly
a
hundred
billion
dollars
was
lifted
from
some
account.
It
means
that
the
cost
to
repair
the
various
damage
and
to
undo
and
redo
things
and
basically
eliminate
replace
whole
programs
will
may
end
up
at
that
number.
B
I
I
don't
know
how
much
to
believe
that
number,
but
it's
quite
clear
that
it
was
substantial.
Unsurprisingly,
when
something
is
subsubstantial,
you
know
to
an
organization:
the
organization
wants
to
do
something
about
it.
B
I
think
it's
so
the
likely
contents
are
that
first
of
all,
companies
that
do
business
with
the
federal
government
will
have
to
meet
some
security
standards
and
swiftly
report
incidents,
they're,
probably
going
to
strengthen
dhs
and,
in
particular
cyber
security
infrastructure
security
agency.
This
is
a
give
it.
You
know,
more
power,
more
authorities,
more
money,
federal
agencies
are
being
asked
to
move,
to
multi-factor
authentication
and
to
improve
fedramp,
which
is
the
us
government's
way
to
examine
cloud
services.
They
want
to
move
more
to
what's
called
zero
trust.
B
I
realize
that's
a
weird
name,
but
the
idea
here
is:
don't
you
just
because
you're
on
a
network
it
doesn't
matter.
We
can't
trust
any
everything.
That's
going
on
a
network
by
the
way
open,
ssf
courses
can
help
them
do
that,
because
okay,
cisa
and
nist
are
trying
to
identify
critical
software
for
more
action.
B
That's
very
connected
to
what
the
openness
critical
projects
working
group
is
doing,
and
I'm
hoping
that
you
know
the
I
okay,
let's
see
here,
require
software
build
the
materials
in
some
cases
you
know
so
just
having
software
build
materials
doesn't
automatically
fix
problems,
but
it
means
that
there's
visibility
to
know
about
things
like,
for
example.
Oh
you
use
that
component.
What
version
do
you
use
that
has
a
known
vulnerability?
B
Is
it
vulnerable
in
this
context,
or
not
it's
hard
to
answer
that
question
without
knowing?
What's
in
the
software,
a
a
larger
software
or
a
software
component
or
a
larger
system
with
software
in
it,
there
may
be
some
sort
of
software
security
grades?
We
don't
know
very
much
about
this,
but
I
will
note
that
the
open,
ssf's,
best
practices,
working
group
and
security
threats
working
group
has
been
working
on
those
sorts
of
things.
B
What
package
managers
can
do
to
help
prevent
attacks
like
dependency
confusion?
I
think
the
quick
answer
here,
as
I
mentioned
earlier,
actually
kay,
can
you
jump
back
up?
Real
fast
is
basically
security.
By
default,
there's
different
ways:
a
package
manager
can
implement
how,
when
it
sees
a
request
for
a
package,
how
to
make
sure
that
the
dependency
confusion
can't
happen,
but
it
basically
you
can
make
sure
that
when
you
load
in
a
package
it
comes
from
one
and
only
one
source.
B
There
are
many
specific
ways
you
can
accomplish
that
and
which
one
depends
on
various
factors,
but
the
the
final
goal
really
is.
When
you
say
I
want
internal
package,
you
shouldn't
have
to
say:
I
want
a
two-year-old
package
and
hope
that
the
package
manager,
guess
is
the
right
source.
You
want
to
make
sure
the
package
manager
knows
how
to
get
the
correct
source
and
ignores
the
wrong
one,
and
with
that
I'm
done.
Thank
you
very
much.
A
Okay,
great
so,
let's
move
on
then
and
we'll
have
ryan
talk
with
us
about
some
of
the
great
work
that's
happening
in
our
working
groups
in
openssf,
ryan.
D
Thank
all
you
next
cyphers,
so
real,
quick
to
kind
of
kick
off.
Give
you
a
little
bit
of
background
for
those
that
are
already
familiar.
The
openssf
has
a
group
called
the
technical
advisory
council,
and
one
of
the
things
that
we're
responsible
for
is
oversight
for
all
the
various
technical
initiatives
working
groups
that
open
ssf
is
working
on
and
what
we
do
is
we
try
to
facilitate
the
collaboration
between
those
groups?
You
know
make
sure
that
we
don't
have
overlap
and
that
each
one's
working
together
sort
of
towards
this
common
goal.
D
Next,
please
so
some
of
the
things
that
the
the
tac
the
advisor
council
has
been
working
on.
In
the
past
few
months,
we
actually
went
through
and
we
reviewed
all
of
the
different
working
groups
that
we
have
and
their
charters
and
we
approved
them.
So
you
know
we
went
through
kind
of
a
formal
review
process.
We've
worked
on
our
code
of
contact
and
then
what
we've
got
coming
up
is:
we've
sort
of
been
building
a
lot
of
this
work
towards
a
road
map.
D
So
in
the
last
town
hall
we
talked
about
a
mission
statement
and
a
vision
and
now
we're
working
towards
this
roadmap
of
technical
initiatives,
and
this
roadmap
is
kind
of
like
a
giant
backlog
of
all
the
things
that
we
could
be
working
on.
You
know,
as
we
all
know,
there's
a
ton
of
work
that
we
could
do
for
security,
particularly
for
open
source,
and
so
we
have
a
large
collection,
a
big
document
that
we're
working
through
to
create
that
roadmap.
D
We're
also
starting
to
focus
on
improving
and
incorporating,
I
should
say
the
diversity
civility
and
inclusion,
and
so
we're
going
to
start
taking
that
on
and
formalizing
that
process
and
then
we've
also
been
working
on
the
budget.
So
the
governing
board,
like
I
mentioned,
is
driving
budget
and
then
we're
coordinating
with
them
and
the
working
groups
to
get
all
that
figured
out
next,
please
all
right.
So
what
have
the
working
groups
actually
been
doing?
So
we
have
six
different
working
groups
currently
the
best
practices
group.
D
They
include
things
like
the
cii
best
practices,
badge
that
existed
before
open
ssf
and
we
brought
that
in
and
what
they've
been
doing.
Is
they
have
some
new
tooling
that
they
released
to
work
on
some
automation
they
worked.
They
began
work
on
a
swahili
translation
which
I'm
personally
excited
about,
and
then
we
added
a
new
quote
project
is
maintained
criteria.
D
D
For
some
folks,
we
enable
others
to
build
on
top
of
the
educational
materials
and
then
we're
also
working
with
the
cra
team
to
integrate
some
of
those
requirements
and
then
the
other
another
group
is
the
security
tooling
working
group
and
they've
created
a
draft
of
a
guide
to
security
tools,
so
they're
kind
of
iterating
through
all
the
different
tools
that
exist
out
there
and
putting
them
into
a
document
that
makes
it
easier
for
people
to
find
tooling
that's
available
to
them.
Next,
please.
D
We
also
have
the
vulnerability
disclosure
working
group
and
they're
working
on
a
couple,
different
documents,
so
they're
identifying
all
these
different
pain
points
around
vulnerability,
disclosure
and
they'll
leverage
that
to
incorporate
towards
different
requirements
and
ways
to
improve
that
they've
got
white
paper
that
they're
working
on
that
for
oss
projects,
around
vulnerability,
disclosure
as
well,
and
then
they're,
creating
guidance
for
what
the
vulnerability
management
process
should
be
for
oss
projects.
D
As
we
know,
that's
a
huge
challenge
on
how
to
disclose
and
disclose
responsibly
and
identify
these
things
so
they're
working
on
that
and
then
the
next
group
is
the
identifying
security
threats
working
group
and
so
they've
got
some
really
cool
things
coming
up.
Then
I'm
not
going
to
spoil
it
from
mike
scaveda
he's
got
the
big
demo,
but
some
of
the
things
are
security,
reviews,
project
and
you'll,
find
out
more
in
a
moment
and
also
the
metrics
dashboard
of
which
mike
is
going
to
give
a
really
really
cool
demo.
D
For
that
next
slide,
please
and
then,
additionally,
one
of
the
things
that
the
tac
has
kind
of
been
working
on
that
affects
all
of
openssf.
Is
we've
been
working
with
osf?
Who
has
a
proposal
for
managing
audits
for
open
source
projects,
so
they've
created
a
from
a
vast
number
of
resources
and
different
materials,
a
list
of
critical
open
source
projects
that
would
be
useful
to
have
security
audits
performed
on.
D
So
they
have
a
process
and
a
lot
of
experience
doing
this
type
of
work,
so
we're
coordinating
with
them
to
get
a
lot
of
that
work
done
and
then
that
the
results
of
that
initiative
will
actually
be
published
within
the
open,
ssf
security
reviews,
repo,
which
michael
is
going
to
talk
about
here
in
a
minute.
D
So
we've
discussed
it
at
the
tac
and
everybody's
excited
about
it.
So
we'll
be
working
on
that
in
the
coming
months
as
well
all
right
and
with
that
I
will
hand
it
over
to
mike
scavena.
E
Awesome
thanks
so
much
next
slide,
so
hi
everybody
I'm
mike
scaveda.
I
lead
the
identifying
security
threats
working
group.
I
have
two
nice
announcements
here.
First
is
the
the
open
source
security
metrics
project
that
we've
been
that
we've
been
working
on,
so
this
is
available
today
at
metrics.openssf.org.
E
E
So
what
does
this
really
mean?
So
this?
This
is
a
relatively
new
project.
I
think
we
we
officially,
you
know
it
became
live
about
a
week
and
a
half
ago,
or
so
so.
At
this
point
we
are
collecting
existing
metrics,
which
means.
Fortunately,
we
have
nice
data
sources
right
now,
they're
all
from
the
from
the
open
ssf.
So
the
scorecard
project
which
collects
information
on.
E
It
analyzes
open
source
projects
looking
for
things
that
that
can
be
where
the
the
posture
can
be
pulled
out
automatically
criticality
score,
which
is
a
measure
of
the
importance
or
influence
of
a
project.
E
E
So
if
you
next
slide
so
what's
the
purpose
of
this,
we
really
see
kind
of
three
main
consumers
of
this,
and
first
is
an
author
themselves.
So
you
know
I
I
run
an
open
source
project
and
I
would
like
to
know
like
how
good
am
I
doing
relative
to
my
peers.
Are
there
simple
things
that
I
can
do
to
improve
my
security
posture
for
consumers?
You
know
if
I
have
a
choice
of
you,
know:
12
different.
E
If
security
were
one
of
my,
you
know,
top
order
bits
there
and
then
in
aggregate
for
organizations
that
have
you
know
security
or
risk,
or
maybe
even
compliance
teams
to
say
we're
using
a
whole
lot
of
open
source,
because
they're,
probably
using
a
whole
lot
of
open
source
which
of
those
projects
are
the
are
the
riskiest
which
of
those
need
investment,
perhaps
for
open
ssf
and
the
the
you
know.
Part
of
the
critical
projects
working
group
is,
is
focusing
investment
and
attention
on
the
most
critical
projects
out
there.
E
It's
a
little
self-referential
because
we're
using
their
data
to
populate
the
dashboard.
But
you
know
having
a
having
a
a
view
per
component
and
then
ultimately,
some
sort
of
aggregate
view.
We
think,
will
help
everybody
make
better
decisions
and
and
have
things
be
less
less
objective,
but
we're
also
looking
for
further
ideas.
So
if
you
have
a
need
for,
if
you
think
that
this
kind
of
data
would
be,
would
be
useful
to
you,
please
let
us
know
we.
E
We
think
this
is
the
kind
of
thing
that
you
know
as
we
build
it
and
as
it
gets
more
populated
and
better
more
uses
will
will
become
apparent
next
slide.
So
current
state
is.
We
have
about
105
000,
open
source
projects
that
have
at
least
some
data
for
one
of
the
data
sources.
Most
of
that
data
comes
from
the
criticality
project.
They
have
about
over
a
hundred
thousand
that
they've
that
they've
collected
themselves.
E
It
has
really
basic
features
like
searching,
and
things
like
that
we
use
graphana
as
a
as
the
dashboard
front
end,
so
that
helps
us
be
very
agile
and
we
can
change
things
and
improve
them
very
quickly
and
we
have
a
very
simple
json
api
which,
if
you
hit
the
next
slide.
Oh
sorry,
maybe
maybe.
E
Have
a
really
simple,
json
api
that
that's
helpful,
so
next
slide,
but
this
is
really
important.
This
is
super
alpha
quality.
It
can
change,
it
can
go
down.
Please
don't
integrate
this
into
anything,
really
important,
where
you'll
be
woken
up
in
the
middle
of
the
night.
If
our
you
know
service
goes
down,
it
will
absolutely
improve
over
time.
E
E
E
You'll
see
a
list
of
projects
that
we've
that
we
have
data
for
that
have
kubernetes
in
the
name.
There
are
two
links.
The
link
on
the
left
takes
you
to
the
dashboard.
The
link
on
the
right
takes
you
to
the
api
view.
This
is
a
super
clujie,
not
great
ux,
but
that's
what
it
is
right
now,
so
you
can
copy
the
url
that
you
saw
on
the
right
and
figure
out
how
to
call
the
json
api
yourself.
E
But
if
you
click
on
the
on
the
link
on
the
left
next
slide,
this
is
the
the
the
grafana
front
end.
So
what
you
have
is
you
know
on
the
the
top
left
or
kind
of
that
that
top
section
is.
You
know,
project
details
that
we've
been
able
to
pull
out
of
the
various
data
sources,
urls
that
we
that
we
found
in
in
descriptions
and
another
kind
of
metadata
security
reviews.
E
If
we
have
any
we'll
be
there
and
then
the
three
rows
below
come
from
well,
one
comes
from,
as
the
text
suggests.
One
comes
from
the
criticality
score,
one
from
the
best
practice
is
badge
program
and
one
from
the
scorecard,
and
these
are
color
code
coded
to
make
sense.
So
you
know
in
most
of
these
a
check
mark
is.
E
This
is
a
good
thing,
so
for
the
kubernetes
project,
if
you
see,
if
you
see
the
blue
square,
that
says
2017
and
you
look
to
the
right
of
that,
that
says,
static
analysis
used.
That
is
saying.
E
The
best
practices
badge
program
asserts
that
the
kubernetes
project
uses
static
analysis,
that's
great,
but
it
also
says
that
now
one
one
more
to
the
right
that
the
kubernetes
does
not
use
dynamic
analysis
now,
as
I'm
speaking
now,
I'm
curious
as
to
how
it
could
be
not
used,
but
the
one
below
is
fixed,
so
perhaps
there's
some
data
issues.
The
point,
though,
is
that
you
should
be
able
to
trust
that
a
check
mark
means
it.
E
It
meets
the
thing
and
that
x
means
it
doesn't
meet
the
thing
where
we
don't
have
data
or
the
data
is
incomplete.
You
get
some
sort
of
a
a
question
mark
or
something
that
says
like
no
info,
so
we
want
to
improve
this
over
time.
We
don't
know
if
these
are
the
right.
You
know
all
exactly
the
right.
Metrics
we've
chosen
a
subset
of
the
things
that
are
collected,
there's,
certainly
more
that
we
want
to
do.
E
We
want
to
collect
things
directly
from
github.
Actually,
if
you
go
to
the
next
slide,
we
have
some
kind
of
next
steps
on
this
so
yeah.
So
we
want
obviously
to
improve
the
ux.
We
want
the
additional
metrics
to
tell
the
right
story.
There
was
a
there
was
a
comment
on.
You
know:
hey
what
about
the
the
chaos
working
group,
the
chaos
risk
working
group,
I've
already
kind
of
started
to
reach
out
there,
I'm
planning
on
attending
the
next
meeting
david.
I
think
you
you're
a
regular
attendee
there.
E
So
right
now
consider
this
kind
of
a
an
experiment
that
went
live
so
where
the
we're
the
future
holds,
and
you
know
this
project
versus
chaos.
I
don't
think
anybody
benefits
by
having
multiple
groups
doing
exactly
the
same
work,
but
I
think
there's
also
room
for
experimentation
all
around.
So
we
are
aware
of
the
the
chaos
working
group
and
we're
gonna
have
a
better
story.
Hopefully
in
three
months.
E
The
final
point
there
is,
you
know
perhaps-
and
this
is
this
is
contentious-
you
know,
where
does
the
where's
the
line
between
objective
data?
So
you
know
everything
you
saw
in
the
previous
slides
are
pretty
objective
like
it's.
You
know
you
could
argue
with
the
algorithm
that
comes
up
with
the
score,
but
the
score
is
the
score.
What
what
none
of
those
are
doing
is
saying
this
thing
is
good.
E
This
thing
gets
an
a
plus
because
we
because
we
say
so,
but
I
think,
there's
there's
a
good
argument
for
why
consumers,
especially
you
know,
risk
teams
and
compliance
teams
and
and
true
consumers
that
they
would
really
benefit.
I
think,
by
having
the
ssl
lab
style,
you've
got
an
a
plus.
These
are
all
a
plus
projects,
and
these
are
b
plus
projects
and
they
can
get
to
an
a
minus
by
doing
x,
y
and
z.
E
We
don't
have
specific
plans
on
implementing
this
today,
but
it's
the
the
type
of
conversations
that
we
want
to
have
in
the
near
term
about
where
how
to
make
this
more
useful
and
more
valuable,
because
I
think
sometimes
you
can
have
a
dashboard
that
has
a
huge
amount
of
data
that
ordinary
humans
just
don't
know
how
to
profit.
Like
is
this
a
good
thing
am,
I
am.
I
am.
I
supposed
to
be
happy
when
I'm
looking
at
this
or
sad
so
we
want
to
we
want
to
want
to.
E
Our
next
working
group
is
on,
may
10th.
Please
join.
There
are
instructions
at
the
link
below
or
if
you're
interested
in
this
project,
in
particular
the
link
to
the
project.
Is
there
you
there
are
instructions
on
how
to
set
up
a
local
development
environment
and
suck
in
all
the
data
and
and
play
yourself
and
really
looking
forward
to
where
this
is
going
in
the
future,
and
I
just
wanted
you
know
huge
thanks
to
the
folks
on.
E
You
know
that
that
put
together
the
scorecard
and
the
criticality
and
and
david
on
the
on
the
badge
program.
You
know
this
project
would
be
nothing
without
those
data
sources
already
being
collected
and
and
provided.
E
E
Do
I
should
I
worry
about
this,
and,
and
you
can
do
all
the
work
yourself
and
you
can
run
tools,
you
can
look
for
you
know
public,
cves
and,
and
things
like
that,
but
and
let's
suppose
you
do
that
and
you
spend
you
know
x,
number
of
days
doing
this
work
and
then
you
decide.
You
know
what
this
thing's
actually
pretty
good.
E
Well,
today,
lots
of
organizations
do
things
like
that,
but
they
keep
this
information
to
themselves,
they'll,
throw
in
an
excel
spreadsheet
or
a
database,
or
something
and
maybe
other
users
within
their
organization
or
maybe
just
their
team
would
be
aware
of
it
and
that's
great
for
them,
except
that
now
someone
else
wants
to
take
the
same
component
and
put
it
in
in
a
you
know,
self-driving
car
and
does
the
same
work.
E
So
at
best
this
we're
super
inefficient
here
and
you
know:
wouldn't
it
be
better
if
those
results
were
shared
among
these
organizations
so
that
the
second
organizations,
if
they
don't
trust
it,
they
could
do
it
again
or
maybe
you
know
they
can
make
whatever
decision
they
want,
but
at
least
it'll
be
more
information
that
they'll
have
saying
that
you
know
you
know
bob
reviewed
a
component
and
thought
it
was
okay
or
thought
yeah,
it's
okay,
but
not
in
this
way,
or
you
know,
make
sure
like
this
component
is
not
meant
for
direct
internet
connectivity.
E
So
make
sure
you
you
put
it
it's
a
back
end
component
or
something
so
I
I
think
that
these
things
are
it's
something
that
would
be
useful
to
to
readers
also,
today,
a
lot
of
these
things
exist
in
the
form
of
third-party
audit
reports,
so
you'll
have
ncc
group
or
trail
of
bits
or
or
austif
they'll.
E
Do
a
security
review
and
they'll
have
the
results
available
on
their
website,
and
these
things
are
pretty
distributed,
because
if
you
don't
know
that
trail
of
bits
did
an
assessment
of
zlib,
you
won't
really
know
to
go
there.
You'd
have
to
like
splunk
around
the
maybe
the
z-lib
repository
there
might
be
a
reference
there
or
something,
but
at
least
collecting
references
to
all
these
things
in
one
place
also
makes
it
easier
to
find.
So
I
say:
has
anybody
looked
at
zlib?
Oh,
it's.
E
It's
been
trail
of
bits
and
these
three
other
organizations,
and
this
is
what
they
have
to
say.
So
we
think
that's
useful
and
the
second
part
of
this
is
similar,
which
is
right
now.
Most
security
data
that
you
get
is
is
either
absent
or
negative.
So
if
you,
if
you
wanted
to
say,
has
anybody
ever
looked
at
foo
and
you
go
out
and
you
don't
see
anything?
E
So
it's
kind
of
like
you
know
all
news
is
bad
news
in
in
some
ways
for
for
security
data,
while
the
very
you
know
the
the
reflection
of
of
you
know.
We
looked
at
this
component
and
it
looks
fine
is
something
that
other
than
audit
reports
that
occasionally
come
back
with
like
gold
stars.
There
isn't
much
of
that.
So
next
slide.
E
This
community
collection
of
security
reviews,
but
I
want
to
make
it
really
really
crystal
clear
here.
This
is
not
a
place
to
to
disclose
new
vulnerabilities.
If
you
found
something
we're
not
a
zero
day
collection
agency,
we
are
not
going
to
notify
projects
when
you
report
them
to
us.
That's
not
this
project's
place
in
the
universe.
Please
report
things
responsibly
and,
and
we
have
a
you-
know,
workflow
on
the
on
the
project
page.
E
So
this
is
an
example
of
of
one
that
I
stiff
contributed
for
open,
ssl
1.1.1
and
it's
a
summary,
and
you
know
it
references,
I
believe
at
the
bottom
of
this
review.
It
references
the
the
original
review
content,
but
the
idea
would
be
that
this
would
be
searchable.
It
would
be
in
one
place,
it
would
be
easy
to
consume
and
with.
Actually,
if
you
go
to
the
next
slide
with
the
we
have
a
you
know,
a
simple
you
know:
quick
start
where
you
can.
E
You
know
put
in
a
review
and
you
you
choose
like
what
have
you
done
well,
I
did
static
analysis
or
I
just
I
did
a
code
review
or
I
just
looked
at
the
website.
You
know
you
disclose
things
like
you
know.
How
severe
are
the
issues
that
you
found?
Is
this
something
that
you
know?
Technically?
Yes,
it's
a
vulnerability,
but
you
know
it's
not
anything
that
that
folks
should
worry
about,
or
is
it
something
that,
like
nobody
should
be
using
this
component
at
all?
E
Because
of
you
know
this
this
horrible
thing
and
then
so
really,
what
I'm
saying
is
you
know
we're
looking
for
contributions
here.
E
We
want
organizations
that
that
do
this
for
a
living
to
be
willing
to
contribute
at
least
references
and
a
summary
and
point
back
to
their
original
content
for
the
for
the
projects
that
for
the
assessments
that
they've
done
individual
security
researchers,
if
you're
part
of
an
organization
where
this
is
what
you
do,
you
know,
consider
consider
submitting
it
and
helping
the
next
team
out-
and
you
know
hopefully
you'll
you
know
someone
else-
will
help
you
out
and
we're
all
kind
of
better
off.
E
This
way,
I
don't
think
any
of
any
of
our
organizations
succeed
by
the
institute
by
open
source,
you
know
being
less
secure
or
less
or
their
their
insecurity
being
less
understood.
E
So
you
can
just
fill
out.
This
form
submit
a
pull
request
and-
and
if
you
have
questions
obviously
ask
this
is
also
a
very
new
project
that
we
will
be
learning
learning
as
we
go
next
slide
yeah.
So
we
have
a
small
but
growing
set
of
reviews.
Yeah,
as
I
said,
please
consider
contributing
our
project
website.
Is
there
ossf
security
reviews?
A
Okay,
great
so
one
last
slide
and
then
we'll
open
up
for
questions
about
you
know
anything.
We
did
get
some
questions
as
we
were
going
along,
but
we'll
leave
a
little
bit
of
time
for
others
there.
You
know,
in
addition
to
all
the
work
that's
going
on
in
openssf,
we
do
have
a
few
things
that
are
upcoming,
that
I
thought
I
would
mention
to
you.
One
is
david
mentioned
during
some
some
of
the
slides
on
open,
ssf
and
and
ryan
mentioned
in
in
his
session.
A
We
are
working
on
some
budgeting
from
opens
open,
ssf
to
security,
related
projects
and
specifically
we're
working
on
a
grant
program
where
we
allow
members
of
the
community
to
provide
funds
to
this
grant
program,
and
then
we
do
some
vetting
of
projects
and
then
decide
which
which
projects
we
will
allocate
funds
to,
and
then
those
projects
will
get
in.
You
know
we,
you
know
we
make
them
part
of
the
official
open,
ssf
funded
projects.
So
that's
pretty
exciting.
A
A
One
is
a
project
originally
initiated
by
google,
but
it's
open
for
anyone
to
participate
and
there's
some
discussions
about
this
going
on
in
the
what
is
it
developer,
identity,
attestation
working
group?
This
is
called
security
levels
for
supply
chain,
artifacts
slsa,
and
this
also
fits
nicely
with
the
work
that
we're
looking
at
for
security
metrics.
Where
we
try
to
define
these.
You
know
levels
of
security
that
projects
can
be
measured
objectively,
for
so
anyway,
have
a
look
at
that.
It's
just
a
project
to
watch.
A
There's
another
project,
that's
doing
some
exciting
work
around
supply
chain,
end-to-end
supply,
chain
integrity
and
that's
in
toto,
and
so
you
can
learn
more
about
that.
There
there
are
a
couple
of
new
ites,
which
is
in
total
enhancement
that
are
being
driven
by
google,
but
are
getting
some
some
community
support,
strong
community
support.
So
just
you
know
if
you're
looking
at
what's
happening
in
supply
chain
security,
those
might
be
a
couple
things
to
look
at.
A
So
those
are
some
upcoming
things
and
now
we'll
leave
open
in
case
people
have
questions
you
might
have
seen
just
pop
up
on
your
screen.
We'd
love
to
have
you
provide
some
feedback
to
us
about
how
the
session
went
today
and
and
we'll
take
that
seriously.
We'll
also
have
a
note
we'll
send
out
an
email
that
will
get
you
to
a
link
where
you
can
provide
some
comment,
form
feedback.
This
is
just
a
you
know,
quick
one
or
another.
How
would
you
rate
the
session
so
so?
A
Please
fill
that
out
any
questions
from
anyone,
we'll
we'll
leave
a
couple
of
minutes.
A
Oh,
let's
see
we
might
have
some
trouble
with
the
poll.
I
think
it
got
started
and
then
might
have
ended.
A
Prematurely
any
other
questions
or
thoughts
about
anything
you
saw
in
the
in
the
news
or
things
that
the
group
is
working
on.
B
If
you
know
I,
I
think
we're
going
to
just
relaunch
the
poll
clearly,
unless
you
have
another
idea,
there.
C
Just
it's
since
it's
closed,
it's
no
longer
an
option
to
relaunch,
but
I
I
can't.
A
All
right,
yeah
take
a
minute
to
do
that.
Everyone
and
then,
while
you're
doing
that
I'll
just
mention
that
we
would
again
that
we'd
love
to
have
people
get
involved.
So
you
can
join
a
technical
working
list.
You
can
join
a
mailing
list.
We
have
a
calendar
that
shows
our
public
meetings
and
you
know
all
of
those
meetings
where
you
know
anyone's
welcome.
There's
a
slack
channel.
You
know
an
easy
way:
low
investment
way
to
just
ask
a
question
for
any
of
the
working
groups.
A
We
record
all
of
our
meetings
and
you
can
find
meeting
recordings
on
the
youtube
channel
and
then
for
organizations.
You
can
become
a
corporate
member
right.
Now,
it's
it's
pretty
easy
to
become
a
corporate
member.
Just
be
a
member
of
the
linux
foundation
and
open
ssf
is
you
know,
an
option
to
join
at
no
cost
next
year
we're
we
are
evaluating
some
member
fees,
and
this
will
help
support
the
work
that
we
do
to
fund
open
source
projects
and
we'll
announce
more
about
that
shortly.