►
From YouTube: Securing Critical Projects WG (November 19, 2020)
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
B
Sweet
okay,
so
we
have
a
couple
presentations
today
and
the
first
one
is
from
josh
who's
from
the
isrg.
Is
it
I
can't
remember
the
relationship
with
let's
encrypt
and
isrg,
but.
B
It
okay
cool,
so
he's
gonna,
be
talking
to
us
today
about
some
of
the
work
he's
doing,
and
then
we
also
have
another
guest
jordan.
Who's
gonna
go
over.
Some
of
the
work
he's
been
doing
on
a
tool
looking
for
malicious
packages
in
pie
pie,
so
it
should
be,
should
be
a
good
hour.
So
josh
do
you
want
to
take
it
away?
You
can.
I
can
stop
sharing
and
you
can
take
over
the
screen.
C
C
All
right,
peace,
everyone,
I'm
josh
from
internet
security,
research
group-
we
are
probably
best
known
for
running
watson,
but
we
do
some
other
things
as
well,
and
we
have
a
memory
safety
initiative
that
we
have
started
on
over
the
past
year.
It's
not
a
full-blown
project
for
us
yet,
but
we
are
getting
there
learning
a
lot,
so
I'm
glad
to
get
a
chance
to
speak
with
you
about
it.
Today,
let's
go
to
the
next
slide,
so
I
think
a
lot
of
people
in
this
call
are
probably
going
to
be
very
familiar
with
the
problem.
C
Memory.
Safety
is
a
big
problem
for
the
internet
software
infrastructure,
mainly
when
I
say
that
a
lack
of
memory
safety
is
a
problem.
I'm
talking
about
c
and
c,
plus
plus
code,
which
you
know
the
cause
of
new
vulnerabilities
every
day
it
is
sort
of
a
plague,
the
safety
of
the
internet.
So
let's
go
to
the
next
slide.
C
And
when
I
talk
about
this
internet
software
infrastructure
and
that
being
unsafe,
we're
talking
about
this
from
top
to
bottom,
so
web
browsers,
you
know
they're
at
the
very
top
of
the
stack.
That's
how
a
lot
of
people
experience
internet.
It
is
millions
and
millions
of
lines
of
c
and
c,
plus
plus
code.
We
know
that
code
is
not
safe.
We
know
we
cannot
make
it
safe.
The
vulnerabilities
are
non-stop
and
you
go
down
from
there
until
you
get
all
the
way
down
to
the
kernel.
C
C
C
C
Google
is
saying
about
90
for
android
vulnerabilities
and
you
know
a
general
look
at
zero
days
found
being
exploited
in
the
wild,
but
80
were
memory
safety
vulnerabilities.
C
C
And
well,
I
think
that
we
all
know
about
these
things.
I'm
not
sure
that
we
quite
gross
yet
how
big
the
consequences
of
our
choice
of
programming
languages
is.
Let's
go
to
like
slide,
and
we
know
how
to
solve
this.
It's
it's
not
an
unsolved
problem,
because
we
don't
know
what
to
do.
C
C
It
is
a
lot
of
work
when
I
talked
to
people
about
this
starting
a
couple
years
ago.
You
know
really
common
reaction
was
this
sounds
like
a
lot
of
work
that
seems
almost
unreasonable,
we're
not
going
to
go
rewrite
all
the
code
that
the
world
depends
on.
C
You
know,
c
and
c,
plus
plus,
are
here
to
stay
they're
not
going
anywhere.
I
don't
think
that's
true
at
all
yeah.
It
is
a
lot
of
work,
but
we
have
a
lot
of
smart
people
in
this
world
and
a
lot
of
resources
and
if
we
decide
we
want
to
solve
this
problem,
we
can
solve
it.
So
it
is
a
lot
of
work,
but
it's
not
that
much
work.
It's
doable
and
I
think,
with
the
right
partners.
C
C
So
we
have
this
memory
safety
initiative
at
isrg.
We
have
two
goals:
move
the
internet,
software
infrastructure
to
memory
safe
code
and
the
second
one
is
change,
the
way
that
people
think
about
memory
safety
and
I'm
gonna
go
through
what
we
plan
to
do
about
both
of
these
goals.
Let's
go
to
the
next
slide,
so
we're
a
small
organization
we're
about
16
full-time
people.
Today
we
are
not
going
to
go
out
and
rewrite.
C
You
know
all
the
c
and
c,
plus
plus
plus
code,
that
is
critical
to
the
infrastructure
ourselves.
That's
not
going
to
happen,
so
we
view
our
role
as
coming
up
with
a
strategy
facilitating
and
coordinating
the
work
that
needs
to
be
done,
raising
the
money
that
needs
to
you
know
we
need
to
raise
to
do
this
work
and
then
communicating
with
the
public
about
what
we're
doing
our
engineers
might
do
a
little
bit
of
work
here
and
there,
but
for
the
most
part,
we'll
be
enabling
community
and
maintenance
to
do
the
work
next
slide.
C
C
Another
one
is:
can
we
do
this
in
a
modular
way?
Can
we
replace
particular
components
of
a
piece
of
software
instead
of
a
wholesale
rewrite,
because
wholesale
rewrites
are
even
more
difficult
to
pull
off
and
the
last
one
is
maintainer
project
and
cooperation,
so
the
more
a
maintainer
is
willing
to
participate
in
this
effort
and
cooperate
the
easier
it
is
to
do
this
and
we'll
get
into
that
a
little
more.
Let's
go
to
the
next
slide.
C
C
If
you
can
get
the
maintainer
on
board
and
fix
the
actual
project,
users
can
be
protected
through
just
a
normal
software
update,
which
everyone
should
be
running.
So
it's
very
important
to
have
maintainers
on
on
board.
Whenever
you
can,
if
you
can
fund
those
maintainers,
it
helps
you
to
create
buy-in
and
alleviates
resource
concerns.
C
So
we
really
try
to
talk
to
maintainers
and
see
if
we
can
get
them
to
participate
in
this.
It's
not
going
to
work
for
every
project,
something
maintainer
just
aren't
going
to
do
it
for
whatever
reason,
then
we
might
have
to
come
up
with
another
strategy,
but
whenever
possible
you
want
to
work
with
them.
C
C
C
C
So
it's
hard
to
walk
into
a
project
and
say
you
know
we
want
you
to
ultimately
move
this
project
away
from
sea
and,
let's
say,
for
example,
towards
rust.
You
know,
maintainers
may
not
know
rust.
They
don't
know.
If
russ
is
the
right
choice
could
be
some
other
language,
I'm
just
using
russell
as
an
example
here,
but
they
may
not.
D
C
C
For
example,
curl
had
maybe
had
the
same
set
of
concerns
as
you
and
here's
how
we
dealt
with
those
concerns
and
here's
how
everything
played
out
and
how
we're
able
to
deliver
a
safer
product
so
we're
starting
with
some
some
projects
where
we
have
maintainer
buy-in
and
have
you
know
a
good
path
to
success
and
we're
going
to
use
those
stories
to
help
other
maintainers
make
the
decision
to
go
down
that
road
and
we're
going
to
build
up.
C
You
know
a
corpus
of
success
stories
and
hopefully
that
will
convince
more
maintainers
to
help
out
next
slide.
C
So
we're
focusing
on
three
things
at
the
moment.
We
may
do
some
more
things
shortly,
but
the
first
one
is
curl,
so
we've
publicly
announced
the
project
to
make
curls
hdp
and
tls
code
memory
safe.
This
project
was
our
first
project
because
it
really
embodies
all
the
things
that
we've
talked
about
here
in
our
approach.
C
C
We've
got
a
module
approach
going
on
here,
so
we're
not
rewriting
curl
from
scratch,
but
we're
replacing
openssl
with
russells,
and
we
are
replacing
the
networking
libraries
with
a
safe
networking
library
called
paper-
and
this
is
you
know,
the
work
is
actually
getting
pretty
close
to
done
now.
So
a
couple
more
months,
we'll
have
something
ready
to
go
and
that
can
ship
in
a
relatively
short
amount
of
time.
You
don't
have
to
spend
years
rewriting
curl.
C
So
this
is
a
perfect
example
of
what
I'd
like
to
accomplish
and
the
kinds
of
projects
we
want
to
work
on.
First,
and
once
we
deliver
these
success
stories,
we
can
move
on
to
projects
that
are
a
little
more
difficult.
C
C
So
I'm
not
quite
ready
to
talk
about
the
details
of
that
yet,
but
we'll
be
making
some
announcements
soon,
and
the
last
example
is
the
linux
kernel
and
honestly,
when
I
started
working
on
this,
we
made
a
list
of
all
the
projects.
That
might
you
know
be
something
we
could
help
with
and
at
the
bottom
of
my
initial
list.
C
C
Those
things
need
to
be
built.
They're
important
because
we're
not
going
to
rewrite
all
this
code,
certainly
not
in
a
short
amount
of
time,
and
these
things
really
do
help.
These
just
are
not
what
we
are
working
on.
So
that's
just
not
this
project,
we're
glad
that
those
things
are
happening,
but
it's
not
us.
C
They
do
introduce
a
bunch
of
overhead,
so
projects
have
to
run
them,
make
sure
they're
running
you
got
to
keep
running
them
and
they
don't
ultimately
solve
the
problem
programs
projects
that
you
fuzz
and
apply
static
analysis.
They
still
have
more
safety
vulnerabilities,
so
they're
important
mitigations,
but
they
do
not
solve
the
problem
and
they
introduce
some
overhead.
C
C
We've
learned
this
lesson
a
million
times,
but
it's
still
pretty
stuck
pretty
standard
practice
to
stick
millions
of
lines
of
cnc
plus,
whilst
on
the
edge
of
your
network
that
has
to
change,
we
can't
we
can't
continue
doing
this,
we're
just
going
to
get
been
over
and
over
again
and
again,
real
people
pay
the
price.
C
Think
back
to
that
slide.
You
know
it's
financial
losses.
It's
hospitals
getting
shut
down.
It's
massive
privacy
breaches.
We
can't
keep
doing
this.
We
need
to
get
to
the
point
where
you
know:
sticking
apache
or
nginx,
or
something
else
that's
written
in
c
in
the
edge
of
your
network
is
seen
as
irresponsible.
C
That
is
the
end.
I
believe
I've
come
in
just
under
the
20
minutes,
so
I
want
to
thank
paul
care
and
alex
gainer
for
the
work
that
they
have
done
in
helping
us
with
this
problem
and
also
daniel
sundberg
who's,
the
maintainer
of
curl,
who
has
been
a
great
initial
partner
in
this
project,.
B
F
Yeah,
so
if
I,
if
I
can
jump
in
david
wheeler,
I
agree
with
you
that
the
modular
approach
in
general
is
better.
One
challenge
with
the
modular
approach
is
that
now
you
have
the
challenge
of
conversions
and
dealing
with
the
date
with
differences
in
data
structures
and
approaches
between
different
languages?
C
It
really
depends
on
your
language
choice.
You
know,
I
don't
think
that
the
answer
to
every
memory
safety
problem
is
rust,
but
rust
has
really
made
it
possible
to
solve
this
problem
in
the
way
that,
like
I,
don't
think
this
project
will
really
would
have
made
sense
prior
to
having
rust,
mature
and
one
of
the
most
important
things
about
russ
is
that
it
works
really
well
with
cnc
plus
code.
So
the
foreign
function
interface
is
fantastic.
C
C
So
you
know
in
the
module
approach
one
of
the
nice
things
that
a
logic
projects
can
benefit
from
the
same
module
so,
for
example,
we're
investing
pretty
heavily
in
the
russell's
tls
library
to
replace
openssl,
so
we're
building
a
c
api.
That's
going
to
be
in
the
russell's
repository
that
any
c
program
can
use.
C
C
B
So
I'm
curious,
I
I
know
it's
a
modular
approach
and
it's
something
we've
talked
about
in
this
working
group
before
too,
as
every
project's
different,
you
know
everything
needs
a
different
thing.
Do
you
do
you
have
ideas
how
you
might
like
scale
your
work
like?
Is
it
is
it
funding?
Is
it?
Is
it
people
like
doing
these
negotiations
with
maintainers
or
have
you
have
you?
Have
you
thought
about
that?.
C
Yeah,
I
think
we
need
to
find
ways
to
focus
on
the
most
important
things.
The
most
important
things
for
us
right
now
are
to
build
up
some
success
stories
so
that
we
can
refute.
You
know
concerns
that
people
have
about
this
stuff.
So,
for
example,
we
need
to
get
curl
done
when
curl
is
done
and
it's
working
well.
C
You
know
that
gets
us
the
ability
to
talk
to
more
people.
We
need
to
focus
on
high
value
projects.
So
if
you
want
to
reach
a
lot
of
people
and
scale,
I
mean
depends
on
what
you
mean
by
scale.
But
if
the
goal
here
is
to
deliver
memory,
safe
software
updates
to
a
lot
of
people,
then
you
just
got
to
pick
the
right
project.
So
you
know
focusing
on
something
like
russell's
and
saying
you
know:
russell's
is
our
choice
for
a
memory
safe,
tls
library.
C
We
can
do
all
the
work
here
and
we
don't
have
to
convert
eight
different.
You
know
open
ssl
libraries
to
memory
save
code.
That's
not
gonna
happen.
So
beyond
that.
If
we're
talking
about
scale
in
terms
of
how
many
people
can
we
get
working
on
this?
In
parallel,
part
of
the
answer
is
just
having
the
funding
to
do
the
projects
that
we
need,
but
I
think
we
also
just
need
to
be
a
little
patient.
C
If
you
rush
out
there
and
try
to
you,
know,
fund
everything
and
get
a
thousand
engineers
working
on
this
tomorrow,
it's
going
to
be
a
bit
of
a
mess
right.
We
need
to
be
careful
and
build
up
our
case,
so
we
want
to
get
the
funding
that
we
want,
for
you
know
the
next
set
of
projects
that
make
sense.
We
want
to
have
that
available,
but
I
don't
think
we
want
to
scale
too
fast.
B
That
makes
sense
I'd
be
curious
how
how
you're
defining
what
projects
are
important.
I
know
you
said
based
on
usage.
If
there's
you
know,
if
we
could
collaborate
on
some
of
that
together,
what
you're,
using
for
heuristics
to
come
up
with
your
list
could
be
interesting.
C
So,
let's
say
you've
got
you
know
several
billion
instances
of
linux
kernel,
that's
its
weight.
You
know,
then,
there's
a
certain
amount
of
weight
to
nginx,
based
on
how
heavily
that's
used.
How
many
instances
of
nginx
are
out
there
in
the
world,
so
you
want
to
look
at
how
common
something
is
obviously
we're
only
looking
at
things
that
are
written
in
c
and
c,
plus
plus.
C
So
you
know.
I
know
that
in
a
lot
of
vulnerability,
analysis
that
your
working
groups
do
you're
talking
about
a
lot
of
javascript
projects
and
dependencies,
and
things
like
that-
and
certainly
there
are
security
concerns
there.
But
you
know
luckily
memory
safety
is
not
one
of
them.
So
we're
really
only
looking
at
commonly
used,
cnc
plus
plus
applications.
D
C
C
G
G
Okay,
I'm
I'm
I'm.
I
was
trying
to
look
it
up
and
get
my
question
answered
and
I'm
I'm
failing
so
far.
So
anyway,
yeah.
C
We
haven't
actually
published
it
yet
this
is
pretty
recent,
so
google
is
funding
the
work
with
curl
and
some
of
the
work
of
apache.
C
Right,
we're
non-profit
our
you
know
we
are
not.
We
don't
have
the
funds
to
fund
all
the
work
that
we
would
like
to
do
ourselves.
So
our
role
here
is
to
figure
out
the
projects
and
figure
out
the
best
return
on
investment.
Where
is
the
best
place
to
put
in
resources
and
then
we'll
go
raise
that
money
and
make
sure
that
it
gets
to
the
right
people
to
get
the
work
done.
G
Yeah,
I
I
yeah,
no
I'm
I
I
love
it
come
come
talk
to
us.
I
I
this
is.
This
is
sort
of
a
continually
depressing
topic,
because
we
were
just
launched
some
fuzzing
efforts
and
found
some
code
that
it's
like
yeah
memory
safety,
and
I
was
looking
with
the
over
the
shoulder
of
the
researcher
and
staring
at
the
code
and
I'm
going
wait.
They're
not
checking
any
of
the
input
parameters
here
and
they're
using
it
to
index
into
arrays,
and
it's
like
wait.
They're
like
and
then
she
sees
the
researchers
said
yeah.
G
This
is
crap
code.
So
it's
like
awesome
and
it's
code
that
it
was
written
by
an
intel
person.
That's
maintained
by
an
intel
person
today,
not
like
it's
part
of
their
sort
of
day
job
right
and
and
it's
what's
distressing.
G
If
we
hadn't
launched
this
fuzzer
effort
for
a
separate,
you
know,
project
they'd
be
like
well,
we
would
never
have
found
this
or-
and
I
think
these
bugs
have
been
around
at
least
until
since
2016
and
maybe
since
2011,
so
that
doesn't
make
me
that
makes
me
incredibly
depressed,
because
if
we
hadn't
done
this
fuzzing,
we
would
never
have
found
it
until
well.
Maybe
it's
being
exploited
every
day
who
knows
by.
C
G
Criminals
or
something
like
that,
so
yeah,
it's
it's
a
and
it
started
getting
us
really
interested
in
gee.
I
wonder
what
other
parts
of
the
code
you
know
not
necessarily
stuff
that
we've
done,
but
other
parts
of
the
code
that
might
be
similarly
crappy.
So
it's
a
it's.
D
G
And
and
nobody
would
have
in
their
day,
job
said:
oh,
you
should
rewrite
this
intel
and
rust
right.
It's
like
it's
good.
C
C
Well,
it's
an
important
point
about
fuzzers,
which
is
that
you
know
part
of
their
value
is
in
helping
you
make
cnc
plus
plus
code
a
little
safer,
but
part
of
their
value
is
in
pointing
out
to
you
how
bad
this
code
actually
is.
You
know,
and
it's
not
about
intel
engineers
or
google
engineers,
all
human
engineers,
writing,
c,
c
and
c,
plus
plus
code
they're,
not
good
enough
at
it.
It's
not
a
particular
engineer.
Humans
do
not
write
good
enough
code
to
keep
your
memory
safe.
The
answer
here
is
to
have
the
compiler.
C
G
G
About
this,
but
I
I
probably
better,
not
you
know,
okay,
I
I
I
I
have
a
lot
of
intellectual
curiosity
relative
to
this,
and
the
rust
conversation
is
one
that
is
it's
not
a
new
one
in
my
mind,
but
anyway,
that's
that's
great
thanks.
Thanks
for
your
presentation,.
A
C
C
This
is
a
place
where
you
know
the
compiler
can
do
the
work
and
you're
not
adding
a
step,
and
we
don't
have
to
add
this
layer
of
fuzzing
and
static
analysis
from
memory
safety
on
top
of
it.
So
I
think
another
benefit
of
all.
This
is
removing
complexity
from
the
general
best
practices
for
software
development.
F
Yeah
josh,
if
I
can
push
back
a
little
bit,
you
can
have
memory
safety
and
you
still
need
fuzzers
and
you
still
need
static
analyzers.
You
just
don't
need
them
to
detect
for
memory
safety
properties,
there's
other
properties.
A
F
You
want
memory
safety,
but
and
that,
but
you
know
that
statement
I
just
said
doesn't
mean
that
this
switching
to
memory
safe,
is
a
bad
idea.
F
Depends
on
what
it's
doing,
but
so
it's
it's
still
better
how's
this!
I
you,
but
my
point
still
stands.
You
still
need
fuzzers.
You
still
need
static
analysis,
but
this,
but
you're
also
right
that
those
two
are
not
guaranteeing
the
things
that
using
a
memory
safe
language
does.
C
F
B
Yeah
and
then
the
link
to
those
slides
are
in
the
meeting
notes.
If
anyone
wants
to
go
back
and
and
reference
them
jordan,
I
don't
know
if
you
have
slides
or
you
just
want
to
talk
either
is
totally
fine
yeah
all
right.
Do
you
want
me
to
share
that,
but.
E
No,
that's
that's
just
a
link
to
the
blog
post,
so
what
I
was
hoping
to
do
is
just
kind
of
briefly
give
an
overview
of
the
research
that
I
did
and
then
talk
about
why
I
think
that
the
openssf
is
an
awesome
organization
to
kind
of
take
it
to
the
next
step.
E
Talk
about
what
that
looks
like
what
that
means
and
then
try
to
kind
of
figure
out
what
those
next
steps
look
like
and
we
can
kind
of
go
from
there.
So
hi
everybody.
My
name
is
jordan.
I
am
here
just
representing
myself
today,
because
I
recently
moved
from
working
at
duo
security.
You
know
part
of
cisco
and
have
joined
stripe
as
a
security
engineer
on
their
team.
So
I
did
this
research
like
in
the
middle
of
that
week-long
gap.
E
So
so
I
still
haven't
figured
out
the
you
know
the
process
of
coming
on
board
under
stripes
umbrella,
but
I'm
working
on
that
and
already
talking
to
people
about
this
effort,
some
more
to
come
there.
So
today,
I'm
just
here
as
jordan,
so
the
research
that
I
did
I
wanted
to
see
if
I
could
use
dynamic
analysis
to
try
and
find
potentially
malicious
packages
on
pipeline.
E
You
know
we've
all
kind
of
heard
the
incidents
in
the
past
not
just
on
pipeline,
but
on
any
package
manager.
You
know
a
there
may
be
like
a
typo
squatting
package
or
you
know,
reuse
credentials,
give
an
actor
access
to
a
popular
package.
They
try
to
put
malicious
content
in
it
and
that
could
have
pretty
significant
effects.
You
know
I
always
try
to
air
away
from
you
know,
spreading
fud
and
in
in
hyping
things
up,
but
it
could
be
significant.
E
You
know
if,
if
I
ever
feel
you
know
calm,
then
I'll
just
go
to
libraries,
dot,
io,
slash
experiments
and
see
some
of
the
metrics
that
they
post
up
in
terms
of
just
how
often
these
are
downloaded
with
like
one
maintainer.
I
haven't
been
touched
in
in
years
and
and
my
anxiety
goes
right
back
up.
My
hope
was
to
try
to
figure
out
you
know.
Can
we
use
dynamic
analysis
to
find
malicious
activity
with
a
high
signal
to
noise
ratio?
Now?
E
What
does
it
look
like
because
some
package
managers
pipe
included
already
have
some?
You
know
malware
checking
capabilities.
You
know,
for
example,
there
was
just
some
some
effort
done
on
pi
pi
to
do
more
of
the
static
analysis,
work
which
is
great.
You
know
that
that
takes
a
huge
chunk
out
of
the
equation,
and
the
goal
here
is
just
kind
of
a
raising
the
bar
exercise
to
make
it
harder
for
attackers
to
put
malicious
content
on
on
package
managers.
E
That
removes
any
kind
of
potential
obfuscation
that
came
with
the
code.
That's
one
of
the
things
that
makes
static
analysis
a
little
bit
harder.
You
know
to
kind
of
offset
some
of
the
more
expensive
nature
of
this
dynamic
analysis,
actually
installing
the
package,
and
it
was
pretty
successful.
E
You
know
I
show
a
few
case
studies
in
in
my
blog
post
that
show
just
how
powerful
this
is
being
able
to
take
an
obfuscated
block
of
code
and
show
exactly
what
commands
were
executed
exactly
what
network
connections
were
established,
and
I
think
this
is
really
promising
for
a
few
reasons
you
know.
The
first
is
that
it
gives
that
high
signal.
You
know.
Yes,
there
are
a
lot
of
sys
calls
that
happen
whenever
you
install
a
package.
E
You
know
that
that's
expected
right,
but
there
are
some
things
that
would
make
us
at
least
kind
of
raise
our
eyebrows
so
to
speak.
That
would
warrant
further
investigation.
You
know,
there's
network
connections
really
in
general.
I
think
it's
worth
looking
into.
You
know.
Why
is
this
package
reaching
out
somewhere?
E
Just
whenever
I'm
installing
it
likely
is
the
case?
It's
benign.
It
may
just
be
installing
some
helping
components
that
may
be.
You
know
installing
some
other
setup
scripts,
but
that's
always
something
to
kind
of
look
at
with
suspicion
or,
if
they're
accessing
different
places
on
the
file
system.
That
would
make
me
concerned.
You
know,
for
example,
if
something's
trying
to
read
ssh
keys
or
if
they're,
trying
to
access,
you
know
secrets
on
disk.
You
know
in
common
places
those
are
suspicious.
E
E
You
know
many
times,
and
so
the
research
went
well,
you
know,
I
think
it
looks
really
promising,
and
so
what
I'm
excited
about
the
open
ssf
is
that
whenever
we
think
about
the
next
steps,
you
know
that
was
a
one-time
study
and
I
talked
in
the
blog
post.
The
benefit
is
doing
this
continuously
watching
new
packages,
as
they
update
doing
this
same
type
of
analysis,
though
we
can
catch
things
as
early
as
possible.
E
The
benefit
of
doing
this
with
the
openssf
is
that
it's
a
central
organization
that
brings
everybody
together
in
a
room
and
we
can
make
this
one
capability
accessible
to
any
number
of
upstream
package
managers.
Compare
this
with
other
options.
Right.
One
option
is
that
we
ask
each
package
manager
to
run
this
infrastructure
themselves,
that's
difficult
both
because
we're
repeating
work
and
because
I
have
nothing
but
empathy
and
respect
for
all
the
people
maintaining
our
package
managers.
E
You
know
many
of
them
are
volunteers
on
limited
funding,
limited
budget
limited
people-
and
I
would
rather
this
be
done
in
a
place
where
it's
a
centralized
capability
that
everyone
gets
the
benefits
from,
and
the
other
alternative
is
that
I've
had
companies
reach
out
to
me
saying:
hey
I'd
like
to
use
your
thing
just
at
my
company
as
part
of
my
ci
pipeline,
I
said
I
don't
think
you
should
like.
I
don't.
E
I
see
this
as
something
that
we
could
solve
centrally
for
the
ecosystem,
not
something
that
every
company
should
be
doing,
because
it's
not
really
a
special
thing.
You
know
it's
something
that
we're
all
accessing
the
same
open
source
packages.
We
might
as
well
solve
this
and
solve
it
once
and
so
that's
why
the
open
ssf
makes
perfect
sense
because
we're
getting
everybody
together,
we
can
have
really
really
close
ties
with
maintainers
of
upstream
package
managers,
and
we
can
set
up
some
kind
of
just
communication.
E
You
know,
honestly,
you
know
where
we
figure
out.
What
do
we
do
whenever
we
see
something
suspicious
kind
of
who's?
You
know
how
do
we
triage
that
until
we
have
a
a
confidence
level
to
bring
it
to
the
maintainers
upstream,
the
goal
being
that
we're
not
overwhelming
people
with
false
positives?
You
know
that
we're
not
asking
them
just
like
investigate
all
these
yourselves
with
the
already
limited
time
that
you
have
so
where
we
are
now
is
that
the
research
is
done.
E
I
am
starting
to
put
together
a
roadmap
of
what
things
need
to
happen
before
this
is
kind
of
considered
robust.
For
example,
I
just
started
with
pi
pi.
You
know
npm,
ruby,
gyms,
you
name,
it
are
all
great
candidates
and
they've,
I'm
not
the
only
one.
Who's
done
dynamic
analysis
in
the
past.
You
know,
I
think,
there's
even
multiple
people
on
the
call
who
have
done
work
in
this
space
really
similar
to
what
I've
done.
E
So
I'm
excited
that
we
can
all
kind
of
put
our
heads
together
and
figure
out
what
this
looks
like
so
yeah,
making
this
a
continuous
scanning
for
multiple
package
managers
trying
to
increase
the
signal
to
noise
ratio.
You
know
right
now.
My
process
was
very
scientific.
I
got
a
whole
bunch
of
data
and
then
I
just
grabbed
a
lot
like
that
was
that
was
pretty
much
it
because
I
didn't
even
know
what
it
was
that
I
was
looking
for.
E
I
was
like,
let
me
think
about
what
would
be
suspicious
and
see
what
I
can
find.
It
was
very
I
mean,
there's
a
whole
whole
science
oriented
thing,
so
we
would
prefer
for
people
to
not
have
to
do
that
as
much
as
having
that
be
done
for
us
like
in
an
automated
fashion
and
then
use
the
expensive
people
cycles
where
they
could
be
used
most
and
be
the
most
beneficial.
E
So
all
this
together.
That's
that's
why
I'm
here
is
just
to
kind
of
chat
with
y'all
figure
out.
What
is
this
even
the
right
subgroup
to
be
in
because
there
there
are
multiple
subgroups
under
the
open
ssf
for
two
to
hear
from
y'all
about
y'all's
perspectives.
What
you
think
can
go
well
what
you
are
worried
about
and
then
just
try
to
figure
out
what
next
steps
look
like.
G
G
B
Oh
there
he
goes
yeah.
Thank
you,
jordan.
This
is.
This
is
really
interesting.
I
I
love
the
idea
of
being
able
to
take
this
and
scale
it
out
to
other
package
managers
and
everything,
I'm
not
sure
where
we
go
directly
from
here.
I
know
dan
and
I
on
the
google
side,
are
looking
at
some
ways
that
we
might
be
able
to
support
this
and
support
this
work.
So
that's
that's
where
we
are
yeah.
G
Sorry,
I
was
I
I
had
a
family
member
mentioned.
Something
to
me.
I
was
I
I
apologize.
This
is
living
at
work
now.
What
I
was
gonna
say
was
that
yeah,
I
love
what
your
thought
process
here
as
well.
Is
you
know?
One
of
the
things
we
stood
up
a
few
years
ago
was
a
capability
which,
at
least
on
the
kernel
side,
would
try
and
build
the
latest
tip
and
try
and
boot
it
on
a
huge
number
of
machines
right.
This
was
like
the
boot
problem.
G
We
call
it
our
zero
day
lab,
and
so
we
we
did
this
every
night
and
you
know,
based
on
if
we
saw
boot
failures,
we'd
send
mail
back
to
the
the
kernel
mailing
list
and
said
hey
this
patch.
You
know
we
bisected,
and
this
is
the
patch
that
was
was
causing
problems,
so
I
think,
even
even
without
the
research,
I
think
what
you've
basically
said
is.
This
is
not
terribly
scientific,
but
it's
something
at
least
gives
people
an
indicator
where
to
look
and
be
suspicious
right.
G
If
somebody
wants
to
really
you
know
nail
into
this,
but
even
more
so
if
something,
if
a
new
package
or
a
revised
package
that
goes
into
you,
know,
pie,
pie,
you'd
want
to
say:
oh
look,
maybe
a
piece
of
email
that
would
poke
up
a
a
diff
right.
You
know-
and
that's
that's
you
know,
that's
the
sort
of
thing
that
an
organization
like
you
know,
google,
intel,
microsoft,
somebody
you
know
or
even
like
the
psf.
G
That's
that's
another
interesting
possibility,
because
psf
is
in
theory,
you
know
their
their
goal
is
to
try
and
improve
overall
python
right
capabilities
like
this
and
so
psf.
I
think
I
think
in
collaborating
with
those
guys
as
well,
at
least
on
the
python
side.
Now
you
know
node,
php,
etc.
Ruby,
as
you
said,
there's
a
lot
of
other
package
managers
that
that
you
know,
but
this
says
this
says
real
promise
too
yeah.
E
There's
there's
another
thing
to
mention
which
is
there's
the
question
of
whenever
we
see
weird
things
happen,
what
do
we
do
and
and
there's
a
couple
of
answers?
One-
is
that
if
there's
something
that's
weird
in
terms
of
actively
harmful,
you
know,
in
that
case,
there's
an
opportunity
to
let
the
package
manager
maintainers
know
so
that
the
package
can
be
removed
right
and
these,
but
there's
another
option
too.
You
know,
I
know
that
we
recently
being
all
of
y'all
doing
doing
all
the
work.
E
The
open
ssf
has
released
the
scorecard
capability
and
the
goal
is
to
try
to
give
people
answers
to
the
question.
What's
the
risk
of
like
bringing
this
package
into
my
organization
right,
you
know
like
what
you
know.
What
are
some
heuristics
that
I
can
kind
of
get
a
gut
check
on
on.
E
Do
I
need
to
care
about
this
kind
of
information
might
be
useful,
hey
we're
letting
you
know
it
runs
these
commands.
It
makes
these
network
calls.
You
know
anytime,
that
it's
installed.
It
doesn't
mean
that
it's
bad.
It
just
means
that
you
may
want
to
think
about
it,
and
I
think
that's
that's
useful.
So
it's
it's
not
the
only
output
doesn't
have
to
be
just
going
to
package
manager
maintainers.
E
It
could
also
be
giving
people
downstream
more
information
about
the
things
that
they're
installing
and
the
things
that
they're
using
and
one
super
quick
note
you
mentioned
psf.
I
didn't
want
to
give
a
shout
out
because
we
have
dustin
on
the
call
and
dustin
reached
out
shortly
after
the
research
was
published.
So
I
know
that
you
know
I
don't
want
to
speak
for
you
dustin,
but
I
think
there's
interest
you
know
and
just
kind
of
what
this
might
look
like
right.
I
Absolutely
psf
has
a
ton
of
interest
pipia's
tone
interest.
This
has
been
sort
of
ongoing
issue
with,
like
you
said
this
package
manager
and
every
other
one
as
well
so
yeah,
and
I
wanted
to
chime
in
like
not
only
is
having
some
form
of
analysis
of
every
project
on
pipi
interesting,
but
also
if
we
had
a
way
to
sort
of
classify
the
behaviors
at
install
time
of
a
project
like
this
package
makes
network
requests.
The
users
could
potentially
in
the
future,
have
the
ability
to
say
just
don't
install
anything
that
makes
network
requests.
I
G
J
As
you
guys
scale
this
out,
we
just
did
a
assist,
call
monitoring
project
for
one
of
our
commercial
products
at
smallstep
that
plugs
in
directly
to
ebpf
and
then
has
like
a
collection
stack
that
dumps
the
events
into
an
elastic
cluster.
J
It's
like,
I
said
it's
part
of
our
commercial
cluster,
but
our
commercial
product,
but
if
we
could
help
you
guys
scale
this
out
in
any
way.
I
mean
minimally
we'd
be
happy
to
share
experience,
if
not
code.
E
That'd
be
great
yeah
I'll
say
that
the
the
hard
part
about
this
is
the
the
piece
in
the
pipeline
that
installs
the
package
and
watches
calls.
You
know
ideally
right
now.
It's
just
a
c2
instance.
You
know
and
and
I
I
throw
cystic
on
it
and
then
I
spin
up.
You
know
a
couple
of
containers,
one
to
watch
traffic
and
then
one
to
actually
install
the
package
and
systick
is
running
on
the
host,
with
some
filters
set
up
to
only
catch
those
syscalls
right.
E
E
To
share
what
I
know
so
great
that
sounds
perfect,
then
the
oh
yeah,
one
thing
I
wanted
to
mention
was
the
re
one.
Another
reason
I'm
excited
to
be
here
at
the
open
ssf
is
that
another
question
was
given
to.
I
think
it
was
was
josh
about
like
where
funding
comes
from.
That's
hugely
exciting
to
me,
because,
right
now
my
funding
is
my
wallet,
and
so
that
would
be
great
to
not
have
that
be
the
case,
because
I
think
about
that
on
an
ongoing
basis.
F
So
let
me
I'm
gonna
jump
in
real
quick.
I
can't
speak
for
this
entire
working
group,
of
course,
but
in
my
personal
head
at
least,
I
think
that
the
major
repo,
the
major
repositories,
pipi
nodejs,
are
critical
projects
in
their
own
right.
I
mean
you
know
if
I
wrote
down
a
definition
of
what's
critical
and
pi
p,
I
didn't
show
up
there,
you
know
or
didn't
at
least
meet
that
criteria.
That
would
be
kind
of
surprising
to
me.
F
So
so
I
I
mean
you
know
you
could
certainly
feel
free
to
talk
to
other
open
ssf
working
groups,
but
certainly
I
think
this
working
group
would
be
a
perfectly
reasonable
place
to
discuss
this.
I,
I
guess,
there's
there's
several
issues
as
far
as
your
implementation.
You
know
the
the
whole
analysis.
F
The
one
thing
that
concerns
me
is,
you
know,
there's
several
different
kinds
of
malicious
actors
here
involved.
You
know
some
are
just
going
to
try
to
slip
in
and
they're
attacking.
You
know
the
person
who
wrote
the
code
is
actually
getting
attacked
themselves.
F
You
know
their
passwords
or
got
stolen
or
whatever,
but
then
you
have
the
ones
who
are
actually
in
intentional
diligent
initialist
code
or
they're
really
willing
to
put
in
effort,
and
so
they
may
try
to
evade
any
detection
mechanism
you
put
so
it
seems
to
me
that
you
want
like
the
general
mechanism
of
gathering
the
data
open
source,
the
tools
to
implement
rules
to
be
open
source,
and
maybe
the
specific
rules
be
the
hidden
secret
sauce
that
you
hide
off
a
little
bit
somehow
so
that
you
don't
reveal
what
you're
looking
for,
because
if
you
reveal
what
you're
looking
for
some
adversaries
will
try
to
get
right
now,
I
you
know,
maybe
you
know
the
disadvantage.
F
Of
course
is
that
then
you
have
a
lot
less
cooperative
help
on
developing
those
rules.
So
maybe
you
can
make
some
rules.
You
know
fully
released
and
some
not
so
much,
but
you
know
I'm
just
kind
of
thinking
about
the
pros
and
the
cons,
but
I
do
very
much
like
the
idea
of
analyzing
stuff
so
that
in
a
common
way,
so
that
you
aren't
just
easily
emotious
actor,
can't
just
upload
random
garbage,
that's
dangerous.
E
Yeah,
so
so
a
couple
I
I
agree
with
you
and
and
a
couple
things
that
in
the
past,
whenever
I've
thought
about
you
know,
evasion
that
I
kind
of
lean
on
the
first
is.
The
first
is
that
my
goal
is
is
not
necessarily
to
eliminate.
You
know
the
the
problem
entirely.
You
know
it's
very
much
just
to
raise
the
bar
and
to
make
it
more
costly,
more
difficult
to
introduce
malicious
code
into
any
of
these
package
managers
and
at
a
low
cost
to
us
right.
E
You
know
if
I'm
spending
a
dollar
and
they
have
to
now
spend
ten
twenty
dollars.
That's
that's
a
pretty
good
trade-off,
yeah.
The
the
other
thing
is
that
if
we
think
about
the
level
that
we're
monitoring
at
we
think
about
the
rules,
but
the
rules
to
me
is
really
just
what
do
we
filter
down
to
to
look
at
kind
of
manually,
because
if
we
look
at
the
syscall
level
as
soon
as
you
make
a
network
connection,
I'm
gonna
know
or
as
soon
as
you,
you
know,
invoke
a
shell.
E
You
know
and
call
exactly
where
we're
gonna
know
right
and
even
if
that's
probing
the
system
to
look
for
if
certain
files
exist,
if
it's
kind
of
profiling,
the
system,
we
have
the
capability
to
kind
of
know
that
too
right-
and
so
that's
that's
a
benefit.
So,
but
there
is
the.
What
do
we
elevate
for
people
to
look
at?
You
know
we.
F
And
maybe
maybe
one
option
here
is
kind
of
split
this
up
I
mean
if,
if
you
do
an
analysis
and
here's
what
we
see
you
know
here's
you
know
here,
you
know
you
could
download
this
50
gigabyte
file
of
every
syscall
and
you
know
the
in
in
json
form
and
so
on.
You
could
then
stick
a
whole
bunch
of
different
tools
to
look
at
it.
E
You
know
what
I
mean
like
whenever
we
that's
that's
one
of
the
things
that
I
considered
doing
until
I
opened
up
my
wallet
and
I
was
like
there's
not
much
here,
so
I
can't
go
and
give
that
everyone
access
to
like
a
terabyte
of
data
on
s3
and
and
get
away
with
it.
But
there's
there's
tons
of
opportunities
there
to
team
up
with
folks
who
want
to
go
digging
for
stuff
that
we've
missed.
A
So
that
kind
of
ties
into
it's
the
next
step-
and
you
mentioned
this
a
little
bit
before-
but
once
you've
found
a
malicious
packet
or
a
package
we
think
is
malicious.
What
do
you
do
with
that
information?
I
know
this
kind
of
fits
into
the
category.
I
guess
of
typo
squatting
packages
in
my
head
and
I
know
from
a
presentation
one
of
the
other
working
groups
like
the
ruby
gems
people
have
had
actually
a
whole
bunch
of
trouble
getting
typo
squatted
packages
filed
as
cves.
A
It's
meant
to
do,
it's
just
meant
to
be
malicious,
and
so
that
itself
is
not
technically
a
cve,
because
it's
not
a
vulnerability
in
that
package.
It's
by
design,
and
so
it
almost
points
to
a
need
for
a
separate
place.
To
put
these
flagged
malicious
things
that
are
malicious
by
design.
I
I'll
shine
in
the
the
on
the
long
tail
of
all
the
projects
on
these
package
indices.
Basically,
anything
we
can
identify,
as
potentially
malicious,
is
going
to
be
a
valid
use
case
for
a
given
project.
So
I
think
anything
that
you
know
blocks
them
up
front
prevents
them
from
being
installed.
Anything
like
that
is
just
probably
not
going
to
work,
but
raising
awareness
to
end
users
so
that
they
can
sort
of
see
the
behaviors
of
a
package
that
they're,
considering
or
evaluating.
I
A
And
there's
probably
some
that
are
clearly
malicious,
though
that
you
might
consider
taking
down
on
the
pi
pi
side
right
like
some
of
the
type
of
squatting
ones.
I
think
have
been
taken
down
in
the
past,
like
if
a
package
just
uploads
all
the
keys
in
your
home
directory
or
something
like
that
when
it's
installed
and
it's
clearly
camouflaging,
is
something
else.
F
And
then
at
least
and
then
at
least
there's
a
protection
for
joe
average,
who
or
joanne
average,
who
just
you,
know,
they're,
expecting
something
simple
and
they
type
dash
instead
of
underscore
and
didn't
get
what
they
were
expecting.
E
Yeah,
there's
there's
so
much
room
for
us
figuring
out
how
opinionated
we
on
the
open,
ssf
even
want
to
be
in
terms
of
you
know
what
what
do
we
want
to
give
up
stream
or
do
we
just
want
to
try
to
give
high
signal
upstream
and
then
let
package
manager
maintainers
do
what
they
they
feel
is
best
kind
of
for
their
ecosystem
right
and
there's
a
lot
of
a
lot
of
room
in
the
middle,
but
I
do
want
to
do
a
time
check.
E
I
know
that
we're
like
at
time
I
apologize
for
taking
over
20
minutes.
You
know
I'm
happy
to
kind
of
stop
the
discussion
there.
We
can
just
kind
of
work
together
figuring
out
next
steps.
Offline.
B
B
B
F
F
You
know
trying
to
do
these
sorts
of
things,
because
I
think
that
pi
pi
would
be
more
confident
about,
say,
removing
or
not
allowing
install
by
default.
If
they
were
very,
very
confident
in
the
signal
and
if
they're
not
very
confident
in
the
signal,
then
I
think
it's
quite
appropriate
to
say.
Well,
you
know
I
maybe
I'll.
Let
them
find
that
information,
but
I
don't
want
to
you
know
hold
folks
back.
F
H
Proactive,
I
have
a
question
like:
did
you
notice
that
malicious
behavior
only
during
install
time
or
did
you
try
running
some
things
too,
like
like
importing
or
calling
the
api.
E
Or
something
great
question,
so
in
my
research
I
strictly
installed
the
package
that
to
me
was
kind
of
the
trust
boundary.
You
know
the
assumption
was
you
could
make
an
argument,
but
if
I'm
able
to
download
it,
then
I
can
at
least
see
what
it
is
that
I
would
be
installing
before
I
I
start
using
it.
So
there's
there's
a
chance
there
right.
E
E
The
biggest
scalability
issue
was
my
credit
card
and
the
it
does
not
scale
well,
not
the,
but
but
honestly,
you
know,
since
it's
not
running
in
workloads,
it's
running
on
ec2
hosts.
That
was
something
that
I
had
to
consider.
For
example,
on
the
tail
end
of
the
analysis,
there
were
some
weird
issues
where
some
packages
during
install
happened
to
hang
up
or
take
longer
than
expected,
and
then
that
had
some
downstream
effects,
I've
got
to
have
some
orphaned,
cystic
processes
that
I'd
have
to
go
and
clean
up.
E
Overall,
you
know
statistics
wise.
This
took
about
three
to
four
days
of
running.
You
know
the
hosts
running.
Turning
through
packages,
I
had
about
12
to
15,
like
micro
to
medium
ec2
hosts
running,
and
then
it
cost
me
about
120
bucks
all
in
I
think
so
so
scalability
wise,
I
think,
there's
always
the
opportunity
to
scale
that
middle
section,
the
actual
analysis,
but
that
was
running
on
all
the
packages.
If
we
think
about
what
a
continuous
pipeline
looks
like
those
needs
become
smaller.