►
B
A
C
C
E
D
A
A
E
A
A
F
A
A
F
C
A
A
A
I
C
A
A
I
A
A
A
E
E
So
what
I
was
suggesting
was
we
basically
set
up
some
metrics
to
make
sure
that
this
process
is
delivering
on
the
goals
that
we
we
set
out
to
solve,
and
it
would
give
us
kind
of
an
indication
of
you
know.
Is
this
working?
Are
there
adjustments
that
we
need
to
make,
and
if
we
do
make
adjustments,
then
you
know
we
can
measure
whether
or
not
they
were
effective
or
not.
E
If
we,
you
know
anything
from
a
larger
scale,
change
to
something
as
small
as
you
know,
let's,
let's
change
the
text
in
the
emails
that
we're
sending
out.
Let's
change
the
frequency
of
of
how
we're
basically
contacting
people,
but
just
something
that
that
will
help
us
Baseline
where
we're
starting
at
and
then
you
know
it
sets
us
up
for
future
changes
such
that
that
we
always
kind
of
know
are
we
are
we
making
this
process
better
over
time
or
you
know
are,
do
we
do?
We
need
to
re-examine
kind
of
where
we're
at.
A
E
Yeah
and
I
I
think
you
know,
but
what
I
don't
think
should
happen?
Is
approval
of
the
metrics
necessarily
needs
to
block
the
specification
process
right?
You
know
if
it
is
better
lived
in
another
doc
or
you
know
you
can
basically
say
this
is
just
additional
information
on
how
we're
going
to
track
this
process.
E
E
C
E
E
I
So
these
I
don't
want
to
get
too
much
in
the
weeds
of
the
metrics,
because
metrics
can
always
go
down
a
rabbit
hole,
but
these
are
more
focused
on
success,
but
we
have
we
can
gain
learning
objectives
from
failures,
so
to
speak,
lack
of
a
better
term.
So
should
we
also
include
metrics
on
not
resolved
vulnerabilities
but
closed
without
reason,
or
just
identifying
those
that,
like
pager's,
not
working
with
us
and
they
just
closed
it
because
they
wanted
to
close
it.
I
A
E
A
E
A
I
I,
don't
remember
the
process
from
the
top
of
my
head,
but
I.
Remember
that
there
was
you
can
do
it
through
the
GitHub
or
the
the
source
code
management
system,
there's
also
email.
Do
we
want
to
take
account
of
disclosures
that
our
work
through
emails?
Are
we
just
going
to
be
limiting
to
just
the
source
code,
Management
Systems.
A
But
the
the
Well,
if
they
have
pmpvr
enabled,
then
you
don't
use
email,
but
in
all
the
cases
where
email
is
not
is
is
where
pmpavr
is
not
enabled.
You
both
open
an
issue
asking
that
they
enable
pmpvr
and,
and
you
send
them
an
email
with
the
vulnerability
so
for
context
to
those
who
that
are
new
pmpvr
is
defined
as
up
here.
A
A
A
A
E
A
Right
I
agree:
I,
don't
want
to
overdo
it
and
also
the
more
you
have
to
add
them
graphs
and
table
and
stuff
like
that,
yeah,
okay,
there
was
a
general
idea
of
we
wanted
people
to
take.
Have
the
opportunity
to
go
through
so
I
was
going
to
send
an
email
to
the
working
group,
the
larger
working
group,
and
ask
them
to
begin
to
review
this
document.
Did
anybody
have
any
direct
feedback
on
areas
that
we
think
that
were
missing?
A
A
Far-
and
these
are
the
set
of
things
that
you
must
do-
we
have
this
disclosure
flow
chart
which
has
been
I
thought.
It
was
a
topic
for
quite
a
few
meetings,
which
just
shows
like
it
in
different
cases
how
vulnerability
disclosure
would
occur
based
upon
various
conditions,
then
further
down
here
we
have
some
scenarios
that
walk
through
like
this
flowchart
and
explain
what
decisions
will
be
made
based
upon
what
scenarios
down.
A
Here
we
have
information
about
the
patch
and
and
the
code
that
we,
you
know,
let's
talk
about
the
code
that
and
how
it
should
be
fixed
requirement
for
external
review,
that
you
must
offer
disclosure
assistance,
there's
messaging
requirements
for
opt
out
discussions
about
the
vulnerability
fixed
origin
and
who
must
be
generating
the
pull
request,
compute
requirements
to
maintain
communication
responsiveness
with
the
maintainer,
the
ability
to
redeploy
the
changes.
You
know
read
it
rebase
or
redeploy
the
changes.
A
G
A
G
A
This
work,
these
campaigns,
the
behavior
behind
the
opt-out,
is
not
that
they
wouldn't
receive
it's,
not
that
they,
the
the
vulnerability,
wouldn't
be
identified
and
fixed.
It's
just
that
the
maintainer
would
not
be
would
not
be
reported
to.
They
would
not
receive
the
the
vulnerability
disclosure
information
at
all
like
we,
wouldn't
we
wouldn't
ping
them
at
all.
A
It
I'm
just
like
we
had
some
pops
last
week
about
like
a
standardized
flag
in
the
security.md.
We
had
some
thoughts
about
baby,
adding
a
flag
to
the
get
ignore
or
the
get
attributes
file.
I
have
the
GH
robot
standard
that
I
kind
of
created
when
I
started
doing
this
work
way
back
in
the
day,
I'm
wondering
if
we
have.
A
I
E
A
E
I'll
also
add
I
think
it
makes
kind
of
getting
consensus
a
little
easier
when
you
know
if
people
have
different
directions,
they'd
like
to
see
you
know,
implement
it
over
time.
Just
saying
this
is
our
kind
of
our
first
pass
at
it.
This
is
what
we're
going
to
do
initially,
like
we
don't
know
everything
that'll
be
part
of
part
of
us
trying
this.
C
I
B
A
D
A
C
A
A
On
the
other
hand,
on
the
other
hand,
a
simple,
this
is
public,
now
seems
courteous
and
could
help
reduce
prizes
do
not
engage
in
any
way
I'm.
Just
you
know,
I'm
just
concerned
is
like
they've
opted
out
right,
they've
told
us
go
away
like
I
feel
like
that's
like,
if
you,
if
you
ping
them
by
email
or
via
an
issue
tracker
just
a
great
way
for
you
to
get
reported
to
GitHub,
when
they've
explicitly
told
you
to
go
away.
A
So
art
is
saying
that
we,
so
if
you've
opted
out,
you
should
still
send
them
a
simple.
This
is
now
public
message.
When
you
make
it
public,
it
seems
courteous
and
could
help
reduce
surprises,
but
not
turn
not
to
engage
in
any
other
way
in
the
process
and
I'm
I'm
of
the
opinion
that
if
the
maintainer
has
told
you
not
to
not
to
like
they've
opted
out,
then
they
like
it's
like.
I
I
I
A
So,
yes,
I
am
fine
with
that,
but
my
intention
is
communicating
in
the
opt
out
right
like
you,
you
send
them
a
message,
an
email
or
whatever
like
okay.
If
you
want
to
opt
out
in
this,
the
future,
here's
how
to
do
that,
but
included
in
that
original
message,
is
here's
the
implication
of
that
right.
Making
that
very
clear.
So
there's
no
surprises,
though,
instead
of
being
like
reactionary,
hey
you've
opted
out.
E
Could
you
just
do
it
both
times
I,
like
the
proactive
approach
too
I
think
giving
someone
one
last
kind
of
hey.
This
is
happening
is
also
helpful.
You
know,
I,
there
are
people
who
are
at
varying
levels
of
being
on
top
of
their
issues
and
emails
and
I
think
that
two
is
is
giving
everybody
a
better
chance
than
one
shot.
E
I
I
see
where
you're
drawing
the
comparison,
because
they're
both
emails
that
hit
you
multiple
times,
I
think
this
one's
a
lot
more
personal
and
has
you
know
implications
that
you
know
me
getting
spammed
by
marketing?
Does
not
I
also,
you
know
I
think
by
its
nature.
It's
not
quite
a
marketing,
email
and
I.
Think
I,
I,
guess
I've
also
watched
people
to
the
point
of
you
know
like
ascending,
is
right:
A
lot
of
people,
don't
read
their
emails
or
you
know
they.
E
If
they
do
it's
very
spotty,
I
I
think
I've
watched
people
say
wow.
Why
didn't
you
tell
me
and
you're
like
well
I
sent
you
that
one
email,
you
know
I,
think
your
your
probability
of
surprises
goes
down
just
by
increasing
or
rather
going
from
one
email
just
to
two
and
I
I
think
it'd
be
different
if
you're
emailing
them
on
a
weekly
daily
monthly
basis,
without
with
them
saying
I'm
trying
to
opt
out.
E
I
A
So
if
you've
opted
out
is
fine,
do
you
send
them
a
message,
letting
them
know
that
they
have
missed
a
vulnerability
that
is
now
being
disclosed
publicly
for
every
vulnerability
you
disclose
or
do
you
do
it
for
only
the
first
one
and
then,
if
they
remain
it
or
keep
an
opt
out
enabled,
then
then
you
don't
let
them
know
for
any
case
in
the
future
for
any
future
vulnerabilities
that
you're
you
let's
say
you
run
like.
Let's
say:
you're
fixing,
zip.
A
I
If
at
that
point,
if
they
opted
out,
then
they
should
just
go
ahead
and
we
make
it
a
public,
but
I
think
at
the
first
instance
where
they're
like
I'm
gonna,
opt
out.
It's
like
this
is
the
implementations
of
you
opting
out
as
far
as
all
security
vulnerabilities
that
we
will
be
reporting.
It
also
brings
them
that
awareness
of,
like
it's
not
just
this
one,
it's
going
to
be
all
of
them
and
then
they
can
consider
from
there.
A
A
F
A
B
I
I
I
D
A
A
J
A
So
that,
just
for
everybody
else,
the
topic
is
about
trying
to
make
the
account
that's.
Creating
the
polar
Price.
Look
like
an
individual
instead
of
a
bot
or
if
it
is
a
bot
like
buy
account,
make
it
look
like
it's
a
legitimate
account
not
like
some
want,
like
doesn't
have
a,
doesn't,
have
a
profile
picture
sort
of
account.
Yada
yada
like
make
it
look
like
it's
a
legitimate
well-intentioned
account
that
may
be
a
bot,
maybe
operated
by
a
bot
but
yeah
like.
A
A
Well,
that's
another
point
that
I
am
not
necessarily
in
agreement
about
I.
Think
that,
theoretically,
if
the
bot
account
at
mess
or
at
lowers
the
operator,
then
both
people
will
get
the
notifications.
So
the
responses
don't
necessarily
need
to
come
from
the
bot
account.
They
can
come
from
the
operator
as
long
as
there's
a
system
in
place
such
that
you
get
notifications
that
that
there
are
so.
A
A
A
B
D
A
J
C
A
Okay,
any
other
comments:
hey
Arts
there
Brian,
you
left
a
comment
on
March
3rd
about
automatically
fixing
so
avoiding
Pat,
avoiding
patching
test
code
exclusively.
So
this
is
written.
Automatic.
Fixed
campaigns
should
attempt
to
avoid
patching
test
code
exclusively.
If
a
production
source
code
is
modified,
then
it
should
also
be
applied
to
test
code
as
well.
But
if
the
vulnerability
only
exists
in
test
code
generation
of
an
automated
fix
should
be
avoided
or
should
you
know
should
not
be
contributed,
something
should
not
be.
A
A
J
A
J
A
C
D
A
J
No
I
don't
want
to
add
too
many
different
Gatekeepers
on
on
a
document
that
I
think
we
can
still
like,
let's,
let's
not
try
to
go
for
an
acronym
or
anything
to
cutesy
I,
do
think
this
is
a
specification,
but
we're
not
going
to
be
setting
ourselves
up
to
certify
other
other
groups.
Doing
this
or
or
you
know,
passing
judgment
on
them
in
any
other
way,
so
the
word
compliant
does
does
throw
me
a
little
bit.
I.
J
A
A
J
J
This
is
a
little
bit
more
like
guidance
for
how
to
run
coordinated
vulnerability,
disclosure
processes
right
because
so
much
here
is
left
as
a
should.
It's
it's
kind
of
like
I.
Don't
know
that
even
a
third
party
would
really
develop
a
certification
process
around
this,
so
so
just
I
was
thrown
for
the
word
compliant
as
a
part
of
this,
because
that
that
does
suggest
a
certain,
a
certification
regime
that
we're
not
committing
to.
C
A
A
C
A
And
they
we
would
like
to
be
able
to
offer
a
title
associated
with
that,
just
like
the
best
practices
badge
has
like.
You
know
your
you
know,
87
compliant
with
the
best
practices
badge
or
whatever
like
we
would
like
to
be
able
to
say
if
you
follow
these
steps,
you
are
compliant
with
this.
With
this
spec.
J
J
I
suggest
bumping
it
up
to
the
to
the
working
group
and
ceasing
David,
wheeler
and
saying
somewhere
on
the
spectrum
between,
like
the
the
document
that
we
published
for
guidance
for
vulnerability,
disclosure
processes
and
something
a
little
bit
more
rigorous
or
or
you
know,
testable
you
know,
or
or
like
the
the
best
practices
badge
somewhere.
We
just
need
a
term
and
it'd
be
nice
to
harmonize
those
terms
across
the
open
ssf
for
for
the
level
of
rigor
or
the
level
of
seriousness.
B
C
J
Here's
here's!
What
I
don't
want
is
a
third
party
organization
running
an
automated
fix
campaign
and
telling
its
users
don't
worry.
We
use
the
open,
ssf
guidance
because
that
by
using
our
name
when
they
run
their
campaign,
people
who
receive
these
automated
messages
are
going
to
be.
You
know
seeing
our
name
associated
with
something
that
could
be
pretty
annoying.
Even
if
we've
carefully
calibrated.
You
know
a
what
what
you
know
for
for
a
non-annoying
thing:
I'm
nervous
about
the
use
of
the
open,
ssf
brand
for
third
parties
acting
on
on
their
own.
J
You
know,
and
and
merely
self-attesting,
to
this
kind
of
thing.
I
want
to
reserve
that,
for
when
we
run
our
campaigns,
so
I
think
this
looks
a
little
bit
more
like
the
here's.
How
to
here's
our
suggestions
for
how
you
should
run
your
vulnerability,
disclosure,
coordinated
vulnerability,
disclosure
processes
as
an
organization.
Here's
our
recommendation
for
people
who
do
want
to
run
automated
vulnerability,
fixed
campaigns,
the
things
you
really
you
know
you
really
must
do.
I
mean,
must
and
must
not
and
outside
of
a
compliance
regime
is,
is
questionable.
J
A
So
that
was
the
the
intention.
Was
we
write
this?
We
run
it.
We
run
it
by
one
of
those
closures
working
group.
We
get
approval
that
they
sign
off
on
it
and
then
also
say:
hey
Tac.
Are
you
fine
with
people
putting
a
badge
with
this
name
on
it?
You
know
on
their
work,
stating
that
they
are
compliant
with
this.
These
set
of
practices
these
these
protocols
that
we've
defined
that.
A
A
C
A
B
J
B
A
So
we
will
be
meeting
in
three
weeks
and
then
Defcon
is
happening
and
hacker
summer
camp
and
I
presume
most
of
us
will
be
out,
so
we
won't
be
attending.
We
won't
be
meeting
then
so
it'll
be
three
weeks
out
and
then
a
week,
Gap
and
then
two
weeks
and
then
then
we'll
be
back
to
a
regular,
regular
Cadence,
sound
good.