►
From YouTube: OpenSSF Day at OSS NA - Education & Training for Secure Software Development & Distribution
Description
OpenSSF Day at Open Source Summit North America - - Education & Training for Secure Software Development & Distribution - David A. Wheeler, The Linux Foundation
A
No,
I
think
david
is
on
pretty
much
tries
to
make
the
rounds
of
every
working
group
very
busy
in
trying
to
help
us
in
open
source
security
and
also
a
topic.
That's
very
near
and
dear
to
my
heart.
Education,
david
wheeler,
is
the
director
of
supply
chain
security
for
the
linux
foundation
and
is
also
a
passionate
educator,
he's
written
several
books
and
papers
on
security,
and
he
lectures
on
developing
secure
software
at
george
mason
university
david
helped
lead
the
development
of
the
cii
best
practices
badge
which
is
now
branded.
The
openness.
A
B
Okay,
I
actually
think
I'm
we're
starting
slightly
early
is
that
okay,
we're
all
about
five
minutes
ahead.
Are
we
all
good
with
I'm
getting
the
thumbs
up?
Okay,
so
for
those
who
didn't
come
early
they're
going
to
miss
out
at
least
I'm
going
to
I'm
going
to
hopefully
provide
some
information
that
they
wanted
to
see
all
right.
B
So,
as
crow
mentioned,
I'm
david
a
wheeler
I'm
going
to
talk
about
education
and
training,
and
hopefully
this
will
be
more
than
just
education,
good
ignorance,
bad,
although
I
actually
believe
that
that's
true,
but
we
want
delvin,
because
education,
I
think
most
of
us
will
agree,
is
pretty
pretty
important,
but
let
me
drill
down
a
little
bit
further.
I've
read
the
abstract
for
this
talk.
Here's
what
I
said,
I'm
going
to
do,
and
hopefully
I'll
I'll,
be
able
to
do
that.
B
I'm
going
to
try
to
claim
that
many
software
developers
are
receiving
inadequate
education
and
training
on
how
to
develop
and
distribute
secure
software.
I'm
really
thinking
the
whole
mix.
You
know:
development
building
distribution
operations
and
so
on,
I'm
going
to
discuss
both.
Why
it's
the
problem
and
some
steps
that
we're
taking
to
correct
that,
but
I
can't
say
that
we've
got
everything
all
done
or
even
all
the
steps
all
completely
laid
out.
What
you're
seeing
is
the
bridge
being
built.
B
So
first
I
want
to
try
to
argue
and
convince
you
that
we
have
a
more
generally
a
serious
cyber
security,
skill
and
education
gap,
and
that
includes
the
knowledge
about
how
to
develop
secure
software
within
the
development
world,
and
you
know
I'm
a
big
believer
in
trying
to
get
actual
data
where
you
can
so
here's
some
interesting
data
points.
I
think
kind
of
justify
the
claim
so
cso
online
2016
said
that
46
of
organizations
said
they
had
a
problematic
shortage
of
cyber
security
skills.
B
Five
years
later,
you
can
see
it's
going
in
the
direct
direction.
Now
57
of
organizations
are
having
a
problem.
Great
okay,
now
granted
the
tech.
The
questions
weren't
worded
exactly
identical,
but
I
I
think
I
I
don't
think
the
overall
impression
is
wrong.
I
think
what
you're
seeing
right
now
is:
there's
already
a
skills
gap
and
we
have
more
demand
when
you
have
more
demand
and
not
enough
supply.
The
results
are
not
good
if
you're
the
one
trying
to
get
the
supply.
B
B
Ponemon
basically
has
a
report
in
2020
that
53
percent
of
developing
organizations
don't
ensure
that
they
have
development
on
training
on
what
they
call
secure
coding.
By
the
way
it
may
seem
strange,
but
I
actually
don't
I'm
going
to
use
the
phrase
secure
coding
in
several
places,
because
that's
what
my
sources
use,
but
I'm
actually
not
a
fan
of
that
phrase.
B
Coding
is
a
very
narrow
part
of
software
development,
okay
and
when
you're
doing
software
development,
it's
not
just
typing
in
text
into
a
screen.
Okay,
there's
a
whole
lot
more
that
goes
on.
You
have
to
figure
out
what
to
do.
Okay,
but
that's
okay.
This
is
what
my
sources
are
using:
2019,
there's
no
top
40
us
what
they
call
coding
or
top
five
non-us
computer
cert
computer
science
school
requiring
secure
coding.
B
Now
hey
in
2022,
we
had
a
slight
improvement.
Looking
at
all
of
the
top
24.
We
now
have
one
anybody
know
who
it
is:
I'm
sorry
name
names:
okay,
it's
uc
san,
diego,
so
good
for
you,
uc
san,
diego
shame
on
you
for
everybody
else,
shame
on
you
and
and
this
this
report
in
the
bottom.
Here
I
mean
there
you
can
see
the
report,
but
you
know
the
the
interesting
tag
poll
that
I've
phrased
I
pulled
out
is
universities,
don't
train
computer
science,
students
in
security
and
isn't
that
wrong?
B
B
You
know
things
that
well
that
didn't
involve
designing
circuits
yeah,
but
you
still
need
to
know
that,
and
I
think
security
nowadays
is
a
critical
part
of
knowing
how
about
software
development,
all
right,
lf
does
survey,
and
what
we
found
was
that
when
people
were
queried,
they
were
asked
hey,
which
of
these
things
would
be
helpful
for
improving
open
source
software
security
and
now
we're
narrowing
much
more
specifically
on
open
source
software
and
security.
Okay
and
people
are
allowed
to
answer
with
multiple
answers.
B
So
if
this
is
not
you're
not
going
to
see
a
total
of
100
percent,
the
top
three
well,
the
first
one
was
best
practices.
I'm
sure
kroger
is
very
unhappy
to
hear
that
number
two
was
tools:
okay,
but
number
three
in
terms
of
agreement
was
more
training
in
secure
and
memory
safe
programming
for
the
open
source
software
community.
So
there's
clearly
a
widespread
view
that
training
would
be
really
really
helpful.
B
Okay,
and
you
can
look
at
some
of
these
numbers
and-
and
so,
but
you
know,
there's
widespread
agreement
now
when
I
went
out
and
looked
and
hey
what
are
the
data?
What's
the
data
and
statistics
telling
me
there
was
one
study
that
at
first
seemed
to
contradict
this,
but
in
fact
I
think
what
it's
telling
is
a
story
of
of
efforts
trying
but
not
succeeding
to
deal
with
the
problem.
Okay,
if
you
go,
this
is
2022.
B
So
this
year,
secure
code,
warrior
survey,
reports
that
89
of
their
survey
takers
say
that
they've
received
sufficient
training
and
secure
coding
skills
89.
That
is
completely
different
from
what
everyone
else
is
saying.
Now,
whenever
you
do
a
survey,
it's
always
possible
that
the
respondents
set
is
very
different
from
everybody
else's
set.
Okay
and
that's
that's
a
notorious,
it's
always
a
challenge
whenever
you're
doing
research,
okay-
and
that
is
of
course
possible
for
this,
but
I'm
going
for
the
moment
just
to
accept
it
at
face
value,
because
indeed,
that's
a
problem
for
all
surveys.
B
So
what
does
this
say?
89
sounds
like
we're
done.
Well,
wait!
A
minute
mission
accomplished:
yeah!
Let's
stand
in
front
of
that
aircraft
carrier
all
right.
When
you
look
at
that
next
sub-bullet,
let's
look
at
what
they
mean,
they
say:
we've
had
sufficient
training,
but
most
of
them
are
not
familiar
with
the
common
vulnerabilities.
They
don't
know
how
to
avoid
them,
and
they
don't
know
how
they
can
be
exploited.
B
B
B
Maybe
this
is
just
a
very
narrow
understanding
of
coding,
but
guess
what
software
development
is
much
more
than
typing
text
or
drawing
arrows
okay,
81
percent,
that
they
said
they
do
regularly
apply
what
they
did
learn
now,
that's
impressive,
because
it
means
that
whatever
they're
getting
they're
at
least
finding
useful.
So
the
whole
idea
of
hey,
no
one-
will
use
this
stuff.
That's
obviously
not
the
case.
Okay
86
percent
said
that
it
was
challenging
to
practice
secure
what
they
call
secure
coding.
I'm
going
why?
B
Why
is
that
hard?
You
know
buffer
overflows,
you
know,
don't
go,
don't
write
past
the
buffer,
don't
read
past
the
buffer,
sql
injection,
there's
prepared
statements
and
other
kinds
of
things
you
can
use
they're,
not
hard.
Okay,
if
you're
having
that
much
trouble
you
didn't
under.
Maybe
you
got
a
course,
but
you
didn't
learn
what
you
needed
to
know
now.
It's
not
true
that
managers
don't
care.
Nowadays,
managers
are
saying
they're
a
lot
more
likely
to
hire
people
with
these
kinds
of
skills.
So
what
does
this
mean?
B
As
you
can
see
from
my
slide,
I
think
that
when
they
say
hey,
I'm
getting
sufficient
training,
it's
not
because
it's
sufficient
it's
because
they
have
ridiculously
low
expectations.
Okay,
I
think
what's
happening
now.
I
can't
from
just
the
survey
figure
out
what's
happening.
Okay,
I
can
simply
look
at
the
data
and
the
other
data
sets
and
try
to
figure
out
what's
happening,
but
I
have
a
story
that
I
think
I'll
call
this
hypothesis,
that
I
think
it
explains
it.
B
For
the
most
part,
our
schools
are
not
teaching
the
fundamentals
that
they
need
to
know:
okay,
they're
they're.
Our
educational
systems
are
failing
at
education.
So
what
are
our?
What
is
industry
they're,
trying
to
quickly
ramp
up
their
developers
by
giving
them
some
quick
little
courses,
maybe
an
hour,
or
so
maybe
a
nice
cute,
video
and
they're?
Finding
even
that
small
snippet
helpful?
They
are
using
it,
but
it's
not
enough.
B
They
don't
know
what
the
common
vulnerabilities
are,
for
example,
and
so
mary
simpkins
has
this
wonderful
phrase:
I'm
gonna
steal
it,
and
I
mean
I
give
give
her
credit
too.
Whether
she
wants
it
or
not,
but
edutainment
is
not
education.
Okay,
it's
okay!
Oh
thank
you
me,
I'm
not
against
entertainment,
not
at
all
the
more
interesting
we
can
make
our
educational
and
training
resources
the
better,
but
that
should
be
the
secondary
purpose.
The
first
is
aiding
learning.
B
They
did
say
that
one
of
the
top
ways
to
improve,
secure
coding
training
was
get
a
certificate,
basically
they'd
like
to
give
credit
now
again
this.
What
this
suggests
to
me
is
that
a
lot
of
organizations
are
trying
to
create
little
in-house
programs,
but
with
very
short
resources,
so
making
short
little
resources
or
hiring
out
for
a
very
short
course
and
frankly,
what
they're
getting
isn't
what
they
fully
need.
B
How
do
they
prefer
to
get
this?
They
want
codes,
they
want
to
see
code
samples,
they
want
to
see
annotations,
they
want
it
self-paced,
okay,
articles,
books,
okay
multimedia,
if
you
can,
but
they
that's
the
kind
of
thing
they're
looking
for
okay,
and
so
I
this
is
the
this:
wasn't
what
all
the
other
data
says
x,
this
seems
to
say
not
x,
and
I
think
what
this
gives
us
is
a
more
nuanced
understanding.
There
is
a
problem.
B
So,
let's
do
better,
okay,
but,
but
you
could
say-
and
although
I
most
people
don't
really
fight
that
hard
about
education,
there
are
some
folks
who
have
kind
of
questioned
it
and
that's-
and
it's
fair,
you
know
question
your
assumptions:
aren't
there
alternatives
because
educating
developers
that
takes
time
that's
work,
can't
we
do
some
other
things.
Well,
I
mean
some
we've
definitely
tried
other
things,
certain
large
organizations
who
will
remain
nameless.
You
know
you
can
probably
guess
them.
I
have
tried
the
word
just
to
take
some
insecure
software
and
configure
it.
B
B
Yes,
you
can
disable
some
very
insecure
protocols,
but
in
the
end
the
software
has
to
actually
be
designed
to
be
secure.
You
can't
just
twiddle
some
knobs
to
make
it
secure
what
about
training
the
end
users?
Well,
here's
a
problem.
End
users
actually
have
work
to
do
and
it
isn't
babysitting
insecure
software,
okay,
I'm
not
against
education
in
general
and
you're
sure
there
are
cases
where
you
can
at
least
try,
but
frankly,
you're
never
going
to
get
a
lot
of
satisfaction
of
trying
to
train
everybody
uses
a
computer.
B
B
Oh
you
don't
think
they'll
do
it
tomorrow.
You
know
they
will
and
they'll
do
it,
and
you
know
what
here's
the
dirty
secret,
if
you
can't
click
on
a
link
from
an
email
you
design
some
software
badly.
That's
what
links
are
for
okay
to
click
on
them.
That's
what
they're
for!
So,
if
you
can't
do
that
safely,
then
the
software
is
wrong,
not
the
user,
all
right!
Why
not
use
tools
now
here
we
are
getting
some
actually
a
fairly
good
retort.
Okay,
because
I
actually
do
believe
tools
are
absolutely
necessary,
but
they're
not
sufficient.
B
Okay,
false
positives
are
require
human
judgment.
Okay,
almost
every
tool
is
going
to
report
things
that
are
an
issue
from
the
point
of
view
of
the
tool,
but
are
not
really
security
issues.
Conversely,
pretty
much
all
tools
are
going
to
miss
some
things.
Okay,
and
this
means
we
have
to
have
human
judgment,
which
means
we
need
to
have
users
who
can
understand
what
the
tools
are
saying
or
not.
Saying:
okay,
there's
an
old
phrase:
the
fool
with
the
tool
is
still
a
fool
years
back.
B
B
No,
it
did
not.
There
were
two
cves
with
the
name
of
my
tool
in
there
because
it
found
the
vulnerabilities
and
they
didn't
know
what
the
what
the
tool
was
telling
them,
and
then
it's
not
that
the
tool's
hard
to
use
it's
that
you
have
to
the
tool
on
the
other
end
of
the
eyeballs
needs
to
be
ready
all
right.
So
why
not
just
use
secure
by
default
apis
now,
here
we
are
in
total
agreement.
B
We
should
as
much
as
possible
make
things
secure
by
default,
both
for
end
users
and
for
developers,
but
the
problem
here
is:
who
do
you
think
makes
these
apis?
They
do
not
fall
from
the
heavens.
Some
developer
had
to
figure
out
the
apis,
and
if
a
developer
has
to
figure
out
an
api,
they
have
to
know
what
security
is,
so
they
can
make
an
api
that's
secure
by
default.
So
we're
back
to
you,
it's
not
that
this
is
wrong,
but
it's
insufficient.
B
We
need
education
and
really
yes,
we
do
need
tools,
we
need
secure
by
default
apis,
but
we
also
need
education
in
order
for
those
other
aspects
to
work
effectively.
They
work
together,
not
in
opposition
and
really,
I
would
also
say,
prepare
for
the
future.
Attackers
are
intelligent
adversaries
if
you
block
off
one
path,
they're
going
to
look
for
another,
and
so
we
need
to
have
intelligent
defenders
to
deal
with
that
with
intelligent
adversaries,
all
right.
B
B
There
are
basic
principles
that
are
many
decades
old
now
from
salzer
and
schroeder.
They're
still
true,
they're
going
to
be
true
next
10
years
next,
20
years
next,
30
years
too,
so
we
shouldn't
developers
should
know
them
using
accept
lists,
not
deny
lists.
You
should
know
what
the
most
common
common
kinds
of
vulnerabilities
are
and
how
to
prevent
them
from
happening.
Okay,
does
you
know
if
you're
go
if
your
software
is
going
to
have
vulnerabilities
at
least
make
it
a
whole
new
kind?
No
one's
ever
seen
before.
B
B
B
Now
there
the
course
is
free,
but
there
is
a
time
limit
and
if
you
want
to
get
a
certification
of
completion
that
does
cost
money.
This
is
basically
how
adex
works
and
that's
fine,
but
a
lot
of
people
are
saying
that
hey
we're
concerned.
You
know
if
that
those
costs
of
certifications
are
potentially
a
blocker.
We
want
to
encourage
people
to
be
able
to
prove
that
they
took
the
course.
So.
On
march
this
year
we
made
the
material
also
available
on
the
lf
training
and
certification
platform.
B
We
want
this
information
to
get
out
to
all
sorts
of
folks,
so
we're
gonna,
we
plan
to
support
both
platforms
and
I
have
a
cool
new
announcement.
Oh
okay,
so
some
of
you
already
know
about
this.
Okay,
but
all
right
so
as
of
today,
I
want
to
announce
that
we're
going
to
add
and
yet
another
way
people
can
get
access
to
this
course
material.
Something
called
scorm
connect.
Those
who
are
really
into
the
learning
systems
will
already
know
what
that
is.
B
So,
if
you're
in
a
large
organization
or
university,
you
can
just
click
on
it
through
their
system.
It
looks
like
it's
embedded
as
part
of
theirs,
but
in
fact
it's
the
material
that
we've
already
developed.
This
is
going
to
be.
This
is
free
for
open,
ssf
premier
members,
if
you're,
not
an
open,
ssf
premier
member,
but
a
member,
you
can
talk
to
us
later
and
it'll.
Also
we're
also
going
to
make
it
free
to
accredited
educational
institutions,
because
we
want
to
get
this
out.
Other
organizations
can
absolutely
include
it.
We
do.
B
We
would
love
for
lots
of
folks
to
incorporate
we're
trying
to
get
this
information
out
in
as
many
different
forms
and
ways
as
we
can.
It
is
going
to
be
an
annual
fee
for
other
organizations
because
it
actually
turns
out
to
be
costing
a
non-trivial
amount
for
this
sort
of
stuff.
So,
but
this
this
is,
this
is,
as
far
as
I
could
get
people
to
swallow
the
thing
paying
our
our
costs.
But
that
said,
we
don't
feel
like.
If
you
don't
fit
those
categories,
you
can't
do
this
absolutely
come
talk
to
us.
B
Okay
again,
this
provides
another
mechanism,
so
developers
can
learn
about.
This
enables
any
integration
the
course
is
going
to
continue
to
be
free
via
lf's
training,
certification
and
the
edx
platforms.
So,
if
you're
an
individual
or
small
business,
you
don't
have
one
of
these
systems.
Don't
worry
about
it.
Just
take
one,
you
know
through
the
mechanisms
we
already
have
we're,
not
removing
anything,
we're
adding
something.
B
Okay,
quick,
quick
technical
explanation
for
those
of
you
who
are
really
into
this
in
the
educational
world.
Scorm
is
a
common
format
for
training
materials.
It's
been
around
for
a
long
time.
If
you
put
the
learning
materials
quizzes,
everything
else
in
that
kind
of
form,
scrum
connect
is
a
weird
form
of
the
scorm
file,
it's
a
standard
format
and
it
omits
all
the
content.
Now.
How
does
that
work?
Well,
what
it
has
is.
It
has
a
little
redirect
that
says:
hey
to
get
the
content
go
over
here
to
this
website.
Okay,
now,
why
do
that?
B
Because
that
way
we
can
update
the
material
and
instead
of
trying
to
convince
people
to
update
their
material,
we
can
rapidly
update
material
as
new
information
or
as
fixes
become
available.
Okay,
so
all
right,
those
are
no
that's
not
by
only
means
the
only
educational
resource.
Okay
owasp
has
the
security
knowledge
framework.
The
open
ssf
has
actually
provided
some
funding
to
that
group,
and
this
skf
is
a
different
approach,
although
they
actually
embed
the
course
within
their
materials.
B
The
other
course
I
just
mentioned,
but
the
what's
interesting
about
them,
is
they
do
hands-on
code
experience.
The
pro,
of
course,
is
when
you
do
hands-on.
You
learn
better.
The
the
con
is
that
that
takes
a
lot
longer,
so
you
know
there's
pros
and
cons
to
all
sorts
of
approaches,
but
you
know
that's
that's
a
a
pro
if
you're,
if
you're
willing
to
put
in
the
extra
time
a
safe
code
is
some
very
nice
training
materials,
there's
various
expensive
materials.
B
I
should
note
related
that
the
lf
plans
to
soon
release
other
courses
on
six
store
and
devsecops.
I
can't
say
more
yet,
but
it's
coming
it's
coming.
Okay,
now
earlier
brian
mentioned
the
various
street,
the
10
streams
in
the
mobilization
plan
stream,
one
was
specifically
about
development,
education
and
certification.
Okay.
So
what
I
just
talked
about
the
course
is
a
is
a
part,
a
a
support
for
that
broader
goal
and
here's.
B
If
you
look
at
the
stream
here's
some
of
the
things
that
it
mentions,
basically
relatively
few
software
developers,
get
serious,
formal
training.
You
know
that
survey
notwithstanding,
I
think
those
don't
count
for
many
cases.
If
you
don't
know
what
the
common
vulnerabilities
are.
You
got
a
problem,
you
know
a
small
amount
and
ideally
more-
and
there
are
some
modules,
training,
training
modules
available,
free
like
ours
and
the
os
one,
and
basically
we
want
to
make
things
better.
So
there's
three
parts
to
it.
The
first
part
is
collecting
and
curating
content.
B
Let's
get
what's
already
available,
okay
and
make
it
easy
to
find
easy
to
use
work
it
together.
We
very
much
tend
to
intend
to
build
on
the
the
course
I
just
mentioned
the
secure
software
development
fundamentals,
courses,
but
absolutely
add
and
do
gap.
Analysis
second
part
is
to
expand
training,
okay,
where
we
want
to
deliver
training
across
industry.
B
B
B
When
we
first
publicly
released
the
mobilization
plan,
we
intentionally
had
breakouts
to
discuss
each
of
the
streams
to
get
feedback.
Thank
you
on
those,
and
you
know
the
one
one
of
the
ones
that
we
had
was
a
discussion
about
stream.
One
lots
of
great
comments,
one
that
particularly
struck
me-
I'm
not
sure
how
to
deal
with
it,
but
I
find
it
very
intriguing
was
really
a
push
to
say:
hey
wait
a
minute.
B
Most
software
developers
learn
a
lot
of
how
to
develop
software
before
they
even
hit
college
or
universities.
If
they
go
to
them
at
all-
and
I
there
are
various
numbers
but
a
vast-
if
you
know
a
vast
number
somewhere
around
half
of
software
developers,
don't
go
to
universities,
new
colleges,
at
least
for
learning
how
to
develop,
secure
software
it
may
develop.
Software
may
even
be
much
larger,
but
you
know
it's
a
significantly
large
number
and
whether
it's
the
majority
of
minority
doesn't
matter.
There's
a
lot
of
folks.
B
Maybe
we
can
find
ways
for
governments
to
incentivize
this
okay,
and
in
fact
I
mentioned
somebody
earlier,
you
know
translations
for
international
consumption.
You
know
you
know,
I
I
learned
a
little
french
and
the
main
thing
I
learned
is
that
nobody
wants
to
hear
my
attempts
at
french,
so,
okay,
so
so
we
we
really
want
to
make
this
international
and
that
and
part
of
that
is
going
to
need
to
be
figuring
out
how
to
get
translations.
B
So
if
you
are
interested
in
improving
the
world
of
having
software
developers
educated
and
trained
in
how
to
actually
write
secure
software,
what
do
you
do?
Well,
hey?
If
you
develop
software,
there's
an
easy
one.
Take
a
course
at
least
one
okay.
You
know,
software
development
is
a
is
a
as
long
as
you're
gonna.
Do
software
development,
it's
a
lifelong
commitment
to
learning.
That's
not
necessarily
a
bad
thing,
but
if
you're
gonna
develop
software,
you
need
to
learn
how
to
develop
secure
software.
You
know:
go
that
link.
B
Take
the
course
you
want
to
do
the
osskf,
that's
great,
there's!
Others
too.
Obviously
I
developed
one
of
the
courses,
so
I
like
my
course,
but
you
know
what
I'm
way
more
concerned
about
taking
a
course.
I
don't
care
if
you
take
that
particular
one,
I'm
much
more
concerned.
Do
you
know
what
you
need
to
know,
work
with
and
oversee
if
you
work
with
and
oversee
those
developing
software,
encourage
them
to
learn?
Okay,
if
you
manage
other
software
developers,
you
you
probably
already
know
that
this
is
an
issue.
Well,
you
know.
B
Yes,
it's
a
pain
to
stop
for
the
moment
and
learn
something,
but
it
pays
dividends
over
and
over
and
over
again
improve
existing
training
materials.
Okay.
So,
for
example,
if
you
want
to
improve
the
course,
I
just
talked
about
it's
on
github,
okay,
it's
cc
by
file
an
issue
or
even
better
a
pull
request.
B
Okay
and
in
fact
one
thing
interesting
thing:
we've
actually
been
talking
about
for
several
months
about
how
to
make
some
interesting
little
pictures
to
jazz
it
up
and
maybe
help
explain
better.
So
I
just
got
permission
this
morning
from
open
ai,
to
use
dolly
to
create
some
graphical
images
that
we
hope
will
clarify
some
of
the
concepts
that
are
taught
in
the
course
will
it
work.
B
I
don't
know,
but
we're
going
to
do
an
experiment
and
find
out-
and
I
think
that's
really
key
is:
let's
look
for
ways
to
try
to
make
things
better
repeat.
If
the
experiment
doesn't
work
out,
okay,
you
learn
something,
but
if
it
does
work
out,
you've
made
an
improvement.
If
you
do
enough
of
those,
the
results
are
going
to
be
better
for
everyone.
B
If
your
organization
has
a
learning
management
system,
consider
adding
the
open
ssf
course,
okay,
you
don't
have
to
be
at
credit
university
or
an
open,
ssf
premier
member,
but
if
you
are
then
hey
it's
free
and
if
you're
not
come
talk
to
us
anyway.
Okay,
because
we
would,
we
would
like
to
see
more
and
more
developers
know
understand
this
material,
this
kind
of
material
and
finally
open
ssf
best
practices.
B
Working
group
is
creating
within
it
an
education
sig
and
the
idea
is
to
convert
these
higher
level
ideas
into
a
plan
and,
more
importantly,
implement
the
plan.
There
are
way
too
many
people
who
think
that
planning.
If
do
you,
do
create
the
plan
and
then
you're
done,
please,
no!
Okay,
in
fact,
whatever
you
end
up
with,
is
probably
going
to
be
different
than
your
plan
and
that's
all
right.
B
B
B
A
B
B
B
I
see
allen's
very
happy
about
that.
I
will
quickly
note,
though,
that
just
having
an
s-bomb
does
not
suddenly
make
things
secure
and
I'm
I'm
sure,
you'd
agree
with
that
too.
You
you've
got
to.
Actually
you
know
an
s-bomb
simply
gives
you
a
list
of
materials
when
you
receive
it
you're
going
to
have
to
do
something
with
that
information
like
look
at
it
and
you
know,
run
a
tool
and
say:
hey:
are
there
known
vulnerabilities?
Do
I
should
I
worry
about
those,
and
you
know,
are
some
of
these
components?
Hideously
old?
B
B
The
challenge,
more
generally,
it's
very,
very
difficult
to
capture
metrics
people,
don't
want
to
report
a
lot
of
things,
so
we
don't
have
like
you
know
before
and
after
analysis.
That
kind
of
thing
it
wouldn't
be
a
bad
idea,
but
I'll
be
honest
right
now.
This
the
state
of
education
is
so
woefully
bad
that
or
actually
basically
non-existent
for
most
developers,
or
at
least
you
know
it's
on
the
level
of
I
took
a
course,
but
I
don't
know
what
the
common
problems
are,
and
I
don't
know
how
to
deal
with
them.
B
So,
given
that
I
don't
think
right
now
we're
very
worried
about
it.
The
state
is
so
bad
that
anything
isn't
going
to
be
improvement,
and
so
we
haven't
focused
on
that,
but
I
do
think
as
we
as
we
go
on
forward,
it
would
be
great
to
try
to
measure
you
know.
You
know
a
b
testing.
You
know
we
do
it.
This
way,
do
that
which
is
more
effective,
but
we
haven't
made
any
plans
for
that.
Yet,
okay,
I
think
we're
you
know.