►
From YouTube: IETF102-SAAG-20180719-1330
Description
SAAG meeting session at IETF102
2018/07/19 1330
https://datatracker.ietf.org/meeting/102/proceedings/
C
A
A
A
E
F
Didn't
send
it
we're
meeting
in
the
next
session.
Please
do
join
us,
we're
quite
close
on
all
the
informational
documents
moving
forward
either
either
in
working
the
last
call
or
ready
to
be
sent
to
the
isg
and
on
the
2
protocol
drafts.
We're
gonna
work
in
group
last
call
both
of
them
one
for
the
second
time
and
one
for
the
first
time
so
making
good
progress,
cool.
A
G
G
H
I
A
E
Yeah
video
for
Deutsche
Telekom
I,
wonder
you
have
cider
on
the
list,
which
essentially
is
dead
for
a
year
and
the
interesting
thing
that
happened
with
relevance.
There
obviously
has
gone
under
the
radar
for
security
purposes
and
I
wonder
that
the
cider
ops
is
not
on
your
radar
were
the
operational
use
of
the
stuff
that
came
out
of
cider
is
happening
and
I
wonder
whether
that
is
worth
your
attention.
J
A
K
Poll
Hoffman,
chair
of
DRI,
you
bought,
which
is
meeting
in
the
next
session.
So
for
those
of
you
who
have
not
been
following
DOE
and
such
like
that
one
of
the
things
has
come
up
is
that
currently
with
DHCP,
you
connect,
and
it
says
here's
your
here
are
the
IP
addresses
of
your
DNS
resolver
and
those
are
always
over
port
53.
K
For
a
few
years,
we've
had
DNS
over
TLS,
which
runs
on
a
different
port,
but
there's
really
no
way
for
DHCP
to
tell
you
that
so
one
of
the
things
that
DRI
you
will
be
doing
is
how
do
you
do
that
or
how
do
you
get
doe?
But
the
other
thing
is:
let's
say
that
you
have
you've
got
two
things
back
from
DHCP,
one
that
says:
here's
a
plain
port,
53
server
and
here's
an
853
server,
which
you
know
has
some
encryption
privacy
and
such
like
that.
K
How
do
you
rank
them,
so
the
DRI
you
bought,
which
is
in
the
next
session,
is
basically
about.
How
do
we
start
thinking
about
it's,
not
a
working
group
forming
boff.
We
have
a
couple
of
drafts,
but
nothing
big,
but
if
you're
concerned
with
these
kinds
of
things
like
you
know
what
happens
here,
two
different
trust
anchors
you've
got
DNS
SEC
and
you've
got
the
web
PKI
and
such
like
that.
Come
to
the
boss,
thanks
Paul.
A
L
B
M
Well,
so
it
works
so
I'll
just
hold
it
I'm
David
makrooh
from
Cisco.
This
is
a
joint
presentation
with
the
passo
Vaslav
from
nist,
who
will
be
taking
over
about
five
minutes
into
the
presentation
we're
going
to
be
presenting
to
you
today
on
something
called
the
automated
crypto
validation
protocol.
So
this
is
joint
work
between
our
organizations
and
a
bunch
of
other
people.
So
let
me
give
you
some
background
and
you
might
be
already
familiar
with
this.
M
If
you
work
with
cryptography,
but
I
want
to
talk
about
cryptographic,
modules
and
their
validation,
so
cryptographic
module
is
an
encapsulation
of
all
the
functionality
you
need
to
do
to
actually
successfully
do
cryptography,
encryption,
authentication,
digital
signatures,
key
management,
entropy.
All
the
ancillary
functions
all
that
stuff.
M
You
know
it's
encapsulated
in
a
module
right,
so
if
you
want
to
do
cryptography,
you're
gonna
need
that
and
if
you're
either
producing
one
of
these
modules
or
you're
using
one,
you
want
to
either
gain
an
assurance
that
it
was
correctly
implemented,
or
you
want
to
provide
an
assurance
that
it
was
correctly
implemented
right.
So
you
know
testing.
The
correctness
of
this
thing
is:
what's
done
in
validation,
so
before
you
validate
it,
you
have
to
actually
know
what
it
implements
right.
M
So
the
the
validation
process
consists
of
something
like
a
set
of
known
answer
tests
that
you
can
provide
to
it.
Instead
of
you
know,
the
known
answer
test
consists
of
a
query
and
a
response,
and
then
there's
you
know
randomized
tests
and
if
there's
a
remand,
amaizing
algorithm,
then
testing.
It
is
a
bit
more
involved.
M
The
most
widely
known
validation
program
is
FIPS
140,
which
NIST
runs
on
behalf
of
the
US
government,
and
so,
if
you've
ever
implemented
a
crypto
module.
There's
a
good
chance.
You've
participated
in
the
fifth
140
program
with
their
Cryptologic
validation
program
and
they
have
a
process
for
doing
what
I
just
described.
That's
essentially
manual
right
that
there
is
the
implementer
provides
a
PDF
file
that
itemizes
the
algorithms
and
parameters
and
so
on.
M
Al
am
acting
as
an
intermediary
between
the
vendor
and
NIST,
we'll
provide
a
set
of
test
cases
as
use
text
files,
and
then
the
implementer
Maps
these
into
the
crypto
module
and
gets
the
set
of
outputs.
And
then
you
iterate
on
that
so
you're,
all
your
outputs
are
correct
right,
so
I
have
a
link
if
you've
never
seen
this
before
you
can
click
on
the
link.
Once
you
grab
the
slides
and
look
at
the
certificate
for
open
ssl
on
SUSE
Linux,
just
as
a
nice
random
example.
So
what
problem
are
we
trying
to
solve
right?
M
If
there's
validation
program
exists
already,
you
know
what
why
do
we
need
anything
different?
Well,
it's
because
of
the
the
delay
the
lag
between
when
you're
done,
implementing
a
crypto
module
and
when
it
can
actually
come
out
the
end
of
this
process
right
because
there's
so
many
manual
steps
that
it
typically
takes
a
long
time
and
so
you're
speaking
as
a
vendor.
M
The
last
thing
I
want
to
do
is
force
my
customer
to
choose
between
running
a
validated
crypto
module
and
running
a
past
crypto
module
right.
So
if
either
there's
a
software
update
or
a
bug
fix
or
whatever
you
know
that
no
one
no
user
should
ever
have
to
make
that
choice
right.
So
we
want
to
narrow
down
the
the
time
from
when
a
module
is
come.
M
You
know
actually
built
and
when
it's
actually
validated,
so
we
want
to
shorten
that
test
cycle,
and
you
know
there
are
other
considerations
as
well
such
as
you
know,
you
know
C
C
MVP
run
by
NIST.
Is
you
know
what
a
big
substantial
validation
program?
That's
that's
really
useful,
but
it
would
be
nice
if
it
were
really
easy
to
do
a
validation.
N
Okay,
thanks
David,
so
about
two
years
ago
we
reached
out
to
the
industry,
and
we
heard
many
of
these
complaints.
There
are
others
too.
We
don't
have
the
time
to
go
to
the
litany
of
concerns
with
this
program,
but
we
reached
out
to
the
industry
and
said
alright.
Why
don't
you
come
and
work
with
us
on
rethinking
how
we
around
the
validation
program
and
built
a
new
one
that
fits
all
of
our
purposes
and
serves
all
of
our
needs,
and
we
started
the
working
group.
N
We
are
meeting
bi-weekly
and
we
think
about
how
we
should
rebuild
and
the
program
and
not
just
think
about
it,
but
or
working
on
it.
This
is
a
big
effort.
It
covers
not
just
to
the
algorithms.
It
covers
the
module
testing
as
well
we're
trying
to
even
solve
problems
that
currently
the
program
doesn't
do,
for
example,
how
to
validate
cryptography
in
the
cloud
environment,
and
the
goal
of
our
effort
is
not
just
to
improve
the
speed
of
validations,
but
also
to
improve
the
depth
and
coverage
of
testing.
N
So
where
are
we
today?
We
have
a
bunch
of
work
working
areas
within
the
working
group
on
improving
the
the
crypto
validation
programs.
Some
of
them
are
related
to.
One
of
them
is
related
to
improving
the
algorithm
testing
program.
Others
is
for
module,
testing,
cloud,
etc,
etcetera.
The
focus
of
the
conversation
today
is
algorithm
testing,
and
so,
as
David
said,
it's
a
collaborative
effort.
We
have
been
working
on
automating
the
algorithm
testing
developing
a
protocol
for
automating
the
algorithm
testing.
N
It's
it's
an
idea
that
originated
with
David
several
years
ago,
I
assume
and
including
a
prototype
I,
remember,
David,
participating
in
the
very
first
meeting
of
about
working
group,
and
that's
where
our
conversation
on
this
thing
started.
The
effort
now
is
being
led
by
Barry
Russell
and
from
Cisco.
He
is
also
in
the
audience-
and
he
will
be
talking
to
us
later
this
evening,
with
our
Show
and
Tell
on
a
CBP,
much
more
detailed
dive
in
we're
currently
working
on
the
fifth
iteration
of
the
protocol.
We've
been
added
for
a
while.
N
N
What
kind
of
access
credentials
for
access
you
need
and
all
kinds
of
stuff
we're
currently
covering
about
90,
plus
algorithms
and
modes,
believe
it
or
not,
that
there's
this
many
algorithms
that
on
the
approved
list,
are
nest
in
fact
the
the
the
set
of
all
of
them
the
approved
list,
the
of
algorithms,
is
even
bigger,
but
we
only
have
testing
for
so
many
at
the
moment
we
are
targeting
releasing
version
104.
Actually,
our
spec
is
already
in
a
kind
of
an
RSC
format.
N
We
know
it's
not
perfect,
but
it's
good
enough,
because
at
least
there
are
two
independent
client
implementations
from
scratch
implemented.
Based
on
it,
so
we
know
it's
a
workable
specification.
We
also
have
a
client
open,
client
implementation
provided
by
Cisco
it's
on
github.
You
can
take
it
and
use
it
and
jumpstart
your
experience
with
with
HTTP
that
way.
N
So
as
a
set
in
the
beginning,
speed
is
not
the
only
problem,
we're
solving
with
validation,
we're
trying
to
improve
our
testing,
and
currently
we
are
planning
some
future
enhancements
along
these
lines.
One
is
a
project
of
interest
with
cooperating
with
is
the
project
which
approve
at
Google.
They
have
an
interesting
approach
to
testing
cryptography,
much
deeper
testing
based
on
known
attacks,
we're
collaborating
with
the
Hackel
a
project
which
is
a
joint
project
between
India
in
Terezin
and
Microsoft
Research.
N
So
we're
trying
to
see
whether
we
can
leverage
the
power
of
some
of
the
formal
verification
of
crypto
within
our
validation
program.
We
are
trying
to
test
even
more
from
the
approved
list
of
algorithms
and
that's
what
we're
collaborating
with
Japan
to
provide
testing
on
856
B
key
establishment
primitives.
The
message
here
is
not
to
give
you
an
exhaustive
list
of
all
the
future
enhancements,
but
to
simply
say
that
we
are
open
to
learning
and
and
and
using
good
ideas
that
you
may
have
about
how
to
do
testing
better
crypto
testing
better.
N
Very
briefly
about
the
features
of
the
protocol.
This
this,
this
slide
is
deliberately
vague,
I'm
not
going
to
go
into
any
details
of
the
the
protocol.
Again,
we
have
a
show-and-tell
tonight
we'll
go
much
deeper
into
details,
but
it's
any
protocol.
It
provides
a
transport
and
encoding
and
message
format
and
sets
a
particular
set
of
message
exchange.
It
works
over
the
Internet
and
the
name
was
discovery
of
the
capability
of
the
module
being
tested
and,
depending
on
that
in
generates
corresponding
tests.
N
It
provides
a
standard
communication
protocol
such
that
implementers
of
crypto
technology
can
use
the
same
protocol
to
talk
to
various
validation
programs.
We
know
we're
running
one
of
these
validation
programs.
We
know
that
there
is
interest
in
testing
crypto
in
other
parts
of
the
world.
A
lot
of
things
are
happening
in
Europe
Asia
other
places.
We
also
know
of
private
sector
organizations
that
are
interested
in
in
using
validated
cryptography,
for
example,
the
financial
services
industry.
N
There
is
a
PCI
testing
and
all
that
kind
of
stuff,
so
we
see
a
lot
of
possibilities
where
this
this
protocol
could
be
used
and,
of
course,
the
structure
of
the
protocol
is
quite
flexible.
You
can
add
tests
for
new
algorithms,
you
can
add
new
tests
for
existing
algorithms
and
you
can
change
the
protocol
features
without
changing
the
underlying
algorithm
tests.
So,
in
a
nutshell,
that's
the
protocol,
so
why?
Why
do
you
do
as
a
CVP
would
IETF?
Well,
we
think
it
makes
a
lot
of
sense
for
one.
N
We
at
NIST
believe
that
openness
and
transparency
of
crypto
standards
and
validation
methodologies
are
necessary
for
acceptance.
As
you
know,
it's
not
enough
to
have
theoretically
secure
algorithm,
it's
important
to
know
whether
it's
implemented
correctly
and
robustly,
and
so
that's
why
I
developed.
N
O
N
Were
recognized
that
will
only
cover
a
portion
of
what's
out
there
in
terms
of
crypto
technology,
and
what
I
was
saying
before
is
that
we
are
establishing
a
foundation
for
how
to
do
clip,
2
testing
and
we
are
providing
an
extent
extensible
foundation.
For
that
matter.
We
will
have
a
protocol
that
can
be
extended
to
cover
other
algorithms
that
currently
are
not
on
the
NIST
approved
list,
and
it's
not
likely
that
they'll
ever
be.
N
N
The
key
would
be
to
use
a
similar
protocol
such
that,
if
you
have
a
module
that
you
want
to
validate
in
two
different
locales,
you
could
use
it
to
use
the
same
protocol,
and
on
top
of
that,
if
you
have
an
overlapping
set
of
algorithms,
it
should
be
nice
to
have
the
same
kind
of
testing
for
the
same
algorithm,
regardless
of
the
place
where
you
go
right.
So
that's
really
what
we're.
M
P
N
Actually,
they
agreed
to
modify
their
test
vectors
to
match
the
JSON
formats
of
a
CVP.
So
the
plan
is
that
the
ones
we
grow
out
100
we're
going
to
integrate
their
library
into
our
server
and
start
generating
that
their
test
vectors
for
testing.
So
that's
how
we
see
this
evolving
we've
been
talking
to
them
for
close
to
a
year,
but
it's
not
to
make
progress
on
this
front
until
we
set
up
the
foundation
with
what
we
already
have
and
we're
close
to
getting
there.
Ok,
thank
you.
Yeah.
D
D
R
D
B
S
S
If
I
can
understand
this
better
and
essentially
what
we
can
do
is
look
at
a
scan
population
and
collect
together
all
the
IP
addresses
that
have
some
key
in
common
and
that
forms
a
cluster
and
basically-
and
you
do
that-
you
see
every
possible
combination
of
things
being
reused,
except
I
still
haven't
ever
seen.
Anybody
use
an
SSH
host
key
for
TLS,
but
every
other
combination
you
do
see
so
port
25
on
one
host
is
443
on
another,
etc,
etc,
and
so
there's
lots
of
possible
causes
for
this
they're.
S
Not
all
bad
so
multihomed
hosts
is
one
because
we
cannot
borrow
nice,
cutting
IP
addresses,
and
so
you
know
not
all
causes.
This
are
bad
I
kind
of
when
I
create
these
clusters
of
IP
addresses
that
seem
to
have
some
keys
in
common.
It
looks
like
that.
30%
of
them
might
be
dead.
You
don't
look
very
much
like
multi-home
hosts,
but
30%
are
possibly
a
30%,
definitely
not
because,
for
example,
there
might
be
more
than
one
is
involved,
so
you
can
make
little
pictures
like
this,
where
the
nodes
are
reps
up.
S
The
IP
addresses
the
lines
represents
the
key
being
shared
between
the
two
IP
addresses
on
some
ports,
with
different
colors
for
the
different
port
numbers,
and
then
we
will
see
what
the
pictures
start
look
like.
So
again,
not
all
cases
are
reuses
are
bad,
but
some
could
be
clearly
if
you,
if
you
get
one
of
these
keys,
you
could
masquerade
as
any
of
the
other
hosts
in
the
cluster
for
something
and
if
they're,
using
RSA
key
transport.
It's
worse,
it
could
be
that
the
risk
increases
faster
than
linear
with
cluster
size.
S
S
So
it'll
look
like
looks
like
this
rips
this
in
this
case,
there's
a
little
close
and
Ireland
3
hosts
sharing
an
SSH
host
key
into
two
different
day
assets
and
the
SMTP
banners.
Don't
look
like
each
other
like
anything
to
do
with
each
other.
So
that
looks
like
somebody's
SSH
host
key
is
in
more
than
one
place.
If
they
don't
want
us,
which
is
not
that
uncommon.
It's
uncommon,
but
not
that
uncommon.
This
one
is
a
another
cluster
from
Ireland.
S
Yeah
we've
actually
got
to
fix
this,
so
be
nice
to
see
if
they
do
I'm
sure
they
will,
whether
people
deploy
to
fix
we've
been
interesting
to
check.
This
is
just
another
cluster
which
has
a
mixture
of
kind
of
web
and
kind
of
mail
keys.
The
only
other
thing
with
that's
interesting
that
this
one
is
I
started,
November
scan
and
one
in
March
and
two
IP
addresses
got
taken
away
and
one
got
added
to
this
cluster
in
that
time.
So
things
evolved.
S
This
one
is
a.
This
looks
like
a
kind
of
uk-based
mobile
application.
Developer
consultancy
type
company-
and
there
are
you-
know
they
basically
are
using
a
large
kind
of
hosting
company
to
host
a
bunch
of
things
with
this
kind
of
a
metal
key
sharing,
the
blue
colors
are
male.
The
orangey
colors
are
kind
of
port
for
four
three
and
it
looks
for
lots
of
different
SMTP
banners.
So
this
is
for
lots
of
different
people.
S
This
one
is
from
Sylvania
it's
just
like
you're
in
a
prettier
picture.
If
you
look
to
this
some
of
the
metadata
for
this,
it
looks
like
the
hosting
tools
may
be
at
fault.
So
this
is
like
a
whole
bunch
of
things
that
they're
people
are
hiring
hosters
to
do.
The
hosters
are
splitting
up
virtual
machines
in
various
ways.
S
This
is
my
favorite
one.
This
is
from
Finland.
So
this
again,
it's
another
mobile
consultancy
company,
the
black.
There
is
SSH
keys,
being
shared
the
other
colors
mentioned
earlier
and
in
this
case
there's
one
SSH
host
fee.
That's
seen
46
times
here
and
14
times
in
Portugal
in
a
university
for
again,
so
we're
reason
so
again
just
to
go
through
this
quickly.
We
have
some
numeric
results.
S
If
you
look
at
the
link
at
the
top
of
the
slide
deck,
has
a
competent
prepaid,
there's
not
very
much
of
interest
here
just
to
talk
about
now
but
other
than
the
number
there
at
the
bottom.
The
heart:
that's,
the
percentage
of
hosts
were
using
keys
from
those
that
do
some
crypto
in
a
particular
scam.
So
you
can
see
it's
like
50
to
70
50,
70
%,
which
seems
very
hard.
S
The
other
interesting
thing
there
is,
if
you
look
at
the
the
max
cluster
sizes
for
the
Ireland
scan
in
2018
there,
it's
1991
IP
addresses
all
behind
one
single
wild-card
search
and
that
key
also
is
seen
in
Singapore
as
it
turns
out.
So
some
of
these
clusters
are
quite
big
and
there's
some
more
results.
If
accompanies
the
coaster,
size,
distribution
is
kind
of,
as
you'd
expect.
There's
an
awful
lot
of
size
to
cluster,
which
probably
is
some
okay
to
premiering
or
hosted
two
IP
addresses.
S
But
then
you
get
these
very
large
ones,
and
a
bunch
in
between
you
can
check.
How
does
this
evolve
over
time?
So
I
have
scans
from
November
17
and
from
spring
this
year
for
Ireland
and
basically
every
possible
change
that
you
could
imagine
tends
to
happen.
So
if
you
look
here,
we
get
a
hundred
and
sixty
eight
clusters
disappear.
S
Seven
hundred
appear
few
other
odd
cases,
and
then
this
is
kind
of
really
complicated
cases
where,
in
fact,
if
you
look
at
this
one,
the
middle
and
the
bottom,
there
is
the
case
where
two
clusters
evolve
into
three
clusters
later
on.
Now.
If
there
is
any
case
where,
if
you
hire
a
new
VPS
and
you
had
a
chance
to
look
at
the
disk
and
possibly
extract
a
key
that
somebody
else
once
used,
this
might
become
pretty
sad.
S
We
can
look
across
the
ten
countries,
as
I
did
so,
there's
ten
different
little
small
countries,
and
basically,
you
find
keys
in
common
between
every
single
pair
of
possible
things,
even
for
a
Namibia
which
is
really
small,
and
these
look
like
that.
If
you
take
away
this,
some
of
these
are
also
kind
of
reasonable,
there's
a
bunch
of
New
Zealander,
the
singapore
ones,
where
it
could
be
somebody
who's.
Just
you
know,
trying
to
get
around
latency
by
having
a
host
in
two
different
bases
that
it's
otherwise
the
same.
S
That
might
be
okay,
the
ones
that
are
more
than
two
nodes.
Look
like
this
and
basically
they're
all
kind
of
fully
connected
pretty
much
graphs,
which
kind
of
smells
like
hard
coded
keys
or
a
wild-card
search
that
are
being
used
in
many
different
countries
and
there's
like
a
hope,
is
that
there's
like
a
hundred
and
one
of
them
that
have
come
through
about
half
of
them
to
see
what
they
are
and
there's
various
kinds
of
shipping,
with
hard
coded
keys,
going
on
from
a
whole
set
of
different
vendors,
so
I'm
trying
to
contact.
S
S
So
again,
this
is
a
take
the
book
here,
more
hosting
tools
where,
when
they
spin
up
a
new
VPS,
if
they
see
SSH
key
pairs
for
RSA
and
DSA
at
that
point,
they
knew
to
throw
them
away.
But
when
we
added
this
lovely
new
algorithm,
the
hosting
tool
didn't
notice
all
that
one
away,
so
I
inherited
a
ECDSA
key,
and
that
seems
kind
of
common
and
it's
maybe
yet
another
reason
not
to
add
new
algorithms
too
much.
S
This
one
is
a
little
piece
of
open-source
software,
that's
intended
for
developers.
This
is
a
kind
of
a
thing
to
get
developers
to
kind
of
get
started,
really
quickly.
I'm
developing
the
thing
it's
far
and
again,
ships
with
a
hard-coded
key
and
it
looks
like
it's
being
shipped
as
it's,
not
the
intent
and
also
possibly
built
into
other
packages
that
are
more
popular,
which
I
haven't
fully
verified.
Yet
so
I
won't
say
what
package
and
again
I
contacted
them
and
they
said
Oh
we'd
love
to
change.
That
I
mean
money
sometime,
so
yeah.
S
These
are
the
confirmed
cause.
Is
the
SSH
Coast
key
regeneration
of
virtualization?
That
seems
to
be
a
mistake.
That
happens
over
and
over
there
seem
to
be
different
generations.
Amish
hard-coded
keys
is
a
clearly
a
thing.
That's
still
going
on
a
lot
and
amusingly.
There
are
some
vendors
who
release
a
thousand
24-bit
hard
coded
key
and
then
later
on,
they
decided
that
wasn't
good
enough,
so
they
no
ship
at
a
2048-bit,
hard-coded
key
and
even
the
4096
bit
hard
coded
key
because
they're
getting
better.
S
It's
not
clear
to
me
at
what
point
is
using
kind
of
wild
card
search
for
the
numbers
of
addresses
across
different
countries
be
become
a
real
issue
you
can
read
tell
from
the
outside.
It
could
be
okay
depending
how
you
implement
it,
but
maybe
it's
not
I
did
see
some
kind
of
mega
sans.
There
was
one
certificate
with
1500
names
and
the
names
fields.
Any
animal
T
home
hosts
are
kind
of
confirmed
as
I've
talked
to
some
hosts
with
locally
in
Ireland,
and
they
confirmed
at
some
of
the
things
I
see
as
closer.
S
It's
just
our
multi-home
posts,
so
I'm
chatting
with
people
to
try
and
figure
out
more
before
this.
What's
going
on
here,
I'd
love
to
get
more
help
funny
bugs
interested
or
can
suggest
other
causes
for
this,
the
mitigations
I.
Guess
it's
more
interesting
for
the
ITF.
It
seems
like
the
simplest
mitigation
from
CRS.
One
is
just
take
the
model.
That's
there
for
and
let's
encrypt
times
of
generating
a
new
key
every
few
months,
and
then
all
of
this
just
goes
away.
S
I
guess
you
could
argue
that
we
should
build
that
kind
of
thing
into
protocols
a
bit
more
so
an
MLS.
This
morning,
for
example,
there
was
discussion
of
whether
to
kind
of
force
updates
or
to
say
something
about
updates
at
a
certain
frequency.
Maybe
we
want
to
make
sure
the
frequency
cannot
be
zero
in
order
to
avoid
this
kind
of
thing,
measurements
I
think
it's
a
good
way
of
doing
it.
S
Clearly,
if
you
do
measure
these
parties,
you
know
what's
going
on
in
the
network,
you
find
weird
stuff
like,
for
example,
I
Fairweather
SSH
host
key
that
other
people
have
in
theory.
Ssh
clients
could
react
to
seeing
the
same
key
in
different
places
and
say
that's
a
bit
odd.
Did
you
really
mean
that?
S
That's
not
an
ITF
thing,
but
it
could
be
done.
Cas,
I,
guess,
there's
not
too
many
of
these
a
lot
of
these
certificates
for
the
TRS
cases
here
ourselves,
science,
arts,
our
wild
card
search
to
punch
them,
though,
are
kind
of
not
too
many.
But
so
am
I
coming
up
the
number
two
I'm.
Sorry,
our
real
certs,
with
different
names
in
them
and
the
same
key
CAS
could
be
a
little
bit
less
tolerant
to
that
I
guess.
But
that
would
probably
be
a
cab
forum
thing.
S
It
would
be
nice
if
the
SSH
protocol
didn't
provide
a
way
to
rotate
the
host
keys,
but
I
guess
that's
kind
of
hard
and
maybe
I'm
lucky,
but
it'd
be
nice
thing
to
happen.
So
that's
basically
at
the
conclusion
I
had
was
I
was
kind
of
surprised
by
how
much
how
often
this
is
happening.
You'd
kind
of
know
what's
happening,
but
how
often
this
was
a
surprise.
Rotating
Keys
I
think
will
help
you
know.
Measuring
can
help.
This
is
kind
of
part
of
learning
to
manage
crypto
of
scale.
S
L
I
think,
if
you
were
to
study
clients
rather
than
servers,
you
would
end
up
actually
finding
some
reuse
between
TLS
keys
and
SSH
keys.
The
reason
I
predict
this
is
that
OpenSSH
has
pkcs
11
support,
but
it's
rather
limited
and
in
particular
doesn't
let
you
switch
from
the
default
slot.
So
if
you're
trying
to
use
a
smart
card
to
just
authenticate
to
everything,
it
can
be
sorely
tempting
to
use
the
same
smart
card
key
for
an
SSH
key
into
TLS
cert
and
in
fact,
I
swear
I've
come
across
tutorials
on
how
to
do
this.
T
To
Korea,
so
it
sounds
like
you
haven't,
contacted
too
many
people
simply
I.
Think
you
heard
I
heard
you
say
you
contacted
a
couple
yep
yeah.
Did
you
try
and
make
them
aware
of,
especially
like
the
larger
clusters?
You
know
that
compromise
of
one
host
kind
of
compromises,
the
entire
system
and
that
you
know
right
so.
S
S
Really
good
at
that
I'd
be
delighted
to
you
know,
figure
out
a
way
to
share
some
data
with
you
and
have
you
do
well?
Yeah
I
mean
one
of
the
largest
cluster
in
Ireland
is
a.
It
looks
like
a
marketing
campaign
thing,
so
it's
probably
not
top
of
this,
but
if
you
wanted
to
send
those
guys
to
security
report
to
some
sort,
you
have
to
sign
up
for
some
meta
service
that
they
use
to
get
those
things
and
yeah.
U
What
thanks
for
the
presentation
I
wonder?
There
was
also
a
case
where
people
who
are
searching
for
keys
on
github
and
Ellora
open
source
projects
were
actually
you
know,
shipping
with
actual
deployed
keys.
Is
there
some
way
to
act
to
kind
of
check
whether
these
keys
show,
because
now
github
doesn't
allow
you
to
to
search
for
ID
RSA?
But
is
there
still
some
way
to
scrap
and
see.
S
V
V
S
Ct
I
you
kind
of
use
CT
as
I'm
kind
of
manually,
trying
to
validate
things,
I've
used,
CT
and
being
able
to
see
the
same
key
being
used
back
over
time
so
like
over
the
last
five
years.
In
some
cases
so
yeah
you
could
use
it
I,
haven't
kind
of
program
up
an
idea
to
try
and
do
that,
but
sure
yeah.
If
you
could
I
mean
you
could
check,
you
could
verify
in
CT
that
they're
using
the
same
key
right
are
the
same
keys
used
for
different
things.
Right.
V
V
S
T
Yeah
me
again,
sorry,
first
off
West,
one
thing
that
we
had
to
do
an
open
source
software.
It
was
like
deliberately
break
they
config
that
was
distributed.
You
know
so
that
it
would
not
boot
and
would
die
and
because,
let's
just
say
the
word,
public
was
used
widely
at
one
point
in
the
past
for
one
protocol
as
the
secret
key,
not
good.
L
Tim,
how
big
did
you
start?
Somebody
asked
if
there
was
a
CNA
in
the
room.
Unfortunately,
if
you
decide,
I
would
love
to
not
allow
issuance
of
certificates
with
the
same
key
that
was
previously
used.
There's
there's
no
good
reason
for
it.
Unfortunately,
if
a
CA
da,
if
I
ca
did
that
you
are
suddenly
less
competitive,
so
the
right
way
to
do
it
is
get
a
ballot
passed
in
to
see
a
browser
forum
saying
it's
not
allowed
and
then
everybody
can't
do
it
I,
don't
think
that
ballot
would
pass,
but
I
would
love
to
see.
V
It
so
I
want
to
be
clear.
This
is
I
was
not
proposing
that
you
would
not
allow
reuse
of
the
same
key
for
the
same
entity.
I
was
proposing
that
you
could
see
whether
the
key
that
was
being
issued
had
been
used
for
a
different
entity
in
the
past.
There
is
actually
concrete
use
for
reuse
of
keys
in
multiple
certificates
for
the
same
entity
go
over
time.
V
We
ASCII
continuity
right
and
but
but
if
you
see
that
the
key
that's
being
used
has
been
used
for
a
different
entity,
that
would
be
the
thing
that
I
don't
think
there
would
be
much
controversy
about
you
shutting
down
and
you
could
sell
it
as
a
security
service.
Saying
we're
going
to
let
you
know
that
the
key
you're
using
may
have
been
reused
by
somewhere
else.
That.
L
Second,
one
notification
is
actually
much
easier
just
because
of
the
fact
that
it
doesn't
actually
block
their
ability
to
do
anything.
When
you
start
telling
the
customer,
they
can't
have
the
certificate
that
they
want
to
have.
Then
they
get
upset.
Unfortunately,
and
it
also
actually,
if
another
I
understand
your
proposal
better,
that
may
have
a
much
better
chance
of
passing
so.
K
Hoffman,
so
going
back
to
D
K
G's
proposal,
instead
of
saying
to
somebody
who
doesn't
understand
that
they're
reusing
something,
you
know
that
they're
running
software,
you
can't
do
this
and
then
they
they
will
go
to
somebody
who
will
possibly
a
different
thing.
That
would
be
very
useful,
is
either
at
the
time
say.
It
looks
like
you're
doing
this.
This
is
a
problem,
look
into
it
or
even
better,
after
a
scan
where
there
are
these
things,
compare
it
with
with
the
CT
log
and
do
a
thing
saying:
look
what
we
found
among
these
things.
K
W
S
X
You're
using
elliptic
curve
or
any
difficulty
cleek,
there
is
a
solution
to
this
and
that's
use
the
co-operative
key
generation
mechanism
that
I
presented
here
a
few
years
back.
So
basically,
the
idea
here
is,
if
you've
got
a
device
like
an
IOT
thing
that
you
just
can't
trust
it
to
have
enough
randomness.
The
ca
does
not
want
to
generate
a
key
for
that
device,
but
the
key,
a
CA
can
add
some
randomness
into
the
mix
in
a
verifiable
fashion
such
that,
if
the
device
has
chosen
a
strong
key,
the
device
key
will
be
strong.
X
Otherwise
the
CA
would
be
able
to
attack
that
key,
but
nobody
else
would
be
so
it's
not
entirely
foolproof.
You
know
there
has
to
be
some
randomness
in
the
IOT
device,
for
it
could
be
completely
secure,
but
it
does
mitigate
it
to
the
the
extent
that
you
can.
You
can
use
that
with
any
elliptic
curve
and
when
I
showed
it
to
kolinsky
he's
got
one
for
RSA.
Y
Hi
there
Benjamin
down
I'm
a
bit
flabbergasted
about
the
CA
conversation,
because
to
me
that
the
CA
is
job
is
to
is
to
verify
a
is
to
connect
the
identity
of
the
key
to
the
identity
of
the
Oscar
and
this
if
the
CIA
is
not
defending
the
Oscar
against
known
keys,
then
I
to
me
this
seems
like
a
failure
of
the
SIA
to
do
their
job
but
and
I
understand
the
perspective.
It's
just
a
bit
strange
to
me.
Z
Yeah
hi
rich
sauce
I'm
a
bit
confused
by
the
way
some
of
the
conversation
went
in
your
study
was
that
the
actual
key
pair
you
found
reused.
So
the
search
doesn't
really
matter
much
right,
it's
someone
literally
or
is
it
you
actually
found,
or
is
that
the
key
pair
and
the
cert
that
were
reused?
So
you
every.
S
Possible
thing
happens,
yeah
pretty
much,
so
the
cases
and
points
I
think
so.
There's
not
many
cases
for
it's
a
browser,
trusted
cert
and
not
a
wildcards
and
is
issued
to
different
looking
names
for
no
apparent
reason
and
this
M
key.
There
are
some
cases,
but
there's
not
a
huge
number
of
them.
So
it's
those
cases
I,
guess
this.
What
we're
talking
about?
Okay,
thanks.
B
AA
B
AA
AA
We
we
offer
a
web
check
service
for
the
owners
and
operators
of
F
dot
uk'
it
domains
to
check
the
the
health
of
the
TLS
servers.
Looking
for
instances
of
old
versions
of
TLS
old
deprecated
crypt
problems
in
their
x.509
search,
we're
looking
at
tackling
bgp
hijack.
So
this
is
when
your
advert
eyes
roots
in
BGP
that
are
like
a
suspicious.
You
know
we're
looking
at
some.
You
know
automated
analytics
for
detecting
those.
AA
In
aren't
you
in
advertise,
BGP
tables
and
in
the
telephony
world,
we're
looking
at
s
s7,
which
is
signalling
protocol,
has
been
around
since
the
mid-1970s
with
hardly
any
change.
There
are
known
security
problems
in
ss7.
We
don't
think
we
can
do
much
about
them,
but
we
can
offer
advice
to
operators
in
the
UK
and
we're
monitoring
to
see
how
well
that
advice
is
been
taken
up
so
a
bit
more
detail,
so
the
takedown
of
the
phishing
websites.
AA
So
you
know
previously
said
that
these
are
websites
that
are
impersonating
domains,
maybe
by
changing
a
letter
in
the
in
the
domain
name,
so
that
we're
so
that
users
are
fooled
into
interesting
those
sites
when
they
believe
they're
visiting
the
legitimate
site,
so
we're
interested
in
protecting
the
UK
government
brand.
So
if
somebody's
advertising
themselves
to
be
Revenue
and
Customs
and
they're
using
a
variation
of
the
Revenue
&
Customs
domain
name,
then
you
know
we're
we're
spotting.
Those
those
false
domain
names
in
the
registry
data
and
we've
now
got
an
automated
takedown
service.
AA
After
doing
a
manual
examination
of
them
themselves,
that
was
automated
notification
process
has
reduced
the
time
taken
to
remove
phishing
websites
down
from
45
hours
to
7
hours
in
the
case
of
UK
government
brands,
we're
also
looking
at
non-uk
phishing
websites,
but
where
they
are
hosted
in
the
UK
and
we've
detected
120,000
of
those
last
year
and
we've
managed
to
reduce
the
the
share
of
phishing
hosted
in
the
UK
by
our
calculations
from
five
and
a
half
percent
to
2.9
percent.
So
you
know.
R
AA
Think
that
sir
yeah
good
win
for
us
we're
looking
at
stopping
spoofed
email,
where
the
the
from
address
of
the
email
is
coming
from
a
doctor.
You
K
domain,
so
we're
doing
this
by
encouraging
adoption
of
D
mark
and
SPF
across
dr.
dot,
uk'
domains
so
still
at
a
relatively
low
level.
But
we
have
tripled
to
support
for
the
D
mark
over
the
past
18
months.
As
a
result,
we
are
now
stopping
millions
of
spoofed
emails
every
month
from
been
delivered
to
to
users
thanks
to
D
mark.
AA
But
we
know
there's
problems
with
this,
because
there
are
some
messages
that
are
still
getting
through
where
you
know.
We
believe
that
they
should
be
stopped,
but
they're.
Not.
We
don't
understand
why
we're
still
doing
research
on
that
and
when
we
have
the
results
of
that
research
would
we'd
like
to
publish
it.
AA
So
that's
a
low-hanging
fruit.
We
also
tackle
the
more
advanced
threats,
so
they
may
be
state
actors
who
are
trying
to
get
into
UK
government
systems.
So
over
the
the
first
18
months,
we've
we've
dealt
with
over
a
thousand
nationally
significant,
nationally
significant
cyber
attacks
in
the
UK
and
we're
going
to
publish
data
on
those
in
due
course.
We're,
like
you,
know,
pulling
out
the
root
causes
of
what
was
behind
those
attacks
and
we'll
have
some
data
to
to
bring
to
the
ITF
and
the
global
community
later.
AA
On
part
of
the
point
of
doing
this
is
to
emphasize
that
yeah.
If
UK
government
can
be
doing
this
and
yeah
other
nations
could
be
doing
this,
they
could
be
offering
other
systems
similar
systems
within
their
own
countries
or
other
other
sectors.
Maybe
the
financial
industry
could
be
offering
similar
phishing
takedown
services
so
where
we
want
to
get
drive
adoption
of
protocols
like
yeah,
like
the
mark,
so
that
other
people
could
benefit.
AA
Doing
the
research
on
the
way
that
users
interact
with
the
internet
and
technology,
so
yeah
a
couple
of
recent
pieces
of
research.
What
we've
done
show
that
just
educating
users
about
don't
click
on
that
link.
If
it
looks
suspicious,
isn't
enough
I
mean
yes,
you
want
to
do
that,
but
you
want
to
do
it
in
in,
in
addition
to
other
measures,
to
try
and
reduce.
AA
Users
actually
becoming
susceptible
to
those
phishing
attacks
and
realistic
password
policies
as
well.
We've
we've
published
advice
on
yeah
how
to
choose
passwords,
not
saying
things
like
you
know,
make
it
10,
10,
characters
and
mix
of
alphanumeric
situations.
People
don't
do
that.
They
can't
remember
them.
So
this
is
like
realistic,
practical.
AA
My
advice
about
choosing
passwords
that
are
appropriate
to
the
system
that
you
want
to
use,
saying
it
yeah
by
all
means
write
a
password
down
if
you
need
to
because
yeah,
if
you
keep
it
securely
and
it's
done
appropriately,
then
you
know:
that's
that's
yeah,
perhaps
better
than
some
of
the
alternatives
so
go
and
have
a
look
at
our
website.
If
you
want
to
see
some
of
this
guidance,
so
why
we're
here
at
the
idea?
AA
What
we
want
to
do
is
to
share
some
of
the
research
work
that
we've
been
producing
and
encourage
others
to
do
the
same,
to
bring
it
into
the
IRT
IETF,
so
that
when
people
here
are
designing
protocols,
they
have
the
information
that
they
need
to
back
up
some
of
the
decisions
that
that
as
to
how
those
protocols
will
impact
on
cyber
defense.
So,
yes,
some
some
statistics
on
you
know
the
well
the
relative
prominence
of
various
cyber
cyber
attacks,
the
methods
that
he
used
to
detect
and
defend,
defend
against
them.
AA
How
successful
those
measures
are
are
want
to
do
this
yeah
a
systematically
as
possible
across
all
of
the
different
areas
of
the
the
ietf,
so
that
you
know
whatever
new
protocol
has
been
designed.
The
people
doing
that
design
have
cyber
defense
at
the
back
of
their
mind
somewhere
it's
they
can
take
it
into
account
when
they're
doing
that
protocol
design
work.
We
need
some
industry
partners
to
do
this.
AA
We're
looking
for
yeah
companies
that
can
come
and
share
their
research
in
a
venue
like
this
all
doing
this,
so
that
your
protocol
designers
can
make
the
decisions
about
protocols
with
the
evidence
and
if
they
need
to
do
their
job
and
to
take
into
account
cyber
defense.
So
questions
for
the
audience
here.
AA
How
we're
going
to
do
this
so
we've
had
some
thoughts
about
what
we
might
like.
You
know
how
we
could
bring
this
research
to
the
IETF.
We
thought
maybe
a
new
IR
TF
research
group
on
cyber
defense
could
be
the
way
to
go
or
a
workshop,
like
the
a
nrw
that
ran
earlier
this
week.
That
seemed
really
good
there'll
be
a
way
to
encourage
academia
to
come,
and
you
know,
publish
our
results
and
make
them
available
to
the
IETF.
AA
So
we'd
like
to
do
this,
we've
got
researcher.
We
can
contribute
to
the
ITF.
We
we
need
others
to
come
and
help
us
and
do
the
same,
contribute
their
research
be
prepared
to
yeah,
put
time
and
effort
into
you
know
perhaps
running
a
group
here.
So
you
know
really.
My
plea
is
you
know
to
you
all
you
know:
can
you
come
and
help
us
do
this?
Please
come
and
speak
to
us
afterwards,
I'm
here
with
my
colleague,
Kirsty
Payne
Kirsty.
Can
you
stand
up
we're
really
friendly,
so
yeah
we're
want
to
hear
your
ideas.
AB
Hi
Pete
Resnick
I,
really
like
the
idea
of
an
IRT
of
research
group
for
this
particular
kind
of
activity
and
part
of
the
reason
is
I
know
as
someone
who
is
implemented
in
the
past
and
worked
on
especially
the
email
side
of
this.
A
lot
of
folks
won't
come
publicly
to
talk
about
their
data
yeah
because
it's
commercially
since
you
have
to
even
mention
break-ins
like
that.
AB
If
you
all
are
willing
to
do
some
of
the
heavy
lifting
of
convincing
those
folks
to
get
in
the
room,
an
IRT
F
group
can
be
closed
and
can
have
different
kinds
of
models
than
everything
is
open
to
everybody
and
having
the
data.
Even
if
it's
not
easily
identifiable
would
be
great
on
the
implementer
side
of
things.
Yeah.
AA
I
know
that
some
companies
were
being
reluctant
to
bring
that
their
data
here,
because
he
represents
a
propriety
advantage
as
a
government
department.
You
know
we're
not
we
don't
have
any
commercial
advantage
in
this.
We
are
trying
to
protect
you.
Uk
networks
and
yeah
help
a
better
global
ecosystem
for
for
Internet
security.
It's
a
cyber
defense,
so
yeah.
Well,
we
we
need
to
do
that.
Only
a
persuasion
work
with
the
with
the
vendors
hi.
V
There
Daniel
con,
don't
worry,
I
cou,
so
thanks
for
bringing
for
offering
to
bring
more
data
to
the
IETF
I
think
we
can
definitely
use
that
I'm,
not
sure
what
the
best
route
to
go
within
the
IETF
is
in
terms
of
bringing
in
additional
data,
but
from
the
motivations
that
you
presented
on
your
slide
earlier.
I
also
would
be
curious
to
hear
from
you
what
other
kinds
of
things
that
IETF
can
produce.
That
would
help
you
and
your
goal
of
providing
guidance
to
protect
people
against
the
level
attacks.
AA
AA
I
think
so
they,
the
the
web
check
service,
alluded
to
briefly.
So
one
of
the
things
we
do
there
is
look
for
those
old
deprecated
versions
of
TLS
in
particular,
but
you
know
potentially
could
be
extended
to
other
protocols.
So
if
we
have
things
that
are
marked
as
obsolete
or
not
current
best
practice,
then
we
can
we
can.
AA
We
can
use
that
when,
when
forming
the
the
owners
of
those
services
that
yeah
you're
using
something
that's
considered
out
of
date,
because
you
know
we're
the
experts
for
the
day
UK
domain-
we
don't
expect
all
of
the
individual
5000
or
whatever
it
is
individual
domain
domain
owners
two-bit
to
be
able
to
do
that,
so
we
can
callate
it
all
in
one
place,
and
so
those
obsolescence
documents
are
really
helpful
for
that.
Hey.
Y
Y
AA
Either
already
created
their
national
cyber
security,
centers
or
I'm
in
the
process
of
doing
that
yeah,
we
would
like
to
to
talk
to
international
partners.
If
your
country
has
one
that
we
could
open
up
a
dialogue
with
I
think
I'd
be
very
interesting,
so
well,
yeah
coming
yeah
talk
to
me
afterwards
and
give
you
give
me
your
details
if
you
like
to
do
so.
F
Hi,
my
name
is
room
engineer
and
to
follow
up
on
the
last
comment:
yeah
there's
about
90
governments
around
the
world
that
have
national
cybersecurity
organizations
that
are
pierced
peers
to
yours.
I
just
wanted
to
thank
you
again,
as
others
have
for
bringing
its
to
the
ITF
I.
Think
you
purely
driven
OPSEC
advice
they
can
be
given
to
protocol
design
is
gonna,
be
very
helpful.
We
need
to
talk
about
I,
think
the
structural
mechanism
to
sustain
that.
F
S
So
yeah
again,
thanks
I
think
data
is
useful.
The
more
open
the
better
I
know.
Not
everything
can
be,
but
the
more
the
better.
Let's
open
fits
this
process.
Much
better
and
and
then
the
reason
I
got
up
is
cuz
been
forced
me
to
there's
a
thing
called
map
4G,
which
is
a
kind
of
measurement
research
group
already
in
the
ITF.
Q
S
To
me
like
it
would
be
a
fine
place
to
start.
You
know,
bring
some
work
to
bring
some
contributions
to
that
place
and
then,
if
you
overwhelm
them
with
detail,
then
another
research
group
could
be
spun
up
I
think
pretty
easily,
but
map
orgy
seems
like
a
really
good
fit
for
the
kinds
of
information
you're
proposing
yeah.
AA
AC
AC
B
I
A
I
guess
I
passed
all
I,
don't
know
if
I'm
supposed
to
be
advertising,
but
you
sent
also
tonight
there's
a
sign
meeting
on
the
crypt
ever
see
how
7:30
in
Van
Horn's,
there's
also
another
site
meeting
with
the
automated
crypto
verification
protocol
stuff.
So
it's
two
other
conflicting
times.
That's
interesting!