►
From YouTube: IETF105-MAPRG-20190726-1000
Description
MAPRG meeting session at IETF105
2019/07/26 1000
https://datatracker.ietf.org/meeting/105/proceedings/
A
A
Other
people
might
not
know
yeah,
so
this
is
an
IIT
of
session.
We
also
have
basically
a
note
well,
which
is
kind
of
aligned,
with
the
ietf
note
well,
but
slightly
different
terms,
but
I
guess
you've
seen
that
what
this
note
well
is
actually
not
pointing
at
is
the
code
of
conduct
and
harassment
policy,
but
it's
also
important
things
to
note.
So
please
be
aware
of
all
these
things.
B
Just
a
couple
things
I
wanted
to
mention
about
the
status
of
the
group
and
things
that
are
really
healthy
and
mapper
G
as
well.
You
know
we
have
a
lot
of
people.
We
always
have
a
packed
agenda
as
we
do
today
and
we'll
work
our
way
through
here.
One
of
the
things
I
wanted
to
mention
is
the
last
couple
meetings
I've
went
to
PRG
the
privacy
enhancement
and
assessment
research
group.
That's
measurement
related
as
well
and
they're
doing
some
interesting
things.
B
So
look
for
opportunities,
I
think
to
sync
up
with
what
they're
doing
and
watch
what
they're
doing,
especially
when
it's
about
privacy
with
measurements.
The
the
next
thing:
let's
go
the
next
slide.
So
this
is
the
way
I
was
greeted.
When
I
got
my
badge
this
week,
none
Blanca
and
it
may
be-
they
are
anticipating
that
I
won't
be
there
at
the
next
in
Singapore.
B
So
so
we
have
one
Blanca
this
time
in
non-pom
Connect
next
time,
can
you
go
to
the
next
slide,
so
Mary
and
I
were
going
to
call
for
an
alternate
or
additional
map.
Our
gt-r
chair
we've
been
running
for
about
four
years,
I'm,
not
meaning
to
step
from
it,
but
we
could
certainly
use
help
with
another
person
when
we're
covering
meetings
in
different
parts
of
the
globe.
B
So
it's
not
a
requirement,
but
but
if
you
have
that
you
might
like
to
join
and
then,
like
I
said,
you'd
be
able
to
curate
the
upcoming
program,
which
is
kind
of
fun,
and
it's
a
little
bit
of
work
about
like
Miria.
This
time,
for
instance,
did
a
lot
of
legwork
contacting
people
to
get
to
get
the
program
filled
out
so
feel
free
to
write
us
about
that.
I
think!
Is
that
the
last
thing
we
had
there
yeah,
that's
the
last
of
that.
B
C
Hi
everybody,
my
name
is
Yoda,
so
I
will
be
presenting
the
paper,
understanding,
evolution
and
adoption
of
up
level
domains
in
DNS
sick.
This
paper
was
recently
published
in
the
2019
I
Triple
E,
imminent
measurements
in
the
networking
conference
and
other
co-authors
of
this
paper
are
Surya
Ramachandran
and
dr.
Anika
mohanty,
both
also
from
the
University
of
offered
an
excitement.
C
So
in
this
presentation
we
will
classify
generic
top-level
domains
into
three
groups
owed
for
the
ones
created
in
the
1980s
early
for
those
created
between
2001
and
2012.
In
a
new
generic
top-level
domain
for
our
thousands
of
new
generic
top-level
domains
introduced
since
noon,
we
will
also
look
at
country
code
top-level
domains,
OCC
Tod
later
on
next
slide.
C
C
For
data
collection,
we
used
two
different
sources
for
our
data,
so
we
collected
packet
traces
from
the
campus
network
of
University
of
Auckland,
which
is
the
largest
university
in
New.
Zealand,
was
over
40,000
users.
In
addition,
we
also
collected
metadata
for
an
internet
NZ,
the
manager
of
NZ,
New
Zealand
national
top-level
domain
for
three
national
level.
Data
sets
meaning
classified
the
generic
top-level
domains
into
the
three
categories:
old
early
and
new,
based
on
the
time
of
introduction.
C
And
finally,
we
carried
out
pub
level
domain
and
DNS
sick
analysis
in
it
looks
like
so
moving
on
to
the
results
next
slide
for
trace
over
you.
We
found
the
number
of
DNS
queries
to
grow
between
2013
and
2018
was
the
proportion
of
requests
to
early
G
theories,
new
G
theories
and
country
code
top-level
domains
are
increasing.
While
there
has
been
a
lot
of
growth
in
the
new
generic
top-level
domains
compared
to
2013,
they
still
only
account
for
around
0.3
percent
of
total
requests
in
the
local
campus
network.
C
Next
time,
in
terms
of
the
top
10
most
queried
top-level
domains,
we
find
some
common
generic
top-level
domains
that
calm
donate
door
and
in
terms
of
low
country
code
top-level
domains,
the
NZ
not
a
UN
dot
cm
for
New,
Zealand,
Australia
and
China,
or
regularly
appeared
in
at
opt
in
every
year.
There
were
no
new
general
public
domains
at
all
in
the
top
10
in
any
of
the
years
and
for
early
early
generic
top-level
domains,
the
only
one
to
appear
was
not
info
in
2014
and
2018
next
time.
C
So
for
unusual
top-level
domain
usage,
we
notice
the
country
code
top-level
domain
for
Slovakia
or
SK
2,
ranked
4th
in
both
2017
and
2018.
We
found
over
99%
of
these
DNS
requests
to
actually
be
machine
generated
as
part
of
antivirus
software
used
by
the
University
so
different
from
the
fully
qualified
domain
names
of
usual
websites,
such
as
say,
wo
comm.
We
find
a
fully
qualified
domain
names
of
these
machine
generated
requests
to
be
much
longer
and
containing
seemingly
random
combination
of
alphanumeric
digits.
C
C
We
also
found,
in
terms
of
unusual
clearly
usages
that
the
country
called
pop,
like
the
country
code,
top-level
domains
which
were
not
restricted
to
only
the
citizens
of
that
region
may
be
often
used
as
generic
top-level
domains.
Instead,
for
example,
the
thought
IO
country
code
top-level
domain
for
the
British
Indian
Ocean
Territory
was
topped
here
most
most
query
in
both
2017
and
2018,
and
this
was
not
due
to
interest
in
the
region
itself,
but
rather
because
the
top-level
domain
is
quite
popular
among
startups
and
open
source
projects
online.
C
C
Similarly,
we
also
found
that
you,
the
usage,
can
be
used
by
domain
hack
or
concatenating
sections
of
the
fully
qualified
domain
name
to
form
a
different
name.
So,
for
example,
as
is
commonly
seen
in
youtube
link
shortener
YouTube
PE,
which
uses
the
top
PE
country
code
top-level
domain
for
Belgium.
So
what
may
now
be
a
country
code?
Top-Level
domain
may
be
more
accurately
described
as
a
generic
top-level
domain
in
the
future,
so
the
top
level
domain
classifications
we
identified
earlier
will
need
to
be
regularly
updated
to
reflect
that
changes
next
time.
C
Next
slide
for
these
ranked
frequency
plots,
the
top
one
is
fall.
2013
and
the
bottom
is
for
2018.
So
between
these
weekend,
most
notably
see
growth
in
the
number
of
new
generic
top-level
domains.
We
can
also
observe
there.
There
was
this
zips,
though
in
old
and
early
junior
top-level
domains
and
let
learn
new
Jeff
top-level
domains
are
much
less
concentrated,
as
shown
in
the
flatness
slope
in
the
graphs
next
time.
C
In
addition
to
top-level
domains,
we
also
looked
at
second-level
domains,
so
in
this
example
here
the
dock
hole
for
company
before
that
ends
it
New
Zealand
country,
code,
top-level
domain.
It's
the
second-level
domain
so
similar
to
the
new
gTLD
program
by
I
came
the
second
level
domains
in
New.
Zealand
were
open
to
public
registration
in
2014.
So
this
means
that
instead
of
choosing
from
one
of
15
predefined
second-level
domains,
a
company
can
theoretically
use
any
name
as
the
second-level
domain.
C
C
So
right,
so
one
of
the
potential
problems
from
opening
both
general
top-level
domain
and
second
level
domain
registration
to
the
public
is
that
there
could
potentially
be
overlaps
between
the
tree.
For
example,
QE
is
a
term
used
to
describe
New
Zealanders,
and
currently
there
is
both
a
doc
Kiwi
top-level
domain,
as
well
as
a
doc
Kiwi,
the
NZ
second-level
domain
and
the
New
Zealand.
C
The
takeaways
for
the
domain
names,
so
we
find
nearly
introduced
your
top
level
domains
and
not
ends
it
second-level,
domains
to
grow
in
popularity,
but
still
not
very
heavily
used
yet
advantage
of
new
general
public
or
domains.
Is
that
the
cause?
Is
this
publicly
concentrated
in
the
most
popular
domains?
Cyber
cyber
squatting
on
a
large
skillet?
Theoretically,
that's
feasible
compared
to
cyber
squatting
on
the
older
generic
top-level
domains.
C
So
for
Dina
is
sick.
We
found
the
implementation
in
New.
Zealand
is
growing,
but
chain
of
trust
is
incomplete,
main
problem
being
that
the
incorrect
Deus
trickles,
without
which
it
is
not
possible
to
authenticate
the
public
TS
key
records.
In
the
chasm,
we
found
an
improvement
in
reduction
of
expired
signatures
as
a
proportion
of
the
total
signed
domains
which
we
see
as
a
growth
for
a
basic
implementation
in
New
Zealand
next
time.
C
So,
finally,
the
concluding
remarks.
So
in
this
paper
we
presented
a
longitudinal
analysis
on
the
adoption
of
new
generic
top-level
domains
and
the
deployment
of
DNS
sick
in
new
zealand.
New
generic
top-level
domains
are
growing
in
popularity,
but
still
not
very
popular,
yet
we
believe
there
needs
to
be
more
work
done
on
exploring
the
top
level
domain
topic
further
before
the
second
round
of
new
generic
top-level
domain
applications
in
the
future.
Thank
you.
B
D
E
George
Michael
from
a
pinna
cue
can
hear
us,
yes,
so
I
think
a
study
like
this
needs
to
explore
the
concept
of
D
G,
a
or
generative
domain
names,
hash
names,
which
are
a
mechanism
for
command
and
control
in
botnets
and
so
aspects
of
behavior
in
the
new
gTLDs
relate
strongly
to
compliance
checks
on
how
you
justify
your
need
for
a
name.
The
old
gTLDs
and
ccTLDs
tend
to
be
harder
places
to
get
a
name,
and
the
newer
younger
TLDs
often
are
attractive
targets
for
bad
actors.
E
You
talked
about
ESET
using
hash
names
and
machine
generated
names
for
good
purpose,
but
I
think
you
have
to
address
the
question
of
the
use
of
hash
names
for
bad
purpose.
The
second
thing
is
that
many
of
the
new
TLDs
have
no
intention
of
offering
delegation
services
to
wider
public
industry.
Dot.
Ibm
is
not
an
open
domain,
it
is
a
private
closed
domain
and
its
growth
will
be
a
function
of
IBM's
mission.
It's
not
a
function
of
a
market
process,
it's
a
function
of
their
need
for
labels
under
IBM.
E
That,
in
effect,
is
a
UN
chartered
entity
that
has
standing
as
government,
it's
not
about
a
contract,
binding
or
a
belief
of
meaning
it's
about
the
fact
that
they
have
an
army,
and
so
they
are
only
different
in
the
quality
that
they
own
tanks
and
not
specifically,
that
any
legalisms
govern
what
they
do.
But
your
paper
was
very
interesting
and
thank
you
for
presenting
it.
Thank.
F
F
Hi
now
I'm
on
video.
So
thanks
thanks
for
your
talk,
what
I
would
be
interested
in
and
maybe
you
can
a
little
a
little
bit
elaborate
on.
That
is.
Where
did
you
did
that
get
the
data
or
which
kind
of
data
did
you
have
for
your
analysis?
So
you
talked
about
you
had
a
collaboration
with
the
top-level
domain
registry,
but
what
did
they
gave
you
and
how
did
which?
Which
kind
of
measurements
or
analysis
did
you
do
doing
that
data.
C
F
C
F
And,
as
the
last
remark,
I
just
wanted
to
drop
the
ripe
Atlas
Project.
If
you
want
to
do
again
as
like
measurement
stuff,
you
can
also
do
like
any
kind
of
DNS
records,
they're
also
limited
to
direct
geographical
regions,
or
something
like
this.
That
might
be
interesting
for
future
research.
Thanks.
Thank.
C
G
G
So
it
CCTA
these
are
really
very
wrong
over
the
map.
So,
but
back
to
the
topic,
you
said
that
there
is
a
lot
of
domains
more
domains
signing
the
NSA.
Then
there
is
domains
of
sending
India's
records
and
in
that
I
would
like
to
know.
Do
these
newly
signed
domains
support
the
CD
s
CDNs
key,
because
that
is
then
something
that
registries
could
actually
use
to
foster.
The
adoption
of
the
NSA.
H
I
B
And
if
I
could
ask
one
last
question
your
song,
the
you
mentioned
in
your
takeaways
the
problem
of
the
overlap,
for
instance,
the
doc
heated
at
TVNZ
I,
was
just
curious.
How
effect
how
effective
can
one
be
it
at
identifying
that
those
are
essentially
clones
or
what
are
the
methods
that
you
use,
or
you
can
imagine
we
can
use
to
determine
that?
Are
there
essentially
aliases?
Do
you
have
to
do
active
measurements,
or
can
you
determine
it
passively
from
from
the
registrations.
C
B
K
Hi
I'm,
Tony,
Polly
and
I'm
gonna
be
sharing
some
client
measurements
that
we've
taken
I'm
at
Apple,
and
so
this
is
information
that
we
get
from
looking
at
sampling
from
iOS
and
Mac
OS
devices.
Previously
we've
shared
a
lot
of
information
about
how
we
see
ipv6
adoption,
I'm
gonna
be
sharing
a
little
bit
about
that
now.
But
one
of
the
new
cool
things
that's
happening
is
that
we
see
a
new
protocol
version
TLS
one
three
coming
out
and
ramping
up
and
I
wanted
to
share
some
information
about
how
we
see
this
happening.
K
So
as
some
background
on
what
we're
looking
at
here,
TLS
1/3
was
published
as
RFC.
Eight
four
four
six
last
August
been
in
development
for
a
while,
but
things
were
not
really
I'm
deploying
it
at
marked
scale.
Yet
now,
on
our
clients,
we
enabled
TLS
one
three
by
default
for
pretty
much
all
connections
in
March,
and
so
this
was
my
OS
12.2
had
Mac
OS
10
dot
14.4.
K
L
K
So
the
questions
that
we
asked
before
looking
at
the
data
was
first,
how
much
server
adoption
are
we
seeing
of
TLS
1/3
from
the
perspective
of
the
clients
like?
How
much
is
this
growing
and
then
also
one
of
the
key
things
about
TLS
1/3
from
a
measurement
perspective?
Is
that
it's
supposed
to
have
some
performance
benefits?
In
almost
all
cases,
it
can
shave
off
one
round
trip
from
the
handshake
time.
So
we
want
to
see.
Is
this
actually
playing
out
in
reality?
K
How
is
this
actually
improving
the
overall
connection,
establishment
time
that
we
see
from
the
client
so
before
I
get
into
the
data,
just
some
notes
on
the
methodology?
We
are
collecting
this
data
on
a
per
connection
basis
and
in
this
case
it's
a
per
TLS
connection.
We're
not
looking
at
for
this
data.
Anything,
that's
not
a
connection
that
has
TLS
enabled
on
it,
and
this
is
all
going
over
TCP.
K
What
we're
collecting
is
the
TLS
version.
How
long
the
TLS
handshake
took
the
alp
n
value
that
we
negotiated
over
TLS?
Is
this
HTTP
1
or
HP
2,
as
well
as
information
about
the
address
families?
Are
we
going
over
v4
v6
and
our
v4
v6
enabled
on
the
network
that
we're
connecting
over
and
so
because
of
all
of
this?
What
we're
representing
in
general,
is
the
amount
of
TLS
1/3
that
is
being
used.
K
We
imagine,
of
course,
that
there
is
a
long
tail
of
things
that
have
not
turned
on
new
protocols,
but
that
users
don't
often
access
and
the
data
for
this
presentation
is
being
collected
over
around
a
three
month
period.
This
represents
April
to
June
and
it's
a
random
sample
of
for
those
devices
and
those
users
who
have
opted
in
to
data
collection.
K
All
right,
so
here
we
see
a
trend
over
the
three
months.
We
are
sampling.
The
green
is
the
amount
of
teal
s13
that
we
see
compared
to
the
overall
amount
of
TLS
and
I'm,
only
showing
teal,
s12
and
TLS
1/3.
The
older
versions
are
there,
but
they
are
extremely
tiny.
They
are
less
than
half
a
percent.
It's
not
interesting,
and
so
we
see
a
really
steady
march
up
10%
17%
22%.
K
K
K
So
this
data
is
now
only
looking
at
connections
that
are
established
whenever
on
a
dual
stack
network.
So
we
have
both
before
and
v6
available,
we're
asking
for
both
a
and
quad
8
queries,
and
so
we're
essentially
checking
does
this
server
support
v6
and
we
see
that
of
the
arguably
larger
number
of
v4
only
servers,
only
12%
of
them
by
June
had
adopted
and
enabled
1/3,
but
of
the
servers
that
had
previously
already
been
rolling
out
ipv6.
K
That
number
is
of
third,
it's
at
36%,
so
there's
a
fairly
strong
correlation
there
and
kind
of
looking
at
this.
The
other
way
around
when
we
look
at
the
overall
amount
of
server
enablement
of
ipv6.
Today,
when
we're
on
dual
stack
networks,
we're
seeing
about
40%
of
connections
go
over
ipv6,
but
when
we
sample
only
for
those
that
are
going
over
to
us
1/3.
So
if
people
have
been
upgrading
recently,
we
see
that
at
almost
70
percent.
So
if
you're
looking
at
you
know
new
TLS
connections,
these
are
almost
all
happening
over
ipv6.
K
Similarly,
we
look
at
the
correlations
with
a
OPN.
So
how
likely
are
you
to
support
TLS
1/3
if
you
have
also
upgraded
to
support
HTTP
2
and
again,
it's
actually
really
really
similar
number.
So
if
you
remember,
for
v6
and
before
v4
had
I
think
was
12%
and
v6
had
36
or
38,
and
here
we
see
almost
exactly
the
same
numbers.
K
If
you
are
kind
of
a
legacy
server,
if
you're
doing
HP
1
there's
only
13
percent
of
them
have
upgraded
their
TLS,
but
for
it
should
be
2
connections
over
40%
of
them
are
already
on
TLS,
1,
3
and
again
looking
at
things
the
other
way
we
actually
see
overall
from
our
clients
about
a
50/50
split
of
connections,
use
it
should
be
1
or
hb2
on
any
given
network.
But
if
you
only
look
at
the
connections
that
are
using
TLS
1
3,
we
see
75%
of
connections
are
using
HP
2.
K
So
again,
interesting
correlation
and
really
quite
similar
numbers
here
and
then
the
last
thing
we
want
to
look
at
was
the
performance
to
validate
that
other
question:
are
we
seeing
the
performance
gains
that
we
were
expecting
with
TLS
1
3?
And
here
the
answer
is
pretty
clearly
yes.
So
this
is
a
PDF
showing
the
number
of
milliseconds
that
we
see
for
both
GLS
1
3
and
TLS
1
2
handshakes
and
we
definitely
see
a
much
stronger
skew
towards
the
faster
handshake
times
with
TLS
1
3
than
we
do
with
1
2.
K
So
it
is
clearly
a
benefit
all
right.
So
that's
the
data
we
have
to
share.
So
the
overall
observations
we're
making
is
that
TLS
1
3
support
has
more
than
doubled
over
the
past
3
months.
The
performance
wins
that
we
expected
are
being
clearly
demonstrated
and
we're
getting
a
lot
of
value
here
and
then
the
other
kind
of
correlation
insight
is
we're.
K
Noticing
trends
of
these
kind
of
two
populations
we're
getting
bimodal
populations
of
leading
edge
servers
that
are
doing
TLS,
one
3hp
to
ipv6
and
then
some
straggler
populations
that
are
doing
POS
1
to
HB
1.
That
are
ipv4
only,
and
so
you
know,
client
developers
are
looking
at
needing
to
optimize
for
two
different
populations
here,
and
they
should
be
kind
of
expecting
these
general
groupings,
and
it's
also
I
think
interesting.
K
When
we're
looking
at
new
protocol
adoption
I
know
that
these
three
protocols
represent
definitely
different
groups
of
people
at
the
IETF
and
I
think
it
shows
that
we
have
a
lot
of
common
interest
in
common
patterns
that,
when
we
look
at
the
devices
that
haven't
been
upgrading
to
v4
those
are
kind
of
common
enemies
of
apathy
or
inertia
that
all
of
these
protocols
are
trying
to
fight
against
and
we're
kind
of
all
in
it
together.
In
this
so
questions,
please
hi.
E
George
again
can
I
just
check.
Is
this
a
sample
set
mediated
from
your
experiment
in
the
client,
or
are
you
doing
some
traffic
measurement
at
exchange
points,
or
are
you
bound
on
the
server
side?
What
can
I
might
have
missed
it
mm-hmm
it
informs
the
the
next
question.
So
this
is
your
mechanism
in
Apple
for
measurement
of
client
view
of
server
behavior
right.
So
this
is
on
the
population.
E
E
We're
in
the
domain
of
the
apple
handset
in
mobile,
distinct
from
say,
Apple
and
Android,
you
would
have
some
question
around.
Is
this
a
function
of
update
cycle
and
availability
of
things,
because
this
is
a
client
negotiation
right,
the
server
proffers
and
then
you
have
a
downgrade.
So
it
comes
to
a
question
is
if
only
newer
devices
you
recruit?
K
K
M
B
N
What
Google,
thanks
for
presenting
this
data?
It
was
a
similar
question
about
selection
bias.
Actually,
I
was
wondering
if
you
had
attempted
to
do
a
hold
back
experiment.
Where
say:
half
users
like
did
not
have
TLS
1/3
enabled
but
recorded
when
TLS
3
1/3
was
was
available
and
one
the
other
half
like
actually
used
it
to
get
a
better
idea
of
like
because
I
would
remove
the
server
selection
bias
that
might
otherwise
be
present,
and
we
found
it
for
quit.
N
K
So
earlier
on,
before
we
turn
it
on
by
default,
we
were
doing
probing
essentially
detecting
when
TLS
1/3
was
available,
even
if
we
didn't
use
it
kind
of
just
as
a
product
decision.
We
decided
to
turn
on
full-blast
for
everyone,
because
there's
the
performance
when
we
didn't
want
to
hold
some
people
back.
But
yes,
that's
a
very
good
point.
O
Young
lute,
regarding
your
last
measurements,
with
the
PDFs
I,
was
wondering
you
physically
showed
TLS
1.3
server's
seem
to
be
better
connected
more
up-to-date.
What
kind
of
what
is
the
reason
for
the
PDF
being
so
low?
Is
it
that
they
are
just
better
connected
or
is
it
the
protocol
that
is
better
yeah.
K
That's
again
a
very
good
point:
there's
lots
of
biases
that
get
introduced
this
data
and
I
think
the
other
information
that
we
showed
of
like
these
are
clearly
servers
that
are
adopting
on
the
leading
edge
means
that
they're,
probably
optimizing
other
things
more
yeah
I
think
you
know
at
least
isolated
measurements.
We
definitely
do
see
the
benefit
of
one
RTT.
This
particular
data
is
only
looking
at
the
handshake
establishment
time
right,
so
the
RT
T's
are
a
major
factor
in
this.
So
I
do
think
that
does
dominate
this,
but
you
are
correct.
Thank.
O
P
N
Noted
all
right,
Brian
Trammell,
not
as
Banerjee
Koecher,
despite
the
Hat,
the
with
another
bias
question
which
I'll
actually
freeze
it
as
a
comment
because
I
think
the
answer.
The
first
question
we
know
there
are
a
few
things
that
you
could
look
at
in
their
responses
that
might
identify
server
software
so
that
you
could
essentially
Coralie
some
of
these
sort
of.
Like
you
know,
early
adopters
and
stragglers
by
what
the
software
the
earth
are
running
cuz.
One
of
the
things
that
we
saw
like,
for
example,
with
ecn
is
nobody
actually
cares
about.
N
Turning
it
on
is
just
that
the
defaults
are
better,
and
one
thing
that
would
be
really
great
for
this
community
to
know
is
to
what
extent
is
the
work
that
we're
doing
toward
driving
the
defaults
forward
driving
some
of
this
deployment?
Into
what
extent
are
these
people
who
are
making
actual
decisions
based
on
the
merits
of
Chios,
one
three
Thanks
yep.
Q
Gen-I
and
god
I
was
actually
going
to
say
something
very
close
to
that
which
is
have
you.
Have
you
considered
looking
into
fingerprinting
the
the
SSL
libraries
that
are
being
used
at
the
service
to
see?
There's
a
correlation
between
you
know
if
it
is
possible
that
maybe
does
one
SSL
library
that
happens
to
have
bad
defaults
of
that
and
you
turn
it
on,
and
performance
actually
sucks,
which
is
why
people
have
it
turned
off.
K
And,
of
course,
there
would
be
the
concern
about
kind
of
ossified
on
the
fingerprint
that
the
given
implementation
has.
You
know:
we've
been
seeing
actually
issues
on
the
other
side
of
certainly
firewalls,
our
other
measurement
devices
expecting
certain
patterns
of
our
own
TLS
implementation
and
having
changing
behavior
based
on
that.
So
it's
tricky
area
so.
K
R
D
I'm
Corey,
first
and
yep,
we'll
talk
about
quick
I'm
going
to
talk
about
a
some
results
from
a
test
bed.
We
built
we
use
vagrant
to
configure
some
computers,
virtual
machines.
We
ran
tests
with
quick,
we
use
quickly
and
thank
you
for
the
people
who
helped
us
do
that
and
we
put
a
real
Hardy
light
path
between
our
two
endpoints.
This
work
was
jointly
done
by
me
and
Tom
we're
all
from
Aberdeen
University.
So
this
is
a
summary
plot.
D
This
is
a
summary
plot
of
a
lot
of
experiments
and
you
can
see
maybe
look
at
me
right,
left
box
first
and
you
can
see
red,
which
is
straight
using
TTP
to
do
a
W
at
this
time
of
100
kilobytes
across
the
satellite
link,
and
you
can
see
they
takes
six
seconds
to
transfer.
There
are
some
outliers,
and
this
is
working
across
the
hall
access
path
from
the
satellite
and
the
internet
and
doing
the
measurement
across
the
path
they
could
go
to
the
green
box
in
the
left.
One.
D
The
green
box
shows,
with
TCP
using
TLS
1.2
to
get
the
same
thing,
and
the
green
box
shows
the
advantage
of
a
pet,
the
performance
enhancing
proxy,
which
the
satellite
operators
put
in
their
network,
and
you
can
see
it's
much
better
on
the
left
with
the
green
than
the
red.
So
the
real
question
is
now:
what
does
quick
doing?
What
quick
has
to
use
the
full
path
it?
It
has
to
go
over
the
satellite.
The
satellite
can't
benefit
it,
so
it
does
quite
well.
D
It
does
almost
as
good
as
the
pepper
so,
which
is
really
good
news
thanks
everyone
for
working
with
quick.
However,
it's
not
quite
as
good
as
per
happen,
and
you
know
little
bits
make
a
big
difference
when
you
do
web
performance.
Let's
look
at
a
bigger
transfer,
that's
the
one
on
the
right,
though
you
see,
the
red
plot
goes
a
little
bit
wild.
Why?
D
Because,
when
you're
using
TCP
and
you've
got
a
high
bandwidth
delay
product
and
something
goes
wrong,
the
pathology
can
go
many
different
ways:
ssthresh
can
get
clamped
low
and
all
sorts
of
things
can
happen.
So
big
variants
on
the
right,
quick
did
well,
but
not
quite
as
well
as
TLS,
1.2
uh-huh
just
a
minute,
and
we
had
a
previous
talk.
What
about
1.3?
D
D
That's
all
right,
let
me
tell
you
more
about
what
we
are
actually
using
as
our
link.
It's
a
satellite
access
link,
it's
a
commercial
offering
we're
using
an
engineering
platform.
The
forward
link
has
a
bit
rate
nominally
of
10
megabits
per
second,
and
you
can
easily
get
8
point.
5
megabits,
using
iperf
across
this
reasonably
good
service
for
an
isp
returnin
link,
is
1.4
megabits,
pretty
good
and
not
as
much
here
symmetry
as
many
people
will
have
at
home.
D
If
they're
using
soft
light,
but
I
mean
that's,
ok,
bandwidth
delay
for
the
very
large
majority
of
packets
was
just
over
550
milliseconds.
However,
let's
look
at
some
long-term
measurements
of
what
actually
the
RTT
was
you'll
see.
The
blue
bar
is
the
average
RTT
and
it's
a
high
bandwidth
delay
product
link
nearly
always
at
around
5
50
milliseconds,
but
there
are
many
outliers.
This
is
because
there's
a
radio
link
underneath
it's
trying
to
merge
the
resource,
occasionally
you'll
get
resource
requests
which
can't
be
satisfied
because
got
a
contention
busy
time
to
the
day
work
badly.
D
Occasionally,
resource
requests
will
fail.
You
have
to
try
second
time
to
get
resource
or
your
partner
traffic
changes
abruptly.
So
these
are
single
packet
measurements.
You
see
these
large
outliers,
so
I
assert.
This
is
a
wonderful
test
case
for
testing
a
quick
implementation.
If
you
want
to
push
this
implementation
in
a
weird
way
check,
all
the
time
is
work
check
all
the
transmission
stuff
work.
This
is
a
great
environment
and
it's
real
so
we're
carrying
on
doing
this.
D
We've
done
a
lot
of
measurements
and
one
of
the
things
we've
done
is
plotted
and
TCP
as
sequence
number
divided
by
MSS,
because
that's
a
thing
that's
roughly
equivalent
to
the
quick
packet
number
and
you
get
plots
that
look
like
this
and
okay
I
mean
there's
lots
of
these.
There's
lots
more
in
my
dataset.
D
If
you're
interested
in
these
come
and
talk
to
me-
and
maybe
next
ITF
will
organize
a
side
meeting
to
go
through
this
sort
of
data
with
multiple
people,
I'm
not
going
to
take
the
question
to
the
end,
because
I'm
going
to
another
a
rather
different
notes
where
I'm
going
this
is
the
download
and
yeah
all
the
time
quick
does
well.
Sometimes
it
didn't
do
as
well.
Remember
TCP.
Has
this
pathology
as
well,
so
what
we
need
to
do
now
is
look
carefully
at
exactly
what's
going
on.
The
link
logging
is
wonderful.
D
We
can
get
a
lot
about
lot
of
logging
out.
We
can
really
drill
into
it,
and
lists
are
not
wonderful,
so
we
have
to
figure
out
how
best
interact,
how
timers
are
interacting
yeah?
Okay,
so
we
got
lots
of
data.
What
did
we
conclude?
I
thought
I
might
get
some
people
to
like
so
I'm
going
to
be
quick,
quick
quickly
was
wonderful.
D
It
was
a
great
experience
using
it
Thanks
right
and
no
complaints
about
this,
and
we
have
you
to
the
logging
available,
but
then
we're
always
already
talking
about
what
we
should
log
TLS
1.3
had
a
two
RTT
advantage,
one
or
two
RTC
advantage,
always
always
better.
Always
good.
The
fullness
of
quick
or
satellite
was
not
as
good
as
for
TCP
with
a
pet,
but
it's
within
shooting
range,
so
I'd
really
love
to
try
and
get
the
NIT
signed
out.
So
it
is
as
good
downgrading
to
TCP
is
not
a
long-term
solution
for
these
people.
D
These
people
are
currently
actually
pushing
you
to
use
their
pep
in
many
cases,
because
it's
a
predictable
environment,
that's
got
a
stop.
We've
got
to
enable
quick
of
this
network
to
do
that,
we
must
make
quick
work.
Well,
do
that
we
must
understand
quick,
well
and
understanding.
Performance
issues
falls
into
three
parts.
So
this
just
probably
implementations
I
mean
we're
getting
there,
but
there
are
things
that
could
be
changed.
D
We
just
started
I'm
trying
to
say
how
exciting
this
is.
We
just
started
will
continue
doing
measurements,
we're
going
to
talk
about
logging
and
tracing
because
we're
currently
building
our
own
tools.
They're,
not
bad
they're,
passing
JSON
they're,
taking
peak
ups
for
the
TCP,
putting
them
together,
the
Q
log
stuff
could
be
really
fun
and
if
you're,
a
satellite
person
or
a
radio
person-
and
somehow
we
can
run
our
tool
across
that
using
a
virtual
testbed
that'll
be
cool
to
get
a
wider
spectrum
of
results.
D
S
You
going
it's
actually
a
great
great.
Thank
you
did
that,
and
it's
particularly
great
to
of
the
tresses
of
people
can
do
a
simulation
and
chunder
starts
the
the
one
the
little
caveat
I
have
with
your
talk
is
that
you
are
mentioning
quick
when
in
fact,
you
are
testing
a
specific
version
of
a
specific
implementation
and
I
about
human.
We
add
the
19
implementation
of
quake,
a
truss
interrupt.
If
we
tried
all
19
of
them
would
get
19
different
results.
D
D
I
checked
from
that
a
couple
of
points,
and
one
of
them
being
that
maybe
next
idea,
people
who
are
doing
similar
things
with
all
the
other
scenarios
or
other
quick
implementations,
let's
get
together,
let's
spend
some
time
actually
drilling
into
that,
not
at
the
protocol
level,
really
at
the
results
of
what
we're
seeing
to
actually
find
out.
Why
and
how
yep
thank.
Q
Partly
because
and
I
do
to
Christine's
point
you're
using
quickly,
so
I
will
say
you're
using
the
right
implementation,
I'm,
biased,
because
I
implemented
the
loss
recovery
in
and
so
I
must
partly
partly
responsible
for
some
of
the
performance
you
are
seeing,
which
is
also
why
I'm
trying
to
understand
what
the
pep
is
doing.
So
if
the
pepper
is
so
I,
don't
really
understand
that
that
itself
that
is
being
deployed?
Is
it
a
full
Terminator?
Is
it
a
or
is
it
just
doing,
retransmissions
HACCP.
D
D
Q
L
Q
D
Q
D
Q
Do
big
things
one
I'm
super
happy
to
continue
working
on
this,
and
you
know
this
is
happy
to
take
this
back
and
sort
of
fix
the
quickly
implementation
for
any
bugs.
But
second,
we,
you
might
know
that
we
have
Martin,
Simon
and
I,
have
been
working
on
a
simulator
environment
for
testing
these
things,
so
it'd
be
good
to
actually
validate
this
stuff
with
the
simulator
as
well
to
see
if
we
can
replicate
the
same
results
there.
Excellent.
Thank
you.
N
Yes,
right,
Google
I
want
thanks
for
two
things:
one
for
not
using
the
chrome,
quick
implementation,
because
so
many
papers
and
benchmarks
have
been
based
on
it,
and
you
know:
there's
always
my
personal
concern
that
that's
overfitting
and
implementation,
and
maybe
we
have
some
bug
that
turns
out
to
be
a
performance.
Enhancer
number
two.
R
Great
work
gory,
please
keep
it
up.
Do
you
know
the
weather?
Aqm
is
in
use
in
the
networks
or
any
of
this
measurement.
They
were
not
in
these
experiments.
Okay,
so
I'd
be
interested
in
seeing
how
that
compared,
and
the
other
thing
that
I'd
be
interested
in
in
your
future.
Work
looking
at
is
comparison
of
fairness
on
the
tour,
the
terrestrial
network
on
the
far
side.
If
how
quick
behaves
compared
to
sort
of
stock,
TCP,
yeah.
T
Hey
Colin
Perkins,
with
no
hats
and,
first
of
all,
thank
you.
Great
work.
I
can
a
slightly
like
to
echo
the
point.
Christian
was
making
ever
seen
a
lot
of
or
heard
a
lot
of
people
having
read
particular
papers
which
talk
about
particular
quick
implementations,
wildly
over
extrapolate,
the
results,
and
it's
something
that
that
I
don't
know
whether
it's
this
group
or
the
quick
group
might
consider
putting
out
a
statement
on
how
to
benchmark
quick
and
how
to
report
results.
L
T
How
to
how
to
describe
what
you
are
benchmarking,
what
you're
evaluating,
because
I
think
it
is
harming
the
perception
of
quick
and
what
people
are
doing.
Experiments
with
very
early
implementations
of
quick
comparisons
are
very
highly
up
to
most
TCPS
and
writing.
What
looked
like
very
general
conclusions
when
they're,
actually
much
more
specific
yeah.
D
That's
one
of
the
reason
want
to
stand
up
here
because
we
pull
quickly
and
compile
it
and
run
the
test,
so
we
want
to
keep
it
way
up
to
date,
and
anything
we
do
is
patching
that
because
I
think
that's
the
only
way
to
do
it
if
you're
benchmarking
it
now,
please
don't
use
a
G,
quick
benchmark
as
your
baseline.
It's
different
thing,
the
things
that
moved
on.
We
really
need
to
make
sure
we
track
the
latest
version
in
the
quick
way
and.
T
N
Yes,
what
I'm
here
to
talk
about
our
migration
towards
ITF,
quick
from
Google
book,
so
so
where's
chrome,
now
be
46,
which
is
a
Google,
quick
version
that
has
no
real
meaning
but
I'll.
Explain
to
you
approximately.
What
it
is
is
now
a
default
enabled
we
also
enable
be
39
and
43
on
the
server
at
the
moment.
Be
46
is
in
variants,
draft
three
and
below
compatible,
and
it
has
transport
draft
nine
packet
types
which
I
think
actually
are
maybe
but
still
the
same
packet
types
that
we
have
now.
N
A
future
version
will
be
in
variants
for
compatible,
which
is
the
new,
but
that
changes
everything
by
like
one
byte,
which
is
a
rather
relatively
minor
change,
technically
but
I'm
sure
when
we
deploy
that
a
bunch
of
other
things
will
break
before
t6
does,
if
a
quirk
that
actually
only
supports
Google,
quick
style,
eight
byte
connection
IDs
in
one
direction
and
zero
in
the
other.
That
was
just
a
convenience
because
it
allowed
us
to
delay
if
we
went
to
call
the
variable
in
connection
IDs,
which
took
a
ton
of
time.
Thank
you,
david
scamozzi.
N
I
don't
know
if
you're
here
so
the
rest,
this
talk
is
all
about
b43
to
be
46.
So
if,
if
you're
wondering
like
what
versions
are
what
transition
is
you
talking
about?
That's
what
I'm
talking
about
the
first
thing
is
the
change
from
public
reset,
so
Google
quick
has
an
unauthenticated
public
reset
that
sent
anytime.
You
want
to
close
a
connection,
and
you
don't
have
any
state
or
you're,
too
lazy
or
whatever,
but
basically
it's
like
a
TCP
reset
connection
closed
during
the
handshake
and
a
stateless
reset,
post,
handshake
and
idea.
N
N
Also,
why
does
it
matter
so
handshake,
timeouts
and
I?
All
timeouts
are
quite
a
bit
longer
than
getting
an
immediate
reset
in
particular,
idle
time
out
of
our
default
is
30
seconds
right
now,
that's
extremely
user
visible.
So
you
know
getting
this
wrong
even
like
a
few
percent
of
the
time
is
a
is
a
major
source
of
user
visible
like
problems
you.
N
The
connection
as
quickly
as
possible
any
sort
of
circumstances.
So
when
this
and
what
and
do
you
quick,
if
we
have
no
state,
we
just
send
a
public
reset,
that's
pretty
straightforward,
ITF,
quick
header
in
their
state.
You
send
a
Salus
reset.
If
it's
a
long
header,
no
state
and
its
initial,
we
try
to
start
creating
a
connection.
It's
a
long
header
in
those
state
and
it's
handshake.
N
N
Hopefully
that
will
not
turn
out
to
be
a
practical
issue.
So
let's
talk
about
quick
identification.
It
turns
out
at
least
one
major
company
and
increasingly
more
large
companies
are
trying
to
do
quick
identification.
Sometimes
this
is
to
just
give
you
network
monitoring
tools.
Like
your
network.
Is
this
much
HBS,
and
this
much
you
know
quick
in
this
much
DNS
and
this
much,
although
other
stuff,
and
so
it's
also
for
selective
traffic.
N
You
know,
and
enforcement
basically
like
I,
want
to
disable
quick
or
I
wanted
disable
quick
if
I
can't
get
BS
anion,
which
we'll
talk
about
later
so
be
46
when
you
started
rolling
about
at
a
very
large
increase
in
post
hinchik
black
horn,
so
it
was
about
2
X,
be
43,
I'll,
explain
it
a
little
bit
later,
how
we
determine
post
and
shape
by
:.
But
again
this
is
one
of
those
things
we
draw
closely
because
it's
very
user
visible
and
waiting
30
seconds
for
your
connection
to
timeout
before
reissuing.
N
The
request
of
our
new
connection
is
really
really
painful.
It
also
turns
out,
historically,
these
have
been
user
focused.
So
a
given
user
will
experience
this
all
the
time
and
the
other
99.9%
of
the
population
will
never
ever
experience
this.
So
it's
not
like
every
once
in
a
while.
A
web
page
is
slow,
it's
like
one
user
is
having
like
a
bad
month
or
a
bad
year.
So
understandably
we
don't
like
this.
N
The
confounding
factor
is
sometimes
the
network
just
goes
away
right
like
this
happens,
and
so
we
have
this
base
level
of
noise.
That
we
assume
is
the
network
going
away,
but
we
have
no
real
proof.
It's
it's
a
very
you.
Never
getting
this
to
zero
right,
like
especially
with
mobile
devices
like
Android
and
other
things,
and
this
height
is
higher
on
Android,
for
understandable
reasons.
N
Let's
first
talk
about
what
what
too
many
are
tears,
and
then
we
can
tell
you
what
happened
on
the
fifth
RTO
ng-click.
We
found
that
that
was
a
reasonable
signal
for
closing
the
connection.
Kazoo
is
black
hold
as
I
know
at
the
bottom
5
RTO
is,
and
our
current
implementation
is
kind
of
comparable
to
seven
RTOS
in
you
know,
linux,
which
I
think
is
some
time's
configured
as
a
default.
So
you
know
we're
not
so
far
off.
N
N
So
when
you
started
debugging
this
we're
like
how
can
we
get
more
data
about
when
this
is
happening
in
a
connection
and
one
of
our
team
members
have
the
great
idea
that
we
should
add
the
number
of
connections.
Sorry
number
of
packets
that
have
been
sent
or
received
before
closing
the
connection
due
to
too
many
RTS
and
on
the
client
side,
the
receive
metric
was
was
very
indicative.
As
you
can
see,
all
the
extra
black
holing
energy
is
all
between
two
three
and
four
packets.
N
N
Unfortunately,
this
typically
happens
after
you've
completed
the
handshake,
because
four
packets
is
more
than
enough
to
complete
the
handshake
in
Google,
quick.
So,
typically,
the
difference
between
two
and
three
and
four
is
like
this
is
a
case
of
you:
don't
do
zero
RTT,
you
get
a
packet
and
then
that's
a
reject,
and
then
you
do
burn
a
round-trip,
and
then
it
sends
you
to
the
cert
and
that's
somewhere
between
one
and
three
packets,
just
as
an
explanation.
So
that's
why
sometimes
two
three
or
four,
but
they
said.
N
N
N
L
N
You
ensure
that
all
packets
in
one
direction
or
the
other
are
blocked
like
doesn't
matter
which
direction
like
don't
care.
You
can
let
all
the
client
packets
go
through
and
drop
all
those
server
packets.
That's
fine,
just
try
to
drop
all
of
them,
because
anything
else
is
really
user.
Visible
and
chrome
now
has
some
heuristics
to
try
to
avoid
networks
like
this
and
avoid
using
click
on
them,
but
it
basically
goes
from
consistent
and
utter
failure
to
flaky,
hangouts
or
hangs
and
connection
like
loads,
followed
by
like
it
works
suddenly,
and
you
don't
know
why.
N
N
We
eventually
actually
traced
it
to
a
single
company.
At
the
time
we
found
this
out.
V
46
wasn't
blocked
so
we're
like
hey.
Maybe
they
read
the
CFM
variants
graphed
and
said
not
to
drop
the
idea
from
variant
correct,
but
decided
to
drop
Google.
Quick
I
mean
this
is
like
me,
like
thinking
my
most
optimistic
thought.
This
was
not
true.
N
This
was
just
completely
false.
We
turned
on
V
46
and
two
weeks
later,
they
like
stopped
started,
walking
that,
and
the
last
thing
is
anytime.
You
change
the
location
of
sni,
you're
gonna
break
a
bunch
of
stuff,
there's
a
few
more
versions
left,
so
we're
gonna,
probably
break
some
other
things
and
I'm
really
sorry
and
it's
a
hard
transition.
But
we
we're
trying
to
do
this
as
fast
as
we
can.
Okay,
so
I'll
stop
and
give
questions.
Q
Thank
you
heal
from
bringing
this
bringing
this
here.
This
is
something
that
I
been
sort
of
sitting
on
the
edge
of
my
seat
for
a
while
to
watch
what
happens
when
the
G,
quick
format
changes
and
as
expected,
it's
already
been
ossified.
This
is
this
is
like
this
is
exactly
what
we
were
afraid
of,
and
this
is
exactly
what
you
worried
about
and
it
seems
like.
Maybe
some
of
them
are
actually
doing
updates,
which
is
promising,
but
that's
something
that
you
mentioned.
Q
So
the
the
point
that
you
are
showing
there
with
the
energy
in
too
many
audios
is,
is
in
the
early
key
packets
there.
So
that
suggests
a
particular
type
of
behavior
in
middle
box
right
where
they
allow
a
few
packets
to
go
through
and
then
die,
and
we've
we've
seen
this
before
and
I
wonder
if
this
is
actually
it's
important
enough
now,
I
mean
missing
this
again
here
and
then
one
thing.
Q
This
is
important
enough
that
this
is
the
sort
of
thing
that
you
want
to
protect
against,
meaning
that,
if,
if
we
keep
thinking
that
all
packets
are
going
to
go
through
or
none
of
them
are
going
to
go
through,
it
may
not
be,
it
may
not
be
compatible
with
the
way
that
middle
boxes
build
their
architecture,
because
I
imagine
that
some
of
this
is
basically
like.
You
know
the
forwarding
thread
sort
of
parting.
N
N
N
The
intention
in
this
case
was
they
had
a
Enterprise
policy.
They
had
both
monitoring
consoles
and
an
enterprise
policy
and
enterprise
policy
allowed
you
to
drop
quick
and
the
way
they
drop
unknown
protocols.
Is
they
give
it
a
few
packets
in
each
direction?
And
then,
if,
after
two
or
three
packets,
they
haven't
figured
out
what
protocol
it
is,
then
they
start
dropping
it,
and
so
quick
classified
in
be
46
as
an
unknown,
critical
and
then
get
into
this
weird
bin.
So
that's
the
other
thing
I
was
going
to
comment.
N
Is
this
isn't
good
for,
like
any
transport
I
seriously
like
like
unis
people
are
thought
like
whether
it's
you
know
SCTP
over
DT
OS,
like
all
these
things,
have
the
property
that
like,
if
you
complete
the
handshake
and
then
black
hole
it
it's
going
to
be
like
a
really
bad
time.
So
for
the
general
interest
of
transport,
innovation
on
UDP
across
the
entire
internet,
I
would
like
people
to
try
to
avoid
doing
this.
It's
not
just
for
quick
I
mean
this,
isn't
just
for.
Q
N
Q
Little
bit
more
yeah,
it's
basically,
it
seems
like
so
what
I?
What
I've
seen
people
talk
about
is
that
there
are
console
on
the
console
of
the
middle
box.
It
basically
says
packet
received,
cannot
classify,
cannot
classify,
and
then
it
drops
it.
So
it's
possible
that
it's,
you
know
trying
to
do
some
classification
on
a
separate
thread
and
that
it
eventually
times
out
or
gives
up
and
then
says,
I
am
done,
and
then
it
decides
black
hole
but
yeah.
This
is
a
huge
issue.
A
K
N
K
Thing
yeah
cuz
you'd
mention
the
kind
like
the
background
level,
because
we're
always
gonna
have
some
things
that
happen
to
drop
off
after
four
packets.
But
it's
not
really
possible,
then
to
look
at
kind
of
all
the
parallel
connections
and
you
can
see
if
oh,
if
this
other
connection
dropped
off-
and
it
was
further
along-
maybe
that's
just
a
network
impairment,
but
that
would
not
work
in
this
case.
Yeah.
N
I
mean
we
don't
really
mean
our
client
side
is
not
fine
enough
grain
to
actually
even
prove
that
back
but
yeah
I,
don't
know
yeah
unfortunate
I
think
we
can
say
for
sure
whether
or
not
that's
true
yeah
I
burn
a
trammel,
quick
question.
I'll
take
the
rest
of
this
offline,
probably
as
the
co-editor
of
the
quick
manageability
drop.
I'm
gonna.
N
Follow
you
a
couple
of
issues
to
take
this
presentation
into
that
draft
the
so
you
said
that
the
dropping
all
of
the
packets
for
a
connection
is
much
better
than
dropping
some
of
the
package
for
the
connection
from
user
visibility.
That's
kind
of
obvious
and
I
think
we
should
actually
kind
of
write
that
down
and
yell
it
from
the
tree
cops
in
the
IETF,
and
then
you
said
that,
like
so
for
one
of
these
you're,
basically
you
get
the
first
three
packets
and
then
you
drop.
N
Are
you
seeing
other
things
where
you
basically
get
like
intermittent
drops
after
you
have
actually
like
spun
up
data
like
so
data
channels?
Like
you
get
like,
you
know,
you
get
some
actual
data,
it's
like
oh
great,
great,
now,
quick
works
and
then
you
drop
every
other
packet
or
I
mean
like
look.
Can
you
characterize
a
little
bit
more?
The
kind
of
drop
patterns
that
you're
seeing
we
we
do
see
some
level
of
other
like
well
post,
inject
dropouts,
yeah
anecdotally.
It
seems
to
mostly
be
not
related,
we're
out.
N
And
the
fix
for
that
is,
if
you
like
typically
happens
in
the
following
circumstance:
it's
more
like
the
than
that
times
out.
The
server
is
sending
a
bunch
of
packets
to
the
client.
The
client
stops
sending
for
some
period
of
time
and
the
net
timeout
instead
of
being
a
you
see,
timeout
is
a
fixed
timeout,
so
it's
like
yeah.
You
know
at
60
seconds
I'm
going
to
time
out
this
UDP
like
a
couple
instead
of
after
60
seconds
of
idle
I
will
do
this
I
then.
N
B
Okay,
so
we
see
that
you
guys
ask
a
lot
of
great
questions.
We
have
50
minutes
of
content,
this
fit
into
45
minutes
right
now,
so
we're
gonna
ask
Alexander
who
had
22,
maybe
17
minutes
or
something
like
that
and
similar
for
Roland.
Afterwards,
you
cut
a
few
minutes
off
so
that
people
can
ask
you
questions,
but
otherwise
up
next
is
Alexander
about
packet
loss
signaling
for
encrypted
protocols.
U
U
It
may
seem
a
bit
strange,
but
we
are
not
completely
aware
of
what
happens
in
all
nodes
of
our
networks.
That's
a
day
to
day
truth.
So
maybe
for
many
people,
networks
are
just
dumb
pipes,
but
it's
only
true.
If
someone
can
look
at
all
the
nodes
and
if
you
get
some
Oh
am
I'm,
something
that
tells
you.
Okay,
I
got
some
drops
on
that
interface
and
you
have
app
sorry,
no
blind
spots.
It
might
work,
but
in
real
world
in
the
real
world,
it's
not
the
case.
U
So
what
we
do
is
typically,
we
do
some
liquor
to
me.
We
use
some
observer
in
the
middle
of
the
path
and
try
to
say
if
some
problem
occurs
upstream
or
downstream
from
that
point
and
then
reiterate,
we
do
this
in
a
day
to
day
basis
and
have
been
done
so
fro
30
years
on
TCP,
obviously
using
sequence
number
and
X
that's
trivial.
But
what
do
we
do
with
the
encrypted
transport?
Very
quick!
U
That's
a
much
tougher,
of
course.
So
quick,
as
you
may
know,
has
something
for
RT
t
measuring
RTG
with
the
spin
bit,
but
nothing
for
us
for
now.
So
we've
got
a
proposal
to
to
solve
the
problem,
which
is
that,
based
on
only
two
lost
bits,
which
we
named
Q
and
L,
the
Q,
which
is
square
signal
and
L,
which
is
the
lost
Evan
signal.
The
square
signal
is
just
one
toggle,
every
n
packets.
So
that's
like
you're
the
one.
U
The
alternate
packet
packet
marking
also
something
mentioned
by
kazuo
yeah
a
year
ago
about
earlier
in
earlier
loss,
detection
discussions.
So
the
U
cube.
It
is
just
flipped
every
N
packets.
So
when
you
look
at
it
at
a
middle
point,
you
just
count
the
sequences
and
know
how
many
packets
have
been
lost
upstream
from
the
points.
The
last
seven
bit
is
fed
by
the
stack
itself
and
set
to
1
whenever
some
earlier
packet
has
been
considered
lost.
So
by
counting
both
of
them,
you
get
the
upstream
loss
and
the
end
to
end
nose.
U
U
An
experiment,
an
orange
network
through
real
customers
with
thousands
of
Relkin
customers
in
four
countries
with
Akamai
CD
ends
so
Akamai
CD
ends
use
a
quick
stack
which
is
based
on
chromium
with
proprietary
patches
and
in
it
we
implemented
the
the
QL
bits.
As
you
might
know,
in
the
current
quick
specification,
it's
not
possible
to
add
those
bits
anywhere
in
the
packet,
so
we
did
use
some
ugly
level
free
trick.
You
poked
those
bits
in
the
IP
header
in
the
TTL
high
high
order
bits
this
a
lot.
This
is
completely
ok.
L
U
Supposed
to
be
generalised,
but
that's
a
way
of
doing
a
real-life
experiment
today,
with
negotiation
with
version
negotiation
in
quick,
it's
absolutely
not
possible
to
do
a
large-scale
experiment
for
anybody,
but
people
holding
a
browser
or
popular
application.
We
do
not
have
any
of
them,
so
we
were
first
to
do
that.
U
Okay,
the
topology
of
the
experiment
is
the
same
for
all
four
countries.
You
typically
have
an
observer
point,
which
is
just
below
the
Akamai
CBN's
serving
the
access
network,
the
suspected
usual
suspect
in
network
troubleshooting,
of
course,
the
internet
paths
and
the
access
network
paths.
So
whenever
we
start
an
investigation
in
our
networks,
even
today
with
TCP,
we'll
always
start
at
this
point,
because
the
key
question
to
us
first
is
whether
the
responsibility
is
ours
or
not.
U
L
U
So
what
we
do
to
gather
data
at
the
observation
point
is
we
do
traffic
capture
with
step
aggregators
and
we
only
capture
the
downlink
pass?
We
don't
need
the
lag
flow
and
also
we
truncate.
The
full
payload
is
completely
useless.
So
we
only
gather
packet
summaries
with
timestamps
IP
ports,
the
qnl
bits
and
the
payload
size
for
debugging.
The
segmentation
works
by
top
hole
and
time
out,
obviously,
because
the
we.
U
One
compact
text
file
per
flow,
or
would
that
happens
and
then
do
various
tests
with
the
post-processing
of
the
signals
so
both
based
on
those
files.
You
will
see
that
both
statistical
and
unit
analysis
can
be
performed
so
here
before
going
into
the
analysis.
I'll
adjust
the
plotter
CDF
of
the
flow
size,
because
the
flow
size
is
important,
as
you
know
that
the
cubit
just
counts,
sequential
consecutive
packets,
with
the
same
value.
Of
course,
if
you
got
fewer
than
64
pi
packets
in
the
flow,
you
see
nothing.
U
So
what
I've
done
here
is
plot
the
value
of
640,
which
is
1010,
1/2,
K
odds
of
Q,
which
gives
a
boundary
and
the
under
image
that
the
last
chunk
the
importance
sadistically
importance
of
last
rank,
which
obviously
is
incomplete.
So
if
you
just
clip
the
data
set
above
640,
you
get
roughly
10%
in
the
in
the
worst
case.
But
that's
enough,
given
the
number
of
point
captured.
U
So
the
review
the
refusal
we
used
to
exploit
the
desert
are
of
two
kinds.
First
is
statistical.
You
watch
which
calculus
kattappa
lots,
so
we
just
give
the
very
dimensional
distribution
of
the
loss
derived
from
the
queue
and
from
the
l
bits
and
then
the
unit
view
if
we
want
to
debug
what
happened
on
that
point.
On
this
connection,
we
just
plot
an
individual
cumulative
display
of
the
Dallas's,
which
is
over
the
time
in
the
connection
how
many
packets
were
lost,
other
Angwin
or
upstream.
U
So
when
we
look
at
the
first
kind
of
plot,
when
it
is
we
analyze
it,
this
way
we've
got.
Ab
sum
of
stream
runs
against
end-to-end
us.
The
diagonal
of
course
means
pure
upstream
loss,
because
when
the
packet
is
it
is
plus
upstream
is
also
appears
in
the
end
to
end
the
vertical
is
pure
downstream
loss.
It
appears
only
on
the
end-to-end
measurement
and
not
at
all
on
the
cubit,
but
the
bottom
right
is
supposed
to
be
impossible
right.
S
U
We
will
see
well
then,
when
we
do
unit
analysis,
you,
you
select
one
of
these
points
and
look
at
the
the
way
that
the
laws
are
spread
over
time
in
any
one
given
connection.
In
this
example,
you
can
see
that
both
are
completely
synchronized,
which
means
that
it's
pure
upstream
loss,
which
is
every
time
there's
a
cat
which
is
lost.
It's
also
lost
end-to-end
in
this
other.
U
See
only
end-to-end,
absolutely
no
upstream,
so
it's
pure
downstream
this
case
you
see
something
which
shouldn't
happen,
which
is
no
end
to
end
and
they're
completely
insane
and
concentrated
upstream
loss,
which
this
case
was
based
on
miss
segmentation.
The
heuristics
applied
to
the
signal
counting
the
qubits
is
supposed
to
do
something
about
reordering
I
get.
U
U
Now
we
are
going
to
look
at
the
four
countries,
so
these
are
the
four
scatter
plots
quickly.
You
can
see
that
there's
one
of
them,
which
is
an
outlier.
The
bottom
left,
is
completely
insane
and
we
see
why
the
first
one
country,
one
as
you
can
see,
is
concerned
the
main
energies
on
the
vertical
axis,
which
is
pure
downstream
loss.
U
There
are
a
few
points
on
the
diagonal
which
are
really
under
they're
gonna,
which
in
some
events
of
upstream
loss-
and
there
is
also
some
noise
in
the
dragon
area,
which,
after
unit
analysis,
have
been
shown
to
be
due
to
reordering.
So
again,
that's
due
to
when
reordering
happens,
the
boundaries
consecutive
packets
with
Q
equals
zero
and
one
must
be
somehow
accounted
for
one
country.
True,
it's
much
cleaner.
The
noise
from
razoring
is
very
low
and
we
also
seem
some
energy
on
vertical
axis
and
a
few
spots
on
the
diagonal
winning
events
of
strong
actionless.
L
U
U
Flows
on
the
on
the
same
capture
pass
from
TCP.
It
was
shown
that
it
was
completely
saturated
capture
chain,
so
the
method
also
is
able
to
detect
something
that
TCP
could
allow,
which
is
detected
when
your
measurement
purchased
doesn't
work.
Country
3
is
a
cleaner
tool
and
a
few
outliers
due
to
musical
notation.
U
Okay.
So
to
summarize,
what
we've
seen
in
the
dragon
area?
Sometimes
you
some
noise
is
due
to
imperfection
of
the
denoising
heuristics.
Some
is
due
to
miss
Eckman
session
and
some
is
due
to
observe
all's,
which
obviously
should
not
happen
in
your
life
and
the
summer
is
all
these
noises
can
be
disambiguated
by
unit
analysis.
So
we
are
not
at
all
blind
and
also
the
heuristics
have
been
quickly
developed
and
could
be
improved
and
reduce
the
noise
as
a
wrap-up.
U
The
unilateral
deployment
was
made
possible
only
because
we
use
we
didn't
use
the
quick,
vague
vehicle
itself.
That's
all
hack
to
that,
and
that's
the
only
way
to
do
a
large-scale
experiment.
The
signals
were
designed
to
restore
disappear
like
ability
to
debug
networks,
and
they
do
worse
and
short
of.
U
Support,
we
are
not,
we
do
not
expect
to
get
in
anytime
soon
and
other
vehicles
is
needed,
so
any
suggestion
are
open
and
the
mechanism
can
be
applied
to
any
other
protocols.
Quick
centric.
The
only
contribution
that
is
expected
from
the
stack
is
the
Elbit,
which
is
the
the
mechanism
by
which
the
stack
tells
that
it
has.
D
U
N
Yes,
but
Google
thanks
for
this
data,
this
looks
certainly
interesting.
One
question
I
had
is
for
the
spin,
but
there's
a
fair
amount
of
analysis
of
what
happens
if
one
or
the
other
one
of
the
two
end
points
like
tries
to
for
lack
of
a
better
word
subvert
the
signal
it
seems
like
it
would
be
very
easy
for
either
end
point
to
sort
of
convince
a
middle
box,
like
of
whatever
signal
I
wanted,
as
well
as
some
pretty
crazy
signals.
M
P
Thanks
Dave,
so
I
apologize
to
the
people
in
the
video
in
advance.
I'm
gonna
walk
off
the
video
to
start
their
animation
because
my
laptop
didn't
work
with
the
presenter
thing
right.
This
is
a
talk
about
the
RPK
wayback
machine.
My
name
is:
runs
fresh
Sakai,
I
work
for
an
omlette,
labs
and
I
hope.
If
it's
that
somebody
recognizes
this
Logan
Ziggy
says
there
is
a
50%
chance,
we'll
end
up
in
2011
hands
up
who
knows
quantum
leap
there
we
go
good
right,
so
I
don't
know
it
labs.
P
P
It
needs
to
recreate
the
towels,
because
the
towels
are
missing
in
the
tarji
set
files,
and
then
it
uses
a
tool
called
fake
time
to
run
the
route
inator
for
the
correct
date,
and
we
did
this
for
data
starting
January
2011
until
February
2019,
at
least
that
those
are
the
graphs
that
I'm
going
to
show
you.
But
actually,
during
this
week,
I
did
some
more
processing
and
I
have
some
really
nice
animations.
That
I
can
show
you
that
go
until
Monday
this
week.
P
Right
for
those
of
you
are
less
familiar
with
rpki,
a
quick
recap
of
some
of
the
jargon
that
I'm
going
to
be
using
throughout
this
presentation.
The
rpki
is
the
resource
public
key
infrastructure.
This
allows
you
to
set
what
are
called
route
origin
authorizations
and
basically
this
authorizes
a
certain
AAS
to
announce
certain
prefixes.
P
This
is
cryptographically
signed,
so
he
can
sort
of
validate
that
some
a
as
was
allowed
to
originate
that
prefix
and
what
we
typically
deal
with
after
validation
are
called
verified
raw
payloads.
Yes,
that
is
an
acronym
in
an
acronym,
and
this
is
a
cryptographic,
a
valid
statement
about
a
prefix
from
an
hour
away
right.
P
What
is
also
interesting
is
that
the
historic
data
that
we
got
actually
turned
out
to
contain
some
flaws.
You
can
see
little
arrow
on
the
on
the
graph
that
says
first
day
right
data
validates
before
that
date.
Something
was
off
in
the
manifest
files
that
we
got
from
ripe,
ncc
and
it
didn't
validate,
and
it
is
on
a
road
map
to
use
old
or
pti
software
to
also
validate
that
data
and
see
if
we
can
make
this
graph
cover
the
whole
period.
P
Another
interesting
thing
that
you
can
observe
here
is
the
DES
spike
in
the
Green
Line
from
AP
neck,
and
these
were
three
asses
that
suddenly
D
aggregate
all
of
their
rows,
which
basically
means
they
disable
their
max
max
length
attribute
and
rather
than
say.
Oh,
this
is
a
pretty
role,
that's
valid
first
say
sixteen
prefix
and
you
can
have
sub
announcements
of
stretch
24.
P
They
remove
the
max
length
and
basically
converted
that
into
a
whole
bunch
of
slash,
24
rows,
and
apparently
this
was
a
mistake
in
some
software
and
it
got
fixed
pretty
quickly,
but
the
impact
is
quite
visible.
Of
course,
we
also
have
to
look
at
ipv6,
because
that's
a
modern
protocol
and
again
you
can
see
more
or
less
the
same
trend.
It's
it's
far
fewer
prefixes,
for
which
there
are
rows,
but
the
trending
growth
is
pretty
similar.
P
So
what
I
also
looked
at
was
the
prefix
size
in
VR
PS
over
time,
because
the
hypothesis
behind
this
was
that
as
ipv4
exhaustion
continues,
you
might
expect
to
see
fear
piece
for
smaller
prefixes
and
actually,
as
you
can
see
in
this
this
graph.
That
is
true.
So
this
is
a
CDF
for
five
years
and
in
2019
the
fraction
of
prefixes
that
are
24
is
sort
of.
P
Let
me
see,
let
me
say
that
so
that's
going
down
I'm
trying
to
figure
out
what
I
wanted
to
say
there.
So
the-
and
the
same
is
true
for
mix
lengths
right
the
the
when
mixed
length
is
used.
You
can
again
see
that
the
max
length
becomes
smaller
and
smaller
and
smaller
over
time.
So
there
it's
going
down
toward
slash
24
for
a
v6.
We
see
pretty
much
the
same
thing
again.
The
prefix
size
is
decreasing
over
time,
so
we
see
vir
PS
for
smaller
prefixes,
and
this
is
also
to
form
X
length.
P
P
This
one's
a
little
bit
interesting
right,
because
there's
been
some
debate
about
whether
or
not
you
should
use
max
length
or
water
or
or
if
you
should
prefer
2d
aggregate
and
have
lots
of
rowers
that
cover
smaller
prefixes
and
because
the
reasoning
behind
this
is,
if
you
have
a
max
length,
but
you
don't
and
now
announce
all
of
the
sub
prefixes
that
are
underneath.
That
you're
expose
yourself
to
high
checking
attempts.
P
And
what
you
can
see
is
that
max
length
use
over
time
was
going
down
until
about
a
year
ago,
and
it's
slowly
starting
to
rise
again.
And
while
we
can't
confirm
this,
we
think
this
might
be
because
people
are
now
validating
a
lot
more
and
it
turns
out
that
people
were
announcing
sub
prefixes,
that
we're
not
covered
by
the
by
the
rowers
and
their
announcements
were
getting
rejected.
Because
of
that,
and
then
to
fix
that,
if
you're
announcing
sub
prefixes,
it's
easy
to
add
a
max
length
that
is
valid.
P
Again
so
average
prefix
size.
So
there's
a
note
that
says:
compare
this
to
Jeff
story
in
in
routing.
That
was
because
I
use
this
slide
at
the
right
meeting
in
Reykjavik,
where
Jeff
Hewson
presented
something
about
the
BGP
table,
where
you
also
see
a
decreasing
prefix
size
in
the
announcements
that
are
in
there.
Well,
no
surprise:
we
see
a
decreasing
averaged
prefix
size
in
the
VR,
P's
ball
again
max
length
and
the
prefix
size
that
is
in
the
row
up
right
again
for
ipv6.
We
see
pretty
much
the
same
thing
right.
P
P
We
have
a
tool
that
will
exist,
a
video
that
one
no
oh,
okay,
I
need
to
click
harder.
There
we
go!
What
this
shows
you,
this
animation
is
the
coverage.
So
this
is
the
what
we
took
was
route
fuse
data
for
announcements
that
we
see
in
BGP.
We
took
the
veer
piece
for
certain
dates
and
we
use
the
NRO
stats
to
map
those,
the
prefixes
that
we
see
in
the
announcements
and
in
the
rowers
to
certain
countries
and
what
is
animation
will
show
you
is
coverage.
So
these
are.
P
This
is
the
fraction
of
announcements
that
are
covered
by
Aurora,
and
this
animation
runs
until
this
Monday
and
hopefully
it's
kind
of
visible
on
the
screen.
What
you
should
be
able
to
see
in
this
is
that
it's
slowly
getting
darker,
and
that
means
that
we
see
more
and
more
and
more
announcements
that
are
actually
covered
by
Aurora,
and
what
you
should
also
see
is
that
we're
now
April
May
June,
that
this
is
accelerating
towards
the
end
and
I
apologize
that
the
screens
kind
of
mangle,
the
the
nice
colors.
P
P
S
P
P
P
You
is
accuracy
and
accuracy
we
define
as
the
fraction
of
covered
announcements
that
are
valid
right
so
and
what
we
graph
here
are
those
countries
that
have
an
accuracy
of
90%
or
over
so
I
cut
off
everything
that
has
a
lower
accuracy,
because
this
will
show
you
again
that
even
there
is
already
a
high
accuracy.
So
all
of
the
countries
that
you
see
right
now
on
April
1st
2018
had
an
accuracy
of
over
90%
for
the
announcements
that
are
sort
of
for
the
prefixes
that
are
assigned
to
those
specific
countries.
P
But
what
you
will
see
is
that
the
quality
is
really
improving
a
lot
again.
What
you
see
here
is
a
some
kind
of
a
disappeared
oops,
but
the
takeaway
from
this
is
that
quality
is
really
really
getting
good.
So
if
you
are
not
filtering
based
on
our
PPI,
you
really
should
be
doing
that,
because
the
quality
of
the
data
is
very,
very
good
right.
The
accuracy
is
high,
and
can
we
go
back
to
the
slides?
I
have
to
stand
here.
Sorry,
America,
Mario's,
very
strict.
P
Right,
so
those
were
those
two.
Let
me
figure
out
if
I
can
somehow,
if
people
are
interested,
send
me
an
email
I
can
I
can
get
videos
for
that
right.
To
conclude,
we
wanted
to
test
route
inator,
and
it
turns
out
that
RPG
I
took
some
time
to
standardize
properly
right
because
we
couldn't
validate
some
of
the
older
data.
P
The
next
step
that
we
want
to
do
is
to
compare
this
against
routing
information
in
more
detail,
and
we
actually
already
did
that.
We
submitted
a
paper
to
the
ACM
measurement
measurement
conference
that
was
accepted
so
we'll
be
presenting
that
in
October
in
Amsterdam,
and
also
this
is
some
free
advertising.
As
the
one
of
the
general
chairs
of
IMC,
please
come
to
Amsterdam
in
October
to
the
wonderful
internet
measurement
conference
with
that.
P
I
would
like
to
give
a
big
thank
you
to
the
folks
at
the
ripe
NCC
for
actually
collecting
and
keeping
this
data
to
a
meal
from
ripe
to
help
us
make
this
data
available.
This
data
is
now
available
as
open
data.
So,
if
you
are
a
researcher
you
want
apply
play
with
this
rpki
data
go
grab
it
off
the
ripe,
FTP
server
and
and
also
grab
our
free
OSS
tools.
P
Ruminator
the
secure
reading
stats,
which
is
what
I
used
to
do
all
of
the
computation
for
the
the
maps
that
I
showed
and
I
turned
the
repository
for
city
public
earlier
this
week.
So
you
can
grab
that
I
will
actually
now
download
the
data
from
the
ripe
archive,
see
what
you
do.
Is
you
run
it
you
give
it
a
date.
P
V
Robert
Kiyosaki
ripe,
ncc
one
suggestion:
I,
don't
know
about
the
exact
incremental
value
that
you
would
get
out
of
this,
but
if
there
was
an
option
to
say
disable
manaphy
check,
then
you
could
fill
in
the
gap
that
was
before
2014
for
the
MC.
The
reason
is
the
manifest
is
there
to
protect
the
integrity
of
the
whole
set
and
if
it's
fair,
that
it
was
not
compromised.
You
know
in
that
time
period.
Yes,
there
is
a
footnote
there,
but
if
you
assume
that
it
wasn't,
then
you
would
feel
that
yeah,
yes,.
P
Well,
it's
certainly
something
we
want
to
do.
Is
it's
just
that
with
all
the
work
we've
been
doing
on
the
tool
set,
we
haven't
had
time
to
sort
of
add
the
flag,
but
we're
certainly
gonna.
Do
that
and
we're
probably
also
gonna
make
all
of
the
validated
the
data
that
we
file
a
date.
It
will
probably
put
it
up
somewhere
so
that
people
G
can
just
grab
the
vir
piece
rather
than
having
to
do
run
the
validation
themselves,
Oh.
N
N
One
of
the
the
reasons
I
was
in
rpki.
Skeptic
is
sort
of
the
the
base
rate
fallacy
problem
that
you
have
when
you're,
enabling
optional
security
there's
a
risk
to
turning
validation
on
because
the
data
is,
you
know,
there's
problems
with
the
data,
you
might
actually
start
projecting
routes
and
you
have
availability
risk
and
you're
creating
African
security
risk,
the
number
of
it.
So
if
I
were
trying
to
come
up
with
a
risk
number
from
your
data
set,
that
would
be
one
over
accuracy.
Correct.
N
S
N
Because
the
things
that
are
uncovered
or
are
you
know
not
going
to
be
a
problem
anyway:
okay
and
those
and
the
scale
on
the
geographical
data
you're
showing
was
like
ninety
nine,
ninety
percent
to
99
percent
of
aggregated
prefixes,
or
at
announcements,
announcements,
announcements
and
you're
kind
of
hoping
yeah
and
then
the
other
quells
unique
prefixes
in
announcements.
Yeah.
W
It's
more
of
a
comment
on
the
last
question
is
that
when
you
look
at
those
accuracy
numbers,
you
have
to
be
careful
to
see
if
the
route
that's
invalid
might
well
be
covered
by
another
route
that
is
valid
or
not
found.
We've
looked
at
that
and
the
coverage
is
pretty
high.
The
coverage
that
uses
the
same
path
as
pretty
high
coverage
then
ends
up
in
the
same
origin
is
pretty
hot.
W
W
B
So
we
had,
we
had
a
little
a
bit
of
juggling
with
the
agenda
and
we
decided
we
decided
to
have
you
on
come
and
talk
to
us
about
his
recycling,
large-scale
internet
measurements
for
about
10
minutes
and
so
we're
gonna
do
is
bump
the
hackathon
report
that
I
would
have
done,
but
all
is
not
lost.
There's
a
link
to
the
three
minute
hackathon
talk
that
I
gave
you
can
find
it
in
the
agenda
so
in
the
in
their
updated
slides
for
that.
So
take
it
away
on.
Thank.
O
You
so
this
is
work
we
already
presented
at
Pamela's
here
and
basically,
this
all
started
out
well,
I've
been
doing
intuit
measurements
for
the
last
couple
of
years.
I
was
here
at
ITF
in
London
and
presented
some
stuff
about
Google
quick
deployment.
So
yes,
this
is
a
talk,
we're
also
quick
ISM,
but
we
also
scanned
follow
up
more.
We
do
DNA
scans,
TLS
scans
HTTP
to
scans,
and
the
thing
with
these
scans
is.
O
We
are
looking
for
something
very
specific,
but
if
you,
for
example,
do
a
tcp/ip
v4
scan
on
port
81
percent
of
all
IPS
will
actually
give
us
an
EXO
answer.
What
about
the
remaining
99%
right?
So
the
idea,
basically
that
we
had
in
this
paper
Wars?
Is
there
actually
something
we
aren't
scanning
for,
but
we
could
have
scanned
for
all.
Can
we
reuse
the
data
that
we
already
have
and
to
find
that
out,
we
basically
captured
all
ICMP
traffic
that
words
ingressing
our
measurement
networks,
as
we
do
not
generate
ICMP
traffic
for
ourselves.
O
This
is
either
background
traffic
or
traffic
that
is
actually
generated
by
our
measurements.
So
the
idea
was
okay.
Let's
study
ICMP
and
let's
study
what
actually
is
in
there
and
we
were
surprised
because
within
a
week
we
got
what
is
it
six
hundred
thirty
seven
million
icing
penises
from
a
whole
different
number
of
Phi
piece
over
53,000
autonomous
system,
and
you
can
see
a
plot
basically,
where
you
can
see
that
yeah
well,
they
correspond
to
our
measurements,
which
typically
start
at
midnight.
If
people
correctly
set
their
and
yeah.
O
That's
not
going
to
detail
into
this,
and
we
basically
first
thing
that
we
looked
in
Wars.
Okay,
do
it?
Does
it
make
it
different
what
we're
actually
scanning
for
and
yes
it
does.
So
if
we
do
like
a
TCP
scan
on
port
80,
you
can
see
yeah.
We
don't
get
a
lot
of
destination
unreachable
or
port
unreachable
messages,
which
is
obvious
because
you
would
use
probably
reset
for
that.
But
there
are
still
some
of
those.
It
also
makes
a
difference
whether
or
not
we're
simply
doing
ipv4
scans.
O
We
also
do
dns-based
scans,
where
we
have
a
lot
of
zone
files,
for
example,
it'll
be
an
Essbase
and
you
can
see
that
we
get
a
lot
of
destination
unreachable,
because
host
is
prohibitive
messages.
If
we
do
scans
based
on
DNS
data
and
on
the
right,
you
can
see
a
little
table,
it
summarizes
basically
what
kind
of
messages
we
go
out
and
I'm
going
to
from
some
of
those
types
and
see
what
we
actually
found.
When
we
looked
into
those
messages,
I
think
it
should
start
with
redirect
messages.
O
So
if
you
don't
know
what
redirect
messages
are,
they
are
basically
used
to
signal
a
better
pass
in
your
local
network,
so
RFC
1820
1812
actually
states
these
three
things
and
the
most
important
one
it's
in
the
middle.
It
basically
says
yeah.
Well,
if
the
source
address
of
the
packets
on
the
same
logical
network
is
you,
you
may
send
these
things.
However,
we
never
are,
but
we
still
got
18
million
of
those
messages
over
the
public
Internet.
O
100,000
of
those
are
network
redirects,
which
the
RSA
says
you
must
not
send
Pickers,
they
don't
make
sure
we
make
any
lot
of
sense
if
you
look
at
them
well,
but
the
majority
of
host
redirects
and
they
affect
roughly
two
thousand
asses-
and
these
are
roughly
four
thousand
in
these
redirects
roughly-
were
400,000
destinations
and
within
all
our
DNS
data
that
we
actually
have.
We
found
also
that
900
of
them
had
an
a
record
and
some
of
those
actually
redirect
you
to
private
addresses
or
bruised
left
address
spaces.
O
Okay,
this
is
not
good.
Next,
we
looked
at
source
quench
messages.
So
if
you,
you
probably
remember
that
I
didn't
actually
really
knew
what
source
quench
was
when
I
started
this,
and
it's
basically
like
easy
ends
grandparent.
So
the
idea
was
when
the
route
is
congested.
It's
and
the
source
bunch
message
to
the
sender's.
However,
research
found
well,
it's
kind
of
unfair.
O
If
you
do
that,
and
it
can
lead
to
blind
attacks-
and
you
guys
said-
please
don't
send
those
in
1995-
and
you
also
said:
please
ignore
those
messages
in
2012
and
most
actually
do
that
since
2005.
Yet
we
still
found
that
they're,
roughly
2,
now
a
thousand
I
piece
located
in
over
350
asses
that
still
issue
those
messages.
O
Mostly
those
messages
were
issued
in
the
destination
access,
but
we
also
found
some
that
were
issued
on
path.
I
mean
we
look
into
some
vendor
manuals
of
yeah,
some
some
router
manuals
and
we
found
that
most
of
those
vendors
also
state
that
they
removed
it
in
between
2000
2010.
But
you
can
still
see
it
takes
decades
to
get
stuff
out
of
the
internet,
so
whatever
you're
standardizing
people
will
use
it
forever.
O
If
I
once
activated
it
all
right
yeah,
we
also
looked
at
another
thing:
fragmentation
and
if
something
gets
fragmented
because,
as
I
said,
we
also
scan
for
quick.
So
our
packets
are
roughly
1000
fluid
bytes
and
it
as
it
turns
out
or
is
we
use
a
map
default
in
our
scans
and
they
don't
set
the
don't
segment
bit.
So
we
got
some
fragment
time
exceeded
messages
from
another
couple
of
scans.
Where
we
do
set
those
bits.
We
see
that
there's
a
bit
of
fragmentation
so
yeah.
O
O
So
we
ask
yourself
other
paths
very
long.
Are
the
many
middle
boxes
tweaking
our
titi
outs
or
are
there
maybe
routing
loops
and
we
basically
performed
a
whole
bunch
of
trace
routes
to
see
if
there
are
loops
and
we
were
able
to
basically
defined
roughly
440,000
/
24
from
roughly
20,000
SS
that
were
unreachable,
dgo
loop,
yeah?
O
O
O
We
also
found
loops
in
our
upstream
ISP
in
the
German
research
network,
which
was
nice
because
we
can
contact
them
and
they
confirm
the
loops
which
they
didn't
really
like.
They
fix
the
loops
and
for
her
so
we're
specific.
They
could
tell
us.
Basically
it
was
a
static
route
they
had
once
in
one
router,
then
they
removed
it
another
one,
but
it
was
basically
still
forwarding
packets
to
the
other
router,
but
this
rotor
had
no
knowledge
about
the
drought
anymore
and
send
it
to
its
default
route,
which
was
the
first
router
and
yeah.
O
O
The
nice
thing
I
mean
we
all
know
that,
but
the
nice
thing
is
we
could
find
that
without
actually
doing
any
new
measurements.
These
were
all
done
from
apart
from
the
trace
routes,
we're
all
done
using
data
that
we
already
cooked
what
never
looked
at
yeah
lots
of
routing
loops.
Our
data
set
is
partly
available
until
last
month's
will
visually
our
drives
run
full
yeah.
Thank
you.
B
All
right
we've
got
four
minutes
left
in
the
session,
yeah
questions
for
Yann
or
if
we
don't
have
those
any
other
comments
on
work
for
the
group
would
be
great
along
the
lines
of
ice
for
the
last
three
times.
We've
participated
in
the
hackathon,
so
we're
welcome
to
Meaney
what
can
I
talk
about
that
anything
thanks.
Young.
B
If
you
haven't
seen
the
program
there
check
that
out
about
what
can
you
learn
from
an
IP
where
they
surveyed
the
server-side
IP
addresses
and
what
you
can
learn
from
those
and
in
PRG
and
I'm
also
interested
in
doing
this?
Creating
anonymity
sets
somehow
of
the
service
side,
so
people
can't
profile
encrypted
traffic
and
determine
what
you're
talking
to
so.
If
that's
something
you're
interested
in
get
in
touch
with
me
or
talk
about
it
on
the
list
or
you
might
be
able
to
do
it
in
Singapore,
the
hackathon
feel
like
that's
all.