►
From YouTube: MAPRG Interim Meeting 2020-08-05
Description
MAPRG Interim Meeting 2020-08-05
A
A
Welcome
to
the
joint
measurement
analysis
for
protocols,
research
group
and
measurement
analysis
tools,
working
group
meeting,
I'm
dave
planka.
I
chair
with
miria
coolwin
the
map
rg
group
in
the
irtf
and
our
co-chair
for
this
meeting
is
nina
ferguson
and
she
can
introduce
herself
in
a
minute
when
we
switch
to
her
portion
of
the
content
but
she's
she
co-chairs
with
our
colleague
brian
trammell
the
measurement
analysis
tools,
working
group
for
the
ripe
regional
internet
registry.
A
A
This
is
our
recording
notice.
The
meeting
is
being
recorded
now
and
all
the
presentations
will
be
they'll,
be
it'll,
be
available
on
the
ietf
youtube
channel
and
as
usual
for
map
rg
after
the
meeting
I'll
post
individual
links
to
each
of
the
top
mapper
g
wiki,
which
is
available
through
the
memory
page
linked
here.
A
Someone
has
their
audio
on.
If
you
could
please
mute
it,
it
would
be
helpful
in
minnesota.
For
today,
the
charters
for
these
respective
groups
are
at
these
urls.
This
is
the
first
time
we've
tried
this
experiment
of
a
combined
map,
rg
meeting
with
with
ripe
matt
wg,
and
the
idea
was
that
if
we
did
it
outside
the
ietf
meeting
schedule,
then
maybe
more
people
could
join
between
the
two,
when
both
of
our
groups
have
been
having
a
bit
of
trouble
scheduling
in
the
pandemic
era.
A
Both
the
groups
have
mailing
lists
that
are
mentioned
here
that
you
can
subscribe
to
where,
for
instance,
you
would
have
heard
about
this
meeting
being
scheduled.
Today's
slides
can
be
found
at
that
url
there
and,
of
course
all
you
already
know,
but
the
webex
link
is
available
in
the
agenda
and
in
the
slides
at
that.
B
Thank
you.
Do
you
hear
me?
Yes,
yeah
great,
thank
you
yeah,
so
I
am
I'm
nina
barkinson.
I
co-chair
the
reagan,
the
right
working
group
on
measurements,
analysis
and
tools,
along
with
brian
tremel
who's,
also
on
nepal.
Today,
hello,
hey
and
today
we're
gonna
talk
about
so
basically
the
the
wedding
group
mostly
focuses
on
on
measurements
and
on
tooling
and
a
lot
of
the
time.
B
We
also
focus
on
right
atlas
system
and
today
we're
going
to
talk
about
the
mechanisms
and
performance
evaluations
of
the
to
pmax
locations.
But
massimo
is
going
to
talk
and
then
we
will
have
some
measurements
about
what
happened
under
the
the
pandemic
that
we
are
still
running
through,
but
so
far,
and
we
have
vesna
and
lai
is
going
to
give
us
their
observations
on
that.
We
also
thought
that
we
would
have
an
tools
update
from
the
ribbon
cc.
B
People
please
be
mindful
of
muting
their
microphones.
Thank
you,
but
unfortunately
we
we
have
to
cancel
the
tools
update
and
you
will
just
have
to
show
up
at
the
next
right
meeting
to
get
the
latest
news
on
the
atlas
and
other
tools
from
the
vibe
ncc,
but
happy
to
be
here,
and
I'm
really
excited
about
this
joint
meeting
is.
This
is
my
first
itf
meeting.
A
So,
following
the
portion
that
nina
will
be
running
with
the
the
items
that
she
just
enumerated,
that
were
that
that
were
organized
by
matt,
we'll
switch
to
the
map.
Rg
meeting
proper
and
map
rg
is
in
the
irtf,
and
so
we
have
an
intellectual
property
rights
statement
that
binds
you
to
as
a
presenter.
If
you
have
intellectual
property
regarding
it,
you
should
disclose
that
just
the
canned
portion,
so
this
will
be
an
official
map
rg
meeting
as
well,
and
this
applies
only
to
the
map
rg
portion
of
the
meeting.
A
Talking
about
latency
and
aqm
observations
on
the
internet,
then
philip
roon
on
packet,
latencies
and
mobile
networks,
mike
kosek,
is
with
us
to
talk
about
their
paper
and
from
an
academic
conference,
must
don't
care
tcp,
conformance
in
the
wild
and
then
we'll
switch
at
the
it's.
The
last
session
and
portion
of
the
session
to
steven
stros
talking
about
deborganizing,
this
slash
12
prefix,
that
the
rape
ncc
is
now
using
to
before
we
switch
to
some
advertisements
for
various
projects.
A
I'll
talk
a
little
bit
about
how
we're
going
to
run
the
meeting
for
you
to
the
presenters
I'll
switch
you
to
the
best
of
my
ability
to
you
being
the
host
in
the
meeting,
which
will
allow
you
just
to
show
your
slides.
If
you
have
any
trouble
with
that,
no
problem,
let
me
know,
and
I'll
show
them
like,
I'm
going
to
do
for
mark's
presentation
for
those
participants
that
have
questions.
A
There
maprg
invites
projects
that
are
not
presenting
at
the
meeting
to
also
give
us
advertising
slides
for
their
projects
and
for
this
meeting
yuri
and
I
created
a
couple
because
we
have
not
met
so
far
this
year
because
of
the
the
pandemic
and
scheduling
issues
between
ietf,
107
and
108.
This
is
our
first
meeting
of
the
year.
So
let's
bring
you
up
to
date
on
some
of
the
content
that
we
would
usually
have
hosted
in
map
rg
imc.
A
Another
one
of
the
permanent
measurement
conferences
was
tma
that
was
held
in
june
of
this
year.
Similarly,
tma
has
had
recordings
or
videos
of
the
of
the
papers,
and
those
papers
are
up
at
that
url
and
gareth.
One
of
the
chairs
has
created
a
youtube
playlist
that
url
for
the
subset
of
papers,
where
the
presenters
gave
permission
to
share
the
video
so
check
those
out.
A
Lastly,
anna
bremstrom
has
offered
to
us
this
this
advertisement
for
a
project
with
she
and
her
authors,
where
they
studied
narrow
band
iot
internet
of
things,
devices
into
major
metropolitan
areas
and
their
my
understanding
is
their
their
interactions,
for
instance
with
lte
and
cellular
in
that
area,
and
we
included
them
here.
Although
they're
not
involving
specifically
an
ietf
protocol
that
they've
released
a
data
set,
so
there's
a
it
could
be
the
basis
for
other
people
to
do.
A
B
C
Perfect
so
hello,
everybody,
so
my
name
is
massimo
candela,
I'm
a
senior
software
engineer
at
ntt.
First
of
all,
I
would
like
to
thank
the
organizers
of
this
event,
because
I
personally
really
miss
the
math
working
group
at
the
last
right
meeting.
So
I'm
really
happy
that
we
could
do
it
here
so
the
presentation
here
today
I
will
show
you
a
mechanism
behind
this
service
called
write,
ipmap
and
also,
I
will
show
you
a
performance
evaluation.
We
did
it
together
with
the
colleagues
from
keda.
C
This
is
a
paper
that
appeared
in
the
issue
of
april
2020
of
computer
communication
reviews
and
well,
let's
start
from
the
beginning.
The
first
thing
is:
what
is
this
write
it
up
so
right,
ipmap
is
a
service
that
has
been
developed
and
is
offered
by
ripenc,
and
it
is
essentially
a
service
that
you
provide
an
ip
address
and
will
try
to
tell
you
where
such
a
p
address
is
g
located
and
so
the
a
bit
of
of
easter
even
this
service.
C
The
first
attempt
to
tackle
the
geolocation
topic
was
presented
in
2013
by
emil
and
essentially
introduced
the
openip
platform,
where
you
can
use
cloud
source
data
to
improve
ipg
location
and
in
2017
in
another
rhyme
meeting
I
introduced
instead
ripe,
I
team
up,
which
is
the
idea
behind.
That
is
a
a
system
that
has
the
possibility
to
run
various
geolocation
approaches
and
offer
this
platform
to
people
that
they
want
to
provide
a
new
geolocation
system.
C
So,
a
bit
more
of
information,
I
developed
the
single
radius
engine
while
I
was
arriving
to
see
and
not
any
more
ripe,
ncc
stuff,
but
we
got
some
feedback
that
the
system
works,
and
there
is
also
a
paper
that
reports
on
a
country
accuracy
of
99.9
for
their
use
case,
but
we
needed
a
formal
study
dedicated
to
the
in
particular,
to
single
radius
and
in
particular
also
to
city
level.
Geolocation
and
also
the
mechanism
behind
was
not
for
singer
was
not
formally
described
anywhere.
C
The
study
was
a
black
box,
so
at
some
point
they
decided
to
call
me
and
on
board
of
the
team,
to
essentially
provide
more
insight
on
how
his
work
is
working
behind
the
scenes.
The
system,
that's
how
I'm
involved
in
this
research
project,
so
the
just
to
start
with
the
what
what
is
active
geolocation
well,
simply
like
you
want
to
generate
an
ip
address,
and
to
do
that,
you
take
a
host
that
you
know
the
location
of,
and
you
do
a
ping
measurement
to
the
target.
C
In
our
case,
the
host
is,
for
example,
ripotlas
probes
and
after
you
do
a
ping
measurement.
You
know
the
latency,
you
get
the
latency,
you
transform
it
in
a
distance
with
a
coefficient
factor,
usually
expressing
kilometers
per
millisecond.
We
will
see
better
later
and
what
you
know
is
that
the
target
ip
is
going
to
be
around
a
specific
area
around
your
source
of
the
measurement
so
like,
in
this
case
the
circle
so
finger
radius.
Also,
the
as
the
name
says,
does
exactly
this
some
additional
step.
C
You
cannot
select
like
ten
thousand
probes
for
geolocating
one
single
ip,
because
you
will
do
essentially
a
ddos
every
time
you
would
trigger
a
lot
of
icmp
rate
limiting.
You
would
just
also
stress
the
platform
itself,
so
there
is
an
initial
step
that
we
will
see
for
this
election
after
you
have
the
probes,
you
do
the
ping
measurements
to
the
target.
You
select
a
list
single
radius.
Does
that
you
select
the
closest
the
fastest
of
this
ping
measurement
and
it
has
to
be
also
the
router
type
below
10
milliseconds.
C
So
the
round
three
time
divided
by
two
is
the
usual
approximation
that
it's
done
to
calculate
the
one
way
delay,
or
at
least,
as
I
said,
an
approximation
of
it
and
the
two
third
speed
of
light
is
the
upper
bound
of
the
signal
traveling
in
fiber,
which
assumes
that
between
the
source
and
the
target,
there
is
a
straight
fiber
with
no
hop
in
between
them.
So
they
are
a
really
big
approximation.
C
The
last
step
is
that
single
radius
does
tries
to
bring
a
bit
more
accuracy
to
the
process
is
to
select
inside
that
circle
inside
that
area,
100
cities,
if
there
are
at
least
100
and
ranks
them
based
on
distance
from
the
source
number
of
facilities
and
population,
because
the
main
goal
of
ripe
pimp
is
to
geolocate
infrastructure.
So
these
parameters,
they
kind
of
give
an
indication
of
how
much
internet
infrastructure
is
in
such
steady,
see
this.
C
So
one
thing
that
the
single
radius
does
is
selects
like
bgp
neighbors
distance,
one
and
another
thing
that
it
does
is
use
speeding,
db
information
to
detect,
which
exp
the
autonomous
system
of
the
target
is
doing
peering.
So
you
detect
the
cities.
This
is
an
idea
initially
introduced
by
vasilius
that
you
probably
all
know,
and
so
at
the
end,
what
you
do
is
to
have
a
set
of
cities
and
a
set
of.
C
C
C
We
evaluated
the
effectiveness
effectiveness
of
the
probe
selection
method
that
I
showed
before
we
evaluated
the
coverage
we
evaluated
also
the
rather
level
consistency
where,
essentially,
we
ask
a
single
radius
to
gellocate
ip
addresses
belonging
to
the
same
router
in
byte
and
checking
if
we
got
the
same
essential
location
and
overall,
the
goal
is
to
provide
suggestions
to
rapidly
see
it
to
the
entire
community.
The
one
in
bold
are
the
point
that
I'm
going
to
describe
now.
C
So
the
first
thing
is,
we
need
two
data
set.
One
is
for
the
ground.
The
ground
through
data
set,
is
essentially
a
collection
of
ip
addresses
of
known
location,
and
we
need
this
to
validate
the
accuracy
we
use
the
arc
nodes.
The
nl-nog
ring
nodes,
the
m-lab
pods
and
the
arc
proximity
data
set
in
this
table.
C
You
can
see
the
division
of
ips
by
region
and
also
coverage
in
terms
of
unique
autonomous
systems
and
after
we
filter
out
the
ips
that
they
have
wrong
metadata
or
they
were
offline,
so
not
possible
to
do
geolocation,
and
we
reported
some
cases
of
wrong
or
missed
or
corrupted
metadata
to
the
administrators
of
this
platform.
So
this
was
also
one
of
the
benefit
of
this
of
this
exercise.
C
So
in
the
end
we
had
968
ips
and
for
the
coverage
data
set,
where
we
are
not
interested
in
having
information
about
the
geolocation
itself,
but
we
need
some
responsive
ip
addresses.
We
use
the
manic
interconnection
data
set
produced
by
keda,
which
has
more
than
16
000
ip
addresses
active
and
answering
things
so
go
straight
to
the
accuracy
results.
We
you
see
this
cdf.
The
blue
line
is
single
radius.
C
The
orange
one
is
net
tag
within
the
green
is
max
mind
and
you
see
on
the
x
axis
the
error
in
the
geolocation
in
kilometers,
so
we
consider
a
distance
below
40
kilometers,
accurate
at
city
level
at
distance
above
40
kilometers,
not
any
more
accurate
at
city
level.
This
40
kilometer
is
a
trash
that
is,
has
been
used
multiple
times
in
literature
and
it's
considered
kind
of
average
metropolitan
area
size,
and
we
can
see
that
single
radius
outperforms
the
other
two
platforms
and
reaches
an
accuracy
that
is
80.3
percent.
C
But
the
comparison
is
not
yet
finished,
because
if
we
compare
the
median
75th
percentile
and
the
95th
percentile
of
the
three
platform,
we
see
that,
for
example,
for
single
radius
is
6.
26
and
344.
Kilometers
of
error,
while
for
net
equity
is
1080
and
almost
3
000
and
for
max
mine
is
17
278
and
almost
3
000,
also
so
single
radius.
Even
when
produce
a
an
answer
that
has
an
error
above
the
40
kilometers.
C
C
So
if
we
would
change
that
threshold,
so
if
you
reduce
the
threshold
from
10
to
5
milliseconds,
we
drop
the
coverage
to
51.1
percent,
but
if
we
go
to
2
milliseconds
where
we
would
have
a
90,
50,
percentile
accuracy,
a
city
level,
we
have
a
coverage
of
only
24.,
so
you
do
your
math.
If
it's,
you
can
do
your
tuning.
If
it's
convenient
for
you
or
not.
In
my
opinion,
you
basically
drop.
C
C
E
C
The
world,
so
in
our
research
we
do
also
drill
down
for
various
regions
and
we
try
to
split
the
various
results
you
see
here.
Instead,
the
just
the
global
view
for
simplicity,
also
the
ground
truth.
Data
set
of
the
ips
is
really
hard
to
obtain
such
accurate
set
of
ips
that
you
can
test
and
you
know
exactly
location.
So
this
is
also
I
mean
we
had
968,
which
is
a
big
amount
compared
to
a
lot
of
other
works,
but
still
a
small
amount
compared
to
the
entire
address
space.
C
B
F
C
F
F
C
C
So
if
you
look
at
the
measurements
itself,
it
could
be
faster,
but
we
don't
want
to
since
arrive
atlas
has
a
queueing
mechanism
for
the
measurement.
We
don't
want
to
retrieve
data
too
early
and
give
a
wrong
answer.
So
we
wait
the
three
minutes
that
we
calculated
us,
basically,
the
maximum
time
for
a
measurement
to
be
a
one-off
measurement
to
be
executed,
so
that
the
answer
is
is
in
general
three
minutes,
but
the
once
an
ap
is
of
course
calculated.
The
answer
is
cash,
so
it
will
take
milliseconds
to
just
retrieve
it
again.
C
C
F
G
B
C
No,
the
suggestion
where,
like
the
experiment,
overall,
that
we
had
for
the
for
the
use
of
different
thresholds,
we
did
a
good
amount
of.
We
have
also
a
list
of
them.
They
are
more
in
explicit
in
the
article,
so
we
did
various
things
among
which
the
graph
that
you
see
here
can
be
used
by
the
user
to
essentially
tune
their
own.
The
api
provides
the
run
trip
time
used
as
a
threshold,
so
you
can
use
this
to
give
provide
you
your
own
score.
That
is
one
of
the
suggestions.
C
The
other
suggestion
that
I
have
for
writing
cc
would
be
to,
at
some
point,
implement
a
mechanism
with
multilateration.
We
did
some
preliminary
tests
by
using,
for
example,
regional
coefficient
to
transform
the
distances
and
the
the
results
were
extremely
positive
in
terms
also
of
reduction
of
ambiguity
of
the
geolocation,
but
that
would
work
with
a
multilateration
approach
and
another
another
one
could
be
experimented
is
with
the
removing
the
last
mile,
the
last
mile
latency
from
the
measurement
to
improve
the
the
accuracy.
C
B
C
E
Hi
everyone,
my
name,
is
vesna,
I'm
from
the
ripe
ncc,
I'm
a
community
builder,
and
I
am
giving
this
presentation
on
behalf
of
my
colleagues,
mainly
emile
aben,
who
couldn't
be
here,
but
these
are
his
slides
mostly,
so
we
have
been
active
in
analyzing
the
data
from
the
ripe,
ncc
measurement
projects
and
publishing
our
findings
on
ripe
labs.
We
have
a
collection
on
a
slash,
kovit
19,
and
you
can
find
all
these
results
that
I'm
going
to
talk
about
there
so
from
the
beginning.
E
Other
operators
also
contributed
to
this
and
also
published
some
of
their
findings
on
rap
labs,
and
we
try
to
move
our
real
hackathon
that
we
wanted
to
have
to
the
virtual
world
and
that
worked
more
or
less
as
much
as
possible.
So
we
also
publish
some
of
the
results
of
that
hackathon
there
and
some
of
the
recommendations
for
the
security
regarding
most
of
the
applications
for
the
tracking
and
so
on.
E
E
So
the
first
one
is
based
on
a
so-called
risk
of
routing
information
service
and,
if
you're
not
familiar
with
that,
it
is
a
little
bit
like
the
route
views
and
I'm
a
bit
sad
to
actually
compare
it
to
the
other
one
which
is
a
competitor.
But
it's
more
well
known.
I
guess-
and
it
has
a
better
name-
I
must
admit
so.
E
So
there
are
multiple
ways
to
access
this
data,
and
basically
this
is
an
invitation
for
the
other
researchers
and
the
network
operators
to
use
this
data,
because
it's
open-
and
it's
very
well
documented
how
you
can
use
it.
So
normally
what
you
would
use
that
for
is
to
kind
of
troubleshoot
your
network
and
to
show
how
is
your.
E
So
when
we
looked
at
the
corvid
impact
on
the
global
bgp,
we
thought
well,
it's
going
to
flare
up,
because
people
can't
travel
to
their
distant
points
of
presence
to
actually
do
the
maintenance
and
there
was
the
restricted
access
to
data
centers.
So
we
thought
oh
now.
This
is
going
to
actually
explode,
but
we
were
wrong.
So
basically
internet
is
stable,
at
least
on
the
routing
level.
E
So
the
other
way
to
look
at
it
is
by
the
type
of
the
announcement
so
either
by
adding
the
prefixes
or
removing
the
prefixes
or
the
origin
changes
again.
There
were
some
flares,
but
it
was
nothing
systematic
and
actually
it
didn't
coincide
too
much
with
the
lockdowns
per
country
or
anything
because
you
could
see
that
in
december
previous
years.
Actually
there
were
similar
events,
so
our
conclusions
are
that
the
pandemic
didn't
impact
the
internet
on
the
global
routing
table
level.
E
E
E
But
now
you
can
also
get
them
the
latest
data,
and
so
this
is
just
a
a
very
quick
overview
per
country.
And
again
you
can
put
your
own
country
when
analyzing
data
in
your
own
way
or
a
different
time
period
and
so
per
country.
We
also
showed
her
a
specific
provider.
What
were
the
changes
in
latency?
This
is
based
on
the
ripe
atlas
data,
so
the
pings,
the
ping
measurements,
and
so
this
is
the
the
rtt
that
you
can
see
here
and
again.
E
So
there
are
conclusions
to
be
drawn
here,
but
we
didn't
go
into
the
very
specific
details
ourselves.
This
is
more
to
show
what
can
be
what
is
possible
and
to
give
like
a
very
quick
overview
per
country.
How
does
rypatla
see
the
networks
and
how
they
are
impacted
with
the
heightened
usage,
so
the
good
news
is
internet
is
doing
fine
and
if
you
would
like
to
explore
it
yourself
here
are
the
links
to
the
interface
and
to
the
projects
that
I
mentioned.
E
E
So
please
approach
your
communities
and
spread
the
news,
because
we
would
like
to
support
the
existing
projects
or
the
new
ideas
that
work
on
the
good
of
the
internet
contact
us.
If
you
have
any
questions
there
are
my
email
addresses
emails
and
a
lot
of
twitter
details
and
rap
labs.
Thank
you
very
much.
B
F
F
And
after
that
we
performed
some
measurements,
basically
traceroute,
because
we
want
to
to
analyze
the
the
the
traffic
inside
different
parts
of
the
network.
We
we
just
see
the
results,
not
so
much
analyzed
for
for
the
moment,
because
we
we
need
people
to
do
that,
and
it's
a
little
bitty
moment
at
that
time,
because
the
pandemic
makes
so
complicate
everything
and
but
the
mind.
F
E
Yes,
I
would
be
interested
and
if
you
publish
something
we
can
republish
that
in
ripe
labs
and
promote
in
our
community
and
then
maybe
you
can
get
somebody
to
cooperate
with
you
and
help
you
out
and
another
comment
on
this
is
in
the
regions
where
we
have
fewer
probes
it.
It
might
be
possible
to
install
the
so-called
software
probes.
I
have
posted
the
link
in
the
chat
and
that's
just
easier
from
the
logistics
point
of
view.
So
please
consider
that.
A
A
E
E
B
A
I
And
we
can
see
them
yes,
okay,
great
so
hi.
My
name
is
olsen.
I'm
the
project
director
of
measurement
lab
thanks
for
having
us
similar
to
vesna
we're
here,
I'm
here
to
just
share
the
fact
that
our
data
exists
and
give
an
overview
of
how
it's
generated
so
that
researchers
here
can
can
take
it
and
run
with
it.
So
talking
about
what
is
mlab
just
to
speak
briefly
about
the
the
goals
of
our
project,
some
of
you
may
be
already
familiar,
so
apologies.
I
If
so,
but
for
those
of
you
who
haven't
been
introduced,
our
mission
is
to
measure
the
internet,
save
the
data
and
make
it
universally
accessible
and
useful.
I
am
preaching
to
the
choir
that
there
are
many
ways
to
do
this,
and
by
no
means
do
we
claim
to
do
all
of
them,
but
we
do
focus
on
principles
of
openness
and
transparency,
so
all
of
our
data
is
open
and
free
to
access
and
I'll
talk
more
about
how
that
works.
I
I
always
think
it's
useful
to
when
talking
about
measurable
measurement
lab
as
a
project
to
think
about
the
problems
that
it
was
trying
to
solve.
The
project
was
started
in
2008,
so
it's
been
around
for
12
years
and
it
was
responding
to
a
widely
reported
problem
of
not
being
able
to
not
having
access
to
commercially
maintained,
servers
that
were
widely
deployed
enough
with
and
with
ample
connectivity
to
support
the
kind
of
experiments
that
internet
researchers
wanted
to
perform.
I
So
these
are
the
two
kind
of
problem
sets
that
the
project
set
out
to
solve,
and
then,
if
you
fast
forward
12
years,
we
have
a
platform
that
I'll
talk
more
about
a
pipeline
that
archives
all
of
the
data
that
the
platform
that
the
experiments
generate
and
then
the
data
itself
that
we
make
publicly
available.
We
also
maintain
a
suite
of
tools
that
are
focused
on
community-based
data
collection,
and
we,
of
course,
also
try
and
foster
our
community
much
like
ripe
to
get
researchers
involved
in
in
using
our
data.
I
Our
team
is
based
at
code
for
science
and
society,
which
is
an
open
data
and
open
science,
fiscal
sponsor.
We
also
have
google
as
a
core
contributor
and
one
of
the
things
they
do
to
contribute.
Is
they
donate
the
time
of
a
team
of
software
engineers?
So
our
team
is
made
up
of
two
organizations
of
core
contributors
so
getting
into
the
the
nitty-gritty
of
how
the
platform
works
just
to
get
the
sense.
I
On
these
servers
we
host
measurement
services
and
they're,
basically
the
server-side
code,
then
clients
are
developed
to
run
tests
against
them.
So
we
see
this
as
measuring
the
inter
part
of
the
internet
going
past
the
the
eyeball
network
so
to
speak
and
having
the
clients
test
against
servers
that
are
within
the
backbone
anyone
can
develop
clients.
So
our
the
the
server-side
experiments
are
approved
are
mostly
written
by
academic
researchers
and
approved
by
our
experiment
review
committee.
But
anyone
can
write
this
the
client-side
code.
I
One
example
is
the
uni
application,
runs
an
entity
test
and
we'll
talk
more
about
ndt,
the
google
search
client
as
well
as
one
of
the
integrators.
So
if
you,
google,
how
fast
is
my
internet?
We
are
what
pops
up
we're
in
the
ndt
test
that
is
hosted
on
mlab
is
what
pops
up
by
nature
of
ndt
being
on
the
project
the
longest.
I
We
have
the
most
data
for
that
test
or
for
that
measurement
service,
and
so
often,
when
you
hear
about
mlab
data,
we're
actually
probably
talking
about
ndt
data,
and
so
and
for
that
reason
I'll
talk
about
that
a
little
bit
more.
These
slides
are
kind
of
my
standard
slide.
So
I
get
into
a
lot
of
nuances
here
that
actually,
maybe
folks
here
are
more
familiar
with
than
most,
but
to
break
it
down.
Just
briefly
ndt,
which
is
one
of
the
measurement
services
that
we
host
is
measuring.
I
The
single
stream
performance
of
bulk
transport
capacity.
Bulk
transport
capacity
is
trying
to
look
at
the
the
reliability
or
the
ability
to
deliver
packets
reliably
across
that
link.
Just
have
some
notes
here
about
how
often
this
term
is
conflated
with
link
capacity.
I
feel
like
this
is
the
audience
that
understands
the
difference,
but
it's
suffice
to
say
often
both
are
completed
with
speed,
and
so
it's
often
thought
of
as
a
speed
test.
I
It's
also
measuring
the
single
stream
performance
as
a
way
to
effectively
measure
the
baseline
of
the
bulk
stream
or
sorry,
the
volca
transport
capacity
and
put
these
together
and
you
get
ndt
and
so
a
question
that
we
often
get
from
from
kind
of
end
users
is.
Why
is
my
you
know,
speed
test
different
than
other
speed
tests
and
it's
just
an
overview
of
what
ndt
specifically
is
trying
to
measure
and
then
in
combination
with
our
offnet
platform?
That's
giving
you
a
vantage
point
outside
of
the
access
network.
I
I
One
thing
to
note
is
that
ndt
7,
which
we're
actually
in
the
process
of
migrating
the
majority
of
our
clients
to
supports
bbr
so
has
a
better
ability
to
model
the
network
that
it's
based,
basing
its
congestion
control
off
of.
It
also
runs
over
tls
and
uses
modern
web
sockets,
and
actually
today
we're
releasing
a
blog
post
talking
about
the
comparisons
between
ndt7
and
ndt5,
our
previous
protocol
and
there's
some
links
there
too.
If
you're
wanting
to
know
more
about
either
bbr
or
nbc7,
you
can
also
run
your
own
mdp
server.
I
I
We
don't
have
a
research
team,
so
we
would
love
to
work
with
you
to
support
your
research
and
your
use
of
this
data
just
going
through
some
of
the
other
slides.
These
are
just
getting
at
that.
It's
all
open
and
free
and
I'll
actually
go
on
to
the
what
the
slides
about
accessing
the
data,
so
the
most
common
way
is
through
our
visualization
site.
I
Metrics
is
to
use
bigquery
and
that's
just
a
sign
up
on
our
on
our
discuss
mailing
list
and
you
get
a
login
and
it's
free
to
free
to
run
queries
as
well.
So
if
there's
any
questions
on
how
to
access
that,
we're,
always
here
at
support
measurement
lab.net
to
support
your
your
analysis,
I
think
I'll,
just
close
by
talking
about
just
even
outside
of
the
the
cover
19
pandemic.
We're
really
interested
in
engaging
with
this
group,
specifically
actually.
So.
I
This
is
a
really
great
opportunity
to
be
introduced
to
you
all,
and
I
would
defer
to
your
expertise
in
terms
of
how
we
can
be
useful
or
how
we
can
contribute
to
this
working
group.
Some
ideas,
if
you
are
part
of
a
of
a
service
provider,
we'd,
be
happy
to
talk
to
you
about
pod
sponsorship
for
extending
our
platform.
B
A
B
I
J
B
K
Yeah
I
have
no
slides
so,
okay,
if
you
can
hear
me
anyway,
then
I
don't,
we
can
hear
you
just
yeah
just
go
ahead
thanks
robert
excellent.
So
yes
thank
you
very
much
for
for
putting
me
on
the
spot
here,
but
it's
fine.
I
unfortunately
will
not
have
slides
to
to
show
you
what
I'm
talking
about,
but
still
probably
I
will
try
to
be
cohering.
K
So
this
is
the
ripencc
tools
update
and
I
usually
begin
with
ripe
atlas.
It's
still
stable
and
still
growing.
We
at
the
moment
are
floating
around
10
000
results
per
second
coming
in
to
the
system.
So
that's
a
lot
of
result.
Data
from
all
over
the
world.
We
have
about
11
000,
a
bit
more
than
11
000,
end
devices
that
we
call
probes
and
we
also
have
600
650
ripe
atlas
anchors,
which
are
somewhat
bigger
machines
that
you
can
also
target.
K
If
you
are,
if
you
want
to
measure
something
and
you're
looking
for
a
target
in
the
network
now
this
is
a
lot
of
data
that
we're
collecting.
So
one
of
the
feedback
that
we
got
recently-
or
I
should
say
continuously-
is
that
if
you
really
want
to
work
with
all
of
that
data-
that's
relatively
hard,
so
what
we're
doing
nowadays
is
putting
all
of
that
data,
or
almost
all
of
that
data
onto
google
bigquery
somewhat
similar.
K
That
measurement
lab
does,
and
we
plan
to
open
that
up
to
the
whole
world,
to
the
research
community
in
particular,
so
that
you
can
get
access
and
with
the
bigquery
interface,
you
can
really
really
dig
deep
and
very
quickly
into
details
if
you
want
to
so
that's
ongoing,
and
I
expect
that
we
can
make
some
announcements
soon,
probably
or
hopefully
before
the
next
writing.
If
not,
then
the
one
after
meaning
everyone
can
have
access
to
all
of
that
data.
K
This
is
in
addition
to
the
way
we
present
and
make
our
data
available
to
the
community.
Anyway,
on
the
labs
platform,
it's
just
making
it
easier
for
anyone
to
to
analyze
that,
as
you
may
have
heard,
from
vesna,
we
are
working
on
what
we
call
software
probes.
So
these
are
basically
software
packages
that
you
can
deploy
anywhere.
You
want
on
your
home,
router
or
or
your
web
server
or
your
any
kind
of
server
that
basically
acts
as
a
ripe
atlas
probe.
K
K
K
We
are
working
on
improving
the
the
actual
measurement
software.
There
are
some
new
measurements
coming
up
and
we're
trying
to
improve
the
existing
measurements.
So
it
has
been
pointed
out
to
us,
for
example,
that
the
trace
routes
measurements
could
be
more
efficient,
which
is
on
the
agenda
for
us
now.
K
The
vm
anchors
are
still
growing.
We
are
doing
some
life
cycle
replacements,
as
you
can
imagine.
If
you
deploy
hardware
every
now,
and
then
you
have
to
refresh
that-
and
these
are
rec
mounted
pcs
so
every
now
and
then
that
needs
to
be
done
somewhat.
Interestingly,
10
years
of
ripe
atlas
is
coming
up.
It's
it's
surprising
even
to
me
that
we've
been
doing
this
for
10
years,
but
it's
almost
10
years
now
the
exact
date
is
october,
end
of
october
or
mid-november,
depending
on
which
number
you
want
to
look
at.
K
I
think
that
for
a
measurement
platform,
10
years
is
a
lot
and
I'm
sure
the
measurement
lab
is
is
also
in
this
ballpark
and
also
if,
for
this
event,
you
would
like
to
appear
and
show
us
what
you
did
with
ripe
atlas.
So
if
you
can
have
something
like
a
testimonial
that
you
think
was
is
worth
sharing
with
others,
then
please
reach
out
to
us
on
the
usual
channels
and
we'll
try
to
include
you
in
the
information
that
we
will
put
out.
K
Also,
if
you
would
like
to
sponsor
atlas,
that's
always
possible
right
that
that's
one
of
the
tools
that
has
not
been
mentioned
today.
Just
yet
so
that's
our
resource,
explanation
service!
If
you
ask
what
do
you
know
about
an
as
number
or
an
ip
prefix
or
an
address,
then
we'll
spit
back
a
whole
lot
of
information
about
geolocation,
routing,
dns
and
so
on.
This
service
receives
more
than
100
million
queries
a
day,
all
of
which
require
some
number
crunching
on
our
ends.
K
The
user
interface
will
be
revamped,
probably
still
this
year,
maybe
early
next
year.
So
we
will
have
something.
That's
nicer
and
works
better
on
mobile
platforms,
which
more
and
more
and
more
people
ask.
So
if
you
want
to
know
something
about
an
ip
address
on
the
road,
you
can
do
that
very
quickly,
ipmap.
That
has
been
mentioned
both
by
massimo
and,
I
think,
also
by
vestner.
K
They
make
their
own
decisions
about
what
they
think
that
particular
ip
address
is,
and
then
the
system
presents
a
unified
output
of
that
the
the
engine
that
massimo
described
is
one
of
those,
but
we're
still
working
on
more
of
those
to
make
it
more
useful
for
people.
Ris
has
been
mentioned
by
vesna
as
well.
So
that's
our
routing
information
service.
K
There
is
some
internal
thinking
about
you
know
what's
next
for
risk
and
we
we
really
wanted
to
cooperate
with
people
who
actually
want
to
cooperate
with
risk
as
well,
so
both
in
terms
of
peering
at
risk
providing
data
to
the
system,
but
also
running
route
collectors.
If
you're
so
inclined,
that's
one
of
the
features
that
we
would
like
to
explore
a
bit
more.
K
Western
couple
examples,
and
one
more
is
coming
up
at
the
end
of
this
session.
So
that's
measuring
about
2810
12
in
the
ipv6
space,
but
in
general,
if
you
wanted
to
be
up
to
date
about
what
kind
of
interesting
information
we
found
or
are
looking
at
now,
you
can
always
just
look
at
white
labs
and
we
will
publish
most
or
all
of
our
researchers
there.
K
B
A
No,
yes,
oh
you
can't
okay,
good
thanks
for
joining
us
during
your
holiday
robert.
I
think
I
take
better
holidays
than
you.
The
the
this
is
the
intellectual
property
rights
statement
to
just
to
notify
the
following
presenters.
If
you
have
ipr
claims
regarding
your
work
to
disclose
them
in
a
timely
fashion,
what
we
so
we're
running
about
10
minutes
later
than
we
thought
we
would
but
that's
good,
because
we
thought
we
would
need
an
extra
10
minutes,
so
we
should
be
able
to
fit
this
in
time.
A
A
A
A
A
D
D
D
D
I
was
I'll
get
into
what
I
was
looking
looking
at
there
when
I
get
to
that
bit
of
it,
but
just
put
a
pin
in
that
and
my
time
to
work
on
this
was
pretty
limited,
but
so
there
might
be
some
interesting
yet
unaddressed
questions.
So
my
first
conclusion
is
that
ce
marking
is
not
very
prevalent,
I'm
seeing
now
about
about
0.3
prevalence
last
march,
when
you
know
after
most
of
the
shutdowns
had
begun.
But
before
I
had
done
a
lot,
you
know.
D
D
This
is
the
apple's
talk
from
a
few
years
ago,
but
we're
measuring
from
the
server
side
and
we're
basically
looking
only
at
the
downstream,
and
I
organized
it
by
asn
instead
of
by
country,
because
it
seemed
more
meaningful
after
kind
of
looking
at
it.
If
you
follow
the
slides
you'll
see
that
it
had
a
big
observation
in
argentina,
but
that
was
later
discovered
to
be
not
relevant.
D
What
we're
looking
at
here
is
the
proportion
of
the
clients
that
ever
asked
for
ecn
that
actually
saw
ce
marks
and
sorted
by
by
asn,
and
then
this
the
size
of
the
asm
corresponds
to
the
size
of
the
data
point.
So
this
is
just
the
top
35
or
so
asn's,
and
you
can
see
that
it
drops
off
quite
quickly.
There's
a
few
asn's
that
seem
to
have
deployed
something
clearly
managed
by
the
asms
that
are
sort
of
up
in
the
or
by
the
isp.
D
That's
you
know
up
in
the
80
90
percentile
percent
of
clients
who
ask
for
a
seat
for
ecn,
do
sometimes
see
marks
during
a
day
and
but
it
drops
off
pretty
quickly,
there's
a
sort
of
middle
ground
here.
So
you
know
here
at
three
percent,
or
so
it's
still
quite
about
you
know
ten
times
higher
than
the
than
the
global
typical
count.
D
D
D
This
also
is
a
little
bit
covered
in
the
backup,
slides,
there's
a
I'm
filtering
for
isps
that
had
at
least
100
clients
that
ever
asked
for
ecn,
and
so
even
in
the
500
ips,
we
have
at
least
100
clients
that
that
sometimes
ask
for
ecn,
but
so
the
three
percent
that
you're
seeing
here
could
be
as
low
as
three
for
a
very
small.
You
know
three
individuals
for
a
very
small
ips,
so
I
don't
think
that's
particularly
indicative
of
anything.
D
D
D
But
if
you
look
at
the
top
hundreds
so
that
you
know
it
continues
down
to
a
smaller
long
tail
here,
but
but
out
of
the
top
hundred
of
the
you
know,
800
or
a
thousand
isps
that
passed
the
filters
that
only
represents
140th
of
the
ce
marking
paths
on
the
internet.
So
so
I'm
concluding
that
most
ce
paths
on
the
internet
are
not
isp
managed.
Most
of
them
are
probably
these
home
router
kind
of
deployments,
they're
widely
available,
but
but
of
the
ones
that
are
isp
managed.
D
The
reason
I'm
distinguishing
between
isp,
managed
or
not
is
because
we
were
interested
in
the
question
of
whether
a
home,
router
deployment
would
be
able
to
address
the
the
latency
issue
mostly
or
whether
it
really
would
need
something
done
by
the
isp,
and
my
tentative
conclusion
here
is
that
it
probably
needs
something
done
by
the
isp.
It's
not
good
enough
to
do
it
in
home,
routers
predominantly
so.
D
D
One
per
connection,
that's
right
when
the
when
the
connection
opens
and
I'm
not
saying
anything
about
the
home,
router's
aqm
impact
on
on
self
congestion
during
an
active
tcp
flow,
I'm
looking
for
mostly
like
what
you
would
see
in
a
gaming
scenario
where
you've
got
a
lot
of
chatty
back
and
forth,
but
not
a
great
deal
of
bandwidth,
and
I
took
the
tcp
handshake
as
a
decent
proxy
for
this.
For
this
kind
of
measurement,.
D
So
the
the
charts
I'm
showing
are
for
one
day
of
data.
I
was
doing
one
day
at
a
time
just
partly
because
the
way
it's
partitioned
for
for
my
access,
this
covers
I've
done
done
this
separately
on
a
few
different
days,
but
not
in
a
really.
You
know
consistent
and
organized
way.
So
I
just
sort
of
did
spot
checks
for
consistency
between
days
and
and
kind
of
qualitative
judgment
of
the.
D
Done
any
like
putting
the
days
together
to
make
good
sense
out
of
them,
so
I
picked
two
particular
asn's
to
to
go
over.
These
are
chosen
to
be
just
sort
of
from
the
kind
of
leading
edge
and
from
the
middle
of
the
pack
of
the
asms
I
had
available,
and
then
also
I
wanted
to
give
a
shout
out
to
the
presentation
also
a
couple
years
ago
about.
D
D
D
D
So
there
might
be
multiple
different
data
centers
that
talk
to
the
same
client
ip,
but
but
those
would
show
up
as
separate
points
in
the
chart,
but
the
same
client
ip
cannot
appear
more
than
six
times
and
probably
not
more
than
five,
because
it
would
not
have
passed
the
filters
if
it
did
that
so
I'll
be
the
charts,
say
pairs
but
I'll
be
talking,
probably
about
client
ips.
Instead,.
D
So
the
the
quote:
good
isp-
this
is
kind
of
what
I
see
so
there's
there's
eight
different
cdfs
here
essentially,
and
what
they
are
is
the
solid
line
is
the
50th
percentile
of
the
client
ips
ban
of
the
latency
span.
They
observed
over
the
over
the
day.
So,
however
many
samples
they
had
you
know,
we've
got
a
client
ip.
That's
that's
was
in
the
middle
here
and
their
50th
percentile
was
had
an
rtt.
D
And
at
least
the
top
outlier,
but
these
98
percentiles
values
of
the
of
the
latency
span
for
the
client
ip.
Then
each
of
those
clients
would
would
represent
some
point
on
this
chart
and
another
point
worth
noting
is
that
the
same
client
might
have
you
know
a
98
percentile
up
here
and
a
91st
percentile
down
here:
they're,
not
necessarily
the
same
client.
D
All
the
way
across
the
only
real
restriction
would
would
be
that
within
the
client,
it's
it's
monotonic,
but
so,
given
that
what
we've
got
is
the
the
the
green
charts
are
the
baseline.
So
that's
all
of
the
all
of
the
data
points
within
that
asn
and
the
blue
ones
are
just
the
ce
marking
paths,
whether
or
not
the
third
mark
any
client
ip.
That
ever
in
the
day
saw
cme
ce
mark
is
included
in
the
blue
chart.
D
D
Some
of
the
ce
marking
paths
actually
had
a
worse
98th
percentile
than
than
the
overall
paths,
I'm
not
sure
if
that's
selection
bias
or
if
it's
an
artifact
from
running
an
aqm,
but
this
represents
an
isp
where
we
think
it's
not.
There's
not
an
isp
managed
marketing
deployment,
and
you
know
these
are.
These
are
the
kinds
of
numbers
I
have
for
this.
D
I
have
probably
what
1500
of
these
charts-
I
guess
you
know
so
I
just
picked
one
out
to
to
be
illustrative
here
per
day
that
I
run
so.
A
middle
isp
is
just
chosen
from
the
middle
of
the
pack
according
to
their
91st
and
98th
percentile,
and
this
is
about
what
we
see
there,
where
the
it's
the
same
thing,
but
the
the
pink
line
is
ce
marking
and
the
orange
line
is,
is
the
overall
latency
spans
that
we
see
and
the
the
conclusion
comes
from?
D
If
you
just
look
at
the
overall
values
of
the
good
isp
versus
the
middle
isp,
it's
pretty
clear
that
this
is,
you
know
being
in
the
good.
Isp
is
dramatically
better
than
doing
ce
marking
in
the
middle
isp.
So
this
is
pretty
typical.
It's
not
universal
and
again
I
have
a
another
chart
in
the
backup
slides
of
a
different
middle
isp,
where
it
was
a
better,
a
better
value
for
marking
ce
for
ce
marking
paths.
D
D
A
A
D
D
D
That
there's
I'm
not
sure
that
it
never
happens.
Obviously
it
you
know
these
things
could
happen,
and
I've
mentioned
it.
Maybe
they'll
start
happening
who
knows,
but
nobody
was
doing
this
as
far
as
I
could
ever
see
during
the
time
I
was
I
was
looking
at
this.
I
don't
have
you
know
if
you're
talking
about
the
kinds
of
of
traffic
that
was
reported
previously,
where
ecn
mark
were
happening
on
every
packet
or
something?
D
No,
that
also
was
definitely
not
happening
most
of
the
time.
What
we're
seeing
is
a
small
number
of
of
blows
ever
get
ce
marked,
even
in
the
even
on
the
path
they're
doing
highly
prevalent
ce
markings,
but
we
did
not
observe
a
lot
in
the
way
of
you
know
we're
a
lot
we're
relying
on
the
on
the
client
heuristics
that
apple
talked
about
for
when
they
see
a
problem
where,
like
every
packet
is,
is
marked,
then
they
will
sort
of
cut
off
the
negotiation
of
ecn.
D
A
Okay,
let's
jump
to
the
other.
The
other
two
questions,
jay
ignacio,
asked
us
simply:
are
the
ecn
observations
all
in
ipv4?
Do
they
also
include
v6?
This
includes
v6
as
well,
okay
and
then,
and
then
the
third
question
was
also
from
bob
and
he's
asking
could
increase
in
congestion.
Experience
be
due
more
to
congestion
over
an
existing
ecn
node
rather
than
an
isp
having
newly
deployed
ecn.
D
D
Part
of
this
can
be
attributed
to
the
normal
growth
that
we
see
in
traffic
over
time,
but
you
know
in
both
cases.
The
days
I
picked
here
were
sundays.
Since
kovid
started,
you
know
it's,
it's
not
yeah.
I
I
don't
have
a
solid
yes
or
no
on
that,
but
I'm
rating
it
relatively
unlikely,
but
but
not
with
great
evidence
to
do
so.
So
I
would,
I
would
say,
if
I
had
contrary
evidence,
then
I
would
you
know
I
would
change
my
mind
on
that.
A
Okay,
okay,
thanks
jake,
so
just
to
reiterate
jake
is
saying
you
know:
if
there's
a
collaborator,
that's
interested
both
he
and
I
have
worked
on
some
of
the
stuff
in
the
past.
Maybe
it's
something
we
can
do
with
map
rg
in
the
hackathon
and
all
right.
What
I'd
like
to
do
now
is
I'm
gonna
switch
to
making
myself
the
presenter
again
and
then
we're
gonna
check
in
with
mark
mcfadden
and
see
if
we
can
hear
him,
because
we
can
just
squeeze
that.
Can
you
hear
me?
L
L
L
If
you
think
about
it,
and
so
what
we
thought
was
what,
if
you
actually
examined
the
rfc
since
2003,
looked
at
the
securities
consideration
sections
and
tried
to
determine
whether
or
not
there
were
fundamental
and
crucial
changes
to
the
threat
model
for
the
internet
and
also
a
significant
changes
to
the
way
that
people
are
writing
this
security
consideration
section,
and
this
isn't
just
a
sort
of
piece
of
independent
research.
The
idea
was
really
to
inform
some
of
the
other
work.
That's
going
on
in
the
iab.
So
next
slide
dave.
L
Yeah,
so
the
iab
has
already
talked
about
potential
revision
to
3552.
The
iab
currently
has
a
piece
of
work
underway
right
now
called
cleverly
enough
model.
T
that's
talking
about
changes
to
the
threat
model
that
are
in
it.
That's
in
rfc,
3552
and
really
the
the
body
of
work
since
rfc
3552
is
actually
fairly
substantial
since
we're
we're
in
the
neighborhood
now
of
having
five
and
a
half
thousand
more
rfcs
to
look
at
so
next
slide.
L
One
is
a
quantitative
component
and
more
about
that
later,
the
quantitative
mode
component
is
really
a
textual
analysis
of
the
the
words
that
are
in
the
rcs
and
then
a
qualitative
analysis,
and
so
what
what
the
draft
was,
what
the
proposed
piece
of
research
does
is
propose
this
methodology,
and
what
you'll
see
in
a
moment
is
that
this
proposed
methodology,
one
part
of
it,
we've
done
an
experiment
on
so
the
next
slide.
Please
dave.
L
So
what
the
methodology
is
in
search
of
is
comments
for
improvement,
basically,
except
for
an
experiment
that
I'll
report
on
in
a
minute,
the
actual
methodology
hasn't
been
implemented.
Yet
there's
no
one
who's
actually
executed
it.
Although
there
have
been
volunteers,
who've
stepped
up
to
the
plate
and
said
they'd
be
interested
in
doing
it.
L
L
the
thought
was
here,
the
the
hypothesis
was.
Would
you
detect
changes
in
the
words
that
are
actually
being
used
to
indicate
the
threats
and
the
threat
models?
I
mean,
how
is
it
if
you
look
over
the
period
of
17
years?
Has
that
changed
and
would
you
find
patterns
that
indicate
that
some
terms
have
become
more
important
to
protocol
designers
by
virtue
of
appearing
more
often
in
rfc
text,
and
then
one
of
the
things
that
this
last
bullet
on
this
slide?
L
My
co-author
was
exceptionally
interested
in,
was
taking
a
look
at
whether
or
not
security
in
large
scale.
Security
incidents
on
the
public
internet
had
changed
the
way
that
the
pro
that
protocol
designers
had
written
the
security
considerations
of
their
documents.
The
idea
was:
do
events
in
the
real
world
on
the
public
internet
affect
the
way
that
protocol
designers
do
their
work
and
express
that
work
through
the
security
considerations
sections
of
these
documents?
L
So
the
parser
that
we
used
was
a
basically
an
open
source
parser
it
it
removed
all
non-textual
material
and
it
removed
what
we
call
stop
words,
words
that
are
either
words
like
a
and
those
sorts
of
things,
or
other
words
that
wouldn't
wouldn't,
in
our
judgment,
be
related
to
the
protocol
itself.
It
also
removed
some
other
materials,
for
instance
rfc
references
and
anchor
urls
and
so
forth.
L
Then
it
took
all
of
that
text
and
it
parsed
it
into
files
by
year,
so
the
rfc
was
actually
parsed
year
by
year
to
so
that
we
could
actually
have
sort
of
a
time
series
of
what
the
rfcs
looked
like
or
the
security
consideration
sections
of
the
rfcs
and
then
for
each
one
of
those
years.
It
built
a
frequency
list
of
all
the
words
that
weren't
in
the
designated
list
of
stop
words.
So
next
slide.
L
L
Do
new
words
start
to
appear
as
we
as
we
move
forward
in
the
years
or
in
response
to
particular
attacks
and
the
second,
a
second
thing
we
did
was
to
actually
do
an
rfc
count
of
how
many
security
sections
actually
referred
back
to
previous
rfcs,
what
those
rfcs
were
and
what
the
count,
what
the
quantitative
data
was
for
that
so
dave.
Could
I
go
to
the
next
slide.
L
L
I
think
that's
interesting
input
to
the
model
t
effort,
for
instance,
it
shows
that
that
our
c
people
who
are
drafting
rfcs
are
tending
to
use
the
same
words
despite
the
fact
that
the
internet
is
evolving
significantly
so
again,
there's
more
details
in
the
internet
draft,
but
this
gives
you
a
feel
for
what
the
top
10
word
counts
are
in
four
sample
years,
separated
by
five
years.
So
next
slide.
L
L
L
L
And
so
here
are
some
conclusions
we
draw
we
drew
and
I've
already
talked
about
them.
So
I
won't.
I
won't
in
the
interest
of
time
bash
this
too
much,
but
there
really
isn't
much
in
the
way
of
major
changes
if
you're
just
doing
word
accounts
of
the
security
consideration,
sections
and
act.
Look
at
that
in
the
draft
proposes
that
you
also
do
a
qualitative
analysis
and
my
co-author-
and
I
have
not
done
that.
L
Yet
it
turns
out
that
the
word
may
appears
more
often
than
any
other
normative
language
in
security
consideration
sections.
We
found
that
to
be
extraordinary
that
that
that
the
word
may
instead
of,
must
appears
and
appears
much
more
often
in
security,
consideration
sections,
and
so
that
also
is
input
to
the
iab
and
input
to
other
conversations
about
the
internet
threat
model,
and
my
last
slide
is
just
contact
details
dave
I'm
happy
to
first
of
all
have
comments.
L
A
If
not,
I
I
I
just
have
one
I.
I
love
the
idea
of
using
the
artifact,
that
is
the
rfcs
as
a
way
to
determine
whether
the
engineering
is
moving
with
the
real
internet.
Was
there
any
any
real
life
internet
phenomenon
that
that
you
could
correlate
with
the
change
in
either
quantity
of
words
or
just
the
quantity
of
the
security
considerations
itself
I
mean
you
think
you'd
see
evidence
of
the
iab
or
the
ietf's
policy
changes
even
on.
L
L
Certain
changes
to
a
certain
maintenance
for
tcp.
We
would
have
seen
changes
to
the
wording
there
and
we
didn't-
and
we
also
looked
at
the
latency
issue
right
so,
for
instance,
dave.
If
we
look
at
as
an
example,
the
dynatech
one
of
the
things
we
said
was
well.
We
wouldn't
start
to
see
changes
in
the
rfcs
until
about
a
year,
18
months
later
because
of
the
publication,
the
the
pace
of
publication
right.
L
But
when
we
looked
at
that
particular
example,
we
didn't
see
much
in
the
way
of
change
and
we
actually
went
back
and
looked
at.
We
targeted
particular
series
of
rfcs.
For
instance,
we
went
to
a
ietf
area
and
then
limited
the
examination
to
just,
for
instance,
the
internet
area
or,
and
the
results
were
pretty
much
the
same
so
and
I
see
in
the
chat
room
that
bob
suggests
that
you
know
privacy
should
have
been
appearing
more
and
I
agree
with
you
bob,
but
it
it
simply
doesn't
show
up
when
you
do.
L
A
H
H
H
H
H
H
H
H
H
I
have
summarized
the
delay
increase
condition
down
here,
and
the
value
of
theta
is
actually
defined
as
the
minimum
round
to
time
divided
by
8,
but
is
clamped
between
the
values
of
4
and
16..
So
since
in
our
measurements,
the
the
round
trip
time
is
small,
theta
actually
always
takes
the
value
of
4..
H
So
then,
going
back
to
this
condition
here
we
can.
We
can
actually
identify
two
reasons
why
this
condition
can
become
true,
so
either
we
already
have
found
our
minimum
roundup
time
and
then
the
current
runtime
time
increases
making
this
condition
true
or
we
are
outside
of
the
sampling
phase
of
around
so
beyond
the
first
8x
of
around.
H
H
So
here
I
have
plotted
the
roundup
time
samples
that
correspond
to
different
packets
and
so
for
the
first
round-
and
here
I
have
to
say
that
the
definition
of
a
round
from
the
high
start
point
of
view
actually
contains
a
bug.
So
high
start
considers
the
first
11
packets
to
belong
to
the
first
round
instead
of
the
first
10
packets.
H
But
actually
what's
the
case
is
since
the
initial
window
size
is
10.
The
sender
sends
10
packets
in
the
first
round,
not
11.,
but
let's
stick
with
this.
For
for
this
example,
so
first
of
all,
we
have
to
find
the
minimum
rounded
time
plus
theta,
so
the
threshold
value
that
we
compare
the
current
round
time
against,
and
then
we
find
our
current
rounded
time
as
the
minimum
round
to
time
during
the
sampling
phase,
which
is
here
shown
in
gray.
H
Each
time
we
go
for
another
round,
we
head
and
reset
the
current
round
tip
time,
and
then
again
we
find
our
threshold,
which
then
here
drops
on
the
31st
packet,
and
we
find
our
current
rounded
time,
which
is
this
time
20..
So
again,
the
dashed
line
is
above
the
solid
line,
so
we
go
for
the
for.
So
we
again
reset
the
current
round
trip
time
then
and
go
for
the
third
round.
H
H
H
this
in
the
the
example
for
the
second
reason,
is
actually
pretty
similar,
so
in
the
first
round
nothing
else
happens
in
the
second
round.
We
again
see
the
drop
on
31st
packet,
but
this
time
the
round
trip
time
of
the
15th
packet
is
higher
than
before.
So
now
the
current
rounded
time
is
22
instead
of
20,
which
then
makes
the
drop
on
the
31st
packet
to
force.
H
So
now
we
have
understood
these
two
reasons
and
we
want
to.
We
wanted
to
know
okay,
how
likely
are
there?
How
likely
are
these
two
reasons
so,
therefore,
we
did
another
experiment
and
actually
figured
out
that
the
probability
that
we
leave
the
high
start
due
to
an
increasing
roundup
time,
which
is
actually
how
the
delay
increase
mechanism
is
supposed
to
work,
only
happens
in
30
percent
of
the
cases,
but
in
70
percent
of
the
cases
we
leave
the
high
start
when
the
round
time
decreased.
H
H
So
in
a
mobile
radio
network,
the
radio
access
network
actually
controls
all
the
transmissions
so
also
the
uplink
transmissions.
So
the
modem
needs
uplink
resources
in
order
to
transmit
its
data,
or
in
this
case
the
x.
So
then
there
are
two
cases
so
either
the
modem
already
has
some
some
uplink
resources.
H
Consider
the
modem
to
be
idle
from
a
transmission
point
of
view.
So
then
the
modem
has
to
cue
these
10x
and
wait
for
the
next
opportunity
to
send
a
scheduling
request
after
some
time.
The
modem,
the
radio
access
network
will
then
respond
with
a
small
amount
of
with
a
scheduling
ground
that
grants
a
small
amount
of
resources.
H
H
H
So
when
you
look
at
this
graph
here
this
time,
I
have
plotted
the
mean
round
to
time
of
individual
packets
and
by
looking
at
the
first
10
packets.
Here,
you
can
pres
precisely
see
what
I
just
explained
so
the
first
two
packets
have
a
smaller
roundup
time
and
then
the
rounded
time
ramps
up
by
about
eight
milliseconds
on
on
the
third
packet.
H
H
H
H
H
So
since
highstat
still
does
not
consider
the
round
to
time
variance,
it
may
still
not
be
well
suited
for
mobile
radio
networks
and,
in
the
end,
it's
important
to
say
that,
even
though
we
conducted
our
drive
with
lte,
we
presume
that
the
same
effects
will
also
happen
on
5g.
Since
there
we
use
the
same
kind
of
uplink
scheduling,
principles.
A
A
For
instance,
bob
comment:
bob
briscoe
commented
that
high
start
is
on
by
default
in
linux
cubic,
but
according
to
a
mini
survey,
he
says
he
did
a
four
to
five
cdn's
about
a
year
and
a
half
ago,
for
instance,
google,
microsoft
and
apple
all,
but
won
a
disabled
high
start.
Do
you
have
a
comment?
Fellow
before
we
wrap
up.
A
So
it
would
be
a
big
help
to
us.
If
you
would
visit
that
url.
I
just
posted
there.
The
etherpad
map,
our
g
url
and
just
add
your
name
and
affiliation
there.
So
we
get
a
list
that
approximates
the
what
I
saw
about
74
participants
on
today
and
we've
got
56
right
now.
So
in
the
next
half
hour,
if
you
sign
that
that
would
be
great
just
like
you
would,
if
we
were
at
a
real
ietf
meeting
mike
I'm
going
to
hand
it
over
to
you,
you're
ready.
G
A
G
All
right,
thank
you
dave.
So
this
is
a
quick
recap
on
our
work
of
tcp
conformance,
which
has
been
presented
at
palm
in
march
of
this
year.
So
dfb
has
been
analyzed
in
the
past
decades.
However,
every
study
focused
on
different
aspects
of
tcp
and
tcp
extensions,
and
our
question
was
okay:
how
is
the
conformance
to
the
minimum
requirements
so
the
basic
requirements
which
are
phrased
by
the
must
keywords
in
the
rcs
and
for
this
we
chose
an
active
scanning
approach.
We
did
control
testbed
environment
measurements.
G
So
for
our
test
cases,
we
looked
at
the
rc
793
abyss,
the
giraffe
14
version,
and
we
looked
at
different
aspects.
So
we
checked
if
the
check
sounds
are
correctly
validated
if
options
which
are
unknown
are
correctly
ignored.
If
the
mss,
which
are
stated
are
the
rc
stated
defaults,
if
the
effective
sent
mss
is
honored.
If
result,
flags
are
ignored
in
zeros
and
if
the
urgent
pointer
could
be
processed
segments
of
arbitrary
links.
So
with
these
test
cases
we
started
out-
and
this
is
something
we
could
of
course
only
be
observed
on
the
wire.
G
G
What
we
see
here
is
that
only
linux
and
lightweight
ips
showed
full
conformance
to
our
test
cases.
When
we
look
at
windows
10,
then
we
see
that
the
mssd4
stated
in
the
rc
are
used
as
a
lower
bound.
So
you
cannot
actually,
in
this
rear,
handshake
use
something
lower
than
it's
stated
in
the
rc
for
mac
os.
The
1024
bytes
are
used
as
a
default,
regardless
of
the
ip
version,
so
this
falls
short
of
the
ipv6
and
exceeds
the
ipv4
defaults
on
micro,
ip.
G
We
actually
observed
some
pressures,
and
this
was
done
on
urgent
pointer
data,
which
was
pointing
beyond
the
segment
size,
and
we
issued
a
pull
request
for
this.
This
was
especially
interesting
as
contiguous
and
contig.
Os
are
also
vulnerable
from
this
issue,
and
this
pull
request
got
merged
and
is
fixed.
Now,
for
a
c-star,
we
saw
that
the
holster
s
is
supported
of
offline
to
check
some
features
is
not
verified.
G
We
we
reported
this
issue.
However,
to
this
stage
we
did
not
get
any
response
from
this
all
right
for
our
live
scale
measurement
campaign.
We
looked
at
different
target
hosts
from
the
http
archive
we
sampled
cdn
targets.
There
are
28k
unique
targets
house
we
took
around
476
000
unique
target
holes
from
the
alexa
top
run
million
list
and
on
sensors,
which
provides
antenna
white
squad
port
scans.
We
took
around
3.2
million
targets,
so
let
me
start
with
the
notation
here
in
our
three
columns.
G
We
see
the
three
data
sets
and
in
each
sub
column
we
see
the
different
results
which
are
presented
as
a
percentage.
So
for
u
and
k
sub
column
here
this
is
unknown.
These
are
not
clearly
determinable
results.
This
can
have
multiple
issues.
For
example,
if
a
pre-test
was
successful,
but
the
actual
test
of
the
feature
we
wanted
to
check,
we
did
not
get
a
response
to
our
sun
packet.
G
Okay,
digging
right
in
first
of
all,
we
have
the
checksum
test.
We
did
this
in
two
ways.
First
of
all,
which
we
chose
a
random
incorrect
checksum,
and
we
also
did
the
test
with
zero
checksum
and
for
checksum.
We
saw
that
the
cdn
shows
the
lowest
failure
rates
overall
and
actually
no
on
path,
modifications,
which
is
quite
good
news.
However,
alexander
senses
each
show
around
three
percent
target
failure.
So
in
around
three
percent
of
the
targets,
they
did
not
validate
the
checksum
and
responded
to
our
false
false
checksum
package.
G
Next
of
all,
we
have
option
unknown,
and
so
no
singer
s
really
stands
out
here,
but
we
observe
that
the
highest
figure
rates
are
within
isp
networks,
and
this
is
exactly
the
same
behavior
as
we
saw
in
the
mss
missing
test.
So
for
mss
missing
we
saw
around
0.4
percent
percent
failure
on
the
path
of
the
census
data
set,
and
these
are
also
primarily
located
in
isp
networks
and
digging
down
on
this.
We
saw
that
the
mss
is
inserted
here,
and
this
is
most
likely
due
to
the
pppoe
encapsulation
done
by
access
rotors.
G
G
Alright,
so
that's
for
our
quick
recap:
please
have
a
look
at
our
paper
if
you're
interested
there
are
a
lot
of
more
test
cases
and
more
details
in
there.
We
also
released
our
data
set
and
we
released
our
code,
so
our
tool
called
tcp
reg,
is
on
github
feel
free
to
contribute,
feel
free
to
share
your
results.
I'm
happy
to
progress
this
further.
Thank
you.
A
Thanks
mike-
and
we
have
a
couple
of
minutes
for
questions,
the
first
one
that
was
in
the
chat
was
jay
ignacio
asked.
How
did
you
determine
this
was
during
your
results?
Presentation,
for
instance,
the
results
slide
two.
How
did
you
determine
the
middle
box
errors
instead
of
determine
the
middle
box
errors
instead,
targets.
G
So,
for
example,
if
we
wanted
to
test
if
on
a
path
the
the
tcp
checks
on
is
corrected,
then
we
would
get
a
reply
with
a
corrector
checks,
checks
on
with
this
approach,
and
then
we
could
say:
okay,
this
was
a
failure
that
actually
happened
on
the
path
and
not
on
the
target,
and
we
did
this
for
all
of
our
test
cases.
So
this
was
a
lot
of
handcrafting.
There's
no
general
approach
here.
Besides
the
tracebox
approach,
which
generally
can
give
you
a
hint
if
there
is
a
metal
box
or
not.
A
A
M
All
right,
thanks
dave-
this
is
a
this-
is
a
summary
talk
of
work
that
we've
presented
already
tma
on
nrw,
and
this
covers
the
deborganization
of
12.
For
a
little
bit
of
context.
This
was
the
first
slash
12
allocated
by
the
ayanna
to
any
of
the
registries
in
at
the
time
I
think
13
or
nearly
13
years,
which
is
long
enough,
for
you
know,
writer
configurations
to
become
stale
and
thanks
for
you
know
to
be
baked
into
the
writing
system.
M
So
before
we
issued
it
to
members,
we
wanted
to
light
that
up,
get
that
announced
out
there
and
generally
try
and
do
some
checking
to
make
sure
that
it
wasn't
packed
with
pollution,
which
was
unlikely,
but
you
don't
until
you
measure
so
the
timeline
for
this
was
we
actually
got
the
slash
12
back
in
late
2019,
but
in
early
2020,
the
previous
slash
12
was
nearly
done.
We
were
eminently
about
to
start
allocating
from
28
slash
12,
so
we
had
about
one
week
to
advertise
this
and
collect
what
we
could.
M
We
announced
nine
prefixes
in
total,
the
full
slash
twelve,
then
four,
slash
thirty
twos
and
four
slash
four
to
eight.
Each
of
these
were
configured
slightly
differently
in
terms
of
whether
they
had
a
routing
right
object
in
the
writing
database
or
whether
they
had
an
rpki
row
and
the
32s
and
48s
all
had
a
responsive
target.
So
each
of
them,
each
of
the
eight
addresses
in
total,
would
respond
to
icmp
echo
requests.
M
M
The
the
bulk
of
the
tma
paper
is
basically
traffic
analysis
and
the
way
that
we
kind
of
think
about
this
is
with
the
ipv4
space.
You
could
listen
to,
for
example,
a
slash
eight.
If
you
have
a
field
slash
eight
lying
around
and
depending
on
some
of
the
behavior,
you
may
be
able
to
draw
general
conclusions
about
background
noise
to
the
ipv4
space.
Like
each
v4,
slash,
8
is
going
to
be
different
in
terms
of
what
it
attracts,
but
the
v4
space
is
also
small
enough
that
you're
going
to
see
internet-wide
quote-unquote
scans.
M
The
v6
space
is
quite
different.
It's
a
bit
more
like
taking
a
telescope
and
pointing
at
the
sky
and
simply
observing
what
you
see
in
that
part
of
the
sky.
There
may
be
something
generalizable
there,
but
it's
more
likely
that
you
can
report
on
what
you
saw
in
that
corner,
so
that
slash
12
is
a
lot
of
address
space,
but
it's
still
a
very
small
part
of
the
v6
address
in
total.
M
So
what
we
got
is
what
we
got
and
I'll
be
careful
with
drawing
general
conclusions
around
that
the
the
part
that's
kind
of
a
good
start
is
the
bulk
of
the
traffic
that
we
actually
captured
was
our
own
traffic.
We
sent
a
lot
of
measurements
into
this
space
to
the
responsive
targets
from
the
right
atlas
platform,
and
so
the
vast
majority
in
my
head.
I'm
thinking
that
looks
like
around
about
90
to
95
percent
of
the
traffic
that
we
captured
was
generated
by
ripe
atlas.
M
M
The
traffic
can
be
carved
out
in
a
couple
of
broad
categories.
The
bulk
of
the
tcp
traffic
was
what
looked
like
an
orchestrated,
traceroute
campaign
to
locate
responsive
well
to
do
I
guess
to
do
two
things
to
map
out
topology
and
then
find
anything.
That's
responding
in
port
80,
the
dns
sorry,
the
udp
traffic
was
around
about
50
dns
and
the
bulk
of
the
icmp
traffic
was
solicited.
Echo
requests
to
the
addresses
that
we
set
up
so
for
each
of
those
in
turn
around.
E
M
And
a
half
million
of
the
packets
contain
the
tcp
payload
and
they
were
doing
some
kind
of
tcp
traceroute
right
so
trying
to
repeatedly
reach
the
same
target,
but
with
an
ever-increasing
ttl
until
it
eventually
gives
up
all
of
that.
Traffic
has
the
same
basic
characteristics
so,
like
the
same
source,
port
no
payload
send
flag
is
set,
nothing
else
is
active
in
the
packet.
The
msf
value
is
set
as
if
it's
configured
for
an
ipv4
world
there's
a
couple
of
glitches
like
that
and
most
of
this
traffic
in
the
activity.
M
M
Whatever
that
thing
is,
it
was
never
expecting
to
see
a
dress
face
that
large
and
something
bugs
out
in
whatever
the
platform
was,
and
it
started
sending
far
too
many
packets
into
the
space,
and
my
gut
feeling
for
that.
The
reason
I
have
that
gut
feeling
is
that
once
we
let
up
the
32s
and
48s,
there
were
specific
traceroutes
into
those
addresses,
but
at
a
much
lower
volume.
M
So
those
let
up
very
quickly
and
then
stopped
very
quickly,
which
was
good.
Basically,
they
bounced
around
in
the
random
addresses,
though
we
there
was
a
distinct
port
scanning
operation
from
one
origin
to
around
about
sorry,
I
have
targeted
addresses
in
the
next
slide,
around
164
000,
packets,
doing
port
scanning
for
one
origin
and
then
like
tens
of
thousands
with
the
destination
portfolio,
three,
the
night
flags.
M
It's,
I
can't
remember
if
they
have
payloads,
but
I
figure
something
is
either
backscatter
or
is
bouncing
around
looking
for
resets
rather
than
synax
and
there's
a
lot
of
general
noise.
That's
otherwise
uninteresting
beyond
that
point
for
the
traceroute
campaign,
the
addresses
that
that
campaign
selected
appear
to
be
random
bits,
filling
the
bottom
120
sorry
116
bits
of
each
address
and
the
assertion
there
comes
from,
like
the
campaign
selects
in
total
261
000
targets
spread
across
261,
000
64..
M
It's
like
it's
not
trying
to
do
anything
smart
with
particular
addresses
and
say
the
slash.
64
is
just
trying
to
get
something
whether
it's
topology,
mapping
or
not.
The
remaining
targets
are
pretty
well
distributed.
This
graph
on
the
right
is
ranked
like
most
popular
64
target,
I'm
going
down
from
there,
but
there's
133
000
targets,
and
this
forms
a
very,
very
long
tail,
where
many
of
the
addresses
are
targeted.
Maybe
once
or
twice
the
top
target
in
that
list
was
targeted,
the
most
for
the
port
scanning
campaign
from
from
one
origin.
M
M
The
port
numbers
that
were
targeted
are
a
little
bit
interesting,
maybe
like
I've.
I've
taken
poor
ac
out
of
this
plot
because
it
would
swamp
everything
with
the
traceroute
campaign
and
what's
left,
it's
kind
of
broad
like
at
some
point.
Basically,
everything
is
is
attempted
by
something
there's
a
lot
of
noise.
M
The
most
popular
target
is
port
443
and
then
from
there
it
hits
and
I'm
going
to
go
by
memory
here,
6379,
which
I
think
is
redis
5222
is
maybe
xmpp
and
I
lose
the
track
from
there
there's
quite
a
lot
of
traffic
with
the
tcp
for
zero
set,
but
this
is
aside
from
a
general
leaning
towards
the
low
numbers
that
the
port
scanning
does.
A
lot
of
this
is
like
random,
randomized,
payloads
and
weird
targets.
M
The
udp
traffic
was
much
less
interesting.
I
guess
there
wasn't
much
of
it.
50
percent
of
the
dna,
the
udp
that
we
captured
contained
a
dns
payload
that
looked
like
legitimate
dns,
like
there
were
dns
queries
for
like
for
a
and
quality
records
for
popular
names
like
google.com
or
various
cdnc
names
or
whatever
else
this
this
one.
M
We
did
actually
approach
the
network
operator
and
they
found
the
person
who
was
running
a
dns
resolver,
which
was
misconfigured
and
was
sending
these
queries
into
our
address
space,
so
that
one
was
in
theory
fixed,
but
we're
no
longer
announcing
that
address
space.
So
we
can't
verify
that
it
was
actually
fixed.
M
The
this
is
again
a
on
the
right,
our
ranking
of
the
most
popular
targets
and
the
top
eight
targets
are
the
the
eight
basically
reflect
the
eight
responsive
targets
we
created
and
we
requested
the
network
operator.
Send
our
traffic
to
this
graph
is
ignoring
the
ripe
atlas
traffic.
Also,
so
most
of
the
icmp
goes
to
where
we
asked
people
to
send
icmp
echo
requests
the
next
most
popular
is
2810
with
all
the
zeros,
which
was
unresponsive,
but
a
lot
of
people
selected
that
one
rather
than
a
column
called
one.
M
Maybe
people
truly
are
that
lazy,
I'm
not
sure
and
there's
a
long
tale
of
otherwise
essentially
randomized
traffic
targets.
Again,
a
bunch
of
these
in
the
remaining
targets
are
like
genuinely
what
appears
to
be
randomized
payloads
with
invalids,
icmp
types
and
codes
so
forth.
So
a
lot
of
that
stuff
is
just
weird
and
it's
the
numbers
at
that
point
get
so
vanishingly
small
that
we
basically
draw
under
draw
a
line
under
it
and
declare
it
to
be
acceptable.
M
M
We
cover
aspects
of
the
writing
state
in
both
the
tma
and
the
nrw
papers.
Although
not
in
a
spectacular
amount
of
detail,
we
wanted
to
make
sure
that
first,
the
space
was
basically
routable
and
then
from
there
we
used
the
ripe
atlas
measurement
platform
to
ensure
that
it
was
actively
reachable.
If
we
tried
to
get
there
and
in
general
it
was.
M
M
So
that's
completely
okay
and
on
those
ones
we
didn't
see
the
echo
requests
arriving
at
the
at
the
right
collector
that
was
announcing
the
space
so
for
those
ones
they
are
filtering
the
routes
at,
I
guess
3320,
so
our
current
status
and
where
we
are
with
this.
This
was
a
study
in
january.
This
is
basically
historic
at
this
point.
The
ripe
atlas
data
and
the
risk
writing
data
is,
of
course,
all
public.
The
writing
data
in
particular.
You
can
also.
M
M
Yep
sure
so
I'm
about
done
so
the
the
2810
slash
12
is
now
in
use
soon
after
we
ended
the
study,
so
the
week
in
january
is
on
the
left
hand
side
soon
after
we
ended
that
was
issued
to
members
and
then
those
prefixes
started
lighting
up.
The
only
reason
that
the
numbers
go
up
and
to
the
right
is
that
risk
continues
to
add
new
peers.
M
So
we
have
more
peers
simply
listening
to
and
receiving
these
announcements
and
the
address
space
that
we
allocated
our
eights
thirty
thousand
forty
eight
from
was
put
into
the
quarantine
did
because
we
did
active
measurements
to
it
and
we
solicited
traffic
to
it.
That
space
is
now
out
of
quarantine.
The
standard
is
six
months,
so
the
space
has
now
been
reissued
and
it
I
haven't,
checked
today,
but
it's
likely
to
show
up
in
the
wild
soon
if
it's
been
allocated
to
somebody.
M
So
there's
a
potential
comparison
there
between
the
address
space
announced
from
our
red
collector
system
and
the
address
space
announced
from
somebody
else.
If
somebody
wants
an
interesting
writing
site
study-
and
these
are
only
my
endnotes
indicating
what
we
did
here-
there
isn't
anything
in
this
space-
that's
entirely
problematic.
The
writing
and
reachability
seemed
to
be
good,
and
so
we
were
completely
happy
going
ahead
and
issuing
that
address
space
to
our
members,
and
that
is
my
high
level
summary
of
these
two
papers.
A
A
Let
us
know
I
want
to
thank
brian
and
nina
for
doing
this
experiment
with
us,
this
joint
meeting,
let
either
them
or
us
miria-
and
I
know
if
you,
if
you
like
the
joint
meeting
and
and
how
you,
how
would
you
like
us
to
meet
in
the
future-
and
I
want
to
specially
thank
maria
for
encouraging
presentations
and
helping
curate
the
content,
because
she
did
a
line
sure
of
it.
This
time
again
have
a
good
day.
Folks,.