►
Description
ACM, IRTF & Internet Society Applied Networking Research Workshop 2017 - Part 4
Prague, Czech Republic
Saturday, July 15, 2017
A
Okay
time
for
our
last
session
that
we
have
a,
we
have
a
panel
that
came
out
of
a
paper
submission.
The
panel
is
going
to
be
chaired
by
Leslie
Daigle,
who
has
been
around
at
the
IETF
since
1995,
so
for
a
substantial
amount
of
time.
So
she
has
seen
all
kinds
of
ups
and
downs
and
changes
in
the
Internet,
and
that's
probably
puts
her
into
a
perfect
position
to
talk
about
Internet
health.
B
Thank
you
very
much
so
yes,
this
this
panel
came
out
of
a
paper
that
I
had
submitted
for
this
workshop
about
the
network
operator
measurements
activity
that
I've
been
working
on.
You
can
find
more
information
about
that.
Wwh,
org,
slash,
Noma
or
more
likely
talk
to
me
afterwards.
So
we're
not
actually
going
to
talk
about
that,
although
I
will
provide
a
bit
of
context
from
it,
but
instead
we've
got
panelists
here
who
are
very
well
positioned
to
talk
about
metrics
and
internet
health.
B
B
Yeah,
so
the
basic
thesis
behind
me,
the
Noma
work
is
to
get
operators
network
operators
to
self
instrument
to
take
measurements
from
the
edge
of
their
networks
towards
some
well
known
user,
interesting
resources
in
the
center
of
the
network
to
get
a
sense
of
how
well
their
network
is,
is
working.
Let's
put
it
that
way.
Really,
roughly,
the
measurements
are
not
complex.
B
That
is
an
extraordinarily
high
level,
some
nail
sketch
of
what
I
was
talking
about
just
the
sort
of
lay
some
groundwork
for
the
background
that
I
was
coming
from,
so
I
did
actually
have
a
panel
description,
instrumenting
networks,
measurements
to
track
network
and
Internet
health.
This
panel
will
explore
the
challenges
of
establishing
a
measurement
collection
framework
within
an
operational
network,
instrumenting
it
for
the
purpose
of
tracking
the
network
state,
as
well
as
contributing
to
a
picture
of
the
overall
internet
state
of
health.
B
Apart
from
narrowing
down
what
internet
health
actually
means,
challenges
include,
choosing
measurements
that
are
meaningful,
markers
are
for
of
user
experience,
carrying
out
measurements
within
the
network,
without
otherwise
impacting
the
network's
performance
or
perturbing.
Any
target
resources
at
which
measurement
traffic
is
aimed
no
ddossing,
and
these
and
other
challenges
and
insights
will
be
discussed
by
the
panelists
from
operators
to
infrastructure
operators
and
network
architects.
B
C
From
security
background
network
security
in
particular,
and
then
I
joined
the
NCC
to
do
rpki,
which
some
of
you
might
have
heard
of
protocol
work
and
so
on
and
later
on,
join
the
then
forming
science
team,
which
is
now
the
rnd
team
and
I'm
managing
the
R&D
team
ever
since
then.
So.
The
involvement
here
is
that
the
the
NCC
has
been
very
active
in
the
in
the
measurement
and
active
and
passive
measurements
of
the
internet.
C
For
some
time
now,
some
of
you
know
risks
the
routing
information
system
that
has
been
collecting
BGP
data
from
around
700
peers,
or
so
since
1999
TTM,
the
test
traffic
measurements
Oh
public
beta,
is
available.
Soviets
goes
without
saying
the
test:
f
ik
measurements,
which
were
instrumented
rack-mounted
boxes
with
gps
antennas
mounted
on
roofs
and
so
on.
So
they
can
do
one-day
delay
on
the
one-way
delayed
measurements
with
high
precision
that
was
also
running
for
like
12
years
or
so
now
it
has
been
superseded.
C
So
we've
been
in
space
for
for
some
time
now
and
somewhere
around
2008
ish.
We
came
to
realize
that
there
is,
if
network
operators,
which
is
our
primary
concern,
because
we
try
to
serve
the
ISPs
in
this
region
whenever
they
have
questions
about
I
think
there
is
something
wrong
with
the
internet.
It's
not
me,
I,
don't
really
know
who
the
who
causes
the
problem,
and
if
there
is
a
problem,
but
I
feel
that
something
is
going
on.
There
is
a
disturbance
in
the
force
or
some
someone
actually
complains
about
it.
C
How
do
I
go
about
it?
How
can
I
figure
out
what
the
problem
is,
where
the
problem
is
and
and
who
is
to
blame
or
how?
More
importantly,
how
can
we
fix
it?
So
the
only
way
was
to
to
you
know,
talk
to
your
friends
and
say:
I.
Think
you
have
a
machine
in
in
in
Vienna.
Right
can
I
start
pinging
from
there
and
then
you
know
rarely
have
those
kind
of
people
to
support
you.
C
We
do
have
HTTP
measurements
as
well,
but
they
are
not
publicly
available
because
those
reasons
that
we
can
go
into
later,
if
that's
a
topic,
but
our
our
approach
was
to
build
a
basically
a
community
based
network.
So
if
someone
is
willing
to
extend
the
network,
then
we
should
let
them
and
we
should
encourage
them,
but
in
return
they
should
get
some
benefits
out
of
the
network
and,
if
I'm,
assuming
that
most
of
you
know
what
what
ripe
Atlas
is
about.
That's
basically
what
you
get.
If
you
run
a
probe,
you
can
do
measurements
there.
C
C
But
people
at
least
our
target
audience
were
people
who
didn't
really
want
to
spend
the
time
on
figuring
out
how
to
do
a
measurement
and
then
how
to
interpret
the
results.
What
they
want
is
does
it
work
or
doesn't
it
and
if
it
doesn't,
who
do
I
talk
to
and
I
have
to
say
that's
surprisingly
difficult
to
to
achieve.
D
All
right,
so
my
name
is
Victor.
Coursing
I'll
give
you
a
bit
of
my
background
and
I'll
share
some
opinions.
Currently
working
I'll
go
backwards,
I
guess
at
Oracle,
as
you
mentioned,
we
just
got
acquired.
Our
company
was
dying,
so
we
were
very
much
on
the
cloud
services
side
of
the
equation
at
this
point
prior
to
that
smallest
in
an
Cisco
and
then
before
that
about
13
years
in
a
service
provider
up
in
Canada,
so
doing
Wireless
and
cable
operations
and
business
operations.
D
So
I've
seen
the
issue
from
multiple
perspectives,
and
that
includes
seeing
it
from
the
content.
/
service
provider.
You
know
cloud
provider
side
of
it
you
trying
to
offer
what
is
an
increasingly
common
phenomenon
of
a
third-party
network
offering
services
into
SP
networks
and
that
dynamic
of
where,
as
where
does
the
service
issues
lie,
seeing
it
from
a
vendor
perspective
is
how
the
tool
our
equipment
correctly,
so
that
SPS
can
leverage
or
turn
on
the
right
functions
to
be
able
to
metric
your
network
or
take
statistics
out
of
it
and
figure
out.
D
What's
going
on
and
from
the
SP
side
of
the
house,
which
was
think
back
to
the
late
nineties
and
early
2000s,
when
we
did
this
a
lot
of
the
functions
that
we
require
to
understand
that
the
health
these
are
networks
were
quite
segmented
back
then
we
inherently
had
that
was
part
and
parcel
of
that
vertical
integration.
We
had
way
back
then,
and
what's
happened,
is
as
we've
kind
of
made,
this
more
ubiquitous
network
and
data
centers
and
things
have
become
more
common
on
a
on
a
single
fabric.
D
What's
the
issue
we
have
is
we
haven't
replaced
all
of
those
functionalities
with
a
common
set
of
tools
and
metrics
to
be
able
to
understand?
How
does
our
service
now
run
now?
How
do
we
deal
with
an
infrastructure
that
has
many
services
on
it
and
how
do
we
understand
I
know
you
want
to
get
to
the
applications
later,
but
how
we're
going
to
understand,
or
what's
the
right
type
of
data
together
here
to
understand
what
the
implications
will
be
for
the
upstream
applications?
So
that's
that's
kind
of
my
interest
area.
D
You
know
spending
a
lot
of
time.
You
know
in
terms
of
architecture,
design
and
operating
all
these
systems
from
these
different
perspectives
and
I
see
it
as
an
increasing
challenge,
because
I
see
that
not
only
inside
the
SP
or
their
applications
of
their
services
moving
to
these
common
data
centers
and
how
do
they?
How
do
they?
How
do
they
figure
out
what's
actually
happening
within
the
network
as
we
struggle
to
that?
You
know
if
you
go
back,
you've
been
starting
about
five
six
years
ago,
we
struggled
with
how
do
we
convert
all
of
these?
D
What
used
to
be
these
inherent
tools?
We
got
from
the
vendors
to
what
do
we
create
to
be
able
to
understand
that
and
a
lot
of
the
things
happening
with
the
internet
and
then
from
the
service
providers?
Perspective
I
mean
sort
of
the
content
provider
spective
or
you
know
the
upstream
cloud
providers
perspective
is:
how
do
we
access
tools
or
how
do
we
provide
functionality,
cooperatively
with
the
ESPE?
So
we
understand
the
health.
So
we
don't
get
this
finger-pointing
or
we
don't.
You
know
where
does
the
problem
lie?
Is
that
in
the
access
network?
D
Is
it
in
this?
These
increasingly
complex
backbone
networks,
or
is
it
in
the
interconnection
layer
between
all
the
providers
or
is
it
inside
the
cloud
providers
infrastructure?
And
how
do
we
tool
these
two
to
get
meaning
out
of
them
in
a
common
way?
So
we
can
I
think
they
fix
problems
in
in
really
understanding
health
and
attributes
of
applications.
So
that's
my
first
synopsis.
E
First
few
years
of
the
company
were
a
lot
of
you
know:
thankless
work
to
upgrade
the
infrastructure
right
infrastructure
that
actually
had
no
visible
benefits
or
movement
from
the
public's
point
of
view,
upgrading
servers,
operating
systems,
products
what-have-you
so
about
seven
years
ago
six
years
ago,
is
when
we
started
to
really
launch
things
that
we're
gonna
end
up
in
our
customers.
Homes.
E
Around
that
same
time,
when
we
were
getting
ready
to
do
that,
yeah
we
talked
to
it.
You
know
got
all
the
right
provable
internally.
One
of
the
one
of
the
kind
of
the
requests
air
quotes
was
don't
break
him
thing,
got
it
don't
break
anything
right
so
so
that
ended
up
kind
of
blossom
me
into
one
ended
up
being
an
opportunity
for
us
to
kind
of
measure
ourselves
right.
Initially,
it
was
from
a
binary
point
of
you
to
make
sure
that
we
didn't
break
anything
right
to
follow
our
you
know.
E
E
It's
you
know
when
we,
when
we
deployed
me
six
as
part
of
kind
of
enabling
it
on
our
infrastructure,
we
did
it
in
a
way
that
we
didn't
just
simply
carry
forward
the
things
that
were
done
in
before
we
looked
at
different
that
you
know,
v6
is
an
opportunity
to
do
things
differently,
better.
You
know
optimizers
if
there's
things
in
baggage.
That
was
happening
that
was
in
place
before
we
wanted
to
like.
E
Take
that
as
an
opportunity
to
perhaps
not
do
that
again,
but
from
a
v6
point
of
view
that
in
turn
led
to
a
series
of
interesting
phases
where
we
said
hey
well,
let's
just
take
this
from
not
a
binary
approach
to
measuring.
Oh
you
know,
you
know,
are
we
broken
or
not?
Hey
wait
a
minute
now,
what's
the
difference
between
v4
and
v6,
oh
and
by
the
way,
in
many
cases,
v6
outperforms
v4
one
of
those
scenarios.
E
What
can
we
explore
there
and
then
finally,
yeah
I
think
the
chapter
that
is
about
to
be
upon
us,
maybe
in
some
places
is
upon
us
is
what
happens
when
before
starts
to
degree
right,
you
can't
go
back
and
measure
right.
You
can't
go
back
in
time
right.
So
you
know
starting
these
measurements
and
letting
them
run
for
extended
periods
of
time
is
critical
right
and
at
least
having
that
you
know,
foundational
baseline
data
is
essentially
you
should
never
go
back
in
time
to
reget
it
right.
So
t0
you
can't
go
back
to
right.
E
B
Thanks
so
I
sent
you
all
some
questions
that
we
had
to
and
let
me
now
start
with
one
that
wasn't
among
those
questions
by
just
sort
of
to
to
put
a
little
bit
of
a
finer
point
on
it,
and
at
least
two
of
you
brought
up
the
notion
of
you
know
doing
measurements
can
can
help.
You
see
where
there
are
actually
problems
in
the
network.
I
wonder
if
one
one
or
more
of
you
would
like
to
give
a
concrete
example
to
the
room
of
where
simple
measurements
can.
E
Have
two
one
that
was
a
big
bet:
it
dates
back
to
world
we
six
launched
was
that
2012
right
yeah
and
when
that's
happening
right
now
right.
So
when
a
good
friend
of
mine
Paul
saw
the
Facebook
right,
so
I
was
visiting
Paul
and
his
office
and
at
the
time
I
guess
was
Palo
Alto
and
he's
like
hey
it's.
So
it's
a
month
before
the
event
because
hey
let's,
let's
turn
b61
and
comp
case
nowhere.
I
said:
okay,
let's.
E
Said:
listen,
let's,
let's
you
know
refill
the
beer
first
and
let's
give
it
again:
let's
get!
Let's
give
it
a
go
right
so
and
I
said
hold
on
a
second
I
said.
Let
me
let
me
plug
in
you
know,
dub-dub-dub,
that
peaceful
calm
into
kind
of
like
the
system
right
talk
about
what
the
system
is
later
and
he
turns
it
on.
We
go
back
at
him
by
T,
come
back
and
I
said
said
dude
you
might
want
to
turn
that
off.
E
So
it
turns
out
in
that
case-
and
this
is
exactly
the
kind
of
relations
that
I'm
describing
that
blossom
yeah-
that
really
helps
the
internet
ecosystem
get
together
and
fix
things
that
affect
millions
of
people
on
the
Internet
turns
out.
You
know
the
v4
traffic
was
being
served
at
is
something
relatively
close
to
Chicago
and
then
Chicago,
but
v6
was
being
served
someplace
and
like
the
northwest
of
the
countries
I'm
like
yeah,
that's
not
horrible,
it's
not
broken,
but
it's
not.
You
know
the
best,
so
turned
it
off.
E
F
E
And
a
CDN
where
I
happen
to
be
measuring,
you
know,
so
these
relationships
have
kind
of
teamed
to
blossom
the
CDN.
The
condom
owner
is
basically
LinkedIn
right
and
then
I
and
I
get
a
bunch
of
requests
from
them.
That
says,
hey
you
know.
What's
it
look
like
when
your
network
how's
things
going
because
we're
getting
some
reports
of
some
some
weird,
you
know
issues
right,
so
you
plug
in
you,
don't
LinkedIn
URL
and
then
collect
some
data
for
a
while,
and
what
we're
finding
is
is
that
there
are.
E
There
are
issues
both
on
my
side
and
on
you
know
another
third-party
side,
where
it's
potentially
affecting
you
know
kind
of
the
customers
again
not
not
broken
but,
like
you
know,
again,
opt
opportunities
for
optimization
right
and
the
measurements
Leslie.
You
know,
and
if
you
want
to
get
into
this
later,
that'd
be
great
they're,
not
sophisticated.
You
know
it's
basically
a
wrapper
built
around
Lib
curl
right.
That,
basically,
does
you
know:
DNS
lookup,
TCP,
connect
and
download
time
very
simple
for
v4
v6,
which.
D
Grab
on
to
examples,
one
will
actually
reiterate
something
that
John
just
said
and
then
I'll
jump
into
a
second
one
from
a
different
perspective.
So
the
same
event:
different
network
ipv6,
Day
2012,
when
we
were
amping
up
to
it.
We
saw
a
very
similar
issue
and
on
the
dns
to
go
into
the
details.
What
we
found
on
our
SP
side
network
was
that
you
know
one
can
argue,
just
came
down
to
design
Delta's.
This
will
continue
to
be
true
between
the
46.
D
D
You
know
what's
going
on
here:
why
is
this
happening?
Wasn't
instrumented,
you
know
we
find
discovered.
Well,
you
know,
there's
issues
within
the
network
and
how
the
config
was
done
and
we
found
out
was
a
lot
of
manual
processes
on
the
v6
side
was
quite
new
back
then.
So
you
know
that's
just
to
reiterate
one
of
the
things
that
John
said
that
we
found
that
so
it's
more.
D
The
same
or
similar
kinds
of
issues
we
were
ramped
up
there
and
it
was
one
of
those
Wow.
You
know
this
is
a
this
condition
actually
happen
and
we've
had
other
examples.
Another
perspective
is
I,
find
implementations
are
different,
differed
between
vendors
I'm,
not
going
to
give
away
any
vendor
quip.
D
In
this
point,
but
one
of
the
things
we
found
more
recently
was
we
had
an
issue
where
before
actually
kind
of
sucked-
and
we
didn't
have
a
v6
problem
with
this
specific
traffic
flow,
when
you
were
at
frame
rail,
you
know
was
what's
going
on
our
UDP
traffic,
it
looks
like
it's
being
dropped
from
the
floor
or
going
into
this
data
center
what's
happening.
Well,
we
realize
now.
This
might
be
that
the
v6
implementation
of
a
ologies
was
broken.
A
D
It
sure
performed
very
differently
between
v4
and
v6,
so
we
got
a
totally
fundamental
that
from
behavior
between
v4
and
v6,
in
terms
of
how
UDP
traffic
was
being
managed,
and
this
there
was
some
asynchronous
nature
going
on.
Without
going
to
the
big
details
now,
had
we
been
able
to
instrument
network,
we
could
have
found
this
problem
a
lot
sooner
than
then,
by
the
way,
SPS
will
buy
the
exact
same
equipment.
We
were
using
instances
we
could
have
found.
This
problem
allows
sooner
not
waited
for.
D
You
know
incredible
number
of
tickets,
a
lot
of
problems,
a
lot
of
issues,
a
lot
of
trolling,
a
lot
of
man-hours-
and
you
know
I'd
rather
spend
that
be
honest,
you
doing
actual
network
implementation
or
as
opposed
to
just
troubleshooting
troubleshooting
troubleshooting.
So
it
took
a
long
time
to
find
this,
so
we're
seeing
just
implementation
issues
are
hard
to
find
so
instrumenting
things
can
help
us
find
things
in
this
particular
case.
You
know
before
actually
sucked
more
than
v6
I.
C
Am
constantly
surprised
about
how
much
value
researchers
can
squeeze
out
from
the
same
data
set
they
today
and
that
that
just
it's
just
it's
just
the
thing
that
we
are
still
using
the
tools
that
we've
been
using
40
years
ago,
literally
ping
trace
route,
DNS
lookup
and
the
volume
has
changed
a
lot
and
I.
Think
in
terms
of
you
know,
detecting
problems
with
v4
with
v6,
which
one's
better.
What
has
changed,
having
a
significant
amount
of
constant
data
inflow
helps
a
lot.
C
We
are
measuring,
something
like
4,500
results,
a
second
that's
what
we
collect.
So
400
million
data
points
a
day
which
is
more
than
enough
to
pinpoint
problems
when
they
occur.
The
trick
is
one
of
the
tricks.
Is
that
it's
nice
to
to
collect
baseline
there?
So
you
know
referring
back
to
previous
examples.
If
you
knew
how
the
network
behaved
before
the
problem
came,
and
now
you
measure
how
it
is
behaving,
then
you
can
compare
and
say
that's
the
problem
right
there
once
you
have
a
problem
and
then
you
start
to
measure.
C
B
C
Think
that's
really
hard,
especially
if
you,
if
you
look
at
the
fact
that
the
technology
that
the
end
users
interact
with
changes
so
quickly
quickly.
Even
so,
it's
really
difficult
to
have
measurement
points
inside
your
network
or
inside
your
customers
network
that
try
to
simulate
what
the
users
will
see.
Basically,
you
have
to
re-implement
the
heavy
eyeballs,
as
those
guys
did
it
over
there.
But
if
your
population
happens
to
prefer
a
different
browser,
then
their
experience
will
be
different.
So
in
that
sense
the
only
real
user
specific
metric
like
well.
C
How
do
the
users
feel
about
this
change
would
be
the
actual
end,
client
that
the
user
is
using
and
that's
kind
of
difficult
to
get
to
and
limited
people
try
that
and
some
some
plugins
still
work,
but
I
do
feel
that
on
the
network
level,
we
can
do
a
lot
of
measurements
and
a
lot
of
guesswork
and
some
of
that
guesswork
is
pretty
accurate.
But
it's
never
going
to
be
the
same
that
the
user
experiences.
So
that's,
it
might
be
a
personal
opinion,
but
I
think
that's
difficult.
E
Yeah,
so
it's
fascinating
even
the
way
that
we've
done
some
of
the
measurements
that
we
were.
We
were
doing
for
a
while.
Now
you
know
we
can.
We
can
kind
of
initiate
many
of
these
and
that's
frankly,
why
we
chose
HTTP,
because
HTTP
vertically,
when
you
include
all
the
protocols
that
go
into
kind
of
an
HTP
exchange,
it's
like
it's,
it's
pretty
good
place
to
start
I
mean.
Is
it
perfect?
Is
it'll
come
saying?
No,
but
it's
it's
yeah.
It's
got
a
like
a
very
long.
E
Runway
of
things
to
factor
in
I
mean
there's,
there's
plenty
of
work
that
can
be
done
just
on
that
vertical
space
alone
and
what
we've
also
noticed
in
our
in
error
in
our
network.
You
know
ours
is
pretty
big
nationwide
network.
So
we
have
these
regions,
you
know
Boston
Philadelphia,
Atlanta,
Chicago
Denver.
You
know
San
Jose,
San
Francisco.
E
We
have
probes
that
are
kind
of
generally
in
each
of
those
regions
right
one
of
the
things
that
we
think
that
could
you
know
really
help
to
augment
that
is
kind
of
pushing
it
closer
to
where
the
customer
resides
topologically
in
a
network
right
and
to
Robert's
point.
Even
then,
it's
not
exactly
perfect,
but
I
mean
it's.
It
still
represents
a
kind
of
cornucopia
of
data
opportunities
to
really
analyze
and
evaluate
what
that
experience
was
like.
Ultimately,
after
they
is
done,
it's
really
something
like
this
right.
E
E
Exactly
the
the
issue
that
we
had
with
with
LinkedIn
recently
right
somehow
with
you
know
the
the
anycast
DNS
servers
that
we
said
to
the
customers,
the
cluster
in
a
particular
region
was
not
go
was
not
enabling
the
proper
geo
and
people
in
a
certain
part
country
would
go
and
just
accept
the
work
going
completely
somewhere
else.
So
that
could
be
a
byproduct
of
you
know
routing
policy
of
the
Comcast
network.
You
know
something
with
the
DNS
infrastructure.
E
D
I
felt
like
I,
wanted
to
say
something,
and
this
was
kind
of
lines,
and
this
was
more
like
I
raise
a
question
we
were
talking
about
in
the
background.
Thinking
back
to
the
content.
Imagine
you
were
watching
some
kind
of
streaming
content
from
a
very
popular
provider
at
home.
You
know
one
of
the
things
that
I
grappled
with
is:
how
do
we
understand
what
the
quality
is,
that
you
do?
I
understand
the
infrastructure
side.
D
That's
is
if
we
understand
what
we
need
to
instrument
to
kind
of
get
a
a
course
view
of
how
the
infrastructure
is
actually
working.
So
you
know
John
and
I
could
be
in
our
relative
homes.
We
could
both
be
in
Philadelphia
and
I
moved
there,
and
if
we're
both
watching
this
content,
you
know
this
content.
You
know
suddenly,
for
some
reason,
there's
a
a
red
drop
somewhere
in
the
network.
He
drops
down
and
bandwidth
for
his
stream.
You
know
so
he's
no
longer
getting
his
4k
or
HD
signal
on
that
on
that
stream.
D
D
Probably
not
an
issue
in
the
content
side
may
be
somewhere
in
the
middle
of
his
network.
Maybe
it
was
on
the
access
network
where
that
happened,
but
the
question
I
have
is
I.
Don't
you
know
is
that
good?
That
means
that
there
was
a
problem
at
work.
Probably
at
that
macro
scale.
There
really
was
no
issues
there,
but
his
experience
was
that
there
was
an
issue
he
he
no
longer
saw
his.
You
know
high-definition
video
come
through
on
a
streaming
service.
B
B
Sounds
like
some
homework
where
I
was
gonna
go
and
I
will
go
since
I
I'm,
not
gonna
Google.
That
right
this
minute,
it
kind
of
sounds
like
what's
being
described
here.
Is
that,
from
the
perspective
we're
talking
about
today,
it's
not
really
particularly
possible
to
simulate
or
measure
the
exact
user
experience.
Because
of
all
the
reasons
that
you
raised,
Robert
for
instance,
but
we
might
be
able
to
start
getting
to
the
question
of
how
well
is,
is
the
network
for
some
value
of
the
network
serving
and
customers.
B
B
All
right
so
as
we're
sliding
downhill
towards
a
sense
of
despair
and
a
morass
of
negativity.
So
how
about
the
health
of
the
Internet?
Is
there
any
way
in
which
we
can
I'm
gonna
be
positive?
We
can
talk
about
health
metrics,
not
from
the
standpoint
of
being
able
to
answer
all
questions,
but
at
least
from
the
standpoint
as
John
was
articulating
it
creating
baselines.
B
Let
me
let's
make
this
T
zero
and
start
measuring
some
things,
some
pretty
basic
things,
so
that
when,
when
we
subjectively
decide
that
things
are
not
working
well
in
the
internet,
we
can
turn
to
some
objective,
metrics
and
say
this
is
what
has
changed
or
this
is
what
is
different
in
this
network
versus
that
network
over
there,
and
if
we
were
gonna.
Do
that
sort
of
thing?
What
might
be
some
of
those
things
that
we
could
measure
and
find
useful.
E
E
Over
the
past
couple
of
several
ITF,
so
he
said
hey,
you
know
we
think
there's
something
up
with
kind
of
DNS.
The
DNS
in
general
on
the
air
quotes
capital.
I
Internet,
we're
like
you
know
it
seems
to
be-
and
this
is
a
few
years
ago,
seems
to
be
like
just
you
know,
grossly
biased
towards
I
could
be
forward.
No,
no!
You
know
that
can't
be
the
case
you're
like
yeah.
E
We
think
it
is
so
we
did
some
did
some
digging
and
obviously,
if
you
don't
have
a
good
path
to
you
know
recursion
over
v6-
and
you
know
the
DNS
will
do
the
right
thing.
It'll
use
before
turns
out.
There's
there's
a
lot
more
than
that,
even
internally
on
our
network,
our
recursive
name
servers
we're
not.
There
was
a
bug,
plus
it
was
not
configured
as
such
to
prefer
recursion
over
v6.
First
wait.
You
know
I
think
where
I
were
described.
E
Happy
eyeballs,
maybe
was
Victor,
so
we
ran
some
tests
and
we
found
out
that
you
know
recursion
DNS
recursion
over
v6
was
actually
faster
than
IP
v4.
There's
it
there
ten
cases.
The
Comcast
network,
two
Asian,
authoritative,
DNS
servers
that
were
pretty
bad,
but
you
know
I
can
pick
up
the
phone,
make
a
few
phone
calls
to
say:
hey,
you
know
something
at
dot,
I
in
or
something
dot
or
whatever
you
got
it.
You
gotta
fix
this
thing
up
right,
but
the
vast
majority
of
it
was
like
v6
was
outperforming
before
right.
E
B
E
D
Just
to
kind
of
jump
on
one
of
the
things
that
you're
saying
I
think
DNS
now
I
have
to
remember.
There's
some
bias
here
at
this
point,
because
I
have
to
be
working
for
coming
in
makes
money
serving
the
DNS
so
trying
not
to
take
off
the
bias
head.
Try
and
take
off
the
bias
I
cannot
have.
A
bias
is
I
think
that's
a
really
good
space,
because
it's
it's
a
fundamental
part
of
the
experience
of
a
lot
of
usage
of
the
internet
right
so
you're,
not
often
the
dark
web
somewhere.
D
So
it's
it's
a
fundamental
part
of
that
experience
right.
A
lot
of
a
lot
of
what
the
customer
will
see
has
to
do
with.
You
know
DNS
performance,
and
it
also
to
make
that
work
right.
A
lot
of
infrastructural
things
have
to
be
right
or
wrong.
Now
we
get
escalated
on
quite
a
bit
because
you
know,
if
you're
hosting
some
big-name
brands.
What
happens?
Is
you
get
a
quick
aggregate
of
problems
and
you
notice
it
pretty
fast
and
we've
had
to
troubleshoot
with
other
with
other
providers.
D
You
know
anomalies
in
their
v4
and
v6
routing
a
lot
in
the
visa
product
face
you'd,
be
amazed
at
how
reformed
be
sick
to
perform.
Fundamentally,
if
in
terms
of
how
people
happen
to
route
that
across
their
networks-
and
so
you
know,
as
we've
had
to
fix
a
lot
of
infrastructure
things
that
seem
to
have
nothing
to
do
with
Dean
s,
you
know,
like
your
red,
you
know
your
PGP,
you
know
you
say
no
export,
you
do
an
you,
do
an
export
or
you
know
fundamental
things.
D
D
B
So
I'm
gonna
open
up
the
floor
to
questions
from
the
audience
and,
while
you're,
organizing
yourselves
towards
the
mic
and
I'll,
throw
out
one
question
to
you:
Robert
with
the
Alice
infrastructure
or
the
measurements
infrastructure
in
general.
Have
you
have
you
noticed
differences
over
the
last
few
years?
Shall
we
say
of
how
who
is
using
the
infrastructure
and
how
they're
using
what
kinds
of
questions
they
seem
to
be
trying
to
answer,
or
is
that
not
something
that
you
have
any
can
get
any
insight
into.
C
C
So
so
who
are
using
right
but
less
than
Y
and
what
kind
of
behavior
they
they
show
and
they
identified
I'm
not
alternative
to
talk
about
the
paper,
but
they
identified
two
classes
of
people,
one
who
really
know
what
they
are
measuring.
So
they
have
a
specific
problem,
and
now
they
are
shooting
out
a
bunch
of
measurement
requests
and
then
trying
to
parse
the
results
and
figuring
out
what's
going
on,
and
then
the
other
class
of
people
is
I'm
trying
to
stay
away
from
their
terminology.
C
C
So
so
so
those
still
exist.
Our
original
goal
was
to
address
operators
because
we
thought
that
they
need
this
tool
and
I'm
very
happy
to
report
that,
like
two
or
three
years
after
we
started
and
we
had
to
push
a
bit
at
the
beginning,
but
two
or
three
years
after
we
started
seeing
talks
appearing
from
operators
at
various
conferences.
C
Besides,
this
I
did
see
a
huge
take
up
in
researchers
using
FS
and
I'm,
very
proud
of
that,
and
it's
just
one
thing
that
researchers
will
try
to
get
their
hands
on
anything
they
can
when
they
think
that
there
is
value
to
be
extracted
there,
but
I
think
there
is
more
and
more
and
more
conscious
users
of
Atlas,
because
there
are
specific
questions
that
the
researchers
feel
that
this
platform
may
not
be
the
best.
But
at
least
it's
there
it's
usable
and
it
can
provide
insight
and
that's
useful
for
for
the
operator
community
as
well.
C
E
Like
one
of
the
things
that
you
said,
you
know
measuring
everything.
So
when
you
have
a
generic
routine
that
you
just
like
plug
in
a
URL
to
test
it's
easy
and
tempting
to
like
want
to
measure
everything's
right
to
Robert's
point,
but
man
I
got
to
tell
you
when
you
do
that
and
take
it
out.
Take
it
from
me.
E
You
know,
because
we
got
there
at
one
point:
the
the
volume
of
data
that
you
end
up
having
to
kind
of
sift
through
it's
not
only
is
it
distracting,
but
it's
almost
becomes
an
impossibility
honestly
so
like
there
is
so
that
there's
some
of
the
measurements
that
we
ended
up.
You
know
that
we're
still
doing
you
know
we
ended
up
re
configuring
them
to
be
very
explicitly
targeted
towards
like
internal
endpoints,
so
that
we
kind
of
get
back
to
the
basics.
But
there
is,
there
is
a
dangerous
side
effect
alike.
This
temptation
of
measuring
everything.
F
D
Talk
a
little
bit
about
any
casting
that
we
use
a
lot
of
it.
Yeah
I
mean
I,
think
getting
tooling
around
and
how
any
cast
because
the
Trish
you
know
traditional
traditional
networking,
you're,
not
gonna,
necessarily
understand
how
in
any
cast
excretion
works
very
well.
You
obviously
can
test
for
multiple
points
and
etc.
The
other
thing
that's
unique,
unique
about
any
cast
depending
if
it's
inside
a
network.
It
behaves
one
way
when
it
behaves
across
networks.
It
behaves
totally
different
again.
D
So
when
you're
doing
you
know
between
between
networks
and
that
behavior
has
to
do
with
while
you're
it's
the
you
know
the
political
layer
of
how
we
route
traffic
so
things
that
you
think
we're
going
to
happen,
don't
actually
happen
and
it's
a
fundamental
thing.
We
grapple
with
every
single
day
and
that
we're
one
of
those
networks
that
have
to
go
between
networks.
So
what
we
did
with
it,
there's
not
what
the
next
provider
will
do
it
with
it
and
it's
it's
a
major
issue.
D
The
other
thing,
too,
is
you
know
without
giving
away
the
farm
people
who
run
you
know
in
our
inner
domain,
anycast
networks
do
things
to
specifically
try
to
circumvent
certain
behaviors
like
you,
don't
want
to
block
whole
traffic,
so
you
don't
distribute
things
in
you.
Geez.
Do
you
have
very
unique
distributive
distribution
patterns
to
make
sure
you
get
certain
key
trying
to
block
against
black
holing,
but
you
also
want
to
try
to
get
performance,
so
you
have
to
then
understand.
Not
all
things
are
equal.
D
So
if
you
have
a
provider
it
has,
you
know,
a
number
of
any
class
blocks
doesn't
mean
you
should
expect
to
see
the
same
thing
from
each
one
of
them.
The
answer
is
you
shouldn't
expect
to
see
the
same
thing
and
if
you
did
well,
they
might
be
they
might
be
in
trouble
if
they
have
some
issue
on
their
data
centers.
D
So
that
I
think
that's
actually
a
very
interesting
question,
but
I
think
it'd
make
me
different
for
inside
and
SP
and
across
in
their
domains,
because
there's
different
behavior
you
see
and
that
political
layer
issue
is
that
is
everyone's
entitled
to
route
traffic.
The
way
they
want
and
they
sure
as
heck
do
it
differently
at
times,
and
this.
E
Idea
is
my
friend,
so
so
the
any
cases
we
use
a
lot.
You
know
it's
it's
you
know
all
of
our
DNS
servers
have
any
case
addresses
and
it's
an
area
though
it
does
add
a
level
of
complexity.
To
be
honest
with
you,
but
like
it's
I
think
the
tools
need
to
evolve.
One
of
the
things
that
I
did
over
the
pit.
E
This
past
holiday
is
I,
so
some
of
the
measurement
stuff
that
we
were
talking
about
here
I
updated
a
lot
of
that
code
to
basically
capture
and
inside
the
information
right
so
like
when
I'm
working
with
you
know,
folks
that
are
at
your
date.
You
know
a
dollar
sign
day.
Job
right,
I
have
sufficient
level
of
information,
but
even
with
that,
you
had,
like
you
know,
labels
that
come
in
the
form
of
anxiety,
a
not
available
everywhere
and
be
it
still
is
only
like.
E
A
D
So
I
just
want
to
add
one
thing:
I
first
mentioned
earlier
and
something
that
John
said
reminded
me
of
it.
The
other
thing
about
anycast
depend
I,
think
John
says
something
very
important
which
is
get
as
much
data
as
soon
as
you
can,
because
one
of
the
things
that
changes
as
where
you
go
change
over
time,
there's
one
very
large
value
that
changes
latency,
suddenly
changes
drastically
on
you
potentially,
and
that
could
be
by
design
and
to
understand
you
know
what
the
limits
are,
because
suddenly
something
that
looked
close
as
far,
but
is
that
good?
D
C
Line
is
getting
longer
so
I'm
going
to
keep
it
short
from
ours
like
different
measurement
network
point-of-view,
anycast
versus
unicast
is
like
v4
versus
v6.
We
just
do
it,
and
so
it
doesn't
matter
for
us
which
one
it
is.
That
said,
that's
true
for
the
most
part,
but
some
consequences
we
have
to
deal
with.
For
example,
we
can,
when
you
want
to
visualize
stuff,
then
you
cannot
claim
that
it
is
in
a
single
place.
C
B
J
Al
Morton
I've
got
a
broad
view
of
network
health
and
I
want
to
try
to
broaden
it
for
everyone
its
hardest
to
keep
track
of
when
everything
is
green,
that
we've
been
given
to
monitor
our
networks.
So
what
that
usually
means
is
the
problem
if
it
exists,
is
outside
of
our
operator.
Control
John
touched
on
this
with
the
DNS
problem
and
to
go
and
get
some
help
from
folks
outside
right,
and
it's
still
going
to
be
the
same
thing
with
our
new
partners,
the
content
distribution
networks.
J
J
Cd
and
the
there's
many
media
players
there's
all
sorts
of
things
that
they
could
tell
us
about
stalling,
which
is
the
most
heinous
thing
that
can
happen
apparently,
and
but
we
all
put
up
with
it
and
and
there's
actually
some
CTA
work
in
the
u.s.
going
on
to
try
to
instrument
that
and
get
that
get
that
part
of
the
piece
shared
with
all
of
us,
but
so
I'm
wondering
you
know.
Let
me
let
me
just
turn
it
over
now
to
the
nails.
What
were
you
guys
doesn't
see?
It
CD
ends
I'm.
C
E
Call
co-worker
barons,
Aaron,
Nygren,
he's
probably
not
here
so
so
that
I
mean
you're
right
men
I
mean
it's
true
statement
right
but
like,
and
that's
one
of
the
things
I
mean
a
lot
of
the
work
that
I've
done
over
the
you
know.
Personally
over
the
past,
you
know
ten
plus
years.
You
know
Comcast
not
only.
A
E
If
I
build
it's
a
great
relationship,
a
lot
of
a
lot
of
those
relationships
are
not
being
really
great
friends
right.
You
know
people
I
hold
in
high
regard
right
so
like
what
ends
up
happening
is
it's
you
know
I,
let
my
fingers
do
the
walking
right
and
we
see
a
problem.
I
think
of
a
phone
call
I
pick
up
the
phone
and
make
a
call,
but
that's
my
point
about
about
what
I
was
saying
that
I
think
that
Aaron
and
I
would
go
back
and
forth.
That
works
great.
E
G
G
I
So
I
work
for
inside
the
Animus
to
dine
in
ale
TLD
operator,
and
we
had
a
person
who
dressed
many
casimir
cause
that's
what
we
did
to
on
and
I
we
do
it.
We
have
medium
bunch
of
work
in
that
Jenna's.
Optimization
got
a
bunch
of
papers
out
there,
working
from
USC,
yes,
I
am
but
I
would
what
I
want
to
tag
on
any
cast?
Is
it
like?
I
If
you
try
to
optimize
Meo
client
I
may
try
what's
gonna
happen,
make
other
times
gonna
be
shift
around
a
different
data
centers
and
then
they
suffer
so
they
problem
just
gets
much
complex,
so
I'm
trying
to
say
any
classes.
That's
what
I've
seen
we
were
looking
in
the
data
where
sacred
it's
way
more
complex,
that
is.
E
Yeah
and
the
other
thing,
depending
on
the
size
you
network,
and
how
you're
you
know,
configured
the
you
you
may
have
it
set
up
the
right
way
on
your
end
right,
you
may
have
an
intermediate
network
that
makes
the
make
some
decisions
that
end
up
having
implications
for
right.
It's
kind
of
makes
the
argument
for
some
of
these.
You
know
background
active
measurements
that
are
always
going
out
and
like
settings,
I
mean
I,
think
you're
kind
of
reiterating
some
yeah,
so
you're
gonna
add
what.
D
I
first
know
that
yeah
and
maybe
you
know,
Robert
or
John,
Marin
I
agree.
The
other
thing
too,
is
this:
there's
the
I.
What
I
mention
earlier,
which
is
you
know
you
have
things
happen
other
people's
that
week
you
can't
control,
but
even
down
to
the
fundamental
SP
networks.
Their
networks
operate
a
lot
time
differently.
His
decision,
like
egress
and
ingress
in
an
SP
network
typically
is
not
aligned
right.
It's
his
asymmetric
and
that's
maybe
there's
some
networks
that
are
symmetric,
maybe
have
one
transit
peer
and
that
is
pretty
ease
of
it.
D
A
lot
especially
have
a
lot
of
peering
or
even
number
of
transfer
riders.
You
know
where,
where
what
how
you
decide
where
you
want
to
send
your
traffic
typically
based
on
some
economic
reason,
doesn't
match
how
it
comes
back
to
you,
because
someone
else
decides
for
their
economic
reason.
They're
gonna
send
trough
in
a
different
way,
and
so
now.
A
D
Different
over
time
as
well,
so
what
network
use
the
hood,
transitory
networks
you
may
go
through
is
not
always
constant,
so
it
doesn't
always
go
from
A
to
B
the
in
go
A
to
C,
and
that
would
be
that's
a
version
of
normal
as
well
so
again,
just
to
reiterate
I
think
any
anycast
provides
it
an
interesting
nature
which
I
would
like
to
see
attacked.
Yeah.
I
A
C
That,
if
you
add
the
new
anycast
instance,
then
things
will
get
better
right,
so
you
know
intuitively.
You
would
say:
oh
now,
I'm
going
to
be
closer
to
more
clients,
in
fact,
yeah,
maybe
but
you're
attracting
traffic
onto
already
saturated
links,
because
VDP
doesn't
really
care
about
saturation
of
links
and
all
that,
so
you
know
we
had
cases
where
we
had
to
shoot
down
a
cable
instance,
because
it
made
things
worse
and
that's
actually
an
interesting
research
area.
If
anyone's
interested
I
have
lots
of
data.
C
I
C
C
H
K
I
listened
to
it
also
started
working
with
for
Nikki,
so
I've
got
two
questions,
or
maybe
just
a
bit
of
background
to
that.
So
my
interest
has
been
measuring
Internet
performance
in
Africa,
mostly
and
so
working
in
the
region
way.
God
you
don't
have
that
many
vantage
points
for
four
minutes.
Sometimes
you'd
want
to
use
various
platforms,
so
I've
used
wipe
a
truss
and
use
Qaeda
and
recently
has
used
speed
checker.
So
one
of
the
frustration
sometimes
is
when
you're
trying
to
make
these
different
data
sets.
K
You
gotta
fill
formats
and
you've
got
to
code
number
of
steps
to
try
and
bring
these
datasets
together.
So
my
question
would
be
any
efforts
or
do
you
think,
there's
an
importance
to
trying
to
come
to
a
point.
We've
got
sort
of
a
standardized
or
founded
for
Reforma,
so
you
can
have
some
these
different
art
forms
in
general.
C
C
When
I
first
saw
that
l
map
started
up-
and
you
know
there
is
going
to
be
standardization
in
this
space
and
that's
awesome-
I-
don't
think
that
the
actual
measurement
results
are
standardized
yet
maybe
there
is
more
work
to
do
yet
there.
What
I
can
say
from
from
our
perspective
is
that,
if
that,
if
there
is
a
standard
that
says
this
is
how
you
should
disseminate
your
treasure
on
your
DNS,
your
pink
results.
I
would
be
happy
to
provide
translations
from
what
we
have
to
the
standard
thing.
That's
not
a
problem.
E
Yeah
so
I
think
as
much
as
it
may
be
technically
true
about
the
wire.
It's
not
practical,
though
right,
Leslie
and
I
have
been
talking
about
this.
For
a
while,
I
mean
I.
Think
the
idea
would
be
perfect
if
there
was
like
a
JSON
template
that
says:
hey
if
you
wish
to
publish
your
data
and
you
want
to
self
publish
you
know,
here's
the
format,
send
it
here.
Right
and
yeah.
Some
organization
will
gladly
digested
and
help
summarize
it
for
you
right,
I,
don't
think
we're
there.
Yet
I
think
you
know.
E
E
So,
but,
but
you
know
it,
doesn't
it
shouldn't
stay
that
way
right?
There
should
be
a
way
for
us
to
kind
of
generically
represent
these
things
so
that
you
can
you
can
just
you
know
you
can
pass
it
around.
The
people
I
mean
I'd
like
to
be
able
to
work
with.
You
know
with
Aaron
I
said:
hey
man,
you
want
feet.
You
know
some
of
this
measurement
data
that
you
know
you
can
subscribe
to,
so
that
you
can
see.
E
What's
going
on
or
even
like
late,
LinkedIn
or
Facebook
I
mean
be
pretty
pretty
sweet
because
right
now,
every
single
consumer
of
that
data
has
now
have
their
own
parsing
routine.
To
figure
out,
like
you
know,
what
does
field
number
12
and
what
happens
when
John
is
prints?
It
starts
a
field,
new
field,
number
12,
then
I
can
go
back
and
read
to
all
these
things.
That's
Doc's
man
right.
Yes,.
B
K
K
We
are
also
having
lots
of
Internet
users
that
are
accessing
the
internet
from
mobile
devices,
and
so
so
one
of
the
challenges
also
is
trying
to
measure
much
more
from
from
those
vantage
points
from
the
end
users,
mobile
devices,
there
are
quite
limited,
choose
distribution
of
vantage
points,
anything
that
speaks
I,
don't
know
what
you've
got
ideas.
We've
tried
to
use
some
some
to
speak
checker,
one
of
them
very
hard
to
get
users
to
install
the
app
on
their
on
their
phone
because
it
uses
data
and
so
on.
I,
don't.
D
So,
to
share
some
of
my
so
back
when
I
was,
you
know,
designing
an
Arctic,
neuron
mobile
network.
One
of
the
challenges
from
an
internet
perspective
is
a
lot.
Mostly.
Infrastructure
is
actually
invisible
to
the
Internet
in
a
mobile
network.
You
pass
right
over
it.
You
know
the
first
point
of
access
to
the
Internet
to
is
ik
or
3gpp,
guys
call
it
the
PD
n
gateway.
Is
those
I
ran
where
most
of
your
latency
likely
isn't
a
lot
of
your
rerouting
will
happen
and
then
to
mobile
core.
D
So
you
know
one
of
the
challenges
there
is
to
get
the
mobile
riders
to
help
provide
tooling
on
the
the
substrate
infrastructure
that
actually
prizes.
Most
of
what
is
the
experience
is
once
you
get
up
to
the
once.
You
get
up
to
your
first
gateway
that
you
know
the
mobile
networks
think
that's
just
a
link
right,
but
it's
not
it's
actually
a
complex,
long,
large
network.
That
typically
has
especially
in
earlier
infrastructure,
is
probably
a
significant
amount
of
aggregation.
So
there's
a
lot
of
back
hauling
and
then
just
to
throw
one
more
piece
in
there.
D
So
I
think
getting
a
substrate
tools.
Is
important?
The
other
thing
too,
is
if
any
way
to
get
some
of
the
exchange
like
the
GPX
IPX
net.
It
changes
to
actually
tool
that
part,
because
with
roaming,
unless
you're
doing
local
breakout,
you're
actually
crossing
back
to
your
home
network.
So
there's
there's
versions
where
understanding
how
that
how
that
performs
is
relevant
to
what
is
an
Internet
experience
that
makes
sense.
C
Just
to
confirm
I
fully
agree
with
also
with
the
metadata
argument
and
we're
trying
to
do
our
best
to
do
that.
You
know
every
piece
of
result
that
we
deliver
has
like
what
kind
of
finger
I
did
do
that,
and
if
that
firmer
had
the
bug
you
can
look
it
up
and
so
on
and
so
forth,
as
it
has
been
pointed
out
before,
we
can
always
do
better,
but
at
least
we
try
to
do
that
as
well.
K
L
Make
it
brief,
actually
I
think
I
think
that
that
the
metadata
conversation
kind
of
went
the
direction
that
I
wanted
to
I?
Think
all
the
questions
that
I
have
are
answered.
If
I
I
could
make
a
comment
on
that
I'm
actually
gonna
get
up
here
and
not
abuse.
Robert
I'm
gonna
praise
one
of
the
things
at
the
right
balance.
L
Guys
have
done
yeah
in
that,
so
there's
I
mean
there's
a
couple
of
definitely
there's
a
couple
of
dimensions
here
right
on
how
you
make
data
available
and
one
of
the
one
of
the
really
nice
things
about
ripe
Atlas
is
what's
stored
in
the
database
is
what
came
directly
off
the
probe.
So
you
have
you
know
not
ground
proof.
You
don't
have
any
problems
where
there's
some
sort
of
intermediate
data.
L
If
you
talk
about
interchange
and
I,
think
that
you
know
I
mean
I
think
we
know
how
to
do
a
lot
of
these
things
with
metadata
and
keep
the
metadata
as
together
with
the
data
and
make
sure
that
we
don't
make
gratuitous
changes
to
the
data
as
its
stored
I
mean,
we've
learned
how
to
do
these
things
and
and
I
think
we
keep
not
in
the
idea.
If
we
keep
not
solving
that
problem.
L
Right,
I
know:
I'm
gonna
point
to
some
I'm
gonna
point
to
a
draft
that
I
haven't
read
but
I
read
the
I'm
gonna
put
out
on
the
spot
here
and
point
out
that
we're
actually
taking
on
the
traceroute
problem.
Well,
we
might
people
are
trying
to
take
on
the
tracer
a
problem
again
in
AI
ppm
and
we
might
actually
take
it
on
that
turns
out
to
be
really
difficult.
Like
you
know
formats
there
there
are
an
infinite
number
of
possible
arbitrary
choices
that
you
can
make
and
how
to
represent.
It's.
M
L
There's
a
there's
half
a
PhD
in
that
statement,
but
actually
like
getting
people
who've
worked
on
measurement
people
of
work
on
standards
and
people
who
actually
use
routing
information
for
debugging,
especially
some
of
the
really
interesting
stuff
that
can
happen
with
any
cast
together
to
kind
of
bang
on
that
problem
might
be
worth
doing
and
everyone
who's
nodding
gets
to
help
contribute
to
that
effort
and
I
guess.
I
have
to
do
so
as
well,
because
I
said
that
it
has
to
be
done
so.
E
E
L
E
Have
one
other
questions
so
everything
that
we've
I
think
we've
all
talked
about
here,
I
put
into
the
category
of
an
active
measurement,
something
that
we
run
some
code
it
does.
Some
things
brings
back
some
data
right,
so
I'd
like
to
understand
and
I'm
kind
of
new
did
to
this
group
a
and
are
W
right,
one
of
the
questions
and
then
obviously
this
is
actually
something
that
I
don't
have
an
answer
to
so
I'm.
E
Looking
to
you
guys
for
some
advice
and
some
guidance
here
so
there
there's
lately,
there's
been
some
discussions
around
people
who
say
you
know
what
you
know.
You
don't
need
to
actively
measure
okay.
So
then
what
do
you
do?
Rea
so,
like
Robert,
said
something
about
raw
bits
on
the
wire
right.
So
are
there
any?
E
Is
there
any
work
being
done
where
people
can
say,
hey,
there's
this
research
in
this
space,
where
somebody
says
all
they
do
is
look
at
interface,
counters
and
the
time
and
increment
by
which
they
change,
not
an
active
measurement
but
they're
looking
at,
like
just
you,
know,
raw
data
and
making
assertions
about
the
raw
data.
So
it's
a
step
away
from
what
I
would
call
kind
of
a
bonafide
users,
yeah
user
experience
right
like
there's.
E
There
is
no
user
we're
just
looking
at
white
counters,
which
is
on
the
wire
for
a
protocol
and
doing
you
know
just
simple
delta
math
between
different
points
in
time
and
there's
some
assertions
that
are
being
made.
I,
don't
I,
don't
know
if
I
totally
agree
with
that,
like
you
can
do
the
same
thing
that
way
as
an
active
measure
with
you
and
I
struggle
with
that
a
little
bit
right,
so
I
like
to
get
either
a
reaction.
Gordon
Heiser.
Is
there
like
a
research
paper
that
you
can
point
to?
G
G
J
J
Note
the
order
that
they
put
that
in
it's
not
going
to
active
passive
measurement
conference.
It's
passive,
active,
so
I,
there's
tons
of
good
stuff
in
there
papers
over
the
years,
but
the
thing
that
put
the
kibosh
on
that
is
kind
of
the
Snowden
revelations.
You
know
when
we
were
doing
the
large-scale
measurement
effective.
You
know
of
access
performance,
Elm
at
everybody
said:
oh
yeah,
we're
not
gonna,
be
able
do
this
passive
stuff
anymore.
Nobody
wants
their
stuff
touched
and
we
can't
even
talk
about
it.
L
Yes,
I
was
gonna
say
that
that
question
opens
an
entirely
different
workshop.
That
apparently
will
take
place
here,
starting
at
5:00
p.m.
but
there's
no
I
mean
yeah
ya,
know
it'll
start
upstairs
at
5:00
p.m.
I
would
I
would
what's
what's
actually
go,
have
a
beer
and
talk
about
this
there's
like
also
every
paper,
our
common,
well
half
the
papers.
Mark
Roman
wrote
in
the
mid
90s
talking
about
doing
TCP
inference,
and
this
is
actually
one
of
the
big
discussions
that
we're
having
with
quick
right
right
now
is
there
are
a
lot
of
there?
L
Are
there?
Are
people
who
build
boxes
that
will
give
you
this
information
off
of
TCP,
I,
think
thousand
eyes
thousand
eyes.
Actually
does
this
that
Randy
mentioned
and
when
you
sometimes
they
do
it
wrong
right?
So
this
you're
saying
you're
a
little
bit
skeptical!
It's
like
you!
If
you
do
the
really
stupid
thing,
looking
at
sequence,
numbers
and
acknowledgment
numbers,
you're
gonna
get
a
bad
data
stream.
It's
gonna
give
you
a
bad
estimate
of
what
the
but
Lawson.
L
If
you
are
a
little
bit
smarter
about
that
and
you're
willing
to
make
deeper
and
deeper
inferences
about
what
the
endpoints
are
doing,
then
you
can
get
better
data
off
of
that
and
we're
actually
looking
at
some
work
on
how
we
can
design
that
in
two
quick
in
such
a
way
that
you'll,
you
won't
have
to
do
too
much
inference
to
make
it
work
yeah.
So.
E
E
A
So
quick,
quick,
quick,
final
word
to
close
this
yeah
thanks
all
of
you
for
attending
things.
There
are
things
to
met
for
his
support
on
running
things
in
the
back,
while
making
better
sense
of
what's
happen
and
the
generous
support
from
Isaac
to
our
other
sponsors
from
UK
meet
upstairs
again
and
also
to
Stephanie
from
the
ITF
team,
who
has
been
doing
a
tremendous
job
in
getting
us
organized,
sometimes
on
short
notice,
less
than
24
hours
and
yet
a
year.