►
From YouTube: IETF111-BMWG-20210726-1900
Description
BMWG meeting session at IETF111
2021/07/26 1900
https://datatracker.ietf.org/meeting/111/proceedings/
D
A
F
A
A
Okay,
all
right,
that's
that
was
a
good
good
troubleshoot
bill.
Thank
you,
no
problem
all
right.
A
Well,
I
guess
we're
gonna
get
rolling
here.
A
The
the
appointed
hour
has
has
arrived
and
I
think
we've
got
most
of
the
key
players,
although
I
I
don't
see
my
my
co-chair
yet
sarah,
but
I
think
we'll
we'll
start
with
some
of
the
preliminary
stuff
here
and
hope
that
sarah
is
able
to
join
us.
So
welcome
everybody.
A
A
The
meeting
will
be
chaired
by
me
and
sarah,
and
if
you're
not
subscribed
to
the
bmw
ge
mailing
list,
you
can
do
that
with
the
link
in
the
slides.
All
the
slides
are
available
online
in
the
meeting
materials
and
all
the
drafts
we're
going
to
talk
about
today.
They've
all
been
highlighted
there,
so
we're
in
pretty
good
shape
administratively
and
so
now
I'll
move
on
to
the
to
the
next
slide.
A
Okay
slide:
two
most
important
is
this
little
sentence:
I've
added
we
work
as
individuals
and
we
try
to
be
nice
to
each
other
and
and
that's
becoming
very,
very
important
as
we
as
we
get
deeper
and
deeper
into
the
you
know
the
the
zoom
phobia
that
we've
we've
all
got.
So,
let's,
let's,
let's
do
that
so
then.
A
The
rest
of
the
note
well,
is
that
it's
a
it's
basically
a
reminder
of
the
ietf's
policies
on
topics
such
as
patents
and
code
of
conduct
meant
to
point
you
in
the
right
direction,
and
we,
the
patent
policies,
periodically
updated
the
one
we're
working
with
on
this
as
a
on
this
slide
is
as
of
march
2018
and
so
by
participating
in
the
ietf
you,
you
agree
to
follow
the
ietf
processes
and
policies
you're,
aware
of
that
any
ietf
contribution
is
covered
by
its
patents
and
and
so
forth,
of
what
you're
a
sponsor.
A
You
must
disclose
that
fact
or
not
participate
in
the
discussion
and
as
a
participant
or
attendee.
You
acknowledge
that
any
written
audio,
video
or
photographic
records
may
be
made
public
and
as
a
participant
or
an
attendee,
you
agree
to
work
respectfully.
A
The
personal
information
you
provide
will
be
handled
according
to
the
privacy
statement
and
and
really
the
most
important
thing
is
that
anything
that
that
happens
here
beyond
just
listening
and,
of
course
our
meetings
are
recorded,
is
a
contribution
and,
and
therefore
it's
covered
by
this
policy
and
all
the
details
for
this
policy
are
in
the
list
of
bcps
and
the
privacy
policy
link
there
below
any
any
questions.
A
So
here's
our
our
agenda
for
today,
what
we've,
what
we've
basically
got
planned,
is
the
working
group
status,
which
is
two
items.
The
working
group
drafts,
which
we've
shuffled
the
order
a
little
bit,
multiple
loss
ratio,
searches
is
going
to
go
quickly
and
first
and
and
then
on
to
the
next
generation
firewall
benchmarking
discussion,
which
is
going
to
take
longer,
but
we
we
hope
to
be
able
to
complete
that
very
productively
today,
and
then
we
have
a
couple
of
proposals
that
have
really
had
a
certain
amount
of
discussion.
A
Some
you
know
more
than
others,
but
a
certain
amount
of
discussion
on
the
mailing
list.
Recently-
and
it's
probably
you
know
it's
probably
time
for
us
to
start
thinking
about
taking
up
some
additional
work
here
in
in
the
bmw
g
and
since
these
items
have
garnered
some
interest.
It's
worthwhile
thinking
about
those.
So,
with
this
agenda,
are
there
any
bashes
to
the
agenda.
A
Hearing
none,
let's
run
it
as
it's
written,
we've
got
bill,
cerveny,
taking
notes
and-
and
maybe
somebody
else
in
on
on
code
emd
that
I
I
saw
was
was
on
there.
I
think
you
know,
I
think
we've
got
a
fairly
easy
way
to
cover
jabber
now.
A
Oh,
let's
see
well,
let's,
let's
just
let's
sort
of
assume
that
you
know
whatever
whatever
facility
here
is
available
for
use,
that
folks
will
find
it
and
use
it
and
and
it
will
be
available
in
the
meeting.
Oh
look
here,
it
is
the
chat
panel
yeah,
that's
exactly
right.
A
So
so
this
is
the
this
is
where
we're
basically
taking
on
our
our
jabber
inputs
and
I'll,
be
looking
for
the
chat
panel
to
to
flash
red
at
us,
and
we've
got
27
people
online
at
the
moment.
So
that's
that's
great
good
good
turnout
for
this
first
session,
all
right
so
jabber
and
we've
covered
ipr
all
right
on
to
the
next.
A
So
here's
our
status
and
we're
we're
basically
looking
at
the
fact
that
the
evpn
draft
is
gone
through
its
iesg
review
period.
There's
some
discuss
ballots
to
resolve
and
the
authors
are
hopefully
working
on
that.
I
think
there
there's
initially.
I
saw
some
progress
there,
but
not
much
recently.
A
So
then,
the
the
next
generation
firewall
benchmarking
document
that
I'm
I
may
have
forgotten
to
update
some
of
these
numbers
here.
I
know
that
we
had
a
confirm
the
confirmation.
Last
call.
I
think
was
was
on
o6
and
now
we're
all
the
way
out
to
version
nine,
I
think,
is
that
right.
That's.
A
Okay,
so
there's
been,
I
mean
since
the
since
the
last
call
and
and
actually
leading
up
to
it.
Actually,
what
I
see
here
is
in
my
notes
is
that
there
was
a
working
group
last
call
on
on
08
in
may,
so
that's
that's
where
we
stand
and
we
have
another
version
and
but
we've
still
got
a
few
comments
to
resolve
and
we're
going
to
try
to
resolve
them
in
this
session.
A
One
one
point:
I
wanted
to
point
out
just
an
additional
comment
here:
we've
got
one
sentence
in
the
draft
that
that
we
agreed
on
it
says
it
obsoletes
rc
3511,
but
we
need
a
sentence
in
the
abstract
all
by
itself.
That
says
that,
and
we
should
also
add
that
status
in
the
header
and
if
you,
if
the
author
team
needs,
help
doing
that,
just
let
me
know
I'll
I'll
try
to
help
out
all
right.
A
So
the
as
I
said,
the
proposals
keep
coming
and
shall
we
make
way
for
new
work
well
by
you
know,
moving
moving
stuff
in
and
out
of
the
iesg
and
and
so
forth.
That's
how
we're
doing
that
so
we're
making
way.
We
need
to
decide
what
we
take
up
next,
so
here's
some
good
news
back
in
may
rfc
9004
on
the
back-to-back
frame
benchmark
was
published.
A
It's
an
update
of
of
that
that
particular
section
in
rfc
2544,
and
so
we're
done
with
that
one
and
the
charter's
been
stable.
Thanks
to
the
last.
A
With
warren's
joining
us
says,
a.d
advisor
warren's
here
warren
is
here
today.
A
We
we
arranged
this
thing,
so
that
would
be
fairly
stable
for
our
work
and
didn't
require
a
lot
of
maintenance.
So
I
think
that
was
the
right
choice
and
you
know
it
just
lets
us
spend
our
time
on
on
doing
the
work
instead
of
chartering
it,
and
we
still
have
a
nice
supplementary
working
group
page
that
sarah
has
provided
for
us
so
in
the
milestone
review.
Obviously
there's
some
few
things
that
we're
still
a
little
bit
behind
on
and
we
may
have
updated
some
of
these.
A
I
I
forgot
to
check
that,
but
we've
also
got
a
few
of
them
that
are
done
and
we're
fairly
close
to
picking
up
work.
That
would
satisfy
some
others
so
well,
but
as
always,
there's
kind
of
a
need
for
a
milestone,
reconciliation,
even
just
updating
them,
and
and
that's
it
for
the
working
group
status.
Any
questions.
H
A
And
masiak,
if
you,
if
you
put
yourself
in
the
queue
or
or
enable
your
microphone
there,
we'll
be
able
to
hear
you
in
time
to
present
this
okay.
I
Hello,
hi
sarah
hi,
everyone
hi,
so
radko.
I
couldn't
make
it
today,
so
I
will
be
presenting
the
update
on
the
draft
and
next.
So
this
is
the
the
multiple
loss
ratio
search,
drought
that
we've
been
working
within
this
work
group
for
for
a
while
now
next
slide.
Okay,.
I
We
have
posted
an
updated
version,
zero
one
earlier
in
july
and
I'll
quickly
cover
the
the
main,
the
main
changes,
but
we're
looking
for
more
reviews
and
and
comments
on
the
on
the
list
or
any
other
and
the
other
way.
So
the
main
changes
are
really
related
to
the
at
the
points
that
we
have.
I
think
briefly
announced
on
the
in
the
previous
meeting
in
the
atf-110
and
bringing
the
draft
in
line
with
the
open
source
implementation.
I
We
have
in
lfn
fast
data
io
project,
so
there
are
some
terminology
updates
related
to
the
to
the
updated
algorithm
and
the
logic.
I
The
main
the
main
draft
update
is
associated
with
the
the
logic
improvements
specifically
support
for
configurable
number
of
target
packet
loss
ratios
versus
before
the
actual
sample
implementation
was
based
on
on
on
two
of
those
and
and
also
associated
aspects
related
to
the
way
that
how
the
measurements
are
tracked
and
versus
the
versus
the
bounds
for
the
rates
that
we
are
searching
for.
I
So
that
is
that
that
that
resulted
in
the
updates
in
the
mlr
search
overview
section
and
the
sample
implementation
section
and
also
reflected
in
the
in
a
sample
run
that
we
have
also
updated
in
the
example
mr
search
run
section.
I
So
these
are
the
the
main
updates
next
slide
in
terms
of
the
the
pointers
on
where
we
are
using
this
approach
in
the
open
source,
so
we're
operating
we're
using
this
implementation
on
using
a
t-rex
traffic
generator
in
the
in
the
elephant.
I
Fdio
project,
as
I
have
mentioned,
there
are
some
pointers
to
the
to
the
to
the
links,
including
description
of
methodology
and
how
we're
using
the
mlr
search
and
also
the
the
python
package
in
that
in
the
pi
pi
has
been
updated
at
the
version:
zero,
four,
zero
and,
and
also
it
is
staying
a
bit
behind
of
for
what
we're
running
in
the
fdio
project.
I
It
does
capture
the
stuff
that
we
have
updated
in
the
in
the
draft,
so
the
the
support
for
more
than
two
targets
loss
ratios-
and
that's
that's
really
the
update.
So
I'm
looking
for
for
comments,
reviews
and
if
there
are
any
questions.
A
Very
good:
well
thanks
for
that
masiak
any
any
questions
and
or
or
comments
at
this
point,
I
think
I
saw
brian
come
through
there
briefly,
but
please
cue
yourself
up.
If
you
want
to
ask
some
questions.
A
All
right-
and
I
mean
there's-
it
looks
like
there's,
there's
plenty
of
stuff
to
try
out
here
that
that
might
be
beneficial
to
create
you
know
to
or
to
prompt
some
comments
on
the
draft,
and
so
that's
kind
of
what
I
recommend
to
folks
is.
Is
there
anybody
who'll,
put
their
hand
up
and
volunteer
to
to
review
the
draft?
A
I
mean
we,
we
picked
this
up
with
working
group
support,
and
so
certainly
anyone
who
said
they
would
support.
It
should
be
willing
to
review.
A
A
Yes,
so
all
right
and
vladimir
all
right
so
now
I'm
trying
to
not
remove
from
q.
What
do
I
want
to
do
so?
This
is
the
thing
I
didn't
get
practice
with,
because
I
didn't
have
a
cue
gabor,
I'm
going
to
try
to
I'm
going
to
try
to
remove
you
from
the
queue
and
see
what
happens
all.
G
A
All
right,
okay!
So
in
any
case
I
saw
two
people
join
the
queue
gabor
and
vladimir,
and
if
they're
willing
to
review
the
draft,
that's
great
so
feel
free
to
open
up
your
mics
and
and
confirm
that
if
you
can.
K
A
Yeah
yeah,
I
think,
I
think,
and
and
of
course
I'll
I'll
review
this
when
when,
whenever
and
and
wherever
I
I
get
some
time
thanks
for
your
your
effort
on
on
this
massacan
and
please
pass
it
along
to
viratco
too.
A
A
So
it
looks
like
looks
like
sarah
is
still
having
some
trouble
joining
the
the
call
here
today.
Let.
F
L
L
L
A
So
we're
we're
about
to
pick
up
the
benchmarking
methodology
for
a
network
security
device
discussion,
and,
let
me
ask
brian:
are
you
going
to
present
the
slides
today.
B
Not
all
the
slides
I'll
present
the
first
first
couple
of
them,
then
carson
and
battle
will
pick
it
up
from
there.
A
A
And
so
I
mean
it's
right
next
to
the
number
of
participants
in
your
on
your
meeteco
screen.
L
A
L
A
All
right
so
go
ahead,
kick
us
off
here,
brian
and
and
and
incidentally,
what
I
want
to
do
is
I
want
to
leave
about
about
15
minutes
each
for
the
remaining
two
presentations.
If
we
can
so
by
my
calculation
that
gives
us
it
gives
us
an
hour
and
and
five
minutes.
B
Okay,
just
so
you
know
I'll
I'll
be
I'll,
be
taking
some
handwritten
notes
as
as
well
once
I
stop
stop
speaking
so
I
can
share,
can
transcribe
and
share
them
with
you.
If
you
want.
A
Except
that,
okay,
that's
great!
No
I'm
sorry!
I'm
thinking
I'm
doing
two
things
at
once
here
and
the
silly
countdown
clock
appears
right
in
the
middle
of
the
slides
for
me
and
I
couldn't
move
it.
So
that's
not
going
to
work
I'll.
Just
keep
track
of
the
time,
all
right,
so
yeah
figure
on
a
little
more
than
an
hour
and
please
go
ahead
brian.
Whenever
you're
ready.
B
All
right
just
go
to
the
next
slide.
Please
all
right.
B
So
the
the
agenda's
pretty
straightforward,
it's
a
pretty
quick
presentation
on
the
timeline
and
the
draft
status
and
then
carson
and
bella
will
review
the
open
comments
and
then
discuss
net
next
steps
at
the
at
the
end
of
the
discussion.
Next
slide.
B
All
right,
so
just
so,
you
know
I'm
going
to
just
you
know,
talk
about
this
slide
mainly.
B
We
have
a
second
slide
or
the
next
slide,
and
the
one
after
that
presents
timeline
a
lot
more
detail,
but
I
I
won't,
you
know,
go
through
those
ones
at
the
same
time,
but
the
initial
proposal
for
the
draft
was
november
of
2017,
where
it
was
proposed
and
accepted,
and
we
uploaded
the
first
draft
about
a
month
about
a
month
after
that
and
then
received
initial
comments
from
folks
in
the
bmw
g
about
a
a
month
a
month
after
that,
so
the
first
five
drafts
were
under
balance,
bala's
name
until
they
were
formally
adopted
by
the
bmwg
and
in
in
december
of
2018.,
so
it
took
about
a
year
and
a
bit
before
it
became
a
a
formal
work
item.
B
As
far
as
the
bmw
goes,
we
released
the
first
draft
against
that
in
march
2019
between
march
2019
and
march
2020,
we
had
five
draft
versions
had
a
lot
of
good
discussions
at
ietf
104
through
109..
B
I
received
comments
from
a
number
from
a
number
of
people,
as
you
can
see
itemized
here,
I
won't
bother
reading
it
out,
but
it
was
a
significant
number
of
contributions
and
comments
made.
B
So
we
really
appreciate
that
an
awful
lot
then
the
working
group
last
call
was
started
last
last
december
and
then
the
second
working
group
last
call
was
was
started
in
and
in
may
of
of
the
sh
of
this
year,
we've
received
in
total
about
150
comments
and
suggestions
from
about
12
12
contributors
during
the
last
calls,
via
email,
post
or
post
to
the
benchmark,
working
group,
mailing
lists
and
all
of
the
comments
have
been
were
resolved.
B
With
the
exception
of
the
of
comments
that
we
received
from
sarah
on
may
20th
and
july
12th
of
those
comments,
I
think
we've
resolved
pretty
much
everything
with
the
exception
of
about
six
of
them.
So
we
have
so
this.
B
Today's
discussion
will
focus
on
that
six
plus
issues
that
that
that
need
to
be
resolved
in
order
to
to
move
forward.
Does
anybody
have
any
questions
or
comments
before
we
move
on.
B
All
right,
then,
so,
if
you
go
thanks
a
lot
so
around
what,
as
I
said,
around
30
comments,
worse
and
suggestions
were
posted
to
the
mailing
list
by
sarah
on
march
20th.
It
says
here
11.,
I
guess
my
math
isn't
all
very
good,
but
but
there
were
some
initial
responses
that
where
were
sort
of
not
agreed
to
in
in
toto,
so
it
gives
this
gives
you
a
basic
idea
of
where
sort
of
what
bucket
the
comments
fallen.
B
The
scope
definition
is
a
testbed
environment
and
and
the
traffic
mix,
and
it's
worth
noting
that
none
of
these
comments
that
we
believe
actually
relate
to
the
core
sections
of
the
draft.
That
is,
the
the
actual
specifications
of
the
the
bence
benchmarking
methodology.
Next
slide.
B
So
at
this
point,
I'm
going
to
pass
this
over
to
bella
and
carson
who
will
discuss
the
the
technical
details
of
the
comments
guys.
M
Thanks
brian,
so
this
is
carson
speaking
yeah.
Let's
dive
right
into
the
remaining
open
comments,
and
I'm
not
sure
I
mean
we
have
just
you
know
grouped
them
in
the
way
they
came
in
and
maybe
some
of
them
are
related
to
each
other.
But
let's
just
start
so
we
have
a
couple
that
are
related
to
the
scope
definition,
and
so
I
think
we
don't
need
to
reread
the
text
or
anything.
M
Sarah
can
probably
comment
by
herself
and
so,
from
our
point
of
view,
the
the
reason
to
replace
rce
35
11
is
really
that
you
know
we're
recovering
the
old
scope
of
firewalls,
we're
covering
additional
ones,
and
the
question
was:
what
is
this
term
next-gen
firewall?
M
M
L
If
that's
okay
with
you,
sarah
yeah,
so
I
really
appreciate
gabor,
I
did
see
your
email.
I
just
hadn't
had
a
chance
to
respond
yet
I
did
see
that
and
I
appreciated
the
due
diligence
and
hey
if
the
scope
is
around
ng
firewalls,
and
this
is
the
definition
you
want
to
use.
I
think
that
makes
a
lot
of
sense
and
I
also
like
saying
by
the
way
sometimes
optional
features,
because
you
know
not.
Everybody
has
everything
right,
so
yeah.
L
Yeah
I
I
could
almost
hear
scott
bradner's
sort
of
comments
to
us
here
right,
but
the
notion
that
hey
we're
trying
to
classify
it
as
putting
the
year
around
that
I
thought
really
helped
and
then
being
specific
around
application
level,
traffic
inspection
and
then
having
optional
features.
I
thought
some
of
that.
There
made
a
lot
of
sense.
M
M
Be
seen
so
we've
had
these
question
of
what
is
a
security
device
security
device.
Sorry-
and
I
think
that
was
actually
very
good
comment,
because
we
didn't
think
about
the
possible
confusion
with
you
know
passive
devices
that
would
not.
You
would
just
sit
there
outband
somewhere,
and
our
suggestion
is
to
update
the
list
of
the
security
features
that
any
security
devices
in
scope
can
have
and
to
describe
that
they
must
be
configured
in
inline
mode,
so
be
active
devices
actually
transiting
or
you
know,
forwarding
the
data
actively
that
runs
through
them.
L
Yeah,
so
look,
you
know
from
my
perspective,
I
feel,
like
you
know
it's
one.
I
know
it's
weird
I
sort
of
picked
on
the
hey.
What
does
ng
mean
because,
frankly,
I
couldn't
find
a
definition
either.
So
it's
really
to
be
able
to
say
my
device
is
ng,
regardless
of
in
line
or
not
I
struggled
with,
but
if
we
put
a
pin
in
that
for
just
a
sec,
I'm
not
sure
I
100
agree
that
in
order
to
be
sort
of
a
next
generation
security
device,
you
have
to
be
active
in
line.
L
I
think
there's
the
notion
of
next
generation
security
devices
and
some
of
them
are
in
line
and
some
of
them
aren't.
But
I
think
what
you're
trying
to
say
is
look
if,
if
we
could
go
with
that
that
hey
there
are
next
generation
security
devices,
some
that
work
in
line
some
that
work
out
of
out
of
band
or
or
passive,
and
this
draft
is
going
to
focus
on
the
active
inline
ones.
L
I
think
that's
a
really
easy
way
forward,
because
it
keeps
your
focus
clear,
so
you're
not
saying
that
hey
an
ng
ids,
which
hasn't
been
defined
and
isn't
in
scope
of
this
draft.
I
know,
but
it's
not,
that
anything
on
the
passive
side
couldn't
be
ng.
It's
that
this
draft
specifically
is
examining
the
things
that
are
configured
for
active
inline.
Would
that
be
a
fair
statement
of
where,
where
you
guys
are
seeing
the
world
at
two
and
would
that
potentially
work
as
a
way
forward.
M
I
think
so.
Okay,
I
mean,
if
anybody
else
has
other
comments,
that
that
would
be
the
last
chance
now,
because
otherwise,
I
think
we're
going
to
to
close
that
discussion.
M
Any
other
okay
alex.
I
don't
know,
l
how
you
manage
the
queue.
A
C
Okay,
yeah,
I
wasn't
sure
if
I
should
say
anything
before
or
after
you
know.
Definitely
the
you
know
I
I
agree.
Next
generation
doesn't
really
have
anything
to
do
with
whether
whether
a
device
is
in
line
or
not.
You
know
I'm
from
a
network
security
vendor.
You
know
we
have
both.
You
know
firewall
next
generation,
firewall
ips
and
any
of
those
devices
can
be
inline
or
not
in
line.
C
So
if
you
have
a
if
a
device
is
forwarding
traffic,
as
carson
just
said,
you
know
it,
it
certainly
you
know,
can
qualify
here,
but
just
because
somebody
has
a
ips
or
next
generation
ips.
That
doesn't
necessarily
mean
it
is
a
passive
device.
There
are
plenty
of
them
that
are,
you
know,
active
and
you
know,
can
operate
in
both
modes.
So
I
don't
you
know,
I
don't
know
if
ng
implies
any,
you
know
inline
or
not
inline
capabilities
of
the
device.
M
For
the
purpose
of
defining
the
scope,
I
think
that's
what
we
want
to
cover
and
what
we
actually
do
cover
in
the
test
methodology.
Primarily
in
this
version-
and
maybe
you
know,
some
of
the
confusion
came
from
this
big
list
of
the
grand
scheme
of
what
we
might
want
to
cover
in
the
future.
But
after
all
of
this
revisions,
I
think
we're
really
focusing
on
the
firewalls
at
this
time.
L
Yeah
that
makes
person
that
makes
a
lot
of
sense
right.
I
definitely
for
for
what
it's
worth.
My
confusion
definitely
came
from
the
top,
where
what
I
interpreted
in
what
I
read
was
that
hey
an
ng
fire,
an
ng
security
device,
is
this
and
by
the
way
it
has
to
be
active
in
line,
and
I
was
just
taking
a
little
exception
to
look.
I
I
really
like
the
idea
of
an
ng.
L
I
really
like
the
original
premise
of
the
draft
to
sort
of
be
a
little
broader,
but
then,
when
we
sort
of
restricted
it
to
only
in
line
that's
when
I
was
saying
well,
I
don't
know
that
I
agree,
but
it
definitely
was
clear
in
your
methodology
that
you
were
certainly
thinking
only
of
active
in
line,
and
so
that's
why
I
was
saying
hey.
If
it's
the
notion
that
look
we
we
can
talk
about
what
all
of
these
devices
are,
but
then
say
our
scope.
L
Is
this
and
it
sounds
like
we're
agreeing
then
I
think
that
makes
a
lot
of
sense
and
it
removes
all
of
the
the
confusion
for
me.
M
M
C
C
L
Yep
totally
agree
there.
M
Is
a
comment
from
warren
about
next
gen
fire?
What
is
a
stupid
term?
I'm
not
recommending
or
suggesting
it
to
be
changed,
but
it's
an
annoying
marketing
term
grub
and
I
completely
agree
warn
and
sympathize.
We
really
thought
about
this
and
they
also
thought
well,
it's
just.
It
sounds
like
marketing,
but
unfortunately,
or
unfortunately,
or
whatever.
It
is
the
term
that
people
use
right,
and
I
recently
sent
a
message
on
the
mailing
list
about
the
google
search
and
that's
just
overwhelmingly
the
term
that
people
use.
L
I,
for
what
it's
worth
so
carson
bear
with
me.
I
totally
agree
you
can
google
the
heck
out
of
it,
but
for
what
it's
worth
when
I
googled
the
heck
out
of
it,
I
actually
couldn't
find
anybody
defining
what
it
was.
Everybody
says
they
have
one,
because
I
think,
frankly,
warren
the
analysts
really
call
it
ng
firewall.
So
you
kind
of
get
lumped
that
way,
and
I
think
that's
why
customers
come
at
us
as
vendors
with
you
know.
L
Are
you
an
ng
firewaller,
or
are
you
not
right,
so
I
totally
understand
wanting
to
use
it,
and-
and
I
want
to
be
clear-
I
was
a
little
not
super
comfortable
with
it,
because
it
was
originally
the
scope.
I
thought
was
really
broad
for
everything
and
with
it
being
specifically
around
ng
firewalls.
I
think
that
makes
a
lot
of
sense
and
I
totally
accept,
I
think,
carson's
early
on
comment
or
brian
wrote
it,
but
somebody
on
the
author
team
had
said
hey.
This
is
sort
of
a
well-known
thing.
L
It
kind
of
is
it's
it's
a
really
weird
example
of
everybody
knows
what
an
ng
firewall
is,
even
though
nobody's
really
defined.
What
the
heck
it
is,
but
because
we're
narrowing
that
scope,
I
I
think
for
me,
I'm
okay
with
this
now.
M
Because
it's
there's
always
this
danger
to
get
hung
up
on
naming
naming
discussions
right,
okay,
so
the
next
one
is
still
kind
of
related
and
it
we
already
discussed
the
inline
part.
There's
also
the
question
of
fail,
close
or
fail
open,
so
fail
open
means
that
in
the
worst
case,
if
the
device
is
overwhelmed
by
too
much
tracking
incoming,
for
example,
it
was
just
open
the
floodgates
and
let
everything
through
we've
actually
seen
that
in
our
lab
to
our
utter
disgrace.
But
that's
not
what
we're
looking
for
in
this
draft.
M
M
One:
okay
and
please,
let's
go
to
comment
number
four.
M
It's
actually
a
different
topic
right
and
now
we're
entering
the
area
of
these
ancillary,
or
you
know
helper
devices,
and
I
understand
that
rates
some
concerns
or
and
also
confusion.
Why
do
we
need
them?
Why
do
we
want
them?
Actually,
nobody
wants
routers
or
switches
in
the
test
bank,
but
sometimes
they're
needed,
and
I
summarize
like
three
scenarios
when
they
will
be
needed.
M
The
aggregation
could
be
understood
as
ethernet
or
in
a
different
way
on
eip
level,
but
we
don't
care
for
this
here,
but
in
fact
that's
the
first
point.
The
first
option,
when
these
auxiliary
routers
or
switches
are
needed,
the
second
option
might
be
sometimes
we've
seen
measurement
equipment,
for
example
in
the
virtualized
space,
of
course,
not
t-rex
if
magic
is
still
listening,
but
some
other
measure
virtualized
measurement
equipment.
M
I
assume
that
each
of
these
10
gigabit
ethernet
physical
links
can
generate
at
most
2.5
gigabit
per
seconds
frames
for
a
specific
test
scenario,
and
then
four
of
each
links
are
required
to
fill
110
gigabit
each
network
towards
the
switch,
and
the
last
example
would
be
if
we
have
advanced
scenarios.
M
For
example,
when
we
use
specifically
open
source
tools,
which
might
one
of
them
might
be
the
traffic
tool,
the
other
one
might
be
the
attack
tool
and
the
threat
emulation
tools,
and
in
that
case
we
would
have
the
traffic
coming
from
two
different
sources,
and
so
there
is
no
auxiliary,
switch
again
needed
to
combine
the
traffic
towards
the
system
under
test.
So
these
are
the
three
examples
that
I
can
think
of
where
these
routers
and
switches
unfortunately
are
needed,
and
that's
why
we
introduced
them.
L
So
carson,
if
I
could
say
look
I
totally
agree.
I
think
you
know.
From
my
perspective
at
least
I
think
you
know
I
was
always
taught.
You
know,
minimize
the
amount
of
variables
that
you
have
in
your
test
bed,
because
not
for
these
tests,
for
example,
but
I
have
a
test
going
in
my
lab
right
now,
where
my
auxiliary
switch
is
actually
introducing
a
500,
millisecond
delay
and
maybe
that's
acceptable.
L
Maybe
it
isn't,
but
it's
just
another
variable
in
the
equation
that
you
have
to
take
into
account
for
or
not,
and
so
maybe
the
advice
is
look
where
you
can
simplify,
simplify,
but
we
do
understand
when
we're
testing
that,
for
these
three
reasons
these
are
the
three
different
use
cases
we
see
where
you
might
have
to
use
them.
It
did
when
I
read
that
I'll
have
to
go
back
and
reread
it.
So
maybe
it's
there
and
I'm
I'm
just
not
remembering
it.
I
just
when
I
read
it.
L
I
knew
that
I
figured
that's
where
you
were
going,
but
I
was
always
mindful
of
in
past
years
where
scott
would
get
up
and
he
would
say:
hey
well
the
more
boxes
you
have
in
your
test
bed.
How
do
you
know
the
accurate
your
the
results?
You're
getting
on
the
sut
are
accurate
versus
introduced
by
something
else,
and
I
think
that's
kind
of
where
you're
going
anyways
like
look.
If
you
can
simplify
and
directly
connect
your
measurement
equipment
to
the
sut
do
it.
L
M
C
Two
other
scenarios
that
I
you
know
have.
I
totally
agree
with
sarah
that
you
want
to
keep
it
simple,
but-
and
this
is
more
one's
gonna
be
on
the
switch
side.
One
is
gonna
be
on
the
router
side,
just
in
general,
in
placement
of
these
network
security
devices
that
are
in
people's
network,
they
are
generally
you
know,
behind
a
router
or
switch,
and
one
of
the
things
that
we've
seen
that
happens
on
testing
is,
if
you
have
the
test
equipment
that
which
is
simulating,
you
know,
lots
and
lots
of
clients.
C
You
end
up
getting
a
flood
of
mac
addresses,
which
does
not
generally
happen
in
the
customer
environment,
because
there
are
layer,
three
devices
between
the
the
end,
clients
and
the
the
actual
security
device.
The
second
one-
and
this
is
one
you
know
more
for
on
the
on
the
router
side-
is
when
I
have
a
virtual
router,
and
this
is
usually
within
the
test
tool.
It
becomes
extremely
easy
for
me
to
do
a
back-to-back
test,
which
is
what
I
will
do
in
order
to
eliminate
any
other
devices
in
line.
C
You
know
like
I
want
to
do
a
back-to-back
test.
Make
sure
the
the
system
under
test
is
not
in
there.
I
can
characterize
what
the
what
the
you
know
normal
behavior
looks
like
and
then,
when
I
introduce
the
system
under
test,
then
all
I'm
seeing
is
the
difference
between
those
two,
so
that
both
of
those
things
are
are
you
know
why
I
personally,
like
virtual
routers
and
switches
available
in
the
test
tool.
A
Thanks
yuri
something
quick.
N
This
is
yuri,
I'm
with
one
of
the
test
vendors,
so
I
just
wanted
to
add
a
quick
note
that
for
most
of
our
customers,
we
see
them
use
switches
in
between
our
test
equipment
and
devices
under
test,
and
that's
just
from
a
practicality
point
of
view,
so
that
they
can
easily
test
lots
of
different
equipment,
setups
and
versions,
and,
generally
speaking
today,
the
switch
latency
is
is
a
rounding
error
relative
to
the
latency
of
a
next
generation
firewall.
L
L
You
inherently
open
up
the
variables
of
what's
impacting
your
test,
and
I
like
what
alex
said
right,
establishing
your
baseline
as
close
as
you
can
back
to
back,
allows
you
to
understand
or
or
rule
out
frankly
that
yep
using
that
router
didn't
change
a
thing
and
I'm
good
to
go,
but
either
which
way.
I
think
all
of
the
comments
make
sense.
I
just
wanted
to
make
sure
that
color
comes
through
because
there's
a
lot
of
testing
know-how
on
this
call.
L
M
G
M
I
think
if
you
maybe
look
at
section
five
once
more,
we
we
do
recommend
the
pre-test,
because,
yes,
I
do
agree,
the
latency
should
not
be
a
problem
anymore,
but
of
course
there
could
be
surprises.
There
could
be
other
types
of
surprises
and
to
eliminate
this
and
to
make
the
these
auxiliary
switches
a
known
component
and
the
well-defined
component
of
the
testbed
not
influencing
the
actual
results.
M
A
So
we,
I
think,
we're
agreeing
that
we
want
to
add
some
notes
into
the
draft
based
on
today's
discussion
and
the
recording
will
be
available
soon.
L
Yeah-
and
the
previous
comment
addresses
here
too
right
because
carson
if
the
or
authors,
if
the
the
goal
is
look,
try
to
keep
it
as
simple
as
you
can.
But
if
you
need
to,
you
could
add
a
switch
here
or
you
can
add
a
router
there
again.
I
think
we're
leading
with
trying
to
keep
things
things
as
minimal
as
possible
and
or
how
do
you
establish
your
baseline
to
alex's
point
and
doing
things
back
to
back?
So
now
you
understand,
as
you
add
these
other
layer,
three
bumps
on
the
wire.
L
A
Yeah
and
and
as
yuri
pointed
out,
the
next
generation
switches
are
in
the
rounding
error
for
their
latency.
That's
cool
all
right,
so
I
think
we've
dealt
with
this
one.
L
A
And
then,
on
to
five
of
eleven
here,
karsten.
M
Right
so
there
there
are
two
more
basically
focusing
on
the
same
topic
of
these
additional
waters
and
I'm
not
exactly
sure
what
you
would
want
us
to
change
in
this
case.
Sarah.
So
the
question
is,
you
know:
can
we
describe
the
one
and
only
testbed,
and
I
think,
from
these
five
use
case
scenarios
you've
seen
that
there
are
multiple
different
options
and
that's
why
we
described
things
as
optional.
L
Yeah,
you
know
what
this
one,
I
think
is
is
fair.
I
think
I
said
I
I
wasn't
sure
I
think
I
wasn't
sure
what
you're
meaning,
but
now
I
am
so.
Why
don't
I
take
a
look
carson
and
go
back
and
see
if
I
can
come
back
with
it's
fair
to
say,
hey,
I
might
have
some
proposed
text.
What
do
you
guys
think
would
that
work
for
you.
M
Yeah
in
principle:
yes,
I'm
just
interested
to
close
the
whole
process
soon
yeah.
I
know
right
now.
A
A
Thank
you,
al
all
right.
So
that's
a
good
one.
M
Next,
one
is
on
the
choice
of
emulated,
routers
versus
real
routers.
I
think
it
doesn't
really
matter
from
the
terms
of
from
the
point
of
understanding
the
reference
testbed,
whether
they're,
emulated
or
real,
and
so
again
you
know
a
pretest
can
figure
out
what
the
performance
impact
is,
I'm
not
sure
bella.
If
I,
if
I'm
on
the
right
spot
here,
maybe
you
can
help
me.
L
I
I
would
still
say,
though,
proactively
hey
again,
if
it's
about
limiting
and
keeping
things
as
simple
as
we
can
and
or
having
a
baseline
and
then
adding
pieces,
so
you
understand
the
deltas
or
not,
then
I
think
this
comment
goes
away
with
that
as
well.
If
that
helps
you
guys
move
it
along.
N
Already
but
again
that
we
see
from
the
test
vendor
perspective
that
virtual
routers
are
most
commonly
used
for
for
two
reasons:
one
to
avoid
overrunning
mac
tables
and
things
like
that
in
the
switch
infrastructure
between
the
test
equipment
and
the
duty,
but
also
from
a
realism
perspective,
because
normally
in
a
typical
I.t
environment,
routers
or
firewalls
are
not
exposed
to
tens
of
thousands
of
mac
addresses.
Usually
there's
routers
in
between.
M
So,
let's
go
to
number
eight,
so
number
eight
was
about
ids's
and
we
just
discussed
this
among
the
authors
and
other
contributors,
and
we
said:
okay,
it's
it's
easiest
to
just
remove
intrusion
detection
systems
here
from
the
from
the
scope,
because
we
really
don't
want
to
open
the
whole
document
again
and
start
reviewing
it
for
the
purpose
of
another
type
of
devices.
L
L
Maybe
we
crossed
our
wires
there.
I
think
brian's
first
response
was
hey.
We
agree
we're
going
to
remove
ng
ids
from
the
list,
so
the
minute
that
table
comes
out.
The
notion
I
mean
never
mind
an
audience,
doesn't
typically
decrypt
ssl
anyways,
but
it
would
matter
if
you're
removing
that
from
the
list
it
sort
of
organically
is
gone
right.
So
I
think
you're
you're
good
to
go.
F
L
M
Okay,
cool
all
right:
okay,
then
we
had
the
question:
you
know
which
test
equipment
would
actually
work
and
no,
why
should
we
have
this
section
in
here?
Does
it
actually
matter
or
not?
We
have,
I
think
I
think
we've
got
a
lot
of
discussions
not
only
with
ixia
and
spirent
or
keysight
and
sparring,
but
I
think
it's
also
good
for
people
wanting
to
use
open
source
generators
and
emulators.
L
Yeah,
I
honest
to
goodness
didn't
don't
remember
so
I'll
have
to
go
back
and
look
and
then
I'll
take
note
carson,
you're
asking
hey.
Do
I
want
something
to
change
which
I'll
propose
or
if
I
wanted
it
removed?
L
I
vaguely
remember
thinking
it
was
sort
of
internal
details
that
I
didn't
think
anybody
would
give
up,
and
I
know
you
guys
responded
and
said
actually
inspiring
did,
but
I
need
to
honestly
go
back
and
read
four
three
one:
one
and
I'll
circle
back
with
the
timely
feedback.
Would
that
work.
G
Okay,
so
I
represent
the
other
test
vendor
in
this
group
and,
yes,
we
are
working
not
only
between
ourselves
and
you
know,
sharing
as
much
technical
details
as
it
can
be
to
help
the
overall
purpose
of
this
draft
you're,
also
working
with
other
test
vendors
that
that
may
not
have
the
all
the
technologies
yet,
and
you
know
we
really
welcome
any
third
parties
and
in
fact
we
are
right
now
working
on
creating
a
methodology
that
would
be
as
much
test
tool
independent
as
possible.
G
We
are
only
discussing
the
parameters
and
and
stuff
like
that
and
and
removing
any
any
vendor
specific
or
test
vendor
specific
details
out
of
it.
Okay,.
M
Perfect-
and
I
must
I
need
to
add,
like
with
my
test
lab
cat
on
that,
you
know-
sometimes-
we've
had
customers
come
to
us
and
say
like.
Why
are
your
results
like
this?
We
measured
with
the
another
test
equipment
from
another
leading
vendor
and
the
results
are
totally
different
and
we
figured
out
in
long.
A
M
Specified
in
the
test
equipment,
oh
tcp
tcp
stack
attributes.
So
so
this
section
four
three
one
one
is
about:
ccp
stack
attributes.
So
it's
about
you
know
the
window
sizes
receive
window
sizes,
delayed
x,
retries,
different
gcp
flags
and
so
on.
And
yes,
it
looks
a
little
bit
like
very
detailed
to
a
reader
that
might
stumble
upon
this.
The
first
time,
but
we
found
in
in
many
many
test
cycles
that
this
these
parameters
really
make
the
difference.
A
It's
it
would
be
great
if,
if
all
the
vendors
sort
of
gravitated
to
one
set,
that's
because
then
we
wouldn't
have
these
differences
to
worry
about
at
least
not
so
much
and
that's
kind
of
at
the
root
of
the
problem.
L
No,
I
think
well,
so
I
think
what
carson
was
trying
to.
I
think
the
goal
here
in
this
section
then
was
about
having
them
at
least
tell
you
what
the
parameters
are
so
then
to
carson's
point:
we've
all
run
into
that
scenario
where,
when
you
run
the
test
with
vendor
a
versus
vendor
b,
you
get
wildly
different
results
and
you're
like
what
is
going
on
carson.
Honestly,
that's
why
I
was
surprised
because
in
all
of
my
time,
in
working
with
several
of
the
large
main
vendors,
I
couldn't
get
that
info.
L
So
I
was
shocked
that
they
would
give
it
up,
and
I
guess
it
sort
of
begs
the
question:
would
they
give
it
up
to
the
the
you
know:
sarah's
bike
shop
and
not
just
carson,
because
he
works
at
the
ea
ntc
or
and
if
it
is,
you
know,
it
still
makes
a
question
of
hey.
Well,
what
happens
with
the
open
source
tools
where
they
may
or
may
not
have
that
info
and
just
in
general,
getting
docs
out
of
them
is
tougher,
but
either,
which
way
I
still
stand
by
hey.
L
Let
me
go
back
and
read
it
again,
get
the
thoughts
clear,
because
even
if
the
open
source
folks
didn't
have
it
or
wouldn't
give
it
to
you
or
for
some
reason
you
can
have
it
documented.
I
still
like
the
idea
of
being
able
to
have
it
to
your
point.
So
let
me
circle,
sir.
I
stand
by
hey
I'll
circle
back
with
text
for
proposals
for
9
11.
issue
911
on
the
screen.
Excuse
me.
A
Right
good.
Thank
you,
sir
yuri
a
quick
comment.
N
The
the
given
up
comment
means,
but
the
way
I
read
the
spec
is
it
just
defines
key
stack
parameters
that
you
can
implement
in
test
tools.
You
know
bday
from
my
tool
or
somebody
else
or
open
source,
and
it
just
defines
key
parameters
that
have
a
certain
outcome
to
how
tcp
behaves
and
then
the
test
tool
like
myself,
we
we
implement
those,
so
you
have
control
over
that,
but
it's
not
a
tesla
vendors
gave
up
to
be
in
the
draft
right.
It's
what
the
draft
defines
us
as
key
to
configure.
L
N
L
Yeah
yeah,
I
totally
didn't
read
it
that
way.
So
again
I'll
go
back
and
reread
and
if
it's
not
clear,
then
I'll,
I
don't
know
I'll,
propose
the
sentence
that
says
some
version
of
what
you
just
said:
I'll
steal
your
words
totally,
but
I'll
also
go
back
with
feedback.
That's
that's
really
good
clarification.
Thank
you.
A
Yeah,
that's
that's
exactly
what
I
was
trying
to
get
at
and
apparently
not
saying
it
clear
enough
that
you
know
even
testing
with
tcp
is
hard
to
begin
with,
and
if
everybody's
gonna
use
different
parameters,
it's
going
to
be
a
mess
yeah,
you
know
I
can
I
can.
I
can
feel
I
can
feel
scott
bradner's
heat
on
me
as
as
I
say
that.
M
Right
all
yeah
exactly
under
the
next,
I
think
actually
another
example
is
yeah.
We
can
move
to
the
next
one,
but
just
to
wrap
this
up.
Another
example
that
we
had
challenge
with
a
long
for
a
long
time
before
we
started.
This
draft
was
the
way
to
close
tcp
sessions
and
in
some
cases
they
were
close
in
a
kind
of
you
know
interesting
way,
just
to
increase
the
benchmarking
results
for
public
data
sheets,
and
we
discussed
this.
M
M
Right
and
that's
the
way,
it's
good
to
specify
it,
and
then
the
open
source,
open
source
contributors
and
open
source
programs
can
actually
look
at
this
and
say
this
is
the
way
they
want
to
do
it
and
they
can
implement
it
in
the
same
way.
So
I
think
it's
probably
the
easiest
for
open
source,
because
that's
normally
software
based,
whereas
some
of
the
implementations
we've
even
looked
at,
are
using
accelerator
cards,
and
sometimes
it
was
not
so
easy
to
change
these
parameters.
M
Okay
anyway,
I
think
that's
going
beyond
the
discussion
here.
So
the
next
one
is
discussion.
Sorry
documentation
of
this
configuration
and
ancillary
devices
on
both
sides.
M
We
think
that
you
know
if
we
define
that
the
testbed
must
be
well
known
and
that
the
auxiliary
switches
and
routers
should
not
have
an
impact
or
should
have
a
known
impact
to
be
subtracted
from
the
results.
Then
there
is
no
reason
to
add
a
requirement
to
to
add
the
configuration
of
these
switches
to
the
documentation.
L
So,
look
on
one
hand,
I
see
what
you're
saying.
On
the
other
hand,
I'm
not
entirely
sure
I
100
percent
trusted.
Could
it
be
negligible?
Maybe,
but
the
notion
of
at
least
where
I
was
coming
from
is
hey.
If
I'm
going
to
repeat
a
test,
I
want
to
configure
in
my
lab
exactly
what
karsten
configured
in
his
and
when
you
have
ancillary
devices.
How
routing
is
configured,
for
example,
can
matter
or
not
how
large
your
mac
tables
are
can
can
matter
or
not
so,
if
you're
using
a
physical
versus
a
virtual.
L
So
I
thought
look
rather
than
get
into
all
of
that
and
having
to
have
that
debate.
If
you
just
ask
them
to
document
what
you
use,
so
it
can
be
repeated.
Apples
to
apples
across
any
number
of
test
runs
then
you're
clean,
and
I
was
a
little
surprised.
You
guys
pushed
back
on
that
because
I
thought
it
was
just
low
hanging
fruit,
like
sure
document
it
move
on.
But
that's
that's
where
I
was
coming
from
with
that.
M
I
think
it's
not
and
from
my
point
of
view,
it
would
actually
be
a
little
bit
dangerous
or
counter
productive.
To
add
these
details,
for
example,
let's
say
at
entc
we
might
be
using
a
cisco
router,
for
example
right
and
then
we
will
put
into
our
results.
We
use
a
cisco
router
and
then
it
might
become
like
required.
For
you
know
anybody
who
wants
to
reproduce
our
tests
to
buy
a
cisco
router,
although
there
is
absolutely
no
reason
to
buy
one.
M
L
Right
but
carsten
one
way
to
handle
that
exact
example
that
you
gave
is
instead
of
saying
a
cisco
router.
You
could
say
we
used
a
router
that
held
the
number
of
for
whatever
the
test
is
right,
so
these
number
of
routes,
or
it
was
configured
for
bgp
and
ospf
or
just
so
it
whatever
you
could
cover
the
configuration
without
calling
out
the
actual
vendor.
You
see
what
I
mean.
M
L
It's
just
generic,
I
didn't
mean
to
say,
of
course,
yeah.
I
totally
understand
where
you're
coming
from.
I
I
for
sure
understand
why
you
wouldn't
want
to
say:
well
it's
a
juniper
or
cisco
or
whoever
and
now
you're
mandating
they
buy
it
but
generically
what
that
device
was
performing
so
that
I
could
use
you
cisco.
I
use
juniper.
Who
cares
right
so
long?
It
was
as
they
were
configured
with
the
same
features
and
for
the
same
functions.
I
think
that's
your
point
then
fine,
I
can
repeat
the
tests.
M
I'm
not
entirely
sure
because
we
really
tried
very
hard
to
make
this
draft
independent
of
any
specific
test
equipment
or
auxiliary
equipment,
and
we
try
to
specify
the
methodology
detailed
enough
so
that
it's
reproducible,
no
matter
what.
So.
From
from
my
understanding,
maybe
the
other
authors
can
comment,
but
from
my
understanding
the
draft
itself
should
create
a
reproducible
environment.
If
it's
properly
followed,
it's
not
necessary
to
document
additional
stuff
in
detail,
alex
is
raising
his
hand.
I
think.
A
Yeah
go
ahead,
go
ahead,
alex.
C
I
had
a
question
for
sarah.
You
when
you
said
you
know
documenting,
do
you
just
mean
you
know
hey
the
these.
Were
the
devices
in
the
test
bed
or
do
you
want
the
full
configuration
for
the
all
the
all
of
the
ancillary
devices
for.
L
Every
block
that
you
have
in
the
test
bed,
although
I
think
carson,
is
saying
it
might
already
be
there
for
every
device-
that's
configured
in
your
test
bed
specifically
outside
of
the
sut,
so
any
ancillary
you
have
to
have
a
reason
for
having
it
and
now
it
occurs
to
me.
Well,
maybe
he's
saying
yeah
well,
we
already
did
that
right.
Like
we're,
saying
look
you
need
to
have
this
many.
I
don't
know.
L
I
don't
remember
what
the
specific
examples
are,
but
if
I
pick
on
routers
like
okay,
so
you're
using
these
protocols
or
you're
using
these
number
of
macs
or
you're,
using
these
number
of
clients
or
whatever,
and
if
that's
actually
already
there,
then
I
would
retract.
I
have
to
go
back
and
take
a
look.
I
guess,
but
I
would
say
then
that
covers
it,
because
yeah
I'm
less
worried
about
having
a
specific
device
that
I
can
load
on
a
you
know:
vendor
blah's
router.
L
C
So
the
only
thing
I'm
I'm
just
concerned
is,
I
I
don't
think
we
you
know,
while
some
of
that
is
in
there,
I
don't
think
we
go
into
that
level
of
detail.
So,
let's
just
use
a
router
as
an
example.
You
know
we
may
have
a
router,
that's
that's!
In
there.
That's
just
doing
layer.
3
forwarding
has
some
static
routes
in
there,
for
whatever
the
client
and
server
networks
are,
but
we
don't
necessarily
ask
that
to
be
documented.
C
C
I
don't
think
we
are
looking
at
the
level
of
detail
to
try
and
and
document
that,
we're
assuming
that
when
we
do
the
the
pre-test
or
the
you
know,
like
the
the
you
know,
back-to-back
or
something
else,
you
know
as
long
as
everything
is
working
in
that
configuration,
adding
the
the
system
under
test
you
know
is,
is
that's
the
piece
that
we're
really
looking
at
and
to
measure
so,
hopefully,
whatever
the
characteristics
and
behavior
during
the
back-to-back
test.
Is
you
know,
that's
your
baseline
and
then
every
you
know
anything
else.
C
L
I
think
the
the
the
rub
sort
of
comes
in
like
look
well,
if
I
as
vendor
a
say
that
my
device
achieves
performance
of
x
and
then
a
customer
goes
out
and
says
well
great,
I
want
to
reproduce
that
in
my
lab,
I
think
one
of
the
goals
generally
that
we
promote
in
bmwg,
is
having
that
repeatability,
and
so
I
guess
I
was
coming
at
it
from
the
perspective
of
look
having
the
ability
for
alex
or
carsten
or
whomever
to
take
a
test
that
I
ran
in
my
lab
and
then
go
run
it
in
yours
and
ostensibly
come
out.
L
The
other
end
with
the
same
result,
provided
that
that's
how
the
device
is
behaving,
and
so
that's
why
I
was
sort
of
airing
on
the
you
know.
How
do
you
document
it?
So
that's
the
case
I
see
where
you're
going,
though
I'm
not
saying
that
hey.
This
is
something
I'm
super
married
to.
I
think
repeatability
is
you
know
a
keystone
of
bmwg,
but
if
you
guys
want
to
reject
the
feedback,
I
mean.
That's
your!
That's
your
choice
as
well.
C
Sure-
and
I
would
also
just
like
to
say
that
you
know
I
think
we
have
the
same
goals.
I
I'm
a
big
fan
of
repeatability
I'd
like
to
you
know
if
somebody
else
can
run
the
same
test
and
get
the
same
results,
but
I
know
even
with
let's
just
say
in
my
perfect
world,
where
everything
is
documented
and
two
people
have
the
exact
same
set
of
equipment
and
configurations.
They
may
not
necessarily
be
able
to
repeat
the
same
things
you
know.
C
In
both
cases,
we
want
to
make
it,
as
you
know,
get
to
that
same
point
as
much
as
possible,
but
we're
also,
as
carson
said,
trying
not
to
prescribe
people
into
a
you
have
to
have
this
set
of
equipment
in
order
to
run
these
tests,
because
that
will
lock
a
lot
of
people
out.
You
know
from
being
able
to
do
any
testing.
L
C
Good,
maybe
we
can
find
a
balance
or
I'm
not
sure
you
know
what
the
what
the
resolution
of
this
one
is.
A
We've
got
and
and
and
then
decide
how
much
more
to
add.
If
anything,
that's
the,
I
think,
that's
that's
where
we
are.
What
what
do
you
think
carson.
M
Well,
the
suggested
resolution
is
to
to
keep
things,
as
is
because,
from
from
my
point
of
view,
the
whole
the
way
the
whole
draft
is
written,
reproducibility
or
repeatability
is
achieved
by
specifying
the
methodology
well
enough
and
by
requiring
a
reference
test
to
verify
before
any
actual
benchmarking
measurement,
that
the
testbed
is
working
as
expected
right,
okay-
and
I
mean
there
is
always-
there
is
always
something
below
the
horizon
of
what
any
benchmarking
document
in
the
ietf
can
define.
For
example,
it
could
be
fiber
types
to
blame.
M
You
know
your
cables
could
be
too
long,
and
I
don't
know
what
you
know
and
and
but
but
that's
kind
of
assumed.
I
think
that
there
is
a
minimum
level
of
what
we
can
describe
and
and
above
that
we
just
need
to
make
sure
it's
very
useful,
and
then
it
doesn't
matter
whether
it's
bgp
or
ospf,
if
it's
reproducible
and
the
testbed
is
referenced
tested
before,
because
otherwise
you
know
we
would.
We
would
write
pages
and
pages
and
pages.
A
Detail
here
that
that
people
are
gonna
have
to
appreciate,
so
I
mean
okay,
so
let's
let's
leave
it
at
that
and
and
if,
if
folks
have
serious
misgivings
later,
then
I'm
sure
they'll.
Let
us
know
thanks.
M
L
Yeah,
I
I
was
literally
just
asking
you
a
question
not
making
a
critique
for
what
it's
worth,
but
the
proposed
text
here,
I
think,
is
super
clear
as
well,
so
that
works.
That's
really
nice.
Thank
you.
M
So
these
actual
application
traffic
mixes
are
a
subject
of
discussion
and
I
think
that
goes
beyond
the
draft.
So
the
draft
opens
up
the
opportunity
to
have
different
traffic
mixes
but
relevant
for
different
use,
case
scenarios
or
industries
or
whatever,
and
actually
the
netsec
open
group
is
working
on
a
couple
of
application
traffic
mixes.
So
we
will
be
able
to
provide
examples
in
the
future,
but
that's
explicitly
beyond
the
scope
of
this
document
to
actually
define
the
traffic
mixes.
L
Yeah,
I
was
actually
going
to
say
when
you
guys
came
back
to
this,
to
potentially
propose
some
examples,
but
I
think
no
matter
what
examples
never
make
everybody
happy
anyway.
So
having
one
as
a
starting
point,
is
nice
agreed
it's
not
required
and
your
point
documenting
what
the
heck
you
did.
So
you
can
repeat
it
is,
I
think,
the
key
thing
and
that's
what
you
cover.
So
I
think
I
think
you're
good
here.
N
M
Okay
cool,
so
that's
about
that
and
I
think
I
mean
on
one
hand
there
could
be,
could
be
other
discussions.
Maybe
we
should
ask
first
if,
if
there
are
any
other
comments
to
the
draft
at
this
point,.
A
All
right,
as
as
co-chair
I'll,
ask
that
question:
are
there
any
other
comments
on
on
this
version
of
the
draft?
The
next
generation
security
devices
here
09.
A
All
right:
well,
then,
we
I
think
we
we've
got
our
answer
for
today
and
we
do
have
a
few
things
to
to
resolve
offline.
So,
let's,
let's,
let's
look
to
the
availability
of
the
next
draft
to
set
our
schedule,
but
we'll
try
to
try
to
move
this
forward
as
quickly
as
we
can.
O
M
A
And
the
well,
the
I
think
the
key
thing
is
that
in
the
last
working
group
last
calls
we've
we've
you
guys.
Actually
you
guys
actually
resolved
a
lot
of
the
comments
during
the
last
call
and
and
then
sarah's
comments
came
and
and
now
we're
we're
working
on
resolving
those.
And
you
know
what
I
hear
is
a
lot
of
agreement
on
on
the
direction
that
we're
going
in
and
the
development
that's
been
done
here
and
the
and
most
importantly,
the
scope
and
the
and
the
coverage.
A
So
you
know,
I
think
that
I
think
the
working
group
is
behind
us
if
we
move
this
forward,
and
so
we
have
a
few
less
details
to
to
straighten
out
and
and
we'll
we'll
go
from
there.
A
A
B
Question,
if
I
might
so
we
do
al,
do
you
do
you
think
it
would
be
best
for
us
to
sort
of
like
resolve
all
the
items
that
aren't
quite
completely
resolved
worth
work
directly
with
sarah
then
produce
the
draft,
and
you
know
go
from
that
point
because
I
think
I
think
it's
reasonable
to
say
that
you
know
sarah's
is
provided
a
lot
of
interesting
feedback
that
you
know
I'd
like
to
see
resolved.
A
B
So
once
once
once
draft
or
draft
version
10
has
been
posted.
What
happens
then.
A
So
let
me
ask
the
working
group:
we've
got
25
people
assembled
a
couple
of
people
on
jabber
and
so
forth.
A
Today
doesn't
sound
like
it
now,
as
document
shepard,
I'm
going
to
have
to
take
a
last
look
at
it
right,
because
that's
because
that's
my
job
and
and
that
gives-
and
that
gives
people
time
to
read
it
if
they
want
and
and
and
to
say
something
if
they
must.
A
But
I
I
don't,
I
don't
foresee
the
need
for
yet
another
working
group
last
call
here:
we've
had
two
really
productive
ones
and
and
and
lots
of
really
productive
discussions
I
have
to
you
know
I
have
to
add
one
of
my
concerns
early
on
was
that
when
we
took
this
work
up
that
all
the
work
would
be
done
in
in
netsec,
open
and
we'd
be
presented
with
a
finished
project
here,
product
here
and-
and
you
know,
and
bmwg
wouldn't
be,
you
know,
able
or
or
there
wouldn't
be
much
bmwg
participation.
A
Well,
that
hasn't
happened.
The
in
fact
the
what
we
want
to
happen
happened,
and
that
is
that
that
the
working
group
participants
have
contributed
to
this
extensively.
So
it's
a
it's
a.
I
think
it's
a
great
product
of
of
the
working
groups,
effort
with
a
great
start,
all
along
the
way,
with
new
features
at
various
points
from
the
netsec
open
group,
and
you
know
we
we
shouldn't
hold
on
to
this
any
longer.
We
should
get
the
next
steps
going.
B
What
are
the
next
steps
said
once
once
once
version,
10
has
been
posted
and
people
are
everyone's,
provided
that
last
little
bit
of
speak
now
or
forever
hold
hold
your
peace.
What
what
happens
then
so.
A
And-
and
I
may
I
may
come
back
with
comments
related
to
you-
know,
nits
checking
and
some
other
some
other
things
like
that.
You
know
like
that,
update.
I'm
sorry,
obsoletes
3511
point
that
I
made
early
on
in
the
working
group
status
and,
and
I
mean,
and
we
and
we
probably
need
to
fix
those
before
we
hand
it
to
warren
for
his
area,
director
review
and
one
more
and
warren's
gonna
review.
A
This
and
warren
is
very
likely
to
gonna,
have
some
comments
and
and
areas
where
we
can
improve
the
draft
before
it
goes
to
the
next
step,
which
would
be
ietf
last
call,
and
then
once
we've
resolved
the
last
call
comments,
and
during
the
last
ietf
last
call
we
usually
get
the
directorate
reviews
from
from
places
like
the
general
area,
review,
team
security,
review
team,
and
you
know
transport
area
and
and
others
that
you
know
try
to
help
us
out
and
then
and
then
it
goes
to
the
iesg.
A
The
internet
engineering
steering
group
where
more
area
directors
like
warren
will
get
their
first
chance
to
read
it
and
and
so
expect.
Some
comments
at
that
level
expect
some
more
work
to
do
at
that
level.
I
expect
some
of
it
to
come
at
the
last
minute
because
I've
still
got
you
know
bricks
that
hit
you
know
bricks
on
the
floor
here
that
hit
my
head
hours
before
something
was
supposed
to
be
approved
and
then
spent
a
good
amount
of
time
dealing
with
them.
A
So
so
it's
a
I
mean
it's,
it's
a
it's
an
approval
process
with
many
steps
and
that's
why
I'm
trying
to
get
us
to
the
point
where
we
can
or
we
can
move
this
along,
and
I
realized
that
I'm
in
the
critical
path
for
the
next
step.
A
B
A
Well,
there's
some
fixed
times
in
our
time
intervals
in
the
process
like
the
like
the
ietf
last
call
that
usually
takes
two
or
three
weeks,
something
like
that
right
and
then
the
and
then
there's
you
know
some
uncertainty
about
the
amount
of
time
to
resolve
the
comments
and
we're
we're
about
to
hit
the
august
august-ish
vacation
period
where
people
are
going
to
disappear
for
weeks
at
a
time.
A
But
you
know
I'm
I'm
really
hopeful
that
we
could
get
this
through
last
call
and
in
front
of
the
iesg,
let's
say
like
sometime
in
october
and
then
and
then
and
then
it
really
pays.
It
really
pays
off
for
the
authors
to
be
really
attentive
to
the
area
director
comments
that
come
at
that
point
and
resolve
them
with
the
area
directors
as
quickly
as
possible
and
and
then
you.
A
A
then
you
can
reach
an
approval
point.
Once
it's
been
on
the
iesg
telechat
agenda,
you
can
reach
an
approval
point
very
quickly.
After
the
the
telechat
you
just
start
putting
stuff
into
working
text
propose
resolutions,
you
know
we're
we're
not
going
to
get
into
the
we're
not
going
to
get
any
show
stoppers.
At
that
point,
it's
it's
really
about
it's
really
about
dealing
with
things
that
the
area
directors
read.
A
That
was
unclear
things
that
they
they
thought
of,
that
we
haven't
thought
of
and
and
I'll
I'll
now
turn
it
over
to
warren.
To
give
his
perspective.
E
I
was
just
going
to
add,
especially
once
it
reaches
80
review
stage.
You
know,
don't
be
afraid
to
send
reminders
and
promptings
of
like
yo
get
get
a
move
on
get
this
done.
So
that's
something
I'm
very
good.
At
okay,
great.
L
The
the
other
thing
I'd
add
hey
if
I
put
on
my
rsaw
cap
for
a
sec,
is
when
you
guys
do
get
the
off
48.
So
after
warren
signs
off
isg
is
good
to
go
once
you
guys
are
in
the
queue.
You
have
a
long
list
of
authors
and
I
think
we
still
the
the
rfc
editors
still
want
to
hear
or
excuse
me,
the
oh
gosh,
not
heather.
Oh
warren,
keep
me
honest.
What
do
we
call
sandy
and
alice.
L
Thank
you.
So
I
had
it
right.
So
look
they're
looking
to
make
sure
everybody
agrees,
so
make
sure
that
everybody
is
attentive
and
when
they're
sending
hey,
are
you
guys
good
or
hey?
Do
you
agree
or
those
those
that
type
of
language
as
long
as
you
guys
are
responding
to
that
pretty
quickly?
It'll
help
auth
48
go
faster,
because
I
think
you
know
the
typical
stage
for
a
draft
this
size.
I
don't
think
it
has
any
contingencies
here.
L
E
L
That's
why
I
was
saying
it
it's
it's
technically
speaking
that
entire
process
can
take
12
but
yeah,
or
I
think
75
percent
of
our
rc's
fall
into
that,
but
just
making
sure
hey
be
attentive,
it'll
go
as
fast
as
it
can.
I
think
we're
saying
the
same
thing.
F
Indication
is
to
make
your
document
as
rfc
editor
ready,
as
you
can,
because
the
more
opportunities
you
make
for
the
rfc
editor
to
make
changes
the
more
the
greater
the
possibility
that
what
they
change
may
not
be
what
you
intend.
That's
just
my
humble
opinion.
B
F
So
we'll
we'll
definitely
be.
B
Leaning
on
the
folks
that
are
on
the
call
for
help
in
that
definition
of
what
rfc
editor
ready
actually
means.
So
this
is
our
first
kick
at
the
can,
I
think
so.
L
So
al
is
really
good
at
that
he
talked
about
making
sure
your
knits
are
clean.
You
have
a
really
good
document
shepard,
so
I'm
pretty
sure
when
you
guys
come
at
the
end
of
that
process,
you'll
be
relatively
good
good
to
go
there.
The
english
is
pretty
good.
The
grammar
is
pretty
good
already
so
yeah
and
if
you
have
questions,
ask
us
we're
happy
to
help
great
thank.
A
You
yeah
and
but
but
I'll
also
tell
you
that
you
know
the
one
of
the
drafts
that
I
just
wrote,
that
has
37
pages
ended
up
with
37
questions
from
the
rfc
editors
and
and
the
last.
A
Parts,
a
through
g
and
it
took
about
six
hours
to
deal
with,
so
you
can
never.
You
can
never
know
exactly
what
they're
going
to
ask
and
and
but
you
know
it's
all
about
improving
the
document
so
take
it
with.
B
A
H
A
Okay,
we
doctor,
we
notice.
A
Okay,
thank
you.
Thank
you,
too,
for
the
excellent,
slides
and
leading
a
really
productive
discussion
today.
A
B
I
thank
thanks
to
carson
and
battle
for
helping
pull
this
together.
A
Absolutely
all
right
so
we're
right
on
time,
and
that
means
that
I'm
going
to
move
us
over
to
let's
see
where
is
it
yeah
interconnect,
tester
management
and
it's
this
one
and
I
think
we're
we're
ready
to
go.
If,
if
you
are
oh
where'd,
he
go
hi.
Oh
vladimir,
yes,
yes,
yeah!
Please,
please!
Please,
go
ahead
and
and
I'll
try
to
keep
up
all
right.
J
Well,
we
have
worked
on
the
the
draft
during
the
the
hackathon
as
usual,
so
this
is
the
the
front
page
and
if
you
you
can
move
to
the
next
one
where
there
is
actual
content
right
donate
to
next
slide
yeah,
so
the
project
has
been
presented
already.
This
is
the
state
we
have
a
new
new
version
of
the
draft
we
have.
J
J
So
this
is
our
proof
of
concept
and
we
have
the
device
which
is
a
hardware
device
with
the
necessary
server
side,
instrumentation
code,
we
use
fpga
to
do
a
hard,
real-time
traffic
generation
and
timing,
and
we
have
some
if
you're
interested
in
the
hardware.
There
are
these
two
links
that
you
can
check
out
about
how
we
build
it.
So
it's
if
you
go
to
the
next
slide
yeah.
J
This
is
what
it
looks
like
when
it
is
assembled.
This
is
what
we
used
during
the
hackathon,
and
this
time
we
we
had
two
setups
one
is
using
the
same
tester
and
doesn't
doesn't
need
gps
time
synchronization.
J
J
Which
deals
with
that,
so
this
is
the
architecture
of
the
implementation
which
we
have
presented.
We
just
augmented
that
with
this
sync
pps,
which
now
we
actively
use
to
be
able
to
synchronize
the
real
time,
clock,
module
and
next
slide,
please,
okay,
so
this
is
what
got
done.
Actually,
the
most
important
is
that
we
completed
the
rfc
2544
script
and
validated
the
binary
search
algorithm.
So
now
you
have
a
complete
command
line,
two
that
connects
to
these
devices
and
performs
performs
the
test
with
the
report
and
the
other
things
are
improved:
gps,
real-time
code
synchronization.
J
There
we
worked
on
the
hardware
implementation,
doing
them
some
disciplining
of
the
real-time
clock
and
we
use
some
of
the
the
rfcs
that
we
deal
with
that
to
to
find
the
right
way
to
do
it
yeah
and
we
we.
We
still
have
public
access
to
this
net
conf
devices,
so
different
vendors
can
actually
connect
to
them
and
validate
the
the
net
config
young
implementation.
J
Yes,
so
I
did
post
generated
reports
some
time
ago
on
the
mailing
list,
and
this
is
how
how
it
looks
like
now,
this
probably
too
little
to
see,
but
those
who
are
interested
then
can
open
the
document
itself.
J
You
can
see
the
design,
even
if
you
are
not
familiar
with
junk
and
don't
use
it
actively.
The
command
line
tool
has
a
corresponding
parameter
list
to
to
what
actually
gets
into
the
model
and
the
actual
source
code
was
was
listed
previously.
So
this
script,
rfc
2544.python,
is
a
single
script
that
does
the
entire
test.
So
it's
very
simple
to
use
as
a
reference
designed
for
proving
the
concept,
and
this
is
the
the
report
you
can
see.
J
The
throughput
section,
the
latency
framework
rate
back
to
back
frames,
and
one
thing
that
is
interesting
is
like
we
try
to
make
the
minimum
minimum
amount
of
time
per
trade,
which
is
commanded
by
the
rfc
and
the
minimum
amount
of
trails.
So
if
you
look
at
this
report
and
at
the
skipped
trails,
it's
exactly
41
trails
that
a
test
running
on
a
loopback
connection
of
one
gigabit
takes.
J
So
this
is
kind
of
a
private
case,
of
course,
but
it's
interesting
to
take
it
into
consideration
when
considering
rfcs
that
optimize
that
process.
So
if
we
take
that-
and
we
apply
this
mlr
search
algorithm,
how
many
trails
are
there
going
to
be?
So
that's
what
I'm
going
to
use
when
I
do
the
review
of
that
draft
I'll
try
to
apply
that
to
our
tool.
So,
but
I
think
it's
a
good
starting
point
for
a
discussion.
J
A
J
A
Here
that
that
I
don't
completely
understand
is
the
result
under
back
to
back
frames.
What
can
you
tell
me
a
little
more
about
that.
J
Okay,
now
there
are
two
choices
like
starting
from
the
least
possible
amount
of
back
to
back
frames
or
from
the
maximum,
so
we
have
chosen
the
minimum.
So
first
trail
is
with
two
back
to
back
frames.
Then
you
have
four
and
then
you
end
up
with
a
full
second
of
back
to
back
frames
where
there
is
a
limit,
so
the
test
doesn't
work
for
back
to
back
frames
burst
longer
than
one.
Second,
that's
the
only
limitation
that
is
kind
of
not
into
the
rfc.
J
That's
something
that
you
have
to
come
up
in
your
implementation,
but
we
use
one
second
and
the
maximum
back
to
back
frames.
You
can
put
in
one
second
on
a
one
gig
internet
interface
is
that
number.
So
it
goes
up
to
21
trails
and
figures
that
so
it
stops
there.
If
there
was
something
that
drops
frames
it
earlier
point,
then
it
would
go
into
binary,
search
and
find
the
exact
number.
A
J
It
is
the
running
test
that
is
printing
it,
so
we
have
taken
one.
Second,
that's
the
only
thing
that
is
like
predefined
without
being
part
of
the
rc,
okay,
okay,
yeah.
J
J
J
A
A
A
Yeah
more
more
complex
than
that,
because
you
know
this
was
this
was
what
they
could
do
in
the
90s.
A
And-
and
it's
it's
better
if
I
mean
a
lot
of
a
lot
of
the
current
generators
claim
to
measure
delay
on
on
every
frame,
some
of
the
software
generators
measure
delay
on
a
sample
of
frames,
but
in
any
case
they
all
try
to.
A
They
all
try
to
characterize
the
the
min
and
the
max
and
the
average
of
a
more
complete
distribution
of
the
frames
so
they're
looking
at,
I
mean
they're
looking
at
a
lot
more
frames
than
just
one
in
the
middle
of
the
test.
Let's,
let's
just
leave
it
at
that.
J
Yeah,
that's
actually
easier
to
implement
with
that
more
complex
detection
than
the
one
that
is
prescribed
because
there
you
have
to
detect
exactly
a
frame
at
a
certain
point,
but
it's
interesting
to
mention,
because
we
took
that
as
like
it
was.
I
thought
we
should
be
able
to
do
exactly
what
the
classical
rfc
recommends.
So
it
was
just
a
walk
of
functionality
in
the
model
that
was
causing
that
and
we
need
we
needed
to
have
a
like
time.
J
Stumps
frames
to
be
able
to
do
that
unless
we
had
something
that
allows
us
to
capture
particular
frames
and
we
added
that
to
the
model.
Actually,
this
is
the
part
we
added
to
the
configuration
in
the
new
model
in
the
latest
release.
So
if
you
go
to
the
next
slide,
there
is
no
more
questions
for
this
one.
J
If
you
should
stop
or
if
you
should
overwrite
that
so
this
functionality
every
test,
every
tester
has,
and
we
added
that
to
the
model.
So
now
we
actually
are
able
to
implement
the
rfc
like
it
was
implemented,
but
it
was
defined
with
a
single
frame
specified
at
a
certain
offset.
We
can
capture
that
and
we
can
compare
the
timestamps,
so
we
haven't
done
the
implementation
yet,
but
something
to
do
on
the
next
category:
okay,
okay,
yeah!
So
next
slide.
Please.
J
So
now
the
question
is:
should
should
we
start
working
on
a
more
complex
filter
configuration
model,
because
at
the
moment
we
have
left
this
on
the
site,
and
we
probably
should
do
that.
One
thing
is
to
consider
if
we
should
add
it
success
control
list,
which
is
the
itf
specification
for
filtering,
and
the
other
option
is
to
use
bit
fields.
So
that's
just
mentioning
the
the
options
probably
will
add,
like
a
young
construct
called
the
choice,
so
you
can
actually
implement
the
model
with
both
of
them
or
either
of
them.
So.
A
Cool
all
right,
well,
any
any
feedback
for.
A
Vladimir
right,
well,
you
know
just
as
as
a
as
a
working
group
chair
here
vladimir,
I
wanted
to
wanted
to
recognize
that
your
you
know
your
progress
with
this
has
been
continuous
and
prolific
and
you've
caught
the
interest
of
several
people,
providing
comments
on
the
list
and
and
so
forth.
A
I
I
think
that
we
might
want
to
take
this
up
as
a
as
a
working
group
draft,
but
is
there
any
feedback
on
on
that,
particularly
I'm
looking
to
my
co-chair
here,
but
but
anyone
else
who
wants
to
comment
at
this
point.
Do
you
think
we're
at
the
stage
where
we
could
adopt
this
draft
as
a
as
a
working
group
item.
L
Yeah,
I
definitely
agree
with
that
and
I
also
appreciate
the
the
continued
progress
and
very
quick
progress
actually
on
the
draft.
So
thank
you
for
that.
Vladimir.
A
Well
then,
well
that
then
that'll
be
an
action
item
that
that
we
take
going
out
of
this
meeting
is
to
start
working
group.
Adoption
call
on.
A
And
hopefully
we'll
we'll,
you
know
it'll,
it's
gonna,
be
it's
gonna,
be
something
that
happens
in
august,
but
hopefully
we'll
we'll
get
plenty
of
interest
and
enough
people
kind
of
signing
up
to
review
the
draft
and
consider
some
of
the
other
things
that
you're
thinking
about
here
and-
and
I
think
I
think,
quick
feedback
is:
let's,
let's
keep
it
kind
of
keep
it
minimal,
and
then
you
know
we
can
always
do
a
a
secondary
version
of
it
vlad
later
to
to
add
some
more
material.
J
A
Very
good:
well,
it's
a
it's.
A
very
synergistic
relationship,
then
awesome
all
right.
Well,
thanks
again
for
your
presentation
and
all
your
good
work
today
and
next
up
is
gabor
and
gabor.
You've
got
15
minutes
and
about
15
slides.
If
I
remember
correctly
14,
it
looks
like
so.
Let's
keep
that
in
mind.
H
H
Thank
you.
So
the
first
question
is:
why
do
we
need
a
new
measurement
method?
As
you
know,
we
have
rsv
2544
and
rfc
5180
as
a
general
benchmarking
methodology
for
network
interconnect
devices
and
for
ipv
version,
6
specific
things,
and
we
also
have
isd
8219,
which
addressed
the
ip
version,
6
generation
technologies
where
stateful
n864
belongs
to.
So
why
do
we
need
other
methodology?
H
This
is
because
none
of
them
discussed
how
to
apply
the
rfc
48,
14,
7
port
numbers
to
any
stateful
n80
gateways.
N80
safari
can
be
the
very
old
traditional
net,
4
4,
or
an
864,
or
even
something
else.
So
I
would
like
to.
Could
you
go
to
the
next
slide?
Please.
H
Thank
you
so
first,
I
would
like
to
discuss
why
it
is
a
hard
problem
to
use
cellular
random
port
numbers
with
stateful
n80
gateways,
and
then
I
recommend
some
solutions,
including
the
test,
setup
and
terminology
and
the
two-phase
measurement
method
and
its
limitations.
Could
you
go
to
the
next
slide?
Please.
H
Thank
you.
So
why
is
it
a
problem
to
use
special
random
port
numbers?
Let's
imagine
that
we
have
a
very
old
traditional
nad44,
let's
say
a
home
gateway
or
a
carrier-grade
net
device
doesn't
matter,
so
it
has
one
private
side
and
one
public
side
and
in
the
private
to
public
direction.
It
is
no
problem
that
we
can
use
any
source
and
destination
port
numbers.
H
However,
if
we
consider
the
size
of
these
ranges-
and
we
multiply
the
two
sizes-
we
got
two
three
billion
more
than
three
billion,
and
it
means
that
if
we
have
so
many
source
for
destination
number
combinations,
then-
and
we
use
them
as
just
random
numbers-
we
will
exhaust
the
connection
tracking
table.
So
it
will
be
denier
of
service
attack
agents
the
device.
So
we
may
not
do
that
and,
of
course,
in
the
other
direction.
H
It's
triviality,
it's
a
problem,
because
if
you
just
invent
any
random
port
numbers,
the
stateful
net
devices
drop
the
packets
because
they
do
not
belong
to
any
existing
connection.
But
we
will
overcome
this
issue
too.
So
could
you
go
to
the
next
slide?
Please.
H
Yes,
here's
a
test
setup,
so
this
methodology
with
any
ip
versions,
but
just
for
simplicity,
we
use
the
traditional
nat4.
For
so
on
the
left
side,
you
can
see
some
private
ip
version
for
addresses
and
on
the
right
side,
you
can
see
some
public
ip
version
for
addresses
and
you
can
see
two
devices,
the
tester
and
the
device
under
test,
and
the
test
has
two
ports
and
the
two
ports
have
different
roles.
One
of
them
be
named
initiator
and
other
the
named
responder.
H
H
And
of
course,
as
we
do,
black
box
testing
its
size
policy
and
content
is
not
known
for
the
tester
and,
and
the
fourth
upper
mean
the
four
numbers
that
identify
a
connection.
H
They
are
source
ip
address
and
destination,
ip
address
and
source
port
number
and
destination
port
number,
and
of
course
it
would
be
a
five
table,
including
the
protocol
tcp
or
udp,
but
we
always
use
udp
and
at
the
end
we
discuss
the
difference
from
tcp
and,
of
course,
as
I
mentioned
before,
and
the
tester
has
two
sides
and
on
the
responded
side,
there's
a
state
table
and
that
when
the
tester
receives
frames,
it
extracts
the
four
tuple
for
each
receive
frame
and
as
the
source,
the
four
toppers
it
stays
stable.
H
Yes!
So,
as
I
mentioned
in
the
initial
the
time
the
tester
has
to
let's
say
ports
or
or
sides,
one
of
them
is
called
initiator.
I
don't
know
what
is
called
responder
and
the
initiator
is
the
part
of
the
tester
that
may
initiate
a
connection
through
the
state
for
device
under
test
in
the
private
to
public
direction,
and
the
responder
is
the
other
part.
Is
that
the
portal
or
ends
where
you
can
connect
the
the
device
under
test
and
it
may
not
initiate
a
connection
through
the
stator
device
on
the
test?
H
In
the
public
to
private
direction,
but
it
may
send
frames
that
belong
to
an
existing
connection
and
to
achieve
this,
it
uses
the
four
tappers
from
the
previously
exhausted,
extracted
frames
and
use
ip
addresses
and
port
numbers
from
there.
Of
course,
it's
selected
randomly
from
stage
table.
Could
you
go
to
the
next
slide?
Yes,
thank
you.
H
So
we
defined
a
new
test
phase.
Of
course,
the
real
test
phase
is
the
one
which
we
use
for
the
actual
test.
It
can
be
throughput
or
latency
on
any
other
test,
and
before
the
rear
test
phase,
we
use
a
so-called
preliminary
test
phase.
We
introduced
this
test
phase
to
support
the
stateful
test.
So
in
this
primary
phase,
the
test
frames
are
sent
only
by
the
initiator
to
the
responder
through
the
device
under
test
to
fill
both
the
connection
tracking
table
of
the
device
under
test
and
the
state
table
of
the
responder.
H
Slide,
thank
you
so
to
overcome
the
problem
of
the
dna
service
attack.
We
believe
that
the
initiator
should
use
restricted
ranges
for
source
and
destination
port
numbers,
but
not
in
asymmetric
way.
The
size
of
the
source
port
number
range
should
be
somewhat
larger
and
the
size
of
the
destination
port
number
h
should
be
smaller.
H
I
can
explain
the
reasons,
but
they
are
details.
I
don't
go
into
details
and
the
product
of
the
two
sizes.
Two
ranges:
the
size
of
the
two
ranges
is
a
parameter,
and
the
performance
of
the
states
for
an
atxy
gateway
may
be
examined
as
a
function
of
this
parameter.
Could
you
go
to
the
next
slide?
H
Yes,
so
this
is
the
primary
test
phase,
so
it
serves
two
purposes:
it
fears
the
connection
tracking
table
of
the
device
under
test,
and
it
is
important
because
it
we
expect
that
its
maximum
connection
establishment
lead
may
be
lower
than
its
maximum
frame
forwarding.
That
is
throughput.
So
we
have
to
feel
it
before
we
can
test
the
throughput
and,
of
course
we
have
to.
H
So
these
it
means
that
the
primary
frame,
the
primary
test
phase,
is
always
necessary
before
the
rear
test
field.
However,
it
can
be
used
without
the
uterus
phase
to
measure
the
maximum
connection
instead
establishment
rate.
The
next
slide
will
be
about
that
no
yeah
yeah.
Yes,
so
it's
a
separate
measurement
which
is
worth
doing
and
publishing
it's
it's
a
result
because
it's
it's
an
interesting
characteristic
feature
of
a
stateful
gateway.
H
The
maximum
connection,
establishment
rate
is
is
interesting
by
itself
and
this
determination
is
necessary
for
a
safe
execution
of
the
of
the
parameters
phase
when
we
use
it
before
the
real
test
phase,
because
we
have
to
fill
in
the
tables,
and
we
have
to
know
that
what
rate
we
can
do
that
and
its
measurement
procedure
is
similar
to
that
of
the
throughput
measurement
procedure.
We
define
it
in
detail
in
section
4.4.
H
Yes,
and
the
latest
phase
is
when
we
do
the
actual
test,
but
it
must
be
preceded
by
the
parameters
phase
during
which
the
frame
rate
is
not
higher
than
the
maximum
connection
established
madrid
and
then
all
the
tables
are
filled
and
then
may
come
the
real
test
and
we
have
given
some
details
about
it
in
section
4.5.
H
Yes,
of
course,
we
have
given
all
the
details
of
the
measurement
methods
in
the
draft
and,
of
course,
we
also
give
the
limitations.
The
most
important
innovations,
I
think,
is
that
the
recommended
usage
of
udp.
We
do
it
because
all
rfcs
which
we
built
upon
use
udp
for
its
simplicity.
H
H
So
it
also
means
that
if
we
do
some
benchmarking
measurements
using
udp,
it
may
happen
that
our
results
do
not
perfectly
characterize
or
don't
very
much
characterize
the
behavior
of
nat
xy
gateways
when
they
are
used
for
forwarding
intent
traffic
because
on
the
internet
traffic
many
times
you
can
see
udp,
for
example,
for
for
for
for
carrying
http
or
something
like
that.
H
And,
of
course,
if
it's
necessary,
we
can
adjust,
for
example,
timeout
values.
But
we
have
to
be
careful
with
that
and,
of
course,
if
we
would
like
to
to
examine
highlander
protocols
very
com,
we
recommend
the
benchmarking
working
group
working
item
about
network
with
an
edge
and
firewalls
which
we
have
discussed
today.
H
So
it
was
what
I
wanted
to
say
just.
I
think
we
have
a
more
slide
yes,
so
all
the
text
is
written
in
the
draft
and
we
have
one
partial
implementation.
H
In
fact,
the
implementation
supports
all
what
I
was
talking
about.
The
documentation
is
in
this
paper.
It
was
only
partial
because
it's
what
was
written
a
few
months
ago
and
after
that
I
I
implemented
some
some
other
parts
committed
last
week,
the
the
part
which
which
solves
the
problem
of
unique
random,
so
unique,
south
random
port
number
combinations,
which
is
necessary,
but
it
was
not
included
in
that
paper
yet.
H
So
this
is
what
I
wanted
to
say,
and
thank
you
very
much
for
for
listening
and
my
I
would
like
to
ask
you
to
to
comment
and
and
say
your
opinion,
if,
if
you
find
it
useful,
meaningful
or
whatever
you,
you
recommend
to
to
change
or
add,
so,
thank
you
very
much.
A
Well,
thank
you,
gabor.
That
was
a
a
very,
very
complete
presentation.
We
have
a
few
minutes
here.
Does.
Does
anyone
have
any
comments
or
or
feedback
they
they'd
like
to
give
gabor.
A
Well
then
I
I
know
there
was
some
feedback
on
the
list
about
this
topic
right,
gabor.
H
Yes,
there
were
some
some
some
answers
and
thank
you
very
much
for
those
who
who
answered-
and
I
would
like
to
get
more
of
course,
right
right.
A
So
I
I
think
it's
probably
going
to
be
more
productive
for
us
to
to
talk
about
this
a
little
bit
more
on
the
list
and
try
to
try
to
get
some
try
to
get
some
reviews
on
on.
Maybe
this
version
or
the
next
version
of
the
draft
and
then
we'll
take
it
from
there.
A
H
Yes,
so
there
were
some
comments
and
we
answered
and
on
the
basis
of
the
comments,
I
would
like
to
to
submit
a
new
version,
maybe
by
the
end
of
august,
very
good.
A
Very
good
yeah.
I
noticed
that
that
was
one
of
the
things
that
I
didn't
see
included
in
your
in
your
slides
today.
A
In
other
words,
some
synopsis
of
the
of
the
discussion
that
you
had
on
the
list,
but
perhaps
perhaps
you
could
just
do
that
in
a
quick
in
a
quick
email
to
the
list
and
and
then
we'll
have
that
information
in
our
records,
and
then
you
know
we'll
we'll
we'll
see
we'll
see
what
the
interest
level
is
in
the
next
version
of
this
okay.
Thank
you
very
good.
A
All
right
well
we're
down
to
our
last
20
seconds,
so
I
want
to
thank
everybody
for
attending
and
participating
today.
It
was
a
good
meeting
and
lots
of
good
preparation
and
good
development
results
in
that.
The
thing
that
will
make
it
better
is
is
when
we've
got
more
people
reading
the
drafts
that
are
new
and
so
forth.
So
anything
anything
more
to
add.
Sarah.
A
Very
good,
and
thanks
for
all
thanks
for
attending
everybody,
that's
what
makes
it
worthwhile
bill
thanks.
So
much
for
taking
notes
today,
much.