►
From YouTube: IETF112-BMWG-20211108-1600
Description
BMWG meeting session at IETF112
2021/11/08 1600
https://datatracker.ietf.org/meeting/112/proceedings/
A
Okay,
everybody
it's
the
top
of
the
hour
and
I
think
we'll
get
going
here.
I'm
al
morton,
I'm
one
of
the
co-chairs
of
the
benchmarking
methodology
working
group
sandra
banks-
I'm
sorry,
sarah
banks
will
be
will
be
joining
us
in
about
15
minutes.
She's
got
another
engagement
early
this
morning
in
pacific
time
that
she,
I
think,
couldn't
avoid.
So,
let's,
let's
begin
everybody
got
here
on
time,
including
all
the
the
folks
whose
clocks
changed
on
sunday.
A
So
that's
good
and
I
see
we've
got
about
18
people
here
right
now,
which
is
another
good
thing.
So,
let's
so,
let's
proceed
if
you're
not
thus
currently
subscribed
to
the
bmw
g
mailing
list
and
would
like
to
be
please
go
to
the
link
on
the
first
slide
there
and
you
can
subscribe
all
right.
A
So
it's
very
early
in
the
week,
and
that
means
we
need
to
go
through
the
the
note
well
fairly
carefully,
so
that
folks
understand
that
as
we
go
into
the
into
the
meeting
and
also
we
you
know
folks
will
see
this
all
as
we
work
along.
A
A
You
know
some
very
spirited
discussions
shall
we
say
and
we
we
need
to
follow
our
code
of
conduct
and
and
and
and
it
boils
down
to
this,
so
I
hope
that
you
know
we'll
just
continue
on
as
as
we
have
been
in
our
friendly
and
and
a
working
group
where,
where
folks
recognize
that
you
know
the
technical
arguments
are,
are
what
we're
trying
to
make
here
and
the
best
technical
argument
wins
the
day,
all
right,
so
you
by
participating
in
the
itf,
you
agree
to
abide
by
its
policies
and
processes
and
any
contribution
you
make.
A
It
is
covered
by
the
patent
policy
of
the
itf,
and
we
ask
that
you
acknowledge
the
patents
that
might
be
part
of
the
work
or
or
or
any
contribution
you
make.
Contributions
are,
are
basically
anything
other
than
sitting
on
your
hands
and
saying
nothing.
It's
comments.
You
make
in
the
meetings
email
you
send
to
the
lists
comments
on
the
chat.
A
If
you
help
out
with
the
minutes,
that
would
be
a
contribution
as
well
and
it's
all
covered
by
our
policies.
And
if
you
have
any
questions
about
these,
you
can
talk
to
me
or
our
ad
advisor,
who
is
warren.
Warren
kumari
warren
warns
here
and
and
also
if
you,
if
you
love
to
read
about
all
of
these
things
in
the
ietf
best
current
practices,
there's
a
whole
list
of
them
here
that
you
can
check
out
so.
A
But
here's
here's
what
we've
got
the
working
group
status,
we'll
we'll
mostly
talk
about
the
evpn
draft,
then
we've
got
a
couple
of
working
group
drafts
to
talk
about.
We've
got
a
working
group.
Adoption
call
which
went
I'll
say
differently
than
other
adoption
calls
that
we've
had,
and
we
definitely
need
to
talk
about
that,
which
is
why
I
didn't
compose
a
message
to
the
list.
A
Further
proposals:
we've
got
the
benchmarking
methodology
for
stateful
nat
xy
gateways
using
the
hash
and
stuffing
pseudorandom
part
numbers.
That
was
one
of
my
favorite
drafts
and
rfcs
in
this
group
and
and
a
brand
new.
What
I
would
call
a
considerations
draft
it's
in
our
history.
It's
titled
the
problems
and
requirements
of
evaluation
methodology
for
integrated
space
and
terrestrial
networks.
That's
biting
off
a
lot
and
but
we're
going
to
hear
about
it
today
in
the
time
available.
A
All
right,
I
see
no
requests
for
the
floor,
can
can
somebody
help
me
out
with
the
note
taking
today
and
and
of
course,
the
way
you
do
that
is
with
the
note
taking
tool
and
the
note-taking
tool
has
already
been
populated
with
the
agenda
and
the
various
topics
we're
going
to
talk
about,
so
it
would
actually
be
very
easy
to
do
it's
just
something
that
you
have
to
capture
our
agreements
at
the
end
of
discussion.
A
If
there's
a
big
question
and
a
big
answer
that
follows,
then,
would
be
nice
to
capture
that
as
well,
but
otherwise
we'll
do
the
best
we
can
here
and,
as
I
said,
sarah
is
going
to
join
us
a
little
bit
late.
Can
I
can
I
get
a
volunteer
to
help
out
with
the
minutes.
A
All
right:
well,
I
hope
I'm
gonna
ask
that
question
again
shortly
and
hope
that
we
can
move
along
here
a
bit.
So
basically,
our
brief
benchmarking
working
group
status
is
the
ethernet
vpn
draft
that
went
up
to
iesg
review
quite
some
time
ago
received
five
disgust
ballots
which
are
blocking
ballots
and
they
need
to
be
resolved
by
the
authors.
A
One
of
the
authors
basically
said
that
that
they
were
working
on
resolving
the
comments,
but
then
he
was
looking
for
his
co-author
to
agree
and
and
then
not
much
happened.
So
warren
imposed
a
deadline
of
october
31st,
where
where
we
would
declare
the
document
dead,
if
there
was
no
progress
and
unfortunately
october
31st
came
and
went,
and
we
have
seen
no
progress
on
this
so
warren,
I'm
gonna
ask
you
to
sort
of
step
forward
here
and
looks
like
you're
joining
the
queue.
B
Yep
the
queue
is
under
the
list
of
under
the
participant
list,
which
is
to
the
right.
Oh.
B
Well,
yeah,
I
mean
I
don't
really
want
the
document
to
die,
but
we've
been
waiting
on
the
authors
for
a
really
really
long
time
and
I
can't
seem
to
get
a
hold
of
them.
I've
sent
them
a
bunch
of
mails
both
through
the
data
tracker
and
directly
and
similar.
B
So
I'm
not
sure
if
the
working
group
wants
to
add
some
additional
authors
who
might
be
willing
to
help
deal
with
the
discussed
positions
or
what
I
mean.
I'm
okay,
even
in
its
current
state,
a
bit
longer.
But
I
would
really
really
really
like
to
see
it
moving
along
just
because
it
shows
up
as
a
big
red
thing
and
data
tracker
keeps
sending
me
grumpy
emails.
A
Yeah,
I
think
that's
I
mean
to
cover
all
the
bases.
I
I
like
your
suggestion
of
asking
if
there's
anyone
else
in
the
working
group
who
would
like
to
pick
this
pick
this
work
up
and
and
deal
with
the
disgust
ballots.
A
I
know
that
you
know
one
of
the
one
of
the
authors
has
lost
his
place
and
I
saw
another
announcement
too.
That
led
me
to
believe
that
maybe
maybe
the
second
author
had
the
same
problem
so
we'll
we'll
have
to
see
whether
whether
anyone
is
willing
to
take
this
up
and
yeah.
I
I
propose
when
sarah
rejoins
us
or
joins
us,
maybe
at
the
end
of
the
meeting.
Let's
let's
revisit
this
and
and
discuss
it,
then
oh
sarah's,
here
good
hi,
sarah.
B
B
I
mean
I'd
sort
of
said
by
october
31st,
if
nothing
happens,
is
going
to
going
to
market
dead
and
we
haven't
managed
to
reach
the
authors,
but
I
really
really
really
don't
want
to
market
dead.
So
at
the
moment,
I
think
we're
trying
to
see
if
there's
anybody
else
who
wants
to
volunteer
to
step
forward
and
sort
of
help,
carry
the
document
over
the
finish
line.
B
I've
sent
probably
five
or
six
attempts
through
the
data
tracker
to
reach
the
authors
and
then
once
on
the
list
with
a
somewhat
passive
aggressive.
You
all
need
to
do
stuff
for
al
or
our
market
dead,
and
also
about
seven
or
eight
off
list
emails
with
with
no
luck.
So
apart
from
actually
just
getting
all
of
the
comments
addressed,
there's
also
going
to
be
the
problem
of
at
auth
48th
time.
B
We
need
all
of
the
authors
to
acknowledge
that
they're,
okay
with
the
changes,
so
we
do
still
need
to
reach
them
in
some
manner,
but
I'm
not
sure
if
you
can
speak
yet.
B
Okay,
so
hopefully
sarah
has
more
like
managing
to
reach
the
authors
than
I
did.
I
got
some
bounces
and
the
rest
of
them
just
sort
of
seemed
to
disappear
into
a
black
hole,
so
yeah.
A
A
So
the
I
mean
the
the
real,
so
the
real
question
is
whether
whether
anyone
anyone
else
in
the
group
will
help
with
this
and
that's
the.
D
A
Okay,
all
right,
so
I
you
know
I
I
can.
I
can
certainly
remember
that
sarah's
always
had
some
difficulty
getting
the
real-time
media
going.
So
let's
see,
let's
see
how
that
works
out
good,
but
thank
you
for
that
suggestion
warren.
I
I
think
we'll
we'll
try
that
and
as
a
last
ditch
effort
and
then
let
you
know
what
happened.
Maybe
a
couple
weeks
request
see
if
see
if
this
is
important
enough
for
anybody
else
to
pick
up,
okay,
so
on
the
on
the
next
generation
firewall.
A
Actually
this
is
I
mean
this
is
just
our
status
part
of
this
we've.
We
had
our
last
working
group
class
call
on
this.
The
comments
were
resolved,
went
on
to
the
document
shefford
shepard,
and
we
need
some
things
added
to
the
latest
version
of
the
draft
based
on
the
shepard's
review,
which
is
me
so
that's
the
status
there.
A
We
got
a
lot
of
new
comments
on
multiple
loss
ratio
search,
which
is
the
other
new
working
group
document
and,
of
course,
the
proposals
keep
going.
So
that's
that's
where
we
stand
in
general
in
the
working
group,
we've
got
no
new
rfcs.
A
We've
we've
had
a
nice
stable
charter
for
about
four
years
and
and
we
like
it
that
way
so
unless
anybody's
got
problems
with
the
current
charter,
we'll
just
leave
it
as
is,
and
we
have
a
supplementary
bmw
g
webpage
that
I
haven't
visited
in
a
while,
but
I
hope
it's
still
reachable
so
brian
brian
monkman,
I
see
you're
in
the
queue
sir.
Please
go
ahead.
Yes,
can
you
hear
me
okay?
E
Good
good,
quick
question:
first
off
our
apologies,
we
totally
missed
the
attachment
when,
when,
when
you
sent
it
out,
I
mean
all
all
of
us
did
so
I'm
not
too
sure
what
happened
there,
but
we
reviewed
it
and
everything
makes
makes
total
sense.
E
The
one
question
I
do
have
is:
should
we
create
a
new
version
before
go
before
the
it
gets
passed
over
to
the
area
director
for
review,
or
should
we
wait
until
we
get
the
area
director
comments
back?
What
should
we
do.
A
Let's,
let's
I
I
suggest
that
let's
send
warren
our
cleanest
version
possible,
and
then
I
mean
they're,
basically,
all
editorial
things,
the
only
you
know
the
only
one
that
I
thought
would
involve.
A
Some
work
was
checking
one
of
the
references
to
see
if
an
updated
version
could
be
used
and
that
came
out
of
the
nits
check
right.
E
Right
but
yeah,
that's
pretty
sure,
it's
pretty
pretty
straightforward,
but
okay,
fair
enough
I'll.
Once
I
think
we
have
have
something
I
mean,
if
we're
going
to.
If
we're
going
to
change
the
the
references
to
something
you
know
newer,
then
we
should
probably
just
do
everything
and
then
so
it's
it's
done
and
we
could
pass
it
on
with,
with,
with
all
of
your
comments,
addressed.
A
Good
good
and
then
I'll
push
the
button.
There's
a
there's,
a
button
to
request
a
you
know,
publication
request
and
that's
when
warren
gets
it
for
his
review
and
we'll
take
it
from
there.
E
Okay,
all
right
I'll,
send
you
an
eta
once
once
I
have
that
finalized.
A
That's
great,
thank
you.
Okay,
great
all,
right!
Well
that
so
that
covers
the
that
covers
the
next
generation
firewall
part
of
our
our
agenda,
and
it
it's
I've
mentioned
this
a
couple
of
times.
It's
important
to
include
that
obsolete
3511
in
the
abstract.
It's
just
the
way
thing.
It's
just
the
way
people
do
things.
A
So
let's
do
that
all
right-
and
let's
see
here
so
the
authors
will
will
will
do
that
and.
F
A
Search
and
yeah,
what
I
want
to
do
there
is
is
I'll
I'll
basically
share
my
screen,
which
is
going
to
be
a
little
different
than
this.
Let
me
go
away
from
the
slide
share.
A
Well,
I'm
guessing
I'm
guessing.
This
is
really
small.
The
my
notepad
here
is
that
is
that
the
general
impression
folks
in
in
the
audience
should
I
make
this
try
to
make
this
bigger.
A
Well,
you
know
it
might
actually
be
yeah,
it's
actually
more
visible
here.
So
for-
and
this
is
the
this
is
the
notepad
that
I
mentioned
before-
in
the
editing
mode,
so
on
the
multiple
loss
ratio,
search,
mossyack
posted
a
status
late
last
week,
and
what
we've?
What
we've
also
got,
is
some
comments
that
have
been
made
by.
A
Gabor
and
verrattko
has
been
commenting
or
sort
of
responding
to
those
comments,
so
we've
gotten
we've
gotten
some
reviews
and
then
vladimir
posted
comments
over
the
weekend.
A
So
that's
great,
I
know
they
haven't
had
a
chance
to
to
respond
to
those
a
flat,
but
it
appears
that
there's
plenty
of
material
for
the
authors
to
take
back
and
consider
and
and
work
on
an
update
of
this
of
this
particular
document
and
we've
had
you
know:
we've
had
lots
of
discussion
of
this
thing
on
its
way
to
working
group
adoption.
So
I
think
that's
you
know
it.
This
is.
This
is
a
document?
That's
that's
on
its
way,
so
we're
just
looking
forward
to.
A
You
know:
we've
gone
through
these
pretty
heavily
we're.
You
know,
developing
a
multiple
loss
ratio
search
toward
you
know
toward
a
working
group.
Last
call:
we've
had
many
last
calls
on
the
security
device
performance,
and
I
think
we
can
be
happy
with
with
what
we've
accomplished
with
both
of
these.
So
far,
we'll
see
a
little
more
we'll
see
a
little
more
work
on
on
both
of
them
before
we
send
them
on.
A
All
right
so
we're
making
good
time
here
during
the
interim
period
between
our
our
session
at
itf,
111
and
ietf
112,
we
had
a
working
group
adoption
review
for
the
yang
model,
a
young
data
model
for
network
interconnect,
tester
management
and
you
can
see
the
document
is-
is
here
it's
been
updated
october,
25th
thanks
to
the
author
for
his
for
his
update,
and
this
is
what
I
meant
when
I
said
that
the
working
group
adoption
was
a
little
bit
unusual.
With
this,
we
we
got
some.
A
There
were
some
comments
in
september
early
september,
when
I,
when
I
started
the
adoption
call
tom
petch,
basically
wrote
in
an
objection,
but
it
seemed
to
be
a
manageable
of
objection
about
the
the
prefixes
to
some
of
the
names
and
jurgen
schoenwalder.
Had
many
comments,
including
this
comment
about
the
status
but
just
checking
here.
A
Oh
yeah,
vlad.
You
know
I've.
I've
done
a
little
summarizing
here
already,
but
I
I'd
be
glad
to
invite
you
to
the
microphone
to
to
add
some
additional
detail.
C
C
Yeah,
I
I
think
the
the
summary
that
you
did
is
precise
and
I
don't
think
we
can
add
the
words
right
now.
The
the
comments
from
you
again
are
something
that
the
work
you
can
can
work
with
and
decide
upon.
C
Tom's
objection
was,
as
you
said,
many
jeru
and
I
think
also
the
work
group
will
have
the
say
about
if
we
modify
the
prefixes
further,
but
at
least
we
have
two
people
that
have
read
the
drafts
in
detail
and
I
I
don't
see
any
explicit
support.
Anyone
who
has
has
done
that.
So
that's
a
bit
unfortunate,
but
I
think
a
lot
of
people
have
read
the
draft
and
no
one
has
come
with
some
obvious
objection
that
that
proves
this
draft
not
to
be
a
work
group
item.
So
this
is
positive.
A
Good
thanks
thanks.
I
I
appreciate
you
joining
us
today
to
add
that,
and
so
then
the
next
question
I
want
to
talk
about
in
particular
is
this:
this
idea
of
the
changing
the
status
to
standards
track.
A
I
I
I
have
to
agree
that
if
we're
going
to
have
a
yang
model,
as
jorgen
said,
then
it
should
be
a
standard
track
model
like
the
rest
of
the
yang
models
and
the
the
big
surprise
here
is
that
we've
never
produced
a
standards
track
document
in
the
benchmarking
methodology
working
group.
That's
because
all
we've
produced
up
to
this
point
is
you
know,
considerations,
drafts
and
terms
and
and
definitions,
drafts
terminology,
basically
and
then
methodology
drafts
where
we
talk
about
the
procedures
and
and
so
forth.
A
But
there
was
at
the
time
the
benchmarking
methodology
working
group
started.
There
was
no
real
way
to
move
material
like
that
along
the
standards
track.
A
That's
no
longer
necessarily
true,
but
we
haven't
had
much
interest
in
following
what
the
ip
performance
metrics
working
group
did,
which
is
to
test
implementations
of
the
the
rfcs
for
their.
You
know
their
measurement
equivalents.
You
know
back
when
we
were
working
on
that
heavily
in
in
the
ip
performance
metrics
working
group.
I
opened
that
possibility
up
to
to
this
working
group
and
and
didn't
get
a
lot
of
takers,
so
we've
continued
on
with
the
informational
drafts.
A
So
I
I
I'd
like
to
propose
two
things
and
that
we
were
so
we're
discussing
the
the
adoption
outcome.
Now
I
think
it's
clear
that
we
need
a
yang
doctor
to
work
with
us
work
with
bmwg
on
this
and
that's
a
question
for
warren
really
d.
Do
you
agree
warren
that
that
would
be
helpful,
and
maybe
you
can
help
us
get
a
yang
doctor
assigned.
B
A
Yeah,
so
I
think
I
think
that's
that's
one
of
the
problems
that
we
we
don't
have
a
lot
of
yang
expertise
here
in
bmwg,
and
so
we
should
at
least
have
a
you
know
like
a
yang
doctor,
to
advise.
A
If
you
know
if,
given
that
the
comments
that
came
back
in
in
adoption,
were
you
know
very
specific
yang
comments
and
and
not
so
much
on
the
benchmarking
aspect,
which
you
know
which
which
those
of
us
who
have
looked
at
it,
seem
to
be
okay
with
so
it's
we,
we
need
this
additional
perspective.
I
think-
and
so
that's
I
mean
that's
one
part
of
this
and
I
I
think
I
think
we
should
probably
do
another
short
working
group.
A
So
that's
that
would
then
be
the
question.
Another
question
for
the
working
group.
G
I'm
I'm
sorry,
I'm
sorry
vlad.
I
missed.
I
think
I
missed
what
you
said
there
yeah.
I
agree
with
with.
A
B
A
Yeah,
I
think
that's
true,
I
I
I
I've
never
used.
I've
never
used
that
terminology
in
the
in
the
charter.
Although
you
know
I've,
I
think
I've
I've
said
a
number
of
times
in
the
document.
Shepard's
form
that
bmwg
has
only
produced
informational
documents.
So
far
you
know
I
just
basically
have
to
stop
saying
that
in
the
in
the
shepherds
form.
A
Given
that
we,
I
I
I
think
I
think
the
yang
doctor
part
of
this
is
important,
so
I
mean
alongs.
So,
alongside
the
new
working
group
adoption
should
we
cross
post
this
adoption
call
to
another
working
group
to
try
to
get
a
yang
doctor
to
help
us.
A
Like
I
don't
know,
net
mod.
B
A
Yeah
yeah
and
that
that
last
one
was
was
cc
yang.
B
B
A
Area
working
group
right
right
right,
oh
okay,
so
there's
some
chat
here
that
I've
been
missing.
Sarah
says
I
also
agree.
I
thought
the
ch.
I
thought
our
charter
had
at
one
point
said
something
about
information,
but
maybe
yamaron
all
right,
so
good,
all
right
and
we've
got
some
other
things
here:
yeah
mailing
list,
young
doctors
very
good.
A
Good
all
right
cool
all
right,
so
we've
got
a
path
forward
and
I'll
start,
the
I'll
start.
The
working
group
last
call
and
cc
all
these
people
as
quickly
as
I
can
and
then
we've
we've
got
a
got
a
good
decision
about
this.
I'm
sorry
for
the
sorry
for
the
delay
about
this.
You
know
coming
to
a
closure
about
the
first
adoption
call
vlad.
You
know
too
many
notes,
but
thank
you
for
your.
Your
input
and
participation.
Much
appreciated.
A
Okay,
so
let
me
go
back
to
the
the
slide
sharing
mode
here.
I
guess
I
gotta
stop
this
and
I'll
start
this
and
I'm
I'm
I'm
warming
up
to
show
your
slides
gabor.
So
please,
please
get
ready
on
your
end.
F
Okay,
so
thank
you
very
much
for
the
opportunity
to
present
this
draft,
and
this
draft
is
about
the
benchmarking
methodology
for
stateful
n80
xy
gate
phase,
where
xy
means
four
or
six
as
you
find
as
you
want,
and
the
specialty
of
this
draft
is
that
we
also
consider
the
seven
random
port
numbers.
Could
you
go
to
the
next
slide?
Please?
F
F
F
However,
the
situation
now
is
a
bit
more
complicated
because
you
can
see
that
on
the
left
side
there
are
private
ip
addresses
and
on
the
right
side,
the
public
ipv
version
for
addresses.
And,
of
course,
if
the
tester
from
the
left
to
the
right
would
like
to
send
some
test
frames
through
the
device
and
the
test.
It's
no
problem.
F
It
can
use
any
port
numbers
and
just
send
a
frame
through
and
it's
a
duty
of
the
device
on
the
test,
the
stateful,
let's
say
an
84
for
gateway
that
it
performs
the
translation,
changes
the
let's
say,
the
source,
ip
version
for
address
and,
if
necessary,
it
also
changes
the
port
numbers
and
sends
out
the
frame.
And,
of
course,
the
tester
on
the
right
side
will
receive
the
frame
all
right
and
if
it
was
a
frame
belonging
to
a
new,
not
yet
seen
session.
F
Then
the
device
in
the
test
store
the
session
in
the
connection
tracking
table
and
when
they
responded
when
the
tester
receives
this
frame,
it
also
extracts
the
ip
addresses
and
port
numbers,
and
it
also
stores
the
four
type
of
I
mean
the
source
and
destination
ipad
addresses
and
port
numbers
into
its
states
table.
Because
then
it
would
like
to
send
a
frame
in
the
reverse
direction
from
right
to
the
left
through
the
device
under
test.
F
It
just
cannot
invent
arbitrary
port
numbers,
but
it
has
to
use
evade
for
tumble
and,
of
course,
change,
source
and
destination,
and
then
it
can
can
send
a
test
frame
through
the
device
on
the
test
which
belongs
to
an
existing
connection,
and
thus
the
stateful
nat
gateway
is
able
to
translate
its
addresses
and
port
numbers
and
forward
back.
F
So
this
is
how
it
works,
and
so
you
have
seen
so
far.
One
thing
that
the
right
side
of
the
tester
just
may
not
invent
port
numbers
but
have
to
have
to
know,
has
a
knowledge
stay
stable
about
the
port
numbers
which
can
be
used
and
there's
another
thing.
It's
in
relationship
with
rfc
4814
that
it
recommended
the
user
usage
of
random
port
numbers.
F
So
even
from
the
left
to
the
right,
you
may
not
not
just
use
arbitrary
number
of
port
number
combinations
because
it
would
would
have
been
denier
service
attack
against
the
connection
tracking
table
of
the
device
under
test.
So
we
have
to
limit
the
port
number
ranges
both
the
source
and
extension
upon
number
edges
to
a
smaller
range,
so
that
all
the
combinations
should
be
limited
number
and
not
exhaust
the
connection
tracking
table.
F
Thank
you
and
how
a
measurement
happens.
So
we
invented
a
preliminary
phase
which
happens
before
the
real
test
phase
and
in
this
phase
the
last
left
part
of
the
test,
which
was
called
initiator,
sends
primary
frames
to
the
through
the
doot
to
the
right
side
of
the
tester,
and
two
things
happened
during
this
primary
phase.
F
F
But
it's
not
always
that
situation,
because
if
we
would
like
to
measure
the
maximum
connection
connection
establishment
rate,
then
we
just
use
the
primary
phase
and
we
just
make
a
binary
search
to
find
the
highest
rate
at
which
all
the
frames
test
frames,
which
initiate
a
new
connection,
are
forwarded
and
successfully
right
back
to
the
tester
and,
of
course,
our
traditional
tests,
such
as
throughput,
framelessly,
etc,
are
done
in
the
real
test
phase,
which
must
be
preceded
by
a
perimeter
space.
F
Slide,
thank
you.
So
this
method
was
presented
already
at
the
last
ietf
meeting
and
since
then
we
have
made
two
new
versions:
version
zero.
One
includes
the
comments
on
the
mailing
list
before
the
last
iitf
meeting
and
zero
version.
Two
includes
some
interesting
improvement
or
refinement
of
methodology,
because
I
have
made
a
lot
of
measurements
for
the
ip
version.
6
operations
working
group
for
this
draft.
F
So
in
the
first
test
series,
I
used
one
two,
four,
eight
and
16
cpu
cores
and
I
examined
how
the
performance
of
ip
tables
scaled
up
with
the
number
of
cpu
cores
and
in
the
second
test
series,
I
used
a
different
number
of
connections
in
the
connection
tracking
table
and
I
examined
how
the
performance
of
ip
tables
degraded
with
high
number
of
connections
such
as
one
100
million
400
million,
even
800
million
connections
in
the
connection
tracking
table,
and
during
this
experience
I
have
had
some
more
insight
how
the
method
should
work
and
using
this
experience
we
have
reverb
section
4.3
and
made
some
changes.
F
Also
in
4.4
and
as
al
wrote
a
few
days
ago,
I
should
have
changed
some
other
parts
and
I
will
update
those
parts
also
because
there
became
some
inconsistencies.
But
now
I
would
like
to
talk
about
section
4.3.
Could
you
go
to
next
slide?
Please.
F
Thank
you,
so
our
experience
has
shown
that
there's
a
significant
difference
between
the
processing
power
necessary
for
a
test
frame
when
it
generates
a
new
connection
and
when
it
doesn't
generate
a
new
connection
in
the
connect
on
the
contact
table
of
the
device
and
the
test
and
what
is
even
worse.
F
We
have
experienced
that
if
I
fill
the
connection
tracking
table
and
it
requires
a
given
amount
of
time,
its
depletion
requires
even
more
time.
So
it's
a
problem.
If
some
connections
are
time
out
and
some
new
connections
are
established
during
the
test,
because
their
proportion
may
significantly
influence
the
results
of-
let's
say
throughput-
tested
oil
tests.
So
for
this
motivation
I
think
we
managed
to
make
the
measurement
process
purchase,
such
as
a
procedure
more
clean,
and
I
would
like
to
explain
it
in
the
next
slide.
Will
you
go
to
the
next
slide?
F
F
Yes,
so
the
consequence
is
that
we
can
ensure
two
extreme
situations
and
they
can
be
ensured
easily,
and
unfortunately
I
must
say
that
I
cannot
ensure
any
other
situations
easily.
So
the
first
exam
situation
is
when
all
test
frames
create
a
new
connection,
and
this
is
ideal
for
measuring
the
maximum
connection
establishment
rate,
and
the
second
situation
is
when
test
frames
never
create
a
new
connection.
F
Of
course,
I
understand
it
for
the
second
phase
for
the
real
test
phase
and
this
idea
for
the
throughput,
latency
and
all
other
traditional
tests
and
due
to
blackbox
testing,
I
must
say
that
we
are
not
able
to
ensure
other
situations
in
between,
for
example,
it
would
be
natural
to
say,
let's
say
10
of
the
new
test.
F
Trains
keys
in
your
connection
and
90
percent
belong
to
the
existing
connection,
but
I
cannot
kind
of
do
that
because
of
blackboard
testing,
so
the
tester
is
not
able
to
examine
the
connection
tracking
table
of
the
data
under
test.
For
this
reason,
I
don't
see
a
good
chance
to
to
implement
such
kind
of
tests,
just
the
two
extremes-
okay,
and
how
to
how
to
ensure
this.
These
two
extreme
situations-
I
will
cover
this
in
the
next
slides.
Could
you
go
to
next
slide?
Please.
F
So
before
the
method,
how
to
ensure
that
two
excel
situations,
we
make
some
assumptions.
The
first
assumption
is
just
for
simplicity.
It
is
that
a
singular
source
addressed
and
destination
despair
is
used
for
all
tests.
I
know
that
rfc
2544
requires
also
testing
with
256
different
destination
networks,
but
currently
we
do
not
support
it.
F
F
So
it
is
something
to
be
ensured
by
the
one
who
performs
the
test
and
if
it's,
if
ensured,
then
all
other
things
can
be
ensured
easily.
Could
you
go
to
the
next
slide.
F
So
excellent
situation,
one
which
is
for
measuring
the
maximum
conduction
establishment
rate-
is
that
we
need
to
ensure
that
all
test
frames
initiate
a
new
connection
in
the
connection
tracking
table,
so
it
just
for
the
first
phase,
the
primary
phase
and
it's
very
easy,
because
we
must
use
all
different
source
port
number
destination.
Port
number
combinations
in
the
primary
phase,
and
we
also
need
to
ensure
that
the
udp
timeout
of
the
gateway
is
higher
than
the
length
of
the
duration
of
the
premier
phase.
F
Of
course,
if
some
connections
would
time
out,
the
new
combinations
would
still
result
in
in
new
connections,
but
we
must
ensure
that
no
connections,
time
out,
because
timing
of
the
connections
would
impose
some
processing
power
need.
So
we
would
like
to
do
not
like
to
disturb
the
measurement
with
some
some
timeouts
and
and
the
deletions
of
connections.
That's
why
we
need
to
ensure
that
connections
are
not
deleted,
and
this
is
enough
for
the
measurement
of
the
maximum
connection
establishment
rate.
F
Could
you
go
to
the
next
slide
and
now
comes
the
most
interesting
thing
we
would
like
to
perform
the
traditional
measurements
such
as
throughput
and
frame
low
slate
and
latency
etc.
So
how
can
we
do
that?
F
This
is
these
are
the
conditions.
So,
in
the
prima
preliminary
phase
we
use
all
different
source
port
number
destination,
port
number
combinations
and
in
the
preliminary
phase
we
enumerate
all
possible
source
point
number
and
decision
point
number
combinations.
F
F
F
This
is
the
length
of
the
preliminary
phase
and
the
gap
between
the
two
phases
and
the
length
of
the
real
test
phase,
and
if
we
ensure
that,
then
in
the
real
test
phase,
we
will
have
no
new
connections,
no
timeouts
and
all
test
frames
will
belong
to
existing
connections,
so
it
it
makes
testing
more
simple,
because
this
spring,
when
I
made
some
some
other
tests,
I
had
to
have
to
take
care
if
things
were
time
out
or
not
timeout
and
refresh
the
connections
by
private
practically
scanning
the
connection,
the
the
state
table
of
the
device
of
the
tester,
but
now
this
this
high
time
mode
eliminates
all
other
troubles,
and
I
think
in
this
way
these
two
measurements,
these
two
types
of
measurements,
can
be
executed
in
a
way
that
they
produce,
I
think
meaningful
and
especially
repeatable
results.
F
Thank
you
and,
of
course,
some
considerations
regarding
the
southern
random
port
numbers.
Earlier
I
have
made
also
some
tests
when
I
enumerated
the
port
number
combinations
in
increasing
order.
I
mean
that
let's
say
that
let's
take
the
source
point
attestation
on
port
numbers
and
make
a
four
digit
number
and
of
course
there
are
some
some
bad
and
invite
combinations.
F
I
I
just
eliminated
the
valid
combinations
in
increasing
order,
but
I
experienced
that
when
I
enumerate
all
the
combinations
in
a
random
order
and
if
I
animate
them
in
an
increasing
order,
the
result
is
not
the
same
and
it's
probably
some
special
things
in
ip
tables,
but
I
experienced
that
was
there
really
a
significant
difference,
so
southern
random
enumeration
is
is
a
must
and
of
course
we
may
use
a
linear
increasing
enumeration.
F
Also
an
editorial
metric
but
epsilon
random
animation
is
a
must
and,
of
course,
there's
a
good
algorithm
for
that.
So
we
can
use
that
stanford's
random,
shuffle
algorithm.
It
means
that
we
just
emulate
all
the
port
number
combinations
in
an
area
and
then
make
the
random
shuffle,
and
then
we
have
all
the
port
numbers
and
during
the
test
we
just
read
the
array
in
the
in
the
tester.
So
we
have.
F
F
So
so
far
I
have
told
what
we
can
do
and
my
question
is:
what
do
you
think
about
it?
So
do
you
think
that
it
is
a
useful
thing?
Do
you
think
that
they
provide
meaningful,
reasonable
results
and
are
these
results
satisfactory,
characterize
the
performances
of
the
device
under
test
or
is
there
anything
missing,
and
I
have
a
first
question:
what
do
you
think
about
measuring
the
connection
tiered
our
performance?
F
Unfortunately,
I
can
offer
only
a
way
that
I
fill
and
number
of
connections
in
the
connection
tracking
table
and
then
I
did
this
whole,
the
entire
connection
tracking
table.
It
can
be
done
with
ip
tables
by
deleting
the
ip
tables
rule
and
then
removing
the
kernel
module,
or
it
can
also
be
done
with
you,
which
is
a
nat64
implementation
by
by
restarting
joule,
and
we
can
measure
its
time
so
I
have
experienced
that
it
lasted
longer
than
filling
in
the
state
table.
F
A
F
A
Gabor
I'd
like
to
open
it
up
for
questions
and
and
comments
at
this
point
I
think
you've
got
you've.
You've
brought
us
up
to
date
and
you've
got
reasonable
requests
for
feedback
here.
So
let's
hear
some
feedback
on
on
this
on
your
work.
That
is
pretty
fundamental
to
the
kind
of
thing
we
normally
do.
In
benchmarking
methodology
working
group.
A
F
Well,
if
you
think
about
the
measurement
results
of
the
ipt
bus
testing,
I
have
some
slides.
I
invented
them
for
the
ip
version
6.
F
D
F
I'm
sorry
I
just
can't
find
my
slides,
but
anyway
I
will
be
able
to
do
that.
So
I
I
have
uploaded
my
I
don't
know
I
okay.
I
will
find
I
started
to
fred
baker.
The
other
slides
and
I
will
find
it
here.
A
F
I
have
I
have
got
now
the
the
file,
if
I
just.
F
H
F
Yes,
I've
got,
I
got
the
slice
now.
I
think
I
would
like
to
ask
permission
from
you
to
share
the
screen.
Yeah.
F
F
A
So-
and
these
are
the
number
of
cores
that
are
allocated
to
the
to
a
virtual
machine-
that's
that's
doing
the
nat
function.
A
40!
I
I'm
asking
I'm
asking
in
in
the
case
of
number
of
cores,
you
are
you're
allocating
a
certain
number
of
cores
to
the
nat
processes.
F
Yes,
it
means
that
I
use
the
physical
machine
and
I
just
ditched
one
of
the
cores.
So
with
the
kernel
parameter,
I
could
switch
on
and
off
the
cores
right.
So
first,
I
measure
the
single
core
and
then
with
two
cores
four
cores:
eight
core
and
16
cores.
F
And
you
can
see
that
it
scaled
up
quite
well,
so
it
from
one
quarter
to
core.
Do
you
also
see
my
my
little
pointing
hand?
Yes,
yes,.
B
F
So
when
going
to
two
chord,
it
doesn't
double
and
from
for
two
chords
to
to
focus
on
double,
but
it's
significant
significantly
increased
and
I
also
calculated
the
relative
scale
up.
So
it's
not
the
double
and
at
the
end
it's
not
not
16
16
times
more,
but
you
you
can
see
it
about
10
times
more,
so
it's
quite
good
scaling
and
here's
the
throughput
and
also
it's
in
double.
But
similarly
it's
about
the
performance
of
of
10
cores.
So
you
know
16
cores.
I
could
achieve
the
performance
of
10
cores.
So
it's
not
bad.
F
A
F
A
A
Was
this
testing
was
this
testing
on
on
a
an
intel
architecture
where
you've
got
two
pneuma
nodes?
Something
like
that.
F
Yes,
it
was
intel.
I
can
show
you
the
the
previous
slide
this
here.
It
is
a
dell
poweredge,
r430
server
and
the
clock
was
set
to
fixed
2.1
gigahertz.
This
is
the
ram
size
and
there
were
two
10
giga.
Do
a
port
nics.
It
is
the
internal.
D
So
the
the
dell
bios
actually
has
virtualization
features
were
those
turned
on
to
help
out
the
cpus
here.
F
F
Had
32
cores,
but
I
didn't
didn't
use,
didn't
include
the
measurement
research,
of
course,
because
there's
some
some
kind
of
problem,
because
if
I
use
32
cores
no
interrupts
were
scheduled
for
the
last
16
cores,
so
I
excluded
those
scores.
I
didn't
include
it
didn't
it
didn't
really
increase
after
16
chords
because
of
that
problem.
D
You
know
hey
first,
thank
you
for
presenting
results.
I
think
if
we
were
in
person,
you'd
have
a
line
of
people
sitting
in
the
front
chairs
leaning
over
to
to
see
your
results.
It's
always
fun
when
we
get
these
results
in
the
working
group.
So
thank
you
for
sharing
them.
D
F
But
it
was,
it
was
important
for
me
at
the
ip
version,
6
operations
group,
how
it
scales,
because
there
are
some
opinions
that
states
for
technologies
don't
scale
there
and
stateless
scale
well,
and
I
think
it's
a
quite
a
good
skill
to
kill
up.
So
I
did
some
some
alternative,
dns
server
benchmarking,
and
even
that
was
not
better,
but
here
it
is
a
stateful
thing.
You
know
we
have
to
handle
the
connection
timing
table
and
how
well
these
cpus
can
cooperate
on
the
same
connection
tracking
table.
I
think
it's
a
really
good
result.
F
If,
if
you
have
some
time,
I
can
show
you
some
other
results.
These
are
the
scalability
agents,
the
number
of
current
sessions
concurrent
sessions,
so
how
many
sessions
are
stored
in
the
state
table
you
see
here
is
the
number
of
connections,
a
million,
and
I
could
test
up
to
800
million,
but
it
was
really
the
limits
of
the
hardware.
F
When
I
used
only
400
million
connections,
it
was
somewhat
less
than
150
gigabytes
the
usage
of
memory
and
it
was
somewhat
below
300
gigabytes,
but
all
together
we
have
only
384
gigabytes.
So,
in
this
case,
in
the
last
column,
you
see
the
results
when
non-numerical
memory
was
also
used,
and
there
was
another
other
limit
that
I
couldn't
set.
The
hash
hash
table
size
higher
than
this
one,
so
this
is
the
highest
one.
You
see
that
we
always.
F
Quadrupled
the
numbers
because
of
increasing
the
port
numbers-
and
I
couldn't
do
it
anymore,
because
it
gave
me
an
error
message,
so
you
know
what
it
works,
that
iptables
has
a
hash
table
and
from
every
entry
of
the
hash
table
the
link
list
is
is
started
and
usually
the
link
list
has
less
than
one
element.
But
here
it's
on
average.
F
All
linked
list
has
three
elements,
so
it
it
was
a
factor
which
it
slows
down
the
operation
and,
of
course
you
can
see
some
degradation,
some
performance
regulation
here,
but
here
you
can
see
that
this
loop
is
near
constant
here,
4.4
milli,
4.5,
4.5
and
so
and
so
on.
It's
just
measurement
error
is
4.4.
I
think
this
is
the
constant
4.5
and
here
at
the
beginning
I
think
the
the
first
column
is
higher
because
of
the
level
3
cache.
It
still
could
do
something,
but
it
means
that
in
a
wide
range,
it's
quite
scalable.
A
Thank
you
yeah.
It's
always
interested
to
interesting
to
look
at
some
actual
results
and
and
then
we
can,
you
know,
get
a
feeling
for
the
value
of
the
benchmarks
you've
proposed.
So
that's
a
that's
a
great
follow-up
gabor
and
I'm
glad
we
took
an
extra
moment
or
two
to
get
this
on
the
screen
and
make
it
part
of
our
meeting
thanks.
E
A
So
going
back
to
going
back
to
the
questions
you
raised,
I'm
gonna
see
if
I
can
hijack
the
screen
here.
I
stop
sharing.
A
A
So
I
think
I
think
we
can
kind
of
answer
this
first
question
now:
do
they
provide
meaningful
and
and
reasonable
results
in
the
in
the
use
case
where
you've
applied
them
to
ask
to
try
to
answer
a
question
about
you
know
the
difference
between
stateless
and
and
stateful.
I
think
the
answer
is
yes
and
you've
got
a
couple
of
good
metrics
there.
I
think
that
there's
some
ability
to
think
about
this
more.
A
Yeah,
I
I
think
in
my
comments,
I
was
wondering
about
the
size
whether
the
size
of
the
whether
the
size
of
the
table
itself
was
a
was
a
metric
that
was
worth
measuring.
Did
this
is
the
table
capacity?
Is
that
one
of
the
ones
that
you've
proposed.
F
F
My
student
works
with
that
and
we
couldn't
find
any
way
to
set
its
connection
tracking
table
size.
So
it
may
be
an
interesting
metric
and
yes,
we
can
measure
it,
but
it's
not
very
easy.
You.
You
saw
that
as
I
knew
that
it
as
I
knew
that
it's
the
size
was
large
enough.
I
could
be
sure
that
if
I
try
to
add
new
entries
and
the
problem
can
only
be
that-
I
do
it
too
fast,
and
this
is
why
how
I
could
this
is
how
I
could
measure
the
maximum
connection
establishment
rate.
A
Yeah,
at
least
at
least
you
can
you
can
report
the
number
of
connections
that
you've
used
during
a
test
and
and
in
the
follow-up
testing.
So
that's
at
least
you
can
infer
something
about
the
capacity
there
from
from
that.
A
Okay
and
oh
yeah
connection
teardown
performance,
so
so
then
you're
working
with
timeouts,
again
and
and
the
efficiency
of
connection
tear
down
in
the
in
the
device
it.
I
think
it's
reasonable
to
look
into
that,
but
I
can
certainly
imagine
some
some
difficulties
trying
to
make
a
trying
to
make
a
repeatable
benchmark
out
of
it.
What
do
you?
What
do
you
think
gabor.
F
Yeah
the
trouble
is
that
I'm
not
sure
if,
if
how
many
of
them
times
out
and
how
many
new
I
should
should
include
so
it's.
F
I
can
just
delete
the
ip
tables
rule
and
I
can
remove
with
rm
mode
linux
command
the
the
canon
module
and
then
it
lasts
for
a
long
time
for
minutes
to
to
do
to
execute
the
command,
and
I
can
measure
its
time
and
of
course
I
can
do
the
measurements
with
different
number
of
connections,
and
I
can
estimate
how
much
work
on
average
to
destroy
one
connection.
A
I
see
I
see
and
and
and
that
that
manual
deletion
of
the
entire
table
is
it
should
be
a
process
maybe
similar
to
the
time
out
process.
A
But
it's
kind
of
surprising
I
get
maybe
not
so
surprising
when,
when
you've
got,
you
know,
connections
in
in
the
sizes
that
you're
talking
about
here
and
really
large
numbers.
So
in
any
case,
that's
a
it's
just
something
to
think
about.
I
I
I
can
see
that
you're
already
thinking
about
it
and
you
know
if
you,
if
you
try
some
things
and
and
there's
something
useful
comes
out
of
it.
A
I
think
that's
worth,
I
think
that's
worth
investigating,
because
it
is
you
know
it's
the
it's
the
other
side
of
connection
establishment
and,
and
that
feels
feels
good
feels
symmetrical
to
to
understand
both.
F
F
So
if,
if
you,
if
you're
an
eisp
and
you
operate
a
gateway,
then
new
connections
are
coming
and
some
connections
are
deleted
and
it
matters
how
much
effort
it
it.
It
takes
to
date
a
connection
right.
A
It
seems
like
if
you
filled
well
yeah,
if
you
knew
the
capacity
of
the
you
have
to
know
the
capacity
of
the
table,
and
then
you
fill
it
and
then
you,
you
wait
a
certain
amount
of
time
where
a
number
of
the
connections
should
be
a
number
of
the
first
connections
established
should
be
cleared
and
then
you,
you
know
it
should
have
timed
out,
and
then
you
try
to
establish
a
whole
new
block
of
of
connections
and
determine
whether
the
device
is
ready
for
those
yet
or
things
along
those
lines.
A
It
seems
to
me
that
we've
had
we've
had
some
testing
like
this
proposed.
You
know
of
this
ilk
proposed
in
the
past
and
I'm
I'm
not
I'm
not
connecting
it
to
a
particular
rfc
that
we
did,
but
I'm.
F
A
That
there's
one
out
there
where
we
or
we
had
to
grapple
with
this
so
anyway,
that's
something
to
think
about
too
and
yeah
so
far.
I
think
the
aggregate
measurements
are
are
a
great
start
here
and,
as
you
say,
can't
do
anything
else,
but
the
depletion
time
is
a
that
might
be
a
an
interesting
thing
to
investigate
further
okay.
A
Well,
thank
you,
gabor
for
your
for
your
talk.
I
I
just
wanted
to
quickly
summarize
that
in
the
may
july
period
I
saw
five
people
discussing
the
draft
on
the
list.
There
was
you
know
the
references
to
the
procedures
in
rfc
8219,
which
our
friend
marius
worked
on.
A
I
believe,
right
and
and
also
there
was
sort
of
some
interest
in
the
tcp
based
testing,
and
that
was
you
know
something
that
that
was
talked
about,
but
right
now,
you're
all
udp,
and
I
think
that's
that's-
probably
a
good
scope
for
for
now.
A
The
I
guess
the
question
to
the
group
is
with
the
with
the
this
traffic
that
we've
seen
and
the
reasonable
results
that
we've
that
we've
seen
and
and
actually
the
open
source
implementation
of
some
of
this.
That
gabor
has
prepared
and
made
available
to
us
on
on
several
occasions.
A
A
A
D
D
I
I
think
it's
gabor
I
apologize.
I
hadn't
had
a
chance
to
go
through
this
in
detail,
but
I
will
take
a
look
and
frankly
between
what
have
piqued
my
interest
to
put
it
on
the
reading
list
and
I'm
running
out
of
time
and
this
presentation
today.
I
do
think
it's
worth
asking
the
the
working
group
for
the
support
and
I
would
also
I
mean
I
always
read
the
drafts,
but
I
just
wanted
to
weigh
in
officially
and
say
thank
you
for
doing
this.
D
Thanks
for
presenting
the
results-
and
I
will
absolutely
read
it
formally
and
share
my
feedback
with
you
on
the
list.
A
A
Okay,
well,
then,
we'll
try
to
we'll
try
to
ask
that
same
question
on
the
mailing
list.
Gabor,
I
think
you're
I
mean
I
think,
you're
basically
doing
a
good
job
of
that,
and
if
you
you
know,
if
you,
if
you
have
some
more
results
to
share
with
us,
that
may
help
get
some.
A
Commentary
going
and
I
would
share,
I
would
share
these
these
questions
on
the
mailing
list,
as
well
as
a
kind
of
a
prompt
for
folks
to
read,
read
the
draft
and
and
then
maybe
we
can.
You
know
depending
upon
what
happens
in
the
remainder
of
november
and
december,
we
can
maybe
take
up
a
working
group.
Adoption
call
in
january
or
sometime
in
2022..
A
In
20.2
yeah,
that's
good,
because
you
know
this
is
the
kind
of
thing
that's
right
up
our
alley.
It's
it's
very
you
know,
obviously
very
benchmarking,
centric
and
and
your
your
your
work
here
to
advance
our
state
of
the
art
is
very
much
appreciated.
So
thanks.
A
Okay,
very
welcome
all
right.
So
the
last
topic
today
is
a
is
a
brand
new
topic.
A
It
was
proposed
by
zeki,
lai
and
his
co-authors,
and
I'm
gonna
share
some
slides
here,
which
were
which
were
prepared
for
this
talk,
and
you
can
all
read
the
the
title
of
the
talk
here
with
me:
together:
problems
and
requirements
of
evaluation
methodology
for
integrated
space
and
terrestrial
networks.
A
So
we're
going
to
learn
a
lot
about
this
today
and
you
know
I
I
I
really
thank
zeki
and
his
co-authors
for
bringing
this
to
us
to
our
attention,
but
I
will
say
one
thing
in
advance
and
that
is
that
our
work
on
benchmarking
is
limited
to
what
we
can
do
in
the
the
isolated
test
environment
and
that
that
means
that
you
know
we're
obviously
not
going
to
be
running
any
benchmarks
over
satellites
at
least
things
that
we
kind
of
the
things
that
we
kind
of
create
here
but
and
I'll
say
more
about
this.
A
At
the
end,
I
I'd
just
like
to
think
I'd
like
folks
to
think
about,
as
as
zeki
eli
describes,
this
work,
what
what
kinds
of
pieces
of
it
we
might
be
able
to
tackle
in
the
near
term
within
within
our
charter,
if
anything
and
and
and
if
more
development
or
or
research
or
other
activity
is
needed,
then
then,
let's
try
to
figure
out
what
those
things
are
and
and
give
that
feedback
today,
but
with
that
zeki
and
his
co-authors
are
are
all
new
to
the
working
group.
A
As
far
as
I
know
so.
Welcome
and
you
can
begin
your
presentation.
H
Okay,
so
thank
you
very
much
al
and
thank
you
in
advance
for
controlling
my
slides,
because
maybe
in
the
next
presentation
I
may
continuously
say
next,
please
next
page,
okay,
thank
you
in
advance,
so
hello,
everyone
in
the
bmw
fox
good
morning
and
good
afternoon,
and
maybe
good
night
for
myself,
because
this
is
about
1am
in
my
place.
So
this
is
30
life
from
the
institute
for
network
science
and
cyberspace,
ching
hai
university-
and
this
is
my
first
time
to
participate
the
online
meeting
of
the
our
benchmarking
working
group.
H
H
A
number
of
companies
have
disclosed
efforts
along
these
slides,
including
spacex,
amazon,
one
web
and
so
on.
So
these
mega
constellations
are
proposed
to
provide
global
internet
service.
For
example,
elon
musk
sent
the
first
tweet
through
the
space
where
starting
satellites
and
recently
google
said
that
it
signed
a
deal
with
spacex
to
use
the
space
company's
growing
satellite
internet
service,
starting
with
its
cloud
unit.
H
As
of
november
2021,
there
are
already
four
thousand
and
five
hundred
active
satellites
launched
in
the
orbits
according
to
the
ucs
database.
So
look
at
this
two
table.
Two
two
figures
as
the
usa
has
about
two
thousand
and
seven
hundred
active
satellites
and
china
have
about
400
satellites
and
the
united
kingdom
has
about
340
satellites,
and
we
can
found
that
most
of
these
satellites
are
for
from
mega
constellations
for
the
internet
service.
H
H
So,
with
such
plans
in
progress
for
large-scale
satellite
to
provide
a
global
internet
service,
the
internet
is
potentially
taking
one
year's
leap
into
space.
What
might
the
internet
be
in
the
next
decade?
The
future
is
up
in
the
sky
here,
our
understanding
for
internet
for
the
internet
in
the
next
decade,
which
we
call
that
integrated
space
and
terrestrial
networks
or
istn
as
a
high
level,
the
istm
will
contain
two
segments.
The
first
one
is
the
space
segment,
including
satellites
and
ground
stations.
H
H
The
integration
of
satellite
is
constellations,
and
terrestrial
network
opens
a
lot
of
great
new
opportunities.
The
first
one
is
that
istn
can
potentially
provide
pervasive
network
accessibility
for
various
users.
According
to
a
recent
broad
report
from
statistics,
the
global
internet
penetration
rate
is
about
only
about
60
percentage.
H
There
are
still
a
large
portion
of
population
still
have
no
internet
access
in
the
world.
Istn
is
promising
to
provide
internet
success
for
remote
or
rural
users
and
many
other
situations
like
maritime
global,
iot,
airplane
users.
In
addition,
istn
can
also
be
used
for
many
new
scenarios,
such
as
space
flights,
deep
space,
navigation
and
exploration.
H
H
Laser
links
can
communicate
at
the
speed
of
the
light
in
a
vacuum
which
is
much
faster
than
that
in
the
terrestrial
fiber
second
evently
evenly
distributed
leo
satellites
can
establish
near
to
optimal
space
roads
and
enable
lower
latency
as
compared
with
meandering
terrestrial
fiber
roles,
for
example.
This
right
picture
shows
an
example
on
one
hand,
assume
two
communication
ends
locate
at
beijing
and
sydney
for
for
for
some
isps.
H
The
next
page,
next
page,
please
and
the
third
opportunity
is
that
istn
can
be,
can
enable
high
through
throughput
assess
free
space.
High-Speed
inter-satellite
links
can
provide
several
gbps
data
rates
and
the
new
ka
or
ku.
Radio
frequency
links
can
also
provide
very
high
speed
ground
station
links.
So
this
picture
shows
a
speed
test
from
the
individual
styling
beta
test,
so
one
individual
can
user
can
obtain
about
400
mbps,
downlink
data
rate
over
the
current
version
of
starting
constellation
next
page.
H
But
while
promising
istn
also
involves
several
unique
characteristics,
that
might
be
the
roadblocks
of
realizing
such
new
opportunities,
the
first
characteristic
characteristic
is
the
high
dynamics
on
a
global
scale.
These
two
figures
use
the
first
share
of
sterling
constellation
with
more
than
one
thousand
and
five
hundred
of
stars
find
hundreds
of
stress
satellites.
H
This
satellite
moves
in
very
high
optical
velocity,
and
it
takes
about
19
minutes
to
make
what
make
a
one
path
around
the
earth.
That
means
the
space
ground
connectivity
could
change
in
every
about
every
three
minutes.
Such
dynamics
may
cause
many
connect,
for
example,
connectivity,
loss,
routing,
reconvergence
and
other
issues
next
stage.
H
H
Just
like
this
draft,
we
pr
we
propose-
maybe
the
first
pro,
maybe
the
first
for
the
version
is
awful,
so
we
need
many
rounds
to
discuss
and
modify
our
draft
to
polish
it,
and
the
styling
constellation
today
is
not
the
same
as
its
first
version,
and
we
believe
it
is.
It
is
not
the
final
version
and
it
will
still
evolve
rapidly
in
the
future.
H
Finally,
as
compared
with
many
other
kinds
of
terrestrial
networks,
the
operational
cost
of
a
satellite
network
could
be
very
high.
Here
I
mean
here
at
the
cost.
I
mean
the
time
and
money,
for
example,
it
may
take
years
from
a
trunk
check
to
the
rear
satellite
launch
and
if
we
have
deployed
the
satellite
in
outer
space,
updating
a
deployed
constellation
is
costly.
H
So
here,
when
we
look
back
at
the
history
of
internet,
we
found
we
can
found
that
evolution,
evolution
and
benchmark
play
an
important
role
in
the
evolution
of
our
internet.
If
we
look
ahead
since
istn
is
still
revolving
very
rapidly
and
not
in
the
final
stage,
we
argue
that
rs
istn
necessitates
the
right
evolution
storage.
H
H
So
next
page
please
so
next,
let's
do
look
at
the
possible
evaluation.
Storage.
The
first
one
and
the
most
straightforward
approach
is
to
leverage
leave
satellite
networks
to
evaluate
new
sites.
For
example,
starling
has
started
its
beta
test
recently.
However,
this,
however,
this
approach
suffers
from
several
limitations.
H
First,
the
leave
satellite
network
lacks
the
flexibility
to
explore
various
experimental
requirements.
For
example,
it
is
very
difficult
to
change
the
constellation
exchanged,
maybe
change
the
inter
satellite
or
ground
satellite
connectivity
to
evaluate
various
internet
working
technologies
of
istm.
H
Second,
the
accessibility
of
a
live
satellite
leaves.
Satellite
network
could
be
constrained
in
many
regions
around
the
world.
As
far
as
I
know,
for
example,
studying
their
tests
is
only
available
in
certain
regions
around
the
world.
Finally,
directly
conducting
experiments
in
the
leave
network
has
many
constraints.
For
example,
it
is
difficult
to
conduct
top
large
topology
or
login
test
in
a
live
satellite
network.
H
The
second
approach
is
using
simulations
to
evaluate
or
benchmark
new
designs
for
ists,
both
aerospace
and
network
community
have
many
simulation
tools
to
create
an
experimental
network
environment
for
evaluation.
For
example,
stk
is
a
rapid
representative
of
orbital
anesthetic
tool
that
can
perform
complex
analysis
of
spacecraft
as
well
as
ground
stations.
H
H
H
So
the
third
approach
is
emulation.
Emulation
is
a
hybrid
approach
that
integrates
real
applications,
protocols
and
operating
systems
in
the
network
environments
similar
to
the
leave
networks,
emulators
emulators,
run
real
codes
with
interactive
network
traffic,
similar
to
simulators
emulators
can
support
controllable
and
diverse
topologies,
and
their
virtual
hardware
requires
less
resource
and
compared
with
leave
networks.
H
H
So,
given
that
all
these
existing
approaches
have
their
limitations
on
evaluating
isatn
here
we
summarize
the
problem.
Existing
evaluation
methodologies
from
both
aerospace
and
the
network
community
may
have
their
limitations
and
are
insufficient
for
comprehensively
and
systematically
evaluating
istns
next
page.
H
H
The
first
requirement
should
be
realism
at
least
constellation
and
real
network
realism.
That
means,
if
we
create
an
experimental
environment
or
an
isolated
environment
in
our
terrestrial
laboratory
laboratory.
The
terrestrial
presentation
is
expected
to
have
the
same
skill
and
dynamicity
like
a
real
constellation,
for
example,
in
an
experimental
environment
for
studying
satellite
network
is
expected
to
have
the
same
size
of
the
experiment:
nodes,
for
example
the
first
shell
or
of
the
starting
phase.
One
has
about
1500
satellites.
H
Also,
it
is
because
it
is
expected
that
the
environment
can
load
realistic
network
traffic
functionalities
protocols,
not
just
the
numeric
simulation
next
page,
please
so
the
second
requirements
we
call
it
flexibility
at
mega
constitution,
scale
testers
may
have
various
experimental
requirements
to
evaluate
various
internetwork
technologies
under
different
network
architectures,
topperage
and
so
on.
So
the
mass
storage
is
expected
to
flexibly
support
various
experimental
situations,
in
addition,
consider
that
the
scale
of
mega
constellation
network
could
be
much
higher
larger
than
many
other
terrestrial
networks.
H
For
example,
a
satellite
network
could
be
a
global
wide
area
network
with
thousands
of
networks
nodes.
So
the
map
storage
is
expected
to
be
scalable
to
mega
constellations
next
page
and
the
third
requirement
could
be
maybe
low
cost
and
easy
to
use
to
advance
the
progress
of
isd
and
technologies.
The
methodology
is
expected
to
be
technically
and
economically
feasible
for
normal
researchers
and
engineers.
H
And
the
last
one
is
that
the
methodology
requires
realistic
data
and
test
cases
to
improve
the
fidelity
and
standardize
the
benchmark
steps.
For
example,
if
the
evaluation
is
driven
by
realistic
constitutional
information
and
isdn
traffic
users
and
user
and
ground
station
distributions,
it
can
produce
more
meaningful
recommendations
concerning
the
key
performance
characteristics
of
internet
working
technology.
H
Next
page,
please
so
here,
if
we
look
forward,
what
we
really
need
is
an
evaluation
methodology
tailored
for
ists,
and
how
could
we
design
such
an
evaluation
storage,
satisfying
all
the
requirements
above
a
viable
path
might
be
might
be
to
build
an
integrated
reality,
realistic
database
community,
driven
evaluation
storage
for
future
istm.
H
Data
driven
here
means
that
the
laboratory
environment
is
expected
to
be
driven
by
real
data,
for
example
based
on
information
of
real
ground
station
satellites,
orbits
and
constellations.
Here
we
show
these
three
pictures
should
introduce
some
open
database,
for
example.
This
is
the
data
satellite
database
of
itu
and
an
orid
in
this
open
database.
You
can
search
the
satellite
information,
for
example
the
satellite
location.
I
mean
latitude,
longitude
and
attitude.
H
H
H
If
we
continue
to
look
at
hand
who
can
benefit
from
the
social
evaluation,
storage
for
satellite
operators
such
as
spacex
or
one
wipe,
they
can
use
this
evaluation
methodology
to
analyze
and
understand
network
performance
under
various
constitution
designs,
for
example,
under
different
numbers
of
satellites
and
different
orbital
heads
second
for
internet
service
providers.
They
can
use
such
methodology
to
evaluate
the
reliability,
latency
support
convergence,
time
and
other
metrics
under
various
inter-domain
routing
protocols.
H
H
Also,
we
ask
a
question
what
can
be
centralized
guys
here?
We
conclude:
maybe
first
the
evaluation,
storage
for
ii,
students
and
the
benchmark
steps
and
test
case,
and
also
some
maybe
the
performance
metrics
for
stm
evaluation,
for
example,
constellation
utilization
can
be
standardized
in
buy
some
the
draft
or
work.
H
Future
integrates
space
and
terrestrial
networks
or
stn
holds
great
promise
for
pervasive
low
latency
and
high
throughput
internet
service,
but
realizing
such
promise
needs
the
right,
evaluation
and
storage
to
comprehensively
and
systematically
evaluate
and
understand
new
network
designs
for
istn
before
the
real
deployment
and
launch
existing
assessment
tools
or
benchmark
tools.
Plan
phones
and
test
cases
are
insufficient
for
comprehensive
and
systematic
evaluation
for
ictn
and
we
need
the
right
evaluation.
Storage,
turret
for
tailored
for
the
future
icn,
for
example,
constellation
and
network
realism,
flexibility,
low
cost
and
data
trace
and
test
case.
H
A
Well,
thank
you.
Zeki,
you've,
you've
run
through
this
very
efficiently
and
and
all
your
references
are
available
here,
yeah.
So
we've
we've
got
all
that
material
available
and
this
might
be
a
slide
to
leave
up
for
for
folks
to
kind
of
refer
to.
Let
me
open
the
four
up
to
to
comments
and
if
folks
have
any
questions
or
or
additional
observations,
they'd
like
to
make,
maybe
we've
got
some
satellite
well-worth
orbit
system
users
out
there?
Who
would
like
to
comment.
D
Well,
I
am
definitely
not
qualifying
into
the
latter
part
of
al's
comment
there,
but
I
would
ask
you
know
first
thing:
this
is
super
interesting,
I'm
wondering
if
you
have
any
participation
from
operators
or
folks
who
are
running
these
satellites
or
ist
and
networks
for
for
lack
of
a
better
term
here.
Do
you
have
any
involvement
from
folks
outside
of
the
educational
forum.
H
Okay,
thank
you.
Sarah.
Thank
you
for
this
question.
Actually,
our
team
have
been
working
on
the
ice
tn
for
many
years
and
as
as
we
have
listed
in
the
reference
page,
so
recent
so
over
the
past
decades.
Maybe
we
have
some
research
work
and
from
maybe
from
paperwork,
or
we
also
submit
some
discussion
or
draft
in
the
standard
group,
for
example
itf
or
iq,
and
for
your
question
I
mean
today.
Maybe
most
of
our
work
are
done
by
our
groups,
but
in
the
future.
Maybe
if
we
can
agree,
this
is
a
very.
H
After
after
we
clarify
this
problem
statements
and
requirements.
If
it
is
needed,
we
can
ask
for
some
further
collaboration,
maybe
with
the
satellite
operators
or
maybe
the
cloud
operators
and
the
industry
satellite
industry.
D
Yeah,
I
definitely
see
I
definitely
see
some
room
for
clarification
here
and
we
can
provide
those
comments
to
you
on
list
right.
There
is
a
very
high
level
view
of
requirements,
but
I
think
getting
to
the
next.
D
And
merging
what
you,
some
of
what
you
said
into
text,
would
make
a
lot
of
sense
and
then
I
think,
once
there's
sort
of
a
cohesive,
hey,
here's
what
we
want
to
do,
and
here's
particularly
what
we
want
to
measure
and
why
the
existing
rfcs
that
we
have
don't
cover
those
methodologies
and
you've
touched.
F
D
I
think
then,
my
based
on
what
I
know
today
ahead
of
the
working
group
taking
this
on
or
us
having
a
conversation
around
if
we're
the
right
ones
to
take
this
on,
would
definitely
be
around
hey,
making
sure
that
there's
a
need
for
this
in
the
industry-
and
it
certainly
seems
like
there
is,
but
getting
that
feedback
directly
from
the
folks
who
would
own
these
satellite
or
space
networks,
I
think
is,
is
probably
going
to
be
key
so
that
we
understand
hey
putting
the
time
into.
D
This
makes
a
lot
of
sense
and
again,
that's,
I
think,
a
secondary
point
to.
I
think
there
might
be
a
bit
of
a
question
there
over
whether
or
not
we're
the
right
group
to
do
this
or
not
benchmarking.
Yes,
it's
the
first
bullet
point
on
that
previous
slide
that
you
were
on
around
hey
where
the
satellites
are
positioned
at
once,
we
start
getting
into
space
specific
technologies.
D
But
I
I
to
be
clear:
what
I'm
proposing
is
hey
if
we
have
a
crisp
and
clear
set
of
problem
statements
and
the
requirements
that
you're
looking?
What
are
you
looking
to
test
and
measure
and
why
what
we
have
today
doesn't
help.
I
think,
once
that's
crisp
it'll
be
easy
to
have
or
what
I
propose
is.
Then
we
could
have
that
conversation
around.
Does
this
make
sense
for
us
or
if
you
can
remove
certain
things
out
of
out
of
scope?
Would
it
make
sense
for
us
al?
D
I
was
thinking
ahead
a
little
bit
there
to
hey
if
they
want
to
test.
You
know
where
satellites,
how
closely
they're,
positioned
or
not,
or
what
have
you
that's
probably
way
beyond
our
purview,
but
if
they
remove
that
from
from
purview
or
make
it
sort
of
a
a
superfluous
detail,
would
that
change
our
stance
and-
and
I
actually,
I
honestly
don't
know
I
I
could
take
a
guess,
but
I'm
proposing
that
hey,
we
might
want
to
have
that
conversation
as
a
group.
A
Sure
sure
yeah
the
yeah,
the
reason.
The
reason
I
put
this
slide
back
up
is
that
you
know
if
you,
if
you
take
away
all
the
all
the
indications
that
things
are
flying
here
and
earth
stations.
A
There
are
there's
some
fundamental
networking
problems
to
solve,
and
that
is
that
there
are,
you,
know,
backbone,
satellites
and
relay
satellites,
and
you
can
think
of
these
as
backbone,
routers
and-
and
you
know
you
know-
maybe
routers
near
the
edge,
but
not
quite
at
the
edge
and
and
and
and
then
you
know,
you've
got
the
terrestrial
network
here.
A
Where
there's
there
are
going
to
be
routers
that
are
connected
to
the
earth
stations
that
are
at
the
edge
of
the
network
and
and
there's
you
know,
there's
real
path,
benefits
that
path,
length,
propagation
delay,
benefits
that
can
be
derived
from
the
additional
flexibility
that
that
a
a
dense
leo
network
can
provide.
A
So
there's,
there's,
there's
clear,
there's
clear
advantages
to
you
know
to
this
work
on
on
paper.
You
know
from
from
many
perspectives,
but
as
as
zeki
lai
mentioned,
there's
there's
also
a
lot
of
expense
involved
in
in
building
these
networks
and
maintaining
these
networks.
Just
and
you
know,
the
main,
the
maintenance
part
is
is
hard
because
these
satellites
are
in
space
and
they
can't
be.
You
know
easily
serviced
beyond
a
kind
of
like
a
reboot
that
brings
online
redundant
equipment.
A
Things
like
that.
So
it's
a
you
know
it's
a
it's,
a
cold
lonely
problem
to
to
try
to
fix
these
things
and
and
the
you
know
the
satellite
processors
in
space.
I
I
think
you
know
I
think
zeki
it
might
help
a
little
bit
for
me
to
add
my
personal
perspective
here
I
spent
I
spent
the
first
eight
years
of
my
career
working
in
satellite
communications
and
and
back
then
I
mean
back.
H
H
A
That's
right,
that's
right,
but
the
but
the
most
I
think
the
most
important
part
of
it
was
that
we
found
ways
that
we
could
test
in
test
the
satellite
network
in
the
laboratory
with
without
some
of
the
without
some
of
the
pieces.
Let
me
let
me
let
me
just
put
it
that
way.
A
We
we
had
a
laboratory
where
we
could
test
all
of
the
ground-based
systems
and
the
you
know,
and
the
and
the
networks
that
connected
up
to
the
satellite
network
and
all
of
the
frequency
conversion,
and
then
maybe
the
big
surprise
is
that
we
had
the
actual
what
they
called
the
brass
board
version
of
the
satellites.
A
And
so
we
were
we
dispensed
with
antennas
and
high-powered
amplifiers,
but
we
connected
right
up
to
the
right
up
to
the
functional
satellite
with
you
know,
using
waveguide,
and
you
know
that
was
a,
and
that
was
a
very
valuable
thing
to
do,
because
very
often
most
of
the
questions
we
were
asking
had
to
do
with
you
know
how
many,
how
many
channels
can
we
get
going
on
this
satellite,
with
this
new
form
of
modulation
and
and
and
so
forth?
A
And
and
finally,
we
did
bring
in
some
antenna
based
testing
with
the
third
generation
of
the
satellites
that
that
we
tested.
We
ended
up
with
two
two
big
brass
board
versions
of
the
of
the
satellites
there
with
a
traveling
wave,
tube
and
silicon
based
amplifiers.
You
know
the
technology
changed
over
time.
It
was,
I
mean
all
I'm
saying.
Is
there
were
ways
to
scale
down
from
the
you
know,
the
the
complexity
of
actual
satellite
live
network
usage
and
and
still
get
very
meaningful
results.
A
So
you
know
that's
the
challenge
that
we
would
be
faced
with
here
in
in
trying
to
get
this
working
in
a
laboratory
and
in
a
benchmarking
kind
of
environment
that
we
could
that
we
could
call
on-
and
maybe
that's
the
kind
of
thing
that
that
would
only
answer
a
limited
range
of
questions,
but
it
still
might
be.
It
still
might
be
useful
to
go
after
that.
So
you
could
imagine
you
know.
A
I've
been
I've
been
thinking
about
the
the
dynamics
of
this
here,
the
the
dynamics
of
the
the
multiple
shells
of
the
orbits
that
are
talked
about.
I
mean
all
the
all
the
orbits
you're
talking
about
are
are
higher
than
the
international
space
station
and
and
the
international
space
station
is
at
220
miles
about
330,
kilometers
or
something
like
that
and
and
it
traverses
the
visible
sky
in
less
than
a
minute
it.
It
really
seems
to
fly
by
you're
talking
about
the
low
earth
orbit
with
three
minute
visibility.
A
So
that's
I
mean
that's
a
that's,
actually
a
big
step
up
you
could,
you
could
potentially
use.
You
know
one
satellite
for
three
minutes,
and
that
starts
to
give
you
the
kind
of
you
know
the
pace
of
of
connection,
changing
that
that's
going
on.
Also,
I
attended
a
workshop
back
in
september
and
at
the
end
of
the
workshop
one
of
the
participants
who'd
been
you
know,
very
active,
very
vocal
and,
and
you
know
really
nice
guy.
A
He
he
revealed
to
us
that
he
was
using
a
a
leo
satellite
service
and
yeah.
We
did
see
a
couple
of
interruptions
when
he
was
speaking
and
and
things
like
that,
but
no,
no
more
than
what
you
see
when
you
know
when
somebody's
trying
to
use
the
terrestrial
background.
A
So
so
I
I
thought
that
was
quite
amazing.
He
walked
around
the
the
farm
where
he
was
and
showed
us
the
dish
up
on
on
top
of
a
barn
and
stuff
like
that
it
was
really
cool.
So
you
know
this.
This
stuff
is
a
reality
right
now.
That's
the
most
important
comment
I
wanted
to
make
and,
and
it's
it's
it's
just
a
matter
of
of
designing
the
right.
You
know
a
scope
of
testing
that
would
produce
some
meaningful
benchmarks
and
and
reasonable
results.
A
H
Okay,
thank
you
al
and
thank
you.
Sarah.
Thank
you
very
much
for
all
your
comments,
so
I
want
here.
Maybe
let
me
explain
why
we
want
to
submit
this
draft,
because
during
our
previous
studies
in
istn,
we
found
that
there
are
a
lot
of
internet
working
technology
and
challenges
facing
the
istm
because,
as
we
have
introduced
before
the
the
high
dynamics
and
the
constraint
resource
and
the
low
the
high
operational
cost.
H
So
if
we,
if
we
want
to
unleash
such
a
lot
of
potential
of
the
istn,
for
example,
pervasive
connectivity,
low
latency
and
throughput,
we
need
a
lot
of
new
research
and
we
know
the
experimental
or
the
evaluation
storage
play
a
very
important
role
during
the
evolution
in
the
evolution
of
our
internet.
So
we
believe
the
same
thing
for
istn.
We
need
the
right
evaluation
methodology
and
test
case
for
the
istn
to
evaluate
evaluate
a
lot
of
internet
working
technologies.
H
We
would
believe
that
that
fits
the
charter
of
the
bmw.
So
that
is
why
we
write
loads.
We
wrote
this
draft
and
some
here,
but
I
also
agree
with
sarah
that
maybe
we
need
some
clarifications
for
the
problem
requirements
and
the
most
most
important,
maybe
the
performance
metric,
for
example,
whether
we
can
use
the
traditional
metrics
like
the
latency
or
support
coverage
and
other
metrics
to
quantify
the
performance
of
the
benchmark
of
the
isdn
or
maybe
we
need
to
seek
for
some
new
specific
metrics
for
istn,
maybe
like
the
constellation
just.
H
I
have
very
simple
question.
For
example,
starlink
has
already
launched
about
1500
satellites
in
the
orbit
and
they
plan
to
launch,
for
example,
43
thousands
of
satellites,
but
how
many
satellites
really
we
need
to
support
the
global
internet?
So
what
is
the
constellation
utilization
and
how
can
we
evaluate
manning
the
network
performance
from
the
network
perspective
of
istn,
so
such
metric
or
such
test
case
and
benchmark
methodology
can
guide
us
to
construct
the
future
istn.
H
So
that
is
why
we
want
to
roll
this
draft
and
submit
this
draft
to
this
group
and
I'm
very
happy
to
if
I
can
get
some
further
recommendations
or
suggestions
from
the
chair
of
the
and
other
bmw
folks.
So
you
are
more
than
welcome
to
give
any
questions,
recommendations
and
suggestions
from
the
mail
list
or
just
direct
mail
to
me.
Thank
you.
A
Well,
you're
very
welcome,
thank
you
and-
and
you
know,
I
think
I
think
one
of
the
one
of
the
other
points
that
came
up
is
is
if
we
can
recruit
some
new
members
of
the
audience
from
these
new
networking
providers,
the
the
space
leo
network
providers,
I
mean
that,
would
that
would
give
us
a
real
leg
up
in
trying
to
make
the
decisions
of
what's
valuable
to
this
audience
and
what
would
they
like
to
know
now
and
what?
A
How
would
this
help
them
choose
the
equipment
they
want
to
add
to
their
networks
and
and
modify
existing
protocols
and
so
forth?
So
these
are
the
kinds
of
questions
that
you
know.
We
could
possibly
begin
to
help
with
in
in
bmwg,
but
but
there's
a
there's
a
ways
to
go
here.
A
I
I
mean
most
of
most
of
what
I
hear
is
is
research,
and
I
think
I
think
everyone
would
generally
agree
agree
with
that
at
you
know
at
this
exact
point
of
time,
although
you
know
there
are
production
networks
out
there,
so
that's
the
you
know
that
that's
the
other
side
of
it
yeah
it's
it's
it's
it's
it's
way
way
out
of
the
lab.
A
It's
people
are
using
these
things
right
now
and
and
40
4500
satellites
is
nothing
to
sneeze
at
so
very
good,
all
right,
so
we're
a
little
bit
over
time,
but
I
I'm
really
glad
that
in
this
session
we
had
some
chance
to
give
some
extra
time
to
this
techie,
because
you
know
this,
it
just
happened
to
work
out
that
we
had
some
extra
time
in
this
in
this
session
to
handle
it.
So
usually
we
don't
so.
F
A
Worked
out
well
all
right!
Well,
we
encourage
you
to
keep
working
on
it
and
try
to
try
to
draw
in
some
audience
members
and
then
we'll
you
know
from
the
industry
and
then
we'll
take
it
from
there.
Thank
you.
A
And
thank
you,
okay.
So,
at
the
end
of
our
agenda,
any
other
business.
A
Okay,
well
thanks
very
much
for
participating
today,
everybody,
it
was
a
useful
session
and
we
shared
a
lot
of
information
and
test
results
and
progress,
and
I
think
that
that's
what
we're
always
looking
for
here
during
our
sessions.
A
So
again,
thank
you,
we'll
see
on
the
mailing
list
and
bye
for
now.