►
From YouTube: Workshop on Analyzing IETF Data (AID), 2021-12-02
Description
Day 2 of the IAB's Workshop on Analyzing IETF Data (AID), 2021-12-02
Workshop page: https://www.iab.org/activities/workshops/aid/
Day 1: https://youtu.be/AAliAmBpzII
A
A
So
you
can
access
the
video
through
the
following
link.
Okay
and
then
I
have
a
brief
review
of
the
hack
group.
Okay
and
thanks
for
the
mirror's
work,
and
also
this
summarizes
the
possible
questions
and
to
set
up
the
possible
the
hexane
groups.
So
here
is
the
brief
review,
so
the
group
one
is
for
the
improved
toolings.
A
A
A
A
nick
is
response
for
this
group.
Okay,
okay,
so
this
is
just
a
brief
review
of
this:
the
hexane
groups.
Okay,
during
the
two
days
of
the
hacksaw
activity,
we
also
have
the
two
sync
ups
meeting,
so
that's
in
the
second
meeting,
so
they
will
have
this.
The
extensive
discussion
most
of
the
discussion
is
about
the
date
and
also
the
tools
and
also
introduce
this,
the
progress
of
the
hacksaw
rules
and,
besides
the
data
and
the
tools
about
the
action,
we
also
have
this.
A
The
extensive
discussion
of
the
different
topics,
and
also
this
is
to
be
honest
beyond
my
expectation.
So
that's
we
talked
about
how
to
make
the
work,
suspend
and
also
how
to
provide
the
necessary
data
and
also
that's
when
do
the
analysis
work
and
provide
the
possible
data.
The
privacy
issues
are
involved
and
also
be
discussed.
A
Okay,
so
this
is
just
a
brief
review
about
the
monday's
workshop
and
also
the
hacksand
groups
and
the
syncopa
meetings.
So
then
this
is,
that
is
the
agenda,
so
we
will
go
on
to
discuss
the
environmental
sustainability,
so
this
is
according
to
the
position
paper.
We
have
the
initial
result
discussion.
A
A
B
B
C
C
So
in
this
in
this,
in
the
beginning
of
the
workshop,
we
were
yeah
mapping
out
a
strategy
and
a
methodology.
How
we
can
try
to
yeah
assess,
assess
our
the
question
that
we
post
ourselves
and
we
take-
I
guess,
a
little
bit
of
a
similar
approach
as
we
have
seen
that
that
of
livers
did
in
the
ncc
group.
Research
on
on
potential
security,
vulnerabilities.
C
C
Which
is
used
to
filter
out
relevant
pieces
of
text,
so
you
know,
for
example,
so,
and
also
I
mean
you
will
find
some
of
these
words
are
maybe
more
ambiguous.
They
can
appear
in
different
contexts
and
that's
why
it's
important
to
later
on
yeah
filter
out
the
the
relevant
pieces
of
text
that
are
really
concerned
about
the
environment.
Of
of
you
know,
ecology
and
not
the
environment
of
you,
know
computer
infrastructure
and
next
slide.
C
Please,
and
so
yes,
the
the
data
acquisition
has
been
taking
up
a
lot
of
time,
so
we
have
tried.
So
here
again
it
can
be
argued.
I
mean
we
have
tried
to
to
you
know
to
to
make
an
approach
based
on
volume.
I
mean
something
that
we
can
finish
this
week
and
doesn't
take
too
long.
So
you
know
if
you
would
go
through
all
the
mailing
archive.
I
mean
we
would
be
busy
for
for
months
if
you
don't
have
the
necessary
cpu
power.
C
So
so,
basically,
the
the
smallest
amounts
of
text
are
that
can
be
found
within
the
rfcs
and
the
active
group
drafts.
And
while
we
have
already
finished
our
initial
keyword,
search
for
rfcs,
we
are
still
running
our
code
for
the
drafts
of
rfcs
and
the
active
working
group.
Crafts,
while
going
through
meeting
minutes
and
mailing
archives,
is
something
that
we
still
need
to
do.
C
And
then,
in
the
third
step,
next
slide,
please,
after
the
initial
keyword
search
where
we
narrowed
down
the
body
of
texts
that
we
have
to
go
through,
we
then
have
to
find,
as
I
say,
you
know,
those
keywords
they
can.
Some
of
them
are
more
and
bigger
than
others.
They
can
appear
in
different
kind
of
contexts.
C
So
then
we
have
to
find
the
more.
We
have
to
go
through
a
more
complex
procedure
by
using
nlp
to
to
filter
out.
You
know
what
kind
of
environments
the
text
is
speaking
to
and
yeah
in
that
respect.
That
is
something
we
have
not
been
able
to
to
to
work
on
at
all
during
this
time
and
then
the
very
last
step
in
visage.
B
D
C
As
I
say,
yes,
I
mean
we
were,
we
finished
our
first
analysis
of
our
seas
and
I
mean
they're
there
yeah,
we
have,
I
mean
we
yeah
of
all
the
rfcs.
We
I
mean
we
went
through
I've.
Seen
one
to
you,
know
the
most
recent
one
and
we
have
basically
I
mean
we
can
counter
it
on
our
hands.
The
number
of
documents
where
we
have
seen
those
keywords
popping
up.
C
I've
only
had
time
to
look
into
very
few
of
them
personally
and
see
in
what
kind
of
context
they
appeared
and
well
yeah.
There
were
one
some
there
were
some
oceanographers
involved
from
the
university
of
somewhat
santa
barbara,
on
ucl,
a
and
and
so
yeah
I
mean
we
have
found
some
body
of
text
already
now,
but
it's
really
sparse.
I
mean
it's
very
few.
C
C
E
D
Do
you
have
an
idea
about
what
what
kind
of
recommendation,
what
kind
of
insights
might
come
out
of
this
for
the
itf
or
for
anything
that
is
actionable?
Like
I
mean
maybe
sure
the
itf
should
should
take
more
into
consideration
the
energy
consumption
of
the
protocols
we
develop
or
whatever,
or
do
you
think,
there's
anything
more
concrete.
C
Well,
I
mean
at
the
moment
I
just
maybe
yeah,
maybe
making
it
making
it
a
point
in
the
minutes
at
all
is
maybe
already
like
something
that
is
helpful.
What
I
just
see
at
the
moment
is
that
that
you
know
icts
is
being
considered,
as
you
know,
something
as
the
solution
for
the
climate
issue
like
we
ne
like,
if
you
see
a
and
policy
reports
or
how
the
yen
talks
about
the
energy
transition,
then
it's
all
based
and
founded
on
the
necessity
for
smart
grids
and
iot.
C
So
just
just
maybe
you
know,
keeping
it
or
like
keeping
it
or
introducing
it
as
a
point
in
the
minutes
or
in
the
discussion
is
already
helpful
enough
in
the
first
instance
and
then
later
on.
We
maybe
we
see
that
there
are
some
other
steps
that
can
be
taken
to
to
yeah
make
yeah
give
some
more
concrete
results.
B
C
C
B
H
H
H
B
H
The
rfcs,
the
recent
one
that
I
checked
actually
like
I
have
I
have
actually
like
because
I
sorted
them
out
based
on
the
rfc
numbers.
Some
recent
rfcs
like
starting
like
the
most
recent
one
that
in
talked
about
climate,
is
rfc-8752
or
something
like
that.
But
that's
only
like
talking
about
environment
and
bracket
climate,
so
it's
a
false,
positive
and
and
also
like
some
other
drops
did
mention.
H
But
I
haven't
looked
at
the
the
drafts,
actually,
the
names
of
the
drafts
and
what
they're
talking
about,
but
yeah
like
from
seven
thousand
cities
and
seven
thousand
two
between
seven
thousand
to
nine
thousand.
I
can
see
that
six
or
seven
six
drafts
actually
mentions
climate
or
ten
times
in
total,
but
I'm
curious
to
know.
Actually,
maybe
that's
it's
good
to
see
like
in
which
context
they're
using
climate.
B
I
I
I
We
found
a
couple
of
rfcs
with
some
concern,
but
we
actually
found
a
bunch
of
active
internet
drafts
that
had
more
dangerous
things
in
them
than
in
the
rfc.
So
I
I
encourage
you
to
look
through
you
know,
even
though
rfcs
have
a
higher
standing
in
the
itf,
I
encourage
you
to
keep
looking
at
the
drafts
and
to
dig
down
further
there,
because
that
when
we
fixed
a
few,
we
didn't
have
to
fix
much
for
the
y2k
things.
I
A
A
So
that's
the
accordion
to
collect
the
our
power
consuming
information
will
calculate
the
possible
pass,
so
that
time
is
that
time
is
a
very
popular
topic,
but
later
they
found
this,
the
topic
is
the
the
requirement
may
be
too
early.
So
that's
this
work
you
does
not.
It
did
not
move
on
so
and
also
all
the
or
almost
most
of
the
drought
are
the
individual
drought
and
expert.
A
B
E
Hey
colin,
it's
corinne,
can
you
hear
me?
I
raised
my
hand
but
I'll
jump
in
because
you're
you
lost
your
connection.
I
wanted
to
do
the
same
thing
as
the
previous
speakers,
and
that
is,
you
know,
really
encouraged
his
work.
I
think
it's
incredibly
important.
I
really
appreciate
it.
It's
also
great
to
hear
that
you've
been
speaking
to
michael
ogia
about
this.
Another
thing
that
I
wanted
to
maybe
flag
is
that
the
ford
foundation
is
actually
also
working
specifically
on
issues
of
like
internet
infrastructure
and
sustainability.
K
And
if
I
could
add
to
the
choir
is
that
there
have
been
so
like
alternative
internet
infrastructures
such
as
scion
have
been
making
quite
some
claims
about
more
efficient
routing
which
would
cause
less
carbon
emissions.
I'm
not
sure
what
that's
actually
been
substantiated
and
not.
It
could
be
done
with
other
approaches
such
as
as
sets,
but
I
do
hope
that
your
analysis
could
perhaps
spark
such
discussions
and
in
the
anrp
there
have
already
been
interesting
papers
on
this
topic
so
yeah.
B
G
G
So
we
had
some
chat
conversation,
but
I
tried
to
to
try
to
to
provide
some
numbers
around
that,
so
how
much
co2
do
we
have
so
the
problem
I
mean
so
I'm
currently
only
focusing
on
f
light.
The
co2
emissions,
as
I
said,
to
earth
flight
and
the
problem
I
had
is
that
well
we
have
to
estimate
which
flight
each
of
the
attendees
are
taking,
and
I
mean
flights
are
not
always
direct
from
the
origin
of
the
attendee
to
to
the
the
meeting
location.
G
G
G
So
I
did
that
for
every
participant
and
every
itf
meetings
and
on
the
diagram
here
I
plotted.
According
to
so
each
attendee
can
be
on-site,
an
on-site
attendee,
a
remote
attendee
or
an
attendee
that
has
not
arrived
so
for
each
of
these
participants.
I
took
the
origin
location
and
I
estimate
a
flight,
so
a
real
flight,
multiple
segments
and
so
on,
and
I
estimate
from
that.
The
co2
for
each
of
those
participants.
G
So
for
that
I
I
tried
to
estimate
I
I
took
some
data
about
how
much
co2
per
capita
are
being
emitted
for
different
countries,
and
I
try
to
compare.
I
mean
how
much
is
one
me
attending
to
one
meeting,
two
meeting
three
meetings
and
so
one
way
to
represent.
That
is
when
you
see
on
the
map,
so
I
mean
one:
two
2.7
is
here
for
one
itf
meeting
2.
two
meetings
attending
to
two
meetings.
G
You
you,
you
likely
changed
category
and
I
mean
three
meetings
here
is
over
there.
So
I
mean
we
do
actually
get
into
the
dark,
color
and
and
and
just
to
be
clear.
It
only
represents
the
co2
that
is
needed
to
attend
the
meeting
versus
the
co2
that
one
person
in
a
given
country
is
needing
to
to
basically
leave
a
year.
G
G
If
you
attend
two
meetings,
you
have
some
european
countries
such
like
greece,
italy,
uk
one
meeting
I
mean
in
this
area,
you
have
a
the
co2
per
capita
is
closer
to
mauritius
or
venusaur,
but
I
mean
it's
hard
to
to
to
figure
out
what
people
living
in
venezuela
what
kind
of
life
size
they
have,
but
it's
just
a
number
of
found
yeah.
So
that's!
G
G
G
G
So
after
2024
and
then
you
have
the
long
term
decrease
decline,
which
is
a
10
percent,
so
a
shorter
growth
and
then
you
have
a
decrease
and
that
scenario
is
a
responsible
for
only
0.04
degree
increase,
and
what
what
we
need
to
consider
is
that,
even
if
you
stop
aviation
now
and
no
flight
at
all,
aviation
will
still
be
responsible.
In
2050
of
I
mean
some
increase
of
temperature,
because
it's
what
really
matters
is
the
accumulation,
so
I
mean
whatever
solution
we
find.
G
G
G
G
I
mean,
I
know
there
are
a
lot
of
effort,
especially
I
mean
the
mythical
teams,
or
even
I
mean
things
that
are
not
obvious,
but
we
during
the
virtual
meetings,
I
think
doing
the
agenda
and
choosing
the
time
slot
and
all
this
is
as
have
not
been
part
of
that
effort.
But
I
am
I
I
can
see
that
a
lot
of
effort
has
been
put
on
that.
G
G
B
G
K
G
B
K
B
D
D
G
Also,
so
that's
that's
really
to
discuss.
I
I,
I
think
yeah.
Of
course,
if
we
go
to
zero,
that's
better
in
terms
of
co2,
but
I
I
think
we
I
mean
I'm
the
the
way
I
see
that
is,
we
might.
We
might
have
a
transition
with
a
one
meeting
per
year,
and
then
I
mean
we
have
a
few
years
to
decide
if
you,
if
we
go
to
zero
or
if
we
keep
to
one.
G
I
think
the
most
important
thing
for
now
is
not.
I
mean
to
try
to
to
to
limit
the
number
of
meeting
and
to
lower
that
number
and
and
then
I
think
that
requires
more
work.
I
would
say
it
might
be
better
to
have
one
meeting
everyone
goes
to,
but
then
of
course
I
mean
it's
probably
I
I
think
hybrid
meeting
are
going
to
be
very,
very
challenging.
So
that's
a!
G
G
So
so
it
might
be
interesting
because
it
removes
this
local
aspect,
and
maybe
it
gives
us
something
smoother
to
to
see
some
trends
or
maybe
considerations.
So
I
think
it
might
also
be
a
nice
angle
to
to
look
at
to
analyze
the
evolution
of
the
itf
or
the
growth
potential
growth
of
the
itf,
but
from
the
graph
here.
I
I
think
it's
pretty
clear
at
least
to
me
that
there
is
a
a
growth
associated
to
the
remote
participation.
G
I
mean,
if
you
you
say
the
student
yeah
you're,
going
to
work
for
that
draft,
but
you
you're
never
going
to
go
to
the
itf
meeting.
It's
not
as
motivating,
for
example,
to
say.
Yeah
I
mean
itf
can
be
virtually
attended
and
we
are
doing
those
two
meetings.
We
can
do
that
meetings
together
so
for
local
engagement.
I
think
it
might
also
be
helpful
to
and
and
very
beneficial
to
for,
the
itf
to
have
those
virtual
meetings.
B
B
C
Yeah,
this
work
is
very,
very
great
thanks
for
presenting
it,
and
I
was
just
wondering
I
mean
I
didn't
understand.
I
think
I
didn't
fully
understand
our
aspects
of
it.
I
was
wondering
how
you,
how
you
you,
how
you
think
about
carrying
it
further,
are
you
thinking
about?
I
mean
sophie
if
I
mean
there's
a
lot
of
work
going
on
and
I
mean
they're
already,
you
know
it's
already
used
like
sustainable
aviation
fuels.
Are
you
taking
this
into
consideration?
C
I
think
I've
taken
this
into
consideration.
Maybe
if
you
you're
going
to
evaluate
it
further,
I
mean
it's
going
to
be
there
by
by
you
know
by
2030
for
sure
and
earlier
of
course,
as
well
so
and
whether
there's
no
kind
of
like
policy
interventions
you
you're
envisaging.
Is
this
also
yeah?
As
you
say,
I
mean
you
know,
if
someone's
traveling
locally,
they
will
consume
less.
So
you
know
when
you
say
that
an
attendee
is
restricted
to,
like
you
know,
attending
one
meeting
per
year.
C
G
G
I
mean
it's
thoughts
to
be
very
complex
and
it's
a
and
and
then
it's
very
easy
to
I
mean
I
mean
I
am
not
working
in
the
aviation
sector,
so
I
didn't
want
to
make
a
a
complex
mode.
That
is
maybe
completely
out
of
scope,
so
I
I
mean.
Currently,
this
is
the
most
I
mean
basic
model
I
am
applying
to
and
but
yeah
it
might.
It
might
change
the
the
data.
G
G
G
I
mean
I
mean,
assuming
that
the
the
more
connection
you
have,
the
more
time
you're
gonna
spend
on
earth
on
airports
and
the
the
higher
the
probability
would
be
that
you
either
catch
the
virus
or
that
you
participate
to
the
spreading
of
the
wires
is
so
I
looked
at
the
number
of
legs
per
location.
It's
not
very
clear
what
the
strategy
could
be,
but
further
work
is
probably
needed
around
that.
I
think
I
found
that
the
japan,
china
and.
B
B
G
G
L
G
L
So
you're
you're,
then
you're,
relying
on
the
metric
coming
out
of
your
underlying
tools.
That
is
a
per
passenger
metric
and
bake
into.
That
is
some
assumption
about
the
averaging
all
the
passengers
out
over
all
of
the
existing
flights,
so
that
yeah
there's
an
assumption
that
the
distribution
of
the
ietf
passengers
on
these
planes
matches
the
general
distribution
of
the
population
on
planes.
And
we
know
that's
not
true.
L
G
G
So
the
difference
would
be
it's
not
the
same
that
10
person,
10
attendee
flight
from
sf,
to
let's
say
london
as
one
person
from
san
francisco,
another
one
from
atlanta,
another
one
from
johannesburg
and
one
from
beijing
yeah
so
yeah.
So
I
agree
so
the
current
model
I
am
applying
to.
Yes,
they
do
define
some
general.
They
quantified
that
and
it's
it's
the
same
everywhere
around
the
world,
so
one
for
example.
What
could
change
the
data
would
be?
G
G
G
L
G
G
G
G
G
G
I
mean
I
I
mean
those
have
been
slightly
cons,
even
if
you
consider
short
short
hole,
planes
and
long
haul
planes
which
you
could
think
makes
a
huge
difference.
Well
for
a
given
flight.
The
difference
is
not
that
much
I
mean.
If
I
saw
I
mean
if
I
plot,
for
you
the
co2.
According
to
the
distance,
using
those
two
types
of
planes,
I
mean
there
is
a
small
difference,
but
it's
not
a
huge
one.
You
you
will
probably
not
see,
see
it
until
I
I
I
give
you
the
equations.
G
G
L
L
K
J
K
J
Okay,
so
this
was
a
diversity
and
inclusion
group
in
hackathon
we
sort
of
splintered
off
into
individual
sort
of
specific
projects
that
we
each
worked
on
and,
as
a
result,
we've
got
a
sort
of
set
of
slides
that
we've
each
added
a
couple
of
slides
to.
So
if
we
go
to
the
next
one,
I
don't
know
how
slide
this
is,
admittedly,
but
whoever
it
is.
If
you
want
to
talk
through
it,.
O
That
was
my
slide.
I
can't
even
recognize
my
own
flight.
Well,
I
originally
put
it
later
in
the
deck,
so
I
thought
there
was
no
way.
I
was
first
all
right,
I'm
I
apologize
so
one
of
the
things
I've
been
looking
at
measuring
is
international
cooperation
in
a
number
of
different
ways,
and
I
took
some
past
work
that
I
did
were
analyzing
email
archives
to
extract.
You
know
what
systems
are
communicating
with
other
systems,
so
I
took
the
top
20
mailing
lists
from
2020.
O
Thank
you
to
someone,
I'm
blanking
on
the
name
who
supplied
me
that
list
of
of
the
top
most
mailing
lists
list
included.
Things
like
you
know,
get
quick
issues
and
things
like
that
and
I
purchased
the
headers
to
look
through.
You
know
where,
where
is
traffic
coming
from,
and
then
I
created
a
node
node
edge
graph
out
of
that
and
then
plotted
it.
Of
course,
the
the
hardest
part
about
that
is.
O
So
in
the
results
you
know,
I
do
have
some
interesting
proof
of
concept
success
that
makes
me
think
this
is
worthwhile
to
continue
looking
at
there's
a
lot
more
work
to
be
done
in
terms
of
collecting
data
as
well
as
cleaning
it
and
and
better
parsing
it
that's.
Actually,
I
think
where
the
majority
of
the
effort
is,
and
then
you
know
it
would
be
nice
to
tie
it
to
other
data
systems
as
well.
L
O
So
this
is
my
initial
results
and
it's
both
interesting
in
what
it
shows,
and
it
also
shows
that
how
broken
it
is
so
all
of
the
diamond
shapes
which
are
light
and
hard
to
see.
But
you
can
see
the
name
of
the
working
groups.
In
particular,
you
can
see
sort
of
dns
up
at
the
top
middle
atd
on
the
far
left,
working
group
chairs,
tx,
off
and
v6
offer
on
the
far
right
and
then
they're
connected
by
country,
dots
for
who
actually
contributed
to
that
particular
working
group.
Now.
O
The
reason
I
know
this
is
very
broken
is
because
it
shouldn't
be
the
segmented
right.
Almost
all
countries
should
be
contributing
to
most
groups
and
yet
for
some
reason,
ro
on
the
right
hand,
side,
which
is
that
romania,
I'm
not
sure,
is
like
the
only
country
contributing
to
v6x
tx,
often
working
group
chairs.
That
makes
no
sense
right.
You
know
I
happen
to
know
that
I'm
in
the
united
states
and
contributed
a
lot
in
2020
to
dns
op,
but
yet
the
us
isn't
in
there,
which
is
where
I
live.
O
So
there's
work
to
be
done
to
increase
the
data,
but
it
at
least
begins
to
show
me
that
this
sort
of
you
know
graphic
pictorial.
I
actually
intended
to
to
duplicate
all
the
countries
around
each
of
the
working
groups
just
to
make
it
cleaner
in
the
long
run,
so
it
wouldn't
be
such
a
mess,
but
the
initial
plot
came
out
so
clean
that
I
decided
to
include
that
instead
because
there
was
no
need
to
do
that
that
separation,
so
you
know
the
other
interesting
thing
I
realized
the
study
would
be.
You
know
if.
L
O
Can
do
this
just
for
2020,
with
even
more
working
groups
and
cleaner
data,
comparing
it
to
2018
or
something
like
that
to
see
you
know.
Is
there
any
differences
between
in
pandemic
and
out?
I
doubt
it,
but
it
would
be
interesting
to
see
so
that's
where
I
am
a
lot
more
work
is
needed,
but
I
actually
did
get
some
stuff
done
this
week
and
that's
it
for
me.
O
I
pursed
the
received
headers
for
where
mail
was
actually
coming
from
and
then
from
that
I
either
tried
to
you.
If
if
the
ip
address
was
in
it,
then
I
would
use
standard
databases
for
determining
ip
addresses.
I
didn't
use
standard
database
from
2020,
so
the
other
thing
to
do
would
be
you
know,
collecting
mappings
over
time
and
trying
to
pick
one
from
near
that
time
period
because
address
allocations
do
change
around
the
world.
O
If
the
server
name
was
not
available,
I
would
actually
try
and
resolve
it
to
an
ip
address,
and
if
that
wasn't
available
I
would
actually
use
the
two-letter
country
code
if
it
came
through
something
with
a
two-letter
tld.
So
you
know,
unfortunately,
things
like
gmail
are,
you
know
completely
useless
for
doing
determination
right?
O
Q
O
Yeah
and
that's
actually
what
I
sort
of
said
that
tying
into
the
data
tracker
would
be
another
interesting.
You
know
way
to
approach,
and
it
would
actually
be
interesting
to
figure
out
if
data
from
somebody
identifying
in
a
country
in
the
data
tracker
is
actually
sending
mail
from
a
large
number
of
places
or
or
you
know,
routinely
not
living
in
the
country
that
they
say
that
they're
associating
with,
for
example,
yeah
yeah
yeah.
I
think.
P
P
One
would
rather
want
to
go
to
the
highest
quality
libraries
available
elsewhere
in
the
world,
so
it's
possible,
but
maybe
not
high
priority,
but
one
thing
that
I
I
did
learn
this
week
and
this
is,
I
guess,
more
generally,
the
type
of
thing
that
this
workshop
is
for,
that
we
learned
that.
Oh
those
people
have
this
interesting
data
that
I
could
use
for
this.
P
So
so
I
understood
that
niels
and
nick
and
so
on
were
working
on
some
classifications
of
organizations,
I'm
sure
they
will
talk
about
more
in
a
bit.
I
could
really
use
this
because
I
have
you
know:
company
crafts
and
country
and
continent
crafts,
and
I
could
easily
make
like
operator
versus
vendor
versus
academic
plot.
I
think
that
would
be
really
interesting.
P
So,
but
I'm
not
going
to
do
it
myself,
I
will
wait
for
for
the
source
data
of
the
classification
to
be
available
and
then
hack
on
it
in
the
rest
of
the
year.
I
think
maybe
we
go
the
next
slide
also
because
I
added
this
slide
as
well,
and
this
is
because
we
did
have
some
discussion
of
the
privacy
considerations
and
I
guess
this
is
not
sort
of
a
comprehensive
treatment
of
the
topic.
It's
just
to
flag
this
that
this
is
also
important.
P
The
ietf
has
been
relatively
conservative
about
collecting
any
extra
data
and
we
we
ask
for
permissions,
but
at
the
same
time
it's
important.
We
actually
have
some
transparency
and
we
can
understand
like
what's
actually
going
on
like
if,
let's
say
one
company
is
doing
all
the
all
the
standards
or
half
of
the
standards
in
the
internet.
That
will
be
interesting
information
or
if
some
you
know,
parts
of
the
world
or
types
of
organizations
are
not
involved
at
all.
P
P
P
Previously
they
listed
for
each
person
like
their
employment
and
so
on
that
that
information
is
gone,
so
it's
only
publications
available
and
then
for
companies.
There's
more
information,
yeah!
That's
that's
what
I
wanted
to
flag
and
if
people
can
discuss
we
can,
but
I'm
not
sure
I
have
any
particular
answers
on
this
privacy
aspect.
At
least.
E
If
I
can
hop
in
just
because
the
the
issue
that
yeah
I
just
mentioned
in
terms
of
ethical
guidelines,
I
mean
obviously
all
the
people
who've
been
presenting
here
have
been
doing
so
at
least
the
academics.
Obviously
I
assume
having
gone
through
some
sort
of
ethical
review
of
the
university
system,
but
that
is
the
thing
that
we
might
want
to
consider
for
our
next
meeting.
I
Yep,
where
we
go
away
from
what
corrin
just
said
that
to
me
is
an
especially
interesting
question,
not
just
in
the
ietf
but
in
every
non-academic,
multi-stakeholder
community,
which
is
who
would
be
the
ethics
committee.
Wes
pointed
out
the
other
day
that
icann,
you
know,
has
a
lot
of
participants
and
such
like
that,
and
we
have
an
ombudsman
and
such
like
that,
but
it's
much
much
less
formal
than
any
academic
community
has
so
a.
D
I
mean
this
is
a
very
sensitive
topic,
but
has
kind
of
extensive
guidelines
about
how
to
use
their
platform
correctly,
to
do
studies
that
are
ethically
acceptable.
So
I
think
we
could
actually
put
a
little
more
guidance
about
what's
okay,
to
do
with
the
data
and
what's
not
okay,
to
do
with
it
there
and
how
to
do
it
correctly.
J
Yep,
so
this
is
this
is
me:
I
was
looking
at
using
python
libraries
to
try
and
determine
the
gender
of
meeting
registrations,
the
participants
that
have
registered
for
meetings.
So
basically
I
fetched
the
meeting
registrations
using
the
itf
data
library
and
then
fed
them
into
one
of
these
python
libraries.
J
The
one
I
used
was
gender
guesser.
Just
a
few
sort
of
caveats.
I
guess
the
library
simply
doesn't
have
a
mapping
for
around
a
quarter
of
the
names
in
the
meeting
registration
data
set,
there's
almost
certainly
biases
in
the
disc
that
the
library
is
using,
but
those
aren't
well
documented
and
they're,
not
clear.
It's
not
clear
how
they
line
up
with
the
the
biases
that
are
in
the
itf
data
set
and.
J
It
would
clearly
be
better
to
to
use
sort
of
a
self-co
self-declared
gender
identity
field,
and
that's
that's
something
that
the
itf
community
survey
has
actually
started
doing
in
2021.
So
the
most
recent
survey,
and
so
that
data
should
be
available
starting
this
year.
But
of
course
it
it
misses
out
all
of
the
historical
data,
and
so
something
like
this
is
maybe
useful
for
looking
at
that
that
old
data.
So
if
you
go
to
the
next
slide,.
D
D
J
J
J
I
think
the
interesting
thing
and
you
know
it's
not
a
significant
increase,
but
I
think
if
we
look
at
the
virtual
and
online
meetings
and
towards
the
the
bottom
of
the
plot,
we
can
definitely
see
a
sort
of
increase,
if
not,
as
I
say,
significant
increase,
but
an
increase
nonetheless
in
female
attendance
and
registration
at
online
meetings.
I
wonder
if
this
feeds
into
questions
around
whether
or
not
access
to
meetings
when
they're
online
is
is
better
for
typically
underrepresented
groups
in
the
itf
yeah.
I
haven't
any
run.
J
Any
statistical
tests
see
if
it's
a
significant
result
but
just
sort
of
squinting
at
the
graph,
and
it
looks
as
if
we're
seeing
an
increase,
but
it's
definitely
something
to
dig
into
further.
I
think
so.
I
had
fun
to
extend
this
analysis
to
different
subpopulations
for
beyond
meeting
registrations.
Looking
authorship
and
mailing
this
participation-
and
I
didn't
get
a
chance
to
do
that.
E
P
Okay
to
ask
a
question,
so
I
I
was
wondering
if,
if
you
had
looked
into
the
not
just
the
unable
to
classify
case
but
also
classified
irrationally
case,
because
it
would
seem
to
me
that
if
you
have
something
that
is
in
the
order
of
some
percents,
for
instance,
and
and
you're
trying
to
evaluate
how
many
of
those
are
in
in
the
data
set.
And
then
if
your
error
rate
is
in
the
same
order,
then
you
might
actually
get
interesting
results
and-
and
you
couldn't
really
trust
them
too
much.
J
So,
certainly
in
terms
of
the
the
percentage
of
registrations
that
it's
not
able
to
classify,
that
has
definitely
increased
so
at
the
start
of
the
data
set
so
say,
itf
80,
it
was
about
20
of
registrations,
it
couldn't
map
to
a
agenda
and
by
112
that's
about
28.
I
think
so
it's
definitely
going
up
again.
I
don't
know
what
the
the
biases
are
in
the
sort
of
particular
data
set
behind
the
library.
J
I
think
I
need
to
dig
into
them
a
little
bit
more
to
see
whether
that's
a
significant
result.
If
the
itf
is
becoming
more
diverse
and
we're
using
I'm
using
a
tool,
that's
you
know
using
a
a
data
set
that
isn't
then
you
know
you'd
expect
the
error
rate
to
increase.
But
again
I
would
need
to
dig
into
that
to
see,
if
that's
the
case.
E
I
think
I
remember
that
one
of
the
biases
in
this
database,
because
I
have
a
friend
who
who
wrote
her
phd
about
gender
and
in
github,
specifically
one
of
the
biases-
I
think
she
found
but
I'll,
have
to
double
check
with
her.
So
don't
pin
me
down
on
this
is
that
it's
heavily
biased
towards
names
that
are
common
in
the
west.
So
in
that
sense
the
question
that
yari
asks
in
terms
of
you
know
who
can't
be
classified
could
actually
speak
to
diversity
issues
as
well.
That
are
interesting.
J
K
A
This
is
to
me,
so
I
do
the
hacksaw
work
yeah
and
the
two
days,
because
this
is
always
how
have
to
stay
up
very
too
late
in
china,
because
this
is
a
time
issue,
so
I
have
to
do
some
this,
the
same
for
analysis
to
hear
this
data
source.
So
this
you,
the
most
active
authors
and
also
that's
the
history
of
the
ib
members,
so
I
do
some
of
the
interesting
work
and
so
because
of
the
time
so
you
this
manual
spread
sheet.
So
that's
you
just
this
simple
functionality
for
the
statistics:
okay,
next
slice.
A
Okay
from
this,
the
most
active
authors
according
to
the
yaris,
the
well
website,
so
we
can
see
that
the
average
number
of
the
document
of
most
active
authors
per
area.
Here
we
can
see
that
the
number
of
the
documents
from
the
rtg
area
is
verified.
This
is
almost
the
30
documents
for
the
most
active
authors,
but
for
others,
other
area
is
almost
similar
either
just
use
the
five
between
the
five
and
the
eight.
So
this
is
the
interesting
some
findings.
A
Next,
one:
okay,
so
here
we
also
because
from
the
china
I
have
this
the
advantage
to
do
some,
these.
The
statistics
about
this,
the
chinese
most
active
authors.
So
here
from
here,
we
can
see
that
the
chinese,
the
authors
so
in
the
routine
area,
there's
the
most
number
of
the
active
authors
from
the
china,
but
it
seems
that
the
most
active
authors
know
the
active
authors
for
the
inter
area
and
the
application
area.
This
call
also
shows
the
interest
of
the
research,
and
also
we
see
that
the
sun
is
the
female
of
the
active
causers.
A
From
this
example,
we
can
see
that
the
so
that
we
can
see
this
of
the
20
percentage
of
this.
The
chinese
authors-
so
I
think
this
to
some
instance
is
similar
from
the
findings
from
the
from
this.
The
previous
slides,
okay,
next
one
okay.
So
here
this
is
because
we
have
this.
The
history
of
the
ib
members.
E
A
The
in
the
website
there's
the
name
and
also
there
is
the
tenure
and
this
they
take
this
the
position
of
the
ib
members
that
is
from
antu.
So
let's
have
this
interesting
one.
So
we
can
see
that
the
most
of
this,
the
ib
members,
the
january,
is
the
two
years
four
years
or
six
years,
but
we
also
have
some
the
extreme
this
the
keys
and
also
this
is
the
30
years
or
the
12
years,
but
most
of
this
happens
before
the
200.
But
after
that,
so
this
is
the
tenure.
This
is
always
the
average
okay.
A
K
That
is
excellent,
so
much
so
much
work
done
by
group
two
that
is,
that
is
really.
It
is
really
really
impressive.
It's
really
excellent
work
and
stuff.
We
can
build
on
and
yeah
that's
even
more
than
I
expected
to
be
happen
in
general.
Actually,
but
now
this
is
only
one
part.
So
now
we
go
over
to
the
work
that
has
been
done
in
group
three
sebastian.
Would
you
be
so
kind
to
share
your
slides
and
present
the
work
done
by
group
three.
K
S
And
some
of
those
links
are
explicit
in
the
data
they're
part
of
the
structured
data
that
we
get
from,
say
the
data
tracker
or
from
other
sources,
and
some
some
links
have
to
be
built
or
inferred
sort
of
statistically
but
based
on
the
kind
of
you
know
they
say,
carve
the
world
at
its
joint.
So
based
on
the
kind
of
structure
of
that
of
that
data,
we
were
able
to
subdivide
into
into
smaller
groups.
S
And
then
there
was
another
group
of
us
that
was
interested
in
looking
at,
say,
data
tracker
records
to
understand
people's
affiliation
with
those
organizations
over
time,
how
they
change
and
using
that
data
to
just
a
map
between
organizations
and
email
domains,
which
often
can
be
done
through
individual
data
sets,
and
then
christoph.
Would
wind
up
forming
zone
group
and
developed
a
really
interesting
visualization
that
sort
of
built
on
all
of
this
work
together?
S
S
As
I
mentioned,
this
kind
of
database
schema
almost
or
taxonomy
of
of
what,
if
we
had
the
ideal
database
of
this
stuff,
what
would
it
look
like,
and-
and
this
is
worked
by
nick-
exploring
the
wiki
data
data
set,
which
has
some
of
these
categorizations
itself,
so
in
the
future,
we
might
try
to
standardize
on
on
sector
labels
from
based
on
some
external
standard
of
what
those
sector
labels
should
be.
That
would
allow
us
to
draw
on
these
external
data
sets
a
bit
better
from
that.
S
I
apologize
for
the
you
know
the
pii
and
the
slide.
You
know
don't
tell
anyone,
but
this
is
all
just
extracted
from
attendance
records
from
the
data
tracker
and
I
guess
what
I'm
trying
to
show
here
is
some
process
this.
This
isn't
exactly
what's
in
the
data
tracker,
but
with
some
transformation
we
can
get
these
records
of
individuals
with
their
affiliation,
their
their
email
address,
with
their
domain
email
domain
extracted
and
the
dates,
and
and
from
that
we
can.
S
You
know,
because
we
don't
actually,
for
the
sake
of
this
analysis,
care
that
much
about
the
individuals
we
care
about.
What
are
the
statistical
relationships
between
the
affiliated
organizations
and
their
domain.
So
this
is
showing
that
you
know
out
of
128
people
or
registrations
for
this
span
of
meetings,
which
I
think
is
like
nine
or
ten
meetings
that
say
that
from
cisco
systems,
124
of
those
registrations
use
cisco.com,
so
that
that's
a
pretty
strong
signal.
S
You
know
weekends
for
for
other
companies,
but
it's
a
pretty
strong
signal
that
that's
the
domain
associated
with
the
company,
which
is
helpful
because
we
can
then
match
that,
with
the
kind
of
hand
generated
data
set
that
group
a
was
doing.
So
I
want
to
point
to
two
things.
So
this
is
this
is
a
sort
of
join
of
the
data
sets
on
a
normalized
institution
name.
So
this
is
using
a
script.
That's
in
big
bang.
S
It
has
a
kind
of
sort
of
a
lexical
entity
resolution
resolver,
that's
tailored
specifically
to
these
kind
of
organization
names
getting
a
match
on
those
normalized
entity-resolved
organization
names.
A
lot
of
joining
the
data
sets,
and
that's
pretty
good.
I
mean
these.
These
domains
are
are
pretty
good
matches,
except,
as
you
can
see,
like
a
personal
email
domain
that
slips
in
so
we
still
have
some
issues
with
the
the
noise,
given
the
way
that
this
particular
dataset
is
done.
S
But
it's
it
is
progress,
and
then
this
is
work
by
christoph
becker.
So
once
the
domain
names
are
associated
with
things
like
company
categories,
it's
possible
to
both
do
these
very
beautiful
plot,
showing
that
say
within
a
particular
working
group
mailing
list.
You
know
it's
mostly
a
conversation
between
ericsson
and
huawei
or
erickson
and
samsung
or
hawaii
and
samsung,
but
you
can
also
say
so,
zoom
out
and
say:
okay!
S
Well,
this
is
between
you
know,
telecom
providers
and
research,
institutions
etc
and
there's
another
plot
which
might
be
interesting
to
wes,
which
is
a
similar
kind
of
plot,
but
with
sort
of
country
of
origin.
So
we're
very
interested
in
in
figuring
out
how
to
get
the
most
value
out
of
these
data
sets,
as
we
sort
of
combine.
E
S
We
think
that
this
proves
the
value
of
combining
these
data
from
from
multiple
seos,
as
well
as
external
sources,
as
well
as
gold
standard
hand,
curated
data
and
all
the
data
and
scripts
for
so
the
automated
parts
of
this
being
added
to
this
big
bang
open
source
software
project,
and
we
really
look
forward
to
working
more
on
this
in
the
future
sort
of
systematically
populating
this.
This
notional
database
to
support
research
as
well
as
administrative
insight
into
organizational
involvement
in
these
stfs.
K
E
I
mean
it's
it's
an
old
hand,
other
than
to
say
that
it's
incredibly
cool
work,
one
of
the
things
that
I
wanted
to
ask
you
specifically
and
again.
This
is
a
question
that
I
could
have
asked
with
any
of
the
sessions,
but
I
think
it's
appropriate
here
because,
like
both
niels
and
nick
who've,
been
working
on,
who've
been
hacking
with
you
over
the
last
couple
of
days
have
experience
doing
qualitative
research,
and
one
of
the
things
I
was
wondering
is
like.
How
do
you
see
that
to
be
complementary?
S
E
K
Well,
so
so
I
think
that
this
was
very
much
if,
if
I
may,
that
a
lot
of
this
work-
and
maybe
elizaveta
can
also
comment
on
this-
that
there
has
been
quite
a
lot
of
hand
coding,
and
so
I
think
that
this
work
has
been
based
on
a
lot
of
mixed
methods,
approach
for
looking
into
this
data,
and
that
then
informed
also
how
how
further
code
should
be
developed.
But
I
think
then
verifying
it
with
ethnographic
methods
and
see
where
it
makes
sense
and
where
the
data
doesn't
work
really
helps.
K
E
Yeah,
if
I
could
just
follow
up
with
them
really
quickly-
and
this
is
also
with
the
knowledge
from
having
set
in
your
living
room-
as
you
guys
were
doing
this-
I
could
hear
you
here-
you
do
the
work
in
the
background,
one
of
the
things
that
I
was
wondering
about,
especially
for
the
hand
coding
one
is
like.
Did
you
keep
a
a
log
of
how
you
made
which
decisions?
Because,
obviously,
especially
when
you
hand
code
you,
you
are
the
classifier
right.
E
You
are
the
person
that
decides
that
a
particular
bit
of
data
falls
within
a
particular
category
that
the
system
subsequently
runs
on,
which
is
a
hugely
impactful
decision,
and
people
are
probably
going
to
push
back
on
it
right,
which
means
that
they
can
say
like
well
yeah.
Your
outcomes
might
seem
interesting,
but
you
made
decisions
that
you
need
to
justify.
So
I
was
wondering
how
how
you've
gone
about
that
sort
of
thorny
issue.
K
Yeah
we
took
notes,
but
I
think
christoph
might
say
we
did
not
take
enough
notes,
but
elisaveta-
and
I
did
did
our
best
to
to
agree
on
the
note
to
to
to
do
that,
but
that
that
but
then
luckily
seb
and
nick
came
back
and
said
like
we
should
maybe
not
rely
on
this
way
and
then
provide
and
then
scrape
wiki
data
to
actually
back
this
up
and,
for
instance,
affiliation
and
there.
There
again.
These
things
inform
each
other.
Whether
we
can
then
see
where
wiki
data
fails
or
where
our
thing
fails.
K
And
then
the
conflicts
arise.
And
those
are
then
actually
interesting.
Points
for
research
because
already
like
combining
the
3gpp
datasets
with
ietf
data,
showed
us
that
a
lot
of
companies
had
registrations
in
different
countries
would
allow
them
to
have
more
voting
rights
in
the
3gpp
and
that's
why
we
included
subsidiary
data
so
see
where
the
parent
company
is.
But
how
does
this
then
change
over
time
with
mergers
and
acquisitions,
and
that
just
shows
the
messiness
of
the
reality
and
the
data
and
helps
us
make
choices.
K
But
then
we
also
came
to
very
much
the
realization
to
what
kind
of
research
question
you
have
will
also
then
choice
how
you
structure
the
data.
So
there
is
not
one
data
source
that
that
is
authoritative
for
everything,
so
you're,
very
right,
so
that
that
that
really
depends.
But
we
try
to
make
notes.
So
that
is
clear,
and
that
would
also
allow
us
to
restructure
the
data
set
to
answer
different
kinds
of
questions.
S
I'd
really
love
it.
If
moving
forward,
we've
settled
on
a
kind
of
sort
of
labor
data
automation
pipeline
that
we
could
all
agree
on.
As
being
you
know
as
valid
as
possible
and
tracking
the
provenance
of
the
data
as
we
build
it
and
then
and
all
the
mixed
methods
that
are
using,
like
that's
a
incredibly
valuable
thing
to
do.
My
current
answer
is
just
that.
I
think
it's
going
to
take
work
to
sort
of
like
you
know
architect
that,
but
I
think
we
should
do
that
moving
forward.
R
Yeah,
I
can
finally
learn
how
to
do
it.
Just
a
couple
of
comments,
so
I
was
thinking
that
they
related
to
a
to
the
command
from
korean,
and
maybe
what
could
be
useful
is
for
every
affiliation
to
have
a
say
like
a
vector
where
each
number
represents
one
of
the
potential
sources.
So
you
know
for
every
affiliation
what
has
been
the
soror.
So
in
many
cases
that
would
mean
that
there
is
a
very
good
source.
So
you
don't
need
to
worry
too
much
in
other
cases,
maybe
more
heuristical.
S
R
Yeah
and
the
the
thing
I
was
wondering
is-
and
I
don't
know
how
I
idf
people
feels
about
it-
whether
it
would
make
sense
going
forward
to
have
a
more
standardized
way
of
collecting
affiliation
data,
because
that
probably
would
solve
quite
a
lot
of
time.
Energy,
and
I
guess
that
this
is
also
relevant
for
those
that
they
have
to
file
in
the
patents
that
might
be
related
to
the
standards
they
are
pushing
and
stuff
like
that.
L
L
R
Yeah,
I'm
I
mean
yeah.
I
totally
understand
I'm
wondering
like
maybe
something
something
a
little
bit
in
the
middle,
where
you
know
I
registered
for
the
itf
and
now
I
let's
say
I'm
from
microsoft
as
I'm
about
to
type
microsoft,
because
we
already
have
one
record
that
says:
microsoft,
microsoft,
pops
up
as
an
self-complaint
completion.
So
if
I
write
it
with
a
typo
that
would
be
avoided.
K
P
T
Okay,
can
you
see
this?
Maybe
if
I
go
to
excellently
slideshow,
is
that
even
better,
probably
even
better?
Yes,
all
right
and
it
moves
along
by
itself,
I
think
I'll
go
to
full
screen
instead
of
slideshow,
it's
probably
probably
nicer.
Okay,
yeah!
So
here
are
the
discussions
that
we
that
we
well
the
questions
that
we
discussed
to
kickstart.
The
whole
thing:
these
are
text
clips
from
the
papers
that
were
submitted
and
there
were
some
pretty
obvious
cases
of
overlap.
T
Well,
the
roles
of
seniors
in
the
itf
or
senior
participants,
people
that
have
been
long
active
and
then
and
whether
they
have
had
more
influence
or
less
influence.
Does
it
improve
the
chances
of
success
or
not.
So
that
was
something
that
well.
There
was
a
common
interest
on
that
and
also
as
we
go
further
along
towards
rfc
publication.
Well,
there
was
also
this
this
interest
in
looking
at
deeper
at
the
mailing
lists
and
understanding
working
group
last
calls
specifically.
T
T
Influencers
clustering
people
into
groups
colin
showed
us
a
bit
of
ongoing
work
that
he
has
on
his
side
and
there
seems
to
be
a
bit
of
overlap
between
already
ongoing
work
on
his
side
and
then
what
they
have
started
to
do,
and
then
it
was
essentially
carson
and
me
working
on
the
analog.
A
bit
of
an
analysis
of
working
group
last
call
with
technical
support
from
colleen,
because
we
use
this
ietf
data
library.
T
T
I'm
not
sure
I
can
represent
this
correctly,
because
that
wasn't
me
trying
it
later,
but
then
there
was
something
with
the
with
a
version
number
so
making
this
run
with
mongodb.
I
think
they
managed
to
do
it
and
that
was
only
possible
using
python
3.9
and
the
python
3.7
version
was
related
to
the
itf
data
version
that
I
don't
know
was
itf
data
wasn't
available
at
the
latest
version,
or
something
like
that
I'll
leave
it
to
my
peers
to
explain
that
in
greater
detail.
T
What
did
work
is
the
itf
data
library.
We
were
having
a
lot
of
fun
with
that
we
were
we
did
have.
We
should
have
put
this
in
the
past
at
some
point
and
regarding
next
steps.
Well,
I
mentioned
already
that
I'm
going
to
have
a
phd
student.
Finally,
this
is
not
settled
that
really
a
person
is
going
to
be
here,
working
on
some
nlp
stuff
and
working
with
nlp
on
this
on
mailing
list.
T
Analysis
of
the
itf
needs
to
find
your
models,
because
this
the
language
of
these
models,
they're,
usually
pre-trained
on
newspaper
texts,
news
texts
in
general,
so
they
need
to
be
adjusted
to
work
with
the
itf
specific
language
and
it
seems
pretty
useless
for
separate
groups
to
do
this
kind
of
work
independently
and
not
collaborate.
So
I
think
we'll
definitely
stay
in
touch
working
on
that,
but
that
is
future
work.
T
U
Yeah,
that's
great
so,
like
michael
was
saying,
we
did
try
to
have
a
collaborative
notebook
as
suffolk
had
suggested,
so
that
we
could
collaborate
and
the
we
were
able
to
run
mongodb,
which
is
used
by
the
itf
data
library,
with
persistent
storage
in
google
drive.
But
we
were,
I
wanted
to
work
with
the
itf.
The
latest
library
version
the
default
one
which
uses
with
the
python
3.7
is
the
0.4
version.
U
Earlier
groups
have
tried
to
see
affiliation
and
gender
and
other
aspects,
specific
aspects,
informed
aspects
from
the
domain
and
then
try
to
understand
users
and
their
and
their
homophily
like
the
similarity
aspects,
but
I
I
did
it
from
a
temporal
activity
perspective
and
if
we
move
to
the
next
slide,
we
can
see
that
so
I
I
did
a
very
rough
highlight.
This
is
a
hierarchical
algorithm.
U
So
if
you
run
the
algorithm
again
on
these
clusters,
they
will
come
into
more
more
fine
grained
clusters
wherein,
as
you
see
these
peaks,
this
is
a
time
series.
So
this
is
the
60
65
months
that
the
working
groups
this
working
group
has
been
active
and
on
the
y-axis,
is
the
number
of
emails
that
that
person
has
sent
in
that
month.
So,
as
you
can
see
that
some
people
I
mean
those
within
a
cluster,
they
have
overlapping
peaks.
U
That
is,
they
were
both
or
they
were
all
together
active
during
that
particular
month
with
that
volume
and
the
the
overlap
in
the
peaks,
whereas
people
in
different
clusters
who
are
in
different
clusters
do
not
have
that
overlapping
peak
with
the
other
cluster,
so
this
corresponds
also
to
the
topics
that
they're
interested
in.
So
it
is
that
people
who
are
in
one
cluster
one
group,
they
are
more
interested
in
topics
that
are
being
discussed
within
that
like
by
the
people
in
that
group
versus
people.
U
In
the
other
group,
I
was
able
to
find
some
interesting
observation
that
such
like
eri
and
media
are
both
clustered
together
in
one
group,
I
did
not
use
any
of
their
affiliation
information
or
any
other
demographic
information,
just
their
temporal
activity,
and
they
come
up
in
the
same
cluster.
Things
like,
but
michael
richardson
or
paul
vixxi
are
in
a
different
cluster.
So
even
within
the
v6
ops
groups,
they
are
in
different
sub
groups.
U
Let's
say
and
fred
is
in
a
completely
different
cluster
altogether,
so
the
the
top
level
I
had
10
clusters.
Obviously
this
gets
mine,
finer
and
finer
to
find
whether
there
are
any
discrepancy,
even
within
the,
whether
there's
any
diversity
of
thought,
even
within
the
cluster
etc.
Surprisingly,
I
was
able
to
also
identify
this
is
something
new
that
I
did
not
know
before,
that
it
helped
in
entity
disambiguation,
so
people
with
the
same
name
but
different
email
address
were
found
to
be
in
the
same
cluster.
U
U
T
E
T
T
We
got
per
rfc
quite
nicely
with
this
library.
We
got
the
working
group
last
call
number,
so
how
many
last
calls
there
were,
and
also
the
date
of
the
working
group
last
call
that
happened
in
the
data
tracker
I
mean
from
the
data
tracker.
We
did
analyze
the
mailing
list.
Discussions
related
to
the
first
working
group
last
call,
so
we
took
the
date
of
the
first
working
group
last
call
and
then
said:
well,
look
at.
T
Let's
look
at
the
emails
that
happened
before
this
and
after
that
and
for
the
set
of
preceding
drafts
of
an
rfc.
We
did
that
so
well.
We
took
the
emails
in
this
in
the
range
of
of
the
time
from
the
very
first
preceding
draft
that
eventually
became
an
rfc
until
the
working
group
last
call
date
and
then
from
the
working
group
was
called
until
the
rfc
publication,
and
then
we
filtered
these
emails
by
the
subject
line,
which
of
course
you
know
is
not
a
hundred
percent.
T
T
So
in
that
case,
a
stupid
idea
is
the
part
that
I
wanted
to
keep.
We
may
have
added
draft
in
the
beginning
as
well,
but
we
were
we
filtered
it
like
that.
Now
this,
the
part
that
you
see
in
blue
in
this
example
down
here
and
started
this
left
it
running
today
on
the
emails
that
were
downloaded.
I
don't
think
that
even
all
the
emails
came
complete.
T
T
Q
T
We
have
the
number
of
participants
in
these
email
discussions.
That's
by
using
the
person
id
field
from
the
data
tracker.
We
excluded
the
authors
of
the
rfcs,
because
these
are
not
so
interesting
to
understand
if
the
discussion
was
lively
or
not
to
me.
If
people
just
send
emails,
saying
please
read
my
draft.
Please
read
my
draft,
that's
not
that's
not
so
so
I
mean
these
are
excluded
here
and
well.
You
see
that
the
result
is
kind
of
expected
right.
T
T
Q
Well,
I
mean
it
would
have
been
really
nice
to
go
through
our
question
of
how
many
drafts
succeed
and
if
it
relates
to
seniority,
but
even
just
trying
the
list
to
create
get
the
list
of
all
the
drafts
independent
of
whether
they
come
from
an
rfc
or
not,
is
just
a
call
that
is
hanging
forever.
So
that
definitely
needs
a
couple
of
days
to
even
start
running
the
script.
K
Indeed,
very
interesting,
but
also
very
great
outcomes
already-
and
I
think
part
of
the
aim
of
the
workshop
is
also
to
get
interesting
questions
and
paths
for
future
research.
So
in
that
you
definitely
succeeded
and
enlightened
all
enlighted
us
on
all
on
the
on
this
group
five,
so
that
was
excellent.
K
H
D
D
I
hope
you
can
see
it
so.
First
of
all,
I
put
some
notes
in
here
based
on
my
own
note
and
my
own
impression
and
I'm
sure
I'm
missing
a
ton
of
things
because
there
was
so
much
going
on,
but
I
think,
like
a
big
volume
of
this
workshop
was
just
connect
to
everybody,
to
each
other
and
to
sparkle
some
ideas
for
some
new
work
and
that's
also
reflected
in
my
slides.
I
think
so.
D
We
started
the
session
with
talking
about
data
and
tools,
and
this
is
just
like
a
very
brief
summary
there's,
like
all
this
data
people
are
already
using
from
the
main,
is
from
the
rfc
index
from
the
data
tracker,
but
there
were
also
a
couple
of
things
additional
things
that
we
talked
about,
for
example,
they're,
the
visitor
stats
from
the
web
page,
there's
github
that
we
could
probably
utilize
more
and
there's
data
about
interrupt
testing.
That
could
be
quite
interesting.
We
have
the
survey
data,
download
statistics
for
rsc
and
so
on.
D
D
We
just
discussed
in
this
question
already
about
how
to
derive
name
and
gender
and
how
to
integrate
this
into
the
existing
tools
and
make
these
things
useful,
and
this
also
drove
a
little
bit
of
discussion,
not
so
much
at
the
main
session
on
monday,
but
in
the
site
meetings.
The
last
two
days
about
privacy
and
legitimate
use
of
the
data,
so
this
is
also
something
we
should
look
into
further.
D
However,
I
want
to
point
out
again
this
slide
from
robert,
so
with
the
data
tracker
robert
was
here
and
he's
like
the
best
contact.
If
you
have
any
questions,
problems,
proposals
for
enhancements
or
whatever,
so
I
think
I
can
freely
say
that
him
feel
free
to
contact
robert,
because
he
said
this
multiple
times
already.
E
D
Always
happy
for
contribution.
There
is
a
statistic
side
in
data
tracker
that
only
shows
a
very
few
statistics,
so
work
on
that
is
super.
Welcome
and
there's
also
something
that
is
not
in
the
slides
here,
which
is
called
the
code
sprint,
which
is
like
a
hackathon
that
is
focusing
on
enhancing
the
data
tracker
and
which
happens
at
every
meeting
as
well.
So
keep
that
in
mind.
E
D
Do
companies
compete
on
leadership
and
these
kind
of
questions
and
in
the
discussion
there
were
even
more
more
things
that
we
were
interested
to
look
on,
and
then
people
took
this
up
and
started
to
look
at
this
at
the
hackathon.
That
was
about
affiliation
trends,
the
best
in
looked
into
that,
and
we
talked
about
having
other
sources
of
fundings,
which
would
also
be
interesting
to
get
data
about
not
sure
where
the
data
comes
from.
D
We
talk
about
rather
than
looking
at
companies
looking
at
stakeholder
groups
and
how
to
characterize
companies
into
stakeholder
groups.
Some
work
has
been
ongoing
there
and
again
the
point
about
comparing
this
with
other
seos
or
enriching
our
data
from
with
data
from
other
sdos
community
diversity.
This
session
it
has
a
little
bit
more
on
the
slides
and,
as
I
said,
it's
like
some
of
my
notes,
something
from
the
hackathon
and
so
on,
and
there
might
be
things
missing
because
we
talked
about
quite
a
lot
of
things.
D
So
we
talked
about
things
like
you
know:
who
who
are
we
and
what
is
the
diversity
and
how
to
improve
diversity
in
the
ietf
and
also
you
know?
What
would
would
the
itf
be
better
if,
if
diversity
would
be
stronger,
and
so
we
talked
about?
How
can
you
actually
measure
diversity
with
the
existing
tools?
And
we
also-
I
like
this
comment-
I
think
from
mallory
saying
like
what.
D
Why
are
we
measuring
it
right
so
measured
because
one
on
one
side,
we
want
to
have
better
protocols,
because
diversity
should
make
better
protocols,
but
also
to
make
our
work
more
relevant
because
having
more
people,
aware
of
it
and
being
involved,
leads
to
better
outcome.
That
is
more
relevant
for
more
people.
So
this
is
important,
and
so
we
also
discussed
about
understanding
the
organizational
structure
behind
this
and
started.
D
So
we
had
a
couple
this
was
just
presented.
We
had
a
couple
of
of
hacking
groups
that
started
looking
into
different
things
like
gender
country
distribution
and
also
like
talking
about
these
privacy
points
again
and
then
the
thing
that,
where
we
just
talked
about
in
the
last
hacking
group,
basically
was
more
related
to
process
and
the
rfcs
itself
or
decision
making
within
the
ietf.
And
then
the
question
is
really:
how
do
we
come
to
a
decision?
D
How
do
how
can
we
detect
when
a
decision
was
taken,
and
how
can
this
be
used
to
improve
the
process
and
also
you
know
what
what
is
actually
makes
rfc
successful
or
what
makes
the
product
we
are
producing
in
the
ietf,
successful
and
michael
was
looking
into
last
calls.
He
just
presented
that,
but
then
what
there
was
also
a
question
about
tenure
like
who's
writing,
successful
rfcs.
D
Looking
at
code,
fragments
code,
similarity
downloads
from
libraries,
references
to
rsc,
everything
that
is
outreaching
so
yeah,
I
think
there's
more
work
here,
right,
sorry,
one
more
session
that
we
had
today
right
the
environment
of
sustainability.
So
here
there
were
like
two
aspects
and
some
very
early
results.
One
is
about,
like
you
know
how
what's
the
impact
of
our
technology?
How
green
is
the
ietf?
Do
we
consider
sustainability
enough
in
developing
our
protocols?
D
And
there
was
some
initial
analytic
keyword,
analysis
done
and
there's
more
work
to
do,
but
it's
important
to
create
awareness
of
this
topic
in
the
ietf.
So
this
is
a
good
starting
point
and
then
there
was
also
this
whole
discussion
about
sustainability
of
itf
meetings,
which
is
a
little
bit
ongoing
in
the
itf
already
in
the
small
working
group,
but
probably
more
work
to
be
done
here
as
well.
D
So
you
know
this
is
like
the
very
quick
summary
and
I'm
pretty
sure
current
wants
to
add
a
few
more
words
but
like
from
my
side,
thank
you
for
all
the
input
we
provided
in
the
paper
in
the
discussion
and
all
the
hard
work,
and
it
seems
like
there's
more
work
to
be
done.
So
hopefully
we
can
figure
out
a
future
venue,
probably
hackathons
at
the
next
itf
meetings,
but
we
could
also
think
about
other
ways
to
organize
organize
ourselves
or
more
workshops
or
whatever.
So
maybe
this
is
something
a
little
bit.
E
Yeah,
I
just
want
to
hop
on
and
amplify
maria's
point
about,
thanking
all
of
you
done
the
done
the
hard
work,
and
also
you
know
stress
that.
I
I
think
that
if
this
week
has
shown
anything
that
we
have
raised,
perhaps
more
new
questions
than
we
have
answers,
and
I
do
believe
that
there
is
a
real
community
forming
around
itf
data
and
and
what
it
is
useful
for,
but
also
what
it
tells
us,
what
we
don't
yet
know
and
also
including
building
new
tools.
E
So
I
hope
that
that
we
can
do
that
and
we'll
make
sure,
on
our
end
as
the
organizers
to
follow
up
in
terms
of
what
you
would
need
and
what
that
would
take.
And
if
you
have
any
kind
of
interest
in
in
perhaps
hosting
a
next
meeting
or
somehow
having
your
organization
be
involved
in
that,
please
do
reach
out
that
we
would
very
much
welcome
that.
K
K
You
never
know
how
it
goes
with
a
hackathon
right
where
people
show
up
or
just
check,
email
and
people
really
did
not
check
a
lot
of
email,
but
we're
very
much
on
the
slack
and
doing
a
lot
of
code
and
presenting
really
cool
outcomes
and
really
really
appreciated.
Also
robert
and
and
yary,
and
I
ib
also
to
integrate
this
and
and
and
to
really
see
a
connection
between
academic
community,
ietf
leadership
and
and
industry.
So
yeah
we're
already
at
a
really
good
point.
K
So
if
you
are
interested
in
helping
us
co-organize
that
and
making
that
happen,
either
co-located
with
an
ietf
which
then
can
be
stressful,
but
then
also
other
people
might
be
there.
Please
do
reach
out
to
me
and
then
we'll
set
up
an
email
thread
on
that
and
then
hopefully
also
also
organized
again
with
the
iab,
but
that'll,
of
course,
also
take
discussion
with
the
iab,
but
I
would
definitely
think
everyone's
enormous
contributions
to
this.
D
And
so
we
will,
we
have
the
github
with,
hopefully
all
the
slides
there,
so
we
can
look
up
what
we,
what
we
did
here
and
find
all
the
information
there
and
we
are.
We
will
also
write
and
report
for
this
workshop,
as
we
do
with
all
iab
workshops
and
at
least
until
the
report
is
finished,
we
will
keep
the
workshop
mailing
list
open.
So
if
you
have
any
outcome,
any
findings,
anything
you
want
to
share
and
please
do
on
the
mailing
list.
K
And
I
see
yari's
hand
is
up,
but
before
that,
I'd
like
to
especially
thank
kate
and
corrine,
also
for
taking
my
new
details
of
the
notes
of
the
of
the
meeting
and
cindy
for
making
it
all
possible,
and
that
will
really
help
us
in
the
report
writing
and
further
follow-up.
So
we
also
know
how
how
troublesome
logistics
can
be.
So
thanks.
So
much
for
that
and
yari.
Please
come
in.
P
Yeah,
I
also
wanted
to
thank
the
organizers
and
everybody
who
pitched
in
amazing
amount
of
work,
so
much
achieved
even
even
this
week
by
other
people,
but
I
wanted
to
say
a
few
things
additional
observations.
So
so,
obviously
one
thing
here
is
not
just
the
achievements
but
sort
of
getting
to
know
each
other
and
and
then
maybe
this
is
start
for
collaboration
in
that
sense
and
also
when
it
comes
to
the
tooling,
at
least.
P
For
me,
a
sort
of
takeaway
is
that
we
don't
actually
live
in
this
monolithic
tool,
world
anymore,
that
there's
many
components.
Idf
data
is
one
example
of
that,
but
not
the
only
ones
also
like
this
gender
analysis
and
other
other
other
useful
things.
So
maybe
that's
also
another
start
that
we
can.
We
can
collaborate
more
in
that
sense
that
we
have
useful
pieces
that
can
be
used
in
in
several
different
contexts
and
and
indeed
many
many
new
questions
opened.
P
Also,
I
was
kind
of
inspired
by
this
session
on
sustainability
as
well,
but,
of
course
it's
not
just
sustainability
issues
for
the
ietf
organization.
It's
also
about
our
technology.
I
was
kind
of
like
thinking
about.
Could
we
compare
some
of
the
potential
things
we
could
do
or
maybe
not
compare
but
sort
of
quantify
what
what
are
the
impacts
of
various
kinds
of
practices
that
we
run
in
the
internet
today
and
how
much
savings
could
we
get
through
those?
So
I
think
that's
also
interesting
direction,
maybe
for
somebody
to
consider.
G
Yeah-
I
am
one
comment
I
I'd
like
to
make.
Is
it
was
very
nice
to
to
see
some
of
the
tools
that
have
been
developed
and
if
I
have
known
them
earlier,
I've
probably
I
mean
you
probably
have
helped
me
to
do
some
passing
of
html
pages
and
so
on.
So
I
think
it's
important
that
we
know
those
tools
exist,
so
people
can
build
on
the
specific
question
they
asked
and
they
don't
redo
everything
from
scratch.
So.
C
C
Wanted
to
thank
the
organizers
for
this
workshop,
as
danny
just
said,
I
mean
also.
I
stumbled
upon
upon
other
people's
work
that
really
helped
also
big
bang
a
lot
to
progress
and
yeah
for
all
the
nice
feedback.
I
I
got
for
the
the
work
that
safik
and
I
are
doing
now.
I
think
we
will
put
some
more
effort
in
there
and
see
where
we
get
and
maybe
can
present
some
results
next
year
and
first,
maybe
hopefully,
let's
see
thank
you.
D
A
A
So
I
think
that
this
is
very
important,
but
because
of
the
idea
of
the
volunteer
work
so
that
we
we
in
the
history,
we
cannot
get
the
enough
this
the
moment.
But
I
think
this
is
the
workshop.
This
is
a
very,
very
good
chance
to
that's
the
user.
To
exchange
this,
the
information
that's
between
the
ietf
and
the
academia,
and
also
this
is
very
appreciated.
This
academia
work
to
do
the
research
on
the
iet
of
data,
so
I
think,
and
also
that's
your
work
is
very
important
and
also
there's
your
this.
A
V
O
I
just
want
to
thank
everybody
for
for
actually
being
willing
to
try
this
virtually
you
know
this
was
an
experiment
and
it
actually
showed
that
it
worked.
We
actually
did
things
you
know
somewhat
together,
I'm
sure
it
wasn't
quite
as
nice
as
in
person,
but
you
know
if
we
want
to
both
increase
productivity
and
simultaneously
decrease
our
travel.
For
you
know,
environmental
reasons,
her
personal
health
reasons.
O
We
have
to
keep
trying
things
like
this
in
order
to
figure
out
what
works
and
what
doesn't,
and
you
know
it's
just
not
going
to
magically
fix
itself
without
us
trying
things
I
found
you
know
gather
was
it
was
interesting.
You
know
it
actually
worked
as
sort
of
a
hacking
space,
so
it
was
kind
of
neat
to
do
and
though
I
did
get
distracted
by
my
day,
job
more
than
I
might
have.
If
I
was
in
person,
I
still
got
stuff
done
right.
O
F
V
V
Oh
okay,
we
should
get
the
the
itf
data
people
together,
because
a
week
of
hacking
with
occasional
check-ins
can
potentially
be,
I,
I
think,
really
impactful
for
for
the
long-term
work,
including
you
know
sure
many
of
us
know
about
other
work.
That's
going
on
now,
or
or
maybe
we
had
some
of
these
connections
already,
but
all
of
us
working
on
new
code
and
new
spreadsheets
and
data
sets
together
for
a
week
could
could
make
a
big
difference.
So
I'm
eager
to
see
how
we
continue
it.
D
Yeah,
which,
which
reminds
me
of
one
more
point,
so
I
think
we
have
made
some
really
good
connections
here
and
and
got
a
really
nice
group
of
people.
But
I
think
one
thing
we
should
do
is
try
to
do
some
more
outreach.
I'm
sure
there
are
more
people
who
working
or
start
working
in
this
space,
so
we
should
definitely
try
how
to
connect
there,
maybe
for
the
for
the
hackathon
or
for
any
kind
of
other
event.
Where
might
be
organizing
in
future.
D
L
You
know,
I
do
think
that
the
efforts
that
people
have
started
here,
don't
don't
stop
pushing
on
them
just
because
we
got
to
the
end
of
this
particular
event.
Keep
the
inertia
up
and
the
outreach
that
this
instance
of
this
workshop
achieved
is
is
very
impressive,
and
I
I'm
glad
that
we
attracted
the
the
group
of
people
that
we
have
and
that
we've
gotten
to
know
each
other
for
those
of
us
that
met
this
for
the
first
time.
R
Yeah
well
just
to
join
everybody
else
in
the
green
that
this
was
great.
Thank
you,
organizers,
thank
you
for
the
for
everybody
who
participated
and
looking
forward
to
the
next
workshop,
and
maybe
we
could
take
the
next
workshop
as
an
opportunity
to
try
to
see
how
to
organize
this
as
a
more
regular
thing
that
goes
forward
in
time.
D
E
K
Yeah
and
as
a
final
note
thanks,
everyone
for
also
making
this
really
a
constructive
thing
where
everyone
got
to
speak
and
everyone
got
to
contribute.
So
I
really
felt
that
there
was
a
really
good
contribution
from
everyone
and
there
was
space
for
that.
So
thanks
so
much
for
making
that
happen
and
have
a
great
weekend
and
see
you
on
the
next
one
see
you
on
the
list.
Happy
hacking,
bye.