►
Description
The Internet Architecture Board (IAB) held an open meeting on 29 July 2021 at 23:30 UTC. This session will provide an opportunity for more direct interaction and technical discussion between the community and the IAB, and reporting back on work done as well as gathering input about on-going work.
A
D
B
B
B
So
we
have
peterson
andre
has
volunteered
to
take
notes.
Initially
he
was
going
to
take
it
for
half
of
the
session,
but
he
just
very
kindly
stepped
up
for
the
entire
session,
so
we're
good
there.
Traditionally
we
have
a
java
scribe
for
those
folks
who
are
not
on
audio
but
want
to
ask
questions.
Do
we
have
someone
who
can
just
monitor
the
chat
in
case?
There's
someone
who,
for
whatever
technical
reason,
can't
speak,
needs
to
have
a
question
taken
to
the
microphone.
B
B
B
B
I
Fantastic,
I'm
actually,
despite
what
it
says
on
the
on
the
agenda,
I'm
going
to
merge
the
discussion
of
the
things
we're
trying
to
accomplish
with
the
discussion
of
technology,
because
I
think
it's
easier
to
understand
the
motivating
cases
so
I'll
just
fold
them
right
in
right.
Having
a
separate
use
cases,
presentation
so
I'll
say
I
sent
a
link.
I've
written
up
quite
a
bunch
of
interactive
material
on
this.
I
So
if
people
want
to
read
that
I
sell
a
link
in
the
java
chat
and
obviously
something
is
unclear.
Please
stop
me
and-
and
you
know,
ask
so:
okay,
let's
move
on
so
there
you
go.
I
like
to
start
so.
Here's
just
like
the
traditional
like
this
is
what
I'm
going
to
talk
about
kind
of
in
order.
First,
I
want
to
talk
about
like
the
kinds
of
things
like
one
actually
wants
to
measure
in
these
typical
settings,
then
I
want
to
talk
about
anonymous
measurement.
I
That's
about
all
the
science
and
plan
to
do
so.
There's
a
lot
of
situations
where,
like
one
would
like
to
learn
about
people
right,
you
know
we
have
like
the
the
census
or
you
know,
there's
a
lot
of
public
research.
You
know
and
you
learn
things
like
demographics
and
you
know
people's
income.
You
know,
maybe
they
have
medical
issues.
You
know
companies
want
to
do
product
development
so
to
see
what
features
that
you
know,
people
using
don't
use
how
much
they
use
them
like.
I
Are
the
products
not
working
in
some
way
and
then
you
also
want
to
take
like
behavioral
measurements
like
like,
so
you
know,
say
you
want
to
discover
like
new
websites
that
no
one
knows
about
or
what
information
people
care
about.
So
you
can
tune
your
product
to
be
like
more
like
what
people
actually
want.
So
I
mean
there
was
a
good
example
of
this
on
the
other
day
when
brave
doesn't
posting
about
like
their
new
search
engine
and
how
they
like
want
to
discover
websites
for
the
search
engine.
I
So
so
all
these
problems
involve
like
collecting
data.
The
information,
of
course,
is
like
very
useful,
but
it
also
is,
you
know,
can
be
very,
very
sensitive.
You
know
people
often
don't
want
to
and
shouldn't
have
to
disclose.
You
know
their
medical
issues
in
order
in
order.
We
want
to
learn
how
many
people
have
some
medical
condition.
You
know,
that's
that's
good,
because
we
want
to
know
you
know
where
we're
finding
should
be
targeted,
for
instance,
but
we
don't
want
to
know
what
individuals
have
like.
I
I
This
is
often
called
like
high
dimensional,
sparse
data
sets
and
figure
things
out
that
people
would
be
surprised,
and
so
I
had
this.
This
is
you
know
from
an
article
a
few
years
ago
about
you
know,
target
inferring.
You
know
a
girl
was
pregnant
by,
like
looking
her
other
other
sort
of
pershing
behavior.
I
So
there's
been
a
lot
of
you
know
research
on
this
and
what
would
hope
to
do
better
and
so
like
the
historical
way
that
you
know
one
does
these
things
is
that
you
just
gather
all
the
data
and
then
you
promise
not
to
disclose
it,
and
you
know
this
is
like
not
working
out
super.
Well,
you
know
data
breaches.
There's
this
the
famous
case
of
the
census
information.
I
I
The
data
you
want
to
know
is
usually
what's
called
aggregates,
so
say
you
wanted
the
distribution
of
people's
income,
maybe
in
a
particular
region,
or
maybe
you
want
to
look
at
their
relationship
between
like
income
and
height
which,
by
the
way
there
is
one
or
you
want
to
know
what
the
most
popular
websites
are.
But
I
don't
care
like
what
less
any
individual
goes
to.
I
Often
you
need
to
like
slice
the
data
in
multiple
ways
to
say:
look.
I
just
want
to
look
at
a
given
region
or
when
I
compare
two
variables
I
want
to
regress
them
against
each
other.
So
in
these
cases
you
know
it's,
as
I
say,
it's
actually
not
useful
to
have
to
have
the
individual
data
beyond
what
lets
you
compute
these
aggregate,
metrics
and
and,
of
course,
it's
very
harmful
individual
data.
I
If
you
misuse
it
and
from
the
perspective
of
a
researcher,
not
only
you
know,
it's
not
just
a
matter
of
it's
not
just
a
matter
of
of
harmful,
but
it's
a
matter
of
dangerous,
because
now
you
have
to
have
also
controls
and
procedures
around
hand
on
the
data
that
make
it
very
hard
to
work
with,
and
of
course
also
it
makes
it.
People
unwilling
to
you
know,
share
the
data
with
you
if
they
know
if
they
think
you're
not
going
to
handle
responsibly.
I
I
I
I
want
to
tell
you
a
little
bit
about
the
kinds
of
it's
the
kind
of
output
measurements.
You
might
want,
there's
a
number
of
common
measurement
tasks
that
that
we're
hoping
to
achieve
in
this
working
group.
So
the
first
is,
what's
often
called
a
simple
aggregates.
This
is
the
stuff
you
would
like
learn
in
an
introduced
class
of
you
know.
You
know
single
figures,
group
statistics
that
capture
some
data
so
mean
median
sum
histograms.
I
I
think
then,
there's
sort
of,
like
you
know,
relationships
between
values,
correlation
coefficients
ordinary
squares,
that
kind
of
stuff-
and
you
know,
people
talk
like
federally
machine
learning.
That's
kind
of
out
of
scope
for
this,
but
simple
stuff
is
in
scope
and
and
then
there's
this
specific
problem.
That's
often
called
heavy
hitters,
which
is
collecting
common
strengths
that
a
lot
of
people
have.
I
It
turns
out
to
be
a
very
useful
technique
for
numbers
and
number
settings
so,
like
I
said
you
can
notice
that
these
are
all
like
the
aggregate
things
I
don't
depend
on
any
anybody's
individual
did
it.
So
we
just
like
to
find
some
way
to
gather
those
these
aggregates
without
having
to
be
infected
by
people's
individual
data,
which
is
like
basically
toxic
waste
at
some
level.
I
So,
like.
Let
me
give
you
like
one
motivating
use
case,
it's
very
useful
to
know
what
kind
of
sites
you
should
visit,
because
then,
if
you're
like
a
web
browser,
so
that
our
backlight
use
case,
if
you're
like
web
browser,
you
like
to
know
what
kinds
of
stuff
people
visit.
So
you
can
make
your
web
browser
work
well
on
those
sites,
and
so
you
can
spend
and
spend
time
and
say:
okay,
well,
people
like
really
watch
a
lot
of
videos
like
making
video
really
work
is
important.
I
And
so
there's
been
a
lot
of
talk
about
this
in
the
ads
context
of
like
what
topics
are
safe
but
like
in
an
ideal
world.
If
you
didn't
care
about
privacy,
what
you
want
to
do
is
like
bucket
the
sites
by
topic
and
then
count
the
number
of
like
minutes,
but
on
each
topic,
but,
like
I
said
we
can't
just
do
that.
But
so
this
is
another
problem
statement
is
to
collect
the
distribution
of
time
spent
on
each
type
of
site
without
actually
seeing
the
individualized
people
on.
I
So
that's
like
one
motivating
this
case.
Another
mode
of
use
case
again
for
a
browser
is
to
see
which
websites
are
having
problems
of
one
kind
or
another.
In
some
cases,
these
problems
are,
I
guess,
innocuous,
so
like
web
compatibility
is
a
big
problem.
Some
sites,
just
don't
render
properly
and
like
mozilla,
operates
like
a
thing
which
you
can
press
in
the
upper
right
hand
corner
of
your
browser
somewhere.
I
That
says,
like
this
site
is
broken
from
me
but
like,
but
we
depend
pretty
heavily
on
people
volunteering
information
because
we
obviously
don't
want
to
collect
what
url
everybody's
going
to.
This
is
like
a
bigger
problem
for
browsers
with
smaller
market
shares,
because
things
will
often
work
on
like
one
engine
or
not
another.
So
in
many
cases
we
can
detect
breakers
on
the
client.
I
We
know
that
something's
wrong,
like
they
try
to
use
a
property,
we
know
it
doesn't
exist,
or
these
are
saying
reload
constantly
like
rage
clicking,
but
we
can't
do
anything
about
it
because,
like
the
browser
knows,
but
it
can't
tell
us-
and
so
that's
one
example.
Another
example
is
that
we
know
there's
a
lot
of
like
what's
called
fingerprinting
going
on.
I
So
an
example,
people
often
talk
about
is
what's
called
canvas
fingerprinting
where
you
like
render
some
fonts
and
then
you
read
back
from
the
canvas,
and
that
gives
you
information
about
like
the
gpu
uses
the
machine,
so
you
can
use
this
to
build
up
a
like
single
value
which
you
can
use
use
like
follow
the
user
on
the
internet.
So
this
is
like
a
big
tracking
that
is
not
addressed
by
the
kinds
of
you
know.
Third-Party
cookie
blocking
the
browsers
like
firefox
and
chrome.
I
Do
this
is
again
off
the
detectable
on
the
client,
because
you're
like.
Why
is
this
person
like
doing
a
lot
of
canvas
readbacks
when,
like
they're,
not
actually
just
playing
anything
meaningful
and
or
like
they're?
Actually
loading?
Some
some
like
script,
which
you
know
is
a
tracking
script,
but
you
just
can't
like
report
it
back
for
exactly
the
same
privacy
reason.
I
So
we're
stuck
in
the
situation
where,
where,
if
you
have
like
the
sort
of
accurate
information
about
which
sites
are
having
problems,
you
could
just
think
about
it,
could
you
go
to
the
site
and
you
could
like
download
the
script
and
find
it
yourself,
but
but
doing
that
would
entail
collection,
browsers,
distribute
problematic.
One
thing
I
would
say
is
people
often
say
like:
why?
I
I
And
I'm
just
gonna
preview
on
the
the
rest
of
people
here
were
talking
about
on
a
number
of
other
cases.
I
guess
I
I
hit
this
first
one,
so
I
should
have
like
that.
My
slides
got
changed,
but
there's
a
bunch
of
talk,
I
think
other
use
cases
involving,
but
some
advertising
work
and
also
some
work
on
covenant.
Exposure
notification
measurement
so
you'll
see
those
later,
but
I
mean
this
should
give
you
a
sense
of
breath.
I
So
it's
important
to
recognize.
There
actually
are
two
kinds
of
privacy
threats
with
these
kinds
of
data
collections.
The
first
is
when
you
collect
sensitive
data
and
it's
directly
tied
to
identifying
information,
so
you
say:
look
I,
like
you
know,
did
a
survey
and,
like
I
called
people
on
the
phone,
and
they
told
me
you
know
they
tell
me
this
sensitive
stuff
and
now,
like
I
have
the
phone
number
and,
like
you
know,
they
told
me
like
you
know
whether
they
have
a
particular
medical
condition.
I
I
The
client
like
and
it
encrypts
the
data
to
the
collector,
and
then
you
have
some
proxy
in
the
middle
that
removes
all
the
metadata
like
the
ip
address,
and
so
this
this
avoids
the
collector
like
seeing
that
meta
information
but
still
gets,
and
because
that
is
encrypted,
the
proxy
never
sees
the
report
so
that
these
things
are
split
up
and
we
talked
about
the
trust
model
for
that.
So
I
won't
go
into
that
in
much
detail
and
so
the
number
of
ways
to
do
this.
I
It's
really
good
for,
like
sort
of
boosting
the
privacy
of
semi-sensitive
data,
like
did
you
collect
anyway,
you
say
well
like
I
wish
it
had
the
ip
address
you
can
get
rid
of
it,
and
so
it's
very
common
now
for
like
browsers
to
collect
telemetry
and
we
have
the
ip
address
which
you
just
throw
away,
because
we're
going
to
have
that
involved.
There
are
also
a
bunch
of
cases
where
you
want
to
collect
individual
values
and
these
free
from
data
blobs
that
you
want
to
really
dig
into.
I
I
I
So
I
can't
use
this
if
I
want
to
do
this
kind
of
correlation,
because,
unfortunately,
demographic
information
is
self
defined
and
the
same
thing
is
true:
if
you
want
to
do
like
crosstabs
and
subgroups,
it's
also
not
good
for
collecting
this
kind
of
heavy
hitter
stuff,
and
the
reason
is
because
the
the,
even
though
it
is
anonymized
you
get
all
the
values,
even
the
low
cardinality
values.
A
lot
of
the
cardinality
values
are
problematic,
so
they
might
be.
I
I
So
you,
the
the
what
the
client
does
is.
The
client
wants
to
report
some
value
and
it
takes
the
value
and
it
splits
it
up
into
two
shares,
with
a
secret
sharing
technique
of
some
kind
and
the
way
the
shares
are
constructed
is
that
knowing
only
one
share
doesn't
give
you
any
information
about
the
other
value
and
that
that
splitting
is
what's
called
information
theoretically
secure.
Maybe
it
doesn't
depend
on
computational
assumptions,
but
when
you
put
the
shares
together,
they
of
course
represent
the
entire
value.
I
So
what
you
do?
Each
client
sends
like
one
share
to
one
server
another
share
another
server
and
then
the
servers
take
the
shares
themselves
and
they
aggregate
them.
They
compute
the
aggregate
value
but
again
you're
just
working
on
the
partial
data.
So
you're
not
learning
anything,
and
then
you
could
take
the
aggregate
so
like
say,
for
instance,
you
want
the
sum
you've
got
a
partial
sum,
and
then
you
take
each
partial
sum
and
you
put
them
together
and
you
get
the
final
output.
I
I
So
let
me
just
pause
for
a
second
before
I
talk
about
the
crypto,
which
is
the
trust
model,
because
this
is
like
really
important.
I
know
this
comes
up
a
lot.
We
talk
about
these
systems,
so
the
client's
requirement
is
the
two
servers
do
not
collude.
If
it's
a
service
clue
they
can
be
digital
values
and
it's
game
over
right,
and
so
that's
very
hard
to
operate
between
people.
Obviously,
and
the
client
has
to
trust
exactly
one
of
them.
The
client
is
great.
I
The
client
trusts
both,
but
as
long
as
one
of
them
doesn't
cheat,
it's
fine
and
you
can
do
end
servers,
but
two
is
the
most
common
number.
Obviously,
the
servers
also
have
to,
for
various
reasons,
to
do
a
little
bit
of
enforcement
about
like
minimum
batch
sizes
and
query
limits
and
stuff
like
that.
To
avoid
some
attacks
that
we
will
talk
about
here
for
the
collector's
requirement.
Both
servers
have
to
actually
be
executed
protocol
correctly
because
either
server
can
like
distort
the
results
that
they
don't.
I
But
again,
this
is
only
a
correctness
requirement
that
only
one
server
is
required
to
behave
correctly
from
the
client's
perspective.
So
I
just
want
to
recognize
right
up
front.
It's
like
difficult
to
verify
from
the
client's
perspective
that
the
servers
are
colliding.
That
conclusion
could
happen
through
side
channels
depending
on
the
architecture.
Sometimes
it's
the
sideshows
are
small
signs.
They're
big!
I
You
can
do
point
in
time
on
this
to
verify
that
someone
is
behaving
correctly,
but
you
can't,
but
it's
like
not
possible,
you're
not
colluding,
and
I
just
want
to
like.
So
I
want
to
highlight
that,
but
I
also
want
to
say
that,
like
this
is
like
a
very,
very
common
scenario
on
the
internet,
where
you,
where
people
have
data-
and
you
have
to
trust
and
behave
correctly-
I
mean,
if
you
think
about
like
you
know
your
data
is
in
gmail.
I
Like
you
know,
google
has
like
your
entire
email
record
right
and
your
questions
and
behave
correctly,
and
you
know
even
you
know,
even
if
the
server's
running
on
your
machine,
like
you
know,
generally
people,
think
of
that
as
behaving
correctly,
but
like
your
ability
to
verify
the
software
running
on
your
device,
extraordinary
limited
and
so
like
wallaby
wallace
it'd
be
great
to
have
a
situation
in
which
we
never
had
to
trust
anybody.
That's
simply
not
the
situation.
I
Most
of
us
find
ourselves
in
so
we're
talking
about
here
is
trying
to
like
alleviate
the
situations,
trust
people
and
make
the
number
of
people
have
the
trust
smaller
or
the
number
of
people
have
to
cheat
larger.
I
guess
we're
not
talking
about
a
limiting
trust
entirely.
It's
not
possible
the
state
and
our
technological
development
and
so
we're
trying
to
improve
the
situation,
but
we're
not
trying
to
like
boil
the
ocean.
I
So
I
want
to
talk
very
briefly
about
like
one
cryptographic
protocol
to
give
you
a
sense
of
like
the
situation.
This
is
sort
of
the
one
that
started
it
off.
It's
called
prio
and
it's
useful
for
computing
like
numeric
aggregates,
like
sum
and
mean
that
kind
of
thing
and
like
this
is
like
the
one:
that's
the
most
apprehensible
and
like
we're
gonna
punt,
the
crypto
to
cfrg,
but
this
one
is
like
understandable,
like
normal
humans.
So
we
assume
each
client
has
some
value.
I
That's
in
the
numeric
value
and
like
called
x
of
I
right,
and
so
the
client
does.
Is
the
client
splits
up
that
value
in
the
following
way?
It
generates
a
random
value,
sorry
about
the
fancy
math,
but
basically
a
random
value
is
smaller
than
a
prime
and
then
it
basically
does.
It
sends
server
one
like
the
value
minus
the
random
value
module
of
the
prime
and
it
sends
server
to
the
random
value.
I
It's
like
it's
quite
easy
to
convince
yourself
that
if
you
know
that
oh
no,
the
random
value
is
not
enough
and
knowing
and
knowing
that
the
the
subtraction
is
not
enough,
and
that
is
sufficient.
So
now
each
server
takes
all
the
shares
I
get
from
everybody
and
they
add
them
up
right
and
again
like
because,
because
because
because
these
are
like
information,
theoretically
they're
allowing
anything
and
then
they
and
then
they
basically
exchange
the
exchange,
the
sums
or
really
like
one
sentence
on
the
other.
Probably.
I
And
then,
if
you
take
the
sums,
you
add
them
up
and
you
like
do
a
bunch
of
like
much
like
relatively
simple.
Like
you
know:
middle
school
math.
You
could
just
convince
yourself
because
addition
is
commutative
that
when
you
add
the
sums
up,
you
actually
get
some
of
the
initial
values
so
admission
accomplished,
which
all
this
all
this
computation
is
zero
knowledge
and
we
have
the
output
at
the
end.
I
By
doing
some
logs
geometric
means
coming
from
product,
you
can
do
variance
and
standard
deviation
by
computing,
some
some
of
the
values
of
the
squares
there's
a
button,
there's
a
bunch
of
fancy
stuff
for
doing
like
or
and
mid
max,
and
even
ordinary.
Obviously,
these
squares,
so
the
trigger
is
just
finding
the
writing
coding
and
there's
like
papers
now
about
how
to
do
all
this
stuff.
I
I
Unless
you
can
date
yourself
and
you
just
live
with
it,
you
just
live
with
the
noisy
data
right,
because
people
don't
lie
consistently
and
then
there's
like
consistent
completely
ridiculous
data,
or
I
like
say,
like
I'm,
a
kilometer
tall
or
worse,
I
say
like
I'm
negative,
kilometer
tall
right
and
ordinarily.
What
you
do
is
you
just
like
have
some
filter
mechanism
where
you
said
you
know
you
just
said
like
well,
I
I
I
just
rejected
anything.
This
is
a
kilometer
tall
right
and
but
like
with
the
pre-it
is
encrypted.
I
You
can't
do
that
right
and
so,
and
so
instead,
what
you
do
is
like-
and
this
is
the
fancy
math
part.
Each
submission
comes
with
a
zero
knowledge
proof
of
validity,
and
the
proof
says
something
like
this
height
report
is
like
between,
like
100
and
200
centimeters
right.
The
servers
work
together
value
the
proof
and
you
only
aggregate
the
submissions
that
have
ballot
proofs
right
so
like
you've
got
to
trust
me.
I
I
So
okay,
so
like
going
back
to
my
use
cases
right
say:
I
want
to
collect
these
user
interests
right.
So
basically,
what
you
call
user
interesting
thing
like
preo
is
that
every
user
interest
is
a
bucket
and
you
have
like-
I
don't
know:
100
200,
500
buckets
right
and
the
client
individually
reports
time
spent
in
each
bucket.
You
have
to
report
the
ones
that
are
zero
too
by
the
way.
I
Otherwise
you
can
just
look
at
which
buckets
are
reported
and
then
use
prior
sum
up
and
end
up
with
a
bunch
of
sums
one
for
each
bucket,
and
now
you
know
exactly
how
much
your
time
was
spent
on
each
bucket
for
each
pocket.
But
you
don't
anybody's
individual
time
spent
and,
as
I
know
you
can
always,
you
can
also
report
t
squared.
You
could
be
standard
vision
as
well.
I
I
So
the
idea
is
that,
like
each
client
submits
a
string
like
a
url
and
you
want
to
output
the
end
most
frequent
strings
and
I'm
going
to
like
very
very
aggressively
handle
this,
but
the
basic
idea
is
that
the
servers
can
join.
You
compute
the
number
of
strings
that
any
given
prefix
p,
and
so
what
you
do
is
you
start
with
like?
I
I
I
But
then
you
can
say:
okay,
computer,
I
get
over
the
subsets,
so
that's
like
a
very
powerful
today.
That's
one
reason
why
this
is
like
a
powerful
technique
and
ways
that
like
ohio,
is
not,
but
of
course
it
means
that
repeated
queries
can
be
used
to
determine
values
by
like
querying
for
the
subset
like
includes
them
and
then
excludes
them.
There's
defenses
against
this
having
minimum
batch
sizes,
anti-replay
randomization
for
differential
privacy.
This
is
a
piece
of
work.
I
I
So
what
is
the
state
of
the
play
here?
A
number
of
us,
some
of
the
people
you're
representing,
have
basically
developed
a
generic
protocol
that
is
designed
for
doing
privacy
pure
measurement.
What
I
mean
by
generic
is
that
it's
a
framework
protocol
that
then
you
can
plug
in
you
know
individual
cryptographic
technologies,
so
it's
compatible
with
the
basic.
I
What
these
things
called
verifiable
should
be
variation
functions
which
we'll
be
talking
about,
but
initially
it's
tuned
to
work
without
preo
and
heavy
hitters,
it's
built
on
top
of
https.
It's
going
to
smell
a
lot
like
you
know
any
rest
kind
of
protocol
like
acme
or
whatever
you've
seen
before
that's,
what's
easy
to
implement
with
physical
services
infrastructure
and
about
a
number
of
people
working
on
this,
like
you
know,
having
work
from
regular
infrastructure
to
design
and
work
well
with
this
so
yeah.
I
So
let
me
just
give
you
like
system
architecture
picture.
That's
what
it
looks
like
you
know,
there's
a
bunch
of
clients
they
send
their
shares
and
then
there's
like
a
leader
and
and
the
leader
like
basically
takes
the
shares
and
parcels
about
the
helpers
and
orchestrates
the
whole
computation.
And
then
the
data
is
like
sent
to
a
collector
as
the
clutch,
because
I
look
give
me
like
the
output
for
this,
this
subset
of
the
system,
and
then
it
splits
the
results
out
right,
and
so
one
way
to
think
about
this.
I
I
think
there's
a
number
of
playing
modes
compatible
with
one
of
which
is
that
you
know
the
collector
and
the
leader
are
the
same
person
and
they're
trying
to
do
it.
The
data
collection
and
they
outsource
the
helper
job
to
one
other
person
so
that
they
can
make
guarantees
about
the
privacy
of
the
system.
Another
possibility
is
the
whole
is
a
whole.
Like
leader,
collector
helper
box
is
like
a
service.
That's
provided
people.
I
The
trust
model
here
you
should
be
assuming
is
that
the
clients
have
to
know
what
the
helpers
later
are,
and
so
the
clients
know
who
the
who
the
data
is
being
encrypted
for,
and
so
they
can
make
their
own
assessment
of
whether
or
not
they
trust.
One
of
those
people,
though,
of
course
in
a
real
world
scenario.
I
So
you
know
people
are
actually
going
to
ask
like
what
like
the
situation
of
ill-high
is
because
oh
hi,
as
I
mentioned,
is
useful
for
many
of
these
kinds
of
settings.
These
are
complements
and
not
substitutes.
So
you
know
good
cases
for
ohi
are
like,
as
I
say,
sort
of
standing
sensitive
data.
This
will
kind
of
rich
freeform
data
that,
like
you,
couldn't
really
you
know
aggregate
this
way.
Anything.
I
Of
course
it
needs
any
kind
of
responsible
right
because,
like
none
of
this
technology
gets
your
response,
it
just
measures.
Good
cases
for
ppm
are
like
really
sensitive
data,
hopefully
simpler
data
because,
like
as
you
sort
of
find
any
idea,
it's
not
like
really
easy
to
like
do
very
complicated
stuff
and
settings
where
you
just
kind
of
do
drill
down
you
do
regression
or
you
do,
do
subset
correspondence,
icons
and
things
so
like
these
are
like
you
know,
these
are
two
great
hits
that
tastes
great
together
right.
I
You
can
use
ohio
to
talk
to
ppm
server,
that's
like
boosting
the
privacy
of
the
system.
So,
like
you
say,
okay
well,
I
do
want
to
collect
this
data,
but
I
actually
store
an
ip
address,
so
you
can
remove
that
as
well,
and
so,
in
fact,
I
think
you'll
see
you'll
see
some
of
the
some
of
that.
Like
later
talks,
you
can
also
use
sort
of
a
front-end
proxy
server
to
do
a
bunch
of,
like
you
know,
kind
of
misuse.
I
Detection
of,
like
you
know,
spamming,
attacks
and
stuff
like
that.
So
so
I
think
these
are
complementary
techniques
not
but
not
not
competitive,
which
is
why
you
see
some
of
the
same
people
working
on
them.
I
don't
know
why.
Oh
yeah
right,
I
was
like.
Why
do
I
have
two
more
slides,
so
I'm
now
done.
I
think
I
hit
my
target
time
target
quite
well.
I
have
a
little
time
for
questions
which
I'd
be
happy
to
take.
I
Oh,
I
do
want
to
say
one
more
thing,
but
while
I'm
waiting
to
see,
if
anybody
will
say
anything
which
is
our
assumption,
is
the
vdf
functions
we
standardized
or
defined
in
cfrg,
not
an
itf.
And
so
the
idea
is
we'll
build
the
framework
here
and
we'll
defer
the
work
of
defining
the
v
dash
to
cfrg
and
that
work
is
already
probably
going
on
or
is
being
presented
as
a4g
west.
K
Boss,
thanks
good
presentation,
I
enjoyed
you
laid
it
out
really
well,
the
one
thing
that
I'd
like
to
hear
more
about
is
sort
of
the
deployment
scenario,
for
example
on
slide
14,
don't
go
back
you
you
specifically
said
the
client
wants
to
report
some
value
right.
So
would
your
expectation
be
that
application,
authors
and
and
servers
would
sort
of
make
use
of
the
same
way
that
sort
of
ohio
is
considering
being
deployed
by
you
know
various
organizations
to
get
into
stuff
or
that
you
know
doe
is
being
used
for.
K
I
Absolutely
yeah,
I
think
so
I
think
I
think
the
most
likely
settings
for
this
initially
will
be
the
kind
of
like
kind
of
measurements
that,
like
people
are
already
taking
via
the
software
disseminate.
So
you
know
things
like
browser
telemetry.
You
know,
I
think
the
the
case
of
the
case
of
the
the
tim.
What
we're
talking
about
next
involves
measurement
of
cover,
exposures
and
yeah.
Those
things
are
all
being
done,
like
sort
of
sort
of
like
you
know
automatically.
I
You
know,
potentially
by
asking
these
are
automatically
by
the
software.
I
think
there's
some
possibility
that
in
the
future
you
know
you'd
see
like
this
views
for
surveying
for
direct
surveying.
Where
you
say,
okay,
you
know,
are
you
willing
to
participate
in
surveys
and
then
we'll
walk
down
to
the
client?
And
you
can
do
it,
but
I
mean,
like
you
know
the
the
yeah
I
mean
I
think
yeah
you
know
these
are
these?
I
It
basically
cannot
be
kind
of
be
contained
with
ohio,
because
you,
because
you
learn
the
you-
learn
a
bunch
of
the
you-
don't
want
to
learn
any
little
cardinality
sites,
those
those
are
very
dangerous
right,
because
if
you
like
collect
like
every
url
from
what
it
goes
to,
if
this
is
behaving,
then
there's
a
real
probability
that
you
learn
a
bunch
of
like
say:
google
docs,
you
know
capability
urls
or
you
learn.
You
know
you
know
some
document
quality.
I
I
That
would
be
80
80
counters,
like
really
problematic
with
the
high,
because,
basically,
if
you
decide
to
get
every
single
value
and
report
it
separately,
otherwise
you
end
up
a
situation
where
you
can
build
these
high
dimensional
data
sets
and
use
them
to
to
do
production.
So,
like
those
are
both
cases
where
all
hype
does
not
perform
particularly
well,
though
the
second
like
you
could
maybe
get
away
with,
but
it's
probably
problematic
andrew.
L
B
I
J
J
All
right,
thank
you,
okay,
so,
let's
get
started.
My
name
is
tim
and
I'm
an
engineer
at
the
internet
security
research
group
we're
the
non-profit
that
operates
the
let's
encrypt
certificate
authority,
not
to
be
confused
with
the
irs
g.
Incidentally,
the
dilemma
that
mr
riskorla
just
introduced,
in
which
the
essential
function
of
gathering
telemetry
from
the
field
introduces
significant
privacy
risks
for
users
is
of
great
interest
to
the
isrg.
J
Given
our
mission
of
reducing
barriers
to
secure
and
private
communications
on
the
internet
next
slide,
please
so
ecker
covered
that
you
know
the
ways
in
which
telemetry
is
a
privacy
risk
for
users
and
the
you
know
the
implied
value
benefits
to
users
of
using
these
new
technologies.
But
I
think
it's
worth
noting
that
many
data
collectors
also
want
to
do
the
right
thing
and
respect
the
privacy
of
their
users.
Besides
that
being
a
decent
thing
to
do.
J
The
large
amount
of
personal,
identifying
information
stored
by
conventional
telemetry
systems
is
a
significant
liability
for
the
data
collector.
There's
new
privacy
regulations
emerging
in
various
jurisdictions,
all
the
time
which
require
expensive
and
complicated
controls
around
user
data
and
all
that
pii
makes
for
a
very,
very
tempting
target
for
attackers.
J
J
So
we
envision
running
a
standards
compliant
aggregator
as
a
service,
with
the
same
focus
on
automation
and
use
of
integration
that
drive
let's
encrypt.
We
expect
that
some
customers
will
want
to
run
their
own
aggregator
and
have
it
work
with
hours,
but
others
will
want
to
avoid
running
any
servers
at
all
and
will
instead
choose
two
existing
aggregators,
say
one
run
by
isrg
and
one
run
by
some
other
organization
or
a
company
that
chooses
to
participate.
J
So
in
support
of
that,
we
we
are
hoping
to
provide
an
open
source
implementation
of
a
ppm
aggregator
with
the
aim
of
making
it
easy
for
some
data
collector
to
interoperate
with
isr
use
aggregator
or
anybody
else's.
So
hopefully
it
ends
up
being
a
matter
of
grabbing
a
container
image
from
some
public
registry.
J
We
are
also
aiming
to
provide
open
source
client
libraries,
targeting
like
a
variety
of
languages
and
frameworks
chosen
to
you
know,
facilitate
adoption
for
the
most
likely
interested
parties.
So
you
know
you
can
imagine
like
a
swift
sdk
for
ios
apps
javascript
for
web
app
for
a
single
page
application
on
the
web
and
so
on.
J
So
right,
so
with
this
goal
in
mind,
an
open
standard
through
the
ietf
is
acutely
valuable,
because
a
proprietary
single
vendor
solution
wouldn't
be
terribly
useful,
since
the
privacy
guarantees
are
contingent
upon
independent
and
non-colluding
aggregators
next
slide.
Please
thank
you.
So
we
talked
a
lot
about.
You
know
the
benefits
of
these
new
technologies,
but
there
are
some
trade-offs,
some
drawbacks
to
these
systems.
So
for
one
thing
there
are
more
servers
involved,
so
it's
more
likely
more
likely
to
fail
somewhat
necessarily.
J
Finally,
metrics
gathered
under
these
schemes
are:
these
skins
are
necessarily
less
flexible
than
conventional
telemetry
systems.
You
can't
make
arbitrary
post-hoc
queries.
Sorry,
you
can't
make
arbitrary
queries
post-hoc
against
your
your
corpus
of
data.
You
have
to
know
up
front
before
you
begin
collecting
any
data.
What
are
the
aggregations
you're
interested
in
computing?
This
has
to
do
with
the
construction
of
the
proofs,
as
well
as
enforcing
some
of
the
privacy
guarantees
of
the
system.
J
Enpa
allows
back
anonymously
and
privately
back
hauling
that
data
to
your
regional
public
health
authority
so
that
they
can
get
information
on
how
many
people
are
getting
the
notifications,
how
many
people
are
on
their
mobile
devices,
how
many
people
are
interacting
with
them
and
all
sorts
of
interesting
metrics
about
the
spread
of
covet
itself,
as
well
as
the
effectiveness
of
the
of
the
en
system.
So
this
is
currently
deployed
in
13,
u.s
states
and
the
district
of
columbia,
and
at
the
moment
it's
gathering
2.1
million
measurements
per
hour.
J
We
also
heard
last
night
to
throw
in
one
more
interesting
number
that
sometime
over
the
night,
we
gathered
12
billion
individual
metrics
that
have
been
aggregated
since
the
system
launched
we're
also
about
to
deploy
this
internationally.
So
beyond
the
united
states.
So
we're
hoping
to
soon
turn
this
on
in
four
states
in
mexico.
Okay,
I'm
already
well
over
time,
so
I
will
see
the
floor.
Thank
you.
B
B
H
Okay.
So
I'm
going
to
talk
very
briefly
about
some
of
the
use
cases
in
advertising,
specifically
the
conversion
measurement
one.
I
think
charlie's
going
to
follow
up
with
some
more
details
on
on
this
one.
So
conversion
measurement
is
something
that
happens
on
the
web
quite
a
bit
when,
when
someone
shows
an
advertisement,
it's
kind
of
nice
to
know
if
that
advertisement
is
having
the
intended
effect.
H
H
So
the
way
this
works
today
is
pretty
simple:
we
assign
a
user
an
identifier,
so
everyone
gets
their
own
unique
identifier
and
every
time
they
visit
a
website
and
an
advertisement
is
shown
or
they
do
something.
Then
we
just
create
a
little
little
log
record
that
records
all
the
details
from
the
context
and
the
user
identifier
time
stamps
all
those
sorts
of
other
things.
H
And
then
you
look
at
the
log
and
you
can
answer
all
sorts
of
questions
about
what
people
have
done.
It's
it's
really
great
for
getting
the
information
that
you
need.
It's
all
quite
precise
and
leaving
aside
all
of
the
complications
of
anti-fraud
and
all
those
sorts
of
other
things
you
can
answer
all
the
questions
that
you
have
fairly
precisely.
H
The
current
status
of
this
work
is
that
there's
lots
and
lots
of
requirements
that
are
coming
through.
This
is
obviously
a
lot
more
complicated
when
it
comes
into
practice.
There's
lots
of
ideas,
people
have
all
sorts
of
wonderful
proposals
and
some
competing
requirements,
but
a
lot
of
the
really
promising
ideas
include
something
like
what
echo
described
earlier.
H
B
M
Let's
see
if
this
works
great,
you
guys
can
see
it
yep,
okay,
great
hello,
everyone.
My
name
is
charlie
harrison.
I'm
a
software
engineer
working
at
google
looking
at
ppm
like
solutions
for
doing
ads
measurement
on
the
web
to
satisfy
similar
use
cases
that
martin
was
just
talking
about
so
I'll
kind
of
breeze
through
some
of
the
background.
Just
because
there's
a
lot
of
overlaps
with
martin's
slide.
M
Is
that,
like
many
ads
use,
cases
are
actually
like
totally
fine
with
aggregate
data
and
they
don't
actually
need
to
track
you
kind
of
around
the
web
and
learn
all
the
data
exactly
we,
we
could
be
fine
with
aggregate
data
in
many
cases,
so
I
I
want
to
go
over
just
in
a
little
bit
more
detail.
How
attribution
measurement,
which
is
also
called
conversion
measurement,
happens
today
with
cookies.
M
M
So
you'll
read
this
cookie
when
an
ad
is
placed
and
you'll
also
read
this
cookie
when
there's
a
conversion
like
when
you
buy
something
later
on
down
the
road
after
you've
seen
an
ad
for
it
and
so
right
now,
this
cookie
id
is
used
as
the
join
key
to
join
these
two
cross
site
events
right
and
but
they
they
they
join.
Arbitrary
events,
so,
like
all
of
your
browsing,
can
be
linked
up
to
this
one
cookie
in
theory.
M
How
could
it
be
improved?
We
could
internally
join
this
data
in
the
browser.
So
when
you
see
an
ad,
we
could
register
something
in
like
custom,
new
browser
storage
and
when
you
buy
something
that
was
like
pointed
to
by
that
ad.
That
would
join
up
with
that.
With
that
event
in
the
internal
browser
storage-
and
you
could
have
a
communication
path
from
the
browser
to
something
like
ppm,
where
we
we,
you
know,
we
have
this
data,
we
want
to
compute
something
like
a
histogram.
M
Maybe
we
want
to
learn
something
like
you
know:
what
are
the
accounts
of
conversions
like
purchases
per
ad
campaign,
so
the
x-axis
is
that
campaign.
The
y-axis
is
the
number
of
conversions
we
can.
We
can
use
ppm
to
kind
of
generate
a
data
data
share,
split
that
encodes
that
histogram
contribution
send
that
up
to
ppm
and
the
ad
tech
could
learn.
Aggregate
statistics
like
just
a
histogram
of
of
the
of
the
counts,
and
this
doesn't
reveal
any
user
data
directly.
M
M
So
that's
something
we're
looking
into
there's
a
lot
of
interesting
research,
that's
related
to
kind
of
the
heavy
hitter
stuff
about.
How
do
we
report
these
histograms
when
they're?
Like
really
really
really
big,
like
you,
are
running
millions
of
campaigns
or
you
want
to
do
like
all
sorts
of
different
crosses
we're
really
interested
in
systems
that
can
help
train
machine
learning
models
in
some
cases
there's
their
results
that
show
that
even
just
aggregate
histograms
could
be
used
to
train
like
logistic
models.
M
But,
like
are
like
we're
looking
into
more
sophisticated
mechanisms.
There's
you
know
in
rather
than
conversion
measurement.
We
could
look
at
reach
measurement,
which
is
asking
like
how
many
distinct
users
saw
my
ad
across
many
different
websites.
So
this
is
kind
of
like
removing
a
like
remove,
duplicates
operation,
and
you
know
we
we're
we're
interested
in
exploring
like
you
know.
B
N
G
All
right
so
yeah
this
is
this
is
pretty
different,
so
ecker
talked
about
protocol.
We
had
a
couple
talks
about
use
cases,
I'm
here
to
talk
about
kind
of
the
substrate
and
the
the
label
we
are
using,
for
this
is
verifiable,
distributed
aggregation
functions
thanks
to
chris
patton.
For
that
that
name
context
here
is
a
like
many
things
in
the
in
the
ietf.
We
need
some
complicated
crypto
for
this,
and
so
we're
doing
some
parallel
work
in
cfrg,
and
it
goes
alongside
the
crypto.
G
Now,
if
you
look
at
the
draft,
it's
kind
of
in
two
parts,
we
define
an
api.
That
is,
the
abstraction
ppm
is
supposed
to
rely
on.
So
the
idea
is
that
we
have
multiple
instantiations
that
do
various
flavors
of
this
private
aggregated
measurement,
dance
that
all
behave
in
a
close
enough
way
that
we
can
define
a
common
api
over
them
and
build
protocol
around
that
api,
and
this
provides
a
way.
G
So
the
idea
of
this
vdaf
is
to
kind
of
capture
the
critical
things
we
need
the
cryptography
to
do
inside
of
ppm,
so
kind
of
working
back
to
front
the
aggregation
bit
is
the
idea?
That's
you
know
the
inputs.
This
process
are
individual
measurements
and
what
comes
out
of
it
at
the
end
is
an
aggregate
measurement
over
those
individual
measurements
distributed
in
the
sense
that
it's
distributed
among
a
bunch
of
aggregators.
G
You
know
that
the
computation
of
that
aggregate
is
distributed
over
those
aggregators
and
the
privacy
properties
of
the
individual
measurements
are
assured
by
non-collusion
among
those
aggregators
and
finally,
it's
verifiable
in
the
sense
that
the
aggregators
can
check
that
the
inputs
have
meet
some
properties.
As
ecker
pointed
out.
You
know,
there's
risk
in
this
sort
of
distributed
scenario
that
the
measurement
results
that
get
reported
in
by
the
measurement
points
could
be
corrupt,
it
could
be
invalid
and
they
that
could
flow
through
and
lead
to
a
garbled
aggregate.
G
G
So
looking
at
how
this
kind
of
plays
out,
you
know
this
is
the
kind
of
classic
measurement
scenario.
This
is
how
a
lot
of
telemetry
works
today,
where
you
have
a
client
out
there.
You
know
actually
many
clients,
but
I'm
only
going
to
portray
one
on
this
slide.
You
have
many
clients
out
there
collecting
measurements
of
something
you
know
where
people
clicked
in
an
application
or
what
what
not
and
reporting
that
back
to
a
collector
and
the,
as
you
know,
you've
seen
various
takes
on
this
on
the
last
few.
G
But
the
idea
here
is
that
we're
introducing
this
aggregator
tier
in
between
the
client
and
the
collector
and
the
idea
of
these
aggregators
is
that
the
client
shards
their
each
measurement
out
to
individual
shards
of
the
aggregators
such
that
you
can
only
make
sense
of
that
measurement.
If
you
have
all
the
shares,
that's
how
we
get
that
non-collusion
guarantee
the
aggregators
then
go
through
this
process
of
what
I've
called
prepare
the
term
using.
G
This
is
kind
of
where
you
do
that
stuff.
That
echoes
over
you.
I
add
up
the
shares
into
a
share
of
the
sum
and
then
finally,
the
aggregators
send
their
aggregate
shares
over
to
the
collector
who
unshards
them
to
get
the
final
results.
So
this
is
kind
of
the
data
flow
view
of
the
same
thing.
I
think
the
interesting
thing
here
from
the
kind
of
protocol
point
of
view
is
you
notice,
there's
a
kind
of
back
and
forth
thing
at
the
preparation
stage.
G
There
is
a
need
for
some
some
chattiness,
some
interaction
between
the
aggregators
in
that
verification
process,
to
enable
a
distributed.
Verification
of
the
correctness
of
the
input
shares,
but
otherwise,
hopefully,
this
kind
of
explains
the
overall
process.
Also,
this
the
kind
of
little
gray
squares
that
parallel
to
the
output
shares
are
other
and
then
to
indicate
other
output
shares.
G
G
In
the
draft
we
define
a
couple
of
constructions.
I
think
ecker
probably
described
described
these
in
a
little
bit
more
detail.
Prio
is,
is
the
one
where
we
that
I
think
enpa
is
is
based
on.
The
diversion
in
the
draft
is,
is,
I
think,
has
a
few
more
changes
on
top
of
what's
in
the
npa,
but
that's
one
where
you
have
a
fair
bit
of
deployment
experience
and
then
the
hits
protocol
is
the
one
that
lets
you
get
distribution
on
strings.
G
There's
been
some
discussion
in
chat
about
whether
you
know
possibly
we
could
fit
something
like
star
in
here.
That's
I
mean
it's
kind
of.
The
idea
of
the
draft
is
that
we
can
look
at
not
just
these
two,
but
also
fitting
other
things
in
under
this
umbrella,
so
that
we
can
have
ppm
as
a
generic
construct
into
which
we
can
fit
multiple
different
cryptographic.
Realizations.
G
G
If
you
look
at
the
papers
in
which
they're
published
there's,
you
know
it
might
look
intimidating
in
terms
of
the
amount
of
data
that's
getting
sent
around
the
amount
of
computations
being
done,
but
with
these
implementations
we
have
some
early
data
about
the
expense
of
in
terms
of
computation
and
communications
overhead
of
doing
these
private
schemes-
and
you
know
your
mileage
may
vary,
but
it
seems
seems
tolerable
to
first
order.
Let's
say
that
so
yeah
I
think.
That's
all.
I
had
yeah
a
couple
references.
B
O
F
F
G
Yeah,
I
I
I
I
admit
that
I'm
not
an
expert
on
kind
of
what
the
ppm
protocol
is
doing
right
now.
I've
mainly
been
stuck
down
at
this
layer,
but
I
think
my
understanding
is
that
you
can
emulate
if
this
kind
of
looks
like
a
broadcast
channel,
and
if
you
have
a
leader
that
is
connected
to
multiple
helpers,
you
could
have
basically
a
star
topology
that
over
which
you
can
emulate
a
broadcast
channel.
So
if,
if
a
helper.
L
G
To
it
for
us,
for
instance,
in
the
in
the
in
the
pro
in
the
api
we've
sketched
out,
we
structure
the
preparation
process
in
rounds
where
the
input
to
each
round
is
the
output
of
the
previous
run
from
all
aggregators.
So
I
think
the
in
the
ppm
content
kind
of
communications
model.
You
can
envision
all
helpers,
submitting
their
outputs
from
the
previous
round
and
then
the
the
leader
distributing
those
outputs
out
to
all
of
the
helpers
for
the
next
round.
I
F
I
B
B
So
we're
going
to
move
on
to
a
call
for
expression
of
interest
from
folks
in
this
just
just
to
make
sure
that
there
is
enough
critical
mass
behind
this.
Aside
from
the
people
who
have
presented
here,
we'd
like
to
ask
who
in
attendance
is
interested
in
working
on
this
technology
trying
to
figure
out,
we
probably
don't
need
people
to
speak
to
this.
Necessarily,
although
we're
very
happy
to
hear
what
you
have
to
say
on
the
topic,
I'm
also
going
to
throw
up
a
quick
show
of
hands.
Q
Hello,
all
right,
I
I
did
the
show
I
just
wanted
to
speak
up
a
little
bit
to
mention
from
my
perspective.
You
know
we
definitely
have
or
already
we're
using
stuff
like
this
and
seeing
something
like
this
be
standardized
and
taken
on
by
the
word
group
would
be
something
that
would
be
a
positive
thing.
So
you
know
we
didn't
do
any
of
the
presentations
here,
but
we're
definitely
interested.
I
R
R
In
other
words,
what
systematic
approach
can
you
adopt
for
the
design
and
innovation
process
that
ensures
that
ethical
factors
and
values
are
considered
at
each
relevant
step,
and
I
think
that
would
that
would
help
clarify
a
lot
of
the
comments
in
the
chat
about
along
the
lines
of
yes,
but
this
happens
already
or
well.
This
only
works.
If
you
can
assume,
there's
no
collusion
and
so
on,
because
what
it
does,
is
it
flushes
those
assumptions
out
and
lets?
S
Yeah,
I
just
want
to
briefly
mention
that
I
do
think
this
is
exciting
technology
and
we
should
work
on
that.
I
can
see
other
types
of
applications.
We
mostly
talked
about
sort
of
application,
type
browser
type
of
things,
but
also
many
types
of
networks
could
probably
use
this
this
technology
to
do
a
better
job
at
collecting
information
that
needs
to
be
collected
for
various
debugging
and
other
other
reasons.
S
The
only
piece
that
I
was
sort
of
a
little
bit
concerned
about
was
was
this
advertising
piece
and
I
sort
of
just
as
a
personal
opinion.
I
had
some
reservations
about
browsers
working
with
advertisers,
even
though
I
do
recognize
that
I
need
to
do
something
better
than
we
do
today,
but
you
know
maybe
there's
also
other
paths
for
the
advertisement,
problems
trying
trying
to
prevent
information
flow
rather
than
collaboration,
but
I
don't
work
for
the
advertisement
in
the
industry
so
anyway.
I
I
like
this.
This
should
go
ahead.
B
B
D
D
Doesn't
really
involve
tying
two
actions
together
in
the
same
way
that
an
ad
conversion
does
so.
It
may
be
that
there
are
simpler
mechanisms
or
only
a
partial
use
of
this
system
that
would
still
satisfy
those
where
invoking
the
full
thing
might
be
more
difficult
and
not
necessarily
buying
you
everything
that
it
would
need
to
buy
you
when,
when
you
had
the
conversion
case,
so
I
think
as
a
set
of
problems
it's
interesting
to
work
on,
but
I
would
prefer
that
we
take
it
as
a
set
when
we
take
the
work
in.
T
T
I
am
a
bit
worried
that
we
seem
to
have
gone
straight
for
the
real
high
falutin
cryptography,
rather
than
measures
like,
let's
just
encrypt
log
files
as
they're
produced
rather
than
having
them
sit
in
plain
text,
ready
to
be
stolen,
so
there's
a
bunch
of
real
bread
and
butter
issues.
I
think
that
we
need
to
do
as
itf
before
we
do
the
stuff
that
really
excites
us.
The
other
thing
that
I
point
to
is
this
has
been
presented
in
terms
of
a
network
realization.
T
I
think
that
the
more
immediate
and
more
useful
use
for
this
would
be
to
assist
work
like
the
stuff
that
my
wife
does.
Analyzing
workers
compensation
claims
in
that
there
you've
got
enormous
amount
of
really
privacy,
sensitive
data
that
is
aggregated
together
and
then
she
analyzes
it,
and
so
it
that's
not
a
network
application,
but
that
transition
from
I
have
this
large
amount
of
data.
T
I
would
like
to
pre-process
it
into
a
form
that
it
can't
then
leak
in
a
dangerous
way
and
analyze
it
in
that
form.
I
think
that
that
technology
would
be
very
useful,
disconnected
from
the
whole
network
case,
and
it
might
make
for
some
starting
problems
that
are
rather
simpler
than
trying
to
think
about.
How
do
we
secure
the
web.
E
That
is
to
say,
we
we
want
to.
We
have.
We
need
to
collect
these
aggregate
statistics
to
answer
certain
questions
and
it's
whether
or
not
you
know
this
is
harmful
helpful
for
the
purposes
of
web
advertisements
and
conversion
measurements
in
a
privacy
processing
way,
I
think,
is
a
good
question
to
ask,
but
that
does
not.
I
don't
think
that
takes
away
from
the
the
very
valid
use
cases
that
were
also
presented
here.
E
That
could
certainly
be
improved
by
a
more
privacy
preserving
protocol.
So
I
I'm
very
strongly
supportive
of
this
work.
It
cuts
across
so
many
different
problem,
domains
and
use
cases,
and
it's
kind
of
inevitable
that
you
know
it'd
be
standardized
somewhere,
given
how
important
it
is.
So
that's
what
I
want
to
say.
Thank
you.
I
Yeah,
so
I
would
make
a
number
of
points
so
so
first,
I
think
you
know
ted's
point
yeah
I
mean
this
is
a
toolbox.
That's
got
that
that
has
a
variety
of
tools,
and
I
think
that
that
you
know
that
the
one
we
think
about
this
is
that
the
ppm
protocol
is
the
toolbox
and
the
feed
apps
the
tools.
And
so
the
idea
is
not
it's
not
it
doesn't
define
any
particular.
You
know.
I
Measurement
measurement
you
know
thing
is
to
define,
is
to
define
a
set
of
mechanisms
which
can
be
used
for
taking
different
kinds
of
measurement,
and
then
you
know
individual
applications
they
layer
on
top
of
those
tools.
So
that
would
be
true
both
in
terms
of
the
you
know,
which
b
dash
exists
and
it'll
also
be
true
in
terms
of
like
how
how
you
build
applications
on
top
of
the
top
of
the
vdf,
because,
like
you
know,
like
I
mean
one
thing
about,
this
is
like
you've
got
a
thing
that
basically
doesn't.
I
I
can
collect,
counts
right
and
but
like
what
do
you
do
with
those
counts
is
an
important
question
and
one
that
we
don't
pretended
to
define
the
you
know
it's
I
I
do
see
some
over
indexing
on
the
on
the
I'd
use
case
here
as
well.
You
know,
you
know
there
are
a
lot
of
use
cases
they're
not
ads
and
in
particular
like
there's
nothing
ad
specific
to
any
of
this,
all
the
ad
specific
work
which
exists
or
somewhere
else
in
in
past
cg.
I
So
I
mean
this
is
about
building
a
set
of
fulfillment
measurements
and
as
a
really
one
application
measurement
with
that
said,
I
do
think
that
you
know
this
question
about,
like
you
know
about.
This
is
for
letting
good
people
do
do
good
things.
I
think,
though,
there's
two
points
about
that.
One
is
that
there
are
a
lot
of
applications
where
I
as
wes
says.
I
I
I
think
application
case
two
is
that
there's
been
a
lot
of
like
attempts
to
look
at
what
it
would
take
to,
like
you
know,
reproduce
partially
out
ecosystem
with
better
privacy
because,
like
I
think,
we'll
agree
as
they're
not
going
to
go
anytime
soon
and
one
of
the
pushbacks
that
one
gets
when
one
says
doing,
that
is,
it
will
have
a
negative
impact
on
the
ecosystem
as
a
whole.
And,
like
you
know,
I'm
not.
You
know,
I
think
you
know.
I
Firefox
is
already
quite
a
bunch
of
anti-tracking
technologies,
so,
like
I'm
not
like
all
in
on
that,
but
I
think
there's
a
real
concern.
People
have,
and
so
the
idea
is
not
that
when
we
simply
offer
you
know
these
alternative
technologies
and
then
people
use
them
and
they're
part
of
cookies.
The
idea
is
that
the
idea
is
that
you'll
be
offered
alternative
and
that
one
would
eventually
have
to
deprecate
the
existing
privacy-based
technologies.
M
M
I
think
the
platform
can
really
mediate
this,
and
if
we
think
that
the
platform
provides
a
good
enough
foundation,
then
we
can
we
can
remove
the
stuff
that
we
think
is
is
bad
for
user
privacy,
and
this
is
like
our
our
strategy
that
we're
trying
to
do
on
chrome,
where
you
know
we're
investing
all
this
time,
trying
to
come
up
with
new
browser
primitives
and
also
we
have
like
a
timeline
where
we
want
to
disable
third-party
cookies
and
deprecate
them.
So
yeah
we're
we're.
B
B
Okay,
so
let's
go
ahead
and
move
on
to
the
charter,
then
I
have
it
up
here
in
front
of
us:
we've
broken
into
two
sections:
it's
probably
not
terribly
useful
for
me
to
try
to
read
through
it
directly
as
it's
on
your
screens,
but
we
want
to
go
ahead
and
take
some
comments
here.
I
see
our
area
director
has
stepped
up.
D
B
U
Thank
you.
So,
thanks
for
all
the
presentations,
it's
been
really
interesting.
So
most
presentations
today
were
about
use
cases,
but
there's
nothing
about
that
in
the
charter.
I
don't
think,
although
I
can
only
see
the
second
side
now
or
I
don't
think
currently
in
the
draft,
I
think
that
would
be
a
useful
thing
to
kind
of
document
somewhere.
I
Thanks
ecker,
I
think
we're
up
for
that.
I
guess
you
know.
I
think
the
the
one
thing
we're
trying
to
do
with
the
use
case
is
motivational
because
I
think,
like
the
these
are
generic
techniques,
but
I
think
be
for
I
think
I'd
be
more
than
happy
to
think
to
like
write
down
like
what,
like
I'm
not
going
to
visualize.
I
You
understand
why
you
want
to
collect
making
a
statistic
and
just
to
say
something:
siobhan
should
ask
in
the
chat
is
there's
room
for
like
for
like
multiple
for
like
for,
like
multiple
instantiations
of
the
same
basic
task.
I
think
absolutely
something,
for
instance,
if
there's
a
there's,
a
better
mechanism
for
for
collecting
heavy
hitters
than
than
hits,
which
is
relatively
expensive.
That
would
like
something
like
that.
I
think
room
for.
I
think
you
know
you
know,
like
I'd,
certainly
be
interested
like
like.
I
I
think
I
want
to
build
something
where,
like
as
new
photography
gets
adults,
we
can
add
it
and
I'm
going
to
build
something
where
we
have
multiple
mechanisms
that,
like
support
the
same
basic
thing,
I
think
you
know
the
only
the
only
question
I
I
I
want
to
make
sure
it's
like
you
know.
If
we
have
something
radically
different,
it's
all
like
some
light
weight
it
doesn't
match
up.
I
V
V
I
think
if
you
go
to
the
previous
one
previous
slide,
sorry,
I
think,
like
splitting
measurements
between
multiple
non-including
servers,
like
I
think
that
is
like
a
class
of
for
techniques
for
achieving
the
scope
of
privacy
preserving
measurements.
But
I
think
you
could
do
it
in
other
ways
as
well,
and
I
see
there's
some
support
to
rewrite
text
and.
V
W
I
think
the
charter
needs
to
say
something
about
use
cases
and
threat
models.
I
don't
have
any
formal
words
for
that
at
this
point,
but
I
do
think
that's
the
important
thing
to
capture
here,
because
if
we're
going
to
be
developing
multiple
solutions,
as
ted
was
alluding
to
everyone,
there
might
be
different
threat
models
that
apply
to
them
rather
than
the
single
overarching
one,
and
I
think
it
would
be
good
to
make
sure
that
information
is
clearly
captured
when
things
have
developed
and
obviously
that
needs
to
fit
into
the
charter.
Cheers.
M
L
Yeah
just
adding
to
the
the
early
point.
I
completely
agree
with
the
idea
of
adding
in
the
use
cases,
but
just
as
echo,
appointments
come
up
in
the
chat,
I
think
also
consider
adding
abuse
cases
and
ideally
mitigations
as
well,
which
might
help
alleviate
some
of
the
concerns
about
some
of
the
dark
practices
in
this
general
area.
Thanks.
B
X
Yeah,
just
under
the
abuse
cases
use
cases
thing.
I
think,
given
that
this,
I
think,
is
probably
pretty
good
technology
that
could
be
used.
Well,
I'm
not
too
worried
about
documenting
use
cases,
and
that
could
be
a
bit
of
a
time
sink,
but
given
that
this
good
technology
could
be
abused,
I
think
effort
spent
in
that
direction
would
be
much
more
valuable,
particularly
if
we
find
ways
of
mitigating
abuses
that
we
see.
O
I
I'm,
I
guess
I
think
we
should
be
discussing
it
if
we're
going
to
put
a
lot
of
effort
into
into
the
work
of
the
protocol.
That
depends
on
those
things.
The
other
concern
is
about
the
name.
I
think.
Maybe
it's
come
up
a
little
bit
already
in
the
chat.
I
I
think
priv
is
is
both
very
confusing,
does
not
describe
the
work,
that's
happening
about
privacy,
preserving
measurement
and
actively
misleading
that
this
is
the
group.
O
B
B
X
So
again,
pardon
my
ignorance,
it's
vast,
but
so,
for
example,
if
this
tech
this
this
technology
could
be
used
to
measure
things
into
small
in
aggregates
that
are
too
small
and
that
become
exposing
or
if,
if
some
application
had
some
kind
of
opt-in
mechanism,
and
then
it
was
possible
to
change
what
gets
measured
in
ways
that
are
damaging
for
users.
Those,
I
think,
would
be
kind
of
abuses
that
are
a
bit.
X
X
S
Thank
you
so,
plus
one
on
the
name,
confusion,
issue,
plus
one
on
talking
about
abuse
cases,
and
I
I
think,
that's
mostly
by
the
you
know.
Whoever
is
doing
the
measurements,
not
not
so
much
about
the
abuse
by
by
the
user.
Users
should
still
be
able
to
produce
the
data
that
they
want
to
produce
as
long
as
it's
sort
of
within
bounds.
S
In
order
to
create
this
set
of
users
that
don't
do
a
particular
action
and
similar
techniques
might
actually
apply
here
that
you
automatically
exclude
some
set
of
users,
and
then
you
sort
of
create
this
opportunity
to
not
do
this,
and
users
can
opt
out
without
being
targeted.
As
of
doubters,
and
so
I
think
that
that
kind
of
thing
would
be
important.
I
don't
have
the
specific
language
for
the
charter,
or
maybe
something
about
you
know
enabling
opt-in
and
opt-out
or
realistic
opt-in
and
opt-out
possibilities.
I
I
The
the
poison
is
making
are
absolutely
correct,
like
you
know,
there's
like
a
whole
pile
of
material
that
has
to
be
done
about
ensuring
that
the
system
the
the
system
is
in,
does
not
allow
for
abuse
by
by
the
collector
or
by
or
by
the
various
aggregators,
and
that's
like
a
that's
like
a
big
topic
that
that,
like
absolutely
have
to
make
sure
we
hit
properly
there's
some
work
already
in
the
document
about
it,
but
like
it's,
it's
a
grease
insufficient.
So
I
think
I
would.
I
H
Just
to
just
a
yary's
point,
a
lot
of
the
systems
that
are
being
described
here
allow
people
to
to
opt
out
of
the
system
without
apparently
opting
out.
They
can
generate
the
inputs
to
the
system
that
appear,
to
all
intents
and
purposes
as
valid
inputs
to
the
system,
but
they're
not
actually
contributing
any
any
values
to
the
system
and
that's
actually
a
useful
property
that
is
exploited
in
various
ways.
So
I
think
that
that
capability
exists,
I'm
not
sure
that
we
need
to
put
those
sorts
of
things
into
a
charter.
H
I
do
think
that
we
need
to
include
some
of
the.
I
think
we
need
to
include
something
generic
about
abuse.
There
is
something
very
specific
about
abuse
in
here
about
proofs
of
validity,
but
I've
spent
quite
a
lot
of
time.
Thinking
about
things
like
civil
attacks,
on
various
manifestations
of
these
sorts
of
protocols
and-
and
those
would
be
interesting
things
to
to
have,
at
least
in
the
auspices
of
the
group.
K
M
K
And
steven
stand
up
our
own
helpers
and
leaders
and
expect
everything
to
accept
data
that
we
submit
to
it,
because
the
two
of
us
agree
that
we
trust
each
other.
But
but
the
collectors
you
know
are
the
collectors
only
going
to
accept
data
from
certain
places,
and
so
we
end
up
with
either
a
very
small,
very
centralized
ecosystem
or-
or
you
know,
a
much
more
flexible
framework.
H
S
S
D
I
Answer
this
question
I
think,
there's
a
bunch
of
like
compatible
models.
I
think
the
important
thing
to
remember
is
that
the
person
who
I'm
going
back
to
the
right
model
I
had
I
I
put
up
earlier
right
that
the
the
collector
has
to
trust
the
to
help
the
aggregators,
but
the
clients
also
trust
the
aggregators,
because
the
click
because
aggregators
responsible
for
the
client's
privacy
and
so
as
a
practical
matter
I
would
expect.
I
would
anticipate
that
you
know
that
that
that
you
know
there'll
be
a
a
a
non
like.
I
You
know,
a
non-gigantic
number
of
collector
of
different
aggregators,
because
fundamentally
you're
you're
trusting
them
to
behave
correctly
right.
So
you
just
you
know
what
you
and
steven
probably
has
worked
up,
except
for,
like
maybe
you
know,
maybe
I'd
be
willing
to
death,
but
because
people
probably
rely
on
relying
on
the
rotations
of
the
collectors.
Oh
sorry,
the
aggregators,
but
I
mean
this
is
compatible
with
like
an
arbitrarily
large
number.
I
So
I
put
this
to
those
two
models,
but
there's
nothing
really
centralized
here.
There's
nothing!
There's
like
no
there's!
No
reason
you
know
like
every
you
know
like
there
are
plenty
of
like
trustworthy
entities
in
the
world
and
and
so
there's
nothing
that
basically
requires
like
that.
Like
a
good,
the
collector,
a
work
with
it
same
set
of
aggregators,
clutter,
b,.
B
P
Bump
what
eckerd
just
said,
but
also
also
kind
of
point
out
that
something
that
we've
been
working
on
in
the
protocol
is
lowering
the
bar
of
entry
to
running
this
this
system
as
much
as
we
can
so
there's
a
sort
of
asymmetry
between
leader
and
helper.
So
the
leader
is,
you
know,
and
and
this
could
change
depending
on
what
p,
what
people's
needs
turn
out
to
be.
P
How
much
you
have
to
store
and
how
long
you
have
to
store.
It
depends
on
the
vda
if
you're
running
so,
but
I
think
this
is
one
of
our
goals,
and
I
think
this
is
important
thing
to
keep
in
mind,
regardless
of
like
how
the
ecosystem
pans
out.
We
want
it
to
make.
We
want
to
make
the
bar
bar
to
entry
as
low
as
possible.
I
think.
B
All
right,
thank
you
very
much
so
at
this
point,
we'd
like
to
go
ahead
and
move
on
to
some
questions
to
inform
the
the
area
directors
in
the
isg
about
community
interest.
For
this,
I'm
going
to
go
ahead
and
bring
up
polls
for
each
of
these,
but
also
at
certain
points,
we're
going
to
ask
that
if
you
are
not
in
support
of
forming
the
working
group
that
you
go
ahead
and
put
yourself
in
the
queue.
C
B
D
D
B
O
R
And
again,
a
personal
statement
not
on
behalf
of
isoc.
I
think
the
discussion
in
the
chat
about
even
the
name
of
the
group
and
most
of
the
proposed
names
did
not
have
the
word
value
in
values
raises
questions
about
the
the
scope
and
intent
of
the
group
that
are
fundamental
enough.
That
question
two
is
very
difficult
to
answer.
As
nick
said.
I
I
I
like
got
punchy
and
like
trying
to
figure
out
how
to
say
priv
so
like
like
really
like,
there's
like
like,
like
that's,
what's
going
on
here
so
like.
If
you
want
to
like
that
likes,
it's
just
something
else
because,
like
like
the
the
like
every
every
letter
in
there
was
like
constructed
to
make
it
say,
priv,
not
for
any
actual
reason
other
than
that.
B
Okay
and
then
the
final
two
questions
that
we're
going
to
have.
Typically,
we
would
have
raised
of
hands
of
this
in
a
physical
room
because
we
do
like
to
keep
track
of
who
exactly
answered
yes,
but
to
do
that
effectively
here
we're
going
to
ask
people
in
the
chat
if
you
are
willing
to
review
documents
associated
with
this
working
group.
Please
respond
with
review.
B
B
D
Hi
everyone
I'd
like
hi,
everyone,
I'd
like
to
repeat,
I
think
we've
had
a
successful
buff
and
I
want
to
thank
everyone
for
kind
of
all
their
input.
I
think
the
chat
was
just
as
lively
as
as
the
mic
line.
If
not
more,
I
mean
generally.
What
I
heard
was
there
is
a
critical
mass
of
interest
in
working
in
this
kind
of
particular
problem.
There
was
repeated
the
tension
I
heard
around.
This
idea
of
you
know
are
we
okay
working
on
ad
tech
and
the
ad
use
cases?
D
You
know
at
the
same
time
recognizing
that
that
is
the
reality
on
the
ground
and
that
there
would
be
wider
applicability
of
this
tech
much
larger
of
these
protocols,
much
larger
than
that
particular
kind
of
use
case.
I
think
in
the
charter
discussion
we're
not
ready
to
go
with
this
particular
charter.
D
There
were
a
number
of
really
kind
of
helpful
suggestions
about
how
to
polish
that
and
we'll
double
check
and
kind
of
confirm
that
so
specific
things
I
heard
reading
off
my
list
is
things
like,
let's
generalize
the
charter
text
to
make
sure
we're
sufficiently
flexible,
so
we
can
swap
approaches.
So
it's
not
just
pre-owned
the
heavy
hitters.
We
need
to
make
sure
that
how
we
define
aggregation
doesn't
restrict,
you
know
other
kind
of
alternatives.
D
We
talked
about
the
need
for
work,
items
to
document
the
abuse
cases
and
talk
about
the
threat
models
against
all
the
portions
of
the
architecture,
and
then
we
had
quite
a
lot
of
conversations
that
we
need
to
tune
the
working
group
name.
I
don't
know
what
the
solution
is,
but
that
we
would
like
a
change
there,
and
so
I
think
the
proponents
seem
open
to
to
making
kind
of
those
changes
and
that
will
those
changes
will
be
made
and
then
we'll
bring
that
back
for
our
confirmation.