►
From YouTube: 2021-10-21 meeting
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
A
B
Hi
everyone-
I
am,
I
think
we
might
wait
another
minute
and
I
would
be
glad
to
start
our
meeting.
I
usually
run
this
meeting
and
I
have
we
have
an
agenda.
B
Someone
put
one
in
there
and
I
like
how
this
meeting
has
had
a
bunch
of
conversation
about
probability,
sampling
and
if
we
have
conversations
about
other
forms
of
sampling,
that's
great
or
other
questions
about
sampling,
that's
great,
and
then
I
have
one
hoping
I
was
going
to
see
atmar
here
today,
because
it's
really
a
mouthful
and
math
question,
and
otherwise
I
would
like
to
pass
this
meeting
time
to
whoever
it
is
that
wrote
this
bullet
ability
to
see
full
failure,
problematic
choices,
even
when
stamping
is
enabled.
B
In
the
current
state
of
my
computer
right
now,
hi
and
here
put
that
item
on
the.
C
Agenda
hello:
this
is
mate
from
microsoft.
I
wrote
the
agenda
point
so
maybe,
let's
start
I
have
a
quick
kind
of
presentation
or
just
a
couple
of
slides
to
go
through.
So
you
can
see.
B
Someone
else
is
projecting
great.
Can
you
see
the
screen.
C
Yeah,
yes,
thank
you,
okay,
all
right,
so
I
wanted
to
talk
about
about
the
problems
you
are
facing
in
microsoft
and
the
so
basically
in
microsoft,
we
have
a
really
large
distributed
systems
which
will
require
kind
of
some
form
of
the
sampling
with
the
current
status,
the
head
based
sampling,
as
per
part
of
the
specification
it
works,
but
for
problems
when
which
appear
during
execution
of
the
request
in
this
kind
of
large
distributed
system.
C
Even
when
there
is
the
sampling
defined
and
it
doesn't
matter
if
it's
a
head-based
like
standard
head-based,
sampling
or
probabilistic
sampling
yeah.
C
So
without
the
ability
to
see
the
full
failure,
or
at
least
the
origin
of
the
failure,
we
are
not
able
to
kind
of
timely
fix
these
issues,
because
we
will
the
probability
that
we
will
have
the
trace,
which
is
sampled
out,
and
we
will
not
see
the
root
cause
is
high
right,
so
just
because
of
the
load
of
the
distributed
system
overall
load
and
the
kind
of
low
sampling
which
will
have
to
be
required
for
this
kind
of
large-scale
services.
C
Another
problem
is
that
the
current
sampling
specification
is
biased.
Is
this
from
what
I've
read?
It
seems
to
be
biased
towards
the
head-based
sampling
and
it
seems
the
reason
is
at
least
what
I
see.
The
reason
is
that
it
prefers
the
performance
or
computer
resource
optimization
over
these
kind
of
observer
observability
needs.
C
So
maybe
let's
go
forward
to
how
we
can
imagine
the
assembling
specification
to
be
a
bit
different
so
that
we
can
still
have
the
head-based
sampling
according
to
specs.
These
are
being
written,
but
also
with
the
ability
to
have
the
additional
traces
emitted.
C
So
what
we
see
is
that
each
series
in
the
system
should
be
able
to
actually
decide
at
the
end
of
processing
or
end
of
procedure,
logic
within
the
scope
of
each
span,
whether
that
span
or
the
logic
was
problematic
or
not.
What
I
mean
by
problematic
is
that
did
any
exception.
Exception
occurred
within
that
span
or
did
actually
the
execution
took
too
long,
and
it
took
more
than
what
is
the
prescribed
sla
or
for
dependency
calls
the
same.
C
The
dependency
call
took
longer
than
what
is
the
prescribed
sla
for
latency,
so
for
this
kind
of
issues
in
the
system,
we
would
like
to
see
what
was
happening
where,
where
was
the
bottleneck
so
with
head-based
sampling,
the
really
the
probability
that
we
would
be
able
to
know
or
find
out
that
this
problem
is
low?
Now
each
service
again
should
be
able
to
emit
the
spans
collected
within
the
scope
of
the
service.
C
Basically,
on
top
of
what
the
head-based
sampling
instructions
are.
So
what
do
we
see
here
that
when
instructions
came
that
you
need
to
sample
out?
I
think
there
should
be
still
a
way
to
pro
basically
emit
additional
traces,
which
would
be
not
counted
into
the
statistics.
C
Of
all
the
samples
collected
yeah
because
we
don't
want
to
have-
we
don't
want
to
skew
the
statistics
which
will
be
computed
based
on
kind
of
a
probabilistic
and.
B
We
have
associated
with
the
sample
spans
in
this
current
draft
spec
and
I'm
enjoying
this
presentation.
I
I
have
some
thoughts
and
mostly
agree
with
you.
B
I
just
want
to
interject
that
there
is
a
concept
in
the
current
proposal
for
probability
sampling
that
is
meant
to
enable
what
you're
talking
about,
although
it
doesn't
answer
every
question,
so
I
want
to
keep
talking
about
this,
but
this
idea
of
a
zero
adjusted
count
has
been
baked
into
the
probability
spec
sampling
spec,
so
that
if
you
have
any
reason
to
record
a
span
other
than
the
probability
decision,
you
can
give
it
a
zero
count
and
still
record
it
and
not
skew
the
counts.
B
C
Right
so
yeah
just
account
that
that
would
work.
However,
at
the
moment,
I
think
that
the
current
implementation
of
sampler
is
that
they
will
sample
out
or
decide
decide
basically
at
the
beginning.
So
there
is
no
way
to
say
that
you
should
emit
the
trace
for
the
failure.
B
A
B
Much
further,
I
want
to
add,
there's
just
a
bunch
of
baggage
in
this
spec
that
I
want
to
mention,
because
it's
related
so
there's
this
concept
in
the
sampling
sampler
api,
that
you
can
return
three
values
for
your
decision.
One
is
don't
sample,
one
is
record
only
and
one
is
sample
and
record,
and
I
I'm
not
the
terminology,
I
don't
love
the
terminology,
but
but
sample
and
record
means
do
sample
this
and
record
only
means
just
sit
around
in
memory
and
hold
on
to
this
span.
B
You
might
want
to
sample
it
and
the
spec
is
fairly
incomplete.
It
doesn't
say
how
to
do
that,
especially
not
probabilistically
speaking,
but
now
we
have
that
coming
in
the
zero
adjusted
count
is
meant
to
let
you
record
something
that
you
didn't
sample.
According
to
the
sampling
logic,
probability
sampling
logic,
please
continue.
Okay,
I
see.
C
Right
so
yeah,
basically
what
you
said
that
this
kind
of
adjusted
count
is
what
you're,
what
you're
mentioning
exactly?
What
I'm
mentioning
in
this
card
in
this
slide,
that
the
extra
spawns
should
not
be
counted.
C
That
one
thing
is
that
it
would
be
great
if
the
service
should
be
able
to
inform
the
caller
that
the
request
request
was
marked
as
problematic,
and
there
seems
to
be
also
some
preliminary
proposal
for
how
to
propagate
the
information
from
callie
to
back
to
the
caller
that
sampling
occurred
and
the
traces
were
emitted.
Are
you
referring.
B
To
w3c
efforts
on
trace
response,
headers;
yes,
yes,
okay!
As
far
as
I
know,
nothing,
no,
that
conversation
has
not
entered
open
telemetry
at
this
point
in
time.
It's
certainly
relevant
to
what
you're
saying
I
just
want
to
say
that
we
haven't
talked
about
it
at
all.
As
far
as
I
know,.
C
Okay,
right
and
basically
also
the
logic
which
is
executed
on
the
caller
side,
should
be
also
based
on
the
these
kind
of
hints
coming
from
the
dependencies
so
that,
when
the
dependency
decided
to
sample
in
an
extra
extra
traces
extra
spans,
then
the
caller
could
also
emit
additional
trace
or
partial
trace,
and
that
would
actually
work
also
within
the
execution
of
you
know,
of
the
single
service
workers
in
the
single
service.
C
So,
basically,
the
information
collected
in
the
child
scope
should
be
provided
back
to
the
parent
scope
so
that
you
can
emit
that
that's
fun
as
well.
I
have
a
quick
and
yeah
quite
clear
diagram.
What
I
meant
what
I
just
described,
so
I
can
just
zoom
in
here
so
yeah.
C
C
Now,
in
this
diagram,
you
can
see
that
the
service
first
there
is
one
called
dependency
service,
which
is
the
second
column
which
start
processing
the
request.
Then
some
business
logic
is
executed,
cause
dependency
and
the
sub
dependency
again
execute
some
logic
and
evaluate
at
the
end
of
the
processing
whether
the
processing
was
healthy.
C
If
not,
it
would
emit
the
trace
and
return
the
indication
of
the
unhealthy
processing.
If,
yes,
it
would
return
back
indications
of
healthy
processing
same
would
occur
on
the
dependency
service
that
it
would
take
into
account
this
hint
from
the
dependency
calls
and
would
decide
to
emit
the
trace
or
not.
So
this
is
kind
of
different
from
the
tail
base
tracing
where
you
collect
all
the
spawns
from
all
the
all
the
calls
and
decide
at
the
very
end.
So
this
would
be
kind
of
a
creation
of
the
full
trace
from
the
deepest
scope.
C
Now
the
generalized
form
could
exist
also,
if,
if
you
look
at
marking
the
requests
problematic
as
just
assigning
the
category
as
a
problematic
to
the
given
span-
and
there
could
be
other
categories
defined
for
particular
sponsor,
let's
say
for
incoming
requests-
we
could
potentially
identify
the
the
partner
service
as
a
category
and
additionally,
we
don't
have
to
have
like
100
percent
sampling
for
those
extra
spans.
There
could
be
possibility
in
ideal
world
to
define
sampling
per
this
category,
which
could
be
static
or
yeah.
C
That
would
mean
that
you
would
be
able
to
say
you
want
to
sample
all
the
problematic
spawns
and,
let's
say,
10
percent
of
spawns
coming
from
the
partner
service
a
or
you.
If
you
are
investigating
issues
figure
out
on
your
api,
then
you
could
also
specify
that
you
want
all
traces
to
be
admitted
for
that.
C
Now,
a
quick
pros
and
cons.
I
think
you
you
are
already
familiar
with
that.
So
the
pros
are
that
any
kind
of
a
head-based
sampling
can
be
used.
The
ability
to
see
if
we
promote,
problematic
or
categorized
traces
would
be
retained
and
actually
the
sampling
capabilities
would
be
much
much
more
flexible.
C
Of
course,
it
has
a
potential
performance
impact
due
to
the
need
of
a
collector,
collecting
extra
response
and
not
skipping
the
sampling
logic,
and
I
also
also
there
are
some
related
issues
or
somewhat
related
issues
to
to
this
and
oteps
as
well,
where
people
were
requesting
adding
additional
kind
of
hooks
for
sampling,
which
is
similar
similar
to
what
I
just
described.
B
Thank
you.
I
I'd
like
to
first
respond
to
the
one
of
the
higher
level
statements
which
I
agree
with,
but
it's
it's
a
little
bit
murky.
The
the
claim
that
open,
telemetry
sampling
has
been
prioritizing
performance
over
observability
from
the
from
the
start.
Here,
I
think,
is
true
in
the
extent,
but
it's
not
widely
a
100
agreed
to,
and
that's
why
you
see
these
issues
are
still
filed.
B
After
all,
these
attributes
are
present,
provided
because
it's
very
expensive
to
do,
and,
of
course,
if
you're
trying
to
sample
based
on
those
attributes,
you
just
need
those
attributes
and
it
costs
you
performance
and
I
think
the
the
the
type
of
arrangement
that
you've
described
here,
where
you
hold
your
spans
until
the
end
when
you
can
see
if
there
were
unusual
exceptions
and
so
on,
or
if
your
children
caused
had
unusual
exceptions
and
so
on
that
just
costs
you
the
cost
of
tracing,
which
is
we
keep
it
low.
B
But
it's
never
zero,
and
so
I
think
what
what
we
have
just
a
is
like
a
very
complex
situation
where
some
people
want
a
performance
and
some
people
want
observability
and
it's
hard
to
there's
just
tension
there,
and
so,
as
far
as
the
mechanisms
you're
asking
for
most
of
it
is
kind
of
in
place.
We
have
this
ability
to
record
but
not
sample.
B
We
have
the
ability
to
mark
zero,
adjusted
count
coming
in
our
probability,
sampling
spec.
We
don't
have
this
ability
to
respond
from
a
client
back
to
a
server
or
from
a
from
a
colleague
back
to
a
caller
with
some
state
that
might
help
in
this
situation.
B
That's
the
stuff
I
mentioned
from
the
w3c
about
trace
response
headers,
and
I
would
I
would
say
it's
a
welcome
idea,
but
I
think
the
hotel
group
was
waiting,
for
you
know
its
own
house
to
get
like
1.0
for
tracing
and
so
on,
get
the
like
the
forward
propagation
to
work
first
before
anyone
wanted
to
talk
about
it,
but
I
know
the
w3c
group
moved
forward
on
that,
because
it's
useful.
For
this
reason,
I
think
we
should
talk
about
it
when
people
are
ready
in
hotel.
So
I
think
maybe
now
it's
time.
B
The
biggest
the
biggest
picture
I
can
pay
for
you
in
response
to
this
is
that
we
have
a
lot
of
technology
and
we
know
how
to
sample.
We
know
what
the
mechanism
should
be
if
you
want
the
super
fast
head,
sampling,
you're,
going
to
disable
the
span
most
of
the
time.
If
you
want
this
type
of
sampling
that
you've
described
you're,
going
to
pull
the
span
and
not
record
it
most
of
the
time.
If
you
want
to
sample
based
on
http
attributes,
you're
going
to
evaluate
the
http
attributes
before
you
decide
to
sample.
B
If
you
don't,
you
might
not
evaluate
the
sample.
The
attributes
before
you
decide
to
sample,
and
so
there's
just
all
this
complexity
around
many
different
ways
are
viable
and
desirable,
and
unfortunately
coding
for
them
all
just
makes
a
sophisticated
problem,
and
I'm
not
saying
we
won't
do
that.
But
it's
when
we
get
to
this
room
with
a
few
people
who
know
what
we're
talking
about
and
look
at
all
that
complexity
and
talk
about
how
long
is
it
going
to
be
before
we
have
this
solved?
B
I
can
just
predict
it's
going
to
be
a
while,
but
I
fully
support
this.
Would
anyone
else
like
to
comment.
D
I
I
just
want
to
comment
on
the
sort
of
performance
versus
expressiveness
trade-off,
so
I
think,
like
in
the
jaeger
sdk,
we
were
able
to
struck
a
good
balance
there,
that
if
you
use
a
standard
head
based
like
probabilistic
sampling,
then
the
path
was
fairly
cheap
and
at
the
same
time
it
did
not
prevent
this
sort
of
more
detailed
sampling.
When
you
want
to
take
into
account
some
other
attributes.
D
But
if
you
were
in
that
situation,
then
you
had
to
pay
the
cost
of
sort
of
like
always
collecting
stuff
in
memory,
at
least
until
the
the
old
attributes
are
available,
and
you
can
make
a
sampling
decision
so
and
from
the
sort
of
like
the
api
point
of
view.
D
But
then
the
sampler
implementation
could
easily
optimize
it
for
the
apk,
although
the
most
performant
case
and
then
be
less
even
more
expensive
when
it's
not
when
it's
more
complex,
like
sampling
criteria,.
C
C
Now
is
there
a
way
to
yeah
decide
on
a
solution
which
would
be
very
configurable
in
this
case,
so
yeah.
B
Configuration
is
basically
the
biggest
problem
that
we
have
across
open,
telemetry
right
now,
and
it's
especially
acute
in
sampling.
I've
been
saying
this
in
in
the
same
meeting
here
for
for
months
now
is
that
the
metrics
group
went
and
tackled
this
question
that
we
call
view
configuration
it
says,
given
a
bunch
of
instrumentation
you
may
you
may
want
to
suppress
some
of
those
metrics.
You
may
want
to
change
the
aggregation
of
those
metrics.
You
might
want
to
just
do
something
different
for
each
one
of
those
metrics
and
then
in
sampling
and
spans.
B
We
just
have
this
like
idea
that
you
can
batch
them
and
send
them,
but
there's
no
open
telemetry
machinery
that
we've
specced
out
and
implemented
across
the
sdks.
That
says
well
I'd
like
to
select
this
span,
but
not
that
span
I'd
like
to
sample
this
spin
at
10,
but
that's
been
100
like
that
type
of
just
configuration
is
one
question
that
we
haven't
really
focused
on
in
spans.
Whereas-
and
the
only
reason
it
happened
in
metrics
is
that
it
came
out
of
open
census.
B
That
way,
just
like
it
was
a
spec
saying
there
would
be
a
view
for
metrics,
and
so
we
did-
and
I
think
now
is
the
time
for
open
telemetry
to
build
views
into
span
the
span
machinery
and
then
you
could
and
then
there's
some
way
to
configure
this.
This,
like
wide
array
range
of
behaviors,
that
we've
talked
about
in
the
past
10
minutes
and
and
it
becomes
a
question
of
like
here's,
some
yaml
that
comes
in
and
you
read
what
you
want
and
then
you
configure
a
sample.
B
That's
going
to
be
as
efficient
as
possible
to
do
what
you
asked
it
to
do,
and
and
probably
we
need
to
see
prototypes
and
configuration
languages
and
and
and
all
that
setup
that
happens.
D
D
Josh,
I
I
think
the
specifically
this
presentation
is
seems
to
be
well
yeah.
There
is
a
configuration
aspect
right
on
the
decision
making
of
what
you
consider
sort
of
interesting
span,
but
more
fundamentally,
it
requires
back
propagation,
potentially
a
different
mode
of
so
like
one
example,
and
I'm
actually
curious
if
anyone
on
the
call
or
in
the
industry
kind
of
had
experience
with
this
back
propagation
solution
being
valuable
in
practice
right
because
we've
talked
about
it,
a
lot.
D
There's
always
this
use
case
that
if
you
call
two
dependencies
and
the
first
dependency
returned
normally,
even
though
it
may
have
returned
garbage
data
but
like
there
is
no
latency,
there
is
no
error
code,
then
your
sampling
will
essentially
say
no.
I
I
wasn't
gonna
sample
that
so
because
it
was
happy
path,
it
wasn't
interesting
and
then
your
second
dependency
actually
fails
because
of
the
first
one,
and
then
you
sample
only
that
path
in
the
tree,
because
you
never
got
the
original
data.
B
I
noticed
mate
said
in
one
of
his
phrases
was
at
least
the
exceptional
caller
so
like.
I
think
it
was
acknowledged
that
you,
you
may
get
the
the
trace
from
the
error
down,
but
not
from
the
error
up
not
to
the
caller
the
root
essentially.
C
I
think
initially,
what
the
biggest
value
which
would
be
from
from
this
additional
emission
would
be
to
do
it
in
case
of
exceptions,
at
least
as
a
starter,
and
have
a
way
for
samplers
to
configure
that
exception.
When
exception
occurs,
it
should
go
and
be
sampled.
In
that
way,
you
don't
need
to
have
like
full
trace,
but
at
least
you
could
be
able
to
see
in
what
service
when
the
request
was
propagated
through
the
system.
In
what
service
there
was
some
exception
or
yeah.
B
I
guess
I
want
to
point
out
that
I
think
you
can
do
something
like
that
in
open
telemetry
today,
but
it's
just
not
an
out
of
the
box
configuration
so
you'd
have
to
configure
a
sampler
to
do
the
record
decision
and
then
you'd
have
to
configure
the
exporter
to
check
if
there
was
an
exception
and
then
record
it
depending
on
whether
it
was
sampled
or
not.
Originally,
you
may
adjust
the
count
to
zero
and
so
on,
but
that
just
sounds
crazy
complicated
for
some.
B
You
know
you,
you
came
with
a
simple
request,
and
now
we
told
you
to
do
three
things
and
it's
not
very
easy.
So
in
some
sense
I
think
what
you're
saying
is:
it's
just
not
an
easy
configuration
to
achieve,
and
then,
moreover,
if
you
have
an
exceptional
condition
in
the
middle
of
your
span
and
you're
still
starting
children,
do
you
then
begin
tracing
the
children?
That's
not
something.
We
have
a
support
mechanism
for
either.
E
What
I
think
can
make
sense
is
you
have
to
have
the
sampling
rate
yeah
yeah,
to
choose
the
sampling
rate
according
to
attributes
which
are
locally
available.
So
if
there's
an
error
on
the
current
span,
you
would
maybe
record
it,
but
I
I
do
not
believe
that
the
back
propagation
to
to
the
caller
is
works
in
a
large
scale,
because
all
the
the
callers
have
to
buff
all
the
spans
and
and
spain's
can
yeah
can
complete
after
a
long
time.
E
Basically
so
it's,
but
of
course
I
mean
the
information
would
be
valuable,
but
I
don't
know
if
it's
technically
feasible.
B
D
A
So
josh,
I
didn't
really
understand
your
comment.
You
said
you
would
have
started
the
child's
children's
pants.
You
cannot
control
those,
so
you
would
have.
B
Said
so,
what
I'm
saying
is
that
the
open
telemetry
api
there's
an
sdk
spec
for
api
for
the
sampler
and
it
gets
to
make
a
decision
once
in
the
beginning
of
its
lifetime,
and
then
it
set
the
context
for
the
children
to
use,
there's
an
implied
machinery
here
that
has
been
described
where,
during
the
middle
of
a
span
the
child
reaches
out
and
makes
an
rpc
or
something
the
the
children
of
that
call
have
an
exception.
They
propagate
backwards,
somehow
magically.
I'm
just
like.
B
Let's
add
that
later
they
propagate
back,
that
there
was
an
exception
and
now
you're
in
the
middle
of
a
span.
You
its
child,
got
the
exception
and
now
you're
you're
still
making
outbound
calls
and
you've
found
that
there's
an
exception.
Do
you
want
to
start
tracing
your
children?
If
so,
we
have
no
way
to
do
that
in
open,
telemetry.
B
So
I,
if
I
was
to
summarize
what
I'm
what
I've
heard
here
I
I
do
agree
with
the
the
motivation
I
think
opmar
just
presented
the
counter,
the
counterpoint,
which
is
there's
always
somebody
who
wants
the
best
performance
and
there's
always
somebody
who
wants
the
best
observability
and
we
find
ourselves
saying
you
can
configure
either
if
you
want,
but
we
we
can't
right
now,
so
I
do
still
think
that
the
biggest
problem
we're
having
is
about
configuration
of
sampler
behavior,
because
there's
such
a
range
of
that
of
it
and
so
there's
a
configuration
problem.
B
There's
a
back
propagation
question
is
that
something
that
open
telemetry
wants
to
bring
in
from
w3c
and
I'm
open
to
it?
I
just
haven't
studied
it
and
I
haven't
thought
through
it,
because
I'm
thinking
about
that
that
that
child
span
that
that
goes
through
and
calls
the
rpc
and
now
it's
back
propagated.
B
It
gets
back
propagated
to
a
context
not
to
a
span,
there's
still
a
connection
to
a
parent
that
you
have
to
establish
somehow
and
I'm
I'm
thinking
through
how
that
would
work
and
it's
we'd
have
to
grow
our
model
quite
a
bit
for
like
how
you
get
from
that
context
back
to
the
parent
in
in
at
runtime,
which
is
a
few
questions
that
would
have
to
be
asked,
and
then,
if
I
may
just
to
say
the
third
thing
that
brought
this
conversation
brought
out,
that
I
don't
think
we
have
talked
about
enough.
B
Yet
comes
back
to
some
questions
that
I
have
been
pondering
for
the
w3c
and
the
questions
about
trace
parent
mate.
If
you
could
go
to
your
slide,
number
six
generalized
form
of
the
solution.
You
you
list
here,
something
like
it
sounds
like
you
might
want
multiple
sampling
policies,
yes
and
multiple
sampling
policies.
B
If
they're
non-overlapping
can
be
boiled
down
to
one
sampling
policy,
but
when
multiple
sampling
policies
overlap,
we
now
have
a
position
where
you
may
be
sampled,
with
different
probabilities
by
different
sampling
pieces
of
logic,
and
at
that
point
the
proposal
that
I've
written
for
open
telemetry
falls
apart.
It
doesn't.
B
It
doesn't
support
multiple
probabilities,
not
because
I
don't
know
how
to
do
it,
because
it's
just
complexity
that
I
didn't
think
we
were
going
to
try
and
spec
out
right
now,
and
so
it
crosses
my
mind
that
this
is
the
area
where
our
ideas
about
how
we
might
extend
w3c
trace
parent
fall.
B
Apart
like
if
we're
going
to
definitely
support
multiple
probabilities
head
probabilities,
then
I
think
we
will
be
stuck
with
a
trace
state
solution
forever,
because
you're
going
to
have
multiple
probabilities
and
it's
a
variable
length,
header
and
so
on.
Just
a
thought.
I
we
haven't
really
discussed
that
much
multiple
sampling
policies
in
action.
At
the
same
time,.
E
E
B
Yeah,
I
think
your
I
think
I
see
your
point.
It
just
requires
reminding
that,
in
a
consistent
probability,
sampling
regime,
like
two
decisions,
are
the
same.
That's
kind
of
what
you're
that
that's
the
requirement
for
what
you
said.
E
But
basically,
the
the
samples
are,
in
theory
free
to
choose
the
sampling
rate
based
on
local
span
attributes.
So,
and
so
this
flexibility
is
not
used
yet
in
the
current
sample
implementations,
because
the
sampling
rate
is
always
fixed,
but
in
it
all
the
the
estimation
algorithms
would
work
fine
with.
If,
if,
if
you
choose
the
sampling
rate
dependent
on
some
attributes,
yeah.
B
Yeah,
I
think
I
agree,
I'm
not
even
sure
why
I
said
earlier
that
I
I
mean
you
could
imagine
a
simpler
architecture
for
sampling
where
you,
just
like,
I'm
sampling,
spans
labeled,
a
at
10
and
any
span
100,
and
then
I
I
end
up
recording
that
fact
on
the
context
that
I'm
sampling
this
in
two
ways
and
then
I
record
it
to
two
different
places
at
the
end
and
and
what
you're
saying
is
that
you
shouldn't
do
that.
B
B
Microsoft,
people,
you
may
be
wondering
what
is
it
that
causes
open
telemetry
to
not
do
exactly
what
you're
wondering
wanting
from
sampling
and
open
telemetry?
I
I
think
we,
since
many
of
you,
seem
to
be
here
from
microsoft
today.
I'd
love
to
talk
a
little
bit
more
about
what's
holding
us
back.
B
There's
ludmila
is
one
of
your
colleagues
who
is
working
on
http
attribute
sampling
and
she
has
been
running
into
a
wall
and
I'm
not
sure
exactly
what
the
wall
is,
if
it's
just
we're
all
over
overloaded
or
or
whether
people
aren't
really
sure
about
this
cost
versus
the
the
observability
cost
versus
performance
question.
But
it
has
been.
B
There's
been
this
extensive
debate
over
whether
http
attributes
must
be
specified
at
the
beginning
of
a
span
and
if
so,
it's
very
expensive
and
then
ludmila
ended
up
not
getting
her
pr
in
and
there's
still
a
debate
about
whether
it
should
be
possible
to
have
like
whether
whether
htp's
instrumentation
must
provide
those
attributes
and-
and
it's
basically
deadlocked-
and
I
one
proposal
is
to
to
allow
lazy
attributes
in
it.
B
Just
requires
work
on
the
specification
and
I'd
say
that
this
this
the
as
I
was
saying
at
the
beginning,
you
can
have
both
you
can
have
high
performance
and
high
observability.
It
just
becomes
a
configuration
problem
to
get
what
you
want
and
a
complexity
problem
to
make
sure
that
you
can
have
both,
and
I
think
we
can.
We
can
do
that.
It
requires
more
work
on
open,
telemetry.
A
So
we
do
work
quite
closely
with
lord
miller
and
actually
ludmilla
and
dennis
are
working
on
this
thing
together,
so
we
can
sink
back
with
her
as
well.
This
is
a
specific
need.
So
far
we
have
been
aligning
with
open
diameter.
I
think
one
of
the
specs
that
ludmila
had
pushed
maybe
more
than
a
year
ago,
which
we
still
it
keeps
coming
back,
is
how
can
we
have
different
services
like
the
migration
story?
A
How
teams
are
going
to
be
migrating
from
previous
sdks
over
to
open
telemetry
and
if
you
have
different
sampling
or
gardens,
how
they're
going
to
communicate
and
still
get
consistent
sampling?
So
that's
still
another
open
item
that
keeps
coming
back,
but
for.
B
B
So
her
otep
was
number
135
and
it
was
about
consistency
and
we've.
I
believe,
replaced
that
with
a
functional
equivalent
and
and
that
that
work
has
progressed.
I
wrote
those
taps
more
recently,
but
I
I
believe
that
we
have
fully
replaced
one
of
hers
at
this
point
in
time.
So
that
part,
I
think
we've
got
taken
care
of
okay.
A
B
B
At
the
end
that
we
did
functionally
solve
the
problem
originally
presented
in
her
in
her
135,
which
is
about
allowing
consistent
facing
decisions
to
be
made,
we've
we've
proposed
something,
but
I
don't
know
that's
going
to
help
you
with
your
migration
strategies.
So
so
we
should
look
into
it
into
it.
B
I
also
just
want
to
say
I'm
I'm
supportive
of
what
she's
asking
for
it's
just
that
it
requires
convincing
a
large
number
of
people
that
we
should
do
this
across
the
board
and
so
on.
The
the
idea
that
I
feel
is
a
winner
is
that
we
should
make
it
so
that
open
telemetry,
sdks,
support,
lazy
attributes
and
then,
when
you're
in
http
instrumentation
package,
you
will
provide
your
attributes
as
a
lazy
function.
B
A
B
Sounds
details
and
then
I
I
think
someone
if,
if
you're,
really
excited
about
back
propagation,
it's
looking
for
a
champion
to
just
to
take
it
and
make
a
proposal
for
how
we
would
potentially
use
it.
But
I
it's
gonna
when
you
get
into
the
code,
you're
gonna,
I
think
you're
gonna
see
how
tricky
that
is.
So
I'm
looking
forward
to
prototypes
and
so
on
to
see
how
we
might
use
that.
I
think
yuri
expressed
the
same.
B
A
B
Yeah,
I
think
that's
right,
I
think
you're
going
to
want
to
there's
going
to
have
to
be
a
hook
back
to
the
parent
span.
Somehow,
and
I
think
that's
the
question
and
is
it
it's
entirely
in
the
sdk?
Probably
I
don't
know
this
is
the
question
of
what
the
api
looks
like
that
accomplishes
what
you're
looking
for.
C
Thank
you,
yeah
from
our
side.
I
think
more
is
not
the
big
propagation,
but
the
ability
to
sample
in
with
the
with
the
adjusted
count
zero
at
the
basically
based
on
the
decision,
which
is
after
the
execution
right
at
the
end.
B
That
is
technically
possible,
but
it
requires
bridging
a
sampler
with
an
exporter
and
so
on.
So
it's
not
it's
definitely
custom
logic
like
a
heavy
amount
of
custom
logic,
which
is
problematic
from
the
user's
perspective,
but
at
least
for
the
kind
of
early
adopters.
I
think
we
should
be
able
to
show
that
this
can
be
done
and
then
my
goal.
My
hope,
is
that
in
the
long
term,
open
telemetry
will
adopt
a
view
configuration
mechanism
for
spam,
so
that
you
can
say
I
want.
B
B
B
Thank
you.
This
is
exciting.
I
I
I'm
very
supportive
I've.
Always
I
got
in
this
field
to
do
debugging
more
than
to
do
production
monitoring,
just
just
as
a
background
story,
so
any
technology
aimed
at
better
debugging
excites
me,
I
put
in
a
link
to
my
favorite
paper
from
microsoft
over
the
last
12
years.
That's
just
you
know.
The
next
item
I
had
on
the
agenda
was
just
to
ask
ottmar
a
question:
it's
mostly
a
math
question,
since
I
don't
think
of
myself
as
a
mathematician.
B
I
have
been
writing
this
draft
and
it's
just
not
ready
for
wide
review
that
captures
everything
we
put
into
otep
168
and
170
as
requirements
and
so
on,
and
I
was
and
one
of
the
challenges
that
was
given
in
some
of
the
feedback
from
the
the
general
open
telemetry
group
was
gosh.
This
is
complicated
and
hard
to
test.
How
do
we
know
we've
got
it,
got
it
right,
and
so
I've
been
trying
to
figure
out.
B
B
A
test
specification
that
says
you
will
test
your
samplers
to
behave
according
to
the
spec,
and
it
occurs
to
me
that
I
can
test
all
the
sort
of
corner
cases
and
stuff
that
I
know
about
like
if
p
is
not
present
and
so
on,
like
corner
cases
legitimately,
I
can
test,
but
when
it
comes
to
the
actual
performance
of
the
sampling
and
the
adjusted
count
inference
that
happens,
it
comes
down
to.
Did
you
implement
a
proper
algorithm
and
it
comes
down
to?
B
Is
your
random
number
generator
good
and
then
it
comes
down
to
I
don't
know
it's
just
a
random
number
generator
built
into
the
to
the
library.
So
my
question
is:
can
we
come
up
with
like
a
validation?
That's
perhaps
not
very
strict,
but
at
least
more
than
nothing
that
that
says.
You
know
I'm
going
to
run
a
simulation
of
a
million
sampling
decisions
and
I'm
going
to
calculate
my
adjusted
count
and
I'm
going
to
repeat
this
a
thousand
times
and
the
number
of
that
it
should.
B
The
the
accuracy
should
be
within
some
percent
more
than
fewer
than
it
should
fail
fewer
than
so
many
times.
I'm
looking
for
that
type
of
test,
and
I
know
that
math
can
help
us,
but
I
I
I
find
myself
not
the
best
person
to
calculate
these
things.
So
I'm
wondering
atmar
if
you
would
be
interested
in.
E
Actually,
I've
already
written
some
unit
tests
in
this
proof
of
concept,
implementation
which
do
some
statistical
tests
or.
E
B
B
Perfect,
if
you
could
help
me
with
a
link
to
that,
I
will
take
a
look
at
it
and
try
and
write
it,
and
that
was
always.
That
was
all
I
was
asking
here.
B
B
That
was
my
agenda
item
and
anyone
else
would
like
to
speak
about
sampling
today.
B
Thank
you
mate
for
the
presentation.
I'm
always
excited
to
see
more
interest
in
high
performance
observability.
It's
definitely
what
got
me
into
this,
and
I
remember
the
high
performance
stuff
from
google
days.
It
seems
less
important
to
us
here
today
at
our
my
employer
today
is
not
as
excited
about
the
cost
of
traces
as
the
cost
of
network
bandwidth,
but
it
all
matters.
So
I'm
glad
to
see
that
microsoft
is
here
promoting
this
particular
performance
problem.
B
Thank
you
for
the
time
and
for
the
responses
all
right
I'll,
be
here
next
week.
Thank
you
all
and
see
you
again.