►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Keynote: (Open)Telemetry Makes Observability Simple - Sarah Novotny, Open Source Wonk, Azure OCTO, Microsoft & Liz Fong-Jones, Principal Developer Advocate, Honeycomb.io
Observability is a fundamental requirement for sustainably developing and operating cloud native applications. It must be accessible to a diversity of users and support a robust ecosystem of tooling around these common needs. Liz and Sarah explain how OpenTracing and OpenCensus merging benefits the entire cloud native ecosystem. With OpenTelemetry, users and vendors alike can focus on distilling insights out of their data rather than duplicating instrumentation work. Sarah and Liz will show the progress so far, integrations with peer CNCF projects, and how you can participate!
https://sched.co/UdJS
A
All
right,
hello,
San
Diego,
so
we
are
hoping
to
tell
you
today
how
you
can
make
all
of
this
complexity
of
the
cloud
native
system
easier
to
develop
and
easier
to
run.
So
our
cloud
native
systems
are
really
really
complex
animals
and
we're
adding
more
complexity
all
the
time,
because
we're
solving
the
business
challenges
of
scaling
up
and
scaling
out.
A
But
there
are
some
challenges
that
we're
introducing
that
are
new
problems
that
we
need
to
solve,
because
we
have
micro
services
now,
where
we
are
sending
Micra
service
transactions
across
dozens
of
different
services
that
are
required
to
complete
end-user
requests.
Our
systems
are
now
in
the
words
of
Ben
settlement
they're
deep
systems.
Now
we
no
longer
can
hold
them
all
in
one
person's
head.
We
also
have
to
think
about
these
interactions
with
our
cloud
and
serverless
providers,
because
your
faults
are
no
longer
localized.
Only
in
your
system,
they
may
be
localized
in
someone
else's
system.
A
So
how
do
we
actually
debug
these
things?
And
on
top
of
that,
we
live
in
a
world
where
every
single
user
experience
matters
where
it
no
longer
is
fair
to
say,
ok,
99%
of
the
ads
have
served
rate
I,
don't
care
about
the
1%.
We
care
about
every
end-users
experience,
so
we
need
to
measure
and
debug
service
level
objectives,
but
it's
really
hard
to
see.
A
Clearly
when
we've
built
all
of
this
complexity,
we
need
new
tooling
and
we
need
the
ability
to
observe,
to
look
inside
our
systems
and
ask
them
new
questions,
so
we
can
understand,
what's
happening
under
the
hood
and
how
we
can
repair
things
to
keep
our
users
happy.
So
all
of
these
things
are
things
that
we've
been
busy
developing
and
it
may
sound
obvious
right
that
you
need
to
see
into
your
systems.
But
if
we're
obvious,
why
haven't
we
done
it
already?
We.
B
B
Observability
is
the
ability
to
ask
questions
of
your
system
and
learn
from
it,
and
instrumentation
is
the
tooling
that
provides
your
telemetry
metrics
logs
and
traces
are
the
components
of
the
telemetry
data
that
you
receive,
but
they've
often
been
stored
in
and
visualized
in
different
systems.
If
we
had
trouble
agreeing
on
the
definitions
of
two
words,
oli,
observability
and
instrumentation,
imagine
the
trouble
we
had
defining
what
an
event
is.
B
This
has
led
to
lots
and
lots
of
confusion
among
the
solutions
lock-in
with
particular
choices
and
a
fear
of
needing
to
reinstall
with
your
entire
system
in
order
to
make
a
change,
and
so
many
many
developers
decided
to
roll
their
own
tooling.
Oh,
this
is
a
dark
story.
Sorry
take
a
breath,
okay,
so
let's
refocus
on
the
users,
because
that
is
a
super
easy
way
to
start
this
conversation,
you
are
all
our
users,
you
develop
systems,
you
need
to
introspect
about
them.
B
B
No,
it's
pretty
uncontroversial
yeah.
We
thought
so,
but
it
was
hard
to
do
so.
You
know
it's
an
important
start.
So,
since
the
spring,
with
this
newfound
refocus
on
users,
we
brought
together
two
projects
that
had
been
contributing
to
some
of
the
confusion.
There
were
two
projects
here
in
the
CNC,
F,
open
census
and
open
tracing,
and
these
two
projects
are
now
open,
telemetry,
the
two
of
them
had
different
specifications,
differing
focuses
and
even
implementations
of
how
they
thought
about
observability
in
telemetry
and
so
we're
bringing
them
together
with
consistent
vision.
B
This
means
clarity
around
a
common
language
and
telemetry
outputs
and
a
one-time
effort
to
add
instrumentation,
regardless
of
the
visualization
providers,
which
is
always
great
fun.
These
providers
now
also
win
because
you
can
choose
having
a
clear
area
to
differentiate
and
delight
users.
This
this
seems
like
a
win-win
for
everybody,
so
Liz
there
are
so
many
accomplishments.
What
are
you
gonna
share
so.
A
We
also
focused
on
making
sure
that
there
are
migration
paths
from
the
past
solutions
make
sure
that
to
get
started
playing
around
with
this,
you
don't
need
to
necessarily
write
a
lot
of
new
code,
and
this
plays
very
nicely
with
your
existing
network
topologies.
So
you
can
collect
and
then
have
proxy
and
forward
on
all
of
your
traces
and
metrics
to
where
they
need
to
go.
A
We
also
have
been
focusing
on
making
sure
that
all
of
the
languages
are
on
the
same
page,
that
all
of
them
feel
like
a
native
library
in
your
language,
but
all
have
the
same
commonality
of
the
same
idioms
of
open
telemetry.
So
we
support
seven
languages
today,
in
addition
to
having
the
common
language
neutral
collector.
So.
B
A
So
what
I
have
here
is
I
have
an
application
that
computes
the
Fibonacci
series
using
a
micro
service
or
a
set
of
micro
services.
So
a
second
Fibonacci
number
is
to
the
third
Fibonacci
numbers.
Three.
Anyone
want
to
guess
what
the
fourth
one
is:
I've,
five
very
good.
So
this
computes
Fibonacci
series
numbers,
but
it
gets
a
little
slow
when
you
start
acting
for
the
sixteenth
Fibonacci
series,
number
I
wonder
why!
Well
maybe
it's
because
I
unnecessarily
introduced
some
complexity,
let's
try
to
measure
and
understand.
A
What's
going
on
so
I'm
going
to
add
in
the
OT
HTTP
plug-in,
this
enables
us
to
go
ahead
and
measure
the
performance
of
an
individual
RPC
or
web
Handler,
and
it's
just
a
simple
drop-in
wrapper.
So
I
drop
this
in
on
top
of
my
Fibonacci
Handler,
give
it
a
descriptive
name
and
I'm
off
to
the
races.
So
now
this
does
mean
absolutely
no
good
if
I
can't
actually
output
this
somewhere.
A
Fortunately,
we
have
Yaeger
yaer,
which
recently
graduated
is
now
the
project
that
I'm
going
to
be
demonstrating,
how
we
can
actually
add
this
instrumentation
with
so,
let's
first
of
all,
recompile
to
make
sure
they
didn't
break
anything
and
can
I
still
compute
the
Fibonacci
series,
no
I
cannot.
The
reason
is
I
suspect
that
I
failed
to
import
the
right
things.
Let's
have
a
look
at
my
console.
This
is
the
danger
of
doing
live
demos
on
stage.
So
what
am
I
missing
here?
I
am
missing
an
additional
parenthesis
right
there.
A
There
we
go
much
better.
So
let
us
wait
for
that
to
load
up
and
then
meanwhile,
let's
talk
about
what
we're
going
to
do
next,
so
what
we
need
to
do
next
is
we
need
to
hook
up
the
open,
telemetry
SDK
and
we
need
to
connect
to
it
the
Yeager
back-end.
Yes,
this
indeed
works
great.
So,
let's
connect
in
be
a
go
back
end,
so
I
didn't
want
to
type
this
all
out
by
hand.
A
A
Flashcards
so
we
need
a
new
provider,
so
let's
go
ahead
and
instantiate
the
provider
and
we
need
to
give
it
a
configuration
which
tells
it
to
sample
everything,
because
I
don't
want
to
have
to
hit
this
endpoint
10,000
times
before.
I
show
you
any
results.
I'd
rather
show
you
this
after
one.
So
let's
go
ahead
and
do
that
real
fast,
let's
give
it
a
sampler
that
has
a
value
of
always
sample.
A
A
We
have
a
new
trace
provider
and
now
we
need
to
pass
this
into
the
global
trace
provider
so
that
the
open,
telemetry,
HTTP
instrumentation
knows
where
to
find
it.
Let's
stop
there
and
see
whether
this
works,
if
it
doesn't
I'll
get
a
lovely
compiler
message
and
I'll
know
that
I
need
to
do
a
little
bit
more
work.
A
Okay,
so
we
were
having
an
error
because
I'm
not
yet
using
the
ignorant
exporter.
Well,
that's
easy
enough
to
fix.
So
let's
go
ahead
and
add
the
jäger
exporter,
so
I
need
to
configure
it,
but
fortunately
I've
already
done
the
hard
work
to
configure
the
secrets
of
where
to
send
the
data.
So
all
I
need
is
to
pull
up
my
cheat
notes
and
go
ahead
and
paste
in
the
data
about
how
to
access
the
opened
the
open,
telemetry,
jäger
backend.
So
we're
going
to
add
in
the
instantiation
of
the
Jaeger
exporter.
A
Well,
I'm
doing
pretty
good
in
time
great,
so
we're
going
to
go
ahead
and
pass
this
in
here
and
it
also
needs
to
know
a
service
name
which
I
happen
to
know
is
the
host
name,
which
is
Otel
cubic
on
demo.
So
now
that
I
have
this
working,
let's
go
ahead
and
indent
that
correctly.
So
now
we
just
need
to
tell
this
thing
hey
by
the
way
you
are
supposed
to.
Also
in
addition
to
taking
the
config,
you
need
to
also
take
in
a
exporter
and
send
every
span
you
collect
over
da
ker.
A
Okay,
great
so,
let's
see
whether
that
works.
Okay,
that
come
keep
to
know
you.
Let's
have
a
look
over
in
the
eager
UI
and,
let's
see
what's
recent
I
find
ooh.
This
is
not
actually
a
collection
of
traces.
This
is
a
collection
of
individual
trace
fans,
I'm
missing
an
ingredient.
What
do
you
think
I
might
be
missing?
Sarah
I'm.
A
What
I'm
missing
is
contacts
propagation
I
need
to
pass
along
the
trace
identifier,
saying
hey
by
the
way
that
initial
requests
I
started
from
the
plan
line
here
that
actually
pertains
to
all
of
the
downstream
requests.
I
was
gonna
say
that
so
let's
go
ahead
and
add
some
instrumentation
for
that.
So
what
we
need
to
do
is
we
need
to
do
two
things.
A
We
need
to
go
ahead
and
add
in
the
tracer
and
we
need
to
go
ahead
and
tell
it
to
trace
all
of
this
code
so
down
here,
where
I'm
calculating
this
in
a
really
silly
way.
I
currently
have
a
function
that
takes
an
integer
and
runs
a
and
and
runs
the
client
calls.
So,
instead
of
doing
this
in
this
way,
what
I
hate
to
do
instead
is
I
need
to
do
it
like
this.
A
So
right
now,
I'm
calling
this
with
the
offset.
So
let's
go
ahead
and
instead
of
doing
this,
let's
go
ahead
and
wrap
it
inside
of
something
else.
So
what
we
need
to
do
is
slightly
different
than
the
code
that
I
originally,
so,
let's
go
ahead
and
add
in
the
HTTP
client
instrumentation.
So,
in
order
to
add
the
HTTP
client,
instrumentation
I
also
need
HTTP
tracer,
which
I
have
here.
Let's
go
ahead
and
look
at
my
flash
cards
and
see
what
I
was
I
was
supposed
to
do
with.
A
Yes,
that
is
exactly
what
I
was
supposed
to
do
with
this.
So
inside
of
here
what
I
have
is
I'm
issuing
an
outbound
hgp
request,
so
I
need
to
pass
it
all
of
my
trace
headers,
and
this
means
that
I
need
to
be
aware,
even
in
here
of
what
my
request
context
is
so
I
need
to
find
our
class
context
and
then
what
I
need
to
do
is
I
need
to
go
down
here.
I
need
to
go
ahead
and
say:
I
am
going
to
you.
I
am
going
to
pass
in
this.
A
Right,
something
like
that
should
do
the
trick
and
it
just
needs
to
be
indented
correctly,
which
is
easy
enough
to
fix.
Okay
and
the
other
beef
that
I
was
looking
for
was
actually
in
here.
There
we
go
now.
This
is
looking
much
more
familiar
great,
so
now
I'm
passing
in
the
trace,
headers
and
making
sure
that
they're
propagated
along
nicely.
A
So
the
final
thing
that
I
need
to
do
here
is
I
need
to
make
sure
that
I'm
wrapping
this
that
I'm
wrapping
this
client
call
so,
instead
of
instead
of
just
doing
the
client
call
by
itself
now
what
I'm
going
to
do
instead
is
I'm
going
to
do
the
tracer
that
with
span
and
I'm,
going
to
pass
in
the
fib
client,
so
gonna
keep
that
this
is
a
client
in
her
client
span,
I'm,
just
passing
the
bear
function.
This
does
leave
one
missing
thing,
which
is
that
I
also
need
a
trace
provider.
A
If
I
don't
have
a
trace
provider,
all
of
my
span
names
would
collide.
So,
let's
fix
that
really
quickly
and
do
this
up
here
so
I
have
a
tracer
and
I'm
just
going
to
fetch
the
tracer
from
named
after
my
own
domain
name:
okay.
So
hopefully
this
will
compile,
if
not
we'll
have
a
little
bit
of
debugging
to
do.
I
did
not
use
API
trace.
I
know
why
I
did
not
get
used.
Api
trace
I
did
not
get
a
news
API
trace,
because
I
also
wanted
to
do
one
more
thing.
A
A
A
A
So
now
I
got
a
result:
I
can
go
look
over
here
and
I
can
see
some
glorious
straight
spins,
but
wouldn't
it
be
nice
if
I
could
send
not
just
to
one
provider
but
instead
four
different
providers?
That's
what
I'm
about
to
show
you
I
am
about
to
show
you
exporting
not
just
to
Jaeger,
but
also
exporting
to
multiple
different
ecosystem
partners.
They
are
all
working
together
on
open
telemetry.
A
A
Then
what
I
will
get
in
a
moment
as
soon
as
I
copy/paste
this
out
of
my
sneaky
little
buffer?
What
I'm
going
to
do
is
I'm
just
going
to
grab
all
this
code
here,
I'm
just
going
to
paste
right
into
my
function.
This
will
never
possibly
go
wrong
said
this
annoying
loss
of
of
everyone,
who's
ever
read
written
code.
So
let's
go
ahead
and
add
some
more
exporters
and
let's
go
ahead
and
also
add
in
the
additional
calls
to
use
those
exporters.
A
You
could
say
visualize
this
so
I
think
that
should
work,
so,
let's
restart
it
and
see
whether
this
works.
This
might
come
back
with
an
error.
Oh
no,
it's
fetch!
It's
fetching
all
of
these
things.
That's
fine!
Let's
go
ahead
and
disable
my
read-only
mod
and
see
whether
this
works
and
after
that
we'll
go
ahead
and
tell
you
about.
What's
ahead
for
open
telemetry,
so
I
go
back
over
here,
we
are
fetching.
Oh
I
see
that
is
my
bad
I
accidentally
typed
honeycomb
by
yo.
A
A
A
Main
thing
that
we
need
to
work
on
is
stability.
We
have
a
lot
of
stability
work
to
make
sure
that
people
can
safely
put
open
telemetry
into
production.
We
have
seven
different
languages
to
synchronize,
as
well
as
the
collector,
and
we
need
to
make
sure
that
everything
is
rock-solid
going
into
q1,
so
hopefully,
by
next
cube,
Khan
will
have
a
beta
or
stable
release
to
announce.
We
also
have
been
working
on
integrating,
not
just
with
Jaeger,
but
also
with
services
like
Prometheus
and
Zipkin,
and.
B
That
we
have
to
say
on
behalf
of
the
Governance
Committee,
where
both
Liz
and
I
said
we
want
to
thank
both
the
open,
telemetry
and
there's
both
the
open
Tracy
and
the
open
census
contributors
for
their
laser
focus
and
identification
of
user
needs
in
this,
bringing
the
project's
together
as
well
as
being
egoless
and
willing
to
collaborate
on
open,
telemetry
sunsetting.
These
two
projects
that
had
competing
standards.