►
Description
Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Jaeger: Present and Future - Pavol Loffay, Red Hat & Jonah Kowall, Logz.io
In this session, we will start with an introduction to the Jaeger distributed tracing project and the basics of distributed tracing. For the deeper dive, we will discuss the future of Jaeger and its relation to OpenTelemetry. We will cover how OpenTelemetry and Jaeger work together to unlock new use cases for operational monitoring using the new monitoring tab, which adds metrics capabilities to Jaeger UI. We will also take a closer look at the Jaeger Kubernetes operator and discuss deployment strategies. Jaeger is the most popular open-source distributed tracing backend. We are always seeking new collaborators, contributors, and users. We need your help! Whether your goal is to get acquainted with Jaeger and distributed tracing or to keep abreast with the latest and greatest, please join us!
A
All
right,
I
guess
I'm
gonna
get
started
since
the
room
is
full
here
and
hopefully
those
of
you
that
are
online
and
couldn't
get
in
or
online
and
can
at
least
listen
or
catch
the
replay
yeah.
I
think
there
was
about
700
people
that
registered
for
the
event
and
when
I
saw
the
room
I
said
this
is
going
to
be
a
lot
of
unhappy
people,
but
it's
fine
yeah.
So
thanks
everyone
for
joining
and
coming
in
person,
they
gave
me
lots
of
room,
but
you
guys
are
pretty
crammed
in
out
there.
So.
A
A
Let's
put
them
together
service,
one
of
the
tools
that
we
offer
is
jaeger,
along
with
prometheus
and
open
search,
and
so
I
spend
a
lot
of
time
working.
But
when
I'm
not
working,
I
try
to
be
underwater
some
photos.
I've
taken
recently
spend
a
lot
of
time
diving.
I
live
near
miami
florida
and
yeah
a
lot
of
work.
I
have
povel's
bio,
that's
why
I'm
pulling
it
up
and
so
pavel.
A
Pavel
contributes
to
several
cm
cncf
projects,
including
open
telemetry
and
jager
and
pavel,
spends
a
lot
of
time
in
the
mountains,
which
is
why
you
see
these
awesome
pictures
he's
always
sending
me
photos
of
him,
skiing
and
climbing,
and
making
his
dog
do
uncomfortable
things
in
the
mountains.
So
it's
always
enjoyable
pavel
is
great
to
work
with
he's.
Definitely
part
of
the
team
working
on
jaeger
and
great
to
work
with
him
there
and
on
open
telemetry.
A
So
you'll
hear
from
pavel
a
bit
and
I'll
also
talk
a
little
bit
through
the
presentation
today
we're
going
to
cover
a
few
different
things.
I'm
going
to
talk
about
some
of
the
basics,
I'm
not
going
to
go
into
a
whole
tutorial
about
distributed
tracing
because
you're
going
to
get
that
in
many
of
the
open,
telemetry
talks
going
on
this
week.
A
So
I'm
going
to
talk
about
that
and
then
pavel
will
discuss
the
operator.
He
spends
a
lot
of
time
and
obviously
it's
super
important
for
those
running
on
kubernetes
or
openshift
to
use
the
operator
and
so
he's
going
to
talk
about
that
and
then
I'll
touch
on
the
roadmap
and
where
jaeger's
going
for
those
of
you
that
are
using
it
and
want
to
see
kind
of
what
the
vision
is
and
then
we'll
do
some
q
a
so
without
further
ado.
I'm
going
to
jump
in
here.
A
So
a
few
semantics
one
jaeger
was
created
and
it
was
a
project
created
by
uber
and
then
donated
to
the
cncf.
It's
one
of
the
few
graduated
projects
of
the
cncf.
So
it's
considered
mature,
it's
been
around
for
a
while.
It
was
actually
built
around
open
tracing,
which
is
the
predecessor
to
open
telemetry.
A
So
some
of
the
things
in
the
ui
and
some
of
the
semantics
might
use
slightly
different
terminology.
But
I'm
going
to
talk
in
open
telemetry
words,
because
that's
what
you're
going
to
instrument
your
applications
with
and
it's
by
and
large,
very
similar
to
open
tracing.
So
the
trace
is
basically
think
about
an
end-to-end
transaction.
That's
going
through
a
bunch
of
different
microservices
and
that's
what
the
trace
is.
Each
of
those
microservices
and
the
pieces.
They're
executing
is
a
span.
A
So
that's
what
the
tags
are
used
for,
and
people
stuff
all
kinds
of
things
into
tags.
You
can
even
like
stick
a
whole
log
message
into
a
tag.
It's
not
always
the
best
idea,
but
I've
seen
some
crazy
stuff
and
people
also
will
create
a
lot
of
different
types
of
span.
Data
related
to
a
trace,
because
it's
your
code,
you
can
essentially
instrument
and
build
out
that
visibility
in
any
way
that
you
want
so
open.
A
So
like
another
way
to
look
at
this
visually
and
I'll
show
you
how
it
looks
in
jager
is
basically
we
call
this.
The
root
span.
That's
the
first
part
of
the
transaction
up
at
the
top
and
then
essentially
can
cascade
through
asynchronously
or
synchronously,
and
as
these
things
execute,
you
basically
build.
A
A
That's
really
what
tracing
is
useful
for
and
to
see
where
errors,
latency
or
other
problems
are
within
the
trace
itself,
so
in
the
jaeger
ui
here
we're
basically
just
visualizing
over
time.
This
is
a
relatively
short
transaction,
700
milliseconds,
and
then
you
can
see
where
the
time
is
spent,
but
a
good
example
of
you
know.
Potentially
some
issues
is
when
you
see
things
that
are
stacked
up
like
this.
A
A
C
B
D
F
B
D
D
About
a
bit
more
about
open
telemetry,
I'm
pretty
sure
there
is
a
different,
open,
telemetry
presentation
at
this
conference.
But
just
for
the
context
of
this
talk,
I
want
to
introduce
you
to
open
telemetry
and,
as
I
mentioned,
open
telemetry
is
all
about
the
data
collection
and
data
collection
is
very
complex
and
difficult.
E
B
B
D
E
D
D
B
D
D
E
D
D
One
that
allows
you
to
you
know,
process
the
data.
What
is
super
cool
is
that
you
know
a
single
stream
of
data.
You
can
process
it
and
then
export
to,
for
example,
to
multiple
systems.
So
you
can
keep
some
data
in
or
export
it
to
jager
that
you
have
deployed
locally
in
your
in
your
environment
but
as
well
export
the
same
data
to
to.
D
However,
those
clients
are
deprecated
right
now
and
we
encourage
users
to
use
open,
telemetry,
sdks
and
apis
and
to
better
kind
of
support
migration
to
open
telemetry.
We
have
done
bunch
of
work
on
the
open,
telemetry,
sdks,
and
so
the
hotel
sdk
supports
the
jager
context,
propagation
header
or
format.
B
D
D
D
Then
there
is
jaeger
exporter
that
can
export
data
to
jaeger.
There
is
as
well
jaegerino
sampler
extension,
so
essentially
you
can
just
grab
your
sampling
configuration
from
eager
collector
and
use
it
in
open,
telemetry,
collector
and
last
but
not
least,
open
telemetry
collector
integrates
with
kafka
and
you
can
configure
it
to
use
jaeger
as
a
payload
type.
D
B
A
Cool
thanks,
pablo,
hopefully
he's
listening,
but
he
has
one
other
section.
You
can
hold
the
applause
until
he's
actually
done,
but
yeah
I
mean
pavel's
done
a
lot
of
work
and
I'll
also
talk
about
where
things
are
going
between
jaeger
and
open
telemetry,
because
there's
definitely
overlap
the
sampling
implementation
for
those
that
are
really
into
tracing.
A
A
We
renamed
it
a
little
while
ago,
but
the
idea
is:
how
do
we
integrate
these
two
things
together?
Instead
of
building
metrics
in
jager,
we
just
said
everyone's
using
prometheus:
let's
make
this
part
of
the
way
that
people
already
work
and
then
once
it's
in
prometheus.
Not
only
can
you
visualize
it
in
jaeger,
but
you
can
use
whatever
front
end.
You're
you're
used
to
using
probably
grafana
for
the
ui.
A
So
the
thing
that
we
tried
to
solve
is
how
do
we
generate
the
metrics
from
the
traces?
This
is
where
otel
comes
in,
so
what
we
built
is
a
processor
when
pavo
was
talking
about
the
pipeline.
Your
trace
data
comes
in
from
the
application,
and
then
we
built
a
processor
which
is
called
the
span.
Metrics,
processor
and
I'll
I'll
have
a
link
to
that
and
then
the
idea
is
you
export
the
metrics
from
open
telemetry
into
any
format
that
you
want.
A
Even
if
it's
not
a
prometheus
backend,
you
can
export
it,
but
in
jager
we
only
support
prometheus
based
back
ends
for
the
query
side,
and
so
this
means
in
our
case
at
my
company.
We
use
m3
db,
but
if
you're,
using
thanos
or
cortex
or
victoria
metrics
or
whatever
kind
of
prometheus
back
end
as
long
as
it
supports
prom
ql
in
remote,
right,
it'll,
work
seamlessly
and
then
the
idea
is
we
derive
the
metrics
from
the
traces
automatically
and
then
we're
actually
able
to
store
them
and
visualize
them.
A
So
I'm
going
to
show
you
how
this
works
in
configuration.
This
is
a
open,
telemetry,
collector
sample,
config
and
basically,
in
here,
you're
we're
talking
about
that
span.
Metrics
processor,
the
up
at
the
top.
You
can
actually
define
what
kind
of
buckets
you
want,
and
so,
when
you're
looking
at
a
histogram
depending
on
how
you
want
to
segment
the
data
and
how
you
want
to
store
it,
that's
totally
configurable
based
on
what
makes
sense
for
you.
A
This
is
kind
of
the
default
buckets
for
histograms,
that
we've
set
up
and
then
down
towards
the
bottom
is
the
pipeline,
where
you
would
listen
for
jaeger
traces.
In
this
case
you
would
process
it
and
create
the
metrics
and
then
in
the
case
of
and
then
you
would
basically
define
the
exporter
now
down
here
on
the
bottom.
We
can
send
it
to
prometheus.
You
can
send
it
to
whatever
back
end.
You
want-
and
this
essentially
will
create
that
and
there's
a
link.
A
But
this
is
part
of
the
contrib
collector
distribution,
if
you're
using
open
telemetry,
and
so
you
can
visualize
it
a
couple
of
different
ways.
In
this
case
as
an
example,
we
visualized
it
inside
of
grafana
just
to
show
you
how
you
can
use
the
histograms
and
then
inside
jaeger
we
created
a
new
service
called
the
metric
query
service
and
that
will
query
any
prom.
Ql
back
end
that
that
understands
the
prom
ql
query
language
and
today
it
supports
you,
know
whatever
kind
of
back
end.
A
A
You
know
the
top
services
and
then
there's
a
few
out
of
the
box
kind
of
columns
that
we're
looking
at
here
that
we've
defined
it's
relatively
basic,
but
at
least
it
starts
to
get
people
thinking
about
how
you
can
operationalize
trace
data,
which
not
a
lot
of
open
source
tools
can
do
so.
This
makes
it
a
bit
more
useful
and
we
hear
great
things
from
users.
A
A
lot
of
people
have
actually
taken
that
open,
telemetry
piece
that
we
built
and
they're
using
it
for
all
kinds
of
other
use
cases,
because
there's
a
lot
of
useful
data
when
you
create
metrics
from
traces
cool
and
I'm
gonna
pass
it
back
to
pavel
to
talk
about
the
operator.
He's
done
a
ton
of
work
on
this
over
the
last
few
years
and
it's
it's
definitely
one
of
the
main
ways
that
people
are
deploying
jager
and
deploying
other
parts
around
the
jaeger
ecosystem.
D
B
D
Which
brings
us
to
the
cr
and
I'm
pretty
sure
that
almost
everybody
here
is
familiar
with
the
term
custom
resource
definition,
and
on
this
slide
we
see
the
custom
resource
definition
for
the
jaeger
and
it
is
essentially
a
yaml
file
where
you
can
kind
of
describe
how
the
jaeger
deployment
or
jager
cluster
should
look
like,
and
we
will
take
a
look
at.
You
know
parts
of
this
crd
to
explain
the
operator
features
and
the
first
part
that
we
will
explain
is
the
strategy
and
there
are
three
strategies
right
now.
D
B
D
E
D
D
Take
a
look,
how
we
configure
the
storage
in
the
cr,
the
storage.
You
know
you
can
configure
it
in
under
in
the
crd,
under
spec,
dot,
storage,
node,
and
there
are
two
most
important
kind
of
configuration
fields.
First,
one
is
type
where
you
can
define
which
storage
type
you
want
to
use.
There
is
in
memory
elasticsearch,
cassandra,
pager,
even
grpc,
plugin
and
kafka,
and
the
second
option
is
the
options
that
is
loosely
coupled,
and
here
we
put
the
the
storage
related
flux
for
the
given
storage
that
we
want
to
use
and.
D
Do
we
find
out
which
flags
the
jaeger
supports
right?
There
are
kind
of
two
ways
that
I
can
think
of.
You
can
either
go
to
our
documentation
on
egotracing.io
or
you
can
run
the
jaeger
collector
or
the
query
docker
commands
or
the
binaries,
with
the
spam
storage
type
set
for
your
storage
of
choice
and
use
the
health
flag,
and
it
will
print
you
all
the
supportive
flags
so.
D
E
D
C
D
A
A
This
is
something
that
that
we
came
out
with
recently
in
open
telemetry,
which
pavel
touched
on
the
service
performance
monitoring,
which
I
touched
on.
That's
also
a
new
feature,
that's
key,
and
then
we've
actually
decided
as
a
project
to
really
move
towards
open
telemetry
we've
gone
through
deprecation
of
the
jaeger
clients
and
the
sdk,
and
that
leads
me
to
where
we're
going
from
there,
which
obviously
has
a
lot
to
do
with
with
open
telemetry
in
the
second
part
here,
but
we're
we're
doing
some
work
on
the
dependency
graph.
A
So
those
of
you
that
are
using
jager,
there's
kind
of
three
different
dependency
graphs
in
jager,
they're,
all
kind
of
mediocre-
or
let's
say
not
that
useful,
so
we're
actually
going
to
be
normalizing
these
together
and
really
creating
a
nice
service.
Topology
view
that's
interactive
has
metrics
overlaid
on
it,
so
we're
going
to
really.
A
So
that'll
allow
you
to
write
directly
to
open
telemetry
from
any
anything
that
supports
otlp
and
we
can
take
it
into
jager
natively.
So
it's
definitely
more
coming
and
I
would
offer
questions,
but
I
don't
know
if
I
have
time
for
it
and
I'm
going
to
put
up
a
few
links.
Do
I
have
time
to
take
a
couple
questions?
I
think
I
do
so.
Does
anyone
have
a
question
and
I
will
bring
a
mic
over
to
you.
B
A
A
A
A
It
is
in
the
data-
it's
just
not
in
the
ui,
so
there's
no
plan
to
make
the
ui
work
with
that.
Okay,
if
you're
using
something
like
open,
search
or
elasticsearch,
you
could
do
something
in
cabana
or
open
search
dashboards
to
visualize
it,
and
I
know
the
open
search
team,
which
is
open
source
they're.
Building
some
trace
analytics
capabilities
that
we're
going
to
be
that
are
going
to
be
compatible
with
jager,
where
it
might
actually
solve
that.
A
So
I
would
check
out
open
search.
They
have
a
trace
analytics
plugin
that
might
do
a
good
job
and
you
can
use
it
together
with
jaeger
soon,
not
quite
yet.
Right
now
there
are
two
formats
in
the
storage,
but
they
are
going
to
support
jaeger
format.
It's
part
of
the
plan
there
so
yeah
sure,
thanks,
guess
one
more
until
they
kick
me
out.
I'm
gonna
keep
taking
questions.