►
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Building Prometheus Metrics Support in OpenTelemetry - Alolita Sharma, Amazon
Speakers: Alolita Sharma
OpenTelemetry is a popular open source, cloud-native observability framework with more than 900 active contributors from over 200 organizations. OpenTelemetry components and libraries can be used to instrument, generate, collect, and export telemetry data to monitor and understand application and infrastructure behavior. OpenTelemetry contributors are building a stable metrics pipeline currently. This means the OpenTelemetry data model and components including the Collector and language libraries are being enhanced to provide full interoperability with Prometheus and features to support both delta and cumulative metrics. Join in for this talk to learn about design and implementation to ensure a complete metrics pipeline with full Prometheus interoperability. This talk will discuss the Prometheus remote-write push pipeline and enhancements in the Prometheus receiver, remote-write exporter, core Collector and OpenTelemetry Protocol (OTLP) to ensure full Prometheus metrics compatibility.
A
A
The
best
modern
solutions
today
in
observability
live
in
open
source
and
open
telemetry,
as
well
as
prometheus,
are
great
examples
of
such
amazing
and
powerful,
open
source
projects,
tracing
metrics
and
logging,
as
you
know,
are
the
core
data
signals
in
observability
and
they
give
the
opportunity
for
the
user
to
be
able
to
use
metrics.
You
know
for
aggregation
tracing
for
each
request,
which
is
transaction
based
as
well
as
logging
and
events
that
are
collected
for
telemetry.
A
So
observability
in
kubernetes
is
an
area
that
we
is
near
and
dear
to
all
of
our
hearts,
because
we
have
been
working
in
that
space
and
it's
something
that
you
know
really
is
important
for
kubernetes
developers
to
understand
and
use
telemetry
data
to
understand
in.
In
turn,
what
the
behavior
of
their
code
is
during
runtime
and
having
more
observability
data,
whether
that's
metrics
or
traces
or
logs,
really
helps
in
the
effectiveness
of
kubernetes
monitoring.
A
Analyzed
and
visualized
down
the
pipeline
to
better
understand
what
your
applications
and
your
infrastructure
is
functioning
as
is
it
healthy?
Is
it
does
it
need
provisioning
or
is
there
something
that
can
be
done
automatically
so
open?
Telemetry
obviously
provides
the
ability
to
collect
that
data
and
and
process
it,
and
then
send
that
over
to
a
very
important
you
know
and
to
to
and
very
important.
A
A
Agent,
open
telemetry
is
committed
to
having
full
support
for
prometheus
and
we
needed
to
support
prometheus
protocol
compatibility.
That
is
at
the
given
that
open
telemetry
actually
has
a
full
data
protocol
to
recognize
the
prometheus
exposition
format
to
interoperate
with
it
and
similarly
have
the
components
that
are
able
to
support
service
discovery
in
a
scalable
way.
A
So
the
collector
specifically,
which
is
used
in
conjunction
with
the
apis
and
sdks
for
collection
or
by
itself
dynamically,
and
this
this
particular
component,
should
be
able
to
support
scalable
service
discovery,
data
scraping
receiving
and
exporting
prometheus
data
and
being
able
to
use
push
and
pull
mechanisms.
Exporters
to
be
able
to
send
an
ex
that
data
to
a
any
kind
of
a
service
backend
or
the
prometheus
server
server
itself.
And
then
you
have
the
from
open,
telemetry,
apis
and
sdks.
A
We
should
be
able
to
ingest
and
export
prometheus
data
interoperably
and
then,
of
course,
using
otlp
on
the
internally,
but
being
able
to
completely
support
and
ingest,
as
well
as
export
prometheus
data
formats
right.
So
you
can
see
on,
for
example,
applications
that
are
running
in
the
kubernetes
space
or
kubernetes
itself,
with
its
orchestration
and
different
containers,
as
well
as
services
which
are
which
could
be
hosted.
A
In
this
case
the
example
is
of
managed
prometheus
from
aws
and
then
being
able
to
even
visualize
it
with
an
service
such
as
managed
grafana,
which
is
a
visualization
platform
commonly
used
in
in
the
kubernetes
space.
So
you
can
see
that
you
know.
A
whole
pipeline
requires
that
end-to-end
meteors
support,
especially
when
users
are
using
metrics
for
observability.
A
A
The
operator
also
we
needed
to
improve
and
have
modified
to
be
able
to
now
handle
a
diversity
of
deployments
types
such
as
deployment
even
set
and
stateful
set
support
to
be
able
to
then
trigger
any
collector.
You
know
using
the
open
telemetry
operator
and
with
the
crd
and
and
be
able
to
run
multiple
instances,
similarly,
modifying
and
enhancing
the
exporter
that
exists
for
the
specifically
the
prometheus,
remote
right
exporter
to
be
able
to
handle.
A
The
other
part,
which
was
which
has
been
really
cool
in
working
towards
adding
interoperability
support
in
open
telemetry
for
prometheus,
has
been
working
with
the
prometheus
community
and
closely
and
facilitating
enhancements
and
prometheus,
such
as
adding
https
support
that,
however,
that
rolled
out
in
2.29
of
prometheus
version,
as
well
as
defining
remote
right
compliance
tests,
compatibility
tests
and
helping
also
define
some
of
the
receiver
compliance
tests.
So
again,
a
lot
of
collaboration
here
and
a
lot
of
you
know
really
cool
improvements
on
the
open,
telemetry
side
to
reinforce
full
support
for
prometheus.
A
So
some
of
the
wins
that
have
been
very,
very
you
know,
exciting
on
the
open,
telemetry
side
have
been
successful
coordination
working
with
talented
engineers
across
you
know
both
projects
open
telemetry,
as
well
as
prometheus,
to
discuss
and
solve
some
technical
challenges.
You
know,
sometimes
a
single
engineer
may
not
know
all
the
moving
parts
of
these
complex,
open
source
projects,
and
so
learning
we
have
had
over.
A
You
know,
working
on
this
interoperability
requirement
over
the
last
few
months
has
been
that
every
org
has
limited
resources
right
and
and
combining
these
resources
really
leads
to
the
creating
a
larger
whole,
which
is
essential
to
collaborating
successfully
and
and
bringing
in
and
creating
results,
and-
and
we
have
been-
you
know
very
successful
in
in
being
able
to
build
out
the
support
in
a
very
clear,
clear
way
on
both
the
protocol.
The
data
specification,
as
well
as
the
implementation
on
the
open
telemetry
side
to
ensure
prometheus,
is
fully
supported.
A
Similarly,
some
of
the
challenges
have
been
the
you
know.
There
was
a
bunch
of
code
that
was
inherited
from
open
census
in
the
open,
telemetry
collector
and
as
we
looked
at
it
and
we
worked
through
it,
it
really
required
some
rewrite
and
really
iterating
through
that
slowed
us
down
a
fair
bit,
because
you
know
there
were
a
lot
of
assumptions
that
were
made
in
open
census,
which
necessarily
you
know,
have
don't
apply
anymore
or
have
changed,
and
and
the
assumptions
in
the
open,
telemetry
project
are
different.
A
They're
extended,
they're,
more
scalable,
you
know
so
taking
those
requirements
into
consideration
has
been
very
important.
Similarly,
one
of
the
other
challenges
that
we
have
you
know
ran
into
and
have
been
working
through
has
been
that
there's
never
enough
resources
right
even
on
open
source
projects.
You
know
we
are
completely
contributing
contributor,
driven
work
and-
and
there
is
no
not
enough
resources
in
any
particular
contributor,
so
we
had
to
effectively
figure
out.
You
know
how
to
get
so
many
different
players
to
collaborate
productively.
A
The
process
that
we
have
followed.
You
know
in
in
adding
this
interoperability
support,
which
is
something
that
you
know
at
an
open
source
project
scale
as
large
as
open
telemetry
is,
is
something
that
really
works
is
has
has
been
having
effective
work
groups
and
and
being
able
to
have
a
weekly
discussion.
A
Open
telemetry
has
some
excellent
sigs,
where
you
know
maintainers
and
contributors
come
and
discuss
different
areas,
they're
working
in
or
different
questions
they
have
and
what
are
some
of
the
architectural
design
decisions
that
are
taken.
So
the
work
group
that
we
created
for
weekly
prometheus
specific
discussions
has
been
really
very
useful
and
very
instrumental
in
making
taking
decisions
that
you
know
engineer
experts
coming
in
from
different
companies
different
projects.
You
know
discussing
these
issues
together
and
figuring
out.
A
A
The
code
itself
for
the
collector,
as
well
as
apis
and
sdks,
and
as
well
as
the
technical
documentation,
including
providing
more
clarity
on
use
cases,
which
is
always
you
know,
a
very
tough
area
for
projects
to
collect,
but,
on
the
other
hand,
open
telemetry,
you
know,
has
really
had
the
benefit
of
having
a
huge
number
of
different
users.
You
know
both
developers
and
end
users
coming
and
discussing
the
use
cases,
which
has
been
very
helpful
for
us
in
identifying
what
are
the
specific
areas
we
need
to
enhance.
A
Technically,
you
know
in
terms
of
implementation
first
and
prioritize
that
and
and
the
other
aspect,
which
has
been
very
useful,
has
been
regular
communication
as
you,
as
you
know,
on
open
source
projects.
Communication
is
key
and
working
on
such
large
components
which
have
an
industry
impact,
requires
regular
communication,
so
open
telemetry,
collector
and
api
sdk
groups
which
meet
in
other
sig
meetings.
A
We
have
updated
them
regularly
from
the
prometheus
web
group,
making
sure
that
everybody
is
on
the
same
page
as
their
decisions
taken
technically,
as
well
as
from
a
design
standpoint
or
a
data
protocol
standpoint,
so
that
all
changes
are
not
duplicative
or
repetitive
right.
So
again,
communication
is
key
there.
A
The
commitment,
also
from
the
open,
telemetry,
collector,
approvers
and
maintainers,
to
ensure
that
code,
reviews
and
design
reviews
are
done
regularly
and
you
know
keeping
prometheus
support,
as
has
first
class.
Interoperability
on
the
project
has
been
very
instrumental
in
making
sure
that
we've
made
good
progress
so
far.
That
said,
work
is
still
in
progress
and
we
are
tracking
the
backlog
at
the
prometheus
work
group
on
open
telemetry.
A
You
can
go
check
it
out
anytime,
to
see
you
know
what
the
project
group
is
working
on
and
you
can
track
the
issues
right
here
on
the
work
group.
There
also
are,
you
know
we
track
and
pre-order
the
backlog
regularly.
So
again,
there's
some
some
element
of
project
management
here,
which
is
very
essential
in
making
sure
that
across
you
know,
multiple
components.
A
A
Support
has
been
very
key
in
being
able
to
drive
and
and
build
out
metric
support
for
open
source
observability
on
open
telemetry
and
the
status
of
the
project
in
terms
of
moving
tracing
stability.
Metric
stability
and
log
stability
is
work
in
progress.
We
maintain
that
very
regularly
on
open
telemetry.
I
o
status
from
a
metric
standpoint.
Again.
A
The
open
telemetry
protocol
is
now
stable
as
of
april
2021.
The
data
model
is
also
now
stable
with
the
as
of
july.
2021
and
work
on
the
prometheus
support
is
still
work
in
progress
with
some
more
enhancements
that
need
to
be
made
in
the
collector,
so
that
work
is
in
progress
as
well
as
the
apis
and
sdks.
The
metrics
support
you
know
being
built
out
as
well
as
full
interoperability
with
prometheus.
Both
for
push
and
pull
mechanisms
is
working
progress,
so
that's
supposed
to
be
landing
in
in
november.
A
There's
also
a
lot
of
areas
that
we
need
more
contributors
on
so
again,
if
you're
super
interested
in
open
source
observability
or
in-
or
you
understand
you
know
from
as
an
end
user
or
a
developer,
who's
worked
on
prometheus
or
on
open
telemetry
before
please
come
join
us
help.
Us
you
know
build
some
of
these
components
as
well
as
code
review,
super
valuable
to
have
expertise
being
shared
across
all
of
us
to
really
build
out
a
stable
and
steady
support
for
not
only
prometheus
but
metrics
in
overall.
A
So
thanks
for
listening-
and
I
really
appreciate
the
time
from
all
of
you,
for
you
know
this
update
on
what
progress
we've
made
on
in
prometheus
support,
the
future
is
bright.
We
are
actually
looking
forward
to
having
full
support
and
continuing
to
maintain
that
for
the
two
projects
to
work
closely
together
and
also
building
out
end-to-end
protocol
support,
so
that
any
large
scale,
implementations
of
telemetry
and
ingestion,
as
well
as
export
of
petabytes
of
data
coming
in
for
observability,
is
fully
operable
and
usable
on
the
open,
telemetry
site.