►
From YouTube: Lightning Talk: Managing OpenTelemetry Through the OpAMP Protocol - Mike Kelly, observIQ
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Lightning Talk: Managing OpenTelemetry Through the OpAMP Protocol - Mike Kelly, observIQ
Managing thousands of data collection Agents across just as many servers can overwhelm DevOps teams. Open Agent Management Protocol (OpAMP) is a new network protocol from the OpenTelemetry Project that enables remote management of OpenTelemetry collectors, allowing them to report their status to and receive configuration from a Server and to receive agent package updates from the server. This eliminates the need to create new custom distributions and redeploy, drastically simplifying Agent management. OpAMP is also vendor-agnostic, so the Server can remotely monitor and manage a fleet of different Agents that implement OpAMP, including a fleet of mixed agents from different vendors. In this talk, Mike will detail the OpAMP specification and future possibilities being discussed by the community.
A
Welcome
back
from
lunch,
everyone,
I'm
Mike
Kelly,
and
we're
going
to
kick
things
off
talking
about
managing
open
Telemetry
with
op-amp,
and
that's
a
new
open,
Telemetry
effort
around
managing
agents
and
defining
the
protocol
for
managing
those
agents.
So,
first
a
little
bit
about
me.
I'm
Mike,
as
I
mentioned
I'm,
the
CEO
of
observe
IQ
I've,
been
working
on
observability
for
the
past
10
years,
primarily
on
the
the
agents,
the
Integrations
and
the
pipelines.
A
So
everything
about
Telemetry
and
getting
data
in
and
at
Observer
Q
we've
been
entirely
focused
on
open
source
Telemetry
for
the
past
couple
of
years
started
with
developing
an
open
source
log
agent
stanza.
That
was
eventually
contributed
to
open,
Telemetry
and
then
more
recently
released
bind
plane.
Op
I'll
talk
about
in
a
second
which
is
an
open
source,
observability
Pipeline
and
agent
management
system,
and
it
uses
open
Telemetry
at
really
at
the
core.
A
Well,
let
me
jump
in
and
talk
about,
op,
amp
and
I'm
going
to
apologize
to
all
the
electrical
engineers
in
the
room,
because
opamp
is
an
overload
determin.
This
is
not
operational
amplifiers.
This
is
open
agent
management
protocol
and
it
is
a
network
protocol
for
Remote
Management
of
large
fleets
of
data
collection
agents,
and
it's
worth
noting
I
think
that
it
starts
with
large
fleets
like
that
was
a
key
consideration
of
this
from
the
very
beginning.
A
It's
a
protocol
definition
it's
part
of
the
open,
Telemetry
project
and
currently
consists
of
that
protocol
implementation
and
go
and
then
applications
that
are
utilizing
this
and
one
thing
I'll
note
here
is
you
know
just
because
it's
part
of
open
Telemetry,
this
really
isn't
and
I'll
show
an
example
of
it
managing
open,
Telemetry
collectors.
It's
intended
as
a
a
general
solution,
a
generic
solution.
So
the
idea
is
that
this
should
be
used
for
any
agent
proprietary
open
source.
A
A
So
what
does
it
include?
The
short
answer
is
everything
that
you
need
to
manage
the
life
cycle
of
Agents,
so
that
includes
things
like
definitions
for
remote
configuration
of
Agents.
Obviously,
things
like
status,
reporting,
agent,
Zone
Telemetry,
so
what's
going
on
in
the
internals
of
that
agent,
agent
upgrade
so
allows
for
packages
to
be
deployed,
and
then
secure
security
considerations
like
connection
credential
management
and
secure,
auto,
update
capabilities
and
again
that
was
part
of
the
focus
of
you
know,
making
sure
that
this
is
something
that
can
be
used
by
the
the
largest
organizations.
A
And
I'm
going
to
go
through
a
little
bit
about
the
the
communication
model
and
a
little
bit
more
about
the
protocol,
but
you
know
a
lot
of
the
work.
The
hard
work
goes
into
defining
the
protocol,
but
it
can
be
more
interesting,
seeing
it
in
a
actual
application
demo.
So
I'll
do
a
short
demo
as
well.
A
This
can
be
embedded
in
an
agent
I'll
talk
about
other
ways
that
it
can
can
operate
in
an
op-amp
server.
Communication
is
typically
via
websocket,
but
also
supports
http,
and
it's
fully
asynchronous.
If
you're
using
websockets,
all
the
payloads
are
binary,
serialized,
protobuf
messages
and
then
the
as
you
might
imagine,
the
op-amp
protocol
is
well
defined
for
all
the
scenarios.
I
just
showed.
A
But
then
it
layers
on
an
additional
component,
which
is
own
Telemetry
for
the
agent,
so,
as
you
might
imagine,
folks
that
are
involved
in
open,
telemetry
resistant
to
Reinventing,
otlp
and
definitions
for
Telemetry
signals
being
sent
to
another
service.
So
the
way
this
operates
is
op-amp,
Will
configure
an
otlp
exporter
in
the
agent
and
that
will
be
sent
then
to
the
otil
otlp
receiver
in
the
agent
management
service
and
the
little
box
in
the
the
bottom
that
I
neglected
to
mention
the
first
time
other
clients,
that's
really
everywhere.
A
A
This
was
one
other
model
that
the
specification
describes.
It
doesn't
doesn't
provide
full
full
specs
for
what
this
would
be,
but
a
common
scenario
is
we
have
an
agent,
that's,
and
maybe
it's
proprietary.
Maybe
it's
something
that
you
don't
want
to
open
up
the
internals
and
embed
a
new
library
in
that,
and
so
in
this
case
we
have
separate
supervisor
process
that
is
intended
to
be
an
op-amp
client
and
also
to
manage
agents,
whether
they're
proprietary,
they
could
be
open
source.
A
A
A
It
implements
an
op-amp
server
right
and
it
also
works
with
an
open,
Telemetry
collector,
specifically
the
observe
IQ
distrib
Rotel,
which
includes
an
embedded
op-amp
client
and
some
other
functionality.
So
as
we
look
at
this,
you'll
see
that
there
is
already
an
embedded
op-amp
client
in
those
collectors,
but
it
takes
advantage
of
the
Opium
protocol
to
provide
full
life
cycle
management
and
configuration
of
your
observability
Pipelines.
A
Okay,
so
this
is
bind
plane
op
and
what
you're
looking
at
here
is
several
hundred
agents
that
are
under
management.
So
this
is
a
demo
environment.
It
could
very
easily
be
several
thousand
agents,
but
you
can
see
all
of
these
some
details
about
them:
the
status
of
those
agents,
the
versions,
the
configuration
and
even
the
amount
of
data
flowing
through
them.
A
So
it
had
a
predefined
configuration,
but
I
also
have
a
demo
configuration
that
we
can
switch
it
to
easily
to
show
off
some
of
the
other
capabilities
so
air.
You
know
you
know,
since
you
can
report
status,
you
can
then
get
in-depth
error
messages.
So,
if
I
introduce
a
simple
error
to
this
file
right,
something
that
happens
very
often
when
we're
trying
to
configure
complex
agents
like
this,
it
shows
us
exactly.
What's
going
on,
tells
us
exactly
how
to
resolve
that.
A
That's
all
standard
in
or
defined
in
the
op-amp
protocol
so
go
through
and
resolve
this.
So
we
don't
have
any
more
red
on
the
screen
and
then
you
can
see
it
pushes
that
new
configuration
and
we
get
the
immediate
status
response
and
part
of
that
is,
you
know
with
websockets.
It
is
very
fast,
immediate
response
communication,
but
a
lot
of
this
we're
talking
about
scale
so
I'm
going
to
look
at
this.
It's
a
configuration
that
was
predefined
and
we
want
to
apply
this
to
a
large
Fleet
of
Agents.
A
So
if
I
go
and
and
look
at
all
my
production
instances
I'm
going
to
select
all
of
those
again,
it's
130,
it
could
be
a
thousand
or
two
thousand
as
soon
as
I
select
those
it's.
It
already
has
connections
and
it
can
deploy
that
at
scale
very
quickly,
but
then
they're
also
managed.
So
we
know
what's
going
on
with
those
individual
agents.
A
A
This
defines
everything
that's
happening
in
the
configuration
file,
all
the
data
that's
flowing
from
The
Source
through
the
processors
to
the
destination,
and
so
when
you,
when
you
have
that
little
detail,
you
can
make
changes
in
real
time
that
are
going
to
impact
the
the
flow
of
data
through
your
system
and
give
you
a
lot
of
control
over
that
data.
So,
for
example,
here
we
saw
there
are
a
lot
of
logs
coming
through.
We
can
add
in
a
simple
severity
filter
to
filter
out
anything.
A
That's
beneath
the
warning
apply
that
and
get
real-time
feedback
on
the
data
reduction
that
we're
seeing
through
the
pipeline.
You
could
do
that
at
every
stage.
You
can
do
it
on
an
individual
basis.
You
can
do
it
across
hundreds
or
thousands
of
agents
and
see
the
changes
in
real
time,
so
I
think
I'm
at
time,
but
yeah
there's
a
lot
of
power
in
this
I
wanted
to
share
a
couple
resources
that
are
available
here.