►
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
The Prometheus Conformance Program - Richard Hartmann, Grafana Labs
Prometheus is the standard for cloud-native metric monitoring and beyond. We're a victim of our own success inasmuch as everyone claims to be compatible with Prometheus when many are not. In the best case, this leads to user confusion. In the worst case, this leads to data loss, data change, and missed alerts. In our experience as Prometheus team supporting end users, the latter is scarily common. To fix this situation, CNCF and Prometheus team have launched the Prometheus Conformance Program. We will walk through its design, through current test suites available, through current test results, and show you how to apply for the official mark of Prometheus compatibility. Depending on overall timeline, we might use this talk to publish official marks for projects and products.
A
Hi
and
welcome
to
the
prometheus
conformance
program.
You
might
never
see
this
talk
actually,
in
fact,
I
hope
you
don't,
because
if
you
don't
see
this
talk,
this
means
we
figured
out
how
to
splice
this
in
as
a
neotalk,
simply
because
we
will
have
pretty
fresh
results
which
currently
we
don't
know
yet,
because
it's
a
month
before
coupon,
so
I'm
kind
of
optimizing
towards
having
a
really
short
actual
talk.
A
In
the
past,
we
will
most
likely
also
end
up
publishing
those
results,
just
to
have
a
little
bit
of
a
fair
level
playing
field
between
between
all
the
different
offers
and
and
vendors
and
such
as
of
this
recording.
I
don't
know
who
is
making
that
cut
or
was
making
the
cut
my
we
just
don't
know
it's
too
early.
A
The
intention,
obviously,
is
that
we
want
the
end
users
you
most
likely
to
be
able
to
choose
between
different
implementations
between
different
offerings
with
confidence,
without
having
any
confusion
about
about
what
is
actually
compatible
with
purposes
and
what's
not,
we
have
a
little
bit
of
a
problem
of
success.
We
are
victims
of
our
own
success.
A
A
How
does
this
work?
We
created
a
suite
of
tests.
You
can
see
them
here
just
in
the
promises
or
compliance
and
anyone
can
run
them.
Obviously,
part
of
the
design
cncf
or
more
to
the
point.
The
linux
foundation
is
using
trademark
law,
which
I
found
interesting
when
I
first
realized
how
this
is
structured
to
grant
the
usage
of
a
mark.
In
this
case,
the
mark
promise
is
compatible
with
little
logo
and
such
so
those
vendors.
Those
projects
can
showcase
that
yes,
they're
actually
compatible
with
prometheus.
A
To
get
this,
they
need
to
sign
paperwork
with
cncf
or
the
linux
foundation,
which
include
clauses
like
good
faith
testing.
You
can't
cheat
on
the
tests,
blah
blah,
blah
blah
and
lots
of
dry
stuff,
which
you
need
to
sign
if
you,
if
you
want
to
actually
have
that
mark,
but
then
you
agree
on
the
terms
of
the
usage
of
that
mark.
That's
a
prerequisite!
A
So
if
you,
if
you
would
try
to
stop
agreeing
with
this,
you
already
signed
a
contract
in
the
past,
which
is
kind
of
nice,
this
this
mechanism,
in
this
automated
automation
in
in
contracts
nice.
When
I
realized
how
this
works,
you
can
either
to
submit
your
test
results
yourself
or
a
prometheus
team,
runs
them
for
yourself.
For
you
with
things
like
remote
write,
it's
a
lot
easier
to
run
it
yourself
and
to
write
tests
yourself
and
just
run
them
fully
automated
with,
for
example,
prongql.
A
You
need
quite
a
bit
of
interpretation
of
the
result.
Sets
we'll
get
there
and
hopefully
there'll
be
more
automation
over
time,
and
then
you
really
receive
a
time
limited
mark
to
actually
show
that
you
are
prometheus
compatible
and
if
we
on
the
prometheus
side,
update
our
our
test
suite-
and
you
run
out
of
that
out
of
that
grace
period,
where
you
have
that
mark
lose
that
mark
under
the
trademark
contract,
which
again
is
for
a
nice
design,
you
might
stumble
across
three
different
words,
which
look
almost
the
same
conformance.
A
A
Why
do
I
say
if
it's
relevant,
if
you
don't
offer,
for
example,
a
storage
course
instrumentation
library,
it
would
be
kind
of
unfair
to
have
alert
generation
or
something
as
part
of
this.
So
that's
where
this
is
coming
in.
You
might
not
actually
be
implementing
all
the
interfaces
of
prometheus
for
your
thing
to
be
considered
compatible
with
prometheus
if
you're
an
end
user.
It's
quite
easy,
look
for
prometheus
compatible
and
ignore
everything
else.
A
The
nice
thing
is,
they
all
have
defined,
meaning
in
this
contract
in
under
this
program,
which
means
that
if
you
signed
the
thing,
you
implicitly
acknowledged
that
those
have
specific
meanings
and
you
cannot
in
some
potential
future,
become
clever
with
how
you
use
other
words,
which
is
a
little
bit
of
a
trick.
But
it
has
a
nice
forcing
function
to
just
keep
everyone.
A
Honest,
there's,
also
incentives
here,
there's
like
a
lot
of
incentives,
as
we
in
prometheus
team
have
found
over
the
years
again
and
again
and
again,
there
is
a
ton
of
informal
knowledge
among
vendors
and
such
who
is
actually
compatible
and
who
might
not
be,
but
they
don't
really
want
to
raise
a
fuzz
or
have
a
public
fighter.
What
have
you?
A
Even
if
some
people
are
aware
that
some
things
are
not,
as
they
might
seem
to
appear,
but
remember
we
have
these
huge
amounts
of
money
which
the
market
is
worth
like
the
market
segment
in
which
we
are
moving.
So
there
is
a
direct
monetary
interest
for
people
for
enterprises
for
managers
to
improve
the
tests
to
keep
everyone
else.
Honest.
A
A
A
It
might
actually
be
the
case
that,
as
we
introduce
new
tests
or
new
aspects
of
testing
that
some
products,
some
projects
even
lose
their
their
mark
of
compatibility,
and
then
they
have
to
re-implement
something
to
or
implement
something
fix,
something
to
actually
retain
or
regain
that
mark,
which
is
not
nice
and
like
again.
This
is
hopefully
what
what
part
of
the
incentive
is
for
a
lot
of
people
to
keep
each
other
honest
and
to
just
keep
moving
the
complete
ecosystem
at
a
nice
velocity.
A
If
you
need
to,
if
you
need
help
to
convince
your
manager
to
let
you
personally
write
stuff
here,
talk
to
us,
we
can
most
certainly
make
a
good
case.
Why
you,
as
that
person,
should
be
doing
this
and
that
work
on
this?
Invest
that
test
suite.
We
can
help
you
with
design
document.
We
can
help
you
with
everything
on
purpose.
We
want
to
structure
this
as
openly
as
humanly
possible
matter
of
fact.
We
already
have
two
quite
large
cloud
providers
who
committed
to
working
on
a
prometheus
remote
write
receiver
test
suite.
A
Maybe,
as
we
talk
it's
already
done,
I
don't
know
so.
We
just
opened
the
design
document,
so
they
have
a
place
where
you
can
write.
Everything
in
everyone
can
get
your
feedback,
no
matter
if
they're
from
those
vendors
or
prometheus
team
or
from
someone
else
entirely,
it
doesn't
matter,
and
once
there
is
consensus
on
how
that
test
should
be
structured
watching
what
needs
to
be
covered.
All
those
things
people
can
just
get
going
all
develop
in
the
open,
all
even
designed
and
discussed
in
the
open
to
make
this
as
honestly,
fair
as
possible.