►
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Lightning Talk: Prometheus rules validation one step further - Martin Chodur, Seznam
Introduction of a new tool for validation of Prometheus rules metadata and expression properties to align with the infrastructure setup.
A
You
may
know
me
as
fusakla
from
github
I'm
a
devops
engineer
at
cessnam.czect
and
recently
we
did
some
major
refactoring
of
our
alert
routing
and
we
found
out
that
we
need
one
thing
additional
to
test
in
our
stack,
alerting
stack
to
make
sure
everything
works.
As
expected,
you
can
see
all
this.
You
can
test
with
chrome
tool
or
am
tool,
but
this
one
still
isn't
possible,
and
this
was
the
motivation
to
create
a
tool.
I
would
like
to
show
you.
A
You
need
somehow
ensure
that
the
alert
that
the
labels,
the
alerts,
have
are
those
that
you
expect
in
the
routing
tree
of
the
alert
manager.
So
the
tool
is
called
permeable.
You
can
just
simply
go
get
it,
and
here
is
simple
example
of
alert
which
has
label
severity
with
value
info,
and
the
second
snippet
shows
the
validation
yaml
file,
and
this
is
actually
a
configuration
file
for
the
tool
promoval.
A
Here
you
can
see
I'm
using
has
labels
validator,
which
ensures
that
each
rule
has
the
label
severity
which
we
see
the
alert
does
have
and
the
other
one
validator
is
label
has
allowed
values
and
it
checks.
If
the
rule
has
the
label
it
checks.
If
the
value
is
one
of
the
allowed
values,
we
can
see
that
this
is
not
the
case,
so
the
validation
should
fail,
and
now
we
can
try
to
run
the
two.
A
You
can
use
the
prebuilt
binaries
from
github
and
run
just
permeable
validate
passing
the
configuration
file,
which
is
the
validation
yaml.
You
saw
on
the
previous
slide
and
the
rules
yeah
mo
with
the
rules.
Actually,
you
want
to
validate
it
prints
out
some
human
readable
description
of
the
validation
that
will
be
done
and
the
result
which
we
can
see
is
invalid
should
be
again
human,
readable,
easy
to
fix
and
some
statistics.
A
I
show
you
just
just
two
of
those
validators:
these
are
all
of
the
currently
supported
validators.
These
are
for
labels
for
annotations.
I
would
point
out
that
there's,
for
example,
validation
of
annotation
value.
If
it
comes,
it
contains
contents
valid
url,
which
can
be
handy.
If
you
put
playbook
links
in
your
annotations,
it
can
also
resolve
the
url
actually
or
you
can
validate
your
expressions.
A
There
are
just
three
of
those
validators.
I
would
point
out,
for
example,
the
does
not
use
older
data,
then
I
actually
bump
into
this
myself
many
times
writing
alert,
which
uses
longer
more
data
than
the
retention
of
the
prometeurs
has,
which
you
can
forbid
by
this.
You
can
avoid
this
error
missed
this
issues,
which
is
really
handy.
A
Yeah
d2l
actually
embeds
the
prompt
ql
code,
they
code
from
from
prometheus,
so
it
actually
really
parses
the
from
ql
and
analyzes
it.
You
can
also
disable
the
rules
temporarily.
You
can
do
it
by
well-known
annotation
and
passing
the
names
of
the
validation
rules
separated
by
commas
or
using
the
command
line
flag,
and
the
last
feature
is
that
the
validation,
yaml
configuration
can
grow
a
lot
and
it's
not
easy
easily
readable.
A
So
if
you
want
to
provide
your
users
or
anyone
creating
an
alert,
some
human,
more
human,
readable
form
of
the
validation,
you
can
use
the
command,
promoval
validation,
docs,
pass
it.
The
configuration
file
and
set
the
output
currently
supported.
Output
is
html,
markdown
and
text,
and
it
looks
like
this.
So
this
is
all
there
are
some
future
ideas
and
make
sure
you
check
out
the
github
record
repository.