►
Description
How Prometheus indexes Data and Why You Should Care - Harkishen Singh, Timescale
Prometheus is capable of ingesting and storing large amounts of metric samples. Prometheus users define queries and dashboards to extract insights from all that data that help them ensure their systems are up and performing as expected. Good query performance is important and that’s why Prometheus indexes incoming data. In this talk we will dive into how Prometheus indexes incoming data. We will aim to give you a visual understanding of the on-disk layout and data structures used to store samples. The aim is to develop an intuitive understanding of data access complexity and costs. This will inform you about how to manage cardinality and how PromQL queries leverage the index to speed up query execution.
A
A
brief
about
me,
hello,
everyone
I
am
singh.
I
live
in
bhubaneshwar,
which
is
an
east
coast
of
india.
I
work
as
a
software
engineer
in
a
time
scale
and
contribute
to
prometheus
upstream
regularly,
while
developing
and
maintaining
prom
scale,
which
is
an
open
source,
observability
backend
that
can
store
prometheus
metrics
and
open
telemetry
traces
when
I'm
not
working.
I
read
books
and
play
computer
games.
A
That's
about
me.
This
talk
will
cover
three
topics.
We'll
start
with
understanding.
The
prometheus
data
model
then
learn
about
the
indexing
strategies
that
prometheus
uses
to
index
its
data
and
finally
learn
the
querying
execution
flow
with
the
help
of
prometheus
indexing
strategies
that
we
learnt
in
part
2
and
the
effect
of
high
cardinality
on
it.
A
Let's
begin
with
understanding
the
prometheus
data
model,
for
me
just
uses
a
simple
data
model
that
contains
labels,
values
and
timestamp.
In
this
example,
we
have
a
metric
go
underscore
go
routines,
which
has
two
labels
instance
equals
c
advisor,
called
8080
and
job
equals
c
advisor,
which
has
a
value
125,
followed
by
a
timestamp.
A
Now,
what
are
series
series
are
basically
sets
of
labels.
Now,
if
you
change
one
label
or
even
its
label
value,
it's
a
completely
new
series
in
prometheus
data
model,
for
example.
The
third
label
over
here,
which
is
instance,
has
its
value
changed
towards
the
right.
This
leads
to
completely
new
series
for
prometheus.
A
A
A
A
A
posting
index
is
a
relationship
between
labels
and
a
set
of
series
that
contains
that
particular
label.
For
example,
name
equals
up.
This
label
is
contained
in
two
series
and
similarly
the
next
example
is
instance
equals
9091.
We
have
once
we
have
one
series
in
our
example,
which
contains
this
particular
label.
A
A
A
Let's
understand
a
basic
query
execution
strategy,
so
a
query
has
two
things
as
we
learned,
which
were
labels
and
time.
The
first
step
uses
the
time
component
of
the
query,
which
basically
means
we
see
the
queries,
time
range
and
figure
out
which
block
overlaps,
the
time
range
of
the
query,
and
then
we
consider
those
blocks
for
query
evaluation.
A
A
Hence
we
move
to
the
next
step,
which
is
looping
through
each
of
the
series
list,
which
we've
got
from
step
two
and
using
the
help
of
series
index
which,
if
you
recall,
was
relationship
between
series
and
a
set
of
chunks
that
belongs
to
that
particular
series.
Only
so,
while
we
loop
over
the
series
using
the
series
index,
we
find
the
chunks
that
belong
to
that
particular
theory
and
then
retrieve
samples
from
the
chunks
in
step.
Four.
A
For
understanding
the
querying
process,
it
is
important
to
understand
that
the
steps
that
I
said
happens
for
each
block,
then
each
series
and
finally
each
chunk.
This
means
any
delay
on
step.
1
will
have
multiplicative
effect
on
the
following
steps.
Similarly,
for
step
two
and
so
on,
for
example,
if
there
are
a
lot
of
blocks
to
to
consider
for
a
query
evaluation
that
will
slow
down
the
series,
because
we
have
to
go
through
each
of
the
index
file
in
the
block
and
then
read
the
the
respective
posting
syntax
and
the
respective
series
index.
A
A
Next,
we
use
the
postings
index
to
find
which
series
satisfy
the
labels
of
the
query
over
here.
The
query
as
up
which,
in
other
words,
is
name
equals
up.
So
we
see
the
postings
table
and
find
out
which
series
contains
this
particular
label.
In
our
example,
we
have
two
series
for
that
in
the
next
step.
A
A
A
A
A
A
It
gets
a
match
for
the
first
three
entries,
because
these
three
entries
satisfy
the
queries
labels.
Now,
what
the
query
wants
is
a
series
that
contains
name
equals
up
and
job
equals
prom
and
instance,
equals
9090..
So
we
do
an
intersection
between
the
three
entries
and
we
get
series
three
as
the
result.