►
From YouTube: Grafana Loki Community call [2023-02-02]
Description
Monthly Grafana Loki community call NASA timezone. It's all about Loki OSS community.
Few highlights from the talks
1. 2022 Recap for Loki
2. Loki 2.7.2 release (multi-tenant querying)
3. Grafana 9.4: New features and updates for Loki data source
For original notes please take a look at the google doc
https://docs.google.com/document/d/1MNjiHQxwFukm2J4NJRWyRgRIiK7VpokYyATzJ5ce-O8/edit#
A
A
272
Ivana
is
going
to
talk
to
us,
probably
about
some
new
features
in
grafana
9.4
super
exciting
and
then,
if
anybody
has
any
other
topics,
anything
you
want
to
talk
about,
throw
them
on
here.
So
what
happened
in
2022?
This
is
a
good
question,
because
time
goes
by
really
quickly
the
big
things
that
came
to
my
mind.
Obviously
the
changing
of
the
index
that
was
probably
about
a
year-long
project
from
designed
to
we've.
Now
all
of
our
clusters
internally
run
on
tsdb.
A
Now
we'll
probably
be
looking
to
do
another
release,
maybe
the
3.0
release
for
Loki,
where
we
essentially
promote
that
to
being
production,
ready,
I
would
say
at
this
point
it's
as
good
as
production
ready,
although
you
want
to
be
probably
running
from
a
newer
version
than
2.7,
because
I
think
we've
got
a
couple.
Fixes
in
but
actually
probably
what's
in,
2.7
is
also
going
to
work
well,
but
very
exciting
for
us
we're
seeing
all
kinds
of
improvements
both
in
smaller
indices.
A
You
know
as
much
as
a
third
of
the
size
so
on
some
of
our
tenants
that
are
in
the
like,
you
know,
four
or
five
six
hundred
thousand
streams
in
a
24-hour
period.
The
index
went
from
about
a
gigabyte
per
day
to
to
300
megabytes
per
day,
so
big
wins
there
on
size.
Also,
the
ability
for
it
to
do
the
sharding
dynamically
on
queries
is
allowing
for
much
higher
query
throughputs.
A
We
see
some
queries
on
big
clusters
in
the
three
four
five
hundred
gigs
a
second
range
now
I,
don't
know
that
we've
had
one
crossed
a
terabyte.
A
second
we've
been
very,
very
close.
A
couple
900
gig
a
second
queries
in
there,
and
that
largely
is
because
we
can
parallelize
more
The.
Sharding
Factor
on
the
bolt
TV
index
was
only
16,
so
it
most
we
could
split
by
time
and
then
by
16..
A
Now
we
can
split
by
time
and
by
I
forget
what
our
the
most
will
Shard
by
10
000
or
something
really
big.
So
if
you
have
the
hardware
resources
configured,
you
can
throw
a
lot
more
parallelization
at
it.
The
other
end
of
that
Spectrum
now
is
that
we're
gonna
probably
start
looking
at
the
storage.
That's
in
I'll
get
to
that
in
a
second
I
guess
and
I'll
tell
you
about
why
when
I
get
there
but
other
fun
things,
you
know
we're
I
would
say
largely
as
a
team
we've
grown
a
bunch.
A
But
Trevor
is
here.
Trevor's
done
a
ton
of
work
on
our
Helm
chart
now,
which
has
been
moved
and
to
the
Loki
repo
again,
so
that
we
can
hopefully
keep
closer
eye
on
it.
We're
trying
to
really
curate
it.
An
experience
around
the
home
chart
that's
more
opinionated
to
get
to
where
folks
can
have
a
similar
install,
so
that
we're
hoping
that
with
more
people
running
Loki
the
same
way,
it
makes
it
easier
to
flesh
out
problems
and
have
a
you
know,
a
more
I,
don't
know
easy
to
operate
experience.
A
A
When
you
start
getting
into
the
500
Plus
or
maybe
a
terabyte
a
day,
you
really
do
need
to
have
a
cluster
with
enough
resources
and
configured
well,
and
that
stuff
is
tricky
and
we're
basically
trying
to
do
our
best
to
continue
to
abstract
configs
away,
so
that
Loki
can
do
that
more
automatically.
Ssd
embeds
a
lot
of
the
complexity
of
microservices
tries
to
take
that
away.
B
Yeah
I'll
say
something
correct:
sorry
for
the
background
noise
yeah,
one
of
the
other
main
goals
of
the
work
in
the
house
artists.
You
know
before
this
we
had
like
I
think
the
total,
if
you
includes
the
Enterprise
charge,
was
like
seven
different
time
charts,
and
so
you
had
this
like
the
community.
Contributions
were
spread
out
over
seven
different
charts
and
so
each
chart.
B
A
A
A
So
if
you
find
a
bug,
you'll
have
to
fix
it
yourself
kind
of
thing
right
now:
I,
don't
think
we
communicate
that
super
clearly,
and
you
know
it
It's
tricky,
because
I
I
don't
want
to
take
features
away
like
I,
don't
want
to
remove
capabilities
and
agents
and
stuff
and
I
understand
the
frustration
when
someone
like
is
using
a
thing
because
it
meets
their
requirement.
Doesn't
work
I,
just
there's
no
sort
of
perfect
answer
here
for
us,
but
we
have
to
try
to
focus
at
least
or
narrow
our
Focus
some.
A
If
we
want
to
be
able
to
move
things
forward,
otherwise
we
could
just
mired
down
in
being
a
thousand
miles
wide,
an
inch
deep,
okay.
So,
what's
coming
in
2023
I,
don't
know
I
put
question
marks
next
to
all
of
these
things,
because
we
haven't
totally
figured
it
out
yet,
but
we
are
looking
a
lot
more
into
how
the
query
execution
works
and
probably
building
like
a
real
query.
Planner
I
just
alluded
to
the
chunk
storage.
A
You
aren't
going
to
run
into
this
with
object,
Stores,
but
you
know
we're
kind
of
figuring
out
how
to
there's
an
aspect
of
Loki
storage.
That's
always
kind
of
bugged
me
a
bit
is
that
it's
really
opaque
and
really
hard
to
do
manual
operations
around,
and
so
that's
something
to
like,
considering
putting
in
some
like
human
readable
date,
prefixes
around
how
we
store
things
so
that
it's
a
little
bit
less
opaque.
It's
not
there's
a
lot.
You
can
do
as
a
person,
but
it
it's
kind
of
tough.
A
You
know
if
you
did
find
yourself
wanting
to
like
look
at
chunks
or
move
things
and
do
physical
operations
on
it.
It's
pretty
tricky
to
do
that,
because
it's
just
a
bunch
of
hashes
and
folders,
and
things
like
that,
and
we
continue
to
talk
a
lot
about
open
Telemetry
support
the
open
time
open,
Telemetry
collector
has
a
Loki
output
plug-in.
A
A
You
know
whether
it's
configuring
it
at
the
source
or
configuring
it
at
the
destination
or
you
know,
do
you
want
to
store
all
that
metadata
or
not.
So
that's
that's
in
the
plans
for
what
we
want
to
do
and
I'm
sure
there'll
be
a
whole
bunch
more.
But
if
you
have
things
you
really
want
for
Loki,
you
have
feature
requests
that
you
really
need
or
want
go
open.
A
So
those
needs
are
those
things,
don't
always
over
overlap,
but
Loki's
been
doing
really
well
at
both
I
mean
I
sort
of
still
run
on
Raspberry
Pi's
like
three
or
four
low-key
setups.
You
know
it
does
just
fine,
it's
very
happy
with
small
hardware
and
and
small
setups,
and
we
want
that
to
be
the
case
forever
so,
but
it
would
be
helpful
if
you
have.
A
A
That
was
a
good
deal.
We
kind
of
missed
the
2.8,
so
we're
it's
a
bit
busy
for
the
next
few
weeks
here.
So
I
would
suspect
that
a
2.8
should
be
following
after
that.
So
like
late
February,
early
March,
I,
don't
think
we'll
do
the
3.0
next,
because
I
we
gotta
kind
of
organize,
there's
a
number
of
things
within
Loki
that
we'd
like
to
fix,
renaming
metrics,
for
example,
that
are
tough
for
operators
and
we'll
probably
put
them
under
a
3-0
release,
not
a
huge
amount
of
stuff.
D
So
hey
everyone
I
would
like
to
share
with
you
new
features
and
updates
that
are
coming
in
graph
and
994
and
actually
I,
think
beta
and
for
beta
was
released
either
yesterday
or
a
couple
days
ago.
So,
if
you
are
interested
to
try
them
out
it's
available,
but
the
stable
is
coming
in
the
next
couple
of
weeks.
D
Okay,
so
there
is
a
couple
of
updates:
I'm
gonna
point
out
the
more
visible
ones,
but
definitely
go
into
the
change
lock
to
see
all
all
updates
and
new
features
related
to
walking
and
logging.
So
I
would
start
with
label
browser,
so
label
browser
is
quite
popular
feature
to
interact
with
your
labels
to
filter
them
based
on
other
labels,
and
previously
these
was
available
in
cold
mode.
D
So
we
moved
it
and
now
it's
available
in
both
mode,
so
it's
place
and
while
we
were
working
on
it,
we
also
decided
to
move
it
into
the
model,
and
this
is
especially
useful
in
dashboards,
because
in
dashboards,
when
you
are
editing
query,
you
have
your
panel
options,
you
have
your
visualization
and
query.
Editor
is
kind
of
just
like
one
third
of
the
screen,
so
label
browser
was
quite
like
cramped
in
the
space,
and
now
you
have
like
almost
full
screen
where
you
can
interact
with
the
with
your
labels
and
filter
them
out.
D
Moving
on
to
a
thing
that
was
also
mentioned
on
the
previous,
it's
going
to
be
released
in
the
info,
so
I
decided
to
mention
it
as
well.
So
we
have
implemented
implemented
and
added
query
validation
in
the
code
mode.
So,
as
you
are
typing
your
query,
we
are
doing
real-time
validation
and
in
case
you
write
something
incorrect
or
something
that
produces
incorrect
block.
Ql.
D
D
So
that's
a
second
thing,
I'm,
looking
into
notes
what
okay
so
log
details,
we
have
done
quite
a
lot
of
work
related
to
the
log
details,
and
this
is
coming
from
a
lot
of
like
historical
contexts.
The
log
details
were
kind
of
messy
because
we
were
adding
features
and
then
Loki
was
implementing
like
the
features
that
log
details
was
substituting
and
when
I'm
talking
about
features
I'm
talking
about
the
technique
build.
D
So
in
the
past
we
used
to
have
labels
and
detected
fields
and
a
lot
of
users
were
confused
about
like
where
is
it
detected?
Is
it
coming
from
Loki?
Why
are
there
different
sub
features
and
so
on
and
we
implemented
detected
Fields
because
at
that
point
of
time,
Loki
wasn't
implementing
tarsers.
So
we
try
to
help
users
and
show
them
key
value
pairs
in
the
log
details,
but
in
I
think
it
was
lucky
too
they
implemented
parsers
and
now
these
detected
fields
are
kind
of
obsolete
and
we're
causing
much
more
confusion
than
it's
needed.
D
So
that's
the
reason
why
we
have
decided
to
just
show
field
which
are
basically
labels
and
we
are
implementing
all
features
that
were
implemented
more
important
detected
field
and
therefore
we
hope
that
it's
going
to
be
more
straightforward,
more
understandable
for
users
and
that
you,
users
per
user,
is
going
to
be
easier
to
understand.
Where
are
these
coming
from?
D
D
D
So
if
you
run
metric
query-
and
you
would
like
to
see
samples
of
log
lines
that
contributed
to
these
metrics,
you
can
click
here
on
the
log
sample
toggle
and
it's
going
to
give
you
and
show
you
samples
of
those
worklines
in
the
case
that
you
would
like
to
more
interact
with
these
log
lines
or
maybe
change
them.
You
can
use
open,
Works
in
split
view
button
and
it's
going
to
open
split
view
with
the
log
query
for
you.
So,
depending
on
your
you
on
your
use
case.
D
If
you
need
to
go
deeper
into
your
log
lines
and
change
the
query,
you
can
do
it
in
split
view,
but
if
you
put
just
like
an
overview
that
helps
you
to
figure
out
like
what
labels
you
should
use
or
if
you
are
debugging
and
looking
for
like
specific
loglines,
maybe
just
walk
samples.
Would
be
enough
for
you
for
this
case,
so
that's
locks,
them
pause
and
last
thing
and
last
feature
that
I
would
like
to
mention
is
estimating
the
size
of
the
query.
D
So
maybe
you'll
notice
that
we
have
here
this
new
message.
That
is,
that
is
telling
you
approximation
of
how
much
data
is
going
to
be
touched.
When
you
run
this
query,
we
are
using
for
this
new
stats
endpoint.
That
block
is
providing
and
visualizing
it
for
you
in
the
current
implementation.
It
is
showing
you
how
much
data
will
be
approximately
touched,
but
in
the
future
we
are
also
thinking
about
making
it,
like,
maybe
more
obvious,
like
showing
you,
okay,
this
is
probably
too
much
data
or
hey.
D
D
A
D
I
actually
have
a
question,
so
in
the
in
the
past
we
used
to
have
this
GitHub
issue
where
I
was
like.
What
are
you
looking
for
from
walking
2021
22.?
Is
there
a
plan
to
do
something
for
2023,
because
it
was
also
beneficial
for
our
grafana
team,
as
a
lot
of
people
also
mentioned
features
that
would
be
helpful
for
their
flow
and
in
some
cases
it
was
a
feature
that
would
be
implemented
in
grafana,
so
yeah.
A
A
But
it's
hard
right
like
it's,
so
you
you
get
really
long
credit,
so
GitHub
discussions
would
be
a
better
place
for
this,
but
we
we
don't
have
GitHub
discussions,
enabled
on
the
Loki
repo
there's
been
some
internal
back
and
forth
because
it
would
lend
itself
nicely
to
some
things,
but
it's
like
another
place.
We
have
to
go
and
keep
track
of,
and
we
already
have
too
many
places
that
we
have
a
hard
time
keeping
track
of.
A
A
A
A
A
Yeah,
let's
make
another
one
of
these
and
see
what's
sort
of
relevant
or
new
things.
People
would
like
I
think
it's
a
good
idea.
It's
always
hard
to
get
feedback
from
the
community
like
it's
just
sort
of
not
a
great
way
to
advertise,
and
you
know
we
tend
to
hear
when
things
don't
work
and
it's
nice
sometimes
to
hear
when
things
do
work
too.
So
you
know
I
could
start
an
issue
for
like
tell
us
what
you
like
about
Loki,
so
that
we
don't
break
that
stuff.