►
From YouTube: LogQL v2 Public Design Call 2020-06-08
Description
Public design call around LogQL v2
A
A
B
By
the
way,
something
I
think
might
be
interesting
to
share
with
all
of
you,
so
we
had
something
interesting
that
happened
last
week
where
we
had
like
a
performance
regression
and
it
like
showed
like
the
kind
of
slow
through
all
the
signals,
kind
of
worked
extremely
perfectly.
In
our
case
there,
where
we
had
like
we
deployed
something
we
had
an
increase
in
latency
which
alerted
us.
We
went
to
logs
found
a
trace
id
from
that
used
the
trace
id
to
look
at
jager
and
from
jaeger.
B
We
could
find
the
exact
instance
where
something
was
spending
a
lot
of
time
right,
and
then
we
used
that
that
time
and
the
instance
to
call
profile
in
controv
and
actually
find
the
exact
spot
like
within
two
minutes
where
the
performance
regression
had
happened.
It
was
pretty
cool.
I
think
we're
probably
going
to
schedule
a
blog
post
about
this
and
kind
of
share
with
everyone,
but
yeah.
A
A
A
A
C
So
the
I
think
the
challenges
is
also
mean,
if
you
so
the
the
challenge
with
the
the
pipelines.
Once
you
get
to
kind
of
very
exp,
very
complicated
expressions,
it's
like
it's
really
not
clear
like
how
you
would
do
binary
operations
in
a
in
a
pipeline
right.
You
would
end
up
having
to
bracket
things
anyway,
yeah,
in
which
case
it
starts
to
look
like
the
prometheus
expressions.
I
understand
agree
like
in
the
prometheus
world.
Matching
brackets
is
a
pain
in
freaking
ass,
especially
on
large
queries.
C
D
C
C
D
Yeah
I
mean
I
would
question
that
that,
from
a
user
point
of
view,
there's
there's
maybe
a
point
where
you
also
perceive
loki
a
bit
more
as
a
drop-in
replacement
for
some
other
logging
solution
that
you
have
and
then
the
sort
of
the
the
prometheus-isms
is
they.
Maybe
then
they
stand
a
bit
in
the
way
right.
So
it's
kind
of
like
do.
D
We
want
to
embrace
the
sort
of
early
crowd
now
who,
like
obviously
know
prometheus
well
and
no
know
all
the
like,
yeah,
more
all
the
background
on
why
the
things
are
the
way
they
are
or
do
are
we
do?
We
want
to
be
a
bit
more
forward,
looking
and
say
like
listen,
this
is
ultimately
a
logging
query
language.
Maybe
we
should
look
at
other
login
query
languages.
C
I
mean,
but
I
can
show
you
other
login
query
languages
that
have
brackets
like.
I
don't
think
it's
the.
I
don't
think
we
should
necessarily
use
the
fact
that
other
people
do
this
way
as
an
argument
to
do
it.
That
way,
right,
we
shouldn't
cargo
cult,
how
to
build
the
language,
but
we
should
look
at
like
if
we
want
to
be
like
aws's
cloud
cloud
watch
logs,
then
we
should
look
at
what's
powerful
in
doing
in
their
language.
C
What's
easy
to
do
in
their
language
isn't
easy
to
do
in
ours,
not
that
they
just
do
it
that
way
and
like
to
be
clear,
my
argument
for
prometheus
isn't
just
prometheus.
Does
it
that
way?
It's
the
the
familiarity
means
you'll
have
to
learn
less
means.
We
can,
you
know
effectively
try
and
map
one-to-one
between
prometheus
and
loki
and-
and
hopefully
people
are
already
very
familiar
with
prometheus-
should
be
very
familiar
with,
with
with
loki
like
that.
Doesn't
work
for
cloud
watch
right
because,
like
we're
not
trying
to
map
one-to-one
with
cloudwatch-
and
I
don't.
B
The
opposite
would
be
true
in
that
way,
then,
as
well
right
like
if
there
are
people
who
are
learning
loki
first
and
it
is
close
to
prometheus,
then
it's
also
easier
to
learn
prometheus.
That
way
right
sure
yeah,
because,
like
that's
where
david
was
coming
from.
As
I
understand
right
like
if
you're
just
starting
to
learn
this,
then
it
should
work
the
other
way
around
around
as
well.
A
A
B
E
Yeah,
so
the
the
interesting
thing
is,
I
think,
there's
a
couple
use
cases
here
where,
like
from
an
ad
hoc
query
standpoint,
I
think
the
pipe
syntax
is
a
lot
more
fluid
like
normally
I'm
just
writing
out
a
query,
and
I
want
to
manipulate
it
and
manipulate
it
more
and
like
this
is
what
I
do
from
cli
right,
but
you
know
from
alerting
and
binary
operations
and
sort
of
more
elaborate
examples.
That's
where
the
function
syntax!
E
It's
like
you
know
it's
it's
sort
of
right
once
and
reuse.
You
know
in
an
alert
or
something
like
that
or
you
have
a
query
you
save
or
it's
ad
hoc.
They
both.
I
think
I
don't
know,
there's
pros
and
cons.
That's
maybe
why
supporting
both
makes
sense,
because
you
know
writing
an
alert
rule
around
a
chain
of
pipes
and
things
might
look,
you
know
obviously
less
it
might
look
goofy
or
harder
to
read
than
a
properly
like
line,
separated
and
indented
function.
Syntax.
D
It
did
the
big
problem
there
was
that
the
filtering
was
difficult
where
you
had
to
where
you
had
to
name,
because
with
flux
is
like
basically
table
modifications
and
to
filter
something
you
had
to
reference
a
field
or
like
a
certain
column
and
then
say
a
filter
for
that
one,
and
that
just
that
just
made
those
filter
expressions
really
annoying.
I
think
if
that
hadn't
been
there
to
fill
these
pipes,
I
think
they
made
a
lot
of
sense,
and
I
heard
I
heard
some
good
feedback
from
other
people
that
writing
flux.
D
C
And
the
I
mean
the
reason,
I'm
keen
like
I'm
not
keen
to
jump
to
jump
to
an
answer
to
this
question
to
be
clear
because
we're
gonna
have
to
live
with
whatever
we
decide
here
for
years
right,
so
the
the
other,
the
other
pro
pipes
use
case
is
actually
in
the
metric
space
is
graphite
right
like
the
way
the
graphite
experience
in
grafana
works
is
effectively
stacking.
You
know,
select
some
queries,
then
aggregate
them,
and
then
you
know,
do
other
stuff,
like
you
effectively
stack
up
a
pipe
and,
for
instance,.
D
I
mean
it's
it's
more
like
it's
more
like
an
sql
expression
right.
So
if
you
it's
kind
of
difficult
to
say
or
yeah
yeah,
it's
kind
of
difficult
to
kind
of
use.
The
graphical
editor
as
the
basis
for
like
a
conclusion
here,
because
because
that's
like
a
simplification
on
top
like
it's
like
the
equivalent
of
syntactic
sugar
right
right
on
the
kind
of
write
sql
anyway
or
like
the
graphical
language,.
C
And
I
think
we're,
irrespective
of
what
we
decide,
we're
going
to
end
up
building
a
graphical
query
editor
following
that
kind
of
model
like
select
the
metrics
select
some
filters
select
some
aggregations
and
select.
Some
group
buys
like
we're
going
to
end
up
building
that
for
prometheus
and
for
loki.
D
C
That
important,
but
well
I
mean
and
and
both
of
these
proposals
on
the
table
are
equivalent
right,
like
the
choice,
is
one
of
style,
almost
right,
yeah,
because
I,
whilst
yes,
I
agree
like
binary
operations,
can
be
easier
in
a
bracketed
language.
I
don't
think
it's
necessarily
easier.
It's
just
you're
more
used
to
using
brackets,
whereas
in
a
pipeline
language
you
might
not
be
used
to
using
brackets
and
you
would
have
to
potentially
use
them
for
a
for
a
binary.
A
C
D
Yeah
I
mean
to
the
experienced
user,
it
might
feel
more
natural
to
just
go
start
typing
rate
parentheses,
open
yeah.
I
don't
think
well.
I
had
another
question
before
like
do.
We
know
any
case
where
this
would,
where
the
pipe
syntax
would
like
or
that
wouldn't
work.
Right
now
I
mean
you
mentioned
binary
operations,
but
maybe
that
can
be
solved
for
parentheses.
Was
there
anything
else.
D
B
A
A
E
C
C
B
D
D
B
D
B
A
A
B
A
C
D
D
B
C
I
mean
it's.
I
was
never
particularly
happy
with
there
being
two
ways
of
doing
filtering
with
the
original
proposal.
Right,
one
was
if
you
can
select
streams
based
on
labels,
and
then
you
have
a
separate
filter
keyword
that,
like
is
for
handling
the
this,
this
new
third
type,
which
was
a
record
or
something
right.
I
didn't
I
don't
I
don't.
I
don't
think
the
third
type
is
necessary
and
I
don't
like
that.
It
has
a
separate
set
of
syntax
for
handling
it.
C
If
the
json
and
the
regex
pipeline
stages
or
functions
whatever
we
want
to
call
them
were
to
extract
labels
and
values
into
labels
and
values,
not
into
a
new
record
type
yeah.
So
that's
like
that's
something
I
feel
pretty
strongly
about,
but
then
the
question
becomes.
How
do
you
filter
on
this
on?
You
know
how
do
you
select
some
streams
with
a
selector
pipe
it
through
json
to
turn
those
all
those
fields
into
labels,
and
then
how
do
you
run
those
again.
A
Yeah,
that's
something
that
I
removed
from
the
original
scope,
but
it's
probably
the
next
improvement.
That's
going
to
come,
and
it's
exactly
that
it's
you
extracted
new
labels,
but
you
now
need
to
filter
on
them,
and
this
is
like
a
real
use
case
that
all
you
have,
the
the
the
the
biggest
example
is
the
path
like
you
have
an
nginx
path
and
you
want
to
select
only
the
path.
That
is
this
one
right.
So
I
actually
this
one
is
very
easy
to
do
with
just
the
filtering,
but
there
is
the
json.
D
C
A
So
so
would
you
would
you
say
that
it's
weird
that
the
filtering
after
on
the
extracting
label
is
not
a
function
or
is
it
fine
if
it's
a
pipe
which
which
one
you
you
will
prefer
in
that
case,
because
you
haven't
really
created
magic.
Yet
if
we
say
the
example
like
you
have
rig
x
and
then
after
it's
a
filtering
on
a
label
extracted,
it's
not
yet
metric,
you
still
need
to
sum
or
count
on
top
of
it
to
make
it
a
metric.
C
I
think
if
we
went
with
you
know,
filtering
is
done
with
pipes
and
maths
is
done
with
functions.
If
we
went
with
that
rule,
then
this
would
have
the
this
would
have
to
be
part
of
the
pipe
operations.
It's
more
like.
I
didn't
like
the
actual
fact
that
we
had
a
separate
filtering
syntax
from
our
label
matching
syntax.
I
feel
like
they
should
be
the
same
yeah,
but
it's.
C
C
D
A
C
C
A
C
E
A
C
I
think
it's
very
important
to
understand
that
the
two,
the
two,
the
two
things
going
on
in
my
head
right
is
one.
Is
I
don't.
I
don't
like
like
a
set,
a
different
way
of
filtering
your
logs
to
the
first
way,
but
the
reason
I
don't
like
the
second
one,
which
is
by
using
the
same
method,
is
that
it
suddenly
that
method
becomes
context.
Sensitive.
D
D
C
C
C
Yep
yeah,
so
that
that's
what
that's!
What
leads
me
down
that
this
has
to
put
labels
and
and
values
in
in
the
existing
data
model,
for
for
the
rest
of
the
maths
to
work
right,
but
then
that
still
leaves
a
question.
What
you
know
this
kind
of
you
know,
maybe
maybe
we
get
rid
of
the
pipe
there.
Maybe
that
makes
more
sense,
because
suddenly
this
almost
matches
a
metric
name.
B
A
C
Then,
like
the
question
becomes,
should
the
filter
be
a
filter
on
the
should
he
I
mean,
I
like
the
idea
that
it's
not
right,
because
I
like
the
idea
that
log
format
is
an
atom
like
jason
like
like
regex
right,
it
would
look
weird
if
that
was
regex
and
the
specified
both
the
regex
and
the
filter,
so
I
think
it
make.
I
think
this
makes
sense.
This
is
a
this
is
a
note.
C
D
D
B
C
C
A
E
A
A
A
C
A
Yeah,
it
is
there's
one
thing
I
wanted
to
say
about
the
regex,
because
in
the
park,
the
proof
of
concept
that
I
did
this
is
something
that
I
realized
it's
at
hostly,
slow
and
there's
nothing.
That's
gonna!
Save
us!
There's
no
trick.
That's
to
save
us
on
that
part,
and
I
expect
people
to
switch
to
something
else
than
regex
as
soon
as
they
have
a
lot
of
logs.
A
C
B
C
D
A
D
C
A
B
C
B
Like
I,
I
can
give
you
a
very
concrete
example
where
it's
going
to
be
difficult
to
do.
Something
like
this.
Like
kubernetes
is
gonna
start
introducing
like
references
to
objects
as
a
nested
thing,
and
some
objects
are
namespace
and
some
are
not
so
while
they
all
will
have
the
ref
object
or
something
like
that
in
their
log
line,
some
will
have
a
namespace,
some
won't
and
it
will
be
from
the
same
log
stream.
A
A
C
C
We
have
to
this
is
just
one
suggestion:
yeah
right,
one
suggestion
would
be.
You
have
a
way
of
assigning
the
you
know.
An
entry
has
a
value
right
has
a
timestamp
and
a
value
and
labels
and
a
stream
is
a
well
sorry.
A
stream
has
labels,
an
entry
is
a
timestamp,
and
a
value
and
of
the
value
is
string
right.
So
one
way
to
do
it
would
be
to
tell
log
format
that
the
new
value
of
that
entry
is
this
field.
C
B
C
What,
within
a
single
yeah
yeah,
I
mean
imagine
if
there
was
two
messages,
both
king
jason,
you
want
to
filter
on
one
and
the
other
or
even
worse,
conditionally
filter,
one
based
on
the
other
yeah.
I
mean
you
can't
do
that
in
in
prometheus
right.
You
can't
query.
You
can't
conditionally
select
a
metric
based
on
the
value
of
one
label
like
in
another
label,.
C
B
C
B
C
A
C
C
D
B
D
D
C
D
D
C
D
C
D
A
Yeah
and
the
second
one
has
a
another
use
case
that
is
very
interesting
and
probably
match
up
with
jason.
Is
you
have
a
json?
There
is
a
message,
it's
unreadable,
if
you
don't,
if
you
keep
it,
as
is
in
graph
accurately
and
being
able
to
say
that's
only
the
json
that
I
want
to
show
in
my
log
panel
the
the
message.
Sorry,
it's
no,
I
think
it
has
a
lot
of
value.