►
From YouTube: Grafana Agent Community Call 2022-09-30
Description
Excited to announce the release of Grafana Agent Flow.
https://grafana.com/blog/2022/09/29/introducing-programmable-pipelines-with-grafana-agent-flow/
A
A
A
So
that's
related
to
the
operator,
which
Craig
is
probably
the
go-to
person
there,
and
we
probably
don't
have
an
answer
right
now,
but
we
can
definitely
dig
into
it
and
try
to
figure
out
why
you're
not
getting
correct
errors,
because
it
should
definitely
be
spitting
out
something
and
if
you
have
any
additional
kind
of
contacts
around
that
it
would
be
much
appreciated.
To
put
it
on
that.
B
A
B
So
he
said,
wow
fantastic
I
have
logs
there
and
I
can
fix
it.
So
so
so
he
gave
up
on
on
on
on
using
graph
an
agent
and
I
think
it's
quite
a
pity,
and
that's
a
little
bit
about
the
point
where
I've
already
talked
about
people
during
low-key
Community
course,
and
they
say
that
it
was
a
weak
point
of
the
agent
that
it's
quite
hard
to
debug
it,
because
because
the
the
the
the
base
tools
are
hidden
inside
it
like
from
Taylor
and
and
stuff
like
that.
B
So
when
I've
seen
the
the
announcement
about
the
the
the
new
interface
and
and
the
the
the
flow
stuff
I'm,
not
I'm,
not
much
interested
in
the
flow
stuff.
Maybe
because
I,
don't
I,
don't
have
the
use
for
now,
but
I
see,
there's
an
interface
and
there's
some
health
information
and
maybe
some
some
help
on
how
to
debug
the
the
the
fainting
targets.
And
for
me
that's
really
interesting.
B
A
Absolutely
we'll
get
in
the
foil
here
in
a
bit,
but
the
you
know,
one
of
our
key
goals
of
flow
is
just
offering
more
inner
information
about,
what's
going
on
in
the
system
in
a
more
easy
to
digest.
Manner
that,
just
simply
you
know,
scattering
through
the
logs
are
even
worse
in
this
case,
where
there
appears
to
be
no
logs
that
are
kind
of
being
spit
out.
So
I
think
there's
a
big
opportunity.
You
know
as
we're
reevaluating
things
to
really.
A
C
I'm,
looking
through
the
issue,
it
seems
like
this
problem
is
coming
from
Upstream,
because
it's
not
that
you
couldn't.
It
seems
like
it's
not
that
you
couldn't
scrape
anything.
It's
just
that
the
discovery
didn't
work
right,
so
it
wasn't
discovering
any
PODS
of
scrape
and
you're
right
like
that
is
hidden
flow,
wouldn't
fix
that
right
now,
but
I
think.
C
A
C
Pascalis,
well,
it's
a
Prometheus
operator
Works.
A
little
bit
differently,
like
Prometheus
operator,
is
using
the
crds
and
it'll
log
things,
but
once
it
reaches
the
Prometheus
SD
level,
the
errors
will
get
hidden.
Actually
in
pascalis.
You
tagged
this
as
operator
or
survey.
Are
you
actually
using
the
operator
to
your
the
foundation
operator.
B
No,
no!
No,
apart
from
that,
we're
not
using
it,
but
but
the
thing
is:
oh,
we
are
instrumenting
a
lot
of
clusters
and
we
are
deploying
the
the
Prometheus
operator
and
promptail
and
different
different
tools,
and
if
it
would
all
be
packaged
as
graphene
agent,
it
would
be
better
and
and
low-key
people
told
me
that
the
way
to
go
is
graphene
agent.
They
were
even
thinking
about
dropping
promptail.
So
so
why
invest
on
prompted
right
now.
C
Right
and
yeah
so,
like
sorry
as
Matt
said
like
one
of
the
things
we
want
to
do
with
flow
is
expose
information
more
directly,
more
easily
to
the
user,
don't
hide
it,
but
this
is
one
specific
case
where,
like
we
need
to
make
an
upstream
change
so
that
we
can
eventually
expose
it
to
users
but
I
think
I
think
I
misunderstood
first
of
what
was
going
on
here
and
now
that
I
have
a
better
idea.
We
can
hopefully
help
resolve
this
more
quickly.
C
C
B
C
Also
I
I
believe
there's
no
danger
of
like
prontel
being
discontinued
or
anything
we're
really
using
promptailing
the
agent,
but
I
think
that
we're
just
gonna
continue
using
that.
Don't
don't
say
that
okay,
okay,
the
Loki
team
have
been
considering
whether
it
makes
sense
to
donate
promptail
code
to
us.
B
A
All
right
does
anyone
else
have
any
new
comments
or
issues
I
want
to
bring
up
if
not
I
will
go
into
a
little
demo
of
kind
of
what
we've
done
with
flow
since
last
time
we
met
as
I'm
going
through
this.
If,
at
any
point
you
have
any
questions
or
comments
or
feedback
feel
free
to
interrupt
me
raise
a
hand.
Put
it
in
chat.
A
A
A
That
was
the
truly
an
exciting
one
for
us,
because
it
pushes
this
concept
that
I
think
probably
dates
back
to
like
February
on
the
initial
talking
about
it,
which
is
kind
of
a
reimagining
of
interfacing
with
the
agent
that
we've
turned
flow
and
we
released
kind
of
the
initial
usable
portion
of
it
yesterday.
After
talking
about
it,
so
that's
super
exciting
and
just
to
bring
everybody
up
to
date.
A
A
It's
consists
of
grafana
agent,
mamir
and
grafana
itself
kind
of
all
running
in
a
Docker
composed,
and
it's
been
running
for
a
few
hours,
and
this
is
the
default
page
you
get
when
you
just
go
to
the
grafana
agent.
In
my
case,
it's
localhost
one,
two,
three,
four
five
and
my
camera
just
died.
Fantastic.
A
A
So
this
is
the
graph,
and
this
is
more
of
a
how
the
components
interact
with
each
other
as
opposed
to
what
we
might
call
a
data
flow
graph,
where
you
would
see
something
like
node
exporter
feeding
to
scrape.
But
this
gives
you
a
very
good
idea
of
how
they
connect.
It
gives
you
a
quick
viewpoint
on
whether
things
are
healthy
or
not,
and
you
kind
of
see
so
we're
just
going
to
click
on
one
of
these.
Let's
do
the
node
exporter
and
it
takes
you
to
this
page.
A
A
What's
going
on,
you
know:
there's
small
little
UI
things
like
this
line
goes
way:
okay
passed
there,
but
it
gives
you
that,
like
ability
to
interact
with
the
state
of
the
system
and
see
what's
going
on
in
a
pretty
good
way,
so
we
have
all
these
arguments
here.
Arguments
are
things
that
are
fed
to
it,
and
then
exports
are
things
that
you
know
are
the
outputs
of
a
component.
A
A
C
Yeah,
so
basically
flow
is
really
like
terraform
inspired,
but
since
grafana
agent
inherently
deals
with
Telemetry
pipelines,
it
doesn't
make
sense
for
us
to
represent
everything
as
like.
A
common
type
like
not
everything
can
be
a
string
or
a
bull
or
a
number.
So
we
need
ways
to
represent
more
complex
types
users
stuff
that
like
they
can
pass
around
things
that
move
data
in
a
pipeline
like
streams
of
data,
but
there's
no
really
way
to
show
that
value
to
the
user.
So
those
are
called
capsules
because
they
encapsulate
something
internal
to
Nirvana
agent.
C
The
Prometheus
receiver
capsule
is
basically
a
stream
of
metrics
coming
into
the
right
ahead
log.
Basically,
the
only
way
for
us
to
show
it
is
just
say
like
hey:
this
is
some
capsule
value,
and
it's
given
like
this
light
gray
font.
So
you
know
that
it
that
it's
like
a
you
know
not
a
real
thing.
You
could
construct
and
config
files,
just
kind
of
read
information.
A
A
C
Yeah,
so
this
is
one
of
the
places
where,
from
the
discussion
earlier,
we
could
expose
those
errors
where
you
would
see
that
one
in
your
pods
isn't
being
scraped
for
some
issue.
There's
usually
a
health
like
an
Airfield
here.
So
if
that
Health
has
down,
there
will
be
an
Airfield
for
that
Target
saying
what
the
scrape
air
was.
C
But
earlier
we
were
talking
about
service,
Discovery,
being
an
issue,
and
in
that
case,
at
the
top
here,
where
you
see
like
the
health,
the
health
message,
we
could
probably
find
a
way
to
move
like
I,
couldn't
even
discover
pods
because
of
some
reason
and
that'd
be
at
the
very
top
of
this
page.
Instead
of
kind
of
in
the
debug
info.
A
And
then
we
can
kind
of
keep
walking
through
this,
like
you
can
go
to
the
relabel
service.
This
is
basically
just
runs
relabel
configs,
we've
renamed
them
rules,
same
exact,
syntax
and
then
finally,
we
have
the
remote
right.
I
should
have
made
this
bigger,
and
this
is
just
pushing
to
mamir
instance.
Now
one
thing
that
I
think
with
flow-
that's
been
a
big
push
for
us
and
shout
out
to
Karen
here
is
helping
us
with
documentation
beforehand.
A
Our
documentation
for
a
lot
of
things
that
were
Prometheus
or
Loki
related,
would
have
a
link
that
was
essentially
go
look
at
their
documentation
and
sometimes
that
could
get
out
of
sync.
So
what
flow
and
has
forced
us
to
bring
in
is
wrapping
all
the
configuration
and
allowing
us
to
put
really
good
documentation
on
top
of
it.
A
You
know
that
save
you
a
Google
Search
and
maybe
that
Google
search
sends
you
to
an
out
of
date
or
wrong
one
by
just
integrating
there.
We
get
some
really
good.
Findability
I'm
gonna
go
through
all
this,
but
in
in
flow
we
have
this
concept
of
attributes
and
blocks.
Attributes
are
kind
of
just
setting
discoverability
good,
that's
a
good
one
and
then
blocks
are
just
kind
of
a
collection
of
settings
or
attributes,
and
they
can
include
other
blocks.
A
Obviously
like
this
kind
of
tree,
not
tree
here
grow
author,
the
child
of
HD
decline,
and
then
we
just
kind
of
identify
everything
with
notes,
anything
that
might
be
tricky.
We're
trying
to
put
here
and
basically
make
it
to
where
you
don't
have
to
go
outside
of
kind
of
this
point
to
get
the
documentation
and
everything's
documented,
and
there
may
not
be
a
one-to-one
kind
of
transference
because
we
may
want.
We
may
found
better
ways
to
wrap
the
configuration
in
a
way
that
makes
more
sense.
A
Just
to
give
a
tease
on
how
the
the
documentation
look
or
the
configuration
looks,
this
is
a
file
that
represents
what
we're
just
looking
at.
As
you
can
see,
it's
very
HCL
terraform
like
and
her
goal
is
to
be
much
more
concise
and
readable
and
usable
than
yaml
you
know
and
I
think
we've
done
a
pretty
good
job
with
that,
any
other
questions
or
comments
so
far.
B
C
C
Imagine
we'll
open
it
up
and
make
it
a
separate
project
for
other
things,
people
to
use,
but
it's
really
going
to
be
up
to
different
teams
if
they
want
to
switch
away
from
the
yaml
based
configs
to
this
I
think
also,
there's
really
a
movement
right
now,
around
kind
of
Q
being
more
and
more
popular.
So
it's
likely
that
if
these
other
projects
do
switch
away
from
yaml
I,
don't
even
think
that
this
might
be
the
best
choice
they
might
want
to
go
for
something
like
you
or
Jason.
C
So
I
don't
know.
I
I
do
get
that.
It
is
different.
I
think
the
the
growing
list
of
things
that
the
agent
does
facilitate
it
or
like,
what's
word,
facilitated,
necessitated
the
need
for
a
a
more
flexible
language
than
what
yaml
usually
provides
so
we'll
have
to
kind
of
revisit
in
the
future.
If,
like
what
problems
it
causes
by
having
something,
that's
just
quite
a
bit
different
than
how
you'd
configure
our
other
things,
yeah.
B
C
Well,
good
news:
one
of
the
things
we're
planning
to
do
is
supporting
pod,
monitor
like
retrieving
pod
monitors
probes,
the
Prometheus
operator
resources
like
reading
those
from
this
config.
So
that
way,
if
you're
using
crds,
you
can
continue
to
use
them
as
is
well
depending
I,
guess
it
depends
on
what
the
crds
are
talking
about
are
but
I'm
talking
about
the
Prometheus
operator,
crds
yeah.
B
B
Okay,
okay,
and
that
that
brings
me
to
to
my
my
second
question
that
I
had
from
the
beginning
today
for
users
that
really
have
a
complex
agent
setup
written
in
yaml
and
would
like
to
moving
to
River.
Is
this
a?
Is
there
an
easy
way
to
migrate,
or
do
we
have
to
rewrite
all
the
configuration,
or
can
we
do
a
little
bit
of
it
or.
C
So
right
now,
because
it's
experimental,
we
didn't
write
any
tooling
to
migrate
once
this
becomes
maybe
maybe
beta
may
be
stable,
but
at
some
point
we're
going
to
provide
tooling
to
convert
the
yaml
config
as
it
is
today
into
this,
and
then
that
way
people
can
just
rely
on
that.
We
just
didn't
write
it
now,
because
things
are
still
changing
pretty
quickly
and
we
don't
want
to
have
to
maintain
that
tool.
While
things
are
unstable,.
C
A
Yeah
and
I
I
think
internally,
we've
we've
built
some
Json
at
scripts
that
essentially
convert
from
yaml
to
flow.
So
that
can
be
a
a
pilot
example
idea.
We
wouldn't
I
found
it,
but
something
to
help
convert,
while
also
you
know
allowing
you
to
proceed
I
think
long,
long
term.
We
we
want
this
to
be
the
future,
but
that's
a
good
ways
away,
and
we
understand
people,
you
know,
have
a
a
large
investment
in
the
current
yaml
script.
A
A
Going
by
my
notes
here,
yeah
and
we
haven't
Pascal-
wrote
a
awesome
blog
post.
That
I
would
recommend
anybody
read
at
that
goes
into
a
little
bit
more
in
depth
and
links
to
some
other,
the
rfcs,
some
other
assorted
documentation.
We
have
we've
been
trying
to
develop
this
as
much
in
the
public
as
possible,
so
there
is
rsc's
or
or
issues
or
you
know,
feel
free
to
jump
in.
We
definitely
want
anybody's
feedback,
and
then
you
may
have
noticed
with
what
we
released.
A
It's
you
know:
node
exporter,
scraper,
relabel
rules
and
remote
right.
So
it's
a
very
you
know,
kind
of
minimum
viable
product.
That's
an
overuse
term
for
metrics,
so
you
can
use
it,
scrape
it
and
use
node
exporter,
but
definitely
we're
bringing
more
things
to
the
table.
As
time
goes
on.
We
just
wanted
to
get
something
out
to
start
soliciting
feedback
so
fleshing
out
metrics
more
so
adding
logs
and
traces.
A
This
has
been
formed
from
you
know.
Our
years
of
working
with
the
agent
and
trying
to
figure
out
like
what
are
the
best
path
forwards
and
how
to
build,
you
know
the
agent
better,
but
we
definitely
want
to
try
to
figure
out
how
to
resolve
all
the
use
cases
that
the
agent
currently
solves.
We
want
to
apply
them
all
to
flow,
maybe
not
the
exact
technical
solution
or
the
exact
way
it
goes
around
it.
But
we
want
to
handle
any
of
the
use
cases
that
agent
currently
does
within
flow
and
now
go
ahead.
C
Want
to
add
the
big
disclaimer
here,
I
think:
okay,
I
mean
I'm,
biased
because
I've
been
working
on
the
agent
the
longest,
but
I
think
that
flow
is
a
really
solid
step
towards
reaching
one
out
and
reaching
like
a
unique
agent.
That's
not
just
a
combination
of
all
these
different
open
source
projects
that
we're
importing
directly.
C
That
being
said
like,
if
the
overwhelming
reception
is
we
hate,
this,
don't
do
this
like
we're
going
to
listen
to
that
feedback
so
far,
I
mean
it's
only
been
less
than
a
day,
but
the
the
reception
seems
between
neutral
and
positive.
So
that's
that's
mostly
a
good
thing,
but
we're
still
paying
attention
to
feedback
and
we're
we're
more
than
happy
to
backpedal
from
this.
If
it
turns
out
that
people
feel
like
it's
the
wrong
direction
for
us
to
take.
C
A
All
right,
if
no
one
has
anything
else,
I'll
go
ahead
and
end
the
call,
and
then
we
will,
if
you're
interested
in
the
recording
it'll
be
posted
up
sometime,
Monday
I
really
appreciate
everybody
coming
out
and
join
us
here
and
letting
us
talk
about
flow.
A
little
bit
love
to
see
you
back
and
I'll
see
everybody
later.