►
From YouTube: Hands-on Workshop: Getting started with Fluent Bit and Metrics with P... Anurag Gupta & Gibbs Cullen
Description
Hands-on Workshop: Getting started with Fluent Bit and Metrics with Prometheus - Anurag Gupta, Calyptia & Gibbs Cullen, Chronosphere
The Fluent Ecosystem has recently added support for Prometheus and Open Metrics. This hands-on session will give attendees a chance to both learn and try out the new capabilities while being guided by a Fluent maintainer and Prometheus community expert. In this session we will cover the following:
An introduction to Fluent ecosystem and Prometheus metrics
How to spin up Prometheus as well as visualization platform (Grafana)
Setting up Fluent Bit to collect and send metrics to the backend; Viewing and visualizing data that is sent by Fluent Bit
Question and answer section
This session is suitable for learners of all skill levels, and attendees can expect to leave having a better grasp of how these Cloud Native projects can be used together to strengthen observability.
A
B
A
So
before
we
jump
into
the
workshop,
just
a
quick
primer
on
fluid
bit
and
metrics-
I
know
eduardo
gave-
gave
a
a
little
bit
of
this
background
earlier.
But
actually
one
of
the
fun
facts
is
metrics
were
the
first
plugins
with
fluid
bits.
So
the
first
plugins
that
ever
existed
were
cpu
memory,
disk
thermal
all
geared
towards
embedded
linux
environments
to
iot
environments
and
scraping
that
metric
data,
so
that
it
would
be
usable
by
folks
who
need
it
in
in
their
analytics
environments.
A
Well,
we
took
those
log
based
metrics
and
said
what
is
the
the
main
factor
that
folks
are
using
today
and
it
really
was
open,
metrics
prometheus
and
we
said,
let's
extend
and
and
make
sure
we
can
collect
metrics
in
that
format.
So,
lastly,
we
extended
to
to
open
metrics
based
plugins.
Those
include
prometheus
exporter,
remote
write
and
most
new
in
in
1.9
is
our
prometheus
scraper,
so
making
sure
we
integrate
really
well
with
that
ecosystem
and
throughout
the
workshop,
we're
going
to
go
ahead
and
walk
through
all
of
that.
B
Okay,
so
for
the
for
the
workshop,
this
is
a
kind
of
high
level
overview
of
the
architecture
that
we
have
set
up.
So
you
have
your
instance
of
prometheus
here
on
the
right
side
and
then
instance
of
grafana
on
the
read
side,
and
you
can
see
that
we
have
our
three
tiers
main
tiers
for
m3
set
up
here.
So
you
have
your
coordinator,
which
will
then
send
metrics
over
to
the
m3
db
tier
and
then
for
querying.
You
have
the
m3
query
tier,
which
will
fetch
metrics
from
the
db
tier.
B
A
Yeah,
so
I
went
ahead
and
pasted
the
instruct
link
in
the
slack,
so
you
can
copy
it
real,
quick
here.
If
you
need
it
or
you
know,
get
it
from
the
slack.
If
you
want
to
access
it
immediately,
I'm
going
to
go
ahead
and
click
into
it
and
we're
going
to
just
walk
through
it.
So
we'll
walk
through
this
entire
lab,
but
it'll
be
available
afterwards
as
well.
So
if
you
want
to
try
it
out
there
too,
you
know
happy
happy
to
have
have
you
do
so
there.
A
A
We
also
have
an
instance
of
grafana,
that's
going
to
be
used
for
for
visualization,
and
then
we
also
have
m3
running
there
as
well.
So
first,
what
we're
going
to
do
is
look
at
specifically
at
fluid
bit
and
how
do
we
collect
metrics
with
with
fluid
bit
so
with
fluid
bit
when
we
collect
metrics?
We
use
this.
A
This
idea
of
plug-ins
right,
you
have
an
input
and
you
have
an
output
and
one
of
the
simplest
ways
that
you
can
input
and
output.
Your
metrics
is
by
sending
all
of
that
to
a
standard
output.
So
what
we're
going
to
do
is
run
fluid
bit
via
command
line.
I'm
going
to
go
ahead
and
copy
and
paste
this,
it's
really
small.
So
let
me
see
if
I
can
increase
the
size
here,
and
this
command
is
is
pretty
simple.
I'm
running
fluent
bit.
A
I
have
dash
I
to
indicate
an
input
and
that
input
here
is
going
to
be
node
exporter
metrics.
So
you
can
imagine
this
will
be
metric
similar
to
what
you
would
expect
from
the
prometheus
notice,
no
exporter
and
then
we're
outputting
that
straight
to
standard
out.
So
if
I
go
ahead
and
click
enter,
there
flip,
it
will
start
running
and
it
will
go
scrape
the
metrics
that
it
traditionally
goes
and
gets,
and
it
just
prints
them
out
here
in
the
terminal.
So
again
a
very
simple
way
that
we
can
see.
A
A
And
in
the
next
challenge,
we're
going
to
do
something
just
a
bit
more
complicated.
So
now
what
we're
going
to
do
is
we're
going
to
take
fluent
bit
instead
of
just
outputting
it
to
the
terminal,
because
in
the
real
world
we
can't
just
watch
the
terminal.
What
we're
going
to
do
is
output
that
data
into
a
prometheus
endpoint,
so
to
do
that
we're
going
to
modify
fluid
bit
configuration
and
so
in
the
lab
again
this.
A
This
is
all
there,
so
you
can
access
it
later
too,
but
we
have
this
configuration
file
that
allows
you
to
export
those
metrics
on
a
specific
port.
So
here
what
we've
done
is
we've
said:
hey
we're
going
to
export
our
prometheus
metrics
that
we're
capturing
from
node
exporter
and
we're
going
to
export
them
on
the
port
2021.
A
A
A
So
now
I'm
going
to
go
visualize
that
by
switching
into
grafana
and
we're
going
to
go
ahead
and
use
the
super
secure
password.
I
think
devsecond
is
right
next
door,
so
they'll
they'll
know
how
secure
admin
admin
is
and
we'll
just
skip
all
those
warnings
and
then
within
grafana,
as
you
would
expect
you
have.
You
know,
folks
have
gone
out
in
the
community.
They've
built
these
great
dashboards.
A
A
Last
24
hours
is
probably
a
bit
extreme,
so
let's
just
switch
to
five
minutes
and
five
seconds.
So
you
know
within
you
know
this
super
small
set
of
time
we've
already
gone
ahead.
Taken
those
node
exporter,
metrics
that
we
were
collecting
earlier.
We've
exposed
them
over
a
port,
prometheus,
has
scraped
them
and
now
we're
visualizing
them
just
like
we
would
with
your
traditional
node
exporter.
So
you
can
see
how
easy
it
is
to
kind
of
plug
this
into
the
workflows
that
you
already
have
existing
and
and
basically
get
started.
A
Now,
the
one
thing
I
will
say
as
well
is
you
know
we
capture
a
lot
of
what
node
exporter
does,
but
it's
not
a
100
percent
match
to
match.
There's
still
some
things
there
that
that
are
not
yet
out
of
the
box,
but
we'd
love
for
the
community
to
to
tell
us
hey.
These
are
the
things
that
are
really
important
to
us
and
here's
what
we
would
like
to
see,
so
that
is,
with
prometheus
x,
exporter,
plug-in
and
now
we've
seen
a
lot
of
the
community
switch
to
remote
right
right.
A
So
now,
instead
of
prometheus,
scraping
everything,
a
lot
of
folks
are
switching
to
prometheus
remote
right.
How
does
that
work
versus
just
exposing
stuff
over
port?
So
if
we
go
into
that
challenge,
click
in
here
the
nice
thing
about
this
sandbox
environment,
too,
is,
if
you
already
have
a
service
that
accepts
remote
write.
You
could
try
this
exact
sandbox
with
that
live
service.
So,
if
you're,
using
things
like
chronosphere
or
new
relic
or
logs
io
grafana
cloud,
anything
that
has
that
prometheus
remote
right
endpoint,
you
could
modify
the
configuration
with
your.
A
You
know
your
your
tokens,
your
url,
and
actually
see
what
that
data
would
look
like
in
in
your
live
service.
So
in
this
case
we've
also
also
provisioned
an
m3
instance
for
you
and
with
m3.
You
know
we
we
showed
that
diagram
earlier,
there's
a
lot
of
different
components,
and
I
won't
pretend
I
know
a
lot
about
m3.
But
one
thing
that
that
we
didn't
need
to
make
sure
is
that
m3
has
the
proper
name
space
set
and
that
namespace
is
ready
and
accepting
of
of
remote
write
data.
A
A
So
how
do
we
send
prometheus
remote
right
data?
Is
it
vastly
different
than
when
we
were
exposing
data
over
a
port
earlier?
Actually,
no,
it's
it's
still
very,
very
simple!
So
we're
going
to
go
ahead
and
now
modify
the
configuration
again,
but
this
time
with
the
configuration
you'll
see
another
output
here
where
it
says
prometheus,
remote
right
and
with
prometheus
remote
right,
we're
now
pointing
to
this
host,
which
is
m3.
A
We
have
the
port
there,
and
then
we
have
a
specific
uri.
So
you
can
imagine
if
you're
sending
to
a
specific
service
like
chronosphere
or
fanacloud
or
new
relic,
which
all
our
documentation
has
instructions
for.
Most
of
these
services,
you'll,
put
with
tls
on
https
you'll,
put
a
specific
uri.
You'll
put
a
token
all
these.
All
these
things
are
configurable
there
for
you.
So
I'm
going
to
go
ahead
and
copy
this
and
we'll
go
ahead
and
do
is
we'll
switch
into
configuration,
we'll
switch
into
flimpit.conf.
A
A
A
And
it's
running
so
it's
now
sending
data
over
to
m3.
How
do
we
go
and
check
that
that
data
is
actually
there
we'll
switch
over
to
grafana
again
and
then
within
grafana,
we'll
go
back
to
our
our
dashboards,
we're
going
to
use
the
same
node
exporter
dashboard,
but
this
time
instead
of
using
our
default
data
source,
we're
going
to
switch
to
m3
and
within
m3,
let's
switch
to
last
five
minutes
and
last
five
seconds.
So.
A
A
Okay
and
the
last
challenge
that
we
have
within
this
workshop,
very
easy
again
to
try
is
scraping
a
custom
endpoint
and,
as
I
was
putting
this
this
workshop
together,
I
was
trying
to
think
what
type
of
application
I
could
spin
up.
That
would
be
really
cool
to
showcase
that
has
a
dashboard
and
eventually
it
became
too
much
of
a
hassle
that
I
said,
let
me
just
make
this
a
little
more
meta
and
I
will
I
will
monitor
the
application
of
fluent
bit
itself
so
within
fluent
bit.
A
But
in
this
case
you
know,
say
you
weren't
trying
to
monitor
fluid
bit
itself.
You
want
to
monitor
another
application,
third-party
app,
you
know,
we've
had
folks
do
things
like
hashicorp,
vault,
mysql,
nginx,
all
sorts
of
applications
that
you
want
to
collect
prometheus
data
from
export
it
over
a
different
port
or
remote,
write
it
to
your
service.
A
It
will
go
ahead
and
and
pop
all
these
metrics
out.
So
if
we
zoom
in
to
this,
you
can
see
you
have
the
ability
for
oops
didn't
mean
to
copy
that
you
can
see
hey
we
started
nine
seconds
ago.
This
is
what
we're
capturing
here.
All
the
labels-
and
that
also
brings
me
to
to
a
good
point,
is
if
you
want
to
do
some
more
customization
or
or
additions
on
top
here.
A
lot
of
these
things
include
things.
Like
add:
label
oops.
A
You
can
do
add
label
and
on
top
of
the
exporter.
So
if
you
have
custom
labels
app
labels
that
you
need
all
of
that's
available
in
your
configuration
as
well
so
yeah,
I'm
happy
to
have
folks
run
through
the
workshop
and
we
can
walk
around.
You
know
if
you
run
into
any
issues.
We
have
questions
for
the
next
five
minutes,
but
yeah
again,
very
quick
showcase
of
of
how
to
run
oh
looks
like
something
happened
with
the
with
the
http
client
pulling
from
itself,
but
yeah.
There's
it's
a
really
quick
workshop.
A
It's
going
to
be
available
after
the
conference
as
well!
So
if
you
don't
want
to
run
through,
it
now
feel
free
to
run
through
it
after
and
we're
always
looking
for
improvements.
So
we're
going
to
try
to
build
as
many
of
these
workshops
as
possible
to
make
it
easier
to
learn
it
and
integrate
it.
And,
of
course
it's
it's
accessible
via
third
party
services
too.
So,
if
you
want
to
pump
data
to
your
service,
see
what
it
looks
like
I'd
say:
go
for
it.