►
From YouTube: Hands On Workshop: Getting Started With Fluent Bit and Metrics With M... Anurag Gupta & Gibbs Cullen
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Hands On Workshop: Getting Started With Fluent Bit and Metrics With M3DB - Anurag Gupta, Calyptia & Gibbs Cullen, Chronosphere
The Fluent Ecosystem has recently added support for Prometheus and Open Metrics. This hands-on session will give attendees a chance to both learn and try out the new capabilities while being guided by a Fluent maintainer and Prometheus community expert. in this session we will cover the following: * An introduction to Fluent ecosystem and metrics * How to spin up an open source metrics backend (M3DB) as well as visualization platform (Grafana) * Setting up Fluent Bit to collect and send metrics to the backend; Viewing and visualizing data that is sent by Fluent Bit * Question and answer section. This session is suitable for learners of all skill levels, and attendees can expect to leave having a better grasp of how these Cloud Native projects can be used together to strengthen observability.
A
Okay,
everyone
welcome
back,
so
this
next
session
is
actually
a
workshop,
so
we
did
fluentcon
in
europe
earlier
this
year
and
one
of
the
big
pieces
of
feedback
that
we
got
was
folks
want
some
more
hands-on
sessions,
be
able
to
get
their
hands
dirty,
try
things
out,
experiment
with
stuff,
and
so
we
thought
let's
go
ahead
and
build
a
session
that
does
two
things.
One
exemplify
some
of
the
new
behavior
we've
added
within
fluent
bit:
two
gets
you
familiar
with
other
cloud
native
technologies
and
three
something
that
you
could
try
out
live.
A
Getting
the
workshop
running
should
only
take
about
five
minutes,
but
then
we'll
demonstrate
some
of
the
capabilities
showcase,
how
you
can
start
to
tune
this
within
your
your
own
site.
So
we'll
we'll
be
watching
the
slack
channel
we'll
be
watching
and
if
you
have
questions
love
to
make
this
more
of
a
discussion
workshop
versus
us
just
just
presenting.
A
A
A
There
we
go
okay,
so
agenda
we're
going
to
walk
through
fluent
prometheus
m3.
Give
you
a
little
bit
of
what
the
hands-on
workshop
overview
looks
like
what
you
might
need
on
your
laptop
and
then
within
the
workshop,
we're
going
to
set
everything
up,
walk
through
it
and,
of
course,
throughout
the
session.
Please,
if
you
have
questions
and
answers
we'll
be
here
and
we'll
be
looking
at
the
slack
channels
as
well.
A
Okay,
so
I
wanted
to
just
talk
a
little
bit
about
flipped
and
metrics.
I
I
know
in
the
previous
session
this
was
mentioned
a
little
bit
about
hey,
metrics
are
brand
new
and
we've
been
doing
some
some,
you
know
great
things
with
them,
but
actually
with
fluent
bit.
One
of
its
main
use
cases
when
it
was
built
in
in
2015
was
for
embedded
linux
things
like
wind
turbines.
A
You
know
other
linux
machines,
things
that
are
embeddable
in
in
your
house,
and
the
first
plugins
were
actually
all
metrics.
There
were
cpu
memory
disk
thermal,
and
these
are
all
what
we
consider
log
base
metrics.
They
essentially
take
json
and
send
that
over
into
your
log
pipelines,
whether
to
an
output
like
elasticsearch,
etc.
A
The
advantages
with
this
is
it's
compatible
with
old
plugins.
It's
also
compatible
with
some
of
the
more
advanced
features
and
functionality
that
fluent
bin
has
like
sql
stream
processing,
but
the
disadvantages
are,
as
we
see
the
evolution
of
this
prometheus
ecosystem.
You
know
they're,
it's
not
compatible
and
it's
also
a
little
more
expensive.
Bite-Wise
right,
you
have
json.
Json
is
a
pretty
heavy
and
expensive
compared
to
prometheus
format.
A
So
with
fluentpit
and
prometheus
in
the
the
integrations
didn't
just
start
with
the
latest
release.
We
do
have
a
lot
of
prometheus
formatted
metrics
within
fluentpit,
and
so
we'll
do
a
quick
intro
for
prometheus
for
those
who
might
not
be
as
familiar
and
fluent
bit.
Internal
metrics
were
exposable
via
prometheus.
You
know,
since
2019
and
before
so
you
can
see
how
many
input
records
you
can
see.
Filtering
records
output,
you
can
get
storage
information,
you
can
get
all
sorts
of
various
internals
all
exposed
in
a
prometheus
format
today
and
with
1.8.
A
A
We
take
the
the
great
work
the
prometheus
no
exporter
team
has
done
and
have
ported
some
of
that
to
fluent
bit
so
we
can
collect
things
like
cpu
network
disk
file
system
stats,
and
then
we
have
metric
outputs.
So
things
like
remote
write,
which
is
more
of
a
push
method
for
sending
that
data
over
to
prometheus
or
prometheus,
compatible
back
ends
and
prometheus
exporter.
So,
if
you're
doing
the
pull
model
in
the
workshop,
we'll
actually
be
running
through
both
of
those
but
yeah.
So,
however,
you
need
to
get
your
prometheus
data.
A
We
can
support
that
and
then
the
formats
for
the
existing
plugins.
So
things
like
you
have
forward
plug-in
where
you
might
be
sending
to
a
fluentd.
We've
made
sure
that
that's
compatible
we've
ensured
things
like
influx
if
you're
starting
to
influx
back
end
with
line
format
and
the
advantage
here
is
it's
very
lightweight
and
it's
convertible
into
multiple
formats
and
the
disadvantage.
A
B
B
It
also
does
metric
congestion,
it's
scrape
based
so
pool
based
metric
congestion.
It's
also
very
efficient
at
metric
storage,
so
very
good
at
storing
metrics
inside
the
instances
themselves,
and
it
does
have
the
ability
to
kind
of
out
of
the
box
do
visualizations
and
graphing
of
metrics.
Most
people
choose
to
use
another
solution
like
grafana,
but
it
does
have
that
capability
as
well,
and
then
you
can
do
kind
of
aggregation
with
prometheus,
using
the
prometheus
recording
rules
and
then
doing
the
alerts
on
your
communities
metrics
as
well.
Using
the
prometheus
alert
manager.
B
And
some
of
the
advantages
of
prometheus,
which
is
which
has
made
it
so
popular
first,
is
that
it's
very
easy
to
get
started
so
having
a
single
binary
for
ingestion,
storage
and
query
and
then
you'd
have
another
single
binary
for
alerting
it's
also,
the
cncf
recommended
monitoring
tool.
So
that's
made
it
have
a
very
large
community
around
it.
B
So
and
that's
you
know
made
that
exposition
format
that
it
uses
widely
accepted,
as
well
as
the
query
language,
prom,
ql
and
some
other
reasons
why
it
has
become
so
popular
is
because
of
its
dynamic
endpoint
discovery
which
can
be
found
on
many
different
platforms,
including
kubernetes,
and
then
it
has
a
very
large
ecosystem
of
exporters
which
are
essentially
just
existing
software
integrations
and
most
major
software
projects,
especially
open
source
ones.
Have
these
existing
integrations
and
they're
huge
lists
of
them
on
on
the
prometheus
docs.
B
So
we're
gonna
and
as
part
of
the
workshop,
we're
gonna
use
m3,
which
is
an
an
open
source
metrics
engine.
It
was
created
back
at
uber
in
2016
to
help
with
their
metrics
monitoring
use
cases
internally
and
now
it's
used
by
many
other
companies,
including
chronosphere,
which
uses
m3
as
its
back
end,
but
just
to
kind
of
quickly
run
through
the
architecture
of
what
m3
looks
like
at
a
high
level.
So
you
have
three
main
components
in
here.
We
have
the
ingest
and
kind
of
down
sampling
tier,
which
is
the
m3
coordinator.
B
Then
we
have
a
distributed
custom
time
series
database
called
m3db
and
then,
on
the
query
side
we
have
an
optimized
query.
Engine
called
m3
query,
so
we'll
be
spinning
up
as
part
of
the
stack
and
instance
of
each
of
these
or
a
node
of
each
of
these,
with
like
a
single
node
of
m3
db
and
then
typically,
how
metrics
are
sent
into
and
out
of
m3
is
we
have
on
the
on
the
right
side
of
things.
B
We
can
accept
both
prometheus
metrics
through
remote
write
and
then
we
also
can
accept
carbon
or
graphite
metrics
as
well
from
statsd,
and
then
on
the
read
side.
You
can
again
send
queries
in
using
graphite
or
promql
through
grafana,
and
then
you
can
also
kind
of
alert
on
your
metrics
via
the
alert
engine
through
the
coordinate
or
through
the
query.
Tier.
B
And
so
yeah,
so
I
think
I
just
kind
of
ran
through
a
lot
of
this,
but
some
of
the
guest
key
benefits
of
m3
were
in
some
of
the
design
choices
around
around.
That
were
to
be
very
large
scale,
and
that's
because
it
was
built
back
in
uber
so
and
you
know
to
beat
that
level
of
of
scale
was.
It
was
a
requirement,
so
it
does
have
the
ability
to
accommodate.
You
know
billions
of
time
series
metrics
per
second.
B
We
do
use
a
reverse
metric
index
inside
the
the
db
tier
and
also
kind
of
a
very
efficient
compression
compression
algorithm,
and
that
kind
of
makes
it
so
your
your
metrics
can
be,
you
know,
retrieved
very
quickly
and
makes
it
very
efficient
for
storage
and
then
yeah
cost
efficiency
as
well.
That
kind
of,
I
guess
that
kind
of
falls
into
that
as
well.
So
just
some
of
the
key,
I
guess
features
of
m3
there,
okay
and
then
that's,
I
think,
that's
all
for
our
slides.
B
A
A
Then
we
will
go
ahead
and
do
docker
compose
up,
launch
everything,
we'll
kind
of
walk
around
make
sure
everyone
has
as
everything
good,
we'll
check
on
the
slack
channel,
for
those
joining
virtually
make
sure
things
are
are
great
and
then
we'll
reconvene
in
like
10
minutes
after
after
that
see
what
we
ran
into
with
with
running
everything.
So,
first,
let's
walk
through
the
configuration
for
flint
bin
and
with
flintbit.
A
This
is
what,
for
those
who
might
not
be
familiar,
what
that
configuration
looks
like
there's
been
a
lot
of
work
to
make
this
very,
very
simple,
easy
to
understand.
High
readability
and
you'll
see
these
big
brackets
that
denote
what
this
might
be
doing.
If
it's
an
input,
it's
an
output
is
it
reading
data?
Is
it
sending
data
and
in
this
case,
we've
added
this
input,
which
is
node
exporter
metrics
we're
going
to
give
it
a
tag.
A
So
all
the
data
within
fluid
bit
even
the
metrics,
are
tagged
with
metadata,
which
then
says
this
is
where
it's
going
to
route
to.
This
is
how
it's
going
to
be
sent
out,
and
then
we
have
two
outputs.
The
first
output
is
for
prometheus
exporter,
so,
if
you're
scraping
from
node
exporter,
you
might
want
to
expose
these
metrics
on
a
port
with
a
listening
address.
You
might
want
to
add
labels,
so
labels
can
really
help
with
the
queries
we'll
show
some
problem.
Ql
queries
here
later
and
then
also
we
support
remote
writes.
A
So
if
you
want
to
go
ahead
and
send
this
to
a
service,
you
know
say
the
m3
service,
that's
running
locally
or
even
other
services
hosted
services.
So
if
you're,
using
things
like
hosted,
prometheus
managed
prometheus
by
vendor,
typically
they'll
accept
prometheus
from
alright
okay.
So
what
we'll
go
ahead
and
do
is
I'll
switch
into
the
root
of
this
directory
after
doing
a
git
clone
and
very
simply,
I'm
just
going
to
do.
Docker
compose
up.
A
And
it's
going
to
go
ahead
and
start
both
m3
provisioner
coordinator,
query,
flintbit
and
grafana,
so
this
should
be
a
very
simple
step,
but
ideally
what
is
happening
is
we
are
running
fluentbit.
We
are
starting
to
capture
those
metrics.
We
are
plugging
in
both
to
the
scraping
and
we're
also
sending
that
data
to
m3
where
it'll
be
searchable
and
accessible
via
grafana.
A
So
we've
gone
ahead
and
run
this
command
here.
We're
going
to
take
a
quick,
10
minute,
walk
around,
make
sure
everything
is,
is
running
well
and
we'll
reconvene
and
we'll
check
the
slack
channel
as
well.
Thank
you
yeah.
So
we
were
walking
around
looked
like
most
folks
got
got
it
situated
again.
This
should
be
really
quick,
so
we've
made
the
workshop,
so
you
can
kind
of
turn
key
and
then
start
doing
some
expiration
after
it
is
hopefully
working.
A
So
here
we
had
run
the
docker
compose
up
about
10,
10
or
so
minutes
ago.
Some
quick
corrections
again
for
those
joining
virtually
the
localhost
and
the
readme
was
30
30.,
it's
actually
3
000.,
and
so
let's
go
ahead
and
move
over
to
that.
Let's
go
ahead
and
log
in
localhost,
3000
and
actually
for
most
folks.
It
should
give
you
a
quick,
username
login.
A
You
can
use
admin
admin,
you
can
reset
the
password
and,
as
part
of
this,
we
wanted
to
use
the
exact
same
node
exporter
dashboard
that
you
typically
find
when
you
use
prometheus.
So
when
you
log
into
grafana,
you
can
come
into
the
dashboards,
you
can
take
a
look
at
them
and
you'll
find
this
node
exporter
full
dashboard
and
then
within
this
you'll
automatically
see
that
data
loaded
up.
So
what
has
happened
is
fluent
bit
again
was
configured
with
the
node
exporter
input
and
on
the
output
side.
A
We
had
two
ways
to
capture
that
data,
whether
you
want
to
scrape
that
data
via
prometheus
exporter
or
you
can
use
the
remote
write
2
to
m3,
and
we
can
look
at
the
last
five
minutes
of
data.
You
can
look
at
what's
going
on
here
and
you
know
your
your
dashboard
is
is
filled
out
here.
So
if
folks
have
you
know
other
metrics
or
types
of
metrics
they
want
to
collect.
This
is
a
area
that
we're
always
looking
to
improve
from
a
community
standpoint
and
then
from
an
ease
of
use
right.
A
Let's
go
ahead
and
just
check
if
there's
any
questions
on
the
slack,
nothing
so
far.
So
next,
what
we'll
showcase
here
is.
Maybe
some
quick,
prom
ql
queries.
Add
some
labels
we'll
pass
it
off
to
to
gives
you
can
maybe
showcase
some
of
that
within
within
graffana?
Maybe
what
you
can
do
with
some
of
this
data
after
it
arrives?
B
B
B
A
There's
a
couple
of
drop
downs
there
where
you
could
say
hey,
I
want
to
look
at
potentially
labels
that
match
a
specific
app
profile,
a
particular
environment,
and
so
the
main
thing
we
wanted
to
just
showcase
here
as
part
of
this
workshop,
is
we're
not
doing
anything
like
too
fancy
nothing.
That's
super
flintbit
specific
that
it
prevents
you
from
utilizing
all
of
the
processes
that
you
might
have
built
on
prometheus
before.
A
So,
if
you
already
have
node
exporter,
you
have
all
the
alerts
defined
out.
You
have
everything
going
for
it,
you'll
be
able
to
quickly
go
ahead
and
basically
plug
fluent
bit
into
that
ecosystem,
and
then,
secondly,
right
if
you
are
expecting
to
traverse
and
explore
the
data,
you
haven't
really
had
too
much
experience
with
with
node
exporter
or
prometheus.
It's
pretty
simple
to
get
started
right.
There's
a
really
large
ecosystem
around
prometheus.
A
Really
large
amount
of
folks
and
tools
out
there
to
kind
of
help
explore
that
data,
so
we're
gonna
go
ahead
and
hang
out
on
the
the
slack
here
for
for
a
little
bit
more
and
just
make
sure
that
things
are
running
well
for
others,
but
other
than
that.
We'll
we'll
go
ahead
and
answer
questions
things
about
metrics
and
then
we
will
be
breaking
here
for
lunch
right
after
so
we're
the
only
thing
that
stands
in
your
way.
A
A
Awesome
awesome
yeah
and
for
the
for
the
recording
yeah
it
was
saying
hey.
I
have
a
lot
of
raspberry
pi.
I've
been
trying
to
do
a
lot
of
note,
exporter
setup
and
what
we
can
do
here
with
with
flintfid
looks,
looks
a
little
better.
So
yeah,
that's
that's
great
news,
and
this
is
something
where
we
want
to
keep
on
building
as
a
community
for
for
the
metrics
integrations.
And
so,
if
you
have
those
requirements,
you
have
those
needs.