►
From YouTube: [Webinar] Fluent Bit Operations & Best Practices
Description
Fluent Bit is a powerful tool that is more than capable of scaling to meet the needs of your mission-critical data and enterprise growth.
In this session, we explore:
1. Agent and aggregator patterns to maximize performance
2. Tuning Fluent Bit for cloud-native environments
3. Best practices in operating Fluent Bit in production
4. Key lessons learned from supporting Fluent Bit on 100k+ servers
A
Okay,
so
a
brief
introduction,
most
of
you
will
not
know
me
so
I'm.
Currently
the
support
leader
at
caliptia.
We
are
the
as
Austin
said,
the
creators
and
core
maintainers
of
fluent
off
the
fluent
ecosystem.
A
I've
been
working
on
engineering
and
in
support
a
bit
over
21
years
already
and
I
joined
calypsia
a
bit
over
a
year
and
I
have
been
having
a
lot
of
fun,
dealing
with
issues
and
use
cases
from
some
Fortune
10
customers
that
actually
have
deployments
over
100k
servers
or
nodes.
A
So
I,
it's
been
a
a
pretty
nice
trip
so
far,
also
on
the
open
source
side,
I've
been
working
with
the
documentation
and
issue
triage
in
the
in
the
GitHub,
repo
and
I'm,
also
active
on
slack,
along
with
Pat,
the
anurag
and
some
other
folks
and
and
from
time
to
time,
going
to
or
stack
Overflow.
That's
not
a
huge
community
in
our
case,
but
still
there
are
some
interesting
questions
there.
So
if
you're
around
you
have
any
further
questions
you
can
reach
out
there
directly.
My
handle
will
be
lecados
on
those
places.
A
Okay.
So
what
are
we
going
to
talk
about
today?
What
we
are
trying
to
to
share
with
you
today,
first
of
all
common
architecture,
parents
for
the
deployment
of
fluid
how
to.
B
A
These
once
you
are
in
production
and
a
little
bit
on
tuning
flame
beat
for
Max
performance
and
to
to
avoid
unknown
issues,
I'll
be
done
troubleshooting
once
you
are
in
production,
some
some
tips
and
tricks
from
supporting
these
large
deployments
and
a
couple
best
practices
when
deploying
in
production
that
we
have
learned
by
supporting
these
large
deployments
of
our
users
and
customers
here
in
California,
so
before
diving,
deep
into
that,
a
little
introduction
to
flu
and
beat
for
those
that
may
not
know
follow
me
that
well
so
fluency
started
as
a
as
a
log
collector
a
long
time
ago,
but
then
we
have
added
metrics
and
traces,
so
it
is
now
a
full
Telemetry
agent
and
and
fully
open
source.
A
Of
course,
these
these
deployments
allow
users
to
have
a
very
high
performance
these
compared
to
the
the
very
good
performance
that
flindy
has
in
the
past,
or
still
has,
but
now
it's
from
mid
here
around.
So
it's
high
high
performance
with
millions
of
messages
per
second
being
delivered
from
point
A
to
point.
B
body
is
not
just
a
simple
moving
data
from
A
to
B
right.
A
We
also
allow
the
users
to
filter
to
further
process
the
the
log
records
to
enrich
these,
or
even
to
drop
data
that
you
don't
need
in
in
your
data
backhand
data
that
it's
not
relevant
for
your
troubleshooting
or
for
your
auditing
need.
So
that's
something
that
is
very
useful
too.
A
So
far.
This
has
been
deployed
over
8
billion
times
only
from
the
docker
half.
So
we
are
not
counting
those
where
a
user,
a
company
May
download
only
once
and
then
download
from
their
internal
Repository.
A
So
this
is
a
a
very
exciting
thing
to
share
with
you.
A
We
know
that
there
is
a
huge
Community
behind
this
and
we
appreciate
all
your
your
feedback
that
you
provide
through
opening
tickets
and,
of
course,
opening
PRS
on
the
public
GitHub
a
little
bit
on
this
8
billion,
the
we
have
seen
how
the
the
fluid
adoption
has
increased
since
March
2022,
and
now
we
should
be
already
close
to
the
9
billion
downloads
and-
and
you
can
see
there,
that
the
the
adopters
are
big
cloud
providers
and
observability
providers
as
well.
A
So
how
does
flowbit
work
in?
In
general
terms,
we
have
sources
from
where
you
are
going
to
graph
your
data,
your
logs,
your
metrics,
your
traces
and
you
have
at
the
B
part.
You
have
the
things
where
you're
going
to
move
all
your
data
and
and
in
between
you,
you
can
transform
this
data,
as
I
said
before,
and-
and
you
can
do
this
from
any
number
of
sources
and
then
move
this
data
to
again
any
number
of
destinations
or
syncs
in
between
you.
A
Have
your
data
save
by
by
providing
a
a
mechanisms
to
to
deal
with
a
problem
back
pressure
or
when
your
service
is
not
available.
A
So
you
can
have
different
configurations
to
to
keep
your
data
when,
when
that's
needed,
when
you
deploy
to
a
kubernetes
containers
with
a
single
demon
set
deploy
there,
you
can
graph
all
the
logs
metrics
and
traces
that
that
you
are
creating
in
in
that
particular
host
and
and
then
you
can
of
course,
start
working
over
that
we
have
support
for
Lua
and
Wasim
for
even
a
more
complex
processing.
A
A
So,
as
I
was
saying,
this
could
be
one
engine
to
to
rule
them
all
so
and
and
part
of
these
plugins
that
we
have
are
generic
tcpe
and
HTTP
inputs
and
outputs
that
you
can
easily
connect
to
your
legacy
sources
of
information
that
only
know
how
to
solve
TCP
or
HTTP
or
even
new
new
destinations
that
you
may
need
to
move
your
data
to
and
also
you
will
have
a
specific
vendor
or
technology.
A
Plugins
like
like
Splunk
like
open
search,
elastic,
Azure
log,
analytics,
datadog,
Google,
Chronicle
Etc,
you
have
the
flexibility
to
send
to
any
of
those
with
specialized
plugins
and,
of
course,
we
we
work
with
common
data
structures
if
you
need
to,
for
example,
send
on
gizi
GC
format.
A
A
A
Karen,
will
you
please
okay,
so
we're
gonna
now
talk
about
the
common
architecture,
parents
that
flu
and
B
that
is
has
been
deployed?
We
have
seen
this
through
our
experience
with
users
and
with
customers
and
we're
gonna
talk
about
two
of
these
more
common
architecture
patterns.
A
The
most
widely
adopted
is
as
a
collection
agent,
where
you
will
have
one
agent
in
in
one
node,
and
this
Asian
will
collect
the
information
from
all
your
applications
and
also
from
the
operating
system
and
from
whatever
source
that
you
may
have
there.
A
If
you
also
need
your
the
information
from
your
from
your
nose
like
using
no
exporter
flow
and
bead
can
also
collect
that
and
then
send
it
out
to
to
whatever
thing
that
units
and
that
information
right
this
this
agent
is
the
the
equivalent
of
sidecar
container
on
on
kubernetes
there's
some
advantages
and
disadvantages
on
using
disparing
or
or
other
on
this
case
you,
you
can
have
all
the
processing
done
on
this
note
and
then
send
that
information
to
your
to
your
things
already
processed
already
enriched
or
already
filter
out.
A
So
this
should
be
a
pattern
where
you
will
have
the
the
processing
completely
here
in
the
in
the
agent,
and
it
will
come
with
the
with
the
same
advantages
and
disadvantages
of
having
a
sidecar
right.
So
it
will
consume
resources
from
your
path.
So
you
will
notice
the
the
resources
consumption.
If,
if
it
is
consuming
a
lot
of
resources,
you
will
notice
in
the
in
the
Pod
performance.
A
And
this
these
sidecars
could
raise
against
the
the
application
pod
right.
So
if
one
started
before
then
the
other
there
could
not
read
all
the
logs
Etc
some
some
things
to
to
keep
in
mind.
Usually
it
will.
It
will
work
fine,
but
it's
something
to
keep
in
mind
when
you
start
seeing
unexpected
behaviors
right
and
then
the
the
other
pattern
will
be
using
as
an
aggregator.
A
So
on
this
pattern
you
will
have
again
one
Asian
per
node,
but
the
processing
that
you
will
do
in
the
node
will
be
minimal.
These.
This
is
a
kind
of
centralized
processing
where
you
will
have
these.
This
fluent
bit
agents
like
forwarders
of
the
data
only
so
they
will
collect
and
immediately
or
with
a
very
low
processing,
send
the
data
to
your
aggregator
and
your
aggregator
could
be
one
or
more
instances
of
flu
and
beat
or
even
fluency.
A
So
this
this
will
allow
by
moving
your
process
into
the
aggregator,
you
will
have
a
Max
throughput
here
at
the
origin
of
your
data
and,
of
course
the
resource
utilization
will
be
will
be
really
low.
Bloombeat
is
already
lightweight
compared
to
fluency.
A
So
only
with
with
this,
with
these
improvements-
and
with
this
pattern
you
you
can
maximize
the
the
throughput
from
the
origin
to
your
aggregator
and
then
you
can
have
here
all
the
all
the
configuration
that
you're
required
to
preserve
your
data
to
further
processing
Etc.
So
the
the
actual
processing
power
here
in
the
aggregator
will
be
able
to
escalate
independently
right.
A
If
you
need
further
power,
you
will
not
have
to
do
any
modifications
in
your
Source,
but
you
will
grow
here
as
desired
on
only
in
the
aggregator
side,
yeah,
and-
and
this
of
course
will
require
to
have
more
power
here
right
it.
It
is
a
violence
if
you
have
deal
with
performance
issues,
if
you
have
deal
in
the
past
with
tuning
scenarios,
you
know
that
it
is
a
concession
game
and
where
you
try
to
balance
what
your,
what
you're
improving
and
what
you
can
lose
because
of
that
Improvement.
A
So
it
is
the
same
here.
The
resources
are
limited,
so
we
have
to
balance
all
these
variables
and
and
yeah.
The
single
point
of
configuration
here
will
allow
you
to
do
modifications
as
needed
and
to
not
have
to
restart
or
move
your
configuration
at
the
original
of
the
data
and,
of
course
again
as
fluenbid
is
Bender
agnostic
or
vendor
neutral
agent.
A
If
you
decide
to
change
your
sink
or
to
add
a
new
one
or
even
keep
sending
to
different
endpoints
at
the
same
time
say,
for
example,
you
will
send
your
the
debug
level
logs
records
to
cold
storage,
because
you
don't
need
it
immediately,
but
you
may
need
them
later
or
if
you
want
to
send
other
live
information
to
any
other
service
lab.
You
only
send
that
information
that
you
require,
and
you
will
change
only
here
in
the
aggregator
component
of
your
architecture,.
B
Yeah
I
was
just
about
to
send
a
quick
response.
He
was
asking
about
extensions,
but
I'll
I'll
respond
directly
in.
A
The
in
the
chat-
okay,
okay,
okay,
I'm
gonna,
move
then
to
monitor
influence
in
in
production,
so
fluid
allow
us
to
to
send
logs
matches
and
traces
to
to
have
an
overview
to
observe
our
systems
right,
but
we
also
need
to
to
observe
the
The
Observer
here
to
monitor
flu
embed
and
and
for
that
we
have.
We
have
exposed
some
metrics.
We
exposed
these
metrics
like
uptime.
A
We
expose
metrics
per
plugin
instance
about
the
the
breakers
process
about
the
use
memory,
the
bytes
process.
We
publish
both,
so
you
will
see
number
of
Records
processed
number
of
bytes
of
the
records
processed.
A
We
also
provide
metrics
about
the
filters
because
the
filters
could
add
or
remove
records
and
and
also
how
how
much
data
we
have
processed
in
terms
of
bytes
and
in
terms
of
records,
and
we
also
have
metrics
about
this
storage,
which
will
become
important
as
we
progress
here,
we'll
see
what
the
storage
has
to
do
with
this,
but
those
metrics
are
used
to
be
in
the
past
in
another
HTTP
endpoint.
Now,
with
the
API
version
2.,
all
the
metrics
are
in
the
same
endpoint.
You
can
consume
them
from
one
endpoint.
Only.
A
Estimate
that
over
90
percent
of
the
users
will
use
Prometheus,
so
we
publish
our
metrics
using
Json
and
Prometheus
formats.
You
can.
You
can
have
both
if
you,
depending
on
what
you
want
to
do
with
them
right,
but,
for
example,
for
Prometheus
we
can
scrape
the
the
data
from
a
single
host
or
a
single
pod
with
a
really
a
basic
scrape
configuration
in
there.
And
you
will.
You
will
query
this
this
path
in
the
server
and
flu
and
bit
exposes
this
metrics
to
an
HTTP
server,
the
the
default
Port.
A
A
For
these,
you
will
have
to
make
some
configuration
in
both
in
the
following
bit:
demon
set
configuration,
which
are
these
annotations,
that
you
can
see
on
the
screen
and,
of
course
you
will
also
have
to
configure
your
Prometheus
deployment
on
that
same
cluster
like
provide
the
permissions,
the
relabeling
that
you
may
need
Etc.
This
is
a
a
well-known
pattern
to
to
gather
information
from
floam
bead,
so
there
are
a
couple
blog
posts
on
the
internet.
A
If
you
are
interested
on
configuring
this,
and
also
as
fluenbid
may
speak,
a
lot
of
languages,
let's
say
fluent
bit,
can
also
write
to
your
Prometheus
instance.
You
can
use
the
output
plugin
promise,
use
remote
right
and
send
your
metrics
from
your
phone
bit
instances
to
to
your
server
and,
of
course
you
can.
As
shown
on
this
example,
you
can
add
your
Bureau
labels
to
that
data
that
you
are
sending.
A
Okay,
so
with
this
Matrix
in
place,
once
we,
we
have
already
started
to
to
watch
our
metrics
of
flu
and
bit
there's
some
patterns
that
we
can
look
for
and
see
if
fluid
is
presenting
any
issues
before
we
receive
those
calls
that
we
are
not
expecting
from
our
users
right.
A
So
one
of
the
important
patterns
to
look
out
for
will
be
the
back
pressure.
So
what
is
the
back
pressures
the?
Basically
we
can
Define
these
when
in
in
certain
scenarios,
your
ingestion
capacity
is
bigger
than
the
the
throughput
that
you
could
have
in
your
output
side.
So
let's
say
you,
you
are
able
to
ingest
50k
records
per
second,
but
your
data
endpoint
is
only
capable
of
receiving
30k
per
second
or
even
when
your
endpoint
is
down.
So
what
happens
here?
How
can
we
detect
this
and
what
happens
at
fluency?
A
Is
that,
as
it
is
start
to
retrying,
to
send
this
this
data
to
your
endpoint,
which
is
not
available
or
not
capable
of
cope
with
with
the
Drupal
that
you're
trying
to
send
a
fluid
buffering
mechanism,
will
start
consuming
more
resources,
of
course,
because
it
has
to
deal
with
this
back
pressure
in
place.
A
So,
basically,
you
could
see
if
you
are
consuming
your
data
from
the
the
metrics,
something
like
this.
The
the
first
the
first
chart
is
for
input
records,
so
you
can
really
see
at
about
6K
per
second
there
in
the
input
and
it
was
similar
on
the
output,
but
then
in
the
in
the
third
one
you
can
see
the
difference
between
both
right.
A
A
Where
we're
sending
this
this
information
and
and
yeah,
we
can
deal
with
this
using
a
file
system
storage
mechanisms
that
mechanism
that's
the
most
recommended
we
are
going
to
so
it
could
be
more
about
that
later
and
once
you,
you
see
this
back
pressure
in
your
monitoring
chart,
you
should
make
some
questions
to
see
if
you
need
to
actually
deal
with
it
or
how
to
deal
with
it,
it
could
be
that
you
have
peaks
of
data
ingestion
or
that
the
the
end
point
is
down
or
how
the
resources
look
in
your
in
your
node,
in
your
server,
where
fluent
bead
is
running,
probably
is
having
a
different
kind
of
issues
that
is
not
allowing
to
to
send
the
data
to
your
endpoint,
sync
and
another
part
to
to
look
out
for
will
be
the
retries.
A
For
this.
We
also
have
metrics,
you
will
will
have
in
the
flu
and
bit
Matrix
the
retries
per
output
plugin
and
what
is
a
retry?
Basically,
when,
when
you
are
sending
your
data
to
your
endpoint,
fluent
B
will
try
to
flash
this
this
data
to
your
endpoint
right
and
then
there
could
be
three
answers
to
this
operation.
One
could
will
be
okay,
meaning,
yes,
I
took
this
chunk
of
data,
I
send
it
over,
and
the
server
replied
with
an
okay,
200
right
or
2001.
A
Whatever
the
server
responds
and
the
other
will
be
an
error,
it
means
that
for
some
reason,
I
send
something
that
is
wrong
from
my
side
and
the
the
service
rejected,
and
it
rejected
it,
because
there
was
something
wrong
with
this
data.
It
could
be
a
format,
it
could
be
an
index
issue
in
your
in
your
data
back
end
or
any
other
reason
that
makes
this
shank
non-retrial
okay.
So
it's
an
error.
I
will
not
retry
this
and
the
third
answer
could
be.
A
Yes,
there
was
an
error
while
sending
the
information,
but
I
can
retry.
This
is,
for
example,
when
the
end
point
is
down
or
when
the
end
point
returns,
500
there
or
or
even
when
the
server
not
responded
with
the
500,
but
with
a
400
saying
that
hey
I'm
receiving
too
too
much
connections
at
this
time
too
much
data
coming
my
way
so
I
cannot
serve
you
now.
So
this
will
be
the
the
three
and
and
in
the
case
of
the
third
one,
when
I
can
retry
to
send
this
chunk.
A
Flume
bit
will
schedule
retry
task
so
internally,
the
the
fluent
bit
engine
will
ask
the
the
scheduler
to
to
a
schedule,
a
retry
task,
and
it
will
be
retried
in
the
future.
Okay
and
yeah,
and
the
other
will
be
the
the
drop
records.
A
Disrupt
records
are
counted
when
I
have
records
that
are
not
longer
retrievable.
This
could
happen
because
of
this
400
errors
right
where
I
send
in
proper
data
to
the
to
the
endpoint
or
when
the
retries
were
exhausted.
You
can
tell
who
embed
hey
if
the
end
point
replies
with
with
an
error
which
is
retrievable,
then
retry
three
times
five
times
or
retry
forever
or
no
retries
at
all.
A
So
in
any
of
these
cases
on
the
retrievable
that
already
exhausted
the
retries
or
when
in
in
the
case
of
an
irrecoverable
error,
we
will
count
the
drop
records
and
then
yeah
you
will
have
to
start
looking
into
the
logs
into
the
the
flu
and
bead
logs.
What
is
going
on
here?
A
So
these
are
the
the
both
common
parents
that
we
have
reserved.
The
reason
for
that
are
are
multiple
are,
as
mentioned
before,
that
the
endpoint
is
not
available,
or
we
made
some
errors
in
the
in
the
formatting
or
even
in
the
index
Creation
in
our
data
endpoint
I'm,
seeing
more
questions
sustained
yeah.
B
There's
a
there
was
a
quick
question
on
what
logging
is
expected
for:
retries
dropped
Etc,
and
then
there
was
a
follow-up
question
to
that.
That
was
asked
before
by
John.
B
I
believe
we
we
just
covered
some
of
it
actually,
but
for
the
question
yeah,
the
has
the
V2,
metrics
and
Prometheus
endpoint
been
tested
against
Telegraph
John
said
that
he
recently
tried
using
it,
but
got
an
error
about
duplicate
help
statements.
A
Okay,
no
I
I,
don't
have
this!
That
I
think
is
a
good
idea
to
to
reach
out
to
GitHub
or
to
Slug,
and
we
can
take
a
look
into
that
that
that's
interesting
I
haven't
used
that
against
with
Telegraph.
B
Yeah
kamola
we'll
follow
up
with
you
afterwards
and
see
if
we
can
help
help
with
that
error.
A
A
A
Yeah,
okay,
a
little
bit
on
tuning
for
a
bit
on
this
Cloud
native
environment.
A
Before
going
into
that
a
brief
overview
of
Alpha
limit
works
right,
so
we
have
multiple
sources
that
we
can
graph
data
from
then
in
the
middle,
is
flu
and
bit
processing
and
and
then
send
into
two
different
endpoints
and
to
to
do
these
flu
and
bit
will
instead
of
just
saving
the
the
Json,
the
most
common
format
out
there.
A
Instead
of
saving
this
data
through
the
processing
through
the
pipeline,
it
will
use
message
pack
that
that
we
use
to,
of
course,
say
space
and
and
the
one
that
we
use
during
the
whole
pipeline.
So
fluent
bead
will
have
the
ingestion
size,
the
input
plugins
then
we'll
have
the
filtering,
then
a
buffer,
and
then
it
will
route
the
data
to
the
different
endpoints
and
and
in
the
middle.
Then
it
will
use
message
pack
to
to
save
the
data.
As
you
can
see
there,
there
are
savings.
A
These
are
simple
examples
of
how
message
spark
use
less
space
than
Json.
So
this
is
what
what
we
use
for
data
serialization
inside
flu,
embed
and
and
then
once
we
we
have
this
data.
Converted
to
this
format.
We
will
have
our
shank
files,
sorry
chunks,
which
are
also
chunk
files
if
you
use
the
file
system
storage,
but
the
the
base
structure
for
this
is
a
chunk.
A
So
a
shank
will
be
a
group
of
events
that
belong
to
the
to
the
same
tag
right
you'll,
do
processing,
you
will
convert
it
to
message
pack
and
you
will
put
it
into
into
the
chunk
and
if
there
is
any
other
filter
in
the
pipeline,
it
will
repeat
this
operation
will
unpack
from
message
pack
we'll
do
what
it
requires
to
do
with
this
event
and
then
we'll
pack,
it
back
into
message
pack.
A
So
the
the
the
format
that
fluent
use
will
be
this
I
shank
with
attack
with
with
the
records,
and
we
made
that
data
as
well
inside
it.
A
So
when
flambeat
is
working
with
this,
it
will
have
chunks
of
two
megabytes.
This
is
kind
of
a
soft
limit,
so
let's
say
that
from
those
two
Megs
you,
you
have
only
15
kilobytes
free,
but
your
next
event
is
of
20
kilowatt,
so
it
will
save
it
to
the
to
the
same
shank,
most
probably
so,
it
will
move
a
bit
over
the
limit,
but
the
the
soft
limit
for
this
is
of
two
megabytes.
A
This
is
something
hard-coded
on
fluid,
not
a
configuration
parameter
yet,
and
film
bit
will
process
this
data
and
we'll
append
to
shank
based
on
this
buffer
chunk
size.
This
is
configuration
setting
that
the
users
may
change.
We
have
basically
to
two
parameters.
There
they've
offered
chunk
size
and
also
the
max
size,
but
these
are
the
units
that
will
will
tell
fluid
how
much
data
at
a
time
it
will
process
and
all
this
processing
is
done
in
memory.
A
Even
if
you
have
a
file
system
back
storage,
it
will
do
all
the
all
the
processing
in
memory
fluent
bead
has
this
concept
of
having
chunks
up
and
chunks
down.
So,
while
flu
and
bit
is
working
with
a
shank,
it
will
be
up
in
the
state
of
up.
It
means
it
is
in
memory
right,
not
not
on
the
disk.
In
case
you,
you
are
using
file
system,
storage
and
once
fluid
is
done
with
that,
it
will
put
it
down
right.
A
That
could
not
be
immediate,
because
fluid
will
then
pass
this
instruction
to
the
kernel
and
the
kernel
may
decide
to
do
it
later
on
time,
but
this
is
how
a
fluent
bit
works
and
how
it
treats
this.
This
chunk
structures.
B
And
Lucario
there's
a
quick
question:
if
messages
are
over
two
megabytes,
is
there
any
issues
with
the
chunking
process.
A
Okay,
yeah
good
question,
so
what
will
happen
is
that
you
will
have
chunks
of
over
that
size.
Okay,
fluent
bead
will
not
cut
it
out.
The
the
option
where
the
the
event
or
the
log
event
could
be
carried
out
will
be
or
left
out
is,
if
you,
if
you
use,
skip
long
lines.
Okay,
that's
a
parameter
that
you
may
use
for
that.
But
if
you
you're
not
skipping
long
lines-
and
you
have
a
message
over
to
Max,
the
the
chunk
will
grow
enough
to
to
allocate
that
log
event.
A
And
yes,
as
Ryan,
says
there
in
the
question,
yeah
it's
about
pattern
and-
and
you
should
try
to
to
troubleshoot
that
to
know
why
you're
having
such
big
log
events,
we
have
seen
even
funny
stuff
there.
We
have
seen
like
a
full
PDF
files
written
to
the
log
file,
so
yeah,
it's
something
that
you
need
to
to
look
into.
B
I'll
I'll
grab
that
one
and
answer
it
in.
B
There's
another
one
from
Francisco,
though,
so
what
is
the
use
of
using
different
sterilization
mechanisms
with
fluent
bet
pipeline,
since
it
will
again
be
destarily
or
de-serialized
at
the
output?
Is
it
to
reduce
memory
usage
of
float
bit.
A
Yeah
exactly
as
we
mentioned
before,
throughmb
was
born
after
the
experience
with
fluency
right.
Fluency
was
written
in
Ruby
and
with
a
with
a
bigger
footprint
on
the
system,
and
now
yes,
fluent
bead
is
a
lightweight
Telemetry
agent.
So
yeah,
that's
one
of
the
reasons
we
are
using
a
message
pack
to
actually
compress
the
data
yeah
thanks
for
that
question.
Francisco.
A
Okay,
so
some
settings
that
flew
me
disposes
to
to
deal
with
this
will
be
the
the
aforementioned
buffer
chunk
size.
It
means
flow
and
bead
will
process
this
amount
of
data
each
time
for
each
file.
Okay,
this
thinking
that
we
are
using
the
the
tail
input
plugin.
This
configuration
parameters
are
per
file,
so
per
file
fluid
will
process
this
amount
of
data,
this
amount
defined
by
buffer
chunk
size
and
the
buffer
Max
size.
A
So
it
will
pause
the
ingestion
if
this
limit
is
Rich
and
you
are
not
using
their
file
system
storage
and
then
you
have
a
storage
limits,
mostly
at
the
at
the
output
side
and
and
also
on
the
on
the
input
side.
But
this
could
be.
These
have
been
proven
more
useful
in
the
in
the
output.
A
So
in
the
in
the
input
side,
you
can
improve
performance
by
enabling
threaded
input,
plugins,
slow
and
beat
before
these
run
in
the
the
input.
A
In
the
main
thread
of
the
application,
so
of
course,
if
you
have
Parts
in
there
or
any
other
operations,
or
you
have
a
lot
of
input
instances,
yes,
you
will
notice
that
that
the
ingestion
was
not
as
performant,
but
now
you
can
Mark
a
input
plugin
as
threaded,
and
then
it
will
create
a
different
thread
or
quarantine
to
to
process
the
the
input
instance
and
recently
we
have
introduced
processors
that
will
allow
to
have
more
filters
in
the
in
the
input
side,
not
not
only,
for
example,
a
parser
that
you
can
Define
on
the
tail
plugin,
but
you
can
put
there
any
any
other
filter
that
you
you
may
need,
and
those
will
run
on
this
on
this
thread,
not
on
the
on
the
main
thread
and
in
the
output
size.
A
You
have
workers
fluid
used
to
run
again
in.
It
was
a
different
thread,
but
still
only
one
thread,
but
now
you
with
the
workers
configuration
parameters,
you
can
tell
fluent
B2
open,
let's
say
eight
different
workers
for
the
same
output
plugin
and
they
will
run
on
their
own
on
their
own
thread
and
you
also
have
the
processors
there
I
think
anurac
talk
about
this
in
on
the
previous
webinar.
A
A
So
these
this
could
help
you
if
you
were
doing
rewrite
tag
or
sending
from
one
pipeline
to
to
the
other
for
further
processing
before
sending
to
your
data
endpoint.
So
now
you
can
do
this
at
the
end
of
your
pipeline
right.
So
do
the
common
processing
from
the
left
to
right,
and
only
once
you
are
at
the
output
you
can
yeah
do
further
processing.
A
Let's
say
you
only
want
to
send
the
in
for
log
level
data
to
one
output
yeah,
you
can
filter
those
out
and
only
send
what
you
require
and
in
the
output
the
storage
limit.
This
is
to
deal
with
with
the
back
pressure
right,
how
much
disk
or
will
will
you
use
to
deal
with
this?
Will
you
allow
losing
records
Etc,
so
this
will
allows
you
to
to
say
how
much
you
want
to
use
from
your
from
your
system.
A
We
know
that
some
users
even
will
not
use
any
disk
at
all
and
we'll
only
want
to
have
something
in
memory
right.
So
so
you
can.
You
can
manage
this
settings
in
the
output
side
to
to
limit
the
storage
usage.
B
A
Okay,
good
questions
yeah.
Well,
actually
they,
this
sequence
of
the
the
log
events,
is
not
guaranteed
when,
for
example,
you
have
retries
right
if,
if
you
have
retries
and
and
one
new
chunk,
which
has
new
log
events,
if
that
one
succeeds
before
the
retry
of
the
previous
one
yeah,
they
will
not
be
in
order,
so
in
general
we
do
not
guarantee
the
the
order
there.
Fortunately,
thanks
for
that
question,.
A
Okay
and
and
also
related
to
this
compacting
data
on
on
fluid.
A
If
you
have
multiple
filters
running
in
in
your
pipeline
for
for
your
pipelines
in
your
configuration,
this
could
degrade
the
performance
because
it
will
be
constantly
decompressing
and
compressing
again
from
Json
to
to
message
pack
and
and
for
these,
what
we
proposed
was
to
use
Lua,
so,
instead
of
having
one
filter
after
the
other,
you
will
have
only
one
Lua
script
that
will
do
all
the
Transformations
required
by
your
pipeline.
We
did
in
in
conjunction
with
with
one
of
our
partners.
A
They
did
a
lot
of
testing
on
these.
They
they
have
amazing
images
about
that,
and-
and
they
proved
this
and
and
we
did
some
testing
and
yeah.
This
is
the
the
recommendation
that
we
can
do
now
to.
If
you
have
multiple
filters
there,
it's
better
to
use
one
Lua
script
to
to
do
all
the
Transformations
required.
A
If
you
miss
the
the
previous
webinar
there
Thiago
our
Lua
expert,
he
he
mentioned
this
way
and
he's
also
around
on
the
slack
and
on
GitHub.
If
you,
if
you
have
further
questions
about
the
use
of
Lua-
and
we
have
this
multi-line
support-
and
we
have
basically
two
ways
of
supporting
these
one-
is
the
built-in
multi-line
parsers-
we
have
a
parser
for
python,
go
Java
and,
and
you
can
use
them
right
in
the
in
the
tail
input
plugin
and
in
in
the
other
case.
A
Another
scenario
you
can
use
your
own
custom,
multi-line
parsers
there
you
will
Define
your
multi-line,
parser
or
regular
Expressions
right.
You
will
first
Define
the
start
state
which
is
actually
required.
That
does
must
be
the
state
name
and,
and
you
define
yeah
how
the
the
first
line
of
a
multi-line
message
will
look
into
this
log
that
we
are
gathering
right
and
then
I
could
look
the
second
one
and
and
so
on.
These
are
regular
Expressions.
A
So
you
will
not
go
over
each
line
of
of
your
multi-line
parses,
but
you
will
have
to
Define
them
to
to
match
with
the
with
the
body
of
it
right
this
being
open
source.
You
can
take
a
look
into
the
into
the
code
how
the
different
multi-line,
built-in
multi-line
parsers
are
defined
and,
of
course
we
are
there
if
you,
if
you
need
help
with
with
it
through
the
community
and
long
lines.
We
mentioned
that
with
the
with
the
previous
question.
A
You
can
skip
long
lines
if,
for
some
reason
there
is
no
end-of-line
character
in
your
log
record.
You
can
skip
that
if
it's
too
too
long-
and
we
have
also
seen
one
issue-
called
the
clubbing
issue
where
we
see
that
records
that
are
supposed
to
be
or
not
multi-line
are
growing
immensely
right.
We
are,
we
have
detected
logs
over
5
Max,
and
that
was
because
the
the
regular
expression
was
not
correctly
defined.
A
We
used
to
recommend
the
the
use
of
rubler.com
to
to
test
your
regular
expressions
or
or
even
we
are.
We
have
seen
some
users
and
and
part
of
the
team
asking
to
a
AI
shot,
whether
or
not
the
the
regular
expression
is
is
good
or
if
it
is
optimal.
So
that's
something
to
to
look
into
as
well.
A
Okay,
a
little
on
the
troubleshooting
in
in
production.
These
are
common
things
that
we
see
in
both
their
community
and
some
of
our
customers.
So
let's
say
that
your
data
is
not
flowing
right.
A
Our
recommendation
is
to
start
a
simple
and
then
increasing
the
the
tooling
to
to
find
the
root
cause
of
why
this
is
not
flowing
right.
If
you
start
with
the
flu
and
bit
log
file,
you
should
have
very
good
Clues
on
what's
going
on.
There
pay,
of
course,
attention
to
errors,
but
also
look
into
the
warning.
A
Messages
through
embed
will
tell
you
that
it
could
not
deliver
a
chunk
of
data
as
a
warning,
because
it's
retrial
right,
but
it
could
be
that
your
employees
down
but
fluent
bid,
is
not
yet
capable
of
detecting
that
pattern
and
act
upon
it.
So
you
should
yeah
look
into
the
warnings,
see
what
they're
saying
and
see.
If
there
is
something
that
you
can
do
to
remediate
that,
and
if
is
nothing
there
or
nothing
is
clear
there.
Then
we
recommend
to
increase
the
log
level.
A
B
B
Great,
is
there
anything
like
fluent
D's
secondary,
so
logs
aren't
lost.
A
So
sorry,
I
cannot
answer
that.
We
can
probably
follow
up
later.
B
A
And
okay,
let's
say
we're
dealing
with
corrupted
and
rejected
messages
yeah
in
increasing
the
the
debug
level,
because
we
have
some
extra
information
there
from
Splunk
elastic,
open
search.
A
You
can
have
the
trace
error
and
Trace
output
options,
so
you
will
see
the
request
and
the
response
from
from
that
service
and
you
can
find
there
that
there
could
be
because
of
mapping
or
indexing
issues
in
your
data
back
end
and
the
the
always
very
useful
standard
output,
blue
and
bit
has
a
an
output,
plugin
called
STD
out
that
will
print
the
messages
to
your
standard
output.
So
you
can
see
what
is
actually
trying
to
send
over.
A
And
another
common
case
that
we
have
seen
is
that
the
formatting
is
seen
correct,
and
for
this
we
recommend
first
check
the
source
check
that
you
didn't
change
anything
in
the
architecture
or
the
there
were
no
changes
in
the
application
for
that.
Of
course,
you
use
this
standard
output
plugin
and
look
for
changes,
for
example,
on
kubernetes.
We
have
seen
lately
an
increase
of
these
cases
where
they
changed
the
runtime
format.
A
So
now
it
seems
that
floam
bit
adds
some
extra
data
to
the
to
the
log
record,
but
it's
actually
that
the
runtime
change
so
the
format
of
the
logs
change
and
if,
if
your
log
events
are
multi-line,
yeah
take
a
look
whether
or
not.
This
is
a
a
built.
Sorry
if
this
is
a
built-in
parser,
if
it
is
a
custom,
if
it
is
correct,
validate
your
regular
expressions
and
yeah
that
that
will
be
the
the
recommendations
on
on
these
cases.
A
A
In
production,
this
is
something
that
we
we
have
learned
together
with
users
and
huge
deployments
in
in
our
customers
over
100k
servers
deployed.
There
have
shown
us
a
couple
things
if
you
are
not
allowed
to
to
data
loss,
yeah
use
storage
file
system.
A
So
if
your
backend
is
not
available
or
phone
beat,
for
any
reason
cannot
send
the
data,
you
will
have
a
space
literally
a
space
to
retry
that
in
the
future,
so
use
a
bit
of
of
your
disk
and
to
to
save
that
records
until
flu
and
beat
can
retry
or
successfully
retry
that
what
happens
when,
when
there
is
an
error
from
the
from
the
endpoint,
is
that
it
could
be
retrievable
or
not
right.
So
this
will
determine
whether
or
not
fluent
bead
will
use
your
storage
to
save
those.
A
While
it
is
waiting
for
the
for
the
retry
right.
If,
if
the
data
is
finally
not
sent
and
yeah
flame,
it
will
count
that
we'll
say:
hey
I
had
to
drop
this
I
could
not
send
them
for
further
information.
You
have
to
check
the
log
files
right,
but
it
will.
It
will
delete
this.
These
records
that
are
not
retrievable
or
that
were
retried
to
the
Limit
configure
and
then
is
the
the
retry
logic
that
enters
into
the
game.
A
This
retry
logic
by
default,
we'll
retry
only
once,
but
you
can,
you
can
tell
to
either
retry
Without
Limits
or
a
fixed
number
greater
greater
to
one.
Of
course,
a
fixed
number
of
rejoice
and
this
retries
will
will
be
managed
by
the
the
scheduler
not
directly
by
the
flu
and
bit
engine.
A
And
in
the
ingestion
side
start
simple,
then
specialize
right.
If
you
deploy
fluid,
as
a
demon
said,
it
will
graph
all
your
logs
and
and
then
you
will
have
to
start
learning
from
your
locks
right
know.
Your
logs
know
how
the
different
applications
that
are
sending
data
to
your
to
your
pipelines
or
that
they're
consuming
through
your
pipelines
learn
how
their
is
their
data
if
it
is
a
structure
or
what
will
be
the
regular
expression
that
will
actually
parse
that
data
that
comes
in
your
log
events
and
again
again
check
your
regular
Expressions
yeah.
A
We
have
noticed
that
regular
expression
can
have
a
great
impact
whether
they
are
greedy
or
not.
They
can
impact
in
the
in
the
processing.
We
have
done
changes
there
that
somehow
help
to
deal
with
this,
but
by
removing
the
the
processing
from
the
main
main
thread,
but
yeah.
You
should
always
always
check
your
regular
expressions.
B
Great
thanks
girls.
So
if.
B
Feel
free
to
use
this
QR
code,
I'll
drop
some
or
drop
some
links
in
the
the
chat.
If
you
aren't
able
to
access
this.
So
so
don't
worry
we'll
leave
this
up
here
for
a
second
for
anyone
who
wants
to
join
us
on
slack,
it's
a
great
place
to
ask
questions.
B
It's
a
great
resource
to
reach
out
to
lucaros
Pat
on
rug.
Anyone
else
on
our
team
we're
all
in
that
chat
and
respond
relatively
quickly
to
any
and
all
troubleshooting
issues.
So
it's
a
great
place
to
be
yeah
Let's.
Let's
jump
to
the
next
slide.
B
Big
announcement
for
our
fluent
bit
summer
series
so
we're
excited
to
announce
a
full
half
day,
training,
that's
covering
a
lot
of
the
content
that
we
covered
in
the
three-part,
webinar
Series.
So
if
you've
enjoyed
this
content-
and
you
want
to
get
Hands-On
we're
doing
essentially
a
half
day
lab
with
Eduardo
on
our
team-
probably
Jose
and
honorag
as
well
on
our
side
and
we're
going
to
be
going
through
an
intro
to
fluent
bit
fluent
bit,
Advanced
we're
going
to
be
covering
a
lot.
B
Some
processing
and
things
of
that
nature
in
that
session,
and
then
operations
and
monitoring
really
really
excited
to
kind
of
offer.
This
training
to
you
guys
and
kind
of
get
you
some
hands-on
experience
with
all
of
the
content
that
we've
covered
in
our
webinar
so
far.
So
if
you're
interested
another
QR
code
for
you
to
sign
up
right
there
and
we'll
include
this
with
our
our
follow-up
from
the
webinar
today
as
well
and
I'll
include
it
in
the
links
that
I'm
about
to
throw
in
the
chat
too.
B
Great
and
without
further
Ado,
let's
move
into
q
a
we
had
a
lot
of
great
questions
during
the
conversation
today.
Guys
so
really
appreciate
that
if
you
have
any
last
last
minute,
questions
or
anything
you're
curious
about
feel
free
to
throw
it
in
the
chat
now
or
throw
it
into
q.
A
and
we'll
cover
it
here.
B
There
is
one
that
that
just
came
in
and
Jose.
So
if,
if
you
wouldn't
mind
when
there
are
logs
in
queue
on
Startup,
does
flumpit
handle
those
the
same
way
that
it
would
logs
that
are
received
post
startup.
A
Okay,
so
yeah
kind
of
fluid
will
do
basically
the
same
right.
It
will
process
these
files
that
already
exist
there.
We
call
them
static
files
instead
of
live
files,
which
has
which
are
the
ones
that
we
tail.
But
there
are
some
tuning
options
in
there.
A
It
will
basically
try
to
ingest
as
fast
as
possible,
but
if
you
see
that
it's
using
a
lot
of
resources
in
your
in
your
node,
where
it's
deployed,
you
can
tweak
that
a
little
bit,
you
can
restrict
the
amount
of
data
that
fluent
bead
will
process
from
these
static
files
on
each
loop
of
the
loop
event.
Loop
sorry.
B
And
and
one
last
question
here:
is
there
a
way
to
process
different
log
formats
when
they're
all
coming
from
the
same
source.
A
Yeah
yeah,
that's
another
thing
we
have
seen
with
our
users.
Yes,
you
can
tell
fluid
in
the
in
the
parser
or
in
the
multi-line
parser
parameter.
You
can
say
hey
here:
I
I
could
have
either
go
python,
Java
format,
so
you
will
just
put
the
those
three
parsers
there
separated
by
a
comma
and
flu
and
bit
yeah,
we'll
we'll
try
those
different
formats
and
we'll
apply
the
the
corresponding
parser
to
them.
B
Awesome,
thank
you
so
much
so
Carlos.
If
you
wouldn't
mind,
jumping
into
that.
The
last
slide
that
we
have
here
today.
Thank
you
guys
so
much.
Thank
you
for
coming
to
the
three-part
webinar
series.
I
noticed
a
lot
of
you
kind
of
join
for
all
of
the
the
three
webinars.
B
So
we're
really
grateful
to
have
you
guys
and
excited
to
producing
some
of
some
more
content
with
the
film
bit
summer
series
so
look
forward
to
seeing
a
lot
of
you
at
the
half
day,
training
session
as
well,
if
you're
interested
curious
and
have
questions
feel
free
to
reach
out
to
us
at
hello
caliptia.com,
but
with
that
I
will
wish
you
guys
a
good
afternoon
good
rest
of
your
morning,
good
rest
of
your
day
and
we'll
see
you
guys
soon.