►
From YouTube: Enhance your Kafka Infrastructure with Fluvio
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
Hello
and
welcome
to
the
enhance
your
kafka
infrastructure
with
fluvio
webinar,
I'm
grant
swanson
your
host
for
today's
session.
We
will
start
with
a
few
educational
slides
on
kafka.
Vs
fluvio
then
go
into
a
live
demo
and
then
finish
up
with
q.
A
please
enter
questions
anytime
during
the
session
in
the
questions
window.
A
A
A
A
A
We
will
dive
deeper
into
smart
module
functionality
in
the
upcoming
slides,
but
first
we
wanted
to
cover
a
recent
poc
that
we
just
completed
with
a
company
that
serves
over
three
thousand
global
banks
that
enables
over
1.2
billion
people
to
carry
out
their
daily
banking
needs
in
the
poc.
They
did
benchmark
testing
on
fluvio
versus
kafka
and
the
results
were
outstanding.
A
A
A
A
A
B
Thank
you.
As
grant
mentioned,
we
are
going
to
demonstrate
the
fluidial
data
processing
pipeline
and
we
are
going
to
do
it
in
two
stages.
B
The
first
one
we
go
in
by
http
connector,
connecting
to
filmhub
api,
then
populating
fluidville
topic
and
then
via
calculating
connect,
writing
data
into
kafka
topic,
and
then
we
are
going
to
expand
it
by
18
aggregate,
smart
module,
which
will
add
a
specific
data
transformation
into
before
writing
data
into
calculating
connector.
So
the
transformation
will
happen
on
the
topic
before
data
moves
into
kafka.
So,
as
grant
mentioned,
the
kalka
will
receive
already
clean
data
and
we
are
going
to
start
from
installing
local
cluster.
B
So
this
is
a
fresh
instance
in
ec2,
and
what
I'm
going
to
do
now
is
to
install
fluvio
using
our
open
source.
B
Fluvia
cluster,
so
as
you
can
see
it's
downloading
and
installing
which
is
50.
so
then
we
are
going
to
start
with
your
cluster.
B
So
that's
for
the
local
cluster
is
spinning
up
and
installed
inside
of
the
mini
cube,
so
that
instance
already
had
the
mini
cube
installed,
but
you
can
follow
our
website
for
prerequisites.
So
now
I'm
going
to
test
floyd
plaster.
So
I'm
going
to
create
a
topic
topic
greetings.
B
Then
I'm
going
to
create
a
single
message,
hello
here
by
a
fluvio
produce
command,
so
currently
I'm
using
cli
and
then
I'm
going
to
consume
the
greetings
topic
again
by
the
way
cli.
So,
as
you
have
seen,
is
quite
straightforward,
so
we
installed
the
fluid
cluster
and
we
created
a
topic
called
greetings.
We
updated
it
and
then
we
consumed
it
from
the
beginning.
B
Now
that
instruments
already
have
a
mini
q
yeah
and
it
also
have
a
pre-installed
kafka
local
instance,
which
is
a
development
instance.
It's
nothing
resembling
to
production,
but
it's
already
have
multiple
topics
populated.
So
this
is
the
local
kafka
cluster.
B
So
what
I'm
going
to
do
next
is
I'm
going
to
create
an
http
connector
which
will
read
the
fin
tab
api
and
that's
going
I'm
going
to
do
it.
Building
up
on
the
demo,
which
sebastian
already
demonstrated
in
the
previous
webinar,
so
you
can
clone
our
previous
webinar.
C
B
But
the
idea
is
that
you
need
to
register
on
financial,
thin
hub,
io
and
obtain
api
token.
Then.
B
You
need
to
create
a
configuration
file
which
looks
like
this
for
http
connector.
As
you
can
see,
that's
the
name
of
the
connector,
that's
the
type
of
the
connector.
This
is
a
topic
which
is
fluviotropic.
B
The
data
will
be
red
so
also
it
all
says
that
it
will
create
a
topic
which,
if
it
doesn't
exist,
and
then
it
will
query
the
api
endpoint
using
get
method
for
every
three
seconds
and
we
have
a
make
file
which
can
very
conveniently
demonstrate
you
how
to
run
multiple
commands
and
this
one
we
are
going
to
create
a
connector
by
a
fluid
connector
create
command.
As
you
can
see,
it's
quite
straightforward
for
your
connector
creator
and
now.
B
I'm
going
to
check
so
the
connector
is
running
and
let's
check
that
it
is
actually
producing
a
topic.
So
I'm
using
fluvia
consume
command
again
much
in
a
topic
named
gme
stocks
and,
as
you
can
see,
that's
populated
with
stock
data.
Every
three
seconds.
B
So
now
what
we
are
going
to
do
is
we
are
going
to
write
into
cluster.
So,
as
I
mentioned,
I
already
have
a
kafka
cluster
which
we
created
using
docker
compose
and
docker.
Compose
file
is
provided
with
repo,
which
will
be
shared
after
webinar,
and
the
important
point
is
that
it
have
a
advertisement
host
which
is
matching
the
gateway
for
mini
cube,
so
our
fluvio
cluster
can
actually
connect
into
kafka.
B
So
what
we
need
to
create
first,
is
the
game
configuration
file
for
kafka
and
it
looks
pretty
similar
to.
B
Http
connector,
except
it's
so
name
type,
is
kafkasim,
picks
up
the
data
out
of
topic
gme
stock
and
then
writes
it
into
kafka
process.
Blue
geotopic.
B
So
plural
connector
locks
command
can
help
the
debug
if
there
are
any
issues
with
a
kafka
connector.
But
there
are
no
issues
so
far
and
let's
check
if
we
are
actually
populating
the
topic
so.
B
So
I'm
going
to
change
a
topic.
This
one
should
be
processed.
B
And
yeah,
as
we
can
see
every
three
minutes,
we
are
populating
every
three
seconds.
We
have
good
eating
kafka
together
right,
so
that
was
a
basics
introductions.
So
we
have
created
two
connectors,
including.
B
One
is
http
connector
and
another
kafka
scene
connector.
So
what
we
are
going
to
do
next
is
more
interesting,
so
we
are
going
to
add
the
smart
modular
and
it
is
aggregate
and
it
will
perform
the
aggregate
function
so
that
smart
model
pretends
that
we
purchased
some
stock
volumes
and
we
are
going
to
populate
them
using
the
text
file
warrants.txt
and
it
will
reshape
our
tick
options
and
try
to
create
potential
profit
on
all
laws.
B
B
So
that
model
is
the
rust,
is
written
in
the
rust
and
what
we
are
going
to
do
is
to
compile
it
into
web
assembly,
and
then
we
are
going
to
upload
the
basemte
into
smart
modules,
repository
in
local
fluid
cluster,
and
then
we
are
going
to
meditate
it
and
attach
it
to
a
kafka.
Sync
connector.
B
So
again,
there
is
a
convenient
make
file
which
we
will,
which
is
already
online,
so
that's
already
applied.
We
come
to.
B
Upload
smart
modelling
and
as
as
you
can
see,
those
are
just
commands
which
are
pretty
long,
but
this
mac
file
on
the
wrapper
for
a
fluvior
cli,
and
what
I'm
going
to
demonstrate
that
we
now
uploaded
price
warrant
aggregator
and
now
we
need
to
produce
warrants.
I
assume
that
we
actually
make
the
purchase,
and
that
is
very
simple
command
which
again
uses
fluid
cli
to
late
hdmi
stocks
topics,
and
then
we
are
going
to
check
if
actually
our
smart
model
is
working.
B
And,
as
you
can
see,
when
smart
monitor
was
applied,
then
the
shape
changed
and
it
also
could
create
in
our
current
profit,
which
is
actually
loss.
So
if
we
bought,
if
we
potentially
bought
the
stocks
at
11th
of
april
at
various
purchase
price,
which
is,
there
will
be
probably
at
laws
at
this
point
in
time.
B
So
what
we
are
going
to
do
next
is
we
are
going
to
deploy
this
smart
motor,
so
it
will
run
inside
kafka,
sim
connector,
and
for
that.
B
We
have
the
only
thing
we
need
to
do
is
to
add
that
smart
motor
plus
aggregator
needs
to
be
applied
and
it
is
aggregate
smart
model.
So
what
I'm
going
to
do
is
again
to
create
a
connector.
C
B
Monitor
and
one
another
of
them
is
running
deep,
smart
model.
So
let's
read
the
kafka
topic
and
meditate
that
our
smartphone
is.
B
Configured
option
so
again:
I'm
reading
directly
out
using
kafka,
console
consumer
and
you
can
see
it's
currently
zero
and
the
reason
for
it
is
that
smart,
motor
and
connector
is
applied
at
the
region
state.
So
what
we
need
to
do
now
is
we
need
to
populate
our
warrants
again
because
they're
already
in
a
topic
and
connect,
started
region
records
and
applying
activity.
B
B
Http
connector,
which
reads
data
into
fluvial
topic,
then
applies
a
transformation
which
is
written
in
rust
and
compiled
into
the
assembly.
It
runs
before
carcassing,
writes
it
into
carpet,
and
that
concludes
the
live
demo
pad
and
we
have
now
can
open
lots
of
questions.
A
Awesome
alex
thank
you
so
much.
That
was
pretty
incredible
that
you
got
all
that
done
in
in
about
20
minutes.
So
give
me
just
one
moment:
I'm
pulling
up
the
questions
from
the
audience
again.
If
anyone
has
any
questions
feel
free
to
enter
them
into
the
questions
window
or
even
the
chat
box
is
fine
and
I'm
going
to.
A
Okay,
great,
we
got
one
question
that
just
came
in
here
so
they're
asking
what
other
types
of
data
transformation
can
be
performed
with
fluvio.
B
So
off
the
shelf,
we
have
a
pre-defined
templates
for
filter
map,
filter
map
and
the
rain
map
and
aggregate
smart
modulus.
Each
of
them
applies
as
described
transformation,
but
you
can
put
more
and
more
complexity
depending
on
your
rust
and
experience
into
the
smart
monitors.
A
Awesome:
okay:
we've
got
a
couple
more
that
are
coming
in
here,
so
I'm
just
going
to
read
the
actual
comment
and
the
question
here.
It
says
incredibly
elegant
architecture,
k-8s
native
and
connectors,
plus
smart
modules
and
wasm.
Look
like
this
could
be
the
building
blocks
of
a
universal
streaming
platform
of
the
future.
C
I
can
talk
about
this.
I'm
sebastian
emily,
I'm
a
software
engineer
here
and
yeah.
This
is
a
high
level
strategy.
Question
we've
discussed
this
type
of
thing
for
a
little
bit
of
like
how
to
visualize
various
streams
and
like
how
to
have
my
more
of
a
graphical
interface
for
defining
your
lego
pieces
that
go
into
a
a
data
aggregate
and
you
know
etl
of
sorts.
So
it's
definitely
something
that
that
can
be
put
higher
in
the
roadmap
should
it
be
needed,
but
for
now
it's
in
the
backlog.
A
Excellent.
Thank
you
sebastian
appreciate
that
we've
got
a
bunch
more
questions
coming
in
here,
so
the
next
question
is
is
so
this
is,
I
guess,
about
a
replacement.
Do
you
have
an
opinion
on
when
kafka
is
suitable,
a
suitable
candidate
versus
fluvio
and-
and
I'm
just
like,
I
said,
I'm
assuming
that's
a
kafka
replacement
for
fluvio
with
fluvio.
C
That's
a
tough
one.
It
really
depends
on
your
use
case,
like
we're
not
quite
feature
complete
with
respect
to
kafka,
one
of
the
things
that
I
will
say:
that's
nice
compared
to
cap,
guys
that
we
were
able
to
deploy
more
easily.
C
In
my
experience,
setting
up
a
cafe
cluster
is
kind
of
a
pain,
so
I
would
definitely
say
that
if,
if
setting
up
a
kafka
cluster
has
been
your
friction,
then
luvio
is
a
pretty
nice
thing
to
deploy
and
we're
not
trying
to
be
entirely
com
future
complete
with
respect
to
kafka
either
because
kaffir,
for
example,
doesn't
have
the
smart
module
features
that
we've
shown
here,
but
I
will
admit
that
kafka
does
have
more
connectors
to
various
services,
because
it's
a
much
the
product
has
existed
longer.
C
So
yeah.
A
Yep,
I
I
agree
with
you
sebastian
and
I'll
just
add
to
that
answer
that
we're
continually
building
out
our
our
connector
catalog,
but
but
definitely
kafka's
got
more
options
to
choose
from
sources
and
connectors,
but
as
time
goes
on
we'll
we'll
we'll
cover
that
gap.
I
believe
okay
next
question
here
when
will
fluvio
support,
windowing
and
processing
windowed
data.
C
That
is
a
that's,
not
a
question.
I
know
off
the
top
of
my
head.
It
has
been
discussed.
I
can't
give
you
a
distinct
timeline
of
this
thing.
I
would
have
to
discuss
with
our
our
product,
ceo
and
ctoceo,
on
aj,
more
yep.
A
Okay,
great
so
we'll
respond
to
that
question
offline
and
we'll
move
on
to
the
next
one.
So
what
are
some
of
the
limitations
that
fluvio
might
have
that
flink
does
not.
B
B
We
are
slightly
for
different
audience
rather
than
for,
like
a
replacement,
so
I
don't
see
a
flower
being
a
replacement
of
link
directly
and
that's
why,
even
if
kafka
and
the
rest
of
the
infrastructure,
we
probably
will
complement
quite
a
lot
of
the
existing
tools
during
various
transition
states
for
organizations
and
as
sebastian
said,
we
think
about
ourselves
as
a
faster,
easier
to
deploy
and
easier
to
manage
excellent.
A
Thank
you
alex.
We
we've
got
a
number
of
questions
coming
in
here,
so
we'll
continue
to
answer
them
as
long
as
we
have
time.
We
have
about
10
minutes
left
in
the
webinar.
So
next
question
is:
can
the
payload
that
fluvio
carries
be
binary,
I'm
thinking,
protocol
buffers
etc?
A
C
Yes,
on
a
low
level,
the
actual
payloads
are
all
binary.
The
I
think
there
was
some
discussion
like
you
mentioned,
basically
some
kind
of
metadata
around
the
payloads.
We
don't
quite
have
typededness
for
the
payloads
just
yet,
but
you
can
building
that
on
top
of
fluvio
streams,
isn't
particularly
hard
yeah.
So
like
there's,
no
reason
why
you
couldn't
like
say,
put
images
or
even
video
into
a
fluvio
stream.
C
You
just
bundle
your
your
frame
of
play
a
movie
or
you
know
a
video
into
a
given
fluvia
record,
which
would
be
big,
but
there's
no
reason
why
you
couldn't
do
that.
So
in
the
case
of
mentioning
grpc,
I
think
I
think
I
have
no
reason
to
believe
that
you
couldn't
like
put
grpc
streams
into
fluvio,
so
yeah
it's
binary,
and
you
can
do
this,
it's
for
the
cases
of
metadata
and
typedness.
You
need
to
do
it
by
convention
rather
than
fluvio
type
checking.
So.
A
Excellent
next
question
does
fluvio
have
a
way
to
de-duplicate
jobs,
to
avoid
modules,
processing
the
same
record
twice.
C
That
has
been
a
a
discussion.
We've
been
looking
into
any
right
now.
No,
but
having
statefulness
on
the
stream
is
an
area
of
work,
we've
been
looking
into
and
it
is
low.
On
the
I
mean
it's
it's
near
on
our
roadmap.
The
examples
of
this
would
be
like
resuming
from
an
offset
for
a
given
consumer
street.
A
B
B
We
are
as
a
grand
set.
We
are
a
platform
upstarting
platform,
so
we
are
building
towards
connectors.
But
that's
where,
if
you
already
have
a
large
infrastructure
then
become
complemented.
But
if
you're
starting
from
greenfield
development,
then
we
probably
will
allow
you
to
progress
way
faster
than
apache
filling
or
kafka.
C
To
expand
on
that,
I
think
one
area
that
flink
probably
lacks
is
is
the
ability
to
do
more
edge
nodes
for
internet
of
things.
Devices
such
as
like
running
blink
on
a
raspberry
pi.
I,
my
hypothesis
or
my
suspicion,
is
that
it
would
be
difficult
to
deploy
flink
on
edge
nodes
of
the
various
infrastructure,
whereas
luvio
we
have
an
installation
for
raspberry
pi.
Should
you
want
to
like
do
some
kind
of
iot
thing?
So
that's
definitely
an
area
that,
where
I
would
say,
flink
is
lacking.
A
Excellent
appreciate
that
answer
sebastian
looks
like
we
have
about
time
for
one
more
question
which
just
came
in
and
the
question
is:
does
fluvio
support
any
flavor
of
sql
or
is
it
in
the
plans.
C
Oh
yeah,
I
can
talk
about
that
the
right
now
we
support
a
very
low
level
representation
of
postgres,
and
it's
mostly
currently
used
for
cdc
between
two
postgres
instances.
It
is
in
the
plans
to
expand
this
to
do
postgres
from
to
mysql,
or
vice
versa.
It
depends
on
on.
If,
if
we,
if
someone
shows
interest
in
in
one
of
these
things,
we
would
definitely
love
to
add
it.
So
it
is
in
the
plans
but
we'd
love
to
make
it
a
higher
priority.
B
I
would
expand
in
the
way
that
we
would
like
to
validate
our
connectors
they've
used
cases.
So
if
you
think
that
fluvio
can
be
useful
for
you
in
particular,
and
have
a
particular
use
case
in
mind,
I
think
that
will
be
a
really
good
opportunity
for
us
to
validate
our
platform
and
for
you
to
prototype
our
use
case.
A
Excellent
appreciate
that
alex,
so
I
will
really
appreciate
the
audience
being
so
engaged
and
asking
so
many
questions.
We're
gonna
we're
gonna
wrap
this
up
with
a
few
call
to
actions.
A
So
it's
very
simple:
if
you're
interested
in
signing
up
for
infineon
cloud,
which
is
our
fully
managed
fluvio
service,
you
can
do
that
at
infineon.cloud,
sign
up
or
you
can
go
to
our
home
page
infinion.com
and
click
try.
Now
we
also
have
instructions
on
how
to
build
a
streaming
app
and
then
we
can.
A
We
also
have
the
ability
for
for
anyone
to
schedule
an
event
streaming
processing
demo
and
we're
happy
to
engage
with
anyone
in
the
audience
if
they're
interested
in
testing
out
fluvio
and
getting
a
poc
up
and
running
we'd
be
happy
to
engage
with
with
everyone.
So,
at
this
time
we're
gonna
wrap
this
up,
and
I
appreciate
everyone's
engagement
and
thank
you
so
much
and
enjoy
the
rest
of
your
day.