►
From YouTube: OpenShift Commons Gathering at Red Hat Summit 2018 Amadeus Kafka on OpenShift Case Study
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
B
Diane
I'm
adduce
we're
doing
IT
services
for
our
industry
for
airlines
and
we
have
been
on
the
road
with
OpenShift
since
I
started
working
on
kubernetes.
So
it's
like
four
years
now.
We
will
have
it
every
year
we
find
new
uses
we'll
talk
about
this
today,
so
Pierre
Olivier
and
myself,
Nannette
and
Pierre.
Please
start.
C
Okay,
so
I
will
introduce
you,
which
is
a
data
streaming
architecture
or
what
is
commonly
called
stream
processing
or
real-time
data
processing.
It's
it's
a
new
kind
of
architecture
which
is
even
driven,
so
everything
is
based
on
an
event
log.
You
need
to
have
this
persistent,
even
Club,
where
every
event
that
your
application
is
is
logging
is
persistently
stone
in
this
event,
log
in
a
strong
milliseconds
manner.
So
you
have
the
grantee,
but
what
you
will
consume
if
consumed
is
exactly
the
same.
C
Otherwise
it
has
been
produced
and
you
need
to
produce
also
immutable
events
because,
as
it
is
persistent,
you
have
this
nice
flexibility
to
be
able
to
replay
events,
so
they
need
to
be
mutable
so
that
you
can
replay
them
so
that
you
can
support
your
application
right.
Events
on
this
7
o'clock
and
you
can
have
different
courses
on,
but
we
don't
always
event
so
very
first
obvious
use
case
to
synchronize
different
data
stores.
C
It's
very
common
those
days
to
have
an
application
writing
on
different
data
store
and
instead
of
putting
stress
on
your
application
to
write
directly
on
was
data.
Storage
application
is
just
logging
so
that
I
change
on
this
log
I
think
honestly,
in
an
even
fully
consistent
manner.
You
can
synchronize
your
account
data
store
here.
I
put
some
example,
but
you
can
put
as
many
process
on
as
you.
C
You
need
invalid
and
chili
on
the
cake,
for
this
kind
of
architecture
is
that
you
can
write
your
own
process
or
applicative
courses,
so
that
will
do
some
kind
of
transformation,
business
logic
on
your
ovens
and
will
produce
a
new
event
log
as
a
result,
and
you
can
compose
a
full
application
with
a
graph
of
post
itself
only
given
producing
new
events,
and
you
can
design
your
application
in
doing
things
in
parallel
in
seconds
that
way.
What
suppose
you
can
see
both
processors
kind
of
micro
services,
but
streaming
make
Authority.
So
it's
a
very
nice.
C
C
It
is
false
for
performances.
It's
very
simple
and
in
terms
of
scalability,
so
the
latency
is
very
low.
You
have
companies
like
Netflix
or
you
go
down
during
more
than
12
millions,
even
per
second
who
set
of
Kafka
Cousteau.
So
in
terms
of
real-time,
a
treatment,
it's
it's
rare
and
after
from
the
processor,
you
have
the
choice.
If
you
you,
you
need
just
stateless.
C
One
of
them
is
Kafka
stream.
That
is
quite
simple.
It
just
Trevor
we
that
you
can
embed
in
your
own
processor,
but
you
have
link
that
is
very
popular
as
well,
and
another
set
of
processes
back
streaming
also
is
very
present
in
this
world,
but
more
on
the
big
data,
even
if
they
do
continuous
process
of
recently
Sparky's
opposition
for
big
data,
so
education
that
aggregation,
I.
Think
honestly.
So
in
term
of
advantage
has
you
you
are
decomposing
application
in
this
kind
of
a
saint
who
knows
what
they
singing
the
harvest?
Clothes
cooking.
C
C
We
worked
any
problem
you
you
will
put
as
many
micro
service
as
work
one
for
each
stage
of
your
application,
it's
very
flexible
because
you
have
a
graph,
so
you
can
plug
in
your
graph
new.
My
question
is
anywhere.
In
the
example,
I
gave
I
had
free
data
store
in
Thailand
to
update,
if
I
need
to
add
a
new
one,
I
just
need
to
plug
a
new
consumer.
For
my
Vienna
event,
Robert
will
write
on
this
new
data
store.
I
will
not
impact
at
all
over
my
co
services
and
for
the
auditability
and
Hawker.
C
It's
very
nice
because
you
have
every
event,
but
he
still
present
in
the
rug.
So
you
you
have
this
a
visibility
of
what
happened
on
your
system.
If
you
put
I'd
a
retention
on
your
event
log
one
week
several
months,
if
you
have
the
capacity
in
terms
of
a
whole
week
of
you,
can
replay
your
events
as
far
as
Cochran
in
the
past
to
recover
your
bag.
C
Big
data
that
will
under
was
data
of
join
that
group
stuff
first
and
you
can
see
here,
but
in
parallel,
so
that
are
stoned
in
a
dupe
in
vs
and
part
of
the
architecture,
all
internal,
so
big
data
on
the
top
and
real
time
business
Leo
on
the
bottom.
Well,
you
can
have
Business
School,
for
example,
interpreting
your
your
for
share
events
and
doing
some
actions
or
any
kind
of
business
process
you
can
plug
on
on
your
graph.
So
we
typically
graph
you
can
compose
your.
C
My
question
is,
as
you
need
this
kind
of
architecture,
it's
very
nice
for
what
we
call
data-driven,
because
in
in
that
model
you
can
have
your
big
data.
Leo
analyzing,
your
data
in
an
offline
process
producing
insights
of
what
you
analyzed
and
pushing
both
inside
again
in
the
detonation
path.
But
your
business
Leo
can
interprets
those
insights
and
decider
take
actions
according
to
them
which,
what
we
call
data-driven.
That's
you
put
intelligence
in
your
application.
C
C
My
question
is
on
during
the
machine
on
in
whether
in
the
business
video
in
aisle
time,
so
it's
very
nice
way
to
do
this
kind
of
of
data
ingestion
data
processing
in
real
time.
So
that's
for
the
high
level
concepts
and
I
will
leave
the
floor
to
Nanette
for
implementation,
part
of
Kafka
on
the
pan,
chieftain.
So.
B
B
Persistent
storage
is
actually
not
a
cluster
which
is
zookeeper
and
in
openshift
you
would
get
your
pod
names
like
random
ish
like
like
there,
so
thankfully
we
have
a
stateful
sets
now
when
we
started
they
were
called
pet
sets
and
they
were
betta
not
supported,
but
they
are
there
now
and
we
can
use
in
some.
What
is
the
great
thing
that
comes
with
a
stateful
set?
So
first
they
provide
stable
pod
identity,
meaning
that
now
our
Kafka
brokers
can
have
a
correct
names
like
Kafka,
zero
Kafka
Wonka
tune
over
longer
random.
B
Do
they
provide
stable
storage?
You'll,
get
your
persistent
storage
for
the
given
pod,
always
even
if
it
moves,
and
there
are
new
things
like
order
to
start
up
or
it
shut
it
down
and
rolling
upgrades.
So
it
actually
runs
fine.
We
have
been
running
for
one
year,
I.
Think
a
load
like
this
becoming
very
well
at
the
weight
and
some
experiences.
So
when
Kafka
is
a
faithful
application,
it's
pretty
much
disk
performance.
Is
this
based
and
network
based?
So
you
want
to
run
it
on
a
good
machines
like
having
SSDs,
so
using
network
affinity.
B
Allows
you
to
do
this
ice
or
not.
Affinity
allows
the
to
do.
Is
loading
on
the
machines
with
the
SSD
with
this
label,
but
also
you
want
to
spread
it
across
different
machines
because
of
lose
your
cluster.
If
one
machine
goes
down,
that's
where
ante
affinity
comes
in
and
then
some
countering
tend
intuitive
findings
so
common
wisdom,
if
you
lose
you're,
persistent
volume
use
position
points
because
we
lose
your
pods.
You
will
lose
your
data,
but
in
the
data
streaming
architecture.
B
If
time
life
of
your
data
is
few
minutes,
you
will
not
have
an
enormous
amount
of
data,
so
you
can
actually
rely
on
cover
a
replication
which
basically
means
you
can
rely
on
empty
the
earth.
So,
on
a
local
storage,
there
are
no
longer
not
yet
their
local
persistent
volumes
becoming,
but
you
can
rely
on
local
volume
on
the
SSD
and
having
very
high
performance,
and
we
will
be
using
primitives
and
Jay
makes
for
monitoring
this
one.
So
it's
fine.
We
can
have
Kafka
brokers
running
there
and
we
can
consume
disk
Africa.
B
But
it's
not
the
only
thing
you
need
when
you're
running
Africa
you
need
have
to
configure
it
for
the
applications
for
the
topics
you
have
have
a
dozens
or
several
dozens
of
micro
services
running
at
once.
In
this
platform,
each
is
consuming
from
a
topic
and
producing
to
one
or
several
topics,
and
we
want
to
be
sure
that
those
topics
exist
in
each
and
every
environment
and
we
want
to
be
sure
that
their
leader,
if
they
no
longer
use
it,
we
want
to
be
able
to
react
on
how
much
disks
is
their
disk
space.
B
Is
there
so
reduce
retention
time
or
increase
retention?
Time,
give
credentials
to
the
clients
and
ideally
want
actually
developers
to
express
this
thing?
It's
not
it's
not
that
they
want
to
write
a
work
order
to
someone,
and
then
someone
has
to
type
these
commands.
What
we
would
like
really
to
use
and
what
we
do
is
I
think
it
would
be
subject
to
any
subject
when
he
talks
is
actually
Muse
operator
just
mentioned
they
talk
before
we
have
a
Kafka
operator,
which
is
basically
monitoring
a
resource
existing
in
a
openshift.
B
In
our
case,
it's
a
config
map
that
can
be
a
customer
source,
and
this
one
describes
the
topic,
its
characteristics
like
how
many
partitions,
what
is
the
replication
factor,
specific
properties
and
whenever
it
changes
it
will
apply.
This
changes
automatically
on
the
Kafka
cluster,
it
controls.
So
there
is
no
actually
human
error
anymore
involved.
The
operator
is
replacing
this
thing.
It's
great
range
is
actually
equivalent
to
the
dimensions
or
Service
Catalog
it's
equivalent
to
the
provision
and
provision
kind
of
a
Service
Catalog,
and
it
can
also
deliver
credentials
to
micro
services.
B
B
There
is
the
next
thing.
Is
you
have
your
platform
with
dozens
of
micro
services
there
and
it
can
get
kind
of
difficult
to
understand
what's
going
on
inside,
because
it's
ok,
it's
no
longer
monolith.
It's
easy
to
deploy
one
service
they're,
all
big
couple,
but
somehow
your
platform
is
not
all
decoupled.
You
have
to
understand
what's
going
on
and
would
be
great
if
we
can
define
your
platform
in
advance
and
you
can
test
it
and
replicate
it
against
so
comes
another
operator.
B
We
call
that
actually
platform
and
plate
tool
operator
where
we
give
possibility
to
our
architecture
and
our
developers
and
designers
to
design
the
workflow
inside
a
data-driven
application.
And
actually,
when
you
look
here,
this
is
a
screenshot
from
their
each
blue
box
represents
the
deployment
config
inside
OpenShift.
B
D
D
Usually,
the
business
is
to
find
new
of
ability,
new
pricing,
and
then
we
need
to
propagate
these
changes
into
the
nodes,
the
thousands
of
nodes.
We
have
a
first
level
of
cash,
but
if
we
have
all
the
nodes
targeting
this
cash
at
the
same
time,
then
it's
the
reside
for
failure.
So
we
have
a
strong
level
of
cash
at
each
node
level
and
we
need
to
stand
in
validation
of
those
caches.
D
With
the
arched
burst
of
invalidations,
we
speak
about
20,000
invalidation
per
seconds,
with
large
bursts
coming
as
all
of
a
sudden,
so
that
the
algorithm
is
clearly
you
send
the
notification
and
then,
if
you
in
twisted,
then
you
pick
up
the
data
for
no
central
cash.
In
case
it's
interesting
for
you
Kafka
in
this
picture.
We
are
not
at
the
streaming
ever,
but
this
is
where
we
target
to
go.
We
use
very
small
messages,
200
bytes,
because
we
know
that
it
will
very
well
scale.
D
We
use
JSON
format
for
sending
the
messages
today
is
deployed
with
ansible
and
typically,
what
we
are
looking
at
is
deployment
with
an
open
chief
operator
and
an
operator
streamin
exists
and
the
decision
mechanism
to
fake
data
is
enriched,
and
then
we
try
to
convey
more
and
more
metadata
in
the
messages
at
the
same
time,
keeping
the
messaging
that
small
as
possible.
In
this
picture
we
do
have
Couchbase.
So
this
is
the
central
cache
which
is
where
we
store
terabytes
of
data
and
sometimes
changes
most
modern.
D
Sometimes
we
dissociate
the
notification
from
the
content
in
this
application,
because
it's
up
to
big
the
number
of
faces
being
largely
smaller
than
the
number
of
notifications
and
to
the
notes.
Where
do
we
see
our
evolution
today?
We
use
Kafka
as
database.
We
use
course
data
center
replication
mechanism
when
we
transfer
the
data
from
one
data
center
to
another
one
to
one
region
in
the
cloud
to
the
other
one.
We
have
a
large
interest
in
again
in
use
of
Kafka
with
operators,
because
it
simplifies
drastically
the
operation
and
the
deployment
of
the
clusters.
D
We
can
speak
about
deploying
a
new
cough
character
on
the
spot
as
an
increase
and
scale
up
the
crystal.
So
basically
we
are
at
the
beginning
of
our
swimming
can
be
used
at
a
very,
very
large
scale.
We
look
at
the
the
architecture
presented
by
Pierre
with
large
interests
and
we
have
a
large
interest
in
the
kubernetes
operators.
So
this
is
the
end
of
my
speech.
We
don't
have
time
for
four
four
questions
you
can
reach
us
we're
there.
We
will
be
happy
to
answer
your
questions.
Thank
you.