►
Description
Speaker: Matthias Niehoff, IT Consultant
CQRS (Command Query Responsibility Segregation) is a pattern, which separates the process of querying and updating data. As a query only returns data without any side effects, a command is designed to change data. CQRS is often combined with Event Sourcing. This is an architecture in which all changes to an application state are stored as a sequence of events.
Because of its great capability to store time series data Cassandra is the perfect fit for implementing the event store. But there a still a lot of open questions: What about the data modeling? What techniques will be used to process and store data in the Cassandra database? How to access the current state of the application, without replaying every event? And what about failure handling?
In this talk, I will give a brief introduction to CQRS and the Event Sourcing pattern and will then answer the questions above using a real life example of a data store for customer data.
A
Actually
we
had
a
use
case
to
come
up
with
security
and
events
or
think
I
would
just
describe
this
use
case
to
you,
then
a
little
introduction
into
then
sourcing
and
CQRS
and
at
the
end,
going
deep
into
Cassandra
into
SPARC,
how
you
use
it
for
storage
for
processing
and
some
benefits
and
pitfalls.
We
have
and
at
the
end,
some
Q
and
a
as
we
have
Q&A
at
M.
Please
ask
questions
at
the
end,
so
I
use
case.
We
have.
We
have
a
customer
which
is
a
wholesale
and
retail
company.
A
We
have
a
lot
of
legacy
old
systems
which
are
not
24/7
able
and
on
the
other
side,
we
have
some
internet
applications
for
the
users
for
the
end
users
to
change
their
data,
to
buy
things
in
the
shop
which
actually
needs
access
to
these
data
in
this
legacy
systems
and
as
it's
not
affordable
to
get
these
legacy
systems
into
24/7
mode.
So
we
get
a
24/7
proxy
as
we
call
it,
which
catches
caches.
The
data
of
the
legacy
systems
provides.
A
These
data
to
the
internet
very
applications
starts
with
changes
which
are
made
into
Internet
applications
and
provides
these
changes
back
to
the
customer
to
the
legacy
systems,
but
there's
no
business,
login
logic
or
validation
inside
this
proxy.
It's
all
done
by
the
legacy
systems.
We
came
up
with
the
following
assumptions:
we
need
to
be
highly
scalable
with
a
lot
of
more
whites
than
weeds
and
whites
with
white,
where
more
than
ten
to
one
and
we
need
to
deal
with
up
to
five
hundred
thousand
five
hundred
millions
or
half
a
billion
customers
and
their
transaction
data.
A
So
it's
not
only
master
data
management
but
also
transaction
data
like
billing
and
turning
items
so
uninvent
sourcing,
the
traditional
pattern
of
saving
applications
state,
for
instance,
you
have
in
store
selling
articles.
You
would
have
a
method,
get
inventory
and
in
method,
update,
inventory,
pretty
simple,
most
efficient
in
a
lot
of
use
cases,
but
once
the
different.
Now,
if
you
used
event
sourcing
measure,
we
have
the
series
of
say
it
simply:
pennies
replenishments.
We
have
a
tablet
which
an
initial
stock
of
60,
we
said
20
and
we
get
in
replenishment
for
10.
A
We
have
a
stove
starting
with
25,
sell
five
and
we
don't
get
replenishment
with
the
normal
saving
only
application
state.
We
would.
We
would
end
up
with
this.
We
have
an
article
inventory.
We
have
some
tablet
with
some
stock
of
50.
We
are
from
gas
stove
with
some
stock
of
20
and
that's
basically
it
which
we
went
sourcing.
We
would
save
events
instead
of
the
state,
so
we
would
have
a
lot
of
events
with
the
timestamp
and
depending
on
which
granularity
you
use.
A
In
the
event,
you
would
end
up,
maybe
with
complete
state
again
like
50
or
20,
or
you
can
end
up
with
events
or
any
having
a
diff
like
okay,
minus
5
or
plus
1,
or
you
could
have
events
like
article
sold
event
or
article
replenishment
events,
the
more
granularity
you
have
in
your
events,
the
more
flexibility
you
gain
in
our
example.
We
have
a
pretty
easy
solution.
A
We
only
use
this
complete
state
which
makes
it
easier
for
us
at
the
end,
but
we
don't
have
the
flexibility
anymore,
but
it's
sufficient
for
us,
but
why
would
you
use
events
or
selling
at
all?
First,
you
have
a
lock
of
all
stock
changes
in
this
case.
So
if
the
producer
of
the
events,
in
our
case
some
of
the
cost
systems
or
some
of
the
legacy
system
produces
one
various
wrong
events,
you
can
easily
switch
back
to
the
state
to
the
time
everything
was
fine
and
start
over
again.
A
On
the
other
side
you
can
have
a
complete
can
do
a
complete,
rebuild
or
state.
So
if
one
of
the
processors,
one
of
the
programs
with
which
actually
processes
the
events,
does
something
wrong.
You
can
also
easily
switch
back
and
start
one
over
again
processing
all
the
events
with
the
correct
processor.
Now
you
can
do
some
time.
Queries
like
what
was
the
stock
at
the
beginning
of
the
month.
What
was
the
standard
end
stock
at
the
end
of
the
month?
A
Anything
like
this
seek
us,
also
starting
with
a
very
simple
default
application:
a
three-tier
layer,
application
with
a
user
interface
using
application
services
to
modify
a
data
in
the
domain
model,
which
is
backed
up
by
some
database
and
also
using
the
application
services
to
retrieve
data
from
the
domain
model.
What
you
basically
do
with
securitizers
splitting
up
the
application
services,
we
have
a
command
service
for
modifying
the
data
and
a
query
service
for
the
retrieving.
The
data.
That's
basically
it
nothing
special,
very
easy.
A
So,
but,
as
a
Chinese
quite
easy,
you
can
think
of
further
changes.
So
you
can
also
split
up
the
domain
model,
so
you
can
have
a
queried
motor.
You
can
have
a
command
model.
The
Kumar
model
could
have
should
make.
The
validation
of
the
events
should
in
should
ensure
data.
Consistency
and
security
model
is
just
true
which
we've
data.
You
can
then
independently
scale
those
models.
If
you
have
a
white,
heavy
or
wheat
heavy
application,
for
instance,
you
have
in
wheat
heavy
applications.
A
You
can
have
much
more
different
wheat
models
also
perfectly
fitted
for
the
query
you
want
to
serve
and
you
have
only
one
command
model,
for
instance
you're
totally
split
up
those
two
and
are
totally
independent
or
further.
You
can't
not
use
the
query
model
anymore,
but
because,
what's
a
benefit
of
a
query
model,
you
don't
have
any
big
model,
you
want
or
a
domain
model.
A
Maybe
you
want
to
validate
anymore,
but
you
have
the
data
in
your
query
view,
and
you
just
want
to
give
it
out
so
just
user
in
wheat
layer,
which
is
a
sequel
or
SQL
whatever
datastore
you
use
to
get
the
data
as
fast
as
you
could.
So
maybe
you
don't
want
to
use
any
ORM,
more
validation,
anymore,
just
get
the
data.
That's
but
not
least,
you
can
use
different
databases
for
the
model.
You
can
use
the
database
for
the
commands
and
event,
and
you
can
use
another
databases
for
all
the
queries
you
want
to
serve.
A
That
could
be
a
racial
relational
database
if
it
fits
your
needs.
It
could
also,
of
course,
be
Cassandra,
but
it
could
also
be
a
search
engine.
So
if
you
combine
those
two
events,
sourcing
and
CQRS,
we
still
have
command
services,
getting
our
forgetting
changes.
We
then
have
a
command
model
which
predates
the
commands
we
get
in
and
then
start
them
to
an
event
store.
Furthermore,
the
command
chronically
gasps
event
processing,
so
every
went,
gets
thought
and
in
parallel
gets
processed.
A
This
went
through
processors,
fill
up
the
query
stores
and
we
have
a
weed
layer
for
to
get
the
data
out
of
the
Christos,
and
you
have
with
one
weed
layer
for
each
query
store
and
you
maybe
have
one
special
database
for
each
tree
store
and
last
but
not
least,
you
have
to
create
services
again.
So
this
short
introduction
into
event,
sourcing
and
secure
succumb
to
su
storage
with
Cassandra.
A
But
first,
why
did
you
choose
Cassandra
or
our
event?
Source
is
not
only
an
stupid
event.
Sync,
we
want
to
do
compaction
on
it.
So
maybe
we
have
what
we.
What
kind
of
events
we
would
have
is
like
a
custom
address
changed?
We
don't
need
any
custom
address
changed
since
the
last
ten
years.
Maybe
maybe
you
just
need
the
last
changes
from
the
last
year
and
anything
else
forget,
so
we
don't
want
to
do
a
compaction
on
it,
but
not
like
not
only
a
time.
A
Compaction
like
forget
everything
which
is
older
than
one
year,
but
forget
everything
that
is
order
bun
than
one
year,
but
keep
at
least
one
copy
of
each
entity
for
to
do
selectively.
We
play
I,
take
the
customer
again
and
also
we
have
articles
and
addresses
and
whatever-
and
we
have
include
you,
which
only
shows
our
customers.
So
we
have
a
new
crew
review,
the
data
of
customers
and
in
our
event,
sauce-
and
we
only
want
to
events
of
customers
to
fill
up
the
new
crew
view.
So
we
want
to
do
a
select.
A
If
we
play,
we
don't
want
to
replay
the
whole
lock
as
a
event.
Lock
is
our
primary
source
of
truth.
We
don't
want
any
point
of
Singapore
fader,
so
it
must
be
totally
reliable,
which
is
with
Cassandra.
Of
course,
you
can
always
like
the
keynote
today.
You
can
always
destroy
our
notes
so
but
at
least
what
at
least
one
note
is
functional,
you're,
fine!
A
Furthermore,
the
company
is
not
only
Germany,
but
also
in
Asia,
our
Middle
East,
not
in
the
US.
Actually,
we
want
also
gave
application
over
different
data
centers,
which
Cassano
supports.
Fine
writing
to
the
commit
log
is
actually
only
always
writing
data
once
and
not
modified.
That's
something.
Cassandra
is
very
good
in
white
data,
ahead
white
white
white
and
don't
modify
the
data
like
typical.
I'm
serious
data
trade
also
as
we
use
park
for
further
and
aggressive
processing
spark
sandra,
plays
very
well
with
the
spark
asunder
connector
and
last
time
on
these.
A
It's
open
source
and
it
said,
has
a
huge
community
which
is,
of
course
good
for
us
developers,
but
also
the
customer.
What
was
really
good
fine
with
ok,
we
use
open
source
technology
with
a
great
community
and
also
the
customer
already
uses
Cassandra,
which
made
it
a
little
bit
easier
for
us
to
get
Cassandra
into
this
project
last
but
not
least
easy
to
operate
most
of
the
time.
A
Of
course,
there
will
be
difficulties,
but
the
cover
customer
actually
has
a
lot
of
our
code,
databases
now
and
Cassandra,
and
some
project-
and
this
fits
quite
good
with
it.
So
no
big
need
to
get
him
to
Cassandra
so
for
the
event
store
to
get
all
entities
of
a
given
type,
for
instance
to
get
air
all
events
of
type
customer.
A
A
We
use
this
update
time
later
on
and
processing
to
decide.
Ok,
it's
this
event
needed
anymore.
I
said
outdated,
their
who
views
this
update
time
and
the
payload
we
plane
safest
text
plaintext.
We
don't
split
it
up
into
Cassandra
columns.
We
just
take
the
payload,
we
use
JSON
for
it
you
could
also
use
XML
or
for
above
or
ofwar.
If
you
need
schema
reliability,
we're
fine
with
Jason.
In
our
case,
we
also
sorting
it.
Ok,
that's
the
normal
stuff.
A
You
want
to
use
size
field
compaction
actually,
as
it's
a
quite
heavy
application,
could
optionally
also
have
second
table
to
access
an
entity
directly.
So
if
you
need
access
on
okay,
I
need
person
with
the
ID
one.
Two
three
you
have
to
the
entity
key
in
a
partition
key
to
get
an
efficient
access
on
the
person.
It's
an
optional
table.
We
don't
use
it
yet,
but
it
might
be
possible
that
in
the
flow
of
our
project
we
might
use
it
for
the
query
stores
you
do
basically
is
just
create
the
table.
That
fits
your
query.
A
So
if
you
want
to
clearly
articles
and
category
computer,
you
maybe
need
to
tailor
which
was
a
primary
key
category
and
the
article
ID
and
the
info,
or
whatever
you
have
from
the
article
beware
of
large
partitions.
One
category
could
be
very
large,
though
it's
just
an
India
and
all
the
information
could
also
be
saved
as
a
JSON
document,
for
instance,
if
you're
from
less
service
at
the
other
end,
giving
out
all
the
data
as
JSON.
Why
not
store
just
JSON
directly?
A
If
you
don't
need
the
data
in
the
crew
store
for
other
analytics
or
search
processes?
There's
no
need
to
start
in
Cassandra
columns.
At
some
point
the
customer
will
come
up
with
so
I
need
specific
queries
with
a
lot
of
different
filters
on
my
data.
Oh
I
need
a
talk,
freebies
I,
don't
know
what
I
want
you
to
search
it,
but
give
me
access
to
all
the
data
and
I
will
look
for
it.
So
basically
solution
is
solo
or
elastic
or
some
other
seek
search
engine.
You
might
want
to
fail.
A
It's
another
clue
view
was
about
stowage.
So,
what's
about
processing
through
you
spark
so
the
command
model,
as
I
said,
as
in
Cronus
lis
triggers
the
processing
of
the
data,
we
have
an
event.
We
have
event,
processors,
which
updates
the
query
view.
We
have
one
event,
processor
preview.
So
if
you
have
a
curve
you
our
customers
with
addressed
data,
we
would
have
one
event
processor,
which
is
in
fact
one
spark
top.
Have
we
been
processors,
one
spark
streaming
application
filling
one
special
database
we
use.
We
started
with
0.
A
A
First,
it's
easy
to
scale
out
it's
basically
the
same
as
with
Cassandra,
just
as
no
nodes
with
more
CPU,
with
more
memory
and
you're
fine,
it's
easy
deployment,
which
you
just
bundle,
a
jar
which
is
your
spark
application,
throw
it
into
the
cluster
and
you
are
ready
to
go
it's
a
important
for
us
as
we
want
to
have.
If
you
want
a
nuclear
view,
we
want
to
deploy
new
crew
you
without
downtime
of
any
other
crew
usage,
so
just
keep
running.
A
Also,
if
you
have
a
new
version
of
a
clear
view,
you
just
want
to
deploy
it
switch
over
to
the
new
version
and
we
are
ready
and
all
this
is
supported
by
spark.
It
is
a
my
opinion,
intuitive
Scala
and
Java
RP,
which
is
important
as
we
as
who
are
building
the
framework.
Most
of
the
time
won't
white,
all
the
clipboard
jobs.
A
A
The
for
tolerance
combined
with
all
of
the
parks,
Kafka
adapter
and
the
integration
with
Cassandra
allows
us
to
have
if
we
want
and
exactly
once
processing,
especially
if
you
use
direct
RP
and
Kafka,
you
get
exactly
once
semantics.
If
you
want
them
so
in
detail,
we
have
in
spark
streaming
application,
as
I
said.
Is
it
only
consumes
topics
of
interests?
So
we
have
one
Cassandra
topic
for
each
entity
type,
one
Kafka
topic
for
each
entity
type.
We
have
one
topic
for
for
customer.
A
One
topic
for
addresses,
one
topic
for
articles,
one
topic
for
prices
and
so
on,
and
every
jobs
just
consumes
this
method.
The
messages
he's
interested
in
and
doesn't
process
anything
else
dot,
the
data
with
which
is
with
the
data,
which
is
thought
in
Cassandra,
too
update
the
state.
So
what's
this
pizza
was
it's
a
talk
in
the
CH
I
said:
join
Wisconsin,
whatever
we
petition
by
Chris
and
whatever's
useful
to
get
join
with
data
stored
in
Cassandra.
A
So
now,
if
you
need
a
new
crew,
some
customers
say
so
I
want
to
see
our
prices
at
of
some
articles
in
Category,
computer
or
whatever.
You
just
create
a
new
two
table
for
your
query
view
which
actually
stores
the
data.
Then
you
create
a
spark
job
which
fills
up
this.
This
table
you
deploy
the
spark
job
and
every
process
of
the
event
database.
We
all
still
have
all
the
events
thought
in
our
database.
A
It
should
be
the
same
to
information
logic
in
installation,
as
in
the
normal
processing.
There
isn't
a
difference.
The
only
difference
is
the
source,
so
you
don't
want
to
put
all
the
events
in
the
kafka
again,
then
every
jobs
get
every
job
gets
notified.
Again
we
use
a
special
Kafka
topic
for
initialization
for
this.
So
if
we
want
to
in
initialize
this
job,
we
have
a
minute
spark
top
and
spark
batch
jobs.
A
Reading
the
data
out
of
the
Cassandra,
putting
it
in
the
into
the
Kafka
and
filling
up
the
topic
which
is
used
for
internet
initialization.
Last
but
not
least,
you
mark
this
view
as
initialized
to
make
sure
none
non
initialized
view
is
delivered
to
consumers.
It
might
be
optional.
If
you
consume
is
fine
with
not
complete
data.
You
can
skip
this
step,
some
of
the
benefits
and
pit
work
pitfalls.
We
have
first
the
most
important
benefits,
of
course,
scalability
for
both
spark
and
Cassandra.
It
comes
down
to
okay.
A
If
you
need
to
scale
out,
you
just
add
more
nodes,
that's
basically
it
and
due
to
the
separation
of
our
query,
model
and
command
model,
and
also
the
databases.
It's
a
more
easy
to
scale
out
those
two
as
I
said,
the
spark
jobs
going
to
be
different
by
the
custom
by
the
client
of
the
system,
for
instance,
than
chopped.
It's
mainly
used
mainly
interested
in
articles
and
master
data
management
is
mainly
interested
in
person
data
and
through
the
spark
jobs
we
get
and
very
easy
collaboration.
A
If
you
just
need
a
new
career
view,
that's
created,
you
don't
have
to
align
with
any
anyone.
It's
so
can
I
change.
This
queer
view
watch
is
the
release
of
your
new
crew
view
and
what
is
mine
release
and
you
just
create
a
new
job,
deploy
it
and
you're
done.
Nobody
else
cares
about.
There
are
also
are
some
pitfalls,
honest.
So
it's
definitely
more
than
complexities
and
a
simple,
create,
read,
update,
delete
application.
If
you
are
fine
with
with
the
create
we'd,
updated
application,
don't
have
any
scalability
issues,
you
maybe
don't
want
to
use
event.
A
Sourcing,
seek
us
but
stay
with
what
you
have.
You
have
side
effects
on
vent
replay.
As
I
said
new
a
new,
a
new
clip
orders
generated
you.
We
play
all
the
events.
What
if,
in
some
case
new
Ben,
would
trigger
emails
because
you
detect
some
credit
limit
which
has
been
reached
and
you
get
send
out
an
email
to
an
department
or
you
lock,
some
credit
card
or
whatever,
so
you
have
to
be
aware
of
it
of
this.
A
The
crew
views
are
eventually
consistent.
That
might
be
okay
for
us,
okay,
it
may
take
one
or
two
seconds
and
two
other
crew
view
gets
updated
or
in
half
and
second
and
one
crew
view
has
more
actual
data
than
the
more
current
data
than
the
other
crew
of
you.
It
might
be
okay
for
us,
but
it
might
not
be
okay
for
some
applications.
A
There
must
be
a
way
office.
We
have
concurrent
rights
and
performance
of
we
play.
I
will
go
deeper
into
these
two
concurrent
whites
without
in
lost
updates.
We
have
two
events:
a
and
B
which
are
in
sequential
import.
Let's
say
a
is
an
update,
PS
and
delete
egg
should
be
first.
Bees
should
be
second,
but
in
our
case
a
is
processed
after
B
show,
so
you
have
an
delete
and
then
an
update.
The
entry
comes
again
like
a
zombie.
We
have
two
solutions
for
this:
one
is
partition
spark
led
by
entity
key
we
open
the.
A
If
we
use
spark
processing
every
entity,
T
gets
processed
on
its
own
and
concrete
entity
like
a
person,
1
2
3
doesn't
get
processed
in
parallel.
It's
pretty
easy
solution
to
achieve
it,
and
it
mainly
helps
us
here.
You
could
also
use
a
lambda
architecture
with
batch
and
a
speed
layer
as
the
speed
layer
always
generating
data,
knowing
it
might
be,
inconsistent
or
might
be
440
and
to
batch
layer
like
that
generates
the
real
data
as
a
complete
data.
Maybe
every
15
minutes
every
10
minutes,
depending
how
long
the
batch
would
take
this.
A
Just
in
a
side
note,
you
can
definitely
fill
a
whole
talk
on
lambda
architecture,
so
justice
and
possibility
possible
solution.
Furthermore,
we
have
to
install
compaction.
The
event
store
gets
very,
very
large.
As
I
said,
we
would
maybe
store
on
last
year,
plus
at
least
one
copy
of
each
entity.
We
could
you
could
use
the
spark
bachelor
for
it
together
with
Cassandra
TTL,
for
example,
to
expire,
the
entries
and,
furthermore,
you
could
have
a
snapshot
or
master
view.
A
Basically,
a
spark
streaming
application
which
connects
to
every
Kafka
topic
and
just
stores
the
data
he
gets
into
and
fled
Cassandra
chaebol
and
that's
the
data,
which
is
the
current
value.
It
could
not
only
be
used
to
initialize
some
new
news,
Park
job
or
new
queue
view,
but
could
also
be
analyzed
for
as
a
back-end
for
restore
like
Sola.
If
you
think
of
Daz
integration,
just
first
Kassandra
tables
use
the
solar
integration
and
you
have
a
search
engine
on
audio
data,
so
the
use
case
and
the
end
solved
with
event,
sourcing
and
cqs.
A
We
have
one
way
to
get
the
data
from
the
cross
systems
gets
processed
by
spark
and
every
internet
ready
application
says:
ok,
I
need
this
data
in
this
format.
I
need
only
this
data
in
this
format
and
every
time
we
just
create
a
small
spark
application
and
on
the
other
way
we
arrive.
The
same
architecture.
Internet
applications
populate
the
changes
to
the
proxy
and
spark
jobs
generated
the
output
ready
to
consume
by
the
legacy
core
systems.
If
they
are
up
again
one
tiny
attachment
we
have,
we
have
been
passed
to
get
a.
A
A
He
wants
to
do
a
transaction
by
some
articles
and
wants
to
deliver
to
them
to
his
new
address,
and
then
the
shop
of
course
needs
a
new
address,
so
we
can
say:
ok
in
the
clipboard
I'm,
not
only
interested
in
data
approved
on
and
processed
by
the
legacy
system,
but
also
on
data
which
isn't
it's
not
yet
approved
and
not
yet
processed
and
it
gets.
It
gets
marked
as
not
a
proof,
but
you
can
work
on
it.
That's
some
of
kinds
of
eventually
consistency.
The
the
client
systems
must
be
aware
of.
A
A
B
A
B
For
example,
an
article
is
being
sold
right,
so
they
are,
like
you
know,
50
events
that
I
have
occurred
and
the
last
demons
was
even
was
where
there
is
no
inventory
available
for
that
article
available
and
you
actually
push
all
the
events
and
then
process
them
and
update
the
master
data
only
afterwards
right.
So
if
the
forty-ninth,
even
it
is
processed
prior
to
the
third
event
at
that
time,
inventory
would
be
42
or
you
know,
and
at
the
last
event
would
be
at
around.
You
have
only
one
item
left,
so
there
will
be
a
inconsistency.
B
C
D
A
Thought
about
it,
yes,
but
coming
up
with
Cassandra
and
the
company
already
has
Cassandra
and
was
like
oh
another
database.
They
only
had
over
cut
years
ago
years
and
it
was
a
big
step
to
get
actually
a
new
database
which
Cassandra-
and
we
said,
okay
Cassandra
totally
fits
it's
okay,
we
get
done
in
the
job
done
with,
but
there
would
be
a
time
series
to
be
friends.
We
also
talked
about
using
HDFS
or
HBase
only
for
the
commits
or
for
the
event.