►
Description
From the Austin, TX Apache Cassandra users group:
http://www.meetup.com/Austin-Cassandra-Users/events/131802852/
Presentation slides:
http://www.slideshare.net/cassandraatx/bv-emodb
Overview:
Fahd Siddiqui from Bazaarvoice describes an internal datastore built on Apache Cassandra.
Description:
Introducing Bazaarvoice datastore (EmoDB)
EmoDB is a RESTful HTTP server used by Bazaarvoice for storing JSON objects and for watching for changes to those events. It also supports a blob store, a queueing service, and a data bus to track events.
It is designed to span multiple data centers, using eventual consistency (AP) and multi-master conflict resolution. It relies on Apache Cassandra for persistence and cross-data center replication.
A
A
So
keeping
that
in
mind
the
name
comes
from
a
word
play
from
story
or
sentiment
and
emotions,
and
all
that.
So
that's
why
we
came
up
with
emodv
a
little
bit
about
bizarre
voice
of
your
austin-based
local
company.
Well,
I
wouldn't
say
local
company
anymore.
It's
a
global
company,
austin-based
global
company.
A
So
I
don't
know
how
many
of
you
are
familiar
with
bizarre
voice
and
what
it
does,
and
this
is
more
than
a
marketing
slide.
I
just
want
to
set
some
context
on
what
kind
of
data
we
deal
with.
What
is
the
scale
of
our
data
and
even
though
it's
not
consumer-facing
site
chances
are
everybody
has
visited
our
servers
and
the
reason
is
that
we
are
on
the
product
pages
of
most
of
the
major
retailers
other
than
amazon.
A
And
so,
if
you
look
at
global
monthly,
unique
visitors,
we're
about
a
half
a
billion
monthly
uniques,
which
is
right
up
there
with
all
the
biggies
that
you
must
be
familiar
with.
I
think
number
seven
just
to
give
you
a
little
bit
more
context:
monthly
stats
and
I'm
not
going
to
go
over
every
one
of
them.
You
can
see
that
it's,
it's
large,
you
know
just
16
billion
review,
impressions,
everyone
and
all
those
nice
stats.
A
Emo
db
is
the
lowest
level
which
is
which
is
directly
interacting
with
the
data.
This
has
the
system
of
truth.
It
has
raw
data.
If
you
will,
you
call
it
system
record
and
it
is
based
off
of
backed
by
cassandra
and
it's
not
just
one
ring,
but
multiple
rings
for
different
things
and
we'll
get
into
the
details
for
that
there
are
massive
global
rights.
A
The
other
thing
that
we
have
interesting
is
we
need
to
watch
changes,
so
we
have
a
data,
bus
layer
right
here,
data
event,
bus
and
every
time
a
change
is
made
in
a
record.
It
should
trigger
an
event
on
that
data
bus.
So,
as
you
may
imagine,
there
are
going
to
be
listeners
on
that
data
bus.
So
that's
point
number
two
first
thing
is
massive
global
rights.
Second
objective
is
that
we
need
to
alert
any
subscribers
who
are
listening
whenever
change
happens.
A
A
Application
that
you
may
want,
so
this
is
not
exactly
database.
This
is
more
you
can
say
when
you
are
trying
to
boot
up
a
new
indexing
cluster
for
yourself.
You
want
to
go
ahead
and
okay
and
and
say
I'm
gonna
start
from
scratch,
and
I
know
you
have
a
lot
of
data.
We
have.
You
know
seven
years
worth
of
data
which
we
were
just
talking
about,
and
I
want
to
go
ahead
and
index
all
of
the
documents
that
you
want.
So
I
want
sequential
scans
and
I
want
to
do
it
quickly
enough.
A
A
Essentially
what
poloy
is
it
is
a
rules
engine
where
anybody
can
create
a
ploy
cluster
and
we'll
create
elastic,
search
cluster
for
them,
and
you
can
provided
rules
so,
for
instance,
we
have
product
reviews
and
ratings.
So
someone
can
say
you
know
what
I'm
not.
I
don't
care
about
reviews
as
much.
I
just
wanted
the
product
catalog,
but
one
small
little
thing
that
I
want
is
with
every
product.
A
I
want
average
ratings
and
some
statistics
on
that
and
that's
all
I
need
so
he
would
write
rules
using
our
domain
specific
language
that
we
wrote
for
floyd
and
he
would
say:
okay,
give
me
all
the
products
and
just
make
sure
that
for
all
reviews
you
do
you
take
the
average
you
take
the
max
you
take
the
minimum
and
so
on
and
so
forth.
Then
poloi
will
create
this
elastic
search
and
it
will.
A
A
You
should
start
storing
any
content
type
so
tomorrow,
instead
of
reviews,
you
want
to
store
recipes
or
something
you
should
not
have
to
re-architect
your
entire
solution,
and
this
provides
us
this
in
this
kind
of
30
000
feet
level
infrastructure
provides
us
that.
So,
if
you
have
another
application
by
the
way
it
doesn't
have
to
be
just
poloi.
I
know
in
the
diagram
it's
just
polaroid,
but
you
can
have
your
own
application
that
can
listen
on
data
bus
and
can
do
a
range
or
a
scan
for
all
the
documents.
A
A
A
D
A
We
are
really
going
to
focus
on
system
of
record
and
databus
reason
is
we
also
offer
queue,
service
and
blob
store,
but
q
service
is
very
much
based
on
the
data
bus.
It
is
you
can
call
it.
Our
internal
sqs
and
blob
store
is
pretty
much
like
system
of
record,
but
it's
more
tailored
towards
storing
images
and
videos,
and
things
like
that,
all
of
which
are
by
the
way,
backed
by
cassandra.
So
one
of
the
things
you
can
think
about
is
system
of
records.
Has
two
cassandra
rings,
one
for
ugc,
which
is
user-generated
content.
A
So
system
of
record
introduces
a
concept
of
tables.
This
is
not
to
be
confused
with
the
column.
Families
that
we
have
in
cassandra
table
is
just
just
that,
like
we
wanted
to
offer
our
consumers
a
way
to
create.
However
many
tables
you
want
to
create
and
it
has
to
be
very
cheap,
it's
basically
a
bucket
that
contains
json
documents.
A
What
you
would
think
the
table
should
be,
but
the
important
thing
is
that
we
can
have
tens
of
thousands
of
tables
and
the
other
thing
is.
It
offers
a
way
to
fetch
any
particular
raw
id.
Just
like
you
know,
you
have
cassandra,
but
here
you'll
have
a
restful
api.
You
can
give
it
a
row
id
and
it
will
return
json
back
at
you
and
then,
as
I
keep
harping
on
complete
cable
scan.
A
A
A
So,
and-
and
this
is
really
important
so
because
they're
tables
they're
going
to
be
roads,
so
concept
of
rows
is
very
interesting
for
us.
We
only
do
append
whenever
we
insert
we're
appending
deltas.
These
are
all
delta.
So,
when
you
are
writing
a
row
and
deltas
can
be
create,
update
or
even
delete,
so
a
delete
operation
for
a
particular
row
is
also
a
delta.
A
A
I'll
also
show
you
the
syntax,
how
it
is,
but
if
you
want
to
add
another
property
and
a
new
property
in
your
delta,
the
syntax
will
just
give
you
just
that
property
and
the
new
value
for
that
property,
or
it
can
change
an
existing
property,
but
regardless.
The
main
point
is
that
when
you
write
to
a
particular
row,
you're
just
inserting
one
delta,
you
don't
care
what
there
was
before
you
don't
care.
A
Due
to
this,
there
are
two
things
so
there's
also
a
concept
of
compaction
which
is
emo
compaction.
I'm
talking
about
this
is
not
cassandra
compaction
and
basically,
as
you
can
imagine,
these
deltas
can
grow
and
every
now
and
then
opportunities
opportunistically.
It
will
go
in
and
try
to
compact
these
delta
and
write
a
compaction.
Then
it
will
delete
a
lot
of
these
columns
and
put
a
compaction
record
in
that
in
the
in
the
place
of
that
and
move
on.
A
So
one
of
the
limitations
of
emodb
because
of
this
structure-
and
it
is
by
design-
is
if
you
have
a
high
update
to
create
ratio,
then
it's
it's,
probably
not
a
good
system
for
you.
So
if
you
think
about
user
generated
content
that
that
we
usually
deal
with
the
review
is
written
and
after
that
there
are
maybe
a
few
more
updates,
so
it
may
go
to
a
moderators
and
moderators
can
approve
it
or
you
know,
add
some
more
metadata
to
that
piece
of
content,
but
pretty
much
after
that,
it's
hands
off.
A
A
A
So
there
are
three
ways
to
read:
data
access
or
three
ways
of
data
access
from
emodb
is
look
up
by
primary
key
there's
bulk
extract
and
then
there's
change
feed,
which
is
the
data
bus.
So
what's
missing,
obviously
it's
a
wear
join
group
by
if
you're
thinking
about
querying
emo
db,
that's
not
what
it
does.
You
would
rather
have
polo
elasticsearch,
which
we
use,
that
that
is
our
main
thing
for
complex
queries.
A
Remember
I
talked
about
creating
tables,
it
has
to
be
very
cheap,
so
anybody
can
send
a
restful
api
and
say:
okay,
create
me
a
review
colon
test
customer
table
or
something
like
that,
and
it
should
quickly
create
that
table.
For
you
and
say
yes,
okay,
your
table
is
created
and
now,
when
you
create
that
table
expectations
are
okay.
Now
I'm
going
to
just
start
start
storing
documents
in
that
table,
and
then
I
should
be
able
to
scan
that
table
now.
A
If
we
were
to
dedicate
one
column
family
for
one
table,
that's
a
megabyte
of
memory
in
cassandra
1.1.
That
is
way
too
much
overload.
So
I'm
going
to
give
away.
One
thing
is
we
will
use
just
one
column
family
for
all
the
tables
now,
assuming
before
going
into
complete
solution
that
we
have
solved
this
problem
by
using
just
one
column,
family,
there
rises.
Another
problem
problem
number
two,
and
this
is
where
the
scanning
comes
in.
A
So
you
can
imagine
that
if
you
have
all
your
tables-
and
let's
say
somehow,
each
row
is
identifying
itself
that
I
belong
to
this
table.
That's
all
fine
and
dandy,
but
now,
when
you
want
a
sequential
scan,
you
have
a
problem
at
your
hands,
because
you
have
only
one
column:
family,
that
stores
all
the
tables
that
stores
tables
of
walmart
and
then
also
stores
table
of
best
buy.
A
A
A
Okay,
I'll,
go
move
on
to
solution,
then
so
a
solution
to
both
problem,
one
and
two,
the
roki,
the
roki
byte
buffer-
that
we
have
contains
a
nine
byte
table,
prefix,
followed
by
utf-8
encoded
key.
So
the
first
byte
is
an
eight
bit
chart
identifier
and
the
second
or
the
next
eight
bits
is
a
64-bit
table
usual
id.
So
it's
not
the
table
name,
but
it's
a
usual
id
that
you
create
for
that
table
and.
C
A
A
A
Your
shards
so
so
basically
chart
identifier,
serves
to
spread
content
for
the
entire
cluster
and
that
avoid
hot
spots
and
for
parallel
scan
the
another
good
thing
about.
It
is
because
we
have
eight
bits
of
chart
identifier,
you
get
256,
ranged,
queries
and
you'll.
You
can
do
those
range
queries
in
parallel
and
you
can
get
you
can
do
the
scanning.
A
The
table
uuids
also
solve
another
problem
for
us.
So
obviously
we
now
know
that
we
can,
you
know,
store
multiple
tables
because
we
have
table
uid,
but
also
because
we're
using
uuids
it
allows
us
to
drop
the
tables
and
right
away,
recreate
a
table
with
the
same
name,
because
this
time
it's
going
to
give
get
another
uuid
and
that
allows
us
to
delete
those
records
lazily.
So
they
may
exist
there,
but
nobody
can
access
there,
because
now
it's
a
completely
different
uid.
Even
if
the
names
are
the
same.
A
A
See
it:
okay,
I'm
not
gonna
go
over
the
code.
All
I'm
trying
to
show
here
is
that
using
java
client
you
make
one
call
to
get
the
splits
you
get
your
splits
here
and
then
in
your
for
loop,
you're,
basically
spawning
off
separate
threads
to
go
over
each
split
and
which
split
is
going
to
give
you
all
the
records.
A
It's,
as
is
it's
as
much
guarantee
as
a
random
order.
Partitioner
will
provide
you
and
that's
the
thing
about
the
8-bit
shard
identifier
that
I
was
kind
of
talking
about
earlier.
Because
of
that,
when
you
do
range
queries,
you
can
add
the
most,
because
you
have
eight
bits.
You
have
at
the
most
256
range
queries,
but
with
that
range
queries
you
can
completely
parallelize
it
and
you
can
just
get
sequential
scans.
A
A
I
A
You
don't
care
whether
I'm
creating
a
conflict
or
hey.
I
need
to
isolate
my
row
first
and
then
write
and
do
whatever
you're
just
writing
a
delta,
and
this
is
true
for
create,
update
or
delete
no
matter
what
you're
doing
that's
just
a
delta,
and
then
I
had
the
graphics
earlier.
You
just
you're,
just
appending
delta's
to
your
row
as
readers
are
asked
to
read
per
row
any
given
row,
they'll
just
go
in
and
they'll
evaluate
because
deltas.
A
So
what
can
happen
is
a
scenario
like
this,
where
you
have
assumed
you
have
two
data
centers
data
center,
a
and
b-
and
you
start
off
with
the
document-
something
like
this
and
we'll
call
that
delta
t
one.
So
so
at
t
one
time
you
have
this
document
and
both
data
center,
a
and
data
center
b
are
aware
of
t1.
A
Then
t2
comes
along
and
this
is
by
the
way
a
review
and
t2
says.
Okay,
I
need
to
add
another
property
status
equals
approved
and
it
goes
to
data
center,
a
that
data
center.
A
has
t1
and
t2
d3
goes
to
data
center
b,
but
d2
hasn't
reached
data
center
b.
Yet
and
what
d3
does
is?
It
adds
facebook
id
to
that
review.
A
So
at
this
point
in
time,
if
somebody,
if
some
application,
goes
to
data
center
b
and
asks
for
this
document,
they're
going
to
get
t1
and
t3,
and
if
somebody
goes
to
data
center,
a
they're
going
to
get
t1
and
t2
so
data
center,
a
does
not
have
facebook
id
data
center
b
does
not
have
status
as
of
this
time,
but
eventually
and
that's
why
we're
living
in
an
eventual
consistent
world.
So
two
seconds
later,
you
will
see
it
does
guarantee
that
you
will
have
of
both
properties
at
both
data
centers.
A
and
b.
A
Typically,
when
you
write
something
to
the
document
or
you're
modifying
the
row,
it's
replication
conflict
and
you're
gonna,
say:
okay,
which
one
is
the
right
one
which
one
should
I
go.
You
can
say:
okay,
I'll
go
with
g3
it's
latest,
but
this
one
doesn't
have
status
in
it,
so
you're
going
to
lose
that
for
us
this
is
much
easier
now,
because
each
delta
specifies
only
the
fields
that
it
modifies
and
so
the
deltas
merge
very
cleanly
and
and
produce
the
desired
result.
A
Also,
if
you
notice
that
there
are
no
cross
data
center
synchronous
communication
required
at
this
time,
all
that
is
required
is
regular
replication.
It's
eventually
consistent!
So
yes,
you,
if
you
there
may
be
at
some
point
where
you're
not
getting
all
the
fields
but
again
in
an
eventually
consistent
world
two
seconds
later
or
two
days
later.
If
the
note
is
down,
you
will
get
everything
the
the
right
document.
A
So
then
it
will
follow
the
regular
resolving
algorithm.
So
let's
say
if
your,
if
your
t3
was
modifying
the
status
as
well.
Remember,
oh
one
of
the
things
I
may
have
forgotten
to
specify
about
deltas
is
we
have
changed
uids
which
are
time
uids,
so
you
always
have
that.
So
if
your
t2
is
status,
approved
and
p3
is
status,
rejected
or
something
it
will
basically
replay
the
entire
thing
in
that
order.
A
G
A
A
A
It
still
does
not
affect
anything
because
guess
what
the
uuid
the
time
uuid
the
change
id
on
the
delta
is
still
t0,
and
when
I'm
writing
it,
the
writer
has
no
concern
whatsoever.
Then
you
know,
I'm
you
know,
have
newer
changes
in
there
or
anything.
I'm
just
going
to
go
ahead
and
write
this
delta
in
the
reader
when
the
reader
comes
in.
It
will
order
it
and
we'll
know
that.
Okay,
this
is
t0.
This
is
t1
this
t2
and
t3
and
we'll
resolve
it
in
that
order.
A
A
A
A
As
you
can
imagine,
there's
no
operation
like
just
modifying
a
list,
so
if
you
have
a
list,
you
can
either
replace
the
entire
list
or
you
can
delete
the
list.
If
you
have
a
situation
where
you
have
a
list,
but
you
want
to
apply
deltas
to
particular
elements
of
the
list,
then
you
model
that
as
a
map
and
if
you
are
not
sure
about
the
keys
to
use
for
this
map.
Usually
time.
Uid
is
a
good
candidate
for
listing
of
the
keys
main
reason
being
it
gives
you
a
natural
sort
order
right
away.
A
A
All
the
rest
of
the-
and
this
is
more
domain
specific
language
for
us,
but
when
we
have
a
dot
dot
in
front
of
it,
that's
that's
basically,
an
incremental
delta
that
you're
applying
and
whenever
you
want
to
delete
something
you
use
the
status
and
you
use
like
a
tilde
sign
which
tells
it
okay
delete.
This
key
from
my
record.
A
The
map,
one
is
pretty
self-explanatory.
Yes,
there's
one
thing:
the
last
one:
that's
a
conditional
delta
and
we're
going
to
talk
more
about
that.
What
basically
the
last
example
is
doing
is
it
removes
a
key
if
the
value
is
approved,
so
only
if
the
value
is
approved
then
go
ahead
and
remove
status.
I
don't
need
status
anymore,.
A
So
you
can
basically
say
that
I
want
to
you
know
regular
write,
but
before
you
do
a
write,
you
can
read
the
object.
You
can
say:
okay,
this
status
is
approved.
I
want
to
do
something
to
this
object
and
I
want
to
write
back,
but
I
want
to
write
back
conditionally
that
the
status
is
still
approved
and
it
hasn't
changed
since
then.
A
So
this
is
essentially
designed
to
help
resolve
the
most
common
concurrent
write
problems
on
modifying
your
objects,
because,
as
an
application,
you
kind
of
realize
that
this
is
my
most
concurrent
right.
There
will
be
multiple
writers
on
this
object,
so
this
will
help
you
the
common
concurrent
right
complaint.
It
you
reduce
is
a
simple
and
reliable
method
to
do
that.
Basically,
for
example,
in
this
one
is
basically
what
I
just
described.
A
If
you
want
to
mark
review
approved
only
if
moderation
hasn't
begun,
so
we
have
some
clients
who
want
optimistic
approach
to
review.
So
if
you
have
written
a
review,
it
will
publish
right
away,
but
then
the
moderator
will
come
later
on
and
if
there's
something
objectionable
about
that
review,
they'll
reject
it.
So
in
this
case,
what
what
what
the
conditional
delta
is
saying
is
that
if
the
submission
text
id
is
this
then
go
ahead
and
set
status
to
approve.
A
A
Basically,
what
it
happens
in
a
happy
state.
Is
you
read
the
original
state
you
compute
the
new
version
based
on
the
original
state
and
the
happy
path
is
then
you
write
conditionally
to
it
asking
it
only
right
if
the
original
condition
holds
and
write
goes
successfully,
because
nothing
has
happened
in
the
middle
or
what
can
happen
is
eventually
the
right
conflicts
and
data
bus
fires,
an
event
for
application
to
detect
it
and
retry
that
right.
So
what
I
mean
by
that
is
what
can
happen
is
the
application
to
the
application.
A
It
may
seem
like
the
write.
Just
went
through
no
problems,
but
later
on,
emodb
can
retroactively,
fail
your
right
and
give
a
trigger
on
the
date
of
us
that
we
talked
about
early
on
saying
that
hey
your
con,
there
was
a
conflict
in
one
of
your
conditional
rights.
You
know
do
something
about
it,
then
it's
up
to
the
application.
A
Downwards
so
initially
data
center,
a
and
data
center
b
are
synced
together,
they're
at
t1
delta
and
then
application
talks
to
data
center
a
and
says
okay
employ
this
or
apply
this
conditional
t3,
based
on
the
fact
that
the
original
delta
is
t1
and
currently
at
the
same
time,
just
a
little
before
that
t2
arrives
in
data
center
b.
That
invalidates
this
conditional.
A
A
A
If
you
will
p1
happen,
t2
happened
and
application
tried
to
write
d3
and
it
failed,
and
this
basically
there
is
no
locking.
Yes,
there
is
a
little
bit
of
different
approach
where
you
get
this,
but
then
you
also
get
a
database
trigger
date.
Of
thus
trigger
is
guaranteed
that
you
will
get
it
so
that
applications
can
react
to
it.
A
Okay,
so
trigger
will
happen
when
you're
pulling
this
data
bus
event.
You'll
know:
okay,
something
happened
and
we'll
tell
you
that
this
there
was
a
conflict
and
if
the
actual
document
is
t1
and
q2,
not
t1
and
g3
anymore,
and
then
basically
the
applications
can
either
retry
the
right
or
they
can.
You
know
forego
it.
F
J
A
A
A
This
is
exactly
what
will
be
stored
here.
This
will
never
get
changed
so
when
e1db
reads
the
record
at
this
time,
it's
going
to
say:
yeah
everything
is
fine,
but
when
t2
arrives
right
here
and
now,
when
the
record
is
red-
there's
a
problem
here,
so
it's
going
to
go
ahead
and
put
that
on
the
data
bus.
H
E
I
A
Because
you're
just
offending,
it's
always
going
to
be
this.
It's
always
going
to
be
that
it's
always
going
to
be
that
nobody's
going
to
change
this
at
all,
and
that
is
where
you
know:
you're
no,
less
compaction
compaction,
yes,
compaction
will
happen,
but
the
thing
is,
nothing
is
changing
in
this.
The
compaction
happens.
Obviously
we're
going
to
erase
everything
and
put
a
c1
here,
but.
A
I
A
C
A
Well,
they
they
have
to
look
into
the
data
bus.
What
we
guarantee
is
that
when,
when
something
like
this
happens,
there'll
be
a
trigger
on
data
bus
and
we'll
we'll
be
talking
about
data
bus
a
little
bit
more.
But
basically,
if
the
event
is
going
to
live
there
for
some
time
and
when
I
say
sometime,
it's
not
one
second
or
two
seconds,
it's
more
like
a
day
to
a
week.
A
K
K
D
A
K
I'm
curious
like
what
that
your
company's
approach
was
to
the
the
idea
that
you
know
universal
time
stamps,
ultimately,
don't
work
in
a
distributed
environment.
You
know
your
your
perception
of
what
universal
time
is
is
very
different
from
another
node.
So
did
you
guys
look
into
like
vector
clocks
in
order
to
do
this,
or
is
that
too
complex
and
alternative?
But
it's
really
the
the
one
correct
way
to
resolve
distributed
nodes,
communicating
and
what
came
first.
A
So
obviously
you
know
there's
a
lot
of
discussion
about
vector
clocks
and
you
can
think
about
it,
but
it
boils
down
to
this
is
there's
a
lot
of
overhead
for
running
vector
clocks.
So,
yes,
it
is
true
that
you
know
ntp
kind
of
employs
a
little
bit
of
vector
clock
behind
the
scenes
as
well.
People
sometimes
don't
know
this,
but
I
understand
what
the
question
is.
I
mean,
but
we
decided
just
to
go
with
like
tiny's.
A
A
We
let
applications
create
subscriptions
for
these
tables
that
they
created
in
udp
and
tables
are
basically
segments
in
column
families.
So
when
you
subscribe
subscribe,
you
can
tell
like
I
want
to
subscribe,
subscribe
to
this
table
that
I
created
and
so
anytime.
There
is
a
change
in
a
row
for
that
table
that
subscription
will
get
an
event
and
after
you
do
that
applications
kind
of
pull
periodically.
A
It's
kind
of
like
a
dvr
thing,
so
system
of
records
starts
dvr'ing
updates
at
that
point
it
supports
multiple
concurrent
writers
and
readers,
so
you
can
poll
and
then
you
can
acknowledge
them
as
well.
There
is
no
guarantee
on
order
on
databus
site
as
well.
So
although
the
guarantee
is
that
you
won't
get
the
trigger,
but
there
is
no
guarantee
that
you
may
get
t3
in
different
orders
and
that's
okay.
A
One
thing
that
to
help
system
of
record
to
help
in
this
regard,
because
nothing
is
in
order-
you
do
get
version
and
version
is
a
little
different
from
signature.
While
signature
is
a
hash
of
your
document
at
that
time,
so
you
can
always
ask
for
hey.
I
want
to
know
what
my
signature
is
at
this
point
and
you'll
know
exactly
what
it
is.
This
is
an
intrinsic
value.
A
A
That's
how
you
basically
subscribe.
There
are
two
parameters
to
look
at.
One
is
the
time
to
live,
and
the
other
one
is
eventually
one
is
for
the
subscription
how
long
the
subscription
is
going
to
be
there,
so
it
could
be
a
week
and
there's
also
to
renew
it.
You
just
basically
call
make
this
call
again,
and
it's
going
to
renew
your
subscription
event.
Time
to
live
is
basically
that's
how
long
events
are
going
to
live
on
the
databus
until
whatever
happens
first,
either
ttl
or
somebody
acknowledges
or
acts
that
event.
A
By
default,
I
think
it's
one
day
for
both
of
those
and
then
you
can
also
apply
cable
filters
on
these
things.
These
are
not.
You
cannot
have
filters
based
on
content
by
the
way,
so
you
can't
say
that
this
particular
status
is
approved
is
what
I
want
or
something
like
that,
but
you
can
use
intrinsic
value,
so
you
know
you
can
always
say
you
can
ask
for
multiple
tables,
so
you
can
say
type
equals
review
and
I
want
to.
I
want
a
trigger
for
that
and
you
can
have
triggers.
A
A
A
G
A
C
A
I
The
claim
information
is
stored
in
memory
when
you
use
consistent
hashing
to
get
you
to
the
server
that
owns
the
claims.
For
that
event,
okay,
and
so
we
can
scale
up
and
down
and
when
we
do
scale
up
and
down
the
server
that
your
consistent
hash
for
that
event
goes
to
might
move.
So
in
that
case,
you
might
get
duplicate.
Events
for.
K
I
I
A
So
this
is
basically
last
slide
that
I
wanted
to
put
for
blob
store,
which
is
pretty
much
similar
to
system
of
records.
I
mean,
of
course,
there's
no
deltas,
but
it's
like
it's
more
for
images
and
the
sweetest
spot
is
for
blocks
for
a
few
megabytes
and
it
replicates
to
all
data
centers
and
the
reason
why
not
amazon
s3,
you
can
totally
use
amazon
s3
with
upfront.
If
you
don't
need
cross
data
center
or
you.
F
K
A
Basically,
we
apply
some
intrinsic
values
for
every
documents
in
modb,
just
for
just
reasons.
One
of
this
is
version
and
first
update
time
last
update
time
signature.
So
if
you
want
to
do
something
like
conditional
write-up
on
signature,
that
is
totally
valid,
so
you
can
say
make
this
right.
You
can
read
the
object.
First,
get
the
signature
and
then
put
a
conditional
right.
There
write
this
only
when
the
signature
is
this
so
you're
kind
of
making
sure,
okay
that
there's
no
dirty
leads
or
dirty
rights
in
the
middle.
A
A
So
you
should
be
able
to
handle
that,
as
I
said,
we
can
retroactively
fail
the
condition
right,
but
what
we
won't
do
is
we're
not
going
to
come
in
the
way
of
appending
deltas,
and
so
one
of
the
challenges
we
have
is
making
sure
that
the
compaction
kicks
in,
because
if
you
have
like
a
really
long
delta
list
and
which
is
why,
right
in
the
beginning,
we
thought
of
it
for
a
minute,
and
we
realized
that
ugc.
It's
not
really
like
a
lot
of
updates
in
it.
B
D
A
B
K
Question
you
mentioned
that
the
applications
it's
multi-tenant
supporting
like
best
buy
and
walmart.
So
how
does
I
understand
you
don't
want
hot
spotting,
but
ultimately
is
what
happens
that
you
end
up
having
kind
of
interleaved
rows
from
best
buy
and
walmart
together
and
then
you're
scanning
over
that
in
order
to
just
wonder,
like
can
somebody
else?
A
A
And
so
you,
basically
once
that's
one
call
you'll
make
it
will
give
you
a
collection
of
splits
and
then
it's
totally
up
to
you
to
completely
parallelize
everything
and
just
get
this
a
split
and
it's
very
possible
that
you
are
going
to
get
a
split.
That
does
not
return
any
document,
because
guess
what
I
mean
it's
like,
we
just
gave
the
entire
space
and
we're
actually
even
thinking
of
depending
on
the
scale
and
performance.