►
Description
Speakers: Jonathan Halliday, Core Developer at JBoss & Rui Vieira, Postgrad Student at Newcastle University
Slides: http://www.slideshare.net/planetcassandra/jonathan-halliday
Performing ranking queries to find the most relevant documents, most popular urls, etc on huge datasets is trivial —if you're willing to wait a while for the answers. For those with less time to waste, this session describes techniques for performing such queries efficiently. We'll describe the ranking queries problem, outline the Cassandra CQL3 data structures and code that can be used to solve it and describe the trade-offs available. We describe intravert, an innovative server-side programming solution for Cassandra, and show how it can be used to reduce network usage and improve performance by filtering data closer to source.
A
So
this
isn't
joint
work
between
jboss
and
Newcastle,
so
I'm
the
jboss
and
the
Newcastle
been
sitting
over
here
and
it's
about
how
to
do
Top,
Gear
queries,
because
thus
far
I
haven't
been
able
to
find
any
presentations
of
these
there's
warren.
I
think
on
video
analytics
have
hinted
it
in
one
slide,
so
I
thought
was
about
time.
Someone
looked
at
this
in
a
bit
more
depth.
A
So
what
is
top
k?
That's
the
first
question
you
ought
to
use
top
queries
every
day.
Google
search
is
a
great
example
of
top
k.
You
give
it
some
keywords
and
it
gives
you
back
the
most
relevant
documents
and
time-series
is
another
one.
So
this
is
all
tool
called
analog.
Anyone
old
enough
to
remember
analog
web
analytics
before
Google
Analytics
took
over
the
market
just
put
in
something
retro
there
to
make
the
point
top.
K
queries
have
been
around
for
ages
and
ages,
and
this
is
really
about.
A
How
do
we
adapt
them
to
work
on
Cassandra?
So
we're
not
too
concerned
about
the
scoring
algorithm
in
the
perfect
for
this
presentation.
Things
have
a
school.
We
don't
really
care
how
it's
arrived
at.
It
could
be
PageRank,
it
could
be
simply
counting
things
that
have
occurred
from
a
log
file
and
there's
two
different
application
domains
for
those
and
this
document
search
where
we're
looking
at
words.
How
often
does
a
word
occur
in
this
this
document?
How
relevant
is
it
or
log
analytics,
which
is
normally
counting
things
over
a
time
period?
A
So
isn't
this
kind
of
straightforward
I
mean
you
give
this
to
any
junior
developer
and
they
go
we're
in
there
right
this
database
query
and
it
takes
them
about
10
minutes
and
yeah
lunchtime
presentation
over
bye-bye
everyone
and
not
that
simple.
Unfortunately,
if
we
look
at
what
this
is
doing,
we
have
a
table
here
that
has
some
kind
notion
of
an
identifier
for
the
entity,
the
document
and
some
kind
of
score
that
we've
computed
by
some
algorithm.
We
don't
really
care
about
and
we're
filtering
it
based
on
things
that
are
relevant
to
our
query.
A
So
I've
done
a
search
for
show
me
all
the
documents
that
have
in
them
and
show
me
the
most
relevant
ones.
I
want
to
order
them
by
score
answer
the
ones
for
the
highest
score.
The
most
relevant
ones
get
at
the
top
of
the
heap
and
I'm
going
to
limited
by
K,
because
I'm
only
wanting
the
first
hundred
or
the
first
ten
or
however
many
very
straightforward
stuff,
except
it's
not
because
potentially
I,
search
on
multiple
terms.
A
I
can
search
for
top
K
an
algorithm,
for
example,
and
that's
going
to
narrow
the
search
down
a
bit.
But
it's
going
to
make
the
search
more
complex
as
well,
because
now
I
have
to
do
group
by
operation,
and
this
is
where
the
problems
begin,
because,
whereas
the
first
query
that
is
valid
both
in
traditional
SQL
databases
and
in
Cassandra,
this
one
isn't
because
I
don't
have
a
group
by
primitive
in
Cassandra
anymore.
So
I'm
going
to
have
to
do
some
more
work
here.
A
A
A
So
if
we
take
efficiency
to
mean
develop
a
tie,
one
of
the
ways
we
can
do
this
is
we
can
implement
one
of
these
tools
that
will
allow
us
to
to
take
sequel,
queries
and
we'll
compile
and
data
Hadoop
MapReduce
jobs
and
we
can
be
another-
were
Cassandra
cluster.
So
if
we
take
something
like
and
Paulo
a
string
or
any
of
this
family
of
tools,
we're
pretty
much
done,
we
can
just
take
out
the
sequel.
Query:
we've
already
got
feed
it
to
the
tool
and
it'll
give
us
an
answer.
A
So
that's
efficient
in
terms
of
the
developer
time.
It's
not
efficient
at
runtime.
It's
it's
going
to
give
us
an
answer,
but
it's
not
going
to
give
us
an
answer
fast
and
that's
a
problem
because
we're
interested
in
real
time
top
k
and
what
I
mean
by
real
time
for
the
purposes
of
this
presentation
is
not
the
old
computing
science
hard.
Real-Time,
miss
our
control
systems,
nuclear
reactors
kind
of
thing.
This
is
web
page,
rendering
real
time
this
is
milliseconds.
A
One
of
them
was
the
the
ingest
bit
of
the
pipeline,
so
the
basic
idea
is
usual
in
Cassandra's
precompute
as
much
as
possible.
We're
going
to
calculate
the
scores
ahead
of
time,
using
whatever
expensive
algorithm.
We
have
that
we're
going
to
store
them
into
to
cassandra's
just
be
computed
values
and
we're
going
to
assemble
these
pre
computed
fragments
the
group
by
operation
the
query
time.
A
We
did
slightly
different,
really
what
we
do
is
we
select
reach
term.
We
issue
a
separate,
select
and
reason
we
do.
That
is
that
some
of
those
are
going
to
return
fairly
quickly,
which
means
we
can
get
on
with
possessing
straight
away
in
the
client,
while
we're
still
waiting
for
the
other
ones.
So
it
cuts
out
a
bit
of
the
latency
that
you
get.
A
If
you
just
issue
a
single
bigquery
and
rely
on
the
Cassandra
coordinate
an
ode
to
pull
that
data
together
from
the
various
nodes
where
it's
actually
stored
in
the
ring,
it's
inefficient
because
we're
still
having
to
pull
in
all
the
data
for
each
of
these
terms.
So
how
can
how
much
data
do
we
actually
need
up
this
big
set?
Can
we
limit
it
so
that
we're
moving
less
data
across
the
network?
We
move
it
more
quickly.
A
So
I'm,
assuming
you
know
a
little
bit
about
cql
three
I
was
told
the
audience
of
these
things
was
highly
technical,
but
I
was
told
that
by
marketing
guys,
data
is
stored,
keyed
on
the
term
of
the
time
period,
depending
on
your
domain.
If
it's
document
search,
you've
got
the
notion
of
a
term
of
word
in
the
document
if
it's
searching
events
in
logs
you've,
typically
bucketed
it
by
time
period,
which
could
be
a
day
or
month
or
whatever,
so
you
can
limit
that
by
creating
over
the
partition.
A
Key
in
your
sequel,
query
in
you,
you
just
say
we're
keen.
You
know
this
term
this
time
period
and
Cassandra
will
limit
the
amount
of
data.
It's
looking
at
the
number
of
rows,
it's
pulling
off
disk
sending
to
you,
but
what
we
really
want
to
do
is
limit
on
the
next
bit.
We
want
to
limit
on
score
as
well,
because
we
have
a
long
tail
effect
here.
We
have
a
lot
of
documents
that
do
contain
the
term,
but
have
very,
very
low
score.
So
they're
never
going
to
appear
in
the
top
cake.
A
There
might
be
scores
over
huge
range,
thousands,
and
it's
only
what
end
we're
interested
in.
We
don't
want
all
the
rest,
because
if
we
get
all
of
it,
we're
going
to
have
to
move
it
across
the
network
and
that
takes
time
serializing
deserializing
we're
going
to
have
to
look
at
it
in
the
client
and
decide
whether
we
want
it
or
not,
which
is
going
to
take
time
for
the
comparisons.
A
It's
not
terribly
efficient.
It's
not
going
to
get
us
down
to
real
time
because
our
data
set
is
so
vast.
So
at
this
point
we
need
a
clever
algorithm.
We
can't
rely
on
Cassandra
to
do
this
for
us,
because
we're
getting
into
application
specific
algorithm,
see
it.
Well,
then,
the
more
general
data
algorithms
that
Cassandra
gives
us.
A
So
this
point
we
go
out
and
see
what's
already
being
done
because
as
I
hinted
out
earlier,
this
is
a
fairly
old
problem.
Smart
people
have
already
thought
about
this,
so
there's
some
algorithms
out
there
that
kind
of
more
or
less
do
what
we
want.
The
difficulty
is,
these
are
fairly
old,
they're,
not
Cassandra.
Specific
they're
actually
turns
out
mostly
for
doing
log
analysis
and
content
distribution
networks.
A
So
if
you
think
about
a
content
distribution
network,
it's
a
bunch
of
servers
that
are
spread
around
the
globe.
They
linked
together
by
a
fairly
slow,
fairly
high
latency
network.
If
you're
serving-
let's
say
video
files
of
these.
Each
of
these
is
logging,
what
it's
doing,
who
are
serving
those
files
to
you,
how
many
times
it's
serving
them,
but
to
get
a
global
picture
of
your
traffic
to
know,
which
of
which
were
the
most
popular
files
in
the
giving
period.
A
You
need
information
mid,
spread
around
all
these
machines,
so
you
need
some
efficient
way
of
getting
back
just
the
top
k.
The
top
hundred
most
popular
files
you've
served
off
your
CDN,
so
these
algorithms
have
been
designed
for
this
kind
of
environment
which
looks
almost
like
Cassandra
and
that
it's
a
bunch
of
distributed
nodes
except
the
network
between
them,
is
different.
What
we
have
is
a
slow
high
latency
network
grows
in
Cassandra
you're,
typically
in
the
same
data
center,
so
you
might
have
much
more
bandwidth
and
it's
much
faster.
A
So
we
want
to
take
these
existing
solutions
and
we
want
to
adapt
them
to
this
new
environment,
because
these
algorithms
were
already
tried
and
tested.
We
don't
really
want
to
cook
up
something
ourselves
because
we're
going
to
make
mistakes
and
it's
going
to
take
us
longer,
but
at
the
same
time
we
need
to
be
aware
that
the
algorithms
were
not
designed
for
exactly
what
we're
using
them
for
so
we
need
to
keep
an
eye
on
on
that
aspect
of
the
problem.
A
So
what
does
one
of
these
algorithms
look
like
I'm
not
going
to
go
into
too
much
depth?
This
is
the
kind
of
non-technical
version,
semi
technical
version-
if
you
want
to
understand
the
algorithms
more
depth,
control
plus
after
but
the
simplified
version
is
this:
they
fetch
you
a
little
bit
of
the
data
and
may
they
sample
it
effectively.
A
They
look
at
the
characteristics
of
it,
didn't
decide
how
much
more
they
need
and
they
go
and
fetch
that
next
chunk,
and
potentially
they
repeat
that
until
they've
got
enough
to
be
satisfied
that
they
can
give
an
accurate
answer
for
the
the
top
K
items
from
this.
Is
you
don't
want
to
do
too
many
cycles
of
that?
Because
each
of
those
is
costing
you
time?
Each
of
them
is
a
network
round
trip
and
the
milliseconds
are
mounting
up
and
your
clients
SAT
there
in
front
of
their
browser
wandering
around
their
page,
hasn't
refreshed.
A
Yet
it's
not
good.
So
what
do
you
want
to
do
to
limit
the
number
of
rounds
of
communication
and
there's
a
trade-off
between
using
a
few
rounds
in
which
you
achieve
a
lot
of
data
versus
using
more
rounds
in
which
you
achieve
only
a
little
bit
of
data
in
each
one
and
there's
some
kind
of
balance
to
be
hit
there
and
that's
essentially
what
the
algorithms
trying
to
achieve.
A
So,
in
terms
of
evaluating
these
algorithms,
the
key
is
to
look
for
ones
that
have
a
fixed
number
of
rounds
of
communication,
because
that
allows
us
to
have
a
predictable
response
time.
There's
an
upper
bound
on
how
long
it's
going
to
take
in
the
algorithms
that
don't
have
this
characteristic
with
degenerate
data
data.
That's
exactly
the
wrong
shape,
you're
going
to
get
into
a
situation
where
it's
taken
a
great
many
rounds
of
communication
and
that's
going
to
blow
your
time
budget.
A
The
other
characteristic
you
want
to
look
for
generality,
particularly
with
regard
to
the
distribution
of
the
data.
Most
data
follow
certain
patterns
and
that's
great,
but
sometimes
it
doesn't,
and
if
the
algorithms
have
been
tuned
or
designed
only
for
certain
that
characteristics
sooner
or
later,
you're
going
to
find
suddenly
a
production
data
doesn't
meet
that
expectation
and
things
are
going
to
start
going
wrong.
So
you
want
a
general-purpose
to
where
you
don't
have
to
worry
about
that.
A
Ideally,
since
I'm
a
middleware
guy,
I
also
want
it
to
be
flexing,
because
I
won't
put
this
out
to
a
number
of
different
clients
and
number
different
use
cases
and
have
them
all
be
able
to
use
the
same
thing
but
ideally
tune
it
for
their
specific
environment.
So
some
kind
of
flexibility,
some
kind
of
tuning
in
the
algorithm
would
be
a
very
useful
characteristic.
A
So
you
can
say
give
me
the
top
k,
with
a
certain
degree
of
accuracy
and
you're
trading
off
accuracy
against
speed.
Essentially,
you
get
a
quicker
answer,
but
it
might
not
be
perfect
and
for
a
lot
of
applications.
That's
fine!
You
don't
really
care
if
doubt
it.
The
bottom
of
you
you're
top
K
range.
The
hundredth
item
is
actually
the
engine
first
and
you're
not
seeing
the
real
hundredth
one,
because
it's
been
demoted
in
the
wrong
place
and
that's
okay
for
many
applications.
A
So,
having
identified
the
right
algorithms,
you
then
sit
down
and
implement
them,
and
you
run
into
certain
problems
and
one
of
the
maze
these
algorithms
assume
server-side
code
execution,
they're
designed
to
have
a
proprietary
server,
sat
on
the
nodes
where
the
data
resides
that
can
execute
code.
That
is
specific
to
the
algorithm
and
that's
a
problem,
because
what
we
have
is
to
cassandra
and
what
little
execute
is
tql
three
and
if
I
can't
express
the
algorithm
in
terms
of
CTL,
three
I've
got
a
bit
of
a
problem
turns
out.
A
There's
certain
steps
in
these
algorithms
that
you
can
only
implement
on
top
of
Cassandra
if
you're
willing
to
incur
extra
RAM
chips
and
extra
transit
over
network.
And
what
you
have
to
do
is
pull
back
more
data
than
you
strictly
need
in
order
that
you
can
call
calculations
on
it
in
client
code,
because
you
can
do
whatever
you
like
in
the
client.
You
can
execute
this
custom
logic
and
then
send
back
additional
cql
through
queries
to
the
node
that
essentially
contain
the
results
of
those
calculations.
A
So
the
problem
with
this
is
that
the
performance
comparisons
in
the
literature
no
longer
valid,
because
they're
assuming
certain
characteristics
of
the
environment,
don't
hold
for
us
in
Cassandra.
So
we
can't
trust
the
evaluation
of
the
algorithms
anymore.
We
can't
say:
oh
look
in
the
literature,
this
algorithm
is
the
best
one.
It's
far
more
performant
than
any
others.
A
Therefore,
I'll
implement
that
work
turns
out,
doesn't
quite
work
that
way,
so
you
have
to
implement
all
of
them
and
see
how
they
compare
so,
fortunately,
the
heavy
liftings
been
done
for
you
here
is
some
some
some
results.
So
this
is
data
transfer
of
the
network.
So,
in
the
context
of
the
academic
literature,
the
algorithms
have
been
published,
data
transfer
is
the
key
metric,
because
they're
working
on
a
lan
data
transfer
is
expensive
and
slow.
A
So
that's
what
they're
optimizing
for
so
the
first
thing
to
notice
is
the
dotted
line
horizontally
across
the
top
of
the
graph.
There
is
the
naive
version
and
it's
way
big
a
lot
of
data
moving
there
that's
going
to
be
too
slow
for
us
most
of
the
other
algorithms.
The
clever
algorithms
are
way
down
at
the
bottom
here,
much
much
much
more
efficient
in
terms
of
the
data
they're
moving
the
one
to
watch
is
that
slightly
sloped
line,
but
the
first
glance
looks
like
it's
the
worst
of
the
clever
algorithms.
A
That's
actually
a
variation
we
developed
specifically
for
cassandra
and
the
reason.
Why
is
that
when
you
look
at
execution
time,
modern
data
transfer?
Suddenly
that
line
is
down
the
bottom?
It's
suddenly
the
most
efficient
algorithm
in
terms
of
execution
time
so
because
we're
evaluating
against
a
different
metric,
because
we
care
primarily
against
execution
time
instead
of
data
transfer,
because
we're
on
land
data
transfer
is
comparatively
cheap.
Now
we
made
some
changes
to
the
algorithm
that
make
it
more
efficient.
Essentially,
what
we're
doing
there
is
in
the
base
version
of
the
algorithm.
A
That
is
Mel
a
sentence.
Yes
very
much
depends
on
your
test
data
right
check
it
with
a
grain
of
salt.
What
are
you
interested
in
is
the
slope
of
the
lines
really
rather
than
the
absolute
numbers,
because
your
deck
is
going
to
differ
your
network
scan
it
differ,
so
that's
as
a
function
of
K.
Most
of
those
lines.
The
line
is
sort
of
low
enough
ingredients
that
you
can
probably
afford
to
calculate
more
than
you
want.
A
So
if,
for
example,
this
high
probability,
the
user
is
going
to
look
at
the
top
10
and
then
click
Next,
you
can,
for
example,
go
for
the
top
20
straight
off
and
just
cash,
the
second
page
ready
for
them
so
execution
time
against
peer.
What
the
heck
has
appears,
the
obvious
question
here
appear
is
a
node.
You
want
to
talk
to.
You
essentially
remember
that
the
data
is
grouped
by
term.
That's
the
primary
key,
that's
giving
you
the
distribution
to
the
ring.
A
A
One
final
one
don't
want
to
use
T
boat,
so
I
said
there
are
three
candidate
algorithms
and
they
mostly
work
fairly
well
they're,
mostly
in
the
same
kind
of
ballpark.
Here
at
least
as
far
as
degree
bit
goes.
That's
the
base
case.
I
said
your
data
can
have
different
shapes
and
one
of
the
cases
where
that
happens
is
where
one
of
the
piers
is
responsible
for
the
bulk
of
the
score
mass.
A
The
bulk
of
the
data,
it's
proportionately
more
important
than
the
others,
and
if
that
happens,
cheaper,
doesn't
work
so
well,
the
other
ones
for
that
kind
of
data
they
actually
get
more
efficient.
That's
a
little
black
bit
at
the
bottom
of
the
clean
hybrid
threshold.
T
boat
doesn't
get
anywhere
near
as
efficient.
In
fact,
it's
very
inefficient
compared
to
the
others.
A
So
your
mileage
is
going
to
vary
just
this
with
your
own
data
test
it
on
your
own
hardware,
because
you're
going
to
have
different
characteristics
to
what
we've
tested
with
in
particular,
try
not
to
use
synthetic
data.
If
you
can
avoid
it,
try
it
with
actual
real-world
data.
If
you
have
it,
but
in
general,
the
recommendation
boils
down
to
if
you
want
an
exact
answer,
use
hybrid
threshold
and
when
I
come
on
the
introvert.
Look
at
this
talk
next
you'll
see
for
implementing
hybrid
threshold.
Introvert
is
kind
of
optional.
A
You
can
actually
express
hybrid
threshold
pretty
efficiently,
just
in
terms
of
cql.
Three
clears
an
approximate
algorithm.
So
if
you're
happy
with
approximate
answers,
it's
more
efficient
than
hybrid
threshold,
you'll
get
an
approximate
answer
faster.
The
downside
is,
it
does
require
some
metadata
after
the
really
efficient
version
of
CLE
there's
an
extra
step
in
the
algorithm
that
makes
it
more
efficient
but
which
can
only
be
implemented
using
introverted,
there's,
no
way
to
express
it
efficiently
in
cql.
Three,
so
you've
got
a
trade-off
here
between
characteristics.
A
A
So
what
the
heck
is
introverted
introvert
is
essentially
an
enhancement
to
Cassandra.
It's
a
way
to
add
extra
functionality.
To
the
server,
so
what
they're
doing
is
swapping
out
the
main
class
oops,
the
node
and
in
the
same
jvm
in
the
same
process
they're
bringing
up
the
server
that's
based
on
vertex
anyone
used
vertex
anyone
heard
of
it
whew,
okay,
so
vertex
is
a
bit
like
nodejs
in
that
it's
a
kind
of
a
container
based
on
a
synchronous
design
exhibit
through
the
JVM.
A
So
it's
polyglot,
many
language
targets,
the
JVM
all
and
happily
in
vertex,
so
using
this
they've
added
certain
functionality
on
to
Cassandra.
The
first
thing:
they've
added
is
a
new
API.
That's
based
on
restful
design
principles
uses
form
Jason,
as
the
data
HTTP
is
to
transport
layer.
If
you've
worked
with
certain
other
no
SQL
databases
you'll
recognize
that
kind
of
design.
It's
great.
If
you've
got
a
JavaScript
client,
you
want
to
talk
direct
to
your
database
without
map
server
to
you
in
the
middle.
A
It's
kind
of
been
done
before
it's
not
absolutely
novel
and
unique
too
introverted.
There's
another
project
called
Virgil.
Did
this
awhile
back,
but
for
our
purposes
it's
really
not
that
interesting.
We
already
have
a
perfectly
good
transport
for
our
purposes
in
the
form
of
the
native
protocol
to
talk
cql
3
to
the
server.
A
We
can
do
multiple
commands,
/
crawl,
which
is
great.
We
can
save
on
some
round
trips
by
patching
things
up,
but
we
can
also
change
some
stuff
sending
a
batch
that
will
essentially
execute
in
parallel.
We
can
say,
executed
this
operation
and
create
what
a
traditional
database
would
call
temporary
table
or
a
cursor.
Some
kind
of
notion
of
something
you
can
refer
to
in
another
query
say:
take
the
operation
here
and
execute
this
next
operation
pass
the
result
fit
set
from
one
to
the
next
using
a
reference.
A
So
introverted
gives
the
server-side
code
execution
in
pretty
much
any
language
theoretically
and
practice.
The
API
right
now
is
groovy.
We
can
do
things
like
filtering
result
sets
it's
fairly
trivial.
We
can
send
a
c
ql
free
operation
and
a
filter
up
expressed
in
groovy.
We
ship
them
over
together
and
it
will
filter
the
results
set
on
the
node
and
send
us
back
just
the
results
of
meat
filled
criteria
we
can
do
triggers.
A
This
is
not
so
important
if
you've
already
made
the
jump
to
Cassandra
too,
because
obviously
now
has
trigger
support,
as
you
saw
in
the
keynote
it's
great
if
you're
still
stuck
on
the
older
Cassandra's
and
want
some
kind
of
right
path
trigger,
but,
unlike
trigger
support,
introverted
will
also
do
re
path.
Work,
not
just
right
path.
Works.
You
can
execute
arbitrary
logical
unit.
A
Are
filters
is
just
a
specialist
case
of
that
you
can
also
effectively
extend
introverted
if
you're
willing
to
hack
it
a
little
bit
to
add
extra
operations,
so
the
basic
operations
it
comes
with
are
the
ones
you
would
expect
from
Cassandra.
You
know
get
some
puts,
and
a
few
new
ones
too
and
chain
result
sets
and
do
triggers,
but
you
can
also
arbitrarily
extend
it
with
your
own
logic.
A
If
you
wish,
and
doing
that
in
introverted
is
a
lot
easier
than
the
way
you
had
to
do
it
in
the
old
days
in
Cassandra,
which
was
to
extend
the
thrifty
APR,
so
in
summary,
introverted,
is
giving
you
a
trade-off
between
additional
power
to
express
your
queries,
additional
power
to
do
operations
on
the
server
versus
operational
complexity,
because
it's
embedded
inside
the
Cassandra
node.
It's
not
adding
as
much
complexity
to
your
operations.
A
People
as
running
a
completely
separate
server,
I
can't
very
well
go
to
the
ops
guys
and
say:
oh
I
want
to
do
this
top
K
thing.
So
here's
a
new
server
process
and
you're
going
to
have
to
put
this
on
all
the
nodes
and
you
can
have
to
monitor
and
manage
it,
and
it's
going
to
have
memory
leaks
and
yeah
like
may.
Instead,
any
tool
they
already
use
to
monitor
Cassandra
will
monitor
inch
of
it
because
it's
the
same
thing
it's
the
same.
Jvm
does
add
some
development
complexity.
A
A
Three
and
you
know,
you've
got
a
performance
problem
good
to
bring
this
in
bring
an
introvert
as
a
tool
to
solve
those
few
cases
where
your
client
is
getting
overly
slower,
overly
complex,
because
the
limitations
of
cql
three
one
of
the
drawbacks,
at
least
for
our
use
case
in
the
client,
is
that
in
the
introverted
clients,
not
really
got
a
great
awareness
of
Cassandras
ring
topology.
So
I
mentioned
earlier.
A
One
of
the
things
we
do
is
that,
instead
of
sending
a
single
query
to
Cassandra
a
random
Cassandra
node,
which
acts
as
the
coordinator
and
expecting
it
to
farm
that
out
to
the
data
nodes,
we
send
a
bunch
of
queries
in
parallel
to
each
of
the
nodes
that
owns
the
data.
Matt
Cutts,
an
extra
network
hop
out
of
the
cycle
for
us,
that's
tricky
to
do
with
introvert,
because
it's
not
designed
to
be
deployed
that
way.
A
The
way
you
normally
do
introverted
is
you
deploy
a
few
servers
as
introverted
servers
and
because
they're
also
part
of
the
ring
they're
communicating
for
you
so
they're.
Both
the
coordinator
in
Cassandra
terms,
aren't
the
the
app
server
if
you
like,
we
instead
deploy
so
that
every
Cassandra
node
is
also
an
introvert
node,
and
we
want
to
talk
directly
introverted
client
to
whichever
one
we
know
owns
the
data,
so
that
adds
a
bit
of
complexity
to
the
client
code.
A
So
let's
return
to
these
clever
algorithms
with
introverts,
we
can
now
do
server-side
executions,
so
we
can
now
have
an
execution
model
that
more
closely
maps
to
what
the
designs
of
these
algorithms
intended.
We
can
implement
this
code
much
more
cleanly
compared
to
the
description
of
the
you
know,
the
algorithm
that's
been
provided
to
us.
We
don't
have
to
put
a
new
round
trips,
just
a
couple
of
limitations
of
the
query
language.
The
Cassandra
gives
us
anymore
trade
off,
for
that
is.
A
We
have
added
some
development
complexity
and
some
operational
complexity,
but
it's
a
nice
middle
ground
between
the
complexity
that
we
would
get
the
much
larger
complexity
we
would
get.
If
we
wanted
to
deploy
a
completely
custom
server.
We
can
reuse
a
lot
of
our
existing
cassandra
tools
to
do
this,
whereas
we
couldn't,
if
we
had
to
write
a
completely
custom
server.
A
Okay,
what
have
we
got
next
pre
aggregation
right,
so
I
showed
graphs
of
the
the
performance
scaling
as
a
function
of
the
number
of
peers.
One
of
the
things
you
can
obviously
try
and
do
is
reduce
the
number
of
years
you've
got
and
that
means
using
denormalization
to
pre
calculate
the
aggregates.
So
it's
hard
to
do
for
text
search
because
it's
very
difficult
to
predict
which
terms
are
going
to
be
queried
together.
There's
a
handful
of
cases
where
you
can
do
it
and
people's
names.
A
A
If
you
try
to
do
all
possible
combinations
time
periods
is
different
time
periods,
you're
almost
always
going
to
query
continuous
time
periods,
so
they
might
be
days
that
are
next
to
each
other
to
form
week
or
weeks
in
the
next
to
each
other
to
form
a
month.
So
pre
calculate
these
robes
because
that's
going
to
reduce
the
number
of
buckets
the
number
of
peers,
you
need
to
query.
A
This
is
where
the
choice
of
algorithms
becomes
important,
because
now
you
might
have
a
query
that
spans
over
a
big
bucket,
like
a
month,
plus
a
number
of
smaller
buckets
like
days
to
bring
you
up
to
the
exact
time
period
you
want.
In
that
case,
11
of
the
buckets
has
a
much
greater
importance,
much
greater
waiting
than
the
others.
So
that's
why
you
want
to
avoid
the
algorithm
that
doesn't
scale
well,
when
your
data
has
that
shape.
A
Okay,
really
really
clever
algorithms.
If
that
wasn't
enough
to
make
your
head
explode,
we
can
get
into
some
more
complex
models.
I've
been
using
the
term
pier
to
mean
Cassandra,
know
a
place
where
data
resides
there
and
said
the
number
of
peers
you
need
to
talk
to.
You
is
the
same
as
the
number
of
creating
terms
you
have
roughly.
A
This
is
not
quite
the
case,
because
the
hashing
might
mean
that
certain
bits
of
data
actually
co-located
certain
logical
peers
in
the
computation
might
have
a
hash
such
that
they're,
actually
physically,
located
and
saying
to
sound
remodeled.
In
that
case,
what
you'd
like
to
be
able
to
do
is
talk
to
that
node
and
say:
hey
I
know
you
own
the
date
of
this
pier
in
for
this
pier
and
what
I'd
like
you
to
do,
is
John
them
together
and
send
me
back
the
pre
aggregated
data.
A
Do
a
local
group
by
over
just
those
terms,
send
me
the
results
and
then
I'll
do
another
group
by
over
all
the
joined
up
results.
I've
got
from
the
nodes,
so
you're
building
a
two
level
hierarchy,
you're
doing
group
by
in
the
client
but
you're.
So,
where
you
can
do
new
PI
on
the
local
data
roaming
note,
and
that
saves
you
a
lot
of
extra
transfer
in
cases
where
you
have
a
very
high
number
of
peers
and
therefore
a
high
probability
that
some
of
them
can
map
the
same
number
of
Cassandra
modes.
A
It's
all
about
the
ratio
between
the
number
of
peers
on
the
number
of
actual
nodes
involved.
Again,
your
mileage
is
going
to
vary
depending
on
how
many
nodes
you
have
in
the
ring
and
how
many
peers
you're
querying
over
some
very
interesting
later
work
on
budget
constraint
top
K.
So
this
is
where
you
can
tolerate
approximate
answers,
and
you
have
some
kind
of
time.
Budget
I
want
an
answer
within
30
milliseconds
within
300
milliseconds.
Whatever
my
time
budget
is
just
give
me,
the
best
results
you
can
in
that
time
period.
A
It
doesn't
matter
if
they're,
not
perfect,
I
have
to
have
them
quickly.
So
there
are
certain
algorithms
that
are
aware
of
budgets.
You
can
express
that
budget
in
timber
terms
of
a
number
of
constrained
resources,
but
most
likely
time
and
potentially
number
of
I/o
operations,
and
so
one
of
the
things
we
want
to
look
at
in
future
work
is:
how
can
we
design
an
algorithm?
How
can
we
implement
an
algorithm
that
is
aware
of
how
much
work
cassandra
is
having
to
do
to
implement
certain
primitive
operations?
A
We
also
want
to
look
at
some
notion
of
fault
tolerance.
The
cases
where
nodes
are
down,
ideally
you've
got
enough
fault
tolerance
just
through
replication
to
deal
with
that.
But
there
are
cases
where
you
haven't
or
where
you
haven't
noticed
failover
and
by
the
time
you
do
it's
too
late
to
reissue
the
query
and
in
those
cases
you'd
like
to
be
able
to
give
back
in
approximate
answer
and
say:
hey
I've
done
the
best
I
can
given
the
data.
A
What
can
say
amount
of
time,
I'll
just
have
to
go
with
the
best
done
so
I
have
so
those
are
very
advanced
bits
we
haven't
got
around
to
implementing
yet
but
show
you
that
this
is
really
just
the
tip
of
the
iceberg
in
terms
of
tuning
this
for
certain
use
cases,
whew
okay
made
it
to
the
end
in
time.
Time
for
questions.
If
you
have
any,
if
I
made
your
heads
explode
now
you
saw
with
me,
you
really
are
a
technical
audience
great.
A
We
hope
so.
Yes,
it's
a
sort
of
research
prototype
stage
right
now,
so
don't
expect
to
be
able
to
put
it
straight
into
production,
but
yeah
it's
a
case
of
cleaning
it
up
a
little
bit.
You
know
wrightington,
making
sure
the
the
tutorial
works
basically
and
slinging
it
out
there.
So
you
expect
it
soon.
A
A
A
So
comparing
that
to
introverted
would
be
really
interesting
and
eventually,
hopefully
taking
introverted
out
the
picture
and
as
all
the
capability
we
need
comes
into
Cassandra
itself,
we
can
phase
out
in
Traverse
introverted,
it's
kind
of
a
stepping
stone
because
Cassandra
doesn't
have
this
functionality.
Yet
another
interesting
benchmark,
I'd
like
to
do
and
have
enough
time
for
yeah
just
to
port
this
to
hbase,
because
H
pace
has
coprocessors.
It
already
has
reading
right
side,
arbitrary
code
execution
on
the
server,
so
benchmarking
against
HBase
would
be
really
interesting.
One
any
more
questions.
A
Right
right,
yeah,
so
there's
there's
a
couple
of
problems
of
that.
You
have
to
know
how
many
there
are
if
you're
going
to
stream
it
right.
So
you
have
to
pre
calculate
that,
but
assuming
you've
got
that
information.
Yes,
because
you
you're
writing
effectively
a
filter,
yeah
yeah,
it's
very
efficient
for
that.