►
From YouTube: C* Summit EU 2013: Building a CQL Driver
Description
Speaker: Theo Hultberg, Chief Architect at Burt
Slides: http://www.slideshare.net/planetcassandra/theo-hultberg
I'm not a database driver expert, I'm just a Ruby developer who really likes Cassandra, but I've written a CQL driver. It wasn't hard, and it feels really good to now have an understanding of how my applications talk to Cassandra. To know what happens when I connect, prepare a statement, or how my data is encoded on its way to the database. When my CQL collections got big and I got weird results back, I could use my knowledge to quickly debug the issue. I'd like to give you an overview of the new native protocol from the driver perspective, maybe it can help you understand Cassandra the way it has helped me.
A
So
we
do
like
analytics
and
other
kinds
of
things
for
publishers
to
help
them
understand
what
their
revenues
are
and
how
they
work
and
their
audiences
and
that
kind
of
thing.
But
I'm
here
because
I
like
ruby
and
I
like
cassandra,
but
they
haven't
really
been
the
best
of
friends.
Historically,
there's
been
ruby
drivers
and
but
they've
built
we've
been
obviously
been
based
on
the
thrift
apis.
A
A
Putting
a
ruby
shell
on
top
of
them
worked
kind
of,
but
weren't
perfect.
So
when,
in
january,
when
cassandra
1.2
came
out,
it
came
out
with
the
new
binary
protocol
also
called
the
cql
binary
protocol
or
just
the
the
cql
protocol.
It's
got
many
names
also,
obviously
it's
not
the
native
protocol
so
that
that
came
out
in
january
and-
and
I
was
really
excited
because
this
felt
like
sort
of
a
great
time
to
make
that
connection
between
ruby
and
cassandra
work
much
better.
A
The
cql
feels
like
it's
more
language,
agnostic.
It's
not
it's
not
a
java.
It's
not
java
oriented
like
the
old
thrift
apis
and
it
feels
like
this.
Could
this
could
actually
be
something
so
I'd,
never
written
a
database
driver
before
I'm,
not
a
database
driver
expert,
I'm
not
a
database
expert.
Even
I
build
big
data
systems,
processing
data
and
that
kind
of
thing
in
ruby.
A
So
well,
you
know
I'm
not
really
an
expert
at
this,
but
how
hard
can
it
be-
and
I
had
this
this-
I
felt
intimidated
just
by
by
the
thought
of
opening.
You
know
opening
a
socket,
pushing
a
few
bytes
down
it
and
see
what
happened
it
felt
like
well,
you
need
to
know
what
you're
doing
to
to
do
that,
but
I
sat
down
with
the
cql
protocol
specification
and
the
ruby
sockets
api
documentation
and
figured
well.
I
can
probably
do
this,
so
what
I
did
was
sell.
A
A
And
it's
also
going
to
be
a
lot
about
the
the
cql
protocol
and
I'm
going
to
show
you
quite
a
lot
of
hex
dumps
like
this
and
it's
not
to
intimidate
you
I'm
showing
them,
because
I
want
to
to
show
you
how
how
that
you
can
actually
understand
this.
I
was
a
bit
intimidated
by
this
and
I
hope
that
when
you
leave
here
today
that
you
will
feel
like
well,
this
is
no
problem.
A
I
can
actually
look
at
the
the
bits
and
bytes
going
back
and
forth
from
between
my
application
and
cassandra,
and
I
can
see
I
can
look
at
that
and
actually
see
what
it
means.
So
that's
sort
of
my
aim-
and
I
hope
I
hope
I'll
I'll
manage
so,
let's
dive
into
to
these
bytes
the
the
first
few
bits
of
the
of
each
message,
so
the
the
one
on
the
top
was
what
I
sent
to
cassandra
and
the
one
on
the
bottom
is
what
cassandra
sent
me
back.
A
So
those
are
called
frames
or
message
frames.
They
they
have
a
header
and
they
have
a
body
and
the
first
part
of
every
every
request
that
you
send
to
cassandra.
Will
be
four
zeroes
and
that
tells
cassandra
well.
This
is
a
this
is
a
request
and
every
response
you
get
back
from
cassandra
will
start
with
the
one
followed
by
three
zeros,
and
that
tells
you
this
is
a
response
frame.
A
Next,
oh
sorry,
that
so
that's
the
sort
of
expanded
bit
pattern
of
those.
So
next
up
is
the
version
number.
So
the
next
four
bits
are
the
version
number
of
the
protocol.
Now,
there's
lots
of
version
numbers
going
on
here,
because
the
the
protocol
has
a
version.
Number
cql
itself
has
a
version
number
and
then
cassandra
has
a
version
number
and
you
could
have.
A
You
could
can
talk
version
one
of
the
cql
protocol
using
cql
version
3.0.3
with
cassandra
2.0
or
you
can
talk
version
2
of
the
protocol
with
cassandra
2.0
using
cql
3.0.0.
So
this
is
just
the
version
number
of
the
of
the
actual
protocol
specifying.
What
is
how
will
this
message?
How
should
you
interpret
this
message?
It
has
really
nothing
to
do
with
cql
or
with
with
cassandra.
A
So
many
versions
of
cassandra
can
support
many
versions
of
the
goal
and
vice
versa,
but
I'm
I
haven't
had
really
had
the
time
to
implement
version
of
version
two
of
the
protocol,
so
this
is
mostly
going
to
be
about
version.
One
they're,
not
very
very
different.
The
second
byte
of
the
of
of
a
message
is
feel
a
flags
field
that
contains
it
can
contain
a
few
flags
in
this
case
it's
zero,
so
there's
no
flags
set,
but
it
could
be
contain
a
flag
for
compression.
A
Is
this
message
compressed
and
it
can
contain
a
flag
saying,
but
I
want
tracing
on
on
this
this
request.
So
if
you
remember
this
morning,
jonathan
showed
you
tracing
in
in
cassandra
and
that's
the
what
the
driver
does
when,
when
you
request
tracings
basically
just
sets
a
bit
in
in
this
field
in
the
cql
sql
frame.
Header
next
up
is
a
the
stream
id,
so
this
can
be
basically
any
number
between
0
and
128,
and
the
nifty
thing
about
this
field
is
that
cassandra
will
respond
with
in
the
response.
A
It
will
have
the
same
number,
so
you
can
sort
of
send
off
a
request
with
with
that
number
set,
and
you
will
know
that
when
you
get
another
request,
a
response
back,
it
will
have
that
same
number,
so
you
can
sort
of
pair
requests
and
responses
together
then
comes
the
the
op
code
that
tells
cassandra
or
the
client.
What
kind
of
a
message
is
this
really
in
this
case
number
one?
It
means
startup
and
number
two.
A
As
in
the
response,
that's
the
the
ready
message
and
that's
really
important
for
knowing
that
how
to
interpret
the
message
body,
because
there's
no
self
description
in
the
message
bodies.
You
need
to
refer
back
to
the
protocol
specification
to
know
what
the
how
the
body
should
be
interpreted
and
that's
sort
of
where
the
this
field
is
absolutely
crucial
to
figuring
out
the
rest
of
of
the
bytes.
The
header
ends
with
four
bytes:
that's
just
the
size
of
the
body.
So
they
had.
A
There
is
a
fixed
length
of
eight
bytes
that
you
can
implement.
When
you
implement
a
a
driver,
you
can
basically
just
wait
for
eight
bytes,
and
you
know
that
you
will
always
get
a
response,
will
always
be
at
least
eight
bytes
and
requests
will
always
be
at
least
eight
bytes,
and
then
you
can
look
at
the
last
four
bytes
of
that
to
know
how
many
more
bytes
to
expect.
So
in
this
case
the
request
is
22
bytes
long
and
the
response
is
zero
bytes.
A
So
the
ready
message
doesn't
have
a
body,
it's
just
zero,
and
this
is
then,
of
course,
the
the
body
variable
length,
as
I
mentioned,
and
for
startup
messages
the
they
contain.
According
to
the
specification,
a
map
of
options
and
the
map
in
the
cql
protocol
is
defined
as
a
as
two
bytes
that
tell
us
how
many
entries
are
in
this
map
and
in
this
case
the
two
first
two
bytes
are
one.
A
So
we
know
that
there's
one
entry
in
this
map
and
then
we
also
know
from
the
specification
that
the
the
entries
will
be
strings.
So
there
will
be
a
key,
that's
a
string
and
the
value
which
is
a
string
and
strings.
I
think,
in
this
case
short
strings,
to
be
exact,
however,
also
have
a
prefix
of
two
bytes
telling
us
how
long
they
are.
So
in
this
case
we
have
11
bytes,
the
the
key
will
be
11
bytes
long,
these
bytes,
which,
if
we
look
at
them
as
ascii,
our
cql
underscore
version.
A
So
the
option
in
this
case
is
cql
version,
and
the
value
for
this,
for
this
key
is
five
bytes
long.
These
bytes
and
those
just
spell
out
the
version
number
that
we
want.
So
it's
it's
actually
almost
human
possible.
It's
not!
I
mean
it's
not
something
you
just
look
at
it
and
you
say
immediately
see
what
it
is,
but
that's
basically
it
from
the
that's
how
the
protocol
works.
A
So,
if
you
ever
see
this
in
sort
of,
if
you're
sniffing
your
network
traffic
and
see
this,
then
you
know
what
it
means,
but
looking
at
it
in
another
way,
just
look
more
like
a
sequence
diagram.
This
is
basically
what
just
happened.
We
we
sent
cassandra
a
startup
message,
and
then
we
got
a
ready
message
back,
very,
very
simple,
handshake
of
open
a
connection
open
your
socket
push.
A
Some
bites
down
get
a
response
back,
make
sure
that
it's
the
ready
message
and
then
you're
sort
of
good
to
go
and
in
another
way
this
you,
you
push
a
startup
message
with
a
map
of
options
that
you
want
to
specify
for
this
connection,
and
basically
these
options
are
more
or
less.
You
just
specify
the
version
of
cql
you
want
and
optionally.
A
So
that's
pretty
pretty
simple
to
to
do,
but
sometimes
now,
when
there's
a
few
versions
of
cassandra
that
use
the
the
the
native
protocol,
you
might
need
to
know
what
options
are
actually
possible.
Before
you
start
the
connection,
then
you
can
send
an
options.
Message
which
will
be
cassandra
will
respond
to
with
sorry
the
supported
message
and
the
supported
message
is
basically
more
or
less
the
same
thing
as
you
send
in
the
startup
message.
It's
a
map
of
a
map
of
a
multi-map.
A
A
And
then
you
send
your
regular
startup
message
using
one
of
these
options
and
and
you
get
ready
back
unless
you're
using
authentication,
because
then
it
gets
a
bit
more
involved
and
you
might
get
an
authenticate
message
back
from
from
cassandra.
When
you
send
your
startup
message,
and
in
that
case
you
just
formulate
your
credentials,
your
username
and
password
and
send
those
to
cassandra.
A
And
then
you
will
get
a
reddit
message
back
and
you
know
that
the
handshake
is
has
worked
and
you're
sort
of
authenticated
to
use
the
database
in
version
two
of
the
protocol
that
has
changed
slightly.
You
can
there's
a
more
options
to
to
authentication
as
jonathan
mentioned
this
morning,
but
essentially
it
it
works
the
same.
So
when
you've
gone
through
the
this
handshaking,
you
want
to
sort
of
get
down
to
business
and
actually
send
queries,
because
that's
sort
of
that's
what
your
application
is
going
to
do.
A
The
startup
phase
and
the
the
handshaking
is
just
a
very,
very
short
duration.
At
the
start
of
your
application.
Queries
is
sort
of
where
what
it's
doing
the
rest
of
the
time
and
that's
also
just
a
one
message
you
send
to
cassandra
and
you
get
another
a
response
back
and
the
response
in
this
case
is
called
result.
A
Result
is
a
very
probably
the
most
complex
message
type
in
the
whole
protocol.
It's
got
quite
a
few
different
subtypes
and
the
structure
of
it
is
is
pretty
big.
It
kind
of
looks
like
this
more
hex
domes
of
of
cql
frames,
but
as
before,
we
we.
We
know
this
format.
Now
we
we
can
see
that
this
is
the
header
of
the
of
the
request.
A
We
know
that
this
is
a
request,
because
it
starts
with
four
zeroes.
We
know
that
it's
the
version
one
of
the
protocol,
and
then
we
can
see
that
this
is
op
code
number
seven.
So
so,
in
this
case
a
query,
query
message:
so
we
we
know
from
the
specification
what
what
to
expect
from
this,
which
is
a
a
long
string
of
which
is
the
the
cql
that
we
want
to
prepare.
A
The
long
string
in
this
case
just
means
that
it's
a
it's
a
string
prefixed
with
the
the
length
of
the
string.
So
the
as
you
see
the
this
one
is
if
our
counting
correctly
19
characters,
long,
which
is
what's
the
one
three
in
hexadecimal
just
before
the
string
means
and
then
there's
the
consistency
of
the
request.
So
we
we
want
to
specify
which
consistency
that
we
want
for
this
request
this
case,
the
consistency
number
four
which
corresponds
to
quorum.
A
A
And
then
we
get
a
response
back,
which
is
up
code,
eight,
the
the
response
type.
As
I
mentioned,
it's
got
lots
of
subtypes.
This
is
subtype
number
two
and
subtype
number
two
means
we
can
expect
a
metadata
block
and
a
a
rose
block.
So
basically,
this
is
when
you
get
rows
back
from
the
database.
This
is
how
how
it
looks,
but
the
the
metadata
block
at
the
at
the
top
is
really
important,
because
that's
how
cassandra
tells
your
driver
or
your
client,
what
kind
of
data
it
will
expect
what
data
types
are.
A
These
are
the
columns
that
are
returned.
In
this
case.
We
have
three
columns
that
are
going
to
be
returned,
as
we
can
see
in
the
in
the
highlighted
number
field
there
and
the
the
fields
are
username
email
and
password,
and
their
data
types
come
just
after
the
after
the
names,
it's
just
a
two
bytes
of
of
types,
the
od
type.
A
That's
what
is
it
14
and
hex?
That
means
a
string
or
the
the
text
type
in
cql,
so
this
actually
corresponds
directly
to
to
text
or
varchar
in
in
in
the
cql
language
and
the
the
last
field.
There
has
the
the
type
three,
which
is
a
blob
type,
so
that
corresponds
directly
to
to
blob
in.
When
you
create
your
table,
you
specify
blob
and
that's
basically,
what
gets
sent
to.
Oh,
that
has
cassandra,
interprets
that
then
we
get
the
rose
back.
A
In
this
case,
we
can
see
that
there's
two
rows
the
and
we
can
see
the
the
the
contents
of
of
those
rows
here.
There's
actually
the
blob
the
blob
field
here
is,
I
didn't
convert
that
to
ascii,
because
that
would
probably
be
just
be
unreadable,
but
that's
sort
of
how
it's
just
it's
just
there
as
bites
blobs
are
are
bites.
A
A
That
can
that
that
means
that
can
be
quite
a
few
different
kinds
of
area.
It
can
be
any
anything
from
the
cql
that
you
sent
is
bad
to.
I
couldn't
reach
the
desired
consistency
or
there's
been
some
other
kind
of
error
in
inside
of
cassandra.
That
cassandra
is
telling
you.
I
can't
fulfill
this
query.
You'll
have
to
try
again
or
fix
your
fix
your
query,
and
this
this
could
be
returned
when
you
authenticate
this
can
be
returned.
When
you
send
a
query,
it
can
be
returned
basically
anytime.
A
One
of
the
best
features
in
in
my
mind
about
the
cql
protocol
is
the
prepared
statements.
I
I
like
prepared
statements,
even
in
in
relational
databases,
they're
a
great
feature,
most
they're
them
a
great
feature
mostly
to
avoid
like
sql
injection
attacks
and
that
kind
of
thing
in
cql.
I'm
not
really
even
sure.
If
that's
it's
possible
to
do
a
cql
in
injection
attack.
A
I
don't
think
it's
legal
to
send
more
than
one
cql
statement
in
in
in
a
message,
but
they're
great
for
performance
and
and
they're
just
great
for
just
sending
just
sending
the
bytes
the
data
over
and
not
sending
sending
all
the
sql.
You
don't
have
to
smash
strings
together
to
to
create
a
query
just
just
pass
on
the
data
and
let
the
driver
encode
the
data
in
as
the
appropriate
data
types
and
prepared
statements
sort
of
they
work
basically
like
queries.
But
you
need
to
to
do
an
extra
round
trip.
A
You
need
to
prepare
the
statement
the
first
time
you
want
to
use
it
and
when
you
do
that,
you
get
a
result
frame
back.
The
result
frame
in
this
case
is
another
another
subtype
of
the
the
result.
So
this
is
basically
the
same
frame
type
as
when
you
do
a
query,
but
it's
got
completely
mostly
completely
different
content,
but
once
you've
done
that
you
can,
you
can
send
an
execute
message
and
just
specify
the
id
that
you
got
back
from
got
back
in
your
result
frame
and
the
the
parameters
to
to
the
statement.
A
Instead.
So,
let's
look
at
how
that
looks
as
a
hex
dump.
We
sorry
we
can
see
that
the
this
is
op
code
number
nine.
So
we
have
to
refer
back
to
the
sql
protocol
specification
to
see.
What's
the
structure
of
message
number
nine,
it's
a
long
string,
which
is
the
cql
statement
that
you
want
to
prepare
and
that's
it.
And
then
you
get
back
up
code
number,
eight
that
we
saw
before
the
result
frame,
but
in
this
case
this
subtype
4.
A
A
So
this
is
this
is
the
id
of
the
statement
that
when
you
want
to
execute
this
again,
you
need
to
send
this
this
id
to
cassandra
and
then
he
knows
sort
of
which
cql
that
the
secure
prepared
statement
that
you
meant,
but
the
rest
of
the
frame
is
actually
the
same
as
in
the
the
previous
result
frame
that
we
looked
at.
It's
a
metadata
block
about
about
this
query.
A
So
when
you
prepare
a
statement,
you're,
basically
sending
it
now
an
opaque
string
to
the
cassandra,
you
have
no
idea
what
what
the
structure
of
that
of
that
string
is.
So
you
don't
know
which
fields
are
are
bound
in
the
statement.
You
need
to
get
that
data
back.
So
cassandra
tells
you
that
in
this
case
it's
the
username
field
that
was
prepared-
and
we
know,
then
that
this
is
sort
of
what
to
expect.
A
So
some
of
you
might
have
already
have
figured
this
out,
but
I
mentioned
this
stream
id
field
in
the
header
and
that
you
could
set
that
to
any
number
between
0
and
128,
and
what
this
means
is
that
you
can
have
128
requests
active
at
any
one
time
over
a
single
connection.
So
if
you
want
to
do
multiple
connections
in
parallel,
you
don't
need
to
open
a
new
connection
for
every
for
every
request
that
you
want
to
do
or
for
every
request
in
parallel
that
you
want
to
do.
A
You
can
actually
have
128
going
on
at
the
same
time
per
connection.
So
if
you
have
a
cluster
of
four
nodes,
for
example-
and
you
and
you
have
a
connection
open
to
each
of
them,
you
can
have
500
over
500
connections
going
on
at
the
same
time.
So
then
the
amount
of
parallelism
just
from
from
those
four
connections
is
quite
big,
and
this
plays
really
well
with
with
like
asynchronous
io
and
that
kind
of
thing
so
having
you
can
have
a
single.
A
A
I
guess
that
just
for
low
numbers,
then
there
wouldn't
probably
not
be
that
much
difference
depending
on
I
mean
it
depends
on
what
how
you
implement
the
I
o,
obviously,
but
with
with
something
like
nettie
or
just
doing
asynchronous
scio,
having
an
I
o
reactor.
Doing
all
that.
I
guess
that
for
low
numbers
that
wouldn't
really
matter
for
larger
numbers,
maybe
it
could
be
the
overhead
of
just
having
to
switch
between
that
many
connections
might
be.
But
on
the
other
hand
I
mean
servers
like
nginx
and
another
high
performance.
A
No,
I
and
I
can't
really
couldn't
really
answer
the
the
question
truthfully
because
I
I
haven't.
I
haven't
implemented
a
for
example,
a
driver
for
the
old
thrift
api,
where
that
would
sort
of
been
the
what
you
would
have
to
do,
but
either
way
it's
a
very
convenient
way
of
just
not
having
to
having
to
keep
many
connections
open.
So
it's
just
a
a
nice
feature
of
the
protocol
and
and
sort
of
it's
one
byte
of
extra
data.
A
I
I
I
in
if
I'd
designed
it,
I
would
have
probably
put
two
bytes
for
that,
so
I
can
have
a
little
bit
bigger
windows
than
than
128
just
because
it's
so
convenient
to
just
have
a
single
connection,
but
the
so
getting
back
to
this.
So
that's
one
byte
of
stream
ids!
You
can
use
all
of
the
positive
numbers.
The
negative
numbers
are
reserved
and
there's
one
special.
A
The
only
negative
numbers
actually
used
is
the
is
-1
which
is
event
server-side
events,
so
you
can
register
to
receive
events
from
cassandra
when
things
like
schema
changes
happen
or
when
the
topology
changes.
So
you
can
actually
register
register
to
to
get
notified.
When
there's
a
new
node
joining
the
cluster,
for
example,
or
if
a
node
that
has
failed,
come
comes
back
again,
so
that's
a
very
convenient
way
of
of
sort
of
discovering
when
you
need
to
when
you
should
connect
to
new
nodes
and
that
kind
of
thing.
A
So
enough
about
the
the
protocol
itself,
I
would
say
that
implementing
a
cql
driver,
the
hard
part,
is
not
actually
implementing
the
protocol.
The
protocol
specifications
are
really
well
written
and
quite
easy
to
to
get
your
head
around.
You
can
start
playing
quite
just
open
a
socket,
as
I
said,
and
try
out
a
few
things
in
your
in
your
terminal.
You
can
even
do
it
with
with
netcat,
for
example,
if
you're,
if
you're,
really
brave
so
the
the
hard
part
is,
is
more
in
the
other.
A
A
A
I
just
want
to
to
release
version
one
and
have
a
driver
that
implements
the
full,
the
full
protocol
so
that
I
can
start
using
it
and
testing
it
out,
and
I
thought
that
other
things
would
sort
of
be
more
important,
so
version
one
of
of
my
ruby
driver.
Actually,
it
would
crash
when,
when
cassandra,
if
a
cassandra
node
went
down,
it
would
say
well
the
connection
closed.
A
So
I'm
going
to
close
the
all
the
connections
and
just
stop,
and
I
mean
why
do
that
if
you
that
just
makes
using
cassandra
into
a
pretty
much
worse
experience
of
just
a
worse
experience
than
than
it
could
be,
and
to
do
you,
you
need
to
to
think
like
cassandra?
Does
you
need
to
think
about
network
petitions
notes
going
down
that
kind
of
thing
so
in
in
the
latest
version
of
my
driver?
A
There's
lots
and
lots
and
lots
of
code
and
probably
the
probably
the
hardest
code
I've
ever
written
that
deals
with
this
trying,
keeping
track
of
which
connections
have
failed.
What
what
other
nodes
are
in
the
cluster
when
new
nodes
join
the
cluster
when
you,
when
nodes
come
back
again
from
after
having
having
restarted,
for
example,
keeping
track
of
to
connect
to
those
again
to
make
sure
that
I
fail
the
right
the
right
requests
when
the
connection
fails.
A
So
if
the
request
has
already
been
sent,
then
obviously
that
request
needs
to
be
failed
to
the
so
that
the
client
can
be
can
be
told
to
maybe
try
the
request
again
or
maybe
not
try
it
by
the
way,
don't
implement
a
thing
like
automatic
retries
of
requests.
That
can
be
very
convenient
in
some
cases,
but
they
can
be
disastrous
in
other
cases.
A
So,
for
example,
if
your
client
is
doing
increment
operations
and
one
of
these
increment
operations
fails,
you
don't
want
to
to
automatically
retry
that
because
the
even
though
the
request,
the
connection
for
example,
might
might
go
down.
But
just
after
you
send
the
request,
but
the
request
can
still
have
ended
up
in
cassandra,
so
the
increment
increment
had
already
happened.
You
don't
want
to
retry
that,
but
many
operations
are
item
potent
it's
fine
to
retry
those,
but
that's
not
something
that
a
driver
can
know.
A
So
you
need
to
defer
that
to
sort
of
the
client
code
to
to
decide
when,
when
a
retry
is
it's
actually
something
you
can
do
or
not
another
really
difficult
part
of
writing
a
driver
is
that
you
get
really
weird
bug
reports
that
are
hard
to
to
reproduce.
So
the
the
what
looks
like
a
white
line.
There
is
actually
a
discussion
around
the
github
issue.
A
It
went
on
for
quite
a
while
and
it
centered
around
a
guy
who
had
this
weird
thing
going
on
when
he
used
and
don't
don't
do
this
at
home,
but
he
used
maps
for
wide
rows.
So
he
had
basically
had
a
single
column
in
his
in
his
table,
which
was
a
map,
and
then
he
just
appended
to
that
map
and
sort
of
used
tried
to
use
that
as
white
rose,
and
then
he
started
getting
sort
of
random.
A
So
obviously
it
must
be
my
driver,
that's
doing
something
wrong
and
I
dug
through
my
code
and
couldn't
find
anything
and
eventually
I
went
back
to
to
the
specification
and
then
it
dawned
on
me.
Well,
a
map
in
cql
is
actually
in
the
protocol,
not
in
cassandra,
not
in
the
storage
model,
but
in
in
the
protocol.
That's
actually
prefixed
with
two
bytes
telling
us
how
how
many
entries
will
be
in
the
map.
So
when
you
build
a
map
over
time,
that's
more
than
65
536
entries.
A
First
of
them
you
will
see
the
first
ones,
because
you
know
the
counter
has
just
overflowed
and
analysis
that
well
there's
around
400
and
what
is
it
445
or
some
entries
instead
of
70
000
and-
and
this
is
just
this
is-
is
this
a
bug
in
the
protocol?
Is
this
a
bugging
cassandra?
Is
this
the
way
it's
supposed
to
work?
Why
isn't
the
is
the
size
of
of
maps
a
full
int
when
many
other
fields
are
actually
prefixed
have
size
prefixes?
That
is
a
full
hint.
A
I
don't
know,
but
the
these
kinds
of
weird
things
crop
up
all
of
the
time
and-
and
you
sort
of
you
never
know
when
it's
most
of
the
time-
it's
your
fault
because
you're,
the
you're,
the
the
driver
implementer.
So
what
what
I
mean?
It's
more
likely
that
your
code
is
wrong
than
cassandra
is
wrong,
but
some
some
of
the
times.
This
is
really
weird
ed
cases
in
cassandra
that
you
weren't
really
prepared.
A
For
the
final
thing
I
want
to
mention
just
one:
the
difficulty
of
writing
a
driver
is
just
performance.
Performance
is
always
really
really
important,
and
I
guess
a
few
of
you
think
now
well
he's
written
he's.
Writing
a
driver
in
ruby.
How
much
performance
can
he
expect,
but
that's
true
and
we'll
never
be
as
performant
as
a
java
driver,
for
example,
but
we
have
production
systems
running
tens
of
thousands
of
operations
per
second
in
ruby.
A
So
I
think
that
the
performance
is
pretty
good,
but
it
wasn't
from
the
beginning
and
there's
not
much
in
ruby
that
helps
you
building
a
high
performance
database
driver.
You
basically
have
to
implement
things
like
bite
buffers
and,
and
that
kind
of
thing
yourself
and
what
you
really
need
to
avoid
in
a
language
like
ruby,
you
need
to
avoid
using
strings
or
arrays
as
buffers,
because
it's
like
cassandra's
wide
rows,
if
just
append
to
the
end
and
delete
from
the
the
beginning.
A
A
So
I
hope
I
haven't
really
scared
you
away
from
writing
a
cassandra
driver
by
showing
you
these
hex
dumps.
I
really
wanted
just
to
to
show
you
that
you
can
actually
decrypt
those
I
now
from
having
worked
with
a
worked
on
writing
a
driver
for
the
better
part
of
10
months.
I
can
sort
of
decrypt
those
in
in
my
head,
not
all
of
them.
I
don't.
I
can't.
A
I
can't
say
that
I
have
the
full
specification
in
my
head,
but
I
sort
of
recognize
the
patterns
and
recognize
a
few
of
them,
and
sometimes
that's
a
great
debugging
feature.
You
can
actually
just
see
what's
going
on
and
sort
of
debug
what's
what's
happening
by
by
looking
at
the
the
the
raw
bytes
going
back
and
forth
and
sort
of
take
the
leap
that
I
did
in
january
and
and
try
writing
your
own
driver
it.
A
You
don't
have
to
aim
for
for
doing
sort
of
writing
a
a
driver
that
will
be
used
by
by
people.
You
can
actually
just
do
it
for
for
cure.
Out
of
curiosity
or
just
for
learning
how
cassandra
works
underneath
or
how
a
protocol
works
underneath.
I
think
it
would
have
been
for
me
a
really
great
experience,
and
this
is
then
just
the
link
to
the
actual
driver
that
I've
been
been
working
on.
It's
just
called
cqlrb
and
version
1.1
release
candidate
0
was
released
just
37
minutes
ago.
A
A
B
A
A
For
the
driver,
it
doesn't
really.
I
doesn't
really
matter,
but
it's
more,
it's
a
hint
to
cassandra
what
to
to
expect.
So
you
you're
telling
cassandra.
I
will
use
this
this
version
of
of
the
sequel,
but
no
the
driver
will
never
never
sort
of
inspect
the
the
cql
strings
and
it
will
it
shouldn't
really
try
to
do
that.
A
So
I
guess
it
will
become
more
important
when,
if
there's
a
ever
a
cql
version
4,
then
it
might
get
more
important,
even
possibly
even
for
the
for
the
driver
to
to
sort
of
know
what
to
expect
back
from
cassandra.
I
guess
not
so
much
what
the
client
will
send,
but
what
to
expect
back
from
cassandra.
It
could
be
something
along
those
lines
in
the
future,
but
right
now
it's
basically
just
a
hint
I'd
say.
A
Yeah,
so
so
in
the
beginning
it
was.
I
was
basically
developing
straight
against
cassandra
just
to
get
sort
of
valid
responses
back
to
be
able
to
to
know
what
sort
of,
because
I
started
off
trying
to
just
from
the
specification
write
a
valid
response
and
then
sort
of
assert
that
I
passed
that
correctly,
but
that
that
wasn't
I
mean
I
made
too
many
mistakes
that
way,
so
I
basically
set
up
integration
tests
that
talked
to
cassandra
sent.
A
What
I
knew
was
a
valid
request,
and
then
I
made
sure
that
the
it
that
the
responses
were
passed
correctly
and
I
can
sort
of
assert
on
that,
so
the
the
hardest
part
of
testing
a
driver
in
in
my
experience
is
at
least
this
testing
the
I
o
stuff.
You
can
test
those
at
the
high
level
and
just
throw
requests
at
cassandra
and
see
what
comes
back.
A
But
if
you
want
to
throw
in
like
what
happens
say
in
error
scenarios,
then
that's
much
much
harder,
and
just
I
mean
in
in
this
case
the
the
it's
got
an
I
o
reactor.
That's
doing
all
of
the
I
o
and
keeping
track
of
all
the
connections
to
all
different
nodes,
just
testing
that
was
just
very
very
hard.
I
haven't
got
any
good
good
tools
to
do
that.
C
A
So
sorry,
you
sound
norwegian.
Do
you
work
for
finn?
Oh
very
good,
we'll
we
should
talk
later
you're,
actually
one
of
our
biggest
customers?
Oh,
you
know
your
your.
Your
colleagues
didn't
know
yesterday
great,
so
I
guess
you
could
use
the
the
java
driver.
That
would
probably
be
more
performant.
You
shouldn't
use
my
ruby
driver
for
that
kind
of
thing.
A
I
think,
because
you're,
not
a
ruby
shop
and
and
the
java
driver
will
always
be
more
performant,
but
I
know
that
netflix
they
use
in
some
of
their
analytical
jobs
in
that
they're
running
in
hadoop.
What
they're
doing
is
actually
they
take
the
raw
ss
tables
load,
those
into
hadoop
and
do
queer
or
sort
of
do
the
extraction
directly
from
the
ss
tables.
So
if
you
want
to
do
massive
etl,
I
haven't
tried
this,
but
I've
heard
that
this.
A
B
A
Yeah,
so
the
the
driver
is
roughly
3000
lines
of
ruby.
The
tests
are
closer
to
7000
lines.
A
lot
of
that
I
mean
a
lot
of
that
will
would
also
be
sort
of
set
up
for
for
the
tests
and
and
just
hex
bite
dumps.
Of
sort
of
this
is
a
frame
let's
pass
it,
but
the
I've
tried
to
do
so.
I've
tried
to
write
some.
So
do
you
know
about
ccm
the
cassandra
cluster
manager
tool?
A
A
I
wouldn't
say
that
that's
a
very
reliable
way
of
testing
and
and
sort
of
that
says
more
about
my
lack
of
experience,
doing
it
than
done
than
it
being
a
valid
value
testing
strategy,
but
I
would
love
to
do
more
around
that
trying
to
do
like
automate
that
kind
of
thing
automate
the
testing
of
failures,
so
that
would
be
if
anyone
has
experience
doing
that.
I'd
love
to
talk
to
you,
so
there
was
one
more
question.
I
think
yeah.
A
Very
good
question:
this
was
one
of
the
the
things
that
I
had
no
idea
before.
I
wrote
this
driver
and
I
was
really
really
happy
to
to
find
there
is.
You
know
there
is
a
system
key
space
it
contains.
For
example,
it
contains
the
schema
information
about
which
key
spaces
and
which
tables
that
exist,
but
it
also
connects
contains
a
table
called
pierce
that
has
basically
it
is
a
a
short
version
of
the
gossip
information
about
the
cluster.
A
B
B
D
A
I'm
actually
using
both
of
them,
but
the
the
notifications
are
good
for
knowing
when
things
change,
but
you
need
the
the
system
key
space
to
know
how
it
is
when
things
are
stable.
So
the
first
thing
you
do
is
just
check
what
how
what's
the
cluster
of
topology
now
and
then
you,
then
you
you
listen
for
the
events
to
know
when
things
change
and
actually
I
don't
use
the
I
I
use
the
event
more
as
a
as
a
notification
that
something
has
changed.