►
Description
Speaker: Matthias Broecheler, CTO at Aurelius
Slides: http://www.slideshare.net/planetcassandra/distributed-graph-computing-with-titan-and-faunus
This presentation introduces Titan, Faunus, and scalable graph computing in general. We present a case study of how Pearson builds an education social network on top of Titan, Faunus, and Cassandra to support learning in the 21st century. Titan is an open source distributed graph database build on top of Cassandra that can power real-time applications with thousands of concurrent users over graphs with billions of edges. Faunus is an open source global graph processing engine build on top of Hadoop and compatible with Cassandra that can analyze graphs, compute graph statistics, and execute global traversals. Titan and Faunus are components of the Aurelius Graph Cluster which enables scalable graph computation and powers applications in social networking, recommendation engines, advertisement optimization, knowledge representation, health care, education, and security.
A
Been
talking
about
our
good
friends,
Oracle
going
come
forth
out
here,
they're
still
going,
but
they
brought
in
some
support
boats,
so
we're
gonna
have
to
we're.
Gonna
have
to
up
our
game
a
little
bit
soda
break,
don't
eat
your
cookie,
we're
gonna
get
out
there
or
getting
get
him.
Somebody
noticed
on
Twitter
right
that
they
brought
a
single
point
of
failure,
so
it's
their
big
sale,
one
sale,
so
we're
gonna
go
out
there
going
to
take
the
sale.
This
is
gonna,
get
the
news.
Isn't
it
a
bunch
of
open-source
hippies
take
over
Oracle
okay?
A
So
our
next
talk
is.
This
is
one
that
I've
been
looking
forward
to
last
year
at
our
summit
we
had
a
talk
by
Matthias
and
it
was
about
Titan
and
it
was
graph
database.
What
are
we
talking
about
a
graph
database
for?
Well,
it's
really
interesting
how
well
these
two
technologies
work
together
and
how
tight
and
uses
Cassandra,
but
that
was
one
of
the
most
viewed
videos
from
that
conference
and
it
really
hit
a
lot
of
people.
I
think
in
their
their
use
case
where
they
wanted
it.
A
C
B
B
Think
that's
a
good
point
to
say
thank
you
to
the
community
that
has
built
around
Titan
without
that
community
wouldn't
have
been
possible
to
accomplish
what
we
have
in
pretty
much
exactly
12
months.
From
the
day
after
tomorrow,
jun
14th,
we
released
Titan
June
fourteenth
of
2012.
That
was
the
alpha
release.
That
was
the
first
prototype
of
Titan.
That
way
that
we
put
out
there,
we
put
it
up
on
hack
and
use
in
a
somehow
bubbled
up
to
the
top,
and
we
were
like
wow.
B
That
seems
to
be
people
actually
interested
in
this,
and
so
we
went
to
work
on
the
0.1
release
that
came
out
a
good
three
months
later
and
ever
since
then
have
been
cranking
on
performance,
trying
to
make
tight
and
better
I'm
getting
more
features
and
features
that
were
requested
with
the
0.2
release.
We
did
a
big
push
on
performance
and
then
just
about
2-3
months
now
we
put
out
the
0.3
leaves,
which
was
a
major
rewrite
of
titan,
and
we
basically
refactor
the
entire
entire
internal
core
of
Titan
and
made
it
a
lot
better.
B
A
lot
stronger.
We
put
an
indexing.
So
now
you
can
hook
up
tighten
with
elastic
search.
You
get
full
text
search,
you
get
numeric
search,
you
get
gon
next
thing,
and
so
on
and
so
forth.
So
with
that
yeah
a
lot
of
new
stuff
came,
came
to
life
and
tighten
also
got
a
brother
with
faunus,
which
is
the
analytic
side
of
Titan.
B
So
fanous
is
a
Hadoop
based
batch
analytics
graph
framework
that
works
nicely
with
Titan
and
I'll
get
back
to
Faunus
at
the
end
of
the
presentation,
but
before
we
we
dive
into
all
of
that,
Titan
is
a
graph
database.
How
many
people
know
what
a
graph
database
is
just
a
show
of
hands,
all
right,
good
I
think
that's
that's.
B
This
time
and
just
say
this
graph
databases
are
vertices
connected
by
edges.
I
just
have
labels
and
both
of
those
have
properties
that
describe
them
further.
As
an
in
this
example,
we
have
people
here,
Hercules
and
a
monster
Cerberus
represented
as
vertices.
We
have
one
edge
between
them,
a
battled
edge
that
has
a
time
point
of
12
and
that's
pretty
much
all
you
need
to
know
in
terms
of
terminology
and
how
you
would
go
about
dealing
with
a
graph
database,
so
that
boils
it
down
to
exactly
30
seconds.
B
One
question,
however,
that
we
haven't
been
able
to
give
a
good
answer
to
is
when,
should
you
use
a
graph
database?
So
when
should
you
consider
using
a
graph
database
and
typically
they're,
you
know
a
bunch
of
use
cases?
I
can
rattle
off
and
a
bunch
of
reasons
for
why
you
may
or
may
not
want
to
consider
graph
database,
but
I
think
we
can
concisely
boil
it
down
to
this.
B
If
you
look
at
your
application,
ask
yourself
the
question
of
how
much
value
is
in
the
relationships
in
your
domain,
or
rather
how
much
value
do
you
want
to
get
out
of
the
relationships
in
your
domain?
If
that
value
is
very
low
or
none,
you
probably
should
be
using
a
key
value
store
right.
That's
the
simple!
As
data
model,
there
is
just
a
key
maps
onto
value
super
simple
to
use
they're
slightly
more.
B
You
get
into
the
big
table
the
document
data
model,
where
you
can
have
key
internal
structure,
so
either
document
structure,
or
you
have
the
wide
rows
that
you
have
in
Cassandra
or
are
the
big
table
type
implementations?
If
you
need
more
structure
more
relationships
than
that,
you
can
go
with
a
relational
data
model.
B
Where
you
know,
relationships
were
sort
of
bolted
on
with
the
concept
of
foreign
keys,
but
if
you
want
to
go
to
fall
in
on
relationships
and
get
every
bit
of
value
out
of
those,
you
probably
best
suit
it
with
a
graph
database,
because
graph
databases
are
built
for
relationship
analysis,
meaning
their
internals
are
highly
optimized
for
graph
for
graph
traversals
relationship
traversal,
but
also
the
interface
is
the
query
language.
Everything
around
graph.
Databases
is
optimized
for
analyzing
relationships.
B
B
This
is
just
to
give
you
an
idea
for
when,
should
you
bring
graph
databases
into
the
conversation,
so
graph
databases
are
forgetting
most
out
of
your
relationships,
and
that
is
what
motivated
Pearson
Education
to
look
into
graph
databases
for
a
new
online
learning
platform
that
they're
developing
a
lot
of
people
probably
know
pearson
education
from
textbooks.
They
are
a
big
publisher
of
textbooks
that
are
used
in
schools
and
universities
all
over
the
world.
Some
of
you
may
know
that
they
also
do
a
lot
of
work
in
virtual
learning
platforms.
B
Where
you
have
one
teacher
and
thousands,
if
not
hundreds
of
thousands
of
students
in
those
classes,
one
of
the
popular
ones
was
peter
norvig
and
sebastian
tune
at
stanford
gave
a
talk
on
gave
a
class
on
artificial
intelligence,
and
I
think
they
had
well
over
a
hundred
thousand
students
in
that
class.
So
now,
with
that
concept
is
possible
to
develop
to
to
bring
world-class
education
to
rule
cities
all
over
the
world.
But
one
interesting
challenge
was
that
is
how
do
you
get
the
social
component
into
learning?
Right,
one
part
is
delivering
the
content.
B
Another
part
is
to
get
the
social
back
into
it.
So
pearson
education
is
looking
at
a
way
to
bring
the
content
delivery
together
with
the
learning
experience
that
is
inherently
social
into
one
learning
platform,
a
virtual
learning
platform
that
marries
concepts
both
from
massive
online
learning
classes,
with
social
networks
and,
quite
naturally,
they
were
thinking
of
about
graph
databases
to
drive
that
kind
of
development
effort,
because
they're
thinking.
B
Well,
if
we
want
to
make
that
a
success,
we
need
to
figure
out
how
we
can
deliver
data-driven
services
to
make
such
learning
environments
effective
for
the
student
and
also
manageable,
for
the
teacher,
because
if
you
meant,
if
you
imagine
your
teacher
with
a
hundred
thousand
students,
that's
a
hell
of
a
lot
of
work
for
you
to
do
so.
You
need
to
have
some
sort
of
support
built
into
the
platform.
B
Let's
look
at
what
their
platform
looks
like
to
see
what
motivated
them
to
look
into
graph
databases
and
tighten
in
particular.
So
this
is
a
very
simplified
version
of
the
schema
that
they're
looking
at
you
will
see
familiar
concepts
like
people,
institutions,
courses
that
people
are
enrolled
in
that
teachers
teach
you'll,
see
discussions
which
are
you
know,
the
discussion
forums.
Discussions
have
can
have
comments
on
them,
and
comments
have
comes
on
comments,
there's
the
concept
of
a
share,
which
is
something
that
is
bored
from
the
bar
from
the
social
network
domain.
B
Similarly
to
how
a
Twitter
work,
so
student
can
share
any
piece
of
content
or
anything
that
would
like
to
share
with
other
students,
and
then
those
students
that
follow
them
will
see
that
piece
of
content
as
well
right.
So
we
have
both
elements
from
online
education
and
social
networking
mirror
together
in
this
simplified
schema
here
when
they
started
off
developing
this
platform,
they
had
a
bunch
of
different
teams
developing
these
components
and
with
that
they
had
a
bunch
of
different
storage
solutions
that
entered
the
picture.
B
So
now
all
of
that
can
be
stored
in
Titan.
Titan
has
a
very
integrative
data
model
as
a
graph
database.
It
has
a
flexible
schema,
which
means
it
does
have
a
schema.
So
Titan
has
a
schema.
It
will
enforce
a
schema,
but
the
schema
can't
be
defined
at
runtime,
so
you
can
tell
Titan
at
runtime
how
to
extend
the
schema
and
therefore
be
able
to
accommodate
additional
services
or
additional
additional
concepts
in
your
domain.
B
But
what
makes
graph
databases
really
interesting?
Is
the
kind
of
services
data-driven
services
that
you
can
now
develop,
one
of
which
I'll
explain
here
it's
called
discussion
rank
discussion
rank
is
something
that
came
out
of
a
sort
of
a
lab
environment
inside
Pearson,
a
collaborative
collaborative
adventure
where
we
try
to
figure
out.
How
can
we
support
teachers
in
driving
discussions,
for
course,
so
it's
well
known
that
one
of
the
best
ways
to
make
students
understand
new
concept
is
to
have
those
students
engage
with
one
another
and
explain
the
concepts
to
each
other.
B
If
you
have
to
explain
something,
you
learn
it
much
better,
so
that's
well
known.
But
how
do
you
do
that
in
a
virtual
environment?
So
one
of
the
ways
you
can
do
that
is,
you
can
engage
students
in
a
discussion,
so
a
teacher
can
kick
up
a
discussion
and
then
try
to
motivate
those
students
to
engage
in
the
discussion
and
interact
with
one
another.
B
Now
that
works
very
well
at
a
small
scale,
but
when
you
have
a
class
by
the
hundred
thousand
students
again,
that
becomes
very,
very
tedious
because
you
might
have
hundreds,
if
not
thousands,
of
discussions
happening.
At
the
same
time.
Now
a
teacher
cannot
possibly
look
at
all
of
those
discussions
at
the
same
time.
So
what
do
you
do?
B
The
idea
behind
discussion
rank
is
to
rank
discussions
based
on
their
fruitfulness
or
how
likely
it
is
that
those
discussions
are
on
track
and
therefore
don't
require
any
further
teacher
intervention,
whereas
other
discussions
might
be
derailed
and
therefore
the
teacher
should
step
in
look
at
those
discussions
and
probably
get
it
back
on
target.
What
we
did
to
accomplish.
That
is.
We
looked
at
multiple
factors,
and
this
is
where
graph
databases
are
really
great
to
inter
integrate
those
all
into
one
picture.
B
If
you
look
at
the
discussions
and
if
you
look
at
the
discussion
tree
that
is
being
spawned
from
the
from
a
starting
discussion
topic,
you
can
see
that
you
get
a
tree
structure
for
each
individual
discussion
right,
there's
a
starting
topic.
Let's
say
you
know,
was
what
Shakespeare
one
of
the
greatest
poets
in
England,
let's
say
as
an
individual
starting
point,
and
then
you
have
in
students
argue
pro
and
con
on
that
particular
question.
So
you
get
a
tree
structure
out
of
that
and
you
can
look
at
the
structure
of
that
tree.
B
So
is
it
a
very
bushy
tree
so
a
tree
that
has
a
high
branching
factor?
Is
it
a
very
thin
narrow
tree,
so
I
send
essentially
just
one
individual
branch,
and
that
structure
of
the
tree
gives
you
some
indication
of
how
wealthy
or
how
healthy?
That
discussion
is.
Another
thing
you
can
look
at
is
who
is
engaging
in
the
discussion.
Are
those
just
students
who
know
each
other
ready
so
students
who
follow
each
other
or
are
the
interactions
between
students
who
have
not
previously
interacted
so
we
can
pull
in
the
Fellowship's
relationships?
B
Another
thing
we
can
do
is
we
can
try
to
understand.
Is
the
discussion
on
topic.
We
can
look
at
the
concepts
that
are
being
mentioned
in
the
individual
discussion
posts
and
relate
those
to
a
concept
map
that
we
have
extracted
from
Wikipedia.
There's
a
project
called
dbpedia
which
tries
to
codify
the
human
knowledge
captured
in
Wikipedia
as
a
graph.
So
we
can
look
at
the
concepts
and
how
they're
related
to
another
friend,
since
the
fact
that
page
rank
is
related
to
eigenvector
centrality,
which
in
turn
is
related
to
linear
algebra
right.
B
So
we
have
those
relationships
captured
and
then
we
can
look
at
discussion
topics
and
see
whether
they
are
on
the
topic
of
the
course
which
we
have
also
captured
in
this
graph.
So
you
can
see
we're
traversing
various
relationships
from
concepts
and
their
relatedness
to
discussions
their
comments,
people
following
each
other
and
people
authoring
the
discussions
to
understand
whether
or
not
a
discussion
is
healthy
or
whether
discussion
is
possible
intervention
by
teacher
bottom
line
being
that
Titan
allows
you
or
gives
you
the
capability
to
do
that
kind
of
relationship,
analysis
very
effectively.
B
It's
a
number
of
transactions
and
how
many
people
can
currently
interact
with
the
graph
and
the
size
of
the
graph
is
just
the
sheer
amount
of
data
can
hold
to
answer
this
question.
We
started
a
benchmark,
and
basically
we
said
okay,
if
we
want
to
add
it
to
educate
the
planet,
let's
do
a
benchmark
where
we
simulate
3.5
billion
students.
B
So
that's
about
half
of
the
world's
population
being
on
our
learning
platform,
so
hopefully
we'll
get
to
that
point,
but
for
now
well
we're
just
simulating
it
and
we
ended
up
with
a
graph
with
6.2
billion
vertices
and
121
building
edges
connecting
those
vertices.
So
this
is
a
massive
graph
and
in
fact
we
don't.
We
don't
know
if
any
graph
data,
any
social
network
graph
data
of
that
size-
that's
really
available.
So
we
had
to
generate
it.
We
used
census
data
to
generate
names.
B
We
used
to
scale
free
data
generators
that
we
implemented
and
subsequently
open
source
to
generate
the
network
structure
and
make
it
as
realistic
as
possible.
We
used
MIT
OpenCourseWare
to
generate
course,
titles
and
the
concept
in
those
courses
and
so
on
and
so
forth
to
end
up
with
3.5
billion
students
about
a
hundred
and
780
million
courses
about
180
million
teachers
at
1
million
universities
that
we
generated
first
we
had
to
so
this
is.
B
Wrong
button,
we
had
to
load
it
into
a
Titan
Cassandra
cluster.
What
we
did
is
we
used
16
heiio
instances,
so
those
are
the
new
Amazon
ec2
instances
that
have
the
ssds
in
them
the
two
terabyte
SSDs,
and
we
put
them
all
in
one
placement
group,
one
of
the
things
that
Titan
allows
you
to
do
it.
It
allows
you
to
run
cassandra
embedded,
so
you
can
start
a
titan
cluster.
That
internally
will
start
cassandra
and
then
the
interaction
between
Titan
and
Cassandra
are
not
over.
B
We
also
did
a
bunch
of
optimizations
to
Cassandra
itself,
like
increasing
the
size
of
the
mem
table
for
for
the
loading
and
removing
the
compaction
throttling,
so
that
made
it
possible
to
load
that
data.
It
sat
in
about
1.5
days,
which,
which
is
pretty
fast,
considering
the
size
of
the
data
at
honor,
21
building
edges,
and
then
we
moved
on
to
a
transactional
workloads.
So
now
all
the
data
is
in
the
database
and
so
now
we're
simulating
a
lot
of
students
from
currently
interacting
with
the
system.
B
B
This
is
written
in
gremlin,
which
is
the
query
language
for
Titan
and,
as
you
can
see,
what
we
do
here
is
we.
We
define
a
set
x
and
a
map
m,
and
what
we
want
to
do
is
we
want
to
find
out
good
recommendations
for
a
particular
user.
That
user
would
be
randomly
sampled,
as
I
mentioned
before,
so
what
we
do
is
we
Traverse
out
from
the
user?
B
We
look
at
everybody
that
that
user
follows,
and
then
we
aggregate
all
those
users
an
hour
set
X
to
make
sure
that
we
don't
recommend
any
of
those,
because
that
would
obviously
be
pointless,
but
then
we
take
one
further
step
out,
so
we
look
at
who
do
they
follow
and
then
do
a
group
count
on
that.
So
we
look
at
everybody
who
I
follow,
who
do
they
follow
and
then
I
aggregate
those
by
counting,
you
know
counting
the
duplicates
essentially
and
then
the
logic
behind
this
recommendation
is
well.
B
If
might
if
the
people
I
follow,
follow
these
people
more
frequently
than
probably
those
are
good
recommendations
for
mia
me
as
well,
and
so
that's
what
we
do
down
there.
We
sort
in
this
in
the
second
third
line.
We
sort
those
results
and
return
back,
the
top
number
of
when
number
it
can
be
specified,
so
let's
say
the
top
ten
and
we
would
return
those
to
the
user.
So
this
is
a
fellowship
recommendation
that
we
implemented.
B
There
are
other
kinds
of
recommendation,
algorithms
that
we
implemented,
all
of
which
were
implemented
in
gremlin,
which,
as
I
said
as
the
core
language
for
Titan
and
actually
not
part
of
Titan
itself.
It's
part
of
the
Tinker
pop
ecosystem
and
that's
an
important
part
of
Titan
as
well,
because
it
constitutes
the
ecosystem
in
which
Titan
is
embedded.
We
have
tightened
as
a
native
blueprints
implementation,
which
means
it
benefits
from
all
the
components
of
the
tinker.
B
Bop
stack,
Graham
them
being
one
of
those,
probably
the
most
important,
but
there
are
others
like
the
the
register
grav
server,
which
allows
you
to
expose
any
Titan
server
at
the
rest
endpoint.
So
it
makes
it
very
easy
for
web
development.
There
is
an
algorithm
package
called
furnace
which
implements
a
bunch
of
standard
algorithms
and
we're
now
working
on
a
very
new
type
of
algorithmic
framework.
That
will
be
coming
out
shortly.
B
So
a
lot
of
a
lot
of
things
you
need
when
you're
dealing
with
graphs
are
provided
in
this
ecosystem,
so
Titan
itself
being
a
graph
database
is
important,
but
as
important
is
the
ecosystem
in
which
Titan
is
embedded
going
back
to
the
transactions,
we
ended
up
executing
this
transaction
benchmark
and
we're
able
to
execute
about
ten
thousand
two
hundred
transactions
per
second
on
average.
Now
that
is
obviously
lower
than
numbers
that
Jonathan
shared
with
us
earlier,
but
remember
that
those
are
not
just
individual
sort
of
atomic
transactions.
B
Those
are
complexed
reversals
of
the
graph
like
the
fellowship
recommendation
that
I
just
showed
you,
which
is
a
multi-step
traversal
over
the
entire
graph
and
in
fact,
just
to
give
use
it
like
a
rough
idea
of
what
those
transactions
are.
All
of
those
are
specified
in
the
blog
post.
I
just
gave
you
sort
of
a
an
excerpt
here.
You
see
that
we
have
transactions
here,
like
a
student,
retrieving
all
of
the
content
for
single
course
in
their
content
list.
So
we
have
multiple
pieces
of
contact
that
will
being
traversed
to.
We
have
adding
fellowships.
B
We
have
student
reading
their
streams,
we
have
students
reading
the
most
recent
comments
for
all
of
their
courses
and
so
on
and
so
forth,
and
you
can
see
in
the
column
over
here
we
have
the
average
transaction
times.
For
those
queries
you
can
see
those
very
widely
and
also
the
standard
deviation
of
those
varies
widely,
because
the
complexity
of
the
individual
transactions
varies.
With
that
now
we
talked
about
the
benchmark
and
the
fact
that
we
can
scale
Titan.
One
of
the
things
I
want
to
get
into
now.
B
There
are
three
things:
I
want
to
talk
about,
the
first
being
vertex
representation,
so
the
way
we
represent
the
graph
is
that
we
store
each
vertex
as
a
row
in
Cassandra.
So
here
we
give
an
example
of
an
example
vertex
for
Hercules
that
vertex
is
assigned
id-5,
and
so
we
have
a
row
in
Cassandra
with
that
ID
five
and
then
we
have
one
column
for
each
property
associated
with
that
vertex
and
one
column
for
each
edge
associated
with
that
vertex,
so
essentially
restoring
the
adjacency
list
of
each
vertex
with
the
vertex.
B
One
of
the
important
aspects
here
is
that
there
is
an
induced
order
on
the
edges
and
in
fact,
Titan
allows
you
to
define
what
that
order
should
be
by
defining
primary
keys
on
for
a
particular
edge
label.
So,
for
instance,
here
we
have
to
find
a
primary
key
as
time
on
the
battle
edge,
meaning
we
want
battle
edges
to
be
sorted
by
time.
B
What
that
allows
us
to
do
is
allows
us
to
affect
it
to
efficiently
query
for
all
of
the
recent
battles
that
Hercules
engaged
in
or
for
all
battles
that
fall
in
a
certain
interval,
and
that
makes
courting
very
effective,
in
particular
when
you
have
super
nodes,
sub
vertices
or
the
larger
edges
incident
upon
them,
and
what
this
translates
into
our
row
index
retrievals
in
Cassandra.
So
we
use
the
row
index
a
mechanism
that
comes
with
Cassandra
to
execute
those
cores
efficiently.
B
Now,
with
the
adjacency
list
representation,
one
of
the
downsides
is
that
we
effectively
storing
each
edge
twice
right,
one
on
the
one
on
the
incoming
one
of
the
outcoming
outgoing
vertex.
In
order
to
mitigate
that
duplication
in
in
storage
we're,
we
invested
a
lot
of
effort
into
making
sure
that
we
can
represent
edges
compactly.
So
we
use
variable
long
encoding
for
all
of
these
IDs
that
we
have.
B
Lastly,
one
important
aspect
of
Titan
is
graph
partitioning
and
that's
one
of
the
things
that
we're
spending
a
lot
of
time.
Think
about-
and
you
know,
trying
out
various
approaches.
It's
it's
a
very
challenging
problem
as,
as
we
all
know,
it's
actually
NP
complete
and
we're
looking
into
how
we
can
effectively
petition
gratz
the
way
we
do
that
in
cassandra
is
we.
We
essentially
map
the
partitioning
strategy
onto
the
ID
scheme
and
thereby
onto
the
tokens
that
get
assigned
to
a
particular
vertex.
B
We
use
byte
order
partitioning
at
cassandra,
so
we
have
control
over
where
a
particular
ID
or
token
for
that
matter
gets
assigned.
So
we
know
what
machine
and
particular
vertex
gets
assigned
to,
and
so
what
we
can
do,
as
illustrated
in
this
example,
if
we
know
a
good
partition
cut
for
a
graph,
we
can
then
assign
the
IDS
such
that
we
know
that
vertices,
which
should
be
close
together,
I
actually
allocated
to
the
same
physical
machine
or
in
fact,
to
the
same
denote
by
which
factor
they
will
then
be
on
the
same
physical
machine.
B
One
of
the
key
questions,
of
course,
then
as
well,
how
do
you
decide
whether
or
not
two
vertices
should
be
on
the
same
machine?
And
that
in
fact,
is
a
very
challenging
problem
for
the
Pearson
study?
We,
we
used
a
transactional,
bound
petition
mechanism,
so
we
said
that
you
know
vertices
that
are
assigned
in
the
same
transaction.
B
We
essentially
just
do
partitioning
on
those
vertices
which
works
pretty
well,
because
a
lot
of
the
relationships
are
centered
around
universities
and
the
students
contained
in
those
but
other
use
cases
might
have
other
petitioning
schemes
that
are
more
effective.
Titan
allows
you
to
plug
and
play
your
petitioning
mechanism
into
the
into
the
database
itself,
so
you
can
optimize
the
petitioning
strategy
for
your
particular
use
case,
but
most
certainly
I.
B
Now
we
talked
about
Titan
a
lot,
which
is
a
real
time
graph
database,
as
I
mentioned
in
the
beginning
of
the
talk.
They
are
further
components
to
the
o.r,
alias
graph
cluster
Titan,
being
one
of
them
for
the
real
time
graph
database,
but
there's
also
fanous
faunas
is
Titans
brother
in
a
way
and
that
it
is
a
based
graph,
analytics
platform.
It
works
by
compiling
gremlin
queries.
So
that's
the
query.
B
Language
I
was
talking
about
earlier
into
MapReduce
steps,
so
you
can
run
MapReduce
graph
analytics
for
graph
analytics
jobs
on
top
of
Titan
by
sucking
the
data
out
of
Titan,
then
doing
the
batch
analytics
and
then
either
writing
the
results
back
into
Titan
oil
into
some
into
HDFS
or
some
other
output
format
to
get
the
results
of
your
computation
farness
inner
of
itself
is
it's
this
whole
own
Beast.
So
I
could
give
a
whole
talk
about
fondness,
but
you
can
also
check
it
out
online.
You
know,
documentation
all
is
available.
B
This
is
just
to
say
that
Titan
is
one
component.
Faunus
is
a
whole
separate
world.
It
enables
you
to
do
batch
type
analytics,
so
you
can
look
at
degree.
Distribution,
so
you
can
look
at
any
kind
of
statistics
on
the
graph
or
you
can
do
very
extensive
traversals,
where
you
have
to
traverse
out
of
every
vertex
in
the
graph.
B
So
those
are
the
kinds
of
things
that
funds
allows
you
to
do
and
then,
lastly,
currently
under
development
and
coming
out
pretty
soon
as
full
gora,
which
is
a
graph
traversal
framework
which
is
optimized
put
in
memory
traversals.
So
if
you
have
very
extensive
traversals
where
it
is
too
expensive
to
go
to
disk
or
any
even
just
any
kind
of
intermediate
representation,
then
fog
or
will
make
it
faster
because
it
resides
entirely
in
memory.
The
point
of
this
crowd.
B
The
point
of
this
light
is
really
to
give
you
an
idea
for
how
all
these
pieces
fit
together
and
the
fact
that
we're
building
one
coherent
in
korean
graph
framework
that
allows
you
to
address
various
Grafton's
cases
within
the
context
of
just
one
framework
with
that
I
want.
To
conclude
my
talk.
I
want
to
thank
Pearson
for
their.
You
know
continued
support
on
this.
D
B
Sapporo
bagel
yeah
so
yeah,
so
this
is
so
pretty
good
when
you
bring
up
giraffe
so
so
GF
is
actually
not
so
similar
to
tighten
its
more
in
the
Faunus
framework,
meaning
it's
more
comparable
too
fanous,
meaning
it's
it's
a
batch
graph
analytics
framework.
So
it
also
allows
you
to
analyze
graphs,
but
it
does
so
by
looking
at
a
graph
as
a
static
graph,
and
then
you
run
a
batch
job
over
that
graph.
B
So
you
can
extract
statistics
or
you
can
do
any
other
kind
of
computation,
but
the
graph
is
static
and
your
computation
is
a
batch
computation.
So
you
know
with
the
overhead
of
Hadoop
and
so
and
so
forth.
You
would
expect
your
results
in
minutes
or
hours
depending
on
the
size
of
the
graph.
Titan
is
an
OLTP
graph
database.
So
the
point
here
is
really
real
time,
so
you
can't
answer
cores
and
the
results
should
become
should
come
back
to
you
in
milliseconds,
so
you
can
drive
web
applications
or
any
other
kind
of
user
facing
application.
B
Alright.
So
so
that
is
the
comparison
to
giraffe.
There
are
other
graph
databases,
obviously
that
are
also
oltp,
so
they
are
real
time
graph
databases
to
those
I
think
one
of
the
characteristics
of
Titan
is
it's
open
source,
it's
Apache
tube
and
it's
one
of
the
first
graph
databases
that
is
looking
at
scaling
graph
data,
meaning
scaling
it
across
multiple
machines.
B
E
So
no
coming
back
to
the
question
on
that
massive
simulation
seems
very,
very,
very
impressive.
That
simulation
that
you
did
so
you
mentioned
prior
to
talking
about
the
simulation
you
mentioned
about
this
being
used
for
real-time
social
analytics.
So
how
are
you
able
to
simulate
I
didn't
catch,
how
you
were
able
to
simulate
the
the
social
interactions
in
that?
How.
B
So
we
basically
had
these
templates
that
are
characteristics
for
an
interaction
that
a
user
would
have
with
the
social
network,
and
then
we
would
substitute
the
randomly
chosen
user
with
that
template
and
then
execute
that
transaction
against
the
graph
database
right
and
thereby
we
would
simulate
a
complex
interaction
like
a
student
logging
in
clicking
on
a
course
clicking
on
everybody
in
that
course
or
student
logging,
and
looking
at
all
the
content.
You
know
these
types
of
interactions
would
be
simulated
in
this
way.
All
right,
then
we
measured.
Basically,
how
long
does
it
take?
F
B
So
know
that
the
techniques
we
looking
at
for
graph
partitioning,
the
ones
we
found
most
effective,
are
labeled
propagation
based
mechanisms
and
and
that's
sort
of
the
direction
we're
going
in
as
I
said.
However,
we
decided
to
make
the
partitioning
mechanism
plug
and
play
and
tighten
because
we
realized
that
an
optimal
petitioning
strategy
is
always
dependent
on
your
domain.
C
C
B
So
20,000
students
is
a
lot
I
mean
just
to
materialize.
That
results
that
I
mean
that's,
that's
a
lot
of
data
that
you
need
to
get
out
of
the
database.
Typically,
we
find
that
in
the
context
of
a
particular
algorithm
or
particular
goal
that
you're
trying
to
achieve
there
are
certain
subsampling
strategies
that
you
can
use,
of
instance,
for
the
follow
recommendation
that
I
showed
you.
One
of
the
things
we're
doing
is
we're
subsampling
by
recency,
so
we're
looking
at
people
who
you
have
recently
started
to
follow
and
not
at
everybody.
B
You
have
followed
all
right,
so
that
allows
you
to
constrain
that
that
search
space
and
allow
you
to
rein
in
on
that
combinatorial
explosion
that
you
often
have
with
graphs
and
Titan
can
do
that
very
effectively
because
it
amounts
to
the
induced
order
on
the
edges
that
are
talked
about.
So
you
can
therefore
use
the
row
indices
that
come
with
Titan
to
effectively
extract
those
edges
and
therefore
those
neighbors.
G
B
The
replication
factor
is
configurable.
In
fact,
Titan
has
a
configuration
option
for
that
allows
you
to
configure
that
replication
factor
and
a
lot
of
other
things
that
you
would.
You
know
typically
configure
with
Cassandra
like
you
know
what
what's
your
right,
consistency,
we
consistency
levels,
those
kinds
of
things
also
expose
that
the
Titan
level,
because
they
directly
map
onto
each
other.
H
H
D
I
have
another
question:
actually
this
is
more
about
full
gora.
Is
it
so
I
just
recently
came
across
something
called
reckful
J,
which
is
a
recommendation
engine
on
top
of
neo4j?
Are
you
planning
on
something
similar
for
full
gora,
like
where
you
take
common
applications
like
recommendation
engines
and
so
on
and
yeah.
B
So
actually
it
is
my
understanding
that
req
for
j
is,
is
sort
of
generalizing
their
underlying
storage
infrastructure,
so
that
they
can
interact
with
any
blueprints
implementation
or
they
have
something
similar
to
that
where
they
abstract
away
the
actual
database
and
just
have
an
interface
to
that.
So
in
that
sense,
you
could
run
rec
for
j
in
you
know
on
something
like
Titan,
but
that
has
not
been
done
yet.
We
have
an
algorithm
package
called
furnace,
which
it
essentially
has
a
bunch
of
these
standard,
algorithmic
templates
implemented.
B
We
don't
have
anything
like
a
you
know
like
a
standard
recommendation
engine
yet
because
we
don't
I
mean
with
the
customers
that
we
interact
with.
We
often
see
that
they
use
these
standard
algorithms,
but
the
recommendation
engine
itself
is
typically
tailored
to
their
particular
domain
or
to
their
particular
liking.
So
we're
not
aware
of
anything
that
is
like
a
you
know
like
a
one-size-fits-all
recommendation
engine
so,
but
we
have
these
kinds
of
pieces
that
you
can
then
use
to
assemble
the
own
recommendation.
E
You
talked
about
partitioning
and
how
easy
it
is
to
change
the
partitioning
in
your
current
setup.
You
know,
because
I
can
see
for
certain
types
of
problems.
You
know
you
might
want
to
change
the
way.
The
partitioning
works,
okay
for
the
very
same
application
yeah.
So
you
are
looking
at
one
type
of
questioning.
You
know
you
need
one
kind
of
partitioning
to
make
this,
so
is
it
easy
to
change
the
partitioning
or
what
is
your
experience
so.
B
So
right
now
you
would
that
the
petitioning
strategy
would
be
fixed.
So
once
you,
you
know,
once
you
decide
on
a
petition
strategy,
would
be
using
that
partition
strategy.
I
have
not
yet
come
across
a
use
case
where
you
would
swap
it
out
for
another
petition
strategy.
There's
actually
no
reason
that
that
wouldn't
work,
because
you,
basically
you
incrementally
assign
vertices,
and
so,
if
you
change
the
logic
behind
that
assignment,
there's
nothing.
You
know
in
the
existing
data
that
we
get
impacted,
but
quite
I
mean
at
this
point
in
time.
B
Titan
works
with
the
most
recent
version
of
blueprints,
so
Titan
zero
point:
three
point:
one
is
compatible
with
blueprints
two
point
three
point:
oh
and
in
fact
we're
trying
to
sink
that
up
a
little
better,
so
so
that
the
the
sub
versions
match,
but
essentially
a
lot
of
a
lot
of
the
stuff
that
we're
doing
in
Titan
is
also,
to
some
extent
driving
the
developments
and
blueprints
and
vice
versa.
So
so
the
two
have
been
tightly
following
each
other
in
a
way,
and
so
tightness
has
always
been
up
to
date.