►
From YouTube: C* Summit 2013: Big Data Analytics - Realize the Investment from Your Big Data Clusters
Description
Speaker: Mark Davis, Principal Engineer at Dell
Slides: http://www.slideshare.net/planetcassandra/c-summit-2013-big-data-analytics-realize-the-investment-from-your-big-data-clusters-by-mark-davis
The term "big data" seems to be everywhere these days. With the ever growing number of attendees at big data and Hadoop events, it's clear big data is here to stay. But what does that mean for the analytics market, and how does big data fit into the picture? This session, featuring Mark Davis, Sr. Product Architect at Dell, will explore what big data means in a practical sense to the IT department. It will also explore the many ways that big data affects an organization's picture of performance. Plus, see how big data analytics, using technologies like Cassandra and Hadoop, will converge with traditional business intelligence to create a complete picture of the enterprise's information assets, thereby giving the business a complete and insightful view of its operational efficiency.
A
It's
funny
because
every
time
I
hear
adrian
talk,
I
hate
it
because
I
always
feel
like
I'm
not
doing
enough
get
like
six
months
of
busy
work
out
of
it.
So,
and
it's
been
like
that
for
years,
I
mean
I've
seen
these
talks
for
years
and
years,
and
it
just
seems
like
I've,
get
all
this
extra
work
every
time
I
see
them
talk,
but
it's
cool
stuff,
I'm,
really
glad
that
they
do
the
open
source
of
for
those
of
you
don't
know
we
have
a
Cassandra
committer
that
works
at
Netflix.
A
So
some
of
the
code,
that's
in
Cassandra,
is
you
know
stuff,
that's
real
world
experience
in
Netflix,
so
you
know
this
is
part
of
the
community
effort
that
we
have
open
source.
It's
great.
So
our
next
speaker
is
Mark
Davis.
He
he
works
at
Dell,
but
it's
funny.
We
were
just
talking
in
the
back.
He
started.
He
was
a
co-founder
of
Katanga,
which
was
then
bought
by
quest
which
was
bought
by
dell.
So
that's
how
he
got
there,
but
we're
gonna
be
talking
about
big
data.
Analytics
thanks
mark
thank.
B
You
very
much
and
yeah.
I
just
want
to
mention
straight
away
that
in
terms
of
dell,
this
is
all
part
of
a
sort
of
long-term
sort
of
mission
and
evolution
of
dell,
where
we
began
as
a
supplier
of
servers
and
in
laptops
initially
and
then
gradually
over
time,
becoming
more
of
a
full
service
enterprise.
Some
software
company,
I'm
going
to
talk
about
a
lot
of
use
cases
and
some
high
level
stuff,
and
then
I'm
going
to
talk
a
little
bit
of
because
about
Cassandra
at
the
end,
not
initially.
B
But
there
is
some
Cassandra
in
here
so
first
off
in
terms
of
sort
of
high-level
perspectives.
There's
some
people
have
done
some
calculations
and
said
well.
If
you
took
all
of
the
human
spoken
language
up
until
about
nineteen
hundred
and
you
were
to
encode
it
at
a
certain
rate
and
how
many
people
were
they
and
how
often
did
they
talk
and
communicate
with
each
other
and
so
forth?
Just
broad
estimates,
maybe
ten
exabytes,
which
is
one
times
ten
to
the
18th.
As
you
all
know,
and
and
then
something
happened,
and
that
is
our.
B
Our
data
generation
began
to
expand
and
grow
and
grow
to
the
point
where,
as
we
all
know,
we're
in
the
realm
of
Big
Data
technologies,
where
we're
confronting
this
edge.
But
one
of
the
one
of
the
interesting
up
takes
about
this
is
in
terms
of
sort
of
the
economic
consequences
of
data,
RJ,
Gordon,
who's,
a
economist
and
therefore
part
of
the
dismal
science
at
Northeastern
or
northwestern
talks
about
the
notion
of
industrial
revolutions
and
the
impact
that
those
have
had
on
the
US
economy
and
so
industrial
revolution.
B
Number
one
was
what
really
prosaic
in
to
our
minds,
but
the
invention
of
steam
technology
steam,
locomotives
and
so
forth.
And
with
that
we
saw
an
incredible
disruption
of
traditional
ways
that
people
did
things
and,
at
the
same
time,
tremendous
hop
economic
opportunity
and
that's
sort
of
a
one
of
Gordon's
themes
is
that
with
technological
change
comes
enormous
disruption
but
great
opportunity
as
well.
B
Industrial
Revolution
number
two
then
happens
with
electrification
in
the
United
States,
as
well
as
something
as
important
as
telephony
and
running
water
and
flush
toilets,
and
this
is
this
is
kind
of
may
seem.
You
know
like
a
very
basic
thing,
but
there
was
a
time
where
the
in
New
York
City,
the
sanitation
workers
would
march
in
parades
and
people
would
cheer
them
for
what
they
had
achieved
in
terms
of
liberating
the
city
from
the
continuous
stink
of
humanity.
B
Industrial
Revolution
number
three
then
comes
about
with
the
world
wide
web
and
the
availability
of
portable
electronic
devices
and
they're
in
a
connectivity
within
that,
and
here
we
start
to
see
an
enormous
acceleration
of
the
data
that's
being
generated
and
we
see
technologies
like
Cassandra
and
other
Big.
Data
technologies
become
relevant
as
people
began
to
outpace
and
outstrip
the
capabilities,
and
sometimes
the
price
of
our
DBMS
and
other
technologies
that
grown
up
around
the
same
sort
of
time
so
for
Gordon.
Now
this
is
the
end
of
the
line
he
says
well
yeah.
B
We
achieved
all
this,
but
there's
no
major
disruption
that
we
can
foresee
in
the
future
and
so
he's
as
I
say,
he's
a
pessimist,
he's
part
of
the
dismal
science
of
of
economics,
but
other
people
have
said.
Well,
maybe
the
big
data
revolution
is
really
the
next
Industrial
Revolution,
and
this
has
included
dismal
scientists
like
Paul
Krugman
in
the
New
York
Times,
who
has
had.
B
Perhaps
it
is
big
data
that
is
going
to
shape
the
unlocking
of
intelligent
systems
from
the
kind
of
brittleness
that
we
we've
seen
historically
from
artificial
intelligence
and
bring
it
on
into
a
scale
where
it
is
effective
in
terms
of
being
able
to
disrupt
large
numbers
of
low-level
workers,
large
numbers
and
knowledge
workers
and
to
replace
them
with
with
machines.
And
so
this
is
where
we're
sort
of
we
stand
today
and
we're
moving
up
into
the
the
zettabyte
and
the
range
at
the
moment.
B
In
terms
of
the
data,
some
people
say:
20
40
people
like
Ray
Kurzweil,
and
this
will
be
a
watershed
event,
a
singularity
and
so
forth.
Them
I
tend
to
think
it's
probably
a
little
later
than
that,
but
along
the
way,
we
have
this
kind
of
zoo
of
Big
Data
technologies
and
these
range
from
technologies,
techniques
and
ideas.
B
Even
that
have
come
about
whether
this
is
the
notion
of
eventual
consistency
of
the
one
hand
or
companies
that
are
involved
in
big
data,
as
well
as
the
confronting
of
different
types
of
data
that
we've
never
really
processed
before,
like
unstructured
data
in
the
sense
of
raw
text
and
and
blog
post
and
so
forth,
to
extract
sentiment
to
extract
named
entities,
people
places,
organizations
from
it
and
all
of
our
customers
are
confronting
the
same
kinds
of
challenges.
How
do
they
align
the
Big
Data
technology
with
their
their
future?
B
Looking
business
objectives,
their
current
business
objectives
and
they
they're
looking
for
for
guidance
in
terms
of
doing
that?
So
a
lot
of
a
lot
of
what
I'm
going
to
talk
about
has
to
do
with
intelligent,
intelligent
systems
and
use
cases
for
them,
but
also
data
integration.
And
how
do
you
integrate
data
for
companies
like
like
Netflix?
They
have
some
data
integration
challenges,
but
they
mostly
have
a
volume
sort
of
challenge,
and
so
variety
is
is
perhaps
less
important.
But
what
we
find
is
that
we
have
all
of
the
different
dimensionality
of
big
data.
B
That's
involved
so
I'm
going
to
talk
about
some.
Some
use
cases
run
through
a
whole
bunch
of
them
and
just
talk
a
little
bit
about
how
we
help
to
solve
them
and
some
of
the
challenges
that
were
involved
in
that
and
then
I'll
wrap
up.
I
don't
want
to
do
too
much
of
a
product
pitch,
but
just
talk
a
little
bit
about
my
company,
what
we
created
and
what
were
moving
forward
in
terms
of
integration
with
the
entire
del
capabilities.
B
So
we
got
our
start
in
around
two
thousand
eight,
as
que
tenga
and
our
primary
customers
were
in
the
US
government
and
they
were
focused
on
defense
intelligence.
The
the
goal
for
them
was
to
improve
force
effectiveness,
and
that
meant,
when
Humvees
roll
out
in
the
conflicts
in
Iraq
at
the
time
and
then
into
Afghanistan,
and
then
the
officers
come
back
and
they
write
an
after-action
report
on
the
day's
activities.
In
order
to
do
this
sort
of
thing,
they
write
narratives.
B
They
have
semi-structured
data,
the
locations,
information
of
that
sort
who
was
involved
and
who
they
encountered,
what
the
day's
activities
were.
If
there
was
any
kinetic
events,
which
is
the
polite
way
of
saying
that
the
gunfire
was
exchanged,
and
they
record
all
that
material,
but
in
order
to
make
use
of
that
then
down
the
line
so
that,
as
I'm
doing,
mission
planning
later
on,
I
want
to
know
what
to
expect
as
I
enter
a
village.
B
What
to
expect
as
my
these
roll
down
the
road
and
how
to
essentially
turn
that
into
actual
intelligence,
we
need
to
do
different
things
to
that
data.
That
is,
for
the
narrative
fields.
We
need
to
make
it
searchable.
We
also
would
like
to
extract
name
to
entities
from
that
using
natural
language
processing
to
be
able
to
extract
the
names
of
the
people
that
were
encountered.
B
The
information
processing
task
tend
to
be
fairly
heavy
weight,
as
you
can
imagine,
and
so
being
able
to
distribute
that
load
over
multiple
machines
in
a
big
data.
Sense
is
a
critical
part
of
that,
and
on
top
of
that,
we
want
to
provide
tools
that
don't
just
populate
sort
of
templates
associated
with
this
data,
but
make
it
visually
understandable.
B
So,
in
terms
of
our
initial
use
case,
we
were
working
with
US
Army
Intelligence,
the
I
2wd,
which
is
the
intelligence,
information,
warfare,
Directorate
and
they're.
The
system
that
they
had
for
many
years
did
not
scale
over.
They
were
overwhelmed
as
the
army
encountered
more
and
more
engagements
in
more
and
more
theaters
and
they
needed
analytic
ability,
capabilities
that
that
merge
with
search
all
in
the
same
environment.
B
So
there
there's
one
use
case,
one
of
our
primary
use
cases
initially
and
but
the
same
theme
can
be
applied
in
sort
of
different
realms,
for
example,
in
competitive
intelligence.
We
wanted
to
change
our
indie
priorities
and
in
improve
marketing
approaches
for
a
major
Silicon
Valley
company.
They
want
to
be
able
to
look
at
their
competitors.
B
Information,
whitepapers
and
material
on
the
web,
be
able
to
extract
what
they're
doing
with
ddr3
memory
and
things
of
that
nature
to
crawl
web
sites
and
to
be
able
to
make
all
of
that
information
actionable
at
some
level
and
so
for
a
technology
company.
We
created
custom
natural
language
processing
capabilities
that
do
things
like
extract
the
names
of
flash
memory,
memory,
types,
electronic
devices
and
so
forth
from
them
the
mass
of
their
websites,
and
we
actually
found
for
for
some
of
the
companies
we
were
working
for.
B
They
were
surprised
that
there
was
that
much
material
on
the
web.
So
it
was
a
big
big,
surprise
and
great
bind
all
this
metadata
now,
and
this
is
critical
in
terms
of
reaching
the
end
user,
the
the
business
analyst
and
the
line
of
business
decision
maker
binding
all
that
metadata
and
capabilities
into
a
visualization
capability
so
that
they
can
interrogate
the
data
they
can
drill
down.
They
can
understand
the
factors
that
contribute
to
the
way
in
which
the
data
is
presented
and
so
that
they
can
make
better
decisions,
drug
discovery
so
we're
moving
far
afield.
B
Now,
we've
gone
from
military
intelligence
to
competitive
intelligence
for
drug
discovery.
A
the
same
sorts
of
themes
occur,
published
research,
reports
genomics
and
whole
proteomics
databases.
We
want
to
be
able
to
extract
what
are
the
compounds
that
are
involved?
What
are
the
drugs
and
the
downstream
biological
consequences
of
those
drugs,
as
they
are
metabolized
throughout
the
human
body
and
so
forth?
And
this
means
taking
a
look
at
different
data
sources
and
data
integration
turns
out
to
be
an
enormous
challenge.
B
You
also
can
go
to
the
WWF
da
govt
site
and
you
can
do
a
basic
keyword,
search,
presenting
ranked
list
of
results,
and
it's
a
very
simple
page
through
sort
of
thing.
One
thing
that
they
found
was
that
internal
regulators
could
not
access
drugs
by
the
regulatory
number.
What's
called
the
NDA
number
of
their
accession
number,
and
that
was
due
to
the
fact
that
a
lot
of
search
engines
take
numbers
and
throw
them
away
before
indexing,
and
they
do
that
because
numbers
are
very
productive
and
they
would
pollute
the
database
in
the
door.
B
The
the
index
for
the
search
engine
and
the
they
would
have
index
bloat
and
nobody
wants
that
in
their
search
index.
So
they
threw
it
away.
But
that
meant
that
nobody
could
figure
out
where
in
the
regulatory
pipeline
any
of
their
drug,
the
drugs
were,
and
so
by
using
natural
language
extractors,
we
were
able
to
extract
patterns
that
included
things
like
NDA
numbers.
What
is
the
status
of
the
drug
in
the
regulatory
pipeline?
B
Is
it
approved
as
a
recall
and
so
forth
and
bind
that
into
a
interrogate,
able
and
search
resource
that
you
can
now
see
at
a
glance?
What
are
all
the
ingredients
in
the
drugs
related
to
that
contain
acetaminophen
and
what
is
their
status
in
the
regulatory
pipeline
and
what
is
the
the
the
company
that
manufactures
them
and
CRM
analytics?
B
This
is
a
ongoing
work
that
we're
doing
with
a
major
telecom
provider,
and
your
sources
are
customer
relationship
management
records.
They
are
typically
in
narrative
fields,
despite
as
much
training
as
you
can
possibly
give
to
call
center
of
people
who
are
escalating,
calls
in
about
their
service
and
their
phones
and
and
so
forth,
they're
going
to
write
things
in
idiosyncratic
ways
and
it's
going
to
be
difficult
to
extract
meaning
from
it.
B
But
the
goal
is
really
to
normalize
and
extract
some
sort
of
value
out
of
it
and
to
provide
the
ability
to
build
to
drill
down
on
that,
and
these
records
for
a
large
telat
of
telco
provider
are
typically
not
used
in
any
way.
At
the
current
moment,
they
fall
on
the
floor.
They
are
recorded,
they
are
in
oftentimes.
You
there's
a
Hadoop
cluster,
that's
being
used
to
dump
that
data
and
sort
of
the
notion
of
the
the
rubbish
sheet
of
heap
of
HDFS,
but
very
little
is
done
with
that
data
network
analytics.
B
This
is
a
canonical
use
case,
of
course,
for
things
like
Hadoop
and
pig
being
able
to
pull
data
together,
Cassandra's
well
to
a
degree,
and
we
worked
with
a
router
switch
vendor
who
revealed
to
us
that,
in
order
to
understand
network
failures
and
managed
network
switches
and
devices,
they
did
not
have
sufficient
Oracle
capacity
to
store
the
data
and
embedded
analytics
was
too
slow,
and
so
they
had
a
tiger
team
halfway
around
the
world.
That
would
dump
the
entire
database
and
they
had
a
library
of
perl
scripts
that
they
would
run
across
the
database
dumps.
B
That
was
a
bit
of
a
an
eye-opener
for
for
us
and
I'll
just
mention
a
couple
more
I
want
to
I
want
to
get
to
showing
you,
rather
than
describing.
You
and
I
usually
do
a
live
demo,
but
I
don't
have
that
option
today
by
signals.
This
is
a
classic
sort
of
you
case
associated
with
targeting
sales
and
doing
recommendations
associated
with
with
products
and
crawling
an
extracting
sentiment
associated
with
Twitter's
blogs
websites
and
things
of
that
sort.
How
do
you
make
use
of
that
data
once
you
have
acquired
it?
B
What
is
the
the
output
that
you're
providing
in
order
for
people
to
make
better
decisions?
Sometimes
it's
a
statistical
outcome,
but
oftentimes
it's
a
discovery
process
where
people
are
using
visualization
tools,
combined
with
search
to
be
able
to
drill
down
on
that
kind
of
data
and
there's
more
use
cases
around
here.
So
in
terms
of
my
goal
with
this
talk
is
how
do
you
make
use
effectively
of
big
data
I
just
want
to,
and
in
terms
of
your
cluster,
to
kind
of
germinate?
B
Some
seeds
have
thought
about
these
kinds
of
use
cases
the
the
companies
that
are
involved
in
in
big
data
and
as
part
of
their
line
of
business.
You
know
they
understand
what
they're
trying
to
achieve,
and
a
lot
of
you
understand
what
you're
trying
to
achieve,
but
for
the
broader
Enterprise
world,
they're
still
sometimes
struggling
to
understand
how
to
make
use
of
these
technologies.
B
However,
we
got
our
start
working
on
top
of
Hadoop
and
so
batch
style
analytics,
and
our
goal
is
really
to
transform
big
data
into
actionable
intelligence,
and
that
means
being
able
to
do
things
like
aggregate
and
count
things
at
a
clearly
a
critical
use
case
associated
with
big
data,
but
where
we
really
kind
of
differentiated
ourselves
was
by
providing
in
intelligent
systems
on
top
of
that
machine
learning
and
natural
language
processing
and
then
binding.
All
of
that
metadata.
B
So
we
we
talked
about
on
to
users
of
our
technology,
and
we
found
that
this
sort
of
resonates
with
almost
any
technology,
that's
trying
to
achieve
analytics,
/,
big
data.
So
on
the
left,
as
that
stage
right,
we
have
the
analytic
analytical
producer
and
the
analytical
producer
is
the
data
scientist.
It's
the
business
intelligence
person
who's
been
tasked
with
performing
data
science,
but
it's
somebody
who
needs
to
take
big
data
resources
that
they've
perhaps
never
used
before,
to
try
and
get
some
traction
on
and
produce
an
analytical
product
out
of
it.
B
And
then
on
the
other
side,
we
have
the
analytical
consumer
and
that's
a
line
of
business
decision
maker
who
needs
to
take
the
results
of
those
analytical
products
and
be
able
to
make
a
decision
about
out
of
it,
and
so
we
have
capabilities
in
this
kind
of
Mickey.
Mouse
been
diagram
here
that
range
from
analytics,
including
the
some
of
the
features
that
I
talked
about:
natural
language,
processing
and
machine
learning.
We
build
on
top
of
Hadoop
all
of
our
pipelines
and
everything
work
across
the
do.
B
We
have
visualization
capabilities
and
then
search
that
that
goes
beyond
the
rank
list
and
really
binds
faceted
visualization
directly
into
the
search
experience.
So
what
does
that
look
like
well?
So
here
you
have
an
example
of
how
to
create
a
workflow
and
the
workflow
is
doing
some
extraction
of
HTML
links
and
HTML
text.
We
have
a
variety
of
tools
that
you
drag
and
drop
onto
a
stage.
You
wire
them
up,
you
parameterize
them
and
you
configure
them.
B
And
what
do
we
mean
by
bind
intelligently
to
that?
Well,
we
may
need
to
in
generating
a
geo
spatial
visualization.
We
need
to
do
some
sort
of
intelligent,
geospatial,
name
disambiguation,
and
so
we
take
a
term
like
London
and
it
turns
out
there's
a
one
in
Ohio,
San,
Francisco
I
believe
there's
10,000
san
francisco
mint
in
Latin
America
alone.
B
How
do
you
rank
those
and
provide
the
ability
to
filter
down
on
it
to
be
able
to
provide
the
visualization
that
you
want,
whether
it's
by
population
level,
filtering
or
country
level
filtering
and
things
of
that
sort
to
provide
the
ability
to
bind
these
visual
tools
directly
to
Hadoop
data
and
the
shorten?
Essentially
that
data
science
workflow
cycle,
where
you
ramp
up
a
team?
B
They
have
an
analytical
product
task
is
confronting
them
and
they
have
to
assemble
the
expertise
and
the
tools,
and
then
they
ultimately
have
to
bind
that
into
some
sort
of
visual
representation
for
the
line
of
business
decision
maker.
And
then,
in
addition
to
that,
we
have
a
search
driven
capability
that
combines-
and
some
of
you
recognize
some
d3
in
there
if
you're
from
the
open
source
community
that
binds
visual
tools
directly
into
a
search
experience.
B
So
when
you
start
to
do
a
search
now,
all
of
the
metadata
gets
populated
into
different
visual
tools,
and
you
can
see
some
very
interesting
outcomes.
You
can
see
what
essentially,
what
the
lay
of
the
land
is
with
regard
to
your
search
results,
if
you
do
an
initial
query
on
one
term
and
then
you
get
a
visual
representation
and
I
go
back,
go
back
far
enough
and
I
have
a
few
minutes.
I
think
that
I'll
just
mention
an
anecdote
about
this.
B
This
goes
back
to
the
early
days
of
information,
retrieval
research
in
America
and
say
the
92
295
range
and
sudha
may,
who
is
later
my
boss
at
at
Microsoft
Research,
did
a
study
where,
where
she
found
that,
if
you
put
people
in
front
of
an
information
retrieval
system
and
at
the
time
information
retrieval
is
only
of
interest
to
librarians
and
spies.
Hardly
anybody.
There
was
no
web.
B
If
you
put
them
in
front
of
that,
and
you
ask
them
to
guess
at
their
search
terms
and
start
typing
and
so
forth,
that
their
ability
to
use
the
search
engine
was
directly
proportion
to
their
s80
verbal
scores,
in
other
words
their
their
ability
to
guess,
words
was
proportional
to
their
verbal
fluency.
But
what
she
found
was
that
if
you
provided
more
like
this
context,
something
as
simple
as
do
you
do
an
initial
query
on
Saturn
and
you
get
cars
and
you
get
some
documents
on
planets
and
there's
a
lot
of
ambiguity.
B
Why
of
going
from
a
ranked
list,
result
to
other
tools
that
sort
of
fulfill
a
broader
agenda
associated
with
data
interaction,
to
look
at
the
geospatial
layout
of
search
results
to
look
at
the
companies
mentioned
within
search
results
here,
in
this
case,
we're
looking
at
thousands
of
tweets
and
Facebook
postings
that
have
had
their
sentiment
analyzed
and
we're
binding
into
a
visual
representation
sediment
bearing
terminology
in
the
middle
and
topic
areas
over
in
pie,
charts
and
so
forth.
So
you
can
drill
down
on
it
and
according
to
what
your
needs
are.
B
And
this
sort
of
expands
on
that-
and
we
can
see
here,
the
use
of
heat
maps
and
other
visual
representations
to
be
able
to
expand
and
enhance
that
so
just
to
find
out
finalized
and
don't
know
how
long
I
have.
Is
anybody
monitoring
that
a
couple
minutes?
Okay,
great
I
did
want
to?
I
want
to
touch
on
cassandra
since
we're
here
at
cassandra
summit
and
how
were
we're
integrating
into
kissena
with
cassandra,
and
it
turns
out
that
that
dell
and
its
wisdom
of
purchasing
quest
was
in
quest
purchasing
us
all.
Within
five
days.
B
It
is
that
we
have
the
ability
now
to
mash
that
up
cassandra
with,
for
example,
user
login
information
with
crawls
of
the
web,
with
crawls
of
feeds
of
social
media
and
to
be
able
to
get
a
360
degree
view
of
the
combination
of
internal
data
and
external
data,
all
using
the
sort
of
zoo
of
different
technologies.
That
has
been
coordinated
in
terms
of
its
some
ability
to
pull
that
together
and
without
I'd
like
to
thank
you
and
send
you
off
to
your
lunch.