►
From YouTube: OpenStack on Ales 2013 - Ceph on Openstack
Description
By Ian Colle
As the size and performance requirements of cloud deployments increases, storage architects are increasingly seeking new architectures that scale.
Ceph is a fully open source distributed object store, network block device, and file system designed for reliability, performance, and scalability from terabytes to exabytes. Ceph utilizes a novel placement algorithm (CRUSH), active storage nodes, and peer-to-peer gossip protocols to avoid the scalability and reliability problems associated with centralized controllers and lookup tables.
Ceph's architecture is based on RADOS, an object store with support for snapshots and distributed computation. Ceph offers a Swift and S3-compatible REST API for seamless data access to RADOS. Ceph's network block device can be used to store large images and volumes in RADOS, supporting thin provisioning and snapshots. Ceph has native support for Qemu/KVM, libvirt, CloudStack and OpenStack which makes it an attractive storage option for cloud deployments.
As a result of participating in this session, attendees will gain a full understanding of the Ceph architecture, its current status, and plans for future development. Attendees will also learn how Ceph integrates with OpenStack to provide a unified object and block storage service.
Help us caption & translate this video!
http://amara.org/v/FG0G/
A
This
is
amazing,
if
any
of
you
guys
aren't
from
here
haven't
been
in
this
area
that
I
was
just
astounded
by
by
the
views
of
the
mountains
around
here
and
honestly,
I'm
a
little
bit
jealous
and
I
live
in
colorado,
which
I
think
is
pretty
beautiful
place.
So
I
like
to
said
we
are
at
an
on
ales
event,
so
please
to
tip
your
waitresses
and
drink
drink
up
heavily,
not
just
because
of
my
presentation
but
because
you
paid
for
it
so
hey
you
deserve
it.
A
A
So
what
if
there
was
another
way?
That's
that's
our
vision
at
inktank
and
the
reason
that
we've
got.
That
is
because
of
the
philosophy
and
the
design
that
has
been
baked
into
our
dna
by
our
founder
and
cto
sage.
Weil
sage
is
probably
the
most
prolific
and
committed
person
to
open
source
software
that
I've
ever
met.
In
my
life,
he
by
far
has
more
commits
than
the
rest
of
the
entire
cef
community.
Combined
he's
been
developing
ceph
since
somewhere
between
2002
and
2004.
A
It
started
out
as
a
grad
project
and
he
has
built
a
large
community
around
it.
So
while
he
is
the
primary
committer
and
the
primary
contributor
we've
seen
a
growing
uptake
in
the
amount
of
different
companies
that
have
been
supporting
ceph,
and
so
his
focus
has
not
been
to
build
ink
tank
as
a
sort
of
one
one
stop
shopping
where
everything
all
good
stuff
comes
from
there,
but
really
to
be
a
home
base.
A
A
The
open
source,
goodness
we
want,
and
if
all
that
is,
is
to
be
used
as
a
quote
to
offset
what
my
emc
guy
comes
in
and
sells
me.
So
I
can
so.
I
can
beat
him
up
on
price,
then
nobody
wins.
Our
goal
is
to
transform
the
industry
to
really
free
up
people
that,
when
they
get
a
quote
from
whether
it's
gluster
or
swift
or
ceph,
as
part
of
a
project
that
the
customer
looks
at
that
as
a
serious
implementation
and
not
just
as
some
option
to
beat
their
favorite
sales
guy
down
on
price.
A
A
A
A
Can
you
can
you
even
imagine
what
that
could
do
if
we
unleash
that
in
the
open
source
community
a
billion
dollars,
that's
how
lucrative
they
they
think
this
industry
is
they're,
not
spending
that
just
because
they
feel
some
sort
of.
Oh.
We
want
to
advance
the
cause
of
the
future
of
storage.
No,
they
want
to
make
money.
A
Now
we
think
there's
a
different
way
and
I
I
believe
that
many
of
you
here
in
this
room
are
on
board
with
us.
We
think
you
should
be
able
to
buy
whatever
kind
of
hardware
you
want
and
you
should
be
able
to
swap
it
out
anytime
that
you
want
and
yeah
there
may
be
performance
considerations
and
why
you
go
with
a
specific
vendor,
but
you
should
not
be
locked
in
and
feel
like.
Oh
my
god,
my
cluster
is
three
years
old.
Now
things
are
starting
to
really
go
tango,
uniform.
What
do
I
do?
A
A
Don't
you
guys,
like
that
name?
It's
an
open
stack,
okay,
duh!
It's
not
proprietary,
you're,
not
locked
in,
and
we
believe
that
that
ceph
is,
is
the
best
fit
for
storage.
But
again,
I'm
not
here
to
just
pump
us
up.
I
think,
as
long
as
you're
going
with
a
non-proprietary
vendor
that
that's
a
win
there
and
then
finally,
on
top
of
that
enterprise
subscription
and
see
the
big
white.
Well,
it's
a
little
tiny
white
box,
but
optional.
We
don't
think
you
should
be
forced
to
buy
support
for
your
software.
It's
open
source.
A
If
you
want
to
hack
it.
If
you
want
to
play
with
it,
if
you
want
to
get
if
you
just
want
to
get
on
a
mailing
list
and
get
support
that
way,
fine
do
it.
There
are
many
customers
that
will
not
want
to
do
that.
They'll
want
just
for
that
peace
of
mind
to
know
that
someone's
there
when
they
call
the
phone
want
to
pay
for
that.
But
we
don't
think
you
should
be
forced
to
do
that,
just
because
you're
using
the
software.
A
A
It's
all
based
upon
rados
rados.
Is
this
cute
acronym
that
sage
came
up
with
in
grad
school?
That
is
our
just
basically
explains
our
object.
Storage.
That
is
the
base
level
of
everything.
So,
whereas
some
vendors,
you
heard
john
mark
talk
earlier
about
how
they
have
a
file
based
semantic
and
that's
just
kind
of
built
into
their
dna
and
how
they're
wired
we
come
at
it
from
an
object
level,
that's
that's
kind
of
our
basis
and
that's
how
we're
hardwired
and
everything
builds
upon
that
built
upon
rados.
A
A
We
do
have
some
people
using
ext4
and
just
with
the
with
a
future
release
here
that
doesn't
show
up
that's
weird.
Well
now
the
bottom
of
my
slides
are
cut
off.
That's
going
to
be
intriguing,
so
the
little
asterisk
translates
to
the
squiggly
stuff
that
you
can
see
at
the
bottom
of
the
slide,
and
that
means
that
that's
coming
in
our
emperor
release,
which
is
coming
out
next
month,
well
yeah,
the
end
of
october,
and
so
we've
got
now
zfs
support
also
for
your
base
file
system.
A
So
we
have
this
object:
storage,
daemon
not
to
be
confused
with
the
entire
rest
of
the
storage
industry.
That
calls
an
osd
and
object
storage
device
again.
For
some
reason,
we've
got
our
favorite
little
use
of
that
acronym
and
so
osd
for
us
stands
for
daemon.
That
runs
on
top
of
this.
This
local
file
system
that's
sitting
on
a
disk,
so
you've
got
one
osd
per
device.
A
So
how
do
you?
How
do
you
interact
with
this
cluster
and
that's
what
the
cute
little
logo
is
there
between
the
human
and
the
cluster
here?
Is
that
you've
got
these
monitors?
That's
what
the
m
stand
for
the
monitors
are,
I
won't
say
it's
hard
to
say
the
brains,
but
they
are,
if
you
will
I'll
say
the
parents
and
the
grandparents
that
everybody
checks
in
with
when
they
get
home
safe
at
night
from
a
night
of
drinking,
and
they
say
I'm
here-
I'm
safe,
everything's,
okay
and
they're
like
okay
and
they
tell
each
other
hey.
A
We
have
an
odd
number
for
a
reason.
Somebody
talked
about
split
brain
earlier.
If
you
have
an
even
number
of
monitors,
they
can
fight
each
other.
I
say
the
answer
is
this:
I
see
the
answer
is
that,
well
you
have
an
odd
number
they're
using
a
paxos
decision
making,
and
so
that's
how
they
decide.
Okay,
what's
the
right
answer,
I
say
is
up,
I
say
is
down
what
do
you
say
I
say
sub?
A
Okay,
we
all
agree
now
the
osds,
as
you
saw
at
a
bunch
on
the
slide,
but
you
can
have
tens
to
tens
of
thousands
all
in
a
single
cluster.
The
monitors
we
recommend
a
low
number
three
is
is
optimal.
I
would
suggest
in
some
maybe
edge
cases
you
might
want
to
consider
five
but
again
you're
increasing
the
communication
because
they
all
have
to
talk
every
time,
something
changes.
So
if
an
osd
goes
down
and
they
all
want
to
vote
again
to
say,
is
it
really
down?
Is
it
up?
A
A
A
A
And
you
can
use
your
favorite
language,
but
that
being
said,
we
we
know
that
some
people
want
that
swift,
s3
type
of
compatibility.
They
don't
want
to
do
all
the
hard
work
of
having
to
play
with
a
library.
Just
give
me
something
that
I
can
shove
my
data
in
and
import
it
that
responds
something
like
swift
or
like
amazon.
A
A
A
Now,
here's
a
specific
example
from
our
former
parent
company
dreamhost
and
what
they've
got
here
is
they
put
a
couple
of
load
balancers
out
there
in
front
of
their
gateways,
so
they've
got
four
gateways:
that's
a
cute
little
icon.
We
we
created
with
the
arrows,
going
both
ways
and
they
just
have
software
load,
balancers
handling
the
load
for
ingest,
and
then
it
goes
to
the
monitor.
A
A
A
A
A
A
A
Here
you
can
have
the
host
that
just
executes
that
locally
and
you
and
uses
that
kernel
module
same
access
to
the
object
store,
except
instead
of
coming
in
via
the
user
space
library,
lib
rbd
via
librados
here,
using
the
kernel
module.
Now
there
are
little
there
are
some
functions
that
are
only
in
user
space
that
haven't
been
backported
the
kernel.
A
Yet
just
because
the
kernel
development
cycle,
we
don't
control
and
obviously
linus
puts
those
out
whenever
he
feels
like
it
and
so
there's
a
little
bit
of
a
backlog
between
getting
our
features
into
the
kernel,
as
opposed
to
our
own
library
that
we
can
just
crank
out
whenever
we
want
so
again.
What
is
this
rbd
thing?
A
And,
to
be
honest,
that's
the
reason
I
didn't
stand
up
here
and
you
know
critique
the
swift
guys
about
how's
their
long-term
data
integrity,
and
I
didn't
ask
john
mark
how
what
about
the
death
of
posix?
You
know,
because
those
who
live
in
class
houses
should
not
cast
stones
and
I'll
be
the
first
one
to
tell
you
that
we
we're
fully
aware
our
file
system
is
not
production,
ready,
that's
not
to
say
it's
not
going
to
work
for
you.
A
We
have
users
using
it
production
right
now,
kind
of
scares
a
hell
out
of
me,
but
you
know
if
they
want
to
do
that.
That's
fine
and
really
it's
just
a
matter
of
the
fact
that
we
have
not
been
able
to
put
the
testing
resources
against
it
to
where
we
feel
that
we're
ready
to
stand
behind
it
as
a
supported
product.
A
A
One
of
the
examples
that
we
often
use
is
where
in
the
hell
did
I
leave
my
car
keys
today
and
one
of
the
things
that
my
wife
and
I
have
done
is
we
have
a
little
bowl
right
by
the
back
door
which,
unless
our
kids
start
messing
with
other
keys,
which
then
you
never
know
where
they're
going
to
be,
but
that's
where
we
typically
put
them.
So
the
question
is
well:
how
do
you
find
out
where
your
car
keys
are
now?
A
It
could
be
that
maybe
you
always
put
in
the
same
spot,
so
you
know
that
oh
there
you
are,
I
always
put
my
data
right
there.
Well,
that's
gonna
fill
up
pretty
quickly.
I
don't
have
a
never
ending
supply.
You
know
of
my
my
bowl
that
sits
by
my
back
door
can
only
handle
so
many
sets
of
keys
before
it
overflows.
A
So
pretty
soon.
You
have
to
come
up
with
another
another
idea,
so
this
is
called
the
deer
diary.
Today.
I
put
my
keys
here
and
that's
if
you
have
many
many
many
keys,
okay
and
that's
kind
of
a
typical
metadata
server,
you
look
up
and
you
say:
okay,
I've
got
an
f.
It
goes
here,
my
that
goes
into
this
compute
node.
With
the
storage
there
you
go.
A
Now
this
is
a
little
bit
different
to
my
scenario,
but
you
get
the
idea
if
you
always
hang
them
on
a
hook
by
the
door.
But
the
big
question
is
again:
that's
not
a
that's,
not
sustainable.
When
we're
talking
about
systems
that
that
are
operating
at
the
petabyte
and
in
you
know
the
next
five
to
ten
years,
exabyte
era.
A
And
the
answer
the
cute
lego
man
says,
is
crush,
which
is
another
fun
acronym
for
our
hashing
algorithm.
So
you
got
an
object
that
comes
in
here.
It
says:
hey
crush.
I
got
to
play
some
data
and
based
upon
the
rules
that
you've
given
it.
It
sticks
it
there.
So
there's
no
sort
of
sort
of
look
up.
It's
a
pure
calculation
based
upon
the
rules
that
you've
set
in
this
case
we've
done
just
one
replication,
so
you've
got
the
original
and
one
backup.
A
We've
created
some
rough
rules
that
say
and,
however,
you
want
to
define
it
either.
The
top
row
is
is
maybe
one
row
in
the
data
center
and
the
second
row
is
another
row
in
the
data
center
and
then
maybe
we've
got
different
bus
rows
going
up
and
down
so
we've
said
for
our
data.
We
want
it
shaped
like
this
and
that's
how
it
replicates
it.
A
But
so
that's
just
one
example.
Well
now,
here
we've
followed
that,
through
with
all
the
different
colors,
so
you
can
see
how
it
easily
gets
to
be
complex
with
satisfying
all
the
rules,
but
the
thing
is
that
it
doesn't
have
to
look
up
anywhere.
It
just
calculates
all
the
time.
It's
pseudo-random,
it's
deterministic,
so
it's
always
going
to
get
the
same
answer
as
long
as
you
give
it
the
same
inputs.
A
A
A
A
Well,
we
know
that
our
rules
say
we
always
have
to
have
two
copies
of
everything
and
we
know
that
they
can't
be
in
the
same
row.
So
what
do
we
do
all
right?
Let's
move.
Let's
make
some
copies
so
again,
totally
no
intervention
from
you.
It
just
does
it
so
it
ensures
the
replication
rules
that
you've
set,
and
this
is
why
we
say
that
it's
self-healing.
A
A
A
A
Is
talking
to
hypervisors
and
vms,
so
typically,
we
don't
just
have
one
vm
again
talking
to
one
block
device.
We
have
hundreds
and
hundreds
of
these.
So
how
do
you
spin
up
hundreds
and
hundreds
of
vms
quickly
and
efficiently
well,
with
rbd
you've
got
instant
copy?
What
does
that
mean?
Okay,
you
look
here.
You've
got
your
one
stripe.
A
A
A
A
Okay,
that's
that's
where
we
are
now,
but
what's
coming
up
what's
on
the
road
map,
these
are
kind
of
the
things
that
that
get
us
a
little
excited
is
gateway.
Disaster
recovery
is
coming
up
in
our
in
our
next
release
and
that
will
allow
you
not
just
the
multi-site
single
name
space
that
we've
had
in
our
current
dumpling
release,
but
this
will
allow
you
to
do
a
full
metadata
and
a
data.
Sync
of
all
your
data
from
multi-regions.
A
In
a
gateway
additionally,
now
these
are
really
there's
so
much
baked
into
these
these
next
two
that
I
made
another
couple
slides
for
erasure
coding,
but
caching
and
tearing
I
mean
that's
huge,
we're
talking
about
being
able
to
pool
your
cold
dead.
Nobody
ever
touches
it
data
onto
who
cares
what
your
cheapest
commodity.
You
want
and
then
pools
of
fast
ssds
that
you
can
put
the
stuff
that
everybody's
trying
to
get
that
they
really
are
bummed
if
it
doesn't
get
to
them
very
very
quickly.
A
So
you
can
set
up
those
pools
not
only
as
individual
caches,
but
you
can
have
a
front
end
so
that
it
moves
stuff
back
and
forth
and
we'll
have
apis
that
will
be
exposed
so
that
if
anybody
wants
to
set
up
their
own
tools
to
control
that
migration,
from
hot
storage
to
cold
storage
and
back
again,
they
can
do
that
now.
Erasure
coating
is
really
exciting,
because
right
now
one
of
the
biggest
knocks
I'd
say
fairly
on
ceph.
Is
that
well
yeah?
A
Not
only
that,
but
this
only
costs
you
about
1.4
1.5,
maybe
times
the
amount
of
storage
as
opposed
to
three
times.
So
you
get
better
data
integrity,
cheaper,
cost.
Well,
you
know
you
can't
have
everything
so
there's
got
to
be:
what's
the
catch,
the
catch
is
obviously
it
costs.
You
a
lot
more
of
processing
and
memory.
A
A
We're
we've
been
kind
of
negligent
in
the
quotas
area
and
people
have
been
wanting
it
for
a
while
and
so
in
the
emperor.
We're
not
sure
if
it'll
make
it
into
emperor,
but
it'll
definitely
be
in
by
by
firefly,
will
be
the
bucket
and
user
quotas,
so
you'll
be
able
to
set
that
so
that
you
won't
just
have
somebody
that
okay
you're
only
if
you're
a
hosting
company
and
they're
only
paying
you
for
10
terabytes
and
they
just
start
blowing
up
their
favorite
torrents
spewing
into
their
cloud
account.
A
A
Additionally,
in
the
ice
house
time
frame,
we
really
want
to
focus
on
enhanced
security
testing
right
now.
There's
nothing
really
stopping
you
from
breaking
out
a
hypervisor
and
kind
of
running
amok
and
whoever
storage
you
want.
I
think
that's
the
case
with
just
about
everybody's
storage.
We
really
want
to
make
sure
that
that
you
are
locked
down
to
the
storage.
That's
allocated
to
your
area,
if
you
happen
to
find
a
way
to
hack
out.
A
A
It
seems
like
we,
we
really
just
kind
of
copied
amazon
may
for
good
reason,
but
in
the
openstack
community
we
said
the
thermal
storage
block
storage,
just
never
the
the
twin
shall
meet
and
we've
created
only
almost
our
own
split
brain
we've
got
cinder
handling
some
stores.
We
call
it
the
storage
product,
but
then
we've
got
nova
handling.
A
Honestly,
I
don't
think
noah
should
care,
it's
just
storage
and
if
it's
storage
cinder
should
handle
it,
that's
just
me,
but
I'll,
throw
that
out
and
see
what
you
guys
think
so
we're
kind
of
running
out
of
time
again.
This
is,
I
feel,
like
I'm
preaching
to
the
choir,
but
for
any
of
you
that
are
still
agnostic-
and
you
know-
maybe
you
you
really
like
getting
your
emc
rep
a
new
car
every
three
years.
A
A
But
I
want
to
invite
you
to
get
involved
in
stuff,
so
here
are
some
ways
that
you
can
quickly
get
a
cluster
up
and
running.
We've
even
got
one
of
my
new
guys
set
up
some
ansible
playbooks
you've
got
juju
to
play
with
it's
all
up.
There
go
to
the
links
and
you
can
get
up
and
running
quickly.
I'm
kind
of
envious
of
john
mark's
four
commands
and
that's
gonna
be
a
goal.
I'm
gonna
have
to
shoot
for
now,
but
we're
getting
there
so
and
you
know
we're
an
open
source
project.