►
From YouTube: CDS G/H (Day 1) - Calamari Intro
Description
https://wiki.ceph.com/Planning/CDS/CDS_Giant_and_Hammer_(Jun_2014)
24 June 2014
Ceph Developer Summit G/H
Day 1
Calamari intro session
A
All
right
looks
like
we've
got
a
pair
of
calamari
talks
now
brought
to
you
by
John
spray
and
Gregory
me
know:
I
guess
the
first
one
is
going
to
be
an
introduction
to
calamari
and
we'll
wander
through
and
hopefully
get
a
little
bit
of
a
demo
and
I'll
look
at
how
it
works
and
how
all
the
moving
pieces
fit
together
and
and
maybe
a
little
bit
of
what's
going
into
what
goes
into
deploying
it.
So
if
you
guys
want
to
take
that
away.
B
B
I'll
just
put
that
I
see,
so
that's
that's
some
sort
of
development
topics.
I
do
actually
have
a
little
overview
slide
here
that
so
I'll
just
down
my
screen
them
it'll.
Allow
hopefully
give
us
the
terminology
for
what
follows
that
coming
through.
Ok,
ya
go
so
from
top
to
bottom
calamari.
Clients
is
the
JavaScript
user
interface,
but
that
runs
in
your
browser
and
talks
to
a
calamari
server
over
HTTP.
But
I
was
talking
to
the
rest
interface
Apache
acts
as
the
front
end
to
everything
running
on
the
calamari
server.
B
B
Calamari
rest
is
the
python
module
that
actually
implements
the
calamari
rest
interface
and
calamari
web
is
another
python
module
which
is
just
responsible
for
serving
the
static
files
for
camry
clients
am
behind
calamari
rest.
We
have
the
only
component
of
this
that
has
a
truly
silly
name,
which
is
Lulu
and
that's
the
sort
of
business
logic
and
and
that's
root
for
doing
things
like
running
remote
operations
and
checking
their
progress
and
that
sits
on
top
of
Dalton
master.
B
It
was
chosen
for
calamari
because
it
has
particularly
nice
integration
with.
You
should
go,
see
ms
messaging
bus
and
so
we're
not
using
it
in
the
same
way
that
they
used
to
with
views
like
chef
recipes
of
the
system,
we're
using
it
as
an
SMS.
Primarily
now
that
component
told
to
remotely
to
its
counterpart
insulting
megan,
which
runs
out
on
the
servers,
is
essentially
the
agent
for
Halle
Berry
and
we
have
a
module
for
specific
to
SEF
the
runs
inside
the
minion
and
that
talks
to
the
modern
cluster
and
also
talks
directly
to
the
services.
B
C
A
couple
couple
of
quick
questions,
mm-hmm
or
here
so
on
the
whole
graphite
component
and
wis
wis,
pretty
or
whatever,
where
all
the
stats
are
being
aggregated,
that
it's
currently
all
intended
that
that
all
lives
on
one
on
one
host.
But
you
have
all
the
all
of
the
real-time
information
Reuters
being
fed
back
to
11
data
repository
that
I
guess
carbon
is
acting.
Is
the
the
warehouse
for
that.
B
B
Not
actually
sure,
if
that
happens
at
the
diamond
blow
or
or
if
oh
right
yeah,
so
you
need
you
need
something
that
speaks
the
pros
called
so
so,
some
of
the
time
series
databases
also
accept
the
graphite
protocol.
It's
sort
of
a
primitive
lingua
franca
type
thing
it,
but
that
that
part's,
like
super
simple,
it's
just
a
dict
of
statistics.
So
if
we
needed
to
switch
fatto
everything
else,
it
wouldn't
be
a
big
deal.
Okay,.
B
Will
there
a
couple
kind
of
frequently
asked
questions
here?
One
is:
how
does
this
relate
to
surf
rest
api
and
does
it
replace
it?
Is
it
deprecated
that
kind
of
thing?
Clearly
there
is
a
bunch
of
overlap,
but
at
the
moment
we're
not,
you
know
we're
not
saying
one
has
to
replace
the
other
necessarily.
The
key
difference
is
that
surest
API
runs
directly
on
a
mon
and
it
exposes
the
exact
same
command
set
as
Seth
command-line
interface.
B
If
you
don't
want
to
have
a
separate,
calamari
server,
if
you
want
to
run
it
directly
on
a
mon
mon,
then
the
codes
pretty
flexible
and
you
can
actually
run
it
without
a
database.
You
can
run
it
without
graphite.
You
can
run
it.
You
can
even
run
it
without
salt
just
about
on
my
development
branch,
those
we're
not
married
to
salt.
So
that's
another
frequently
asked
question:
can
I
run
it
with
puppet?
A
A
B
It
would
also
be
great
to
get
it
built
as
part
of
fedora
that's
on
stage,
but
the
idea
would
be
that
we
would
get
all
of
these
sort
of
myriad
Python
dependencies.
We
need
packaged
for
us
by
the
distros,
although
possibly
not
backported,
necessarily
to
the
long-term
support
distros.
But
that's
the
hard
part.
B
Both
help
with
packaging
and
testing
calamari,
and
also
help
with
from
upstream
distro,
maintain
errs
to
get
the
dependencies
of
calamari
available
in
the
distro
so
that
we
can
streamline
the
packaging,
because
currently
the
packaging
of
calamari
server
is
rather
monolithic.
It
builds
a
Python
virtual
and
with
a
lot
of
the
dependencies
inside
it
that
was
done
in
order
that
we
could
pick
it
up
and
run
it
on
all
distributions
that
don't
have
the
more
recent
dependencies.
B
But
when
you
package
something
that
way,
it's
not
going
to
pass
muster
with
you
know,
package
maintained
as
from
live
stream
distributions
right
yeah.
They
won't
let
a
package
like
that
into
a
distribution
because
it
just
breaks
all
the
rules,
though
there's
a
there's
kind
of
a
need
to
meet
in
the
middle,
where
you
know
our
packaging
needs
to
be
cleaner
and
many
distros.
We
want
to
run
on
need
to
have
more
recent
dependencies
for
us
as
well
and
as
ads
for
the
actual
plan.
B
And
yeah
it
has
been
quite
quite
noticeable,
especially
on
the
mailing
list,
but
some
you
know
we
have
people
coming
and
sort
of
looking
for
packages
to
download,
so
they
can
just
go
ahead
and
install
it,
and
at
the
moment
you
you
can
install
it,
but
you
have
to
build
your
own
packages
and
that
was
a
result
of
our
decision
to
open
source
as
soon
as
possible.
But
what
that
meant
was
that
at
the
point
where
we
open
sourced
it
it
just
men,
you've
got
the
sauce,
it
didn't.
B
C
A
B
A
A
D
A
C
B
I
couldn't
tell
you
my
thing:
they
publish
it
pretty
shortly
after
they
heard
them.
Yeah.
B
B
You
get
a
lot
of
nice
detail
from
CLI,
but
it's
not
so
easy
to
make
sense
of
at
a
glance,
especially
if
you're
not
an
expert
on
Seth,
the
equivalent
view
in
calamari
the
front
page
dashboard
gives
you
likely
less
information,
but
hopefully
in
a
much
clearer
way.
So
it's
giving
you
a
series
of
headlines
about
the
health
of
your
system,
the
bit
of
rundown
of
numbers
of
things
which
are
ok,
and
this
neat
little
colored
block
in
the
middle
is
showing
you
the
status
of
your
pgs.
B
So
if
you
have
things
which
are
in
a
bad
state,
you
can
see
little
sort
of
red
bashes
going
through
that
we're
we're
also
exposing
graphite
stuff
in
our
own
user
interface,
though
you
can
go
directly
to
the
graphite
dashboard
that
it
comes
with.
We've
had
a
suggestion
of
bundling
graph
on
there,
which
is
an
alternative
dashboard.
That's
pretty
interesting,
but
for
the
moment
we
have
this
sort
of
fairly
simple
way
of
selecting
network
disk
darts
on
a
per
host
basis
and
they're
also
/
pool
darts
in
here
as
well.
B
So
you
can
see
the
I/o
and
usage
on
on
a
per
cool
basis
or
across
the
whole
cluster.
Another
command
line,
interface
example
for
comparison.
This
is
your
list
of
OS
DS,
as
you
get
them
from
Seth
in
the
command
line
and
the
equivalent
in
calamari.
Is
this
OSD
workbench
view
where
we're
showing
a
list
of
colored
dots
depending
on
the
status
of
an
OSD?
And
this
is
a
filterable
view.
B
The
management
aspect
of
the
user
interface,
as
opposed
to
the
monitoring
aspect,
includes
a
view
of
what
hosts
you
have
in
the
system
and
if
I
was
adding
some
new
hosts,
they
would
be
some
user
interface
to
go
with
that.
You
can
also
get
some
detail
about.
The
hosts
this
stuff
is
actually
getting
passed
through
be
much
verbatim
from
salt,
though
one
of
the
things
salt
gives
us
is
a
whole
bunch
of
metadata
about
the
host
server
running
on,
and
we
pass
that
on
up
through
calamari
into
user
interface
as
well.
B
The
ceph
config
is
visible
in
the
user
interface.
This
is
an
example
of
something
that,
in
the
long
run,
it
would
be
great
to
make
this
right
of
all
as
well
as
readable
and
but
for
the
moment,
we're
just
sort
of
providing
what
we
can
in
the
user
interface
in
terms
of
stuff.
We
can
just
read
straight
out
the
South
cluster
and
put
on
the
screen
global
OSD
settings
and
there's
an
another
view
of
OSTs
where
they
listed
on
a
per
host
basis.
B
So
you'd
have
another
service
for
our
BD.
You
have
a
service
for
argue
w
and
there's
sort
of
a
little
bit
analogous
to
the
the
monitor
classes
that
you
have
within
the
SEF
mon
that
you
have
/
subsystem
one
of
these
services
and
they
they
expose
RPG
up
and
salt
down
you.
So
you
can
edit
pools
and
there's
a
little
bit
of
smart,
auto
choosing
for
them
a
replacement
groups
in
the
pool
that
goes
on
here
as
well.
In
the
user
interface
and
the
the
SEF
dash
WQ
is
visible
in
the
user
interface
as
well.
B
That's
example
of
something
that's
kind
of
low
level.
We
don't
really
want
it
to
be
front
and
center,
but
it's
no
problem
to
just
expose
that
verbatim.
It's
free,
easy
thing
to
add.
B
So
the
rest
of
this
deck
is
waxing
lyrical
about
how
wonderful
rest
api
is,
but
really
it's
just
a
kind
of
an
example
of
what
we
mean
when
we
say
REST
API,
that
you
have
the
address
of
a
resource
at
the
top,
where
we're
referring
to
something
by
the
cluster
SSID
and
the
object
type.
So
it's
the
big
long,
uuid
/
OSD
or
give
us
a
list
of
OS
DS,
and
most
of
this
data
is
similar
to
or
directly
taken
from
the
content
of
the
OSD
map.
B
But
we
also
have
the
opportunity
to
be
a
little
bit
cleverer
and
fuse
information
from
multiple
sources
together.
So,
for
example,
if
you
go
and
look
at
the
servers
in
the
calamari
REST
API,
they
will
not
only
tell
you
what
services
are
running
on
a
given
server,
but
they'll
go
look
at
the
front-end
back-end
OSD
addresses
resolve
that
those
two
network
interface
names
and
tell
you
what
the
front
end
and
back
up
back
in
network
interfaces
are
on
a
server
and
what
that
lets
you
do
is
go
look
up.
B
The
statistics
for
those
network
interfaces
from
graphite,
and
the
upshot
of
that
is
that
you
can
have
a
nice.
You
can
very
easily
using
that
API
write
a
nice
user
interface
page
that
shows
you
front-end
back-end
network
bandwidth
per
per
host,
but
I
have
a
piece
of
example:
code
for
building
these
are
interface
like
that,
but
I'm,
keeping
share
or
ugly.
That's
in
so
the
camera
API.
B
The
idea
is
that
if
you're
a
system
integrator-
and
you
have
an
external
system
that
wants
to
provision
some
some
logical
storage
within
SEF
and
then
start
using
it,
you
want
that
higher
level
view
of
not
not
just
telling
the
user.
Hey
I
updated
the
OSD
map
for
you,
but
telling
them
your
storage
is
ready
to
use.
So
the
idea
is
that
both
views
are
useful
right.
The
low-level
view
and
the
high
level
view
and
calamari
leans
toward
the
high
level
view
and
the
API
has
its
own
documentation.
B
This
diagram
and
the
technology
is
used
in
calamari
are
principally
a
Python
back
end
and
a
JavaScript
front
end
and
on
the
back
end,
we're
using
saltstack
graphite.
The
diamond
statistics.
Asian
and
zeromq
is
used
pretty
heavily,
so
0
and
Q
is
what
sits
underneath
salt,
and
we
are
also
using
a
library
called
zero
RPC,
which
is
based
on
0
mq
for
RPC
between
services.
In
the
Python
side
of
things,
we
have
a
postgresql
database,
but
it's
a
key
design
choice
that
it
is
not
in
line
with
any
business
logic
in
the
system.
B
So
you
can
actually
switch
the
database
off
entirely
and
calamari
continues
to
work.
It
just
has
to
redirect
your
cluster
the
next
time
that
you
restart
server.
That's
worth
noting
as
well.
Actually
that
you
don't
tell
calamari,
add
this
AF
server
or
a
sorry,
you
don't
tell
add
this
seth
cluster
or
at
this
service.
You
just
tell
it
about
your
servers
and
you
install
the
software
on
the
servers
and,
from
that
point
onwards,
calamari
will
detect
your
clusters
and
start
talking
to
them
and
detect
your
services.
B
B
If
you
want
to
have
additional
remote
execution
capability,
you
can
write
an
additional
salt
module
and
then,
once
you
have
that
code
out
there
on
the
system,
you
can
write
these
new
small
Python
services
within
the
calamari
server,
which
call
out
using
the
salt
message
bus
to
your
remote
code
and
then
expose
that
up
into
the
rest,
api.
The
REST
API
has
a
very
natural
means
of
extension
that
everything
is
a
URL
namespace.
B
B
You
would
go
ahead
and
implement
that
as
a
separate,
so
you
could,
for
the
most
part,
talk
to
your
own
bits
of
the
rest
api
in
your
own
app
and
not
worry
too
much
about
fitting
in
with
the
rest
of
the
application.
If
you
didn't
want
to
or
need
to
there's
a
couple
more
slides
here,
but
they're
actually
one
of
these
is
about
today
and
where
we
are
right
now.
B
Yeah
I
mean
it's
doing:
role
based
access
control
systems.
The
user
interfaces
is
something
that
is
often
asked
for,
but
it's
relatively
far
outside
the
sort
of
core
function
of
what
calamari
is
for
you
know.
Calamari
is
not
about
managing
complicated
systems
of
user
accounts.
It's
it's
really
about
dealing
directly
with
staff.
B
So
there
is
a
lot
of
interest
in
plugging
calamari
into
other
Access
Control
regimes.
So,
for
example,
if
you
have
an
ldap
server,
plugging
calamari
into
that-
and
it
might
be
that
within
that
you
have
multiple
groups
and
roles
and
that
Camry
could
respect
those,
and
so
we've
we've
done
a
little
bit
of
thinking
about
that.
But
it's
pretty
early
days
there,
because
calamari
uses
the
Django
authentication
system,
which
is
itself
pluggable.
B
If
you
had
a
system
that
you
wanted
to
plug
into
and
a
Django
plug-in
existed
for,
that,
you
would
be
able
to
do
a
custom,
calamari
deployment
that
did
that.
But
as
for
the
question
of
confirmation,
when
you
delete
pools,
I
mean
at
the
moment
the
the
user
interface
will
ask
you
to
confirm.
You
know
it'll,
ask
you
to
click
yes,
I!
Yes,
I
really
mean
it.
I
think
we.
There
are
some
suggestions
here
about
putting
text
boxes
in
that
you
have
to
type
something
into
I.
Think
that's
probably
quite
a
good
idea.