►
From YouTube: CDS G/H (Day 1) - Calamari Development
Description
https://wiki.ceph.com/Planning/CDS/CDS_Giant_and_Hammer_(Jun_2014)
24 June 2014
Ceph Developer Summit G/H
Day 1
Calamari development discussion
A
All
right
and
we're
back
with
more
calamari
discussions,
so
this
will
be
the
calamari
development
discussion,
talk
about
some
of
the
moving
pieces
and
kind
of
where
it's
headed
and
the
road
map.
So
john
greger,
you
guys
want
to
take
it
away.
B
So
there
are,
there
are
two
pieces
here:
there's
gregory's
got
a
blueprint
for
adding
crush
management,
which
is
a
high
priority
feature
for
next
release,
and
I
have
a
probably
less
exciting
list
of
things
that
we
need
to
look
at
when
it
comes
to
packaging
and
dealing
with
upstream
dependencies
where
we
currently
have
branches,
and
that
kind
of
thing.
So
do
you
want
to.
C
A
D
D
That's
weird
like
on
the
phone:
it
wasn't
working
anyways
all
right.
Let
me
try
for
the
third
time
third
time's
the
term
so
in
the
agenda,
there's
a
blueprint
for
adding
crush
management
to
the
calimary
rest
api
and,
as
john
said,
this
is
fairly
urgent.
Work.
Firefly
has
landed
for
steph,
and
that
gives
us
the
ability
to
do
erasure,
coded
pools
and
cache
tiering
and
in
order
to
use
those
features,
preferably
and
samely.
D
D
Get
the
pieces
that
comprise
a
crush
map
and
do
basic
crud
to
it.
So
there's
three
points.
The
recommendation
really,
which
is
the
crush
maps,
made
up
of
something
nodes
and
rules.
So
the
node
is
basically
a
way
that
you're
drawing
the
infrastructure
that
describes
how
you've
created
your
storage
cluster
so
that
you
can
describe
what
rack.
What
data
center
et
cetera
your
osds
live
in
and
the
crush
is
a
an
algorithm
that
helps
you
kind
of
partition.
D
Your
failure
domains
in
such
a
way
that
you
have
good
durability
or
you
have
certain
pools
that
may
live
on
osds
so
that
they
can.
You
know,
accomplish
the
things
that
you
want
with
say:
a
cash
tier
or
an
eraser
coded
pool,
for
example,
something
fast
or
something
slow
like
cold
storage,
and
so
we're
just
trying
to
put
this
into
the
calamari
api
so
that
we
can
start
making
steps
toward
getting
that.
D
D
So
there's
a
number
of
view
sets
that
are
named
and
that's
where
I
expect
the
work
to
happen
for
this
and
there's
that
and
then
another
piece
kind
of
an
enhancement
to
calamari
is
going
to
be
to
resolve
osds
and
hosts
osds
and
servers
to
crush
nodes,
so
that
when
you
ask
for
those
resources
in
the
api,
it
helps
you
understand
where
they
live
in
the
crush
map.
So
that's
an
enhancement
there
and
then.
B
So
I
think
the
the
seth.com
manipulation
might
be
one
of
them
hairier
parts
of
that,
because
people
will
have
their
copy,
that
they've
deployed
with
seth
deploy
when
we
go
and
edit
local
ones,
maybe
they'll
get
overwritten
next
time.
Somebody
calls
intercept
deploy
that
kind
of
thing,
so
I
don't
know
if
we,
if
we're
going
to
want
to
come
up
with
something
I
don't
know
like
it
would
actually
be
really
useful
to
have
like
a
kind
of
conf
d
thing.
B
That
would
let
us
pop
things
in
there
without
having
them
overwritten,
but
I
don't
know
that
might
not
be
necessary.
Yeah.
B
Okay,
so
on
the
ether
pad,
there
are
quite
a
few
points
there,
but
I
think
they're,
all
pretty
quick,
so
I'll,
just
whip
through
them,
some
sort
of
general
reminders
for
anyone
who
is
getting
involved
with
development.
This
is
stuff
which
is
hopefully
expressed
in
the
readme
and
has
been
on
the
mailing
list,
but
it
doesn't
hurt
to
reinforce
it.
So
there
are
two
git
repositories:
one's
called
calamari
and
one's
called
calamari
clients.
Calamari
is
the
back
end
and
calamari.
Clients
is
the
front
end.
B
So
if
you
build
and
install
calamari,
then
you
don't
get
a
user
interface
until
you've
built
and
installed
calamari
clients
as
well.
The
documentation
camry
is
at
calimory.readthedocs.org.
B
There
is
quite
a
lot
of
it
and
if
you
have
issues
with
it
or
you
want
to
add
to
it,
then
it's
all
right
there
and
get
so
just
like
with
the
def
links
documentation,
you
can
send
pull
requests
to
update
documentation.
B
There's
there's
been
a
little
bit
of
confusion
about
the
steps
involved
in
setting
up
a
development
environment,
because
the
documentation
includes
a
a
sort
of
manual
way
of
doing
it.
There
is
also
a
non-manual
way
of
doing
it,
which
is
much
much
quicker
using
vagrant.
So
vagrant
is
a
tool
that
lets
you
set
up
a
sort
of
canned
virtual
machine
with
certain
environment
in
it.
I
really
want
to
steer
people
toward
the
vagrant
option
unless
you
have
a
solid
reason
for
wanting
to
do
it
by
hand.
C
For
building
the
packages
as
well
they're,
it's
certainly
possible
to
go,
build
the
packages
by
hand
and
and
also
to
examine
what
the
vagrants
do
to
build,
both
the
calamari
and
culinary
clients
packages.
But
please
try
the
vagrants
first
and
only
go
to
manual.
If
there's
no
other
option.
B
Yes,
that
a
thousand
times
over
so
other
housekeeping
stuff,
so
we
we
have
a
little
bit
of
code
which
uses
private
interfaces
of
salt.
So
this
is
something
that
python
kind
of
lets.
You
do,
it's
clearly
not
encouraged,
and
it
means
that
at
the
moment,
calamari
is
pretty
sensitive
to
the
exact
version
of
salt
you're
doing
you're
using.
So
we
need
to
clean
that
up,
and
that
is
probably
going
to
mean
sending
some
stuff
upstream
into
salt,
to
expand
the
interface
to
include
everything
that
we
need
for
it.
B
I
was
just
checking
on
that
earlier
today
and
it
looks
like
they
have
a
new
module
interface
called
net
api
that
lets
you
add
your
own
api
to
the
salt
master
with
the
interfaces
you
want,
so
that
that's
going
to
need
doing
otherwise
in
a
few
versions
of
salt,
we're
probably
going
to
see
stuff
start
breaking.
B
We
currently
have
a
branch
of
diamond
with
the
latest
dangers
for
the
theft
collector
in
that's
that
there's
absolutely
no
reason
for
that
not
to
go
upstream
other
than
that.
We
probably
want
to
update
some
unit
tests
around
that.
Before
doing
it.
Absolutely
anybody
could
could
pick
up
that
task
if
anyone's
looking
for
a
way
to
help
out,
and
we
currently
have
a
branch
of
graphite
as
well.
There
are
two
reasons
for
that:
graphite
branch.
B
One
is
that
there
were
bugs
in
graphite
that
we
needed
to
fix
for
calamari.
Those
bug
fixes
have
gone
upstream.
So
that's
okay,
but
there
was
also
a
custom.
B
Sorry
about
that,
so
the
other
thing
with
the
graphite
branch
is
that
we
added
a
custom
format
for
the
for
our
user
interface.
So
when
you
query
statistics,
it
was
giving
them
in
a
format
that
we
didn't
really
like.
We
added
a
custom
one
we
sent
it
upstream
and
they
didn't
like
it.
Unfortunately,
they
were
basically
saying
that
they
have
enough
formats
and
they
don't
want
any
more.
B
So
we
probably
need
to
refactor
that
code
to
load
the
data
out
of
graphite
in
their
native
format
and
then
do
that
conversion
inside
calamari,
and
at
that
point
we
should
be
good
to
then
use
upstream
graphite.
Although
the
graphite
project
hasn't
done
releases
in
quite
a
long
time,
so
we'd
probably
still
be
using
a
git
hash
of
graphite,
rather
than
an
official
version
of
graphite,
which
is
yet
another
reason
to
switch
to
a
better
time
series
database.
B
So
those
things
feed
into
the
the
possibility
of
doing
nicer,
cleaner
packaging
and
getting
upstream
distros
involved.
So
the
point
where
we
have,
when
we
don't
have
all
these
dodgy
branches
of
things,
then
we
would
look
to
have
a
glimmed
down
calamari
server
package,
which
didn't
have
this
monolithic
embedded
virtual
env,
with
all
the
python
dependencies
packaged
separately
as
part
of
the
upstream
distribution.
B
Currently,
we
sort
of
take
over
the
salt
master
on
the
machine
that
we
install
on
and
for
clean
packaging.
We'd
want
to
have
a
cleaner
way
of
doing
that,
I'm
not
sure
exactly
what
it
is,
but
there's
a
thing
that
needs
doing
there.
B
Currently
we
have
some
post
installation
configuration
tasks
such
as
setting
up
the
database
and
connect
and
connecting
things
together,
which
we
do
from
a
tool
called
calamari,
ctl
and
distributions
will
have
policies
about
whether
it's
appropriate
to
do
that
in
the
script
or
whether
you
should
be
doing
it
in
package
scripts
and
the
ui
packaging
could
use
some
simplification.
Currently,
you
require
a
fairly
heavyweight
javascript
stack
in
order
to
build
the
user
interface.
B
So,
in
order
to
make
packaging
easier
for
for
especially
upstream
packages,
we
might
want
to
look
at
producing
a
compiled.gz
that
they
would
treat
as
static
files
and
then
put
in
a
package
for
us,
and
these.
These
are
all
things
that
it
would
be
great
if
anyone
who
is
interested
in
seeing
calamari
packaged
for
their
favorite
distribution
could
get
involved
with
that
that
would
that
would
be
fantastic.
D
I
have
for
that
list.
Let's
not
forget
the
one
thing.
Other
thing
that
we
were
talking
about
the
other
day,
which
is
there,
is
kind
of
a
vestigial
piece,
do
sulu,
which
is
called
the
plugin
manager
before
calamari,
was
open
sourced.
That
was
a
way
that
we
had
developed
for
third
parties
to
add
functionality
to
calamari,
but
at
the
moment
no
one's
using
it
and
we
think
that's
probably
going
to
add
some
confusion.
D
D
Absolutely
so
I
just
wanted
to
kind
of
throw
that
out
there
to
allay
any
confusion
for
the
people
participating
today.
You
should
know
plugin
manager
is
not
used
at
the
moment
and
that's
not
the
right
way
to
integrate
with
it.
Unless
you
have
a
really
good
reason,
we're
probably
going
to
take
it
out
in
the
short
term,
because
we
don't
have
a
really
good
reason.
A
So
what
have
we
done?
Well,
I
was
going
to
say
what
kind
of
work
or
or
or
or
appetite
are
you
guys
having
for
bringing
in
some
of
the
other
stuff
that
was
in
the
community?
Like
I
just
pasted
two
into
the
chats
here,
there
was
dashing
ceph
and
sef
dash
that
were
both
community
started
projects
during
the
window
in
which
calamari
was
closed
source,
so
I
don't
know
like
are
we?
Are
we
looking
at
maybe
integrating
some
of
the
stuff
that
they've
done?
A
I
know
most
of
it
is
probably
not
as
advanced
as
calamari,
but
I
think
that
there
were
some
pieces
that
they
had
that
we
didn't,
and
I
know
that
both
of
those
projects
were
interested
in
getting
involved
in
calamari
development.
So
I'd
be
interested
to
see
like
both.
How
are
we
planning
on
rolling
that
kind
of
work
in,
but
also
in
terms
of
the
actual
ui?
How
flexible
or
plugable
is
it.
D
So
I
think,
that's
a
great
question,
patrick
and
there's
another
project
that
you
didn't
paste,
which
we've
actually
had
some
traction
from
someone
who's
involved
in
it
called
ink
scope
and
on
the
mailing
list,
one
of
the
one
of
the
people
who's
working
on
that
is,
you
know,
expressed
interest
and
something
like
that.
So
the
answer
is,
you
know
it's
definitely
great.
Now
that
we've
released
it
as
open
source.
You
know
we
can
work
toward
making
this.
D
You
know
work
for
as
many
people
as
possible,
so
we
don't
duplicate
as
much
effort.
You
know,
building
these
other
dashboards
before
people
had
access
to
that.
That
being
said,
ui
is
something
we're
a
little
bit
soft
on
in
the
near
term.
We'd
love
some
help
and
support
from
the
community
for
ui
development
as
we're
short
on
resources
there.
So
if
anyone
wants
to
step
up-
and
you
know
contribute
to
that-
we'd
love
to
have
you
so
insofar
as
incorporating
what
they
have
visually
in
the
short
term,
we
don't
plan
to
do
much
of
that.
A
Yeah,
I
was
thinking
more
along
the
lines
of
just
making.
You
know
the
gui
stuff
pluggable
so
that
you
know
if
someone
wanted
to
you
know
normalize
their
dashboard
to
utilize
what
we
had,
but
they
wanted
the
metro
style.
Look
I
mean
I
can't
imagine
why
someone
would
want
that,
but.
B
If
you,
if
you
have
a
way
of
doing
generic
plugable,
javascript
rich
client
user
interfaces,
you
should
do
a
startup
and
make
about
100
million
dollars.
It
is
a.
It
is
a
technical
issue
that
the
the
frameworks
in
use
for
building
rich
client
interfaces
in
general
are
quite
hard
to
plug
into
one
another.
B
You
know
it's
sort
of
a
nice
idea
that
one
would
be
able
to
pick
up
a
widget
from
here
and
put
it
down
there,
but
I
think
we're
we're
going
to
be
seeing
probably
more
traffic
in
ideas
that
you
know
if
somebody's
gone
through
the
process
of
working
out
how
to
expose
something
in
a
user
interface,
then
taking
those
ideas
and
bringing
them
into
calamari.
B
I
think
that
that's
a
very
fertile
kind
of
avenue,
but
in
terms
of
us
going
out
and
trying
to
directly
take
code
from
other
projects
that
there
are
quite
some
technical
obstacles
to
doing
that.
Yeah.
A
Okay,
no,
I
guess
that
that
answers.
The
question
then
like,
if
the
the
best
way
in
to
calamari
is
definitely
just
to
say,
hey,
I
want
to
do
this,
and
this
is
how
I
did
it
and.
A
Yeah
yeah
cool
all
right
were
there
any
other
questions
from
folks
floating
around
out
there.
B
Yeah,
it's
all
on
the
readme
for
github.com
calamari
has
links
to
the
the
mailing
list
and
the
tracker
and
the
documentation
as
well.
Yeah.
A
Just
posted
the
mailing
lists
info,
it's
listed
on
our
mailing
list
and
irc
page
I'd
also
like
to
see
us
aggregate
our
documentation
somewhere
too.
You
know
having
it
over
on
read
the
docs
may
be
all
well
and
good,
but
it
would
be
good
to
have
it
on
stuff.com
somewhere
too.
B
Yeah,
I
mean
that's
just
a
build
thing,
so
I
mean,
if
that
that
can
be
made
to
happen.
Read
the
docs.
It's
just
it's
very
convenient
because
it's
a
it's
a
third-party
hosted
site,
so
it
doesn't
require
us
to
stand
up
any
additional
infrastructure.
B
We
also
have
travis
ci
in
use
for
running
the
sort
of
linting
stuff
against
four
requests
for
calamari
again,
because
it's
straightforward
to
use
a
third-party
hosted
service
for
that,
and
it
doesn't
require
us
to
stand
up
any
new
infrastructure
to
do
it
so
but
yeah,
I
agree.
Convergence
would
be
a
good
thing.
Yeah.
A
Although
that
said,
if
there's
anybody
out
there
that
really
gets
off
on
on
web
infrastructure,
ceph.com
could
use
a
new
coat
of
paint.
We
need
to
figure
out
all
the
moving
pieces
there.
I
know
the
ceph
community
user
committee
was
going
to
take
a
pass
at
it,
but
it
is
several
orders
of
magnitude
more
complex
than
I
think
it
needs
to
be,
so
I
think
that
probably
stymied
any
help
there,
but
hey.
If
anybody
really
wants
to
jump
in
there.
I
I
welcome
the
the
help.
E
E
I'd
say
I
I
had
to
drop
off
so
I
missed
the
last
45
minutes
or
so,
but
yeah
that's
fine.
I
don't
think
so.