►
From YouTube: CDS Jewel -- Calamari/API/Storage/Hardware
A
Alright,
on
to
the
next
one
Gregory
and
Joe
for
calamari
API,
hardware
storage,
all
the
things
though
Greg
you're
gonna
start
us
off
yeah.
C
B
So
this
session
is
going
to
be
about
the
planning
and
implementation
of
a
hardware
api
for
calamari.
The
overarching
motivation
for
doing
this
is
to
provide
calimary
with
the
ability
to
know
about
the
underlying
storage
behind
it
with
these,
so
that,
when
things
happen
to
that
hardware,
we
can
figure
out
what
the
impacts
are.
B
The
cluster
and
help
advise
for
root
cause
analysis
and
make
the
life
of
storage
administrator
is
easy,
so
that
they
can
identify
the
appropriate
hardware
to
replace
and
to
know
what
to
replace
it
with
and
whether
that
hardware
warranty
or
has
recalled
or
any
that
sort
of
stuff.
So
basically
it's
the
full
cycle
if
I
know
something's
wrong
with
hardware,
how
does
it
affect
my
cluster
and
the
other
side?
I
know
something's
wrong
with
my
cluster.
B
What
hardware
is
likely
the
culprit
so
to
that
end,
I'd
like
to
direct
your
attention
to
the
etherpad,
which
was
probably
easiest
way.
They
consume
this
data
as
I've
pasted
it
in
some
of
the
the
potential
API,
endpoint
and
fields.
I
want
to
discuss
those,
so
the
setup
here
is
basically,
you
know
in
one
of
us
distributed
systems
invented
always
happening
and
you're
mostly
about
you,
know
the
causes
of
these
things,
rather
than
just
a
large
stream
of
events,
because
we
want
to
give
the
quality
of
information
or
just
the
noise
of
a
distributed
system.
B
Things
always
fail
and
that's
some
degree
acceptable
in
the
South
world
because
of
the
way
that
is
designed.
So,
let's
imagine
for
a
moment
that
nihd
goes
down.
You
know
that
it's
in
some
hosts,
but
what
drive
is
that
and
is
that
a
software
failure
or
hardware
failure?
What
are
you
replace
it
with
always
have
been
failing?
So
those
are
some
of
the
questions.
I
want
to
try
and
answer
with
this
API.
B
You
can
see
that
we've
got
it's
a
kind
of
a
generic
listing,
endpoint
online
27,
API,
v2,
hardware,
storage
and
it's
all
39
of
a
specific
detail
for
the
same,
which
is
you've,
picked
up
one
piece
of
storage
by
skies,
ewww
ID,
and
so
these
are
just
kind
of
a
list
of
proposed
fields
that
we
think
John
I
think
are
useful
left
here.
Some
feedback
on
whether
there's
anything
we're
missing
there
and
there's
a
list
of
questions
that
I'll
go
over.
B
C
B
Good
to
me,
excellent,
and
so
of
course,
I
didn't
caught
up
in
the
excitement
of
CD
s.
I
realized
the
one
to
make
an
announcement
beginning
of
this
total
just
plow
through
it.
There
are
packages
for
kalmar
be
available
on
download.com
/
calamari
top
of
the
YouTube
pad
I've
got
some
questions.
I
expect
to
get
about
this
and
some
answers
about
those
questions,
and
you
can
read
them
and
I'll
kind
of
answer
them
here
again
my
voice,
which
is
what
packages
and
specifically
it's
the
packages
that
tongue
now
you've
had
to
build.
B
If
you
want
to
interact
with
calamari,
so
calimary
server
appliance,
which
is
becoming
romana
and
diamond,
which
is
a
package
that
we
have
a
fourth
kind
of
that
hasn't
made
an
upstream,
so
that's
great
get
packages
would
I
do
with
them.
Well,
it
turns
out
we
have
a
read
the
docs
pitch
for
calamari
and
as
guidance
on
how
to
install
and
provide
the
link
in
the
aether
pad.
This
stuff
is
also
available
on
the
github.
Read
me
so
that'll
be
front
and
center
for
anyone
who
checks
the
product.
First,
that's
all
well
I'm
good!
B
You
can
see
if
you
actually
go
to
that
URL
packages
that
there's
stuff
available
for
son
toss
and
a
bunt
too.
What's
the
plan
going
forward,
so
we've
released
the
packages
from
our
latest
stable
version
and
that's
a
point
in
time
and
to
start
I
want
to
go
ahead
and
get
some
Knightly
test,
Suites
running
into
these
packages
and
some
public
facing
build
infrastructure
so
that
we
can
go
forward
and
start
filling
out
the
like
the
master
branch,
build
packages
and
other
releases.
B
So
this
is
a
very
small
start,
but
it's
a
accomplishment
nonetheless,
and
I
want
to
thank
Dan
McCann
getting
these
out
there.
So
then,
you
might
say
well
sent
us
a
bunt
too.
It
does
don't
support
my
favorite
distributions.
When
is
that
going
to
happen?
My
answer
to
you
is:
when
volunteers
emerge
to
leave
the
effort,
our
plans
do
Fedora,
21,
plus
and
I'm,
going
to
be
making
progress
on
let
the
rest
of
this
year.
D
B
B
B
Knew
the
reserve
answer
okay.
So
if
there's
no
more
questions
about
packages
which
I'm
really
excited
about,
let's
go
ahead
and
go
back
to
the
harder
API,
also
from
feedback
on
the
mailing
list
from
John
spray
and
others.
We've
gone
ahead
and
tried
to
include
some
more
information
available
in
the
hardware
endpoints,
but
mostly
you
can
see
by
the
URL
structure
that
retargeting
storage
I've
left
it
kind
of
open-ended
the
namespaces
hardware
storage.
B
C
A
fair
question:
I
mean
I
mean
it
would
be
brought
probably
the
way
they've
implemented.
It
is
that
the
there's
basically
a
faked
out
controller
in
front
of
the
device
and
it's
just
a
single
controller
to
a
single
disc,
but
that
would
be
worth
following
up
on
because
yeah,
if
you
don't
have
that
for
say
a
fusion-io
car
or
something
else
and
vme,
and
this
breaks
down
for
that.
C
B
C
From
my
perspective,
I
don't
have
any
better
input
than
you
do
about
what
a
standard
calamari
server
would
look
like
so
but
but
yeah,
you
know
your
point
I.
My
speculation
is
that
we
would
probably
want
as
much
of
the
smart
data
as
we
can
get
if
it
does
overwhelm
a
standard
installation.
The
way
we
store
information
for
calamari
today,
then
we
probably
want
to
push
it
down
to
an
into
a
log
file
somewhere.
So
people
can
go
back
in
in
history
and
take
a
look
at
trends.
B
That's
a
fair
observation.
It
definitely
brings
to
mind
an
old
discussion
that
John
I
had
was
the
Palomar
mostly
cares
about
the
current
state.
What
do
you
do
with
historical
data
you're
currently
answered
as
we
try
to
provide
really
fast
access
to?
You
know,
get
information
that
cluster
state
by
providing
lists
of
architecture.
B
Maybe
this
is
one
thing
where
we
would
want
to
explore
serving
data
from
a
persistent
storage
and
cash
weren't
necessary.
So
the
idea
being
that
you
have
a
kind
of
slower,
end
point
where
you
can
get
a
lot
more
historical
data,
so
maybe
that
you're
not
gripping
log
files
but
you're
using
it
and
point
that
you
know
really
is
going
to
return
slower
responses.
D
Don't
think
you
should
throw
away
data
if
you
can
possibly
avoid
it
sure
hun
and
as
for
whether
a
normal
server
will
hold
up
I
mean
the
starting
point
is
to
emphasize
you
know:
10,000
hard
drives
worth
of
data,
and
you
know
run
that
on
your
laptop
and
if
it
works
on
your
laptop
great,
you
know
we
can
smoke
around.
Oh.
C
Other
the
other
concern
I
have
about
smart
data
is,
is
like
John
said.
It
is
something
you
really
don't
need
to
pull
all
that
frequently,
but
that
doesn't
mean
people
don't
try
to
pull
it
all
that
frequently.
So
we
probably
need
to
think
about
whether
we
want
to
put
in
some
mechanisms
to
protect
people
from
themselves
effectively.
C
B
B
So
there's
only
one
endpoint
in
my
mind
that
doesn't
obey
that
principle
and
that's
the
do
I
endpoint,
which
is
basically
a
pass-through,
so
allows
you
to
issue
Steph
CLI
commands
directly,
which
means
that,
if
you
issue
a
large
number
of
those
can
currently
are
in
short
order,
they
they
can
affect.
You.
C
Okay,
well
then,
that's
good
news.
Well,
so
are
we
not
letting
people
query
smart
on
the
fly
actually
go
down
to
the
set
cluster
itself
and
issue
that
command
over
and
over
again,
where
we're
having
it
be
something
consumable
that
we
are
pulling
on
the
back
end,
instead
of
something
that
they
are
actively
going
and
requesting,
via
the
get
request,
submitting
a
command
down
to
the
lower
levels.
B
I
think
it
makes
sense
to
implement
that
way,
because
my
understanding
of
the
way
that
a
lot
of
Simmons
use
smart
is
that
just
either
yeah
I
think
there's
even
a
smart
d,
but
basically
it
just
ends
up
in
crown
and
it
reports
and
they
look
at
reports.
Not
boys
I
mean
I.
Imagine
in
your
disaster
recovery.
You
might
run
it
and
expect
to
do
it
interactively
so.
D
I
think
the
main
practical
reason
you
know,
leaving
architecture
aside
for
not
gaining
a
whole
demand
is
that
when
a
drive
fails
you
don't
that's
when
you
would
like,
perhaps
to
know
what
the
smart
status
was
before
it
failed.
Brang
arm,
though,
you
need
to
have
been
collecting
it.
In
the
background
the
whole
time
and
at
the
point
that
you
want
you've
been
doing
that,
then
you
know
you
have
it
stored
somewhere.
D
So
when
the
user
asks
for
it,
you
you
don't
have
to
go
and
hit
the
drive
again
and
I
would
echo
the
point
you
were
making
joke
about
some
hitting
dried,
frequently
being
bad
news,
because,
while
some
good
controllers
will
do
that
efficiently,
some
bad
controllers
will
like
stop.
I,
oh
while
they
go
check
smart
stuff,
yeah.
B
B
So,
let's
see
then
I
guess
when.
D
C
B
Well,
I
mean,
I
think,
I
think
the
the
current
direction
for
telomere
is
gonna.
Go
along
the
lines
of
what
your
last
piece
of
feedback
on
analyst
was
John
yeah
calimary
is
too
complicated
to
be
useful,
and
so
in
the
future,
we're
talking
about
paring
down
so
that
it
basically
provides
the
rest
api
and
doesn't
have
a
lot
of
a
lot
of
additional
trappings,
so
kind
of
continuing
your
work
in
the
past
on
calamari
on
monitor
so
making
some
of
the
other
things
that
it
does
right
now
optional,
I'm
trying
to
modularize
it.
B
So
it's
just
it
provides
the
smart
api,
not
smart
sword.
It
provides
an
intelligent,
high
level,
API
a
little
wells,
and
so
in
that
future
I
mean
there's
a
there's
protein
of
the
product
calamari
lives
on.
It
would
still
do
something
like
that,
but
it
if
you
shed
the
summer
that
functionality
than
it
makes
less
sense
to
say,
calamari
schedule,
salt,
our
schedule,
Smart
View,
salt.
C
We're
talking
about
a
couple
different
at
some
point.
We
have
to
abstract
that
away.
Right
is
that
that
that's
it
in
rat
us
or
do
we
go
and
target
some
other
kind
of
library
or
API
that
exists?
That's
already
doing
this,
particularly
well,
where
we're
throwing
around
a
couple
of
options,
I
mean
Gregory.
Do
we
want
to
talk
about
the
one
that
you
were
talking
to
me
about
earlier
at
this
point,
bring
that
in
as
a
possible
buffer
possible
intermediate
consumer
of
the
smart
data.
C
B
B
D
So
I
mean
well,
there's
bears
so
that's
one
question,
you
know
what
library
do
you
use
for
querying
smart
data,
and
you
know
you
also
need
to
think
about.
How
is
it
going
to
get
encoded
and
sent
back
to
the
Mons?
Is
it
going
to
be
something
that
is
and
back
periodically
or
is
it
going
to
be
like
a
tell
command
using
existing
OS
detail
mechanism?
D
C
Okay,
that's
fair
I
mean
they're
there.
The
other
way
we
could
implement
it,
I
suppose,
is
just
as
something
that's
running
in
the
background,
and
then
you
match
up
whatever
the
OSD
is
aware
of.
With
regard
to
an
end
point,
you
know
a
device
note
essentially
with
whatever
you
got
from
that
device
node
and
that
smart
query
running
in
the
background
hitting
all
the
devices
periodically.
D
C
C
C
C
C
C
So
beyond
that
there
there
are
other
things
we
can
do
beyond
smart
right.
We
don't
have
to
just
be
pulling
smart
data,
but
there
are
other
operations
that
may
be
valuable
on
a
controller
array
or
or
on
a
disk
storage
endpoint
of
some
sort.
Is
there
anything
you
know
John
or
anybody
else?
Is
there
anything
you
want
to
see
us
pulling
up
or
querying
from
disks
periodically?
That
would
offload
some
past
that
storage
admins
are
doing
today.
C
C
Directly,
Kiki,
that's
and
that's
part
of
the
other
things
that
we're
trying
to
think
about
here
is
we're
going
to
have
to
abstract
more
than
just
a
command-line
interface
or
an
executable
right.
We
also
need
to
figure
out
how
to
talk
to
Sage,
pssa
CLI
or
what
megaraid
uses,
for
example.
Those
are
those
are
tools
that
we
need
to
interact
with,
and
that's
partially,
why
I
live.
Storage
management
came
into
play
because
it
does
do
it
has
some
of
the
the
framework
for
that
built
up
already.
C
F
I
get
it
if
I
ever
correct
in
mine
for
somebody
else,
I
control
laws
can
be
set
up
to
run
small
anyway
on
regular
basis
and
can
report
it
then
back
on
the
drive.
So
you
only
need
to
call
the
as
I
tools
to
get
this
model.
Information,
for
example,
and
for
sure
information
like
it's.
The
battery
for
better,
very,
very
big
operation
for
us
too,
okay
was
in
the
Fed
state
would
be
some
useful
information
for
real
okay.
C
C
F
Be
used
with
raid
0
preclosing
yeah,
everything
wasn't
ready
to,
but
ya,
better
evict
up
stood
and
yeah
and
I
could
be
useful.
To
note,
I
mean
that
I,
wouldn't
probably
by
such
thing
again
I
didn't
buy
it
so,
but
but
it's
still
there
and
I
guess
they
have
a
lot
of
people
that
may
we
use
all
of
our
existing
heartland
and
there's
often
an
array,
controller
and
Malaysia
informational.
That
would
be
me
maybe
useful.
The
equation
for
me
is
what
kind
of
smart
information
you
really
want
to
collect?
Is
it
only?
F
C
You
can
get
some
hints
about
about
the
health
of
a
drive
from
smart
right.
You
can.
You
can
get
information
about
iOS
that
fail
from
smart
and
that's
the
kind
of
thing
you
can.
You
can
track
and
then
you
you
may
be
able
to
see.
Okay,
this
drive
is
really
not
behaving
correctly.
You
may
even
be
able
to
preemptively
intervene
and
avoid
more
significant,
downtime
right
being.
B
F
B
Yeah
and
to
that
point
is
currently
reason
for
this
design
is
to
try
and
make
it
so
that
we
can
point
to
the
parts
on
calamari.
So
no
vendor
comes
on
it
says
you're
doing
their
own
thing
with
our
hardware.
You
don't
know
how
to
predict
failures.
Let
me
show
you
how
to
do
it.
Let
me
say:
well
we're
you
know
we're
collecting
the
data
here
and
here's
where
we
analyze
it
and
then
there's
where
we
report
it.
So,
okay,
it's
the
right
and
I'll
work.
Well,.
C
D
On
the
point
of
our
raid
controllers,
if,
if
you
are
on
a
system
using
right
erase,
life
gets
complicated
in
some
other
ways
as
well,
so
potentially
for
somebody
and
let's
say
they've-
got
a
10
disc,
great
sex
and
they've
sliced
it
up
into
two
lumps
and
each
one
is
a
different
OSD,
and
so
each
of
those
lumps
has
a
worldwide
IE,
but
so
does
each
scuzzy
desk
within
the
array
for
both
one's
a
carved
out
of
europe
and
at
that
point,
you're
gonna
have
to
come
up
with
a
molar
schema
for
expressing
that
topology
of
you
know,
you've
got
a
lung.
D
It's
got
disks
in
their
part
of
a
raider
rain
as
another
loan.
That's
part
of
the
same
rhetoric
that
that
can
get
pretty
hairy.
If
you
/
specially
trying
to
do
it
generically,
so
would
it
I
mean?
Would
it
be
unreasonable
to
to
say
actually
it's
somebody's
using
raid
arrays
there
are
a
manufacturer
should
give
them
tools
for
this,
and
the
the
functionality
ncf
should
only
be
for
their
desks.
I
mean
that
that
might
it
depends
on
what
people
are
using,
whether
that
it's
another
ceptable
limitation.
C
You
know
I
I
do
and
duel
yeah
multipathing
is
a
total
different
beast
and
that
that's
I
would
say:
that's
certainly
outside
the
scope
of
what
we're
expecting
force.
F.
Is
anybody
actually
using
multipathing
and
stuff
today
that
we
know
we're
old,
so
I
I,
don't
know
it's
something
that
we
may
have
to
think
about.
C
See
what
else
we've
got
here,
so
I
think
the
the
the
generic
elements
that
we're
adding
underneath
the
worldwide
ID
or
whatever
we
consider
the
abstraction
to
be
eventually
make
sense.
You
know
drive
device
node.
What
kind
of
disk
it
is
what's
capacity
is
manufacture.
Former
version
serial
number,
that's
pretty
much
the
basic
stuff
that
you'd
want
to
know.
You.
C
D
D
C
B
F
I
don't
know,
for
example,
I
know
some
people
at
information
about
the
machine
to
the.
What
is
it
called
ever?
Who
you,
or
something
like
that
could
put
information
about
machine
on
with
your
own
inventory,
inventory
information
on
the
different
machine,
and
maybe
that
was
something
you
may
also
want
to
add
to
the
host
to
identify
it
easily
instead
of
animals.
Name
I
mean
in
a
large
set
up.
A
host
name
doesn't
mean
anything
so
yeah,
that's
a
good
point.
What
you
can
with
the
next
boot
your.
B
C
Yeah
and
that's
that's
kind
of
a
another
stretch
goal
for
what
we're
doing
here.
It's
not
there.
There
is
other
Hardware
in
this
environment.
That's
not
just
storage
disk
base
and
I
PMI
is
a
great
way
to
get
at
that.
There
are
other
future
looking
tools
too.
If
you
want,
you
can
go
and
look
up
what
red
fish
is
there,
a
bunch
of
other
vendors
collaborating
on
DMC
capabilities
outside
of
just
the
basic
ipmi
set
of
data?
C
D
Think
it's
it's
it's
a
kind
of
phrase
this
it's
very
easy
when
writing
a
storage
management
tool
to
end
up
writing
a
data
center
management
tool.
C
D
Enough
I
mean
this
I.
It's
it's
open
to
debate.
Of
course.
What
what
idea
that
I've
had
is
things
which
are
on
the
IO
path,
our
sex
business,
so
the
block
and
ice
is
SAS
business.
The
network
card
potentially
assess
business,
the
supporting
infrastructure,
probably
less
soap
and
the
the
Linux
hosts
themselves.
D
That
I
mean
the
presume.
Presumably
most
environments
have
some
other
tool.
That's
monitoring
their
estate
of
service,
though
we
I
think
we
should
be
mindful
that
calamari
and
Seth
unlock
the
only
things
running
on
these
systems
that
any
normal
administrator
will
so
have
deployed
best
side,
wide
monitoring
tools
that
they
use
all
their
servers.
Yeah.
D
That's
that's
a
little
bit
different
if
somebody's
building
like
Anna
clients
is
that
everything
that
well,
then
they
would
probably
want
anything
much
more
all-encompassing,
but
for
it
for
a
general
purpose,
Seth
monitoring,
tool,
I
think
as
a
starting
point.
Maybe
we
should
limit
it
to
IO
devices
yeah.
B
That's
a
good
point
and
I
think
that
you're
right
about
that
one
future
direction
of
calamari
is
definitely
to
be
the
the
answer
about.
What's
going
on,
you
know
from
sess
perspective,
to
those
larger
monitoring
frameworks,
so
we're
just
sitting
there,
alongside
as
peers
of
other
things,
about
toast
status
under
management
and
all,
but
but
we
don't
want
to
get
our
fingers
into.
We
want
to
just
be
offering
the
best
opinion
we
can
about
what's
going
on
with
the
stuff
in
the
IO.
B
B
C
I
think
I
got
the
answers
that
I
needed
and
especially
about
scope
of
what
we're
doing
here.
I
think
we
still
have
some
decisions
to
make
about
whether
we
plug
into
some
other
more
general
purpose
tool.
That's
already
implemented
around
some
of
the
other
tools
that
we're
trying
to
include
here:
smart
HBS,
a
CLI,
etc.
That's
all
still
TBD,
but
I
think
we
have
a
pretty
good
start
on
what
the
right
answers
are
to
look
for.
D
D
A
A
A
C
B
Okay,
the
next
session
mon
calamari
has
fun
title:
let's
make
calamari
easier
to
troubleshoot,
so
I
can
say
that
about
this
topic
myself
and
I
know
John.
Does
a
great
job
of
participating
in
the
calamari
mailing
list
spend
a
large
amount
of
time,
helping
other
people,
troubleshoot,
calamari
and
I.
Don't
think
that's
really
the
best
use
of
my
time,
because
I
want
to
make
calimary
better
and
have
time
to
write
features.
So
this
session
is
really
about
some
ideas.
I've
had
and
some
ideas.
Maybe
the
community
has
about
how
to
improve
that
interaction.
B
How
to
get
past
the
trouble
you
know,
overcoming
the
difficulty
of
getting
calimary
set
up
so
I
think
something
that's
going
to
go.
A
fair
amount
of
distance
than
that
is
having
offered
packages
for
calamari
on
download.com,
because
so
far
the
conversation
once
I
have
calamari
is
usually
ok,
we're
going
at
the
packages
and
until
now
our
answer
has
been
well
you've
gotta
build
them
yourself.
So
in
the
future,
that's
less
the
case
because
we're
offering
them
going
forward
I'm
to
improve
them
and
where
did
not
even
live
question
phone?
B
B
The
class
of
issues
that
we
usually
have
with
salt
is
the
people
areas.
They
are
usually
running
versions
that
are
far
too
old
or
unfortunately,
in
the
recent
case,
far
too
new,
so
2014
anything,
it's
a
pretty
good
bet
for
salt
and
all
the
other
versions
cause
this
from
me,
so
to
men's,
I've
included
in
the
docs
that
bit
and
I've
engaged
the
salt
community
to
work
on
what's
causing
us
trouble
with
the
latest
versions.
B
D
A
bit
maybe
relaxing
that
in
some
way,
so
the
underlying
issue
that
you're
having
with
salt
is
that
calamari
uses
norm
stable
api's
and
behaviour
insult.
So
you
know
some
of
some
of
the
method
names
that
calls
start
with
an
underscore
and
as
long
as
that's
the
case,
new
versions
of
salt,
organically
break
from
calamari
notice.
B
B
You
know
into
my
acceptance
in
minions
to
the
master
then
proving
that
the
transport
0q
transports
actually
functioning
and
then
finally,
proving
that
fool
is
able
to
do
meaningful
work
on
stuff
cluster,
so
I
agree
the
non-public
interfaces
of
problem,
but
in
order
to
make
calories
your
troubleshoot
that's
kind
of
low
on
the
order
of
I
care
about
that
anyway,
because
most
of
the
time
it's
just
well,
okay,
what
is
what
assault
key
she'll
tell
you.
What
does
salt
star
test
pain?
B
D
B
C
I
think
that
would
get
you
a
long
way.
Gregory
I've,
just
I've,
seen
it
from
the
time
that
I've
paid
attention
to
what's
going
on
in
calamari
just
there.
You
know
there
are
some
errors
that
come
back.
That
are
not
obvious.
You
know
it
comes
back
from
this
Python
file,
but
it
really
has
nothing
to
do
with
this
Python
file.
C
It's
really
this,
what
salts
doing
five
layers
up
or
something
or
what
cthulhu's
doing
somewhere
over
here
and
so
really
knowing
how
to
go
back
all
the
way
back
to
where
the
problem
might
be
via
you're,
really
just
your
experience
and
debugging.
This
sort
of
thing
would
probably
help
a
lot
of
people
myself
included.
B
Yeah
I
feel
like
too
long
I've
been
guilty
of
just
synthesizing.
Some
troubleshooting
steps
and
one
offing
at
each
time
whenever
I
should
be
doing,
is
improving
the
docs
and
always
point
the
docs
is
if,
if
they
can't
help,
then
it's
on
our
shoulders
or
my
shoulders
or
anyone
who
spends
time
on
the
mailing
list
to
come
up
with
solutions
every
time
right,
I
agree,
I
want
to
say
that
that's
definitely
one
more
working
for
this,
which
is
through
the
doc
troubleshooting
section.
C
May
be
the
easiest
one
and
I
don't
know
is
there,
you
know
you've
hinted
to
it
and
making
calamari
lighter
weight,
but
that
is
part
of
the
part
of
the
difficulty
in
it
right
there.
There
are
these
other
pieces
running
around,
be
its
salt
or
some
other
piece
of
Python
code.
That's
come
in
and
really
understanding
what
the
right
answer
is
to
debug
all
those
different
little
pieces
or
finding
some
way
to
remove
them,
if
at
all
possible,
right,
remove
complexity,
make
it
easier
to
get
down
to
the
core
of
the
problem.
C
B
In
for
sure,
I
can
save
it
from
you
know
the
from
the
front
leg
standpoint,
that's
a
viable
solution.
I
can
reduce
the
complexity
and,
from
my
day,
job
is
the
end
point
the
product
that
this
is
based
on
isn't
going
away.
So
it
looks
like
I'm
going
to
be
supporting
that.
So
the
good
news
is
is
that
you
have
a
lot
of
flexibility
with
what
callum
or
you
can
become.
I'm
still
gonna
figure
out
how
to
spend
less
time,
dealing
with
the
frustrations
of
salt
and
some
other
problems
with
calimary.
B
D
C
I
think
I,
you
know
we're
making
a
little
bit
of
progress
towards
getting
away
from
the
the
underbar
prepended
api's
the
intended
to
be
internal
api's
of
salt.
That
was
that
was
what
happened
without
that
one
salt
loader
issue
that
I
submitted
that
pull
request
for
it.
You
know
if
just
did
translate
to
the
new
bit
that
was
in
2015.
No,
unfortunately,
it
turns
out
that
2015
has
problems
that
we
have
to
back
pedal
from.
So
we
can't
really
fully
migrated
over
to
that,
but
you
know
we're
trying.
D
D
D
And
yeah,
but
very
specifically,
to
get
this
behavior
of
holding
a
lock
for
the
duration
of
processing
a
particular
event
coming
from
salt.
So
I'm
not
talking
about
going
through
all
the
data
structures
and
adding
lock,
lock
members
to
and
that
kind
of
thing
I'm
talking
about
you
get
an
event
from
salt.
You
take
a
lock,
you
deal
with
it.
B
B
We're
going
to
well
done,
I
mean
that's,
that's
pretty
easy.
Ideas
are
being
that
we
have
the
nicely
decorated
to
guide
us,
essentially
that
we've
got
the
locking
correct,
though
it's
my
understanding
that
we,
if
we
put
it
a
walk
around
the
event,
handling
code
and
right
way,
then
it'll
start
itself
out.
You
can
move
that
decorator,
where
it
is.
Let's.
B
B
B
D
So
I've
to
Colette's
assure
a
little
one.
So
the
short
one
is
that
there's
a
piece
of
low-hanging
through
here,
which
is
to
get
the
diamonds
changes
upstream,
there's
nothing
wrong
with
that
code.
It
just
needs
someone
to
take
on
the
task
about
streaming
up
the
longer
one
is
that,
in
order
to
make
sure
that
things
aren't
breaking
with
new
versions
as
they
come
out,
the
development
of
calamari
has
to
happen
on
the
upstream
district.
There
has
to
happen
in
fedora,
so
going
and
doing
practice
for
fedora
is
absolutely
the
right
thing
to
do.
D
D
But
if
you,
if
you're
developing
the
right
of
the
bleeding
edge
there,
then
you
can
ensure
that
breaking
changes.
Don't
go
into
the
distro
and
you
can
object
to
things
on
the
basis
that
hey
my
package
depends
on
zeromq.
These
guys
made
a
change
the
broke
it
you
know.
Potentially
they
have
to
back
out
there
change
rather
than
you
having
to
make
a
walk
around,
and
that
certainly
requires
a
lot
of
work,
but
I
must
be
sort
of
the
ideal
way
of
doing
it.
B
Yeah
and
I
support-
that's
kind
of
where
I
want
to
go
is
the
idea
that
if
we
make
it
so
that
the
development
environment
works
on
fedora
and
there's
packaging
available
all
this
packaging
available
on
fedora,
and
we
can
be
part
of
that
discussion
instead
of
boys
reacting
to
the
well.
This
thing
changed
and
now
I'm
going
to
deal
with
it,
because
our
previous
answer,
which
wasn't
so
great,
was
you
know
we
also
were
very
sensitive
to
version
of
all
the
way
to
solve.
B
B
C
C
C
B
Actually,
just
I.
E
What
I'm
saying
need
to
set
up
a
test
so
that
it
doesn't
try
to
do
an
external
cluster?
Because
testing
has
an
external
cluster
yeah.
You
probably
want
to
ecology,
but
if
you're
not
testing
its
internal
cluster,
if
you
just
want
to
test,
go
and
learning
itself,
there
shouldn't
be
any
requirement
for
utility.