►
From YouTube: CDS Infernalis (Day 2.2) -- Ceph Performance
Description
Videos from Ceph Developer Summit: Infernalis (Day 2.2)
04 March 2015
https://wiki.ceph.com/Planning/CDS/Infernalis_(Mar_2015)
A
All
right
welcome
to
session
one
of
day
two
at
two
for
the
SEF
developer
summit.
I
will
be
talking
about
a
standard
framework
for
Steph
performance
profiling
with
the
latency
breakdown
I,
believe
our
friends
from
Intel
have
have
submitted
this
blueprint
and
Mark
Nelson,
obviously
will
be
weighing
in
heavily
so
I
think
you
guys
had
some
slides.
If
you
want
to
give
us
an
intro
of
what
you
were
thinking
and
then
we
can
take
it
from
there.
C
Them
it
lit
properly
Malik
about
we
want
to
the
foot.
We
want
to
focus
on
two
things:
the
first
device
we
want
to
integrate
cough
into
cbt
at
the
performance
benchmarking
tool
and
the
other
thing
is.
We
want
to
share
our
findings
and
the
experience
with
current
to
treat
profiling
and
increasing
framework.
We
have
done
a
lot
of
performance
analysis
of
work
on
staff,
and
then
we
found
latches
in
some
were
to
the
dto
profanity
and
reaching
framework
will
be
very
helpful.
So
we
already
all
right
already
have
some
meeting
with
mark
and
I.
C
Think
from
our
interest
point
of
view.
We
in
the
future
we
will
average
the
cbt,
as
than
a
little
self
of
benchmarking
tool
and
I
think
it
is
good
for
quick
performance
testing
and
doing
airport,
where
poor
performance
comparison
across
different
platforms
with
different
set
of
verses
and
the
clever.
If
we
took
a
look
around
current
current
benchmark
of
current
to
walk
around
cbt
supported,
we
think
we
need
an
object,
storage,
benchmarking
tool,
so
India
has
developed
a
cost
benchmarking
tool
called
the
coffins,
which
is
the
cloud
object,
storage,
benchmarking
tool.
C
C
We
want
to
integrate
cough
engine
hood
to
cbt
and
for
the
for
the
later
one
is
we
have
trying
to
do,
listen
to
break
down
a
fall
from
a
lotta
from
for
several
months
after
the
first
week,
we
added
to
the
trip
onto
Hanuma
and
manually
and
from
I
think,
is
from
dumpling
version,
and
it
seems
right.
It
is
very
complex
and
very
difficult
and
time-consuming
to
add
attitude
risk
on
ship
manually.
C
So
we
are
thinking
that
if
we
already
have
a
cheating
from
work
like
the
lack
of
interest
or
portraits,
are
something
like
the
general
treaty
from
working
half.
In
this
way,
we
can
simply
leverage
in
that
framework
to
do
detailed,
latency
breakdown.
It
will
have
the
developer
to
understand
the
performance
bottlenecks,
laterally.
C
A
A
C
This
is
a
birth
instruction,
cotton
bench
so
currently
on.
We
can
see
from
the
bottom
picture
that
actually
cost
mean
to
provide
a
web
console
to
monitoring
the
Taft
and
view
the
result
from
the
bottom
left
picture.
We
can
see
many
how
many
job
of
the
week
we
have
where
the
controller
and
it's
for
the
bottom
right
picture.
It's
actually
about
to
the
performance.
Magics
cosmetic
are
reported,
including
the
reader
writer,
the
through
food
to
the
respond
fam,
the
success
ratio
penalizing
something
like
that.
C
It's
also
maintain
a
history
history
blog
over
all
the
all
the
runs
that
we
have
been
scheduled.
So,
although
Kaufman's
have
a
command
line
spot,
for
example,
from
the
apparatus,
we
can
see
that
today
either
script,
which
can
submit
a
configuration
file
for
coughs
pinch
and
then
click
off
the
run.
Okay,
this
is
a
general
introduction
of
pinch,
and
so
considering
our
goal
is
to
integrate
cough
into
cbt.
So
we
provided
a
simple
approach
Oh
with.
We
think
that
the
current
this
approach
would
be
the
most
efficient
and
fast
one.
C
We
we
would
prefer
to
depend
on
the
inner
user
to
deploy
cough
into
themselves.
Following
the
cough
into
catalan,
that's
it
because
that
you
deploy
crossbench
is
somewhat
complex
job.
If
we
leverage
CBG
to
do
the
deployment
to
work,
we
need
to
adding
the
controller
since
the
the
GBM
dependencies
elaborations
all
to
cbt.
So
if
we
can
separate
the
two
from
the
job,
it
would
emit
much
matter.
This
is
like
kind
of
like
Oh.
What
have
I
all
is
doing
so
Sabine.
C
He
only
needed
to
give
a
binary
link
to
over
crossbench,
and
so
in
that
way,
kaufman
can
be
run
by
cbt,
so
confidential
will
be
deployed
by
the
user
separately.
The
next
step,
we
need
to
add
a
plug-in
model
to
cbt,
and
we
prefer
to
edit
the
parameters
for
Kaufman
gene
the
yam
fell
and
then
like
the
module
to
extract
the
parameters
and
affiliate
in
a
xml
file
that
cough
inch
can
can
be
used.
C
C
We
can
add
a
cough
inch
party
in
there
and
we
can
define
the
different
fashions
like
the
vocal
hello,
the
name,
the
authenticating
part,
if
they
are
insulting
of
indicating
involved
for
for
a
GW,
and
we
can
then
describe
the
object
style,
for
example,
cc's
of
512k
objects
test
test
here,
and
we
can
define
the
mode
either
read
or
write
or
mix,
and
we
can
define
the
ratio,
which
is
one
hundred
percent
rate
or
one
hundred
percent.
Oh
right,
I'll
mix
the
mixer
mode.
After
that
we
can
define
the
list.
C
The
timing,
part
of
the
ramp
ramp,
Apple,
the
Rembrandt
I'm.
On
the
run
time,
the
the
concurrency
is
controlled
by
the
worker,
the
number
of
workers-
and
we
also
have
the
continuous
and
object
to
to
to
describe
how
cops
pinched
is
going
to
testing
the
object.
Storage
performance,
for
example,
while
is
so
too
only
only
rather,
data
to
a
specific
number
of
containers
are
randomly
choosing
one.
With
this
cost.
With
this
example,
configuration
configuration,
the
plug-in
module.
Will
extract
it
and
it
generated
the
the
coffee
inside
time
about
anna
translated
to
into
that
format.
C
After
that,
we
we
can
use
mullen
to
invoke
crossbench
with
that
configuration
file
to
do
the
performance
hatching
and
then
generates
the
result.
The
result
is
like
the
bottom
picture.
It
will
contain
atheria
a
csv
file
neiman
after
the
walk
alone,
a
workload
name
so
the
other
jobs,
the
plug-in
module,
need
to
be
in
to
be
complete,
completed
to
extract
the
the
results
from
the
cross,
pin
to
csv
cell
and
and
then
read
to
and
very
integrity
into
the
common
cbt
result.
So
this
is
a
general
proposal
for
the
detail.
Implementation.
C
D
For
just
one
second
mark
I,
don't
I,
don't
actually
know
very
much
about
cbt.
What's
that,
does
that
model
make
sense
where
I
understand
correctly?
That
cause
bench
instance
will
be
persistent.
Cvt
will
be
starting
up
for
each
run,
a
fresh
south
cluster
and
then
triggering
a
cause
bench
workload.
Does
that
yeah.
E
D
E
All
right,
yeah,
yeah
and
part
of
this
is
figuring
out
in
costs
bench.
How
we
actually
kind
of
specify
that
we
want
specific
test
run
that
specific
times
right
now,
you
kind
of
give
cost
bench
a
a
configuration
file
that
also
could
potentially
allow
cost
bench
to
like
sweep
through
different
things.
So
we
kind
of
have
peter
out
who
does
what
and
how
we
actually
do
the
submission
to
cost
bench.
C
D
D
In
that
case,
then,
for
each
for
each
set
of
specific
parameters,
it'll
submit
a
job.
Cosmos
will
do
it
right.
Then
DBT
will
just
there
and
wait
for
it
to
order
of
the
plug-in
or
whatever
would
wait
for
it
to
finish
and
pull
out
the
CSV
and
yeah
dump
it
into
whatever
BBT
is
doing.
I,
don't
actually
know
how
CVT
stores
step
4
simply.
F
E
Yeah
CBT
is
very
dumb
right
now.
It
just
stores
the
the
data
and
a
nested
directory
hierarchy
that
then,
basically
is
copied
back
to
the
wherever
you're
running
the
script
from
I
think.
Eventually
we
we
need
to
do
something.
Smarter,
I
was
kind
of
thinking
like
local
SQL,
Lite
databases
that
could
then
be
output
for
each
run
and
then
through
tools
combine
them
into
much
larger
databases
that
you
could
do
like
some
kind
of
mining
operations
on
I
think
some
of
the
things
we
have
to
think
about
two
are
in
cbt.
E
D
E
E
Yeah,
alright,
and
what
we
can't
have
to
decide
those
do.
We
want
cost
bench
to
sweep
through
its
own
set
of
parameters,
or
do
we
want
cbt
to
sweep
through
those
parameters
and
invoke
a
single
instance
of
a
benchmark
in
cost
bench.
I
think
we
might
be
able
to
do
either,
and
maybe
it
would
be
useful
in
cost
bench
to
actually
allow
someone
to
execute
like
a
specific
test,
so
that
other
tools
could
maybe
make
use
of
it
in
that
way.
But
women
have
the
castle
is
possible.
D
I,
don't
think
so.
I'm
not.
D
C
Configuration
the
young
fowl
would
be
only
one
section,
but
it
could
be
generating
multiple
cough
insects,
files,
for
example.
If
we
define
like
many
test
cases,
we
define
multiple
runs,
then
it
will
generate
a
multiple
Cosmi
to
xml
files,
and
in
that
case
the
the
cottage
will
be
wrong.
For
several
several
line
is
like
the
previous
picture.
I
have
soon
okay
dynamics
in
okay,.
E
E
Configurations
almost
will
say
so
this
factory
will
go
through
and
yeah.
It
will
only
launch
each
one
with
the
specific
set
of
parameters.
So
if
we
want
to
do
it,
the
other
way
around
we'd
actually
have
to
change
kind
of
the
way
that
it
works.
When
it's
running
benchmarks
to
save
you
know
now,
in
fact
something
else
is
going
to
handle
this
they'll
launch
this
particular
one
with
all
these
parameters,
not
just
a
subset
of
them.
G
We're
now
talking
about
the
latency
breakdown
in
the
current
chef
coat.
When
we
do
latency
breakdown,
it
will
be
slow
in
time
consuming,
because
the
the
chase
points
in
the
lab
coat
has
no
semantic
and
and
also
we
need
a
homemade
script
to
do
the
post-processing,
especially
when
some
checkpoints
are
in
the
OSD
level
and
some
are
in
the
file
store
levels.
They
have
different
identifiers.
G
So
this
is
why
we
want
to
do
a
general
transport
and
tracing
framework
to
to
make
it
more
general
ender
can
visualize
it
so
a
hair
and
in
the
current
ltd
energy
codes
inside
staff,
as
we
as
we
just
talked
about
that.
It's
without
the
semantics,
so
in
the
current
blocking
patch,
which
is
using
the
LT
TNG
and
the
block
and
block
as
it
came
to
do
the
latency
breakdown
and
follow
the
Google
paper
dapper.
So
it
has
this
man
take
and
another
Church
Point
format
will
be
more
general.
G
D
G
C
D
Glow
had
I'm
just
I
just
want
to
make
sure
I'm
understand,
because
that
I
know
we
added
trace
points
in
liberate
us
and
love
our
BD.
That's
basically
just
mirrored
the
actual
function
calls
so
you
can
trace
who's
using
the
API
and
what
their
arguments
are.
I,
don't
remember,
actually,
I'm,
not
I,
don't
think.
We've
added
anything
on
the
OSD
side
that
uses
el
TT
in
G.
Yet
let's
what
I'm
again:
yeah:
okay,
okay,
so
you're
talking
about
the
rbd
and
the
brightest
ones,
I.
C
G
In
this
page,
you
can
see
because
they're
using
the
unit
United
ID,
to
identify
the
whole
I'll
request
and
also
the
it
way
of
using
the
span
ID
and
parents,
man
ID
to
level
the
chase
point.
So
when
using
the
semantics
is
there
only
have
the
timestamp
and
key
value,
two
kinds
of
semantics
to
to
describe
the
chase
point.
So
it
will
be
such
things
simple
to
to
do
the
post-processing
and
to
understand
this
cheap
one.
G
So
when,
when
we
chased
using
the
block
in
patch,
we
can
get
this
if
we
can
print
out
the
results
in
the
zucchini
web
page.
It's
kind
of
like
this
this
page
and
if
you
click
one
I'll
request,
you
can
see
the
whole
they
tend
to
break
down
picture
here
and
because
we
use
lesbian
I,
do
it
so
the
latency
is
leveled
in
different
levels.
Oh
and
you
can
see
just
the
OSD
level
as
the
injure
level
two
and
the
pastor
is
the
level
five.
B
E
G
Okay
expert:
we
need
to
produce
the
chase
point
using
the
LT
TNG.
Then
it
will
store
in
the
LT
TNG
file
and
we
using
a
pison
to
to
send
this
algae.
Tng
chase
pointed
to
the
lib
k.
The
King
has
three
demons:
one
is
the
collector
and
then
the
query
and
the
web
page.
So
the
tractor
will
receive
these
chase
points
and
install
into
his
day
to
pay
at
his
database.
I.
G
Then
I'll
go
is
building
a
general
framework
to
include
most
important
trade
point
and
using
the
clock
in
touch.
So
our
approach
would
be
in
this
respect.
Steps
one
is
to
provide
a
script
to
deployed
LT
TNG
in
the
zipper
King,
because
they
are
two
different
separate
module
and
then
we
need
to
add
more
chase
points
in
the
step
codes
to
cover.
G
Other
part,
like
message,
part
of
the
part
journal
part-
and
the
third
part
is
we
need
to
do
some
modification
to
the
duplicate
to
view
the
latency
mo
better
and
more
clear,
and
the
current
gap
is,
the
blocking
patch
is
not
murdered
to
death
master
code,
yet,
but
the
days
days
ago,
Andrew
the
author
of
the
blocking
patch.
Is
it
already
sent
out
this
version?
Five
though
he
said
it
will
be
promising
too
much
like
minutes,
I
guess.
G
So
this
is
a
gap,
and
if
the
blocking
patch
merge
into
the
safe
code
we
can
do,
we
can
cover
other
messages.
Besides
the
mos
do
p.
Maybe
we
can
do
some
points
in
the
pairing,
something
like
that
and
and
the
insert'
gap
is
when
we
do
the
zip
key.
It
only
shows
the
one
specific
I
Oh
latency,
but
in
our
test
there
will
contain
like
ten
thousands
of
Io
in
wine
wrong.
So
we
think
we
need
to
add
a
summary
page
to
make
it
more
clear
and
that's
how
I.
D
C
G
B
D
If
there's
like
a,
if
there's
a
script
that
you
could
use,
that
would
take
the
to
trace
points
and
look
at
look
at
events
that
are
clearly
the
same
or
you're,
sending
an
event,
a
message
and
receiving
a
message
and
then
try
to
look
at
a
bunch
of
those
and
just
post.
Do
a
post
processing
step
that
basically
adjusts
all
the
time
spans
to
be
sort
of
the
minimum.
D
D
D
Okay,
just
an
idea
so
I
just
one
and
two
or
one,
especially
so
that
the
goal
is
definitely
to
get
this.
This
merged
in
I'd
Andrew
isn't
here,
unfortunately,
but
he
was
blocked
because
he
was
he
was
seeing
crashes
when
he
was
running
it.
There
was
like
a
some
sort
of
memory:
corruption
that
I
think
he
worked
with
Josh
recently
and
and
fix
that,
but
I'm
not
sure
so
I
mean,
if
you're,
if
you
guys,
are
actually
running.
G
D
D
C
D
G
D
D
Ok,
awesome!
Ok,
so
I
think
I
think
probably
the
way
forward
is
just
did
to
get
andrews.
Patches
study
posted
the
list.
The
latest
version
in
the
form
of
a
pull
request,
and
we
can
merge
it
into
a
test
branch
and
run
it
through
the
key
ways.
We
can
make
sure
it
actually
doesn't
break
under
all
of
our
stress
tests
and
if
that's
okay,
then
we
should
be
in
good
shape.
D
G
G
D
G
D
H
This,
like
post-wedding
tradition,
so
my
understanding
is
that
close
range
will
insist,
is
rgw
path
right,
so
so
all
right
and
and
and
then
so.
We
are
planning
to
actually
see
the
how
much
deposition
pardal
is
wise,
but
but
the
RDW
itself
is
actually
very
low,
performant
and
and
and
if
there
is
any
declaration
on
the
NSA
actual
cluster
side
right,
it
may
not
able
to
be
take
because
any
weights
bottle
like
an
ostrich
right.
So
in
that
case,
how
actually
he
was
able
to
figure
it
out.
That
is
there
any
declaration
on
the
road.
H
What
I,
if
you
actually
integrate
rattles
range
sorry
this
cause
bench
with
to
get
this
performance
trepidation
released
by
the
lease
right.
So
so,
and
my
understanding
is
that
it's
from
the
RDW
side,
so
rgw
itself
is
actually
the
garlic
on
the
oxygen
level.
So
if
there
is
speculation
on
the
radio
soup,
which
way
still
able
right
so
it
won't
be
actually
will
be
detect
from
the
from
the
app.
C
B
C
E
H
B
D
D
C
D
F
C
Avantika
see
the
in
our
than
initial
plan.
We
also
want
to
do
this.
For
example,
we
only
care
about
the
OSD
level,
only
added
program,
only
adding
gambling
to
respond
to
why
the
went
to
ink
into
the
OSD
part,
and
after
that
I
went
to
into
the
journal
alf
after
part,
but
that
basically
depend
on
how
how
many
trees
font
viewing
enabling
our
teaching
g.
So
it
is
quite
simple
to
implement
that,
but
hidden
different
people
may
have
different
opinion.
What
level
you
want,
we
can
propose
a
draft
on,
for
example,
we
can.