►
From YouTube: 2018-MAR-08 :: Ceph Performance Weekly
Description
Weekly collaboration call of all community members working on Ceph performance.
http://ceph.com/performance
For full notes and video recording archive visit:
http://pad.ceph.com/p/performance_weekly
A
A
A
A
There's
an
async
messenger
PR
here
by
how
am
I
that
that
I
think
it
mostly
is
just
look
at
reducing
lock
contention
and
apparently
has
pretty
pretty
impressive
latency
reduction
effects,
and
I'm
really
curious
about
that,
because
we
we
do
actually
see
a
fair
amount
of
cpu
time
spent
in
the
messenger
for
certain
workloads.
So
that's
good.
A
Let's
see,
I
don't
know
very
much
both
this
stupid
allocator
discard
PR.
I
know
we've
looked
at
discard
in
the
past
and
not
really
done
much
with
it,
but
it
looks
like
igor's
reviewing
it
so
next
time
we
see
him.
Maybe
he'll
update
us
on
it,
but
I'm
a
little
nervous
anytime.
We
do
anything
too
stupid
alligator.
A
B
But
still
working
progress
anyway,
it
compared
the
competition
on
on
arms
and
clinging
memory
is
on
the
board.
So,
even
if
someone
will
click
the
button,
there
would
be
no
Greek
tragedy
and
we,
the
idea
there
is,
is
to
remove
mostly
stood
among
the
lock.
That
looks
perfectly
unnecessary.
It's
rich
right,
lock
that
is
taken
only
in
the
non-exclusive
mode,
but
taken
on
the
main
path
on
the
hot
purpose
of
execution,
removing
it
and
unveiled
high
contention.
B
We
got
on
the
spin
lock
in
on
the
spin,
lock
guarding
the
communicated
community
communication
part
with
op
history
thread.
Also,
there
are
some
some
non-related
at
the
moment
fixes
mostly
for
our
spin
orientation.
It
was
using
only
the
compare
exchange
instruction,
invert,
I
Club,
that's
not
the
best
idea
and
just
implemented
some
some
proposals
from
from
Intel
and
also
from
GBC.
A
C
B
A
All
right,
let's
see
so,
there's
a
couple
closed
ones
here,
both
the
first
one,
the
second
one
I
made
to
talk
about
them
a
little
bit
more
in
terms
of
what's
the
discussion
topics
below
and
then
that
third
one
adds
minimal
tracing
for
cash
activity.
I
think
stage
just
closed.
That
I
think
he
maybe
just
didn't
do
anything
with
it,
so
that
one
is
no
longer
there
a
couple
of
updated
ones.
My
PR
sounds
like
I,
failed,
I.
Think
for
unrelated
reasons
that
look
like
I
need
to
ask
you
about
that.
A
But
yeah
look
like
some
other
stuff
was
going
on
and
then
yeah
there
were
just
a
couple
other
tests
that
it
looks
like
maybe
failed,
or
at
least
one
this
ec
back-end
one
and
then
the
other
one
said
I,
don't
think
a
whole
lot
was
going
on
yet
with
them
all
right.
So
maybe
before
I
get
into
this,
if
I
have
does
anyone
else?
Have
anything
they'd
like
to
talk
about
this
week
or
bring
up.
A
All
right
so
then
I've
got
I've
got
two
things.
The
the
first
is
related
to
this
change
to
you
of
us
buffered
I/o,
true,
so
the
gist
is
that
we
have
a
user.
We
had
a
user
on
the
the
IRC
channel,
who
migrated
from
file
store
to
blue
store
and
was
seeing
all
kinds
of
background
work
happening.
It
was
like
22,000,
read,
I,
ops,
I've,
just
you
know,
apparently
nothing
see
didn't,
have
any
client
traffic
going
on.
Well,
it
turns
out
that
this
was
work
created
by
the
migration
process.
A
He
based
on
on
the
instructions
on
the
soft
page.
He
he
removed
the
old
file,
store,
OST,
wait
for
the
cluster
to
heal
and
then
put
the
new
blue
store
OST
back
in
and
did
this
for
every
single
OST
and
it,
you
know,
created
a
lot
of
extra
background
work
that
he
did
do
be
done
to
clean
everything
up
and,
as
it
turns
out
when
in
blue
store,
when
I
got
him
to
a
wall
clock
profile.
A
The
SSDs
into
into
memory
of
all
the
data
and
I,
don't
know
how
much
was
thrashing,
but
I
suspect
it
was
thrashing
quite
a
bit,
but
that
that's
what
the
work
that
was
going
on
is
tons
and
tons
and
tons
of
little
8k
reads
so
I
had
him
enable
buffered
reads,
and
that
seemed
to
help
not
entirely.
It
was
still
doing
a
lot
of
work
and
some
of
the
threads
were
still
really
busy
doing
reads.
A
Not
all
of
them
were
anymore,
so
I,
I
kind
of
suspect
that
maybe
he
didn't
have
enough
buffer
cache
to
really
like
fully
catch
everything,
but
maybe
was
caching
some
things,
so
it
was
better,
but
look
how
the
really
the
problem
right
is
that
it's
doing
these
little
8k
reads
and
maybe
maybe
it
it
it
wants
to.
But
maybe
it's
really
just
doing
sequential
reads.
A
C
A
A
C
A
A
A
That's
a
really
simple,
PR
I
think
it's
just
looking.
If
something
is
sequential
and
then
if
he
sees
sequential
stuff,
then
yeah
I'm,
reading
two
sequential
layers,
yeah.
A
A
C
A
A
So
what
I'm
doing
now
is
modifying
some
of
that
in
rocks
DB
to
try
to
make
it
so
that
I
can
kind
of
control,
know,
monitor
and
control
that
more
in
a
more
flexible
manner
and
then
in
blue
store
I
can
piggyback
on
the
the
the
mempool
thread,
where
we're
already
flushing
the
cash
and
and
do
some
rebalancing
in
there
I'm
kind
of
a
periodic
basis.
So
you
know-
maybe
maybe
this
is
it
once
it
returns
seconds,
maybe
once
every
60
seconds
I,
don't
know
we'll
see
how
how
much
overhead
it
is.
A
A
I
think
it's
going
to
be
a
lot
better
if
we
can
kind
of
get
into
that
mindset,
so
I'm
gonna
try
to
have
that
at
least
a
prototype
for
that
sometime,
the
next
week
or
two
here
and
then
try
to
get
whatever
changes
in
rocks.
Tbh
I
need
submitted
upstream
and
between
those
this
new
adaptive
thing
adaptive
read
ahead
and
then
Radda
Slav
also
has
a
PR
for.
Oh
sorry,
I
already
forgot
what
it
was
that
you've
made.
B
A
A
Then
I'm
gonna
try
to
start
making
decisions
based
on
it,
okay,
but
the
good
news
is
that
I
can
actually
get
all
these
stats
now,
whereas
before
we
didn't
really
have
anything
so
so
now,
I'm,
actually
you
know
I'm
running
different
workloads
and
actually
watching
the
the
high
priority
cache
items
and
racks
DB
increase
and
sometimes
like
spill
over
into
the
non-priority
pool.
But
you
can
you
can
tweak
that
kind
of
stuff
like
how
much
space
you
want
each?
So
you
know
partes
will
be
just
kind
of
trying
to
figure
out.
You
know
what.
A
Yeah,
so
not
not
a
whole
lot
yet
there,
but
but
it's
it's
kind
of
the
starch
and
I
I.
Think
from
what
I'm
seeing
so
far,
it
all
seems
like
it's.
It
kind
of
makes
sense
and
I'm
hoping
it
will
well
we'll
have
a
nice
graph
showing
that
the
the
caches
are
rebalancing
and
the
hit
rates
are
better
looking
for
it.
Beth.
C
B
Maybe
just
a
quick
note
note
on
the
mutex
implementation
from
from
GBC:
basically
it
doesn't,
it
doesn't
try
to
it
doesn't
try
to
spin
even
for
a
second.
All
it
does
is
to
check
the
atomic
the
atomic
value
inside
and
and
if,
if
it's,
if
it's
locked,
then
just
go
to
Cardinal
I
created
as
a
part
of
the
abstract
early
work,
I
put
a
small
thing
called
adopt,
adopt
grad
I,
guess
the
better
name
would
be
trying
guard
or
something
like
that.