►
From YouTube: 2018-08-23 Ceph Performance Weekly
Description
Weekly collaboration call of all community members working on Ceph performance. http://ceph.com/performance
B
C
B
C
B
A
C
C
B
C
Right
well,
I
think
we
have
our
crowd
for
people
today,
but
it's
not
much
going
on
because
sage
has
been
gone,
so
I
think
it's
fine.
Maybe
we
get
started
so
the
first
new
PR
and
the
list
Radek
is
your
PR.
That
was
just
this
morning
for
the
perfect
counter
work.
I
think
you
had
mentioned
and
stand
up
that
there
is
some
additional
things
that
need
to
be
done.
Yes,.
B
C
C
B
C
B
B
With
instances
of
such
structures
of
such
structure
that
are
made
context,
it
means
it
can
be
used
as
a
unique
identifier,
but
it
comes
to
putting
those
single
perf
counters
into
a
group
health
counters
T,
which
is
just
something
very,
very
mm
similar.
What
we
have
before
the
camel
cased
health
councils
class.
B
Guess
the
most
the
thing
that
I'm
most
satisfied
with
it's
not
the
performance
is
that
all
those
validations
we
have
we're
counters
and
have
been
moved
to
compile
time,
though.
Okay,
if
you,
if
you
are,
if
you
are
going
to
misuse
method
for
for
increment
of
a
counter,
but
it's
not
compliant
with
the
counter
type,
then
you
will
compiler
will
tell
you
about
that
excellent
compiler,
not
a
runner
time
time.
C
B
But
there
are
also
some
restrictions
you
need
to
know
about.
You
need
to
know
about
the
met.
You
need
to
know
the
metadata
of
per
counter
at
compile
time
that
stage
in
chrome,
in
contrast
to
what
we
had
before.
However,
personally
I'm
not
entirely
sure
whether
we
really
need
to
have
the
feature
of
declaring
new
counters
dynamically
I.
C
Do
use
that
sort
of
right
now
all
know.
What
well
is
it
that
the
the
grouping
of
counters
also
can't
be
dynamically
declared,
or
is
it
like
just
that
you
can
so
so
right
now,
I
did
change
my
stuff
around
a
little
bit
based
on
what
you
had
said
earlier,
so
that
I'm
I'm
not
totally
dynamically
creating
counters,
but
I
do
have
different
groups
of
counters
that
get
kind
of
created.
C
B
Can
have
multiple
numbers
of
instance
of
country?
Scrapping
it's
not
absolutely
a
problem.
All
you
need
to
do
is
to
have
a
knowledge
about
how
any
group
of
their
counters
should
look
like
you
need
to
that
in
compiled
time.
It's
because
you
are
dependent.
The
perf
counters
group
is
basically
a
template
over
over
a
set
of
metadata
config
like
that,
okay.
B
C
C
So
we
move
the
the
priority
cache
interface
to
that
thing,
rather
than
the
key
value
database,
which
was
kind
of
crazy
having
it
there
anyway,
and
then,
on
top
of
all
that,
we
we
start
building
in
the
bindle.
Are
you
cache
a
new
priority,
cache
manager
that
utilizes,
that
cetera,
et
cetera
the
good
news
with
it?
Is
that
it
it
does
kind
of
seemingly
what
we
want?
C
I
think
it
does
a
good
job
of
kind
of
rebalancing
caches
based
on
the
use
case,
if
there's
or
megabyte
sequential,
writes
coming
in,
it
really
starts
favoring
data
cache.
If
there
are
4k
random,
writes
coming
in
you'll,
see
either
the
metadata
cache
or
the
kV
cache
get
the
lion's
share
of
the
available
memory.
Depending
on
whether
or
not
you
have
enough
cash
to
cash
all
of
the
metadata
in
the
medic,
so
it
the
the
behavior,
looks
like
it's
kind
of
seemingly
what
we
want
in
terms
of
performance.
C
C
It
could
be
that
maybe
no,
this
really
mattered
that
much
but
I
kind
of
doubt
that,
just
based
on
what
we've
seen
in
the
past
we've
seen
a
lot
of
cases
in
the
past.
We're
changing
these
KB
value
or
the
the
ratios
of
cash
do
have
an
effect,
so
I
think
it's
gonna
be
important
to
dig
into
it
a
little
bit
more.
C
One
point
is
that
I've
been
doing
all
the
testing
with
the
data
cache
enabled
and
there's
a
lot
of
overhead
in
the
mempool
thread
for
trimming
the
data
cache.
So
it
could
be
that
just
doing
all
that
trimming
in
the
mempool
thread
itself
is
kind
of
masking
anything
else.
So
anyway,
there's
there's
more
investigation
to
do,
but
I
think
the
behaviors
is
good.
C
The
only
question
regarding
behavior
is
whether
or
not
we
might
be
imposing
the
memory
allocator
overhead
by
kind
of
swapping
or
switching
from
one
kind
of
cache
type
to
another
kind
of
cache
type,
and
now
there's
more
fragmentation
or
something
because
of
it.
So
anyway,
that's
that's
out
there
I'm
still
running
tests
on
it.
Well,
we'll
see
where
it
goes.
C
Those
are
the
only
two
new
PRS
that
I
saw
this
week
for
performance.
There's,
you
know
other
stuff,
but
that
was
is
been
cast
low
in
terms
of
close
stuff.
There's
some
new
lock
lists
off
work
that
key
foo
did
I.
Think
Casey
reviewed
it
and
thought
it
looked
good.
I
know.
Sage
was
scan.
Look
at
that
too,
but
I
think
he's
been
out
this
week
in
Boston
or
something
so
I'm,
not
sure
he
has
a
chance
to
look
at
it.
C
There's
this
optimize
recover
order,
PRF
that
I
think
Neha
look
in
primary
log,
PG
I,
don't
know
how
that
actually
intersects
with
any
of
her
work.
But
anyway
she
she
look
at
I,
guess
it
must
look
good
because
it
got
merged
and
then
Jason
merged
have
lebar
BD
thing
that
I
haven't
looked
at
at
all
so
yeah.
That
was,
and
that
was
it
so
now
now
a
whole
lot
new
or
going
on
this
week.
But
you
know
people
are
pretty
busy.
C
It's
can
the
end
of
the
summer
and
and
we've
got
a
conference
coming
up
that
a
lot
of
people
are
going
to
so
perhaps
after
that
will
pick
up
a
little
bit.
So
I've
got
a
couple
of
things.
We
can't
already
talked
about
the
bindle
ru
cash
I
mean
try
to
present
a
little
bit
more
about
how
that
works
and
and
present
graphs,
and
that
kind
of
thing
at
some
point,
but
it's
not
ready
yet
so
I'll
just
skip
that
for
now
I.
C
Over
the
past
week,
while
I've
been
running
tests
on
that
stuff,
I
started
looking
at
replacing
the
standard
list
in
in
buffer
list
with
small
vector
it.
It
turned
out
that
radicand
already
done.
Basically
everything
I
was
already
doing,
and
there
was
a
issue
with
iterator
and
validation,
moving
away
from
standard
list
and
the
way
I
was
trying
to
solve.
It
was
kind
of
ridiculous
compared
to
what
Radek
had
already
done.
So
I
ended
up
just
adopting
radix
code
for
it
and
and
yeah
everything
works.
C
Fine
after
that,
oh,
when
I
did
that
the
results
I
saw
were
almost
always
just
a
little
bit
slower
than
the
master
again,
it
was
maybe
within
the
margin
of
noise,
but
it
was.
It
was
always
on
the
low
side
of
it,
whereas
master
was
always
on
the
high
side
of
it,
so
I
think
there's
probably
an
effect
there
and
I
think
graddic,
based
on
what
we
had
talked
about.
You
are
also
seeing
not
consistently
higher
performance
at
the
very
least
right.
Yes,
exactly.
B
I
will
sink
also
regression
at
Miraval.
It
was
whirring
with
with
multiple
things
testing
this
sturdy
podcaster
I
was
seeing
different
results
in
comparison
to
I
had
to
I
had
on
CB
t
CB
t
runs,
I
speculate
it
because
of
the
very
extensive
usage
of
buffer
list
in
semtex.
We
eradicated
that
so
maybe
retesting
might
be
a
way
to
go,
not
sure
how.
C
B
C
B
Was
also
funny
was
that
the
benchmarks
we
have
for
a
buffer
least
told
huge
improvement,
but
but
in
real
situations,
well,
I
will
sink.
Also
regressions.
It's
very,
very,
very
weird.
Behavior
also
I
think
that,
basically,
the
branch
I
was
testing
not
only
switching
to
a
small
vector,
but
also
improved
in
line
ability
of
the
buffer
list
before
we
for
demons
for
OSD
before
we
merged
before
we
switch
to
linking
itself
come
on
ethically
because
because
of
necessity
to
go
through
PLT
after
merging
yep,
quite
I
think
that
I
haven't
retested
incest
in.
B
The
early
months
of
this
year,
or
something
like
that,
it
was
half
a
year
ago-
something
like
that,
but
maybe
it's
worth
to
get
back
to
the
idea
at
least
to
understand.
What's
what
is
closing,
what
exactly
is
causing
the
regression?
We
see
I'm
I'm
poking
a
bit
with
with
the
branch
rebase
it
to
master
stripped
all
in
in
lining
related
stuff,
leaving
just
the
more
vector
transition
along
I
plant
to
give
it
to
give
some
profiling
to
the
branch.
Maybe
we'll
know
a
mark.
B
C
C
B
C
B
C
Yeah,
it
could
be
it
didn't.
Look
to
me
like
we
do
it
very
often
and
I
think
in
some
of
the
other
cases
that
that
looked
bad
for
small
vector
right,
where
you're
shifting
memory
around
or
whatever
I
I,
don't
think
we
do
it
horribly
regularly.
I
mean
we
should
probably
look
and
make
sure
but
it'd
be
it'd,
be
good
to
know
what
our
actual
usage
patterns
of
buffer
list
really
look
like
and
if
it
makes
sense
how
often
everything
that
we
do,
and
it
makes
sense,
makes
sense,
I
guess.
B
C
Wondering
too
a
little
bit:
I
need
to
go
back
and
look
at
the
profiling
results,
but
even
just
the
the
creation
and
deletion
of
small
vector
itself,
if
it's
more
higher
overhead,
potentially
than
standard
list
or
something
maybe
the
constant
time
overhead
is
actually
enough
to
make
it.
You
know,
look
bad.
B
C
B
Memory
management
pattern
you
mentioned
in
a
mempool
effort.
I
also
took
a
look
on
this
guy
a
bit
and
it
seems,
although
it's
not
per
se,
that
mr.
member
is
not
so
critical.
It
takes
around
10%
of
CPU.
However,
it
does
a
lot
of
a
lot
of
pls
costly
from
from
the
TC
malach
point
of
view,
mostly
because
of
the
traffic
that
needs
to
go
to
the
central
cash.
B
B
Touched
just
for
experimenting
whether
the
whether
is
whether
there
is
any
difference
or
not,
but
it's
basically
not
a
solution,
just
a
a
measure
to
verify
whether
it's
worth
digging
more
in
this
area
or
not
I
haven't
tested,
I
haven't
tested
the
pride,
the
branch
it
I
plan
to
do
that
as
soon
as
possible.
Okay,.
C
C
C
B
B
C
B
Then
the
problem
would
be
reduced,
just
the
slapped
allocations.
We
already
had
poor
requests,
bringing
support
for
that.
We
could
resurrect
that
not
a
problem
at
all,
but
if
the
size
is
constant
and
we
want
to
fight
fights
that
we've
introduced
by
introducing
a
memory
allocator,
we
would
and
I
am
afraid
we
would
end
with
our
own
implementation
of
malloc
and
free,
which
is.
C
Quite
possible
yeah
ultimately
comes
down
to
the
the
C
string.
Underneath
it
all
right,
I
mean
everything
else
is
kind
of.
You
know.
You've
got
some
number
of
pointers.
You've
got.
You
know
a
couple
of
other
data
types
in
there,
but
it's
all
pretty
straightforward.
It's
the
the
variable
length.
You
know
C
string,
underneath
it
all
yeah
or
a
number.
B
C
B
C
C
B
C
I
wonder
how
much
like
all
this
other
junk
is
polluting,
TC,
Malik's
space.
You
know,
I
remember
how
a
female
corpse
compared
to
J
mouth
Jay
Malc
tries
to
like
create
like
groupings
of
like
sized
objects
together
in
memory.
If
I
remember,
right
and
I
wonder
with
TC
malloc,
if
it
does
something
kind
of
similar
or
not
I
forget,
but
but
anyway,
I
just
I,
wonder
I've
got
to
think
that
the
more
of
that
kind
of
stuff
you
you
remove
from
from
TC
malloc.
C
B
C
C
C
Next
week,
ac
said
that
he
and
Adam,
and
maybe
Matt,
can
come
and
I
don't
know
if
it
will
I
guess,
maybe
not
next
week
as
everyone's
gonna
be
at
Mountain
point.
But
but
at
some
point
soon
in
the
future
he
said
that
they
could
come
and
talk
about
what
they
did
with
the
sea
star
buffer
implementation,
because
they
somehow
well
they've
got
something
there
that
they've
wrapped,
where
it's
it's.
C
Compatible
or
or
works
with
our
encode
and
decode
existing
encode
decode
stuff.
So
potentially,
maybe
that's
a
segue
into
that
conversation
of
what
does
the
sea
star
world
look
like?
Are
we?
Are
we
using
it
with
our
encode
decode
code,
and
maybe
that
will
help
us
and
help
inform
us
on
what
we
potentially
could
do
with
the
the
you
know,
existing
existing
OST.
C
Alright,
the
only
other
thing
I
was
gonna
say,
is
I,
am
starting
to
work
on
a
better
result,
parsing
code
for
CBT
or
actually
making
something
real,
rather
than
a
bunch
of
ox
Crips
that
do
it.
So,
hopefully
that
will
be
your
soonish,
alright,
any
anything
else
guys
for
this
week,
or
should
we
wrap
this
up
almost.
B
B
B
A
B
B
C
B
B
C
B
C
B
B
B
B
C
A
B
B
B
C
What
I,
don't
understand
is
why,
after
parted
has
completed,
why
is
it
that
we
we
can
end
up
in
this
situation?
Well,
how
is
it
that
part?
It
is
not
like
ensuring
that
everything
is
is
synced
before
it
returns,
because
that's
what
it
seemed
like
to
me
is:
that
was
an
issue
with
parted
rather
than
anything
else.
C
B
C
Be
maybe
that's
it,
but
I
swear
I've,
seen
it
where
I
haven't
even
have
a
part.
I
have
a
file
system
open
like
I've
closed
it
long
long
ago,
there's
nothing
open,
but
if
I
run
multiple
iterations
of
parted
over
and
over
again
I
can
hit
it
I
think
so
long
times,
I
looked,
but
but
I
I'm
not
sure
it's
the
file
system,
it
might
be,
but
I
I,
don't
remember:
I
haven't
looked
at
it
in
like
a
year.
Well,.
C
B
C
B
B
C
C
B
C
B
Sent
an
email
ever
asking
about
about
real
usage
pattern
of
those
of
those,
the
bugs
and
basically
from
my
from
my
understanding,
people
could
live
without
without
that
without
the
default.
We
currently
have
sure
I
won
your
gamma
to
go
ahead,
and
many
of
them
were
surprised
that
settings
could
drink
my
drink.
Such
such
results,
provement,
yes,.
C
C
B
B
We
will
definitely
yes
I
think
so
we
will
definitely
remove
the
overhead
coming
from
get
name
info.
It's
the
dominant
guy
for
each
of
these
de
box.
Second,
one
is
mos
deep
print
pacifying
this
drink
is
slightly
more
costly,
I'm
afraid.
The
question
is
how
much
because
we
would
have
a
trade
off.
You
would
just
copy
more
to
avoid
some
some
processing
of
locks,
but
I.
B
Yeah
I'm
still
afraid
that
we
are
putting
a
lot
of
complexity
just
to
just
for
the
just
for
the
box
to
be
honest,
I
would
I
would
prefer.
I
would
really
love
to
go
with
the
absolutely
simplest,
the
very
very
damp
solution,
which
is
just
moving
the
stuff
from
from
the
hot
bath
moving
them
all,
and
they
maybe
we
wouldn't
need
to
be.
We
don't
need
to
mess
with
the
back
a
mess
at
all,
maybe
just
bumping.
B
These
levels
of
this
reader
box
is
a
way
to
go
because
well,
most
of
I
got
a
feeling
they
are
just
to
make
like
info
level,
not
not
something
that
something
critical,
not
something
that
signalizes
a
problem
issue.
Something
wait.
No
it's
a
it's!
Those
are
common
messages
in
for
messages,
I
would
say:
yeah.
C
B
C
B
I
guess
we
already
discussed
that
I
correctly.
The
most
important
thing
was
just
to
not
confuse,
do
not
surprise
operators
and
we
are
going
to
have
other
changes
in
messenger.
That
required
just
require
some
attention
from
operator
like
reading
stuff
about
the
new
connect
option.
Maybe
we
could
reuse
that
opportunity
to
also
signal
eyes
that
we
changed
I'm
thinking
our
locks
in
our
lock
priorities.
B
B
A
B
B
C
You've
convinced
me
that
just
get
them
out
of
the
hot
path,
and
then
you
know
if
it.
If,
if
it
turns
out
that
was
the
wrong
decision,
we
can
always
put
them
back
in
it's
not
the
end
of
the
world.
If
there
are
problems
on
someone's
cluster,
they
can
always
try
to
restart
with
those
things
changed.
Oh
I
think
I,
think
you're,
right,
I,
think
you're
ma
the
hot
path.
I
think
that's
the
right
way
to
go.