►
From YouTube: Ceph Performance Meeting 2020-08-13
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
So
open
all
right,
not
not
a
whole
lot
new
stuff.
Here
there
were
two
new
prs
that
I
saw
this
week.
One
was
for
me
that
was
a
result
of
the
collaborative
work
that
that
radek
and
I
did
looking
at
trying
to
make
bufferless
depends
faster
and
we,
we
kind
of
ended
up
with
two
complementary
changes
that
I
think
are
both
worth
doing,
but
this
is
only
the
first
one,
which
is
a
smaller
change.
A
This
basically
changes
it
so
that
in
refill
append
space
we
dynamically
change
the
length
of
the
append
buffer
that
we're
adding
based
on
how
much
we,
how
big
the
old
one
was.
So
basically,
this
is
kind
of
like
in
c
plus
plus,
where
vectors
grow.
They
double
each
time,
that's
more
or
less
the
same
thing
we're
doing
here.
A
We
do
bound
it,
but
but
the
the
end
result
is
that,
hopefully
we're
we're
doing
that
much
less
often,
if
we
have
really
big
buffers
that
have
a
lot
of
depends
happening
and
it
seems
to
help
fairly
significantly.
The
only
interesting
thing
is
that
tc
malik
actually
ends
up
slowing
down
at
some
point
when
you
have
larger
append
sizes-
and
I
don't
know
why
exactly
it
changes
depending
on
how
we
tweak
different
things,
but
it
it
consistently.
A
We
seem
to
hit
it
sooner
or
later,
so
it
overall,
it
still
seems
to
be
better
than
what
we
have
in
master,
though
far
better,
especially
for
pen
hole
with
the
ring
buffer.
It's
even
better.
Yet,
especially
when
we
stay
within
the
ring
buffer
and
don't
kind
of
go
outside
of
it,
we
can,
if
we
have
too
many
buffer
lists
at
once,
build
memory,
but
but
it
overall.
The
combination
of
the
two,
at
least
according
to
our
micro
benchmarks,
looks
really
good
in
practice.
A
I
see
a
lot
less
tc
malik
overhead,
but
it's
not
really
speeding
things
up
yet,
as
far
as
I
can
tell
at
least
in
our
test
case
with
the
mds,
so
anyway,
that's
kind
of
where
that
is
at.
I
guess
sorry.
I
just
went
over
the
whole
thing,
but
that's
that
any
any
questions
or
thoughts
on
that
pr.
A
So
this
may
actually
help
if
you
guys
have
any
cases
where
you're
doing
like
a
pen
hole,
and
you
know
lots
of
little
pens.
I
don't
know
if
you
do,
but
that
was
what
we
were
noticing
in
the
mbs
is
that
it
was
doing
a
lot
of.
A
A
A
All
right,
well,
next,
pr
that
I've
got
for
new
pr's
is
one
from
yan,
and
this
is
shoring
up.
Some
of
the
ephemeral
pinning
work
that
just
went
in
a
little
while
ago.
The
idea
here
is
to
distributor
franks,
so
that's
kind
of
like
what
I
was
doing
with
the
the
pre-sharding
and
pre-distribution
of
different
eggs,
but
they're
doing
it
for
ephemeral,
pinning
and
theoretically.
A
I
guess
the
idea
here
is
that
we're
going
to
reduce
the
number
of
sub-trees
because
of
it
so
anyway
need
to
see
how
that
works,
see
if
it
works
well
or
not,
and
that
was
it
for
new
pr's
updated.
A
I
think
adam
and
I
both
that
we
should
use
the
majin
peng's
pr
to
avoid
flushing
too
much
data
at
once
in
blue
rock's
environment.
That
means
qa,
hopefully
kifu
can
add
it
to
his
qa
suite
and
then
this
d3m
caching
thing
that
got
reviewed
by
a
couple
of
people
and
then
has
had
updates
based
on
that
review.
That's
still
in
in
review
and
we'll
be
in
testing.
Assuming
everything
looks
good
and
then
radic
is
still.
A
I
think,
looking
at
majin
peng's
reduced
buffer
list
rebuild
pr
in
blue
fs,
trying
to
do
that
more
generically
than
just
in
the
bluefs
code.
So
he's
working
on
that,
though
I
believe-
and
that
was
it
for
updated
prs.
A
C
All
right,
so
let
me
just
quickly
first
link
what
I'm
talking
about
in
the
chat.
So
people
can
take
a
look.
So
this
week
I
came
across
a
paper
published
at
the
fast
conference
like
five
months
ago.
I
think
back
in
march,
and
this
paper
talks
about
sort
of
like
an
expansion
to
the
crash
algorithm.
C
So
first
they
they
discuss
some
weak
points
of
crush
in
the
case
when
there's
a
cluster
expansion
and
then
there's
a
lot
of
data
migration
happening,
and
that
was
basically
that
their
motivation
to
to
making
that
extension
to
to
crush
and
the
idea
is
making
sort
of
like
a
centralized
authority
responsible
in
addition
responsible
for
for
the
data
migration
in
the
cluster
expansions.
C
So
those
were
kind
of
like
that.
That
was
the
main
idea
and
they
were.
They
introduced
a
number
of
ways
to
to
handle
the
load
balancing
and
at
the
end
they
talk
about
some
of
the
results
they
got
and
it
seems
like
the
the
algorithm
they
introduced
is
achieving
much
better
results
than
then
crush.
They
were
performing
the
test
on
on
on
cfs
and
and
rbd.
C
So
I
saw
I
I
saw
it
like
two
days
ago
so
they're
still,
I
I
still
need
to
go
over
it
more,
but
I
kind
of
wanted
to
use
this
meeting
to
to
show
it
to
you
guys
and
and
give
you
time
to
go
over
it.
If
you
want
to
and
tell
me
what
you
what
you
think
about
it.
D
I
I
hadn't
seen
this
before
myself
and
it
looks
very
interesting
I'll
have
to
like
read
the
paper
to
kind
of
understand.
What's
going
on
here,
if
you've
already
looked
at
it,
a
bit
is
like
the
main
idea
that
they've
kind
of
reduced
the
amount
of
data
being
migrated
or
that
they're
spreading
the
migration
over
a
longer
period
of
time
or.
C
Yes,
so
so
the
idea
is
they
are,
are
they
are
reducing
the
the
amount
of
data
migrated
because
they
are
adding
another
virtual
sort
of
layer
under
root?
You
can
see
there.
There's
the.
There
are
multiple
figures
in
the
paper,
so
they're,
adding
a
virtual,
a
virtual
layer,
so
they're
looking
at
the
different
shelves
and
cabinets
as
as
layers.
C
A
C
Only
goes
to
the
new
layer,
and
then
they
are
discussing
how
how
they
are
handling
the
load
balancing,
because
because
there's
sort
of
a
trade-off
there
and
they're
presenting
three
three
methods,
namely
pg
remapping,
cluster,
shrinking
and
layer
merging.
So
yeah.
D
Than
all
data,
to
avoid
moving
too
much
around.
C
Yeah,
so
they
are
so
yeah
exactly
so
they
are
using
this.
I
have
it
here,
one
second
they're,
basically,
timing:
they're,
adding
like
timestamps
they're,
calculating
the
time
that
the
data
is
coming
in
so
that
they
that
they
can
distinguish
between
all
data
and
new
data.
So
when
you're
doing
the
expansion.
A
A
That
new
data
is
always
landing
on
like
osds,
that
are
are
that
they
won't
have
to
go
back
and
then
do
like
future
migration
for
or
is
it
how?
How
is
it?
I
guess
I'm
still
trying
to
understand
how
it
how
what
are
the
bounds
on
data
migration,
like
for
new
data
and
old
data,.
C
Yeah,
so
they
did
they.
I
I
didn't
go
into
specifics
of
like
their
exact
calculation
of
what
did
they
determine
as
as
old
data
and
new
data.
I
kind
of
went
over
like
the
the
main
idea
and
the
results.
C
So
the
main
idea
was,
like
I
said,
doing
those
those
timestamps
and
adding
a
a
virtual
layer
under
the
root
kind
of
grouping,
all
the
the
the
cabinets
and
shelves
in
in
a
virtual
layer
so
that
when
you
expand,
for
example,
you
add
another-
you
had
another
shelf,
then
it
only
affects
that
virtual
layer
and
it
doesn't
affect
the
weights
of
the
other
layers.
So
then
crush
doesn't
do
the
the
load
balancing
or
the
map
x.
C
Yeah
yeah
there's
definitely
a
lot,
there's,
definitely
a
lot
to
read.
I
myself,
like
I
said
I
I
came
across
that
just
two
days
ago,
so
I
thought
this
will
be
a
good
forum
to
to
bring
that
up
to
give
everybody
time
to
everybody's
interested,
give
them
time
to
go
over
it.
I'll.
Definitely
look
further
into
that
as
well
before
next
week's
meeting
and
then
next
week
we
can
all
kind
of
have
a
better
idea
of
of
the
entire
paper,
and
we
can
discuss
it
further.
A
B
D
D
When
we
describe
things,
I
mean
it's
certainly
controlled
in
some
sense
like
that,
there's
a
bounded
amount
of
data
being
moved
and
you
could
there
is
control
over
how
fast
it
moves
outside
of
crush
itself.
A
D
It
sounds
like
they're
kind
of
describing
it
being
uncontrolled
in
the
sense
that
it's
not
directly
controlled
just
by
crush.
C
Yeah
they're
saying
that
it's
basically
like
crushes
using
like
a
decentralized
placement
methods
and
they
are
sort
of
adding
like
a
another
centralized
authority
once
cluster
expansions
are
happening
and
they
are
showing
that
you
know
once
you
add
that
centralized
authority,
you
essentially
you
don't
hit,
you
don't
hurt
the
load
balancing,
so
that's
good,
but
at
the
same
time
you're
also
getting
better
performance,
because
data
is
not
migrating
over
between
the
different
layers.
C
D
Yeah,
I
think
I'll
have
to
read
more
to
understand
the
tradeoffs
involved
like
if
it
has
potential
implications
for
the
failure,
domains
or
kind
of
the
balancing
of
a
via
load
as
well
yeah.
Definitely
that's
that's
the
reason
why.
C
I
thought
I'd
bring
that
up
here.
A
And
and
when
you
were
reading
through
this,
did
it
sound
like
for
new
objects?
Do
we
end
up
what
happens
to
new
objects
like
it,
while
you're
in
a
degraded
or
recovery
scenario,
what
what
is
the
effect
on
new
objects
when
they
come
in
with
this
scheme
versus
traditional
crush?
C
C
Okay,
okay,
but
I
think
I
think
that
in
section
3.2,
when
they
are
talking
about
migration
control
and
they're
kind
of
introducing
the
three,
the
three
method
methods
they
are
using
to
to
address
the
potential
load
imbalance
between
the
layers.
I
think
that
the
answer
to
your
question
might
reside
there,
I'm
pretty
sure.
Okay,
I
just
didn't,
have
enough
time
to
to
really
go
into
detail.
C
But
I
just
yeah,
I
thought
it
it.
The
results
were
pretty
surprising.
That's
why
I
thought
it
would
be
pretty
interesting
to
to
have
a
look
at
that
because
it
seems
like
just
the
results
of
significantly
better
in
terms
of
of
the
iops
they
are
getting
and
the
latency
yeah.
C
Yeah
yeah
in
section
four,
you
have
evaluation
and
they
are
describing
both
the
machine
they
are
using,
the
they
are
describing
the
cluster.
The
storage
devices
stuff
like
that
and
they're
saying
the
ceph
version,
which
is
a
luminous
yeah.
A
A
C
Matter
yeah,
the
thing
is
that
the
way
they
described
it,
it
seems
like
implementing
their
algorithm
was
pretty
easy.
So
so,
in
the
case
that
you
know
they
didn't
run
into
ssds,
maybe
we
can
test
it
ourselves
and
and
see
how
it
how
it
performs
this.
D
Thing
is
just
kind
of
the
the
impact
they're
trying
to
reduce
on
flying
tio
or
left
something
with
the
work
quality
of
service.
Where
we're
balancing
background
io
versus
client
io
as
well,
and
we.
D
Wondering
how
much
of
that
of
the
kind
of
impact
of
this
would
would
already
be
captured
by
a
better
quality
of
service.
C
Yeah
I
get
what
you're
saying
so
that
that
that's
why?
I
think
that
that
it's
it's
worth
to
to
go
into
more
detail
like
read
through
it
again
and
maybe
even
try
to
to
recreate
their
their
tests
on
on
different
hardware
and
see
and
kind
of
see
the
results.
I
don't
think
it
should
be
that
complicated,
because
they
didn't
completely
change
crush
they
sort
of
like
just
extended
it.
C
Yeah
me
personally,
no
not
really:
okay,
okay,.
A
C
Okay,
so
I
guess
we
should.
We
should
discuss
this
further
like
next
week,
once
everybody
or
I
don't
know
in
two
weeks,
once
everybody
has
everybody
in
the
call
has
some
time
to
to
go
over
it
if
they
want
to
so
then
we
will
all
have
a
better
sense
of
of
exactly
what
what
is
going
on
in
the
paper
and
see
if
there
are
any
like
potential
issues
there.
D
Yeah
sounds
great.
Thank
you
yeah.
Actually,
I
certainly
will
take
a
look
and
try
to
get
a
better
understanding
of
the
tradeoffs
involved
with
data
placement
like
this
there's
always
tradeoffs
yeah.
A
Sounds
good,
so
I
mean,
I
guess
one
one
thing
here
right
would
be
that
they
they
talk
about
performance,
but
really
what
we
want
to
know.
At
least
what
I
want
to
know
is
with
like
this
migration
free
expansion
and
avoiding
migration
in
general.
You
know
what
how
much?
What's
the
reduced
amount
of
data
movement
right?
Because
performance,
I
don't
really
care
about-
I
mean
that's
kind
of
arbitrary,
but
but
what's
the
the
real
effect
on
data
movement
in
the
cluster?
That's
what
I
really
want.
C
I
see
here
a
figure
talking
about
the
the
number
of
affected
pgs
yeah
number
of
like
the
pgs
in
layer
merging.
So
after
after
four
expansions
of
the
cluster.
C
I
don't
see
here
anything
specific
to
to
the
percentage
of
of
data
migrated,
but
they
might
have
like
just
you
know,
wrote
that
and
didn't
add
like
a
a
graph
or
something
it
might
be
discussed
in
one
of
the
sections.
So.
A
It's
just
a
really.
It
surprises
me
that
they
don't
have
a
figure
showing
the
reduced
data
migration,
given
that
that's
like
the
whole.
As
far
as
I
understand
it,
the
fundamental
benefit
of
this
is
that
I
don't
care
about
latency
or
I
have
in
the
you
know
grand
scheme
of
things
that
much
for
this
specific
thing
that.
C
From
the
beginning
of
the
paper
was
that
they
are
more
interested
in
the
effect
that
reduced
data
migration
would
have
on
on.
You
know
stuff
like
iops
and
latency,
so
that
I'm
assuming.
That
is
why
the
figures
they
supplied
were
mainly
discussing
about
those
sort
of
things:
iops,
latency,
yeah
and
all
the
rest
of
the
things
we
have
here.
A
Yeah,
I
mean
that's
all,
that's
all
kind
of
you
know
nice,
but
that
that
you
know
comes
as
a
result,
hopefully
of
the
reduced
date
of
migration
and
then.
C
A
Yeah,
the
the
big
reason
why
I
like
that,
potentially
more
even
than
iops
and
and
throughput,
and
all
these
other
benefits
latency,
is
that
especially
on
nvme
drives,
you
have
a
reduced
amount
of
right
endurance
right.
You
have
a
limited
number
of
amounts
of
right
endurance,
and
so,
if,
during
healing
events,
you
can
reduce
that
you
can
make
your
your
nvme
drives
live
longer.
C
Yeah
exactly
I
guess
I
will
try
to
you
know
it's
hard
to
me
to
believe
that
they
didn't
supply
even
like
some
sort
of
number
of
of
the
reduced
migration.
You
know.
Maybe
they
didn't
dedicate
a
a
full
figure
for
that,
but
I
do
believe
that
they
somewhere
mentioned
they
reduced
migration,
but
that
is
just
me,
hoping
I'll
have
to
I'll
have
to
to
search
for
that.
A
C
Yeah,
if
you
take
a
look
at
figure
number
two
before
they
even
discuss
map
packs,
they
are.
I
think
this
shows
like
the
the
data
migration
in
in
crush.
It's
yeah
figure
number
two.
A
C
A
C
Yeah
they
discuss
it
in
the
paper.
I
think
I
know
what
you're
talking
about.
It's
called
there's
a
flat
or
there's
a
variable
called.
Oh,
it's
the
max
the
rsds
max
backfill.
I
think
I'm
trying
to
to
grab
that
I'm
pretty
sure
they.
C
They
mentioned
that
that
ceph
has
some
sort
of
variable,
limiting
the
migration
happening
and,
and
they
do
address
that
in
their
experiments
actually,
and
they
show
that
that
even
with
that
sort
of
optimization,
their
algorithm
still
outperforms
once
again
in
terms
of
iops
and
latency.
D
Yeah
that
max
bag
tells
us
about
the
rate
of
recovery,
not
about
the
amount
of
data
moved.
I
think
our
mark.
What
you're
talking
about
may
be
more
related
to
kind
of
the
modular
stuff
and
crush
to
try
to
limit
migration
to
like
half
a
subtree.
A
Yeah
yeah
exactly
it's,
it's
been
a
while
a
long
time
since
I
really
looked
at
this,
but
when
I
was
trying
to
make
like
a
a
halton
sequence
based
distribution,
algorithm
that
all
all
it
ultimately
just
didn't
work.
It
was
one
of
the
big
problems
with
it
is
the
amount
of
data
migration.
D
A
D
Have
a
more
of
an
impact
on
a
smaller
cluster
for
sure,
just
because
you
are
usually
changing
a
larger
percentage
of
the
cluster
at
a
time.
A
D
C
I'm
pretty
sure,
let
me
take
a
look.
I
think
it's
at
the
end
of
the
paper.
C
Yeah
yeah
there
is
the
figure
number
eight
yeah
in
page
seven
yeah.
It
talks
about
the
different
different
experiments
and
then.
A
A
A
Good
well,
what
I
really
want
is:
I
want
that
result
from
figure
two,
adding
one
rack,
80
osds
to
240
osds.
I
want
to
see
the
crush
result
and
then
I
want
to
see
the
map
x
result.
That's
that's!
Basically
what
I
want
yeah
okay,
so
I'm
trying
to
figure
out
like
in
in
figure
eight
now
what
exactly
that
means.
C
You're
gonna
have
to
read
through
where
they
discuss
about
player
merging
in
section
number
three
3.2
to
be
exact.
To
really,
I
guess,
understand
what
they're,
comparing
there.
A
D
A
D
A
A
I'm
open,
I'm
not
what
do
you?
What
do
you
think
would
you
be
interested
in?
That
would
be
useful
for
you.
C
Yeah,
it
would
be
really
useful
because
the
thing
is
in
the
conference
they
actually,
they
were
supposed
to
also
present
their
work
in
the
conference,
but
they
couldn't
they
couldn't
make
it
because
of
coronavirus.
So
it
was
just
somebody
else,
just
reading
their
slides,
so
you
couldn't
really
understand
anything
because
he
was
just
reading
the
slides,
so
I
think
they
would.
I
think
they
would
also
be
interested
in
presenting
that
I
know
I
would
definitely
so
I
mean
I
can
contact
them
or
josh.
A
Yeah,
it
doesn't
matter
to
me
either
if,
whichever
you
guys
want
to
do
just,
let
me
know
and
we'll
we'll
set
up
a
time
slot
on
one
of
the
performance
meetings
for
it.
C
Okay
sounds
good
and
then
we
between
us,
we
can
talk
about
it
more
next
week.
I
think.
A
All
right,
we've
got
a
lot
of
latecomers
from
the
course
stand
up
here,
any
any
thoughts
or
questions
folks.
A
C
Link,
I
just
attached,
has
well
it
has
a
link
to
the
paper
itself
and
also
to
the
presentation
video
from
the
conference.
If
anyone
is
interested,
but
once
again
like
I
said,
the
person
presenting
is
he's
just
like
reading
the
slides,
because
he's
not
one
of
the
one
of
the
people
who
worked
on
the
paper
because
they
couldn't
make
it
to
the
conference,
though
okay,
okay,
but
if
you
want
more
slides
there,
you.
A
A
Guys,
if
not
then
have
a
good
week
and
looking
forward
to
discussing
this
more
next
week,
have
a
good
one
have
a
good
week.