►
From YouTube: Ceph Crimson/SeaStor OSD 2020-10-28
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
Let's
start
last
week
I've
been
reviewing
johannes,
pierre,
introducing
interpretable
future
and
hopefully
I
will
wrap
up
the
review
and
also
I
was
discussing
the.
A
A
When
serving
them,
we
have,
we
have
the
pr
sorry
that
the
pg
being
target
experienced
some
some
some
changes
in
the
acting
sets
per
person.
First,
discussion
with
them.
We
should
study
them
from
scratch
by
working
on
it
and
that's
it.
B
Hi,
so
I
just
I
made
some
changes
to
the
to
the
pr
I
haven't
updated
like
I
haven't,
pushed
the
changes
actually
yet,
but
I've
made
the
the
changes
according
to
our
discussion,
careful
and
now
I'm
just
trying
to
build
it.
B
C
B
Apparently
there
aren't
any,
we
don't
have
any
tests
available
for
that
at
the
moment,
so
we
might
just
have
to
run
it
on
the
entire
trash
test
and
yeah.
So
it's
been
working
on
that
and
another
update
that
starting
from
tomorrow.
So
tomorrow
I'm
going
to
canada
so
I'll
be
in
a
different,
a
time
zone
for
a
while
until
mid-december.
A
Okay,
like
are
you
going
to
attend
the
the
stand
up
when,
while
you
are
in
canada,.
B
Yes,
I'm
still
it's
all
the
same.
I'm
still
like
working
and
studying
at
the
same
schedule,
just
like
different
different
times,
so
I'll,
probably
either
attend
like
I'll,
probably
attend
the
meetings
on
tuesday.
I.
B
C
The
unit
for
the
omega-3,
the
next
step,
I
will
clean
up
code
rebase
and
the
provided
pr
for
review
and
when
doing
the
only
trade
in
test,
I
found
a
bag
in
the
in
the
kitchen
country
the
same
in
the
omatrix,
so
fix
the
bag
and
provide
a
reply
test
case
and
also
there
is
missing
in
the
in
the
allocate
exchange
by
by
type
for
omap3,
for
instance,
extend
three
node
node
tab,
so
I've
already
provided
the
updated
pr.
C
C
C
Hip
overflow,
something
happened,
so
I'm
not
sure.
What's
the
problem
still
thinking
debug,
I'm
late,
so
maybe
ask
sam
to
help
me.
Take
a
look.
C
Can
produce
when,
when
I
add
a
very
big
number
of
extent,
my
ping
and
remove
remove
one
by
one
and
after
remove
about
over
1
000,
and
it
has
a
problem
in
the
airbn3
lookup
range.
They
they
reported.
The
hip
is
overflow
or
something
like
that.
I'm
not
sure
if
it
is
my
code
is
your
own
ability
issue,
but
it
very
big
external
map
tree
in
there,
because
I
want
to
tr
trigger
the
in
internal
node
as
blade
and
the
merge.
C
So
because
this
insert
key
and
the
weather
is
too
small
under
the
atom's
interest
is
very
big,
so
I
need
a
very
big
insert
and
to
trigger
into
the
node
slate
and
emerge.
Internal
split.
Merge
has
happened
before
the
1000
remove
k,
but
after
the
1000
remove
k-
and
I
mean
some
problem-
I
couldn't
remove
all
the
other
items
and
if
there
is
a
problem
happened,
but
if
I
just
remove
1000,
it
is
okay
and
all
the
internal
node
merged
and
the
balanced
and
explained
happened
before
we
fight.
D
I
am
okay,
so
it
I
decided
the
easiest
way
to
write
a
good
performance.
Tester
for
different
elements
of
c
store
was
to
implement
an
nbd
server
with
a
back
end
that
could
talk
to
either
the
transaction
manager
or
c-store.
Once
it's
more
complete.
D
The
problem
is,
I
wanted
to
create
an
fio
back
end
that
could
talk
to
something
in
c-star,
but
that
turns
out
to
be
basically
impossible
because
you'd
have
to
write
an
inner
process
queue
and,
fortunately,
for
us,
there's
already
a
protocol
that
does
that
which
is
the
network
block
device.
So
I
wrote
a
simple
nbd
server
like
extremely
simple,
that
just
hands
off
the
relevant
ios
to
transaction
manager,
because
you
know
block
offsets,
look
a
lot
like
lbas,
so
it
just
hands
off
the
same.
D
D
B
D
Have
been
some
discussions
regarding
how
to
accommodate
nvme
over
fabric
in
rados
going
forward,
so
I've
been
gaming
out
some
different
strategies
for
changing
the
raido's
protocol
itself
to
permit
clients
who
do
more
direct
hardware,
assisted
io.
D
D
B
Okay,
I
I'm
just
confirmed
that
you
are
using
nvd
server
instead
of
ipod
fl
to
test
system
right.
D
I'm
using
fio
to
talk
to
an
nvd
server
and
the
fio
has
an
nvd
backend
that'll
just
behave
as
a
normal
network
block
device
client.
That
seems
like
the
easiest
way
to
get
fio
to
talk
to
other
stuff.
B
Linking
oh
backhand
to
talk
to
mvd
server
that
is
running
the
store
exactly.
D
And
then
we'll
just
write
multiple
interface
implementations
as
we
get
further
along
like
I'll
write,
one
at
the
transaction
manager
layer
and
then
we'll
write
another
one
that
talks
to
c
sort
itself.
So
the
transaction
manager,
one
of
course
bypasses
the
oh
node
stuff
and
then
c
store
as
a
whole
would
also
test
the
extent
manager,
onode
and
omap
right.
So
we
can
get
a
better
sense
for
where
the
bottlenecks
are.
B
D
D
There
are
two
processes:
there's
the
fio
process,
which
is
perfectly
ordinary
fio,
like
you'd,
find
in
any
modern
linux
distribution.
If
you
just
give
it
a
unix
domain
socket
and
the
nvd
protocol
it
just
opens.
You
know
a
connection
over
that
unix
domain
socket
and
talks
nvd
over
that
the
convey
block
commands
to
the
other
side,
which
in
this
case
is
a
fairly
short
handwritten
server
in
c
star.
That
just
speaks
the
simplest
version
of
the
nbd
protocol.
I
could
come
up
with.
D
I
downgraded
it
to
the
old
server,
so
I
didn't
have
to
worry
about
the
options,
whatever
it's
very
simple,
not
a
lot
to
it.
There's
already
an
implementation
for
rpd
in
the
seftree,
though
it
was
partly
a
matter
of
translating
that,
although
I
couldn't
use
the
linux
layer
which
is
annoying
but
whatever,
because
it's
synchronous
it
accepts,
commands
over
unix
demand,
socket
sends
them
to
c
store
and
then
sends
the
response
back.
When
it's
done,
there's
not
a
lot
going
on
here.
D
A
D
There
are
reasons
why
we
don't
want
to
use
why
we
don't
want
to
use
that
approach
for
one
thing,
if
you've
ever
used
it,
it's
a
giant
pain
in
the
ass,
it's
really
prone
to
crashing
fio,
because
fao's
memory
handling
is
kind
of
complicated
internally.
I
spent
like
three
weeks
debugging
this
last
year.
It
was
really
frustrating,
but
the
big
reason
is
that
fio
and
c-star
cannot
share
an
allocator
period.
D
A
D
D
D
It
doesn't
need
to
like.
I
don't
need
a
custom
version.
We
don't
have
to
build
a
custom
version
of
fio
to
use
it,
for
instance,.
C
Fl
only
test
the
c
style
right
now
to
include
the
chromosome,
the
other
layers.
D
Yeah,
when
we
get
further
along,
we
don't
need
fio
to
or
we
don't
need
anything
special
for
fio
to
talk
to
crimson
fio
speaks
rbd
natively,
okay,
like
current
distributions
of
fio,
already
included
rbd
back-end,
that's
not
a
problem.
The
purpose
for
this
is
that
c-star
isn't
mature
enough,
yet
to
run
a
whole
osd,
and
even
if
it
was,
I
don't
actually
want
to
test
the
osd.
I
don't
care
about
that
part
right
now.
I
just
want
to
test
the
product
testing.
D
A
D
Exactly
and
and
it's
it's
got
a
little
interface
so
that
when
c-star
gets
a
c-store
gets
a
little
bit
more
mature.
We
can
write
a
little
back-end
for
c-store
too
that'll
do
the
same
thing:
it'll
just
be
into
like
four
megabyte
objects
or
something
with,
like
some
token
x,
adder
right
each
time
to
simulate
what
the
osd
would
be
doing.
A
One
more
one
more
questions
is
a
unique
domain
socket
that
could
be
a
bottleneck
between
file
and
the
c
store.
D
A
D
Well,
it's
not
really
using
bandwidth.
What
it's
doing
with
the
ux
domain
socket
is
using
a
little
shared
memory
buffer
inside
the
kernel
and
some
shuffling
back
and
forth.
So
we're
a
little
memory
bandwidth,
but
nbd
itself
is
in
some
sense
is
sequential.
Let
me
put
it
this
way.
I
don't
plan
on
using
this
to
prove
that
c-store
is
fast
for
the
moment.
B
A
No,
no,
I'm
not
worrying
about
the
serialization,
I'm
worried
about
the
ordering,
because
it's
like
tcp,
a
single
channel
tcip.
You
need
to
order
all
the
packages
along.
D
A
D
D
D
B
D
A
A
But
regarding
to
the
nvme
over
bad
fabric,
I'm
just
wondering
if
it's
like
inform
the
client
which,
which
endpoint
would
it
be
talk,
would
be
talked
about.
Talk
talk
to
in
this.
D
D
Really,
I
wouldn't
worry
about
that
yep
so
bluntly.
The
the
the
biggest
and
best
version
of
this
would
never
see
the
crimson
code.
What
what
crimson
would
do
is
it
would?
You
would
set
up
conditions
that
make
this
work
and
then
nvme
over
fabric
would
handle
the
actual.
I
o
that
I
o
would
never
see
a
cpu,
that's
the
goal,
so
it's
this
isn't
really
a
crimson
efficiency
thing.
This
is
a
protocol
level
thing
to
enable
some
kind
of
full
bypass
from
the
client
all
the
way
through
to
the
device.
A
D
D
D
Extra
small
bit
of
information
in
the
existing
hosting
map-
that's
like
a
tiny,
tiny
change
to
radios.
That's
that's!
That's
current
rados
with
a
little
bit
of
extra
information,
the
oc
map-
that's
not
really
what
I'm
talking
about
here!
So
right
now,
when
we
do
a
write,
we
write
a
pg
log
entry
right.
A
A
I
think
we
had
probably
I'm
I
failed
to
to
clear
myself.
My
point
is
that
one
one
problem
we
we
were
we
were
we
were
discussing
regarding
to
m2n-
is
that
the
kind
the
the
the
messenger
cannot
be
shared
yeah,
true,
of
course,
so
one
of
the
solutions
is
that
we
have
a
multiple
we,
we
expose
more
multiple
ports
for
multiple
pgs
and
we
shot
pgs
by
ports.
A
B
D
C
D
Much
like
current
radios,
it
would
be
pretty
radically
different.
For
instance,
it
may
not
be
the
case
that
you
even
talked
right
now.
For
instance,
we
do
primary
replication
client
talks
to
primary
primary
talks
to
replicas
replicas
talk
to
the
primary
primary
talks
to
the
client.
Imagine
if
the
client
did
an
nvme
over
fabric
operation
directly
on
an
exposed
device
with
no
intermediate
coordination,
no
osc
cpu
wakes
up,
no
one
processes,
a
message:
there's
no
log
entry
and
there's
no
metadata
rhythm.
D
That's
what
I'm
talking
about!
It's
not
really
about
our
current
message
protocols.
It's
about
not
using
them
at
all.
In
other
words,
there
would
be
control
messages
that
happen
in
the
osd,
but
for
block
device
operations.
Hypothetically.
If
we
could
make
this
work,
then
we
simply
wouldn't
wake
up
the
crimson
osc
process
at
all.
D
D
D
Three
of
them,
so
the
question
is:
what
would
we
have
to
do
to
the
way
we
do
like?
Under
what
conditions
would
that
be
possible,
and
how
would
we
maintain
an
appropriate
level
of
consistency
and
atomicity
guarantees
to
clients?
What
would
that
look
like?
What
do
we
have
to
do
to
the
raido's
protocols
to
make
that
work?
What
does
peering
look
like
if
clients
are
allowed
to
do
that?
How
much
work
do
we
have
to
do
when
an
interval
changes,
bring
things
back
to
consistency?
That
kind
of
thing
this
isn't
really
about
ports.
A
D
B
D
Literally
there
there
are
four
of
them.
I
think
two
public
ips
and
two
private
or
four
some
number
of
heartbeat
ones,
that
I
can't
remember
the
number
of
one
public
one
and
some
number
of
cluster
ones
for
replication
and
they're
all
just
listed
in
the
osd
map.
Everyone
who
has
the
same
ocd
map
agrees
on
the
same
iport
combos
yeah.
So
what
we
do
here
is
we'd
say
instead
of
saying,
clients
talk
to
osd
81
on
ip,
whatever
or
whatever
we
say.
D
D
D
It's
published
in
the
osd
map,
and
it's
just
that's
just
what
it
is
until
the
next
interval,
by
contrast
with
the
nvme
over
fabric
stuff,
you
can't
just
say
this
is
the
device
you
you're
talking
to,
because
the
dynamic
location
of
an
object
on
a
device
is
not
something
you
can
predict
in
advance
at
least
not
easily.
That
would
be
I
there.
May
there
might
be
ways
to
do
that,
but
it
would
give
up
properties
of
radios
we
like,
so
we
probably
wouldn't
do
that.
A
They
commute
this.
This
message
over
and
via
me
also.
D
There's
their
approach,
it
actually
does,
but
that's
not
the
the
rado
side
still
needs
to
happen,
because
we
do
need
to
maintain
enough
metadata
to
allow
us
to
complete
peering
in
a
timely
fashion.
So
it's
not
nothing
about.
This
is
really
about
nvme
over
fabric.
It's
more
about
taking
the
protocol
that
nvme
over
fabric
is
and
figuring
out
whether
we
can
make
a
protocol
like
that
work
without
coordination.
D
In
summary,
this
really
has
little
to
do
with
an
m
versus
or
m10.
I
really
wouldn't
think
about
them.
At
the
same
time.
Okay,
things
that
come
into
this
message
queue
will
look
pretty
much
like
what
they
look
like
now.
There
will
be
an
operation
that
needs
primary
coordination
on
a
pg,
and
there
will
be
some
kind
of
a
mapping.
D
D
You'd
be
able
to
run
a
full
throughput
pci
express
device
on
armed
on
arm
hardware.
That's
what
because
the
cpu
wouldn't
be
woken
up
for
every
I
o.
D
Or
like
a
good
like
a
modest
but
beefy
intel
chip
could
drive
an
entire
gigantic
high-density
nvme
device.
It's
the
same
thing
crimson
is
about
just
more
so,
and
this
is
not
something
we
started
talking
about.
Yet
this
is
just.
This
is
just
a
thing
greg
and
I,
and
some
of
us
got
talking
about
after
the
80
n
presentation
at
cdm
a
couple
of
weeks
ago.
D
I
just
want
to
flush
out
a
design
document
the
next
couple
of
months.
This
is
not
like
a
plan
yet
does
that
make
sense?
The
idea
is
that
if
you
could
somehow
drastically
reduce
the
number
of
situations
where
an
osd
needs
to
be
involved
in
an
I
o,
then
the
cpu
cost
for
those
ios
goes
from
a
lot
to
virtually
nothing
and
even
crimson
as
like.
Even
if
we
did
crims.
B
B
B
B
D
There
might
not
be
a
video
this
time
it
might
not,
it
might
not
have
been
recorded
yeah
this
one
didn't
get
recorded,
I'm
sorry!
The
notes
from
it
are
here.
D
The
design
that
intel
was
presenting
was
they
use.
The
the
the
question
was:
how
do
you
use
a
bare
metal
host
to
access
an
rbd,
so
the
obvious
answer?
Is
you
run
nvme
over
fabric
on
the
host
and
attach
it
to
a
software?
Oh
found
it
sweet.
B
D
Anyway,
the
idea
is
that
you
use
have
you
ever
had
me
over
wrapper
to
talk
to
a
software
terminating
thing
on
a
gateway
host
and
then
that
gateway
host
speaks
rbd
back
to
the
cluster
right.
So
easy,
peasy,
simple,
but
kind
of
crappy,
because
you
have
to
do
the
second
hop
over
fabric,
so
their
design.
Was
you
embed
a
bunch
of
information
about
where
each
block
of
an
rbd
image
is
located?
D
Basically,
by
giving
it
the
little
bit
of
math
required
to
map
the
object
name
to
a
pg
and
then
giving
it
a
lookup
table
from
the
pg
back
to
the
osd
numbers
and
then
have
it
talk
directly
to
a
one
of
these
nvme
over
fabric
terminators
on
the
end
on
the
osd,
which
then
speaks
rbd
right
back
to
the
locally
hosted
primary?
D
So
you
will
avoid
the
second
hop.
But
if
you
think
about
generalizing
this-
and
I
talked
about
this
a
bit
in
the
thing
in
the
question
port
part
after
you-
can
eliminate
the
scalability
problem
by
constricting
where
objects
are
allowed
to
map.
D
D
D
No
and
it
won't
support
sffs
either
because
the
protocol
will
be
inherently
weaker,
so
rado's
off.
The
problem,
essentially,
is
that
radius
is
really
good
at
what
it
does,
but
it
offers
guarantees
that
rbd
doesn't
need,
like
block
devices,
don't
actually
offer
ordering
guarantees
they
just
don't
like
they
don't
order.
They
don't
even
necessarily
offer
read
after
write.
That
depends
because
cashing
on
the
client
will
typically
handle
that
they
don't
handle
or
they
don't
offer
like
guaranteed
order,
delivery
of
of
operations.
D
So
if
you
send
a
prefix
of
operations
and
then
get
a
path,
failure
and
then
resend
the
same
prefix,
you
might
actually
get
a
different
outcome
from
what
you
got
the
first
time,
depending
on
details
of
which
portions
of
your
write
got
through
the
first
time,
and
if
there
are
parallel
rights
to
the
same
object.
Really
all
bets
are
off.
It's
very
complicated,
the
outcomes
that
are
possible.
D
For
these
reasons,
the
network
protocols
associated
with
block
devices
are
typically
much
simpler
than
rados's
and
they
offer
far
fewer
guarantees
and,
as
a
result,
they're
typically
faster
kefu.
You
were
alluding
to
this
earlier,
which
you
mentioned,
that
using
a
single
stream
is
less
efficient
than
using
parallel
streams.
That's
a
correct
observation:
that's
why
block
device
protocols
are
usually
designed
the
way
they
are
it's
so
that
you
can
exploit
parallel
streams
across
a
network
or
fiber
channel
or
whatever.
D
So
for
things
like
radio,
gw
or
cephas
metadata
operations.
Rados
is
awesome
because
it
would
be
very,
very
expensive
to
implement
this
kind
of
atomicity
without
help
from
the
cluster.
So
in
reality
we
would
want
both.
D
We
would
want
to
be
able
to
use
the
strong
guarantee
protocol
when
we
want
it
like
in,
for
instance,
in
our
gw,
when
we
use
the
conditional
rights
to
ensure
that
we
get
atomic
rights
for
objects
and
be
able
to
degrade
to
weaker
guarantees
when
that's
what
we
really
want,
for
instance,
for
rbd,
or
perhaps
even
for
some
cfs
files,
though
that
is
less
clear
to
me,
so
we
would
want
to
design
it
in
such
a
way
that
the
protocol
is
general
enough
to
be
adapted
into
ffs
and
maybe
even
rgw
for
some
of
its
rights.
D
That's
not
totally
clear
to
me,
but
efficient
enough
that
rbd
is
able
to
operate
well
or
is
able
to
operate
it
close
to
what
we
think
is
the
best
that
is
practical,
given
the
design,
constraints
and
hardware
realities
of
life
like,
for
instance,
if,
like
one
design
constraint,
I
think,
is
very
unlikely
to
go
anywhere
is
that
we
offer
consistency
and
that,
as
long
as
you
have
one
out
of
three
remaining
replicas,
you
may
continue
doing
reads
and
writes
without
noticing
any
anomalies.
D
It's
a
that's.
That
guarantee
alone
is
something
that
many
storage
systems
don't
offer,
but
it's
one
that
we
will
likely
want
to
continue
offering,
as
it
is
important.
So
it's
not
really
about
weakening
the
guarantees.
Rbd
offers
it's
about
not
offering
guarantees,
it
doesn't
need-
and
hopefully
that
gives
us
enough
freedom
to
perhaps
monitor
some
hardware
configurations,
maybe
at
least
design
out
what
a
direct
client-to-disk
protocol
would
look
like.
D
A
This
also
reminds
me
a
discussion
with
with
on
the
rdma
anime
solution.
D
Well,
this
would
be
in
many
ways
rdma
right.
One
thing
this
is
not
a
thing
the
crimson
project
is
committing
to
it's,
not
even
a
thing
red
hat
is
committing
to.
This
is
just
a
thing.
I've
been
looking
into
the
last
two
weeks.
A
Thanks
for
sharing
again,
but
if
you
happen
to
know
scott
peterson.
D
E
Last
week
I
was
on
major
testing
and
fixing
encryption
recovery
related
bugs,
and
this
is
the
branch
I'm
working
on.
E
I
fixed
the
I
fixed
those
bugs
based
on
the
interpretable
future
branch,
and
I
think
once
the
test
goes
goes
well
and
I
will
submit
all
the
commits
into
the
interrupt
interoperable
future,
the
interoperable
future.
Pr.
No
that's
fine.
E
I
I
implemented
those
figs
on
the
on
the
interpretable
future
branch
and
it
seems
there
there's
some
extra
work
needs
to
be
done.
That
needs
to
be
done
if
I
need
to
separate
them
into
another
branch
and-
and
since
I
think
both
the
interoperable
future
commits
and
those
bug
fix,
commits
all
all
for
the
purpose
of
a
smooth
running
of
recovery-
and
I
I
think
maybe
they
should
be
submit
together.
E
A
Because
I'm
still
reviewing
your
your
interpret
future
because
because
it
could
be
controversial
regarding
to
the
granularity,
because
I
kind
of
agree
with
the
same
and
the
concerns
raised
by
sam
and
the
redec,
because
they
they
believe
that
there
is
a
chance
that
there
should.
There
could
be
some
other
pgs
which
is
not
related
to
the
ex,
which
is
not
related
to
easiest
one.
To
to
to
sneaking
to
the
the
intrepid
interrupted
continuation.
A
D
D
That
should
prove
that
it's
impossible.
I
think.
Okay,
I
mean
that
and
the
asserts,
although
I
need
to
read
it
carefully,
of
course.
If,
but
again
you
should
satisfy
yourself
with
that.
I
could
easily
be
wrong.
This
kind
of
code
is
really
really
tough,
but
that
was
my
thought
at
the
time
at
least,
and
it's
there
are
asserts
right
what
happened
so
that,
if
something.
A
D
A
E
B
A
So
you
need
to
scratch
square
some.
Some
preliminary
implementation
for
first,
for
example,
to
persist
the
the
index
into
into
uptime.
B
D
B
D
Okay,
my
I
have
a
suggestion
that
you
may
want
to
do.
First,
I
suspect
that
one
of
the
most
common
use
cases
for
optane
at
least
early
will
be
for
caching
metadata
and
the
journal.
D
Unless
you
think
that
the
beach
reimplementation
in
yeah,
it
doesn't
seem
appropriate,
though
I'm
not
sure
you
might
want
to
put
together
a
bit
of
a
document
that
we
can
talk
about
it,
yeah
yeah,.
B
D
A
You
mentioned
that
you
send
a
some
question
to
me
and
to
sam
did
you
yeah?
Do
you
send
it
over
email
or
an
email?
Okay,
so
email,
okay,.
C
D
A
I
I'm
I,
I
believe
that
it's
a
it's
a
way
to
to
shut
the
pg
pipe
course.
That's
that's
it,
and,
and
to
the
premium
design
changes
to
to
embed
this
information
in
is
in
embedded
information
in
in
hd
maps.
D
D
D
I
think
we
should
skip
the
radius
protocol
part
for
now
and
just
go
ahead
and
run
it
on
multiple
cores
with
either
one
messenger
forwarding
messages
to
all
of
the
cores
or
n
of
them
and
a
they
all.
Listen
on
the
same
port.
D
With
randomly
balanced
connections
across
them
and
then
just
go
ahead
and
implement
a
crossbar
which
again,
we
have
to.
A
A
D
A
A
D
It's
ugly,
but
I
think
it's
okay
for
now,
and
the
advantage
to
this
is
that
we
can
get
to
theology
coverage
on
multi-core
crimson
before
we
get
the
radius
protocol
changes
done,
which
will
be
much
harder
and
once
we
have
the
radius
protocol
changes
nothing
changes
in
the
crimson
code.
It
just
means
that
this
handoff
mostly
doesn't
happen.
D
A
D
D
I,
the
the
design
I
designed
or
the
design
outline
before
with
the
thing
we
just
like,
take
the
thing
mod,
the
number
of
pgs
actually
kind
of
sucks,
because
it
means
we
have
a
static
assignment
for
every
interval
of
pg's
decor,
which
might
act
which
might
not
be
what
we
want
to
do.
We
might
want
to
actually
change
the
messenger
v2
protocol
itself
to
embed
some
information.
That
says
actually,
mr
client,
if
it
would
be
all
right
with
you,
please,
please
initiate
any
further
operations
on
pgx
to
this
other
address.
Please.
D
D
D
D
B
B
D
Well-
and
we
can
definitely
start
like
if
the
the
in
parallel,
we
can
write
a
design
document
about
what
we
want
to
do
to
either
the
messenger
or
the
osd
map
protocol
to
figure
out
what
we
want
to
do
to
change
this,
that's
the
other
half.
We
can
totally
do
that
in
parallel,
but
I
don't
want
to
block
it.
D
I
think
the
more
immediately
important
part
is
getting
test
coverage
on
multi-core
crimson,
because
once
we're
sure
that
works,
we
kind
of
just
need
to
change
the
protocol.
It
won't
really
cause
bugs
the
risky
part
is
making
sure
that
we
have
thread
local
variables
everywhere
we
need
them.
We
haven't
made
any
assumptions
about
memory
allocation
like
all
that
stuff's
tricky,
especially
with
bluestora,
and
it's
very
likely
that
we
messed
some
of
it
up.
So
the
sooner
we
shake
those
bugs
out.
The
more
confident
we'll
all
feel,
I
think.
A
A
So,
by
the
way
I
the
the
the
change
to
to
collect
to
collocate
the
the
default
allocator
with
the
satellite
allocator
has
been
merged
to
system.
So
I
will
pick
combat
resistor
to
some
module
to
include
the
changes
to
enable
us
to
to
run
the
sita
and
lipstick
allocator
in
our
in
our
crimson.
A
Okay,
okay,
I
will
add
the
some
of
the
text
to
our
trello,
so
we
can
work
on
it
later
on.
D
B
D
So
the
reason
why
I'm
not
worried
about
this
is
that
my
anticipation,
or
is
that
I
anticipate
that
on
four
cores
there
will
be
either
four
or
two
or
one
c
store
instances
and
those
instances
will
have
separate
metadata
structures,
also
keith.
This
is
the
other
reason
I
wanted
to
make
sure
we
had
the
ability
to
do
cross
core
operations,
it's
that
for
128
core
machine.
I
really
don't
think
we're
gonna
run
128
c
store
instances.
D
I
think
we'll
probably
run
16
with
groups
of
pg
threads
mapped
to
them,
we'll
run
whatever
is
a
direct
number
to
maximize
throughput
to
the
nvme
device,
but
not
not
more
so
that
will
require
a
core
handoff
and
we
probably
won't
even
run
pg
code
on
those
cores.
It'll
just
be
c.
D
A
I
have
a
write-up
in
in
the
past
due
to.
A
A
To
to
tweak
the
the
test
to
enable
the
the
module
called
deployment
deployment,
so
we
can
contest
it
to
see
what
happens.