►
From YouTube: January 2021 OpenZFS Leadership Meeting
Description
At this month's meeting we discussed: release roadmap; RAIDZ expansion; marking vdevs non-allocatable; zpool import performance; drive vendor SCSI priority bits.
meeting notes: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit
A
B
Roadmap
for
opengfs
releases:
do
you
have
some
perspectives?
What
features
should
be
available
on
next
release,
or
in
this
year?
It
will
be
interest
to
me,
because
we
try
to
port
messy
cities
to
deals
and
I'm
interested
in
some
perspectives.
When
you
have
plans
trying
to
release
something.
This
opens
the
professors.
C
A
A
I
mean
d
raid
is
the
main
one,
so
that's
in
that's
in
master
now
and
the
next
release
2.1
will
have
that
so
that's
kind
of
the
big
exception
to
our
guidelines
about
minor
releases
not
having
new
features
and
another
thing
that
I
know
brian
was
hoping
to
get
into
2.1
was
the
forced
pool,
export
so
being
able
to
export
a
pool?
That's
that's
hung
because.
A
And
we
would
expect
those
I
think,
on
a
roughly
quarterly
basis
and
then
we're
hoping
to
get
open,
zfs
3.0
the
next
major
release,
hopefully
by
the
end
of
the
year,
that
would
be
the
fastest
major
release
ever
in
the
history
of
zfs
on
linux.
So
you
know
I
wouldn't
I
wouldn't
I
wouldn't
count
on
that.
But
that's
that's!
The
goal
is
to
get
out
this
this
calendar
year,
but
realistically,
I
think
that
the
cadence
has
been
more
like
18
months
than
12
months.
A
E
E
E
F
E
Yeah,
so
that's
still
in
progress,
we're
want
to
get
that
into
the
2-1
release
as
well.
Brian
atkinson
was
going
to
refresh
it
again.
There
was
some
feedback
on
that
and
I
think
it's
moving
in
the
right
direction.
We
got
some
feedback
on
the
freebsd
side
as
well,
and
it's
still
moving
forward.
So
that's
what
I'm
helping
him
work
on
for
the
next
few
weeks
or
months
to
just
to
get
it
in.
So,
yes,
we'd
like
to
get
that
into
2-1
and
get
it
out
in
the
first
quarter.
E
A
E
A
A
I
didn't,
make
my
goal
of
getting
a
new
pr
beta
release
of
that
out
by
the
end
of
last
year,
but
I
I
am
still
working
on
it
and
actually
putting
in
putting
time
on
it
so
expect
to
see
progress
on
that
soon.
I
I'm
work
right
now,
specifically
I'm
working
on
breaking
out
some
of
the
abd
related
changes
to
some
of
the
abd
infrastructure.
A
A
B
E
B
Where
we
can
see,
we
will
see
some
additional
changes
to
the
fs
tests
code,
where
you
can
find
we
can
use
the
same
userline
parts
on
different
platform.
It
is
our
goal:
try
to
make
a
dls
platform
to
be
more
easy
for
ports
of
user
length
applications
from
linux
d
from
debian
mainstream,
something
like.
F
B
And
at
this
moment,
zfs
code
on
geos
and
illumis
are
different,
because
I
have
portraits
additional
features.
Both
are
missed
on
illumos
and
ilumas
contained,
for
example,
l2r
persistent
controlled.
I
did
not
pull
to
deal
with
yet
because
I
tried
to
specify
some
changes
and
we
have
to
be
more
stable
in
gfa
specific
code
for
our
business.
B
I
want
to
use
open,
zfs
code
deals
and
try
to
prepare
a
special
build
environment
where
we
can
make
changes
only
in
the
fs
code
and
use
additional
build
board
where
we
can
try
to
test
additional
changes
from
opencfs
to
dealer.
Spotlight,
that's
cool,
it's
a
wonk.
Yes,
it's
a
long
term
project.
What
I
thought
to
work!
It
is
why
I
try
to
ask
you
about
some
features
and
some
perspectives
to
be
a
more
informative
about
what
the
way
I
want
to
try
to
use
for
our
specific
to
be
more
compatible
with
opencfs.
A
Yeah
so
the
long
term
like
using
the
opencvs
code
based
on
illumos,
that
sounds
really
cool.
I
know
that
joshua
kulo
was
also
interested
in
that
I
don't
know
like
what
it
sounded
like.
He
was
getting
more
real
in
his
mind.
Last
time
I
talked
to
him
about
it,
so
you
might
want
to
you
know,
coordinate
with
him
if
you're
not
already.
B
G
A
A
A
B
Thanks
for
your
information-
and
I
hope
additional
platform
can
help-
try
to
find
additional
issues
and
we
have
found
additional
issue
with
infinite
loop
in
receiver
stuff.
If
checkpoint
present,
I
will
provide
additional
information
and
because
we
try
to
minimize
some
steps
with
the
small
drives
for
reproduces
issue.
B
For
example,
we
are
using
for
testing
a
storage
about
one
petabyte.
It
is
why
I
can
try
to
find
a
more
additional
issues
when
we
can
try
to
find
the
difference.
Small
drives
based
on
the
fs
code,
and
some
issues
cannot
be
introduceable
with
a
small
data
and
can
be
reproducible
only
with
the
big
data
about
5000
of
terabytes
or
something
else
well.
All
right.
B
I
can
try
to
help
because
we
have.
I
can
try
to
help
with
testing,
because
we
have
our
environment
where
we
can
try
to
test
some
specific.
It
is
why
I
am
interested
in
communication
with
the
open
gfs,
where
we
can
try
to
provide
some
issues,
try
to
understand
it
and
fix
it
and
open
the
first
will
be
more
stable.
We
are
all
interested
in
it.
A
B
Additional
issues
additional
issue:
it
is,
for
example,
if
you
are
using
a
local
drives
connected
to
lsi
sba.
For
example,
you
can
see
some
specific
device
where
drive
can
respond,
but
if
you
try
to
use
network
device,
for
example,
from
iscsi
or
something
else,
you
can
see
network
delay
where
you
will
be
wondering
how
it
can
impact
of
some
cfs
works
and
where
we
need
some
retry
operation.
B
Also
additional
issue
with
a
network
drive
if
you
have
stripe
of
drives
in
range
network
drives
and
one
drive
was
offline.
If
you
try
to
do
import,
you
try
to
retry
to
this
drive
over
network
again
and
again.
If
you
try
to
do
import
jf,
the
import
and
additional
feature
will
be
interest
to
me.
If
we
try
provide
a
specific
vdf,
try
to
move
it
to
offline
state.
B
H
H
B
A
A
To
mark
devices
as
ones
that
we
don't
want
to
allocate
from
so
this
is
you
could
think
of
this,
maybe
as
sort
of
an
extension
of
that,
although
I
mean
the
way
that
we're
you
know
be
able
to
do
that,
import
time
is
another
special
thing
and
whatnot.
So
it's
not
exactly
the
same,
but
it's
kind
of
closely
related
yeah
exactly.
I
F
I
G
I
I
It
doesn't
get
allocated
from,
and
I've
added
like
assertions
in
the
cio
pipeline
stuff
and
so
on
that
we
crash,
if
we
actually
try
to
use
it.
The
nice
thing
is
that
we
don't
do
these
assertions
for
the
vdf
label
space.
So,
like
all
the
normal
z
point
management
stuff
that
works
and
that
code
is
pretty
much
done.
I
just
felt
it
was
too
hecky
to
upstream,
but
maybe.
F
A
A
The
customer,
may
want
to
remove
several
devices
with
top-level
device
removal
and
we
want
to
mark
all
of
them
so
that,
like
today,
what
you
do
is
like
you
move
the
first
device.
Then
you
move
the
second
device,
then
you
remove
the
third
device
when
you're
moving
the
first
device.
It's
like
copying
data
onto
you
know
the
fourth
and
fifth
devices
where
you
want
it,
and
also
the
second
and
third
devices
that
we're
just
trying
to
move
anyways,
which
is
one
problem,
and
then
another
problem
is
like.
A
A
Class
to
kind
of
move
its
labs
off
to
the
side
to
where
we're
not
going
to
do.
It
sounds
like
a
way
to
implement
that.
There's
also
additional
logic
that
we
need
to
do
like
the
space
accounting
and
you
know,
make
sure
they
will
not
run
you
out
of
space
kind
of
similar
to
what
we
do
with
device
removal.
I
A
So
in
the
in
the
current
implementation,
that's
in
the
pr
we
don't.
We
don't
try
to
move
them
between
classes.
So
that
is
a
lot
simpler.
So
yeah
I
mean,
maybe
maybe
that's
not
the
right
way
to
go
about
it
in
terms
of
moving
metastops
between
classes.
I
think
that
we,
I
have
to
check
the
design
dock,
but
I
don't
think
that
we
had
envisioned
doing
that.
A
A
G
I
I'm
not
not
really
convinced,
that's
a
better
idea
that
I
have,
because
this
is
the
first
time
I
heard
that
metaslab
still
carry
the
attribution
to
an
allocation
class.
My
impression
was
that
it's
like
something
that
is
only
relevant
before
the
allocation
is
done,
but
apparently
I
misunderstood
so
yeah
I
don't
know
I
can
share
the
code
and
somebody
can
take
a
look.
Okay.
H
I
H
A
Yeah
yeah,
if
you
I
think
that
the
way
we
designed
it
like
the
if
you
get
that
stuff,
if
you
get
that
pr
out
before
then
we
can
take
advantage
of
that
for
the
user
interface.
But
if
not,
then
we'll
have
like
a
new.
You
know
sub
command
or
something,
and
then
we
can
map
that
back
onto
the
properties
when
they
integrate.
B
Yeah,
we
can
see
additional
issue
with
performance
on
the
pool
import
command
when
we
have
scrap
or
receiver
is
in
progress.
If
you
try
to
import
the
pool
after
crash
and
as
we
work
around
right
now,
we
are
using
external
variable
where
we
can
stop
a
scan,
operation,
import
tool
and
start
scan
operation
again
it
can.
B
B
A
B
A
A
It
should
be
proportional
to
the
number
of
txgs
that
go
by
before
the
pool
is
like
you
know,
fully
available
for
use
and
it,
but
it
should
be
like,
like
the
txt
timeout
30
seconds
or
whatever
times
number
of
txgs
and
not
proportional
to
any
sort
of
performance,
but
it
sounds
like
my
understanding
may
be
wrong,
so
something
else
there
is
going
on
that.
We
should
investigate.
B
F
D
A
B
A
B
B
Yeah
yeah
yeah,
but
you
cannot
reproduce
this
issue.
If
you
have
a
small
of
data,
you
can
reproduce
it
if
you
have
a
big
data
and
some
network
device
with
some
device
for
the
producer
of
this
issue,
I
probably
I
can
try
provide
additional
information
or
access
to
our
test
environment
where
you
can
try
to
play.
A
H
H
H
A
A
That's
the
question
I
was
trying
to
ask,
but
I
presume
like
I'm
guessing
since
I'm
guessing
that
this
only
happens
when
you
have
a
scrub
in
progress,
and
it
doesn't
happen
if
you
are
just
doing
that
couple
txj's
check,
but
I
mean
another
question.
I
think
that
something
that
could
be
at
play
here
is
like,
like
you
said,
looking
at
all
of
the
data
sets,
so
it
would
be
interesting
to
know
how
many
data
sets
are
in
you
know:
igor's
system
file
systems
and
snapshots
and
z-volts.
E
We
have
seen
similar
issues
like
this,
where
we
put
some
minor
mitigations
in
place
like
there's
a
delay
before
the
scrub
starts
again
during
import.
It
delays,
like
I
don't
know,
10
or
20
transaction
groups,
or
something
like
that.
Just
keep
it
out
of
the
way.
Initially
before
it
starts.
I
think
that
might
have
made
it
tunable.
You
could
try
to
increase
it
for
testing,
but
nothing.
Quite
this
severe.
A
B
F
B
A
B
A
D
A
J
So
what
he
was
trying
to
do
was
to
use
the
ata
and
scuzzy
priority
scheduling
as
part
of
the
spec
to
do
some
background,
work
and
offload
some
of
that
at
the
device
level
so
send
you
know,
commands
with
a
specific
priority
so
that
they
don't
not
not.
All
commands
are
created
then
equally
in
this
in
the
system
and
what
I
found
out
from
because
he
was
having
some
issues
with
some
drives
and
he
was
seeing
inconsistent
behavior,
and
so
he
wanted
to
get
an
explanation.
J
So
we
dig
a
little
bit
deeper
and
we
found
out
from
the
firmware
team
here
at
seagate
is
that
it
totally
depends
on
the
implementation
of
that
product,
and
you
know
the
firmware
being
used
and
and
most
of
the
time
what
we
do
is
that
we
work
with
the
ecosystem.
So
I
won't
name
names
from
the
customer
side,
but
you
know
the
west
coast,
people
that
are
huge
and
taking
on
our
you
know
near
line
drives.
J
They
basically
request
and
test
a
certain
firmware,
a
certain
way,
the
way
they're
using
the
drive
and
the
priority
bits
in
both
scuzzy
and
ata,
and
then
the
firmware
is
released
to
them.
So
if
we
want
to
put
it
in
a
generic
one,
the
firmware
team
was
saying
that
they
need
to
know
a
little
bit
more
on
how
zfs
ecosystem
would
be
using
it
so
that
it
doesn't
implement
the
generic
case
and
that's
why
it's
not
turned
on.
J
If
you
just
go,
buy
an
airline
drive
from
amazon
and
so
matt
was
like
hey
yeah
I'll,
be
definitely
interested
in
talking
to
you.
What
I'm
trying
to
do
and
what
I'm
seeing
in
the
drives
that
he
had,
and
the
question
to
this
forum
is
that
if
it
is
an
interest
to
the
general
body,
I
would
much
rather
have
one
discussion
where
we
set
some
agenda
and
then
I'll
bring
in
the
core
firmware.
J
B
J
So
there
is
some
priority
scheduling
features
that
are
part
of
the
scuzzy
and
ata
spec,
both
t10
and
t13.
But
the
thing
is
that
it's,
they
are
not
always
worded
the
same
way
so
there's
some
of
it
is
open
to
interpretation,
but
also
it's
not
implemented
in
all
the
drives.
So
you
have
to
kind
of
go
figure
out
like
hey.
Do
my
drives
actually
support
it,
and
if
they
do,
then
you
actually
send
that
command.
But
then
it's
even
between
us
and
wd
and
toshiba
and
others.
J
A
Have
to
know
that
you're
using
those
drives
and,
like
you
said,
customers,
your
customers,
who
are
designing
a
whole
whole
hardware,
software
stack
and
they're
buying
specific
drives.
They
set
it
up
because
they
know
that
that's
what's
going
to
happen
right,
but
for
a
generic
software
solution
like
zfs
it.
A
You
know
it's
hard
without
any
special
knowledge,
it's
hard
for
us
to
take
advantage
of
that,
because
it's
like
well
like.
Why
would
I
set
these
special
bits?
Because
I
don't
even
know
what's
going
to
happen
when
it
goes
down
to
the
drives,
because
I
don't
know
what
kind
of
drives
there,
but
in
theory
I
guess
we
could
build
in
something.
That's
like
we
query
the
drives.
If
the
drives
are
running
the
right
firmware
that
we
know
how
what
it
does,
then
we
take
some
special
behavior.
You
know
right.
J
So
I
mean
all
you
would
need
to
do
is
actually
look
at
the
you
know
the
identify
or
the
whatever
bits
and
say:
oh
yeah,
that
feature
is
enabled.
That
means
that
that
drive,
regardless
of
the
way
the
finger
is,
would
actually
behave
the
way
the
spec
says
it
would
behave
from
a
from
that
perspective,
but
I
think
there's
some
in
bigly
in
the
spec
and
then
there's
some
other
stuff
that
needs
to
happen.
So
I
was
offering
a
forum.
J
A
It
would
be
great
to
get
like
a
just
a
high-level
description
of
like
what
the
drive
can
do
based
on
that
and
and
then
we
can
think
about.
Like
you
know
what
would
be
like,
let's
just
assume
that
we
knew
that
we
were
always
using
these
drives
or
a
significant
portion
of
the
time
we're
using
these
drives.
A
How
would
we
take
advantage
of
that,
and
you
know
how
much
you
know
benefit?
Would
we
get
and
then
try
to
figure
out,
because
there's
two
problems?
One
is
like:
how
do
we
know
that
we're
in
this
situation?
Is
it
worth
knowing
that
we're
in
this
situation,
then
it's
like
okay.
If
we
are,
then
what
are
we
actually
going
to
do
about
it?
What
you
know,
how
is
the
ifs
going
to
take
advantage
of
that
information,
so.
J
K
K
In
particular,
the
biggest
step
was
to
separate
background
such
as
scrap
receiver
and
other
activities
from
interactive
ones
initiated
by
user
science.
The
first
has
latency
acceptable
up
to
means
potentially
or
who
knows
hours
forever,
just
to
not
cause
common
payments,
while
second
still
has
to
be
in
seconds
and
milliseconds.
So
obviously
it
could
be.
K
It
would
all
could
also
be
interesting
potentially
to
separate
even
synchronous
requests
from
asynchronous,
but
it's
like
emotique
science,
I
think
chronos
read,
maybe
potentially
promote
to
synchronous
when,
if
we
first
issued
prefetch
request,
but
then
has
to
escalate
it
to
actual
actual
synchronous
request
when
that
even
for
which
we
prefetch
it
has
arrived,
when
actual
application
request
has
appeared
and
that's
making
maybe
complicated,
but
for
rights
we
have
really
two
kinds
of
rights:
one
is
synchronous,
rights
and
other
eisenchronos
rights.
I
don't
know
what
the
drive
can
differentiate.
K
K
K
B
B
K
K
Red
card:
no,
if
there
is
a
significant
right
cash
and
if
it's
battery-backed
it's
it's
more
significant
investment
to
just
give
practically
different
different
class
of
cards.
None
of
our
no
none
of
our
systems
use
that.
But
I
agree
that
that
information
could
be
useful
for
a
card
but
barely
buying
an
additional
card
with
additional
memory.
K
Would
compensate
things
that
could
be
done
by
the
drive
directly
since
drive?
Is
practically
the
only
place
in
a
system
that
knows
about
where
it's
head
right
now?
How
can
it
be
scheduled
in
what
proper
order
requests
should
be
executed
and
to
better
handle
that
drive
should
have
as
deep
queue
as
possible,
but
as
soon
as
we
submit
more
requests
to
the
drive,
latency
start
growing
and
we
have
no
control
over
those
latencies
and
supplying
priorities
to
the
drive
is
a
way
to
say
at
which
point
latencies
are
acceptable
and
which
they
are
not.
B
K
B
K
B
K
Of
okay,
okay,
but
to
each
specific
drive,
we're
sending
some
mix
of
commands.
Some
of
those
commands
are
high
priority.
Some
of
those
commands
are
low
priority,
but
drive
has
no
idea
about
that.
Drive
execute
them
all
evenly,
while,
obviously
we
don't
need
it,
it
needs
them
to
be
even
no
matter
how
we
distribute
load
between
drives.
K
There
are
still
concurrent
ios
from
one
side
and
from
another.
I
am
heavily
trying
to
reduce
that
concurrency
to
keeps
latency
in
touch,
but
reducing
queue,
depth
reduce
performance
of
the
drive
science.
The
only
drive
knows
how
to
schedule
it
properly.
So
I'm
trying
to
bring
knowledge
to
the
point
where
it's
usable.
K
A
J
Yeah,
that's
what
I
was
looking
for
your
guidance
on
like
if
I'm
just
trying
to
get
an
agenda
together
so
that
you
know
my
folks
know
what
they
need
to
come
prepared
with
and
for
somebody
from
the
office
say
you
know
if
it's
just
math,
just
explaining
the
actual
need
and
how
zfs
would
actually
use
what
is
currently
out
there.
That's
the
kind
of
guidance
I'm
looking
for.
So
if
it's
just
like
me,
mailing
it
to
the
mailing
list,
saying
hey:
this
is
the
agenda
if
you're
interested.
J
A
J
A
A
C
A
E
A
That,
with
with
nothing
on
the
agenda,
we
somehow
managed
to
fill
the
hour
useful
discussions.
So
I
I
think,
that's
a
sign
of
a
healthy
community
and
I
enjoy
everyone's
ideas
and
discussion,
and
I
hope
that
we're
able
to
get
to
the
bottom
of
some
of
the
strange
import
performance
stuff
that
igor
brought
up.
So
let's
please
continue
that
offline
thanks
a
lot
thanks.