►
From YouTube: April 2021 OpenZFS Leadership Meeting
Description
At this month's meeting we discussed: corrective zfs recv; blake3 checksum; LXD containers; ZFS on Object Storage
meeting notes: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit#
A
Welcome
everyone,
it's
one
after
the
hour,
so
we'll
get
started.
I
see
that
alan
is
wearing
a
great
t-shirt
and
perhaps
has
rearranged.
You
rearranged
your
office.
Yes,
I
turned
my
desk
sideways
in
order
to
try
to
feel
like.
I
wasn't
in
the
same
place
anymore,
cool
yeah,
so
you
have
a
new
office.
That's
good
excuse.
A
A
D
C
A
C
A
Yeah,
that's
great
yeah.
This
is
the
code
that
you
posted
quite
a
while
ago,
but
yeah
great
to
see
that
making
progress
on
that.
Do
you
wanna,
because
it's
been
a
couple
years,
maybe
give
just
a
quick
overview
of
like
what
the
point
of
this
is
and
how
and
how
it
works
so
that
people
who
might
be
interested
can
can
realize
that
they
are
interested
and
help
you
review
it.
C
C
On
the
pool,
so
without
really,
if
you
have
permanent
corruption
in
your
data
set,
you
can't
do
much
about
it
other
than
destroy
and
re-receive
it.
If
you
have
a
backup
right,
but
this
enables
you
to
use
a
send
file
from
a
well
somewhere.
Maybe
you
have
it
backed
up
that
data
set
somewhere
and
you
can
use
a
send
file
from
the
backup
system
to
heal,
corrupted
data
and
some
of
the
things
this.
So
this
patch
can
re-encrypt
data
re-compress
it.
C
So
if
the
those
settings
don't
match
between
source
and
destination,
it
takes
care
of
those
things
of
those
details
and
there's
an
arbitrary
restriction
of
the
guide
of
the
snapshot
has
to
be
the
same
as
the
snapshot
you're
healing.
So
basically,
the
send
file
has
to
come
from
the
same
snapshot
we,
which
doesn't
it's.
C
I
say
it's
arbitrary
restrictions,
since
it
doesn't
quite
always
have
to
be
that
way
to
be
able
to
heal,
but
it
seems
to
make
sense
to
kind
of
enforce
that
and
make
sure
that
the
data
you're
you're
ingesting
is
can
be
used
for
healing
for
sure
yeah,
that's
kind
of
the
the
overview.
I
think
that
should
be
cool
starting
point.
Yeah.
A
C
A
Yeah
cool,
it
seems
like
this,
the
there's
a
very
also
very
old
and
incomplete
pr
from
a
deflex
intern,
about
being
able
to
better
identify
which
blocks
were
damaged
and
like
tell
you
which
snapshot
it
appears
in
and
stuff.
It
seems
like
that
would
be
really
helpful
for
figuring
out
like
what
send
you
need
to
do
to
do
the
healing
properly
or
even.
E
A
A
B
C
B
D
Yeah,
that's
me:
I'm
from
germany,
hello,
welcome,
yes,
break
three!
I've
been
in
for
about
three
weeks
or
four.
I
don't
know.
Currently
it
was
very
fast
to
implement,
but
then
I
had
a
problem
with
the
freebsd
support
and
because
there
are
undefined
references
in
the
open,
cfs
pumped
cargo
motor,
I
don't
know
how
to
implement
and
implement
this
properly
into
the
bsd
stuff.
I'm
a
linux,
only
user,
so
I
run
into
problems
to
make
it
ready
to
to
to
to
submit
really
submitted
it
because
yeah
now
this
is
the
problem.
D
D
A
A
D
A
I
posted
a
it's
a
bit
of
an
old
patch
now,
but
it
looks
at
plugging
that
in
on
the
freebsd
side,
and
you
can
just
see
there's
a
we
made
a
os
linux
version
of
it
that
just
returns
e,
not
sub
for
so
far
but
could
be
plumbed
to
whatever
function.
Would
you
would
call
in
the
linux
kernel
to
do
the
the
checksum.
A
G
A
Yeah
so,
unfortunately,
using
that
using
the
the
kernel
encryption
routines
on
linux,
because
you
know
the
opencvs
project
is
not
using
any
gpl
only
exports,
that's
not
really
a
great
option,
which
is,
I
think
why
the
the
icp
stuff
is
from
illumina.
So
that's
all
code,
that's
cddl
license
that
was
copied
from
lumos
all
the
c
code
and
I
think,
there's
maybe
some
very
light
acceleration
in
there
but
yeah.
A
I
totally
believe
that
there's
more
optimized
routines
that
we
could
use
and
if
we
can
find
ones
that
are
appropriately
licensed,
I
mean
we
could
bring
them
in
like
if
these
freebsd
ones
are
bsd
licensed.
We
could,
potentially
you
know,
copy
those
into
the
zfs
source
space
and
replace
the
icp
illumios
ones.
The
main
bits
of
it
are
assembly
files
from
intel
or
something
that
are
yes.
A
D
A
So,
just
before
you
go
to
the
next
thing
for
the
blake
three,
the
can
you
give
a.
I
remember
I
researched
this
a
while
back,
but
can
you
give
a
summary
of
like
how
does
blake
three
compare
to
the
other
tech
swimming
algorithms
like
which
is
which
ones
is
it
faster
than
which
ones
is
it
more
secure
ish
than
like?
What's
what's
kind
of.
D
D
It's
and
and
on
r,
is
also
a
lot
of
faster.
I
I
I
checked,
but
not
not
really
a
bit
faster,
but
there's
also
there
must.
The
block
must
have
some
more
bits
as
at
about
4k.
Also,
then,
screen
is
also
a
lot
of
faster,
but
that
would
be
also
a
question
which
block
sizes
should
we
benchmark.
Currently,
this
is
very
fixed
within
fletcher,
and
this
shouldn't
be
fixed
because
you
can
set
up
different
record
sizes
and
then
you
have
also
a
different
speed
for
different
algorithm.
D
A
D
A
I
mean
I,
I
would
see
a
lot
of
value
in
just
seeing
the
results
like
if
you
just
made
a
table
and
we're
like
hey
like
on
a
modern.
You
know
cpu
that
has
all
the
extensions
here's
how
each
algorithm
performs
for
each
block
size.
I
think
that,
like
being
able
to
regenerate
that
on
demand
all
the
time
you
know,
I
don't
think
that's
something
that
needs
to
get
run
more
often
than
like
they
come
out
with
new.
You
know
new
instructions
or
whatever,
which
is
only
maybe
once
every
five
years
or
something.
D
E
A
A
A
D
I
could
make
some
code
and
do
it
into
into
opencfs.
I
have
no
problems
with
this.
I
I
would
reuse
other
ones
code
make
it
so
that
it
fits
and-
and
then
it's
it's
in-
but
I
when
there's
a
need,
I
think
for
for
linux,
there
is
a
need
because
I
we
can't
use
the
fast
stuff
from
the
camera.
So
I
think
I
would
do
it
with
extra
pull
requests.
A
D
A
A
A
A
So
basically
it
hooks
up.
The
enforcement
of
you
know
is
global
zone
and
the
similar
primitives
that
are
in
zfs.
So
when
you
call
dev
zfs
from
inside
the
username
space,
you
can
only
see
the
data
sets
that
have
been
delegated
to
that
namespace
and
the
parents
that
you
have
to
see
to
get
back
to
the
root
or
whatever,
and
you
can
create
new
data
sets
and
so
on,
and
they
happen
all
inside
the
namespace.
A
The
next
thing
we're
working
on
now
is
hooking
up
the
user,
id
and
group
id
mapping
currently
without
that,
when
you
create
a
file
as
one
of
the
kind
of
pseudo
user
ids
inside
the
namespace,
the
file
ends
up
showing
up
as
being
owned
by
user
id.
You
know
264
thousand
and
one
which
is
the
actual
uid
in
the
system,
but
in
the
namespace
that's
supposed
to
show
up,
as
you
know,
a
low
number
normal
user
id
and
adding
the
mapping
there
and
so
on.
A
The
main
goal
behind
this
for
our
customer
is
to
be
able
to
run
docker
using
its
zfs
driver
inside
of
an
unprivileged
lxd
container,
so
they
can
run
their
ci
stuff
for
multiple
projects
on
the
same
machine
and
inside
the
container
docker
can
do
the
zfs
crates.
It
needs
to
to
run
that
way,
but
we're
interested
in
what
other
use
cases
people
might
have
for
this
and
what
you
know
test
cases
would
make
sense
for
it.
A
Is
you
know
what
makes
sense
here
and
what
can
we
expect
to
exist
on
the
the
ci
machines
and
what
won't
you
know
we,
you
know
our
clients
use
case
is
mostly
the
the
ubuntu
lxd,
tooling
and
so
on,
but
I
imagine
we'll
mostly
use
the
I
think
it's
the
unshare
command
or
whatever
they
use
to
create
a
namespace
in
linux,
for
the
tests
to
keep
them
basic
and
so
that
they
should
work
on.
You
know
all
the
supported,
kernels
and
so
on,
but
any
input
people
have
about
this
would
be
helpful.
G
A
I
Kind
of
related
to
this-
I
don't
know
if
there
would
be
interest.
I
haven't
touched
this
in
forever,
but
there
is
a
change
and
I'd
have
to
ask
to
see
if
it
was
who
originally
did
this
at
giant,
but
for
anyone
that's
used
zfs
on
newer
versions
of
solaris.
You
know,
there's
you
for
things
like
zones
which
may
be
applicable,
then
for
jails
or
whatnot.
There's
a
basically.
I
I
guess
it's
done.
It's
like
an
alias
so
that,
instead
of
having
when
you
when
you're
in
your
container,
whatever
you
want
to
call
it
instead
of
seeing
the
whole
path
to
whatever
that
dedicated
data
set
is
since
it
can
be
long,
it
can
basically
aliase
it
within
that
container
to
like
just
the
name
of
it.
So
so,
then
you
can
all
your
commands
and
everything
use
that
and
like
and
at
least
in
this
implementation
I
don't
know
how
it
was
done
on
in
solaris,
but
with
this.
I
Basically,
there
is
a
an
alias
property
which,
in
this
case
on
you,
know
in
a
lumos,
there's
also
zone
specific
data
which
is
kind
of
like
thread
specific
data,
but
it's
per
zone,
and
so
it
kind
of
sets
the
alias.
So
then,
when
you,
you
issue
the
octals
it'll
unalias,
the
data
set,
you
know
before
it
issues
it,
but
so
there's
code
for
that,
like
I
haven't,
touched
in
forever
because
it
I
believe,
though
it
pretty
much
worked
aside.
I
A
I
Yeah,
it's
the
only.
I
guess,
potential
issue,
which
really,
I
guess,
is
something
that
can
be
done
when
you're
creating
the
container
is.
Obviously,
if
you
have
two
different
data
sets,
you
know
where
the
you
know
in
terms
of
the
the
last
portion
of
the
name
is
the
same.
That
could
cause
a
conflict,
although
I
think
that's
you
know
probably
good
enough
just
to
say
you
know,
don't
do
that.
A
I
Well,
I
just
mean
like
if
you
try
to
like
okay,
you
delegate,
two
data
sets
to
your
container
and
they
happen
to
have
the
same.
You
know
the
last
component
of
each
of
those
names
happens
to
be
the
same.
Of
course
when
you
alias
it,
but
you
know
it's
probably
one
of
those
things
where
you
could
say
you
know
what
you
know.
Don't
do
that
you
are
just
you
know,
just
fail.
I
You
know
when
you're
setting
up
a
container,
it
says
you
know,
hey
that's
ambiguous
yeah,
I
mean
I
it's
probably
you
know
something
to
note,
but
I
don't
know
if
it's
something
that
would
really
be.
You
know
an
issue
in
practice,
but
just
you
know
that
is
out
there
that'd
be
the
only
little
gotcha
and
again
I
don't
since
I
don't
know
how
solaris
did
it.
I
don't
know
you
know
if
they
have
the
same
issue
or
if
they
did
something
else
where
then
that
doesn't
cause
that
or
whatnot,
but.
I
But
yeah
I
can
try
to
maybe
something
here
the
next
couple
weeks.
I
can
try
to
update
that,
and
maybe
it
may
put
it
up
for
review
than
if
there's
interest,
although
the
only
thing
I
guess
I'll
have
to
figure
out
for
the
open,
zfs
piece
is
like
I
said
it
does
rely
on.
You
know
what
we
call
like
the
zone,
specific
data
to
hold
the
some
of
the.
A
I
I
E
A
E
A
A
A
H
I
met,
if
possible,
I
hoped
ryan
murdered
to
do
that,
since
it
was
his
project,
but
I'd
like
just
to
attract
attention
to
apr,
which
is
11919,
which
is
we
talks
about
problem
with
extended
attributes
porting
between
freebsd
and
linux.
Originally
in
illumis,
science
historically
happens
that
we
have
two
completely
incompatible
implementation
and
we
at
existing
particularly
heat.
This
compatibility
issue
science.
We
have
two
products
on
freebsd
and
linux,
which
are
at
this
point
incompatible.
H
So
all
the
things
described
in
a
request,
if
short
on
linux,
I
think
user
space
attributes
are
prefixed
with
a
user
dot
and
and
then
name
on
freebies
users
knows
this
prefix
and
question
how
to
make
it
com
compatible.
So
if
somebody
have
interesting
area
of
extended
attributes-
or
I
have
ideas,
I'd
like
to
welcome
you
to
that
pr.
H
A
Yeah
I
haven't
looked
at
that
very
deeply,
but
thank
you
for
taking
on
that
bit
of
incompatibility.
That
is
one
of
those
things
that
hopefully
we
can
avoid
those
in
the
future.
You
know
through
the
the
coordination
of
open,
gfs
and
obviously
freebsd
and
linux
are
using
the
same
repo
now,
but
for
the
other
for
the
other
operating
systems
as
well
cool.
Well,
I
can
talk
a
little
bit
about
a
new
project
that
we're
working
on
at
delfix,
which
is
zfs
on
object
storage.
A
A
The
I
think
that
there's
been
some
attempts
or
thoughts
about
this
before
the
one
kind
of
interesting
thing
that
we're
bringing
to
is
like
our
use
case
is
databases,
so
we
need
this
to
work
well
and
perform
well,
even
if
you're,
using
small
block
sizes,
you
know
the
the
performance
of
object.
Stores
generally
is
like
the
latency
is
very
large,
and
you
also
need
to
use
large
objects
to
get
good
throughput.
A
A
A
B
A
A
J
J
That
needs
to
be
done
in
either
modifying
the
zil
to
like
you
know,
perform
better
or
some
other
component,
which
is
like
a
right
back
hash
that
that
would
kind
of
be
like
a
step
in
between
you
and
the
object
store
yeah.
So
you
really
get
more
like
a
layered
file
system.
At
that
point,
yeah
we
think
like
for
our
use
case.
We
think
the
zill
will
will
work
well,
but
you
know
the
proof
is
in
the
pudding.
G
A
J
G
A
A
Yeah
I
mean
it
could
definitely
be.
It
could
definitely
be
like
tuned.
The
sweet
spot
for
like
performance
is
a
few
megabytes.
So
like
one
to
eight
megabytes
on
on
s3
and
yeah,
I
mean
that
might
change.
You
know
with
different
cloud
providers
or
with
on-prem
solutions
like
min
io,
and
you
know,
that's
probably
something
that
would
need
to
be
configured
or
you
know
could
be,
could
be
configured
in
terms
of
the
target
object
size
there.
F
A
J
Yeah-
and
the
question
I
was
going
to
ask
for
you
know
with
regards
to
like
feedback
is,
if
folks
currently
are
not
necessarily
using
zfs,
but
using
like
these
object
stores
for
different
types
of
workloads,
and
you
think
that
zfs
might
be
a
fit
there.
It
would
be
interesting
just
to
kind
of
like
send
us
that
feedback.
J
A
Yeah
I
mean
we
want
to
obviously
develop
this
for
our
company's
use
case,
but
we
want
to
upstream
it
and
make
it
part
of
open
zfs
as
well,
and
so
you
know
we
want
to
make
sure
that
we
aren't
missing
anything
about
use
cases
that
are
adjacent,
that
we
can
also
make
work.
Well,
you
know
without
a
lot
of
extra
effort,.
E
E
Maybe
not
exactly
an
object,
storage
use
case,
but
I
was
thinking
about
like
if
there
is
some
infrastructure
to
group
changes
together
into
object,
store
sized
blocks
right.
That
seems
like
it
could
have
some
synergies
with
regards
to
smr
drives
host
managed
smart
drives.
Absolutely
I
see
I
see
some
synergies
there
and
if
the
software
architecture
can
be
made
such
that,
like
the
the
aggregation
of
changes
into
these
larger
sized
records,
can
happen
in
a
way
that,
like
the
next
layer
down,
can
be
either
smr
drive
or
object
storage.
A
Yeah
I
mean,
I
think
that
so
the
all
of
this
object
store
stuff.
It's
all,
as
I
mentioned
intermediated
by
userland.
So
zfs
is
talking
blocks
up
to
the
user
line
agent.
So
so
the
zfs
kernel
is
like
right.
This
3k
block
right
this
three
and
a
half
k
block
right.
This
2k
block
sending
those
commands
up
to
the
agent,
which
is
like
just
a
process
running
on
the
same
computer,
and
then
the
agent
is
coalescing
those
into
objects
and
then
sending
them
over
the
network
to
the
object
store.
A
B
A
A
A
The
sorry
I
just
lost
my
train
of
thought,
so,
oh
so
smr
drives
yeah.
So
you
could
imagine
like
plugging
in
like
teaching
that
userline
agent
to
be
able
to
write
to
smr
write
the
like
objects
to
smr
drives.
Instead,
I
imagine
that
since
a
lot
of
the
smr
drives
are
going
towards
what
they're
called
drive
managed
where
they
like
look
like
a
regular
drive,
they
put
a
bunch
of
smarts
in
there
to
do
the
mapping
themselves,
but
it
only
really
works.
It
only
really
performs
as
well.
A
One
of
the
tricky
things
with
this
is
that
what
if
your
workload
is
contains,
freeze
so
like
what,
if
you're,
actually
freeing
blocks,
freeing
a
significant
amount
of
blocks
over
time,
which
ours
does
for
sure,
then
you
know
in
order
to
actually
release
space
from
the
object
store.
You
have
to
read
and
then
rewrite
the
object
to
remove
those
blocks.
You
can't
just
like
freeze
a
little
bit
of
the
object
right.
You
got
to
read
the
whole
thing
and
then
rewrite
it
out.
A
Omitting
the
blocks
that
are
no
longer
part
of
it
that
that
kind
of
scheme
would
also
work
really
well
with
smr,
where
it's
like.
Okay,
great
we're
just
going
to
like
remove
this
100
meg
file
and
then
like
reallocate
another
one,
and
then
you
know
later
on
we'll
splat
another
different
under
mag
file
over
where
that
one
used
to
be.
A
J
Access
yeah
really
the
way
I
mean
we've
kind
of
architected
this,
because
we
have
this
new
vdf
type.
You
could
you
know
you
can
imagine
the
smr
being
driven
by
you
know
a
user
land
agent
or
some
sub
kernel
component
that
that
bdev
object
now
talks
to
so
you
could
implement.
You
know,
take
the
things
that
we're
putting
into
our
user
lan
agent,
bring
them
into
the
kernel
and
then
do
it
that
way
too,
like,
but
a
lot
of
the
principles
all
apply
here
like.
A
A
J
Yeah,
I
think
that
would
be
great,
I
think,
having
a
way
so
that
you
know
people
can
comment
on
kind
of
what
we're
proposing.
You
know
kind
of
the
keys,
the
new,
like
keywords
that
that
we're
thinking
about
and
kind
of
their
meaning
and
and
the
various
parameters.
I
think
it
would
be
great
to
get
feedback
and-
and
I
don't
know
like-
maybe
if
we
can
put
it
like
as
an
issues
that
would
be
great-
I
think
that's
a
little
bit
easier
to
manage
than
yeah.
Then
the.
A
All
right
here
we
can
do
that.
The
next,
probably
before
the
next
meeting,
yeah,
I'm
very
interested
in
what
the
user
interface
looks
like
and
trying
to
you
know,
make
sure
we
make
something
that
fits
well
and
and
has
the
flexibility
we
want
to
go
forward,
because
you
know
those
interfaces
tend
to
be
once
they're
released.
We
can't
ever
change
it
yeah
I
mean
the
we're
kind
of
doing
it
all
through
just
properties,
so
you
can
always.
I
would
definitely.
H
A
I
mean
we've
talked
about
that
already
in
because
the
new
properties
are
like
the
object,
store,
endpoint
the
like
http
endpoint,
the
bucket
well
sorry,
the
the
region
and
the
keystore
key
location,
which
is
kind
of
similar
to
the
encryption
key
location
where
it
can
be
like
you
know,
file
clone,
slash
and
then
inside
there
it
has
like
credentials
that
are
in
it.
You
know
specific
to
whatever
your
protocol
object,
store
protocol
is,
and
then
the
name
of
the
v
dev
is
the
bucket
name.
A
So
you
do
like
something
like
you
know:
zeus
create
dash
o.
You
know:
endpoint
equals
http
colon,
slash,
slash,
blah
blah
blah
dash
o
region
equals.
You
know
us
west
2
dash,
o
keys.
You
know
key
loc
object,
store,
key
location,
equals
file,
colon,
slash
whatever
and
then
s3
is
the
view
of
type
and
then
the
bucket
name
comes
after
that,
just
like
you'd
be
specifying
like
you
know,
mirror
and
then
device
one
device,
two
or
whatever
yeah.
A
I
think
that's
one
thing
we
haven't
looked
at
yet
with
the
v
dev
property
stuff
is
how
to
specify
properties
on
v
devs
as
you're,
creating
them
yeah.
I
mean
it's
interesting
because
here
you
know,
logically,
these
these
properties
obviously
apply
to
the
vita,
but
not
to
the
pool
as
a
whole,
but
we
can
kind
of
get
away
with
pool
properties,
because
it's
like
there's
really
like
there's
really
no
need
like
we're
planning
to
have
it
just
be
a
requirement
that
if
you're
doing
this
object
store
thing,
you
have
exactly
one
normal
class.
A
That
I
hadn't
really
thought
about
until
you
started
talking
about
it.
Just
now
is
specifying
vita
properties
as
part
of
the
the
pool
creation
and
how
to
do
that
in
a
way
that
doesn't
end
up.
You
know
tacking
a
bunch
of
stuff
on
the
end
of
each
v,
dev
as
you're,
specifying
it
or
whatever
each
member
of
the
vw
as
you're,
specifying
because
that
gets
really
ugly
yeah
yeah
yeah.
We
thought
about
that
and
we're
kind
of
like
well.
We
can
just
get
away
with
having
these
pool
properties
because
we're
only
going.
A
Bytes
scrubbed
number
of
bytes
to
scrub
or
whatever,
and
then,
like
you,
know,
building
something
on
top
of
that.
That
shows
a
progress
bar
or
whatever
yeah,
but
then
also
having
one
for
like
the
the
action
thing,
and
it
says
what's
wrong
with
the
pool
or
whatever
and
a
couple
other
ones
like
that,
and
some
of
those
may
be
applied
to
avidev,
not
the
whole
pool
now
and
you
can
get
really
complicated.
A
But,
yes,
more
properties
is
more
better
which
brought
up
the
other
thing
we
talked
about
at
some
point
was:
do
we
need
an
extra
property
type
like
verbose
or
something
that
isn't
displayed
by
default?
But
is
there
if
you
ask
for
it
if
we're
getting
to?
If
we
get
to
the
point,
we're
having
too
many
properties,
do
we
need
some
like
hidden.