►
From YouTube: CephFS Code Walkthrough: MDS Locker, Part 2
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
A
So
we
have
some
topics
that
we,
you
know
kind
of
left
out,
because
the
locker
is
a
very
big
topic
and
it's
hard
to
cover
it
in
one
code
walkthrough,
so
we'll
probably
have
one
more
of
these
to
cover
much
more
so
before
I
start
with
what,
before
I
start
with,
you
know
the
current
follow
through
of
the
last
walkthrough,
we'll
quickly
skim
over
what
we
discussed
last
walkthrough
for
this.
A
Since
it's
been
quite
some
time
and
it's
easy
to
forget
some
of
the
details-
and
you
know,
locker
involves
knowing
quite
a
lot
of
details.
You
know.
Otherwise
the
understanding
going
forward
is
a
bit
hard.
So
I
wrote
this
document
down.
This
is
basically
this
basically
will
end
up
as
a
pull
request,
maybe
in
tomorrow
or
day
after,
as
you
know,
for
you
know,
for
documenting
the
unlock
internals
for
mds,
locker
implementation
and
design.
A
So
you
know
whoever
you
know,
I
would
love
to
have
eyes
on
it
recommendations
you
know
changes
so
that
we
start
documenting
these
things.
Part
of
the
reason
for
doing
this
is
much
of
the
code
regarding
this
is
pretty
old.
I
mean
it
date
backs
to
some
of
the
source
code,
dates
back
to
2009,
that's
a
good
11
years
12
years
and
has
probably
never
been
touched
post
that
or
maybe
just
few
changes
here
and
then
and
that
took
pretty
minimal.
A
So
the
core
design
is
still
how
it
used
to
be
a
while
back,
and
it
obviously
makes
sense
to
document
this
so
that
you
know
new
journeys
and
people
from
the
community
can
get
them
or
get
themselves
up
to
speed
to
the
design,
which
is
a
much
complex
design,
and
you
know,
hopefully
it
helps
everyone.
Okay,
so
quickly,
it's
going
through
what
we
learned
last
in
the
last
walkthrough
for
locker,
why
we
use
logs?
Well,
you
know,
we
all
know
that
right.
You
know.
A
Basically,
everyone
use
loxes
to
protect
some
state
of
data.
In
this
case
these
are
various
metadata
pieces
in
the
inodes
and
the
dng.
The
thing
and
the
the
way
mds
does
it
is
instead
of
having
a
big
gigantic
lock
for
each
inode.
Well,
mds
has
like
mds
lock.
We
all
know
that,
but
that's
the
entry
point
lock
right,
but
once
it
enters
there,
you
know
there
are.
A
There
are
different
logs
for
different
pieces
of
metadata
in
an
inode
entity.
Entry
and
this
nmds
uses
different
kinds
of
locks,
so
you'll
see
that
different
classes
of
logs
and
since
different
metadata
in
the
inode
and
the
d
entry
have
different
behavior
in
various
situations.
A
So
what
happens?
Is
you
know
there
is
some
metadata
in
for
an
inode.
You
know
say
we
take
the
mode
access,
permissions,
node,
uid
and
the
gre.
You
know,
there's
hardly
any
you
know,
there's
there's
a
there's,
hardly
any
workload
that
does
you
know
parallel.
A
You
know
concrete
updations
to
the
uid
or
the
gid
or
maybe
there's
a
reader,
that
is,
that
is
looping
through
the
uid
and
gid
checking
it
again
and
again
and
there's
a
updater
client
that
you
know
sets
it
and
that
is
setting
it
again
and
again.
So
it's
possible,
but
it's
very
you
know
we
haven't.
B
A
No
real
world
use
case
for
this,
so
the
locker
is
optimized
to
do
all
to
to
to
satisfy
all
these
requirements
in
a
in
an
optimal
fashion.
In
the
sense
locker
will
treat.
You
know
things
like
uid
gid
and
a
bunch
of
other
metadata,
as
you
know,
only
allow
read,
lock
and
exclusive
logs,
which
is
like
you
know.
Many
clients
can
read
congruently,
which
is
fine,
but
when
some
client
wants
to
update
it,
it's
an
exclusive
log.
A
So
all
the
read
logs
are
now
you
know
all
the
gaps
from
the
from
the
readers
from
the
reader
clients
are
now
revoked.
While
there
are
some
other
types
of
metadata
that
need,
you
know,
shared,
read
and
shared
right,
typical,
typically
in
the
mds
you'll,
think
this
this
for
the
recursive
statistics,
information
and
the
directory.
A
So
you
know
very,
very
quickly
and
imds.
We
all
know
it
tracks
a
bunch
of
heat
maps
for
each
particular
inodes
and
the
entries
for
ions
rather
and
then,
when
the
heat
or
the
excess
frequency
for
that
particular
node.
Let's
call
it
node
node,
you
know,
hits
a
threshold.
The
mds
thinks
okay.
This
has
now
been
accessed.
Frequently,
let's
go
and
split
this
directory,
so
so
this
is
the
dynamic
separate
partitioning.
A
So
when
it
does
it,
it
splits
the
directory
into
multiple
charts
and
then,
if
you
have,
if,
if,
if
a
setup
is
running
multiple
active
mdss,
you
know
some
of
these
shards
can
be
migrated
to
some
other
mds
so
that
you
know
client
workloads
are
now
balanced.
So
for
for
this
kind
of
scenarios
the
mds
allows
shared,
read
and
share
right.
How
it
allows
share
to
write
is
for
a
particular
directory
node.
A
You
can
have
a
client
creating
creating
some
files
in
this
directory,
which
is
mapped
to
say,
mds
rank
zero
and
then
a
shard
which
is
mapped
to
mds
rank
one.
Some
other
clients
can
be
creating
directories
in
that,
so
the
recursive
stats
and
the
refractory
are,
you
know
they
can
be
state,
read
and
share
right.
So
the
mds
allows
that
so
it
so
this
particular
log
class
is
now
optimized
for
this
particular
kind
of
metadata.
A
So
instead
of
having
one
big
log
for
an
inode
or
a
d
entry,
it
kind
of
you
know
splits
it
into
into
splits
the
metadata
or
rather
groups
the
metadata
into
similarly
access
configuration
types
in
the
sense
you
know,
so
we
know
that
you
know
like
auth.
If
you
see
this
self
lock
iot,
this
protects
this
mode,
uid,
jid
and
c
time
in
particular,
so
these
are
grouped
into
I
off.
A
So,
whenever
you're,
taking
a
read,
lock
or
say
a
read,
lock
it's
safe
to
read
only
these
three
or
four
metadata
pieces.
If
you
have
taken,
if
you
have
acquired
the
ix
catalog,
it's
safe
to
read
the
excitation
dc
time
and
similarly
for
snaps
and
these
all
so
you
know,
if
you
see,
we
have
like
around
12
to
13
types
and
each
type
covers
a
particular
piece
of
the
metadata
in
that
I
denote
and
the
the
entry.
A
So
these
are
log
types,
so
you
know
quickly
covering
this
is
like
a
split.
These
are
types
these
are
locked,
lock,
types
that
that
that
protect
a
particular
piece
of
information
in
the
inode
or
the
the
entry.
Now
the
interesting
part
comes
as
log
classes.
A
Log
classes
define
locking
behavior
for
the
associated
lock
type,
which
is
necessary
for
handling
distributed
logs
so
mds,
because
we
can
have
multiple
active
mdss
in
a
setup
and
clients
can
be
talking
to
more
one
or
more
than
one
mds
is
even
for
the
same
inode.
You
know
we
need
a
distributed,
lock
mechanism,
you
know
for
safety
for
access,
it's
so
so
how
this
lock
classes
work
is
mds,
defines
three
lock
classes,
or
rather
four
but
majorly.
Three
one
is
the
local
lock.
A
The
local
lock
is
used
for
data
that
doesn't
require
distributed,
locking.
So,
if
you're
trying
to
increment
the
inode
version
or
the
dng
version,
you
really
don't
need
the
distributed.
Lock.
All
you
need
to
be
sure
is
you're
in
the
authoritative
node
for
that
particular
inode,
and
then
you
can
take
this
lock.
It's
like
it's
a
it's
a
version
lock
it
here,
just
counters
you
take
this
lock
increment
the
version
and
you're
done.
The
other
lock
class
is
simple
lock.
So
this
is
the
base
class
implementation
for
distributed
lock.
A
It
defines
much
of
the
locking
rules
or
or
or
or
yeah,
locking
rules
to
implement,
distributed,
locking,
simple
lock,
which
is
what
we'll
be
covering
today
majorly.
A
You
know
later
we'll
see,
scatter,
lock
and
file
lock,
so
simple,
lock
is
used
for
data
that
requires
shared,
read
and
mutually
exclusive
rights,
as
I
mentioned
earlier,
you
know
things
like
the
mode
uid
gid
things
like
the
link
count
x,
hatters,
you
know
snapshot
list
and
all
they
require
shared,
read
and
mutually
exclusive
rights
because
of
the
access
patterns.
A
Right
I
mean
there's
this
there's
hardly
any
excess
pattern
that
that
does
you
know
parallel
reads
and
parallel
rights
for
uid
id,
but
such
a
pattern
is
pretty
common
for
for
for
for
different
files
in
a
directory.
So
we
have
so
that's
how
we
differentiate
the
mds
differentiates.
You
know
different
access
access
types,
so,
as
I
said,
simple,
simple
lock
is
the
base
class
that
defines
rules
for
distributed.
Locking
and
many
metadata
pieces
are
simple
lock,
I
think
majorly,
it's
simple,
lock
and
probably
just
a
couple.
A
Maybe
two
or
three
are
like
catalogs
and
file
logs.
So
we'll
come
a
simple
log
in
this
talk
just
colors
catalog,
so
this
is
used
for
data
that
requires
shared,
read
and
shared
right.
So,
as
I
said,
things
like
you
know,
recursive
starts
and
refractory
and,
like
you
know
those
kind
of
things
they
require
shared,
read
and
share
right.
So
the
mds
also,
can
you
know
kind
of
delegate
some
authority
to
others,
mds
replicas.
A
So
if
a
file,
if
a
particular
I
node
is
getting
really
hot
access
is
getting
really
it's
been
accessed
by
a
client
a
lot,
so
the
mds
can
now.
You
know,
create
a
replica
for
that
particular
inode
and
then
distribute
that
replica
copies
among
other
active
mdss
which
again
can
generate
read
capabilities
for
clients,
so
that
requires
you
know
a
shade,
read
and
shared
right
kind
of
kind
of
access
mechanism.
A
There's
a
catalog
comes
in
the
picture
and
then,
in
addition
to
this,
there
is
a
file
lock,
which
is
like
a
special
case
of
catalog.
Again,
that's
for
sharing
and
share
right,
but
additionally,
with
that,
it
also
protects
pieces
of
metadata
that
requires
share
rate
and
exclusive
right
things
like
m
time.
Things
like
the
size,
and
that
requires
most
of
it-
requires
share,
read
and
share
right,
but
these
additional
metadata
pieces
require
shared,
read
but
exclusive
rights.
So
that's
where
pilot
comes
into
picture.
A
B
A
Log
types
are
mostly
for
for
the
ci
node
structure:
there's
only
like
couple
of
them
for
the
entry
structure,
so
we'll
majorly,
just
you
know,
concentrate
up
with
I
nodes
in
this.
So
if
you
go
to
the
ci
node
source.
A
You'll
see
think
of
this
first,
so
it
defines
a
bunch
of
log
types
and
then
you
know
it
defines
which
are
simple
logs
which
are
local
logs
and
which
has
catalogs.
So
just
you
know
kind
of
rearrange
this
for
privacy,
simple,
look
majorly!
We
have
simple
log,
like
you
know,
on
the
auth,
like
modularity
link,
count,
etc.
Snap
layouts
file
locks.
All
these
require
just
simple
locks.
Then
we
have
scatter
locks,
the
the
the
the
complex
one,
that
is,
for
the
inis
and
idft,
which
is
nothing
but
the
nested
info.
A
The
recursive
stats
and
dft
is
the
date
factory,
so
scatter.
So
these
two
are
scatter
logs,
and
I
explained
why
do
we
need
catalog
for
this
right,
because
we
need
shared,
read
and
share
right
kind
of
scenario
file
lock
again.
This
is
kind
of
a
special
case
for
catalog,
and
this
for
the
I
file
type
again
I
file.
A
You
know
it
protects
m
time,
a
time
size,
truncate,
a
couple
of
things
for
from
handling
truncates
client
pages
and
then
inline
data,
and
then
we
have
the
most
impressive
type
of
log,
which
is
like
local
lock.
As
you
can
see,
these
are
all
version
locks.
The
high
version
is
for
the
inode
version.
D
version
is
for
the
entry
version.
These
are
just
you
know,
local.
These
are
not.
These
are
not
distributed
logs.
These
are
always
taken
by
the
author
mds
for
that
particular
anode.
A
A
So
now
for
each
of
these
log
types,
we
you
know,
mds
allows
you
to
acquire
these
logs
in
three
modes
normally
would
have
seen.
You
know
we
have
just
typical.
If
you
use
a
p
thread
lock,
you
know
you
can
or
a
normal
log
your
text,
you
can
use
it
in
read
mode.
You
can
either
use
it
as
a
as
a
plain
matrix.
Just
like
somebody
has
taken
a
lock
irrespectively,
it
does.
A
read
or
write.
A
Operation
takes
the
log
so
that
guarantees
mutual
exclusivity
and
then
modifies
the
data
in
question
and
then
comes
out
and
that
unlocks
there
is
another.
So
this
can
be
taken
forward
with
read
likes
and
reader
logs,
in
which
you
know,
multiple
readers
are
allowed
to
access
a
particular
data
structure.
So
you
take
read
logs,
you
can
have
congruently
three
for
client
three
four.
You
know
entities
reading
under
that
read
lock,
but
once
you
take
a
right
lock,
you
know
no
readers
or
writers
are
allowed.
So
that's.
The
exclusivity.
A
Mds,
however,
defines
this
in
three
modes:
one
is
the
rd
lock,
which
is
the
shared
read
block
the
w
are
lock,
which
is
shared
right
lock
again.
So
there
is
not
the
exclusive
writer
log
wrlock
is
a
shared
write
lock
and
the
exclusive
right
exclusive
block
is
called
xlock,
so
rd,
lock
and
x
lock
are
pretty
self-explanatory.
A
We
all
know
that
straight
rate
share
right.
We
just
discussed
w
log
is
special,
since
it
allows
congruent
right
writers
and
the
thing
is
wr.
Log
is
only
valid
for
the
log
class
type
scatter
and
pi
lock.
So
you
won't
see
the
wr
lock
been
taken
for
simple
locks,
yeah
my
iauth
or
I-link
types.
You
know
they
only
support,
read
locks
or
exclusive
blocks.
A
Even
when
we
go
to
the
state
machine
transitions,
we'll
see,
you
know
it's
only
the
read
lock
or
the
exclusive
lock,
that
is,
that
makes
sense
for
that
particular
log
class,
so
wr
lock
only
makes
sense
for
the
scatter
lock
and
the
file
lock
and
we'll
see
the
state
machines
and
why
I
kind
of
explained
this.
A
You
know
wlr,
you
know,
allows
multiple
writers.
At
the
same
time
again
when
a
large
directory
is
kind
of
split
into
shards
or
notes,
you
know
each
of
these
charts
can
be
kind
of
you
know,
dedicated
to
a
particular
active
mds
and
when
new
files
are
created
under
these
fragments,
you
know
mds
needs
to
account
for
the
recursive
stats.
So
each
of
these
two
shards,
because
the
stats
for
these
two
shots
are
kind
of
accounted
independent
accounted
independently
by
by
those
mdss.
A
So
what
happens?
Is
the
mds
kind
of
gathers
all
these
accounting
to
the
authentics
and
then
the
read
lock
is
guaranteed
so
scatter
lock
and
file
lock?
Are
they
have
this
scatter
gather
mechanism?
Scatter
is
when
you
take
a
right
lock
and
the
eyes
are
scattered
in
the
sense
you
know.
A
There's
this
special
remote
wr
lock.
That
is
only
valid
for
cases
where
you
do
a
rename
where
the
destination
g
entry
is
actually
on
a
different
mds
than
the
current
one.
So
the
source,
the
entry
might
be
your
time
ds
might
be
on
mts
running
zero,
but
you're
trying
to
rename
it
to
dentry,
which
dng,
which
lies
on
mds
rank
one.
A
So
the
mds0
will
send
a
lock
message
to
mds
rank
one
with
there's
a
so
that
is
not
mentioned
in
all
those
inter
mds
log
transitions
are
not
covered
in
this
talk
we'll
cover
this
in
the
next
talk,
but
just
touching
upon
it
there's
an
inter
mds,
lock
message
that
is
sent,
and
so
the
md0
can
tell
mds
one
to
kind
of
right
lock,
so
that
operation
is
called
the
remote
wr
lock.
A
Okay,
now
we
have
lock
states
and
lock
state
machines,
okay,
so
these
are
the
most
interesting
things
and
we'll
just
stick
to
simple
lock
for
this
we'll
touch
very
briefly
in
scatter,
lock
and
file
log,
so
that
we
get
a
feel
of
the
state
machines
and
you
know
how
the
state
machines
differ
between
simple,
lock,
scatter
lock
and
file
lock,
but
we
won't
go
into
more
detail
because
it's
very
confusing,
so
you
need
to
first
understand
how
simple
lock
works.
Get
used
to.
A
It
may
try
out
a
few
things
on
your
own
and
once
you
know
you
have
like
a
good
understanding
of
simple
log,
we'll
see
how
scatter
lock
works.
Okay,
so
so
mds
defined
various
log
states,
so
we
have
a
whole
bunch
of
states.
The
good
thing
is
for
a
particular
log
class.
A
We
don't
have
to
worry
about
all
these
states,
so
we
have
about,
I
think,
35
to
38
states
total
number
of
states,
but
for
simple
lock
you
know
you
might
just
touch
a
handful
of
them
as
catalog
and
phi
lock
are
much
more
complex
and
they
touch
a
much
more
bunch
of
states.
So
we'll
see
where
all
this
lives
go
to
cmds
logs.h.
A
So
these
are
the
log
states.
You
know
we
have
a
whole
bunch
of
them.
You
know
it's
obvious
to
say
it's
like
not
documented
at
all,
so
the
only
way
to
do
it
is
to
go
through
the
code.
Do
some
experiment
and
see
what
each
of
these
log
states
mean.
A
As
I
said,
simple
lock,
I
don't
have
to
worry
about
much
of
these
dates,
we'll
go
to
the
state
machine
and
see
you
know
it's
just
a
subset
of
these
and
this
lock
state
that
simple,
lock
is
concerned
for
and
okay.
So
one
thing
which
is
very
important
here:
is
these
log
states
what
you
saw
the
log
underscore
line
or
underscore
blind
block?
These
are
not
locked
themselves,
it's
pretty
confusing
because
you,
you
know
by
naming
you
kind
of
expect
them
to
be
logs,
but
these
are
not
locks.
A
B
A
Locks
themselves,
and
if
you
see
there
are
two
kinds
of
naming
convention
here,
if
you,
if
you
paid
in
attention
closely,
you
will
see,
you
had
lock
underscore
a
state
and
then
you
have
lock
underscore
blah
underscore
block,
which
is
like
state
one
and
state
two.
So
the
meaning
for
this
is
how
these
are
defined.
Is
each
state
follows
a
naming
conversion,
which
is
like
log
underscore
state
name.
A
So
if
you
see
here,
the
state
is
like
sync
here:
the
state
is
like
log
here:
the
state
is
x,
log
at
3,
x,
log
and
the
state
is
x
log,
so
log
underscore
state
name
depicts
that
the
current
state
is
the
leading
the
leading
name
after
the
underscore.
The
other
convention
is
lock
underscore
something
underscore
something.
This
is
an
intermediate
state
this,
which
is
also
called
as
a
transition
transition
state.
A
So
when,
when
the
mds
wants
to,
you
know,
move
from
one
state
to
another,
because
a
client
is
doing
a
some
operation,
maybe
it
wants
to
take
an
an
exclusive
block.
It
moves
from
one
state
to
another
and
we'll
see
why
these
intermediate
states
are
required,
but
you
know
think
right
now.
Think
of
this
as
mds
needs
to
go
from
one
state
to
another
by
visiting
a
bunch
of
intermediate
states,
so
these
intermediate
states
are
named
as
so.
A
If
you
want
to
go
from
state
a
to
b,
there's
say
one
intermediate
state,
so
you're
in
state
a
lock
a
say,
say:
lock
a
and
you
want
to
go
to
log
b
and
there
is
one
intermediate
state.
The
intermediate
state
is
named
as
lock
underscore
a
underscore
b.
A
Okay,
so
that
so,
if
you
see
here
log
sync
clock
this
means
it's
going,
the
intermediate
state
is
going
from
pink
state
to
the
log
state
and
the
intermediate
state
is
locks.
Log,
sync
lock
state,
then
it
is.
It
is
a
bit
confusing
with
all
these
log
naming
carried
around
everywhere
and
I
just
hoped
it
was
named
something
else
we
could
do
that.
But
for
now
just
think
of
these.
As
you
know,
intermediate
states
going
from
state
a
to
state
b.
A
A
Okay,
I
think
we
can
go
forward
so
each
of
these
states.
You
know
we
will
not
worry
about
intermediate
states
right
now,
we'll
just
you
know
quickly
see
what
what
each
of
these
states
mean
the
the
primary
states.
These
are
also
called
as
stable
states.
You
know
by
stable
states.
A
So
if
you
see
the
logs.h
we'll
see
the
state
machine,
but
this
is
how
the
state
machine
entries
look
like
you
have
a
next
state
which
you
want
to
which
the
which
the
mds
you
know
says
this
is
the
next
state
we
want
to
go.
The
next
state
is
zero,
which
means
it's
a
stable
state,
stable
state
means
when
an
mds
is
done,
transitioning
from
one
state
to
another.
You
know
when
the
request
completes
and
everything
is
done.
A
You
know,
there's
a
tendency
for
the
mds
to
go
into
the
stable
state,
so
in
the
in
the
state
machine
you
will
see
you
know,
you
know.
Each
of
these
entries
have
something
called
if,
if
it
does
have
another
state,
it
says
you
know
this
is
next
state
to
go
or
if
it's
zero,
it's
the
stable
state
and
there's
always
a
tendency
to
go
towards
stable
state.
A
A
If
you
are
in,
like
sync
state
for
a
particular
lock
type,
when
you
get
a
read
lock,
you
know
you
don't
have
to
transition
between
states.
So
since
it
says
you
can
relock
freely
and
you
know
you're,
you
know
you're
done
acquiring
the
read
lock.
But
if
you
want
to
write
log,
then
obviously,
because
things
think
straight
doesn't
allow
you
to
take
right
or
excuse
log
into
transition
states
which
means
going
into
an
intermediate
state
and
then
to
another
state.
A
So
next,
lock
lock
is
nobody
can
read
lock,
but
only
primary
mds
can
write
lock
or
exclusive
lock.
So
for
simple
lock,
you
mostly
need
to
understand.
Lock,
sync
lock,
lock
and
and
lock
x,
lock,
lock,
mix
comes
into
picture
when
you're
dealing
with
scatter
locks
and
file
locks.
A
So
a
lock
lock
is,
I
mean
somebody
is
some
client
is
already
holding
an
exclusive
lock.
There
are
other
bunch
of
states
which
are
really
not
valid
for
simple
logs.
You
know,
but
most
of
them
are
really
optimizations
four
blocks
which
needs
to
be.
You
know
the
understanding
needs
these
on
the
basis
of
understanding
the
distributed
log
behavior,
there
are
other
logs,
like
logs
xn
x,
in
analog
design.
There
are
kind
of
optimizations
over
the
log
mix
state.
You
know
we'll
cover
those
later.
A
A
So
I
did
mention
you
know
intermediate
stage
like
you
know,
log
underscore
from
state
to
state.
They
knows
the
transition
of
a
log
from
one
state
to
another,
okay
yeah.
This
is
where
it
gets
really
interesting,
where
we
see
state
machines
and
things
like
that.
A
Okay,
so
when
does
the
md
is
you
know,
there's
all
these
state
transitions
transition
of
a
log
from
one
state
to
another
is
mostly
when
a
client
does
a
request
when
a
client,
just
you
know,
does
an
operation
that
reaches
the
mds
and
the
mds
fees
that
it
needs
to
kind
of
not
do
all
these
log
changes
or
the
other
case
is
if
the
mds.
A
Excuse
me
or
the
other
cases
if
the
mds
is
is,
has
picked
up
a
node
from
tree
migration.
You
know,
maybe
because
it
thinks
balance
circuit
got
invoked
and
then
it
needs
to
kind
of
migrate.
This
particular
subtree
to
another
mds.
So
you
know,
even
in
that
case
it
might,
you
know,
prompt
a.
A
Transition
of
logs,
so
we'll
pick
up
a
case
here
where
you
know
we'll
deal
with
simple
locks,
we'll
deal
with
the
iot
lock.
So
iot
is
like
you
know,
it
protects
the
mode
uid
gid.
So
we'll
pick
up
a
case
where
one
client
is
doing
a
read
on
the
inode.
A
On
this
particular
note,
I
know
metadata
piece,
which
is
the
uid
gid
say,
and
another
client
is
confidently
doing
an
update
of
the
uid
and
theory
and
we'll
see
how
the
lock
mds
transitions
they
lock
so
say
the
read
and
the
client
that
that
that
is
doing
the
read
of
the
inode
metadata
piece.
You
know
it
probably
did
a
couple
of
times
and
then
it
you
know
it
got
the
uidj
and
yeah
it's
so
the
client
is
still
array,
which
means
it
has
the
a
s
caps.
A
So
a
is
for
auth
and
s
is
for
shared.
So
I
didn't
cover
more
caps.
The
the
cap
format
in
this
particular
talk.
It's
pretty
well
documented
in
one
of
the
help
documents
that
we
have
you
know
or
in
the
docsuff.com.
So
please
go
look,
look
at
it
up
there.
So
so
you
can
have
like
you
know,
for
each
particular
cap
type
and
so
a
is
for
auth.
A
You
can
have
either
the
shared
cap
or
the
exclusive
gap,
and
because
this
particular
client
did
a
read,
you
know
the
md
has
probably
assigned
a
shared
cap
for
it.
So
if
it
probably
so,
if
it
did
a
start,
you
know
trying
to
fetch
the
module
id
since
it
already
has
the
shared
cap.
You
know
there's
no
need
to
query
the
mds,
although
the
client
can,
you
know
satisfy
that
request
from
the
from
from
its
local
cache
itself,
which
is
the
inode
cache
dyno.
A
So
now,
when
the
other
client
does
a
set
at
a
call
to
the
mds,
you
know
the
mds.
You
know
so
we'll
see
this
where,
where
did
it
do
time
to
see
some
code
mds
server
from
in
the
end?
What
happens?
Is
you
know
when
a
client
does
they
subtract
a
call?
It
passes
through
a
bunch
of
layers
in
the
mds
since
of
source
classes,
handle
client
that
atta
reach,
oh
matter.
So
here
is
this
piece
of
code
here
now,
depending
on
what
flags
have
been
passed.
A
What
kind
of
mask
has
been
passed
by
the
client?
You
know
we
we,
we
add
different
locks.
So
for
the
simplistic
case
we'll
do
here,
you
know.
Obviously
you
know
kind
of
we
are
having
these
file
lock
and
version
logs.
You
know
we
will
not
go
into
detail
into
this.
A
You
know
we'll
just
be
concerned
about
this
case,
because
we
need
to
kind
of
understand
how
simple
lock
works
and
then
we
can,
you
know,
add
these
complexity
layers
of
what
happens
when
a
simple
lock
is
taken
and
then
a
file
lock
is
there.
So
since
we
have
so
the
client
probably
passed,
you
know
he's
interested.
That
client
is
interested
in
fetching
the
module
id
and
the
gid.
So
we
did
a
statics
pass
these
flags
down
and
we
came
here-
and
you
know
this
particular
line
got
sufficiently.
A
You
know
valid
and
then
you
know
we
added
a
xlock
for
the
outlook.
So
a
little
bit
of
information
here
is
the
how
the
locker
class
is
interfaced.
The
locker
class
must
many
of
the
apis
in
the
locker
class
accept
something
called
as
a
lock
op
vector.
A
So
what
you
can
do
is
one
of
the
users
of
the
locker
class
and
pack
in
a
bunch
of
different
lock
types
in
a
vector
which
is
called
the
lockout
vector
and
then
hand
it
over
to
the
acquire
locks.
Api
interface
for
the
mds
to
acquire
the
lock
in
an
optimal
fashion.
B
A
The
lock
off
vector
is
nothing,
but
you
know
if
you
see
in
in
it,
it's
actually
just
a
vector
with
obviously
some
class
typing
around
it,
but
in
the
end
it's
like
a
vector
of
logs,
so
we
can
add
different
clocks.
You
can
add
x
lock
for
the
auth
log.
A
You
can
have
x
lock
for
file
lock
in
this
case,
and
you
can
have
a
write
log
for
the
version
lock
and
then
you
call
the
acquire
locks
api,
giving
the
current
request
and
the
lockout
vector
and
then
so
what
the
acquire
locks
api
does.
Is
it
first
you
know
kind
of
rearranges
the
different
locks
in
the
lock-off
vector
in
a
particular
order.
A
It
always
does
that,
so
you
can
pack
all
these
locks
in
any
order
and
the
acquire
locks
interface
will
kind
of
reorder
it
and
then
for
each
of
the
lock
that
that
we
have
placed
it
will
try
to
acquire
the
locks.
A
Okay.
How
so
here
we
are
only
mostly
concerned
about
the
x,
lock
and
the
outlook.
Let's
see
if
you
see
locker,
where
logs
so
we'll
see
here
so
first,
you
know
it
kind
of
it
does
more
than
that.
Actually
it
doesn't.
It
talk
also
pins
the
inode
that
have
been
that
are
involved
in
the
logs,
so
we'll
skip
that
for
now.
Yeah
here
is
the
shortened
merge
where
it
sorts
the
the
different
locks
in
a
particular
order
and
probably
merges
it.
A
Then
again
it
iterates
over
the
pins
local
opens.
This
is
not
of
interest
for
this
stock.
A
Do
this,
or
if
it's
off
right,
lock
or
remote
clock
or
do
this.
If
it's
like
else,
you
know
the
else
case
is
like
for,
if
it's
a
read,
lock
do
this,
so
it's
pretty,
you
know
three
blocks
that
we
have
to
be
worried
about,
and
it
does
this
for
each
particular
lock
entry
that
we
have
put
it
in
the
lock
that
that
we
entered
in
the
lock
of
factor.
A
So
coming
back.
You
know,
don't
worry
about
the
file
lock
in
the
version.
Lock
right
now
concentrate
about
the
outlook.
So
we
said
you
know
the
satatta
needs
to
modify
the
uadj
and
the
mode,
so
we
need
to
take
an
exclusive
block
for
the
auth
lock.
So
what
happens?
Is
we
come
here?
We
reach
here,
and
you
know
we'll
go
through
the
code
in
a
little
bit
right
now,
but
let's
see
what's
actually
happening
under
the
hood.
A
What
happens
is
so
again,
as
I
saw
you
know.
As
I
said
you
add
this
a
clock
now
the
acquire
locks
thing.
It
goes
to
various
branches
and
you
know
in
the
end.
What
happens
is
it?
It
knows
that
you
know
this
particular
for
for
that
it
knows
that
the
lock
that
it
needs
to
x
lock
is
of
a
simple,
lock
class
type,
so
it
goes
and
fetches
the
it
sees
the
same,
simple,
lock
state
machine.
A
Okay,
yeah,
I
mean
it's
not
wrapped
and
it
needs
to
be
in
a
single
line,
for
you
know
be
really
clear
anyway.
So
this
is
the
simple
log
state
machine,
so
how
the
state
machine
is
written
is
unit
for
each
of
the
entries
follows
this
particular
stock
type.
B
A
A
So
whenever
you
initialize
a
lock,
you
know
the
starting
state
is
always
lock.
Sync,
you
know
it's
hard
to
code
it
and
the
starting
state
is
lock
sync.
Now
what
happens
is
so
the
mds
wants
to
now.
You
know:
go
from
the
log
sync
state
to
the
x,
lock
state,
let's
lock
state,
because
we
need
to
x
lock
that
particular
auth
lock.
We
need
to
go
from
log
sync
to
x,
lock,
state.
A
So
the
thing
is
now
we
go
into
bunch
of
intermediate
states,
okay,
so
how
it's
done
is
first,
you
know.
So
if
you
see
you
know
the,
I
have
detailed
something
here.
So
in
the
log
sync
state,
you
can
only
read
lock.
A
A
So
any
is
like
any
one
author
replica
auth
is
like
auth
only
is
like
the
ex
excel
is
exclusi
is
either
the
auth
or
the
exclusive
client
request.
We
are
not
bothered
about
this
right
now.
So
log
sync
says
log
sync
state
is
you
know
nobody
can
write
our
exclusive
log,
but
they
can.
You
can
only
read
lock
the
lock,
lock
state
says
if
you
go
back
here,
nobody
can
read
log,
but
the
primary
and
mds
can
write
our
exclusive
lock.
Okay.
A
So
if
you
see,
we
need
to
kind
of
go
from
lock,
sync
state
to
lock,
lock
via
to
lock
x,
lock
via
lock,
lock,
so
lock
clock
is
it's
a
stable
state
because
you
know
it's
lock,
underscore
state
name
but
again
it's
an
intermediate
state
kind
of
an
intermediate
state
and
we'll
and
I'll
explain
why
we
choose.
A
While
we
why
we,
you
know,
go
through
various
intermediate,
but
for
now
let's
see
here
so
we
need
to
go
from
log
sync,
to
lock
x,
to
lock
x,
lock
via
lock
lock.
So
the
way
this
is
done
is
so
we
need
to
go
from
sync
to
lock.
So
the
next
state
that
is
chosen
here
is
lock,
sync
lock.
So
if
you
see
talk,
sync
lock,
so
we
jump
to
this
state
there's
a
concept
of
loaner
which
I'll
explain
not
really
required.
A
Now,
it's
good
to
know,
but
not
really
a
requirement
to
understand
the
state
machine.
This
state
says
you
know
you
know.
Log
sync
lock
says
you
know
you
need
to
now
go
from
log
sync
to
lock
sync
lock
as
the
next
intermediate
state.
This
intermediate
state
has
two
purposes
actually
so
the
purpose
first
purpose
is
you
know
at
in
in
this
state
this
state.
A
If
you
see
you
know
it
says
no
and
for
each
particular
intermediate
state,
only
some
kinds
of
some
types
of
caps
are
allowed
to
be
taken
to
be
taken
by
the
client.
A
So
the
log
sync
clock
state
says
that
you
know
these
are
mentioned
here.
If
you
see
these
last
four
fields,
which
are
nothing
but
you
know
the
gaps,
the
loaner
caps
x,
logger
caps
and
the
replica
caps,
you
know
don't
don't
for
now.
For
simplicity,
just
worry
about
these
gaps.
It
says
you
know,
log
six
state
says
you
know
you,
you
can't
have
clients
which
have
taken
the
shared
log
for
this
particular
lock
type.
So
you
can't
have
clients
that
have
taken
the
a
s
lock.
A
A
Requirement
for
having
an
intermediate
state
is
when
you
have
reached
its
intermediate
state,
it
prevents
further
read
locks
to
be
taken
so
when
the
mds
has
reached
locks
and
clock
state,
it
sends
these
revoke
messages.
You
know
cap
rebook,
messages
to
clients,
it's
now
in
the
sync
lock
state,
but
if
some
other
client
comes
in
and
tries
to
read
lock
now,
even
that
should
be
disallowed.
So
this
intermediates
it
solves
two
purposes.
One
is
to
actually
revoke
conflicting
caps
from
the
client
and
the
other
is
to
prevent
further
locks.
A
Certain
locks
to
be
taken,
and
now
what
happens
is
once
the
client
has
replied
to
the
cap
report
message.
So
we
sent
a
cap
revoked
message,
like
you
know,
revoke
the
orchard
caps.
The
client
reply
drops
that
particular
cap
and
replies
back
and
next
what
happens?
Is
we
move
from
the
lock
sync
lock
state
to
lock
lock,
so
the
next
state,
if
you
see
here,
is
the
lock,
lock
state
so
one
we
once
we
move
to
the
lock
clock
state.
A
Our
end
goal
is
to
acquire
the
x
lock,
but
we
can't
so,
which
means
from
lock
lock.
We
need
to
go
to
the
x
lock
state
again.
How
do
we
do
that?
We
do
that
by
going
to
an
intermediate
street,
which
is
lock
lock
and
x
lock,
so
we
go
to
this
state.
Do
the
same
thing
again:
if
there
are
like
conflicting,
you
know,
gaps
that
need
to
be
revoked.
We
do
that
and,
and
once
that
is,
then
we
jump
more
states
like
from
here.
A
A
So
if,
if
you
see
here,
you
know
what
happened,
was
I
I've
taken
this
from
the
logs
again,
for
you
know
for
for
this
case,
where
we
are,
one
client
is
trying
to
get
an
x,
lock
and
some
of
the
clients
have
these
shared
caps,
the
mds.
You
know
the
mds
sends
a
cap
revoked
message,
so
it
says
this
particular
end.
It
has
these
gaps,
so
you
can
see
the
odd
shared
caps,
but
you
know
that
is
not
allowed.
So
these
are
the
only
allowed
caps
you
have.
A
So
we
send
these
to
the
client,
for
the
client
drops
the
oddsshark
caps
replies
back.
So
where
is
this
done?
Cmds
dot
cc
so
most
of
this
starts
with
x.
Log
start
since
we
are
dealing
with
x
log,
but
there's
there's
lots
of
conditions.
Here
we
get
to
a
point.
You
know
where
we
do
a
simple
lock,
because
we
are
in
like
sync
state
and
that's
a
simple
lock.
So
in
simple
lock
we
see.
A
What's
the
current
state,
we
are
initially
we
are
trying
to
move
from
the
sync
state,
so
the
next
state
is
like
boxing
clock
once
that
is
done.
Here
is
the
thing
that
happens
so
here
we,
the
mds,
finds
out.
You
know
that
refers
to
the
state
machine.
This
is
in
c
I
not
by
the
way
cinder
dot
cc.
So
if
you
see
shoot
gaps
need
gather,
you
see
it.
A
You
know
refers
to
that
particular
state
machine
and
and-
and
you
know-
gives
and
gives
back
whether
we
need
to
kind
of
revoke
certain
gaps
or
not
depending
on
these
particular
operations.
A
So
here
we
figure
out
that
you
know
we
really
need
to
know
issue
or
rather
evoke
caps
and
issue.
Caps
is
very.
A
Apps
is
very
where
we
loop
through
all
the
gaps,
and
you
know
for
for
that
particular
client.
We
send
a
revoked
message
depending
on
many
things,
I'm
not
going
to
details
of
each
of
these,
but
you'll
you'll.
You
can
easily
find
out
that
you
know
you
know
you
have
a
bunch
of
gaps
that
are
allowed
and
the
client
this
particular
client
is
holding
those
gaps.
A
Those
are
conflicting
caps,
so
what
you
need
to
do
is
actually
send
a
m-client
caps
message
with
the
new
set
of
caps,
so
the
client
gets
this
new
set
of
caps
compares
with
its
own
set.
It
figures
out
that
it
needs
to
drop
some
gaps,
drops
and
replies
back
so
yeah.
That's
the
crux
of
it
actually.
A
So
once
you
have
done,
you
know
gone
through
a
bunch
of
these
states.
You
know
each
of
these
states
have
a
certain
you
know,
meaning
these
are
actually
grouped.
If
you
see
you
know
there
is
a
grouping
of
states
and
the
mds
kind
of
does
an
automatic
transition
between
these
group
states.
So
you
can
see
you
know,
we
went
into
locklock
xlock,
then
pre,
x,
lock
and
from
that
the
final
state
is
x
lock.
A
So
a
bit
about
this
stable
state
is,
you
know,
as
I
said
you
know,
mds
has
a
tendency
to
move
towards
stable
states.
These
are
when
it
completes
an
operation.
So
lock,
sync,
clock,
lock
and
lock.
Exclusive
are
stable
states
x,
lock
is
not
a
stable
state.
You
know
the
reason
for
that
is
you
know.
Once
the
x
lock,
the
client
has
and
the
x
lock
is
acquired
after
the
operation.
A
You
know
we
go
into
x,
log
done
state
and
depending
on
various
other
factors,
depending
on
what
other
clients
are
requesting,
we
either
jump
to
the
sync
state
or
one
of
the
other
states.
That's
an
implementation
detail
design,
I'm
not
sure
why
was
taken,
but
the
idea
is,
you
know
you
know
once
the
client
has
done
an
x
log.
A
You
know
all
these
new
changes,
like
you
know
if
it
has
added
a
particular
uid
that
are
sent
to
the
other
clients
via
the
place,
information
and
you
know
most
likely
the
clients.
You
know
the
other
clients
will
probably
be
reading
this
information,
and
in
that
case
you
need
not
jump
into
log
transition
states
again.
You
know
like
for
this
case.
What
we
saw
is
you
know
once
it
has
done
with
the
x
lock.
A
A
Well,
yeah,
okay!
So
we
go
into
these
clock
x,
lock,
then
pre,
x,
lock
and
then
finally,
to
x,
log
state.
A
So
yeah
this
was
a
simplistic
case
in
the
sense
you
know
we
didn't
have
you
know
most
of
the
locks
are
like
not
allowed
and
they
had
to
drop
the
lock.
But
if
you
go
to
the
complexions
like
file
lock,
if
you
see
the
lock
clock
state
here,.
A
You
see
the
lock
sync
locks
right
here.
You
know
it
says
you
know.
Clients
are
okay
to
have
the
you
know,
f
c,
which
is
like
the
file
cache
cap,
and
that
is
allowed.
A
A
This
happens
when
you
know
this
is
basically
in
the
file
lock
case.
So
again,
the
locked
state
transition
is
same.
We
start
with
lock
sync,
we
switch
to
lock
sync
clock
again,
we
drop,
you
know,
not
drop,
try
to
revoke
it,
but
during
the
relocation
we
found
that
some
of
the
clients
are
having
file
cache
tabs
and
it
is
fine.
So
we
don't
revoke
file
caps,
cache
caps
and
again
we
go
through
the
same
thing
again
we
enter.
We
enter
lock,
lock,
enter,
lock,
lock,
state.
A
This
lock
lock
state
again
again,
a
bunch
of
you
know:
gaps
are
allowed
it's
okay
to
have
for
some
of
the
clients
to
have
these
caps.
So
this
is
like
the
file
cache
and
the
fcb
cap
cache
and
buffer.
So
those
are
not
dropped.
It's
mostly
an
optimization
so
for
certain
states.
It's
okay
for
the
client
to
have
certain
gaps
like
this,
the
file
lock.
A
So
it's
okay
for
the
for
some
of
the
clients,
you
know
to
have
the
fcb
gaps
so
typically,
when
this
would
happen
with
file
lock
is
you
know
when
somebody
is
when
one
of
the
clients
is
trying
to
read
the
m
time,
which
is
protected
by
the
the
I
file
type
and
another
client
is
trying
to
you
know
kind
of:
do
it
right
which
changes
the
m
time.
A
So
in
that
case
you
know
some
of
the
lock
and
some
of
the
caps
are
allowed
and
safe
to
be
held
by
the
clients,
even
in
between
these
state
transitions.
There
is
this
interesting.
A
State
collection,
which
is
like
mix,
but
the
idea
is
so,
if
you,
if
some
of
the
clients,
if
one
of
these
some
some
clients
have
the
f
b
caps,
which
is
like
they
are
buffering
their
rights
right.
A
So
there's
this
optimization,
where
you
know
for
a
case
where
one
of
the
clients
is
doing
a
a
listing
of
direct
listing
of
directories
and
files
and
directories-
and
you
know
some
of
those
some
of
those
files
have
been
written,
are
being
written
by
one
of
the
other
clients,
but
and
that
client
has
a
fb
cap,
a
file
of
a
cap.
So
it's
caching,
it's
buffering
all
these
rights
instead,.
B
A
The
mds
had
to
revoke
the
fb
caps.
The
client
has
to
flush
out
data
again
and
again,
you
know
when
the
other
client
is
doing
listing.
If
that
is
a
contending
cap,
it
needs
to
flush
out
this
flush
out
the
dirty
data
to
this,
which
is
kind
of
expensive.
If
it
has
to
do
it
again
and
again
and
again
so
what
does
mds
what
the
mds
does?
Is
it
and
it
it's
an
optimization
where
it
it
it
it?
A
It
figures
that
you
know:
okay,
it's
not
really
required
to
drop
the
to
flush
out
the
dirty
buffer,
so
it
enters
this
exchange
state
x,
syn
state
where
the
the
client,
which
is
buffering
the
data,
the
rights
you
know
it
it
it
can
release
the
caps,
which
means
it
has.
The
further
rights
will
be
stopped,
but
the
dirty
data
won't
be
flushed
to
disk,
so
it's
kind
of
an
interruption
to
the
right.
A
You
know-
and
you
know
that's
an
optimization,
so
the
the
other,
the
client
having
the
fb
caps,
it
doesn't
have
to
kind
of
you
know
flush
all
the
data
to
the
osd's.
It
can
just
stop.
Writing
you
know,
release
the
caps
release
the
caps
revoke
or
rather
drop
all
these
caps
and
it,
and
once
it
goes
out
of
the
accent
states
x
in
state
to
back
to
the
mixed
state.
You
know
it
tells
the
client
to
continue
continue,
writing
and
it
proceeds
as
expected.
A
So
these
are
all
these
optimization
states
that
that
the
mds
has
there
is
also
a
decent
state.
That
is
a
tempting
state
which
I,
which
I
have
not
figured
out
yet,
but
there's
that's
another
state
intermediate
state
for
optimization,
so
yeah,
that's
all
I
have
for
this
talk
so
yeah
we
discovered
simple,
lock
here.
You
know
in
the
next
part
three
for
locker
will
we'll
go
into
much
detail
into
this
catalog
and
the
file
lock
and
we'll
pick
up
similar
examples.
A
A
So
we
have
the
start
and
end
and
finish
routines,
so
these
are
called
you
know.
Start
is
called
when
you
want
to
when
you
start
to
want
to
just
starting
to
take
that
lock,
finish
is
done
when
they
actually
you're
done.
Locking
and
a
minor
point
is
if,
at
some
point
of
time,
you
can't
really
take
a
lock
for
whatever
reason
you
know
the
the
the
md
request,
the
the
request
from
the
client
is
actually
put
in
a
queue,
and
you
know
it
waits
for
certain.
A
You
know
certain
events
to
happen,
which
maybe
some
lock
has
been
taken
by
some
other
client
or
something
else
once
that
event
gets
resolved.
You
know
this.
A
This
particular
md
request
is
again
woken
up
and
you
know
it
continues
and
it
starts
again
with
starting
to
take
the
lock
again,
that's
how
it
is,
and
that's
all
I
have
for
this
talk
I'll
put
this
document
out
as
a
pr,
maybe
in
a
day
or
two
probably
tomorrow,
and
you
know
good
to
have
some
documentation
on
the
locking
stuff.
B
So,
thank
you
just
two.
I
mean
to
make
sure
that
I
I
followed
parts
of
it
so
initially
started
off
with
server.cc
code
going
through
the
setup
right
and
you
concentrated
on
the
auth
authx
lock
right
and
the
optics
clock
is
of
type
simple
log
and
you
went
through
the
different
transitions.
A
Yeah,
so
the
log
type
is
called
auth
lock,
not
x
lock.
So
we
have
a
lock
type
and
a
lock
mode,
so
you
can
take
auth
lock
either
in
read,
lock
or
exclusive
lock,
because
it's
a
simple,
lock
type
and
the
simple
lock
class
you
can
add
so
auth
lock
can
be
either
been,
can
either
be
acquired
in
read,
lock,
rd,
lock
or
exclusive
lock.
A
A
Once
that
is,
then
we
went
to
the
locker
class
acquire
locks.
Where
we
see
you
know
the
locks
are
rearranged
and
merge
and
then
for
each
particular,
lock,
lock
type
the
locker
class
consults
the
state
machine
and
then
follows
the
state.
B
B
A
B
A
A
If
you
see
you
know
the
last
one
are
caps,
lunar
xlock
and
the
player
caps.
So
here
these
are
the
last
four
you
have.
These
can
rd
lock
and
right,
lock,
force
right,
clock
lease
and
x
lock
that
is
mapped
to
this
body.
Lock,
part
lock.
A
It
takes
some
time
to
kind
of
map,
but
you
have
to
look
at
each
of
these
states
more
closely
and
map
them
back
to
that
struct
that
I
was
talking
about
to
see
what
each
of
these
state
means
and
what
is
allowed
and
what
is
not
allowed.
Like
lock,
sync,
you
know,
as
I
said,
you
know,
it's
pretty
simple,
you'll
start
with
simple
lock
and
that's
the
most.
You
know.
Other
locks
are
pretty
confusing.
You
know
just
start
with
simple
lock.
A
A
Well,
zero
is
like
you
know
when,
where
you
see
zero
is
like
that's.
That
is
not
allowed
any
is
like
author
replica,
and
if
you
see
author,
that's
only
the
authoritative
nds,
that's
allowed
to
lock.
A
I'll
take
that
as
an
as
I
know,
thanks
guys
and
yeah,
please
go,
and
once
I
put
the
pr
up
you
know
please
make
you
know
it
will
be
good.
If
you
can
put
some
comments
on
it,
we
need
to
really
start
documenting
these
internals,
for
you
know,
for
the
sake
of
yeah,
thanks.