►
From YouTube: data-transfer and fil-markets
Description
Recorded at DataSystems September 2021 Colo. Join us in #retrieval-market on filecoin slack to ask questions or build an open data transfer stack with us!
A
So
I'm
going
to
talk
about
for
a
few
for
a
bit
about
two
separate
libraries
that
exists
in
the
file
code
ecosystem.
They
are
go
data
transfer
and
go
for
markets.
This
stock
is
about
100,
winged
minus
some
like
assembled
resources
and
whatnot
and
half
parts
of
old
talks,
so
so
we'll
just
sort
of
jump
around
as
needed
and
kind
of,
hopefully
we'll
get
to
somewhere.
That
makes
sense,
first
of
all,
just
like
at
the
highest
level.
A
What
are
these
libraries
filecoin?
The
concept
of
a
of
markets
in
filecoin
refers
to
the
deal
making
portion
of
the
system.
So
this
is
the
process
by
which
people
identify
people
to
store
data
with
set
things
up
with
them
and
store
their
data
and
then
also
how
they
identify
people
to
retrieve
data
from
and
set
things
up
and
retrieve
their
data
go.
Data
transfer
is
a
library
that
sits
on
top
of
the
underlying
protocol
for
moving
data.
A
The
underlying
protocol
and
filecoin
is
pressing
and
go.
Data
transfer
is
sort
of
like
the
the
way
I've
started
to
talk
about
it
is
it's
sort
of
a
control
plane
protocol.
Concretely,
what
that
means
is
like
go.
Data
transfer
understands
understands
things
at
a
very
logical
level
like
starting
a
request,
stopping
a
request,
restarting
a
request,
pausing
a
request,
authenticating
a
request.
A
It's
a
very
like
high
level
language
for
talking
about
doing,
transfer
versus
the
underlying
protocol
is
just
a
protocol
and
it
has
like
whatever
mechanics
it
has
for
interacting
with
it
and
so
in
the
overall
system
of
filecoin.
The
way
this
kind
of
works.
If
you
let
me
see,
if
I
can
make
this
one
a
little,
I
think
I
can
open
this
in
a
new
window.
Does
that
do
that
there.
B
A
A
Diagram,
but
I
can't
remember,
who
drew
it
like
forever
ago
and
there's
a
few
bits
that
are
like
kind
of
off
but
like
the
basic
concept
of
like
you
have
when
you
think
about
like
making
deals
in
file
point,
whether
it's
a
and
and
you
you
have
like
here
over
here.
A
A
E
D
E
A
Ago
so
in
this
case
we
have
a
client,
someone
who
wants
to
store
and
retrieve
data
and
a
miner,
and
now
we
call
them
a
provider
who
is
willing
to
store
or
serve
up
data.
And
this
is
sort
of
like
how.
A
A
For
the
time
being
is
graph
sync,
and
then
you
have
a
a
module
that
sits
on
top
of
grassy,
which
is
data
transfer,
and
data
transfer,
translates
the
protocol
syntax
of
grassland
into
a
much
more
semantic
syntax
of
managing
requests.
A
And
then,
on
top
of
that,
you
have
a
you,
have
storage
deals
and
retrieval
deals.
Retrieval
deals
are
primarily
a
transfer.
Storage
deals
are
actually
quite
a
bit
more
than
the
transfer
there's
a
whole
bunch
of
steps
involved
in
storage
steels,
in
addition
to
moving
data
from
one
place
to
another.
A
Actual
semantics
of
storage
or
retrieval
on
the
client
side,
you
have
a
storage,
client
and
a
retrieval
client,
and
on
the
storage
side
you
have
a
storage
provider
and
a
retrieval
provider,
and
what
these
do
probably
in
especially
in
our
case.
We're
going
to
be
thinking
about
retrieval
in
retrieval.
A
For
handling
payments,
so
the
retrieval
client
talks
to
the
data
transfer
layer
and
the
retrieval
provider
talks
to
the
data
transfer
layer
and
essentially
they
monitor
the
transfer
and
the
provider
asks
for
money
when
a
certain
amount
of
bytes
are
transferred
and
the
client
verifies
that
they,
they
they've
tripped
that
they
have
transferred
that
number
of
bytes
so
pre-agreed
upon.
They
send
them
new
money
and
then,
and
then
the
provider
resumes
the
transfer
and
and
sends
them
more
more
data.
A
And
you
essentially
have
this
like
set
of
incremental
payments
for
incrementally
verified
data.
One
of
them-
oh
yeah,
I'll
talk
a
little
bit
more
about
retrieval
in
a
minute
because
that's
obviously
relevant
to
what
we're
going
to
be
doing
with
the
retrieval
client,
the
storage
provider
and
the
storage
client
resource
provider
actually
do
a
whole
lot
more
than
the
transfer
itself.
The
transfer
itself
in
in
the
storage
market
is
pretty
simple,
because
you're
just
moving
data
from
one
place
to
another
and
there's
no
payment
involved
at
that
point.
A
But
the
orchestration
of
setting
up
a
storage
deal
is
actually
quite
quite
complicated.
So
there's
a
number
of
steps
that
are
involved
in
setting
up
a
storage
deal
and
then
negotiating
it
moving
the
transfer
putting
it
on
chain
because
storage
deals
are
unchained
and
then
then,
on
the
provider
side,
there's
a
whole
bunch
more
steps
related
to
relating
to
sealing
data
and
proving
it,
which
is
the
whole
thing
about
the
file
point
system.
A
That
much
about
that
so
yeah,
so
that's
the
kind
of
like
overall
architecture.
I
don't
know
what
this
like
file
and
file
stories
at
the
beginning.
That
feels
a
little
out
of
date
to
me,
but
but
yeah,
that's,
that's!
The
basic
concept
folks
have
questions
before
I
go
into
some
more
in
depth
on
a
few
parts.
These
things
sort
of
make
sense.
A
Yeah,
so
the
data
transfer
has
I'm
going
to
talk
a
little
bit
in
a
moment
about
vouchers.
How
about
just
what
doctors
we've
been?
I
think,
having
I
think
for
a
lo
for
a
lot
of
us.
We've
been
thinking
about
vouchers
in
terms
of
payments,
but
a
voucher
is.
A
A
B
A
Have
any
notes
other
than
just
kind
of
like
me
talking
about
what
this
thing
is?
I
actually,
I
think
I'll
start
by
talking
about
what
it
was,
how
it
originally
came
to
be
the
the
basic
concept
of
grow
data
transfer
when
it
first
was
created,
and
it
probably
eventually
will
continue
to
be
something
along.
These
lines
is
that
we
wanted
to
do
two
things
in
the
context
of
file
point
one
was
like
in
ipfs.
A
We
basically
have
a
system
of
sending
data
around
that
is
effectively
like
completely
cooperative
and
altruistic
and
potentially
vulnerable
to
someone
exploiting
it,
but
though,
there's
mitigation
mitigations
for
that,
but
like
in
in
bitswap,
which
is
the
main
transfer
for
for
ipfs
you,
essentially
anyone
can
ask
anyone
for
blocks,
and
our
current
heuristic
is
just
send.
Everyone
who
asks
blocks
send,
send
them
blocks
back
right.
A
A
Unless
you
get
a
cookie
that
you
know
somehow
authenticates
the
request
and
so
data
transfer
came
about
because
we
knew
it
initially
sort
of
came
about,
because
we
knew
a
that
we
were
going
to
have
to
authenticate
request,
requests
to
transfer
data
in
the
context
of
file
point
because
we
didn't
want
to
just
accept.
You
know
incoming
transfers
on
a
minor
from
anyone
we
needed
to
know
they
were
associated
with
a
storage
deal.
Similarly,
we
wanted
to
make
sure
that
we
only
sent
data
on
a
on
a
client.
A
And
then
we
also
knew
that
we
probably
wanted
to
have
an
abstract
notion
of
that
authentication
and
a
general
abstract
notion
of
managing
the
transfer
without
having
to
specify
the
underlying
protocol.
A
There
is
a
belief-
and
there
probably
is
still
a
belief
going
forward-
that
we
are
going
to
most
likely
support
multiple
protocols
for
transferring
data
and
filecoin
and
other
you
know,
protocol
products,
and
we
need
to
have
a
sort
of
like
protocol
independent
language
for
talking
about,
like
the
stages
of
moving
data
around,
and
so
that's
kind
of
what
data
transfer
is
intended
to
be.
Data
transfer
is
intended
to
be
like
an
abstract
control
layer
for
orchestrating
a
data
transfer
juan
when
he
initially
presented
it
to
me,
like.
A
I
think
it
was
like
two
years
ago,
the
the
image
he
showed
like
the
image
he
had
in
his
mind
was
like
of
a
you
know,
a.
A
And
not
enough
to
netscape
I
mean
which
obviously,
I
guess
would
be
chrome
now.
You
know
but
like
this
idea
of.
A
A
Concept-
and
we
also
knew
that
we
wanted
to
be
able
to
do
orchestrate
transfers
in
ways
that
were
that
were
slightly
unusual,
like
we
wanted
to
be
able
to
initiate
a
transfer
on
the
side
that
was
sending
data
to
essentially
push
data
to
someone
else.
There's
reasons
for
that.
In
the
specific
case
of
storage
deals,
when
the
client
is
sending
data
to
a
provider,
there's
a
much
greater
likelihood
that
the
client
is
behind
a
vpn
and
we're
generally
going
to
do
a
lot
better.
A
If
we
initiate
the
connection
from
the
side
that
is
behind
a
vpn,
so
so
data
transfer,
one
of
the
things
data
transfer
does
is
it
has
a
push
transfer,
which
essentially,
you
open
a
push
transfer
in
order
to
send
data
to
someone
and
the
way
it
works
under
the
hood
is
that
it
actually
sends
a
special
message
to
the
other
person
to
initiate
the
transfer
and
then
the
other
person,
assuming
they
authenticate.
The
request
actually
kicks
off
a
grassland
request
from
their
side
to
request
data
back.
A
A
So
I'm
gonna
just
pull
this
up.
This
is
the
manager.
This
is
sort
of
the
top
level
api
for
a
for
the
data
transfer.
You
can,
you
could
start.
A
You
know,
there's
one
data
transfer
instance
per
per
lid
pdp
host
at
the
moment-
and
you
know
it
has
some
like
really
basic
starting
and
stopping
functions
and
then
the
main
functions
you're
gonna
see
are
you're
gonna,
be
able
to
open
a
push
channel
close
open,
a
push
channel
open
a
pull
channel
you're
going
to
be
able
to
close
a
push
a
channel.
You
can
pause
it.
You
can
resume
it.
You
can
get
states
about.
What's
going
on.
You
can
also
subscribe
to
events.
A
G
A
I
mean
you,
don't
need
there
there.
It
depends
on
what
you
want
to
do.
You
don't
need
to
use
plasma
resume
unless
you
want
to
pause
and
pursue
transfers
and
you
don't
need
to
use
push
and
pull
unless
you
need
to.
I
mean
you
can
only
use
poll
if
you
just
want
to
fetch
data,
but
but
people
do
need
to
use
push
in
certain
cases.
A
It
is
by
design,
in
the
sense
that
a
data
transfer
manager
is
intended
to
manage
all
the
transfers
right.
You
just
create
a
manager,
and
then
you
can
open
channels
and
you
can
pause
them.
You
can
resume
them
I'll,
get
updates
on
all
the
different
transfers
whatnot.
So
that
is
the
design
you.
Why
isn't
there?
An
individual
part
of
the
design
is
motivated
by
the
fact
that
data
transfer
does
currently
have
its
own
lib
p2p
protocol
to
basically
fill
in
the
gaps
and
graph
sync.
A
This
is
this
is
there's
a
refactor
concept
in
data
transfer,
which
it
probably
shouldn't
have
its
own.
It
probably
should
not
have
its
own
underlying
protocol,
and
if
that
is
the
case,
that
will
allow
us
to
potentially
have
one
data,
one
manager
per
request.
If
you
wanted
to
do
that.
E
A
Every
protocol
can
only
have
one
handler
per
host
and
that's
where
you
get
into
like
a
global
handler
like
grassland,
can't
be
initialized
twice
on
the
same,
the
pdp
host
and
because
data
transfer
right
now
has
its
own
pdp
protocol.
It
cannot
be
initialized
twice.
A
Be
initialized
twice
because
it
will
be.
That
will
be
disconnected
the
other
thing
that
data
transfer
like
I
does
like
it
does
some
like
crazy
stuff,
like
it
actually
persists
states
of
every
transfer
so
that,
if
you
shut
down
your
node
and
you
restart,
it
should
resume
the
transfers.
I
believe
that
that's
always
like
it's
always
a
question
mark.
If
that
totally
works.
A
But,
like
you
know,
the
idea
is
that
you,
you
know
you
would
like
it
if
you're
sending
32
gigs
of
data
in
the
storage
field,
and
you
shut
down
your
node
for
you
to
be
able
to
like
turn
it
back
on
and
have
the
transfer
complete
and
then
there's,
like
other
conditions
that
we're
trying
to
deal
with
in
data
transfer
does
most
of
this.
A
F
A
I
want
to
talk
about
vouchers
right,
so
the
way
the
concept
of
how
requests
are
authenticated
and
due
to
transfer
is
this
concept
of
vouchers
and
vouchers.
I
like
a
voucher,
is
just
an
abstract
piece
of
data.
That's
nothing
to
do
with
payment,
there's
a
separate
concept
in
the
payment
channel
world
of
payment
vouchers,
but
like
in
this
case.
The
way
I
think
of
a
voucher
is
it's
basically
the
web3
concept
of
a
cookie.
A
It's
just
a
piece
of
data
that
allows
you
to
look
at
an
incoming
request
and
make
some
decisions
about
whether
to
serve
it
and
basically
the
system
by
which
you
can
have
you
can
delegate
different
like
the
the
way
the
system
by
which
you
can
essentially
specify
different
types
of
vouchers
and
how
you
you
validate
them,
is
you
have
a
concept
of
voucher
types
and
basically,
what
you
do
is
your
voucher
type
is
literally
your
construct
right
so
like
if.
A
Voucher
or
retrieval
voucher
and
I
register
a
voucher
type.
So
I
passed
here,
like
literally
an
instance
of
the
storage
voucher
and
an
instance
of
the
retrieval
voucher,
and
then
I
ask,
and
then
I
pass
it
a
validator
and
basically
what
this
is
going
to
do
is
when
a
new
request
comes
in.
It's.
A
Look
at
the
it's
going
to
look
at
the
encoded
voucher,
it's
going
to
deserialize
it
into
this
specific
type
and
then
based
off
the
type
it's
going
to
call
out
to
whatever
registered
validator
exists
and
and
and
pass
that
to
that
validator.
So
to
answer
will
your
original
question?
That
is
how
that
happens.
Let's
see
what
else
can
we
say
about
this.
B
A
And
so
that's
like
the
there's,
a
concept
of
a
voucher
type
and
that's
like
pretty
much
all
you
need
for
storage
for
retrieval.
We
have
to
have
a
bunch.
We
have
a
bunch
more
stuff
right
because
with
retrieval,
what
we
want
to
be
able
to
do
is
we
want
the
way
retrieval
works
and
again
this
is
just
the
design
of
retrieval
in
the.
A
For
you
know,
in
the
first
version
of
file
coin,
the
retrieval
idea
has
to
address
a
problem
that
I
guess
in
computer
science
is
called
fair
exchange,
which
is
like
I
want
to
pay
you.
I
want
to
pay
you
some
amount
of
money
for
you
to
send
me
some
kind
of
good
goods,
and
I
want
to
be
able
to
do
that
in
some
form
of
trustless
way.
A
Trustless
way
where
I
can
be
where
both
parties
can
be
guaranteed,
that
the
other
parties
can
act
responsibly,
it's
a
fairly
hard
problem
to
solve
there.
There
there's
no
like
magic
bullet.
The
solution
we
use
in
and
file
coin
is
something
that
I
believe
is
called
optimistic,
fair
exchange,
which
is
that
we
essentially
in
a
in
a
in
a
retrieval.
A
We
want
to
send
a
small
piece
of
data
and
then
request
payment
get
paid
for
that,
and
then
you
want
to
complete
that
process
multiple
times
for
the
whole
thing
and
you
can
and
you
can
slowly
increase
the
amount
of
each
each
individual
transfer
for
payment
unit
on
the
theory
that
people
you
know
once
once.
People
are
behaving
well,
they're
likely
to
continue
behaving
well.
D
D
E
A
A
Well
so
what's
gonna
happen
is
the
the
ordering
is
fights
first,
then
then
payment
right,
so
the
provider's
gonna
send
me
ten
thousand
bytes
and
then
they're
gonna
say
you
need
to
pay
for
that
before
I
send
you
any
more.
What's
going
to
end
up
happening
in
that
case,
is
you
know,
let's
say
you're
halfway
through
and
you
run
out
of
money.
A
They
are
going
to
look
at
your
they're,
going
to
send
you
a
request
and
you're
not
going
to
be
able
to
fulfill
it,
at
which
point
the
transfer
is
yes,
indeed,
pause,
because
it
literally
what
happens
is
when
they
send
the
voucher.
Well,
the
first
thing
they
do
is
pause
the
transport
right.
The
other
thing,
though,
the
the
interesting
thing
about
that
is
that's
a
little
bit
of
a
an
ins.
A
What's
an
interesting
question,
because
one
thing
that
you
should
know
is
that
you're
not
actually
sending
phil
back
and
forth
you're
sending
this
thing
called
payment
voucher.
It
is
something
called
the
payment
channel.
There
is
an
entire
concept
in
the
world
of
blockchains
called
payment
channels
and
they're,
a
slightly
evolved
concept
called
state
channels,
and
what
these
are
all
based
on
is
the
idea
of
like
being
able
to
set
up
like.
Basically
it's
like
you.
A
A
That's
a
chain
transaction
right
and
then,
once
things
get
going
what's
going
to
happen,
is
I'm
going
to
send
you
these
things
called
vouchers,
which
will
essentially
be
these
things
that
you
can
submit
to
the
chain
to
actually
unlock
a
portion
of
the
funds
that
have
been
put
in
that
escrow
and
you
don't
have
to
submit
them
right
away
they're
set
up
in
such
a
way
that
you
can
submit
them
whenever
and
that's
important,
because
in
the
case
of
retrieval,
we
actually
can't
block
on
a
chain
train
transaction
for
each
step
of
the
retrieval
chain.
A
A
To
avoid
the
one
of
the
one
of
the
problems
with
retrieval
right
now
is
that
you
have
to
set
up
the
initial
payment
channel
before
you
do
the
retrieval,
which
itself
is
a
delay
so,
like
that's,
that's
a
problem
right
now
in
retrieval
in
any
case
yeah.
So
how
does
that
actual?
Like
pausing
and
whatnot
happen
right?
So
we
have
this
concept
of
a
concept
of
a
revalidator
and
what
the
revalidator
does
is
they
have
this?
A
They
provide
this
interface
to
data
transfer
and
there's
a
couple
of
different
methods
on
it.
This
method
we
can
ignore.
For
the
time
being,
we
have
one
called
onpole
data
sent,
which
basically
means
when
I've
set
some
amount
of
data
right.
I
am
each
time
more
data
is
about
to
be
sent
right.
I
have
the
option
to
inspect
how
much
data
is
sent
and
send
an
additional
voucher
result,
which
is
basically
additional
information
back
to
the
client,
and
I
also
have
the
ability
to
pause
the
request.
A
The
pausing
is
done
with
the
really
terrible
method
right
now,
the
error
value.
If
you
return
a
special
error,
called
air
pause,
it
will
pause
the
request
that
probably
should
change
so
any
in
any
case
so
like
when
I,
when
I
register
a
revalidator,
I
register
this
revalidator.
A
Where
is
it
I
registered
on
the
voucher
type,
so
any
any
any
request
where
the
initial
voucher
matches
this
type
is
now
going
to
go
to
the
revalidator
and
each
time
we
send
data.
This
method
is
going
to
be
called
the
onpole
data
sent,
and
I
am
going
to
basically
look
at
that
and
decide
whether
I
want
to
pause
the
request
and
if.
C
A
Why
I
posted
right?
On
the
other
side,
you
can
call
a
method
called
send
voucher,
which
is
gonna
just
directly,
send
a
voucher
to
the
other
side
in.
A
What
that
the
way
that
happens
is
that,
like
the
retrieval
code,
is
gonna,
look
at
the
information
sent
back
by
the
provider
and
I'm
gonna
say:
oh
they
they
said
to
they
said
they
paused
it
and
they
want
us
to
pay
me
right
and
then
they're
going
to
send
the
voucher
and
then,
when
they
send
the
voucher,
your
the
revalidator
is
going
to
get
this
method
called
revalidate
with
the
voucher
that
was
sent
and
then
they're
going
to,
and
then
that
will
allow
them
to
unpause
the
request
when
saying
that
they've
received
payment.
Yes,.
E
Why
such
a
strong
emphasis
on
validation,
I'm
using
the
character?
What
do
you
mention
about
the
bachelor's
mean
cookies.
A
Well,
because
that
is
effectively
how
you
control
the
behavior.
It's
like
the
word
validate
is
probably
not
the
right
word,
because
you
are
doing
something
between
authentication
and
authorization.
Essentially,
it's
probably
authorization.
It's
the
closest
concept
to
what
you're
doing
in
the
initial
request,
you're
looking
at
that,
you
know
cookie
and
you're,
using
it
to
essentially
decide
who's.
Sending
me
this
thing.
A
Do
I
believe
it's
that
person
and
are
they
allowed
to
do
the
thing
that
I
want
that
they're
gonna
do
and
the
in
the
next
stage
you're,
looking
at
the
voucher
and
you're
saying
what
data
did
I
receive?
Is
it
the
right
data
you
know?
Is
it
the
right
data
for
like
what
I
was
requesting
and
if
so,
I'm
resuming
the
request.
A
A
I
think
that's,
maybe
that,
though
it
only
lasts
for
the
scope
of
a
single
transfer.
Again,
the
language
here
is
sort
of
like
made
up
because
we're
in
a
context
that
like
doesn't
really
completely
match
the
web
context,
but
like
also
like
you
know,
like
yeah
in
fairness.
Sorry,
one
piece
of
history
about
the
data
transfer
module
is
that
it
was
written
like
so
like.
A
They're
like
we
should,
we
should
make
it
work
for
retrieval
before
file
coin
maintenance,
ships
and
so
like.
This
was
like
me,
hacking
out
an
interface
completely
on
my
own
and
with
like
and
to
add
another
layer
to
it
like,
in
addition
to
no
feedback
like
this
got
to
the
point
of
like
we
were,
I
think
at
that
point
in
the
test
net.
What
was
that
thing
called
the
liftoff
space
race
yeah
so
was
like
happening.
A
We
were
very
close
to
launch
and,
like
the
entirety
of
the
current
architecture
of
go,
data
transfer
was
shipped
in
a
single
6
000
line
pr
written
at
like
3
a.m.
This
is
like
some
college
coding.
You
know,
except
that
I'm
not
in
college
by
any
means,
and
so
like
yeah,
there's
god.
This
is
the
like
of
the
various
designs
that
I
shipped
while
working
at
protocol
labs.
This
is
the
one
that
I'm
least
excited
about.
A
Yeah,
so
I
mean
like
it
is
funny
because
everyone
like,
I
feel
like
grassland's,
the
one
that
people
are
most
like.
Oh,
what's
going
on
with
graphic,
this
is
the
problem.
I'm
like
drastic
is
like
relatively
well
designed
at
this
point.
It's
like
you
know,
tested
it
does
a
whole
lot.
I
mean
the
biggest
problem.
Is
it's
doing
a
kind
of
stuff
right
like,
but
but
I
generally
believe
in
it
a
little
bit
more
that
I
plan
to
transfer.
A
I
also
have
slightly
more
belief
in
the
in
in,
like
I
actually
think
it's
vaguely
reasonable
markets
is
like
what
really
market
markets
are
mostly
designed
around
the
storage
market,
because
that
was
the
more
complex
task
and
and
essentially
what
you're
doing
with
the
storage
market
is
you're,
managing
an
extremely
long
running
process,
with
lots
of
stages
and
to
that
process
lots
of
different
things
that
happen
in
each
stage
and
lots
of
things
that
can
go
right
or
wrong,
and
so
markets
is
essentially
managed
by
underneath
by
a
state
machine.
A
There
is
a
there's,
there's
sort
of
a
actually.
Let
me
show
you,
this
yeah
go
state
machine,
so
go
state
machine
is
an
interesting
one,
because
it's
it
has
two
versions
inside
of
the
same
piece
of
code,
one.
The
original
version
of
this
was
written
by
magic
and
that's
this
like
root
version
and
then
there's
like
this
sub
module
called
fsm,
which
I
wrote
most
of
and
fsm
was
like
so
magic
state
machine
is
super
free
form.
A
It's
like
like
it's
basically
like
you
can
submit
any
event
and
like
respond
to
it
in
any
way
and
then,
like
kind
of
do
anything
with
it,
and
you
have
to
write
a
lot
of
code
to
work
with
it.
But
I
mean
it's
very
general
and
in
that
sense
it's
pretty
cool
and
like
you
can
do
lots
of
it
and
he
uses
it
to
super
good
effect
inside
of
the
ceiling
system.
A
A
States
can
happen
out
of
them
and
what
what
you
know,
what
transitions
they
cause
and
all
that
type
of
stuff
and
which
is
like
pretty
cool
and
super
easy
to
use.
But
at
the
same
time
like
I
don't
know,.
C
A
This
is
interesting
trade-off
with
like
magic's
thing,
because
magic
thing
is
really
simple,
and
this
thing
is
more
like
this.
A
Rails
programmer,
where
it's
like
the
convention
over
configuration,
dsl
everything
and,
like
so
yeah,
so
there's
that
yeah.
E
A
Has
a
bunch
of
like
so
like
if
you
look
and
what
the
the
type
of
stuff
that
that
it
generates?
If
you
look
in,
if
you
look
in
film
markets,
there's
a
bunch
of
folders,
but
the
core
folders
are
storage
market
in
retrieval
market
and
inside
of
each
storage
market.
You
can
have
a
bunch
of
interfaces
at
the
top.
You
can
have
an
impulse
directory.
B
And
the
tradition
of
someone
who
is
also
a
java
programmer
at
one
point
and
then
you're
going
to
have.
A
What
are
we,
where
provider,
states
and
client
states
right?
So
this
is
like
the
client
state
machine,
and
you
get
this
like.
This
is
the
fsm
for
the
client,
it's
a
very,
like
dsl,
like
language,
for
dividing
things
to
happen.
You
know
you
start
in
the
when
you
get
the
open
event,
you
go
from
unknown
to
like
the
very
first
stage,
which
is
reserving
the
client
funds
and.
A
You
there's
like
lots
and
lots
and
lots
and
lots
of
things
that
have
to
happen
like
it's
a
very
long
process
with
lots
of
stages
and
the
when
you
enter
one
of
the
core
concepts
in
the
state
machine
is
that
when
you
enter
a
state,
you
can
kick
off
a
function,
that's
sort
of
like
an
asynchronous
like
now,
I'm
in
this
state
I
will
do
stuff
to
potentially
transition
into
the
next
state,
which
is
like
something
that
you
want
to
do
like
you
enter
you
submit
a
chain
transaction.
A
You
enter
the
waiting
for
that
change,
transaction
to
be
done
and
stated.
The
function
you
kick
off
is
something
that
watches
the
chain
and
sees
if
the
transaction
is
done
and
then
fires
a
you
know,
transaction
is
done
event
and
that
triggers
the
state
transition
to
the
next
state-
and
you
have
those
so
you'll-
have
what
you'll.
C
A
A
Structure
of
markets-
I
part
of
this-
is
written
heavily
towards,
like
allowing
testing
of
these
functions,
in
particular
one
of
the
things
that,
for
better
or
worse,
I
was
really
invested
in
like
actually
testing
these
asynchronous
functions
and
making
sure
that
like
given
this,
given
you
start
here,
and
you
run
this
function
and
like
these
things
happen,
you
should
end
up
in
this
state
right.
A
So
that's
that's
a
that's
a
thing
that
happens
here,
and
so
you
have
like
these,
like
client
state
tests
that
test
each
one
and
run
through
it,
which
is
most
cool
and
from
what
I
hear
slightly
annoying,
because
sometimes
there's
a
lot
of
indirection
happening
because,
again,
like
things
are
set
up
to
be
very
uni-testable,
let's
see,
so
that's
the
basic
architecture
of
it.
Just
to
give
you
a
sense
of
like
what
the
overall
path
looks
like
I'm
gonna
see
if
I
can
pull
this
up.
B
A
B
B
A
A
Storage,
client,
as
you
can
see,
there's
a
lot
of
things
that
can
happen,
and
one
thing
you
might
say
is
that
this
is
too
complicated
to
be
a
single
finite
state
machine,
but
as
with
most
finite
state
machines,
you
start
out
thinking.
Oh
yeah,
there's
x
number
of
states,
and
this
is
the
one
thing's
nice
about
this.
This
is
auto
generated
from
the
code,
so
like
yeah,
it
does
stay
up
to
date
and
you
can't
I.
F
A
You
can't
ship
a
pr
without
running
the
update
task.
Unfortunately,
it
does
get.
You
know.
Diagrams
are
only
useful
below
a
certain
level
of
complexity,
so
you
know,
but
you
can
at
least
follow
things.
A
According
to
my
former
co-worker,
when
he
was
working
in
the
code,
he
referred
to
this
a
lot
to
help
him
understand
what
was
happening.
Yeah
and
those
are
auto-generated
and
then
but
yeah.
So
this
is
the
actual
process
of
like
doing
mining.
You
gotta,
I
mean
of
doing
a
deal.
You
gotta.
A
You
gotta
find
out
how
much
they're
charging
you
gotta
start
a
deal,
which
means
the
way
storage
deals.
Work
is
like
there's
essentially
like
an
escrow
account
for
everyone
and,
like
everybody,
has
to
put
money
into
that
escrow
account
before
you
can
even
do
the
deal
and
that's
like
a
that's
a
change
transaction.
A
B
A
Oh
right,
this
is
like
the
what
the
miner
has
to
do.
They
have
to
look
at
all
your
your
proposed
parameters,
decide
if
it's
gonna
work
and
then,
if
they
do,
they
indicate
they're
gonna
in
their
intent
to
accept
it.
Then
you
have
to
do
all
the
transferring
of
data.
The
miner
has
to
put
their
own
money
into
the
actor.
They
need
to
verify
the
data
that
you
sent
them
exactly
matches
the
proposal
you
made
because
they're
gonna
have
to
prove.
A
When
you
do
a
deal
in
filecoin,
you
actually
have
to
prove
that
you
have
the
exact
data
that
was
proposed
to
be
sent.
There's
this
notion
of
a
piece
commitment
that
is
essentially
like
a
hash
of
some
version
of
your
data,
and
so
basically
they
have
to
verify
that
the
data
you
sent
them
exactly
matches
this.
This
piece
commitment
before
they
do
anything
with
the
deal,
then
they
have
to
put
the
deal
on
chain.
That's
called
publishing
once
they
publish
the
deal
they
let
you
know
that
the
deal
has
been
published.
A
A
In
this
case,
an
actor
is
a
it's
a
system
within
the
filecoin
virtual
machine
which
so
I
guess
sorry
it's
slightly
similar
in
the
sense
that,
when
you
think
about
so
the
filecoin
the
filecoin
chain
essentially
has
a
programming
logic
to
it.
It.
A
Smart
contracts,
you
can
think
of
it
as
like,
it
doesn't
have
the
ability
to
program
general
ones,
but
it
is
a
set
of
different
things
that
can
do
different.
Different
people
who
can
do
different
things
in
the
system
right
and
everybody
who
is
in
the
file
plan
chain,
is
considered
an
actor
even
including
every
wallet
is
technically
an
actor.
But
there
are
some
built-in
actors
that
have
predefined
logic
to
them.
A
There's
something
called
the
storage
market,
actor,
the
storage,
miner
actor,
there's
a
few
more
there's
the
payment
channel
actor-
and
these
are
basically
just
sets
of
code
that
you
can
send
messages
to
the
chain
and
they
know
how
to
process
them
based
off
of
the
the
fact
that
it
was
directed
towards
them
and
yeah.
In
that
sense,
I
guess
it's
sort
of
like
an
actor
in
the
sense
that,
like
you
know,
you.
C
D
A
C
A
Yes,
yeah
yeah,
no
totally,
and
if
that
is
not
the
case
here,
also
like
the
fsm,
we're
talking
about
is
an
off
chain
fsm.
This
is
the
fsm
in
the
markets,
module
which
is
coordinating
the
whole
process.
It's
opposed
to
an
on-chain
fsm,
which
would
be
like
a
state
of
the
storage
market
actor,
which
is
this
on-chain
actor
markets.
Is
it
so?
Markets
is
really
interesting
and
like
well,
particularly
for
in
retrieval.
Almost
everything
is
off
the
chain
and
then
in
storage.
A
There
are
things
that
are
on
chain,
but
there's
lots
of
things
that
happen
off
chain
right.
You
have
to
send
like
you
have
like
sorry,
you
have
like
you
know
you
have
to
transfer
data.
The
transferring
of
data
actually
has
nothing
to
do
with
the
file
coin
chain
right
like
and
so
so
the
and
you
know
generally,
they
try
to
keep
as
much
stuff
off
chain
as
possible.
C
I've
been
wondering
how
come
a
few
markets
is
not
part
of
lotus.
What's
the
history,
there
is
any
other
time
using
it.
A
Historical
like
because,
like
a
fun
fact
about
the
development
of
like
filecoin,
there
was
another.
There
was
an
original
implementation
that
was
called,
go
filecoin
and
then
there,
and
then
there
was
lotus
and
lotus,
largely
replaced
profilecoin
for
a
long
time.
There
was
an
attention.
There
was
an
intention
to
ship
at
launch.
Two
different
implementations
of
filecoin
go
filecoin
and
lotus,
which
was
a
little
weird
because
they
were
both
gonna,
be
in
the
same
language
and
like
the
whole
idea
was.
If
you
ship,
multiple
implementations,
you
reduce
risk.
A
A
Relevant
thing,
and
so,
if
they're
sharing
that
it
kind
of
obviated
the
point,
because
of
that,
the
idea
was
that
there'd
be
these
two
implementations,
but
they
would
share
code
that
was
not
considered
like
high
risk
for
security
and
go
film.
Markets
was
considered
one
of
those,
so
the
idea
was
they
were
both
going
to
use
gofundme
markets
and
I
believe
they
both
they
both
did
at
some
point.
Go
filecoin
got
removed
as
a
pl
project
to
have
it
launch
it
actually
got
taken
over
by.
A
I
believe
ipfs
course
yeah
and
then,
and
then
that
is
the
thing.
That's
now
called
venus
yeah
why
to
begin
aspirational,
this
is
like
a
it's.
An
ongoing
thing
in
in
like
in
for
for
blockchains,
is
to
have
multiple
implementations
on
the
theory
that
so
like
ethereum
has
what
they
have
get
and
there's
one
other
one
yeah
yeah,
yeah
yeah.
Sorry
they
have
they.
Now
they
have
lots,
but
there
was,
I
think
it
launched
they
already
had
to
and
like
the
input.
A
The
idea
is
that
that
one
way
to
prevent
to
to
remove
risk
is
that
like,
if
you
have
two
implementations,
the
odds
that
both
have
the
exact
same
security
issue
or
lower,
and
so
in
a
worst-case
scenario,
where
a
security
issue
comes
up
it,
you
can
kind
of
fall
back
to
the
other
one
and
ethereum's
actually
done
that.
A.
E
A
A
Yeah
yeah
the
also
there's
c,
plus
plus
implementation.
I
don't
know
what
happened
to
it
to
be,
but
I
mean
not
to
editorialize
too
much
but
like
the
notion
of
using
the
two
go
implementations
as
like
that
to
solve
that
problem
seemed
always
seemed
pretty
odd
to
me
and
then
also
the
notion
of,
but
and
then
at
some
point.
They
were
going
to
actually
share
the
code
that
ran
in
the
chain,
the
actors,
and
so
then
it's
like.
What's
the
point
at
all,
it
was
a
weird
guy.
A
Let's
just
say
that
there
were
more
more
there
were.
There
were
levels
of
of
there
were
issues.
I
think
that
motivated
that
that
maybe
went
beyond
the
software,
and
that's
literally
all
I
will
say
about
that.