►
From YouTube: Retrieval Client Constraints and Design
Description
Recorded at DataSystems September 2021 Colo. Join us in #retrieval-market on filecoin slack to ask questions or build an open data transfer stack with us!
A
We've
gone
through
now
a
couple
sort
of
larger
group
discussions
and
reviews
and
iterations
on
what
we
want
in
terms
of
retrieval.
There
are
a
bunch
of
constraints
that
we've
discovered
through
that
process,
and
so
we've
got
two
separate
things
that
we
should
be
that
we,
I
think,
are
using
in
terms
of
our
thinking
here.
One
is
what
is
the
end
goal
and
that's
going
to
be
a
longer
thing,
and
so
maybe
we
can
think
of
you
know.
A
I
think
I
think
the
current
aim
is
sort
of
like
this
stretch
of
like
get
to
something
we're
happy
with
at
the
end
of
the
year
and
then
there's
also
this
sort
of
more
immediate
or
short-term
goal
of.
How
do
we
test
that
we
have
the
end-end
retrieval
flow
through
the
conductor?
In
particular?
Probably
we
want
to
validate
that
that
works
for
a
file
point.
A
We
may
also
want
to
be
able
to
validate
that
that
works
for
ips
function,
because
I
think
there
is
some
desire
to
be
able
to
use
this
for
pinata
or
as
a
alternative
to
the
current
for
sometimes,
and
so
the
the
main
trick
now
is.
Can
we
untangle
these
requirements
of
what
is
you
know
the
direction
and
this
longer-term
goal,
and
what
is
things
that
we
absolutely
in
immediate,
short-term
mvp?
A
You
know
for
our
ability
to
have
any
confidence
that
we
have
a
working
index
or
triple
system.
What
do
we
need
to
build
for
that
first
version,
and
if
we
can
get
that
more
limited
set,
where
you
say
okay,
these
are
the.
This
is
sort
of
a
initial
thing
that
we
want
that
we
feel
like
ideally,
isn't,
there's
nothing
that
like
goes
in
the
wrong
direction
compared
to
where
we
want
to
attend
longer
term,
and
if
there
is
like
those
are
things
we
should
identify
and
make
sure
we're.
A
B
Can
answer
that
question?
Yes,
so
we're
building
this
index
error
we're
gonna,
have
providers
running
indexes
who
is
going
to
initially
use
that
like
does
it
just
simply
exist
and
then
we're
like?
Oh
well,
should
anybody
wish
to
retrieve
with
wildlife?
You
can
do
it
this
way
and
build
something
against
this,
and
or
are
we
going
to
look
because.
B
D
B
A
Yeah
who's,
our
user.
For
this
thing,
yeah
beyond
that,
that's
a
fantastic
question
and
that
really
qualified
the
product,
and
so
some
of
this
is
on
us
to
help
the
product
narrow
in
on
whatever
the
use
cases.
Of
that
that
we
think
we
can
support.
First,
are
sure
there
are.
I
think
there
remains
a
desire
for
some
sort
of
gateway
either
run
by
us
or
run
by
the
active
elders,
longer
term.
Think
of
the
current
people
who
are
providing
data
that
there
there's
a
few
different
places
where
filipino
data
is
accessible.
A
There
are
people
who
are
fetching
their
data,
they
push
for
things
like
what
free
content
or
web
storage,
and
so
what
free
storage
is
a
gateway
right
in
that
sense
and
that
it
provides
an
http
interface
to
pull
that
back
out.
So,
even
if
we're
not
doing
it
on
an
icos,
I
o,
if
we
can
provide
either
a
library
that
is
then
used
by
trigger
story
when
it
pulls
it
back
out.
A
Desktop
gap
apps
that
are
doing
it
directly
within
themselves
and
you
could
imagine
just
sort
of
generic
retrieval
things
like
you
know.
Ultimately,
someone
like
brave,
especially
mobile.
They
want
to
only
be
retrieval
and
not
run,
full
might
be
investments,
and
so,
if
they
find
this
component,
that
would
be
a
place
that
they
would
they
would
find
the
sort
of
vibration,
although
is
its
own
thing,.
B
C
A
The
the
current
global
setups-
I
think
this
is
like
mobile
opera
and
things
like
that
that
are
providing
those
things.
They
go
to
a
gateway,
so
they'll
they'll
just
go
to
like
either
a
cluster
or
they
have
or
one
from
one
or
I
think
I'll,
probably
actually
run
one,
but
basically
they
don't
handle
that
protocol,
but
they're
not
running.
C
B
Getting
into
creating
more
browsers,
I
would
suggest
again
because
I
would
suggest
the
one
thing
is
like
those
the
different
user
groups
we
just
mentioned
have
fair,
like
web3.storage
is
very,
they
need
a
ghost
stack
that
will
make
these
things,
or
maybe
a
js
stack,
but
like
they're,
going
to
be
making
the
actual
retrieval
calls
anyone
who's
like
a
mobile
client.
That's
a
javascript
client.
It
talks
to
http.
It's
not.
A
A
B
C
B
A
B
A
B
I
mean
there's,
you
know
I
think,
there's
like
whatever.
Maybe
I
miss,
I
think
there
are
folks
who
have
a
general
dislike
of
how
much
we're
relying
on
gateways
in
general
and
this
just
sort
of
gets
folded
into
it
and
it's
like
well.
If
we
did
this,
we
are
not
just
a
double
check.
The
whole
estuary
proxy
thing
is
like
totally
off
the
table.
People
have
rejected
it.
A
A
Is
that
part
of
what
that
meant
was
not
simply
that
content
should
become
available,
which
is
one
of
the
concerns?
Is
right?
Okay,
if
there's
this
weird
additional
thing:
oh
you
didn't.
You
know
you
didn't
push
it
to
file
a
point
with
the
right
flag,
and
so
now
we
can't
recreate
that
in
that
this
is
mine.
A
A
Story
so
that
it's
rather
than
spreading
that
layer,
oh
yeah,
the
first
version
yellow,
is
just
having
all
the
cost,
and
you
don't
notice
that
oh
yeah
great
yeah,
that
if
you
do
that
and
people
are
blind
and
then
you're
like
actually
you're
stuck
in
there.
It
just
costs
a
lot.
You
guys
have
to
pay
it
hey.
We
don't
even
have
that
solution.
A
B
D
A
There's
been
a
little
bit
of
pushback,
but
but
the
the
flip
side
of
that
is
that
we
haven't
been
given
even
any
direction
of
how
you
would
go
about
building
it
into
ibms
right.
So
we
asked
what
is
the
way
that
you
would
have
other
transports
in
ivfs
and
the
response
was
that's
too
hard,
but
we
will
give
a
proposal
for
how
you
delegate
the
routing
part,
and
so
that
then
was
seen,
I
think
by
everyone
as
and
and
quan
continues
to
push
on.
A
A
D
A
Yeah
I
mean
I,
I
think
that's
right
and
I
think
the
other
thing
is
like
within
the
world
of
ipfs,
and
I
think
for
me,
the
core
of
what
ipfs
is
is
like
content
writing
like
ibms,
helps
you
find
content
by
content
id
and
and
interact
with
it
and
go
ibms
is
like
the
full
move.
That
also
is
a
provider
and
captures
and
does
a
bunch
of
stuff
there.
B
Just
so
you
know
yeah,
I
mean
my
own
approach
to
it
andrew.
Is
that,
like
the
reason
that
you
can't
just
build
it
into
go,
ipfs
is
that
go
ipfs
actually
has
a
solution
to
a
fairly
complex
problem
like
how
do
you
find
your
content
choose
where
to
fetch
it
and
then
like
fetch
it
like
it
like
and
and
be
able
to
like
do
that
for
large
flocks
of
content
that
are
like
dags,
that
you're
gonna
fetch
over
time
and
like
that's,
what
I
think
the
the
problem
is
like.
B
B
It's
a
higher
level
than
than
like
a
single
deal.
Right
like
it's
like
the
the
assumption
is,
you
would
be
making
very
high
level
requests
for
content
without
a
person
you're
asking
from
or
like
a
limitation
of
a
single
transfer
which
is
similar
to
how
you
know
bitswap
sessions
and
the
purpleback
service
works.
You
know,
and
then
this.
B
D
B
B
A
I
offer
the
caution
that,
especially
as
soon
as
any
policy
things
like
pricing
for
people,
their
ideal
change,
that's
going
to.
They
just
need
to
be
careful
about
how
they
design
that
and
when
that's
possible
to
do
and
it's
going
to
potentially
add
more
complex
before
applying
to
trying
to
guess.
If
that's
going
to
happen
or
not,
but
I
don't
think
that
really
matters
if
we
just
go
as
far
as
it
gets
it
to
us.
A
A
D
B
A
D
A
A
Okay,
so
so
there's
a
multiple
peers
like
this,
while
that's
not
an
mvp
level,
and
let's
ignore
that
in
place
for
a
moment,
why
ignoring
these
two
complexities
and
the
policy
of
like
multiple
transfers
happening
potentially
like
does
that
change
the
interface
in
a
major
way
that
we're
afraid
that,
like
we
end
up
like,
I
suspect,
as
we
have
like
it,
will
I
don't
think
more
yeah.
I
don't
think
we
really
lose
what
we
did
before,
like
that's
an
iteration
on
it,
so
I'm
not
too
worried
about
that
as
a
limiting
of
soap.
A
B
So
we'll
get
back
the
list
of
potential
options
from
the
index,
so
the
index
is
going
to
be
given
multiple
results.
Yes,
and
it's
going
to
have
to
look
at
those
and
figure
out
which
one
it
is
and
there's
only
one
indexer
that
we're
talking
to
you
initially,
which
is
just
our
interests.
We
may
end
up
with
a
federation,
but
they
should
all
be.
B
A
Do
you
want
to
get
all
of
those
to
a
transport,
so
let
your
drink
board
get
soaked
in
particular
like
you're
like
great.
I
have
these
three
bits:
does
that
go
to
three
different
bits
transports,
or
does
that
go
into
one
like
bit,
squat
transfer
of
a
section
with
these
three
pairs,
three
different,
so.
B
One
thing
that
I'm
pretty
confident,
though
I
mean
my
one
thing:
I'm
pretty
religious
about
is
refactor
bits
off
so
that
you
can't
you're
not
using
video
obsessions.
We
are
the
best
obsession
right,
at
least
that's
my
again,
but
like
I'm,
assuming
it's
probably
a
post
interview,
I
don't
know
I
could
be
wrong.
A
A
D
B
B
A
B
D
B
D
B
B
If
that's
the
use
case
like
actually
doing
that
gets
us
to
the
is
to
the
end
of
the
year,
because
that
actually
means
it's
like
taking
just
purely
the
file
point
and
like
the
policy
around
that
and
like
testing
it
and
making
it
work
in
production,
which
is
probably
like.
I
could
be
wrong.
I
mean
like
especially
if
people
started,
hitting
up
gateways
to
get
all
the
out.
A
So
my
my
thought
here
right
is
that
that's
maybe
I
could
be
being
under
ambitious.
I
I
I
I
I
don't
think
that's
true.
I
like,
I
think
I
think,
that's
probably
a
wrap
that
you
certainly
take
until
the
end
of
the
year
to
get
there.
My
guess,
though,
is
that
that's
a
couple.
People
like
it
takes
time,
but
it
doesn't
take
all
of
us.
B
C
A
B
A
A
A
B
It
would
be
helpful,
I
think,
in
general,
both
especially
planning
classes
here
like
to
think
about
what
are
the
for
to
understand
what
is
the
product
timeline
for
when
they
need
different
like
like
doing
this
low
level
and
swap
during
the
day,
transfer
factor
delivers
like
absolutely
zero,
like
immediate
product
value
right
like
or
not.
You
know
what
I
mean
like
there's
no
user
feature
that
comes
out
of
that
for
the
most
part.
Immediately,
probably
it
sets
you
up
to
a
user
feature.
A
A
A
B
Channels
integrate
to
get
30
transfers,
I
mean,
but
that's
going
to
be
that's.
My
hope
is
that
the
entire
point
of
go
data
transfer
is
to
provide
a
control
layer
by
which
you
can
orchestrate
a
paper
channel
through
some
kind
of
protocol.
So
there's
there's
external
groups
right,
yes,
and
that
is
that
will
land.
A
A
Because,
like
they're
going
to
get,
but
I
think
there's
two
things
that
we
probably
should
scope
around
this
one
is:
what
is
the
right
books
on
the
client
for
a
client
having
the
interface
to
ask
for
a
voucher,
maybe
introducing
context
from
some
external
process
or
machine
or
something?
And
so
what
does
that
api
look
like
right?
So
so
I
think
the
thought
is
the
in
the
retrieval
client
code
itself.
It
shouldn't
really
think
about
this
beyond,
like
a
voucher,
like
any
state
permission
from
some
external
thing
and
so
then
separately.
A
What
is
that
extra
thing?
That's
like
a
light
like
maybe
it
starts
his
lowest
leg
or
something
even
lighter
than
that
that
it
or
like
something
that
you
know
can
follow
the
chain
but
manages
a
wallet
and
then
just
imagines
like
what
is
like.
Is
there
a
thing
that
you
want
a
developer
to
run?
That
is
they
hear
your
vector
ball?
You've
got
your
payment
channel.
You
can
take
in
requests
for
a
voucher.
Do.
B
A
B
A
For
the
developer,
like
the
developer
is
like
crazy.
I
know
this
tag
is
pretty
big,
so
I'm
going
to
keep
doing
this
optimistically,
like
there's,
potentially
a
lot
of
complexity
here,
where
the
developer
doesn't
want
to
have
to
you
know
blog
on
the
rpc
from
the
client
who's.
You
know
100
milliseconds
away
from
them
every
thousand
likes.
Certainly
right,
like
you,
you've
added
once
you
have
three
parties
in
the
wall:
it's
not
in
the
same
place.
A
You've
got
this
other
latency
source
between
there
and
so
you're
not
interrupting
the
thing.
That's
there
like.
Can
the
developer
optimistic
response
in
the
channel?
Assuming
that
this
is
you
know
this
user
has
already
paid
them.
They
don't
want
to
pause
like
if
the
user,
if
the
counterparty
is
like
great,
I
gave
the
thousand
I'm
waiting
for
the
next
payment
before
I
get
more.
The
developer
may
well
send
that
immediately
right,
like
because
they're
like
right
yeah,
these
are
just
downloading.
A
To
get
the
performance
you
need
some
like,
it
seems
like
if
you
suddenly
got
this
like
simple
thing
like
the
website
like
I,
I
don't
know
how
netflix
is
going
actually,
if
they're
having
to
check
in
regularly
between
some
dap
developers
over
which
the
thought
is,
they
shouldn't
need
a
mic.
Our
server
they're
just
like,
and
potentially
that
needs
to
be
decentralized.
B
D
B
The
idea
is
that
data
transfer
becomes
this
layer
upon
which
you
can
add
any
abstract
from
an
orchestration
mechanism,
and
you
know
it's
left
to
whomever
is
going
to
implement
that
to
do
that,
but
that
can
be
a
potentially
complicated
mechanism
right
now
that
one
in
tribal
climate
of
both
markets-
that
does
you
know
this
basic,
like
checking
of
data
between
two
parties.
But
presumably
there
will
be
more
that
the
intent
is
that.
D
B
Just
in
the
immediate
sense,
so,
first
of
all
so
payment
channels,
I'm
actually
one
thing
I'm
really
interested
in
is
like
what
is
our
view
about
things
about
retrievals
for
things
because,
like
we,
don't
actually
have
anywhere
to
find
that
find
that
out
yet.
But
if
you
get
a,
if
you
do
a
retrieval
that
has
or
that
does
not
have
an
author
copy,
the
user
experience
just
the
joke,
I
feel
you're
just
going
to
be
hanging
for
hours.
B
D
B
You
are
prepared
to
trust
the
person
you're
downloading
from
right,
they're
not
going
to
run
away
with
your
money
and
not
serve
you
right,
like
I
mean
like
the
simple
version
would
be
like.
I
pay
you
x
much
and
you
send
me
the
thing.
The
problem
is
like
like
what,
if
you
just
don't
send
me
the
thing
right
like
that's.
That's
the
fundamental
like
problem
of
their
exchange,
so
like
our
current
solution
for
that
in
a
trustless
world
is
to
essentially
do
it
in
increments
and
essentially
bootstrap
trust.
B
By
doing
a
series
of,
like
you,
know,
micro
payments
and
then
slowly
doing
bigger
and
bigger
pieces.
That's
how
retrieval
works
currently
like
the
first
payment.
Isn't
like
a
thousand
bytes,
two
thousand
bytes,
you
know
and
those
you
know.
The
third
is
is,
for
you
know
more
data,
and
the
fourth
is
for
even
more
data.
You
know
because
at
that
point
you're
like
well
both
parties,
it
responsibly,
probably
they're,
going
to
keep
behaving
responsibly
and
then,
like
the
only
problem.
There
is
the
last.
D
B
B
B
A
Other
thing,
just
as
we
go
deep,
independent
channels,
we
will,
I
think
there
is
this
expectation
or
hope
that
replication
systems
happen
in
this
app
this
year
as
well,
and
that
is
seen
by
product
as
a
potential
way
to
mitigate
some
of
this
trust
stuff,
which
is
there's
a
prioritization
of
minors
that
have
higher
applications,
which
means
things
like
gilbot
have
been
able
to
do
retrieval
and
have
that
work,
and
so,
potentially,
if
you've
got
that,
where
you're
verified
sometimes
that
have
higher
division.
You
can
say
great
above
this
reputation.
A
B
B
A
Well,
unless
you've
got
these
as
separate
places
right,
if
the
wallet
is
in
a
different,
it's
on
a
developer's
machine.
This
is
a
different
like
setup
like
if
you've
got
your
client.
That's
now
in
one
place.
Okay,
your
developer,
who's
got
a
wallet
somewhere
else
has
three
separate
parties,
and
these
ones
are
highlighted
in
c,
like
that
previous
mechanism,
potentially
walks
and
potentially
has
different
brake
systems,
like
we've,
assumed.
B
B
B
A
A
So
potentially
they
could,
I
guess,
themselves,
sort
of
somehow
based
on
the
location.
They
asked
my
clients
the
index
reference
yeah,
but
okay.
So
then
the
other
one
that
I
guess
is
on
the
indexer
thread
is
we
do
at
some
point
want
to
engage
some
of
the
kind
of
people
on
what-
and
this
is
like
this
retrieval
market
thing-
and
this
will
happen-
I
guess
in
the
next
couple
months,
but
we
get
a
retrieval
again,
because
I
think
there
is
some
thought
over
there,
which
is
how
does
the
indexing
system
subsystem?