►
From YouTube: Keeping Estuary Content Available - Elijah Arita
Description
How Estuary keeps content available today and in the future
A
This
presentation
is
about
estra's
work,
with
keeping
our
data
available
on
ipfs
and
a
new
server
we've
been
working
on
called
Auto
retrieve,
which
is
going
to
help
with
that.
So
a
quick
recap
on
Estuary,
if
you're
unfamiliar
it's
an
Open
Source
service
that
allows
users
and
developers
to
easily
pin
your
data
to
ipfs
and
store
on
filecoin
when
you
upload
your
data,
it'll
automatically
make
six
storage
deals
with
different
storage
providers
on
the
filecoin
network
and
pin
your
data
to
our
ipfs
Gateway
and
here's
a
little
sample
of
our
website.
A
A
A
A
So,
in
consideration
of
that
we've
been
working
on
this
server
called
Auto
retrieve
Auto
retrieve
it's
a
standalone
server.
That's
able
to
it's
a
bit
swap
server,
it's
able
to
pull
blocks
from
the
filecoin
network
and
forward
them
to
ipfs
clients.
So
it's
basically
a
file
coin
to
ipfs
bridge
the
server
supports
both
free
and
paid
retrievals.
A
When
it's
done,
Auto
retrieve
will
be
able
to
take
the
place
of
estra's
gateways
right
now
in
most
circumstances
and
allow
us
to
start
unpinning
most
of
our
data
from
our
own
infrastructure
and
allow
us
to
grow
a
lot
beyond
what
we
have
already,
which
is
249
terabytes
we're
looking
to
go
a
lot
higher
than
that
in
more
detail.
This
is
how
Auto
retrieve
serves
our
blocks
to
clients.
A
A
Auto
retrieval
then
go
to
that
file.
Coin
provider
and
download
the
the
deal
that
contains
the
CID
that
was
requested
and
as
soon
as
it
lands
into
the
auto
retrieves
block,
store,
it'll,
go
and
forward
it
over
back
to
the
ipfs
client
and
that
repeats
for
every
block
that
the
ipfs,
client
requests
and
auto
retrieve
does
not
need
a
very
big
block
store
because
all
the
data
that's
stored
on
it
is
kind
of
ephemeral
and
only
used
for
forwarding
directly
to
the
bit
swap
client.
A
A
This
is
kind
of
a
difficult
issue,
because
Auto
retrieve
doesn't
actually
know
what
content
it's
able
to
serve
because
it
hasn't
downloaded
it
yet
in
reality,
yeah
Auto
retrieve
can
serve
any
block
that
is
available
on
the
filecoin
network,
paid
or
unpaid,
and
that
is
a
lot
of
data
indexing
through
the
entire
chain
and
uploading.
Every
multi-hash
to
the
index
here
has
proved
to
be
not
feasible,
so
we
had
to
come
up
with
a
different
approach.
A
Our
current
solution
to
that
is
that
estuary,
the
storage
service
already
knows
all
the
data
that
we
have
stored
on
it,
and
so
we
can
do
is
we
can
have
Auto
retrieve
talk
to
Estuary
and
say
hello,
I'm
here
and
sqa
will
automatically
go
and
make
an
advertisement
to
the
indexer
in
the
name
of
Auto
retrieve
we're
in
the
process
of
implementing
this
feature.
Still
it's
working
progress,
but
it's
almost
done.
We've
gotten
a
lot
of
work
done
at
at
this
event.
Really
with
our
team
together,
it's
been
pretty
cool.
A
A
A
So.
Currently,
there
are
two
full-time
Auto
retrieve
instances
hosted
one's
by
outer
core
engineering,
which
is
us,
and
the
other
is
by
Bedrock
team
for
their
retrieval
metrics
again.
The
DHT
advertising
and
indexer
advertising
is
work
in
progress,
It's
almost
done
and
when
that's
done,
we'll
be
able
to
start
and
unpinning
all
of
our
data
from
Estuary
and
we'll
really
be
able
to
just
serve
that
data
directly
from
the
filecoin
network
to
ipfs
clients.
A
Yeah
I
think
that's
everything
you
can
come
talk
to
us
on
the
filecoin
slack
hashtag
ecosystemdev
or
you
can
talk
to
me
directly
at
Elitch,
and
you
can
also
check
out
the
file
coin
or
the
auto
retrieve
source,
github.com
application.
Research,
slash,
Auto,
retrieve,
yeah,
I,
hope
that
auto
retrieve
can
be
useful
to
people
and
that's
our
solution
for
going
forward
for
keeping
ipfs
data
available
from
Estuary.
Thank
you.