►
From YouTube: IPNI introduction for web3 & Big Data Storage Summit
Description
An introduction to the interplanetary network indexer prepared for the web3 Big Data Conference in Shenzhen, China 07-04-2023 prepared by the Bedrock team at Protocol Labs.
For more information please refer to the following channels on Filecoin Slack:
#IPFS-content-routing-wg
#ipni
Prepared by: Torfinn Olsen
A
Hello:
everyone,
my
name,
is
torphin
Olson
and
I'm,
a
technical
program
manager
on
the
Bedrock
team
at
protocol
labs
first
off
I'd
like
to
thank
sxx
future
data
for
inviting
me
to
the
China
web3
Big
Data
Summit,
so
who
am
I
I'm
a
technical
program
manager
with
a
background
in
building
ground
up
global
data,
intensive
applications
from
web3
all
the
way
to
iot
I
joined
protocol
labs
in
order
to
be
a
part
of
driving
breakthroughs
and
Computing
that
push
Humanity
forward.
It's
our
core
mission
of
protocol
labs.
A
What's
this
presentation
about
you're
going
to
learn
about
the
interplanetary
network?
Indexer
why
you
should
be
excited
about
it
if
you
currently
store
data
on
filecoin
or
if
you're
interested
in
storing
data
for
filecoin
in
the
future,
we'll
also
talk
about
the
future
of
finding
data
on
the
filecoin
network.
What
that
looks
like
for
both
retrieval
clients
and
for
storage
providers
alike,.
A
A
A
A
A
So
what
is
it
exactly
when
we
really
get
down
to
it?
Storage
providers
want
to
advertise
their
content.
That's
stored
in
the
indexer.
The
retrieval
clients
want
to
find
that
content
as
quickly
as
possible
to
improve
content.
Discoverability
indexer
nodes
are
developed
to
store
mappings
of
SIDS
to
content
providers
for
Content
lookup.
Upon
retrieval
request.
A
A
The
ipni
ecosystem
consists
of
three
main
actors:
content
providers.
These
are
the
participants
who
host
content,
addressable
data,
also
known
as
storage
providers
ipni
nodes.
These
are
the
participants
who
ingest
announcements
about
the
content,
addressable
data
and
then
retrieval
clients.
These
are
the
participants
who
find
such
content
via
indexer
nodes
and
they
fetch
up
from
the
providers.
A
A
This
property,
combined
with
the
chaining
of
advertisement,
entries
effectively
captures
a
diff
of
content
hosted
by
the
provider
over
time.
When
a
change
in
content
occurs,
either
new
content
is
added
or
perhaps
some
is
removed.
The
provider
captures
the
change
as
a
new
advertisement,
adds
it
to
the
chain
and
announces
its
existence
to
the
network.
A
The
indexer
node
itself,
ipni
nodes,
are
responsible
for
continuously
listening
to
provider
announcements
once
they
receive
an
announcement.
They
fetch
the
advertisement
and
walk
its
chain
to
effectively
construct
the
current
list
of
content
hosted
by
the
provider,
because
the
advertisements
themselves
are
immutable.
Ip9
nodes
can
infer
seen
from
unseen
advertisements
and
only
walk
the
portion
of
the
chain
that
has
not
been
seen
before.
A
A
A
A
A
The
second
step
is
the
indexer
syncs
new
content.
So
anytime
the
storage
provider
gets
new
content,
it
needs
to
be
synced
to
the
index,
are
known
and
it
can
get
advertisements
in
the
chain
up
to
the
last
scene
or
the
end
of
the
chain.
It
gets
the
context,
ID
metadata
content,
multi-hash
and
the
chunk
chain
for
the
new
advertisements.
A
A
A
Additionally,
we're
working
on
the
ability
to
fetch
from
alternative
advertisement
sources.
So
you
potentially
don't
have
to
you
know,
run
our
advertisements.
You
can
provide
advertisements
via
different
different
means
and
then
also
we're
working
on
reputation-based
ranking
of
providers
we're
starting
with
a
design
plan
for
that
presently
and
then,
ultimately,
we
want
to
help
our
other
index
operators
adapt
to
double
hashing,
so
that
they'll
also
have
the
benefits
of
this
additional
privacy.
A
A
For
that
way,
there's
not
one
Central
index
or
instance,
or
specific
geo-locked
instance
that
you're
looking
up
there's
an
array
of
indexers
that
you
could
call
upon
near
to
the
region
where
you'd
like
to
perform
your
lookup.
That
way,
it's
a
faster
lookup,
there's
less
traffic
created
across
the
network
and
additionally,
you
have
the
benefit
of
decentralization.
A
So
the
Network's
more
secure
by
result
of
having
multiple
nodes
in
different
locations
also
we're
looking
at
getting
Auto
discovery
of
indexers
by
Kubo
nodes,
and
so
this
is
the
ability
for
the
ipfs
network,
Kubo
nodes
to
Auto,
discover,
indexers
and
potentially
validate
the
quality
of
an
indexer
and
choose
the
appropriate
indexer
to
fulfill
a
search.
Query
really
the
longer
term
goal
for
this
team
is
presently.
A
A
All
right
and
ultimately
before
I
bid,
you
all
I,
do
I'd
like
to
provide
you
with
some
resources
to
potentially
learn
more
and
also
to
encourage
you
to
join
us
in
the
filecoin
slack
Channel
hashtag
ipmi.
You
can
ask
us
any
questions
there.
The
members
of
the
team
are
all
very
active
participants,
but
if
you'd
like
to
learn
more
I'd
recommend
starting
out
with
this
blog
post
that
I've
linked
up
at
the
top
on
the
file
coin
blog
about
the
network
indexer,
there
is
a
description
page
of
how
the
ipni
works.
A
Network
indexer
specs
are
on
GitHub.
It's
an
open
source
project,
so
you're
welcome
to
come.
Take
a
look
comment.
Ask
questions
there
we're
happy
to
have
you
and
then
there
are
some
videos
for
Content
routing
that
get
really
into
the
Weeds
on
how
this
all
works
and
what
the
future
holds
for
us
again.
I'd
like
to
thank
s,
access
for
inviting
us
to
participate
in
this
conference
and
I'm
really
hoping
that
some
of
you
would
like
to
reach
out
and
ask
us
some
questions.