►
From YouTube: IPFS Cluster - @hsanjuan - IPFS Implementations
Description
IPFS Cluster - presented by @hsanjuan at IPFS bing 2022 - IPFS Implementations - https://2022.ipfs-thing.io
A
It
was
good
to
see
a
lot
of
new
stuff.
This
is
a
bit
of
an
old
stuff,
because
this
is
started
soon
soon
after
go
ipfs.
Actually,
so
I'm
going
to
tell
you
about
ipf's
cluster,
we
will
go
very
quickly
over
the
features
and
we
will
see
try
to
explain
a
little
bit
of
the
internals
and
about
about
how
it
uses
the
go.
Ipfs
stack.
A
I
don't
think
we
should
call
it
kubo
stack
anyway
under
the
hood,
so
ibfs
cluster
does
data
orchestration
against
a
swarm
of
kubo
demons
as
a
as
a
side
car.
So
what
it
does
is
to
select
given
a
replication
factor
for
a
pin
it
selects,
which
kubo
demons
in
your
cluster
should
be
the
one
spinning
and
it
tells
them
to
pin
it
and
then
it
makes
sure
they
actually
pin
it
if
they
don't
it
retries
and
etc,
etc.
A
The
whole
idea
here
was
that,
if
you're
going
to
deploy
kubo,
you
might
as
well
just
deploy
the
ipfs
cluster
daemon
next
to
it
more
or
less
for
the
same
price
in
a
very
similar
fashion.
It's
also
a
daemon,
it's
configured
more
or
less.
In
the
same
way,
and
you
run
it
more
or
less
in
the
same
way
and
you're
going
to
get
ipfs
coordination
on
top
almost
for
free.
A
What
it
does
print
replication,
you
can
also
do
car
imports
and
ipfs
ads,
but
to
multiple
google
peers
at
the
same
time,
balanced
allocations
apis.
I
can
go
a
little
bit
more
in
detail,
but
the
idea
is
that
every
cluster
in
the
every
period
in
the
cluster
has
a
shared
pin
set,
so
they
they
are
tracking
a
pin
set
and
all
of
them
have
that
estate
between
them
and
we
will
see
how
it
is
coordinated
and
every
pin
in
that
pin
set
has
several
options
among
the
replication
factor.
A
A
It
doesn't
sort
of
balance
the
locations.
So
originally
cluster
will
select
a
location
based
on
the
on
the
amount
of
free
space
reported
by
google.
But
now
it's
a
little
bit
more
elaborate,
because
if
you
attack
your
cluster
peers
right,
you
will
get
a
locations
that
you
can
distribute
around
the
world
and
you
can
say
I
want
to
replicate
this
pin
in
three
places,
but
please
do
it.
A
So
those
three
places
are
not
in
the
same
data
center
and
are
not
in
the
same
availability
zone,
and
once
you
have
selected
places
that
are
distributed
around
the
world,
please
select
those
peers
that
do
not
have
very
high
pinning
queues
and
once
you're
dealing
with
the
ones
that
are
more
or
less
not
pinning
too
many
things.
Then
please
select
the
ones
with
most
available
space,
and
this
is
user
configurable
to
your
needs.
A
A
So
this
is
used,
for
example,
for
for
distributing
package
repositories.
Someone
has
a
collaborative
cluster
that
says
you
can
follow
this
and
by
follow
win
this
cluster
you're
gonna
get
the
latest
pin
set
that
allows
you
to
pin
all
the
packages
internals
a
cluster
appear
is
made
of
a
bunch
of
components.
A
It
doesn't
know
what
is
the
status
on
a
bin
that
is
actually
stored
by
some
other
cluster
beer,
but
you
can
use
the
rpc
to
actually
go
and
ask
the
other
peer.
Hey.
Do
you
have
this
bin,
or
is
it
still
cute?
That's
address
address,
therefore,
you
make
that
request,
and
then
you
come
back
to
the
api
to
the
user
request
on
the
api
saying:
hey
yeah
and
this
bin
is
cute
there
and
it
spins
somewhere
else.
A
This
is
done
through
a
lip
b2b,
so
an
rpc
library
only
p2p
that
we
wrote.
Essentially,
we
took
this
the
standard,
rpc
library
and
made
it
p2b
enabled,
and
then
we
added
features
on
top.
So
now
it's
not
only
calling
methods
and
getting
a
response.
We
can
also
do
a
streaming,
so
we
can
pass
channels
to
methods
and
we
will
stream
everything
on
the
channel
and
then
get
a
channel
as
a
response
and
stream
things
out
comes
quite
handy
and
works
on
top
of
ptp.
A
The
idea
is
that
each
cluster
beer
has
to
agree
on
what
the
pin
set
is,
and
they
all
have
the
ability
to
update
that
pin
set
sometimes
very
heavily
if
they're
ingesting
many
pins.
At
the
same
time,
therefore,
we're
using
in
order
to
manage
that
pin
set,
that
is
a
shared
state
among
all
the
cluster
peers,
we're
using
go
dscrdt,
which
is
an
implementation
of
mercury
crdts
and
it's
essentially
america
dag
is
a
blockchain
except
you
don't
need
consensus,
because
all
the
objects
in
that
blockchain
can
actually
be
merged
together.
A
A
Of
course,
that
means
go
dser
ddt
and
it's
an
ipld
dac
service,
and
this
is
provided
by
ipfslite,
which
is
just
a
minimal
implementation
of
a
dac
service.
Using
the
go
ipfs
stack,
dp2p
host
might
plus
the
hd
plus
datastore
plus
blogstore,
you
wrapped
it
on
a
cache
blog
store.
You
wrap
it
on
a
bloom
filter,
enable
blogstore,
you
add
some
bits
up
to
it.
You
get
a
block
service,
you
had
a
reprovider,
and
now
you
have
a
dac
service
fully
functional.
You
can
grab
anything
from
any
ipfs
network
that
you
want.
A
This
is
essentially
the
list
of
libraries
that
we
use
from
from
the
stacks.
In
the
end,
if
you
noticed
ipfs
cluster
is
a
distributed.
Peer-To-Peer
database
with
the
purpose
just
the
purpose
of
coordinating
binning
and
it's
taking
advantage
of
essentially
every
piece
that
the
that
the
go
ipfs
stack,
which
is
the
the
thing
that
was
made
here
enough
when
we
were
building
this
and
as
we
were
building
this
had
to
offer
and
we
added
the
missing
pieces.
A
So
we
added
the
crdt
part
we
added
the
go
rpc
part,
but
all
the
other
pieces
facilitated
a
lot
a
lot
of
the
tasks
here,
because
we
get
pops
up
for
free.
We
get
content
addressing
and
the
ability
to
receive
blogs
from
anywhere
in
the
network
for
free,
et
cetera,
et
cetera,
so
we,
which
is
covering
small
bridges,
pain,
points
being
a
side.
Car
pain
points
right
now
things
that
bother
me
right
now.
A
lot
of
things
bother
me
in
the
past,
but
right
now,
being
a
side
card
to
google
means
you
pay.
A
A
There
are
many
limits
to
fight
with,
like
the
total
number
of
pins,
that
the
cluster
can
track
the
total
number
of
pins
that
the
cluster
can
ingest.
So
how
can
we
ingest
faster,
but
also,
how
can
we
increase
the
speed
as
at
which
we've
been
contents,
and
how
can
we
increase
the
speed
at
which
we
add
content
and
those
are
those
are
different
paths
and
they
have
like
different
solutions
and
different
optimizations
how
to
maximize
performance?
What
is
the
right
size
of
a
cluster
beer
when
you're
when
you're
hitting
all
those
limits?
A
What
is
the
right
number
of
parallel
pin
at
calls?
What
is
the
right
settings
for
google
for
badger
for
the
file
system
in
the
nodes
that
you're
running
for
the
layout?
What
is
the
right
balance
between
the
disk
size
and
the
memory?
How
many
nodes
do
you
want
to
run?
How
can
you
optimize
and
last
run
less
notes
or
not?