►
From YouTube: How Bitswap Works - dig
Description
Everything The Iroh Team Learned Porting Bitswap To Rust
A
So
I'm
here
to
talk
to
you
about
bit,
swap
because
it's
it's
not
where
we
need
it
to
be,
it's
not
what
we'd
hoped
it'd
be.
I'm
gonna,
be
very
honest
about
that.
The
current
implementation,
I
wrote,
is
14
000
lines
of
rust
code.
Gobit
swap
has
about
20
000
lines
of
go
code,
no
rust
code
in
there.
Sorry,
so
we
got
to
figure
out.
Why
is
this
so
complicated?
And
but
before
you
do
that,
we
should
start
talking
about
what
is
bit
swap.
Who
here
knows
what
bit
swap
is
anybody?
A
Anybody
has
an
idea.
What
pit
swap
is?
Okay,
let's
start
with
what
what
does
Pittsburgh
do?
What
what
track
is
this?
This
is
while
data
transfers
are
here,
but
it's
what
does
more
than
data
transfers
it
turns
out.
It
also
has
content
Discovery,
who
knows:
what's
the
difference
between
data
transfers
and
content
discovery?
A
No,
no,
all
right!
All
right!
One
person,
cool
cool
I
know
there
are
more
people
here
that
know
I'm
sorry
I
know
anyway,
so
that
data
transfer
is
right
is
like
I
know
this
guy
has
my
data
and
I'm
going
to
fetch
that
data
con
Discovery
is
hey?
Who
here
in
this
world,
has
my
data?
So
if
you
go
to
Google
and
you
ask
Google
hey
where
do
I
find
the
nearest
hardest
memes,
then
Google
will
tell
you
and
that's
how
you
do
your
account
discovery,
but
bit
swap
can
also
do
that.
A
For
you,
you
don't
need
to
ask
Google.
You
can
ask
bit
swap
so
bit.
Swab
is
like
hey.
If
you
want,
you
tell
me
the
Box,
you
want
and
then
I'll
ask
the
network
for
you
and
then
magically.
We
also
transfer
those
blocks.
Unfortunately,
there's
there's
a
little
bit
of
problems
with
this.
So
if
you
look
at
what
bitter
actually
does
today
on
the
network,
this
is
how
it
looks
like
so
it
sends
out
what
we
want.
We
want
all
the
cids
so
announce
a
lot
of
cids.
A
Basically,
hey
has
anybody
got
this
stuff
and
then
you
get
back
hopefully
announcer.
Then
most
people
will
tell
you
I,
don't
have
this
stuff
because
usually
you're
asking
something
for
something
that
nobody
has
or
like
this
one
guy
in
the
conference
next
to
you
on
their
laptop,
but
nobody
else
right
and
so
what
happens?
Is
you
get
the
data
and
it
immediately
you
send
out
cancellations
to
everybody
else.
So
if
you
write
a
new
implementation,
you
connected
a
bit
swap
Network.
What
do
you
see
you
sent?
Give
me
all
the
cids.
A
You
look
in
your
store
and
before
you've
looked
into
your
store,
you
get
a
message
like
cancel
all
the
cids,
and
that
happens
about
a
hundred
times
per
second
great,
that's
not
very
cool.
If
you're
on
your
laptop
and
you're
trying
to
like
you
know,
preserved
battery,
maybe
is
that
a
thing
or
still
so
that's
not
great.
The
other
thing
that
you
might
think
a
lot
of
these
protocols
that
we
talk
about
are
request.
A
A
Maybe
I
don't,
but
that's
really
unfortunate,
because,
as
I
said,
we
get
100
messages
per
second,
then
I
don't
know
who
sent
me
these
messages
well,
I
can't
check,
but,
like
I,
don't
know
whether
they're
in
response
to
they're,
probably
not
in
response
to
anything
because
I
haven't
sent
out
anything
yet,
but
that
makes
it
really
hard
for
any
node
to
manage
what
they
actually
want
to
do
on
the
network,
so
cool.
So
some
challenges
all
right.
A
That's
not
so
bad
right,
bitsworks
works,
right,
FPS,
content
data
transfer
that
that's
a
thing
that
happens
right.
Well,
unfortunately,
if
you
go
on
the
internet,
this
is
the
current
state
of
the
art.
This
is
what
people
will
tell
you.
It's
like
waiting
for
bit
swap
to
Finish,
Well
I,
usually
say
ipfs,
but
we
know
that
right
now
bits
what
was
the
main
complementation
of
data
transfer,
so
maybe
a
bit
swap,
but
that's
not
cool
I,
don't
want
to
leave
this
meme
up.
I
wanna,
I,
wanna,
I,
wanna,
I
wanna
get
to
this
day.
A
A
A
We
have
to
break
up,
and
you
know
what
we
have
to
break
up.
We
have
to
break
up
content
Discovery
and
data
transfers,
because
even
the
tracks
are
broken
up
right.
The
organizers
of
this
event
understand
this.
We
we
gotta,
we
gotta
think
differently.
The
talk
previously
right,
we're
talking
about
methods
and
indexers
and
things.
So,
let's
see
what
can
we
do
right?
A
So
the
first
step
right
you
want
to
download
something
on
the
Internet
over
ipfs.
What
do
you
need
to
do?
You
need
to
figure
out
who
has
what
I
want
so
where
the
content
is
is
kind
of
hard.
If
you
talk
to
anybody,
who's
implemented
DHT,
it's
not
the
most
trivial
thing
to
implement.
The
indexes
are
cool
solution.
A
lot
of
nodes
on
the
ipfs
network
today,
unfortunately
have
DHT
providing
turned
off,
which
means,
if
you
go
to
the
DHT
today,
you
actually
don't
find
anything.
A
You
go
back
to
bit
swap
remember
the
I
send
all
the
ones
on
the
network.
That's
where
you
go
back
to,
but
I
think
we're
actually
not
in
a
bad
State
here.
So
thank
you
to
will
Scott
and
everybody
else
who
worked
in
inductor
notes.
We
can
actually
get
a
decent
amount
of
information
from
the
induction
nodes
from
nodes
that
are
very
large
in
the
network
right,
so
we
have
a
peer-to-br
network
and
all
the
peers
are
equal.
No,
we
want
the
period
to
be
equal,
but
the
peers
are
not
the
same
right.
A
My
big
machine
at
home
is
a
little
bit
more
capable
than
my
phone
or
this
laptop,
and
so
let's,
let's
write
software
that
actually
acknowledges
that
and
the
indexer
nodes
are
a
great
thing
right.
If
you
go
to
pinata,
they'll
probably
need
a
different
solution
to
host
data
on
on
this
network
than
any.
When
I
publish
a
meme
from
my
phone
right,
that's
the
thing,
and
so
the
indexer
nodes
are
really
good
step
forward
here,
to
allow
us
to
ingest
actually
large
amounts
of
data
and
make
them
available.
A
It's
not
perfect
right.
It's
not
perfect.
It
is
a
Federate
more
of
a
much
more
Federated
system,
but
we
can
get
to
a
pretty
good
State.
Everybody
can
run
an
injection
node,
we'll
just
need
a
way
of
sharing
and
distributing.
Where
do
you
find
extra
nodes?
Maybe
that's
Google
yeah,
not
that
great,
probably
have
a
solution,
but
hey.
We
could
also
use
the
DHC.
We
could
have
a
DHT
record.
That
says
here
is
the
indexer
nodes.
A
The
other
thing
is,
we
have
the
DHD
and
hey
swap,
went
into
the
direction
of
making
a
lot
of
Discovery
historically,
because
the
dhg
didn't
work
or
it
worked
really
badly
for
large
amounts
of
data.
But
very
clever
people
and
cool
people
have
worked
on
a
DHT
over
the
last
years
and
it's
actually
gotten
better,
and
so
we
can
probably
now
start
publishing
at
least
root
caches.
I,
remember:
ipfest,
chunks,
things
into
many
tiny
pieces,
all
of
those
have
hashes
and
there's
a
route
that
matches
them
together.
A
We
probably
should
only
put
the
root
on
the
DHT,
not
the
whole
thing.
It's.
It's
saves
some
bandwidth,
but
that
works
pretty
good.
The
focus
of
probe
lab,
Giannis
and
Co
have
been
working
on
new
parameters
for
the
DHT,
making
things
a
little
bit
better
and
Adine
has
working
I've,
Been,
Working,
On,
The,
Accelerated,
DHT,
client,
pretty
cool.
A
That
gets
us
actually
an
estate
where
we
can
start
using
the
DHT,
at
least
for
the
stuff
I
want
to
publish
from
my
laptop
sweet
all
right.
Let's,
let's
say
this
works
cool.
We
still
have
the
thing
like
we
need
to.
We
need
to
actually
do
the
thing
that
this
track
is
about
right.
The
guy
comes
up
talks
about
comp
Discovery,
and
we
want
to
do
data
transfers
are
here
all
right,
so
so
so
does
Bob
does
Bob
can
Bob
tell
us
how
we
do
better
data
transfers?
A
Maybe
so
bitter
is
not
the
only
data
transfer
protocol
out
there
today
there
are
other
people
have
clever
ideas,
and
some
of
them
are
built.
Hannah
built
this
thing
called
graphsync,
but
there's
a
spectrum
here
and
probably
there's
there's
stuff
on
either
side
of
this.
This
is
not.
This
is
not
the
whole
Spectrum,
but
I
think
it's
important
to
look
at
this
a
little
bit.
So
on
one
side
we
have
bit
Swap
and
bit
swap,
doesn't
understand
anything
about
your
data.
The
only
thing
that
bit
swap
understands
about
the
data
is
hey.
A
You
have
a
hash
and
you
have
a
blob
of
bytes
and
you
have
a
hashed
blob
of
bytes
I.
Get
that
hash
on
the
other
side
of
the
world.
You
have
graphing,
which
tries
to
understand
a
lot
about
your
data.
It
has
support
for
iply
selectors,
so
you
can
explore
all
your
iPod
data
and
then
stream
them
the
solution
back
to
you,
that's
pretty
cool,
but
it
means
now.
Your
protocol
is
pretty
complicated,
but
now
your
protocol
needs
to
understand
a
lot
about
your
data.
A
The
focus
of
fission
have
been
working
on
something
called
car
mirror,
which
is
a
little
bit
towards
the
Middle,
where
we
start
thinking
about.
Okay,
we're
not
going
to
support
all
IPL
these
actors
we're
going
to
support
a
little
bit
of
less,
but
we
also
support
their
support,
something
where
they
send
just
a
list
of
stuff.
They
already
have.
A
That
is
a
little
bit
simpler
even
than
camera
and
a
little
bit
more
complex
than
bit
swap
because
if
everything
is
chunked
remember
everything
is
chunked
into
tiny
pieces.
So
we
end
up
having
a
lot
of
round
tips,
not
so
cool,
because
latency
is
a
limiting
factor
and
hey
what
if
we
send
actually
something
to
the
Moon.
You
can't
like
that
takes
time.
A
So
my
current
idea-
and
this
is
very
new
I've-
talked
to
some
of
you
about
this.
This
will
probably
change
before
everything
any
thing
ships,
but
this
is
my
plea
for
like
we
need
something
else.
It
looks
a
little
bit
like
this,
so
we're
gonna
do
requests.
I
mean
everybody.
Hopefully
here
has
wants
to
use
the
Gateway,
and
if
you
go
to
the
Gateway,
you
make
requests
like
look
at
the
path
right
and
you
send
this
weird
hash
thing
and
then
slash
my
my
cat
and
then
my
meme
right
cool.
A
What
if
I
could
send
this
over
bit
swap
or
you
know,
cool
new
protocol
now.
This
meme
is
very
small
as
you've
seen.
The
resolution
of
my
memes
is
not
so
high,
so
it'll
fit
in
a
in
a
single
block.
That's
cool
now,
but
I
have
also
a
cad
documentary
that
I
want
to
watch,
and
so
that's
not
going
to
fit
into
a
single
block,
and
but
I
would
really
like
to
avoid
like
having
18
round
trips
to
fetch
the
whole
video
right,
because
I
actually
want
to
watch
the
whole
video.
A
So
what
if
I
could
be
a
little
bit
more
say
a
little
bit
more
about
this
thing,
I'm
calling
this
parameter
recursion!
It's
it's
a
stretch
on
recursion,
but
the
idea
is
I
want
I
want
this
thing,
but
I
also
want
a
little
bit
more
and
that
means
like
expand
on
the
tree
structure.
That
is
behind
this.
In
this
case,
for
two
levels,
so,
okay,
we
can
send
these
requests,
but
what
are
we
getting
actually
back?
Is
that
actually
useful?
A
So
one
big
problem
that
we
have
if
we
start
doing
more
blocks,
that
request
more
blocks
is
trust
right.
So,
if
I
send
you
hash,
you
send
me
a
blob
of
bytes
back
I,
hashtag,
Bob
I
know
it's
correct,
it's
wrong.
If
you're
sending
me
wrong
stuff,
I
stopped
talking
to
you
now.
If
I,
you
send
me
many
blocks,
I
don't
want
you
to
send
me
mini
blocks
before
I.
Stop
talking
to
you,
because
you
sent
me
the
wrong
block.
So
the
idea
here
is
we
do
so.
A
A
First
stop
the
root
block.
Now
the
root
block
is
going
to
have
some
links
so
I
get
through
with
block
I.
Look
at
it,
I
look
at
the
links
sweet.
This
is
just
the
right
lock,
there's
a
link
in
there,
which
says
my
right.
So
the
next
block
that
needs
to
come
is
this.
My
so
I
can
hash
that
next
block
and
block
okay,
my
hash
match
is
cool
and
so
I
go
on
until
the
end.
If
at
any
point
any
block
arrives
in
my
world,
that
is
not
fitting.
A
This
chain,
I,
can
abort,
so
I
don't
have
to
trust
the
other
party,
except
for,
like
while
I,
do
have
to
read
the
data
and
check
that
it
verifies,
but
that's
the
same
thing
you
have
in
bit
swap
today
now
this
is
considerably
less
round
trips
than
you
will
have
in
bit.
Swap
because
remember
for
like
in
this.
This
same
request.
Response
play
in
bit
swap
today
would
at
least
require
you
to
go
for
each
block.
One
round
trip
right,
because
you
get
one
block,
you
parse,
the
block.
A
You
send
the
near
next
request
and
because
bit
swap
is
very
gossipy.
It
will
actually
be
probably
many
many
more
actual
things.
So
that's
the
current
idea.
The
current
ideas
that
I
have
there.
We
need
to
build
a
little
bit
more.
It's
a
purely
request
response
protocol
and
what's
much
cooler,
is
that
I
had
an
idea.
Is
that
other
people
seem
to
like
the
idea
that
we
need
to
move
forward
and
build
a
better
data
transfer
protocol
that
helps
us,
and
so
it
I'm
proud
to
announce.
A
new
working
group
called
move.
A
A
There's
like,
send
us
a
message
and
tell
us
what
actually
looks
is
is
the
data
that
you're
working
on
because
it
turns
out
doing
data
transfers
efficiently,
man,
it
matters
a
lot,
what
your
data
is
and
how
you
move
it
around,
and
so
you
really
want
to
be
sure
that
you
can
manage
these
use
cases.
We
don't
have
all
the
use
cases,
so
you
all
have
the
use
case
so
bring
them
to
us.
Tell
us
how
you're
moving
data
around
we're
gonna
work
on
this.
Hopefully
we're
gonna
ship
something
a
prototype
soon.
A
That's
that's
the
goal.
Important
disclaimer
everything
I've
shown
you
we
might
throw
this
out.
This
is
an
ear,
but
it's
to
get
us
talking
to
get
us
moving
forward,
actually
making
progress
here
such
that
we
can
actually
move
these
bytes
faster
and
we're
going
to
measure
it
right.
We're
going
to
put
it
on
the
wire
we're
going
to
see
hey
how's.
What's
the
overhead
is
this
actually
faster?
Is
this?
Maybe
maybe
the
idea
is
bad
and
this
is
going
to
be
slower,
so
we
like
need
to
figure
out
something
else.
A
It's
not
going
to
replace
bit
swap
overnight
right.
A
lot
of
people
worry
about.
How
do
you
actually
upgrade
to
your
new
data
transfer
protocol
right?
Luckily,
originally,
when
we
built
ipfs,
we
have
protocols
in
place,
so
we
can
add
many
more
protocols.
We've
just
heard
a
talk
about
Heidi
at
your
own
protocol.
It's
not
that
hard!
A
Luckily,
and
we
have
nodes,
we
have
iron
nodes,
we
have
kubernodes,
we
have
jsipfs
nodes,
they're
all
speed
bit
swap
already,
so
they
can
Bridge
the
world
for
a
while,
so
even
of
old
nodes
don't
upgrade.
We
can
still
fetch
data.
We
can
still
use
the
gateways
to
fall
back.
It
worse
comes
to
worst,
but
we
can
search
that
new
nodes
start
talking
new
protocols
and
then
actually
talk
to
each
other
considerably
faster
yeah.
Thank
you.