►
Description
We’ll talk about the challenges of building a data propagation protocol for permissionless blockchain systems. Then, we’ll introduce GossipSub as a message and block propagation protocol for permissionless blockchain environments in general and for Filecoin in particular. Next we’ll discuss the design and test methodology, as well as the results we have taken from our extensive testing campaign using Testground.
Join us for Filecoin Liftoff Week, an action-packed series of talks, workshops, and panels curated by the web3 community to celebrate the Filecoin mainnet launch and chart the network’s future. https://liftoff.filecoin.io/
Events take place all week, October 19-23, 2020. #FilecoinLiftoff
For more information on Filecoin
- visit the project website: https://filecoin.io/
- or follow Filecoin on Twitter: https://twitter.com/Filecoin
Get Filecoin community news and announcements in your inbox, monthly: http://eepurl.com/gbfn1n
A
So
today
we're
going
to
talk
about
gossip,
which
is
I'm
really
happy
about
that,
because
it
has
been
a
great
really
great
effort
and
collaboration
between
the
lip,
b2b
team
and
members
of
the
rest
net
lab
myself
included
in
case
you
don't
know
what
rest
net
love
is.
You
should
go
check
it
out,
because
you
really
miss
a
lot
of
things,
a
lot
of
interesting
things
taking
place
there,
so
we
designed
gossip
sub
for
for
it
to
be
a
secure
message,
propagation
protocol
for
the
filecoin
blockchain,
and,
of
course,
not
only
so.
A
A
So
when
we're
dealing
with
permissionless
blockchain
networks
such
as
the
filecoin
one
or
others,
ethereum
ethereum
or
bitcoin,
we
we
have
several
features
of
those
networks
that
we
need
to
take
into
account.
So
the
permissions
blockchains,
for
example,
are
open
for
everyone
for
anyone
to
join.
There
is
no
access
control,
which
makes
it
really
easy
for
malicious
nodes
to
just
join
the
network
and
try
to
disrupt
the
operation
of
the
network.
A
A
They
need
to
do
verification
of
messages,
so,
as
messages
are
propagating
around,
they
need
they
carry
together
cryptographic
proofs,
and
this
enables
miners
to
verify
that
you
know
some
transaction
is
coming
from
the
person
that
he
claims
to
be
coming
and,
of
course
it
needs
to
keep
the
thing
the
chain
in
sync.
So
you
know
in
order
to
reach
consensus,
everyone
in
the
network
needs
to
have
the
same
view
of
you
know
the
latest
messages
that
are
going
around.
A
A
Now
there
are
several
other
things
depending
and
deadlines,
depending
on
the
blockchain
that
runs
underneath
for
filecoin.
There
is
a
deadline
which
says
that,
within
six
seconds
of
a
message
being
produced
being
created,
it
has
to
reach
all
the
nodes
in
the
network.
As
you
probably
know,
the
filecoin
blockchain
operates
based
on
tip
sets
and
not
blocks.
So
tip
sets
are
groups
of
up
to
five
blocks,
and
this
is
what
miners
are
mining
on
they're
mining
based
on
deep
sets,
so
within
six
seconds.
A
What
this
means
is
that
we
have
from
zero
to
30
seconds
is
the
epoch
when
new
blocks
and
deep
sets
are
going
to
be
produced
and
we've
got
six
second
deadline
to
propagate
the
newly
published
messages
and
blocks
to
all
the
nodes
in
the
network.
This
is
needed
because
the
rest
of
the
time,
the
rest
of
the
24
seconds
is
used
in
order
to
do
all
the
cryptographic
operations
that
are
needed,
put
the
blocks
together,
produce
the
tip
sets
and
go
to
the
next
round.
A
So
what
we
need
to
remember
out
of
that
is
that
the
six
second
deadline
is
a
a
really
important
thing,
at
least
for
file
going
and
gossips
up.
And
what
we're
trying
to
do
is
we
try
to
see?
You
know?
Okay,
it's
an
open,
permissionless
network.
So
what
could
happen?
What
could
go
wrong
and
we?
We
tried
to
approach
all
the
primitive
attacks
that
exist
in
you
know
current
and
previous
blockchain
networks
and
what
we
sorry,
okay,
can
go
back.
Yeah,
we
have
a
range
of
of
attacks.
A
These
are
the
the
eclipse
attack,
which
is
well
known,
attack
a
civil
attack
which
comes
first
of
all.
First
of
all,
you
know
someone
would
need
to
create
basically
a
civil
identities
like
fake
identities,
so
that
they
have
many
many
nodes
in
the
network
and
then
you
can
carry
out
a
number
of
attacks,
so
eclipse
attack
is
one
of
them
where
I
know
these
basically
sinking
messages
and
not
propagating
messages
of
one
node
or
all
of
the
nodes
in
the
network.
A
A
A
If
we
want
to
describe
the
design
of
gossip
sub
in
a
nutshell,
we
would
say
that
it
has
got
these
four
basic
components:
it's
got
a
mesh
construction,
gossip
dissemination,
algorithm
scoring
function
and
some
mitigation
strategies
which
we're
going
to
see
very
briefly
pretty
much
all
of
them,
let's
start
from
the
mesh
construction.
So
what
goes
up
has,
as
the
main
principle
is
a
mesh
network,
so
nodes
are
connected
to
each
other.
No
node
is
connected
to
everyone
else.
There
is
a
limit
to
how
many
nodes
some
you
know.
A
Some
peer
can
connect
to,
and
every
node
has
its
own
view
of
the
network.
So
in
this
figure,
for
example,
we
can
see
you
know,
the
big
kind
of
cloud
is
the
whole
mesh
and
then
every
node,
let's
say
node
a
in
this
case
has
got
its
own
local
view
of
the
cloud
which
is
those
how
many
one
two
three
four
five
peers
that
it
is
connected
to.
So
this
is
the
view,
the
local,
in
a
sense,
the
local
mesh
of
node
a
or
more
accurately
the
view
of
the
mesh
from
node.
A
A
So
this
way
we
can
see
that
you
know
little
kind
of
smaller
meshes
of
each
node.
They
are
interconnected
with
each
other,
and
you
know
how
tight
and
well
connected
the
mesh
is
basically
depends
on
this
d
factor,
the
the
degree
of
the
node,
which
is
how
many
other
pairs
each
node
should
connect
to
we're
going
to
talk
about
how
we
set
this
d,
but
for
now,
let's,
let's
keep
this
important
part
now,
apart
from
that,
what
we
also
have
is
gossip
connections,
which
is
these
dashed
lines?
A
What
these
do
is,
apart
from
a
node
receiving
or
sending
messages
to
its
immediate
peers.
It
also
sends
gossip
messages
every
now
and
then
on
every
heartbeat
as
we're
going
to
see
this
the
what
this
gives
us
is
basically
further
reachability
to
a
wider
to
a
wider
number
of
nodes
in
the
network,
and
this
is
good,
of
course,
if
we
want,
if
a
node
is
is
eclipsed,
for
example,
he
can
send
out
messages
through
the
gossip
channels,
not
only
through
its
immediate
mesh,
so
a
node
that
is
joining
the
network.
A
A
We
have
two
main
well,
we've
got
more,
but
the
main
ones
are
the
graph
message
and
the
prune
message
how
these
work
is
that
when
a
node
wants
to
connect
to
an
extra
node,
so
if
a
wants
to
increase
d
from
five
to
six,
it's
going
to
send
a
graph
message
say
to
this
other
gray
node
here
when
it
wants
to
when
it
read
when
no
day
realizes
that
it
has
got
a
lot
of
connections
at
this
point
in
time,
then
it's
good.
What
it's
going
to
do
is
basically
it's
going
to
start
pruning
connections.
A
It's
going
to
start
cutting
down
connections.
Of
course
we
understand
that
the
more
someone
grafts,
the
more
messages
and
the
more
traffic
he's
going
to
to
create
to
the
rest
of
the
network
yeah.
So
when
we're
grafting,
we
have
a
range
of
a
range
of
peers
where
we
say
that
this
is
an
acceptable
number
of
peers
that
we're
connected
to,
and
if
we
are
within
this
range
of
nodes,
then
we're
okay,
which
is
this
case.
A
If
it's
too
little,
it's
grafting,
more
nodes,
now
yeah,
that's
more
graphically
how
it
works.
There
is
a
range
we
set.
The
d,
as
we
said,
the
d
e,
the
d
factor
or
the
degree
of
a
node,
is
how
well
the
mesh
is
connected
and,
of
course,
the
better
connected
the
mesh
is
the
more
messages,
the
more
traffic
it
propagates
to
the
network.
Now,
if
we
link
that
to
the
security
properties
of
the
protocol,
we
see
that
you
know
if
civil
node,
for
example,
wants
to
start
connecting
and
tries
to
eclipse
a
node.
A
A
So
the
more
connections,
the
more
we
increase,
this
d,
the
more
difficult
we
make
it
for
sybil
nodes
to
come
and
dominate
all
of
the
connections
right
and
the
list,
the
smaller
number
of
nodes
that
we
have,
as
is
the
ideal
degree
of
the
node.
What
we
achieve
basically,
is
that
we
we
we
have
less
traffic
in
the
network
in
general,
but
we
make
it
easier
for
cbnos
to
come
and
dominate
all
of
our
connections.
A
What
happens
is
that
upon
every
on
every
heartbeat,
the
node
is
sending
gossip
messages
out,
so
gossip
messages
are
not
sent
to
everyone
else
that
the
node
has
heard
about
a
node
is,
apart
from
the
direct,
the
direct
mesh
connections.
It
has
got,
you
know
a
list
of
nodes
that
it
has
seen
or
interacted
with
in
the
past.
So
out
of
those
nodes,
the
protocol
is
picking
as
a
fixed
percentage
and
it's
disseminating
gossip
messages
to
those
nodes,
the
gossip
messages.
A
Basically,
there
are
two
types
of
messages
there,
the
I
have
message
and
the
I
want
message,
a
note
that
is
disseminating
the
gossip.
It
says
I
have
those
messages
and
the
notes
that
are
receiving
this
gossip
message
are
checking
whether
they
have
they
also
have
received
the
list
of
messages
that
this
I
have
message
indicates
if
it
finds
that
you
know
it
doesn't
have.
Some
of
the
messages
means
that
these
messages
have
not
propagated
to
this
peer.
Yet
so
what
it
does
then
is.
A
A
Obviously,
as
I
said,
you
know,
if
we
put
sonic
graph
the
mesh
degree
and
the
dissemination
through
gossip,
we
can
arrive
at
interesting
trade-offs.
So
if
we
increase
the
mesh
degree
a
lot
and
we
keep
the
dissemination
factor,
the
gossip
dissemination
factor
a
lot.
Then
what
we
have
is
basically
we
go
to
an
eager
push
approach.
A
The
eager
push
approach,
what
it
does
what
it
points
to
is
that,
because
we
said
that
every
node
is
broadcasting
immediately,
all
the
messages
that
he
had
knows
about.
It's
eagerly
pushing
messages
out,
and
this
creates
more
traffic.
We
have
reduced
propagation
latency
in
this
this
in
this
case,
because
you
know
everyone
is
getting
to
hear
more
messages
quicker,
but
on
the
other
hand,
we
have
increased
propaga
increased
bandwidth
requirement.
A
On
the
other
hand,
if
we
keep
the
mesh
degree
to
a
small
value
and
we
increase
the
dissemination
through
gossip
messages,
what
we
do
is
we
go
to
this
other
kind
of
extreme,
which
is
the
well
the
lazy
pool
approach.
Everything
is
happening
through
the
lazy,
pull
approach
which
says
you
know,
I'm
not
going
to
eagerly
push
stuff,
I'm
just
going
to
wait
and
hear
from
others.
What
are
the
messages
that
are
known
to
the
network
and
I'm
going
to
lazily
ask
them
to
send
them
to
me.
A
In
this
case
we
have
less
traffic
in
the
network
and
less
traffic
redundancy,
of
course,
but
the
propagation
latency
increases
considerably.
So
there
is
a
trade-off
there
that
we
need
to
hit,
and
this
has
to
do
with
the
network
design
the
number
of
nodes
in
the
network.
How?
What
is
the
deadline
in
rk
in
the
file
going
case?
We
said
it's
six
seconds,
so
you
know
you
can
play
around
with
this.
A
So
here
is
just
a
snapshot
of
a
very
interesting
simulation
that
visual
has
done
in
the
past
where
it
shows
you
know.
This
is
basically
when
we
the
left
hand.
Side
is
when
we
flood
the
network.
So
we
can
imagine
that
d
is
very
high,
which
means
that
we
have
kind
of
faster
dissemination
but
more
traffic,
and
on
this
left-hand
side
we
have
gossip
sub.
That
does
not
have
as
big
bandwidth
requirements,
but
the
latency
could
be
a
little
bit
more
than
the
flooding
approach.
So
it's
all
about
a
trade-off.
A
Now,
let's
go
to
the
scoring
function,
which
is
used
by
by
every
every
peer
in
the
network
is
using,
is
keeping
a
score
for
every
other
peer
that
has,
it
is
connected
to,
or
it
has
seen
in
the
network
in
the
past.
So
this
score
function
is
important
to
highlight
that
the
score
that
I
am
keeping
for
all
the
other
nodes
I
know
in
the
network
is
not
shared
with
anyone
else.
So
I'm
not
sharing
this.
A
It's
not
like
a
reputation
system
which
is
very
well
known
from
other
systems
and
from
research
from
research
papers,
because
that
has
got
a
different
set
of
problems,
building
a
correct
reputation
system.
It's
it's
pretty
difficult.
So
what
we
said
was
that
every
node
is
going
to
keep
a
score
for
every
other
pair.
It
is
connected
to
this
pair.
This
score
is
kept
locally
and
the
final
goal
of
the
the
score
is
that
if
I
see
a
node
that
is
scoring
very
very
highly
then
I
will
keep
connecting.
A
A
A
We
have
a
message:
delivery
rate
where
we
monitor
how
many
messages
we're
receiving
from
other
nodes,
and
if
this
is
within
some
known
limit
that
the
peer
has
seen
in
the
past,
then
this
means
that
basically,
the
other
node
is
not
hiding
messages.
It's
not
dropping
messages
off
when
it's
receiving
and
tries
to
eclipse
my
my
node.
A
There
is
message
validity.
If
I
receive
a
message-
and
I
see
that
you
know
I
try
to
do
all
the
cryptographic
proof,
like
all
the
check,
all
the
signatures
and
everything
behind
it,
and
I
say
that
it's
not
a
valid
message,
perhaps
even
because
it's
in
the
wrong
topic,
it's
it
talks
about
a
topic
that
is
not
known
to
the
network.
Then
this
is
as
accounts
as
an
invalid
message
and
is
dropped,
and
the
peer
that
has
that
has
sent
this
message
is
is
penalized
with
less
with
less
score.
A
We
have
the
uptime.
So
for
how
long
have
we
been
seeing
this
node
around?
Because
you
know
the
more
a
node
is
staying
in
the
network?
It
means
that
you
know
it's
unlikely,
or
at
least
the
probability
of
it
being
a
malicious
node
is
going
down
normally.
What
my
issues
now
would
do
would
join
the
network
and
attack
the
network
straight
away,
and
finally,
we
have
the
collocation
factor,
which
is
monitoring.
Basically,
the
iep
addresses
from
where
other
nodes
are
connecting.
A
What
we
want
to
achieve
is
that
we
don't
want
many
civil
identities
that
come
from
a
virtual
environment.
Perhaps
you
know
connecting
through
the
same
connection
and
trying
you
know
creating
many
identities
and
attacking
the
network
from
a
single
host,
so
that's
very
common
in
civil
attacks.
So
if
we
see
this
collocation
factor
going,
if
we
see
a
node
that
is
collocated
with
many
other
nodes,
then
you
know
we
score
this
sphere
negatively.
A
This
is
what
it
looks
like.
The
score
function
is
weighted,
some
of
course,
every
different.
It's
very
interesting
to
see
how
each
one
of
those
factors
is
behaving
over
time.
Yeah
and,
as
I
said,
it's
a
weighted
factor,
it's
a
weighted
sum
now.
Finally,
let's
go
to
the
mitigation
strategies
which,
right
now
I
mean
okay.
These
are
perhaps
examples.
A
A
We
have
adaptive
gossip
dissemination,
which
is
as
I
described
beforehand.
This
is
based
on
two
on
two
main
factors.
One
is
the
gossip
factor
and
the
percentage
that
a
node
is
sending
gossip
to,
and
secondly,
is
the
number
of
rounds.
So
this
the
the
gossip,
the
adaptive
because
of
dissemination,
is
targeting
the
first,
so
the
gossip
factor,
as
I
said
in
in
the
current
cosmos
up,
I
think
it's
25.
A
So
every
node
is
picking
25
of
nodes
now,
basically
multiplying
the
25
or
the
remaining
75
times
three.
We
see
that
the
majority
of
the
nodes
in
the
network
are
going
to
hear
about
some
message
and
finally,
there
is
the
buckle
from
prune.
Oh
only
three
minutes.
Okay,
there
is
the
the
pacophone
on
prune,
which
means
that
if
I
prune
some
other
node,
then
I'm
not
going
to
accept
this
node
again
for
some
set
amount
of
time.
A
So
this
is
basically
saying
that
a
sible
node
that
is
going
to
start
grafting
and
trying
to
connect
many
nodes
once
it
is
pruned
from
some
node
because
of
the
negative
score
value,
for
example,
is
not
going
to
be
accepted
again,
so
we
used
test
ground
and
we
had.
We
carried
out
a
huge
campaign
of
testing
on
aws
we
and
we
compared
the
the
protocol
gossip
sub,
with
the
with
bitcoin
and
with
ethereum.
Now
bitcoin
is
keeping
133
connections
in
total,
inbound
and
outbound
and
ethereum
is
using
yeah.
A
Sorry,
I
should
mention
that
bitcoin
is
using
a
flooding
protocol
that
is
propagating
messages
to
all
of
those
connections
and
therefore
it's
on
the
side
that
introduces
lots
of
traffic
to
the
network.
On
the
other
hand,
ethereum
is
using
a
propagation
protocol
that
is
ethereum.
One
is
using
a
propagation
protocol
that
is
forwarding
messages
to
the
square
root
of
the
number
of
nodes
in
the
network.
So
in
this
case
we
set
it
to
33.,
that's
the
square
root
of
the
number
of
nodes.
A
We
use
test
ground,
which
others
have
talked
about
just
around
here
before
and
let
me
go
very
quickly
through
a
couple
of
results
only
so
this
is
the
network-wide
eclipse
attack
where
we
have
a
very
big
number
of
nodes.
I
think
the
ratio
is
20,
sybil
nodes
to
one
honest,
node
and
they're
joining
the
network
and
they're
trying
to
eclipse
it.
A
So
what
we
see
is
here
that
the
ethereum
and
bitcoin
protocols,
even
with
the
publishing
they
are
severely
affected
and
it
takes
up
to
11
seconds,
to
ethereum,
to
propagate
those
messages
and
up
to
four
plus
seconds
to
bitcoin
to
propagate
the
messages.
On
the
other
hand,
gossip
sub
is
staying
really
immune,
and
you
know
this
is
basically
the
it's
not
even
connecting
to
those
civil
nodes.
So
the
message
propagation
is
staying
as
if
nothing
happens,
really
noon.
A
Now.
What
is
interesting
to
see
here
is
the
the
mesh
status
that
we
have
here.
So
here
the
blue
line
is
the
mess,
is
the
mesh.
Sorry
is
the
is
the
honest
nodes
and
the
orange
line
the
dotted
line
is
the
attacker
appears.
So
we
see
that
the
mesh
is
formed
when
the
when
the
sibil
nodes
join
the
network
and
then,
after
a
while,
after
a
little
while,
when
the
onus
nodes
start
joining
the
network,
they
form
smaller
machines.
We
see
that
this
attack
starts
just
below
that
10
19.
A
And
I
know
I've
run
over
time.
I
need
to
wrap
up
yep
time
for
us
to
introduce
the
next
panel.
So
sorry,
okay,
yeah,
that
was
it
sorry.
I
ran
over
time.
We've
got
no
time
for
questions
but
post
them
and
we'll
be
sure
to
to
to
answer
them
yeah
this.
This
concludes
the
talk.
It's
a
very
interesting
protocol
go
check
it
out.
We
have
a
technical
report.
We've
got
lots
of
things
related
to
to
the
performance,
evaluation
and
yeah.
It's
really
interesting.