►
From YouTube: Consensus Scaling and Sharding for Filecoin
Description
In this talk, we will first cover the general challenges for blockchain scaling related to its security and decentralization. This will help us motivate a hierarchical approach to scaling in Filecoin, which in turn, we would like to pursue in an attempt to make Web3 scale to Web2 workloads. Finally, in this context, we outline steps which PL and ConsensusLab are taking towards making Filecoin scale.
A
Hello,
everyone-
and
thanks
for
the
introduction-
I
hope
you
hear
me
well
because
I'm
really
joining
from
east
africa.
It's
past
midnight
here
and
I
found
a
really
nice
location
here
by
the
pool
where
you
hopefully
cannot
hear
the
party
so
yeah
with
that.
Let's
start
with
discussing
how
do
we
scale
filecoin?
A
A
A
Bitcoin
has
is
limited
to
seven
transactions
per
second
and
ethereum
to
15
transactions
per
second,
and
the
point
here
is
not
depends
on
your
goals
right,
so
choosing
the
right
consensus
protocol
is
important
depending
on
your
use
case,
but
any
consensus
protocol
that
you
choose.
So
you
know,
even
if
you
come
with
a
less
secure
protocol
than
proof
of
work,
but
which
scales
better.
A
It's
going
to
still
be
limited
with
the
single
machine,
single
validator
specification,
because,
if
you're
running
a
consensus
protocol,
this
means
that
you're
ordering
transactions
and
then
you're
doing
with
transactions
on
each
and
every
machine
you're
doing
something
like
the
same
thing
on
each
and
every
machine.
So
whatever
is
your
processor's
protocol
depending
on
your?
What
is
your
use
case?
A
So
we
need
to
basically
support
execution
of
arbitrary
programs
over
the
blockchain,
so
in
that
case,
even
if
you
use
consensus
protocol
to
order
transactions,
what
you're
doing
later
with
them
is
really
important.
So
currently
smart
contracts
supporting
blockchains,
they
execute
sequentially
all
transactions
after
ordering,
and
that's
going
to
be
your
another
bottleneck
in
your
system
right.
So,
if
you're
not
able
to
essentially
remove
the
scalability
bottleneck
in
the
execution
you're
going
to
face
problems
regardless
of
your
consensus
protocol.
A
So
what
are
the
essentially?
The
scalability
goes
for
filecoin.
So
when
I
joined
the
protocol
labs-
and
when
was
when
I
was
thinking
like
before,
I
essentially
established
consensus
lab.
I
was
thinking.
Okay,
let's,
let's
take
the
best
consensus
protocol.
There
is,
let's
put
it
in
file
coin
problem
solved,
but
then
juan
juan
basically
came
with
this
really
really
nice
requirements,
which
is,
let's
get
all
the
entire
web
2
workloads
to
web
3.,
and
this
is
the
basically
the
setting
in
which
we
are
working.
A
So
this
is
our
running
use
case,
which
is
extremely
ambitious,
and
but
it
also
tells
you
something
like
it
drives
the
design
of
consensus
scaling
in
certain
directions.
So
I
won't
read
out
all
the
requirements
that
you
see
on
this
slide,
but
basically
you
shouldn't
sacrifice
like
the
latency
for
in
certain
use.
A
Cases
should
be
very,
very
small,
which
is
completely
opposite
from
what
you
want
in
throughput,
which
you
scale
to
you,
know
the
billions
or
trillions
transactions
per
second,
but
you
should
get
as
much
security
as
you
as
you
can
so,
for
example,
as
we
see
today
with
bitcoin
proof
of
work,
the
fact
that
it
uses
more
power
than
I
don't
know
poland,
which
just
means
that
poland,
you
know
if
it
engages
all
the
power
available
to
them.
It's
not
able
to
attack
bitcoin.
A
So
we
should
be
leveraging
this
security
right.
So
we
should
like
I'm
really
not
against
bitcoin
spending
a
lot
of
energy,
but
we
shouldn't
be
spending
yet
more
energy.
You
know
compared
to
bitcoin,
but
we
should
kind
of
understand
how
to
use
such
a
secure
effort
to
secure
file
point,
as
you
will
see,
because
scalability
involves
trade-offs
when
it
comes
to
security
and
decentralization.
This
is
one
of
the
design
approaches
that
we
are
taking
here
so,
for
example,
leveraging
security
of
other
networks,
notably
bitfire.
A
So
how
do
we
get
to
these
bringing
web
to
world
we're
close
to
web3?
So
one
thing
is
that
what
I
just
mentioned
so
you're
not
going
to
be
able
to
run
basically
because
of
a
single
machine.
Scalability
limits,
you're
not
going
to
be
running
a
single
blockchain
that
basically
runs
the
same
code
on
all
machines.
So
that's
not
going
to
fly.
A
So
what
we
are
imagining
is
a
hierarchical
consensus
approach,
whereas
you
see
here
in
green,
basically
the
green
line
here,
should
you
should
you
could
think
of
it
as
the
current
file
coin
main
chain,
and
we
can
maybe
down
the
road
change.
The
consensus
protocol
deal
to
scale
even
better,
but
even
if
you,
if
you
keep
current
file
coin,
expected
processors,
how
we
are
going
to
actually
do
the
check
do
do
do
scalability
is
by
sharding,
but
not
in
a
classical
way,
but
by
establishing
hierarchies
of
consensus
protocol.
A
So
essentially,
when
we
spawn
here
in
this
picture,
blue
and
later
yellow
shards,
they
are
going
to
leverage
the
security
of
the
basically
their
parent
charts,
and
you
see
the
check
pointing
here
shard,
which
is
an
external
one,
and
I
will
talk
a
bit
more
about
this
here.
We
plan
to
use
the
blockchain,
basically
the
consensus
protocols,
with
even
more
security,
such
as
bitcoin's
proof
of
work
to
to
essentially
increase
the
security
of
the
file
coin
network.
A
Of
course,
when
you
move
to
from
a
single
shard
to
multi-shard
space,
you
have
the
the
main
challenge
is,
of
course,
the
security
of
individual
shards,
but
also
what
we
do
with
crosshard
transaction
semantics.
So
this
is
one
of
the
things
we're
looking
at
and
consensus.
Basically,
the
second
pillar
of
the
problems
that
are
looking
at
in
consensus
lab
is
okay.
What
do
you
do
now
within
us
within
a
specific
shard?
A
So
that
depends
basically
in
you
know,
in
a
shard,
you
will
have
a
different
number
of
nodes
depending
on
whether
this
is
a
file
coil
may
chain
which
currently
has
like
3.5.
Last
time
I
checked
for
fuel
fox.
I
saw
3.5
000
miners
and
this
will
increase
over
the
time,
but
you
can
imagine
that
in
the
child
shards
we
will
have
less
and
less
nodes,
and
this
will
allow
us
to
actually
put
very
efficient
protocols
in
this
child
chart.
A
And
finally,
this
is
extending
basically
what
raul
was
talking
today
on
file
point
vm,
but
basically
it
also
relates
to
what
I
said
two
slides
ago.
If
you
have
this
scalable
sharding
solution
and
if
you
have
really
really
efficient
cross
sensors
protocols
inside
the
shard,
it
still
doesn't
mean
much
if
you're
going
to
be
bottlenecked
by
by
sequential
execution.
So
what
we're
looking
at
the
at
the
consensus
lab
is.
A
Essentially,
we
are
interfacing
with
file
point
vm
team,
where
they
are,
of
course
bringing
the
falcon
vm
and
supporting
wasn't
an
evm
on
top
of
wasm
and
whatever
was
talking
about
today
to
basically
parallelize
this
computation.
So
this
this
this
is
one
of
the
tasks
that
we're
looking
into
in
consensus
lab.
So
I
would
say
if
fvm
team
is
looking
into
to
a
certain
extent
in
this
third
pillar.
A
We
are
also
collaborating
with
them
on
that
and
our
focus
is
basically
on
these
two
pillars:
consensus
hierarchies
and
consensus
proper.
So
this
is
the
focus
of
the
of
this
newly
formed
team.
But
how
do
we
do
that?
I
mean
many
teams
are
looking
into
this
and,
of
course,
it's
easy
to
say
yeah.
Of
course,
we
are
going
to
scale
and
share
the
file
point.
But
how
do
you
do
that?
I
mean,
as
we
know,
the
key
problem
when
you
do
scaling
or
sharding
is
that
you
start
losing
security
and
decentralization.
A
So
this
is
the.
This
is
the
thing
right,
so
it's
by
now
we
have
since
bitcoin
like
well
12
years
of
research
and
we
still
like
if
we
have
a
consensus
protocol,
such
as
proof
of
stake.
Yes,
it
has
better
performance
than
proof
of
work,
but
you're
sacrificing
a
lot
of
security
properties
right.
So
one
of
the
challenges-
and
I
will
illustrate
this
on
a
very
simple
variant
of
proof
of
stake,
but
it
also
applies
to
proof
of
space-based
protocols
such
as
filecoin
expected
consensus.
A
This
is
zero
three
and
the
no4,
a
and
d
transfer
their
stake
to
g
and
h.
Right
and
now
you
see
that
this
con.
This
completely
changes
the
configuration
of
of
a
proof
of
stake
system
or
just
any
byzantine
fault-
tolerance
system.
I
have
here
in
the
picture
bft
system
which
stands
for
byzantine
fault,
tolerance,
which
is
actually
actually
you
can
see.
It
is
a
variant
of
proof
of
stake.
A
If
the
client
knows
only
the
initial
configuration
when
going
to
the
picture
down
when
it
starts
to
basically
talk
to
the
blockchain
configuration
consisting
of
a
b
abc,
indeed
might
not
know
about
efgh
at
all.
Basically
it
can
be
forked,
because
the
these,
a
b
c
and
d
have
no
stake
in
the
system
anymore.
So,
basically,
if
they
keep
cryptographic
keys
that
they
use
to
construct
the
blocks
before
and
basically
in
any
rational
model,
they
will
be
able
to
do
that.
They
can
fork
the
client.
A
So
this
doesn't
I
mean
this
is
something
that
this
is
a
very,
very
basic
attack
on
proof
of
stake,
which
doesn't
appear
in
proof
of
work,
because
in
proof
of
work
you
can
show
basically
the
longest
chain
to
the
client
and
client
will
immediately
switch
there.
But
here
there
is
no
way
essentially
that,
like
the
right,
the
current
configuration
egfh
can
convince
the
client
that
they're
the
right
essentially
configuration
as
compared
to
a
b
c
and
d,
and
this
is
the
problem.
The
proof
of
stake
alone
cannot
solve
right.
A
So
what
we
are
doing-
and
it
applies
also
in
to
to
proof
of
space
right
and
it
could
apply
in
theory
to
current
file.
Fine,
what
we
are
doing
there
is
basically
what
I
was
intuiting
so
far,
so
to
prevent
this
kind
of
long-range
attack
and
to
essentially
have
more
security
in
the
falcon
in
the
main
chain
of
the
falcon
network.
We
are
working
on
anchoring
and
checkpointing
membership
and
essentially
the
state
of
the
falcon
network
into
the
bitcoin
network.
A
If
you
want
inside
the
bitcoin
transaction,
and
then
we
are
putting,
of
course,
the
cid
with
actual
state
to
the
falcon
network
to
resolve
it
right.
That's
basically
checkpoint
in
our
so
reusing
other
consensus
protocols
to
actually
give
more
security
to
falcoid.
But
then,
of
course,
we
need
to
shard
right.
So
coming
to
the
picture
that
I
showed
previously,
all
these
child
shards
are
essentially
going
to
provide
like
less
security
than
the
main
chain,
but
higher
performance
and
they're.
A
Essentially,
you
can
think
of
them
as
a
variant
of
limited
liability
companies,
so
think
of
them
as
a
limited
liability
chase
in
which
essentially,
security
for
a
child.
Child
is
firewalled.
In
some
sense,
so
if
there
is
a
security
violation,
the
child
shall
it's
restricted
to
that
charge,
so
it
doesn't
affect
the
main
chain
at
all.
So
once
we
start
rolling
this
out,
you
know
if
you're
like
okay,
what
the?
What
are
these
guys
doing
I
mean
I
I
don't
want
to
you
know,
endanger
my
my
stake,
which
I
care
for
the
falco
network.
A
This
works,
I
mean
you're
safe,
but
you
don't
need
to
worry
about
that.
Only
if
you
explicitly
essentially
have
an
account
on
a
child,
charlie
will
be
affected
and
then
you
can
profit
from
better
performance
or
whatever
we
are
going
to
put
out
there
depending
on
how
much
time
I
have.
Let
me
just
quickly
check
yeah.
I
probably
don't
have
too
much
time
to
go
into
details,
but
in
this
slide
we
are
talking
about
essentially
details
how
we
checkpoint
into
bitcoin.
A
I
think
I
mentioned
the
the
main
principle
already,
so
we
are
putting
the
content
id
hash,
which
captures
the
entire
state
of
the
filepoint
network
inside
the
op
return,
opcode
of
a
bitcoin
transaction
and
the
address
from
which
funds
move
on
bitcoin
are
essentially
controlled
by
the
old
configuration
of
you
know:
power,
table
storage
power,
table
of
the
old
configuration
and
we're
moving
to
the
address
which
is
controlled
by
the
new
configuration
depending
on
you
know,
miners
joining
network
and
so
on.
So
this
is
the
main
principle.
A
Of
course,
it
will
take
me
a
lot
more
time
to
explain
the
the
the
functioning
here
and
for
sharding,
I
would
just
say
one
slide.
So
what
we
are?
We
are
working
on
udeco
codebase,
so
this
is
soon
to
be
public
and
uniquely
the
lotus
client
fork,
which
differs
from
lotus,
just
in
a
sense
that
it
has
modularized
consensus.
So,
instead
of
just
expected
consensus,
which
is
kind
of
entangled
with
the
other
parts
of
flotus
code,
we
now
have
this
modularized
and
currently
it
is
going
to
support
different
consensus.
A
Protocols
currently
supports,
like
simple
consensus
protocols
such
as
proof
of
work
and
delegated
consensus.
Delegating
consensus
is
where
you
just
give
to
one
validator
the
ability
to
to
you
know
mine
blocks.
So
this
is
not
a
real
fault
or
consensus
protocol,
but
it
serve
us
to
essentially
start
playing
and
start
prototyping
on
the
unique
network.
Of
course,
proof
of
work
won't
be
there.
It's
just
a
placeholder
for
more
efficient
protocols
that
we
are
going
to
do
in
this
consensus,
proper
pillar
that
I
mentioned.
So
to
do
this.
A
We
are
adding
another
actor
to
to
essentially
beautiful
lotus
eureka,
and
this
is
called
the
shard
vector
which
controls
spawning
of
the
shard
joining
leaving
the
shard
and
so
on.
What
is
the
plan
for
2022
just
quickly,
so
we
are
at
the
currently
the
sharding
and
check
pointing
into
bitcoin
poc
stage.
We
expect
first
poc
to
be
finalized
the
q1
next
year
and
then
we
are
going
to
also
so
we
interface
with
filecoin
vmt.
A
Currently
sharding
is
looking
only
into
current
filecoin
networks.
It's
not
looking
into
filecoin
vm,
but
then
once
filecoin
vm
is
ready.
As
roe
was
saying
earlier
today,
then
we
are
basically
merging
this
work
with
falcon
vm
and
then
we
are
continuing
on
supporting
sharding
still
on
poc
level,
basically
in
q2
and
and
after
that,
this
is
again
not
an
announcement
of
of
our
product
feature,
but
we
are
aiming
to
have
by
the
end
of
2022,
progressive
rollout
of
certain
features
and
certain
functionalities
that
they
just
described
into
filecoin.
A
As
for
consensus
proper,
we
are,
you
know,
going
to
put
very,
very
efficient,
scalable
consensus
protocol
into
child
shards.
This
should
be
also
targeting
like
first
rollout
should
be
still
in
2022
and
yeah.
I
think
you
should
have
listened
to
raul
stock
on
falcon
vm,
which
is
related
to
this
right,
just
to
wrap
up
consensuslab.
It
was
just
established
in
q3.
So,
basically,
three
months
ago,
I
joined,
and
we
are
I'm
very
happy
how
fast
we
grow.
A
So
we
have
an
amazing
team
which
currently
has
sarah
as
a
research
scientist
alfonso
as
a
research
engineer,
george
is
our
tpm
and
we
have
two
external
advisors
just
to
you
know
to
try
to
grow
faster.
We
are
relying
here
on
external
advisors
who
are
helping
us
a
lot
with
the
parallel
scalable
execution
and
hierarchical
consensus,
and
we
have
another
research
scientist
who
is
going
to
be
responsible
for
consensus,
proper
joining
us
in
total
early,
basically
q1.
A
If
you
have,
if
you're
you're
listening
to
this
you're
yourself
interested
in
this
or
you
know,
people
who
might
be
interesting,
we
are
hiring
and
please
contact
us.
I
think
you
know
how
we
are
also
not
working
alone
in
the
consensus
lab
right,
so
one
of
our
goals
is
to
really
leverage
the
entire
research
ecosystem
around
the
top
researchers
in
consensus
and
sharding.
On
all
the
topics
I
mentioned,
and
to
this
end
we
had
a
two-day
conference
just
two
weeks
ago,
where
basically,
some
of
the
participants
were
recognized.
A
A
You
know
get
this
larger
research
community
interested
in
the
problems
that
we
are
looking
at
and
hence
have
the
leverage
right
to
to
get
the
most
incentives
to
look
at
that,
and
I
think
this
is
also
a
large
set
of
interesting
problems
and
basically
for
them
to
help
us
make
that
free
scale.
If
you
want.