►
From YouTube: Graph Day: Sharding with Alex Skidanov
Description
Alex Skidanov, the CEO at Near Protocol, gave a presentation entitled "Sharding" at Graph Day in San Francisco on January 25th, 2019. Alex discusses the frontier of sharding research in this technical talk, and how Near compares to Zilliqa and Ethereum's sharding implementation.
Follow The Graph on social media
Twitter: https://twitter.com/graphprotocol?s=20
Instagram: https://instagram.com/graphprotocol
LinkedIn: https://www.linkedin.com/company/thegraph/
GitHub: https://github.com/graphprotocol
Website: https://thegraph.com
#web3 #dapps #thegraph
A
And
let's
talk
about
sharding
before
we
start,
let
me
quickly
introduce
myself.
I
was
I'm
a
co-founder
of
near
protocol,
which
held
briefly
introduced
in
a
second
before
near
protocol.
I
was
working
for
a
company
called
mem,
sequel
and
mem
sequel
was
a
short
database.
I
was
in
play
number
one
at
home,
sequel
and
I
built
most
of
the
early
sharding,
and
then
I
was
driving
charging
when
it
became
more
mature
and
the
form
of
sequel
I
was
at
Microsoft.
That
was
a
decade
ago,
and
we
started
near
protocol
around
August
last
year.
A
It's
one
of
the
younger
protocols
and
as
a
shortened
blockchain,
we
have
a
huge
emphasis
on
usability.
We
invest
heavily
into
developer
experience
into
user
experience
with
the
blockchain.
How
they
can
onboard
easier
and
near
protocol
is
one
of
the
few
teams
that
has
a
very
strong
team.
With
specifically
with
the
industry
backgrounds.
We
have
three
people.
Four,
we
have
four
people
from
one
sequel.
We
have
many
eggs
Googlers.
We
have
many
people
who
have
gold
medals
from
ACMI
CPC,
which
was
a
competitive
programming
and
one
of
the.
A
One
of
the
key
things
worked
near
protocol
is
that
we
are
very
open
with
the
community
and
so
we're
not
when
we're
not
behind.
Like
a
walled
garden,
we
work
closely
with
other
protocols.
We
work
closely
with
the
theory
on
foundation
with
cosmos
and
others,
and
so
the
everything
that
happens
in
the
sharding
research.
We
share
a
lot
of
that
bit
between
us
and
so
in
this
presentation.
I
will
I
will
show
you
what
charting
is
how
charting
works
and
what
sort
of
problems
still
exist
in
charting.
A
So,
first
of
all,
what
charting
is
right.
So
let's
say
we
have
a
blockchain,
so
that
could
be
a
theorem
or
block
chain
or
a
Bitcoin
or
or
any
blockchain,
and
that
and
that
blockchain
has
some
number
of
transactions
per.
Second,
it
can
process.
So
let's
say
it's
12
and
let's
say
12
is
not
enough.
Let's
say
we
want
more.
Maybe
we
want.
Let's
say
we
want
such
as
six,
so
one
thing
we
can
do
is
we
can
run
three
block
chains
if
each
of
them
can
twelve
transactions
that
gives
us
36.
A
If
we
want
to
have
thousand
transactions
per
second,
we
can
run
hundred
blockchains
and
if
we
want
to
have
ten
thousand
transactions
per
second,
we
can
we
can
run
1000,
G
blockchains
and
that's
it.
We
thought
we
thought
the
scalability
problem.
Soft
there's
a
small
problem,
the
slight
problem,
which
is
when
we
have
one
blockchain
and
there's
a
some
population
of
people
who
who
are
willing
to
validate
that
blockchain
and
produced
blocks.
If
you
want
to
compromise
the
blockchain,
you
need
to
corrupt
or
control
51%
of
them.
A
Once
you
have
multiple
block
chains,
those
people
they
were
when
this
blockchain
had
12
transactions
per
second,
there
was
some
limitation.
There
was
some
reason
why
the
blockchain
couldn't
have
processed
more.
Maybe
maybe
the
network
was
at
capacity.
Maybe
processing
was
at
capacity,
maybe
storage,
but
there
was
some
reason
why
the
equipment
process
more.
So
when
we
have,
you
know
ten,
ten
ten
block
chains,
we
cannot
expect
all
validators
validate
all
the
block
chains
because
they
once
they
start
validating
one.
A
They
are
at
capacity,
and
that
means
that
validators
are
split
between
them
and
now
he
want
to
corrupt
one
blockchain.
You
only
need
to
corrupt
sufficient
number
51
percent
of
the
people
who
validate
it,
which
is
for
if
you
have
ten
block
chains
that
becomes
five
percent
of
people.
If
you
have
thousand
block
chains,
you
need
to
corrupt
point,
zero
five
percent
of
all
the
validators,
which
is,
but
that
doesn't
sound
very
hard,
but
that
would
only
work
if
you
can
actually
control.
A
So,
if
you,
let's
say
let's
say
it's
a
half
power
that
is,
that
is
deciding
the
transaction
or
at
stake
for
you
to
be
able
to
corrupt
one
one
of
the
chains.
You
either
have
to
be
able
to
control
where
your
hash
power
goes
to
which
shirt
or
where
your
stake
goes.
Or
you
have
to
be
able
to
to
identify
people
in
a
particular
shirt
and
some
somehow
corrupt
them,
you'd
either
bribe
them
or
hack
them
in
some
way,
and
so
the
most
common
solution
to
this
problem
is
to
rotate
the
validators.
A
We
will
assign
validators
to
shards,
so
let's
say
every
short
has
100
validators.
They
will
produce
some
few
blocks
and
then
we
will
completely
shuffle
them,
and
so
now,
first
of
all,
I
don't
have
control.
Where
am
I
power
goes,
but
also
I
can
actually
corrupt
someone
unless
I
can
corrupt,
very
fast
right,
so
unless
I
can
corrupt
them
during
during
that
timeframe,
well,
the
validating
the
shirt
by
the
time
we
corrupt
them.
A
They
already
got
you
shuffled
and
at
that
point
they
belong
to
different
shirts,
and
so
the
idea
here
is
that
if
we
believe
that
the
total
population
has
less
than
25
percent
that
same
malicious
actors,
then
once
we
sample-
let's
say
100
of
them,
we
can.
We
can
do
some
mathematical
modeling
and
show
that
the
number
of
malicious
actors,
the
probability
of
having
let's
say
33%
of
malicious
actors
in
the
chart,
is
very
low
right.
So
you
know
like
let's
say
the
pool
population
is
10,000
and
you
sample
100.
A
If
you
know
there's
25%
here
at
most,
then
the
probability
that
you
have
33
malicious
actors,
33%
malicious
actors
here-
is
very
low.
It
can
be
considered
to
be
zero,
and
so
once
that
is
the
case,
let's
say
this.
Is
you
and
you
want
to
know
what
this
delete
is?
Look
in
the
blockchain?
You
go
to
the
current
validators
and
you
ask
them
what
is
the
current?
What
is
the
latest
look
and
if
you
hear
the
response
from
two-thirds
of
them
and
you
use
some
sort
of
DFT
consensus
and
that
response
is
the
same?
A
A
That
is,
that
should
be
secure
computation.
That
computation
should
happen
on
chain,
but
we
don't
have
a
chain.
We
have
multiple
chains
which
which
chain
is
responsible
for
actually
rotating
the
validators
and
the
most
common
solution
to
that
problem
is
to
have
the
chain
to
have
one
block
chain
to
rule
them
all
and
in
the
theorem
it
is
called
the
beacon
in
polka-dotted
calculation.
We
will
be
referring
to
this
main
chain
throughout
the
presentation
and
so
that
chain
everybody
validates
it
right.
A
But
it's
forking,
the
only
malicious
behavior
that
can
be
exercised
as
a
matter
of
fact
know
if
you
corrupt
a
shark
in
some
way
you
can
fork,
but
you
can
also
do
another
attack
called
invalid
state
transition.
What
invalid
state
transition
is
is,
let's
say
we
have
a
transaction
in
which
Alice
sends
Bob
10
tokens
and
we
have
the
state
in
which
Ellis
has
done.
10
tokens
and
Bob
has
0
tokens.
A
If
you
get
that
state
and
apply
this
transaction,
the
state
you're
supposed
to
get
is
the
state
on
which
Alice
has
0
tokens
and
Bob
has
ten,
and
so
any
block
which
apply
state,
and
this
way
is
valid.
What
can
happen?
Is
there
could
be
a
block
produced
in
which
you
know
in
initial
State
Alice
had
ten
tokens,
the
transaction
was
applied,
and
now
bob
has
thousand.
That
is
an
invalid
state
transition,
and
you
probably
overheard
about
fourteen
it
is.
A
He
recently
had
a
fork
but
you're
unlikely
to
to
have
heard
about
invalid
state
transition
attacks.
That
is
because
in
non
Charlotte,
blockchains
invalid
state
transition
is
impossible.
The
reason
for
that
is
imagine
that
we
actually
have
majority
of
hair
power
or
majority
of
state
belonging
to
malicious
actors
and
at
some
point
they
produced
an
invalid
block.
So
this
block
a
prime
contains
some
transaction
which
was
applied
appropriately
in
any
modern
blockchain
like
a
theorem
of
Bitcoin.
All
the
validators
are
validating
all
the
blocks
and
all
the
users
are
validating
of
the
block.
A
So
if
you're
using
Bitcoin,
you
would
be
downloading
the
entire
ledger
and
validating
every
single
block,
and
so
the
moment
gives
you
the
a
prime
you
will
see
it
as
invalid.
You
will
discard
it,
and
so
you
will
not
even
consider
B
prime
C,
prime
or
D
prime
as
valid
blogs,
because
you
know
they
built
on
top
of
invalid
State,
and
so
even
though
the
honest
actors
built
a
shorter
chain,
every
single
client
will
know
that.
That
chain
is
correct,
however,
in
the
Charlotte
block
chain.
A
That
problem
does
exist
and
to
understand
why
we
first
need
to
talk
a
little
bit
about
crow
short
communication
right
in
the
in
the
Charlotte
block
chain.
If
we
have
three
different
charts,
we
different
block
chains-
and,
let's
just
say
for
now
that
the
only
kind
of
transactions
that
exists
is
payment
transactions
V,
let's
say
I
have
an
account
in
the
first
chart
and
the
person
I'm
sending
money
to
has
account
on
the
second
shirt.
A
There
is
nobody
who
validates
both
chains,
and
so
the
transaction
cannot
be
in
the
first
block,
because
nobody,
because
because
then
the
validator
is
gonna
change.
The
state
of
the
second
of
the
destination
account
and
the
transactions
can.
The
transaction
cannot
really
be
on
the
second
chain,
because
then
the
validator
cannot
change
the
state
of
the
of
the
sender,
and
so
what
needs
to
happen
is
there
is
there's
got
to
be
some
way
of
change
to
communicate
and
the
way
it's
usually
done
is
to
this
concept
of
receipts.
A
Where,
with
the
transaction
that
touches
multiple
shirts,
it
originates
in
one
of
the
shirts,
for
example,
in
a
Chardon
which,
let's
say
Elvis
is
sending
money
to
Bob.
The
transaction
will
originate
and
the
Alice's
chart
it
will
debit
her
account,
and
then
the
validators
of
this
chart
will
send
some
sort
of
receipt
to
the
second
shirt
with
signatures
confirming
that
the
transaction
was
correct
and
then,
in
the
second
chart
the
validators
will
verify
the
signatures
and
apply
the
transaction
to
Bob.
A
Now,
with
crucial
transactions
in
place.
What
can
happen
is
if
we
manage
to
corrupt
chart,
one
and
I
can
produce
completely
invalid
block
block
B,
in
which
we
mean
some
tokens
out
of
thin
air
and
then
later
we'll
produce
a
block
here,
which
otherwise
looks
correct.
In
Block
C
no
transition
is
applied
improperly
and
from
that
block
with
sending
those
tokens
which
were
minted
to
another
shirt,
the
second
chart
has
no
way
to
validate
the
entirety
of
the
first
shirt
right.
A
They
maybe
are
able
to
validate
block
C,
but
they
cannot
validate
the
entire
history,
because
you
know
if
there's
thousand
shirts
and
everybody
is
sending
cross
card
transactions,
sure
sure
cannot
validate
all
the
shirts
and
so
current
shirt
shirt
has
no
way
to
to
know
that
the
that
the
block
wasn't
correct.
So
one
way
to
solve
that
was
offered
is:
let's
have
let's
not
allow
cross-court
communication
between
any
shirts.
Let's
say
that
only
sound
shirts
can
communicate
between
each
other
and
the
chard.
A
Will
corrupt
corrupt
chart?
One
produced
invalid
block
sent
the
crossroad
transaction
to
chart
two
which
is
also
owned
by
us,
so
we
will
accept
the
crossroad
transaction
and
we
will
send
it
to
chart
three
short.
Three
can
only
valid
a
chart
which
are
two
is
completely
valid.
There
is
no.
There
is
no
invalid
state
transition
in
the
chart,
so
how
it
is
for
today
well
one
way
or
how
it
can
be
solved
in
the
future.
One
way
is
you
can
pretend
the
problem
does
not
exist.
A
Alternatively,
you
can
you
can
try
to
have
some
cryptographic
proof
that
the
entire
blockchain
is
correct
when
you're
sending
a
cross
card
transaction.
There
is
such
cryptographic
primitives,
they
called
snarks,
but
today
they
not
quite
usable,
they
very
slow,
right,
they're
usable
for
private
transactions,
so
zeeky
zeeky
zeeky
caches
using
them,
but
to
valid
to
prove
that
the
entire
blockchain
is
correct.
That
is
not
that
it's
not
quite
there.
Yet
the
long
term
that
will
be
the
best
solution
possible.
A
The
most
promising
solution
today
is
called
fisherman,
where
the
idea
is
that
for
as
long
as
you
have
at
least
one
honest
validator,
that
honest
validator
would
be
monitoring
with
what
is
happening
in
other
blocks
in
other
shirts
and
if
they
observe
the
distant
crush
our
transaction
happening
from
from
the
shirt
that
they
validating
from
the
state
that
they
they
have
not
observed
before
so
effectively.
The
scenario
will
be
the
charges
completely
overtaken
there's
more
than
two-thirds
of
malicious
actors.
They
produce
an
invalid
blog;
they
never
show
this
blog
to
the
owners
validator.
A
They
produce
a
valid
block
on
top
of
it
otherwise
valid.
Besides
the
fact
that
it
is
approving
an
invalid
block
initiate
crash
or
transaction,
the
owners
validator
will
notice
that
this
is
happening.
Download
the
blocks
identify
that
this
block
is
invalid
and
create
some
proof,
some
some
small
proof
that
that
this
is
happening
and
send
this
proof
to
the
short.
So
now,
at
this
point
sure
we'll
quickly
verify
the
proof.
The
proof
is
very
lightweight
and
know
that
the
crossguard
transaction
that
was
initiated
is
incorrect.
The
problem
is
that
monitoring
the
shard
identifying
this.
A
This
problem
happened
downloading
the
block
validating
it.
It
takes
a
lot
of
time
and
so
the
challenge
period,
the
period
during
which
the
period
that
short
will
be
waiting
after
commercial
transaction
was
initiated
before
they
certain
that
no
challenge
will
arrive
should
be
pretty
long
right.
So
so,
let's
say
both
shards
are
producing
blocks
every
three
seconds
so
cross.
Our
transaction
can
can
really
without
fishermen,
be
finished
in
like
six
seconds
with
the
fisherman.
A
It
needs
to
be
on
the
order
of
minutes,
so
the
system
becomes
significantly
slower,
but
the
bigger
problem
is
that
this
block
be
the
owners
valet
that
actually
shouldn't
load
the
blog
B.
The
problem
is
that
the
dishonest
validator
might
have
never
even
published
it,
because
in
the
short
they
they
cannot
be
done.
Loading
all
the
blocks
from
all
the
shirts
in
these
thousand
shirts-
and
there
is
one
megabyte
block
produced
every
few
seconds
in
every
shirt
you
would
have
to
download
one
gigabyte
of
data.
A
Every
few
seconds
that
is
unfeasible
so
far
cannot
rely
on
downloading
this
block
to
verify
the
transaction,
and
so
this
block
might
not
even
exist
anywhere.
Nobody,
nobody,
nobody,
besides,
the
user
knows
validators
might
have
seen
that
block,
and
so,
in
this
case
the
owners
validator
cannot
download,
it
cannot
validate,
it
cannot
craft
the
challenge
that
problem
is
called
data
availability
problem.
Very
few
protocols
actually
really
work
on
this
problem
right.
A
The
only
protocols
that
that
that
publicly
say
that
they
work
on
this
problem
that
they
believe
it
exists,
is
polka
dot
and
the
etherium
most
of
the
other
protocols.
They
there's
nothing
in
their
technology
which
actually
tries
to
to
go
to
work
with
data
availability
to
work
around
the
data
availability
problem
and
the
yeah,
and
the
problem
is
that
it's
not
only
cross
hard
communication
that
is
broken.
It's
also
snap
fitting
to
the
beacon
chain,
your
only
snap,
fitting
to
the
beacon
chain,
the
half
of
the
latest
block.
A
A
Yeah,
and
so
there
are
some
proposals-
the
usual-
they
mostly
work
around
this
idea
of
using
erasure
codes
and
providing
some
extra
redundancy
in
the
data
that
is
available,
and
then
data
is
somehow
split.
Most
of
those
are
the
very
early
stages,
and
so,
if
you
interested
in
this
particular
topic,
you
can
you
can
search
for
a
paper
by
the
italic
called
route
proofs,
and
let
me
finish
my
presentation
with
quick
overview
of
how
people
actually
build
sharding
today.
Right
so
cosmos
is
one
example
of
something
which
is
very
similar
to
charting
right.
A
They
have
a
Kozma
which
is
which
looks
a
lot
like
a
beacon
chain,
and
they
have
zones
which
look
a
lot
like
shorts.
A
few
things
about
coordinates
is
that
first
validators
do
not
rotate
around
zones
validate,
there's
assigned
to
the
zone,
and
it
reduces
the
security
a
little
bit
and
they
the
way
cause
most
positions.
It
is
that
the
users
of
the
zone
have
for
themselves
their
responsibility
to
identify
how
many
validators
exist
there,
how
much
they
trust
them
and
how
much
they
trust
the
zone
and
then
cosmos.
A
The
cross,
shard
communication
cross
down
communication
is
carried
out
through
a
protocol
called
IBC
inter
blockchain
communication,
which
allows
you
to
move
assets
around
between
the
zones,
but
it
doesn't
really
allow
you
to
do
any
more
complex
code.
Execution
polka-dot
is
somewhat
similar
to
the
cosmos.
A
However,
the
big
difference
is
that
a
the
actual
runtime
of
every
blockchain
is
stored
on
on
the
chain,
so
in
cosmos,
every
blockchain
is
completely
different,
binary
in
polka-dot.
In
principle,
it
can
be
built
in
such
a
way
that
the
parity
of
power
chains
of
the
same
binaries-
they
don't
have
to
be,
but
they
could
and
the
runtime
of
the
power
chain,
is
stored
in
the
own
chain,
meaning
the
validators
in
principle
could
rotate.
A
A
You
don't
need
to
trust
the
two-thirds
of
the
validator
disk
is
our
honest.
It
doesn't
have
to
trust
that
99%,
it's
enough
to
try
that
this
is
atleast
one,
honest,
validator,
/,
/
short
I
silica.
Is
you
probably
heard
of
silicon,
the
concept
of
sharding
silica,
only
the
non
short
state,
the
only
short
processing,
meaning
that
every
node
still
has
to
store
the
entire
state,
which
is
a
huge
limitation
and
also
the
for
cross
card
transactions?
A
They
completely
shut
down
the
entire,
the
completely
shut
down,
shorting
and
process
them
serially
and
the
theorem
serenity
near
they
very
similar,
unfortunately
running
out
of
time,
but
they
effectively
everything
I
described
today.
This
is
effectively
how
a
theory
of
serenity
and
near
work
and
cool,
so
anything's
gear.
A
Quick
thing
is
that
we
have
whiteboard
series
where
we
talk
to
two
founders
of
different
protocols.
We
talk
to
cosmos
to
solana
where
we
talk
a
lot
about
technology,
so
check
it
out
here.
Yes,
this
YouTube
and
also
next
Tuesday
we're
revealing
our
definite.
If
you
in
the
city,
google,
it
it's
or
the
link,
is
nearly
I
slash,
definite
and
fortune.
I,
don't
have
it
on
the
slide.
Thank
you.