►
From YouTube: NEAR Core | Episode 05: NEAR Network Routing
Description
Follow the latest from NEAR Protocol on,
Website: https://nearprotocol.com/
Discord: https://near.ai/discord
Medium: https://near.ai/medium
Twitter: https://near.ai/twitter
GitHub: https://near.ai/github
#WhiteboardSeries #Blockchain #FutureIsNEAR
A
Yeah
so
last
thursday
we
did
a
walkthrough
on
how
peers
connect
to
each
other,
as
well
as
some
of
the
essential
data
structures
we
use
in
the
network
code,
and
today,
marcelo
will
share
with
us
how
the
routine
works
in
network
is
it's
very
complicated
and
I
hope
it's
going
to
be
interesting
to
everyone
so
just
hand
over
to
myself.
B
Okay,
thanks
for
the
introduction
bowling
so
well,
let's
give
a
high
level
overview
about
how
it
works
and
speak
about
how
we
discuss
last
week
right
now,
like
last
week,
we
discussed
about
how
to
peer
connect
to
each
other
and
that
they
keep
edges
between
between
them
whenever
they
either
create
or
or
disconnect
from
each
other.
They
create
an
edge
that
contains
that
that
information
on
the
network
level
for
routing
what
will
be
happening
is
that
these
edges
will
be
propagated
through
the
network.
B
B
Oh,
if
it
isn't,
let
me
know
okay,
so
initially,
I
want
to
discuss
what
what
is
happening
immediately
when
two
peers
are
connecting
to
each
other.
They
will,
they
will
try.
They
will
start
to
exchange
certain
information
which
is
the
status
of
their
routine
table.
What
is
the,
what
is
the
relevant
information
over
the
routine
table?
B
We
will
keep
the
list
of
all
edges
inside
some
graph
data
structure,
which
is
just
a
graph
containing
all
edges,
and
this
this
is
the
information
that
we
will
be
exchanging
with,
with
all
the
peers,
all
the
edges
and
with
the
peer
we
just
connected,
we
will
share
all
the
edges
that
are
in
our
graph
right
now,
and
this
is
done
at
the
moment
in
a
very
naive
way.
We
don't
check
whether
there
is
whether
the
intersection
of
our
edges
is
large
or
small.
B
We
use
share
all
the
edges
right
now,
which
is
fine
for
graph
with
a
low
number
of
nodes
with
small
number
of
nodes,
but
we
will
need
to
improve
this
as
our
network
scales
and
as
our
network
skills.
Okay.
So
there
is
a
another
important
piece
of
information
that
we
will
keep
and
is
this
mapping
between
account
ids,
and
this
call
announce
account
which
basically
contains
the
beer
id
of
each
account.
B
What
what
is
happening
here
is
that
we
will
and
we
will
keep
a
mapping
between
each
validator
id
which
evaluates
your
id,
is
just
an
account
id
and
it's
relevant
and
id
on
the
network
like
at
the
network
level.
We
are
referring
to
each
node
with
appear
id,
which
is
a
unique
key,
and
we
will
have
this
mapping
between
validators
and
gear
ids.
This
will
be
relevant
to
root
information
to
particular
validators.
B
While
they
are
well
well,
they
are
validated
well,
they
are
active
validators
on
some
efforts
because
because
we
need
to
root
like
information
such
as
transactions,.
B
Pieces
to
synchronize
etc.
So
this
these
two,
let
me
share
the
code.
They
think
it's
here
so
here
and
it's
the
part
of
the
code.
B
Oh
sorry,
let
me
just
show
this
and
when
we
are
when
we
connect
to
each
other,
we
automatically
send
this
type
of
message,
which
is
a
sync
message
that
contain
all
that,
contain
all
edges
and
contains
all
accounts
known
to
known
to
us
notice
that
let
us
ins,
we
inspected
the
edge
data
structure
in
the
previous
session,
how
they
are
validated
and
how
we
make
sure
the
information
in
that
edge
is
correct.
B
Let
us
inspect
how
nouns
account
work,
so
it
contains
basically
the
peer
id
and
the
oh.
Oh
no!
No!
It's
not
this
sorry
yeah,
it's
very
weird,
not
having
a
confident
yeah.
This
is
the
announce
account
that
contains
the
account
id
of
the
validator,
the
peer
id
of
the
validator,
the
epoch
for
which
this
validator
is
going
to
validate.
This
is
relevant
so
that
we
like,
we
know
when
this
announcement
expires
and
basically
they
expire
after
the
epoch
passed,
and
we
have
this
signature
and
which
is
basically,
this
appears
signaling.
B
The
full
information
this
is
only
this
will
be
relevant
only
for
peers
that
are
not
necessarily
validated
in
the
network
because
other
like
this
is
this
is
relevant
to
yeah
yeah,
because
everyone,
basically
all
validators.
I
think
all
the
peers
can
verify
if,
if
a
validator
belongs
to
the
epoch
id
this
is
necessarily
to
validate,
and
the
signature
is
just
to
check
that
this
validator
actually
controls
the
spear
id
or
basically
control
control,
display
theory.
B
The
opposite
way.
Sorry
like
this
signature
is
for
er
is
segmented
with
the
key
of
the
validator.
Okay,
because,
like
we
want
to
avoid
that,
anyone
can
claim
any
account
id
okay,
so
yeah.
So
this
information.
Well,
let
me
just
make
a
brief
pause
like
in
the
in
the
other
session.
You
can
ask
any
question
in
the
chat
and
I
will
try
to
address
it
so
feel
free
to
do
it.
I
will
be
monitoring
it
so
well,
here
we
we
receive
when
we
receive
all
edges
and
account.
B
That
is
not
in
the
past
that
it's,
the
current
epoch
or
the
or
the
next
epoch,
and
so
we
can
like
this
is
also
required
for
us
to
be
able
to
validate
it,
and
this
information
needs
to
be
passed
to
the
client
actor
to
be
able
to
be
done
either
decline
or
the
view
client
after
to
be
able
to
validate
it,
since
the
peer
manager
doesn't
have
enough
context
to
do
it.
B
A
It
is
passed
to
the
viewer,
client,
actor.
B
Yeah
yeah
because
it
doesn't
need
to
modify
anything,
it's
just
shaking
okay
and
well.
This
is
the
information
that
is
a
exchange
between
two
peers
when
they
connect
to
each
other,
but
another
process
is
going
in
the
background
that
whenever
a
new
edge
is
added
to
the
network,
this
edge
is
automatically
red
casted.
This
should
probably
be
happening
in
peer
manager
and
register
here,
very
likely.
B
In
the
end,
we
yeah
exactly
we
broadcast
this
message,
which
is
basically
a
routine
table,
update
within
table
sync,
but
it
only
contains
it
only
contains
a
single
edge
okay.
So
this
information
is
broadcasted
to
the
network,
to
let
other
peers
know
that
there
is
that
this
connection
exists
and
the
same
is
happening
with
validators.
A
B
Announcements
check
sense,
announce
account
here;
it
is,
and
in
this
function,
if
when
we
are
validators,
this
function
will
trigger
this
event
that
basically
will
be
broadcast
and
the
information
that
we
will
become
validators
in
the
next
epoch.
So
this
information
is
broadcasted
way
way
ahead
of
will
become
validators
so
that
everyone
have
this
information
in
their
reaching
tables
before
we
are
actually
validators.
B
Okay,
this
is
probably
the
most
relevant
part
about
synchrony
synchronizing
information
between
peers.
Let
us
discuss
about
how
how
how
we'll
be
rooting
messages
so,
with
with
this
information,
whenever
we
receive
any
new
any
new
edge
to
the
network
from
the
network,
we
add
it
to
the
routing
table
which
basically
and
we'll
add
it
to
this
graph.
This
is
the
like
the
raw
and
this
graph
will
contain
the
raw
information,
all
the
edges
and
will
be
used
to
process
short
test
path
to
all
the
nodes.
Since
we
want
to.
B
We
want
the
message
to
travel
the
list
through
the
list
and
to
the
path
with
this
number
of
edges.
So,
let's
see
how
the
graph
works.
Basically,
just
a
store
for
every
like
for
every
every
edge
is
just
a
pair
of
peer
ids
from
the
point
of
view
of
the
graph.
It
doesn't
matter
the
signatures
we
at
this
point.
Those
should
be
verified
already,
so
we
have
this
adjacency
matrix.
B
I
expect
you're
familiar
with
that
for
for
every
peer
we
have
the
set
of
all
peers
which
are
connected
to
to
that
appear.
We
also
keep
source,
which
is
basically
our
pred
and
then
in
the
graph.
We
have
this
function,
important
function,
which
is
calculate
distance,
which
basically
for
every
peer
in
the
network.
It
will
give
us
the
set
and
the
set
of
peers
the
the
set
of
peers
connected
to
us
that
belong
to
the
shortest
path
from
us
to
that
to
that
peer,
okay.
B
So,
for
example,
for
if
the
let's
see,
if
I
have
some
example
drawn
here,
I
think
there
is
some
drawings
in
the
test.
So
here
suppose
in
this
test,
that
s
is
connected
to
zero
one
two,
and
these
are
fully
connected
to
three
four
fives,
and
these
are
fully
connected
to
seven
six.
Seven.
Eight
then,
for
the
set
for
the
eighth
set.
We
say
that
all
the
spears
and
zero
one
and
two
and
belong
to
the
shortest
path
to
eight,
because
there
is
a
path
of
length
three.
B
So
there
is
a
path
from
s
to
number
eight
to
node
eight
of
length,
three
and
they're
the
same
from
through
zero
and
through
one.
So
basically,
we
can
send
any
message
from
s
to
eight
through
any
of
those
note,
and
we
will
well
in
this
case
it's
not
like
all
our
neighbors
work,
but
it
doesn't
necessarily.
B
It
doesn't
need
to
be
necessarily
the
case,
for
example,
for
note
2,
the
only
the
only
peer
that
belongs
to
the
shortest
path
is
the
node
itself,
of
course,
because
any
sending
information
through
other
nodes
will
make
the
path
longer
like
if
we
send
it
through
one.
It
doesn't
make
any
sense,
of
course,
if
we
want
to
send
a
direct
message.
So
basically,
this
function
is
running
bfs.
C
B
B
I
I
wanted
to
find
the
like:
there
is
function.
Probably
this
one
is
better
find
root
from
peer
id
which,
when
we,
when
we
call
this
function,
it
will
try
to
it
will
find
to
what
peer
we
we
should.
It
will
find
one
active
peer
that
we
have
right
now.
That
will
be
the
one
that
will
be
sending
the
message
and
this
peer
will
at
the
same
time,
root
the
message
to
the
target.
B
So
in
this
case
this
is
the
target
and
the
result
will
be
the
the
appear
that
we
will
immediately
that
the
next
hop
in
the
in
the
path.
Okay.
So
basically
there
is
some
some
extra
logic
here,
because
what
what
will
happen
is
that
when
we
receive
new
edges,
we
automatically
will
call
update.
This
is.
C
B
Done
on
every
update,
we
will
try
to
make
some
batches
but
the,
but
to
avoid
the
rooting
table
to
go
out
of
sync,
we
don't
wait
more
than
one
second
to
to
to
update
the
routing
table
to
update
the
the
shortest
path
to
all
other
nodes.
So,
basically,
when
we
call
this
function,
we
can
safely
call
peer
forwarding.
B
This
will
give
us
a
list
of
our
routes
to
which
we
we
can
send
the
message
to,
and
then
we
will
do
something
we'll
do
round
rowing
to
select
what
what
is
the
peer
that
we
have
selected
the
list
to
send
message
through
through
it
and
to
try
to
balance
all
massaging
the
all
the
message
we're
sending,
and
this
is
some
sort
of
wrong
drawing
it's
just
that
since
edges
are
coming
and
going
so
we
are
just
having
like
nonsense
about
how
many
messages
we
have
sent
through
node,
but
these
nons,
for
example,
when
we
add
a
new
edge-
and
we
add
it
with
the
nonce
of
the
lowest
of
the
lowest
nones
about
all
connected
period.
B
We
have
right
now
this
is
used
to
avoid
like
sending
several
measures
through
a
newly
created
edge
and
saturating
it.
We
don't
want
that.
A
B
Any
moment
so,
okay,
this
is
the
the
main
idea
about
how
we
are
selecting
a
peer
to
send
a
appear
yeah
to
send
message
through
the
network
and
well.
These
messages
are
very
particular
type
of
message.
Let
me
show
you:
how
do
they
look
like
these
are
rooted
message
body,
but
we
don't
want
that.
We
want
rooted
message
yeah
this
is
it
the
rooted
message
contained
the
target.
This
is
the
the
target
of
the
message.
B
B
Well,
it
speaks
for
itself
and
we
are
the
the
author
of
this
written
message
will
sign
it,
but
if
the
message
contains
any
invalid
like
if
this
message
is
somehow
invalid,
which
I
don't
think
it
can
be
in
any
other
way
than
the
signature
and
the
one
penalize
is,
the
previous
is
the
like.
If
the
signature
is
invalid,
if
we're
I
receive
a
message
from
the
from
appear
with
an
involved
signature.
The
one
penalized
is,
of
course,
the
previous
hop
in
the
in
the
path,
but
otherwise
this
is
not
checked.
B
The
content
of
this
body
is
not
checking
anyway,
so
when
this
message
arrives
to
the
target,
if
it
contains
some
invalid
information,
the
only
one
that
should
be
penalized,
if,
if
should
analyze
at
all,
should
be
the
author
of
the
of
the
message.
Okay,
so
let's
see
and
now
what
this
hash
is.
So
basically,
we
have
this
routing
mechanism,
because
there
are
certain
type
of
certain
type
of
messages
that
arrive.
A
B
A
Network-
and
I
want
to
see
if
people
have
you
know,
questions
regarding
the
how
the
routing
works
and
how
the
graph
you
know
is
completely
up
and
maintained.
C
B
So
for
the
for
the
moment
we
have
like,
we
have
some
background
process
and,
like
I
don't
recall,
if
it's
actually
implemented,
but
there
is
some
garbage
collection
on
top
of
our
routine
table
to
remove
parts
of
the
graph
which
are
not
relevant
to
us
and
basically
the
the
part
of
the
graph
that
are
most
relevant
to
us
are
those
that
are
connected
to
validators
itself
to
peers
that
are
validators.
So
we
don't
necessarily
need
to
keep
a
full
view
of
the
network.
B
C
B
Like
we
have
discussed
how
how
many
invalidators
we
might
be
expecting
in
the
future,
and
it's
on
the
order
of
hundreds,
not
thousands-
and
I
think
hundreds
is
already
a
high
number
and
even
four
thousand
the
current
approach
is
fine.
Probably
the
korean
approach
will
break
with
tens
of
thousands,
and
it
will
be
most
noticeable
will
start
to
be
noticeable,
so
so
for
current
expected
number
of
validators.
That
should
not
be
a
problem.
C
B
B
Okay,
so
just
for
the
record,
the
graph
like
right
now
on
mainnet
boom
may
correct
me,
or
maybe
michael.
There
are
like
hundreds
of
nodes,
validating
and
not
validating,
and
the
diameter
of
this
graph
is
two.
So
routine
message
needs
only
to
travel
to
traverse
through
only
one
extra
extra
hop,
so
so
yeah.
B
We
don't
have
anything
particular
that
ensures
it.
It's
growing
organically
in
that
way,
because
we
have
like
this
threshold,
like
the
number
of
connection,
is
40
for
like
for
every
node.
The
number
of
connection
is
around
35,
more
or
less,
and
the
top
is
40
with
these
numbers,
like
organically
connections
created
at
random
and
the
diameter
is
two.
A
Yeah
I
mean
it's,
surely
the
I
mean
there's
a
formula
right,
so
it's
asymptotically,
roughly
log
n,
divided
by
k,
so
we
only
have
currently
only
have,
I
think
about
200
nodes
on
mainnet.
So
yeah
diameter
is
quite
small.
B
Well,
we
we,
we
cannot
guarantee
that
connections
are
random
for
every
node,
since
we
don't
control
every
node,
but
at
the
moment
in
honest,
not
honest,
but
in
a
node
that
is
ruled
in
our
code
we
sample,
and
we
when
we
try
to
connect
to
our
different
node.
We
sample
connections
at
random.
So
that's
our
warranty
and
we
are
like
assuming
our
implementation
is
actually
using
an
unbiased
sampler.
Basically.
C
A
B
Kind
of
assuming
that
I
mean
the
code
won't
break
or
we
are
not
assuming
that
connections
are
random
at
all.
It
just
happened
that
when
they
are
random,
the
diameter
is
small,
but
we
are
not
trying
to
optimize
the
topology
of
the
network
in
any
way
right
now,
but
that's
something
that
it's
also
doable.
Actually,
we
are
kind
of
handling
the
managing
the
topology
of
the
network.
When,
for,
for
example,
for
archival
nodes,
we
try
to
keep
connections
between
them
active
at
all
moments.
B
B
Let's
just
discuss
the
road
back
cache.
First,
the
route
back
mechanism
first
like
there
is
this
problem
that,
because
we
because
of
the
same
problem
we
were
discussing
before
that,
maybe
all
maybe
the
notes
won't
keep
a
full
view
of
the
network,
and
this
is
something
possible
either
because
they
are
their
routine
table
is
out
of
sync
or
because
they,
actually,
they
don't
want
to
to
maintain
the
full
view,
because
they
may
think
that
they
are
under
some
type
of
attack
of
this
attack.
B
B
We
have
this
mechanism
that,
for
that,
will
that
we
will
store
in
the
routing
table,
not
only
the
we
will
start
in
the
routing
table
for
every
routed
message
that
pass
through
us,
the
hash
of
the
the
hash
of
the
of
the
message
itself,
which
will
be
used
to
root
back
and
to
root
back.
This
message
notice
that
the
most
important
part
of
storing
the
the
hash
of
this
message,
of
course
we
will
store
in
the
hash
and
the
previous
hop
and
the
previous
hop.
B
So
whenever
we
receive
a
response
for
that
message,
we
just
put
the
response
through
the
same
path
but
backward
and
with
this
mechanism
we
don't
need
to,
and
the
target
of
the
message
doesn't
necessarily
need
to
maintain
to
to
have
a
route
to
the
source
of
the
message,
so
they
can
respond
without
without
actually
knowing
the
path
to
the
to
the
target.
Okay,
so
this
itself
might
be,
it's
introduce
some
other
attacks.
This
is,
for
example,
the
reason
we
have
before.
B
A
B
Of
the
message
and
some
identifier
of
it
and
when
we
are
rooting,
the
message
backward,
we
use
same
as
target
and
the
hash,
and
this
hash
is
stored
on
the
on
the
routing
table.
But
of
course
this
is
another
source
for
leaking
memory
on
for
attacking
the
network.
If
you
use
send
dummy,
wrote
a
routing,
routed
message
that
requires
a
response.
They
will
be
stored
on
the
rooting
table
and
you
can
explore
that.
So
for
this.
B
B
So
this
root
discussion
will
like
it's
just
a
sized
cache,
but
in
order
to
avoid
some
type
of
poisoning
that
make
us
and
remove
older
messages,
what
we
are
doing
is
that
we
are
allocating
some
pieces
of
some
sections
of
the
cachet
to
each
of
our
active
connections
and
whenever,
like
the
number
of
messages
is
stored
for
each
active
connection
might
be
more
than
the
allocated
section.
B
But
if
at
any
given
moment
in
time,
we
need
to
remove
something
we
remove
it
from
the
we
remove
it
from
the
from
the
peer
that
have
more
message
allocated
in
the
cache.
We
also
have
some
other
heuristics,
such
as
a
timeout
such
that
a
message
store
in
this
cache
expires,
so
that
we
don't
keep
very
the
messages
in
this
cashier.
So,
basically,
with
this
type
of
cache,
it's
a
kind
of
less
recent
cache
that
it's
the
less
recent
element
this
removed.
B
Notes
so
yeah
right
now,
one
in
like
there
is
a
problem
with
this
type
of
cache
that
is
being
addressed
in
a
pr,
but
it
hasn't
been
merged,
not
with
the
cache
itself,
but
with
the
mechanism
overall,
and
it's
that
when
we
implemented
the
balancing
of
the
network
that
the
network
has
it
balance
itself,
so
that
every
peer
has
from
30
to
35
active
connections
at
any
moment
in
time
and
the
edges
are
being
removed
and
created.
B
So
this
some
message
being
routed
back
and
failed
to
be
rooted
back
because
maybe
some
connections,
no
some
connections
no
longer
exist.
So
the
idea
is
to
enrich
these
messages.
The
root
back
messages
not
only
with
the
hash,
but
actually
with
the
source.
With
this,
the
pid
of
the
source
which,
in
the
backward
direction,
is
actually
the
the
target
and
so
that,
if
it's
not
in
the
cache,
we
will
use
regular
routine
mechanism
and
if
it
is
in
the
cashier,
we
use
the
the
path.
Okay.
B
Sorry,
if
it
wasn't
clear
so
the
hash
of
a
message
that
has
been
rooted,
we
map
it
to
the
peer
id
that
sends
sends
the
message,
like
the
previous
hop,
the
period
from
which
we
received
the
message.
So,
let's
say
a
know,
they
want
to
send
a
message
to
node
d
and
in
the
middle
there
is
no
b
and
c.
So
when
c.
B
Blocks
blocks
are
not
rooted,
but
some
transactions
I
think,
are
rooted,
but
I'm
not
sure
if
they
require
responses.
But,
for
example,
header
requests
require
responses
and
there
are
several
type
of
requests
which
are
rooted
message
because
they
happen
between
validators.
They
require
responses,
but
those
nodes
don't
necessarily
need
to
be
connected
to
each
other.
B
B
Yes,
yeah
and
we
have
this
function,
which
is,
I
think
it's
called
need
response.
B
Expect
response?
Okay,
we
have
this
function.
These
are
the
type
of
the
type
of
rooted
message
that
expects
response.
Basically,
they
are
all
requests.
This
is
just
a
nice
type
of
message
that
is
used
for
the
booking
purposes.
B
A
B
A
Think
the
each
master
each
type
is
well
documented.
You
know.
B
B
A
Remaining
yeah,
I
think,
yeah.
If
there's
questions,
that's
I
mean
if
anything,
martial
said
it's
not
clear.
Let's
discuss
discuss
it
here,
but
if
we're
talking
about
you
know
improvements
to
the
protocol,
let's
not
discuss
it
here
because
you
can't
go
it.
B
B
This
pr
that
it
was
started,
I
can
share
a
link.
It
was
sharing
the
previous
session
and
the
nap
it's
nap,
69
yeah.
So.
B
A
Okay,
so
if
there's
no
more
questions,
let's
maybe
end
it
here
and
thanks
a
lot
of
marcelo
for
working
us
through
the
the
routing.