►
From YouTube: IPFS Camp 2019 - DAGs & Blocks
Description
This section on DAGs from IPFS Camp 2019 goes over how Merkle DAGs are created, read, and used in IPFS. Mikeal Rogers tells us what are the tradeoffs between different block sizes.
This content is created as a part of Protocol Labs' Launchpad program. You can get resources and written instructions in our learning platform at https://curriculum.pl-launchpad.io
Resources
Slides as PDF & Course repo: https://github.com/ipfs/camp/#-core--elective-courses--youtube-playlist
Join Protocol Labs Launchpad: https://pl-launchpad.io/
For more information on IPFS https://ipfs.io
Original video:
Sign up to get IPFS news, including releases, ecosystem updates, and community announcements in your inbox, each Tuesday: http://eepurl.com/gL2Pi5
Chapters:
0:00 Start
00:15 What is a DAG
00:53 How to build a DAG
02:58 Constraints of a DAG
04:00 Big Block Sizes
05:55 Small Block Sizes
07:03 Just-Right Block Sizes
08:50 Q&A
A
A
A
Dag
is
just
a
fancy
acronym
for
direct
acrylic
wrap,
which
is
a
very
fancy
word
for
a
really
simple
concept.
If
you
think
about
a
normal
tree
structure,
you
have
a
root,
and
then
some
branches
and
then
eventually
some
leaves
and
typically
in
a
tree
structure.
A
leaf
would
have
one
parent
in
a
dag
it's
a
tree,
but
at
least
could
have
multiple
parents.
A
You
can't
have
circular
references
we'll
get
into
that
in
a
minute,
that's
impossible,
but
that's
all
that
it
means
unnecessary.
So
let's
build
a
dag.
So
one
interesting
thing
about
building
hash
based
data
structures
is
that
you
have
to
build
them
backwards,
so
get
used
to
writing.
Recursive
functions.
A
B
A
A
Structure
that
you
get
and
then
you
can
break
that
into
blocks
like
this
now.
Obviously,
this
is
a
very
tiny
data
structure,
so
we
wouldn't
normally
break
this
into
a
bunch
of
blocks,
but
when
you
have
really
arbitrarily
large
data
structures-
and
you
want
to
break
them
into
smaller
cement
blocks,
this
is
how
you
do
it.
You
do
it
with
placing
and
the
way
that
links
end
up
working
in
any
API
in
ipld
or
an
ipfs
is
that
they
get
Trend.
They
get
Traverse,
basically
transparently.
A
So
you
know
you
take
the
the
hack
of
qm4,
which,
which
is
our
root
note
here,
and
you
follow
it
through.
You
know
one
through
hello,
you
Traverse
through
the
entire
structure,
and
so
you
hit
this
hello
thing
right
and
it's
just
going
to
grab
all
of
those
blocks
for
you
in
Traverse
it
transparently,
but
you
could
also,
you
know,
grab
the
the
hash
of
one
of
the
leaf
notes
here,
like
here
hash
three
and
look
at
a
property
there.
You
could
grab
the
hash
of
the
leaf
node
itself
to
look
at
property
right.
A
A
Have
the
hash
of
any
of
these
branches?
You
can
Traverse
it
even
more
efficiently.
Here
we
go.
So
if
we
look
at
the
tree,
Now
sort
of
in
the
top-down
way
where
the
links
are
sort
of
pointing
down,
we
can
talk
about
some
of
the
nice
constraints
of
of
making
dags.
So
we
see
here
that
we
have
two
branches
pointing
to
the
same
Leaf.
But
what
happens
if
we
try
to
create
a
reference
back
to
the
parent?
We
basically
try
to
create
a
circular
reference.
A
Well,
it's
actually
impossible
because
you
'll
find
this
you're
going
to
modify
the
hatch
and
because
the
hash
of
this
is
part
of
what
is
like
inside
of
these
leaves
that
are
included
in
the
root
hash.
Then
that
would
change
the
hatch
of
everything
all
the
way
up.
So
this
this
knowledge,
what
the
half
would
be
of
the
root
node
is
just
impossible
to
know
up
front
unless
you
have
a
hash
mark
in
the
sort
of
broken.
So
if
you
don't
use
Shell
One
bad
things
can
happen.
A
A
A
A
So
when
you
have
these
large
immutable
structures,
and
then
you
mutate
them,
you
get
a
new
root
which
we'll
we'll
get
into
in
a
little
bit.
You
end
up
creating
what
are
called
orphan
blocks
if
you've
ever
built
databases
or
dealt
with
any
like
on
disk
mutable
data
structures.
You
deal
with
the
same
problem
where,
like
as
you
change
things
you
end
up
with
data,
that's
just
like
garbage.
That
then
has
to
get
collected
at
some
point
in
time.
A
You
also
end
up
with
a
lot
less
due
duplication,
the
bigger
that
you
make
the
blocks.
That's
because
the
the
only
method
that
we
have
for
you
duplication
is
the
hash
itself.
So
if
you
take
all
these
these
little
particles,
that
would
be
the
same
data
across
many
different
nodes,
but
they're
not
links
they're,
just
embedded
in
the
blocks,
then
those
won't
ever
get
deduplicated
too
small.
So
you,
you
also
end
up
with
issues
when
you
make
a
bunch
of
tiny,
tiny,
tiny
box.
A
So
the
big
one
is
that
you
just
end
up
with
more
requests
for
more
blocks.
If
you
want
to
get
like
a
small
amount
of
data,
you
end
up
getting
like
a
lot
of
different
blocks
that
could
be
out
of
the
network
that
could
be
off
of
disk,
but
it's
always
going
to
incur
some
kind
of
round
trip,
usually
to
I
o,
unless
you
have
like
a
lot
of
the
memory
cache
date.
A
A
Hashes
hashing
functions
have
usually
have
guarantees
about
how
long
they're
going
to
take
for
any
amount
of
data.
So
if
you
end
up
with
a
lot
of
smaller
blocks,
you're
going
to
end
up
with
like
many
more
hashes,
which
is
actually
going
to
spend
like
a
lot
more
compute
time
and
then
similar
to
The
Block
request,
you
end
up
with
just
more
hops
to
a
given
piece
of
data.
A
A
A
good
example
of
this
is
like
when
ipfs
creates
a
file
and
chunks
it
up.
We
always
pull
that
whole
file
out
of
disk
and
then
run
it
through
the
junker.
We
don't
like
look
at
the
existing
chunks
file
and
go
like
oh
I,
see
like
a
particle
that
I
can
just
stick
in
here.
Right,
like
you
know,
in
order
to
figure
out
if
they
match
we
just
re-encoded
every
time,
and
so
you
would
use
a
very
different
algorithm
to
create
that
dag
and
balance
it
eventually
than
you
would.
A
If
you
were
just
trying
to
mute
a
dag
over
time.
Right
have
just
like
a
really
good
example
of
like
something
that
you
mutate
in
place
really
often,
and
so
there
is
a
there's,
an
algorithm
in
the
Hampton
that
guarantees
a
certain
balance
and
a
fixed
width
or
sorry,
a
fixed
depth
of
how
big
that
tree
will
get.
C
A
Big
you
want
to
create
the
bar.
You
also
have
transport
performance
issues
right
if
you're
pulling
all
of
your
blocks,
you
know,
out
of
like
an
HTTP,
do
connection
to
one
host.
It's
going
to
have
a
very
different
performance
profile
than
pulling
them
out
of
peer-to-peer
Network,
and
so
you
might
be
able
to
make
completely
different
trade-offs
when
you're,
when
you're,
creating
those
block
structures
than
you
would
otherwise.
C
So
you're
alluding
to
this,
but
basically
right
now
for
arbitrary
data,
there's
no
optimization
to
try
and
figure
out
if
there's
overlap
like
if
you're
just
mutating
a
little
bit
of
a
file
that
most
of
the
the
blocks
will
be
the
same
at
what
you're
trying
right.
But
if
you
change
something
as
arbitrary,
how
do
you
figure
out
what
that
arbitrary
fit
is?
Can
we
leave
the
rest?
You
know
yeah.
A
So
it
turns
out
that
that
particular
problem
in
the
file
case
is
dependent
on
the
file
type,
and
there
are
different
chunkers
that
you
use
to
potentially
get
better
deduplication
at
a
different
file
types,
and
so
we
will
actually
get
into
that
a
little
bit
later.
Do
it
I
think
we
talk
about
rabbit,
a
bit
right,
yeah
yeah
so
like.
If
you
have
text
Raven
is
like
an
algorithm
that
breaks
it
up
in
a
really
nice
way
to
do
the
duplication.
Next
section.
B
A
Promise
yeah
any
other
questions
about
dags
and
block
sizes.
You
do
you
believe
that
the
current
algorithm
is
very
efficient,
or
is
there
still
a
lot
of
work,
that's
going
on
which
algorithm
so
like
the
production
of
the
dag
for
anything
that's
put
into
ad,
for
example,
I
think
that
we
we
have
some
good
options.
So
the
problem
right
now
is
that
we
are
not
pre-configured
for
different
data
types
to
use
different
tonkers.
You
have
to
tell
it
with
chunker
I
think
that
we.
A
Options
in
which
chunkers
to
use
and
pretty
decent
defaults
in
the
chunker
I
think
that
the
the
thing
that
we're
not
very
efficient
at
right
now
and
you're
supposed
to
do
one
is
the
dag
that
leads
up
to
the
file,
so
the
the
directory
and
the
files
right
so
like.
If
you
have
a
really
tiny
file,
we
don't
have
a
way
to
like
just
inline
that
binary
into
the
file,
and
if
that
file
is
really
tiny
in
line
it
into
a
directory
right
like.
A
Directory
of
five
files
in
the
entire
set
was
like,
maybe
like
one
tenth
of
a
megabyte.
It's
one
block
like
who
cares,
but
we
don't
actually
have
a
way
to
do
that
in
dag
PB
right
now
so
and
in
the
Unison
must
be
two
deep
dive
later
today,
I'll
get
into
sort
of
how
we
might
solve
some
more
info.
So
I
think
that
we,
we
don't
have
a
very
efficient
way
to
create
the
diagnating
into
a
file,
but
once
we
get
into
file
chunking,
we've
done
some
pretty
good
work.
There.
A
B
A
A
So
they
will,
in
the
future,
I
think
that,
like
if
you're
just
thinking
about
dag
sort
of
generically,
the
way
that
we've
done
things
sort
of
in
throughout
the
stack
is
that
a
block
can
contain
an
arbitrary
number
of
what
we
call
nodes
right.
So
a
node
is
like
any
map
or
any
value
within
it.
We
don't
have
in
in
newer
codecs
like
the
XC
board
deck
Json.