►
From YouTube: How IPFS Deals With Files - IPFS Camp Workshop
Description
This course from IPFS Camp 2019 is a deep exploration of the reasons behind immutable data, how we address immutable data, the data structures IPFS creates, and the different ways of interacting with files in IPFS. Follow along with ...
- Slides as PDF: https://github.com/ipfs/camp/blob/master/CORE_AND_ELECTIVE_COURSES/CORE_COURSE_A/IPFS_Camp_Core_Course_A_Slides.pdf
- Course repo: https://github.com/ipfs/camp/tree/master/CORE_AND_ELECTIVE_COURSES/CORE_COURSE_A
Learn more about IPFS Camp: https://camp.ipfs.io/
For more information on IPFS
- visit the project website: https://ipfs.io
- or follow IPFS on Twitter: https://twitter.com/IPFS
Sign up to get IPFS news, including releases, ecosystem updates, and community announcements in your inbox, each Tuesday: http://eepurl.com/gL2Pi5
A
A
A
Tekken
is
one
way
to
ensure
that
data
you
asked
for
is
the
data
you've
got.
The
idea
is
to
calculate
an
identifier
for
the
data
from
the
data
itself.
The
identifier
is
called
a
hash
and
we
use
cryptography
when
computing
the
hash
to
ensure
certain
properties
like
uniqueness
and
determinism.
But
we're
gonna
talk
a
little
bit
more
about
that
in
the
next
session.
A
If
someone
else
gets
hold
of
the
data
and
the
hashes
don't
match,
then
Oh
someone's
tampered
with
your
data,
so
the
guarantee
of
interpreting
checking
integrity
checking
is
that
content
is
immutable.
If
the
data
change
is
the
hash,
I
generate
no
longer
matches.
So
verifiability
is
one
of
the
main
reasons
for
using
immutable
data
in
the
web
of
today,
I
can
put
my
poodle
picture
up:
abc.com,
slash,
poodle,
jpg
and
24
hours
later,
I
can
doodle
on
my
poodle,
so
let's
say
I
add
some
glasses.
Make
him
look
a
little
bit
more
sophisticated.
A
A
A
So
back
to
our
example,
if
we
were
using
content
addressing
up
cute
poodle
has
a
specific
address
derived
from
its
content,
the
hack
and,
if
I,
change,
that
that
poodle
24
hours
later
the
address
also
changes,
but
that's
okay.
It
could
be
verified,
cached
and
fetched
by
anyone.
What,
if
I,
wanted
to
keep
both
files?
Well,
because
the
content
is
immutable,
ID
FS,
any
read
really
needs
to
store
the
changes,
the
differences,
the
two
files
that
we
see
actually
share
many
of
the
same
bytes.
A
So
if
I
have
one
of
those
files,
the
amount
of
data
I
have
to
transfer
to
fetch.
The
other
is
really
minimal
there.
Currently,
if
I
want
to
store
those,
two
very
similar
pictures
I
need
to
store
both
of
them
in
full
on
my
disk
and
if
I
want
to
share
them
with
someone,
and
you
want
both
and
I
want
them
to
see
both
of
those
pictures
and
I
have
to
transfer
them
both
in
full
and
we'll
see
more
about
how
ipfs
does
that
magic,
DG
ping
stuff
later,
okay
effect
from
anyone.
A
If
the
content
is
immutable
and
I
can
verify
its
integrity,
I
should
just
be
able
to
get
it
from
anyone,
and
this
suits
p2p
really
really
well,
but
that's
not
true
off
the
web
today,
so
I
have
two
sites
on
the
on
the
net
hosting
the
same
content.
How
do
I
know
which
of
these
poodles
is
the
correct
poodle?
A
Well,
okay,
so
the
answer
is
that
we
can't
trust
that
any
of
these
poodles
are
correct.
They
are
both
adorable
poodles,
but
we
just
can't
get
it
from
anyone
where
it
needs
to
come
from
a
trust
trusted
source,
because
I
can't
verify
its
integrity
all
right.
So
we
use
content
addressing
in
ipfs
and
we
use
a
special
hash
called
a
CID.
Cid
stands
for
content
identifier
and
we're
gonna.
Take
a
closer
look
now
in
part
two
of
five
okay.
A
A
A
There
are
many
different
hashing
algorithms
that
exist.
Ipfs
uses
char
to
256
by
default.
That's
just
some
of
the
some
of
them
that
are
available
in
ITSs
that
we
could
use
older
aggro
algorithms
like
sha-1,
are
broken,
they're
proven
to
not
be
collision
free,
and
if
and
the
problem
is
that,
if
algorithms
can
break
we're
going
to
want
to
switch
that
hash
that
we
use
by
default
in
the
future,
the
problem
with
switching
algorithms
is
that,
given
a
hash
which
is
just
a
series
of
bits,
what
algorithm
did
we
use
to
generate
that
hash?
A
We
need
a
future-proof
way
of
of
identifying
the
hash
function
used
to
generate
a
hash,
as
well
as
the
hash
name
say
hello
to
multi
hash
it'll
solve
all
your
problems,
so
multi
hash
is,
is
the
hash
which
is
just
at
the
end
there,
but
it's
also
a
prefix,
and
that
prefix
is
one
number
now
go
algorithm
and
which
identifies
the
hash
algorithm
that
was
used
to
generate
the
hack
and
another
number,
which
is
the
hash
length.
These
two
numbers
are
both
variants
and
Virant
is
just
a
compact
encoding
for
integers.
A
The
algo
number
is
a
multi
codec,
because
a
is
Savarin
but
b
it's
it's.
It's
value
is
a
predefined
value
that
we
agree
on
it's
in
the
inner
table
on
the
Internet,
the
multi-coated
identifier
for
char
two,
two
five
six
is
the
number
eighteen
and
our
hash
length
is
two
five
six
in
binary.
It
looks
like
this
you'll
see
that
the
hash
length
is
actually
two
bytes
long
and
that's
because
the
length
is
over
one,
two,
seven
and
in
far
in
encoding
numbers
above
1
to
7
are
encoded
as
2
bytes.
A
A
We
want
to
support
multiple
encoding,
dangerous,
so
you've
got
a
hash
that
I'm
dresses
your
data.
When
you
get
the
data
back,
how
do
you
know
how
its
encoded
we
might?
It
might
be
encoded
in
C,
Bohr,
concise,
binary,
object,
representation.
It
might
be
encoded
in
protocol
buffers,
it
might
be
encoded
as
just
playing
out
JSON.
A
Why
do
we
want
this?
Well,
it
might
be
a
particularly
compact
binary
encoding,
so
really
efficient
for
storage,
but
it
might
be
for
convenience
might
be
just
easy.
It
might
be
really
quick,
but
we
have
this
same
problem
again
like
when
we're
looking
at
some
encoded
data.
How
do
we
know
how
to
decode
it
same
solution?
More
metadata
prefix?
So
this
is
an
IP
LD,
codec
and
ipfs
at
the
moment,
any
any
young,
any
content
that
you
add
gets
is
the
codec
that
used
to
encode.
The
data
is
tagged.
A
B
A
There
are
many
IP
LD
formats
that
you
can
use
and
there
are
too
many
alright.
So
there
are
two
different
CID
versions,
so
that's
mildly,
frustrating,
as
they
often
nowadays
or
going
into
the
future.
You
will
see
version
1
as
the
main
version
that
we're
using
there
is
version
0
and
version
1,
it's
just
a
number
prefix
and
that
we
add
on
to
the
front.
A
So
all
of
you,
if
you
have
a
use
ipfs
before
you,
might
be
used
to
seeing
c
IDs
like
this
version.
0
is
a
this
top
one
and
a
version.
One
one
is
the
bottom
one,
and
so
the
version
0
ones
begin
with
kind
of
qm,
usually
version
version.
Zero
begins
with
qm
version,
one
begins
with
baffi
or
bath
case.
Sometimes
alright.
So
the
thing
is
see:
IDs
are
binary,
data's,
zeros
and
ones
make
for
really
long
strings.
When
they're
printed
out
it
doesn't
even
fit
on
the
saw
on
the
slide.
A
So
what
we
do
is
we
use
a
higher
number
base
encoding
and
to
make
them
shorter
and
easier
to
like
print
out
and
recognize,
even
when
in
string
form,
we
have
another
problem
again.
How
do
we
interpret
the
number
base
encoding
that
this
string
was
create
created
with
characters
in
one
encoding
are
valid
in
another
encoding,
so
we
can't
just
look
at
the
string
and
be
like
that's
that's
a
base.
Buddy
too.
A
So
how
do
we
fix
that?
Well,
you
guessed
it
more
meta
prefix.
This
is
multi
base
and
multi
base
helps
us
know
the
number
base.
The
content
is
encoded
with
the
first
symbol
is
the
identifier
for
the
encoding
B
is
base,
52,
Z
or
Z
is
base
58,
BTC
and
M
is
base
64
version.
Zero,
C
IDs,
like
the
QM
one
you
saw
in
the
last
slide,
are
also
base.
58,
encoded
and
I
just
told
you
that
base
58
should
be
prefixed
with
Z,
but
they're.
Not
if
you're
wondering
about
that,
then
don't
worry.
A
A
This
is
what
it
looks
like
in
binary
form
is
the
CID
version
is
the
IP
LD
format,
multi
codec,
and
it's
the
multi
action.
The
multi
hash
is
split
up
in,
as
you
know,
because
we
discovered
it
into
the
hashing
function
used
the
hash
length
as
a
stream.
We
have
the
multi
base
encoding
and
that
encoding
applied
to
the
rest
of
the
CID
cool
all
right.
So
you
can
try
this
as
well.
C
A
A
So
multi
base,
as
as
I
was
saying,
is
a
it's
code
is
be
start
at.
The
string
is
B,
so
that's
good
for
Thank
You
B
for
the
multi
hash
Curt,
like
the
code
for
char
2,
2
5
6
is
18,
as
you
saw
on
the
slides
so
yeah.
That's
that's.
One.
I've
got
a
base64
encoded
see
over
here
if
I
paste
that
in
remember
I,
said
basic
before
CID
start
with
an
M,
and
they
do
according
to
this.
It's
ok.
So
this
is
another.
D
A
So
this
is
a
v1c
ID
encoded
with
base
58
BTC,
and
this
is
interesting
because
it's
using
a
dag,
see
bor
IP
LD
format,
not
the
dag
PB,
which
is
the
vector
V,
is
a
default
for
adding
files.
Do
not
get
that,
but
you
can
use
the
ipfs
dag
API
to
to
create
structures
that
have
different
encodings
and
this
one's
good.
For.
A
A
Alright,
in
the
beginning,
there
was
only
multi
hash
and
they
were
encoded
as
base
58
BTC,
all
version
zero
see,
IDs,
have
no
multi
base,
and
so
that's
why
they
don't
start
with
Z.
Why
did
I
start
with
QM,
though?
Well?
They
are
just
multi
hashes
and
MA
and
they're
just
multi
hashes
encoded
the
face
58.
A
So
if
you
remember
back
to
the
slides
about
multi
hashes,
multi
hashes
are
the
hashing
function,
plus
the
half
length,
plus
the
hash
and
those
two
things
at
the
start
of
the
of
the
multi
hash
are
kind
of
fixed,
usually
in
ipfs,
so,
which
is
why
you
see
a
lot
of
these
and
and
that
it's
just
the
what
it
is
these
you.
This
qm
is
just
those
two
numbers
encoded
in
base
58
the
multi
base
is
always
base
58
BTC,
like
it
said,
and
so
what
else
do
these
not
have?
A
E
A
They
don't
have
a
format,
they
don't
have
a
CID
version
so
altogether
for
V
zeroes.
This
is
way
simpler.
It's
just
they're,
just
multi
actors
and
it's
drink
form
their
base,
58
encoded,
multi
hashes,
which
is
why
they
start
with
cue
em.
You
can
kind
of
think
of
them
like
this,
so
the
multi
base
is
always
based.
58
feet.
Easy
PTC
is
Z,
it's
implicit,
it's
not
written.
Cid
version
is
always
0.
It's
implicit,
it's
not
written.
The
IP
LD
format
is
always
dag.
Pb,
it's
implicit,
it's
not
written.
A
The
hash
function
is
always
char
to
256.
It's
it's
explicit,
not
implicit.
It
was
a
trick
question
it's
in
the
multi
hash,
so
the
hash
function
is
in
the
multi
fat
patch
it
is
in.
Is
there
the
the
hash
length
is
also
explicit
it's
in
the
multi
hash
and
if
any
of
those
variables
change,
then
you
don't
have
a
V,
zero,
CID
anymore,
all
right
cool.
So
now
you
know
how
I
TFS
addresses
content
using
C,
IDs
and
hashes,
and
we
can.
G
We've
done
this
three
other
times,
so,
what's
a
dag
I'm
sure
that
you've
probably
heard
at
this
point
come
around
a
little
bit.
Dag
is
just
a
fancy
acronym
for
direct
acrylic
RAF,
which
is
a
very
fancy
word
for
this
concept.
If
you
think
about
a
normal
tree
structure,
you
have
a
root,
and
then
some
branches
and
then
eventually
some
leaves
and
typically
in
a
tree
structure.
A
leaf
would
have
one
parent
and
a
dag
it's
a
tree,
but
at
least
it
could
have
multiple
parents.
G
G
So,
let's
build
a
dag,
so
one
interesting
thing
about
building
hash
based
data
structures
is
that
you
have
to
build
them
backwards,
so
get
used
to
writing.
Recursive
functions.
I
hope
you
enjoy
that.
So
we're
gonna
start
here
with
hello
world.
Just
our
little
object,
which
is
the
leaf
node
we're
gonna
get
a
half
of
that.
The
hash
is
cume
hash
one.
G
G
You
know
another
property
is
just
an
arbitrary
metadata
and
both
of
those
are
referred
to
in
the
roof,
so
they've
both
get
patched
and
then
used
as
properties
in
the
root
government.
So
what
you
end
up
with
is
a
full
data
structure
that
looks
something
like
this
right
now
in
this
one.
You
can't
tell
that
these
are
basically
the
same
thing
but
semantically.
This
is
a
big
data
structure
that
you
get
and
then
you
can
break
that
into
blocks
like
this
yeah.
G
Obviously,
this
is
a
very
tiny
data
structure,
so
we
wouldn't
normally
break
this
into
a
bunch
of
blocks,
but
when
you
have
really
arbitrarily
large
data
structures-
and
you
want
to
break
them
into
smaller
Samantha
blocks,
this
is
how
you
do
it.
You
do
everything
and
the
way
that
links
end
up
working
in
any
API
and
IPL
de
or
an
IPSS
is
that
they
get
trend
they
get
traverse
basically
transparently.
So
you
know,
if
you
take
the
the
half
of
qm4
here
and
you
follow
it
through.
H
G
We
go
so
if
we
look
at
the
tree
now
sort
of
in
a
top-down
way
where
the
links
are
pointing
down.
We
can
talk
about
some
of
the
nice
constraints
of
making
days.
So
we
see
here
that
we
have
two
branches
pointing
to
the
same
leaf.
But
what
happens
if
we
try
to
create
a
reference
back
to
the
parent
if
we
basically
try
to
create
a
circular
reference?
G
Well,
it's
actually
impossible,
because
if
you
modify
this
you're
going
to
modify
the
hatch
and
because
the
hatchet,
this
is
part
of
what
is
like
inside
of
these
leaves
that
are
included
in
the
root.
Then
that
would
change
the
halves
of
everything
all
the
way
up.
So
this
is
knowledge
what
the
hack
would
be
of
the
root.
Node
is
just
impossible
to
know
up
front
unless
you
have
a
hash
pump
in
the
slip
broken.
G
So
if
you
don't
use
shell
one
bad
things
get
so
any
questions
before
we
get
into
the
sort
of
block
sizes
in
the
discussion
about
that,
a
little
bit
no
questions,
okay,
block
sizes,
so
yeah
block
sizes
are
like
the
three
bears.
You
know
you
like
to
hot
to
cold,
just
writes
so
Lots
can
be
too
big
and
there's
some
when
you
start
to
make
blocks
bigger
and
bigger,
you
get
a
set
of
different
problems.
One
big
one
is
that
you
can't
download
a
single
block
from
multiple
peers.
G
The
problem
is
that
in
an
untrusted
Network,
anybody
could
just
give
you
bad
data.
The
only
way
that
you
have
to
validate
if
the
data
was
right
or
not,
is
the
hash
of
the
entire
block.
So
if
you
pulled
data
from
two
people
for
the
same
block
and
then
it
doesn't
validate,
who
sent
you
bad
data,
no
idea
so
like
BitTorrent
has
been
dealing
with
this
for
like
longer
than
anybody,
and
they
just
don't
try
to
do
that
anymore
if
it
appears
too
slow
and
they
decided
to
take
it
from
a
new
period.
G
They'll
just
you.
So
when
you
have
these
large
immutable
structures,
and
then
you
mutate
them,
you
get
a
new
route,
we're
get
into
it
a
little
bit.
You
end
up
creating
what
are
called
orphan
blocks
if
you've
ever
built
databases
or
dealt
with
anything
like
on
disk
mutable
data
structures.
You
did
with
the
same
problem
we're
like
as
you
change
things
you
end
up
with
data,
that's
just
like
garbage.
That
then
has
to
get
collected
at
some
point
in
time.
G
Different
transports
may
have
different
limitations
bit
swap
has
issues
with
blocks
over
two
megabytes
right
now
that
will
probably
be
fixed,
but
you
should
still
try
to
keep
that
in
mind
and
keep
in
mind
that
other
transports
are
gonna
have
some.
You
also
end
up
with
a
lot
less
deduplication,
the
bigger
that
you
make
the
blocks.
That's
because
the
the
only
method
that
we
have
for
deduplication
is
the
hash
itself.
G
So
if
you
take
all
these
these
little
particles,
that
would
be
the
same
data
across
many
different
nodes,
but
they're
not
links
they're,
just
embedded
in
the
blocks,
then
those
won't
ever
get.
You
duplicated,
you
small,
so
you
also
end
up
with
issues
when
you
make
a
bunch
of
tiny,
tiny
tiny
blocks.
So
you,
the
big
one,
is
that
you
just
end
up
with
more
requests
for
more
blocks.
G
If
you
want
to
get
like
a
small
amount
of
data,
you
end
up
getting
like
a
lot
of
different
blocks
that
could
be
out
of
the
network
that
could
be
off
of
disk,
but
it's
always
going
to
incur
some
kind
of
round-trip,
usually
to
I/o.
Unless
you've
had
like
a
lot
of
the
memory
cache
day.
You
also
end
up
with
more
encoding
time.
G
When
you
write
different
block
encoders,
you
can
make
them
more
efficient.
The
more
do
you
have
at
a
time.
Usually
so
if
you,
if
you
have
a
much
smaller
sets
of
data,
then
you
may
end
up
with
less
efficient
encoding
times.
The
big
one,
though,
is
hashing
hashes
hashing
functions
have
usually
have
guarantees
about
how
long
they're
going
to
take
for
any
amount
of
data.
G
You
end
up
with
just
more
hops
to
a
given
piece
of
data,
so
whenever
you
want
to
look
at
something
through
a
deep
graph,
you
end
up
like
going
from
one
thing
to
the
next
and
the
more
that
you've
created
the
more
links
in
that
team
that
you
create
the
more
potential
performance
in
that
yeah.
So
what's
just
right,
just
right,
like
very
much
depends
on
your
use
case.
There
just
isn't
one
true
way
to
do
this.
G
Unfortunately,
this
is
why
we've
created
so
many
flexible
structures
and
how
you
create
blocks
and
how
you
create
your
DAGs
is
the
different
use.
Cases
are
going
to
call
for
very
different,
dag
structures.
So
if
you
want
to
optimize
for
reads
versus
writes,
if
you
can
take
a
lot
of
performance
cost
up
front
at
right
time
to
make
the
reads
really
fast,
you
might
do
that
if
you,
if
you
have
as
often
when
you
have
mutable
structures,
you
will
not
actually
mutate
a
particular
part
of
the
graph,
but
you
might
just
recreate
it
every
time.
G
A
good
example
of
this
is
like
when
ipfs
creates
a
file
and
chunks
it
up.
We
always
pull
that
whole
file
out
of
disk
and
then
run
it
through
the
chunker.
We
don't
like
look
at
the
existing
chunked
file
and
go
like
oh
I,
see
like
a
particle
that
I
can
just
stick
in
here.
Right,
like
you
know,
in
order
to
figure
out
if
they
match,
we
just
re-encode
it
every
time,
and
so
you
would
use
a
very
different
algorithm
to
create
that
dag
and
balanced.
G
Essentially,
then,
you
would,
if
you
were
just
trying
to
mutate
a
dag
over
time.
Right
have
just
like
a
really
good
example
of
like
something
that
you
take
in
place
really
often,
and
so
there
is
a
there's,
an
algorithm
and
a
ham
guarantees
a
certain
balance
and
a
fixed-width
sorry
epochs
depth
of
how
big
that
tree
will
get.
So
these
are
just
like.
You
know,
different
trade-offs
that
you
can
make
when
you're
deciding
the
day
and
how
much
and
how
big
you
want
great,
the
one.
You
also
have
transport
performance
issues
right.
G
If
you're
pulling
all
of
your
blocks,
you
know
out,
it
was
like
an
HTTP
Duke
connection
to
one
host.
It's
going
to
have
a
very
different
performance
profile
than
pulling
them
out
of
their
peer
network,
and
so
you
might
be
able
to
make
a
completely
different
trade-offs
when
you're,
when
you're,
creating
those
block
structures.
That
would
otherwise
oh
wait.
I
G
So
it
turns
out
that
particular
problem
in
the
file
case
is
dependent
on
the
file
type,
and
there
are
different
chunker's
that
you
use
to
potentially
get
better
deduplication
at
a
different
file
types,
and
so
we
will
actually
get
into
that
a
little
bit
later.
Dude
I
think
we
talked
about
rabbit
a
bit
right,
yeah
so
like.
If
you
have
text
ravenous
like
an
algorithm
that
breaks
it
up
in
a
really
nice
way
to
yeah.
G
G
I
think
that
we
have
some
good
options,
so
the
problem
right
now
is
that
we
are
not
pre-configured
for
different
data
types
to
use
different
hunkers.
You
have
to
tell
it
which
sucker
I
think
that
we
have
some
really
good
options
in
which
chuggers
to
use
and
pretty
decent
defaults
in
the
chunker.
I
think
that
the
the
thing
that
we're
not
very
efficient
at
right
now
and
you
necessity
one-
is
that
the
dag
leads
up
to
the
file
so
the
directory
in
the
file
right
so
like.
G
If
you
have
a
really
tiny
file,
we
don't
have
a
way
to
like
just
inline
that
binary
into
the
file.
I
know
that
file
is
really
tiny,
you'd
land
it
into
a
directory
right
like
if
you
had
a
directory
of
five
files
and
the
entire
set
was
like,
maybe
like
one
tenth
of
a
megabyte,
just
one
block
Thank
You
Karis,
but
we
don't
actually
have
a
way
to
do
that
in
dag
PB
right
now.
G
F
G
J
J
G
That's
many
blocks
right
now,
so,
right
now
in
use
of
c1,
that's
going
to
be
one
block
for
the
the
file
metadata
and
then
at
least
one
block
for
the
actual
file
data
that
was
leaking
and
then
the
directory
around
it
would
be
interlocked.
So
they
will
in
the
future,
I
think
that,
like
if
you're
just
thinking
about
dag
sort
of
generically,
the
way
that
we've
done
things
sort
of
in
throughout
the
stack
is
that
a
block
can
contain
an
arbitrary
number
of
what
we
call
nodes
right.
G
So
no
there's
like
any
map
or
any
value
within
it.
We
don't
have
in
newer
codecs
like
the
XE
Boren
deck
JSON.
We
don't
have
any
sort
of
limitations
on
yeah.
How
do
you
can
go
with
that?
What
types
that
you
can
use
so
in
the
future,
I
would
say
like
yes,
one
block
could
contain
many
files
right
now
there
are
some
yeah.
There
is
yeah
tend
to
your
question
there.
You
should
not
think
of
a
one-to-one
mapping
between
a
file
on
a
block
or
really
any
data
structure
and
a
block.
G
H
A
G
G
A
Because
you
got
say
if
you
come
back,
we'll
just
do
five
minutes
now
and
you
come
back
a
quarter.
To
then
was
that
against.
Okay,
let's
get
into
some
deep
diving
into
how
important
files
work
scene
I,
give
s.
There
are
fewer
people
here,
but
I'm
sure
they'll
shuttle
in
alright.
Let's
get
started.
What
actually
happens
when
I
use
ipfs
add
my
file
here
is
a
tube
of
bits
and
what
actually
happens
is
that
those
bits
get
divided
up
into
chunks
chunk.
One
chunk
two
and
chunk
free
I'll
be
referring
to
them
later.
A
So,
just
if
you
see
C
1
C,
2,
C
3
this.
These
are
the
bits
of
the
file,
the
chunks
of
the
file
that
get
chunked
up.
So
these
chunks
are
arranged
into
a
tree
structure
which
is
a
dag,
and
so
we've
already
talked
about
like
items
in
the
graphical
node.
Not
every
node
needs
to
contain
a
chunk
of
the
file.
Some
of
these
nodes
are
intermediate
nodes.
They
could
contain
chunks
of
files,
they
don't
always
in
this
example.
They
don't
so
yeah.
A
We've
we've
got,
we've
got
the
chunks
in
the
these
three
trunks
in
the
leaves
of
this
diet
has
been
created
when
building
the
graph.
We
already
talked
a
little
bit
about
this,
but
we
calculate
a
CID
for
every
node
and
that
happens
from
the
bottom
up,
and
that
happens
because
we
can't
calculate
the
CID
for
the
parent
until
we
notice
the
IDS
for
the
children,
because
the
CID
is
calculated
from
the
data
as
well
as
any
links
it
has.
So
the
links
we
don't
have
to
see
IDs
for
the
links.
A
We
can't
calculate
the
CID
for
the
node,
and
so
we
we
have
to
start
at
the
bottom
here
and
and
come
back
up
and
add
a
talk.
We
can't
do
the
top
one
because
we
don't
have
the
one
below
it,
so
we
have
to
traverse
all
the
way
down
to
the
to
the
leaf
and
then
slowly
make
our
way
back
up
again
and
we
finally
get
to
the
root
CID
cqm,
hash,
six
and
so
for
a
single
file.
A
A
Okay.
So
let's
see
it
in
action.
You
can
try
this
out
in
your
browsers
as
well.
Dagda
ipf,
esta
IO
drop
a
small
file
at
less
than
100
K
on
to
it.
People
people
been
dropping
like
five
Meg
pilots
on
it
and
I've
set
the
chunk
size,
pretty
slow,
pretty
low
and
like
their
browsers
start
like
varying
and
because
it's
trying
to
render
oh
crap,
so
what
it
does
is
it
renders
the
dag
for
you
and
it's
trying
to
render
a
graph
as
like
a
million
notes,
so
just
drop
a
small
file.
A
G
K
A
A
Right
we
can
drop
a
fire
alarm.
This
is
to
read
me
from
the
go
I
PFF
repo,
and
it
will
render
my
dag
for
me
and
I
can
hover
over
nose
and
I
can
see
that
on
the
bottom
here,
we've
got
a
bar
which
tells
us
information
about
each
node
that
has
been
created
in
the
bag
and
each,
and
we
can
see
that
each
now
it
has
a
different
CID
in
there.
They're
put
here
now
they're
displayed
here
and
then
I
can
change.
I
can
change
things
like
the
chunk
size
at
the
moment.
A
A
A
We
have
a
default
for
that
and
we've
just
been
changing
that
a
little
bit
and
like
smaller
chunks,
potentially
because
they've
got
fewer
bytes
in
them.
We've
got
a
better
chance
of
DG
Pingdom,
but
it
like,
like
michael's
woody,
said
it's
more
work
upfront
to
create
all
of
those
chunks
you
have
to.
You
have
to
create
hash
past
the
data
and
for
the
CID
for
every
single
chunk
and
the
more
chunks.
A
A
Yeah,
it's
not
so
necessarily
so
great
at
DGP
and
smart
chunk
sizes.
On
the
other
hand,
now
that's
interesting
one
so,
like
maybe
you
have
a
chunker
the
chunks
at
keyframes,
so
you
can
have
better
seeking
and
videos
now
cool
with
that
people
or
it
might
be
a
chunker
that
does
likes
like
very
cleverly
chunks,
the
file,
so
that
so
that,
if
you
add
data
in
the
middle,
it
doesn't
throw
off
all
of
you.
A
So
the
problem
we've
like
fixed
chunks
sizes,
is
that
if
you
add
a
bit
of
data
in
the
middle,
then
everything
like
it
froze
off
the
rest
of
the
chunks
and
those.
So
those
chunks
are
now
different,
and
so
the
deegeu
ping
is
not
necessarily
as
good.
If
you
use
a
smart
chunker
which
will
take
that
into
account.
Maybe
give
you
a
chunk,
that's
a
little
bit
bigger
could
for
that
data.
Then
you
can
keep
the
same
chunk
chunks
that
you
had
before
for
the
rest
of
the
file
and
so
yeah.
A
So
we
have
a
smart
chunker
already.
It's
called
robbing.
You
can
use
it
in
in
J
s
in
the
browser
which
is
brand-new
by
the
way
like
a
little
atom
thing,
which
is
super
cool,
and
it's
in
that
tag
that
tag
visual
as
well.
So
you
can
check
that
out
and
so
yeah
smart
chunk
size
is
really
interesting.
But
again
it's
more
work
upfront
to
do
that.
Work
of
kind
of
figuring
out.
Where
is
the
appropriate
boundaries.
E
A
L
A
Been
around
a
long
time,
it's
not
it's,
not
my
TFS
specific,
but
yeah
this.
So
at
the
moment
we
have
these
two
chunker's
I,
fixed
and
Robin,
that
like
we
could
have
many
different
chunker's
for
in
for
many
different
file
types,
and
that
would
do
the
right
thing
based
on
the
file,
and
you
know
we
could
have
some
really
good
good,
DG
thing
good
chunking
anyway.
A
Okay,
so
back
to
the
example,
let's
look
at
D
duping
in
the
visualizer
all
right.
What
I've
got
here
is
a
I've
copied
the
readme,
and
what
I've
done
is
I've
just
added
some
data
to
the
end
of
it.
So
it's
a
little
tiny
bit
bigger.
What
I
can
do
is
I
can
drag
and
drop
that
on
on
here
and
what
the
visualizer
tool
does
is.
A
It
adds
a
directory
for
these
two
files
to
these
these
two
nodes,
directly
descending
from
the
directory,
are
the
two
files
that
I've
at
the
roots
of
the
two
files
I've
added
basically-
and
you
can
see
from
the
lines
in
this
graph-
they
actually
share
almost
all
of
the
nodes
part
from
so
right
at
the
end
here.
So
we
can
pick
these
up
and
drag
them.
A
You
can
see
that
there's
two
little
chunks
here
and
here,
which
the
only
two
chunks
that
are
not
shared
by
by
these
files,
which
is
kind
of
cool,
and
you
can
see
that
this
this
chunk.
If
you
look
at
the
the
bite-size
hitted,
it's
this
trunk
has
slightly
more
bytes
than
this
one
and
what's
happened
here,
is
that
we
haven't
managed
to
get
it
so
that
it
like
it's
complete
boundary.
A
A
A
Okay,
so
actually
not
told
you
the
whole
truth
about
this.
That
nodes
in
the
graph
are
not
just
made
out
of
the
chunk
of
the
file
file
data.
Every
chunk
is
wrapped
in
this
thing
that
we
call
the
UNIX
FS
wrapper
and
UNIX
FS
wrappers
are.
They
can
have
three
files,
they
can
be
directories.
There
are
other
types,
but
these
these
two
are
the
main
main
two
and
it
allows
to
allows
us
to
distinguish
between
what
is
file
data
and
what
is
a
directory.
A
So
it's
kind
of
a
dream
within
a
dream:
it's
not
just
that!
It's
not
just
the
data,
it's
the
wrapper,
and
that
is
that
it
becomes
the
data.
So
what
that
does?
Is
it
actually
adds
like
a
few
bytes
of
overhead
to
every
node,
because
we've
got
the
UNIX
FS
wrapper
around
each
needs
a
bit
of
data.
So
let's
take
a
quick
look
at
that
in
here.
A
Okay,
so
we
can
see
in
the
bottom
left
hand
corner
if
you
hover
over
a
knows
the
the
UNIX
FS
type
that
that
mode
has,
and
this
this
one
at
the
top.
As
I
said
they
would
it
that
what
the
tool
does.
Is
it
because
two
files
in
a
directory?
So
this
is
a
UNIX
FS
directory
and
these
other
nodes
are
all
kind
of
UNIX.
A
Fs
file
knows
so
the
total
bytes
that
make
up
this
note
and
the
bytes
of
data
in
this
node
slightly
different,
and
that's
because
there
is
this
UNIX
FS
wrapper,
there's
a
few
bytes
of
racking
data
that
that
is
in
every
single
node.
What
we
can
do
is
we
can
change.
We
can
change
the
leaves
to
using
rule
ease.
A
What
that
does
is,
instead
of
adding
the
UNIX
FS
wrapper
to
the
to
the
leaves
it
will
not
do
that
and
just
it
will
just
add
a
row
buffer
of
data
as
the
leaf.
You
can
see
that
the
bytes
total
for
these
nodes
are
exactly
the
same
as
the
bytes
of
the
data.
So
we
don't
have
this
Raptor
anymore,
and
it
means
that
we
save
a
little
bit
of
space
on
every
node
we
create
because
we
don't
we're
not
wrapping
it
with
with
any
information.
D
A
A
E
A
Zeros,
as
we
talked
about
earlier,
they
have
implicit
things
about
them,
one
of
those
things
that
is
implicit
that
is
not
written.
That
means
it's
a
V
0
CID
is
the
IPL
D
format
and
the
IP
of
the
format
for
these
nodes
is
dag
PB
for
the
raw
nodes,
because
the
IP,
our
D
format,
if
you
remember,
tells
us
how
to
decode
the
data.
So
we
know
that
that
what
this
data
is
is
it's
a
it's
a
proto,
buff
encoded
chunk
of
data
and
decode
it
with
using
as
it
as
if
it
were
a
proto.
A
A
A
All
right,
there's
also
different
graph
layouts
that
have
different
performance
characteristics.
Balanced
layout
is
the
default
you
get
when
you
ipfs
ad,
and
it's
simple
to
build
and
really
easy
to
traverse.
But
it's
kind
of
difficult
to
edit,
if
you
were
to
change
in
the
middle
you'd,
have
to
do
a
lot
of
rebalancing
to
balance
it
out
again
where
it's
like
we've
got
trickle
as
well
is
an
option
it's
more
difficult
to
build,
but
it's
really
great
for
like
streaming,
for
example,
because
the
time
to
first
byte
is
a
lot
less.
A
Snowed
and
I
need
to
get
hold
of
this
node
and
I
can
start
streaming.
There's
like
imagine,
I've
got
a
big
file
like
a
big
movie
file
and
it's
made
a
big
balance
deck
and
if
I
want
to
start
streaming
it
then
I'm
gonna
have
to
get
this
node
this
node,
potentially
this
node
isn't
in
this
node
before
I
can
start
streaming
says
time
to
first
byte
is
going
to
be
much
better,
with
a
kind
of
trickle
style
of
dad
alright.
A
But
one
thing
to
notice
is
that
this
cid
up
here,
if
we
check
it
out,
it's
a
QM,
it
kind
of
ends
with
CW
ESU.
If
I,
if
I
change
the
balance
tag
starts
of
QM
doesn't
start
to
end
with
the
same
same
characters,
and
this
is
the
same
data
we've
not
changed
the
data
we've.
All
we've
done
is
change
the
options
here.
Actually,
if
you
change
any
any
of
these
options,
then
you
get
a
different
CID.
A
So
this
there's
something
to
know
there
or
take
take
heed,
or
at
least
you
do
get
different
C
IDs.
If
you
change
the
different
options,
because
the
reason
is
the
it
may
have
more
or
less
children
and
those
children
will
have
different
will
have
different,
see
IDs
because
they
have
more
or
less
children,
so
it
basically
propagates
all
the
way
up
to
the
top,
and
you
just
get
the
difference.
The
idea
for
the
same
data.
B
A
I
I
A
Thing
is
this
this:
this
is
all
available
as
like
options,
but
you
know
99%
of
the
time.
People
will
just
be
a
BFS
adding
and
it
will.
It
should
be
doing
the
right
thing
and
I
think
we
need
to
get
better
at
actually
building
an
appropriate
dag
for
the
file
you've
added
and
for
choosing
it.
You
know
good
chunker
because
of
this.
E
A
J
A
A
Okay,
all
right,
a
mutable
filesystem?
Okay,
what
is
it?
Well,
let
me
explain:
okay,
music,
the
NFS,
the
mutable
filesystem
is
an
abstraction
in
ipfs
that
lets
you
deal
deal
with
files
in
ipfs
as
if
they
were
a
mutable,
unix-style
filesystem.
It
takes
care
of
the
busy
work
involved
when
creating
new
dag
structures.
When
you
change
files,
so
at
a
high
level,
you
can
kind
of
think
of
it
like
a
tracking
there,
that
map's
paths
to
see
IDs
and
like
a
regular,
UNIX
style
file
system.
A
You
work
with
paths
and
you
have
to
familiar
basic
commands
available
to
you
like
move
copy
make
there.
You
know
that
sort
of
stuff
when
you're
addressing
your
cat
gifs
using
the
immutable
hash
like
this,
so
not
MSS
and
you
have
the
whole
ipfs
network
at
your
disposal.
So
if
it's
not
found
locally,
then
it
will
be
delivered
from
another
peer
on
the
network,
assuming
it
can
be
found
when
you
are
using
an
IP,
FS
path
like
this
path
to
my
important
business
document,
you're
asking
just
your
node
for
that
file.
A
A
A
So
what
happens
when
I
want
to
update
my
notes
with
neuf
like
I've
seen
a
new
rare
frog
and
I
want
to
add
it
to
the
end
of
the
file
like
you
can
imagine
the
path,
/
notes
that
frogs
dot
text
to
be
something
like
this
in
the
dag,
so
a
UNIX
FS
directory
for
slash
right
at
the
top,
and
that
is
the
current
EMFs
route
and
ipfs
keeps
track
of
this.
Everything
that
hangs
off
of
that
MF
s
route
is
safe
from
garbage
collection.
It
won't
be
collected.
So
that's
that's
good
to
be
so.
A
A
Well,
we
need
a
new
node.
We
need
an
updated
note
which
points
that
the
previous
chunks,
as
well
as
this
new
chunk,
so
we've
got
a
new
and
you
know
that
new
cid
for
the
frogs
file
and
because
we've
got
a
new
cid
for
the
frogs
file.
We
get
a
new
cid
and
a
new
node
for
the
notes
directory
because,
as
I
said,
that
CID
for
knows
is
created
from
the
data
as
well
as
the
links,
because
my
link
has
changed.
My
directory
has
changed
and
this
propagates
all
the
way
up
to
the
root.
A
So
qm
/
d,
RR
we've
got
a
new
cid
for
the
m
FS
root.
An
IP
FS
keeps
track
of
that.
So
it's
now
this
new
m
FS
root.
Is
this
directory
and
notice
that
I've
grayed
out
here
and
now
eligible
for
garbage
collections,
so
they
may
be
collected
depending
on
the
settings
of
your
ipfs
note
they
could
be
collected.
You
know
whenever
in
now,
when
you've
added
those
more
content
or
never
at
all,
you
never
know
yeah,
okay,
so
back
to
sharing
content.
What
have
we
learned?
A
J
K
M
A
Cool
okay
yeah
talk
about
it,
yeah
I'm,
glad
you're,
all
listening
cool,
okay,
so
the
the
this
is
the
difference
between
code
names
and
addresses
and
MF
s
is
names
for
content,
but
it
says
nothing
about
their
dresses.
Their
continent
dressing
is
like.
Has
this
restraint?
The
content
is
immutable.
Yet
we
are.
We
have
this
mutability
layer
on
top
of
it
and
also
a
location
addressing
is
kind
of
fun,
because
we
can
change
things
and
still
get
them
at
that
URL.
But
there
are.
C
K
A
A
We've
got
a
brand
new
tutorial
for
EM
FS
that
was
built
for
camp,
and,
if
you
head
on
over
to
proto
school
that
on
the
home
page,
you
should
see
a
em
FS
link
to
the
em
FS
tutorial,
which
will
cover
a
bunch
of
things
that
I've
just
talked
about,
but
as
well
as
there's
coding
exercises
and
all
sorts
that
and
that
will
kind
of
keep
you
occupied
and
help.
You
learn
a
bit
more
about.