►
From YouTube: What's with the IPLD Data Model, and what happens when you try and represent it - WAC - @aschmahmann
Description
What's with the IPLD Data Model, and what happens when you try and represent it - WAC - presented by @aschmahmann at IPFS þing 2022 - Data and IPFS: Models - https://2022.ipfs-thing.io
A
I'm
going
to
talk
a
little
bit
about
lessons
from
building
an
ipf
and
ipld
implementation
that
is
based
in
webassembly.
I
will
be
talking
more
about
this
in
a
more
webassembly
oriented
thing
on
friday.
There's
a
lot
to
talk
about,
I'm
gonna,
try
and
condense
it.
So
we
stay
on
time
and
focus
this
on
the
ipld
learnings
from
this
all
right.
A
So
what-
and
why
does
this
thing
exist?
So
there's
two
things
one
like
I
don't
want
to
rewrite
adls
in
all
of
the
languages,
just
as
like.
Please
don't
make
me
write
this
five
times
kind
of
thing
and
unix
fs
has
to
be
in
all
these
places,
and
it's
not
quite
in
all
these
places.
Yet
it's
in
a
bunch
of
them
getting
to
be
more
but
then
you're
like.
Oh
can
I
find
someone?
A
We
this
this
currently
supports
working
with
codex
and
adls
all
right.
What
is
a
kodak,
totally
non-controversial,
I've
just
taken
from
the
website.
Ipld
codecs
are
functions
that
transform
ipld
data
model
to
serialize
bytes,
and
I
guess
any
other
way
around
too.
So
you
can
send
and
share
data
and
transform
the
sterilized
bytes
back
into
the
ipld
data
model.
They
look
like
this
encode
data
model
of
the
bytes
decode
bytes
to
the
data
model
and
whatever
language
you
write,
your
thing
will
look
like
this.
A
If
you
do
codecs,
what's
the
ipld
data
model,
it's
a
collection
of
specific
of
data
types
such
as
these
full
integer
bytes
list
map
it
exists
so
that
developers
can
target
it
as
an
abstraction
layer
without
having
to
care
about
too
much
of
the
particulars
right.
You
know
I
want
to
look
at
get
our
bits,
files
and
list
the
list
of
the
fields
in
them
and
examine
them.
A
You
know
and
have
like
a
uniform
tool
set
to
work
with
them,
or
I
want
to
look
at
both
json
and
see
more
maps
and
have
a
unified
tooling
to
do
with
it
to
work
with
it.
You
could
argue:
should
it
support
more
data,
more
fields,
more
types,
fewer
types
yeah,
maybe
quaternion
should
be
in
there
really
important
for
my
video
games
that
I
support
fraternians,
maybe
not
right
now
it
doesn't
okay.
So
so
what
is
the
ikld
data
model.
A
So
so,
yes
to
back
up
there
is
this
thing
called
apparently
called
whack,
because
I
didn't
know
what
I
was
doing
at
the
time
for
the
webassembly
codec.
It
is
a
wacky
experience
we'll
get
into
this.
This
was
for
for
practicality,
reasons,
both
performance
and
working
with
webassembly
and
for
ease
of
development
and
not
wanting
to
write
like
a
thousand
like
ffi
calls
with
also,
I
guess
the
heads
up.
A
If
anyone
actually
looks
at
any
of
the
code
here,
which
I
will
point
to
it,
will
it
could
be
better.
I
learned
rust,
while
I
was
doing
this
in
like
several
hours
a
week
over
a
couple
of
months,
so,
like
you
know
better
than
me,
if
you've
written
any
rust,
and
so
basically,
what
I
realized
was
that
if
I
could.
A
Use
this
form
to
describe
codex
in
webassembly.
My
life
would
be
easier
instead
of
code
instead
of
doing
data
model
representation
to
bytes
and
error
to
bytes
and
have
to
figure
out
what
the
data
model
means
and
write
all
the
functions.
For
that
I
define
a
serialized
data
model
representation,
and
then
I
let
you
know
the
host
language,
do
data
model,
to
serialize
data
model
and
for
decode
I
can
take
bytes
and
I
spit
them
out
as
the
serialized
data
model
representation,
and
I
let
the
host
language
unpack
that
into.
A
However,
it
wants
to
do
data
model
representations,
you
know,
do
you
like
n64s?
Do
you
like
inch
128s?
Do
you
like
bigints?
Whatever
that's
the
library's
problem
serialized,
we
are
serializing
the
data
model
which
brings
us
to
whack,
which
is
supposed
to
be
a
concrete
implementation
of
the
data
model,
so
that
encoding
and
decoding
functions
work,
just
as
they
would.
A
If
I
was
using,
if
I
wasn't
trying
to
serialize
the
data
model,
what
is
it?
I
largely
stole
this
format
from
from
michael's,
simple,
dag
proposal
from
a
while
ago.
Basically,
it's
like
a
token
for
the
data
type
and
then
the
data,
so
we
have
true
false.
We
have
integer,
we
have
like
negative
integer,
we
have
bytes
strings
links,
maps
flip
and
lists
and
floats
we'll
go
into
the
ones
that
are
somewhat
controversial
here
is
to
back
up
some
things
that
are
not
controversial
right,
true,
false
null.
A
These
seem
like
maybe
yeah.
It
seemed
like
reasonable,
even
integers,
where,
like
the
byte
range,
isn't
defined
right
in
the
data
model.
It's
just
like
it's
an
integer.
We
hope
that
your
library
supports
up
to
two
to
the
53,
which
is
like
the
number
that
javascript
has
blessed,
which
yeah
don't
ask
me
why?
And
so
I've
chosen
to
encode
those,
as
you
know,
var
ants,
because
they
can
be
there.
They
can
be
infinite
length
right
in
the
serialized
form
and
you
can
decide
for
your
implementation,
like
what's
too
big
for
you.
A
So
if
you
read
through
the
spec
pr
on
this,
you
can
notice
some
issues
that,
if
you've
been
following
in
the
ipld
space,
you
might
be
familiar
with.
You
might
have
heard
of
these
issues
before
they
are.
What
is
a
string?
What
is
a
map
and
what
is
a
float?
What
I
will
start
with
the
last
one?
What
is
a
float
is
is
all
right
confirmation
this
these
have
been.
A
A
Okay,
correct
all
right,
so
so
for
for
those
on
the
zoom.
We
we're
rehashing
like
six-year-old
things
which
I
will
boil
down
to
you
as,
but
but
map
keys,
don't
have
any
restrictions
because
strings
are
bytes
and
bytes
can
be
anything
and
then
you
say
well,
but
string
should
be
and
they
say
well,
but
straight
should
does
not
mean
must
and
therefore
maps
can
in
fact
represent
anything
unless
you
want
to
go
and
tweak
your
string
definition,
and
so
what
I
did
was
implement.
A
What
is
the
data
model
which
is
strings
can
just
be
bytes
and
therefore
maps
bytes.
It
works,
it's
pretty
pretty
easy
to
implement
in
any
language.
I
implemented
it
and
go.
I
implemented
it
in
rust,
without
really
understanding
rust.
I
bet
you.
It
would
take
several
hours
to
implement
and
insert
language
of
choice.
A
The
most
complicated
part
is
sort
of
like
uvar
ants,
just
cause
like
they're,
not
just
like
a
bite
array,
but
they're,
not
that
hard.
There
are
specs
if
you're
doing
multi-formats
things
you
already
need
uvar
ants
everywhere.
Granted
the
uvara
inspect
for
multi
formats
is
like
capped
at
nine
bytes,
and
this
is
not.
But
if
you
have
all
the
code-
and
you
just
like
comment
out
the
nine
byte
restriction,
then
you
know
you're
good
to
go
and
then
yeah
floats.
A
What's
an
adl
all
right
from
our
previous
talk,
our
discussion,
codex,
decode,
bytes
and
turn
into
data
model
representation,
adls,
take
data
model
and
spit
out
our
data
model
reminder
because
the
data
model
contains
links
both
my
input
and
output
can
be.
Mult
can
effectively
be
like
multi-block
data
structures
that
I
am
referring
to.
If
the
data
model
representation
is
one
thing,
but
if
I
look
at
like
what
is
the
full
object
being
represented
or
being
talked
about,
it
can
be
a
multi-block
data
structure.
A
Bytes
are
part
of
the
data
model
too,
and
so
like
adls
are
just
a
generalization
of
codecs.
To
some
extent,
if
you
have
a
block
limit
anyhow,
codex
can
only
reference
so
much
data
and
so
on.
Adls
have
to
try
and
map
multi-blocked
to
like
single
block,
which
means
that
anyone
who
wants
to
work
with
adls
needs
like
this
partial
access
stuff,
which
is
just
like
just
interfaces
for
working
with
the
data.
C
A
What
are
adls
so
there
is
no
specification
for
adls
that
we
can
describe
like
the
data
model
specifying
helps
in
some
places
so
like
in
a
subgraph
descriptor
like
selectors
or
paths.
There
are
things
that
sort
of
imply
that
an
interface
should
be
there
right.
Pathing
implies
that
I
need
to
be
able
to
like
do
a
get
on
a
map
element
and
walk
through
it,
or
that
I
need
to
be
able
to
download
an
entire
map.
Object
right.
Selectors
also
have
like
more
expressive.
A
I
think
this
can
just
be
an
implementation
specific
thing
until
we
find
the
need
for
standardizing
these
functions
and
have
them
sort
of
live
in,
like
whatever
subgraph
descriptors
you
choose
or
other
types
of
abstract,
tooling,
you
make,
will
dictate
what
the
adl
interfaces
for
partial
things
need
to
look
like
because,
like
they
don't
need
to
be
part
of
the
spec
they're,
not
they're,
not
a
spec
thing.
What
did
I
do?
It
was
my
pld
only
so
many
hours
in
the
day.
This
wasn't
really
where
a
lot
of
them
were
going.
A
So
I
was
like
hey.
We
have
interfaces.
What,
if
I
just
copied
them
and
see
where
that
took
me
turned
out,
it
wasn't
didn't
take
very
long
in
the
scheme
of
things.
Mostly
me,
learning
rust
and
fighting
with
things
like
a
borrow,
checker,
hey,
that's
a
thing.
The
main
issues
I
encountered
were
like
no
parallel
map.
You
couldn't
get
parallel
map
access
and
only
iterators,
which
means
like
I
can't
enumerate
all
the
entries
in
a
sharded
directory.
A
In
parallel
this
seems
fixable,
others
other
things
that
I
didn't
run
into
explicitly,
but
are
there
again
from
taking
from
the
ability?
Prime
interfaces
only
bytes
have
access
to
sub-ranges
lists.
Don't
really
give
me
a
get
sub-range,
although
you
can
get
the
indices
and
that's
probably
good
enough
strings,
don't
if
you
needed
to
get
parts
of
them,
you
know
integer.
If
you
needed,
like
partials
of
any
of
those
bigger
structures.
You
couldn't
do
that
lists,
yeah,
all
right,
okay,
so
there's
one
other
thing.
A
A
A
The
files
have
names,
including
paths
they
hash.
The
chunks
of
the
hashes
of
the
chunks
are
included
in
the
metadata
block,
and
this
thing
this
metadata
is
called
info
dict,
which
one
hashed
has
the
clever
name
of
infohash
the
goal.
Can
I
load
a
bittorrent
file
the
same
way
I
do
for
uxfs
files,
and
can
I
make
it
so
that,
like
when
I
do
bittorrent,
v2
or
other
things,
it
won't
be
that
much
extra
work
all
right.
What
what
could
I
use
to
do
this,
which
ipld
pieces
might
be
appropriate
or
useful?
A
I
could
just
use
raw
nodes
codex
are
for
chumps
and
make
a
bittorrent
file
or
directory
adl
or
like
a
joint
bittorrent
adl
like
we
have
for
unix
fs.
It's
not
a
unix
file
directory,
there's
just
unix,
and
this
works
because
recall
that
adls
are
just
a
more
expressive.
Version
of
codecs
right.
Codecs
are
bytes
to
data
model.
Adls
are
data
model
to
data
model,
but
the
data
model
includes
bytes,
so
I
could
just
say
raw
nodes
and
move
on
with
my
life.
A
A
It
doesn't
have
hash
links,
it's
a
very,
very
stupid
format
that
we'll
get
to
in
a
minute
very
simple,
very
easy
to
work
with,
for
what
I
did
at
the
moment
was
sort
of
exploration.
I
made
a
bend
code
codec
and
then
a
joint
file
directory
adl
for
bittorrent
v1,
all
right
I'll
do
a
demo.
Briefly,
all
right,
I
have
a
node.
C
A
A
It's
a
bit
torn
v1
koala!
It
like
these
should
all
just
look
the
same
and
that's
like
that's
the
point
of
the
exercise.
Okay.
So
what
do
we
have
here?
This
is
for
again,
for
I
don't
know,
I
don't
know
about
you
guys,
but
I
am
of
course
fluent
in
multi
formats,
and
this
means
this
means
the
ben
code.
This
is
a
cid
of
the
ben
code
codec,
which
uses
sha1,
which
is
160
bytes.
A
Bam
there
it
is
just
works
yep.
So
that's
what
it
is
for
for
those
not
fluent
in
multi-formats
and
then
over
here
we
have
a
selector,
because
I
did
not
have
any
of
the
stuff
that
mob
showed
us
earlier
today,
and
so
my
my
solution
was,
let's
just
like
chuck
a
selector
on
the
end,
because
that
is,
as
of
today,
the
only
signaling
mechanism.
We
have
that's
not
like
the
fact
that
this
says
slash
ipfs
in
front
right,
which
is
the
signaling
mechanism,
which
means
like
hey,
try,
unix
fs.
A
C
A
B
A
A
A
Do
I
think
the
stuff
moves
working
on
his
way
better?
Yes,
I
do,
but
this
did
the
job
yeah
and
then
over.
Here
we
have
another
friend
of
ours.
Oh
no
did
I
lose
it?
Oh
no!
We
did
they
just
look
the
same.
This
is
another
friend
of
ours
also
a
koala,
but
this
one
has
a
different.
This
is
a
different
cid
from
this
one,
and
its
selector
is
different
too,
as
you
can
clearly
see
here,.
A
A
I
I
want
to
I
want
to
do
it.
I
want
to
interpret
the
cid
that
I
have
as
an
adl
and
I
would
like
once
I'm
done
interpreting
it.
I
would
like
to
mash
on
it
and
say
this
is
the
thing
I
care
about
right,
so
we
are
turning
it
into
bytes
and
then
we're
saying
I
care
about
it
and
the
gateway's,
like.
Oh
you
care
about
bytes.
A
You
know
what
looks
an
awful
lot
like
bytes
files
files
look
an
awful
lot
like
bytes,
and
then
it
renders
a
koala
for
us
this
one
over
here
is
a
little
more.
It
says
first
I
have
to
before
I
interpret
the
stuff.
I
have
to
go
in
a
little
bit.
I
have
a
field
and
that
field
is
named.
A
co-op
is
named
koala.jpeg,
because
this
is
a
directory
that
actually
has
a
panda
in
there
too.
A
If
you
look
carefully
and
then
I
want
to
select
that
and
again
this
is
a
bittorrent
v1
thing
that
we
are
traversing
so
interpret
as
bitswornv1,
then
start
traversing
it
it's
hard.
V1
directory
traverse
to
the
koala.jpg,
looks
like
bytes
load.
Bytes
get
koala
very
exciting
many
koalas.
Where
do
we
go
from
here?
A
A
You
know,
which
thing
do
I
use
to
create
the
bitcoin,
the
bits
aren't
codec
and
adl,
like
what
sequence
of
these
things
do
I
use
right.
It
feels
like
giving
people
guidance
on
like
when
to
use
them
would
like
alleviate
the
mental
burden
of
them.
Thinking
about
it,
because
sometimes
when
you
get
people
start
thinking
about
it,
they'll
like
boom
and
give
up,
and
if
you
just
tell
them
kind
of
what
to
do
in
like
90
of
cases
that
will
that'll
do
the
job
for
them.
A
Also,
probably
there's
an
extra
like
layer
to
consider
here.
If
this
is
for
files,
directories
are
a
little
more
complicated.
Everyone
in
the
world
agrees
on
like
a
file
is
bytes.
Basically,
directories
are
are
more
complicated.
You
can
notice
this
from
like
sha,
one
sum
or
sha,
two
sum
or
whatever
does
not
work
on
directories
because
there's
no
like
standard.
This
is
how
to
hash
a
directory.
A
B
B
A
A
Webassembly
becomes
like
almost
a
non-starter.
If
I
cannot
do
parallel
things,
I
need
to
be
able
to
request
blocks
from
the
network
in
parallel
without
blocking
on
one
of
them
to
come
back
linearly
like
we
have
linear
walks
in
places
in
the
go
code
and
the
go
ipld
prime
code
base
for
for
other
reasons,
and
it
it
can
hurt
us
a
bunch.
So.
A
B
A
C
C
C
A
Okay,
so
so
the
so
the
question
the
question
was:
basically
I
made
a
comment
about
how
you
could
you
could
have
for
this
and
for
other
similar
endeavors
just
have
like
have
no
codex
raw
codex
and
then
just
adls,
and
that
you
know
sometimes
you
want
to
have
these
things.
Sometimes
you
may
want
these
things
to
be
reasonably
coupled
together
and
so,
like
sort
of.
A
Is
that
sometimes
that
partial
piece
is
what's
useful
to
you
so
like
as
an
example,
there
are
folks
who
are
like
I
kind
of
wish
that,
like
I
had
a
I
want,
you
know
a
an
abstraction
layer
that,
instead
of
spitting
out
taking
bytes
and
mapping
to
data
model,
took
bytes
and
mapped
out
to
set
of
links
right,
and
that
is
a
subset
of
the
data
model.
Does
that
mean
that's
a
separate
abstraction
like
and
so
there's
there's,
maybe
some
use
for
that
abstraction
separately
from
the
data
model.
A
Abstraction
because,
like
maybe
you
don't
need
or
want
all
of
it
and
that's
kind
of
where
things
go
here,
which
is
like.
Maybe
what
I
want
is
like.
I
could
reasonably
draw
a
line
and
say
the
thing
that
I
want
for
codex
is.
I
want
codex
to
be
able
to
represent
all
of
the
links,
and
I
would
like
as
much
as
possible
that
when
you
run
adls
you
do
not
create
new
links.
A
That
could
be
like
an
approach
I
take
and
the
benefit
I
would
get
out
of.
That
is
to
say
that.
Well,
if
you
have,
if
it's
a
commonly
supported
codec,
but
the
adl
is
more
sophisticated,
then
you
can
just
pin
all
that
you
could
like
request
and
pin
and
walk
the
graph
of
all
the
blocks
that
you
need
to
touch
without
knowing
about
the
adls
and
then
this
starts.