►
Description
Conflict-free Replicated Data Types (CRDTs) are a hot new datatype in distributed programming that promise coordination-free and always eventually-consistent writes. Most of the stuff you will read about them is limited to the context of high-end distributed databases, but CRDT's are more flexible and shouldn't be limited to this field. In the talk, I will demonstrate how CRDT's are great for applications that have partial connectivity (including websites): updates can be applied locally, and when communication is possible, you can send the data back up, and the data will remain consistent even in the most byzantine (or apocalyptic) scenarios. There are even scenarios that can support multiple simultaneous editors.
Beyond that, I will also demonstrate how Elixir's metaprogramming can be used to compose complex models out of CRDT's that themselves exhibit the same exact features. I will also exhibit some newer CRDT features, such as shared-context and delta-operation CRDT's to overcome some of the shortcomings of older CRDT's.
I plan to keep the talk light on theory (the academic literature is sufficient for that).
Help us caption & translate this video!
http://amara.org/v/HO21/
A
You're
giving
talks
about
how
CRT
teens
aren't
advantage
or
the
apocalypse
so
few
disclaimers,
zero
duties
are
strange.
Knowledge
is
bar
logic.
Encourage
learners
on
the
academic
papers
are
very
hard
to
understand.
There
are
very
few
general
purpose:
UNC
libraries
out
there
and
most
people
are
very
interested
in
them,
but
can
barely
understand
them.
A
My
hope
is
that
some
of
you
might
be
interested
in
implementing
some,
but
most
likely
I
want
to
I
want
to
get
people
interested
in
actually
using
them.
I
do
have
a
library
on
hex
called
loom,
it's
a
bit
older,
but
it
supports
the
some
of
the
new
Delta
crd
teas
that
are
recent
and
it
works.
I
have
a
new
API,
that's
coming
out
soon,
but
it
won't
be
ready
for
till
Christmas.
A
A
A
There
are
two
flavors
one
of
them's
commutative
data
types
and
those
are
the
log
based
ones
which
you
just
keep
a
log
of
all
the
updates
that
happen,
and
you
read
over
the
logs
and
apply
the
operations
and
spit
out
a
result,
and
then
there's
once
called
state-based,
which
are
a
lot
easier
to
reason
about,
but
they
they
can
get
very
large
and
it's
very
hard
to
it's
very
hard
to
ship
very
large,
dr
duties
around.
They
can
lead
to
downtime.
Just
like
a
garbage
collection
pause
in
general.
You
issue
an
update
operation
to
mutate.
A
A
Crd
T's
are
trivial
to
communicate
over
a
network.
The
previous
properties
are
great
because
you
can
always
repeat
a
message:
send
it
five
times
send
stuff
from
the
past
send
stuff
out
of
order,
and
as
long
as
the
latest
messages
are
communicated,
you
will
arrive
at
a
consistent
state.
Once
everyone
sees
all
the
messages
and
in
general,
as
long
as
you
have
reasonably
recent
updates,
they
all
the
answers
from
all
your
different
servers
will
be
close.
A
A
There's
a
bug:
okay,
so
flags
are
bullying's
that
are
either
true
or
false.
There
are
counters
they
go
up
or
down
sets
maintain,
maintain
some
list.
There
are
registers
which
maintain
one
thing
most
of
the
time.
Some
of
them
retain
more
than
one
thing.
In
special
cases
there
are
maps
which
associate
a
name
or
a
term
with
a
value
map
side,
the
interesting
property
that
many
of
them
can
nest.
A
Crd
t's,
a
great
thing
about
that-
is
that
you
can
nest
maps
and
maps
in
most
cases,
and
you
can
create
very
large
rich
objects
that
you
can
pass
off
to
the
to
another
system,
and
all
of
your
updates
will
be
preserved
pretty
well.
There
are
also
graphs.
They
help
manage
a
graph
system,
though
the
semantics
for
those
get
kind
of
wonky
in
some
cases-
and
there
are
also
crdt
documents,
though
I-
would
not
look
at
using
them
at
all.
The
storage
requirements
involved
are
often
ten
times
or
more
for
collaborative
editing.
A
A
So
here's
some
examples
of
some,
so
you
can
actually
see
kind
of
how
they
work.
I
wish
I
had
a
laser
pointer,
but
I
couldn't
probably
point
anyway.
So
there's
a
G
counter.
It
counts
up
so
mostly
most
easily
implemented
as
a
simple
map,
single
actor
increments,
their
own
ID
in
the
map
and
the
and
when
they
actors
gonna.
Let
me
take
their
own
value
and.
A
A
Let's
see
if
I
could
actually
whoops
point
so
here
we're
adding
one
array
is
out
adding
one
to
its
counter.
Then
it
adds
two.
So
it's
three
here.
This
is
a
communication
down
here
and
you
can
see
just
the
marriage
has
both
properties,
as
the
communication
goes
on
and
what
happens
when
there
is
an
out
like
say
there
was
an
out
of
order
message
and
this
one
was
delivered
straight
over
here,
because
you're
taking
the
maximum
the
red
one
could
not
overwrite
the
red
three.
A
The
red
three
would
remain,
and
so
all
of
these
messages
can
arrived
out
of
order
in
any
order
and
then
at
the
end
you
just
are.
Whenever
you
want
to
read
the
value
you
sum
up,
each
individual
item
in
the
set
and
you're
done,
one
of
the
most
simple
kinds
of
CRT
tease.
But
there
are
some
problems
that
only
goes
up.
You
can't
go
down,
but
you
can
get
around
this
by
using
two
of
them
together:
14
increments
and
14
decrements,
and
then
you
subtract
one
from
the
other.
This
is
called
APN
counter.
A
This
is
the
kind
of
contortions
you
have
to
do
in
order
to
actually
maintain
crdt
properties
and
in
a
real-life
application.
So
this
is
a
girl
only
set,
which
is
interesting.
If
you
restrict
set
operations
to
only
add
sets
RCR
DTS,
it's
pretty
simple.
If
something
already
exists,
it
exists
problem.
Obviously,.
A
A
The
interesting
part
here
happens
when
I
take
bread
off
the
list
in
this
one's
copy.
Reds
back
here,
I
removed
the
bees
bread
and
then,
when
I
merge
back.
What
happens
is
I
ever
for
every
event.
I
increment
this
counter
and
each
value
is
tagged
with
a
counter
and
if,
when
I
merge
this
with
that
in
the
algorithm
I
check,
if
they
both
should
have
seen
the
update
for
bread
because
they
both
did
because
they
both
didn't
see
it
only
this
one
still
sees
it.
A
That
means
that
someone
must
have
removed
it,
which
means
it
can
disappear
from
the
state
forever
and
as
long
as
I
keep
these
clocks
consistent.
Every
event
that
happens.
I,
add
one
to
this
clock
and
I
tag.
The
values
correctly
I
can
add
and
remove
things
that
will
and
what
you
have
to
do
is
you
have
to
just
decide
on
a
total
order
for
add
and
remove,
there's
a
add/remove
wins
and
a
and/or
edwins
and
remove
wins,
and
then,
when
an
ad
wins.
A
A
This
is
kind
of
like
a
vector
clock,
but
not
really,
there's
a
slight
difference
and
I'll
go
over
them
really
quickly.
So
that's
what
I
explain
each
value.
Each
actor
has
a
value
in
time.
For
each
event,
each
actor
starts
at
zero.
You
bump
on
each
internal
event.
You
have
to
bump
the
counter
and
you
tag
the
value
in
vector
clocks.
A
A
Everything
here
is
in
the
past.
So
when
I
remove
an
element
from
a
set,
that's
tagged
in
this
blue
area.
I
know
it's
not
there
because
it
is
formed
the
causation
into
here,
because
we
don't
really
have
a
unified
time.
Everything
is
actually
based
on
causation.
It's
not
time
as
a
fiction.
Basically,
especially
your
multiple
servers.
Time
can
move
around
on
the
server
itself.
A
Everything
after
here
is
an
effect
of
what
you're
doing
and
then
everything
we're
not
every
value
is
larger
than
the
other
one
or
not
every
value
smaller
than
the
other
one.
These
relationships
are
concurrent
operations,
and
this
is
even
the
case
if
something
doesn't
communicate
for
three
days.
It
might
have
happened
two
days
ago.
It's
still
a
concurrent
operation
because
they're
not
causally
related.
A
A
A
Yes,
this!
This
is
just
like
kind
of
like
reasoning
through
relativity
officer.
The
reason
why
the
word
observation
is
used
is
that's.
What
happens
is
based
on
what
you
see
and
there's
some
pseudo
paradoxes
with
Einstein's
theory
of
relativity,
where
you
can
just
kind
of
you
get
things
that
seem
like
paradoxes.
Never
interesting,
I
went
over
that
see.
A
A
So
there's
a
feature
called
Delta
crd
T's,
where
you
can
actually
take
an
empty
crdt
kinda.
This
is
the
kind
of
methodology
it's
slightly
different
and
apply
operations
to
those
and
instead
of
using
a
dot
context
or
o'clock,
you
have
a
dot
cloud
which
is
just
the
little
atomic
pieces
of
the
future
that
that
Delta
represents,
and
you
can
ship
just
that
Delta
and
if
everything
is
actually
contiguous.
What
will
happen
is
you'll
be
able
to
ship
just
a
subset
of
your
state
changes,
which
is
amazing.
A
It
solves
one
of
the
biggest
drawbacks
of
state-based
co.
Tt's
there's
another
feature
which
is
incompatible
but
also
desirable,
which
is
upsetting,
is
we
can
actually
support
causation
in
a
very
large
nested
crdt
structure
if
we
share
the
context,
so
if
I
were
to
pass
that
context
down
through
some
nested
maps
and
then
way
back
up,
I
could
actually
do
a
lot
of
reasoning
about
the
kinds
of
things
that
change
and
it
would
be
very
easy
to
support
very
complex
data
structures,
but
it's
not
not
easily
compatible
with
delta
CRT
T's.
A
A
One
of
the
things
I
really
like
about
the
elixir
community
myself
is
a
lot
of
the
community
values
where
we
want
to
be
able
to
reach.
You
know
people
at
a
lower
skill
level,
not
only
that,
but
not
only
for
that
sake,
but
because,
when
I
don't
have
to
think
about
something,
I
can
focus
all
my
mental
energy
on
something
actually
hard
that
I'm
interested
in
CR
dt's
really
should
be
a
means
and
not
an
ends.
A
A
A
They
implemented
a
remove
winds
version
of
that
and
they
were
very
pained
at
having
to
implement
this
and
they
actually
implemented
in
Lua
on
top
of
Redis.
So
they
didn't
have
to
do.
They
had,
they
would
have
to
do
the
least
amount
possible
and
they
would
use
Redis
operations,
I
think
to
actually
send
the
updates
I
think
the
status
quo
on
this
can
be
broken.
A
One
of
the
strategies
is
Christopher
Michael
Johns
last,
but
we
I
think
there's
some
also
some
other
use
cases.
We
can
get
some
really
quick
wins
on
that,
aren't
necessarily
the
most
efficient,
but
will
work
in
most
people's
use
cases,
but
some
of
the
solutions
will
require
rich
client
contacts
like
single
page
applications,
native
apps,
etc.
A
You're
going
to
have
to
keep
track
of
state
in
the
client
and
apply
updates
yourself
for
some
of
them
more
complicated
and
nicer
things,
but
the
nice
part
is,
is
you
can
actually
defer
your
updates
and
apply
operations
locally
on
the
device
or
in
the
single
page
application,
and
not
wait
for
a
response
from
the
server
to
get
a
source
of
truth?
So
you
can
actually
make
very
fast
progress
within
the
application.
It's
a
big
benefit.
A
It
kind
of
works,
I
kind
of
took
inspiration
from
the
optimistic,
optimistic,
locking
support
inept
Oh.
So
what
this
actually
does
is
it
creates
an
optimistic
lock
that
can
retry
and
reapply
operations
in
the
face
of
conflict
for
ecto
and
into
the
database,
and
so
basically
in
basically
how
it
works.
A
Probably
easiest
about
this
line,
so
we
add
a
crdt
and
we
just
by
narrated
tournament
code
and
we
sign
it
with
an
h
mac.
So
no
one
can
edit
it
stick
it
in
an
hidden
field
in
a
form
and
in
phoenix
we
can,
when
the
user
submits
the
form,
we
can
dip
the
new
values
from
the
old
ones
in
the
crdt,
when
it's
backed
up
to
the
server
and
the
co
TT
might
have
changed
in
the
database,
but
we're
using
the
copy
that's
submitted
with
the
user.
What
happens
after
that
is?
A
We
can
do
a
diff,
find
the
differences,
extract
operations
and
apply
them
back
on
the
crdt.
That
sounds
complicated,
but
what
we
get
is
whenever
we,
the
third
statement
there
is
the
update.
That's
an
optimist,
lock,
update
where,
if
the
constraint
is
violated,
we
actually
fail
to
insert
so
you
can't
actually
insert
your
new
values.
A
In
time,
so
you
can
still
preserve
some
of
this
stuff
and
keep
it
in
the
very
simple,
very
simple
state,
and
what
what's
really
great
about
this
is:
if
I
go
back,
I'm
still
preserving
on
my
field,
so
I
can
still
run
complex,
select,
queries,
I,
don't
give
up
anything
I
can
still
do
all
the
stuff
I
want
to
do.
The
only
thing
I
can't
do
is
just
insert
directly
without
touching
the
crdt
everything
else,
though,
with
select
statements
and
complex
reporting
I
get
that,
and
it
is
great.
A
A
So,
let's
see
a
lot
of
the
crd
tease
only
become
really
possible,
though,
whenever
we
actually
have
more
cooperation,
because
elixir
is
so
locked
in
on
the
server
and
we
don't
really
have
elixir
iphone
apps,
yet
we're
actually
going
to
need
to
rely
on
some
other
technologies
to
catch
up
before
we
can
really
take
off
with
crd
tease.
So
things
on
my
wish
list
that
we
really
really
need
I
need.
I
I
really
want
a
good
gossip
protocol,
that's
freed
from
another
earling
project
and
doesn't
have
really
complex
dependencies.
A
I,
don't
want
to
include
an
entire
project
just
to
get
its
casa
protocol.
An
external
representation
for
these
crdt
types
is
really
important
and
really
difficult
to
reason
about,
because
they
don't
fit
neatly
into
Jason
and
a
lot
of
the
other
nicer
stuff
nicer,
serialization
formats
they
don't
really
have
much
by
in.
So
we
might
have
to
implement
something
on
top
of
Jason
that
we
would
just
run
a
quick
map
over
and
be
able
to
transform
that
into
something
we
can
use.
A
You
all
would
be
a
great
way
to
do
things,
because
you
can
include
the
crdt
context
with
your
updates
and
you
can
actually
apply
instead
of
inferring
the
events,
you
can
actually
apply
events
directly
from
the
user
and
be
in
the
state
if,
if
they're
well
tested,
the
state
is
going
to
be
well
guarded.
Having
the
browser
be
a
lot
more
stateful,
it's
a
lot
safer
when
you're,
whenever
you're
in
crdt
land.
A
Work
for
some
short
period
of
time
in
in
emergency
operations,
not
an
emergency
operation
center,
but
I
was
working
with
a
vendor
who
sold
stuff
to
emergency
operation,
centers
the
stuff
they
use
and
the
stuff
that
contract
out
is
held
together
with
duct
tape.
It's
very
poor.
The
interfaces
look
like
they're
about
15
years
old
and
probably
are
I,
don't
think
any
of
the
I.
Don't
think
many
of
the
vendors
at
all
higher
permanent
programmers,
except
just
to
support
the
new
version
of
Windows.
A
They
don't
really
communicate.
Well,
they
assume
your
emergency
operation
center
is
online
and
if
you
remember,
during
Hurricane
Katrina
and
I've
read
some
of
the
retrospectives
recently
there's
been
ish
that
one
of
the
biggest
issues
was.
There
was
no
communication.
This
the
cell
phone
networks
are
down
and
they
would
send
someone
out
to
go
find
what
would
happen
and
it
would
disappear
for
two
three
four
days
while
they
were
helping
solve
problems.
But
none
of
the
decision-makers
up
the
tree
knew
what
was
happening
on
the
ground.
Just
people
would
disappear.
A
A
One
use
case
that
would
be
really
great
is
once
you
have
er
DTS
on
the
phone
you
can
communicate
peer-to-peer.
If
you're
worried
about
security,
you
can
sign
everything
with
CA
certificates
and
there's
a
certificate
chain.
You
can
solve
that.
These
are
things
that
we
can
go
grab.
Libraries
off
the
shelf,
the
missing
part
is
actually
the
crd
tees
and
not
many
languages
actually
have
them
and
they're
one
of
the
ways
that
we
could
probably
save
people
a
lot
of
pain.
A
You
get
a
lot
of
resiliency
and
speed
and
you
can
defer
synchronization
for
days
on
some
of
the
models
to
where
there's
not
even
a
business
case.
For
you
know,
if
you've
only
got
like
50
or
60
people
in
your
organization
and
no
one
communicates
for
a
week,
and
then
everyone
comes
into
the
office
and
everyone
crdt
sync
up
together.
A
A
One
of
the
things
like
I
actually
discovered
reading
some
of
her
blog
post
was,
you
can
actually
apply,
what's
called
inflationary
inflationary
updates
without
actually
doing
any
synchronization,
and
you
don't
need
to
preserve
the
commutative
and
associative
properties
of
your
updates.
So
one
example
would
be
in
the
Phoenix
presence
say
you
want
to
protect
against
join
floods
and
you're
sinking,
all
the
server's
every
two
or
three
seconds,
and
so
what
you
can
do
is
when
our
user
joins.
A
You
have
a
local
set
that
you
can
add
and
remove
from
naively
and
then,
when
the
three
seconds
are
up,
you
collapse
all
those
and
merge
them
into
the
crdt
set
as
an
operation.
If
someone
joins
the
channel
10
times
and
leaves
channel
10
times
what
happens?
Is
all
those
events
get
collapsed
down
by
adding
removing
to
just
that
set
to
at
most
one
and
so
at
most
once
they'll
be
able
to
flood
the
channel
with
a
join
or
leave
message,
which
is
a
really
great
property?
A
And
so
I've
heard
a
thing
for
that
christopher
michael
john.
He
has
a
about
twice
a
year,
so
he
publishes
all
the
papers
on
CRT
teas
that
every
that
are
going
around
in
academia,
carlos
peguero.
I'm
really
indebted
to
him
for
the
hour.
Swat
stuff
he's
really
pushed
that
far.
The
react,
ET
source
is
really
interesting.
A
That's
reacts.
Data
types,
that's
the
other
main
inspiration
for
the
current
library.
That's
in
hex
called
loom,
and
this
is
a
summary
paper
that
goes
in
a
lot
more
detail,
and
while
they
have
a
lot
of
Greek
Greek
letters
and
everything,
they
also
have
really
good
diagrams
in
the
paper
and
that's
pretty
much.
It.