►
From YouTube: Gavin & Matthijs - Rust as foundation in a polyglot development environment — RustFest Global 2020
Description
by Gavin Mendel-Gleason & Matthijs van Otterdijk
TerminusDB (open source versioned collaboration database) is a small polyglot software house that switched from C++. The change has enabled big improvements in speed and reduction in hunting for hard-to-find re-entrance, locking and memory safety issues which used to plague us.
More at https://rustfest.global/session/20-rust-as-foundation-in-a-polyglot-development-environment/
B
That's
an
excellent
introduction,
so
I'm
gavin
mendel
gleeson,
I'm
the
cto
for
terminus
tv,
and
I
wanted
to
talk
a
little
bit
today
about
rust
as
a
foundation
in
a
polyglot
development
environment.
So,
first
I'm
just
going
to
give
a
little
outline
of
the
talk
as
with
the
motivation
challenges
in
solution.
B
So
first
you
have
to
know
a
little
bit
about
our
problems,
so
we're
an
in-memory
collaborative
revision
control
graph
database.
So
we
have
fairly
specific
requirements
and
we
we
have
slightly
unusual
features
so
and
that
has
driven
some
of
the
tool
chain
requirements
that
we
have.
Our
software
is
also
it's
a
very
polyglot
house,
so
we
we
have
clients
written
in
javascript
and
in
python
we
have
rust
and
we
have
prologue,
which
is
somewhat
unusual
in
the
modern
day.
B
So
we
also
there's
actually
also
c
and
c
or
yeah.
There's
also
c
involved
there
as
well.
So
some
of
the
unusual
features
that
drive
our
design
requirements
so
we're
an
in-memory
database.
So
that's
that
enables
faster
query.
It's
also
simpler
to
implement,
and
I
I
have
some
experience
in
implementing
on
acid
databases,
and
so
I
know
a
lot
about
the
difficulties
that
you
can
encounter
when
trying
to
to
you
know
paging.
B
B
But
this
requires
a
lot
of
bit
twiddling
so
they're
relatively
complicated
data
structures
and
they're
they're,
very
compact,
but
they
also
they're
not
so
transparent
to
the
developer.
So
you
really
need
to
be
able
to
do
effective
bit
twiddling,
which
of
course
is
where
rust
comes
in.
So
we
also
have
a
bunch
of
git
like
features,
so
we
have
revision
control,
we
have
push,
pull
clone
and
all
of
the
things
that
you
know
from
git.
We
do
those
on
databases,
so
that
also
drives
a
lot
of
our
requirements.
B
We
have
a
data,
log
query
engine
and
we
also
have
complex,
schema
constraint
management.
So,
first,
why
did
we
look
into
rust
in
the
first
place,
so
we
were
not
initially
a
rust
house.
We
didn't
have
any
rust
in
our
in
our
development
at
all.
I
didn't
come
from
a
rust
background
and
although
I
have
a
lot
of
experienced
different
programming
languages,
rust
was
not
one
of
those
programming
languages,
so
our
earliest
prototype
is
actually
in
java.
B
It
was
hard
to
write
and
it
had
mediocre
performance,
and
so
I
started
prototyping
something
in
prolog
and
the
ability
to
write
it
in
prolog.
Actually
because
it
is,
it
was
very
logical,
especially
the
schema
checking
parts
of
it.
It
was
extremely
fast
for
us
to
write
it
in
prologue.
However,
it
had
poor
performance,
so
obviously
it's
not
the
best
for
bit.
Twiddling
later
we
moved
to
a
library
in
c
plus
plus
called
hdt,
and
we
used
that
as
our
storage
layer
and
that
radically
improved
the
performance
of
the
application.
B
B
So
then
there
was
a
secondary
problem,
which
is
that
htt
was
not
designed
for
right
transactions,
so
it
was
really
designed
for
data
sets
and
not
databases.
So
we
were
using
extra
orchestration
logic
on
top
of
it
to
actually
use
where
we
would
journal
transactions
and
stuff
like
that
in
order
to
make
it
so
that
it
could
be
a
transactional
database,
but
it
wasn't
designed
that
way.
B
So
matthias
off
his
own
bat,
went
out
and
wrote
a
prototype
in
rust
of
the
succinct
data
structures
that
we
needed
to
replace,
http
and
sort
of
like
a
simple
library
around
it,
and
it
looked
really
very
promising.
So
I
had
not,
I
had
heard
of
rust,
but
I
hadn't
actually
written
anything
in
rust,
and
so
this
really
drove
me
to
take
a
look
at
rust
and
rust.
Like
I
know,
a
lot
of
languages,
I've
learned,
o'camel
c,
plus
plus
you
know
haskell
prologue
lisp.
B
You
know
I've
been
through
the
gamut
of
all
of
these.
I
I
don't
usually
try
to
learn
a
new
language
unless
there's
something
really
peculiar
that
drives
it
as
something
that
you
might
need
in
your
tool.
Kit
and
rust
really
had
this.
This
kind
of
incredible
aspect
to
it,
which
is
this
this:
the
ability
to
avoid
memory
problems
while
still
being
extremely
low
level,
programming,
language,
so
thread.
Safety
was
one
of
our
major
headaches
that
we
had
re-interest
headaches.
B
We
were
getting
seg
faults
and
this
was
a
problem
and
we
were
finding
it
difficult
and
time
consuming
to
sort
them
out,
and
this
library
was
exhibiting
none
of
these
problems.
So
this
is
this
was
really
promising
and
we
decided
you
know
we
were
just
going
to
take
the
plunge
and
rewrite
the
foundations
of
our
system
in
rust.
B
We
also
ended
up
trampolining
through
a
light
seashin,
and
this
is
probably
not
the
best
approach-
we're
we're
evaluating
a
much
more
direct
approach
currently,
so
I
won't
try
to
tell
everybody:
we've
done
it
right.
We've
done
some
things
right,
but
I
think
we
we
can
improve
a
lot
here
now.
What
we
would
really
like,
though,
is
a
rust
prolog,
because
then
we
could
have
a
nice
clean,
rust,
ffi
and
everything
would
be
beautiful
and
perfect.
B
We
essentially
rewrite
the
kind
of
feature
that
we're
interested
in
there
and
then,
instead
of
just
immediately
going
to
rust
from
there,
we
actually
wait
so
we're
much
more
selective
about
what
we
put
into
rust
than
we
had
initially
imagined,
and
partly
this
is
due
to
the
learning
curve
of
borrow
checking
semantics,
meaning
that
there's
there's
a
difficulty
in
getting
our
developers
to
understand
how
this
stuff
works.
So
that
takes
some
time
and
it
it.
B
But
if
you're
replacing
prologue,
the
sort
of
amortized
costs
are
more
important.
So
you're
gonna
have
to
worry
about
where
you
replace
it,
and
you
have
to
be
more
careful
about
that
once
you've
gotten
the
the
knack
of
the
borrow
checker
things
go
a
lot
faster,
however,
they're
still
slower
than
writing
prologue,
because
you
just
have
to
worry
about
more
things
in
rust
than
you
do.
It's
a
lower
level
language,
which
is
why
we
use
it.
But
it's
also
it's
why
we
don't
always
use
it.
B
B
So
these
include
things
like
you
know,
patch
application
and
squash
operations
and
things
of
that
nature.
So
these
are
larger,
orchestrated
they're,
not
as
low
level,
so
they
have
logic
in
them
and
we're
doing
it
there.
Now
we
also
have
done
some
bulk
operations
that
in,
for
instance,
csv
loading
has
now
been
written
completely
in
rust
as
well
and
this
because,
if
you
have
hundreds
of
thousands
of
rows
in
your
csv,
you
get,
we
get
probably
about
a
10
to
20
times
speed
up
from
going
from
prologue
to
rust.
B
B
So
one
of
the
things,
though,
that
we
we
ran
into
is
a
sort
of
unexpected
bonus
and
we
kind
of
knew
this
was
here
but
are
amazingly
impressed
with
it.
Nevertheless,
so
this
is
the
unexpected
bonus
round,
so
we
got
data
parallelism
from
switching
to
rust
at
a
very
low
cost
using
rayon,
and
it
really
blew
our
minds
because
we
had
things,
we
hardly
changed
at
all.
B
We
had
the
logic
already
written
there
and
we
just
used
these
magic
incantations
into
pirate
or
par
bridge
par,
sword,
unstable
and
suddenly
everything
is
way
way
faster
and
we
didn't
have
to
think
about
it
that
hard.
We
and
I
love
that,
because
I'm
lazy
so
anything
that
that
can
reduce
the
amount
of
time
that
we
spend
writing
things,
while
also
improving
performance
is
just
a
huge
win,
and
I
I
cannot
impress
upon
people
enough
how
awesome
this
is
and
how
much
we
need
other
people
to
start
using
it.
B
B
C
D
So
no
debug
is
definitely
not
fast
enough.
Well,
I
mean
it
is
fast
enough.
It's
fast
enough
when
we're
just
testing
out
things
and
it's
great
sometimes
to
be
able
to
use
a
debugger
or
something
but
like
an
actual
general
use,
also,
while
we're
developing
and
when
we're
not
developing
the
low
level
library,
we
definitely
build
and
release
mode
always,
and
it
is
a
tremendous
speed
up
between
them.
C
D
C
B
Well,
like
I
mean
it's
a
it
types
just
to
introduce
extra
overhead
and
like
in
prologue,
you
have
you
don't
have
to
worry
about
garbage
collection.
You
don't
have
to
worry
about
how
you
allocate
things.
You
just
have
fewer
things
to
worry
about,
and
that
costs
you
later.
It
costs
you
in
terms
of
performance,
but
it's
really
helpful
in
terms
of
developer
time
and
lots
of
things.
It
doesn't
matter
really
what
the
constant
time
cost
is,
because
you
know
it's
just
glue
like
most.
B
D
There's
there's
another
thing
here
which
is
to
compare
with
prolog
in
proloque.
You
would
generally
have
a
running
prolog
instance
with
all
your
codes
with
your
entire
server
running
in
there,
and
then
you
do
like
live
recompilation
of
parts
of
that
program.
So
it's
a
very
short
loop
between
actually
writing
your
code
and
seeing
it
in
action
with
rust.
C
B
Yeah,
so
there's
sort
of
like
machine
learning
where
you're
trying
to
where
you
need
to
have
revision,
control
of
your
data
sets
and
there's
any
kind
of
like
large-scale
graph
manipulation
problems
or,
if
you
want
to,
if
you,
if
you
want
to
keep
revisions
and
and
be
able
to
pipeline
your
data,
that's
that's
where
we
would
use
it
and
we
scale
up
to
quite
large
graphs,
so
you'd
be
able
to
stick
something
large
in
there.
If
you'd,
like.
C
Okay
and
I
think,
we're
running
sort
of
out
of
time-
I
see
like
postponed
messages
popping
up.
Will
you
both
be
active
in
the
chat
still
for
to
hang
around
for
so
answer
some
questions
I
see
otherwise
you're.
Finally,
in
the
chat
as
well,
so
we
have
some
technical
difficulties,
sometimes
which
one
does
with
this
online
experience.
I
would
say,
although
it's
also
a
kind
of
fun
experience
until
now,
I
have
to
say
yeah.
C
I
want
to
thank
you
both
so
much
for
your
time
and
also
the
interesting
presentation
and
please
do
check
out
the
chat,
and
then
I
see
that
in
eight
minutes
we
will
start
the
new,
the
next
speaker
already.
So
that
will
be
interesting
as
well,
so
please
also
for
the
people
watching
the
live
stream
stick
around
for
that,
so
we'll
be
back
in
eight
minutes.
I
would
say-
and
thank
you
so
much
again-
kevin
and
my
dice
great.