►
Description
GStreamer is a highly versatile, cross-platform, plugin-based multimedia framework that caters to the whole range of multimedia needs. It can be used basically everywhere, from embedded devices like phones, TVs or drones to desktop applications or on huge server farms.
In this talk we will discuss how Rust is the perfect match for GStreamer to evolve from its C roots and safely enter the future. Be it for application development on top of GStreamer or for the development of plugins, where the actual media processing is happening. We will give an overview of the current status of getting Rust and GStreamer working together, our experience with Rust so far, what problems we ran into and what is already possible today. In the end we will give an outline of our ideas how to evolve from here, what the next steps are on the way to world domination.
And who knows, maybe in the future we will have a GStreamer completely written in Rust?
Sebastian Dröge
https://twitter.com/sdroege_
https://github.com/sdroege
https://users.rust-lang.org/users/slomo/activity
Luis de Bethencourt
https://twitter.com/luisbg
https://github.com/luisbg
https://users.rust-lang.org/users/luisbg/activity
A
B
B
The
family-owned
part
is
actually
true.
I
had
the
open
source
group
here
in
Europe
and
I
just
wanted
to
say
that
we're
hiring
so
if
you're
interested
so
I've
been
doing
mostly
GStreamer
and
Linux
kernel
development
for
the
last
few
years
and
then
I
help
with
advising
different
teams
and
bridging
communities
and
stuff.
So
I
love
giving
talks
like
this
and
as
both
of
us
in
our
work,
we
do
December
trainings,
so
we're
going
to
do
something
that
would
usually
taken
an
entire
day
we're
trying
to
compress
it
into
30
minutes.
B
B
B
So
the
goals
of
GStreamer
is
to
be
flexible
through
through
this
concept
of
just
having
elements
you
can
shuffle
around
extremely
extensible
which
service
is
going
to
cover
in
terms
of
if
there
is
something
that
isn't
covered
by
this
little
atomic
element.
You
can
add
it
really
easily
and
just
focus
on
your
particular
missing
need,
and
it's
really
easy
to
integrate
with
other
software
in
both
the
direction
of
applications
that
want
to
do
multimedia
things
or
the
underlying
components
of
a
multimedia
stack.
B
It
gives
you
a
common
API
that
doesn't
really
exist
if
you
want
to
use
all
these
different
formats,
the
API,
so
the
mp3
decoders
exposed
is
very
different
ones
and
other
formats
or
other
hardware
devices
will,
but
using
your
microphone
in
your
laptop
and
using
an
mp3
decoder
it
as
a
application
developer
in
gstreamer,
it's
exactly
the
same
API
and
it's
extremely
stable
API.
It's
always
compatible
in
terms
of
once.
B
You
write
that
software,
the
API
is
never
going
to
change
under
you
and
unless
we
have
like
a
major
release
that
only
happens
once
every
six
years.
So
that's
not
really
a
problem,
so
you
have
to
guarantee
that
things
are
not
going
to
be
shuffled.
The
rug
is
never
going
to
be
under
you,
it's
community
driven.
B
So
in
the
last
year
there
was
235
active
contributors
to
the
project
which
I'm
really
happy,
because
part
of
my
job
is
to
be
kind
of
the
cheerleader.
So
I'm
happy
when
the
community
is
vibrant
and
it's
full
of
people
over
the
entire
lifetime
of
GStreamer
the
now
what
14
years
the
project
has
existed.
A
B
So
in
terms
of
like
embedded
devices,
you
will
have
everything
you
need,
but
the
cross-platform
problems
you
would
have
in
other
object-oriented
programming
models.
Would
you
just
don't
have
that
problem
and
we
both
have
experienced
that
running
GStreamer,
an
extremely
small
and
kind
of
obscure
operating
systems,
and
it
works
pretty
fine.
The
most
notable
ones
are
Linux
Windows,
Mac,
OS,
VSDs,
Android
iOS
and
those
all
have
like
really
good
community
support,
I'm
missing
anything,
ok,
doing
a
good
job
so
far
being
a
long-term
and
stable
project.
B
It
has
bindings
for
all
the
languages
that
you
would
expect
and
they
all
try
to
be
as
idiomatic
as
possible,
which
we're
going
to
talk
a
little
bit
later,
how
that
kind
of
links
with
rust,
there's
hundreds
and
hundreds
of
plugins.
So
the
thing
I'll
showing
you
before
that
had
just
like
five
plugins
there's,
hundreds
of
those
that
can
give
you
different
features,
I
like
to
play
the
game
of
I'm,
something
that's
already
not
supported
by
an
element
and
then
I
try
to
find
somebody
that
can
contribute
that
code.
It's
like
a
hey!
B
Why
don't
you
do
this
as
a
first
contribution
it
has
batteries
included
so
the
moment
that
you
install
it.
Do
you
already
have
certain
tools
and
help
like
there's
a
very,
very
useful,
command-line
tool
that
just
lets
you
play
with
pipelines
from
the
command-line,
and
you
can
just
experiment
without
even
having
to
write
code?
You
can
just
do
things
from
your
usual
shell
and
it's
always
good
to
clarify
things
in
terms
of
the
usual
misconceptions
around
GStreamer.
It
is
not
a
media
player
or
a
playback
library.
B
It
is
not
a
codec
and
protocol
library
you're,
looking
more
for
like
SSM
peg,
if
you
want
to
look
for
specific
codecs,
GStreamer
sits
on
top
of
SSM
peg,
so
you
get
all
the
benefits
of
the
work
that
amazing
community
is
doing
in
a
more
stable
API
that
connects
with
other
things.
It's
not
a
transcoding
tool,
even
though
it's
used
a
lot
server
side
for
transcoding,
and
even
though
the
name
is
a
bit
of
a
misnomer
and
confuses
people
in
this
regards
it's
not
streaming
as
in
the
classic
like
VOD
or
music
streaming.
A
Yeah
so
I'm.
The
big
question
now
is
the:
why
rust?
Why
would
we
want
to
use
rust
for
this
stuff,
and
this
is
I
think
the
main
reason
why
we
should
look
into
rust
and
why
we
actually
looked
into
rust.
So
what
happens
in
GCM
is
that
we
have
to
handle
very
complex
data
formats
that
come
from
somewhere.
You
have
no
idea
where
they
come
from
might
be
from
some
malicious
source
might
be
from
you,
don't
know
and
handling
these
media
formats
is
quite
complicated
and
there
are
bugs
everywhere
and
like
in
2016-2017.
A
A
Well,
while
this
gives
you
some
kind
of
safety,
nobody
can
just
execute.
When
quote
on
your
machine
anymore,
it
also
makes
us
user
more
happy.
Probably
you
don't
really
want.
You
are
I,
don't
know
video
player
to
crash
in
some
intense
scene
in
the
movie.
You
just
want
to
watch
it
until
the
end.
So
it's
also
something
that
makes
the
users
happy
independent
of
safety,
and
this
is
not
really
something
that
is
specific
to
this
drama.
A
It's
the
same
story
with
like
ffmpeg,
or
you
probably
saw
the
security
related
issues
in
Android,
stage-fright
library
a
few
years
ago.
It's
something
that
happens
everywhere
and
I
think
we
should.
We
should,
and
we
can
do
better
by
using
more
modern
tools
for
that
which
leads
me
to
the
next
point.
As
Lewis
was
saying
Giusti
miles
all
between
see,
it's
super
inconvenient,
you've
read
a
lot
of
code,
doing
nothing,
it's
mostly
boilerplate
and
it's
becoming
boring.
So
this
is
also
something
the
rust
really
helps
us
it.
You
have
all
kinds
of
modern
language
features.
A
You
have
the
ability
to
build
proper
abstractions
that
you
can't
really
in
C
and
in
that
context
also
the
multi-threading
yeah
features
of
rust,
like
the
pls
concurrency,
is
something
that
probably
will
help
us
a
lot
here,
because
customized
extremely
multi-threaded,
even
in
the
small
example
that
Lewis
was
showing
earlier,
you
would
already
have
attended
a
couple
of
different
sets,
and
one
thing
that
personally
makes
me
very
happy.
Currently,
we
have
some
kind
of
ownership
system
inside
gstreamer
who
owns
a
buffer.
A
When
can
we
write
to
it
when
not-
and
this
is
all
just
convention
with
rust,
we
could
directly
methods
to
the
API
and
the
compiler
would
complain
instead
of
random
things
happening
at
runtime
and
well,
the
last
part
performance
and
slow
overhead,
I
guess
I
just
skipped
that
all
of
you
should
be
aware
of
that
by
now.
So,
let's
start
where
rust
can
actually
be
used
in
a
useful
way
in
the
context
of
GStreamer
and
one
side
is
what
lewis
was
saying:
you
see,
marmax
integration,
easy
in
both
directions
and
it's
the
first
step.
A
We
are
going
to
look
at
writing
plug-ins
for
gstreamer,
with
rust,
so
implementing
something
that
can
read
from
a
specific
network
sauce
that
can
decode
a
specific
video
codec
or
things
like
that-
and
this
is
also,
in
my
opinion,
the
most
important
part
that
we
should
attack
first,
because
this
is
where
well,
the
actual
media
handling
inside
USM
is
happening.
This
is
where
all
the
security
relevant
bugs
in
the
past
happened.
This
is
also
where,
if
you
look
at
our
box
occur
where
most
of
the
crashes
are
happening.
A
So
this
is
something
that
where
we
should
really
try
to
become
better,
and
this
is
why
we
started
with
this
part,
and
the
other
aspect
is
the
GC
Markov
itself
is
by
now
sufficiently
stable
and
simple
and
I
would
say
back
feeling
more
or
less.
So
it's
not
all
these
external
things
that
add
features
to
GStreamer.
That
should
become
a
bit
yeah.
B
A
A
This
direct
mapping
of
the
gusto
ownership
model
to
the
rest
ownership
model,
which
was
very
nice
to
implement
also
things
like
format,
negotiation
like
if
you
have
two
of
these
elements,
they
need
to
agree
on
what
this
one
is
producing
has
to
be
possible
to
accept
by
the
other
one.
That's
also
something
that
is
already
there
and,
in
the
end,
what
you
have
is
a
couple
of
trades
that
you
can
implement.
There's.
B
A
This
is
something
that
can
be
handled
with
one
and
all
that
you
have
to
do
at
this
point
is
implement
this
trait,
implement
the
little
functionality
that
you
have
to
implement,
and
then
you
can
use
that
to
compile
to
a
system
a
plugin
which
is,
in
the
end,
just
a
shared
library
that
can
then
be
used
transparently
by
every
application
that
uses
GStreamer.
And
at
this
point
in
here,
that's
almost
no
C
code
left
I
think
unless
red
lines
are
left
before
it
was
much
more.
A
It's
all
rust
or
well
unsafe
lost,
because
we
have
to
call
in
to
C-
and
all
of
this
is
currently
packaged
into
some
little
standalone
crate.
That
allows
you
and
everybody
else
hopefully
easily
implement
these
to
my
plugins,
and
what
currently
exists
is
a
plugin
to
read
from
files
and
to
write
to
files,
nothing
interesting,
but
that
was
basically
the
first
thing
that
we
started
with.
Just
like
a
proof
of
concept.
Can
we
do
that?
And
yes,
we
can.
A
Yeah.
One
thing
that
might
be
worth
mentioning
is
also
that
the
passing
of
the
file
format
is
done
with
gnome,
which
is
some
kind
of
pathology.
You
just
describe
how
the
data
structures
look
like,
and
it's
automatically
generating
you
coach.
That
passes
this
into
rust
data
structures
very
nice
stuff,
and
you
should
really
look
into
that.
A
You
want
to
pass
something
and
yeah
something
very
nice
about
all
these
plugins
is
there's
not
a
single
line
of
C
code
in
there
there's
not
a
single
line
of
unsaved
last
code
in
there
it's
all
safe
and
should
benefit
from
all
the
safety
guarantees.
Their
trust
is
giving
so
I'm
just
a
very
small
example.
A
This
would
be
the
trait
that
you
would
have
to
implement
if
you
wanted
to
implement
a
source
and
in
the
end
the
only
important
part
is
in
the
that
you
implement
a
small
function
called
fill
and
in
there
you
write
all
the
data
that
you
get
from
whatever
place.
You
are
read
from
into
a
decimal
buffer
and
that's
all,
and
now,
if
you
compare,
for
example,
the
file
reading
plug-in
that
it's
already
in
GStreamer
with
the
one
that
was
written
in
rust.
A
B
B
So
there
is
an
application
API
that
gives
you
that
unlimited
potential
that
you
can
do
by
combining
all
the
elements-
and
this
well
will
handle
all
the
things
inside
in
the
core
that
are
mechanisms
that
are
extremely
complex
multimedia,
has
like
a
huge
breadth
of
things
that
could
be
happening
and
you
just
want
it
to
be
hidden
taken
care
of,
so
where
we
have
some
idiomatic
findings
and
we're
still
kind
of
working
on
this
role.
Puck
in
the
end,
how
you
can
contribute.
B
This
is
always
like
a
work
in
progress,
so
there
is
a
auto-generated
gstreamer
cysts
that
Sebastian
has
been
working
on
that
are
basically
one-to-one
parodies.
But
then
we
have
a
DC
mer
RS
on
top
of
that
that
just
basically
makes
it
idiomatic
or
ET
at
these
sizes,
if
that's
even
a
word,
be
so
people
that
are
used
to
Rusk
and
feel
that
its
multimedia
through
rust
and
that's
the
github,
if
you
want
to
play
with
it,
I
have
a
branch
that
has
been
like
fixing
a
lot
of
stuff.
So.
B
B
So
is
if
the
presentation
multimedia
doesn't
have
a
demo
now,
so
we
should
first
introduce
the
four
concepts
that
we
need
for
this
first
demo,
the
first
one
is
well,
we
need
a
pipeline.
We
need
somewhere
to
store
all
these
elements
that
will
wrap
and
be
like.
Okay,
that's
saying
handles
the
clog
and
handle
just
see
the
messages
and
handles
all
these
like
bits
that
you
need
in
terms
of
like
once,
you
change
the
states
of
the
pipeline.
B
It
changes
the
state
of
all
of
the
elements
inside
and
by
state
I
mean
going
from
stop
to
playing
to
simplify.
Then
you
also
need
to
create
new
elements.
These
the
elements
you're
going
to
want
to
add
to
the
pipeline
and
then
tell
them
which
ones
are
going
to
be
connected
to
whom
that's
going
to
be
easily
done
through
this,
like
extremely
convenient
function,
add
unlink
that,
basically
internally,
it
does
the
add
functions
in
the
link
functions,
but
it
was
just
much
nicer
to
have
one
function
that
does
this
really
common
step.
B
B
It
just
makes
way
more
impressively
previous
talks
that
had
like
live
coding
just
showing
the
code
is
hard
enough,
but
apparently
you
can
even
write
it
on
stage,
so
we
have
this
pipeline
new.
That
I
showed.
Basically,
we
just
call
it
pipe.
The
the
argument
it
takes
is
just
name
you
want
to
give
it
to
it:
we're
going
to
create
a
output
video
source.
B
This
is
basically
a
convenient
element
that
just
takes
the
default
source
you
have
for
video,
so
usually
tends
to
be
the
webcam,
but
you
might
have
other
things
that
are
higher
priority
and
the
same,
but
opposite
will
be
getting
the
auto
video
sync
which
in
this
case
is
going
to
be
just
the
screen,
and
then
we
had
that
atom
link
basically
want
to
add
the
source
and
the
sink
of
video
just
make
sure
it
did
it
correctly.
If
not,
we
should
panic.
Something
went
really
wrong.
We
want
to
get
that
message,
bus
to
see.
B
B
B
B
Just
not
the
ones
that
are
rolling,
but
you
can
block
sayings
and,
and
then
we're
going
to
also
be
reading
and
the
bus
to
do
this
cool
little
concept
like
proof
of
concept,
which
is
we're
going
to
be
listening
to
the
end
of
stream,
which
is
just
like
the
technical
way
to
say.
We
finished
with
the
content
and
in
that,
when
that
happens,
we're
going
to
say
sick,
which
is
a
technical
term
for
moving
inside
the
contents.
B
B
The
same
as
before,
but
just
now
here
we
have
a
URI
that
we're
going
to
get
us
an
argument
from
the
command
line,
which
is
basically
the
path
through
the
video
that
we
want
to
play,
recreate
the
play
VIN.
As
you
can
see,
the
prototypes
and
the
interfaces
are
exactly
the
same
as
a
pipeline
because
it
inherits
from
front
pipeline.
We
get
the
bus
this
before
we
did
it
directly
from
the
pipeline.
Now
we
can
just
do
it
from
the
play.
B
B
Just
ever
so
slight,
it
will
go
up
at
every
single
time,
we're
printing
that
gos
and
see
if
we
can
go
back
to
the
this
guy.
So
every
single
time
it
goes
back.
This
video
is
a
little
bit
of
a
metaphor
of
how
multimedia
works.
It's
like
data
goes
through
one
element
and
then
it's
like
okay,
you
can
now
I
finish,
writing
the
memory.
You
can
start
reading
it
and
then
the
next
one
is
going
to
go.
Spy
I
have
a
second
quick
loop
writes.
We
don't
have
time
system.
A
We
don't
really
have
time,
okay,
so
now
you
might
wonder
what
the
future
of
all
this
is
going
to
be.
We
don't
have
much
time
so
I
will
just
shortly
go
over
all
that,
so
on
many
people
are
asking
us:
do
you
want
to
rewrite
GStreamer?
You
should
just
rewrite
it
in
rust.
It's
going
to
be
so
much
better.
A
The
problem
with
all
that
is
it's
a
huge
amount
of
work
to
do
that,
and
usually,
if
you
rewrite
something
afterwards,
it's
not
going
to
be
better
and
you
still
need
a
lot
of
time
to
stabilize
it
again
and
because
of
that
I,
don't
think
we
are
going
to
do
that
anytime
soon
soon,
especially
because
the
juice
tomorrow
I
said
before
it's
already
quite
stable
and
somewhat
bug
free
and
also
not
very
big.
In
the
end,
sanitation.
A
So
on
also,
there
are
some
technical
and
social
problems
that
have
to
be
solved
first,
socially,
we
first
have
to
convince
all
the
existing
contributors,
or
at
least
the
main
contributors-
that
rust
is
the
solution
to
all
our
problems,
which
will
take
some
time
techni
on
the
technical
side.
One
problem
is
that
Ross
currently
doesn't
have
any
API
stability
or
any
defined
ABI.
So
currently,
everybody
is
doing
static,
linking
which,
in
the
case
of
plugins,
becomes
a
bit
annoying.
A
Every
plugin
is
currently
something
like
three
megabytes
or
bigger,
while
they're
on
comparable
receivers
with
something
like
50
kilobytes
one
kilobots,
and
that's
going
to
be
a
problem
on
embedded
devices.
So
that's
something
that
at
some
point
will
have
to
be
solved.
We
will
see,
but
what
will
happen
in
the
near
future,
at
least
from
us
too,
is
that
more
plugins
for
specific
media
formats
are
going
to
be
written.
A
B
So
something
like
worth
mentioning
one
of
the
reasons
of
why
we
don't
want
to
rewrite
is
gstreamer
is
everywhere.
I
like
to
play
this
game
with
my
non
techie
friends
or
pointing
out
GCM.
Is
there
you
sumers
there
until
they
get
really
tired
of
me
and
want
to
be
like
unfriendly,
so
just
like
notable
ones,
every
single
smart
television
builds
after
2014.
B
If
you
watch
a
video
in
the
flight,
if
you
have
a
GPS
the
thing
that
tells
you
go
and
left
and
right,
if
you
have
read
out
your
books,
if
you
use
a
drone,
if
you
like,
do
video
streaming
online,
it's
just
everywhere,
so
it's
not
trivial.
To
suddenly
tell
this
entire
industry.
Yeah,
let's
like
switch,
is
something
completely
written
from
scratch
in
terms
of
the
application
side
of
bindings.
B
There
are
many
suboptimal
corners
in
the
API
that
I'm
slowly
kind
of,
and
this
week
has
been
really
great
for
us,
because
we
kind
of
keep
like
bouncing
back
and
forth
ideas,
but
obviously
everything
big
in
github.
Please
send
us
issues
in
terms
of
hey
I'm,
a
rust
developer,
and
this
feels
awkward
so
just
tell
us.
B
We
also
want
to
switch
to
a
little
bit
more
of
an
intelligent
of
the
generation,
but
then
kind
of
wraps
things
with
you
like
new
things
that
begin
omec
communities
doing
towards
the
object,
yeah
I
think
that's
pretty
much.
It
I
know
that
we're
like
over
time.
We
usually
try
to
have
the
time
for
questions,
but
not
this
time.
Yeah
find
us
in
the
halls.
If
you
have
any
G,
tumor
or
other
related
questions.
This
is
where
all
the
code
information
is
I'm
loose
BG
in
the
bird
flight.
You
are
s,
Kroger,
I,
think.