►
From YouTube: Meetup Rust Paris #5 RustLex by Leo Testard.
Description
RustLex is a lexical analyser generator syntax extension for Rust. It works similarly to Lex, written in Rust and generating Rust code using a syntax extension. This talk recalls what lexical analyser generators are, and what they are used for inside compilers, then presents the basic usage of RustLex through simple examples.
Leo Testard is currently a student in computer science in the University Pierre et Marie Curie (Paris 6), and is mainly interested in programming languages, especially systems programming and functional programming. He did minor contributions to Rust itself, as well as a few projects around the language, including RustLex.
Help us caption & translate this video!
http://amara.org/v/2Fh3/
A
Okay,
so
hello,
everyone
so
I'm
Leo,
and
they
will
talk
about
one
of
my
longtime
projects
that
I've
made
both
rests.
So
it's
reflects
lexical
analysis
generator
in
rest
that
generates
risk,
so
I've
I
wasn't
sure
about
whether
I
should
do
this
taco
nuts,
because
I
lose
some
motivation.
This
project
recently,
but
anyway,
excel
wanted
me
to
to
do
it.
So
let's
go,
but
I
won't
tell
him
that
everyone
here
knows
what
a
lexical
analyzer
generator
is
know.
A
What
liquor
on
eraser
is
so
first,
what
is
a
lexical
analyzer
generator,
it's
a
generator
for
something
that
is
a
part
of
a
compiler.
So
why
do
we
generate
a
lot
of
the
compiler?
A
compiler
is
made
of
several
parts.
You
have
the
front
end
that
takes
the
rowing
boots
on
your
program
that
will
pass
it
into
an
internal
representation
which
is
called
an
abstract
syntax
tree,
which
is
an
tree
representing
your
program.
A
Then,
on
this
tree
you
run
a
part
that
is
called
a
middle
end
that
will
run
virus
in
the
laces
passes,
and
then,
once
you
analyze
your
tree
and
check
that
it
corresponds
to
a
valid
program.
You
can
generate
intermediate
code
with
it
and
then
optimize
it
and
generate
native
machine
code.
So
when
you
create
a
new
language,
you,
basically
you
don't
want
to
to
worry
about
all
those
spaces
arm
back
hands
on
front
ends
of
the
compiler
or
difficult
to
write,
and
it's
not
the
most
interesting
part
when
you're
writing
a
compiler.
A
What
you
focus
on
is
the
semantics
of
your
language.
So
what
analyzes
passes
you
will
do
what
programs
you
will
reject.
Why
so
the
most
Christine
thought
compiler
is
obviously
the
middle,
and
so
there
is
derived
in
our
needs
for
a
long
time
to
abstract
over
the
less
interesting
parts
of
a
compiler.
That's
all
the
front
end
and
the
back
end
fold
the
back
end
from
many
years
now
we
have
llvm.
A
A
So
a
compiler
front
end
is
also
made
of
several
parts.
First,
you
have
the
programming
good,
which
is
a
row
stream
of
characters
and
the
first
pass
is
called
a
lecturer
which
it
stands
for
lexical
analysis
and
it
will
split
the
real
stream
of
characters
into
tokens.
Tokens
is
like
a
word
of
the
language.
A
But
because
C
is
really
bad
for
writing.
Compilers
I,
don't
know
if
you
already
wrote
see
codes
that
did
a
lot
of
manipulation
and
strings
is
just
super
painful
to
write.
You
have
to
deal
with
buffer
management
and
it
is
really
is
really
bad
right
compared
us
on
this
kind
of
it.
Rest
is
great.
We
have
some
types,
we
have
pattern
matching.
We
have
a
new
olivion
binding
that
is
used
in
the
Sun
dog
dress.
Compiler,
the
rest
could
be
a
bedroom
with
you
for
writing
compilers.
A
Although
what
this
is
like
sauce
and
passes
generators,
so
I
chose
to
the
one.
Despite
that,
there
is
already
rest
back
hands
for
reg
out,
which
is
unreal
times
better
than
the
thing
I
wrote
but
anyway
and
finally,
rest
in
tax
extensions
are
a
really
great
tool
to
create
small
domain-specific
languages.
We
first.
A
How
those
legs
work
in
the
first
place
legs
as
it's
only
in
with
you.
The
legs
on
in
with
you
are
it's
a
language
in
which
you
described,
Lex,
improves
to
run
legs
on
your
lifestyle.
Legs
will
pass
it
and
generate
see
from
it,
if
very
good,
to
integrate
with
the
rest
of
your
compiler
and
the
programming
tools
like
syntax,
highlighting
you're
over
it.
It's
the
features,
work
especially
bad
with
our
legs.
A
Syntax
extensions
or
a
rest
feature
that
allow
you
that's,
allow
you
to
create
some
kind
of
a
compiler
plugin.
It's
compiled
dynamic
library
that
consists
of
compiled
code
and
that
will
be
loaded
by
the
rest
compiler
when
you
invert
some
macros
and
the
the
compiled
code
that
is
inside
the
library
will
be
run
to
expand
the
micro
invocation,
so
it
will
receive
from
the
the
rest
compiler
a
syntax
tree
that
describes
the
arguments
you
pass
to
the
micro.
It's
genuine,
arbitrary,
well
Scott
and
return
anniversary
xray.
That
represents
the
expanded
form
of
the
micro
pop.
A
So
it's
easy
to
integrate
anywhere
in
the
Rebs
code
and
it
works
well
with
tools.
So,
let's
see
an
example,
this
is
what
a
reflects
invitation
looks
like.
So
you
call
the
rest.
Lex
macro
on
at
the
top
of
your
page
will
have
to
write
something
like
face:
syntax,
external
crate,
Ross
legs
which
tells
the
compiler
to
load
the
reflects
plugin
arm.
And
then
you
invoke
the
Restless
macro
this
way
and
you
write
your
wrestling
school.
So
it
looks
arm
a
bit
like
legs
with
for
more
restive
syntax,
you
write
to
set
rules.
A
Each
will
describe
how
to
create
one
kind
of
token.
So
what
you
see
before
the
arrow
is
a
regular
expression.
It
has
a
particular
syntax
for
regular
expressions,
but
this
was
we
need
to
to
integrate
with
the
rest
in
text
so
that
since
I
selectors
and
such
will
work
on
regular
expressions
and
also
to
to
make
it
easy
to
include
escape
characters
inside
regular
expressions.
I,
don't
know
if
you
already
use
regular
expressions
library
in
common
languages,
but
sometimes
it's
awful
when
you
want
includes
a
backslash.
A
You
must
backslash
backslash,
but
if
you
buy
charge
the
backslash,
the
backslash
will
be
interpreted
by
the
syntax
for
strings
of
the
you're,
still
in
with
you
and
not
by
the
regular
expression
enjoy.
So
you
have
to
send
two
backslashes
so
that
the
regular
expression
and
join
will
receive
it.
So
you
like
to
write
something
like
best
side
by
side,
back
side
effects,
/
and
you
end
up
with
a
huge
quantities
of
backslashes.
So
when
you
use
these
kind
of
things,
you
don't
have
this
program,
you
have
sometimes
literal
character.
A
A
A
A
When
it
is
a
sequence
of
one
or
more
digits,
it
will
say
that
it
just
saw
an
integral
a
sequence
of
alphanumeric
characters
beginning
by
later
on,
or
an
underscore
is
an
in-depth
fire,
typically
a
viable
name.
And
finally,
it's
just
matches
the
inch
keywords
if
you
want
to
distinguish
it
from
inter
isn't
right
because
it
matches
both
rules.
So
when
there
is
an
ambiguity,
lexical
analyzers
like
rest
legs,
but
also
lex,
will
also
will
always
try
to
match
the
longest
input
as
possible.
A
So,
for
example,
if
your
input
is
in
a
it
can
match
the
instruction
and
then
the
a
letter
or
the
into
a
identifier
and
we
took,
chose
the
ladder
the
into
a
identifier
and
if
you
write
only
int
so
that
all
interpretations
are
of
the
same
size,
restless
will
choose
the
action
that
was
defined
later
in
the
specification.
So
in
this
case
it's
right
int.
It
will
say
that
it's
so
the
inch
keyword
and
not
the
incident
fire,
because
the
the
room
for
the
in
keyword
was
described
later
and
at
the
top
of
the
definition.
A
The
rule
here
just
eliminates
things
that
are
not
needed.
Spaces
kings
are
yet
that
so
then
the
service
separate
rows
between
tackle.
So
you
write
this,
you
run
the
rescue
compiler,
it
will
load
the
rest
lights.
Library
run
small
piece
of
Roscoe
that
I
wrote
that
will
generate
the
realistic
she
needs
Automator
from
it
then
pass
back
to
receive
rest
code
that
describes
the
transition
tables
of
the
automata
and
some
code
to
iterate
over
it
and
you're
done,
and
then
you
can
use
this
exam.
So
how
do
you
use
the
lecture.
A
A
So
generally,
actions
return,
something
to
say:
I
saw
this
thing,
so
I
chose
to
give
the
generated
lecture
and
need
new
crater
interface,
so
that
each
code
to
the
lecture
function
will
return
the
next
second
as
an
option
token,
each
time
you
called
an
XOR
function,
it
will
start
to
well,
you
stopped
the
last
time
in
the
input
stream.
It
will
try
to
pass
the
input
stream
with
the
set
of
rules
and
once
it
is
sure
it's
matched
the
longest
inputs
possible.
A
It
will
return
some
and
with
the
token
it
matched
or
if
it
didn't
match
anything,
just
written
known,
and
this
would
cause
the
iteration
to
stop
because
in
interest
when
you
write
something
like
for
touch
elixir,
this
will
call
LexA
dots,
Daleks
out
next
method,
Tillie
Tillie,
Trittons
known
and
then
stopped
so
here.
It's
the
same
example
that
on
the
previous
slide,
but
instead
of
printing,
which
they
can
we
just
smashed,
we
return
and
the
black
to
indicate
what
we
sell.
So,
of
course,
the
in
real
program.
A
You
would
have
to
need
to
declare
a
type
before
the
invocation
of
the
reflection.
A
crow
are
currently
the
reason.
Type
of
the
rest
lights
function
must
be
option,
token
and
token
name
must
be
talkin
it
can.
It
can
be
something
else,
but
in
the
future
it
will
probably
override
bar
in
some
way
and
then
in
your
main
function,
you
create
an
input
string.
A
You
give
it
to
the
lecture
I've,
no
idea
why
it
is
a
box
read,
although
it
was
written
a
long
time
ago,
and
then
you
try
to
avert
the
lecture
and
each
time
it
sees
the
can
it
give
it
to
oh
there's,
a
curly
brace
missing
I,
give
it
to
the
to
the
blog.
You
wrote
in
your
follow-up
and
continues
to
the
next
method
returns
known
and
then
it
stops.
A
I
have
other
slides
to
talk
about
specific
features
of
legs
and
relax.
Download,
really
very
interesting
that
I
will
person
quickly
conditions
our
feature
when
you
cannot
completely
pass
your
language
you're
using
only
regular
expressions,
and
it
allows
you
to
define
another
automator
with
manual
transitions
between
the
different
centimeters.
A
Instead
of
having
a
single
state
machine
with
transitions
that
are
encoded
into
the
transition
table,
you
will
have
two
separate
state
machines
and
when
you
are
into
an
action
you
can
say:
okay
go
to
the
other
machine
and
you
have
a
machine
as
a
different
set
of
rules
to
pass
on
good.
So
at
I
pickle
example
of
it,
though
it
is,
conditions
are
not
required
for
this
specific
example.
But
it's
often
used
its
comments
because
yeah,
for
example,
you
might
want
to
continue
to
count
the
lines
of
your
program.
Talk,
error,
reporting.
A
While
you
are
passing
your
comments,
if
you
write,
if
you
write
a
single
regular
expression
that
meshes
with
whole
comments,
you're
the
generators
and
a
laser
won't
do
anything
why
it
matches
this
regular
expression.
So
you
lose
the
information
of
the
line
breaks
that
may
appear
inside
it's,
so
you
use
conditions
you
start
to
write
to.
This
is
always
the
same
example,
except
that
she,
like
syntax,
for
comments,
has
been
added.
A
So
when
you
see
the
beginning
of
the
comments
you
say
begin
and
you
enter
another
Automator
which
is
described
under
the
comment-
intense
fire
in
this
in
this
over
automata.
It
has
been
omitted
here,
but
you
may,
when
you
match
a
line
written
instead
of
doing
nothing
increment
line
counter,
and
when
you
see
the
end
of
the
comments
you
go
back
to
the
initial
tomato,
there
are
fewer
things
named.
Regex
definitions
is
something
like
you
have
a
few
rules
that
use
the
same
regular
expression.
You
don't
want
to
have
to
type
the
war.
A
Regular
expression
every
time,
so
you
can
give
him,
give
it
a
name.
Well,
that's
not
a
revolutionary
and
the
newest
conditions
is
a
tricky
thing
that
I
implemented
the
that
are
useful.
If
you
want
to
include
an
action
on
the
middle
of
a
regular
expression,
and
you
don't
want
to
declare
a
condition
because
you
would
end
up
by
writing
a
lot
under
lots
of
conditions.
A
But
it's
a
very
buggy
right
now,
I
would
I
was
expecting
it
to
to
provide
the
ability
to
match
to
to
extract
the
substrings
that
matches
a
expressions
of
the
tokens,
but
it
don't
work
very
desert
world
very
well
right
now
and
properties
are
a
feature
that
earlier
to
extend
the
it.
Was
you
to
extend
the
generated
structure
that
stores
the
information
about
the
lexer
with
a
custom
fields
that
you
can
then
to
twitch?
You
can
then
assign
in
actions
of
the
of
the
lexical
analyzer.
A
So,
for
example,
if
you
want
to
implement
the
line
counter
instead
of
deterring
a
global
mutable
viable,
like
you
do
with
less,
you
will
ask
russ
legs
to
create
a
field
for
this
in
the
generated
structure,
things
that
are
that
remain
to
do
as
I
said,
being
able
to
extract
the
substrings
that
matched
a
given
sub
expression.
Of
a
token
this
is
currently
I.
I
said
it
was
possible
with
the
anonymous
conditions,
but
it's
very
painful
and
very
buggy
and
reduce
transition
table
size,
because
currently
they
are
absolutely
huge.
A
So
here
thanks.
If
you
have
some
questions
and
if
not
you're
welcome,
if
you
want
to
contribute
to
this
way
because
I
don't
know
if
I,
if
I
will
continue
to
contribute
actively
on
its
I,
was
a
big
depressed
recently,
when
I
noticed
how
many
things
a
lego
was
able
to
do,
and
oh
is
eat,
was
to
do
in
Lego
and
since
regular
eyes.
Now
perfectly
fine
worst
biosens.
This
forget
my
project
last
a
bit
of
interest,
but
anyway,
maybe
if
I
have
timing
will
continue
to
improve
it.
A
bit
Thanks.