►
Description
Oh no! My parser sprouted into an entire compiler! Compilers should not be reserved for programming languages! With the right care (lots of sun, proper hydration) compilers are wildly beneficial outside of their traditional context. In fact we wrote one for GraphQL, a query language! In this talk, I'd like to tell you about the journey of apollo-rs - how we started with a performant, lossless, recursive-descent parser with really good errors that has now become a query-based, diagnostics-first, easily usable modern compiler.
A
Hello:
everyone
now
that
Nell
read
that
out,
I
realized
I
didn't
give
you
a
fun
fact.
I
gave
you
Irina
is
turning
into
her
grandparents
fact,
I
think
in
a
couple
of
years,
I'll
come
back
here
and
I'll.
Tell
you
about
all
the
goats
and
bees
I'm
I'm
having
hi
welcome
to
writing
a
graphql
compiler
in
Rust.
A
A
Thank
you
so
much
for
having
me
here
again.
I'm
really
excited
to
be
here
and
that's
not
just
the
coffee
talking
so
a
little
bit
about
graphql
and
I,
also
Garden,
as
Nell
pointed
out,
but
I
grow
a
whole
bunch
of
things
and
I
also
make
zines
the
zine
I've
printed
out.
You
might
have
seen
about
proc
macros
is
mine.
Please
take
one
if
you
haven't
already
I
printed
200
of
them
at
a
local
Rizo
printer
in
Copenhagen.
A
All
those
things
are
fun
and
nice,
but
today
I
actually
wanted
to
talk
to
you
about
the
thing
I've
been
working
on
for
the
past
year
and
a
bit
called
Apollo
ORS.
It's
a
series
of
crates
that
we've
put
together
at
apolograph,
ql
and
I'm
really
really
excited
to
share
a
whole
bunch
of
stuff
about
this.
This
project,
so
without
further
Ado.
Let's
take
a
look
at
what
we'll
actually
talk
about
today.
A
I
want
to
talk
to
you
about
compilers,
just
in
general,
so
we
kind
of
understand
where
Apollo
compiler
comes
into
play
among
a
whole
bunch
of
other
things.
I'll
talk
to
you
about.
Why
we're
doing
this,
then
we'll
kind
of
take
a
look
as
to
what
exactly
the
compiler
does
we'll
look
at
the
stages
of
the
compiler
process.
And
finally,
my
favorite
bit
is
we'll
talk
about
the
architecture
of
the
project,
so
without
further
Ado.
Let's
take
a
look
and
kind
of
set
the
stage
and
get
a
little
bit
of
background
information.
A
The
preface
so
I
think
it
would
be
important
for
us
to
take
a
look
at
what
graphql
actually
is
and
graphql
what
it
is.
It's
a
query
language
for
apis.
Let's
take
a
look
at
kind
of
the
schema.
The
schema
defines
a
bunch
of
different
types,
and
these
types
are
related
to
each
other
and
are
used
amongst
different
ways.
Graphql
has
a
spec
that
defines
how
each
one
of
these
types
is
meant
to
be
used,
there's
also
a
spec
for
validation
of
these
types
and
so
on
and
so
forth.
So
it's
quite
explicit.
A
A
The
next
thing
that
would
be
good
to
talk
about
is
about
just
give
you
a
glimpse
about
what
Apollo
does
and
apolograph
ql
is,
as
a
name
suggests,
a
graphql
company,
and
what
we
do
is
we
build
a
lot
of
developer?
Tooling
and
infrastructure
around
allowing
people
to
unify
their
graphql
app
and
services,
and
this
should
like
allow
people
to
kind
of
build
on
top
of
their
graphql.
A
So,
but
this
talk
is
not
really
about
the
things
we're
rewriting
in
rest,
but
about
the
backbone
of
lots
of
these
projects,
which
is
apollars
and
Apollo
res
is
a
speck
of
implementation
of
graphql
in
Rust
and
it
is
spec
compliant
and
just
implements
graphql
in
general.
So
this
is
what
we'll
focus
on
today.
A
Apollo
RS
is
a
combination
of
various
different
crates,
a
workspace.
If
you
will
it
composes
of
a
few
important
ones.
One
is
Apollo
parser,
which
parses
graphql
query
language
one
and
the
second
one
is
Apollo
encoder,
which
encodes
programmatically
into
graphql.
We
have
Apollo
Smith,
which
is
a
test
case
generator.
It
is
named
after
wasm
Smith
a
bytecode
Alliance
test
case
generator
for
wasm.
A
But
before
we
kind
of
go
into
that,
I
want
to
set
the
stage
of
what
it
is
that
the
Apollo
compiler
does,
and
here
it'll,
be
good
to
kind
of
talk
about
what
is
even
a
compiler,
something
that
we
are
quite
familiar
with.
Is
we
have
some
C
code
that
compiles
down
to
assembly
and
I'm
here
to
tell
you
that
Apollo
compiler
does
not
compile
down
to
assembly?
I'm,
sorry
that
this
is
disappointing.
A
That
would
be
kind
of
cool.
If
anybody
wants
to
take
that
on.
No,
please
go
ahead
and
I
think
to
kind
of
give
a
bit
of
an
explanation,
because
we
don't
actually
compile
to
any
binaries
at
all
is
to
look
at
the
street
from
five
years
ago
and
I'll
call
this
section.
Irina
does
Twitter
review
to
look
at
this
tweet
by
Steve
klabnik
as
Steve
tweet?
If
you
will
it
goes
you
solving,
please
stop.
A
Let's
look
at
this
other
tweet
that
I
slightly
adjusted
for
this
presentation
by
Ryan
Cavanagh.
This
is
called
X
a
compiler
alignment
chart
and
it
kind
of
outlines
and
I
think
it.
It
is
quite
funny,
but
it
also
kind
of
sets
the
stage
to
think
about
different
inputs
and
outputs
that
we
can
have
as
a
compiler
and
while
I
would
love
a
Paul
compiler
to
be
the
rebel
and
be
compared
to
the
Sun.
A
A
As
our
output,
we
output
instructions,
but
they
are
in
a
form
of
a
database
and
I
promise.
We'll
talk
to
you
we'll
talk
together
about
the
database
portion
in
a
little
bit.
I
know
I'm
kind
of
teasing
you
a
little
bit
but
we'll
get
there.
So
these
are
kind
of
the
things
that
align
us
on
this
alignment
chart
input
as
graphql
output
as
database,
and
at
this
point
what
you're
probably
wondering
is:
why
are
you
going
through
so
much
effort
for
a
query
language?
A
Why
can't
you
just
use
the
parser
because
or
use
any
given
parser
for
graphql,
because
the
graphql
spec
is
usually
implemented
by
various
parsers?
Why
are
you
writing
a
compiler
here?
I
want
to
kind
of
give
you
a
bit
of
a
bit
of
an
explanation
as
to
why
we
went
through
so
much
trouble,
really
and
kind
of,
as
I
mentioned
before,
Apollo
works
with
quite
a
bit
of
graphql.
A
Working
with
increasingly
more
complex,
graphs
and
sitting
in
a
critical
position
in
a
lot
of
people's
infrastructure
requires
us
to
have
a
more
performant
and
reliable
way
to
work
with
graphs
and
work
with
documents
with
work
with
graphql
documents.
Even
we
might
want
to
do
a
whole
bunch
of
different
things
like
compose
different
graphs,
together,
they're
quite
complex.
We
might
want
to
do
query
planning.
We
want
to
perform
all
sorts
of
like
wild
and
unique
schema
manipulations
and
all
of
those
things,
perhaps
or
not.
A
Perhaps
all
of
those
things
are
a
lot
made
a
lot
easier.
If
we
have
a
database
to
ask
these
questions
from
questions
such
as
find
me,
the
join
direct
tip
type.
Where
is
it?
What
is
the
type
of
this
field
in
a
query
operation?
Is
this
type
it
in
scope?
These
are
the
kinds
of
things
we
want
to
be
asking
of
a
database
and
writing
a
compiler
in
such
a
way
allows
us
to
be
able
to
ask
these
questions
easily.
A
A
It
is
incredibly
important
to
be
able
to
have
something,
that's
ergonomic
and
is
done
well,
but
and
while
it's
ergonomic
and
is
done
well,
it
is
also
performant.
But
if
you
really
think
about
it,
performance
doesn't
really
mean
anything
unless
something
unless
we
have
those
ergonomics
in
place,
and
that
is
something
the
that
we
as
rust.
Programmers
really
understand,
because
we
work
with
rust,
C
and
Russ
C
is
performant,
but
the
one
thing
we
really
really
appreciate
about.
A
It
is
the
it's
it's
the
developer
experience
and
the
Diagnostics
we
produce
and
how
the
compiler
sits
there
and
helps
us
along
the
way
with
the
Diagnostics
it
produces,
and
so
one
of
the
things
that
we
would
also
like
to
do
with
the
Apollo
compiler
is
be
able
to
provide
good
Diagnostics
about
a
given
input
given
graphql
input,
and
so
at
this
point,
I
would
like
to
again
go
to
a
Twitter
review.
This
is
officially
a
Twitter
review
talk.
A
This
is
an
Esteban
tweet.
If
you
will
my
streaming,
my
steaming
hot
take
is
at
the
primary
function
of
a
compiler
is
to
take
malfron
code
and
emit
Diagnostics
to
help
the
programmer
turn
it
into
one
form
code.
A
compiler
is
a
narrow
reporting
tool
with
a
cogeneration
side,
gig
and
I
actually
really
liked,
and
this
is
from
a
talk
that
Esteban
gave
at
rusconf
in
2020
a
really
good
talk.
A
A
Here
we
use
a
reporting
library
but
cut
by
cut
Martian
called
miet,
and
it
produces
incredibly
beautiful
reporting
for
Diagnostics
that
use
all
the
sorts
of
information
we're
able
to
feed
it
and
just
it
works
beautifully.
So
we'll
see
it
in
a
demo
in
a
second.
So
what
I'm
going
to
use
here
is
my
colleagues
Avery
harnish's
CLI,
where
he
hooked
up
Apollo
compiler
to
a
CLI
and
a
web
browser
will
just
use
the
CLI
in
the
terminal
to
see
how
the
compiler
reacts
on
a
given
document.
A
A
A
We
should
now
have
three
errors.
The
second
error
is
that
we
have
an
operation
that
is
defined
twice
within
a
document.
Also,
another
helpful
error
message,
I,
wonder
who
wrote
These
can
only
be
defined
once
in
this
document,
so
we're
going
to
remove
another
one
of
these
because
we
shouldn't
have
to
so.
This
should
leave
us,
hopefully
with
two
errors
unless
I
messed
up
another
one
is
that
we
have
an
unnamed
operation,
which
is
only
allowed
out,
if
only
allowed,
if
it's
only
the
only
one.
A
A
All
right,
so
that
kind
of
took
us
through
the
demo
portion
of
the
of
the
stock
next
I
actually
want
to
zoom
in
into
the
the
kind
of
what
is
what
is
compiler
stages,
what
are
the
compiler
process
and
what
the
compiler
stages
are
and
then
relate
those
back
to
our
own
compiler
you'll
see
this
diagram
a
lot.
I'll
reference,
this
quite
a
bit
so
there's
three
different
stages
that
we'll
talk
about
first
is
the
Lexing.
Then
it's
the
parsing
and
then
it's
the
compiler
bit
what
the
lecture
does
is
it
performs
lexical
analysis?
A
What
that
means
is
that,
given
a
particular
input,
we
want
to
make
sure
that
all
the
tokens
that
we
get
in
that
input
all
the
characters
we
get
in,
that
input
is
correct
in
the
context
of
graphql,
for
example,
a
star
character
is
not
allowed.
So
when
the
lecture
encounters
that
it
will
bubble
up
and
say
or
it'll,
create
an
error
and
say
this
character
is
not
allowed.
A
The
parser
performs
syntactic
analysis.
What
that
means
is
that,
given
the
tokens
that
it
receives
from
the
lecture,
what
it
wants
to
do
is
make
sure
they're
in
the
right
order
and
then
the
statements
it
creates
are
correct.
So,
let's
say
if
we
are
looking
at
an
object,
type
definition,
an
object,
type
definition
must
have
fields.
A
A
What
we
then
do
with
the
compiler
is
we
take
that
ASD
and
we
perform
semantic
analysis.
What
that
means
is
at
the
end
of
this
entire
process.
We
just
have
an
entirely
valid
program
if
everything
is
correct.
The
type
of
semantic
analysis
that
the
compiler
will
do
is
they'll,
say
it'll
check
if
the
types
in
use
are,
for
example,
defined
if
you're
using
a
type,
that's
undefined.
A
A
So
let's
relate
those
back
to
Apollo
compiler
and
how
we
manage
our
own
kind
of
compiler
stages
within
the
project.
So
this
is
the
architecture
bit
so
the
first
two,
the
lexer
and
the
parser
live
in
a
crate
called
Apollo
parser.
We
group
those
two
together,
the
next
bit
the
compiler
lives
in
the
crates
shockingly
named
Apollo
compiler.
A
Those
two
projects
coexist
together,
they're
kind
of
sibling
projects,
a
Paul
compiler,
really
depends
on
or
actually
uses
the
AAC
created
by
Apollo
parser.
Lots
of
our
work
is
based
and
is
heavily
inspired
by
the
rust
analyzer
project.
This
is
a
language
server
for
us
that
probably
most
of
you
are
familiar
with.
A
So,
let's
to
kind
of
take
a
look
at
how
exactly
things
are
done
for
the
input
data
to
tokens.
So
what
we
do
is
very
simple:
look
ahead:
parsing,
where
we
just
look
through
the
look
through
the
characters
coming
in
produce
tokens
the
next
little
bit
is
we
actually
have
a
surprise
visitor
and
before
we
create
the
ASC
or
the
type
tree,
we
create
an
untyped
syntax
tree
and
what
we
do
there
is
we
actually
use
a
recursive
descent
parser.
A
This
allows
us
to
have
a
super
easy
to
maintain
and
really
easy
to
diagnose
code
for
us
as
people
working
on
it
to
work
with.
It
also
allows
us
to
be
really
really
specific
about
our
errors,
which
is
exactly
what
we
want
in
a
compiler
that
is
meant
to
be
Diagnostics.
First
I
won't
talk
too
much
about
recursive
descent
parsing,
but
the
blog
post,
I
was
really
inspired
about.
Is
the
one
by
Alexei
cladov
called
simple
and
Powerful
Pratt
parsing.
If
you
Google
for
it,
you
get
a
really
good
post
on
that.
A
A
It's
fast,
but
the
question
you're
probably
asking
yourself
now
is
so
how
do
you
get
to
the
type
syntax
tree?
Because
you
want
to
be
able
to
have
rust
code
to
actually
work
with
in
an
AST?
And
here
we
use
another
rest.
Analyzer
crate
called
on
grammar,
which
is
a
domain
specific
language
which
allows
us
to
quickly
or
quickly
do
a
little
bit
of
code
gen
beforehand
and
take
the
notes
that
Rowan
provides
us
create
a
type
syntax
tree.
A
A
Typed
syntax
tree
at
the
end,
so
it's
fast
and
you
get
an
ergonomic
API
at
the
very
end
for
your
AST,
which
I
really
like
one
of
the
features
of
another
feature
of
our
kind
of
compiler
process
is
that
it's
meant
to
be
fault
tolerant.
This
means
that
we
barely
ever
Panic.
There
is
a
few
times
when
we,
for
example,
reach
recursive
recursion
limits
and
things
like
that,
but
in
all
other,
in
all
of
the
cases,
we're
able
to
gather
errors
and
display
them
to
the
user.
A
A
First
of
all,
we
have
an
unsupported
character
that
Alexa
flags
for
us,
and
the
second
thing
is:
we
have
a
missing
field,
type
that
the
parser
flags
for
us,
but
instead
of
panicking
as
soon
as
we
see
something
that's
wrong,
we
gather
all
of
our
errors
and
all
of
our
and
produce
in
the
AST
still.
So
this
is
an
a
debug
output
of
our
IAC
and
errors.
A
This
also
means
that
we
can
walk
the
AST
afterwards.
This
code
is
not
really
it's
more
for
aesthetic
purposes.
It's
more
to
tell
you
that
we
can
walk
the
AC
I,
wouldn't
leave
you
and
highlighted
code
to
look
at,
so
we
get
the
errors
and
we
get
the
AST
that
we
can
walk
in
and
we
can
see
all
the
Diagnostics
as
much
as
we
can
really,
which
is
kind
of
the
good,
the
good
way
to
go
about
it.
A
In
my
opinion,
the
other
thing
is
that
we
create
the
parser
itself
and
the
compiler
is
lossless,
which
means
we
keep
everything
you
sent
to
us
like
everything,
everything
everything.
For
example.
We
have
this
very
readable
schema,
that's
quite
standard,
but
we
can
also
have
a
schema
that
just
has
a
lot
of
ignore
tokens
that
we
keep.
We
can
also
have
a
different
schema
that
has
a
lot
of
ignore
tokens
that
we
keep,
keeping
all
the
tokens
and
having
a
lossless
parser
allows
us
to
to
be
able
to
essentially
reproduce
back
the
input
we
get.
A
A
So
let's
say
we
returning
back
to
Diagnostics.
Let's
say
we
have
the
schema
that
is
referencing
a
type
that
doesn't
exist,
having
a
lossless
having
a
lossless
parser
means
that
we're
able
to
then
run
it
through
and
get
an
output.
That
says
we
are
using
a
type,
that's
undefined
and
the
type
that's
not
in
scope,
but
also
exactly
output.
Your
schema
as
it
was,
which
means
we're
pointing
to
the
exact
location
in
your
code,
where
you
had
your
error,
which
is
exactly
what
we
want
to
do
when
your
code
is
malformed.
A
A
All
right
so
we've
talked
about
this
before,
but
let's,
let's
kind
of
brush
up
to
it
again.
The
two
features
of
Apollo
compiler
is
having
an
API
to
the
compiler's
database
and
the
ability
to
be
able
to
validate
and
provide
good
Diagnostics
and
malformed
code
or
malformed
graphql.
In
our
example,
to
be
able
to
do
number
two,
we
actually
need
to
be
able
to
do
number
one.
First,
we
need
to
be
able
to
have
that
database
and
having
kind
of
the
relationship
established.
So
the
one
thing
that
the
compiler
does
is
given
your
graphql.
A
It
builds
up
the
context
and
builds
out
their
relationships
between
various
types.
What
does
that
look
like
in
the
context
of
graphql?
Let's
take
a
look
at
the
schema.
This
schema
entry
point
is
type
query.
Human
human,
this
human
is
a
reference
to
a
type
human,
a
type
human
has
some
pets
and
a
pet.
There
is
a
reference
to
another
pet
and
so
on
and
so
forth.
There's
references
to
different
types
everywhere.
Those
are
relationships.
A
We
can
also
have
a
query.
A
query
is
a
direct
reference
to
the
schema,
so
those
are
relationships
as
well.
The
first
bit,
the
human
is
a
reference
to
the
human.
In
our
type
query,
the
name
is
a
reference
to
the
name
field
inside
type,
human
and
so
on
and
so
forth.
So
there
are
really
a
lot
of
relationships
to
be
built
out
and
lots
and
lots
of
chaos.
A
So
here's
a
small
example
of
how
we're
using
it
we're
creating
the
context
and
then
we're
able
to
grab
all
the
object
types
created.
This
is
a
smaller
example
and
then
what
we
can
do
is
just
grab
the
name
individual
name
of
that
of
those
object
types.
So
we
can
also
grab
the
fields
or
the
directives
or
the
implements
interfaces
it
uses
and
all
of
that
is
sort
of
accessible.
A
We
can
also
find
things
in
a
database
like
the
union
or
object
type
by
name
by
ID
by
anything
of
that
sort,
because
we
have
that
full
world
for
World
kind
of
Knowledge
full
knowledge
of
the
context,
and
that
brings
us
to
the
database
that
I
mentioned
about
70
slides
ago
and
I'm
happy
you
stuck
around.
Let's
talk
about
the
database,
so
all
those
relationships
that
we
we
find
out
and
we
we
build
out-
and
we
use
our
part
of
this
last
little
bit
of
the
compiler
bit.
A
A
A
So
salsa
is
a
generic
framework
for
on-demand
incrementalized
computation,
inspired
by
adaptan
glimmer
and
Russ
C's,
query
system
and
I
think
the
query
system
part
is
actually
the
the
more
important
one.
So
with
the
help
of
salsa
RS,
we
build
out
the
database
of
our
facts
and
we
also
build
out
a
query
system
that
goes
with
that
database.
A
What
do
I
mean
by
that?
Let's
take
a
look
at
like
a
small
example
of
getting
all
object,
types
from
a
given
schema.
We
want
to
grab
all
of
them.
This
is
the
query
that
does
that
this
is
a
query
from
our
database.
That's
backed
by
salsa.
What
object
type
does
is
that
it's
a
query.
It
calls
another
query.
That's
called
definitions,
definitions
cause
another.
It's
parent
query
called
document
document
calls
parse
parse
calls
input
string.
All
of
those
individual
ones
are
separate
queries.
A
What
salsa
helps
us
do
is
that
each
one
of
these
queries
is
memoized
and
is
cached,
which
means
if
any
one
of
these
gets
called
again
instead
of
recomputing
it.
We
just
look
for
it
in
the
cache
or
salsa,
looks
great
in
the
cache
I.
Don't
physically.
Do
this
salsa
does
this
so,
for
example,
the
definitions
query
gets
used
by
multiple
other
different
queries:
that's
not
just
object.
Types
be
object,
types,
but
also
input
objects,
operations,
enums
interfaces.
So
after
the
first
time
we
use
it
all
subsequent
times.
A
You
know
when
you
do
a
data
presentation,
you
kind
of
have
to
do
the
data
phone,
so
this
is
the
data
phone,
but
with
this
database
really
allows
us
to
do
is
to
be
able
to
provide
Diagnostics.
Because
for
us
to
be
able
to
say,
is
this
correctly
used?
Can
we
validate
this?
We
need
to
actually
have
all
the
context
of
a
given
input
of
a
given
graphql
schema
of
agreement,
graphql
document.
A
A
When
we
go
into
check
for
a
type
pet
defined
in
various
other
places
after
we've
erased
it.
We're
also
able
to
do
the
same
thing
check
for
whether
that
definition
exists
in
our
database
and
create
Diagnostics
that
follow
it,
and
these
are
just
examples
of
the
Diagnostics
we
could
create.
But
there
is
plenty
more
now
that
we
have
all
of
this
information
and
now
that
we're
able
to
build
out
pretty
much
anything
we
want
out
of
this
database.
A
What
do
we
talk
about
today
was
a
talk
about
Apollo,
rest.
Apollo
s
is
a
collection
of
crates.
We
specifically
focused
on
Apollo,
parser
and
Apollo
compiler,
the
two
crates
that
allow
us
to
kind
of
go
through
this
compiler
process
stage.
I
know
you're,
probably
sick
of
this
diagram
promise
it's
the
last
time,
so
the
two
of
them
make
up
our
compiler
stages
process.
A
We've
built
a
lossless
parser
and
a
fault
tolerant,
parser
and
we've
built
a
database
effect
that
lets
us
ask
all
sorts
of
questions,
but
primarily
what
we're
able
to
do
is
to
create
these
really
useful
Diagnostics,
that
people
should
be
able
to
use
to
fix
their
malform
code
and
have
like
good
and
functioning
graphql,
and
these
are
I'm
gonna,
give
a
flash
to
me
yet
again,
because
this
is
gorgeous
and
I
love
the
reporting
Library
all
right.
A
few
things.