►
Description
Presented by Max Brunsfeld, GitHub.
Developer tools that support multiple languages generally have very limited regex-based code-analysis capabilities. Tree-sitter is a new parsing system that aims to change this paradigm. We're in the process of integrating Tree-sitter into both GitHub.com and Atom, which will allow us to analyze code accurately and in real-time, paving the way for better syntax highlighting, code navigation, and refactoring support. We'll demo some new features that Tree-sitter has enabled in GitHub.com and Atom, discuss its implementation, and share thoughts on ways it could be used in the future.
About GitHub Universe:
GitHub Universe is a two-day conference dedicated to the creativity and curiosity of the largest software community in the world. Sessions cover topics from team culture to open source software across industries and technologies.
For more information on GitHub Universe, check the website:
https://githubuniverse.com
A
Sitter
I
was
worked
on
it
as
a
side
project
for
a
long
time
and
now
I'm
working
on
it
as
part
of
some
projects
at
github,
so
I'm
gonna
explain
what
it
is
and
then
I'll
talk
about
what
problem
I'm
trying
to
solve
with
it
and
then
I'll
show
you
some
of
the
things
that
we're
building
with
tree
sitter
today
at
github
and
finally,
I'll
go
into
some
of
the
algorithms.
The
true
center
uses
that
I
think
are
interesting.
A
Okay,
so
tree
sitter
is
a
library
for
parsing
code,
it's
written
in
C
and
C++,
and
it's
designed
to
be
used
in
applications
like
text
editors
or
github
comm
that
deal
with
code
written
in
many
different
languages.
So
it
can
parse
files
written
in
various
different
programming
languages
and
produce
syntax
trees
that
all
have
the
same
format
to
them.
A
A
Okay.
So
there
are
a
lot
of
great
developer
tools
out
there
in
the
world
that
can
do
really
useful
code
analysis
so
for
any
particular
programming
language
that
you
want
to
use.
There
are
great
ide
s
and
language
servers
and
parsers
that
can
understand
that
one
language
really
well,
and
yet
I
would
argue
that
many
of
the
tools
that
we
rely
on
the
most
as
programmers
still
only
understand
code
on
are
on
a
really
crude
level.
And
so
let
me
give
you
an
example
of
that.
A
One
of
the
biggest
things
that
helps
us
to
read
code
in
life
is
syntax,
highlighting
and
syntax.
Highlighting
is
something
that
we
experience
pretty
much
all
day
every
day,
while
we're
at
work,
whether
we'll
reading
code
on
github
or
on
a
on
Stack
Overflow
or
in
our
text
editor,
whether
it's
atom
or
vim
or
whatever,
and
so
here's
an
example
of
some
syntax
highlighting
that
you
might
see
in
any
one
of
those
places.
A
You
can
see
the
same
thing
happens
in
sublime
text
and
in
Visual,
Studio
code
and
similar
kinds
of
things
would
happen
if
you're
looking
at
code
on
github
or
Stack
Overflow
or
pretty
much
anywhere,
and
so
why
is
this?
Why
is
there
this
inconsistency
here
and
it's
because
none
of
these
tools
really
understand
go
code
or
any
code
for
that
matter?
A
They're
just
kind
of
looking
for
simple
patterns
in
the
code
that
they
can
identify
using
reg
X's,
and
so
they
can
tell
that
string
is
a
type
because
it
has
a
known
name,
it's
a
string,
but
they
can't
recognize
types
in
general
and
so,
as
programmers
most
of
us
are
kind
of
just
used
to
this
situation,
and
we
just
when
we
see
something
like
this.
We
don't
even
think
about
it.
A
Think
that
there's
kind
of
three
main
reasons
the
first
one
that
comes
to
my
mind,
is
performance,
and
so
in
a
text
editor
in
particular
syntax
highlighting
is
something
that
needs
to
happen
on
every
keystroke
and
for
the
editor
to
feel
nice
and
fast
needs
to
happen.
Pretty
fast,
probably
like
a
millisecond
or
less
would
be
ideal,
and
so
that
constraint
alone
kind
of
rules
out
our
ability
to
use
many
standard
parsers
that
we
might
otherwise
use.
A
And
aside
from
that,
in
order
to
use
a
regular
parser
to
parse
a
lot
of
programming
languages,
you
need
to
know
some
other
information
about
the
file
that
you're
parsing.
Besides
just
the
code,
that's
in
it.
So,
for
example,
if
you're
trying
to
parse
some
some
C
code
or
C++
code
or
Objective
C
code
with
clang,
you
would
need
to
know
you
need
to
be
able
to
read
in
all
of
the
other
source
files
that
that
file
includes
as
dependencies,
and
you
would
also
need
to
know
what
macros
are
defined
at
compile
time.
A
What
the
the
thing
that
I
think
is
the
biggest
is
the
issue
of
dependencies
so
for
many
languages,
the
standard
parsing
tool
chain
for
that
language
is
written
in
that
language
and
it
has
its
own
dependencies
that
are
associated
with
that
language
and
so
for
apps,
like
the
ones
that
I'm
talking
about
that
have
to
be
able
to
syntax,
highlight
many
different
languages
out
of
the
box.
If
we
had
to
call
out
to
a
different
external
tool
for
to
do
parsing
anytime,
we
wanted
a
syntax
highlight
some
code.
A
They
run
reg
X's
over
the
file
line-by-line
and
look
for
simple
patterns
and
as
users
we
kind
of
just
get
used
to
the
fact
that
syntax
highlighting
is
not
something
we
can
rely
on
to
tell
us
what
things
are
in
our
code,
and
so
my
question
is
well
what
if
there
were
an
accurate
parsing
tool
that
kind
of
met
all
these
constraints?
It
was
fast
enough
to
run
on
every
keystroke
in
a
text
editor,
and
it
also
didn't
require
any
additional
information
about
files
in
order
to
parse
them
and
it
didn't
have
any
dependencies.
A
A
So
this
is
a
screenshot
of
atom,
but
in
which
I've
ivory
implemented
the
syntax
highlighting
to
base
the
coloring
on
the
syntax
tree
that
is
provided
by
tree
sitter
at
all
times,
while
you're
editing
and
it's
the
same
piece
of
go
code.
That
I
showed
you
earlier
with
the
with
the
person
type.
But
now
you
can
see
that
it
doesn't
have
that
that
problem.
That
I
was
talking
about
earlier.
A
So
this
might
seem
like
kind
of
a
subtle
improvement.
But
to
me
it
really
increases
the
usefulness
of
syntax
highlighting
because
now
I
think
you
really
can
discern
the
structure
of
this
code.
At
a
glance,
because
the
every
element
is
kind
of
outlined
by
the
colors
and
just
to
illustrate
that
a
little
bit
further
I'll
switch
to
the
equivalent
piece
of
code
written
in
C
instead
of
go,
whoops,
okay,
so
go
C
and
you
can
kind
of
just
tell
from
the
colors
that
it
is
an
equivalent
piece
of
code.
A
You
can
see
now
that
it's
C
the
types
are
on
the
left
instead
of
on
the
right,
but
other
than
that.
It
pretty
much
looks
the
same
and
I
think
that
speaks
to
the
fact
that
syntax
highlighting
can
be
a
more
powerful
thing
in
reading
code.
If
it's
consistent
and
specific
like
this,
so
here's
the
same
buttons
are
a
little
twitchy.
Okay,
here's
the
same
code
written
in
C++
and
the
same
thing,
written
in
typescript,
same
thing,
in
Python
and
in
Ruby,
and
so
in
all
of
these
languages.
A
We
are
able
to
produce
more
consistent
and
also
more
specific
syntax,
highlighting
by
basing
it
off
of
this
syntax
tree.
The
tree
sitter
provides
because
that
that
syntax
tree
is
just
a
much
richer
source
of
information
about
the
structure
of
code.
Then
we
can
ever
figure
out
by
running
reg
X's
on
our
code
and
then
another
way
that
this
new
syntax
highlighting
system
is
an
improvement,
is
its
handling
of
long
lines.
A
A
This
performance
constraint
of
reg
X
based
systems
where
they
kind
of
operate
a
line
at
a
time,
and
so
the
longer
a
line
is
the
longer
these
reg
X's
take
to
run,
and
so
they
have
to
truncate
the
lines
so
that
they
don't
take
too
long.
But
with
tree
sitter
we're
able
to.
We
don't
have
that
constraint.
It
doesn't
matter
how
the
file
is
broken
up
into
lines.
A
So
we
can
syntax
highlight
the
whole
file
for
a
file
like
this,
which
is
nice,
and
then
one
last
thing
is
that
when
you
open
up
a
large
file
in
an
editor
like
Adam
or
Visual
Studio
code,
you'll
often
see
that
if
you
scroll
down
to
the
bottom,
it
takes
a
little
while,
before
the
that
code
gets
highlighted,
you'll
see
that
code
in
black
and
white
first,
because
the
the
syntax
highlighting
takes
a
while
this
is
on
this
slide.
I
just
have
an
example
of
parsing
a
large
file
with
tree
sitter.
A
In
this
case
it's
the
development,
build
of
react,
there's
one
big
20,000
line,
javascript
file
and
it
takes
like
69
milliseconds
to
parse.
So
that's
fast
enough
that
we
can
just
do
it
as
soon
as
you
click
on
the
file
to
open
it.
It
doesn't
add
anything
for
the
time
to
open
the
file,
and
so
you
won't
see
that
that
flash
of
unstyled
text
that
you
see
now
when
opening
a
large
file.
A
So
the
next
feature
that
I'll
show
you
is
code
folding.
So
most
of
you
are
probably
familiar
with
code
folding
in
a
text
editor.
It
lets
you
hide
certain
pieces
of
code
that
you
don't
want
to
look
at
right
now,
often
like
the
contents
of
a
function
body.
Things
like
that
and
the
way
that
it
works
in
most
text
editors
today
is
that
it's
based
on
the
indentation
of
the
file
so
lines
that
have
greater
indentation
get
hidden,
but
the
problem
with
that.
Okay,
so
there's
code
folding
in
action.
A
The
problem
with
the
indentation
based
approach,
though,
is
that
the
indentation
of
the
file
doesn't
always
match
what
you
would
intuitively
want
to
hide.
For
example,
in
C,
it's
common
to
write
functions
like
this
second
one
here
where,
where
each
parameter
to
the
function
appears
on
its
own
line,
and
so
actually
the
indentation
decreases.
When
you
go
into
the
body
of
the
function
and
so
different
editors
today
handle
this
differently
sublime
text,
I
think
just
doesn't
let
you
fold
functions
if
they're
written
this
way
and
Adam
in
Visual
Studio
code.
A
I
think
I
even
remove
all
the
indentation
from
the
file
and
I
could
still
fold
it.
Okay,
which
is
useful
because
sometimes
folding
is
a
thing
you
want
to
do
after
you've,
just
pasted
some
code
from
one
file
to
another,
and
so
the
formatting
is
wrong
and
you
want
to
get
an
overview
of
what
the
structure
is
to
see
if
it's
right.
A
So
the
next
feature
that
I'll
show
you
is
this
command
called:
extend,
selection,
that
I
added
to
Adam,
and
it's
a
simple
command
that
when
you
run
it,
it
just
selects
a
bigger
and
bigger
piece
of
your
code
based
on
the
syntax
tree.
So
it's
always
selecting
some
meaningful
unit,
and
you
can
see
here
I'm
able
to
select
bigger
and
bigger
pieces
of
this
JavaScript
data
structure,
and
then
you
can
go
back
down
if
you
go
too
far
up
and
what's
really
cool
is
when
you
combine
this
with
multiple
cursors.
A
So
in
this
example,
here
I
I
could
reformat
this
whole
data
structure
in
one
shot
kind
of
using
a
combination
of
multiple
cursors
and
extend
selection.
In
this
way,
that
would
actually
be
kind
of
time-consuming
just
to
do
if
you
were
trying
to
do
this
in
an
editor
without
extend
selection,
and
so
this
is
an
example
of
a
command.
A
A
But
what
I
like
about
the
way
that
I
did
this
is
that
I
just
hacked
this
in
this
feature
in
in
like
150
lines
of
JavaScript,
that
just
inspects
the
syntax
tree
and
decides
what
code
to
highlight,
and
it's
really
accurate
and
really
fast
and
I-
think
it
kind
of
illustrates
the
kind
of
cool
features
you
can
add
to
atom
as
an
atom
package
author.
Now
that
the
syntax
tree
is
there
at
all
times,
you
know
part
of
the
whole
point
of
atom
is
to
be
the
the
most
hackable
text.
A
Ok,
so
just
recapping
the
atom
features
that
I
added
so
far,
I
was
able
to
do
better
syntax,
highlighting
faster
and
make
code
folding
work,
regardless
of
how
your
file
is
formatted.
Out
of
this
extend
selection
command.
That
is
like
inherently
dependent
on
having
a
syntax
tree
and
added
this
other
cool
little
feature.
The
variable
highlighting
thing
that
illustrates
this
new
level
of
hack
ability
that
we're
achieving
with
atom
by
having
this
syntax
tree.
A
Okay
and
then,
aside
from
Adam,
we're
also
using
tree
sitter
on
github.com,
so
some
of
you
may
have
seen
this
feature
come
out
recently
where,
when
you're
looking
at
a
pull
request-
and
you
open
up
the
menu
of
files
that
have
changed
in
the
pull
request,
you
can
now
see
within
each
file
a
list
of
what
functions
changed
or
were
added
or
removed.
And
so
that
gives
you
this
nice
high-level
overview
of
what
happened
in
a
poor
quest,
and
it
also
lets
you
navigate
the
pull
request
according
to
these
meaningful
units
of
functions.
A
So
this
was
built
by
a
team
in
github
data
science
division
that
is
using
a
whole
bunch
of
really
sophisticated
algorithms
for
analyzing
and
comparing
syntax
trees,
and
they
are
doing
all
of
their
parsing
using
tree
sitter
because
it
gives
them
this.
This
uniform
way
of
parsing
many
different
programming
languages
and
so
between
the
work
that
they're
doing
now
on
github.com
and
the
work
that
I'm
doing
in
Adam.
A
We're
kind
of
building
up
this
shared
ecosystem
of
parsers
for
tree
sitter,
that
that
work
with
more
and
more
different
languages
and
I
think
that
once
we
ship,
those
adam
features
that
I
showed
you
to
the
public,
then
that
ecosystem
of
parsers
will
will
flourish
even
more
because
now
this
big
community
of
adam
users
will
be
able
to
contribute
support
for
more
and
more
different
languages.
And
so
the
plan
is
that
in
the
future,
you'll
be
able
to
use
this
one
tool
tree
sitter
to
parse
any
language
that
you
want
to
parse
in
this.
A
So
when
you
want
to
add
support
for
parsing
a
new
language
with
tree
sitter,
you
have
to
write.
What's
called
a
grammar
and
a
grammar
is
this
specification
of
a
language
that
outlines
what
are
all
the
elements
of
the
language
and
how
those
elements
can
be
built
up
out
of
other
elements
of
the
language?
So
this
is.
A
This
is
a
pretty
small
font,
but
this
is
a
screenshot
from
the
go
grammar
and
it
shows
things
like
a
for
statement
is
a
sequence
of
the
word
for
and
then
either
an
expression
or
a
range
clause
or
a
for
clause
and
then
a
block.
That's
the
idea
of
grammar-
and
this
is
pretty
similar
to
to
a
lot
of
parser
generator
tools.
You
have
to
write
a
grammar
to
be
able
to
parse
the
language,
but
one
difference
with
tree
sitter.
Is
that
there's
a
lot
of
too
popular
partial
generators
like
yak
or
bison?
A
You
have
to
write
the
grammar
in
this.
This
custom
grammar
language
with
tree
sitter.
You,
you
just
write
your
grammar
in
JavaScript,
using
the
few
simple
functions
that
I
wrote
that
build
up
the
rules
and
I
did
it
that
way,
because
well
for
one
thing:
it's
nice
to
not
have
to
go,
learn
somebody's
custom
language
when
you're
trying
to
use
a
tool,
but
on
top
of
that
it
it
lets.
A
So
once
you've
written
your
grammar,
you
run
this
command
line
tool
called
tree
sitter
generate
and
that
tool
runs
your
JavaScript
and
gets
back
the
grammar
object
and
it
converts
it
to
JSON,
and
then
it
passes
it
to
this
C++
library
called
the
tree
sitter
compiler
and
that
library
does
a
bunch
of
analysis
of
you
grammar
and
it
spits
out
this
C
code.
That
is
the
parser
for
your
language,
so
the
C
code
has
two
main
parts
to
it.
A
Like
that
and
it
exports,
this
one
object
called
a
tree
sitter
language,
and
this
is
what
it
looks
like
in
C.
If
you
wanted
to
use
that
object,
you
you
use
this
second
library,
this
small
Steve
library
called
the
tree.
Sitter
runtime
and
the
tree
centered
runtime
defines
a
few
different
types
that
you
use
for
parsing,
you
create
a
document
and
you
set
the
language
on
the
document.
A
The
language
is
the
thing
that
was
exported
by
that
generated
parser
file,
and
then
you
can
parse
code
and
it
gives
you
back
the
syntax
tree
that
has
a
very
simple
API,
that's
kind
of
like
the
Dom,
where
all
the
nodes
are
the
same.
They
have
the
same
interface
and
they
have
a
type.
They
have
an
ordered
list
of
children
and
each
node
knows
where
it
starts
and
ends
in
the
original
file
in
terms
of
what
line
and
column
it's
on
and
what
byte
it's
at
in
the
file.
A
A
Parsing
in
an
IDE,
so
talks
about
what
basic
parsing
framework
to
use
and
then
how
to
modify
that
parsing
framework
so
that
it
can
run
in
real
time
doing
these
incremental
updates
of
the
syntax
tree.
And
then
it
proves
that
those
incremental
updates
have
good
performance
characteristics
and
talks
about
how
to
handle
ambiguities
and
errors
and
a
lot
of
other
cool
topics
in
parsing
and
the
basic
parsing
framework
that
uses
is
called
LR
parsing.
A
It's
this
classic
theory
that
was
invented
in
1965
by
Donald
Knuth
and
it's
the
basis
for
a
lot
of
popular
parsers
today.
So
some
of
you
might
have
learned
about
this
in
college.
If
you
took
a
compilers
class,
but
I
didn't
do
a
computer
science
degree.
So,
in
order
to
build
tree
sit
or
I
had
to
go,
learn
about
this
on
Wikipedia
there's
a
really
good
Wikipedia
article
about
it
that
I
highly
recommend
so
I
thought
I
would
first
go
through
just
how
how
basic
LR
parsing
works.
A
And
then
the
next
token
it
would
find
is
a
plus.
But
when
it
looked
in
the
parse
table
for
what
to
do
there,
it
would
get
back
a
different
action
that
it
needed
to
do.
That's
called
a
reduction
which,
in
this
case,
would
mean
that
it
should
pop
off
three
things
from
its
stack:
the
X
x
and
the
y
and
group
them
together
into
a
new
parent
node
called
a
product
and
push
that
onto
its
stack.
A
And
so
then
it
would
push
again
the
plus
and
the
Z,
and
then,
when
it
got
to
the
end
of
the
string,
it
would
find
again
that
it
needed
to
do
a
reduction.
So
it
popped
off
the
stack,
the
products,
the
Y
or
sorry,
the
product,
the
plus
and
the
Z
and
group
them
together
into
this
sum
node,
and
that
would
be
the
final
tree.
So
that's
just
LR
parsing
and
you
might
ask
like
okay.
This
is
a
great
theory
of
parsing
but
trees
that
are
supposed
to
work
for
all
languages.
A
Like
is
this
one
parsing
theory,
gonna
work
for
every
language
and
answer
is
not
quite
LR.
Parsing
is
really
powerful.
It
can
recognize
a
really
large
set
of
grammars
and
most
programming
languages
are
designed
to
be
really
easy
to
parse
so
that
even
with
a
less
powerful
system
than
LR
parsing,
but
they
often
have
some
weird
quirk
to
them.
That
needs
to
be
handled
via
custom
code
and
so
I'll
show
you
what
an
example
of
one
of
those
weird
quirks.
A
So
these
are
two
pieces
of
JavaScript
code
and
first
one
we're
assigning
to
the
variable
X
the
value
of
the
variable
Y,
and
we
have
it
written
in
parentheses,
just
because
we
can
no
real
reason
and
then
in
the
second
example,
we're
assigning
to
X
on
arrow
function
that
takes
a
parameter
called
Y
and
then
returns
Z,
and
so
the
thing
about
these
examples
is
they
look.
The
code
looks
the
same
from
the
beginning
up
through
the
right
parenthesis,
but
the
syntax
trees
are
different.
A
This
syntax
node
called
an
expression
that
represents
things
that
are
being
evaluated
in
JavaScript,
whereas
in
the
second
example
we're
not
evaluating
why
Y
is
just
the
name
of
a
function
parameter,
so
it
has
this
different
structure
to
it,
and
the
reason
why
that's
a
problem
for
a
normal
LR
parser
is
that,
like
I
said
an
LR
parser,
never
backtracks,
that's
what
makes
it
so
efficient
and
so
once
it
has
pushed
a
tree
onto
its
stack.
That
represents
some
part
of
the
code.
A
It
needs
to
push
something
on
its
stack
representing
the
Y
and
so
that
either
needs
to
be
an
expression
in
a
case
where
we'd
be
evaluating
Y
or
not
in
the
case
where
Y
is
just
a
function
parameter,
and
so
it
doesn't
know
which
one
of
those
things
that
should
do
until
it
would
look
further
ahead
in
the
code
to
see
whether
there's
an
arrow
or
whether
there's
a
semicolon
or
something.
And
so
let
me
show
you
the
way
that
tree
sitter
deals
with
this
problem.
A
A
The
parsers
stack
like
what
I
showed
you
earlier,
when
I
animated
through
them,
but
now
they're
in
this
different
format
that
is
actually
generated
by
the
tree,
sitter
tool
that
it
can
generate
these
stacks
to
help
you
debug,
what's
happening
with
a
parse,
and
so
in
these.
The
names
of
the
trees
are
above
the
lines
and
what's
in
the
bubble,
is
the
number
that
represents
the
state
that
the
parser
is
in,
and
so
you
can
see
it
starts
out
pretty
straight
forward.
A
But
then,
when
we
get
a
little
further
ahead,
we
can
see
that
there's
an
arrow
and
so
an
arrow
wouldn't
be
valid
after
a
parenthesized
expression.
So
at
that
point
we
can
chop
off
one
of
the
branches
of
the
parse
stack
and
just
proceed
on
with
regular
LR
parsing,
and
now
that
there's
only
one
interpretation,
and
so
we
end
up
with
this
arrow
function
like
that,
and
so
that
is
the
that's.
A
A
So
the
file
is
in
an
invalid
state
where
it
wouldn't
compile,
and
so,
unlike
a
lot
of
parsing
tools
that
just
halt
and
give
you
an
error
message
when
the
they
get
to
an
invalid
point
in
the
file,
Teresa
turn
needs
to
actually
return
a
syntax
tree
always
and
just
indicate
where
the
errors
are
as
part
of
the
tree.
So
you
can
see
here
in
this
example.
This
is
some
JavaScript
code,
where
you
have
an
if
statement,
and
then
you
started
typing
a
for
loop
before
it,
and
so
what
Tree
Center
does?
A
A
A
So,
oh
one
other
thing
about
that
that
algorithm
of
using
the
the
parse
tax
splitting
to
resolve
errors
is
a
novel.
As
far
as
I
know,
I,
don't
I
haven't
read
about
any
other
academic
papers
or
popular
parsers
that
do
that,
but
it
works
really
well
for
tree.
Sitter
lets
us
oftentimes
produce
these
really
intuitive
syntax
trees,
even
when
you're
in
the
middle
of
doing
something.
A
Ok,
so
the
last
algorithm
I'll
show
you
is
the
incremental
parsing
algorithm.
So
this
is
what
lets
tree
sitter
quickly
update
the
syntax
tree
after
you
make
an
edit
so
say
that
you
had
some
JavaScript
code
with
these
three
statements:
VAR
a
equals
new,
be
a
dot.
C
return
a
so
it
shows
the
syntax
tree
for
it.
It's
kind
of
small
the
details,
don't
matter
that
much
and
then
say
you
edited
this
code
and
you
inserted
an
argument
to
this
function
here.
A
So
the
way
the
tree
Center
would
process
that
is
when
you
inserted
the
character
tree.
Sitter
would
look
in
its
existing
syntax
tree
to
find
all
of
the
nodes
in
the
tree
that
contained
that
spot,
where
you
edit
it
and
it
would
mark
them
as
having
had
a
character
inserted
into
them.
So
those
are
the
ones
in
green
in
this
diagram
and
then
it
would.
It
would
actually
proceed
to
parse
the
file
as
if
it
were
just
going
to
parse
it
normally
from
the
beginning,
and
so
it
would
start
out
with
an
empty
stack.
A
But
now,
instead
of
having
to
run
that
tokenizer
function
for
every
single
token
in
the
file,
it
has
this
old
tree
as
a
reference,
and
so
what
it
can
do
is
just
look
and
find
the
first
node
in
the
old
tree.
That
is
not
edited
and
it
can
reuse
that
it
can
push
that
directly
onto
its
stack.
So
in
this
case
it
finds
that
the
that
the
variable
declaration,
the
first
piece,
isn't
changed.
So
we
can
just
just
push
that
directly
onto
its
stack
and
skip
a
lot
of
the
intermediate
steps.
A
That
would
happen
in
order
to
build
that
up,
and
then
it
can
reuse.
The
next
few
things
to
this
member
expression,
that's
the
a
dot
C
and
then
it
can
reuse
the
left
parenthesis
and
then,
when
it
gets
to
the
site
of
the
edit
that
you
made,
it
has
to
actually
do
the
parsing
from
scratch.
So
it
has
to
rerun
the
tokenizer
function
and
get
back
a
new
identify
our
token
for
that
letter
D.
A
A
Oops
skip
okay,
reuse,
the
semicolon
and
build
up
a
new
statement
node
for
that
that
one
statement
and
then
it
can
reuse
the
whole
return
statement
afterward
and
then
build
up
a
new
program
that
so
it
gives
you
a
new
syntax
tree,
but
that
shares
a
lot
of
the
elements
of
the
old
syntax
tree
and
in
this
way,
the
time
that
it
takes
to
parse
is
is
proportional
to
the
number
of
edits
you
made
not
proportional
to
the
size
of
the
whole
file.
All
right.
A
So
the
next
steps
for
me
on
adam
is
to
kind
of
get
all
these
features
that
I
show
to
you
that
are
still
sort
of
in
development.
Almost
done
get
them
out
there
and
I
think
that'll
happen
in
a
few
months
and
then
we're
gonna
start
adding
support
for
more
languages.
Right
now
we
have
go
c
c++
java
script,
typescript,
Ruby,
Python,
bash,
and
so
we'll
be
adding
more
and
more
so
we
get
to
all
the
languages
that
Adam
supports
and
then
on.