►
Description
Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Building a Distributed API Gateway with a Service Mesh - Rei Shimizu, Waseda University & Cynthia Coan, Tetrate
API Gateways are going through an identity crisis. As microservice architectures proliferate we're learning that really, we need API Gateway functionality between all of our services, and not just at ingress where API Gateways have been deployed traditionally. The ability to dynamically extend Envoy at runtime via WASM opens the door for many new capabilities, including moving API Gateway functionality into the service mesh layer. In this talk, we'll explore how we helped extend Envoy to support executing WASM as a safe, sandboxed way to run arbitrary code in the proxy. We'll then go further by showing how we used WASM to move functionality typically reserved for the edge of our deployments - API Gateway functions like token validation, rate limiting, and payload transformation - into the service mesh in a real user's deployment.
https://sched.co/ZelP
A
And
we're
going
to
be
talking
about
building
a
distributed
api
gateway
with
a
service
mesh
so
taking
api
gateway
functionality
and
moving
it
into
a
service
mesh.
So
let's
go
ahead
and
get
started
so
we're
going
to
to
give
you
an
idea
of
what
we're
going
to
be
talking
about.
First,
we're
going
to
be
talking
about
what
a
service
mesh
is.
You
know
what
it
provides
next,
we'll
go
over
what
an
api
gateway
is
and
sort
of
degeneralize
that
term,
because
it
means
a
lot
of
different
things
to
a
lot
of
different
people.
A
It's
you
know.
The
name
is
sort
of
going
through
an
identity
crisis.
Next
we'll
be
talking
about
envoy,
which
is
a
particular
implementation
of
a
proxy
that
we
will
be
using
in
this
talk.
So
we
just
want
to
give
sort
of
a
quick
forward
about
what
it
is
next
we'll
be
talking
about
using
wasm
in
envoy
as
well
as
why
we
would
want
to
use
wasm
at
all
in
order
to
build
extensibility
into
these
proxies
and
last
we'll
give
a
demo
of
adding
api
gateway
functionality
into
a
service
mesh
with
envoy.
A
So,
like
I
said,
let's
go
ahead
and
get
started,
so
what
is
a
service
mesh?
A
service
mesh
is
all
about
bringing
a
layer
of
standardization
to
the
network
level
when
you're
dealing
with
lots
of
different
services,
so
you
can
imagine
in
a
sufficiently
large
company
or
maybe
a
medium-sized
company
right.
You
have
20
different
services,
maybe
written
in
three
different
languages
right
and
they
and
you
want
to
apply
a
policy
across
all
of
your
services
at
once.
It
can
be
very
hard
to
do
that
without
any
sort
of
layer
there.
A
For
example,
if
you
wanted
to
enforce
encryption
between
all
of
your
services,
because
let's
say
you
had
a
new
requirement
pop
up
right,
it
may
be
very
hard
to
actually
achieve
that
because
you
have
to
you
know,
implement
a
library
each
in
the
three
languages
and
then
you
have
to
wait
for
all
of
the
20
services
to
adopt
it,
and
then
you
have
to
test
in
each
of
the
20
services
that
they're
all
doing
what
they're
supposed
to
do
I.e.
The
encryption
is
working.
The
way
you
expect
it
to
with
this
with
a
service
mesh.
A
This
problem
is
actually
much
easier
because
the
idea
is,
you
have
a
single
slide
car
that
sits
next
to
every
single
service.
That
actually
does
all
of
that
complex
logic
for
you,
so
this
sidecar
figures
out
how
to
do
mtls
or
service
to
service
encryption
and
your
individual
services
don't
have
to
figure
it
out
themselves.
This
makes
it
really
easy
to
add
in
new
services.
A
So
if
you
want
to
add
in
a
new
service,
perhaps
written
in
a
new
language,
you
don't
have
to
go
build.
You
know
a
library
to
do
transport
encryption.
The
way
your
company
does
it
you
just
let
it
talk
to
the
sidecar
and
the
site
car
handles
it
all
for
you
right.
This
also
is
a
really
nice
way
to
be
able
to
query
policies
across
the
whole
mesh.
Because,
again
you
have
one
configuration
source
for
how
you
do
all
your
networking.
A
This
can
be
really
useful,
also
because
it
sort
of
abstracts
networking
logic
away
from
application
logic.
So,
let's
say
you're
moving
a
service
into
a
new
cloud
provider
to
a
different
cloud
provider.
Maybe
you're
adopting
cloud
you
don't
have
to
go.
You
know
redeploy
all
of
your
applications
anytime.
You
deploy
a
new
service
into
the
cloud.
You
can
just
send
a
configuration
update
that
says:
hey.
You
know
route
all
of
your
traffic
to
this
new
service
on
the
cloud,
and
it's
just
a
single
configuration
update.
A
You
don't
actually
have
to
deploy
a
new
version
of
your
app
or
deploy
an
app
configuration
update
right
because
they're
decoupled,
so
it's
much
lower
risk
and
much
easier
to
make
changes
as
time
goes
on.
This
is
why
a
service
mesh
is
really
useful,
and
why
would
you
would
want
one?
So
an
api
gateway
is
actually
very
similar
to
a
service
mesh
in
our
minds
right.
A
It
just
gets
the
user
authentication
information
in
a
header.
It
doesn't
actually
have
to
figure
out
how
to
do
this.
This
consistency
at
your
front
door
is
also
very
useful,
but
it
got
us
to
thinking
you
know
if
the
service,
mesh
and
api
gateway
are
all
about
providing
consistency
at
the
networking
layer.
They're
just
at
two
different
networking
points.
Would
it
be
possible
to
combine
these
two
things
into
the
exact
same
system.
So
that
way,
you
don't
have
to
maintain
two
separate
systems.
A
For
example,
if
you
want
to
change
how
authentication
is
done,
you
change
it
in
one
place
and
you
don't
have
to
go
change.
You
know
user
authentication
at
the
front
door
and
service
the
service
authentication
for
your
sidecar
implementation,
for
your
service
mesh
right
having
one
place
to
change
that.
If
you
get
an
influx
of
traffic,
you
no
longer
have
to
scale
up
your
api
gateway
and
your
actual
services
behind
the
api
gateway.
You
just
scale
up
your
service
and
your
services.
A
Sidecar
is
handling
all
of
that
front
door
traffic,
as
well
as
the
service
to
service
communication.
We
think
there's
a
lot
of
exciting
things
that
can
happen
here,
and
this
is
what
an
api
gateway
truly.
Is
it's
not
something
that
is
like
combining
responses.
You
know
that's
getting
into
feature
creep
territory
where
it
can
be
really
hard
to
make
changes,
because
the
api
gateway
is
just
doing
too
much,
but
when
you
think
about
an
api
gateway
like
most
api
gateways,
are
where
they're
just
providing
this
consistent
set
of
policies
on
the
networking
front
door.
A
It
can
also
work
really
really
well
as
just
being
part
of
your
service
mesh
sidecar,
where
the
sidecars
are
just
handling
all
of
that
transparently,
because
they're
doing
a
lot
of
the
same
things,
you
know
they're
routing
the
logic
is
exactly
the
same
from
routing
from
a
service
in
as
well
as
a
service
to
service
communication
right,
the
routing
is
exactly
the
same
in
an
authentication,
although
it
may
be
different,
you
know
you're
validating
a
jwt
instead
of
a
certificate.
You
know
it's
the
exact
same
procedure.
A
It's
just
authentication,
you're,
just
authenticating
something
different
right,
but
these
two
things
can
be
combined
without
introducing
too
much
feature
creep.
So
what
is
envoy
for
those
of
you
who
haven't
heard
of
envoy
before
just
know?
It's
a
very,
very
common
proxy
that
is
used
in
a
lot
of
service
meshes
as
well
as
api
gateways.
Today,
in
this
session,
we're
going
to
be
using
envoy
as
the
sidecar
for
our
service
mesh.
A
It's
got
this
idea
of
filters
that
we'll
talk
about
more
here
in
a
second,
and
it's
also
got
this
great
dynamic
configuration
system
which
is
called
xds,
which
will
allow
us
to
not
only
push
updates
very
easily
but
also
track
the
progress
of
when
they
get
applied.
So
it's
very
easy
to
see.
You
know
from
point
a
to
point
b.
A
Has
this
idea
of
filter
chains,
so
an
envoy
is
going
to
be,
it
has
a
listener
that
is
listening
on
a
port
and
is
going
to
have
traffic
coming
on
the
inbound
right
and
that
that
listener
is
then
going
to
pass
off
the
traffic
to
a
series
of
filter
chains
and
a
filter
chain
is
kind
of
a
dense
concept.
But
the
basic
idea
is
it's
a
series
of
filters
and
each
filter
is
going
to
do
one
thing
to
the
request
or
or
response
and
then
move
on.
A
So,
for
example,
it
may
parse
the
request
as
http,
and
then
you
know
let
the
router
now
match
on
http
level
things
right.
So
it's
going
to
validate
that
it's
valid
http
traffic
and
then
continue
on.
Maybe
you
have
another
filter
that
then
takes
an
http
header
and
validates
the
jwt
and
then
passes
it
on
right.
So
you
have
these
series
of
operations
that
are
filters
and
they
they
get
put
in
a
chain
right,
and
so
once
that
chain
completes
on
the
request,
it's
going
to
be
sent
to
a
router.
A
The
router
is
then
going
to
route
the
traffic
to
an
up
a
particular
upstream.
That
upstream
is
then
going
to
respond,
and
its
response
is
again
going
to
go
through
a
series
of
filters
called
the
filter
chain
that
are
each
going
to
do.
One
thing
to
the
response:
maybe
it's
just
adding
a
metric.
Maybe
it's
actually
changing
the
response.
It
is
up
to
you
and
then
it's
going
to
go
back
through
the
listener
and
send
a
response
to
the
user.
A
So
we
have
some
extensibility
in
envoy.
So
envoy
has
recent
recently
announced
that
it
is
going
to
you
know,
support
being
extended
through
wasm.
We
will
talk
about
why
we
chose
asm
here
in
a
bit,
but
this
is
a
very
good
thing.
There
are
official
sdks
out
there
for
c
plus
and
rust.
So
if
you're
developing
with
c
plus,
plus
and
rust,
you
will
have
a
very
well
worn
path
for
development
and
hopefully
it
will
be
very
easy
to
do.
A
There
are
also
unofficial
sdks
for
go
and
assembly
script,
so
you
know
again
a
bit
more
of
not
as
well
worn
of
a
path
but
a
path
that
you
can
still
hopefully
easily
hop
on
to
and
then
because
we're
developing
wasm.
You
know
there
are
many
other
languages
that
can
compile
down
to
wasm
that
you
can
add
support
for
yourself.
A
Wasn't
filters
are
going
to
be
delivered
dynamically
to
envoy
with
the
xds
configuration
system,
so
this
means
it's
possible
to
update
the
actual
filters.
You
know
that
are
doing
operations
on
a
request
such
as
authorization
without
ever
restarting
envoy.
So
we
don't
have
to
worry
about
you
know,
losing
any
connections
or
losing
any
traffic.
We're
just
going
to
deliver
configuration
updates
exactly
as
we
would
deliver
configuration
updates
normally
without
wasm.
A
We
would
normally
have
to
build
our
extensions
as
c
plus
into
the
envoy
binary,
which
means
any
time
we
wanted
to
update
that
particular
extension.
We
would
have
to
rebuild
envoy,
you
know
from
the
ground
up.
This
is
what
things
like
istio
have
done
right.
They
have
maintained
their
own
build
of
istio
and
that's
quite
a
high
bar
in
order
to
be
able
to
do
that.
A
So
what
is
wasm
wasm
is
actually
kind
of
similar
to
java
bytecode
right,
where
it's,
this
generic
typed
binary
that
multiple
languages
can
compile
to
it
was
originally
built
for
the
browser
to
sort
of
be
an
analogous
solution
to
javascript
right.
However,
it's
generally
much
faster
than
javascript,
because
it
has
those
strict
types
that
are
very
easily
optimized,
as
well
as
there's
no
parsing
time
right,
so
it's
very
lightweight
to
deliver
and
then
start
executing.
A
So
why
would
we
even
choose
wasm
if
it
was
built
for
a
browser
as
sort
of
another
version
of
javascript?
Why
would
we
even
want
to
to
use
this
to
extend
our
proxies?
Well,
there
are
a
lot
of
benefits:
first
off,
it's
a
very
isolated
environment.
So
if
the
wasm
filter,
you
know
panics,
it's
not
going
to
panic
the
entire
envoy
proxy
right,
so
the
entire
proxy
is
not
going
to
crash
just
because
this
particular
rasm
filter
crashed.
A
That's
a
very
nice
property,
especially
when
you're
an
api
gateway
and
you're
dealing
with
untrusted
traffic,
and
you
may
have
a
bug
in
your
code
right.
We
all
make
bugs
so
not
having
to
worry
about
bugs
coming
in
and
crashing
the
entire
proxy
and
only
affecting
that
one
request
is
very
useful,
not
only
that,
but
we
can
also
limit
cpu
and
memory
consumption,
so
we
can
validate.
You
know
that
one
person
can't
take
up
all
of
the
cpu
and
prevent
code
from
being
executed.
A
That
normally
would
be
executed
because
one
person
is
taking
up
all
of
the
cpu,
and
you
know
if
you're
writing
filters
and
say
c
plus
plus,
which
is
you
know,
has
a
long
history
of
memory,
corruption
bugs
you
can
sandbox
the
runtime
to
prevent
these
malicious
operations
from
happening
on
the
host
environment.
So
it's
a
bit
safer
than
just
writing.
A
normal
c
plus
filter.
A
There
are
also
many
languages
that
can
be
compiled
down
to
asm,
right
c,
plus
plus
rust,
go
typescript
right
and
a
long
list
of
others.
So
we
get
support
for
all
of
these
languages
out
of
the
box,
so
you
don't
have
to
go
and
learn
a
new
language
in
order
to
develop
a
wasm
filter.
This
makes
it
very
easy
to
expand
for
everyone,
not
only
that,
but
it
gives
us
a
chance
to
be
portable
right,
so
we
can
validate
that.
A
You
know
your
code
isn't
just
going
to
work
on
envoy
or
your
code
isn't
just
going
to
work
on
another
proxy
that
it'll
work
across
all
of
the
proxies.
So
as
you're
as
your
infrastructure
evolves
and
as
you
have
your
needs,
you
will
be
able
to
migrate
your
proxies
with
zero
downtime
right
and
without
having
to
rewrite
any
of
your
code,
because
you
can
just
import
these
filters
from
one
to
the
next.
This
is
a
really
nice
property
that
we
really
want
to
ensure.
A
So
what
is
proxy
wasm?
Well
proxy
wasm.
Is
that
consistent
interface
between
proxies
right
now?
There
are
two
proxies
that
implement
this
particular
interface
right.
This
abi
right-
and
this
is
envoy
and
apache
traffic
servers,
so
any
filters
you
write
using
proxy
wasm
for
envoy
will
also
just
work
with
apache
traffic
server
in
any
apache
traffic,
server,
proxy
wise
and
will
just
work
with
envoy
right
and
hopefully,
as
time
goes
on
more
and
more
proxies
adopt
this.
A
So
again,
as
your
infrastructure
evolves
right,
you
will
be
able
to
take
the
same
code
with
you
as
you
move
to
different
infrastructures,
so
the
basic
architecture,
sort
of
connecting
it
back
to
that
envoy
architecture
we
talked
about
right
is
there's
just
going
to
be
a
wasm
filter
inside
the
filter
chain
right.
The
envoy
worker
thread
is
going
to
come
along.
It's
going
to
hit
the
wasm
filter
and
it's
going
to
be
it's
going
to
talk
to
a
proxy
wasm
sort
of
shim
layer.
This
is
implemented
by
envoy.
A
A
It's
specific,
you
know:
hey
I've
had
request
headers
come
in
and
prox
and
that
little
shim
layer
is
actually
going
to
normalize
it
to
something
that
is
consistent
across
all
of
the
proxies
right
and
again
there
are
official
sdks
for
c
plus
plus
and
rest.
So
if
you
want
to
write
in
either
of
those
two
languages,
you
get
this
nice
consistent
interface
across
all
of
the
proxies
that
implement
it.
A
So
this
is
an
example
filter
that
sort
of
goes
over.
What
we're
going
to
do
right,
so
we're
going
to
match
on
a
path
right
that
contains
private
and
if
there
is
private
in
the
path
we're
going
to
check
the
authorization
header
and
then
we're
going
to
validate
it
and
if
not
we're
going
to
respond
with
a
401,
and
this
code
is
going
to
work
across
apache
traffic
server
as
well
as
envoy,
it's
written
in
rust
and
can
be
dynamically
updated
right.
A
B
Situation,
this
demonstration
is
conducted
on
the
compost
based
environment
to
run
services
from
proxy
and
control
frame.
As
the
container,
I
developed
a
simple
control
frame
to
deliver
wasm
code
to
data
brains
for
this
demonstration,
which
is
built
on
the
top
of
go
control
plane,
which
is
a
library
to
build
control
frame
interviews
from
android
community.
B
B
Production
we
use
ghettoemboy
to
prepare
the
demonstration
environment.
This
tool
provides
the
easiest
way
to
get
involved
binary.
We
can
start
envoy
without
installing
binaries,
by
using
package
manager.
For
example,
we
can
start
washing
enable
the
envoy
with
this
single
wine
command.
This
tool
is
very
useful
to
verify
your
emboy.
B
In
today's
demonstration
we
use
this
tiny
size,
mesh
running
blue
and
green
services
that
have
authentication
required
contents
service
that
returned
json
web
token.
We
provide
the
api
gateway
ability
via
emboy,
wasn't
extensibility
by
providing
authentication
layer
written
in
web
assembly
with
internal
manufacture,
was
the
ck
to
brew
and
green
services
to
preserve
authentication
required
contents.
B
B
B
B
B
B
B
B
B
B
B
B
B
B
C
C
B
Okay,
thank
you
for
listening
recently,
we
released
a
ghetto
emboy
which
has
abilities
to
accelerate
our
webassembly
filter
development.
It
provides
a
boilerplate
with
internal
manufactured
proxy.
Wasn't
sdk
it
supports.
On
last,
the
internal
manufacture
was
the
ck
is
built
on
the
top
of
proxy.
Wasn't
blasted
ck
with
some
conveniences
and
the
easiest
way
to
build
and
test
webassembly
filter
with
docker
container
to
run
developed,
wasn't
filter
with
envoy
without
importing
a
complex
commands
in
this
session.
B
A
A
They
have
an
entire
page
set
on
their
documentation
site.
If
you
just
search
istio
asm,
it
should
pop
right
up
and
they're
continuing
to
do
work
to
make
it
easier
to
integrate
wasm
filters
as
a
user,
including
introducing
new
sorts
of
xdps
apis
that
make
it
very
easy
to
distribute,
was
and
filter
specifically.
A
The
next
question
I
see
is:
is
development
and
maintenance
of
the
service
mesh,
slash
api
gateway,
something
all
developers
in
a
company
end
up
supporting
or
is
it
handled
by
a
specialized
team?
You
know,
I
think
that
depends
effectively
on
the
organization
right.
It's
very
much
an
organizational
structure.
I
think
most
people,
though,
end
up
adopting
sort
of
a
hybrid
where
there
is
one
team
that
runs
sort
of
the
service
mesh
and
making
sure
it
gets
upgraded
and
maintained,
and
then
developers
are
configuring,
their
services
for
the
service
mesh
right.
A
The
next
question
I
see
is,
if
I
understand
correctly,
western
support
is
not
yet
officially
available
in
an
onboard
machine,
mainline
master
version.
Do
you
build
an
envoy
with
wasm
on
your
own,
or
maybe
you
use
the
istio,
which
I
understand,
uses
a
custom
way
for
so
it
is
not
in
the
main,
android
proxy
ford,
slash
envoy
repository
that
being
said,
envoy
does
provide
official
builds
of
it
for
onboard
proxy
slash
wasm.
A
B
A
Next
question
is:
would
you
agree
that
api
gateways
are
still
relevant
for
reasons
like
the
following
api
gateways?
Can
be
region
aware
from
route
traffic
to
multiple
message?
Api
gateways
are
mature
and
api
management
functionalities
like
publishing,
open
api
based
documentation,
supporting
automatic
sdk
generation
providing
mock
endpoints.
I
think
you
know
those
are
certainly
useful
features
of
an
api
gateway.
A
That
being
said,
I
don't,
I
don't
think
they
are
necessarily
the
same
problems
right,
because,
ideally
you
know
combining
those
two
things
I
think
sort
of
gets
you
into
feature
group
territory
where
you
sort
of
have
one
thing
doing
a
bit
too
much.
Those
are
all
incredibly
useful
things
to
have.
I
don't
know
if
they
necessarily
belong
in
an
api
gateway.
It
just
feels
like
a
little
bit
too
much
to
manage
at
one
layer.
For
me.
That
being
said,
obviously
I
you
know.
A
Looks
like
another
high
priority.
One
came
in
just
scrolling
through
here.
Can
different
wasm
code
be
deployed
to
different
paths?
The
answer
is
yes,
they
can
be.
So
if
you
remember
in
the
talk,
we
talked
about
filter
chains,
filter
chains
can
happen
per
route,
so
what
you
can
do
is
you
can
have
a
specific
filter
in
a
filter
chain
that
only
happens
on
a
specific
route,
so
the
wasm
code
will
only
activate
on
that
specific.
C
A
A
A
I
think
I
think
the
service
mesh
provides
more
than
than
just
filters,
there's
a
lot
of
management
that
it
makes
easier
right.
So
the
service
mesh
is
really
about
controlling
all
of
those
side
cards
which
you
you,
ideally
don't
want
to
do
from
within
the
sidecar
itself.
That
being
said,
you
know,
I,
I
don't
think
it's
feasible
for
a
any
particular
service
mesh
to
implement
all
of
the
functionality
for
every
single
person
ever
and
so
having
a
way
to
extend
it
to
fit
your
specific
use
case
with
wasm.
A
B
B
A
A
Okay,
is
it
possible
for
a
filter
to
use
an
external
service
if
it's
sandboxed
in
wasm?
Yes,
the
the
sandboxing?
Is
you
know
not
something
like
a
firewall
or
something
that
would
prevent
access?
Obviously,
I'm
sure
you
could
configure
the
sandbox
in
a
way
that
it
did
block
network
access
if
you
didn't
want
it
to
communicate
out.
But
yes,
by
default
filters
can
talk
to
external
services.
A
B
A
A
That
being
said
with
rate
limiting,
as
we
mentioned,
we
think
it
can
also
provide
benefit,
because
not
only
are
you
removing
it's,
not
necessarily
the
network
step
but
you're,
removing
an
extra
piece
of
infrastructure
that
you
have
just
you
have
to
scale,
because
your
rate
limiting
usually
takes
a
second
to
begin,
because
you
want
to
give
your
users
the
ability
to
use
your
platform.
Well,
so
that's
that's
where
those
links
came
from.
I
think
we
can
do
maybe
one
or
two
more.
A
You
know,
I
think,
that's
a
totally
fair
concern
right.
Any
particular
piece
of
technology
handling
too
much,
I
think,
can
be
dangerous.
I
think
the
I
think
the
best
answer
there
is
making
sure
I
sort
of
brought
it
up
earlier
feature
creep.
Is
making
sure
you
don't
push
too
much
to
the
networking
layer?
Ideally
the
networking
layer
should
realistically
be
you
know
like
it.
It
should
effectively
handle
the
actual,
giving
the
user
identity
and
putting
in
a
consistent
place
retrying
and
then
authorization
right,
and
you
really
retrying
and
rate
limiting.
A
You
really
don't
want
too
much
more
than
that,
and
so
I
don't
necessarily
and
obviously
I
think
you
may
still
have
some
some
code
in
your
app
that
does
a
little
bit,
but
most
of
the
functionality
should
be
handled
by
that
service
mesh,
and
then
you
don't
want
to
grow
it.
So
I
see
a
lot
of
people
doing.
A
You
know
like
combining
responses
and
api
gateway
and
service
meshes,
and
that's
where
I
personally
get
a
little
bit
of
a
sick
stomach,
because
I
I
worry
about
changing
that
code
right
and
having
it
handling
too
much
and
having
it
fail.
So
I
I
think
the
answer
there
is,
you
know
the
network
is
going
to
fail
anyway.
So
having
one
place
where
you
have
a
consistent
failure,
policy
is
good,
but
at
the
same
time
I
I
totally
agree.
I
think
you
need
to
be
careful
in
how
much
you
push
to
that
layer.