►
From YouTube: Building Multi-Tenant Routing And Scaling With Envoy - Yiming Peng, Amazon Web Services, Inc.
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe in Amsterdam, The Netherlands from April 17-21, 2023. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Building Multi-Tenant Routing And Scaling With Envoy - Yiming Peng, Amazon Web Services, Inc.
AWS App Runner is using Envoy underhood for its multi-tenant request-routing, load balancing and auto scaling. In this session, the AWS App Runner service team will share Envoy user experience. Journey of building an Envoy-based scalable request-router from developer point of view. Reason to choose Envoy and benefits it brings to the product. Lessons learnt and best practices for maintaining and operating Envoy-based systems in day-to-day work life.
A
A
I
want
to
share
the
journey
from
the
end
user
point
of
view
by
using
Envoy
I
want
to
also
give
back
to
the
community
to
share
the
lessons
learned
and
experience
by
using
Amway
as
a
builder
and
hope
it
brings
value
to
the
community
and
help
the
products.
Growth
and
I
also
want
to
use
this
opportunity
to
sense
and
appreciate
all
the
support
from
the
Amway
community.
No
matter
it
is
internal
or
external.
A
By
the
way,
Amazon
has
internal
interest
group
for
the
Amway
proxy,
with
like
around
160
people
I'm,
the
founder
and
maintainer
of
lead.
Channel
I
also
want
to
send
to
the
platform
and
I
want
to
use
this
platform
for
the
future
discussion
and
communication
I'm
the
fan
of
kubecon
for
years,
I'm
really
happy
and
excited
YouTube
here
to
talk
with
you
guys
for
the
topic.
A
A
little
bit
introduction
for
apron,
as
I
mentioned
before
so
AWS
App
Partners
G8
in
the
2021,
is
a
fully
managed
service
that
makes
it
easy
for
computers
to
quickly
deploy
container
web
service
and
APS
at
a
scale
with
less
and
zero
infrastructure
experience
required
here
is.
Our
official
website
feel
free
to
check
it
out.
A
So,
from
the
user
point
of
view,
that
problem
provides
managed
experience
for
customers,
app
hosting
request
the
routing
load,
balancing
and
auto
scaling
networking
the
Ingress
egress,
both
observability
cicd
deployment
and
safe
deployment,
and
more
so
customer
just
need
to
have
their
GitHub
source
code
or
the
ECR
image
private
or
public,
and
deploy
the
image
of
source
code
connection
to
a
problem.
Apprent
will
directly
make
it
up
and
running
for
your
web,
app
and
or
API
service
and
I
prom
to
output
outcome
and
public
URL
for
clients
to
access.
A
A
Multi-Regions
region
expensible,
of
course,
we
need
to
make
sure
we
can
build
new
regions
all
the
time
with
the
solution
with
the
existing
architecture,
you
need
to
be
multi-tenant,
we
are
serving
hundreds
and
millions
customers
as
a
platform
and
solution,
so
we
need
a
requested
router
to
be
multi-tenant,
available,
Auto
scaling,
we
need
to
be
scalable
and
we
need
to
fail
because
in
the
future,
we're
going
to
face
more
and
more
traffic
and
workloads
safe
deployment.
Of
course,
it
is
one
of
the
basic
requirements
for
the
customer
product
observable.
A
So
with
the
product
requirements,
the
technical
challenges
will
be
obvious.
They
need
to
be
high,
throughputs
high
performance,
higher
availability
have
reliability.
Think
about
your
website.
You
don't
want
it
to
be
down
for
a
couple
minutes
or
seconds
even
extensibility.
Of
course,
we
need
to
build
more
and
more
features
on
top
of
it
security
large
number
of
concurrence
connections,
as
we
mentioned
that
we
accept
the
concurrent
requests,
which
means
for
the
requested
rotor
there
may
be.
A
large
number
of
connections
happen
in
the
same
time,.
A
A
A
I
want
to
talk
more
a
little
bit
the
auto
process,
architecture,
how
we
provide
it.
So
I
only
works
with
any
programming
language
friendly.
We
can
write
the
application
in
goal
in
Java
in
C,
plus
plus
and
any
other
languages,
and
we
can
Bridge
together
between
all
of
them
and
Its.
Behavior
is
identical
regardless
of
the
applications,
programming,
language
or
the
operating
system.
They
are
running
on
Web
Runner.
A
We
are
using
product
and
we
write
the
service
in
C
plus
plus
in
Java
in
Golden
they
can
iterate
with
Amway
with
no
Gap
and
it
breaks
out
all
the
boundaries
of
programming
languages.
The
auto
process
architecture
is
beneficial,
as
it
gives
the
consistent
see
across
the
language
and
application
Stacks.
You
can
get
an
independent
life
cycle.
A
A
A
A
Roads
so
Amway
will
be
the
CRX
of
load,
balancing
and
layer
7
request
proxy
layer
if
maintaining
a
map
of
the
service
URL
to
the
application
instance
IPS
and
load
incoming
request
to
a
proper
endpoints.
It
also
mapping
a
domain
from
the
service
URL,
for
example
hello
world
here
to
the
Upstream
endpoints.
A
A
A
A
A
A
So
I
will
support
two
types
of
the
health
check,
active
and
passive,
with
active
health
check.
Are
we
periodically
send
the
requests
to
them
points
to
check
its
status
with
the
passive
health
check,
and
we
monitors
hold
endpoints
response
to
connections
you
can
use
either
of
them
depending
on
use
case
load.
Balancing
load
balancing
is
very
important
because
it
is
a
way
to
Distributing
traffic
between
the
multiple
endpoints
in
a
single
service
and
a
single
Upstream
cluster.
A
A
A
Timeout
always
supports
multiple
configurable
timeouts.
Depending
on
your
scenarios.
For
example,
there's
a
request:
timeout
specified
some
amount
of
time,
only
waits
for
the
Intel
request
to
be
received.
There
are
also
Idol
10
miles
represent
one
Downstream
or
Upstream
connection
gets
terminated
if
there
are
no
active
streams.
A
A
A
A
Location
metrics
is
a
Diana
component,
which
is
important
to
collection
all
the
insights
and
activity
data,
how
we
can
configure
exposed,
metrics
locally
and
the
static
sync.
Let's
receive
the
metrics
immediately
from
our
way
and
good
order.
Collection
aggregation
for
the
metrics,
such
as
2xx
5xx
request
account,
can
eventually
forward
it
to
Locus
and
Metric
service.
A
A
A
A
A
A
With
all
the
Army
related
topic
finish,
I
also
want
to
introduce
a
little
bit
on
The
Apprentice
public
roadmap
Runner
as
a
product
has
a
public
roadmap
in
the
GitHub
we
received
and
collect
custom
request,
and
also
we
interact
with
customer
directly
actively
on
roadmap
to
share
our
plan
share
what
we
are
working
on.
Thank
you.
There
are
also
couple
items
which
is
related
to
request
routing
and
boxing
in
a
parameter.