►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Ingress V2 and Multicluster Services - Rohit Ramkumar & Bowei Du, Google
With app modernization, we’ve entered a new phase of structuring services. Services have evolved over time to span multiple clusters as well as hybrid deployments that encompass both on-prem and multiple clouds. There are several use cases for multiple Kubernetes clusters: canarying new versions of your app, or low latency access for your users across the globe. In this session, we will present a new version of the Ingress and the Service APIs. These API’s form the foundational blocks for managing your global, scalable and reliable services close to your users using a single control plane. Moreover, we will compare these new API’s with different models for spreading services across multiple cluster (e.g Istio). We will demo a custom controller we have written that will configure multi-cluster HTTP(S) load balancing given these new API’s.
https://sched.co/MPb6
A
Alright,
so
in
early
2008
we
sent
out
a
survey
looking
at
the
landscape
of
what
ingress
was
to
fulfill.
This
was
a
survey
from
the
users
to
say:
hey
what
features?
Are
you
getting
out
of
ingress?
What
features
do
you
consider
important
and
basically
ingress
describes
an
l7
proxy
based
load
balancer?
Of
course
it
could
be
extend
to
l4,
but
l7
is
the
domain.
It
serves
right
now
now
in
our
mind,
when
we
looked
at
what
is
the
next
ingress
API?
Actually,
there
were
a
couple
of
meta
questions
that
we
also
were
asking.
A
Does
it
make
sense
to
be
part
of
core
kubernetes?
Could
there
be
a
project?
That's
actually
outside
of
kubernetes
I,
couldn't
point
to
and
say:
hey
use
that,
because
that
has
already
solved
the
problem.
The
other
aspect
that
we
considered
is:
could
we
have
a
stronger
statement
on
Portability
and
feature
sets?
For
example,
one
option
would
be
to
say
it's
not
portable.
We
can't
reconcile
all
these
different
providers.
Let's
just
give
up
on
portability.
Another
option
would
be,
let's
standardize
on
a
given
implementation
and
wait
for
everyone
to
converge
now.
A
Looking
at
this,
in
terms
of
the
landscape
of
actual
implementations,
we
have
both
the
cloud
providers
providing
proxy
infrastructure
as
well
as
sort
of
off-the-shelf
user
space
proxies,
and
it
looks
like
it's
likely
infeasible
for
clouds
to
allow
the
kind
of
customization
and
expressiveness
of
a
self
managed
proxy,
but
we
do
see
convergence,
but
it's
slow
and
finally,
in
the
space
in
the
past
two
years,
we
have
basically
the
rise
of
the
service
mesh.
So
this
is
a
third
player
in
the
space
of
l7
load.
A
This
is
not
meant
to
be
exhaustive,
so
if
you
don't
see
your
favorite
system
here,
it's
not
mean
it's
not
cool.
It's
just
I
didn't
put
in
the
slides
okay.
So
what
I'm
gonna
do
is
decompose
each
of
these
models
into
key
attributes.
I
know
there
are
not
a
lot
of
nuances
to
the
API,
such
as
the
features
supported
in
the
specific
way
you
write
them,
but
for
the
purposes
of
this
representation,
we're
going
to
look
at
the
big
picture
right
and
here
we
have
the
good
old
ingress
resource
model.
A
We
have
an
single
resource
that
describes
a
bunch
of
host
path
matching
and
it
goes
to
back
ends
and
the
key
attributes
for
this
is
that
one
it's
self
service,
so
a
single
developer
can
define
and
configure
all
aspects
of
the
front-end
TLS
configuration
and
the
back
end.
At
the
same
time,
it
supports
a
single
role,
which
is
the
developer
kind
of
owns
the
whole
thing,
and
it's
very
simple:
it's
just
one
resource
to
understand
now
as
part
of
the
GA
work,
so
that
was
the
beta
as
part
of
the
GA
work.
A
We
have
kind
of
codified
certain
common
practice
and
also
added
a
little
bit
of
flexibility.
So
we've
added
this
notion
of
an
ingress
class.
It's
very
similar
to
that
of
a
storage
class
in
that
you
can
say
hey.
This
is
my
nginx
ingress
or
my
H,
a
proxy
ingress
or
my
Cloud
ingress,
and
also
we're
currently
in
discussion,
although
this
may
be
punted
to
add
more
flexibility
on
the
backend
side.
So
if
you
imagine
assembling
your
load
balancer,
you
would
probably
want
to
assemble
things
other
than
services
into
your
load.
Balancer.
A
Another
popular
model
that
has
come
up
in
the
past
two
years
is
that
of
Sto.
So
sto
is
a
service
mesh,
but
really
the
set
of
resources
it
uses.
R
is
also
intended
to
serve
traffic
ingress.
So
what
are
the
key
attributes
of
this
T?
Oh
well,
one
big
difference
is
that
they
have
separated
the
representation
of
the
proxy
infrastructure
from
the
rest
of
the
application
description.
So
you
have
this
gateway
that
describes
the
proxy
infrastructure.
A
Then
you
have
a
virtual
service
and
destination
rule
that
describe
how
the
application
routing
works
and
then,
finally,
we
come
back
to
the
services
at
the
other
end.
So
the
key
aspects
of
this
do
is
that
it
actually
supports
separate
roles
for
proxy
infrastructure
in
application
description,
and
then
it
also
has
a
very
rich
feature
set,
which
is
primarily
based
on
envoy.
A
Another
model
that
has
come
about
in
the
past
two
years
is
that
of
ingress
route.
So
ingress
route
is
an
interesting
API
model
in
that
is
recursive,
so
you
have
the
single
resource
ingress
route,
but
ingress
routes
can
connect
to
other
ingress
routes.
What
this
connection
means
is
a
delegation
so
you'll
see
here.
The
red
ingress
route
has
rules
for
just
the
top
level
hosts,
but
the
delegated
ingress
route
has
rules
for
a
specific
host
in
a
specific
path.
So
this
lets
you
model
separate
roles
and
has
this
recursive
composability
feature
of
course.
A
In
addition,
it
has
added
some
features,
such
as
traffic
splitting,
among
other
things,
now,
there's
lots
of
other
API
models
just
for
completeness.
We
know
that,
of
course,
everyone
has
lots
of
annotations
for
describing
their
features,
and
some
implementations
have
actually
implemented
ingress
like
features
using
decorators
on
service
and
then
merging
it
post
facto,
and
also
you
can
just
go
straight
to
custom
resources.
A
So
the
question
is:
what
have
we
learned
so
when
ingress
was
first
designed,
the
focus
was
on
simplicity
and
a
single
developer,
but
really
when
we're
looking
at
kind
of
what
these
api's
are
trying
to
solve,
we
see
that
we
need
to
handle
multi
role,
environments,
for
example,
you
have
infrastructure
people
working
on
infrastructure,
providing
ingress
as
a
service
to
the
application
developer,
and
you
may
not
want
the
application
developer
to
own
everything.
The
second
thing
is
this
question
of
how
much
portability.
A
So,
if
you
look
at
the
results
of
the
survey,
there's
a
very
paradoxical
desire
to
both
have
portability
yet
not
prevent
the
user
from
using
all
the
capabilities
if
they
so
desire.
So
in
some
sense
we
need
to
design
the
API
such
that
this
can
be
a
user
choice
right.
We
can't
the
worst
thing
to
do
is
to
tell
your
user
hey.
You
can't
use
this.
It's
not
portable,
but
you
know
they
know
they
want
to
use
it.
Finally,
just
in
general
we
should
change
the
syntax
a
little
bit
to
support
future
API
growth.
A
So
while
there
are
gaps
today,
the
claim
is
that
in
the
future,
providers
and
features
will
converge
and
we
do
see
convergence,
for
example.
The
cloud
infrastructure
is
slowly
adding
all
the
features
that
the
user
space
proxies
already
have
now,
given
all
that,
this
is
a
modest
proposal
warning
this
is
super
early.
A
This
is
basically
a
shot
in
the
dark,
so
we
really
want
a
lot
of
feedback
from
the
community,
and
the
thing
to
note
is
that
this
takes
heavily
from
existing
work,
so
we
don't
aim
to
try
to
be
clever
and
solve
it
a
different
way.
If
an
existing
model
works
well,
then
we
should
actually
adopt
it
to
be
the
standard
for
kubernetes
now
the
way
I'm
going
to
sketch.
This
is
first
I'm
going
to
talk
about
the
shape
of
resources.
A
Basically,
the
model
that
we
are
thinking
about
then
I'm
gonna
talk
about
portability,
extensibility
how
to
make
that
a
first-class
feature
of
the
API
and
then
finally,
we
will
discuss
future
directions
and
one
of
them
would
be
multi
cluster.
So
in
terms
of
the
model,
if
this
is
nothing
earth-shattering
right,
we
need
to
model
the
infrastructure,
which
is
the
load,
balancing
implementation,
separate
it
from
the
application
description.
And,
of
course
we
have
the
backends
kubernetes
today
now
to
anchor.
This
discussion
would
have
a
basically
a
strawman
set
of
resources.
A
We
take
from
the
sto
model,
this
nice
feature
of
representing
the
Gateway
or
the
proxy
infrastructure
as
a
resource
itself,
and
we
add
on
to
that
a
notion
of
class.
So
you
can
imagine
that
today
we
have
ingress
class
and
ingress
resources.
You
have
something
similar
representing
your
proxy
infrastructure.
You
may
have
a
gateway
class
that
represents
your
nginx
and
another
gateway
class.
That
represents
your
cloud
infrastructure
and
then
you
can
create
gateways
from
there
in
the
middle.
You
have
a
virtual
host.
A
Now
a
virtual
host
may
be
recursive
like
ingress
route,
and
this
is
your
application
description.
So
this
is
what
your
application
developer
will
write
to
describe
their
application
routing
and
then
it
may
be
delegated
to
support
the
different
roles
that
you
may
have
in
your
hierarchy
and
finally,
we
have
the
backends
which
may
be
heterogeneous.
So
let's
talk
about
gateway,
so
gateways
split
into
two
pieces.
One
of
them
is
class
and
gateway
class
really
lets
you
specify
just
like
storage
class
deployment,
specific
options.
A
One
concrete
example
that
we've
seen
a
lot
in
ingress
is
some
ingress
controllers.
Merge
all
the
configurations
together
and
implement
your
ingress
configurations
in
one
proxy.
Other
implementations
take
each
ingress
definition
and
have
a
separate
virtual
IP
and
load
balancer
chain
for
them.
So
that's
that
kind
of
configuration
will
be
attached
to
the
Gateway
class.
Now
the
Gateway
represents
an
actual
instance
of
a
load
balancer
or
proxy
infrastructure,
and
this
is
useful
because
you
can
attach
things
like
capacity
or
different
permissions
to
various
gateways.
A
In
addition,
the
Gateway
will
be
where
you
would
define,
say,
protocol
termination,
ssl
features
and
so
forth.
Now
gateways
need
to
be
matched
up
with
virtual
hosts
right
and
there.
Actually,
if
you
think
about
it,
there
are
a
couple
of
design
options
here.
You
know
if
Gateway
referenced
virtual
host,
you
can
add
virtual
host
reference
gateway
and
so
on
and
so
forth.
A
It
seems
to
make
the
most
sense,
taking
from
the
sto
paradigm
that
the
virtual
host
attaches
to
the
Gateway,
because
if
you
create
a
gateway
that
allows
for
all
virtual
hosts
to
attach
to
it,
then
you
have
preserved
this
self-service
model
that
the
original
ingress
has.
On
the
other
hand,
if
you
really
want
to
lock
it
down,
then
you
can
have
the
gateway
filter,
the
virtual
host.
It
serves
thus
preserving
the
operational
control
of
the
resource.
A
A
For
example,
we
can
just
support
path
based
for
now
and
later
we
can
hang
more
predicates
off
of
it
as
the
cloud
and
portable
features
get
implemented
and
then
actions
again
right
now,
you
can
start
with
an
action
that
just
goes
sense
to
a
back-end,
but
again
it
gives
us
a
space
to
add
more
fancy
processing
later,
for
instance,
in
this
case,
you
see,
an
action
could
be
reply
with
404.
Now
what
would
this
look
like
again?
A
A
Okay,
so
in
terms
of
extensibility,
we
have
the
in
schema
extensibility,
but
really
a
lot
of
people
also
want
the
out
of
schema
extensibility,
and
here
this
is
something
we
have
to
figure
out
as
community.
It
seems
like
we
need
a
decorator
pattern.
Could
there
potentially
be
better
API
machinery
to
do
this,
or
we
could
decorate
every
single
place
with
one
of
these
drinks,
training,
raw
objects,
etc?
This
we
really
have
to
work
out
through
examples
and
UX
with
the
end-user.
A
Now,
let's
talk
about
portability-
and
here
is
where
we're
gonna
propose
a
new
strategy
to
be
pursued
in
terms
of
portability.
This
is
kind
of
a
different
model
than
kubernetes.
The
question
you
might
ask
is
why
why
can't
we
use
an
existing
model,
so
a
lot
of
features
of
kubernetes
have
grown
up
with
the
evolution,
human
rights
itself,
for
example,
container
runtimes,
so
they're
easier
to
bring
in
line
with
a
portable
standard.
A
On
the
other
hand,
we
have
storage
classes,
but
they
have
very
orthogonal
features
that
don't
necessarily
conflict
with
each
other
proxy
implementations,
predate
kubernetes
and
they're
pretty
heterogeneous
in
terms
of
feature
sets.
What
we
want
is
a
way
to
express
in
the
API,
something
that
will
have
a
gravity
towards
portability
towards
a
core
API,
and
it
turns
out
when
we
look
at
the
way
annotations
are
used.
It's
actually
not
very
easy
to
corral
those
annotations
into
a
core.
A
So
what
we're
proposing
is
to
have
an
expanding
ring
of
support,
so
in
the
center
we
will
have
a
core
API.
This
is
guaranteed
portable
across
all
providers
and
really
the
minimal
set,
that's
required
to
say
you
are
an
ingress
provider
outside
of
this
there's
an
extended
API,
the
extended
API
is
still
portable
but
the,
but
only
on
a
feature
by
feature
basis.
A
So
you
a
support,
a
feature
in
the
extended
API,
but
it
must
be
portable
and
this
will
be
made
part
of
the
kubernetes
api
schema
and,
finally
to
catch
all
the
other
use
cases.
We
have
this
custom
layer
on
the
outside.
This
will
not
be
expressed
in
the
kubernetes
api.
On
the
other
hand,
you
have
no
requirements
to
make
it
portable.
So
what
does
this
give
us?
Well,
this
creates
a
gravity
to
pull
things
into
the
core.
A
For
example,
if
we
know
that
say,
80%
of
the
implementations
can
support
a
feature
in
the
extended
API
and
we
generally
have
a
sense
that
everyone
is
headed
in
that
direction.
We
can
put
it
in
the
extended
API.
We
can
have
validations
and
so
forth
for
it,
and
then
this
will
standardize
that
usage
for
when
the
day
comes,
which
we
know
is
coming,
but
everyone's
going
to
converge
and
have
this
functionality.
A
So
the
question
would
be
ok.
This
is
kind
of
complicated.
How
do
we
actually
make
this
operationalize
in
practice?
So
really
you
have
to
have
enforcement
by
conformance
tests
right
and
each
extended
feature
definition
requires
a
contained
conformance
that
will
say:
hey
your
implementation
satisfies
this
feature
set,
and
then
this
is
a
suggestion
to
see
if
it's
going
to
be
possible
is
to
require
all
extended
features
to
be
statically
checkable.
A
What
I
mean
by
statically
checkable
is
when
you
submit
an
API
yamo
to
me,
I
can
look
and
say:
oh
yeah,
you
have
this
feature,
enabled
I
can
check
in
an
admission
controller.
If
my
ingress
actually
supports
this
and
you
can
create
tools
around
it
to
help
people
migrate
and
be
portable.
So
in
terms
of
what
this
would
look
like
in
actuality
right,
we
do
expect
core
ingress
and
maybe
we
can
add
something
like
traffic
splitting
that's
well
implemented
by
all
the
controllers
in
the
core.
A
A
So
let's
talk
about
future
directions
in
terms
of
interesting
backends,
because
I
sketched
out
that
we
will
have
a
resource
that
describes
the
infrastructure.
Then
we
have
a
routing
layer
inside
that
describes
your
applications
and
finally
the
back
ends.
There
could
be
some
interesting
things
there
too.
One
of
the
things
I
described
was
the
storage
bucket,
which
you
can
imagine
the
community
coming
into
a
convergence
about
how
to
describe
storage
buckets
but
another
one.
That
would
be
interesting
as
multi
cluster
services
and
here
to
present
that
Rohit.
Thank.
C
You
sorry
so,
let's
take
a
step
back
a
little
bit
and
go
over
some
use
cases
right
so
note
that
to
preface
this,
when
we
talk
about
a
global
application,
we
mean
where
your
backends
are
deployed
and
say
one
or
more
availability
zones
so
canary.
So
typically
you
want
to
deploy
your
application
in
one
locality.
Do
whatever
checks
you
need
to
do
and
then
roll
that
out
across
the
globe.
C
So
that's
a
that's
one
of
the
more
simple
use
cases
we
see,
and
obviously
you
want
to
serve
your
end-users
as
fast
as
possible
and
to
do
that
you
want
to
ensure
your
backends
are
as
close
to
your
user
as
possible.
So
if
you
have
a
user
in
Australia,
they
get
routed
to
backends
that
are
closest
to
Australia
and
something
that
builds
off.
That
is
high
availability.
You
want
your
users
to
be
served
even
if
the
backends
closest
to
them
are
down
and
in
general,
you're,
okay,
with
a
trade-off
in
that.
C
Balancing
by
this
we
mean
traffic,
that's
coming
in
from
your
end
users
into
your
mesh
of
services,
that's
not
to
say
that
you
need
to
use
sto
or
link
or
D.
This
is
as
opposed
to
east-west,
which
is
traffic
within
your
service.
Mesh
and
you'd
want
the
ability
to
add
and
remove
clusters
to
the
slowed
bouncing
based
on
your
business
requirements,
so
this
gets
back
to
the
use
cases
of
low
latency
and
high
availability,
and
you
want
teams
in
your
organization
to
be
able
to
manage
their
own
ingress.
C
This
is
kind
of
the
self-service
model
and
you
want
there
to
be
obviously
a
clear
boundary
between
your
admin.
Who's
kind
of
controlling
infrastructure
and
the
end
users,
which
are
your
developers
and
you.
Lastly,
you
want
your
teams
to
be
able
to
leverage
the
kubernetes
api.
It's
a
pretty
powerful
tool
and
being
able
to
express
this
in
a
kubernetes.
Api
also
reduces
the
learning
curve
for
your
developers,
all
right.
So
here's
what
we
are
proposing
from
here
on,
if
you
hear
me,
say
MCI
or
MCs,
that's
referring
to
multi
clustering,
rest
Multi
cluster
service.
C
So,
if
you
take
a
look
at
what's
highlighted
in
the
red,
you
will
see
that
this
is
very
similar
to
the
core,
vanilla,
ingress
and
service
specification.
In
fact,
in
our
proposal,
what
we've
done
is
simply
taken
those
api's
and
embedded
them
inside
the
field.
You
see
called
template
and
the
benefit
of
this
is
that
it.
This
becomes
very
pluggable.
As
ingress
and
services
evolve.
We
can
conceivably
replace
their
older
counterparts
with
the
newer
ones
in
kind
of
a
drag-and-drop
fashion.
C
So
you
kind
of
might
be
wondering
how
this
works,
so
this
is
kind
of
where
the
list
of
clusters
implementation
Clemson
for
those
of
you
who
are
active
in
cig,
multi,
cluster,
you've,
probably
heard
of
the
cluster
registry
project
and
a
cluster
registry,
is
really
just
a
list
of
clusters
and
how
that's
implemented
is
typically
using
a
Cooper,
Nettie's,
API
server
and
a
custom
resource
definition,
and
so
how
it
works
is
you're.
You
define
your
clusters
using
a
CR
D.
C
So,
given
that
MCI
and
MCS
are
kubernetes
api
is
you
need
some
place
to
store
them?
This
is
where
the
quote-unquote
config
source
comes
in.
This
is
just
a
standard,
kubernetes
api
server
and
ideally
you
have
an
administrative
figure
like
we
discussed
earlier.
That
maintains
the
API
server,
because
after
all,
it's
infrastructure
and
you
need
kubernetes
clusters
to
run
your
workloads
again.
Ideally,
the
administrative
figure
manages
the
turn
up
and
turn
down
of
these
clusters
and
sidenote.
Here,
nothing
is
precluding
you
from
having
the
config
source
be
on
the
same
cluster
as
your
application
backends.
C
C
Ok,
so
some
common
questions,
we've
gotten
with
this.
First
of
all,
why
not
have
a
multi
cluster
deployment
on
the
surface?
It
seems
like
that
may
alleviate
some
of
the
problems
of
having
to
go
in
and
create
your
deployments
on
each
cluster
and
the
quick
answer
to
that
is
we
don't
want
to
be
opinionated
on
how
users
deploy
their
workloads.
It's
not
it's
not
what
we're
trying
to
deliver
with
this
API
and,
furthermore,
there's
a
lot
of
best
practices
in
the
multi
cluster
space
with
how
to
deploy
workloads.
C
So
there's
no
one-size-fits-all
approach
there,
all
right.
What
about
Federation?
So
again,
those
of
you
who
are
active
in
the
multi
cluster
sig
are
familiar
with
Federation
v2
for
those
who
are
not
Federation
is
designed
kind
of
to
allow
users
to
deploy
their
workloads
across
multiple
clusters
from
a
single
API.
Some
common
use
cases
are
distributing
your
app
with
policy
and
multiple
clusters
disaster
recovery
migration.
So
you
might
be
wondering
how
what
we've
proposed
to
you
is
different,
and
the
answer
to
that
is
a
kind
of
three
parts.
C
The
first
one
is
that
the
assumptions
we
are
making
versus
Federation
are
slightly
different.
One
quick
example
is
that
there's
a
resource
I'm
called
federated
ingress
which
to
keep
it
very
short,
is
deployed
in
each
of
your
clusters
by
the
Federation
control
plane.
In
our
case,
we
do
not
take
that
approach
in
Federation
is
also
opinionated
on
a
lot
of
things,
particularly
across
cluster
DNS
and
service
discovery.
How
you
deploy
your
workloads
and
that
kind
of
ties
in
to
the
Third
Point,
which
it
Federation,
is
trying
to
solve
a
lot
of
use
cases
for
you.
C
What
we
are
trying
to
deliver
is
a
more
narrow,
offering
and
I
say
a
la
carte
here,
because
that's
really
what
it
is,
and
so
we're
kind
of
comparing
apples
to
oranges
here,
but
it's
important
to
call
out
okay,
and
it's
do
right
now.
There
is
no
reference
architecture
for
how
to
do
north-south
load,
balancing
using
the
sto
api's.
However,
we
do
see
that
there's
a
potential
for
those
api's
to
adopt
this
use
case.
At
this
point,
it's
just
a
matter
of
waiting
and
seeing.
A
Okay,
so
in
conclusion
we
have
three
things:
one
is
the
initial
kept
proposal,
basically
illustrating
this
sketch.
We're
hope
is
coming
shortly,
although
we
do
have
to
get
the
ingress
v1
GA
wrapped
up
first,
but
we'll
probably
have
that
conversation
in
parallel,
because
the
v2
is
going
to
be
a
pretty
long
track.
I.
Imagine
the
second
thing
and
I
have
this
in
bold
is
that
feedback
from
the
community
is
key?
We
would
love
to
hear
what
people's
use
cases
are.
A
What
makes
sense
I
think
we
got
some
feedback
from
the
contributor
summit
yesterday
that
oh
no
you're
gonna
make
me
learn
some
more
resources.
Please
don't
make
me
do
that.
That's
always
useful
to
know,
and
then
finally,
we
expect,
as
part
of
this
work,
to
come
up
with
like
a
reference
implementation,
so
everyone
can
look
at
it,
especially
the
libraries
for
doing
conformance
for
various
controller
implementers,
and
that's
it.
So
thanks
for
coming
to
our
talk-
and
this
is
our
contact
information.
A
Okay,
so
two
questions
one
of
the
the
first
one
is:
how
do
we
support
something
like
an
envoy
filter
and
then
the
second
question
is
there
seems
to
be
a
lot
of
duplication.
I
think
I
may
have
to
take
that
one
offline.
It
might
be
hard
to
answer
so
in
terms
of
envoy
filter.
That
would
be
what
I
would
expect
to
live
in
the
custom
land,
because
it's
so
specific
to
envoy.
A
Now,
if
we,
for
example,
let's
say
webassembly
filters
become
somehow
very
standardized,
it
may
be
possible
putting
the
extended
set,
but
for
now
I
would
imagine
that
stuff,
like
the
filters,
would
be
sort
of
out
of
scope
of
the
portable
standard.
Does
answer
your
first
question
any
other
questions.