►
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2021 Virtual from May 4–7, 2021. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Extending Service Mesh to the Edge - Stephen Wong, OPNFV Clover
The advent of edge computing has led to the trend of splitting applications into edge and cloud components, which makes service mesh a great fit to enable unified application network policies for inter-container communications, regardless of where individual microservice is deployed. However, service mesh projects such as Istio are mostly designed for large scale deployment, and some features, such as per pod sidecar, may be ill-suited for inherently more resource constrained edge. And optimizing the use of WAN connection is mostly missing. The talk centers on Clovisor, a project aimed to extend service mesh to the edge. By utilizing eBPF, SDWAN, and Envoy, Clovisor performs functions such as translating service mesh app network policies to edge/cloud network policies, redirecting traffic to proxy on edge, classifying and mapping application flows to WAN connectivity tunnels.
https://sched.co/ekF2
A
The
agenda
is
actually
very
simple
for
this
talk.
We
just
want
to
talk
about
the
motivation
behind
trying
to
extend
service
match
to
run
on
the
edge
some
of
the
key
challenges
that
you
would
face,
a
demo
that
would
provide
a
solutions
of
all
our
solutions
for
one
of
the
challenges
that
we
think
exists
and
then
we'll
wrap
it
up
by
talking
about
some
future
plans
that
we
have
on
the
project.
A
We
are
interested
to
integrate
cloud
native,
related
technologies
or
open
source
projects
and
then
trying
to
see
what
kind
of
energy
use
cases
we
can
address,
and
then,
in
this
case,
what
we
get
feedback
from
users
and
then
for
after
investigation
is
there's
a
very
strong
inclination
from
the
users
that
they
want
to
use
kubernetes
on
all
the
cognitive
technologies
on
mobile
edge,
namely
what
we
call
mec,
which
used
to
stand
for
mobile
apps
computing.
Now
it's
multi-access
edge
computing.
A
If
you
think
about
telcos,
they
actually
have
tremendous
edge
site
presence,
they're
radio
networks.
You
know
when
you're
cell
phone
connected
to
to
att
or
someone
there's
a
close
proximity
base
stations
that
actually
determines
your
radio
connectivity.
So
they
have
those
sites
that
are
in
small
buildings
so
and
then
with
compute
powers.
A
So
they
have
actually,
if
you
will
of
micro
data
centers,
and
that
presence
has
been
around
for
a
very
long
time
and
then
particularly
for
5g
upgrades,
because
the
bandwidth
is
so
much
bigger
and
then
what
kind
of
applications
an
implication
of
the
applications
that
would
come
in.
They
are
the
mec
effort.
Is
there
for
taking
advantage
of
that
access,
compute
power
and
then
allowing
third-party
application
owners
players
to
be
able
to
utilize
those
compute
powers
by
running
their
applications.
A
It
doesn't
necessarily
just
address
the
msu
use
case,
but
it
does
address
the
use
case
where,
if
you
have
servers
on
the
edge
and
you
want
to
run
applications
on
the
servers,
the
benefits
of
actually
running
the
service
match
all
the
way
to
the
edge
and
then
going
back
to
that
use
case.
What
makes
sense
for
us
in
terms
of
deployments
for
kubernetes
is
that
we
want
to
run
cluster
and
auto
control
plane
for
that
cluster
independently
on
the
ash
sites.
A
Obviously
our
benefits
is
the
benefits
are
actually
pretty
obvious
right.
Your
if
you
want
to
restart
a
pod,
you
want
to
reschedule
the
parts
all
the
decisions
are
made
locally,
so
you
don't
have
to
go
back
to
the
cloud
just
to
just
to
restart
and
reschedule
a
part
and-
and
that's
not
anything,
majorly
controversial
right
now.
I
think,
for
the
last
two
three
years,
there's
been
a
number
of
projects
that
are
trying
to
address
the
the
use
case
of
lightweight
kubernetes
engines.
A
So
what
we
see
here
is
something
like
a
k3s
from
rancher
or
micro.
K8S
from
ubuntu
are
all
example,
projects
where
they're
actually
trying
to
address
a
lightweight
kubernetes
cluster
use
case
and
another
one
is.
If
you
have
this
kind
of
deployment,
what
kind
of
application
are
we
looking
at
so
going
back
to
something
like
a
mvc
or
any
kind
of
edge
data
center
micro
data
centers?
You
are
trying
to
run
a
single
applications
that
have
components
that
spans
across
both
edge
and
cloud.
So
so
that's
that's
what
we
get.
A
Actually,
the
feedback
from
some
of
the
users
from
the
open
v
community
also
a
lvl
when,
when
the
control
things
would
be
the
control
analytics
and
all
that
all
that
may
be
running
on
the
cloud.
But
then
the
the
processing
of
frames
and
and
videos
will
be
running
on
edge,
whereas
our
machine
learning
where
the
inference
engine
would
actually
be
running
on
edge.
And
then
all
the
learning
capabilities
are
running
on
cloud
or
iot
gateway,
where
some
sort
of
filter
is
running
on
edge
and
then
the
core
analytics
engines
will
be
running
on
cloud.
A
A
This
presentation
is
that
it
makes
a
lot
of
sense
to
run
a
single
mesh
across
the
cloud
and
edge
sites
for
this
kind
of
applications.
The
obvious
one
is
that
you
know
once
you
do,
that
you
have
a
consistent
network
policy
and
telemetry
formatting
models
across
a
single
applications
across
different
sites.
Your
cloud
or
edge,
so
you
can
write
basically
the
same
framework
you
can.
A
The
major
use
case
for
sdo
really
is
cicd
pipelines,
releases
or
red
black
testing
a
b
testing
type
stuff
for
those
things
for
istio's
particular
single
match
is
actually
running
very
well
spinnaker,
for
example,
is
already
integrated
with
stl
for
over
a
year.
Now
so,
and
then
we
do
think
that,
for
this
kind
of
features,
the
ability
to
do
kind
of
releases,
the
ci
cd
pipelines
is
critical
for
edge
deployments,
because
physical
access
is
very
expensive
for
edge
sites.
A
So
you
you
want
to
automate
the
process
of
deployments
or
only
reciting
parts
or
things
like
that,
all
basically
fully
automated
to
avoid
having.
Actually
you
know,
technicians
accessing
edge
sites
all
the
time,
so
with
all
these
benefits,
that
is,
you
know,
sort
of
obvious
really.
Why
have
no
one
done
it
before
or
why
haven't?
Actually,
so,
actually
we
thought
about
doing
that
as
early
as
I
would
say,
mid
second
half
2018
when
clover
was
in
was
in
major
development
time
at
that
time.
A
Actually
we
had
a
we
encountered.
We
investigated,
we
encountered
a
major
blocker,
and
that
was
actually
that's.
The
component
called
mixer
mixer
is
a
istio
control,
plane
component,
which
it's
actually
an
engineering
model.
It's
a
very
good
idea
as
a
theory.
What
it
does
is
because,
as
you
probably
know
envoy,
which
is
the
proxy
engine
or
the
data
plane,
if
you
will
on
on
istio,
runs
on
every
single
part.
A
So
it's
what
we
call
sidecar.
So
every
part
that
gets
spawned
there'll
be
a
and
voice
a
workload
part
that
gets
spawned
there'll
be
a
steel
proxy,
which
is
an
avoid
running
mixer.
The
idea
would
be,
oh,
you
have
potential
thousands
of
these
running
if
we
have
complicated
policies
or
complicated
policy,
complicated
telemetry
policies
or
if
you
have
a
bunch
of
infra
back
ends
that
does
its
own
thing,
that
is
to
record
the
sessions
you.
A
You
should
centralize
that
capabilities
into
instead
of
spreading
with
a
thousand,
so
it
doesn't
really
scale
a
great
idea,
but
then,
because
mixer
is
the
name
indicator,
because
you
want
to
implement
part
of
your
network
policies
on
the
daily
traffic
in
mixer.
Mexico
is
basically
not
a
control
plane
component,
it's
actually
a
mix.
I
guess
that's
what
the
name
means
a
mix
between
data
plane
and
a
control
plane
that
and
how
android
works
is
every
single
first
request
that
comes
in
for
a
session.
A
They
would
first
get
funded
into
mixer
and
then
they'll
process
it
and
then,
and
then
we'll
see
whether
or
not
it's
a
ona
to
actually
go
before
android
can
proceed.
Just
by
hearing
that
you
would,
you
would
be.
You
would
know
that
it's
actually
pretty
terrible
to
run
that
on
the
edge
you
can
rig,
you
can
actually
run
mixer
on
the
edge
okay
in
multi-cluster
deployment
you
they
do.
Iso
does
actually
assume
that
you
run
maxwell
also
on
the
on
the
the
other
clusters,
but
then
particularly
for
edge.
A
It
doesn't
make
sense
because
your
infra
back
ends
probably
doesn't
exist
on
the
edge
they're,
mostly
pretty
heavy
stuff.
I
would
say
so.
If
that's
the
case,
then
your
mixer
may
be
actually
running
on
cloud
and
if
all
your
requests
has
to
go
to
the
cloud
just
to
get
mixers
to
say
yes,
that's,
that's.
A
The
kind
of
delay
is
actually
completely
unacceptable,
but
thankfully,
for
us
during,
I
think
when
you
know
it's
still
1.41.5
the
comm,
the
istio
community
decided
to
deprecate
mixer,
so
it
and
then
instead
actually
taking
advantage
of
the
rich
set
of
filter
capabilities
on
android.
That
allows
you
to
then
imp,
implement
those
complicated
policies
and
technologies
pro
or
your
custom
protocols
directly
on
android,
instead
of
actually
having
this
centralized
command
components
to
do
so
and
that
in
a
way,
actually
unblocks
us
from
from
moving
forward.
A
So
now
that
we
start
to
resume
looking
at
running
steel,
extending
still
to
the
edge
again
one
of
the
great
things
that
you
can
do
and
a
good
opportunity
to
innovate.
Really,
when
you
run
this
steel
to
the
edge
is
extending
the
istio
capabilities,
the
route
rules,
the
policies
into
influencing
your
choices
of
which,
when
connectivity
links
that
you're
going
to
use.
A
So
in
here
when
I,
when
I
say
when
why
area
networks,
I
don't
just
mean
why
area
networks
with
the
typical
definitions-
and
you
know,
because
going
between
edge
to
cloud,
usually
you
can
go
through
for
networking
terms.
You
can
go
through
access
network,
you
go
through
like
metro
and
then
you
you
and
then
you
go
through
like
the
real
wind,
which
is
either
long-haul
or
the
transcontinental
links.
A
So
so
those
are
being
defined
and
then
and
then
what
we
really
want
to
see
and
then
what
we
see
the
opportunity
on
is
to
map
the
istio
policies
and
railroads
into
different
wind
links
depends
on
what
their
characteristic
is
and
actually,
in
fact
this
is.
This
is
going
to
be
a
very
simple
demo
that
we're
going
to
run
on.
You
know
this
technology
that
we
use
before
we
actually
go
into
the
demo
descriptions,
the
demo
and
this
solution,
actually
that
we
would
propose
actually
utilizes
heavily
on
this
technology
that
we
developed.
A
That's
the
cochlear
visor
horizon
used
to
be
part
of
it
was
developed
as
part
of
opnrv
clover,
but
it
has
since
spun
out
and
it's
an
independent
project.
Now
what
it
does
is
it
talks
to
kubernetes
or
steel
controllers,
and
then
it
the
big
thing
is
it
actually
connects
with
the
linux
kernel
through
bpf
technology.
Iovisor
is
a
open
source
technology
under
linux
that
actually
would
compile
and
load
what
we
call
bpf
code
into
a
kernel
bpf.
A
I
hope
you
know
what
bpf
is,
but
if
you
don't,
I
probably
don't
have
time
to
explain
what
vpf
is,
but
then
in
one
liner
would
be
is
a
kernel
technologies
that
allows
you
to
insert
your
code
safely
into
different
event.
Points
in
a
kernel,
so
for
us,
for
example,
would
be
the
ingress
of
a
network
device
and
the
egress
of
a
network
device.
We
can
insert
our
own
code
in
there
and
it's
it's
kernel
safe.
The
compiler
ensures
that
it
won't
crash
the
kernel
and
then
this
so
this
is
called
visor.
A
A
Each
vm
basically
represents
a
cluster,
so
I
run
a
full
kubernetes
clusters
on
each
vm
untinged
it
so
the
master
runs
here
and
all
the
minions
running
signature
all
the
parts
we
schedule
on
just
this
one
node
and
then
I
also
utilize
the
sdo
multi-cluster
capabilities
by
deployment.
So
there's
three
different
deployment
scenarios,
though
the
one
we
pick
is
replicator
control,
plane,
separate
networks.
This
is
because,
well,
the
the
other
two
are
share:
control,
plane,
separate
networks
and
share
control,
plane
same
network.
A
The
last
one
actually
doesn't
really
make
a
lot
of
sense
for
for
practical
purposes,
it's
probably
more
for
testing,
so
we
didn't
use
it.
I
think
the
second
one,
which
is
the
share,
control,
plane
and
separate
networks
that
actually
reflects
much
better
on
the
cloud
and
edge
relationships.
But
then
we
still
do
the
replicated
control
plane
because
it's
the
easiest
to
set
up.
So
that's,
basically
the
only
reason
what
this
does
is
and
for
a
setup.
A
As
I
mentioned
earlier,
there
are
two
interfaces
that
are
running
between
the
two
clusters,
so
they
similar
to
one
link.
So
this
is
winding
a
and
running
b,
basically
between
the
two
and
that's
so
the
idea
is
this:
is
one
istio
cluster
istio
instance
running
on
a
iso
instance
running
on
a
single
cluster
and
then
there's
another
one,
and
then,
when
sleep
wants
to
access
any
of
this,
actually
we've
been,
it
would
go
through
a
still
ingress
gateway.
A
A
Demo
hi,
so
this
is
the
demo
screen.
What
you
see
here
is
there
are
three
windows
logged
into
the
edge
node,
and
this
is
the
cloud
side.
This
is
going
back
to
that
slide
deck.
This
is
basically
having
actually
been
that
would
actually
maps
minions
to
version
two
I've
actually
been
and
everything
else
to
be
one,
as
you
can
see,.
A
A
A
A
A
A
A
So
what
does
allow
fuel
to
do
what
it
does
is
it's
for
everything
that
matches
in
hdb
rules
or
any.
Actually,
he
thinks
that
matches
port
8000
in
this
case,
because
we
use
cyca
inbound,
it
hasn't
actually
changed
to
port
numbers,
yet
it
extracts
the
actually
header
informations
and
send
it
over
to
to
cloud
visor,
so
cloud
visor
would
for,
for
example,
we
basically
mesh
with
the
rules
that
say
it's
actually
user
equals
to
minions,
then
do
something.
A
Once
that
redirect
rules
is
in
place,
anything
that
sleeps
sends
that
matches
minions
would
then
be
routed
into
s9.
Anything
that
sends
just
anything
other
than
minions
like
boss
in
in
our
demos.
It
would
still
go
through
s8
go
through
a
different
when
link,
so
this
is
actually
the
cluster
of
what's
going
on
inside.
A
A
We
have
a
luau
filter
on
lower
filter
on
on
android.
So
there
will
actually
forward
us
actually
be
informations
on
new
sessions
provider,
implements
that
railroad
classifications,
maps
into
recession,
informations
and
program
bpf,
and
then
subsequently,
all
the
packets
would
then
be
forwarded
into
a
different
rand
link
for
future
enhancement.
So
this
is
cool.
Hopefully
you
think
this
is
good
for
future
enhancements.
A
What
we
some
of
the
things
we're
looking
at
is:
let's
say
you
actually
have
a
very
resource
constrained,
etch
node,
where,
because
envoy
is
being
deployed
per
as
a
container
per
pod
per
workload
pod
it
can
you
may
not
have
locations
that
actually
require
that
particular
edge.
Maybe
it
actually
isn't
needed,
particularly
talking
to
users.
There
really
is
a
difference
between
whether
or
not
your
application
is
cpu
bound
versus.
I
o
bound
your
a
lot
of
times.
Your
applications
in
edge
may
not
be
very
chatty.
It
may
not
be
as
chatty
it
would.
A
A
Sorry
that
basically
runs
on
as
a
single
entity
where
multiple
parts
would
be
able
to
go
through
the
same
envoy,
so
the
clovisa
portion
is
kind
of
more
simplistic.
So
when
you,
when
you
deploy
envoy,
the
first
thing
you
do
is
a
knit
container
is
that
basically
set
it
into
the
to
set
the
ipa
table
rules
so
that
you
can
you
route
all
the
path,
traffic
or
incoming
outgoing
traffic
onto
android?
A
So
cloud
visor
can
actually
do
this
by
using
something
called
sock
map
which
acknowledgement
to
to
project
silium
folks
they
actually
created
this
stock
map
capabilities.
If
you
know
tc
splicing,
it's
basically
the
same
thing
except
it's
programmable,
tcp
splicing
that
allows
you
to
switch
packets
from
one
socket
to
another
socket.
So
we
can.
We
can
utilize
sockback
capabilities
and
basically
create
android
as
a
non-cycle
entity,
but
then
at
least
one
per
name
space
per
node.
A
Why
one
per
name
service
per
node
because
of
security
reasons,
any
things
that
before
going
into
envoy,
are
clear
text.
So
you
would,
you
would
at
least
trust
everything
in
your
name
space
to
actually
do
the
right
thing,
and
that
being
the
case,
then
you
would
send
it
out
onto
a
per
name
say
per
no,
because
if
you
go
out
on
to
the
node
you
do
not
it
is
you
don't
you
do
not
want
unencrypted
things
go
out
to
go
out
on
the
physical
interface,
so
so
this
is
actually
something
that
we
are
exploring.
A
Some
of
the
hard
thing
is
the
to
make
istio
works
with
a
non-psychar
envoy.
Another
thing
that
very
quickly,
because
it's
almost
time
is
how
to
actually
ship
logs
so
right
now.
This
is
not
really
part
of
the
istio
control
plane.
It's
not
a
cld.
The
the
log
collectors,
the
trace
collectors
and
the
metrics
collectors
are
all
separate
entities
that
you
have
to
deploy
yourself.
A
It
really
doesn't
make
sense
to
deploy
that
in
edge,
so
we
would
like
to
figure
out
solutions
that
you
can,
but
you
still
want
the
telemetry
information
to
login
for
information,
the
metrics
informations.
The
trace
information
particularly
are
very
useful.
So
one
of
the
things
to
think
about
is,
you
know:
can
we
store
them
in
batch
and
sentinel,
or
instead
of
you
know,
do
we
do
we
want
them
to
keep
on
sending
it?
When
you
know
I
utilize
when,
but
I
keep
on,
tracers
are
pretty
taxing
because
I
think
diesel
configuration
is
open.
A
So
in
summary
there,
as
we
said,
there's
a
lot
of
benefits
of
running
service
mesh
across
cloud
edge.
We
demonstrated
when
associations
it's
actually.
We
believe
it's
a
very
good
features,
particularly
with
sd-wan.
This
will
definitely
be
very
powerful
moving
forward
and
then
for
the
future.
We
would,
if
you
have
any
resource
constraints
or
concerns
on
site
cars
and
or
the
control
traffic
in
general.
There
are
two
major
areas
to
investigate
and
address.
A
We
do
want
to
conclude
by
saying
that
we
believe
that
edge
computing
is
as
much
if
not
more,
of
a
networking
problem,
as
it
is
a
computing
problem.
So
that
being
the
case,
I
think
you
know
application
needs
to
be
aware
that
they're
running
on
edge
instead
of
on
on
cloud
and
then
and
then
the
infrastructure
needs
to
be
invested
heavily
on
how
to
solve
that
problem
on
a
networking
perspective.