►
From YouTube: Discreetly Studying the Effects of Individual Traffic Control Functions - Lee Calcote & Prateek Sahu
Description
Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Discreetly Studying the Effects of Individual Traffic Control Functions - Lee Calcote & Prateek Sahu, Layer5
Like path-based routing? How about attribute-based whitelisting? Manipulating request headers and routing? (yes, you do.) Service meshes offer granular control over service traffic, allowing you bend network traffic to your will. But, how do these individual traffic controls perform? How much latency do they induce? We present a study on the effects of individual traffic controls in a service mesh. Using a simple application, we will show how users can chain together multiple (fake) microservices, which are configurable in terms of their behavior (i.e. respond precisely and consistently) in performance test environments.
https://sched.co/Zetg
A
B
Hey
guys,
I
am
prathik,
I
am
a
grad
student
at
ut
austin
and
I
was
actually
looking
at
service,
not
looking
at
service
meshes.
I
was
looking
at
function
as
a
service
and
serverless
applications
about
a
year
and
a
half
back.
When
I
met
lee
and
then
we
started
looking
down
this
path
of
service
meshes
and
performance
impacts,
and
here
we
are.
A
Yeah,
clearly,
you
were
focused
on
on
the
well,
I
don't
know
or
not
on
the
right
thing
we
have
to
service
meshes
is
where
it's
at,
so
that
was
what
about
a
about
a
year
ago
now.
So
I'm
just
ecstatic
that
we're
gonna
share
some
of
what
we've
been
studying.
Some
we've
kind
of
I've
sort
of
talked
in
this
space
for
a
while
I've
written
actually
a
couple
of
books
on
on
service
meshes
and
believe
it
or
not.
A
B
No
worries,
so
I
am
kind
of
looking
into
the
research
aspects
of
service
meshes
and
looking
into
interesting
findings
to
share
with
you
folks.
I
have
been
working
on
service
meshes
for
a
while,
and
there
are
a
lot
of
research,
interesting
research
topics
in
this
area,
but
we'll
only
focus
on
a
couple
of
for
them
in
this
particular
talk
and
yeah
feel
free
to
reach
out
to
us
and
join
the
community
to
know
more
and
contribute
and
discuss
interesting
stuff.
B
A
A
A
They
do
now.
Having
said
that,
not
all
meshes
provide
for
non-containerized
services,
but
but
many
of
them
do
the
nice
thing
is
you
can
kind
of
sort
of
within
an
with
an
asterisk?
You
can
kind
of
get
there
for
free,
I'm
just
noticing
a
behavior
of
the
system.
By
the
way
when
I
try
to
advance
the
slide
it
takes
about
10
seconds
to
advance,
so
I'm
going
to
try
to
click
10
seconds
ahead
and
we'll
see
if
it
works.
A
Other
reasons
why
people
come
to
a
mesh
is
to
avoid
baking
in
infrastructure
concerns
into
their
code.
They
they
don't
want
to.
You
know
expend
a
bunch
of
energy
on
duplicative
infrastructure
concerns
that
that
the
infrastructure
can
actually
take
care
of
that
in
the
presence
of
a
service
mesh.
A
lot
of
those
those
features
that
we
just
talked
about
are
there.
They
can
avoid
having
to
bake
that
right,
use
a
client
library
which
we're
going
to
take
a
look
at
one
here
later
to
address
those
concerns.
A
Rather,
they
can
get
retries
and
enforce
deadlines,
cancellations
and
circuit
breaking
and
many
other
things
that
they
might
have
otherwise
had
to
put
into
code.
Your
infrastructure
can
handle
that
for
you.
So
one
of
the
interesting
things
I
think
that
is
arguably
the
most
powerful
phenomenon
about
a
service
mesh
is
the
notion
that
often
the
responsibility
of
who's.
A
A
How
or
any
number
of
these
configurations
like
whether
or
not
a
service
should
be
assigned
an
individual
certification.
A
certificate
have
its
own
identity.
Have
that
identity
rolled
have
that
identity
managed
that
all
of
those
things
I
think
who's
responsible
for
for
that
in
a
given
environment
is
often
diffused
and
and
confused
between
which
which
of
these
two
teams.
A
That
is
the
hope
here
is
that,
when
you've
got
a
piece
of
infrastructure
like
a
service
mesh
you're
able
to
identify
all
of
how
you'd
like
for
that
litany
of
concerns
that
I
just
prattled
off
how
you'd
like
for
those
to
be
configured
and
behave
when
you've
got
that
in
a
set
of
configuration
when
you've
got
that
in
yaml,
it
is
no
longer
a
finger
pointing
game.
Rather
it's
a
it's.
A
singularly
addressable
concern
and
it
becomes
self-evident
who's
who's
taking
care
of
those
things.
A
So
I
think
that's
really
powerful
the
way
that
those
two
teams
are
able
to
begin
to
iterate
independently
of
one
another
they're
able
to
move
more
quickly
and
they're
able
to
both
of
them
rapidly
identify
what
a
given
you
know.
Retry
count
is
for
a
service,
for
example,
because
it's
there
in
the
configuration
it's
not
buried
in
code
or
it's
not
even
in
the
code.
A
It's
not
even
present
so
really
interesting
thing
about
a
service
mesh
as
we
get
into
more
of
what
we're
going
to
kind
of
speak
about
about
the
analysis
of
network
functions
and
how
service
meshes
provide
various
network
filters.
Various
traffic
filters.
It's
important
to
consider
it's
important
to
understand
that
there
are
a
variety
of
different
service,
mesh
architectures
and
that's
in
part,
because
there's
a
variety
of
different
service
meshes
out
there.
A
There's
a
landscape
here
that
tracks
service
meshes
about
20
or
more
and
actually,
as
it
turns
out,
there
will
be
an
announcement
from
a
vendor
that
you
all
are
familiar
with
before
the
year
is
out
a
really
exciting
announcement,
I
think,
but
but
it'll
make
for
the
21st
service
mesh.
So
more
of
them
keep
coming
and
clearly
people
are
seeing
value
in
the
meshes,
because
there
are
as
many
as
there
are
and
which
is
a
great
thing.
A
I'm
quite
pleased
that
there's
choice
out
there-
it's
not
a
one
size
fits
all
necessarily,
but
because
of
that
there
are
service
mesh,
abstractions
specifications
that
are
coming
forth.
I
participate
as
a
maintainer
in
the
service
mesh
interface,
which
is
essentially
a
standard.
Well,
it's
almost
like
a
if
I
can,
if
I
can
we'll
see
if
we'll
see
if
people
get
upset
with
me
for
this,
but
if
it's
almost
like
terraform
is
as
a
utility
to
multiple
clouds
service.
A
Mesh
interface
is
provides
a
standard
interface
behind
which
you
can
plug
in
a
given
service
mesh,
probably
better
analogies
here
or
something
like
cni.
The
container
network
interface
gives
you
one
api
one
surface
to
for
you
as
an
adopter
or
you
as
an
integrator
to
get
at
service
mesh
functionality,
irrespective
of
what
service
mesh
you're
using.
So
that's
great
there's
another
one
here
called
hamlet.
A
A
A
It's
really
a
format
that
standardizes
the
way
in
which
you
capture
and
characterize
the
performance
of
a
service
mesh
so
smps
is,
is
a
vendor-neutral
standard.
It's
one
that
layer
five,
which
is
the
community
that
pratik
was
talking
about.
The
surface
mesh
community
has
been
working
on
for
a
while.
A
We
really
had
gotten
started
in
conversation
with
the
istio
performance
and
scalability
team,
so
with
the
googlers
there,
and
since
we've
been
speaking
about
it,
talking
about
it
and
advancing
it
and
working
through
it,
we've
begun
to
discuss
it
inside
of
the
cncf
inside
of
sig
network
and
the
cncf.
So
there's
a
service
mesh
working
group
that
all
are
welcome
to
come
and
participate
in,
but
where
we
also
advance
this
specification.
A
The
spec,
like
I
said
it
directly,
provides
a
description:
how
to
describe
and
capture
how
to
characterize
the
performance
of
not
just
your
service
mesh,
but
also
the
fact
that
that's
highly
dependent
upon
what
type
of
workloads
your
mesh
is
supporting.
How
your
mesh
is
configured,
how
much
you're
asking
it
to
do?
A
How
big
or
small
of
an
environment
how
many
nodes
and
what
size
nodes,
you're
running
so
many
many
different
variables,
and
so
it
helps
standardize.
How
you
capture
that
from
there
you
can.
It
then
facilitates
some
interesting
things
and
one
of
those
is
an
apples
to
apples
comparison
either
between
service
mesh
deployments
of
the
same
type
or
different
types,
but
maybe,
more
importantly,
it
facilitates
you
gauging
that
for
yourself
and
having
a
standard
way
to
capture
the
performance
of
of
your
mesh,
the
configuration
of
your
mesh.
A
There
is
a
an
implementation
of
this
specification
today.
The
canonical
implementation
in
a
tool
called
mesheri
mashuri's,
the
service
mesh
management
plane.
It's
a
it's
a
multi-mesh
management
plane.
It
is
compatible
with
smi.
It
was
that
compatibility
was
announced,
sort
of
launched
or
announced
at
the
launch
of
smi.
A
There
are
well
gosh,
I
think
three
interns
being
sort
of
sort
of
working
around
smi
and
working
around
envoy
and
around
measuring
through
the
google
summer
of
code
through
the
cncf
and
through
community
bridge,
hopefully
through
google's
season
of
docs
here
shortly
as
well,
and
so
the
part
of
the
there's
a
few
different
goals
of
this
tool,
and
it's
important
that
we
talk
about
this
tool
because
it's
what's
been
facilitating
our
performance
analysis.
The
the
analysis
that
we're
going
to
speak
about
here
in
a
moment.
A
A
Please
disregard
these,
like
I
don't
know,
year
and
something
ago
old
stats.
Rather,
it
was
really
just
trying
to
show
that,
like
hey,
when
console
first
came
out,
really
it
was
only
facilitating
mtls,
so
it
really
wasn't
doing
much
in
comparison
to
maybe
istio.
That
was,
you
were
asking
it
to
do
a
lot
more,
and
so
anyway,
one
of
the
things
that
it
was
intended
to
do
is
that
it
still
still
does
that.
A
A
It
will
analyze
the
configuration
of
your
mesh
and
tell
you
if
you're
you're
doing
it
wrong
or
maybe
it'll
tell
you
you're
doing
it
right
so,
but
well,
it's
I
think,
interesting
to
look
at
where
a
tool
like
this
fits
in.
When
you
look
at
service
meshes,
you
talk
about
the
data
plane
and
the
control
plane.
A
This
is
a
generic
view
of
what
a
data
plane
and
control
plane.
Often
look
like
you
can
also
layer
on
a
management
plane
for
those
of
you
who
are
network
engineers
or
have
done
network
administration.
I
think
that
that's
a
familiar
term
and
in
this
management
planes
are
capable
of
any
number
of
different
things.
The
the
functionality
that
we're
talking
about
here
is
really
around
load
generation
and
statistical
analysis
of
the
performance
of
a
mesh.
A
What
we're
showing
here
is
that
measuring
it
will
gives
you
sort
of
choice
of
load
balancer,
I'm
sorry,
I'm
not
load
balancer
load
generator
between
fortio
wrk2
nighthawk
and
it
will
generate
load
and
do
an
analysis
today.
It'll
do
that
against
any
one
of
these
six
service
meshes
there,
and
it
does
that
through
adapters
through
service
mesh
adapters,
more
adapters
are
being
added
in
the
community,
so
good.
A
A
But
we'll
take
you
through
some
network
performance
instead.
But
the
example
here
is
the
notion
that
the
app
has
much
like
any
sas
offering
that
you
might
use
you'd
sign
up
as
a
user
and
it
and
you
you
have
a
different
subscription
plans
that
you
sign
up
for
well.
This
app
has
subscription
plans,
but
it
doesn't
have
any
way
of
enforcing
people
to
stay
within
the
bounds
of
those
plans.
A
So
hopefully,
this
sample
app
enlightens
expands
some
of
your
horizons
to
well
expect
more
from
your
infrastructure
to
leverage
your
infrastructure
a
bit
more.
This
is
it's
a
very
simple
app.
It's
two
docker
containers,
there's
a
view
ui
or
a
ui
written
in
view
running
in
one
container,
there's
a
back
end.
That's
your
image,
your
docker
image,
storus
storage
service
and
it
runs
a
set
of
apis
and
runs
on
the
storage
service.
A
We're
able
to
take
this
sample
app
and
deploy
it
on
top
of
a
mesh.
The
mesh
that
we've
chosen
is
console
for
a
couple
different
reasons,
one
of
which
is
because
console
uses
envoy
as
its
data
plane,
proxy
and
envoy,
is
capable
of
incorporating
web
assembly
modules.
Wasom
modules
we'll
talk
about
webassembly
more
in
a
moment,
but
there's
a
lot
of
power
and
the
ability
to
load
and
dynamically
reload
web
assembly
filters.
These
filters
are
are
highly
capable,
they're
able
to
see
the
same
network
of
traffic.
A
Just
for
performance
reasons
it
and
I
won't
re-describe
what
the
sample
app
does,
because
that's
kind
of
for
a
different
talk,
but
suffice
to
say
on.
I
guess
I
should
note
that
today
console
doesn't
support
this
configuration
and
the
use
of
awesome
filters
out
of
the
box,
and
I
won't
speak
on
behalf
of
the
the
product
management
team
there.
I
would
just.
A
A
A
A
Plane,
one
is
to
is
to
well
continue
to
use
client,
libraries,
language,
specific
client,
libraries
or
just
the
standard
libraries
of
the
language
itself
to
facilitate
some
of
these
network
functions
in
this
case,
what
we're
talking
about
here
is
rate.
Limiting,
specifically,
this
is
just
what
we
had
studied
here
and
the
statistics
that
we're
seeing
is
in
the
environment
that
we
were
within.
A
B
So
yeah
we
have
run
yeah.
Let
me
take
up,
though,
results
in
in
the
way
that
we
tested
this
in
two
different
environments.
If
I
may,
like,
we
implemented
a
rate
limiting
feature
inside
the
go
application
itself,
and
then
we
also
implemented
a
rate
limiting
feature,
as
in
our
rust
web
assembly
and
plugged
it
in
inside
the
envoy,
sidecar
proxy
and
removed
the
rate
limiting
feature
from
the
go
application.
B
What
we
would
have
expected
is
that,
if
the
rate
limiting
is
inside
the
application
itself,
then
it
should
perform
better.
But
since
it's
web
assembly
and
the
network
traffic
that
is
coming
through
the
sidecar
proxy
and
being
routed
through
the
sidecar
proxy
to
the
application
can
be
processed
in
the
sidecar
itself,
and
it
does
not
need
to
go.
Have
the
overhead
of
being
transferred
to
the
application
again,
because
of
that
we
see
a
fair
amount
of
benefits,
the
experiment,
the
rate
limiting,
is
set
at
100
requests
per
second.
B
B
I,
although
that
this
is
a
very
simplistic
example,
and
but
when
we
see
that,
even
in
a
simple
example,
we
see
such
significant
difference
in
latencies
so
as
and
when
the
functionality
gets
larger
and
more
complex.
These
latencies
will
grow
only
more
significant
one
new
thing
like
not
new,
but
like
a
good
thing
about
wasm
is
you
can
only
use
wasm
filters
as
and
when
you
need
yeah
like
we
do
with
kernel
modules.
In
our
old
age
days,
we
can
insert
kernel
modules
and
remove
them
to
lower
our
binary
overhead.
B
A
And
so
we'll
try
to
get
the
the
slide
advanced
I'll.
Take
a
moment
we're,
I
just
realized,
we're
back
one
and
so
to
critique.
Please
interrupt
me
as
I
as
I
go
here,
but
outside
of
just
looking
at
the
overhead
differences
in
how
you
take
a
given
network
function,
like
rate
limiting
and
how
you
implement
that
and
the
differences
in
the
overhead
of
those
implementations.
A
We
were
seeing
it
really
just
a
single
path,
based
routing
being
having
a
higher
performance
while
on
route
ramad
round
robin
load,
balancing
taking
a
slight
more
hit
in
terms
of
latency
and
in
terms
of
throughput
actually
for
the
given
time
period
and
then
context-based
routing,
as
you
might
expect,
is
a
little
more
powerful
and
it
also
is
comparatively
slower
than
the
other
two
and
so
we'll
see.
If,
by
the
time
we
take
some
q,
a
we'll
have
some
data
points
to
share
there
specifically.
A
A
B
Yeah,
so
this
is
this
was
a
great
experiment
that
came
by
us.
While
we
were
running
other
experiments
is
how
do
we
optimize
our
average
response
times?
So
we
tried
to
run
one
of
the
tests
and,
as
you
can
see
in
this
graph,
we
saw
that
the
late,
the
response
that
we
got
could
be
bucketed
in
separate
chunks.
That
is
in
bucket
one.
We
did
get
a
few
of
these
responses,
but
major
majority
of
the
responses
came
back
in
bucket
three.
B
That
means
that
there
is
potential
in
our
service
to
respond
in
12
milliseconds,
and
we
tried
to
figure
out.
How
do
we
optimize
this
and
if
we
take
the
the
smallest
segment
bucket,
that
is
bucket
one
and
we
try
to
see?
What
is
the
maximum
number
of
requests
that
were
handled
in
the
histogram
that
is
at
14
is
was
around
45
to
50
requests
and
we
divided
by
it
by
the
number
of
threads
that
we
were
running.
B
That
is
two
threads,
and
when
we
did
that,
lo
and
behold,
we
got
most
of
our
responses
in
the
bucket
one
range
lee.
Could
you
share
that
yeah?
So
we
we
re-ran
the
test
with
a
lower
request
per
second
around
25
to
30,
and
we
saw
that
most
of
the
responses
were
received
in
bucket
one.
So
there
is
tons
of
potential
that
we
should
gather
by
looking
at
these
performance
metrics,
and
there
is
tons
of
optimizations
that
we
can
do
in
this
scenario
as
well.
B
A
Your
results
to
baseline
your
environments
to
use
different
load
generators
because
they
do
statistical
analysis
differently
to
compare
your
configuration,
changes
to
test
your
workloads
on
and
off
the
mesh
to
do
it
against
different
meshes.
Like
sort
of
on,
and
on
with
that,
we
recommend
you,
you
give
give
that
tool
a
shot,
but
I'd
also
recommend
that
you
jump
into
this
community
here
where
critique-
and
I
both
hang
out
quite
a
bit
and
speaking
of
hanging
out
we're
going
to
be
hanging
out
taking
q
a
right
now.
B
B
A
A
You
know
if,
if
there
were
folks
from
the
linker
d
project
here,
they
would
espouse
the
the
performance
and
some
other
characteristics
of
rust,
and
I
think
that
by
and
large
this
probably
is
the
case
that
rust
would
be
have
some
both
performance
characteristics
and
mothers
that
that,
if
you're
doing
intense,
you
know
intense
filtering
of
network
traffic
within
within
an
awesome
environment,
you're
seeking
out
rust
or
even
outside
of
wasp
environments.
Linker
d
service
proxy
is
written
in
rust.
B
C
A
B
Expect
trust
to
be
quite
performant
because
the
language
was
developed
to
be
performant,
but
definitely
it's
a
great
question
that
we
can
do
some
additional
research
to
see
which,
among
as
a
wasm
filter
itself,
if
rust
or
c,
plus,
plus,
which
of
them
is
performant
but
like.
I
would
expect
trust
or
cc,
plus
plus,
to
be
one
of
the
best
perform
in
terms
of
performance.
B
So
there's
a
question:
does
the
measuring
help
to
manage
and
configure
wasm
filters,
if
not
extension
able
to
do
it?
It's
a
very
interesting
question.
So
vosm
filters
and
service
meshes
are
still
a
very
much
an
active
developmental
scope
for
a
lot
of
service.
Mesh
meshery
provides
adapters
for
you
to
try
out
a
lot
of
these
service
meshes
and
as
and
when
the
service
mesh
starts,
supporting
wasm
filters.
A
A
The
use
of
wassum
filters,
but
because
it
is
using
envoy
as
its
sidecar
technology,
it
inherently
does
and
so
near.
As
I
understand
that's
a
fairly
near-term
roadmap
item
for
the
console
team,
but
that
image
of
that
sample
app
is
an
interesting
one
to
play
with
in
part,
because
one
part,
because
I
think
it
begins
to
help
expand
people's
horizons
in
parts.
I
think
deepak
had
a
question
earlier
about.
You
know:
filtering
ingress,
traffic
and
manipulating
it
doing
some
things
with
it.
C
A
C
C
B
A
Cool
very
good-
I
don't
know
if
you
guys
can
hear
me
now
or
not,
but
I'm
going
to
say
something
anyway,
which
is
yeah
okay
good.
Well,
since
this
talk
couldn't
have
been
any
more
of
a
disaster
from
the
out
of
things
and
and
the
microphone
going
dead
and
I'll.
Just
I'll
say
that
I
want
to
say
thanks
pratik,
actually
for
all
the
work
that
you've
done
on
this
very
interesting
analysis
that
you've
done
pleased
that
that
you
were
able
to
use
mesherie
as
much
as
you
did.