►
From YouTube: Making Sense of the Mesh: A Conceptual Model for Service Mesh Topolog... Mitch Connors & Mandar Jog
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Making Sense of the Mesh: A Conceptual Model for Service Mesh Topologies - Mitch Connors & Mandar Jog, Google
Service Meshes such as Istio can be used to connect and secure a variety of deployment topologies, but with great power and flexibility comes great complexity and confusion. In this talk we will walk through a conceptual model for thinking about service mesh topologies, including how to think about networks, clusters, control planes, meshes, and federations of meshes. We will describe best practices for building a topology that can scale and grow with your organization, and discuss a number of different patterns that can be used to simplify your service mesh journey.
A
A
Let's
get
started
so
we're
gonna
talk
about
today,
defining
a
service
mesh,
then
we're
going
to
turn
to
some
of
the
best
practices
for
your
service
mesh
that
you
should
be
thinking
about.
As
a
user
or
as
a
developer
of
a
service
mesh
for
users,
it's
imper
ticularly
important
to
think
about
these
best
practices.
Before
you
begin
your
service
message,
Ernie
yeah,
let's
go
so.
This
talk
starts
like
almost
every
other
talk
that
we've
had
today
with
defining
a
service
mesh
and
I.
A
Really
I've
been
surprised
by
that
we
didn't
coordinate
our
slides
like
I,
didn't
preview,
everyone
else's
talks
and
go
oh
I,
better,
make
sure
to
get
a
service
mesh
definition
and
up
front
instead.
I
think
that
this
is
a
product
of
where
we're
at
in
the
industry
today.
In
spite
of
the
fact
that
service
meshes
have
been
available
in
various
forms
for
about
four
years
now,
still
very
early
in
I.
A
Think
the
software
life
cycle,
or
or
maybe
even
the
the
fad
life
cycle,
you
might
call
it
service
mesh,
is
a
very
hot
topic
today
and
what
I
hope
to
show
is
that
one
of
the
reasons
it's
so
hard
to
define
what
a
service
mesh
is.
Is
that
explaining
a
service
mesh
to
someone
who's
in
a
DevOps
role
is
like
explaining
water
to
a
fish
service
meshes
are
everywhere
they're
everything,
and
so
the
question
is:
are
you
sure
that
this
thing
is
real?
A
A
The
first
thing
that
everybody
talks
about
when
I
get
asked.
What
is
the
service
mesh
is,
is
microservices
and
if
their
eyes,
don't
immediately
gloss
over
at
that
compound
word,
then
they
probably
understand
that
micro
services
involve
a
bunch
of
services,
cooperating
together
over
the
network,
usually
using
HTTP
communication
with
one
another,
usually
they're,
implemented
in
multiple
languages
and
there's
all
kinds
of
dependencies
between
them.
So
that's
a
bunch
of
services,
and
this
grid
here
looks
a
lot
like
a
mesh.
Is
this
a
service
mesh?
A
So
what
I'm
clear?
Isn't
it?
Well,
certainly
when
we
get
to
a
little
bit
of
a
more
older
or
more
enterprise-e
style
architecture,
the
n-tier
architecture,
where
each
layer
communicates
only
with
the
adjacent
layers,
some
of
you
might
be
familiar
with
MVC
or
mvvm
or
NV
something
else.
Certainly
these
aren't
a
service
mesh
right.
You've
only
got
two
network
connections
involved
in
the
interior
architecture.
A
Well,
I've
worked
in
into
your
architecture
and
I've
worked
in
large
enterprises
owning
these
sorts
of
apps,
and
the
truth
is
that
in
production,
I've
never
seen
one
that
looked
like
this,
not
once
at
the
very
minimum
and
into
your
architecture,
looks
a
little
bit
more
like
this.
Where
you've
got
a
load,
balancer
handling
client
connections
into
your
front-end,
which
has
a
load
balancer
to
your
application
layer,
which
it
has
an
allowed
balancer
to
your
data
layer.
A
Maybe
you
want
to
be
able
to
do
canary
deployments
at
one
or
all
of
these
layers,
so
you
might
have
multiple
versions
of
that
layer
running
simultaneously.
The
load
balancer
has
to
be
aware
of
that.
Is
this
a
service
mesh?
Well,
there's
a
lot
more
a
lot
more
great
lines
on
this
diagram
now
than
when
we
started.
Maybe
it
is
certainly
the
monolith
is
not
a
service
mesh
right,
there's,
not
even
one
grey
line
on
this
diagram.
There
are
zero
grey
lines,
not
a
service
mesh
at
all.
A
I
think
we
can
all
agree
with
that,
but
again,
I've
never
actually
seen
anything
that
looked
like
this
in
production.
If
you
have
I
would
love
to
hear
about
it,
it
sounds
fascinating
in
real
life.
What
I've
seen
looks
a
little
bit
more
like
this.
You
know
your
clients
sit
behind
a
load
balancer
that
connects
to
multiple
instances
of
your
monolith.
Hopefully
you
have
enough
traffic
to
need
to
scale
out
horizontally
right.
A
That's
what
we
all
hope
for
is
success
for
our
software
projects
and
hopefully,
if
you're
scaling
out
horizontally
you're,
not
storing
state
directly
inside
your
application.
Things
are
gonna,
be
painful.
If
you
do
that,
so
you
probably
have
a
data
layer
now
where
monolith
has
was
that
nine
gray
lines
in
its
diagram
and
actually
over
time?
Even
if
you
started
out
with
the
monolith,
it
ends
up
starting
to
look,
maybe
a
little
bit
more
like
this.
A
A
A
Do
you
already
have
a
service
mesh?
Is
that
what
we're
talking
about
today?
Well
I
think
it
is,
and
it
isn't,
and
hopefully
that
answer
becomes
more
helpful
than
it
sounds
right
now,
when
we
talk
about
a
service
mesh
product,
what
we're
talking
about
is
how
to
manage
change
across
the
service
mesh
that
you
really
already
have
the
mesh
of
services
that
are
already
your
problem.
Everybody
knows
this
for
micro
services.
Everybody
talks
about
it.
The
example
is,
is
over
done
so
I'm
not
going
to
spend
much
time
on
it.
A
A
Here's
all
the
places
you
have
to
touch
to
make
a
change
across
your
interior
environment,
and
here
are
the
places
you
have
to
touch
to
make
changes
across
your
monolithic
environment.
The
truth
is
service.
Mesh
products
are
about
helping
you
with
the
service
mesh.
You
already
have
today.
That's
what
we're
all
here
to
talk
about.
A
For
a
concrete
example,
let's
consider
what
my
boss,
Sean
just
presented
about
needing
to
migrate
to
Mt
LS
right
you're,
see
so
heard
about
a
software
intrusion,
0
truck
or
maybe
heard
about
zero
trust
at
one
of
the
buzzword,
conferences
he
likes
to
go
to,
and
now
your
problem
is
you
to
implement
this
across
your
database,
your
monolith
that
other
guys
monolith?
The
third
party,
that's
on
the
internet,
that
you
don't
even
control
and
your
client
side,
load,
balancer
or
not
client
side,
but
your
front-facing
client
facing
load
balancer.
This
is
a
huge
task.
A
A
So
what
have
we
learned
here
if
we
do
the
math
over
time?
Hopefully
your
application
grows
in
complexity.
You
add
on
business
cases
or
maybe
add
on
businesses
to
your
company,
and
so
the
number
of
nodes
in
your
application
grows.
The
number
of
edges
in
your
application
grows
and
if
you
have
to
address
every
single
one
of
those
nodes
in
order
to
make
a
reasonable
change,
what
you
have
is
as
Tim
Hawken
kind
I.
Think
yesterday,
at
the
cube
con
rejects
conference
a
service
mess,
your
ability
to
respond
to
change
is
going
down
over
time.
A
Your
investment
in
just
operating
your
application.
This
is
not
development
time.
This
is
just
day-to-day
management,
responding
to
pages
it's
going
up
and
up
and
up
so
how
do
we
gain
control
of
this
mess?
Well,
a
service
mesh
provides
an
interface
to
the
mesh
of
services
that
you
already
own.
That
means,
rather
than
addressing
every
single
one
of
those
nodes
individually.
Taking
all
of
that
painstaking
time
you
are
instead
addressing
the
mesh
as
a
whole
in
the
pre,
you
know
the
service
mess
your
mesh
is
ephemeral.
You
can't
address
it
directly
with
a
service
mesh.
A
A
Well,
I
think,
as
far
as
I
can
tell
every
service
mesh
implementation
that
I've
seen
has
some
sense
of
division
between
a
data
plane
and
a
control
plane,
and
there
are
a
lot
of
options
for
how
these
data
and
control
planes
are
run.
It's
really
not
my
goal
here
to
get
into
the
comparative
analysis
of
which
one
of
these
options
is
better,
but
roughly
you
can
run
your
data.
Plane
is
a
sidecar
which
captures
all
network
traffic
from
the
payload
and
mutates
it.
A
You
can
run
a
node
agent
that
sits
outside
of
your
pod
and
captures
all
the
traffic
and
mutates
it.
Netflix
has
a
great
library
called
hystrix
that
if
you
happen
to
be
using
I,
think
it's
Java
correct
me
if
I'm
wrong
provides
some
great
service.
Mesh-Like
features
without
the
need
to
run
a
separate
process
which
is
great,
G
RPC
is
beginning
to
do
the
same
thing
you
can
run
your
service.
Mesh
sidecar
is
a
black
box,
as
in
it
captures
all
the
traffic
no
matter.
A
What
and
your
application
has
no
idea
or
you
can
run
it
as
a
white
box
where
it
listens
on
a
very
particular
port
and
your
application
can
selectively
send
traffic
to
the
service
measure
or
not.
The
control
plane
in
turn
is
responsible
for
applying
the
changes
that
you
request
to
your
data.
The
data
plane
handles
the
data.
The
control
plane
allows
you
to
control.
It
is
really
the
interface
and
again
I'm
within
control
planes
there's
a
lot
of
variability,
some
of
them
scale
or
sorry.
Some
of
them
are
run
as
monoliths.
A
Some
of
them
are
composed
as
micro
services
or
sub
services.
The
key
is
that
you
just
need
your
control
plane
to
be
run.
A
lot
of
people
worry
about
well,
which
cluster
is
my
control
plane
in
it
just
needs
to
run
and
have
access
to
data
planes
if
you're
running
a
control
plane-
and
you
have
access
to
the
information
you
need
from
the
data
plane,
you
can
have
your
service
mesh
if
you
wanted
to
run
it
on
a
VM
run
it
on
a
VM
bare-metal.
A
Good
luck,
some
config
control
planes
actually
also
are
basically
just
files.
If
you
look
I,
think
history
has
a
great
example
of
how
to
run
their
api's
just
from
config
files.
Your
control
plane
can
be
as
simple
or
as
complex
as
your
use
case
for
it,
and
so,
with
that
in
mind,
we
have
these
service
mesh.
Best
practices
run
copies
of
your
control,
plane
in
different
regions
for
H
a
distribute
your
application
across
the
zones
for
each
a
always
use.
Em
TLS,
never
use
em
TLS
with
self-signed
certificates.
A
Use
your
service
mesh
4
canary
deployments,
use
your
service
mesh
for
green/blue
deployments.
It
gets
overwhelming
so
quickly
and
the
terminology
just
becomes
unintelligible,
as
we
get
deeper
and
deeper
I'll
highlight
this.
One
here
use
your
service
mesh
advice
to
isolate
yourself
from
peers.
That's
my
favorite
service
best
practice.
A
It's
very
easy
to
get
lost
in
the
maze
of
best
practices,
especially
as
you
start
trying
to
compare
between
products.
So
what
we
need
is
a
conceptual
model
for
how
to
think
about
service
meshes
any
service
mesh,
regardless
of
the
way
that
it's
implemented.
That
can
help
us
understand
how
that
service
mesh
will
help
us
meet
our
needs
as
users,
and
so
that's
the
goal
of
this
talk
I'm
going
to
propose
that
we
think
of
service
mesh
topology
in
three
primary
dimensions.
A
You
need
to
think
about
how
you're
going
to
scale
your
control
plane,
how
you're
going
to
scale
your
clusters
and
how
you're
going
to
scale
your
meshes
and
we're
going
to
talk
about
these,
not
as
architectural
primitives
but
as
ways
as
means
to
an
end
ways
to
implement
the
three
goals
that
we
have
in
mind
with
service
mesh,
which
is
going
to
be
high
availability
for
your
app
making
sure
that
it's
running
on
compute
in
many
places
resiliency
for
your
app.
Is
it
hard
to
break
this
thing
and
strong
Trust
guarantees?
A
A
You
can
do
this
just
on
one
single
system
with
multiple
CPUs
or
you
can
scale
these
instances
across
zones
and
regions
where
they're
gonna
have
a
little
bit
more
latency
between
them.
But
it's
important
to
notice
that
this
is
not
the
same
as
multiple
logical
control
planes
that
config
that
they're
operating
on
is
still
represented
by
this
one,
nice
generic
database
icon.
You
have
one
logical,
config,
operating
multiple
instances
of
your
control,
plane
and
you've
achieved
some
level
of
high
availability.
Because
of
that
you
can
also
scale
your
control
plane.
A
Logically,
this
is
a
way
for
you
to
isolate
your
configuration
so
that
changes
that
a
user
makes
are
not
going
to
immediately
bring
down
the
entire
service.
You
would
bring
down
half
which
okay,
maybe
scale
it
three
or
four
times
and
bring
that
number
down
a
little
bit.
But
the
point
is
by
running
multiple
config
stores
and
multiple
control
planes.
A
Reading
from
those
stores,
you
now
have
the
ability
to
roll
out
a
change
over
part
of
your
network,
not
all
of
your
network
and
to
test
that
change
and
then
to
expand
the
change
into
the
other
logical
control,
plane.
There's
a
question
mark
in
this
diagram
because
we
do
need
something
to
manage
moving
us
from
the
one
instance
to
the
multiple
law
or
not
instance.
Sorry
from
the
one
logical
control
plane
to
multiple
logical
control,
planes
reconciling
that
config
between
those
storages
and
we've
heard
about
multiple
products.
Today
that
will
assist
you
in
that
journey.
A
The
next
dimension
that
we
want
to
talk
about
scaling
your
service
mesh
is
clusters.
This
is
how
you
scale
compute.
This
helps
us
meet
the
objectives
of
high
availability
and
application,
configure
isolation,
notice,
I'm,
not
saying
service
mesh,
config
isolation,
which
is
a
separate
topic
that
we
just
covered
with
how
you
scale
logical
control,
planes
with
multi
cluster
you're,
able
to
separate
out
your
application
into
distinct
compute
environments
oftentimes.
This
is
what's
necessary
in
various
public
cloud
environments.
A
Be
the
service
for
one
is
one
service:
it's
important
to
note
that
and
service
a
can
selectively
call
service,
B
locally,
which
is
likely
to
be
the
most
inexpensive
networking
call
or
if
service
B
in
cluster
one
is
for
some
reason,
overwhelmed
or
flat-out
down
so
service
a
can,
communicate
with
cluster
to
service
B
using
a
service
mesh.
It's
important,
though,
that
we
respect
the
namespace
and
service
naming
equivalence
in
multi
cluster.
A
If
you
have
the
same
service
name
and
the
same
namespace
name
in
two
clusters
that
are
part
of
one
mesh,
they
are
one
service.
You
can
route
to
them
separately,
using
Geographic
load
balancing
you
can
think
of
them
separately
in
some
ways.
But
you
have
to
remember
that
a
given
request
for
service
B
could
be
served
in
cluster
1
or
cluster
2.
This
also
provides
some
isolation
for
your
operators.
A
If
you're
giving
someone
access
to
cluster
2
notice,
they
cannot
change
the
operational
paradigms
or
the
operational
rules
and
configuration
for
service
a
one
thing
to
keep
in
mind
and
a
few
people
have
called
this
out.
If
you
have
this
model,
if
you
have
this
need
as
a
user,
keep
in
mind
that
your
across
cluster
traffic
may
be
traveling
over
the
public
internet.
So
don't
do
this
without
strongly
in
investing
in
understanding
the
mutual
TLS
or
other
encryption
packages
that
are
available
to
you.
A
What
this
doesn't
give
us
is
the
ability
to
have
hard
multi-tenancy
for
service
mesh
configuration
or
for
service
identity
notice
that
cluster
2
is
able
to
impersonate
service
B
in
this
case,
and
if
they
were
able
to
spin
up
a
service
with
the
same
name
as
service
a
a
name
space,
they
could
impersonate
service
a
as
well.
So
this
is
not
suitable
for
a
hard
multi-tenant
environment,
but
potentially
for
different
teams
within
an
organization.
It
could
be
useful,
but
it's
also
completely
orthogonal
to
control
plane
scaling.
A
You
can
run
Multi
cluster
with
one
control
plane
with
two
instances
of
one
logical
control,
plane
with
three
logical
control
planes
across
only
three
instances
and
two
clusters:
it's
orthogonal.
They
move
independently
and
that's
going
to
be
customizable
to
your
particular
use
case
and
needs
also
we're
talking
a
lot
here
about
pods
and
clusters,
and
things
like
that.
This
is
cube
con.
After
all,
so
we're
talking
a
lot
about
kubernetes,
but
a
cluster
is
really
from
a
service
meshes
perspective,
just
a
context
for
running
compute.
We
have
these
five
needs
from
a
cluster.
A
We
need
something
that
will
run
our
control.
Plane
will
run
our
application,
we'll
make
sure
our
data
plane
is
running
wherever
it
needs
to
run
sidecar,
node
agent,
etc.
Then
we
need
something
to
tell
us
where
to
find
a
service
and
we
need
something
to
hold
our
config
in
some
cases.
Actually,
some
service
messages
come
with
that
prepackaged.
If
you
can
satisfy
those
five
goals,
you
have
a
cluster,
it's
not
a
kubernetes
cluster.
Don't
try
to
create
a
pod
on
whatever
it
is
you've
cobbled
together.
A
A
And
so
now
that
we've
gotten
up
to
the
scale
of
multiple
clusters,
multiple
control
planes
across
logical
and
instance,
scaling
we've
reached
the
ultimate
abstraction
right,
well,
I!
Think,
there's
one
more
level
of
detail
that
we
need
to
talk
about,
and
that
is
multiple
meshes
running
concurrently.
This
is
a
little
bit
more
abstract
and
it's
it's
early.
It's
something
that
is
not
as
well
supported
in
documentation
across
service
meshes
as
it
probably
ought
to
be,
and
as
it
needs
to
be,
as
it
becomes
a
more
popular
alternative.
A
In
this
case,
we've
been
sharing
less
and
less
and
less
as
we
move
forward
in
our
service
mesh
journey
we've
been
sharing,
no
compute,
we've
been
sharing,
no
config,
we've
been
sharing,
not
even
the
same
cluster
and
within
multi
mesh.
What
you're
sharing
is
only
trust,
and
it's
not
trust
for
a
lot.
Frankly,
it's
just
identity
trust
when
you
scale
to
multiple
meshes.
You've
got
multiple
clusters.
Okay,
now
there
are
orthogonal,
you
could
theoretically
run
Multi
mesh
within
a
single
cluster.
Probably,
don't,
though,
you
need
that
sort
of
logical
separation
of
concern.
A
You
don't
want
administrators
to
have
access
to
your
cluster.
The
config
could
just
get
quite
confusing.
In
this
example,
we've
got
services
a
and
B
and
within
our
prod,
these
identities
are
specific.
No
one
calling
production
service
B
should
ever
accidentally
get
routed
over
to
developer
service.
B
right,
a
developer
could
be
whatever
is
at
the
tip
of
github.
A
Your
results
may
vary
wildly
with
how
that
performs.
So
these
identities
are
completely
mesh
specific
there's
no
need
to
worry
about
naming
overlap
or
anything
between
them,
but
they
can
establish
strong
trust
between
them.
When
service
a
receives
a
call
from
service,
be
it
can
be
certain
that
service
B
is
running
in
your
developer
environment
service,
be,
alternatively,
can
also
be
certain
that
it's
reaching
service
a
in
your
production
environment.
A
A
Thank
you,
you're,
not
trusting
anything
other
than
identity,
so
service
B
has
no
ability
not
only
to
impersonate
service
beyond
prod
mesh
on
the
prod
mesh.
It
also
has
no
ability
to
change
its
execution.
You
can't
overwhelm
it
with
too
much
traffic.
You
can't
change
its
application,
configuration
or
change
the
image
that
it's
running
against.
There's
complete
and
total
isolation
between
these,
and
so
you
can
have
trust
well.
I
hope
that
this
has
helped
to
clarify
some
of
the
dimensions.
A
A
A
Yeah,
so
so
the
question
is:
if
we
have
any
real-world
of
examples
of
where
multi
mesh,
Federation
and
Trust
Federation
have
been
used
in
production
and
I,
don't
have
any
of
those
available
today.
What
you're
talking
about
is
I,
think
at
the
bleeding
edge
of
what
service
meshes
have
to
offer,
and
so
I
don't
know
of
any
implementations.
However,
I
mentioned
that
my
boss,
preceded
me,
my
other
boss
will
be
here
after
me,
and
that
would
be
a
great
question
to
bring
up
to
Louie.
As
he
comes
to
the
stage.
Oh.
A
Oh,
do
I
envision
a
scenario,
sure
many
many
enterprises
utilize
workday
and
they
need
to
enter
utilize
a
network
they
in
a
secure
fashion.
There
are
going
to
be
HR
systems
that
need
to
query
the
workday
API
and
there's
going
to
need
to
be
data,
exports
that
come
from
workday
back
to
our
HR
systems.
There's
going
to
be
a
complex
system
integration
for
these
things
in
place.
What
a
multi
mesh
could
do
for
that
is
allow
one
single
route
of
trust.
To
guarantee
to
all
use.
A
Google
is
an
example,
because
I
work
there
to
allow
Google
to
certify
that
they
are
in
fact
contacting
the
workday
api's
and
the
workday
API
is
to
be
certain
about
exactly
what
service
within
Google
is
contacting
them.
Then
they
can
set
up
our
back
rules
to
meet
whatever
their
business
needs
are
to
say
that
the
time
off
service
has
access
to
my
PTO
day
API,
but
it
does
not
have
access
to
my
medical
benefits
API,
for
instance,
Google
assistant,
everyone.
Isn't
it
wonderful?
A
A
A
Okay,
so
to
repeat
the
question:
we're
asking:
what
is
the
use
case
for
a
white
box
data
plane
where
the
application
is
aware
of
the
fact
that
it
is
being
proxied
by
a
service
mesh
rather
than
the
black
box
case,
where
we
simply
collect
all
the
traffic
and
proxy
it
on
their
behalf
as
I
understand
it?
Many
of
the
Envoy
based
service
meshes
have
limited
support
for
traffic
like
UDP,
and
so
there
are
going
to
be
cases
where
a
single
application
needs
to
have
UDP
and
TCP
traffic
that
UDP
traffic.
A
They
would
be
able
to
selectively
bypass
the
proxy
and
route.
It
natively
I
think
that
you
could
probably
come
up
with
other
use
cases
where,
for
various
business
reasons,
you
may
not
want
a
certain
particular
piece
of
traffic
to
be
a
part
of
your
service
mesh,
while
everything
else
should
be.
In
that
case,
a
white
box
solution
would
be
appropriate.