►
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Service Mesh Patterns by the Book- Lee Calcote, Layer5 & Nic Jackson, HashiCorp
Infrastructure diversity is a reality for many organizations. It’s predicted that by 2022, 90% of all apps will feature microservices architectures. A huge range of microservice patterns drives a world of multiple service meshes. As various service meshes have proliferated infrastructures, service mesh patterns and abstractions have emerged. We will break down 60 service mesh patterns into different categories of use, demonstrating and examining a select few using Meshery for deeper review of their problems they solve, discussing caveats, and highlighting anti-patterns. The patterns discussed are being published in Service Mesh Patterns (O’Reilly) by Lee Calcote and Nic Jackson.
A
B
I
think
it's
a
really
good
question
and
I
think
it's
one
that
I'm
kind
of
like
really
hesitant
to
answer,
because
I
think
it's
it's
one
of
those
things
that,
like
I
think,
with
a
lot
of
industry
applications.
You
you
build
knowledge
from
using
the
tooling
now
with
like
service
mesh
service.
Meshes
is
really
new
in,
in
the
sense
that
you
know
it's
been
around
for
for
a
while.
The
concepts
are
not
necessarily
new,
but
the
sort
of
the
implementation
of
like
of
having
this.
B
So
back
at
this
problem
statement
again
distributed
systems
management
is
hard,
and
I
think
this
is
kind
of
like
continually
ringing
true
and
as
we
kind
of
increase
the
complexity
of
our
systems,
it
just
kind
of
gets
more
and
more
of
a
problem
and
like
the
core
things
we
want
to
talk
about
in
this
presentation
is
looking
at
around
the
network
and
actually
how
the
the
network
has
a
major
effect
on
your
application
and,
of
course,
some
of
the
things
that
you
can.
You
can
do
with
this.
B
Networking
has
never
been
more
significant
for
developers
because
it
it
kind
of
is
necessary.
Part.
Now,
we've
moved
to
this
micro
service.
Everything
is
connecting
over
the
network.
Where,
previously
you
know
it
would
have
been
an
in-process
procedure.
Call
let's
take
a
look
at
a
little
demo
of
how
network
can
actually
affect
our
system
that
lee's
going
to
show
us
now.
A
A
B
A
A
A
A
Those
behaviors
that
criteria
can
be
captured
in
a
pattern
can
be
applied
to
different
meshes
surface
mesh
patterns
are
mesh
agnostic.
Clearly,
that's
part
of
the
goal
they
it.
To
that
extent,
they're
intended
to
be
beneficial
to
any
who
are
running
a
service
mesh.
These
patterns
are
reusable
they're
exchangeable.
A
B
Okay,
so
now
you
know
what
we're
trying
to
achieve
with
with
the
patterns.
Let's
take
a
look
at
a
couple
of
examples,
so
the
first
one
I
want
to
show
you
is
probably
the
most
common
pattern
that
you're
ever
going
to
find
inside
of
applications
and
also,
if
misconfigured,
probably
the
one,
that's
really
responsible
for
bringing
your
application
down
to
the
knees.
B
So
in
the
previous
sort
of
demo
that
we
saw
what
we
saw
was
a
an
upstream
connection
to
a
service
and
it
was
intermittently
failing
now
the
the
reasons
behind
the
intermittent
failure.
It
could
be
a
number
of
things.
It
could
be
just
something
on
the
very
pure
network
level
or
it
could
be
some
application
level.
B
The
retry
pattern
actually
operates
at
both
layer,
4
and
layer
7..
So
it
understands
the
concept
of
the
connection.
Can
we
connect
to
an
upstream
service,
but
it
also
understands
what
success
looks
like
with
regard
to
the
layer,
seven
protocol
that
you
are
using
so,
for
example,
the
payment
service
here
is
going
to
try
and
connect
to
the
currency
service.
Now,
with
a
layer
4,
it's
going
to
try
and
establish
a
tcp
connection
to
that.
B
If
it
can't
do
it,
it
will
then
go
and
get
that
connection,
failure
message
and
it's
going
to
go
into
its
retry
loop.
It's
going
to
retry
another
one
of
the
instances
of
the
currency
service.
So
we
see
that
happening
here
now
it
becomes
successfully
connected.
So
what
the
retry
then
does
is
retries
analyzing.
That
request-
and
it
says,
based
on
my
knowledge-
has
the
response
that
I've
been
given
matching
my
criteria
of
success.
So
in
this
instance
it's
an
http
request
and
the
measure
of
success
is
going
to
be
http
status
code
200.
B
now
in
the
instance
here
we
haven't,
got
a
200,
we've
got
a
500.,
so
what's
happening
is
again:
the
retry
is
going
to
go
into
that
loop
and
it's
going
to
try
another
endpoint
and
it'll.
Do
that
for
a
configurable
number
of
times
until
it
gets
success
now
this
isn't
really
kind
of
healing
your
system,
but
what
it's
doing
is
protecting
your
end
user
from
the
exposure
of
these
problems
and
that's
very
important.
B
A
retry
only
buys
you
time
until
you
fix
an
underlying
problem.
It
shouldn't
be
the
fix
itself,
and
I
think
this
is
kind
of
like
a
key
thing.
When
you
apply
a
retry,
it
shouldn't
be
something
which
you're
using
as
a
kind
of
a
permanent
fix.
You
should
always
use
it
as
a
temporary
measure,
because
quite
often,
problems
will
get
worse
and
you'll
get
to
a
point
when
the
retry
can't
help
you.
B
I
think
another
important
point
about
retries
is
item
potency.
So,
in
terms
of
item
potency,
what
we
mean
is
that
if
you
attempt
a
request
multiple
times,
you
should
get
exactly
the
same
response.
So,
for
example,
if
I
try
and
get
a
product
detail,
then
every
time
I
make
a
request
for
that
product
detail,
it
shouldn't
be
changing
any
state
in
the
database.
It'll
just
be
returning
me.
The
same
data
that'll
be
considered.
Item
potent.
B
Therefore,
it's
actually
safe
to
retry
a
number
of
times.
Something
which
necessarily
might
not
be
idempotent,
however,
is
something
like
when
you
take
a
payment
so
looking
deeper
into
to
something
which
might
necessarily
not
be
item.
Potent
is,
is
this
order
service,
so
this
order
surface
has
actually
three
steps.
The
first
thing
the
order
service
does.
Is
it
processes
the
payments
with
an
external
payment
gateway?
B
B
B
B
B
So,
for
example,
if
you've
got
five
instances
of
a
particular
service
and
they're
they're,
exhibiting
a
kind
of
an
error
rate
of
two
percent,
that's
kind
of
evenly
distributed
across
all
of
the
service
instances,
so
given
kind
of
an
at
the
application
of
a
single
retry,
I'm
going
to
just
have
I'm
just
going
to
retry
once
and
then
I'm
going
to
give
up.
What
you
end
up
with
is
is
a
kind
of
a
probability
of
.004
that
both
the
original
request
and
the
new
request
will
will
both
fail.
B
B
So
then
you
can
kind
of
look
at
that
and
say
well,
a
single
retry
isn't
applicable,
so
like
hey,
maybe
maybe
all
I'm
going
to
do.
Is
I'm
just
going
to
do
five
retries?
You
know
it
can
be
really
tempting
to
use
this.
This
high
retry
sort
of
attempt,
but
the
problem
that
comes
with
that-
is
that
whenever
you're
retrying
in
a
system,
your
system
is
running
at
reduced
capacity.
B
So
does
your
system
have
the
capability
to
kind
of
handle
this
load,
because
if
it
doesn't,
what
you
might
actually
be
doing
with
your
retry
is
not
sort
of
increasing
the
reducing
the
failure
rate
from
25
percent
to
zero.
You
might
be
increasing
that
failure
rate
to
a
hundred
percent,
because
the
system
is
now
all
backed
up
and
it
can't
handle
any
of
the
requests
at
all.
A
A
A
So,
let's
head
back
into
measuring
and
look
at
how
it
uses
patterns
it
uses
these
patterns
to
help
you
overcome
those
distributed
systems.
Challenges
recall
that
in
measuring
you
can
use
it
cli
to
import
a
pattern
and
in
the
ui
you
can
also
then
edit
that
pattern
in
context
of
the
retry
pattern
that
dick
was
discussing
earlier.
A
A
A
B
B
B
Now,
there's
a
kind
of
a
number
of
reasons
why
you
want
to
do
this?
The
the
foremost
is
that
if
a
service
is
failing
just
because
it's
busy
actually
breaking
the
circuit
and
not
sending
any
requests
to,
it
will
allow
it
to
recover
in
the
instance
that
it
is
completely
faulty,
then
what
you
can
actually
do
is
you're
removing
it
from
the
the
kind
of
the
process
and
you're
failing
fast.
You
can
handle
that
in
a
number
of
different
ways,
but
it
ultimately
protects
the
system
downstream
from
any
slow
list
which
can
occur
upstream.
B
The
way
that
it
works
is
this,
so
a
request
comes
into
the
system.
If
the
circuit
breaker
is
is,
is
open,
then
you're
immediately
going
to
fail,
but
let's
assume
it
isn't
right.
So
we're
just
kind
of
getting
into
this
situation
where
we're
starting
to
experience
errors,
so
the
circuit
breaker
is
closed.
B
The
request
is
called
upstream
as
normal,
and
if
it's
successful,
it's
returned
as
normal.
However,
if,
if
it
isn't
successful,
then
what
the
circuit
breaker
does
is
it's
starting
to
keep
this
internal
count,
so
it
says:
well,
I've
had
a
number
of
failures
and
this
this
could
be
consecutive
failures
or
it
could
be
failures
over
a
period
of
time,
but
it
kind
of
looks
at
a
threshold
that
you
can
figure
when
that
threshold
is
exceeded.
B
What
happens
is
the
circuit
breaker
opens,
so
no
more
requests
are
going
to
get
sent
to
this
service,
but
then
like?
How
do
you
determine
when
it
recovers,
so
the
circuit
breaker
after
a
certain
period
of
time
and
is
what's
known
as
a
half
open
state,
so
in
the
half
open
state
a
request,
a
kind
of
an
exploratory
request
will
be
sent
to
the
upstream
service.
B
If
it
succeeds,
then
again
the
circuit
breaker
will
will
close
and
we'll
go
back
into
normal
operation.
If
it,
if
it
fails,
then
we
keep
it
open
and
we
can
kind
of
continue
to
assume
that
it's
still
in
that
healing
phase
like
all
of
the
patterns,
they
kind
of
look
simple
on
the
sort
of
the
surface,
but
you
know
to
effectively
use
them.
There
are
a
number
of
considerations
caveats
that
you've
got
to
think
about.
B
Health
checks
are
quite
a
common
one
that
when
you're
kind
of
configuring
thresholds
for
circuit
breaker,
you've
got
to
do
this
with
the
health
check
in
mind.
So,
for
example,
if
you
have
a
threshold
for
a
circuit
breaker,
which
is
greater
than
the
kind
of
the
passive
health
checks
which
exist
in
your
system,
then
circuit
break
is
going
to
have
no
effect,
because
the
health
check
will
detect
the
failure
before
the
circuit
breaker
does
circuit
break
is
designed
to
be
to
detect
failures
quickly,
using
active
requests
as
a
method
to
do
so.
B
Health
check
endpoints
also
don't
report
things
on
such
as
database
logs,
I
mean.
Sometimes
they
do,
but
quite
often
not
they
don't
so
you
can
have
a
failure.
For
example,
that
is
due
to
just
a
lock
on
a
right
on
a
database
if
you
ping
it
as
a
health
check,
a
health
check
might
just
be
saying:
hey,
are
you
alive?
Are
you
okay?
Are
you
up
so
you're
not
going
to
kind
of
get
a
report
on
that
where
the
circuit
break
is
really
effective?
B
B
What
we
really
should
be
doing
is
treating
those
independent
things
as
with
our
own
circuit
breaker.
So
a
right.
Yes,
we
should
absolutely
open
the
circuit
when
we
can't
write
to
to
a
product,
but
we
shouldn't.
If
we
can
continue
to
read,
we
should
kind
of
try
and
keep
as
much
of
our
service
in
circulation.
So
we've
got
to
like
you
know.
We
have
to
think
about
these
things.
Quite
quite
fine
grained.
A
A
A
A
B
So
so
lee
and
I
have
been
been
working
on
a
book
called
service
mesh
patents
for
for
o'reilly,
and
it's
currently
an
early
release.
We
would
love
your
opinion
and,
if
you've
got
any
comments,
or
maybe
some
some
real
sort
of
gems
of
information
or
some
esoteric
kind
of
gotchas
that
you
think
would
make
great
reading.
We
would
love
to
to
kind
of
hear
all
of
those
things
for
you
and
you
can
check
that
out
and
also
with
with
cncf
we're
about
on
slack
any
questions
we
would
be
overjoyed
to
hear
from.