►
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Planning the Zero-Downtime Lifecycle of Your Service Mesh- Christian Posta, Solo.io
A service mesh is a critical piece of application infrastructure that lives on the request path between your services. Once you get past the “hello world” for your particular mesh, you are left having to plan out your production deployment and more importantly, future upgrades. The architecture and patterns for deployment are as important (if not more?) as the specific mesh capabilities you choose. For example, patterns like separating ingress, running canaries, and focusing on limited configuration blast radius are foundational to enable zero-downtime upgrades of your mesh. In this talk, we discuss the vital patterns and practices cultivated working with mesh adopters around the world. The audience should come away with a core set of practices to enable successful lifecycle management of their service mesh.
A
A
I
would
like
to
point
this
out
real
quick,
we've
featured
in
the
surface
mesh
radar
report,
where
closer
to
the
bull's
eye
closer
to
the
middle
here
is
best
and
you
can
see
solar
io
we've
been
working
very
hard,
and
you
know
that
that
shows
in
the
in
the
analyst
observations
of
us
as
well.
So
let's
get
right
into
it
service
mesh
as
you've.
A
You
know,
we've
been
doing
these
conferences
for
a
while
now
mishkan
and
the
folks
that
have
been
coming
to
the
these
types
of
talks
are
probably
familiar
with
service
mesh
technology
and
it
loosely
defines,
as
you
know,
a
framework
that
deploys
so
and
allows
you
to
offload
application.
Networking
concerns
from
the
application
to
a
little
agent
or
proxy
that
runs
co-located
with
the
application
and
uses
some
kind
of
control
plane
to
manage
the
configuration
and
and
drive
the
the
the
policies
and
the
the
behavior
of
the
network.
A
As
observed
and
enforced
by
the
little
side,
cars
or
these
sidecar
proxies
that
live
with
the
applications,
the
operators
or
the
end
users.
They
configure
the
control
plane,
control
plane,
ends
up
configuring.
The
data
plane
now
the
service
mesh
is
a
very
critical
piece
of
application.
Networking
infrastructure,
when
it's
in
place
as
you
can
see,
these
proxies,
live
on
the
data
path
with
the
application
talking
to
the
network.
The
requests
are
going
through
these
l7,
these
layer,
7
proxies
and
since
that's
the
case,
requests
are
flowing
through
these
proxies
configuration
life
cycle
upgrades.
A
These
types
of
things
are
incredibly
important
events
and
need
to
be
planned
for
and
shouldn't
be
taken
lightly,
because
if
you
start
to
take
down
the
request
path
between
your
various
services,
you
start
to
see
outages,
and
that
is
not
something
we
want
right.
So
before
we
get
into
planning
for
the
you
know
how
how
you
go
about
managing
the
life
cycle
of
a
service
mesh,
which
is
a
very
critical
component
in
your
in
your
architecture.
A
A
This
is
a
very
real
scenario,
so
we
got
to
explore
this
this
failure
domain
here
now.
Ideally,
this
doesn't
affect
the
request
path.
You
just
end
up
with
stale
configuration
and
you
don't
see
new
configuration
or
updates
to
the
system
that
have
happened
since
the
communication
went
down
and
modern
proxies
like
envoy,
proxy
and
so
on
can
can
deal
with
this
and
are
expected
to
deal
with
this.
In
fact,
the
model
of
configuration
for
envoy
was
built
specifically
to
be
eventually
consistent
and
can
survive
for
some
period
of
time
in
this
particular
state.
A
So
you
might
miss
updates
to
endpoints
and
and
so
on,
but
the
proxy
can
do
things
like
outlier
detection
and
kind
of
work
around
misbehaving
endpoints
until
communication
is
re-established.
We'll
take
a
look
at
some
approaches
for
dealing
with
this
and
how
to
minimize
this
sort
of
this
sort
of
communication
failure.
A
Another
area
that
we
want
to
look
at
is
how
does
traffic
get
into
the
service
mesh
as
a
service
mesh,
the
proxies
they
can
do
things
like
mutual
tls.
They
can
do
things
like
service
to
service
authorizations
and
enforcing
those
types
of
policies,
but
somehow
traffic
has
to
get
into
the
mesh
before
the
mesh
can
start
to
enforce
these
types
of
things.
Some
service
meshes
have
a
a
a
well
defined
way
for
how
traffic
gets
in
through
some
sort
of
ingress
gateway.
A
Now
another
thing
to
also
watch
out
for
is
the
configuration
domains
and
scopes,
so
you
can
have
you
want
to
watch
out
for
global
changes
or
global
configurations
that
potentially
upon
upgrade
can
take
down
the
configuration
of
the
data
plane,
and
so
this
would
impact
the
request
path
and
whether
services
can
communicate
with
each
other.
So
you
know
keep
keeping
in
mind
what
is
the
scope
of
your
configuration?
A
What
configurations
are
backward
or
forward
forward
compatible
and
how
to
work
around
and
at
least
control
the
blast
radius?
For
these
types
of
changes?
Now,
there's
many
other
things.
You
know
those
are
a
few
of
the
areas
that
we're
going
to
explore
today,
but
a
service
mesh
is
a
critical
piece
of
infrastructure.
That's
likely
highly
integrated
with
other
parts
of
your
stack,
so
things
like
a
dedicated
certificate
authority
for
signing
workload,
certificates
or
setting
up
edge,
ingress
certificates.
A
A
The
first
thing
that
we'll
take
a
look
at
is
getting
traffic
into
the
service
mesh
itself.
As
I
mentioned,
some
service
meshes
have
an
out
of
the
box
gateway
that
allows
you
to
go
from
untrusted
un,
unknown
to
the
mesh
type
traffic
to
trusted
and
and
inside
the
mesh.
You
know
ingress
inside
the
mesh.
A
A
You
know
this
this
sort
of
model
kind
of
baked
into,
but
instead
of
you
know,
you
have
a
few
different
options,
but
you
can
you
can
do
this
through
the
operator
and
have
different
life
cycles
through
the
operator
by
setting
up,
let's
say
configuration
just
for
the
ingress
and
configuration
just
for
the
control
plane.
Have
those
separated
out
come
completely
so
that
they
can
be
updated,
upgraded
and
so
on
independently.
A
Here's
an
example
of
how
you
might
do
that
using
the
istio
configurations.
On
the
left
hand,
side
we
see
configuration
for
the
control
plane.
Note
the
bolded
areas
here
you
know
we're
we're
just
installing
the
control
plane
piece
by
itself,
no
other
pieces
and
on
the
left
side
on
the
right
hand,
side
we're
installing
just
the
ingress
gateway
component,
nothing
else,
and
so
we're
able
to
run
these
in
conjunction
and
have
separate
life
cycles
at
the
operator
level.
A
Now
this
will
come
in
handy
once
we
start
to
upgrade
the
rest
of
the
control
plane,
because
we
can
do
that
independently
of
the
of
the
gateway.
So
we
can
separate
out
the
possibility
of
taking
down
traffic,
that's
coming
into
the
mesh
from
the
upgrade
of
the
data
plane
inside
the
mesh
itself
and
so
to
minimize
those
those
down
times
or
those
stale
configurations
and
and
minimize
the
the
connection
loss
between
the
data
plane
and
the
control
plane.
A
A
We
want
to
avoid
big
bang
upgrades
or
in
place
upgrades
that
have
the
potential
to
to
take
down
the
entire
system.
Even
though
they
might
have
you
know
backward
compatibility
built
into
into
the
implementation,
so,
for
example,
with
with
your
service
mesh
with
again
using
istio
as
an
example,
we
can
deploy
a
canary
control
plane
and
not
affect
any
of
the
the
data
plane
traffic,
whether
at
ingress
or
on
the
east
west
side
and
from
there.
What
we
can
do
is
slowly
introduce
the
the
relevant
components.
A
So
in
this
case
we're
going
to
introduce
a
an
ingress
gateway,
that's
tied
to
the
new
control
plane,
the
canary
control
plane.
It
won't
take
any
traffic
yet,
but
we're
just
we're
going
to
start
getting
the
pieces
in
place
so
that
we
can
slowly
and
surgically
roll
the
the
the
data
plane
pieces.
So
in
this
case
we're
starting
to
see
you
know
one
workload
move
over
to
the
new
control
plane.
Everything
still
continues
to
flow
traffic's
flowing
into
the
load
balancer
and
through
the
original.
A
You
know
the
ingress
gateways
and
we're
slowly
rolling
over
the
traffic.
Now
you
can
see
more
of
those
workloads
and
now
even
the
ingress
gateway
is,
is
taking
traffic
and
bringing
traffic
into
the
various
workloads
and
then
the
more
traffic
we
roll
over
to
the
new
control
plane.
You
know
the
more
that
we
get
closer
to
shutting
down
and
eliminating
the
the
traffic
through
the
older
control
plane
and
if,
at
any
point
in
this
process,
we
see,
we
see
an
issue,
we
can
roll
back
and
go
back
to
the
the
old
control
plane.
A
A
So,
for
example,
scoping
configuration
to
specific
applications
or
in
kubernetes
two
specific
name
spaces
is
highly
desirable.
So
that's
that's
kind
of
step.
One
try
to
avoid
large
scale
global
configuration
in
istio.
That
means
you
know
putting
configuration
into
the
istio
system
name,
space
with
which
is
an
operator,
or
rather
you
know
a
platform
team
user,
restricted
namespace.
We
don't
want
everybody
dumping
their
configs
into
here
kind
of
sculpt
them
down
to
their
specific
namespaces
or
where
applications
live.
Or
you
know
you
might
have
tendency
and
ownership
rules
about
that.
A
You
can
even
use
things
in
istio
specifically
to
scope
it
down.
So
when
you
deploy
configurations
into
istio,
the
seo
control
plane
is
watching
all
of
those
configurations
across
all
the
name
spaces
and
trying
to
configure
you
know
the
mesh
globally.
But
what
you
can
do
in
istio,
for
example,
is
use
the
export
to
a
field
in
the
various
configuration
objects
like
virtual
service
or
destination
rule,
and
so
export
two
controls
where
those
configurations
get
applied.
A
A
Now
this
export
2
is
a
is
a
list,
so
you
can
be
very
specific
about
what
namespaces
should
observe
this
particular
con
configuration,
but
you
know
narrowing
those
down
and
limiting
those
is
highly
desirable
and
keeping
in
mind
where
configuration
could
conflict
with
an
upgrade
like
in
seo.
There's
a
break
last
feature
that
allows
you
to
override
the
capabilities
of
the
underlying
data
plane
in
this
case
envoy
proxy,
with
a
with
a
resource
called
envoy,
filter
right,
so
envoy
filter
also
has
an
export
to
field
which
you
can
configure
to
scope
this.
A
So
you
know
these
are.
These
are
some
of
the
failure
areas
and
life
cycle
things
that
you
should
keep
in
mind
when
managing
operating
a
service
mesh
at
solo,
we're
working
with
some
of
the
largest
largest
service
mesh
deployments
in
the
world.
If
you're
interested
in
seeing
what
we
more
of
what
we're,
what
we're
doing
and
working
with
our
customers
then
or
we're
building
working
on
service
mesh
and
building
the
products
themselves,
please
reach
out.
We
are
hiring
across
across
the
world.
A
As
you
can
see,
our
presence
is
global
and
we're
hiring
for
people
working
in
the
field
with
the
customers.
People
writing
the
code
in
the
back
end
everything
sales
customer
success.
Everything
in
between
so
I
just
want
to
thank
you
for
joining
this
session,
definitely
check
out
some
of
the
youtube
channels
that
we
have
so
we
we
share
a
lot
about
what
we're
doing
with
our
customers.
You
can
learn
a
lot
about
about
istio
service
mesh
in
general,
envoy,
proxy
and
some
of
the
innovative
stuff
that
we
are
working
on
at
solo.