►
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Delivery Done Right with Service Mesh - Brandon Barrow, Shipt & Nick Nellis, Solo.io
Shipt is a personal shopping and delivery company that connects retailers with local customers for everyday essentials brought right to the door. And our service mesh now sparks the connections that shows why every microservice is just as valuable. From grass-roots movement to production, our mesh implementation solved problems around ingress, security, multi-tenancy, and multi-cluster operations at scale. Join us as we delve into our challenges and successes, all to help spark ideas for your implementation.
B
A
Super
excited
to
be
stamped
here
with
a
ship
that
we're
going
to
talk
about
kind
of
the
next
generation
of
service
mesh
and
how
a
shift
is
implemented.
Some
really
amazing
features
to
get
the
like
most
bang
out
of
your
buck
from
service
mesh
and
so
I'm
gonna
hand
it
over
to
Brandon
here
to
kind
of
talk
about
ship.
C
Awesome
yeah
I'm
Brandon
Barrow
senior
engineer
at
shipt
I
work
on
our
developer
platform
team,
so
we
expose
a
self-service
platform
for
developers
that
we
hope
they
enjoy
I'm,
pretty
passionate
about
developer
experience,
so
that's
kind
of
what
I
I
try
to
do
every
day
and
work
primarily
on
kubernetes
go
service,
mesh
and
API
Gateway.
As
of
late.
C
C
We
process
about
3K
RPS
on
average
Services
communicate,
mostly
HTTP,
and
some
grpc
and
yeah.
We
have
some
other
languages
as
well,
but
most
mostly
everything's
written
in,
go
so
beginning.
We
sort
of
reached
the
point
where
we
might
need
something
like
a
service
mesh.
Once
we
started
doing
a
cloud
migration,
so
initially
most
of
our
clusters
taught
sort
of
within
themselves
using
cluster.localdns.
C
We
expose
a
yaml
file.
Tell
me
if
you've
heard
that
one
before
for
developers
in
their
repo
that
allow
them
to
sort
of
Define
what
dependencies
they
have
injected
into
an
environment
variable
and
introduces
the
the
native
kubernetes
DNS.
However,
when
we
started
implementing
multi-cluster,
this
became
problematic,
and
so
we
needed
a
way
to
handle
that
the
yeah.
C
So
initially
we
we
settled
on
sort
of
a
redirect
method
by
implementing
external
Services
on
or
external
name
services
on
every
cluster
for
every
possible
destination,
so
that
we
could
redirect
geographically
to
the
nearest
cluster,
where
the
app
actually
lived.
If
somebody
requested
something
that
wasn't
on
their
cluster,
but
that
was
sort
of
a
stopgap
solution.
So
we
had
an
engineer,
start
pocing
istio
in
our
environments,
if
he's
in
the
audience,
Keith
Maddox
now
at
Microsoft,
and
so
he
sort
of
spearheaded
our
initial
foray
into
service
mesh,
which
inspired
a
nation.
C
So
yeah,
initially
just
Keith
was
working
on
it
and
we
we
sort
of
reached
an
implementation
in
staging
that
worked
pretty
well.
We
deployed
all
of
our
SEO
crds
via
Helm
chart.
It
were
pretty
much
invisible
to
the
developer.
We
did
expose
a
very
loose
abstraction,
Over,
The,
istio
Primitives,
but
for
the
most
part
developers
at
shipped
just
kind
of
want
things
to
work,
so
we
we
definitely
came
up
with
a
default
optimization
for
them.
C
So
after
that,
though,
we
decided
to
partner
with
solo
to
re-architect
the
whole
solution
using
glue
mesh
in
order
to
get
a
few
features.
We
we
really
wanted,
like
we
wanted
to
be
able
to
manage
everything
from
a
single
control
plane
across
multi-cluster,
as
well
as
Define,
our
our
Primitives
in
a
higher
level,
crd,
so
yep.
A
So
yeah
one
thing
we
wanted
to
kind
of
highlight
on
here
is
when
you,
when
you
take
on
implementing
a
service
mesh
or
even
when
you're
getting
started,
that
typically
falls
on
your
operations
team.
Even
though
service
mesh
is
for
developers,
it's
usually
an
added
responsibility
for
your
Ops
Team
and
operations.
A
Teams
are
typically
one
of
the
smallest
teams
in
your
company
and
they're,
really
highly
centralized
knowledge
base,
and
so
something
with
like
ship
is
they
had
somebody
like
Keith
go
out
and
learn
service
mesh,
but
over
time
they
actually
had
to
kind
of
share
that
knowledge
with
the
rest
of
the
team,
and
so
they
kind
of
realize
that
over
time
now,
Keith
says
that
Microsoft
that
the
rest
of
the
developers
and
other
operators
had
to
go
out
and
learn
service
mesh,
and
so
one
of
the
key
learning
points.
I.
A
Think
from
that
experience
was
that
shift
needed
to
kind
of
expand
that
knowledge
base
and
kind
of
share
the
expertise.
And
then
you
know
insulate
themselves
from
you
know,
having
one
person
have
all
the
keys
to
their
service
mesh
and
so
there's
a
lot
of
ways
that
you
can
actually
kind
of
learn
about
service
mesh
from
the
early
onset.
A
If
you're
thinking
about
using
service
mesh
I
would
encourage
you
to
get
a
team
involved
rather
than
just
a
single
person,
I
think
some
there's
some
really
great
books
out
there
that
teach
you
about
service
mesh.
For
example.
The
one
is
an
action
that
Christian
and
Lynn
wrote
is
pretty
much
a
full
stop
zero
to
a
hundred.
If
you
want
to
learn
service,
mesh
or
istio
through
and
through,
and
so
having
your
engineers
kind
of
ramp
up
early
is
going
to
pay
off
immensely
in
the
long
run,
because
you'll
be
spreading
out
that
expertise.
C
Yeah,
so
why
service
mesh?
Ultimately
initially
a
lot
of
our
multi-cluster
solution
depended
on
hairpinning.
So
we
didn't.
We
had
multiple
hops
where
we
didn't
need
them
to
be.
Even
the
the
DNS
redirection
we
were
using
required
hitting
an
Ingress
on
another
cluster
in
order
to
actually
reach
the
service
where
we
wanted
to
talk
service
a
to
service
B.
So
it
worked,
but
it
was
undesirable.
There
were
a
lot
more
optimizations
to
be
made.
C
We
had,
we
have
robust
failover
it
shipped.
It
does
require
Human
Action
right
now,
so
some
developers
like
that
but
I
know
a
large
percentage
of
them
really
would
prefer
things
to
just
kind
of
happen
automatically
so
Envoy
and
SEO
and
glue
give
us
sort
of
that
automatic
failover
capability.
We
needed
to
feel
competent.
C
The
developers
also
wanted
more
options
around
how
their
services
were
routed
to
so
closest
locality,
weighted
routing.
That
sort
of
thing
was
desirable
from
an
implementation
standpoint
for
us,
and
then
we
have
an
infosec
team.
They
love
zero
trust,
they
love
mtls.
They
love
policy
based
enforcement,
so
I
definitely
wanted
that.
That
was
a
plus
for
them.
A
Yeah
I
think
one
thing
that
everything
that
Brandon
listed:
that's
what
service
messages
do,
but
they
actually
had
one
extra
complexity,
is
that
their
applications
are
spread
out
through
a
number
of
clusters,
and
so
this
cncf
survey
kind
of
shows
here
that
the
number
of
people
running
just
one
cluster
in
production
is
decreasing
in
multiple
clusters
in
production
is
increasing
and
so
again
kind
of
service
mesh.
Largely
when
we
talk
about
it
is
just
single
cluster,
but
they
wanted
those
same
features
for
all
their
clusters.
C
Yeah
sure
yeah,
so
initially,
we
were
in
pretty
closely
coupled
to
clusters
to
handle
Ingress
for
this,
given
service
running
on
that
cluster,
each
service
had
sort
of
a
cluster
local
external
URL
that
was
available
internally
for
other
applications
if
they
were
not
on
the
same
cluster
as
that
service.
To
talk
to
so
we
needed
to
decouple
the
cluster
abstraction
from
the
equation.
When
deciding
how
to
route
to
an
application
service
from
outside
or
inside
the
mesh,
we
did.
C
We
wanted
developers
to
really
have
one
URL
to
use
to
you
know,
rule
them
all
inside
outside
whatever
not
care
about
cluster
completely
remove
that
from
the
equation,
and
we
wanted
also
a
single
management
plane,
like
I
said
before,
so
we
were
having
to
in
our
CI
CD
pipelines,
deploy
SEO
crds
to
each
namespace
and
each
cluster
where
things
were
deployed.
So
coordinating
that
you
know
it
worked,
but
it's
much
nicer
to
be
able
to
just
deploy
a
single
crd
to
manage
multiple,
complex,
istio
Primitives
to
a
single
management
plane
per
application.
C
A
Yeah
and
I
think
one
thing
here
is
that
it
wasn't
just
istio
to
make
this
work.
They
needed
to
combine
their
Cloud
DNS
as
well,
so
they
have
a
essentially
a
unique
DNS
name
for
every
service,
and
so
when
your
application
needs
to
reach
another
one,
it
resolves
to
that
DNS
name
and
then
routed
to
a
Gateway
and
I.
Don't
know
if
you
guys
use
https,
but
you
lose
some
once
you
leave
your
cluster
you're
losing
some
of
the
identity
of
your
application.
C
Yeah,
whenever
you
leave
the
cluster
to
go
to
another
cluster
at
the
moment
in
the
the
Legacy
architecture,
it
is
over
https.
So
you
do
have
that
additional
computation
for
for
decrypting
the
https
traffic
and
and
terminating
down
to
the
service
level
after
that,
so
mtls
directly
kind
of
helps
with
that,
and
this
is
the
the
nor
the
kind
of
new
mesh
architecture.
C
So
you
can
see
instead
of
having
to
go
Ingress,
we
go
directly
east
west,
wherever
service
is
using
our
singular
URL,
which
Nick
will
talk
a
little
bit
about
later.
The
glue
knows
how
to
intelligently
route.
If
it
knows
about
it
on
the
mesh
it
can
route
directly.
Even
if
the
URL
is
resolvable
to
an
Ingress
IP
in
the
environment,
pertaining
to
like
a
regular,
istio
Ingress,
it
won't
use
that
if
it
knows
it
can
reach
it
through
less
hops,.
A
Yeah
so
I
think
there's
a
kind
of
a
lot
to
unpack
there
in
terms
of
service
mesh
features,
I
think
one
of
the
big
things
that
most
of
us
start
with
when
you
start
using
service
mesh
is
an
Ingress
Gateway.
We
use
this
dioso
the
issue
Ingress
Gateway
and
it
has
like
some
really
great
features:
service,
Discovery,
observability
to
mtls
and
then
like
a
failover,
outlier
detection
and
that
works
really
good
in
a
single
cluster.
A
But
the
architecture
changes
quite
significantly
when
you
have
multiple
clusters,
I
think
that's
what,
and
so,
if
you,
if
you
decide
to
deploy
two
gateways
now,
you
just
have
two
single
service
meshes
deployed
to
or
two
clusters.
You
have
two
single
service
mesh
deployments
to
those
clusters
and
the
problems
get
a
lot
more
complicated
and
Chip
solve
those
by
using
Cloud
DNS
to
help
the
routing
from
there.
But
it's
not
because
they're
not
connected
to
each
other.
Again,
they
were
able
to
only
use
https
instead
of
mtls
and
kind
of
you
were.
A
So
when,
when
a
request
comes
into
that
Regional
Gateway,
it
knows
where
that
checkout
service
is
so,
it
can
apply
best
path,
routing
and
it
can
actually
use
mtls
from
the
Ingress
gateway
to
the
checkout
service.
And
so
you
can
actually
you're
now
getting
a
multi-cluster
awareness.
You
get
multicluster
observability,
and
then
you
get
full
mtls
from
cluster
to
Cluster.
A
One
great
thing
about
moving
the
Ingress
gateway
to
a
regional
thing
is
now
shipped
has
one
specific
spot
where
public
traffic
enters
their
environment,
and
so
they
can
concentrate
on
that
from
an
operational
side
to
put
all
of
your
security
at
that
one
spot.
And
so
we
have
one
single
Ingress
that
you
can
put
all
of
your
attention
into
to
make
it
more
reliable
and
better
and
put
more
security
at
that
layer.
So
they
use
a
number
of
tools,
oidc
and
jwts,
for
their
external
traffic.
C
Yeah,
so
this
is
more
a
slide
around
sort
of
how
we
did
the
initial
like
stop
Gap
solution
before
implementing
istio.
So
essentially
we
we
created
services
that
corresponded
to
every
potential
destination
that
a
service
could
live
on
every
single
cluster.
That
way,
if
a
service
asks
for
appa.app.service.cluster.local
that
it
would
end
up
getting
handled
by
a
service
with
an
external
name
that
would
redirect
to
the
Ingress
on
the
next
cluster
geographically,
where
the
app
actually
lived.
So
that's
that's
sort
of
how
it
worked.
It
was
it.
It
was
pretty.
C
C
You
know
a
naming
convention
that
pretty
much
is
like
an
a
record
wild
card
that
any
service
can
be
routed
to
Via
host
based
routing
and
that's
all
app
developers
have
to
know
about
in
terms
of
you
know,
knowing
how
to
reach
a
service,
whether
or
not
it's
injected
through
some
sort
of
service
Discovery
or
whether
they
need
to
hard
code.
It
for
some
reason
they
they
know
exactly
how
to
reach
another
service,
regardless
of
where
it
is
across
like
different
app
domains.
C
C
Originally,
we
started
looking
at
using
something
like
DNS
rewrite
with
core
DNS
to
solve
this
problem,
but
glue
does
it
intelligently
through
istio
and
yeah,
using
virtual
IPS
as
long
as
it
knows
about
something
in
the
mesh
you
know.
Instead,
if
it
didn't
necessarily,
it
would
use
the
the
istio
Ingress
yeah.
A
C
Yeah,
so
this
was
a
a
basic
example.
If
a
little
convoluted
of
how
it
worked,
which
I
mean
it
was
convoluted
because
it
naturally
was
so
like
we,
we
basically
in
the
helm,
chart
defined
a
list
of
locations
and
based
on.
If
compute
existed
in
the
region,
we
would
tell
it
how
to
route
based
on
that
Geographic
list
of
locations,
and
so
it
kind
of
looks
something
like
this.
Where
we
we
do.
C
A
You
want
to
go
back.
One
side
actually
I
think
there's
something
really
interesting.
What
they
had
to
do
with
their
Cloud
DNS
is
all
those
load
balancers
that
exist.
They
essentially
have
those
Ingress
gateways
listening
on
all
the
hosts
and
then
if
they
would
receive
a
requests
for
an
application
that
didn't
exist
in
that
application,
they
would
forward
it
to
the
correct
cluster
and
so
using
a
lot
of
like
DNS,
magic
and
then
routing
in
the
Ingress
Gateway.
They
essentially
gave
all
of
their
applications.
A
A
The
second
thing,
you're,
probably
wondering
how
we
were
able
to
Short
Circuit
those
Hops
and
that's
because
of
the
intelligent
DNS,
that's
added
in
istio's
proxy,
so
there's
a
DNS
server
that
runs
in
there.
That
actually
will
take
that
it
will
resolve
host
names
for
you
before
actually
forwarding
on
to
like
the
cloud
DNS.
A
So
we're
hijacking
that
DNS
we're
looking
for
those
host
names
that
they
did
they
made
for
those
services,
and
then
we
can
actually
return
back
the
best
path
for
your
application
to
call
to
get
your
response,
and
so
what
I
put
up
here
is
kind
of
the
flow
pattern
of
the
istio
proxy
and
how
it
determines
where
it
should
go.
So
at
the
top
there,
your
food
application
is
calling
bar.ship.com,
and
so
it's
asking
the
istio
proxy.
A
Where
is
bar.ship.com
and
so
istio
hijacks
that
request
that
DNS
request
and
returns
a
virtual
IP,
and
so
it's
a
just.
An
internal
IP
address
that's
used
by
istio
and
only
known
by
by
the
proxy,
and
so
then
the
foo
application
is
going
to
make
that
request,
and
that
request
then
goes
back
to
the
proxy
and
the
proxy
sees
that
virtual
IP
address
and
then
decides.
Where
is
the
best
path
to
route?
A
And
so
in
this
application,
or
in
this
example,
the
proxy
determined
that
the
bar
application
in
U.S
east
was
the
best
path,
but
it
returned
to
500
error,
and
so
the
proxy
can
automatically
then
retry
and
go.
Oh
there's
a
next
best
path
to
go:
let's
try
the
US
West
one,
and
so
then
it
can
automatically
retry
that
HTTP
request
and
that
one
returned
to
200.
and
so
to
the
end
application.
B
A
A
C
Ahead
yeah,
so
here
I'll
talk
a
little
bit
about
our
getups
pipeline
of
how
we
actually
handle
all
this
from
code
to
to
kubernetes.
So,
like
I
said
earlier
in
the
talk
developers
essentially
Define
what
we
call
an
infraspec
yaml
file
in
their
repo,
which
is
a
pretty
similar
pattern
to
anything
else.
You've
probably
seen
where
we
Define
loose
abstractions
over
kubernetes
and
istio
Primitives,
also
handle
like
clouds,
cloud
storage
and
S3
and
whatnot
that
kind
of
thing
so
develop.
C
It's
like
a
One-Stop
shop
for
developers
to
Define
their
application
environments
and
what
compute
and
data
stores
they
want.
So
there
is
a
service
mesh
abstraction
there
too,
so
that
they
basically
can
Define
all
of
their
weighted
routing.
They
can
opt
in
so
right
now
we
primarily
have
an
opt-in
approach.
C
So
developers
say
they
want
the
sidecar.
They
want
the
mesh
to
be
enabled
at
some
point
in
the
future.
They
they
won't
need
to
do
any
of
that
because
of
ambient
mesh,
but
looking
forward
to
that
day
in
in
the
terms
of
how
the
actual
file
gets
parsed,
it
we've
listened
to
web
hooks
with
a
few
different
services
from
get
pushes,
creates
issue
comments,
that
kind
of
thing
and
parse
that
infraspec
into
a
template
at
Helm
chart
and
then
deploy
those
in
parallel
to
all
of
our
clusters.
C
With
the
addition
of
glue,
we
deploy
an
additional
glue,
mesh
Helm
chart
to
our
management
clusters
that
are
the
control
planes
for
the
multi-cluster
mesh
and
that
sort
of
defines
the
glue
crds
that
are
needed
and
developers
pretty
much
just
use
that
pattern.
They
define
specific
environments
and
regions
they
want
to
deploy
in
and
we
match
whatever
their
strategy
is,
and
it
works
pretty
well.
C
C
A
I
think,
like
one
great
thing,
is
the
architecture
for
ship
didn't
change
and
those
original
objectives
that
they
had
around
multi-cluster
routing
and
Global
host
names
for
their
services.
We
were
able
to
retain
all
of
those
features
that
they
had,
but
we
just
improved
upon
them.
We
made
them
more
secure.
We
made
the
routing
more
intelligent
and
we
allowed
them
to
keep
doing
this.
Their
development
with
all
of
those
same
features
using
service
mesh.
We
just
enhanced
it
and
then
we
with
service
mesh.
A
B
Yes,
I
really
really
enjoy
this
talk
and
congratulations
Brendan
to
your
team
that
you
were
able
to
hide
the
service
mesh
complexity
for
your
application
developers.
That's
really
cool!
So
any
questions
from
the
audience.
All
right,
I'll
pass
the
market
right!
I'll
kill
you
first,
since
you
are
closer
but
I'll
get
to
you.
D
A
A
few
big
things
is
a
service
identity
and
multi-cluster
mtls.
You
know,
I
have
a
lot
more
kind
of
metadata,
that's
traveling
with
the
request,
as
it
leaves
one
cluster
to
another
and
it's
a
lousier
applications
or
your
servers
to
make
more
Intelligent
Decisions
at
in
terms
of
security.
You
know,
should
they
allow
or
deny,
and
essentially
the
with
multi-cluster
mtls.
You
can
now
lock
down
the
entire
environment
and
only
trust,
known
applications,
and
then
secondly,
at
that
Gateway
level,
because
that
Gateway
is
integrated
with
service
mesh.
A
We
were
able
to
focus
our
security
specifically
at
that
layer
and
so
adding
capabilities
like
web
application,
firewall
and
oidc
and
JWT,
and
all
this
stuff
at
that
single
point.
It
kind
of
like
acts
as
like,
a
DMZ
for
outside
traffic
and
entering
ships,
environment
and
then,
in
terms
of
the
routing
decisions
we
kind
of
talked
about
the
decisions
that
the
SEO
proxy
makes
in
terms
of
because
there's
Global
observability,
it
can
look
at
kind
of
your
environment
applications
your
environment
wide,
and
so
we
know,
if
there's
an
application.
A
E
A
great
presentation,
so
you
talked
about
multi-cluster
deployment
I
just
wanted
to
understand
in
terms
of
istio,
providing
this
out
of
the
box
versus
the
implementation.
That
solo
has
done
like
what
are
the
differences
or
additional
features
that
have
been
brought
in
if
any,
and
secondly,
you
also
talk
about
automatic
failover.
Was
it
just
related
to
like
Parts
going
down
and
the
service
taking
care
of
ensuring
that
those
spots
don't
have
traffic
going
in
or
was
there
something
more
than
that.
A
Yeah,
so
your
first
point,
so
what
we
did
we
built
everything
on
top
of
Open
Source
istio.
Essentially,
what
we
were
able
to
do
is
Stitch
all
of
those
individual
istio
deployments
together
and
make
them
aware
of
what
they
needed
to
know
about
in
the
other
clusters,
and
so
all
of
the
kind
of
routing
and
all
that
decisions
is
just
open
source
istio.
We
just
configure
it
at
a
extremely
fine
grain
level.