►
From YouTube: Turning Your Cloud Native Apps Inside Out With a Service Mesh - Adam Zwickey & Liam White, Tetrate
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Turning Your Cloud Native Apps Inside Out With a Service Mesh - Adam Zwickey & Liam White, Tetrate
Cloud native best practices have come a long way from the mid-2010’s when adopters of cloud native patterns were utilizing in-app libraries to manage concerns around load balancing, circuit breaking, service discovery, etc. Many adopters of modern cloud platforms have now realized greater agility and ease of use when those primitives are separated away from the application and managed independently in a cloud platform. This talk will review the pitfalls of those earlier application patterns with an assessment of why we've moved on and, most importantly, the benefits of now operating with an Envoy-based Service Mesh to help you manage your environment and enhance your development practices. It will also address how using a service mesh will help you migrate to this newer development style faster and safer to better enhance your organization's delivery times for updates to products and services.
A
Okay,
good
afternoon,
everyone
great
to
meet
you.
My
name
is
adam.
I
have
with
me
liam
we're
from
touchrate
and
we're
super
excited
to
be
able
to
talk
on
this
topic
of
how
we
can
turn
our
cloud
native
applications
inside
out
using
a
service
mesh.
So
as
by
way
of
introduction.
I'm
part
of
our
solution-
engineering
organization
at
tetrad
really
a
long
time,
proponent
and
contributor
to
open
source
very
active
in
the
cloud
foundry
spring
and
kubernetes
ecosystems,
and
I
have
like
I
said
liam
with
me
I'll.
Let
him
introduce
himself.
B
A
Great,
so
what
we're
going
to
cover
today
is
first
a
little
bit
of
a
history
or
what
we
kind
of
view
as
the
common
patterns
and
building
blocks
that
are
used
for
building
cloud
native
applications,
specifically
using
the
spring
ecosystem
and
spring
cloud,
and
some
of
the
iep
that
netflix
contributed
to
the
open
source.
Then
we're
going
to
cover
how
does
service
mesh
fit
in
with
this
type
of
architecture
and
this
type
of
approach
with
java
applications.
A
Then.
Lastly,
we'll
walk
through
a
little
bit
of
a
migration
example
of
how
we
go
from
potentially
using
some
of
these
netflix
libraries
to
introducing
service
mesh
into
the
app
now
we're
going
to
go
through
a
couple
of
code.
Examples,
architecture,
examples
pretty
quickly,
but
you'll
notice.
There
is
a
link
to
a
github
repo
that
actually
has
the
before
and
after
code
that
you
all
can
take
a
look
at
and
maybe
even
try
and
run
and
use
as
an
example
as
we
start
out.
I
definitely
want
to
preface
this
whole
presentation
with
touch
rate
myself.
A
So
how
do
we
get
to
this
point?
Where
looking
at
the
end
of
the
timeline
where
we
are
in
2020
or
now,
2021,
in
what
I
see
as
most
organizations
looking
at?
How
do
I
build
microservice
or
cloud
native
applications
and
then
spread
them
across
clusters
and
across
multiple
clouds?
Well,
we
rewrite
we've
arrived
at
this
point
because
there's
been
a
number
of
very
important
and
interesting
contributions
to
the
open
source
and
technologies
that
make
building
cloud-native
applications
and
microservices
a
little
bit
easier.
A
It
goes
all
the
way
back
to
2012
when
netflix
realized.
They
need
to
build
smaller
services,
iterate
over
them
quickly,
optimize
for
velocity
and
then
begin
to
be
very
vocal
about
the
patterns
and
technologies
and
the
ways
they
did
this
and
even
open
sourced
many
of
these
libraries
supporting
this.
There
is
the
emergence
of
docker
and
containers,
containers,
scheduling,
platforms
like
kubernetes
and
cloud
foundry.
Then
we
really
saw
the
standard
or
the
buzzword
of
microservices
being
the
way
to
go
for
building
applications.
A
In
the
way
spring
developers
have
known
to
grow
and
love
with
convention
over
configuration,
an
opinionated
out
of
the
box
configuration
that
gets,
you
started
really
quickly
and
then,
a
few
years
later,
istio
was
open,
sourced
and
released
to
1.0
and
we'll
talk
about
why
that's
important
when
it
comes
to
cloud
native
applications
now
the
ingredients
or
why
do
we
actually
do?
This
is
the
fundamental
thing
when
we
move
to
microservices
or
cloud
native
or
just
smaller
units
of
work.
A
Is
we
end
up
in
the
situation
where
everything
we
depend
on
or
anyone
that
depends
on
us
begins
to
communicate
with
us
over
the
network,
and
this
is
especially
problematic
if
either
the
network-
or
we
could
say
more
broadly,
the
compute
the
cloud.
The
infrastructure
that
we're
running
on
becomes
more
dynamic,
more
maybe
somewhat
less
resilient
in
the
terms
of
one
individual
unit
of
commute
compute,
and
so
we
don't
want
to
lose
any
resilience.
A
If
you
take
the
example
of
an
application,
let's
say
depends
on
30
different
microservices
to
build
a
complete
experience
for
end
user.
If
each
of
those
services
has
an
uptime
of
four
nines,
and
then
we
multiply
that,
though,
across
30
services,
the
overall
experience
is
only
slightly
above
two
nines
of
availability.
A
So
that's
why
I
love
this
description
of
one
of
my
friends
and
former
co-workers
duncan
nguyen,
who
he
describes
cloud
native
as
this
way
we
build
software,
that's
designed
to
run
and
scale
reliably
and
predictably
on
on
top
of
unreliable
cloud-based
infrastructure,
or
we
could
also
say,
dynamic,
cloud-based
infrastructure,
so
that
leads
us
to
begin
to
build
into
our
application
and
since
we're
talking
about
spring
applications
and
java
applications
built
right
into
our
jvm
patterns
around.
How
do
I
discover
the
services
I
need?
How
do
I
load
balance
across
them?
A
How
do
I
do
that
in
a
resilient
way,
and
how
do
I
fail
fast
when
there's
problems
that
are
taking
place,
then
how
do
I
get
visibility
into
what
is
taking
place
with
metrics
and
telemetry
and
then
trace
calls
to
service
to
service
to
service,
and
that's
really
what
is
needed
to
begin
to
build
cloud-native
applications
that
are
going
to
be
spread
across
multiple
clusters:
multiple
clouds,
multiple
kubernetes
environments,
and
so
how
would
we
actually
do
this
in
practice?
Well,
here's
a
little
super
simple
snippet
of
code.
A
I
cannot
have
a
cascading
failure
with
my
application,
so
if
we
lay
that
across
how
we
actually
deploy
it
across,
maybe
a
kubernetes
cluster
I'll
point
out
a
couple
common
ingredients
that
we
might
have
in
our
architecture.
Here.
First,
we
probably
get
traffic
into
our
applications
using
a
standard,
kubernetes
ingress
controller
and
that
typically
will
land
at
a
spring
cloud
gateway,
which
then
knows
how
to
talk
with
eureka.
Our
service
registry
find
out
where
our
web
ui
is
where
our
api
service
is
also.
A
Our
api
service
is
going
to
talk
with
that
registry
to
both
register
where
it's
running
and
then
how
it
finds
its
dependencies
and
then.
Lastly,
since
almost
every
application
I
see
in
the
wild
today
is
running
across
multiple
clusters.
This
eureka
instance
is
probably
going
to
be
appearing
with
other
eurekas
in
other
clusters
to
provide
service
discovery
for
either
replicas
of
that
same
application
or
its
dependent
services.
A
So
this
is
a
pretty
standard
architecture,
but
there's
a
couple
of
gaps
or
challenges
that
arise
with
this
first.
This
is
really
optimized
for
the
jvm
or
for
java
applications,
because
these
libraries
to
solve
these
problems
are
built
right
into
the
application.
So
the
polyglot
experience
is
really
less
than
ideal
if
you're
writing
or
want
to
build
a
service
in
something
other
than
java.
A
A
And
then
these
these
patterns
don't
have
the
most
robust
semantics
and
primitives
for
multi-cluster
definitions,
multi-cloud
definitions,
it
becomes
kind
of
hard
to
say
if
you're
running
right.
Next
to
me,
here's
how
you
can
discover
me,
but
if
you're
not
running
in
my
cluster,
you
should
discover
me
a
different
way.
B
All
right
so
transparent
network
layer.
So
if
you
go
and
look
at
the
envoy,
docs
you'll
see
one
of
the
core
tenants
of
the
project
is
that
the
network
should
be
transparent
applications
and
when
network
and
application
problems
do
occur,
it
should
be
easy
to
determine
the
source
of
the
problem.
There
are
two
parts
to
this.
Obviously
there
is
the
transparency.
So
what
this
means
is
the
application
shouldn't
be
aware
or
even
care
that
it's
part
of
a
service
mesh
right
and,
by
extension,
this
includes
developers.
B
If
I'm
a
developer,
my
job
is
to
produce
business
value.
I
shouldn't
have
to
care
about
mutual
tls.
It
should
just
be
done
for
me.
I
might
care
about
retries
or
timeouts
right,
but
beyond
that,
because
that's
maybe
application
specific
beyond
that.
I
don't
care.
I
just
want
to
solve
business
problems,
and
the
second
part
is
about
visibility
and
because
envoy
is
a
tcp
and
udp
proxy
with
cons
with
an
understanding
of
how
to
deal
with
http,
2
and
http
3
for
quick.
B
So
how
does
it
solve
this
problem?
We
can
see
there
on
the
left.
We've
got
the
netflix
oss
model,
where
you
know
we
bundle
in
discovery,
load,
balancing
traffic
management,
resiliency
metrics
tracing
into
the
service
itself.
As
a
library
when
we
are
using
kubernetes
and
envoy,
we
extract
that
info.
We
extract
that
functionality
out
into
envoy
itself.
B
So
what
does
the
request?
Look
like
well
request
comes
into
a
pod.
It
gets
intercepted
by
envoy
gets
forwarded
on
to
the
service.
The
service
when
it
makes
outward
requests,
goes
in
the
reverse
direction.
So,
like
I
mentioned
in
the
previous
section,
it's
separate
container
same
pod.
They
share
a
network
name
space
and
kubernetes,
so
we
can
do
all
kind
of
messing
around
with
ib
tables
and
that's
how
we
do
it
transparently
in
instagram,
excellent.
B
So
envoy
refers
to
this
as
an
out
of
process
architecture,
and
what
this
really
means
is
moving
the
logic
from
the
libraries
within
the
codes
out
to
a
separate
binary
that
can
run
as
a
separate
process
separate
container.
Technically
those
two
things
are
the
same
thing
in
kubernetes
and
in
docker,
but
within
the
same
pod
and
doing
this
has
quite
a
few
benefits,
so
moving
this
out
means
that
it
works
with
any
language
because
it
works.
Like
I
mentioned
at
the
network
level,
it
speaks
tcp
or
udp.
B
You
might
have
some
crazy
protocol,
but
you
can
extend
envoy
to
actually
speak
that
protocol.
If
you
can
write
something
that
compiles
down
to
waser,
if
you're
a
purely
java
shop,
then
this
might
not
be
much
of
an
advantage.
However,
the
next
one
is,
which
is
it
allows
it
to
work
with
any
legacy
in
third
party
applications.
B
So
you
have
postgres,
you
have
redis.
You
have
some
legacy
system
running
on
a
mainframe
that
you're,
never
that
you
don't
want
to
touch.
We
just
deploy
either
envoy
alongside
it
or,
if
you're,
deploying
on
a
mainframe.
Maybe
you
deploy
it
as
an
egress
gateway
on
the
way
out
to
the
mainframe
right,
and
then
we
see,
then
we
have
that
visibility
and
control
of
the
traffic
in
the
same
way
that
you
would
a
modern
cloud
native
application.
B
B
Sometimes
it
doesn't,
but
if
we
decouple
decouple
this
functionality
so
that
the
platform
team
can
just
roll
out
security
fixes
and
roll
out
new
versions
of
envoy,
for
instance,
application
devs
need
be
number
wiser
right
and
again
they
can
just
focus
on
actually
writing
the
core
business
logic.
The
other
advantage
of
this
separation
is,
if
we
aren't,
if
we're,
if
we're
using
an
envoy-based
service
mesh,
nearly
pretty
much
all
of
them
are
api.
Driven
much
like
kubernetes
is
so
in
a
similar
way
to.
B
If
you
needed
to
scale
out
capacity,
you
needed
to
increase
the
number
of
replicas
for
a
specific
deployment,
because
you're
pinning
your
cpu,
you
can
just
scale
it
up
via
an
api
change.
You
don't
have
to
kind
of
like
change
anything
else.
You
just
change
that
one
thing
and
we
can
do
that
using
a
number
based
service
mesh
as
well
right,
like
maybe
you're
something
slow
and
it's
taking
11
seconds
to
respond,
and
you
have
a
10
second
retry
right.
B
Now
there
is
obviously
one
disadvantage,
which
is
relatively
obvious
from
an
out
process.
Architecture
and
that's
latency
right
so
moving
out
process
means
that
when
you're
making
a
request
you're
now
going
through
the
network
stack
within
the
kernel
within
the
kernel
and
user
space
you're
going
in
and
out
of
it
an
extra
one,
two
three
four
times
right
and
this
like
it
legitimately-
does
increase,
except
for
some
circumstances.
B
B
So
this
is
an
issue.
If
your
workloads
have
strict
latency
requirements,
then
the
grpc
team
is
currently
working
on
implementing
the
envoy
apis.
So
on
latency
specific
paths
you
can
choose
to
use
grpc,
you
won't
have
to
do
the
kernel
user
space
network
stack,
jumps,
you
just
kind
of
do
the
thing
I
think
as
well.
Cilia
maybe
have
some
stuff
that
does
a
it
basically
prevents
you
from
needing
to
go
through
from
user
space
into
the
kernel
and
back
out
again
when
you're
doing
this
network
hop.
B
B
So
moving
this
out
process
gives
us
consistency.
This
kind
of
breaks
down
into
four
things:
first,
one
being
traffic
management,
so
it
doesn't
matter
if
it's
a
third-party
application,
postgres
redis
legacy
application,
homegrown,
non-java
application
right,
you
get
this.
You
use
the
same
api
to
configure
all
of
the
traffic
management.
It
doesn't
matter
kind
of
it
doesn't
matter
about
any
of
those
things.
B
So
this
functionality
includes,
like
retries
circuit,
breaking,
request,
shadowing
session
stickiness
locality,
load,
balancing
so
keeping
keeping
your
aws
or
other
cloud
provider
network
costs
down
by
keeping
your
traffic
all
within
the
same
az,
where
possible,
canarying,
ap,
test
and
fault
injection,
whatever
whatever
functionality
that
android
you
need
to
leverage,
it's
the
same
api
for
configuring.
All
of
those
things.
B
It's
also
the
same
api
for
configuring,
security
and
policy
right
so
third
party,
whatever
doesn't
matter
same
api
if
you're
using
istio,
we
leverage
envoys
sds,
so
secret
discovery
service
and
we
allow
you
to
incrementally
adopt
mtls
everywhere.
We
handle
certificate
rotation
for
you.
We
assure
identities
based
on
hours
or
days,
not
weeks
or
months,
and
we
do
this
because
certificate
revocation
is
extremely
painful,
so
we
just
have
short-lived
certificates
and
yeah.
It's
we
have
this
consistent
way
of
writing
policy
for
all
of
all
of
your
applications.
B
The
next
one
is
behavior,
so
the
if
you
have
a
bunch
of
implementations
of
different
client
libraries,
so
you
might
have
a
java
library
that
does
retries
a
go
library
that
does
retries.
Maybe
I
don't
know
postgre
postgres
has
its
own
thing,
but
maybe
you
have
a
legacy.
Application
doesn't
really
do
time,
notes
or
something
retries
right,
whatever
the
behavior
is
going
to
be
different
across
all
of
the
implementations,
or
it
might
not
be
there
at
all
right
and
because
we
have
the
same
binary
everywhere,
save
the
grpc
stuff
we
mentioned
earlier.
B
We
get
consistent
but,
more
importantly,
predictable,
behavior
and
predictable.
Behavior
is
always
better
not
only
that
we
can
move
like
you
know,
and
like
I
mentioned,
we
can
put
the
we
can
put
envoy
at
the
ingress
gateway.
So
we
have
that
consistency
there
and
the
same
goes
for
telemetry
right,
because
envoy
produces
the
same
metrics
across
the
same
protocol.
B
Regardless
of
you
know
third-party
your
own
application
right,
we're
going
to
have
the
same
metrics
we're
going
to
have
the
same
metrics
names
we
aren't
going
to
have
to
coordinate
metric
names
across
teams.
They
just
get
given
metrics
names,
so
we
can
just
generate
dashboards
like
by
usually
by
code
by
templating
right
like
because
the
names
are
always
the
same,
and
the
attributes
are
always
the
same.
We
just
get
kind
of.
We
can
just
auto
generate
all
of
that
stuff.
A
We
now
have
an
envoy
instance:
that's
paired
up
with
every
single
application
container,
that's
running
in
kubernetes.
Secondly,
our
our
kubernetes
ingress
has
changed.
We're
able
to
actually
use
envoy
itself
as
an
ingress
gateway
and
have
some
of
these
capabilities
that
we're
talking
about
at
the
furthest,
edge
ingress
of
our
cluster
and
then.
Lastly,
the
thing
you'll
notice
is:
our
service
registry
actually
has
disappeared
so
inherently
within
whatever
control
plane,
we're
using
to
program
istio.
A
So,
let's
talk
about
this
first
item:
server,
ingress
and
service
discovery,
so
in
our
spring
boot
application
there's
a
couple
changes
that
we
simply
can
remove
and
drop
stuff
out
of
our
application.
Optionally,
we
could
remove
the
gateway.
Well,
that's
not
a
hard
requirement.
There
might
be
certain
use
cases
where
you
leave
your
spring
cloud
gateway
in
the
mech
in
the
mix,
but
optionally
that
could
be
removed.
A
Now,
I'm
going
to
show
you
a
couple
code
snippets
that
are
a
little
bit
more,
how
you
would
program
this?
Imagine
imagining
that
issue
is
your
control
plane,
but
ultimately
this
gets
materialized
into
envoy
configuration
to
actually
program
the
data
path
and
so
any
any
control
plane.
That's
programming
envoy
typically
would
behave
about
the
same,
even
though
the
semantics
might
be
slightly
different.
A
So
if
we're
utilizing
envoy
as
our
ingress
gateway,
we
simply
need
to
tell
our
our
ingress
and
then
program
our
envoy
instances
where
it
can
route
traffic
to
so
it's
a
very
simple
definition
of
my
traffic's
coming
in
to
my
redis
prefix
at
my
endpoint,
send
that
to
my
redistended
destination.
In
my
cluster,
my
code,
I
mentioned
on
the
left,
there's
a
bunch
of
stuff:
that's
extracted
out
of
your
java
code.
A
So
if
we
haven't
already
done
so,
we
can
strip
out
of
our
java
code
out
of
our
application,
those
load,
balance
or
discovery
client
annotations,
potentially,
if
we've
done
something
more
sophisticated,
like
provide
a
custom
implementation
for
the
ribbon
load,
balancers
or
a
more
sophisticated
algorithm
or
configuration
for
that.
That
also
can
drop
out
off,
and
now
we
can
program
our
envoys
of
how
we
want
to
shape
traffic.
So
I
have
three
very
simple
examples
here
of
how
we
can
split
traffic
across
multiple
instances.
A
In
this
case
it's
sending
95
percent
of
our
traffic
to
our
original
version
of
our
cash
service.
Five
percent
will
go
to
our
new
version
using
application
labels
under
the
covers
and
then
we're
giving
it
a
little
bit
of
information
when
things
might
misbehave
at
half
a
second
time
out
retry
three
times
and
what
are
the
type
of
scenarios
we
should
retry
on?
A
We
also
can
tell
envoy
what
is
going
to
be
the
actual
load
balancing
strategy,
the
traffic
policy
that's
going
to
be
applied
when
we
establish
these
connections
and
then,
lastly,
you
know
one
really
interesting
thing
we
can
do
since
we're
controlling
all
traffic.
We
can
actually
control
how
we
connect
to
things
outside
of
our
service
mesh
or
even
outside
of
our
kurendi's
clusters,
and
have
that
do
things
like
tls
mutual
authentication.
A
Now
we
also
want
to
make
sure
our
applications
are
resilient.
Things
are
ejected
from
our
client-side
load
balancers
as
as
needed,
and
so
in
our
application.
We
can
drop
out
any
histrix
or
spring
cloud
circuit,
breaker
dependencies
that
are
in
our
greater
build
or
maven
palm
files,
remove
all
the
annotations
that
are
related
to
that
and
then,
if
we
have
implemented
any
circuit,
breaker
factories
or
potentially
we're
using
more
modern
resilience
for
j,
we
can
remove
the
config
the
properties,
the
annotations
that
we
placed
on
our
methods.
A
A
Well,
we're
able
to
configure
it
so
that
it
can
only
do
ten
connections
at
once.
It's
going
to
time
out
after
five
seconds
and
what
are
the
ways
for
how
should
we
eject
endpoints
when
they
don't
behave
correctly?
So
in
this
configuration
we're
saying
in
any
10
second
window,
if
we
have
10
5
0x
errors,
we
should
eject
that
endpoint
for
a
minute
and
obviously
there's
more
robust
and
verbose
configurations
that
we
can
made
make
for
this,
but
very
easy
to
transparently
interject
this
into
power
traffic's
flowing
through
the
system.
A
Now,
as
a
little
bit
of
extra
credit
I'll
talk
about
some
security
items,
because
that's
one
inherent
capability
that
this
transparent
networking
layer
the
service
mesh
gives
to
us.
So
within
our
application
we
can
strip
out
any
of
the
complex
trustor
key
store
stuff
that
we
may
have
had
to
do
in
our
jvm.
A
We
can
connect
to
services
just
simply
over
tls
and
rely
on
the
service
mesh
to
and
and
our
our
on
voice
side
cars
to
handle
all
the
security
for
inbound
and
outbound.
So
in
this
example,
let's
say
we
only
want
our
caching
service
to
be
talked
to
by
the
api
service
things
closer
to
the
edge
like
our
web
ui
or
our
ingress
shouldn't
directly
connect
to
that.
A
So
we're
very
easily
able
to
author
policy
that
says
for
all
traffic
mp
mtls
is
going
to
be
strictly
reinforced
and
we're
also
going
to
inspect
that
certificate,
that's
presented
by
the
client
and
we're
going
to
only
allow
principles
identified
by
the
to-do's
api
certificate
to
connect,
and
so
that's
the
certificate
in
x509
certificate.
That's
issued
to
every
single
envoy,
a
sidecar
to
our
application,
that's
going
to
have
a
unique
identity,
so
we
can
author
both
off
and
on
z
policy.
A
A
So
this
varies
very
very
widely
of
how
we
would
implement
this
in
our
application
using
spring
security.
So
I'm
not
going
to
cover
what
we
might
be
able
to
remove
from
our
application.
That
could
probably
be
a
30
minute
talk
on
itself,
but
within
the
actual
service
mesh
we
can
author
policy
that
is
going
to
validate
tokens,
and
so,
in
this
case
it's
going
to
be
talking
to
key
cloak
inspecting
the
jot
tokens
that
are
going
to
be
presented
by
the
request.
A
Who
is
invoking
our
service
and
propagate
that
throughout
our
service
mesh
and
enforce
that
using
envoy
as
a
policy
enforcement
point
so
to
wrap
this
up
or
to
kind
of
summarize.
What
is
this
new
architecture
that
we
have
with
service
mesh
and
spring
spring
can
still
bring
to
the
table
the
very
easy
way
to
get
started
and
build
microservice
applications
with
spring
boot
and
all
the
libraries
that
make
consuming
data
services
messaging
services,
reactive
patterns
building,
uis.
All
the
things
that
spring
does
are
great
can
be
built
into
application.