►
From YouTube: Using Istio to build the next generation 5G platform
Description
#IstioCon2021
Presented at IstioCon 2021 by Neeraj Poddar & Dave Lenrow.
Building the next fastest, secure and reliable 5G platform is challenging in its own right but doing that while modernizing your infrastructure and onboarding Cloud-Native Functions (CNFs) from multiple vendors can be a herculean effort
In this talk, we will cover how Istio can be used in 5G platforms to achieve uniform security and visibility across these CNFs deployed in multiple clusters across different sites including edge. However, rolling out Istio at this scale brings its own challenges around lifecycle management, tenant isolation, identity management and visibility beyond metrics and traces. We will cover the lessons learned and tradeoffs between various architectures, in particular:
Tenancy isolation
5G aware telemetry
Uniform identity between legacy and workloads in the mesh
Deep packet inspection
Attendees will gain insight into how service providers use Istio at a large scale and what the future holds for a mesh everywhere world from backend to edge to devices
A
A
A
It's
all
about
software
architecture
and
packet
based
things
and
absolutely
fascinating
for
people
that
are
interested
in
that
technology,
and
so
5g
is
faster
and
there's
certainly
an
element
of
that.
But
a
lot
of
the
big
exciting
things
about
it,
don't
exist
in
the
radio
layer,
and
you
can
see
here
on
the
left-hand
side
of
this
diagram.
A
There
are
really
kind
of
four
major
sections
of
the
topology
that
are
interesting
and
the
ran
describes
the
radio
radio
area
network,
which
is
out
where
the
cell
towers
are
and
away
from
the
traditional
data
centers
and
so
a
lot
of
what's
new
and
interesting
in
5g.
Is
that
we're
putting
digital
programmable
digital
computer
technology
out
at
the
radio
towers
and
in
the
radio
network,
and
that
changes
things
entirely
edge?
A
Computing
is
suddenly
a
part
of
the
telco
network
where
it
was
not
previously
and
one
of
the
the
big
new
technology
components
that's
introduced
with
5g
is
called
network
slicing
and
includes
the
idea
of
having
sort
of
a
virtual
mobile
network
available
to
a
closed
group
of
people
that
has
specific
service
level
agreements
for
security,
things
and
performance
things
and
latency
and
things
of
that
nature,
and
so,
rather
than
kind
of
one
giant
monolithic,
best
effort.
Looking
network,
you
end
up
with
several.
You
know
small
overlay
networks
that
each
have
their
own
properties.
A
You
know,
number
of
endpoints
is
going
to
grow
by
an
order
of
magnitude
and
bandwidth
is
growing
and
all
those
other
things.
So
software
architecture
is
starting
to
matter
a
lot
more
as
the
network
evolves.
It's
much
less
about
dedicated
telephony,
specific
appliances
and
much
more
about
general
purpose.
Compute
we're
going
from
hardware
defined
to
software
defined
in
a
lot
of
cases
and
in
addition,
iot
and
kind
of
the
next,
several
billion
devices
that
are
coming
onto
the
network
have
support
for
their
unique
needs.
So
there's
there's
a
lot
in
5g.
A
A
So
this
is
kind
of
a
big
high-level
picture
of
the
5g
architecture
and
ignore
the
fact
that
it
consists
of
lots
of
strange
three
and
four-letter
acronyms.
That
may
or
may
not
mean
anything
to
you
but
notice
that
it's
very
well
defined
architecturally
in
the
sense
that
there
are
clean
separation
between
the
control
plane
and
the
user
plane
or
the
data
plane.
A
Networking
implementation
and
details
get
moved
out
of
these
specific
tenant
workloads
that
implement
the
network
functions
and
get
moved
into
the
service
mesh
where
they
can
be
applied
consistently
and
be
considered
part
of
the
platform
rather
than
than
part
of
the
application.
So
each
one
of
these
dark
boxes
represents
a
particular
component
in
the
5d
telephony
architecture.
A
The
implementation
that
lives
inside
of
each
of
those
boxes
and
again
the
kind
of
the
really
interesting
piece
of
this
is
out
in
the
radio
area
network,
where,
unlike
a
traditional
data
center,
we're
very
constrained
in
terms
of
real
estate
footprint
power,
cooling.
All
of
those
things
we're
not
able
to
kind
of
throw
unlimited
cpu
at
scaling
out
problems
and
solving
them,
and
so
for
that
reason,
at
least
initially
in
5g.
A
A
Workloads
need
to
be
mobile,
a
single
cnf
might
run
on
a
single
worker,
node
or
it
might
be
spread
across
a
cluster
or
it
might
be
spread
across
multiple
clusters
and
all
of
those
scenarios
need
to
be
managed
smoothly
again.
Getting
that
the
networking
outside
of
the
containerized
network
function
is
really
important.
A
People
who
are
familiar
with
microservices
and
service
mesh
will
see
kind
of
a
similarity
that
the
picture
on
the
right
is
not
actually
a
network
function.
It's
it's
kind
of
a
generic
description
of
microservices
and
their
interconnection
and
their
relationship.
But
if
you
kind
of
lean
back
and
squint
a
little
bit,
you
notice
that
it
looks
a
lot
topologically.
A
The
service
mesh
becomes
the
tool
that
allows
for
easy
interoperability
and
fungibility
within
that,
and
you
could
easily
imagine
sort
of
the
same
thing
happening
if
you
think
inside
every
one
of
those
functional
boxes
in
the
5g
architecture
is
a
bunch
of
microservices
connected
by
a
service
mesh
that
you
can
also
think
of.
You
know
it's
as
a
sort
of
a
recursive
or
a
fractal
pattern,
where
we
have
the
same
sort
of
service
mesh
like
connectivity
between
all
of
the
service,
the
5g
nodes
and
a
similar
kind
of
behavior.
A
That
needs
to
be
supported,
and
if
we
go
to
the
next
slide,
please
I
want
to
talk
about
what
it
is,
in
particular
here's.
What
issio
looks
like
when
you
insert
it
into
that
same
picture.
So
again,
these
are
functional
blocks,
they're,
not
describing
topology.
If
you
look
at
you
know,
for
example,
the
smf
that
might
be
again
running
on
a
single
worker
node,
it
might
be
running
across
a
cluster
or
it
might
be
running
across
multiple
clusters.
A
So
the
actual
low-level
ability
for
these
things
to
communicate
in
a
secure
way
with
consistent
policy
again
needs
some
additional
tool
on
top
and,
as
it
turns
out,
service
mesh
is
just
as
valuable
and
provides
all
the
same
functionality
at
this.
This
sort
of
macro
layer
between
these
different
functions
as
it
would
within
the
functions
to
control
their
own
internal,
behaviors
and
housekeeping.
A
So
in
the
next
slide,
we'll
see
what
what
is
it
that
we
get
from
istio?
That's
so
valuable
in
this
telco
5g
platform
that
we're
building
and
first
and
foremost,
we
get
enforcement
primitives
to
build
zero
trust,
and
you
know
to
oversimplify
zero
trust
is
a
combination
of
really
strong
identity
for
all
of
the
users
and
workloads
and
devices
and
packet
flows
and
then
strong
privacy
between
those
nodes.
When
they're
utilizing
those
network
services
and
the
option
to
you,
know
transparently
and
consistently
encrypt
all
traffic
across
the
entire
mesh
with
mtls.
A
And
then
you
know
the
fact
that
the
sidecar
itself
is
adjacent
to
the
operating
system
and
application
space
rather
than
one
hop
away
in
a
middle
box,
ends
up
in
a
scenario
where
you
have
much
better
context
to
build
strong
identity
by
looking
at
process
tables
and
socket
tables,
and
you
know
things
of
that
nature
next
slide.
Please.
B
Thanks
dave,
this
is
a
wonderful
overview
on
5g
and
its
application
across
industries
and
how
istio
plays
a
role
in
building
a
platform
like
that.
What
I
want
to
talk
about
is
are
the
various
architecture
options
you
have
when
you're
using
history
on
your
platform,
istio
itself
is
quite
extensible
and
for
meeting
any
particular
requirement
that
they've
talked
about.
You
have
various
options
to
understand
these
options
well
and
making
sure
that
you're
meeting
a
requirement
is
the
key
architectural
decision
that
you
have
to
make.
B
Similarly,
I'm
going
to
share
some
of
the
learnings
that
we
have
come
up
with
together,
while
building
a
platform
using
istio
at
its
core.
So
first
off
I
want
to
talk
about
is
how
to
achieve
tendency
or
multi-tenancy,
which
dave
said
was
one
of
the
most
important
things.
So,
in
a
5g
platform,
you
will
have
container
cloud
native
functions
or
cnfs
from
multiple
vendors.
B
At
the
same
time,
the
policy
enforcement
engines,
whether
it's
proxy
or
something
else,
does
not
have
penalties
for
performance
where
our
data
flowing
through
one
of
the
performance
for
policy
enforcement
engine
for
one
network
function
affects
the
performance
of
the
other
right,
so
in
istio,
and
then
particularly
also
in
kubernetes.
The
way
to
achieve
this
tendency
is
to
deploy
your
network
functions
or
cnfs
at
a
namespace
level.
B
So
what
I
mean
by
that
is,
you
can
have
two
cnfs
amf
and
smf,
like
they
have
said,
ignore
that
these
are
three
letter
acronyms,
but
they
just
provide
functions.
So
in
this
case,
it's
accessibility
and
mobility
function
talking
to
session
management
function,
so
each
of
these
network
functions
are
microservices
by
itself
so
and
then
inside.
If
you
open
them
up,
they
will
look
something
like
this.
You
have
a
front
end
talking
to
an
app
which
is
all
part
of
amf.
They
will
have
its
own
database.
B
Similarly,
smf
will
have
a
front-end
application
and
its
database.
Now
you
want
to
make
sure
that
the
traffic
between
these
network
functions
at
inter
cnf
level
is
at
least
encrypted
with
solid
workload
identities.
That
dave
was
talking
about
what
we
found,
that
you
have
a
hybrid
environment
here.
Actually,
when
you
try
to
achieve
it
with
sto,
not
all
vendors
will
be
willing
to
add
side
cars
in
their
cnfs.
They
are
going
through
various
phases
of
migration.
Some
of
them
are
lifting
and
shifting
some
of
them
have
still
not
tested.
B
So
what
you
end
up,
having
is
actually
an
architecture
like
this.
You
have
side
cars
in
some
of
the
network
functions
and
in
the
other
network
functions
you
won't
have
side
cars.
So
what
you
can
do
is,
you
can
add,
an
ingress
gateway
for
those
network
functions
which
can
provide
the
same
primitives
for
encryption
and
then
help
you
migrate
later,
when
you
are
adding
side
cars
to
all
the
components
within
a
particular
nf,
for
example
over
here
you
might
be
adding
side
cars
eventually
to
all
the
components
in
smf.
B
So
now,
in
this
architecture,
which
seems
simple,
there
are
two
or
three
key
challenges
you
will
face
when
you're
building
it
first
is
you
want
to
make
sure
you
have
strong
identities
that
you
are
attaching
to
these
proxies
so
for
the
amf,
so
amf
front
end
has
a
side
car.
So
the
side
car
gets
the
amf
identity.
For
the
smf
side,
you
want
to
have
the
smf
identity
attached
to
the
ingress
gateway.
B
B
Secondly,
we
also
found
that,
as
istio's
architecture
evolved
and
we
went
from
setable
to
sds,
the
istio
generated
certificates
were
themselves
not
available
as
secrets
which
can
be
used
easily
by
the
ingress
gateway.
But
once
you
are
able
to
augment
this
to
you
and
make
sure
that
the
certificates
that
the
control
plane
creates
is
uniformly
consumed
by
both
the
side,
cars
and
at
ingress
gateway,
northbound
interface,
you
can
easily
migrate.
Your
all
your
cnfs
to
later
use
and
inject
side
cars
right
so
from
this
architecture.
B
B
B
So
the
second
thing
that
I
want
to
talk
about
is
legacy
network
functions
and
how
to
make
them
talk
to
cnfs
in
the
mesh.
This
is
a
common
problem
that
you
will
have,
whether
it's
a
5g
platform
or
whether
it's
an
enterprise
application.
You
will
have
legacy
functions
that
have
to
talk
to
containerized
modern
applications.
So
in
this
case
you
have
a
udm
network
function
which
might
be
on
a
virtual
machine
that
wants
to
talk
to
a
cnf
which
is
in
containerized
world.
B
So
the
issue
here
is
as
soon
as
you
add,
the
sidecar
to
smf.
Obviously
you
want
to
do
encryption
and
when
you
try
to
encryption
this
communication
breaks,
because
the
virtual
machine
cannot
talk
the
language
that
the
sidecar
is
talking
over
there.
It
doesn't
understand
the
spiffy
based
science,
it
doesn't
know
how
to
get
the
certificates,
so
there
are
a
bunch
of
options
here
to
solve
this
problem.
B
First
off
this
is
the
istio
community
recommended
way,
so
you
can
add
an
on-voice
site
car
proxy
in
your
virtual
machine
and
extend
your
mesh
to
include
virtual
virtual
machines.
So
it
feels
like
your
virtual
machine
is
participating
in
the
mesh
as
if
it's
in
the
containerized
environment
and
then
the
control
plane
is
transparently
managing
the
certificate
life
cycle
and
identities
for
both
the
side,
cars
in
the
virtual
machine
and
also
injected
it
as
part
of
the
pod
in
smf.
B
Again
in
here,
you
will
have
the
same
problems
that
we
had
when
you
are
trying
to
add
the
ingress
gateway
between
a
side,
car
enabled
port
and
then
a
pod
without
the
side
car
in
the
in
the
egress
gateway.
The
connection
to
the
outside
world
needs
certificates
for
encryption
and
by
default
it
cannot
use
the
skew
enable
certificates,
but
if
you
are
able
to
extend
stu,
you
can
have
uniform
certificate
management
for
for
both
legs
of
this
traffic.
B
Enabling
the
application
to
talk
tls
directly,
so
in
this
case
you
want
the
virtual
machine
to
participate
in
the
mesh
by
getting
the
certificates
and
the
credentials
directly
from
the
seo
control
plane
again.
This
is
easier
said
than
done
in
the
older
versions
of
istio,
where
the
citadel
citadel
was
creating
secrets,
and
you
can
have
the
virtual
machine,
get
the
secrets
based
on
api
or
mounted
as
a
file,
somehow
and
and
get
those
credentials
to
create
a
tls
based
connection
with
sds.
B
It
gets
difficult
so
either
you
have
to
preserve
the
behavior
of
citadel
to
create
those
secrets
just
for
these
use
cases
or
go
some
other
route.
Additionally,
most
of
the
legacy
applications
don't
really
understand.
Spiffy
based
app
spiffy
based
certificates,
even
though
spiffy
is
great
for
creating
identities.
Istio
by
default
only
will
add
stiffy
entities
in
sand.
B
So
these
are
some
of
the
options
that
you
will
have
that's
applicable
in
a
5g
world,
but
also
these
are
the
same
problems
you
might
have
in
enterprises
moving
on
yeah.
So
basically,
in
this
case,
the
udm
identity
is
attached
to
the
udm
virtual
machine
itself
and
it's
managing
the
tls
directly
a
lot
of
tuning
that
you
might
have
to
do
in
sto
itself,
as
this
stu
has
a
lot
of
configuration
knobs
to
make
sure
you're
meeting
the
5g
requirements.
B
So
this
is
some
of
the
things
that
I
would
recommend
you
do,
first,
is
to
make
sure
you
use
cni
so
that
you're
not
having
any
escalated
pod
privileges
by
default.
Istio
adds
a
histo
in
it
container,
which
runs
as
an
init
container
with
sets
of
the
ip
table
rules
for
you,
but
for
it
to
run
the
ip
tables
it
has
to
have
escalated
privileges
in
many
of
the
environments,
especially
like
openshift.
You
might
not
be
allowed
to
have
those
privileges,
privileges
and
in
those
modes
running
cni
is
better
it's.
B
B
Creating
your
own
intermediate
certificate
authority
is
much
better
because
it
allows
you
to
also
extend
your
mesh
easily
by
using
the
same
trust
across
different
clusters
or
different
sites,
which,
as
dave
mentioned,
is
important
and
5g,
as
all
of
these
network
functions,
there's
no
way
to
fit
them
in
just
one
cluster
right.
You
should
be
enabling
ecc
certificates,
so
this
is
one
of
the
functionalities
that
was
added
by
our
team
in
aspen
machine
open
source.
B
Recently,
whenever
we
talk
to
5g
security
architects,
they
will
always
say
we
need
something
like
this
and
I'm
glad
that
now
is
your
support
set.
You
should
be
able
to
configure
your
workload
certificate
ttls
by
default.
Sto
comes
with
around
90
days
of
workload
certificates.
You
should
be
able
to
reduce
the
time.
This
is
again
important
because
there
is
no
way
to
directly
revoke
workload
certificates
in
sdo,
so
having
a
shorted
ttl
can
help
you
work
around
that
problem.
B
There
are
policy
primitives
to
enforce
outbound
policies,
but
those
outbound
policies
and
sto
are
enforced
at
the
client,
and
you
should
not
be
just
relying
on
well-behaved
clients.
You
should
enforce
the
outbound
at
a
egress
gateway,
you
can
use
the
stewards
gateway
or
a
more
capable
big
ip,
or
something
like
that.
If
you
want
to
use,
but
it's
always
better
to
insert
that
in
your
network,
rather
than
relying
on
side
enforcement.
B
B
This
functionality
was
also
again
added
by
our
team
last
permission,
open
source
istio
to
make
sure
when
the
tracing
is
enabled
for
this
cnf
traffic
from
one
cnf
to
the
other.
You
have
the
tags
that
is
surfaced
by
envoy
for
http
and
bunch
of
other
metadata,
but
it
also
has
5g
context
in
it.
There
are
specific
5g
headers
that
each
of
these
network
functions
propagate
and
you
want
to
make
sure
that
context
is
in
those
traces.
So
you
can
actually
understand
what's
happening.
B
When
traffic
failures
happen,
you
should
be
able
to
optimize
your
http
2
stream
and
connection
settings.
This
is
all
more
important
for
network
functions
like
service
communication
proxies
or
scps,
which
are
critical
part
of
5g
flow,
where
all
the
traffic
goes
through.
Interestingly,
we
had
to
optimize
http,
2
stream
and
connection
settings
not
just
for
the
outbound,
but
also
for
traffic
between
the
pod
and
the
workload
on
localhost.
B
B
Moving
away
from
helm
as
the
primary
means
of
lifecycle
management
also
caused
some
churn,
because
many
of
the
vendors
itself
normally
use
helm
for
their
micro
service
deployment
and
using
helm
is
the
common
language
simplifies
a
lot
of
things.
So
I'm
glad
in
sto18
we
were
able
to
reintroduce
helm
as
primary
means
for
installation
and
upgrade
for
sdo,
along
with
the
other
supported
options
as
sqctl
and
operator.
B
B
Certificate
ttls
is
should
be
a
simple
job,
but
sometimes
those
options
are
hidden,
so
you
might
have
to
figure
out
a
few
layers
before
you
understand
how
you
can
configure
them
reliably
as
ecc
certificates
was
added
later
on,
you
have
to
you,
might
have
to
migrate
from
rsa
to
ecc.
This
migration
is
quite
difficult
to
achieve
with
zero
downtime.
It
can
be
achieved,
but
it
it
might
take
few
iterations
before
you
can
get
there.
We
also
realized
some
of
the
critical
headers
for
unauthorized
traffic.
B
When
the
proxies
were
returning
401,
they
were
missing
headers,
like
ww
authenticate
headers,
which
many
of
the
network
functions
relied
on
are
tuning
per
proxy
concurrency
setting
was
added
much
later
in
last
year
before
you
could
only
enable
proxy
configuration
for
the
concurrency
at
a
mesh
level
which
which
works
for
most
use
cases
but
for
network
functions.
Some
of
them
have
critical
requirements
around
concurrency
and,
like
I
said,
consuming
steel
generated
certificates
at
gateways
is
not
very
trivial
to
do.
You
have
to
make
a
lot
of
changes
to
make
that
architecture
work.
B
B
When
I
say
beyond
this
to
you,
you
might
have
to
augment
this
to
you
or
you
might
have
to
add
other
components,
to
make
sure
that
communication
pattern
is
working.
So
there
are
legacy
protocols
in
other
telecom,
related
protocols
like
diameter,
sctp
and
gtp,
for
which
currently
there
is
no
support
in
onward.
So
you
might
have
to
add
a
proxy
like
big-ip
to
make
sure
these
translations
happen
before
the
mesh
traffic
can
begin
for
high
speed
data
power
traffic,
which
relies
on
slrv
or
dpdk.
B
B
One
of
the
last
things
that
I'm
going
to
talk
about
is
deep
packet
inspection,
which
is
the
capability
where
service
providers
are
able
to
recreate
failures
in
the
network
by
capturing
traffic
at
full
fidelity
between
network
functions.
Now
this
is
a
important
feature
for
them,
but
adding
a
service
mesh
or
adding
a
sidecar,
proxy
and
moving
the
encryption
can
often
break
it.
So
I'm
going
to
quickly
go
over
the
same
example
and
see
what
happens
so.
You
have
amf
talking
to
smf.
You
have
encrypted
traffic
in
the
traditional
world
this
encrypted.
B
This
traffic
will
be
tapped
by
optical
taps
and
sent
to
network
analyzers
for
getting
full
fidelity
captures.
But
since
this
traffic
is
encrypted
now
at
the
mesh,
you
can't
really
decrypt
it.
This
is
because
it's
encrypted
with
perfect
forward
secrecy
and
sort
easy
to
maintain
station
keys
for
all
this
information.
Additionally,
these
tabs
don't
really
work
at
layer
7.
B
So
you
are
missing
critical,
http
2
graded
transaction
data,
and
you
don't
really
have
reliable,
contextual
data
about
the
kubernetes
name
space,
for
example
the
metadata
about
the
app
labels
you
just
have
ports
and
ips,
which
itself
can
change.
So
the
only
way
to
really
make
the
same
capability
work
in
the
new
world
is
capturing
these
packets
before
they
get
encrypted.
B
So
this
is
like
moving
some
functionality
that
used
to
be
in
these
middle
boxes
and
moving
them
closer
to
the
workloads,
and
this
is
where
I'm
gonna
hand
it
back
to
dave,
who
is
gonna.
Tell
us
how
we
envision
the
network
topology
changing
with
the
addition
of
sidecast
to
workloads.
Now
over
to
you
dave.
A
So
today,
and
traditionally
you
know
we
had
what
you
could
call
sort
of
a
fat
network
and
skinny
endpoints
inside
of
the
network.
Dmarc,
there
were
all
kinds
of
middle
boxes
with
special
dedicated
function
and
you
could
sort
of
compose
service
chains
by
taking
multiple
hops
through
some
custom
path
through
that
network,
but
as
mirage
showed
as
soon
as
you
move
encryption
into
the
end
point
so
that
it's
truly
end
to
end
you've.
A
A
If
you
will,
and
a
dumber
or
skinnier
transport
network
in
between
this
allows
you
to
reduce
costs
and
to
you
know,
continue
to
have
a
completely
hardware
defined
data
path
inside
the
transport
network,
where
you
have
to
move
things
at
you
know
kind
of
multi-terabit
speeds,
but
a
lot
of
the
high-touch
packet
features
can
all
leave
that
network
and
end
up
running
at
the
edge
and
becoming
software
based,
and
so
the
kind
of
ultimate
evolution
of
that
notion
is.
You
could
have
sort
of
all
of
the
packet
processing.
A
That's
not
very
specific
to
connectivity
and
capacity
all
moves
out
into
the
edge
proxies
and
the
only
thing
left
in
the
network
is
sort
of
connectivity
and
and
bandwidth
and
then,
finally,
if
you
go
to
the
next
slide,
you
know
taking
a
look
at
what's
happening
out
there
in
the
world
and
today
there's
envoy
proxy
projects
for
both
popular
mobile
operating
systems
for
windows,
for
every
flavor
of
linux.
For
everything
else.