►
From YouTube: Stitching a Service Mesh Across Hundreds of Discrete Networks - Jason Webb & Anil Attuluri, Intuit
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Stitching a Service Mesh Across Hundreds of Discrete Networks - Jason Webb & Anil Attuluri, Intuit
Intuit has experienced large growth in its microservices ecosystem over the last few years, which was primarily using a hub and spoke API Gateway for service communication. As the ecosystem expanded, the increased latency and data transfer costs became significant. To facilitate future growth efficiently, Intuit needed a better model. Moving to a distributed Service Mesh running on k8s to enable secure service-to-service communication was the solution. As Intuit was building a migration path for hundreds of services communicating across discrete networks, they faced a host of challenges. While developing a platform to provide end-to-end encryption, they defined a pattern for federated workload identities and learned to manage a federated set of mesh control planes. Jason and Anil will share these learnings and Admiral, a project they are open-sourcing that enabled the migration path.
https://sched.co/UadU
A
A
B
B
So
why
were
we
interested
in
service
mesh
at
Intuit?
So
in
the
beginning
we
started
out
in
our
monolithic
era
right
where
we
had
applications
that
supported
our
big
business
units
that
didn't
quite
have
enough
velocity
and
they
needed
to
be
broken
up
they're
too
thick.
So
we
started
to
move
into
our
micro
services
error
where
we
started
to
decompose
all
of
these
services.
So,
as
we
went
through
that
decomposition,
the
cost
of
authoring,
any
one
service
was
still
too
high
right.
B
So
that's
where
we
really
started
to
look
at
service,
mesh
and
sto
and
how
we
could
bring
that
service
mesh
into
our
enterprise,
so
just
kind
of
a
quick
sample
of
what
a
service
mesh
might
look
like.
So
in
this
particular
deployment
we
have
a
some
toy
services,
a
book
order
or
book
info
service
and
a
product
info
service
and
then
a
payment
service.
So
the
book
and
post
service
is
going
to
talk
to
these
other
two
services.
B
So
we
have
a
single
cluster
here
and
we
have
a
SEO
control
plane
installed
in
this
cluster
and
then
in
between
the
different
services,
communicating
through
a
proxy.
We
can
establish
mutual
TLS.
So
in
this
picture
we
left
an
API
gateway
as
the
ingress
point
as
browser
traffic
comes
into
the
cluster,
it'll
probably
use
a
different
authentication
protocol,
maybe
OAuth
or
some
JA
authentication.
B
So
we
still
having
a
controlled
ingress
point,
but
this
gives
us
the
ability
to
get
observability
and
intelligent
routing
inside
of
our
clusters,
which
is
great,
but
our
enterprise
is
a
little
bit
more
complicated
than
kind
of
this
simple
diagram
right.
So
some
of
the
complexities
that
we
ran
into
as
we
look
to
bring
the
service
mesh
into
our
ecosystem
was
that
we
actually
ran
a
lot
of
kubernetes
clusters.
I
ran
those
clusters
for
four
different
reasons.
Right.
We
wanted
isolation
in
between
our
different
business
units.
B
We
wanted
our
business
units
to
be
able
to
have
autonomy
between
them
if
they
had
different
seasonality,
and
sometimes
they
wanted
to
take
certain
have
more
caution
in
different
parts
of
the
year.
We
wanted
them
to
not
impact
each
other
and
there's
also
compliance
reasons
right.
So,
if
we're
hosting
services
that
are
underneath,
maybe
a
PCI
audit,
we
want
to
limit
the
scope
of
those
audits.
B
But
we
not
only
do
we
have
a
lot
of
clusters,
but
also
all
of
these
clusters.
Don't
have
l3
l4
connectivity
between
the
pods
right,
so
they're
actually
island.
There
kind
of
island
networks
in
between
each
other.
So
as
we
want
to
have
any
services
communicate
with
each
other,
we
have
to
be
able
to
ingress
and
egress
out
of
these
clusters,
so.
B
And
just
to
kind
of
put
some
statistics
on
what
this
actually
looks
like,
so
we
run
about
160
clusters
and
roughly
almost
5,400
namespaces.
So
the
unique
thing
about
these
namespaces
is
that
they're,
actually
globally
unique.
So
every
namespace
that
gets
provision
is
unique
across
all
of
the
clusters
and
that
becomes
important
and
sort
of
how
we
decide
to
stitch
together.
The
mesh
in
the
future
here.
B
So
if
we
take
our
toy
that
we
started
with
and
actually
tried
to
overlay
it
and
does
something
that's
a
little
bit
more
realistic
of
a
topology
remind
the
where
we'll
end
up
with
a
topology.
That
looks
a
little
bit
more
like
this,
where
we
have
three
clusters
hosting
these
different
services,
we've
isolated
our
API
gateway
into
one
cluster.
That
gateway
happens
to
be
a
multi-tenant
system,
so
it
supports
a
lot
of
the
business
units.
B
So
we
don't
want
it
coupled
with
any
set
of
deployments
for
any
of
the
other
clusters
in
this
particular
use
case
for
the
product
info
service
in
the
book
order
service
are
actually
maintained
by
the
same
business
unit,
so
that
went
into
one
independent
cluster
with
itself
and
this
payment
service,
because
it's
gonna
be
part
of
an
audit.
We
broke
it
out
and
stuck
it
into
a
separate
cluster.
B
So
as
we
look
to
sort
of
overlay
a
mesh
onto
this
more
distributed
cluster
topology,
what
do
we
actually
need
out
of
the
mesh
to
be
able
to
make
this
happen?
So
the
first
thing
we
need
to
be
able
to
support
is:
we
need
some
sort
of
identity,
of
what
a
service
means,
and
we
need
this
to
be
able
to
support
HADR
deployments
right
so
because
we
have
globally
unique
namespaces
across
all
of
these
clusters.
B
B
We
also
don't
want
to
lose
strong
point-to-point,
authentication
that
the
mesh
gives
you
with
mutual
TLS.
We
want
to
keep
that
functionality
and
we
also
want
to
keep
end-to-end
encryption,
so
we
don't
want
to
have
any
man
in
the
Middle's
across
the
ecosystem.
We
want
strong
TLS
encryption
between
all
the
nodes
in
between
all
the
east-west
services
service
communication.
B
We
also
want
to
eliminate
single
points
of
failure.
So
we've
we've
created
this
ecosystem
of
all
of
these
clusters
for
different
isolation
purposes
and
autonomy.
We
don't
want
to
then
introduce
another
layer
on
top
of
that.
That
causes
us
to
bring
that
all
back
together
and
break
that
level
of
isolation
and
points
of
failure,
and
then
this
particular
one,
so
by
default,
the
DNS
name
that
kubernetes
is
going
to
give.
You
has
the
name
space
and
the
cluster
suffix
put
into
it.
B
That
sort
of
starts
a
couple:
the
service
discovery
of
how
you
locate
the
service
with
what
we
would
consider
the
administration
boundary
of
the
service
right.
So
if
that
service
changes
ownership
or
moves
in
between
different
namespaces,
we
wouldn't
want
all
the
clients
to
have
to
update
how
they
addressed
that
service.
So
we
want
those
two
things
to
be
decoupled
from
each
other
and
then
also
we
want
to
keep
the
developers
experience
intact
right.
So
we
don't
want
to
have
configuration
separate
from
the
services
actual
hosting
configuration.
B
So
looking
at
these
requirements,
we
started
a
look
at
what's
available
in
sto
to
be
able
to
map
these
requirements
on
to
sto,
so
she
has
a
lot
of
different
multi
cluster
functionality
in
it.
There's
a
lot
of
good
stuff
there
and
be
able
to
connecting
so
the
first
one
we
looked
at
was
using
a
shared
control
plane.
So
in
this
particular
use
case,
or
this
particular
topology,
we
can
see
these
red
lines.
B
All
the
proxies
will
connect
up
into
one
shared
control
plane
between
all
the
clusters,
so
we
can
still
have
mutual
TLS
in
between
the
services
going
across
the
clusters,
and
we
can
do
that
by
sharing
an
intermediary
intermediary
CA
that
trusts
the
same
route,
CA,
that's
deployed
and
Citadelle
in
those
clusters.
That's
the
open-source
way
to
do
it
internally.
B
We
have
an
internal
CA
that
we
use
that
we've
extracted
away
from
all
the
mesh
deployments
so
and
we
can
ingress
and
egress
through
the
networks
pretty
well,
but
it's
just
this
set
up
didn't
quite
meet
all
of
our
requirements,
so
we
didn't
do
anything
here
to
sort
of
be
able
to
identify
a
workload,
that's
being
deployed
in
separate
namespaces.
Actually,
with
some
of
the
mesh
network
functionality,
we
actually
forced
that
we
want
the
way
it's
employed.
B
It
actually
wants
you
to
keep
those
things
the
same,
but
we
got
that
as
a
point-to-point,
authentication
and
end-to-end
encryption
with
this
deployment.
The
single
point
of
failure
is
kind
of
a
warning
sign.
So
technically
you
can
make
that
shared
control,
plane,
high
availability
and
have
multiple
deployments
of
it.
B
But
it's
something
you're
going
to
put
a
lot
of
care
and
time
into
making
sure
that
doesn't
break,
because
if
any
of
the
clusters
can't
connect
into
that
control
plane
to
get
configuration
for
their
proxies,
then
you
won't
be
able
to
Auto
scale
effectively
in
the
cluster
and
then
also
we
didn't
do
anything
to
necessarily
decouple
the
naming
scheme
from
the
namespace
and
now
our
the
way
we
author,
our
SEO
configuration,
is
in
a
control
plane.
That's
separate
from
the
service
goes
control
plane,
which
makes
things
a
little
bit
awkward
for
developers.
B
But
you
end
up
with
a
little
bit
of
a
problem.
Here
is
how
do
you
manage
the
configuration
between
all
these
different
clusters
right
because
now,
we've
partitioned
the
service
discovery.
So
how
can
the
order
the
book
order
service
know
that
the
payment
service
is
actually
deployed
in
another
cluster
and
how
does
they
get
routing
to
get
over
to
that
cluster?
B
But
as
we
look
across
the
points
that
we
wanted
out
of
this,
we
got
a
couple
more
boxes
out
of
it,
but
we
didn't
really
solve
this
problem
of
having
this
single
identity
of
a
service
that
can
span
namespaces
that
are
globally
unique
and
we
didn't
really
decouple
the
service,
the
namespace
from
the
from
the
service
discovery,
but
we
were
able,
but
we
did
eliminate
a
single
point
of
failure
now
that
we
have
a
control
plane
with
every
cluster
from
a
global
services
ecosystem.
There's
no
one
system.
B
Now
it's
all
distributed
which
prevents
that
single
point
of
failure,
and
now
any
developer
would
offer
configuration
local
to
their
cluster.
So
it
would
seem
very
natural
with
the
rest
of
their
kubernetes
deployment
and
also
in
our
evaluation
of
this,
a
lot
of
the
just
the
features
with
SEO
work
better
within
one
cluster.
As
things
go
between
separate
clusters,
there's
certain
features
that
might
work
better
or
not
as
well.
So
it's
kind
of
a
more
stable
deployment
of
of
the
control
plane.
B
So
kind
of
so
we
had
this
appointment
of
multi
cluster,
but
we
had
this
problem
of
how
to
manage
that
configuration
across
all
of
these
clusters.
So
that's
where
we
came
up
with
this
project
Admiral,
so
the
multi
cluster
deployments
that
we
do
use
Admiral
to
do
this
higher-level
configuration
management,
and
so
really
what
Admiral
is
doing.
B
It
can
be
able
to
create
a
globally
unique
name
that
services
can
use
and
then
it
can
be
able
to
set
up
the
routing
to
those
deployments
that
decouples
the
namespaces,
the
namespace
topology
from
the
service
configuration
and
then
the
other
thing
we
created
it
to
be
able
to
do
is
be
able
to
have
multiple
names
for
the
same
service
so
and
to
be
able
to
make
the
create
those
names
easy.
So
a
lot
of
times.
B
You
would
want
maybe
a
end
to
end
environment
that
could
be
pinned
within
different
regional
routing,
so
I
want
an
antenna
environment
to
be
pinned
to
an
East
region
or
a
West
region,
because
that
may
be
I
want
to
run
regression
Suites
in
different
regions
of
that
deployment,
as
I'm
as
I'm
migrating
my
deployment
across
my
fleet,
so
the
way
Admiral
actually
plugs
into
this
multi
cluster
deployment.
Is
it
actually
sits?
B
And
I
guess
one
more
opponent
here
and
Admiral
itself
doesn't
actually
hold
any
runtime
State,
it's
just
sinking,
basically
sto
states,
so
all
of
the
state
that's
used
to
run.
These
clusters
is
actually
pushed
down
into
every
individual
cluster
itself,
and
so
that
means
that
if
Admiral
goes
away,
you're
left
with
the
current
running
state
of
all
of
the
meshes
that
are
across
the
fleet.
So
it
can't
really
it
doesn't
impact
any
of
the
runtime
deployment.
B
So
if
we
look
at
the
kind
of
things
that
we
wanted
to
accomplish,
with
the
multiple
control
planes
and
Admiral
sinking
configuration
and
we're
able
to
solve
all
the
problems
that
we
wanted
to
be
able
to
roll
this
out
across
the
enterprise,
so
Neal
is
gonna.
Do
a
demo
of
Admiral
for
a
a
che
or
an
a
demo
with
Admiral
of
an
H
a
TR
deployment
thanks.
C
Jason,
okay,
so
today
we
would
like
to
demo
the
global
traffic
routing
capability,
that's
powered
by
Admiral.
So
first
we'll
look
into
like
the
same
type
problem
that
Jason
was
talking
about
so
here.
If
you
see
on
the
right-hand
side,
there's
like
a
payments
micro
service,
it
has
been
deployed
in
both
US
questioned
us
east
regions,
and
that
is
for
their
disaster
recovery
issue.
C
And
then
you
have
a
book
order
service,
that's
deployed
in
another
cluster
that
that
actually
calls
a
payment
service
for
the
payment
processing.
And
then,
if
you
see
like
each
of
these
clusters,
have
sto
in
them
and
sto
already
supports
something
called
locality,
load
balancing
which
makes
the
which
makes
the
locality
based
routing
possible.
Basically
like
you
route
the
traffic
to
the
nearest
region
first
and
then,
if
that
fails,
you
route
the
traffic
to
the
next
available
region
or
next
closest
region.
C
So,
given
this
and
Admiral
is
obviously
like
watching
all
these
flusters
it
can,
it
can
watch
the
sto
configurations
and
also
the
services
that
are
getting
deployed
are
to
these
clusters
and
then
it
can
create
this
global
traffic
policies
or
use
basically
to
route.
The
traffic
or
like
to
control
the
traffic
in
individual
clusters,
okay,
so
let's
go
back
to
the
command
line,
so
what
I
want
to
show
is
so
this
custom
CRD
the
custom
resource
definition
that
admiral
understands,
which
is
a
global
traffic
policy.
C
C
That's
going
to
use
payments,
this
name
is
going
to
remain
the
same,
and
then
the
and
then
this
micro
service,
owner
of,
like
the
owner
of
the
payment
service,
can
actually
control
this
configuration
from
their
own
cluster
so
that
they
can
control
how
the
clients
actually
route
traffic
to
their
service.
So
if
you
look
at
here,
you
see
the
DNS
name,
of
course,
and
then,
like
you,
see
the
region.
So
what
we
are
saying
here
is
any
any
at
any
time.
C
If
someone
calls
default
payments
toward
global,
you
would
route
80%
of
traffic
to
us
for
us
to
and
then
20%
no
traffic
to
your
C's.
So
and
then,
if
you
see
the
the
other
interesting
thing
here
is
so
there's
a
selector,
so
you
apply
this
global
traffic
policy
to
the
names
or
like
the
deployments
that
have
this
label,
which
is
the
identity
label.
So
the
identity
is
like
the
unique
identifier
for
that
micro
service
in
your
entire
company.
C
So,
given
that
what
I
would
like
to
do
is
go
back
to
the
slide
that
I
am
showing.
So
if
you
look
at
this,
so
this
is
like
a
browser
or
like
a
desktop
on
the
left
hand
side.
So
that's
calling
a
book
order.
Service
and
bukata
service
is
in
turn,
calling
the
payment
service.
So
now
we
have
built
some
UI
where
the
payment
service,
first
of
all,
like
the
payment
service,
returns
the
look
information
based
on
which
payments
instance
is
serving
that
request.
C
C
So
what
you're
looking
at
here
is
so
once
we
start
the
request,
we
are
going
to
see
the
distribution
of
traffic
that
is
being
served,
but
from
the
payment
service
from
us,
West
and
us
East,
and
based
on
this
proportion
like
how
much
how
many
requests
are
serving
getting
served
from
east
and
west.
These
images
are
going
to
shrink
or
grow
in
size.
C
So
let
me
start
the
request
so
right
now
we
have
the
distribution
of
80/20,
so
if
you
see
a
it
will
stabilize
in
a
little
while,
so
you
can
see
that
eighty
percent
of
traffic
roughly
is
going
to
us
first
and
twenty
percent
is
going
to
us
east
now.
What
we
are
going
to
try
is
go
edit,
this
global
traffic
policy
record
and
flip
it.
So
let's
do
20
and
80
and
then
I'm
going
to
save
this
and
then
go
back
now.
C
So,
let's
get
into
the
actual
edited
or
like
the
modified,
so
I
have
created
a
aliases.
Pardon
me
I
want
to
say
like
seconds
like
whenever
I
don't
want
to
type
like
cubes
ETL
every
time.
So
if
you
look
at
the
destination
rule,
so
we
looked
at
the
global
traffic
policy.
So
this
is
a
destination
rule.
The
ISTE
registration
rule
in
the
cluster,
where
order
service
is
running.
C
The
other
thing
is
the
service
entry
so
default
dot
payments
are
global,
is
the
service
name
that
has
been
created,
it's
not
the
kubernetes
service
name,
which
is
bound
to
the
namespace.
This
is
namespace
agnostic,
which
jason
was
talking
about.
So
we
need
the
namespace
agnostic
names
because
tomorrow,
if
the
service
moves
away
from
that
namespace,
we
still
want
to
have
like.
We
don't
want
the
clients
to
change
the
service
name
that
they're
calling.
C
So,
let's
look
and
and
the
cool
part
about
admiral
is
it
does
the
contextual
configuration
so
let's
say
tomorrow
the
payment
service,
for
whatever
reason
moves
into
the
same
cluster
as
where
orders
service
is
running,
then
you
don't
want
like
that.
Communication
is
local
within
the
cluster,
so
admiral
is
responsible
for
delivering
the
contextual
configuration
depending
on
where
the
where
these
services
and
their
dependencies
are
running.
C
So
that
said,
let
me
show
the
service
entry,
so
service
entry
is
like
bringing
your
owns,
so
sto
watches
East
your
watch
East
your
monitor,
so
kubernetes
api
server
updates
and
it
watches
all
the
services
that
are
being
deployed
in
the
kinetics
lustre.
Now,
if
you
want
to
bring
in
like
your
custom
services
into
sto
service
entry
is,
is
the
configuration
that
you
would
do
since
default,
or
global
payments
or
global
is
a
service
name
that
admiral
generated
it's
going
to
use
something
called
a
service
entry
which
is
a
co
customer
resource
to
do
that?
C
So
if
you
look
at
this
so
so
this
is
the
hostname.
So
whenever
someone
accesses
this
hostname,
what's
actually
happening,
is
you
have
defined
a
couple
of
endpoints
right?
So
this
is
all
automated
by
the
way,
so
this
is
all
being
generated
by
Admiral.
You
see
the
address,
so
this
address
is
basically
the
load
balancers
of
the
clusters
where
payment
services
are
running.
So
this
is
a
cross
cluster
communication,
that's
happening
and
then,
if
you
see
like
there
is
also
a
locality.
C
Information
that
is
being
specified
for
each
of
these
and
and
endpoints.
So
this
is
also
done
by
admiral,
because
admiral
knows
where
these
clusters
are
deployed
based
on
the
nodes.
So
admiral
is
going
to
attach
this
locality
labels
and
and
that's
what
is
making
this
locality
load-balancing
possible
across
the
clusters.
C
C
Okay,
so
Admiral
is
open
source,
so
we
thought
like
based
on
our
learnings
from
the
multi
cluster
service
mesh
journey.
We
have
we
open-source
this
product
so
that,
like
it,
the
SEO
community
can
benefit
from
it
and
not
just
history
community,
the
service
mesh
community
in
general,
so
we
would
love
the
feedback
and
any
like
the
contributions
to
the
project.
So
I'm
going
to
like
go
to
the
Admiral
project,
so
Admiral
project
by
the
way
is
open,
so
standard
SEO
ecosystems
is
tak.
D
C
So
the
question
is:
how
does
the
Admiral
know
if
there
is
a
new
cluster,
so
so
kubernetes
api
server
works
off
secret?
So
what
we
do
is
whenever
there's
a
new
cluster
created
the
cluster
provisioning
piece
of
it
basically
creates
such
secret.
That
is
required
to
talk
to
that
kubernetes
api
server
and
it
drops
it
in
a
central
namespace
where
admiral
is
running.
So
that's
how
admiral
knows
that
there
is
a
new
cluster
and,
from
that
point
on
it's
going
to
watch
the
cluster
and
it's
going
to
do
whatever
is.
E
One
of
the
reasons
you
mentioned
in
the
beginning
why
you
have
multi
cluster
multiple
clusters
in
your
organization
is
one
of
the
reasons
was
compliance.
So
let's
say
if
the
payment
system
is
sitting
in
compliant
area
and
some
other
random
service
is
sitting
in
some
other
non
compliant
area
and
the
the
CA
that
you
mentioned
is
compromised
in
non-compliant
area.
How
will
it
impact
the
service
sitting
in
the
complaint
area
because
it
seems
like
if
they
are
under
also
CA
some
same
route?
Ca
then
the
non
compact
area
compromise
could
impact
the
the
complaint
a.
B
Yes,
from
the
isolation
of
the
the
trust
domains
within
it
yeah,
so
that's
a
really
good,
really
good
question.
So,
in
general,
we've
used
the
same
credential
basis
across
the
across
those
spaces
for
for
a
fairly
long
time,
but
the
within
the
intermediary
CA.
As
long
as
they
trust
the
same
route
CA,
you
could
rotate
the
CA
that
was
issuing
their
credentials
into
the
into
the
compliance
side.
B
The
yes,
so
the
there's,
an
ingress
gateway,
an
in-studio
ingress
gateway,
that's
acting
as
a
layer
for
load,
balancer,
that's
routing
off
of
it's
doing
the
s
and
I
based
routing,
which
is
the
kind
of
the
connected.
It's
the
the
Gateway
connected
example,
I.
Think
in
the
sto
documentation,
cervix
uses
that
technique
and
we
use
the
same
technique.
We
just
automate
the
configuration
for
that
technique.
G
Just
a
two-part
question:
first
one
is
regarding
the
ID
middle
that
you
mentioned
where,
if,
if
the
service
is
in
the
same
cluster,
it
knows
that
it
will
send
the
request
to
the
same
cluster.
I
saw
this
ERD,
you
mentioning
the
location
where
the
service
should
route
like
80
20
whatever.
So,
how
does
I
know
you
know
in
the
same
cluster
so.
C
It
knows
it
because
it's
also
watching
the
kubernetes
api
server
so
when
it
gets
when
it
it's
it's
watching
the
deployments
and
the
services,
just
like
the
sto
like
the
SEO
control
plane.
So
when
it
when
it's
generating
the
entries
for
like,
let
us
say
the
payment
service,
it's
it's
going
to
know
that,
like
so
it
has
the
configuration
or
like
the
state
of
like
what
services
and
deployments
are
running
in
that
cluster.
So
it's
going
to
know
that
auto
service
and
payment
service
are
in
the
same
cluster.
So.
C
Will
not
so
it
will
it
so,
in
our
case,
within
the
company,
like
we
use
some
annotations
to
help
admiral
like,
like
basically
defined
that
global
identifier
that
Jason
was
talking
about,
but
you
can
map
in
any
labels
that
you
want
and
then
admiral
would
use
those
labels
to
identify
your
services.
You
don't
have
to
yeah
modify
yourself
just.
G
One
another
question:
not
the
same
is
about
you
mentioned
about
service.
Entry
need
to
be
created
in
your
use
case,
where
the
names
needed
to
be
different
than
the
knee
space
generated
by
kubernetes
right
does
service
entry
that
your
short
example
yes
manually
need
to
be
created
or
because
I
saw
that
the
load
balancer
name
a
table
is
broadband,
so
name
that
you
had
to
provide
was
obviously
lengthy
and
stuff.
G
C
H
C
So
it
co
is
implementation
of
service
mesh
right
like
this.
Like
yes,
some
said
like
we
we
like
in
the
micro
services
journey
and
service
mesh,
would
bring
in
a
lot
of
capabilities
and
sto
is
one
such
implementation,
so
we
picked
it
co
based
on
our
evaluation
internally
within
the
company
as
well
like
which
service
finish
our
technology,
we
should
use
and
then,
given
that
based
like
based
off
that
we
develop
Admiral,
which
would
automatic
inflation
of
Li
Jason
can
add.
Yeah.
B
I
think
some
of
the
some
of
the
routing
capabilities
were
pretty
useful.
The
telemetry
collection
being
able
to
get
the
point-to-point
mutual
TLS
with
the
certificate
rotation.
I,
think
that's
something
that
we've
have
a
lot
of
application
teams,
implementing
security
protocols
so
being
able
to
externalize
some
of
those
security
protocols.
So
we
can
enforce
credential
rotation
is
a
pretty
useful
feature,
so
those
are
some
of
the
things
that
made
it
pretty
attractive
to
us.
I
B
Add
admiral
is
really
tailoring
sto
to
our
needs.
I
would
say,
I
mean
we
do
have
a
forked
sto
internally,
but
it's
pretty
close
to
the
actual
main
line
other
than
like.
We
said
we
don't
use
Citadel
literally.
We
use
an
internal
CA,
so
there's
some
modifications
that
we
do
to
the
installation
to
set
up
some
of
the
trust
across
the
control
plane
to
use
an
internal
CA.
But
that's
the
main
change
that
we
keep.
I
C
J
C
J
C
It
really
depends
on
so
one
key
thing
here
is
the
load
balancer
names
that
we
have
shown.
So
those
are
like
the
cloud
native,
not
the
cloner,
let
the
provider
native
names
that
get
created.
So
as
long
as
you
have
the
support
for
the
issuing
this
gateway
to
get
a
public-facing,
dns
name,
I
think
you
should
be
good
yeah
and.
J
Note
like
that
parole
is
running
like
it
needs
to
access
to
all
the
clusters
right.
So
in
this
case,
like
the
IM
rod
or
the
instance
profile
should
be
like
much
more
higher
like
you
know,
access
to
this
so
do
you
have
anywhere
like
document
it
like.
You
know
what
are
the
types
of
policies
you'll
be
using
in
that
row
so.
B
J
B
You
can,
in
the
in
that
example,
there's
role
definitions
for
the
access
that
admiral
needs
in
general.
The
admiral
only
has
it
reads,
different
information
from
the
cluster,
but
it
only
writes
some
of
the
sto
configuration,
so
it
has
write
access
based
it's
to
a
configuration
but
reads
some
of
the
kubernetes
types
and
sto
configuration.
But
it's
the
rule.
The
rules
are
fairly
well
locked
down
as
tight
as
we
can
make
em
at
least.
K
What
what?
What
was
the
reason
to
not
use
a
service
registry
for
that
other
than
just
like
kubernetes
here
DISA
register,
because
you
map
for
cod2
a
few
Lelo
balances
right
that
will
then
use
sni
to
and
as
a
segue.
How
would
this
work
if
the
call
was
not
coming
from,
for
example,
the
books,
but
coming
directly
to
the
payments
through
Oh
gob
right?
K
B
So
if
you
address
that
that
payment
service
from
a
point
that
was
outside
of
the
mesh,
that
would
need
an
external
DNS
name
certificates
that
match
it
and
so
on
and
so
forth,
right
so
internal
to
the
mesh,
we're
able
to
manage
that
and
then
allow
on
board
to
give
us
that
layer,
seven
control
over
the
global
routing
pieces
of
it.
So
yeah.