►
Description
Want to view more sessions and keep the conversations going? Join us for KubeCon + CloudNativeCon North America in Seattle, December 11 - 13, 2018 (http://bit.ly/KCCNCNA18) or in Shanghai, November 14-15 (http://bit.ly/kccncchina18).
Istio Multi-Cluster Mesh Expansion BOF - Sven Mawson, Google (Any Skill Level)
Istio is an open platform that provides a consistent way to enforce service security, control traffic, and view service behavior across your service mesh. In this BOF session, Sven Mawson will demonstrate how Istio lets you control your services no matter where they run or what language they are written in. Whether you run in a container or stand-alone, whether you run in the cloud or on-premise, Istio can help you connect, manage, and secure your services.
About Sven Mawson
Sven is a Senior Staff Software Engineer at Google, and one of the founders of the open source Istio project. He joined Google in 2006, and has spent the past 10 years working on several generations of Google's API Management platform, starting with the AtomPub-based Google Data APIs. Prior to joining Google, Sven worked for Salesforce.com, where he helped design and build the SOQL language for querying hosted object databases. Sven received his BS in Computer Science from Stanford University and his PhD in Computer Science from UCLA. He currently resides in a redwood forest in the Santa Cruz mountains, east of Half Moon Bay, and enjoys spending time with his wife and three children hiking through the woods and counting banana slugs.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
Hopefully,
everyone
is
here
to
talk
about
sto,
multi,
cluster,
stuff,
yeah,
okay,
I'm
at
the
right
place.
This
wasn't
really
gonna,
be
a
talk,
so
I
just
have
this
on,
because
we
only
have
one
hand
Mike,
but
I'm,
hoping
it's
more
of
a
discussion.
This
room
is
not
really
set
up
for
discussion,
but
so
people
should
like
come
forward
and
talk.
I
can
go
through
a
little
bit.
What
we
have
so
far,
you
guys
actually
you're
just
wearing
the
shirt.
There's
anyone
from
Cisco
here
that
works
on
yeah.
A
A
B
A
You're
familiar
with
it,
okay,
so
there's
all
people
who
don't
even
know
about
it
now
so
I
guess
we
can
start
with
sort
of
going
through
what
it
is.
I
didn't.
Yeah
I
definitely
did
not
prepare
a
lot
for
this,
so
this
is
just
our
docks
on
the
website.
These
are
the
preliminary
docks
if
you
go
to
SPO
dot,
IO
and
there's
a
little
like
there's
a
little
gear
icon.
A
If
you
click
point
0
and
then
you
can,
you
can
get
some
of
this
stuff,
but
the
basic
idea
is
that
we
have
more
than
one
cluster
and
you
get
it
all
connected
right.
So
you
have
a
master
cluster
and
that's
where
we
run
the
ischial
control
plane
and
then
you
have
a
bunch
of
slave
clusters
or
remote
clusters,
I
guess,
sweet
home
and
those
just
connect
to
all
the
services
in
the
main
one,
and
that's
how
you
get
all
the
goodness
right.
So
this
goes
through
the
instructions
on
how
to
do
this.
A
There's
not,
unfortunately,
any
really
cool
pictures,
but
basically
you
set
things
up.
I,
like
get
the
move
bunch
of
credentials
around,
so
things
can
talk
to
each
other
and
load
things,
but
the
end
of
it
is
that
yeah.
That's
not
that
useful,
there's
sort
of
some
actually
some
really
good
helmet
charts
for
this.
Now
that
lets
you
just
using
helm,
say
install
into
this
cluster
the
main
and
then
install
them
these
ones
remotes,
and
then
it
gets
it
all
set
up
and
what
this
actually
does
for.
A
You
is
a
couple
different
things
right,
so
one
is
around
traffic
management.
All
the
all
the
envoys
in
all
the
clusters
are
talking
to
the
same
pilot
running
in
the
main
cluster,
and
that
one
knows
about
all
the
different
clusters.
It
knows
about
all
the
services
and
knows
about
all
the
endpoints,
and
so
it
can
route
traffic
appropriately.
So
you
don't
have
to
worry
about
which
cluster
things
are
running.
In
you
just
say,
service
a
is
talking
to
service
B
and
that
same
like
I
want
to
talk
to
service
B.
A
You
can
move
service
be
put
in
a
different
cluster.
Put
in
multiple
clusters,
write
multiple
clouds,
none
of
the
clients
change.
They
all.
Just
still
talk
to
it,
so
that's
that's,
probably
the
biggest
benefit.
The
other
thing
this
gives
you
is
the
authentication
part,
so
mutual
TLS
works
across
the
clusters,
assuming
they're
all
on
the
same
network.
There's
ongoing
work
to
work
on
more
network
support
and
then
the
because
you're
sending
everything
to
one
cluster
in
terms
of
observability,
the
like
graph
on
a
dashboard,
is
for
the
entire
mesh.
A
It's
not
for
a
single
cluster,
so
you
get
like
you
can
see
all
the
traffic
for
that
service,
no
matter
where
it
runs.
So
that's
the
basic
stuff.
Let's,
let's
actually
talk
about
stuff,
unless
people
want
to
see
the
demo
I
want
to
get
like
questions
and
discussion,
because
that's
what
this
is
supposed
to
be.
So
do
you
want
to
pass
out?
Anyone
have
questions
or
things
they
want
to
talk
about.
You
yeah,
you
can
hello.
C
Yeah
so
I
have
a
question:
I
guess
for
the
the
audience,
because
this
has
been
something
we've
debated
in
the
community
a
bunch
of
times.
This
model
has
a
single
control
plane
and
all
the
remote
clusters
are
communicating
with
that
single
control
plane.
Well,
a
single
pilot.
Let's
call
it
a
single
pilot
Dhupia
are
people.
C
Is
that
the
model
that
people
really
want
because
they're
another
model
that
we
had
started
to
implement
and
then
we've
gotten
a
little
too
late
to
implementing-
and
we
probably
will
implement,
is
where
you
have
a
more
of
like
a
Federation
Federation
might
not
be
the
best
word,
but
a
federation
of
pilots
and
the
pilots
talk
to
each
other.
Okay
and
interchange
the
service
at
endpoint,
so
each
pilot
talks
locally
to
its
service
registry.
That's
it
Skoob
API,
if
it's
kubernetes
and
then
they
exchange
information.
C
So
that's
another
model
that
we've
explored,
I,
guess
we'd
be
interesting,
a
feedback
is
you
know
the
people
have
a
preference
or
people
have.
You
know
an
idea
what
they
prefer,
as
fen
mentioned,
the
other
one
is
like
the
dashboard
in
the
mixer.
There's
been
some
discussion
about
mix
or
scale
and
how
chatty
it
is.
Another
model
that
we're
likely
to
try
to
implement
is
where
the
mixer
may
be.
We
have
a
common
pilot,
but
we
actually
have
a
mixer
in
each
cluster.
A
A
And
one
of
the
on
the
pilot
side,
one
of
the
things
that
we've
been
talking
about
recently
is
they're
scattered,
so
short,
a
2-mile
two
models
for
having
pilots
in
each
cluster
right,
so
one
model
would
be
where
you
were
talking
about
where
each
pilot
in
each
cluster
is
like
talking
to
its
local
registries
and
getting
the
information
another
model.
Is
you
still
have
this
this
model
where
you
have
like
a
main
cluster,
and
that
one
knows
about
everything,
and
you
just
run
that
AJ
right.
C
F
F
A
That's
a
great
question
because
I
think
we
haven't
really
figured
that
out
and
that's
partly
why
we're
asking
right:
it's
because
when
we
originally
so
when
we
originally
designed
this
stuff,
the
Federation
model
was
what
we
were
thinking
and
like.
We
spent
a
lot
of
time
trying
to
figure
that
out
and
getting
working
and
then
it
was
like
wait.
A
C
That's
exactly
right.
This
was
this
was
very
simple
to
get
started
with
extremely
simple,
because
we
already
had
the
code
was
already
modularized
to
you
know,
handle
kubernetes
console
all
those
even
in
a
single
installation,
so
we
were
just
able
to
expand
it
very
simply.
The
the
federated
approach
was
a
little
bit
more
complicated.
We
had
quite
a
bit
of
debates
and
we
actually
started
doing
some
work
on
it,
but
it
then
you
know
we
got
busy
with
really
improving
quality.
So
that's
slow,
yeah.
A
B
We
in
the
federated
model
the
work
that
they've
been
doing
for
Federation
v2.
It
actually
would
help
a
lot
for
the
deployment
case
so
where
you're
deploying
the
services
in
the
mesh
and
not
so
much
I
mean
the
work
we've
done
already
could
like
coexist
with
that.
Actually,
so
the
pilot,
knowing
about
all
the
you
know,
everything
in
both
meshes
or
multiple
meshes
is,
is
already
there.
B
The
addition,
you
know
it
would
be
cleaner
if
we
you
directly
use
the
same
cluster
registry
and
stuff,
that's
in
Federation
v2,
but
as
far
as
you
know,
after
you've
deployed,
is
do
that.
What
do
you
do
to
deploy
your
apps
across
multiple
clusters
and
in
Federation
is
trying
to
solve
that?
They've
got
some
great
primitives
they've
already
demoed
this
conference.
So
you
can
you
don't
have
to
have
a
homogeneous
clusters
for
deployment
and
that's
the
target
we
were
going
for.
A
A
A
Reason,
yeah
and
I
think
they
they,
unfortunately
do
play
together
and
one
of
the
big
problems
with
kubernetes
Federation
is
it
doesn't
solve
the
problems
we're
trying
to
solve.
With
this,
you
multi
question
work
and-
and
at
least
I
think
that's
a
lot
of
what
people
want
to
solve.
Right
like
I,
want
to
be
able
to
move
an
application
between
different
clusters
and
have
all
that
work
and
and
I
guess
they
we're
gonna
solve
it
at
the
service
level,
but
not
it
like
I'll
set
on
the
whole
time.
There
was
another
comment,
question.
A
Right
so
I
think
there's
two
there's
basically
two
models
that
could
work
here
right
for
for
both
pile
and
a
mixer,
so
there's
two
models
that
would
work
one
model.
Is
you
run
that
in
every
single
cluster
and
they
federate
and
that's
what
I'm?
That's
what
when
I
was
saying
Federation?
That's
what
I
meant
not
good
news
Federation,
but
like
our
Federation
of
the
pilots,
each
pilot
or
each
mixer
only
knows
stuff
locally.
A
It
doesn't
know
about
everything,
it
just
knows
its
own
stuff,
and
it
knows
about
the
other
instance,
the
not
the
other
instances
of
itself,
but
the
things
that's
federated
with
right.
So
you
have
a
pilot
in
cluster
a
it
knows
all
the
stuff
in
cluster
a
if
a
pilot
cluster
B
knows
everything
in
B
and
they
talk
to
each
other
and
they
exchange
information.
So
that's
one
model.
A
Another
model
is
you
just
say:
I'm
only
running
one
kind
of
logical,
mixer
or
one
kind
of
logical
pilot,
but
I
run
an
H,
a
in
multiple
availability
zones,
so
I'm
running
it
in
multiple
clusters,
but
it's
still
kind
of
the
same
instance
right.
Each
of
those
has
full
information
about
everything.
Each
of
them
is
talking
to
everything
they're
not
talking
to
each
other
they're
all
getting
the
information
so
those
either
those
models
would
work
and
I.
Think
we're
trying
to
figure
out
which
model
is
better
and
I.
A
A
Yeah,
so
you
say:
I'm
gonna
run
pilot
in
these
five
clusters,
but
every
one
of
those
is
kind
of
a
mean
master
pilot,
yeah,
yeah
and
I.
Think
that
works
so
I
think
that's
probably
where
we'll
end
up.
At
least
that
seems
like
where
the
conversations
have
been
going
as
far
as
I've
been
hearing
it.
So
my
mixers,
maybe
a
little
trickier,
actually
mixer,
there's
a
bug,
maybe
you'd
say
in
the
multi
cluster
support,
because
mixer
right
now
actually
gets
information
from
the
cluster.
A
A
Well,
it's
a
separate
clusters
in
the
same
mesh
right,
so
not
not
different
meshes
but
yeah
the
same
mesh
so
I
mean
that's
like
yeah.
It
depends
on
what
you're
doing,
but
I
think
the
the
first
thing
we
need
to
fix
is
making
sure
that
you
can
run.
You
could
run
it
in
every
single
one,
but
it's
still
it's
all
kind
of
one
logical
one,
it's
all
the
main
and
then
we
can
and
then
it's
ideally.
If
we
do
that
right,
then
it's
just
up
to
the
user
right.
A
It's
up
to
you
to
decide
how
many
clusters
you
want
to
run
it
and
you
can
run
it
in
all
of
them.
You
can
run
in
some
of
them.
We're
actually
already
looking
at
this
per
node
a
per
node
option
right
for
mixer.
So
you
could.
You
could
scale
from
like
I'm
running
a
single
instance:
I'm
mixer
across
my
entire
mesh
to
I'm
running
a
mixer
on
every
single
node
in
my
mesh,
and
we
can
make
that
all
work
everywhere
and
then,
if
I
make
that
work,
then
I'm
like.
G
G
C
I
think
that's
the
big
problem,
because
when
we
started
mixer
mixer
right
now,
isn't
it's
fairly
easy
for
us
to
do
what
we
did
the
pilot
to
mixer
so
that,
like
as
friends
since
friend
said
the
you
know,
some
of
the
requests
and
stuff
aren't
properly
map
fixing.
That
problem
is
easy,
but
the
reason
we
didn't
do
that,
as
obviously
is
because
of
exactly
some
of
the
reasons
you
said.
There's
a
lot
of
there's
a
lot
latency.
A
Yeah
so
I
think
taking
the
two
parts
of
that.
The
first
part,
that's
probably
true,
I
think
there
is
a
scale
point
where,
maybe
maybe
you
don't
run
it
everywhere
like
internally,
we
have
an
equivalent
of
mixer.
We
run
it
in
every
single
sort
of
zone
availability
zone,
basically,
but
not
in
every
single
cell,
every
single
cluster.
A
But
that's
because
we
have
so
many
right,
there's
so
many
different
clusters,
so
I
think
that's
did
you
get
to
like
if
you
have
a
hundred
thousand
clusters
right,
like
you
have
like
five
in
the
same
building,
you
know,
do
you
really
need
a
mixer?
Never
one!
Maybe
you
do.
Maybe
we
make
it
easier,
I,
don't
know,
I
think
the
second
part
is
probably
really
something
we
need
to
think
about
more
just
how
do
we
deal
with
in
general
adaptors?
A
So
the
the
one
of
the
models
for
mixer
that
we've
been
pushing
people
towards
is
that
adapters
should
new
work.
They
should
mostly
talk
to
a
back-end
that
does
the
work,
but
then
you're
punting
the
problem
to
the
back-end.
So
now
the
backend
has
to
deal
with
the
fact
that
okay,
it's
running
behind
mixers,
running
in
like
ten
different
clusters
right.
So,
if
like,
if
it's
Prometheus,
do
you
have
all
those
mixers
talk
to
one
Prometheus
in
one
cluster?
A
If
you
run
in
Prometheus
in
every
cluster
and
have
them
all
talk
and
then
I
don't
know
what?
And
then
you
have
to
have
solved
this
yeah,
you
punted
the
problem
right.
We
push
the
problem
to
meet
this
quotas,
quotas,
harder
yeah.
So
if
you're
running
quota
in
all,
then
you
have
to
have
a
globally
scaled
quota.
C
A
Yeah,
so
identity
I
think
we
have
yeah.
We
have
a
pretty
good
model
there.
The
authorization
is
where
the
check
hole
would
come
in
and
that's
fine,
because
that's
a
local
decision
where
it
should
be
a
local
decision.
So
again,
it's
only
if
you're
talking
to
the
so
quota
is
a
great
example.
Quota
is
a
lot
harder,
especially
if
you
want
global
quotas.
A
It's
an
interesting
idea:
yeah
yeah
well,
I
think
some
of
them,
if
you
only
run
so,
if
you
do
they'll
like
I'm
running
mixer
in
one
cluster
I
have
ten
clusters,
but
I
only
run
it
in
the
one
I
think
most
things
would
work
and
then
yeah
the
question
is
okay.
So
if
you
running
in
a
multi
yeah,
we
actually
already
have
that
problem
full
quota.
Because
of
so
the
initial
quote.
Implementation
only
worked
if
you
had
one
instance
of
mixer,
not
just
one
cluster
but
I,
did
we
push
out
a
new
one?
A
Anyone
here
now,
if
we
we
were
building
a
new
one,
but
then
we
stopped
and
yeah
I,
don't
know,
I,
don't
know
the
card
state
yeah
it
doesn't.
It
doesn't
seem
like
there's
a
good
quota,
open
source
code
implementation
out
there
for
us
to
use
but
or
more
things
to
talk
about
or
know.
What's
what's
on,
the
roadmap
yeah.
D
C
A
C
Yes,
there's
a
couple
projects,
there's
something
called
zero
VPN,
which
was
a
design
spec.
That
was
attempting
to
address
that
and
we
haven't
gotten
too
far
on
implementing
it.
But
it's
up
there.
It's
you
know
it's
on
the
east.
You
know,
you
know
the
public
drives
that
you
can
go,
take
a
look
and
but
yeah.
We
would
clearly
like
to
solve
that.
Yeah.
A
And
it's
the
that
work
is
actually
helped
by
a
lot
of
work
happening
in
the
networking
working
group
around
the
new
api's.
So
there's
a
bunch
of
new
and
networking
API
is
one
of
those.
Is
this
gateway
thing
and
we
think
we
can
repurpose
a
lot
of
the
code,
we're
writing
for
gateway
to
set
up
a
tunnel
between
different?
A
Basically,
it's
a
VPN,
but
don't
call
it
VPN
between
different
clusters
and
have
so
gateways
talking
to
each
other,
and
then
that
bridge
is
it
and
then
you
can
bridge
them
and
then
the
problem
is,
you
have
to
be
able
to
tell
so,
if
you're,
using
a
single
pilot,
you
have
to
tell
that
pilot
which
network
you're
part
of
so
it
can
segregate
IPs
by
network
and
that
it
can
tell
you.
Oh
ok,
you're
a
network,
a
the
thing
you're
trying
to
talk
to
is
in
network
B
at
some
IP
address.
A
Okay
you're
actually
have
to
go
to
the
Gateway
first
to
get
through
it
and
yeah.
So
setting
that
up
is
interesting,
but
we
can
make
it
work.
That's
that's!
There's
a
bunch
of
options
for
that
in
the
dock.
In
the
dock
we
lists
probably
four
or
five
different
options
for
how
to
do
that.
You
can
do
it
and
end
using
s
and
I
sniffing
throughout
the
traffic.
There's
some
other
options
where
you
don't
do
an
event,
but
I
think
end
end
is
preferred.
H
A
Well,
so
it's
you
tunnel
through
that
gateway.
It's
not
right!
So
it's
not
it's
not
a
traditional
VPN
right.
It
doesn't
look
like
you're
on
one
network.
It's
just
it
has
the
effective
of
VPN,
where
things
can
talk
to
you
Taylor,
but
yeah.
So
you
bounce
through
the
Gateway
right,
so
so
VPNs
try
to
make
this
sort
of
hidden
right,
but
because
we
control
the
end
with
the
envoys,
we
don't
have
to
actually
hide
that
fact
right.
We
can
avoid
well
to
the
application.
It
looks
like
a
VPN
to
the
mesh.
B
Do
you
know
if
the
any
of
the
solutions
preserve,
so
you
know
in
the
load
balancing
that
envoy
does
right?
Is
it
or
you
know
it
makes
the
decision
at
the
sender
of
the
endpoint
it
wants
to
hit,
and
it
does
that,
based
on
all
the
properties
of
the
pod
right,
not
anything
else,
so
it
if
it
goes
to
through
the
Gateway.
Is
that
decision
going
to
be
preserved
by
this?
Any
of
this
approaches
that
we're
talking
about
yeah.
A
I
think
that
so
I,
yes,
we
have
to
solve
that.
Absolutely
and
we've
talked
a
lot
about
it.
I
don't
know
if
there's
been
a
decision
just
because
we
haven't
pushed
on
too
much,
but
there's
there's
a
bunch
of
ideas
for
how
to
solve
that.
So,
yes,
you'd,
like
pass
some
of
the
metadata,
so
you
have
you're
calling
a
gateway
but
you're
telling
it.
You
know.
I'm
actually
I
actually
want
to
talk
to
this
pod
right,
so
the
homeboy
can
communicate.
A
Who
would
actually
wants
to
talk
to
at
the
other
end
and
then
the
Gateway
can
forward
it
appropriately.
So
you
could
do
that
using
so
using
sni.
You
would
basically
have
kind
of
a
mangled
domain
as
the
destination
and
the
Gateway
would
sort
of
unpack
that
and
change
it
on
the
next
hop
or
whatever
so
yeah.
There's
there's
ways
to
do
it
to
get
that
end.
End
behavior.
So
then
you
get
like
you're,
basically
multiplexing
through
the
Gateway
I.
Think
in
there.
I
J
Yeah
the
principles
I
mean
that's
the
different
topic
or
mesh
expansion.
Sort
of
multi
cluster
principles
are
pretty
much
the
same.
You
have
something
like
the
components
of
a
sidecar
running
on
your
virtual
machine,
basically
the
sto
proxy,
which
runs
on
wire
underneath
and
then
you
export
all
your
certificates
and
there's
a
lot
of
manual
things
to
be
done
right
now,
where
you
need
to
populate
your
route
to
go
to
that,
and
then
you
need
to
populate
or
you
need
to
tell
it
what
the
end
points
look
like.
J
Then,
where
is
my
letter
and
where
is
mixer
and
yeah
beyond
that
it
just
sidecar
on
the
Virtual
side,
Carano
VM
and
the
real
sidecar
on
your
pilot,
they're
gonna,
make
a
connection
and
the
traffic
will
be
single.
I
mean
you
still
need
to
manually
register
at
the
end
point
using
sto
CTL
on
the
kubernetes
side.
But
after
all,
these
steps
are
done.
It
looks
like
it's
just
another
end
point:
it's
like
a
headless
important.
E
On
that,
can
you
deliver
something
that
will
set
the
IP
tables
just
like
it
would
do
on
containers?
Because
one
of
the
concerns
we
have
is:
oh
hey:
now
we
have
this
VM
and
we
add
a
sidecar.
We
had
to
use
to
have
end-to-end
TLS,
and
now
we
put
a
sidecar
on
a
VM
and
suddenly
we
have
well
no
TLS
in
between
and
use
a
beauty,
boss
and
stuff
to
prevent
things
flowing
out.
Can
we
get
that
does
well,
or
should
we
do
that
manually
as
well?
So.
A
I
think
the
yeah-
this
is
all
about
mesh
expansion
stuff,
but
and
in
just
BM
support
in
general
right
now,
it's
pretty
manual
I
think
we
provide
some
scripts
to
make
it
easier.
I
want
to
invest
a
lot
more
in
making
that
better
I
think
it's
really
important
to
make
that
easier.
We
kind
of
got
it.
We
got
it
so
cheap
in
kubernetes
because
of
the
way
things
are
set
up,
but
it's
like.
Oh,
if
you
want
to
do
this
on
the
VM,
you
know
run
this
script.
A
J
A
Went
to
a
talk
actually
before
this
about
psyllium
I,
don't
know
if
anyone
here
was
there,
but
they
had
a
their
talk
was
about
how
you
can
get
rid
of
IP
tables
and
use
something
better
much
faster.
So
if
there's
there's
some
really
cool
stuff,
we
can
do
there
too,
there's
also
the
BPP
stuff
that
should
be
possible.
Do
this
so
there's
a
lot
of
really
interesting
things
coming
down.
That
will
make
the
whole
sto
mesh
a
lot
faster
and
also
more
secure
and
also
simpler
to
set
up
so
yeah.
F
C
So
in
fact,
the
one
of
the
key
things
to
implement
like
the
clusters.
We
need
to
know
what
clusters
you
want
to
connect
and
we
started
from
using
signal
to
cluster
work,
but
at
the
time
that
we
attempted
to
do
this,
I
don't
think
they
were
quite
ready.
I
think
the
goal
was
that
we
would
do.
That
would
be
an
API
call
with
the
eight
pilot
would
call
the
cluster
registry
and
say
give
me
give
me
which
clusters
you
want
to
make
a
mess
out
of,
and
it
could
be
multiple.
Okay.
C
We
didn't
get
there
so
right
now
what
we
did,
what
you
see
now
in
the
qur'an
sto.
Is
you
see
flags
that
say
cluster
registry,
but
it's
just
a
file.
That's
was,
and
our
best
guess
of
the
format
of
the
cluster
is
respect
at
the
time
that
we
implemented
it.
But
that's
you
know
the
division
here
would
be
that
we
would
use
that
work
and
we
would
drive
pilot
from
the
cluster
registry
yeah.
Yes,.
F
C
A
I
spent
a
fair
amount
of
time
talking
to
the
at
least
at
Google,
delete
the
leads
of
the
multi
question
work,
and
so
we're
we're
pretty
much
in
st.
got
a
lot
of
this
stuff.
They
know
what
we're
doing.
We
know
what
they're
doing
and
it's
yeah
over
time.
More
things
will
converge.
There's
a
bunch
of
interesting
questions
like
there's
the
multi
cluster
ingress
work
and
I
still
can
kind
of
do
the
same
thing,
and
so
there's
like
how
many
of
these
things?
The
big
question
is
how
many
of
the
things
that
we
can
do.
A
Do
they
build,
or
do
they
just
say
well,
this
do
just
does
that.
Let's
just
tell
people
to
use
its
do
right,
so
that
I
think
that's
an
open,
an
open
conversation
and
some
of
its
around
you
know.
Sp
wasn't
ready
and
like
they're,
not
ready,
and
so
it's
it's.
We
don't
want
to
slow
everything
down
by
forcing
an
alignment
early,
but
I
think
everyone
is
interested
in
having
one
good
story
right
or
not.
We're
not
telling
people
confusing
things.
G
Availability
zones
is
running
the
same
application
in
a
fault-tolerant
way
or
maybe
even
you
might
think
of
it
as
n
copies
of
the
application
in
the
fall
time
way
where
every
cluster
has
the
same
set
of
workloads
running
in
it
and
I'm
curious
like
if
I
could
maybe
pull
the
room
or
something
is
there
a
particular
one
of
those
two
that
you're
more
interested
in,
because
I
think
that
they
need
a
lot
of
different
features
for
misdeal?
Yes,.
A
A
G
Yes,
so
I
mean
mostly,
the
thing
I'm
interested
in
is
whether
individual
services
are
likely
to
be
backed
by
sets
of
pods
running
in
multiple
clusters
or
whether
each
service
is
likely
to
live
in
one
place,
because
things
like
load
balancing,
get
more
complicated.
For
example,
you
know
I
think
right
now
we
don't
quite
yet
have
Geographic
aware
or
balancing,
for
example,
if
you
want
to
run
something
in
multiple
places,
you
need
that
or
you
need
to
think
really
hard
about
your
monitoring
to
try
and
understand
Geographic
monitoring
of
a
service.
G
A
G
I
mean,
for
example,
a
common
way
to
deploy
things.
Maybe
that's
just
because
this
do
is
not
very
popular,
but
a
common
way
to
deploy
things
today
would
be.
You
have
front
ends
that
are
sort
of
globally
load
balanced,
and
then
your
your
service
within
a
cluster
is
configured
to
only
talk
to
Atkins,
with
a
pestle.
A
G
A
No
III,
don't
think
so.
I
think
people
will
move
to
the
the
model
we
do
at
Google
right
is
that
you
prefer
a
local,
but
if
local
is
loaded
and
some
other
cluster
nearby
is
not
loaded,
you
fall
over
there
right.
You
call
that,
and
that
has
a
lot
of
benefits.
Obviously
you
have
cascading
failure
just
because
of
that,
but
there's
a
lot
of
benefits
to
that
too.
So
I
I
think
people
will
get
there.
A
I
don't
think
they'll
stay
and
you
like
each
cluster
is
an
isolated
cluster
and
Cooper
neighs
Federation
is
kind
of
targeting
this
a
little
bit.
Also
right
in
terms
of
deploying
to
these
the
models
that
you
deploy
the
same
thing
across
multiple
clusters,
so
I
think
there's
definitely
an
appetite
for
it.
You
had
a
comment
or
a
question.
My
area.
A
So
I
think
there's
a
and
you
can
correct
me
if
I'm
wrong
here
too,
on
this
there's,
there's
sort
of
two
models
for
how
to
do
this.
One
model
is
I
think
that
the
I
don't
think
all
this
hack
or
not,
but
the
way
we
set
it
up
now
was
that
you
basically
copied
the
key
material
and
have
it
basically
like
as
distributed
CA,
so
you
have
a
distributed
root
CA
in
some
sense,
another
model
is
to
actually
have
a
real
root,
CA
and
then
have
intermediate
CAS
and
the
different
clusters.
A
I
think
both
of
those
can
work.
Probably
the
intermediate
is
more
of
the
right
way
to
do
things,
I'm,
not
sure.
If
we
have
that
all
set
up.
Do
you
know
the
current
state
of
that
yeah?
So
it's
that
the
security
team
is
working
on
that,
but
it's
both
work
and
they
know
about
that
and
they're
working
on
it
but
yeah.
The
model
is
that
you,
you
just
said
where
your
root
CA
is
you
have
your
root
set
up,
and
then
you
have
sort
of
cluster
CAS
that
are
intermediate
and
then
they
all
share.
A
A
Yeah,
so
the
question
is
scaling
problems
right,
like
scaling
as
the
number
of
things
in
the
system
grow,
so
I
think
we
were
somewhat
confident
that
pilot
doesn't
scale
pretty
high
I,
think
that
would
be
where
well.
So,
even
if
you
switch
between
the
federated
model
and
like
global
model
that
doesn't
help
much
because
in
both
of
those
model
models
pilot
is
supposed
to
keep
the
entire
the
entire
mesh
and
I'm
really.
But
at
what
point
does
that
fall
down?
And
if
it's
you
know
a
million
endpoints?
A
C
Yeah,
what's
respect
to
pilot
this,
give
some
of
the
scaling
that
we
were
seeing
and
we're
trying
to
improve
are
less
about.
You
know
multi
cluster
and
how
many
clusters,
but
more
about
the
number
of
endpoints,
how
much
you
have
changed?
How
what
kind
of
protocol
you
run
a
v1
versus
V
to
envoy
those
kind
of
things,
so
the
scale
stress
points
so
far,
don't
seem
to
be
the
number
of
clusters.
It's
the
number
of
the
other
resources,
yeah.
A
So,
like
one
big
cluster
versus
a
bunch
of
small
clusters
doesn't
make
much
difference
here
right,
it's
just
the
the
total
mesh
size
that
matters
yeah
yeah
we're
actually
out
of
time
if
people,
but
we're
happy
I'm
happy
to
keep
talking
I've
nowhere
to
be
so,
we
can
keep
taking
questions
comments,
thoughts,
anything
else.
We
should
ask
the
audience
before
they
all
leave.
A
A
I'm
I'm,
like
crazily,
enthusiastic
and
over
excited
about
this
area,
just
cuz,
it's
like
it's,
it's
something!
That's
so
cool
once
you
get
used
to
it
and
it's
like
getting
people
to
understand
like
you.
Can
you
don't
have
to
worry
about
you
know
getting
everything
set
up
the
same.
You
can
run
things
in
different
laces.
You
can
migrate
them.
You
can
move
them
around.
Like
oh
I
need
to
spin
up
a
new
thing
in
any
region.
You
just
spin
it
up
and
it
works
right
like
that
kind
of
that
level
of
just
awesomeness.
D
A
Yes,
yes,
you
do.
If
you
use
the
same
name,
we
talked
a
lot
actually
early
when
we
were
starting
to
design
this
stuff.
Originally
we
were
to
do
all
this
aliasing
and
like
like
you'd,
have
a
mesh
name
for
things
versus
the
local
name
and
those
can
be
different.
We
may
have
to
go
there
at
some
point
where,
like,
if
you
have,
if
for
some
reason,
you
named
the
same
thing
in
different
clusters,
different
things
but
the
same
name,
and
then
you
want
to
merge
them
like
yeah,
so
we
haven't
really
tackled
that.
A
A
A
C
C
C
A
B
A
To
rename
things
in
the
mesh-
and
so
you
can
use
that
too,
as
the
mechanism
it
so
we
could
say,
oh
only
if
it
has
that
annotation
giving
it
a
name,
is
it
exposed?
Another
thing
would
be.
You
could
do
the
opposite
where
you
like.
Maybe
on
the
previous
example,
you
say
annotate
this
as
a
closed
cluster
local
resource
of
some
sort,
a
cluster
local
service,
and
then
it
has
a
cluster.
It
has
no
mesh
name
or
maybe
it
has
in
the
mesh
name.
It's
actually.
A
A
A
So
pilot
would
tell
it
so
once
we
have
that
part
pilot
will
be
telling
envoy
hey
to
talk
to
this
service,
there's
actually
five
IPS
behind
this
gateway
right.
So
we
usually
give
it
a
wait.
So
it
knows
how
many,
how
to
load
balance
to
it.
You
need
to
give
it
the
region,
but
then
you
also
tell
it
yeah
it's
going
through
the
Gateway.
Here's
the
gateway
happy
know
all
the
homeboys
would
know
that
information,
so
pilot
programs,
all
the
homeboys
and
then
homeboy
would
be
up
would
say
so.
A
I'm
going
knows
to
talk
to
service
acts.
I
have
three
IPS
locally
I
have
another
five
behind
this
gateway
and
another
six
behind
this
other
gateway,
and
then
you
know
prefer
local
depending
on
the
configuration
and
then,
if
that
local
is
overloaded,
it
would
try
to
find
the
closest
one
to
talk
to
you
right
and
then
it
would
load
that
based
on
how
much
load
it
can
take
and
that's
based
on
how
much
is
behind
it.
A
D
A
Yeah,
do
you
know
anyone
no
Lotus
latest
status
on
that?
Is
it
back
in
it's
optional,
though
right
yeah,
yeah,
yeah
and
that's
the
way
it
should
be
and
I
think
yeah
for
so
I
assume
and
hope
that
it
will
work
the
same
with
zero
VPN
you
can.
You
could
require
it
to
be
that
like
I,
guess,
that's
three
hops
or
you
could
support
two
hops
right.
So
if
you
have,
if
you
have
connectivity
to
the
Gateway
on
the
other
side,
you
could
go
directly
where
you
could
go
through
your
egress.
A
You
could
also
yeah.
It
depends.
You
could
do
something.
I,
don't
know
you
might
be
able
to
do
something
where
the
egress
is
pearl
or
the
gateways
are
on
each
other's
network.
Somehow
and
then
you'd
actually
have
one
hop
in
it.
I
don't
know,
there's
lots
of
things
you
can
do
to
make
this
reduce
hops,
but
I
think
the
theory
is
if
you're
going
across
cluster
you're,
probably
paying
a
higher
price
anyway.
So
cool
yeah
last
questions
now
we're
ready
10
minutes
over.
Thank
you
guys
all
for
coming
this
one.