►
From YouTube: Deep Dive: Linkerd - Zahari Dichev, Buoyant, Inc.
Description
Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Deep Dive: Linkerd - Zahari Dichev, Buoyant, Inc.
Linkerd 2.8 introduced new multi-cluster mTLS functionality that can be deployed to securely bridge cross-cluster communication. This deep dive session will be presented by Zahari Dichev, the engineer at Buoyant who led the development of this feature. In this session, he will explain how the feature works, trade-offs, and best-practices to consider as you roll it out to production. He will also review Linkerd's upcoming roadmap, highlighting opportunities for new contributors to get involved in the project.
https://sched.co/cpBH
A
So
the
agenda
for
this
talk
is
we're
going
to
look
at
the
what
service
meshes
and
why
you
might
need
one
we're
going
to
cover
the
multi-cluster
concepts
that
govern
the
implementation
of
this
feature.
We're
going
to
look
at
the
general
architecture
of
multicluster,
then
we're
going
to
do
a
quick
demo
showing
how
this
feature
feels
in
the
terminal
and
what
you
can
do
with
it,
and
then
we're
going
to
look
at
the
life
of
a
request
across
clusters.
A
So,
let's
get
started,
a
lot
has
been
said
about
service
meshes
and
in
fact
I
think
the
best
way
to
describe
it
is
through
a
quote
that
I
read
recently
on
our
blog,
namely
you
might
think
that
when
you
have
a
distributed
system,
the
most
important
thing
is
that
amb
can
exchange
packets
and
that's
true.
It
is
important.
However,
as
your
system
grows
and
becomes
more
complex,
you
need
to
be
able
to
provide
more
guarantees
and
what
linkrd
gives
you
is
the
following.
A
A
So
this
is
a
pretty
concise
quote,
but
I
think
it
captures
the
essence
of,
and
it
goes
and
you
know
in
a
typical
distributed
system,
you
might
have
an
architecture
that
looks
like
this.
You
have
a
client
that
communicates
with
an
api
gateway
that
happens
to
live
in
a
kubernetes
cluster,
this
api
gateway,
then
routes
requests
to
a
number
of
internal
services,
and
these
services
might
talk
to
a
database
or
a
redis
cache,
or
for
that
matter
they
might
be
talking
to
a
third
party
api.
A
A
Its
purpose
is
to
intercept
the
prop
the
traffic
that
goes
in
and
out
of
the
pot.
So
instead
of
your
usual
traffic
pattern,
now
your
traffic
looks
a
bit
like
this.
Everything
goes
through
the
proxy.
This
proxy
communicates
with
a
set
of
components
that
we
collectively
refer
to
as
the
control
plane
through
through
interface
and
that
control
plane
runs
alongside
in
in
in
its
separate
name
space.
So
this
control
plane
gives
you
a
few
things.
It
provides
tls
certificates
for
the
proxy.
A
A
So
it
exposes
an
endpoint
that
provides
you
automatic,
prometheus
metrics
for
http
and
tcp
traffic.
So
now
your
permittee
assistances
can
scrape
this
endpoint
and
give
you
a
bunch
of
useful
traffic
traffic
information
for
all
the
traffic
that
goes
through
the
proxy.
It
provides
latency
aware
layer,
7,
load,
balancing,
as
well
as
automatic
tls
and
an
on-demand
diagnostic
tap
api,
which
purposes
to
allow
you
to
look
at
life
requests
in
the
live
traffic.
A
That's
going
through
your
proxy,
so
it's
probably
normal
to
ask:
how
does
that
proxy
actually
end
up
in
your
parts?
Well,
that's
done
through
something
that
we
refer
to
as
injection,
and
it's
in
this
process
is
usually
accomplished
by
the
proxy
injector
component,
which
is
part
of
the
control
plane,
but
it
can
also
be
done
manually,
so
this
actually
puts
two
containers
in
your
pots,
an
init
container
and
the
proxy
container.
So
this
is
best
represented
with
with
the
diagram.
A
What
happens
upon
injection
is
that
we
add
an
init
container
and
when
that
runs,
it
sets
up
a
bunch
of
iptables
rules
that
are
responsible
for
routing
traffic
to
the
pro
for
making
the
proxy
intercept
all
the
traffic,
and
we
also
add
the
proxy.
So
now
all
your
aiming
incoming
traffic
goes
through
the
proxy
first
and
then
to
the
application
container
and
then,
similarly,
all
of
your
outgoing
traffic
leaves
the
application
goes
through
the
proxy
and
then
to
where
it's
destined.
A
So
some
of
the
core
concepts
of
linker
d
are
observability,
which
is
collecting
actionable
traffic
metrics,
allowing
you
to
pinpoint
problems
and
understand
the
dynamics
of
your
traffic
security,
encrypting
traffic
between
services
automatically
reliability
as
well
as
traffic
management.
So
you
can
use
advanced
patterns
like
traffic
splitting
and
and
all
sorts
of
things,
and
when
we
were
thinking
about
with
multi-cluster,
we
were
wondering:
how
can
we
have
these
guarantees
and
expand
them
to
span
across
clusters?
A
And,
first
of
all,
maybe
it's
worth
asking
why
multiple
clusters?
Well,
there
is
a
number
of
use
cases,
but
but
they
mostly
come
down
to
traffic
migration
across
clusters.
So
this
this
includes
calorie
deployments,
different
environments
that
allow
you
that
allow
your
developers,
for
example,
to
split
traffic
across
the
development
and
production
environment
and
use
the
production
components
while
developing
locally
as
well
as
failover.
A
So
a
lot
of
people
actually
do
a
failover
across
clusters
in
different
availability
zones
and
that
sort
of
thing,
so
we
tried
to
build
upon
the
link
or
d
concepts
and
have
our
multi-cluster
implementation,
be
governed
by
by
other
core
concepts
that
expand
upon
the
linkery
ones.
So
our
implementation
is
secure.
Everything
happens
over
pls,
even
traffic
across
clusters,
it's
kubernetes
first,
so
remote
services
appear
as
kubernetes
services.
There
is
no
special
kind
of
service.
A
A
This
implementation
is
transparent,
so
applications
don't
need
to
know
whether
a
service
is
remote
or
local
and
of
course,
this
implementation
is
also
network
independent.
So,
as
long
as
you
have
connectivity
between
clusters,
then
the
underlying
network
hierarchy
doesn't
doesn't
actually
matter
at
all.
A
A
These
two
clusters
need
to
have
linkard
installed
and
it's
important
to
know
that
linkerd
needs
to
be
installed
with
a
trustroot
that
with
trust
routes,
that's
that
is
shared.
So
this
enables
automatic
tls
across
across
clusters,
and
then
you
have
the
linker
d
multi-cluster
components
which
consist
of
the
service,
mirror
the
gateway
and
cluster
credentials.
A
And
then
you
have
your
local
services
that
might
have
pots
or
other
pots
behind
them.
And
then
you
have
the
remote
services
that
are
effective,
proxies
for
services
that
live
on
the
target
cluster,
so
the
service
mirror
controller
is
actually
responsible
for
monitoring
the
export
state
of
the
target
cluster
and
replicating
it.
This
is
done
through
using
the
clango
and
kubernetes
informers
in
order
to
continuously
monitor
the
state
of
the
services
on
the
on
the
on
the
target
cluster.
A
Then
we
have
the
gateway
component,
which
is
responsible
that
that
usually
lives
on
the
target
cluster
and
is
responsible
for
routing
requests
that
come
in
and
you
know
effectively
taking
requests
on
one
port
and
routing
them
to
the
correct
service
to
the
correct
internal
service.
And
then
you
have
credentials
and
these
credentials
consist
of
two
pieces.
A
A
A
So
now
that
we
have
that
what
we
can
do
is
take
a
look
at
the
services
on
cluster
east.
Again
and
now
you
can
see
that,
apart
from
the
local
back-end
service,
we
have
a
back-end
service
dash
west,
and
this
is
a
service
that
is,
that
is
effectively
just
a
proxy
service
and
all
the
traffic
that
goes
through.
It's
gonna
be
routed
to
the
service
and
the
pods
that
live
on
cluster
west.
So
now
we
can
use
the
gateways
command.
A
A
So
we
have
a
response
from
cluster
west
and
we
have
a
response
for
from
cluster
east
as
well,
which
is
local
now.
The
interesting
part
is
that
we
can
actually
traffic
split
between
these
two
services
between
these
two
clusters
right.
So
we
have
this
traffic
split
resource
here
that
specifies
to
leave
services,
and
the
traffic
will
be
equally
split
between
between
these
two,
so
once
we
apply
it
and
we
get
into
our
pot
again
now
we
can
run
a.
A
So
you
might
wonder
how
is
that
actually
happening?
So
how
does
that?
How
does
that
actually
work?
Well,
if
we
go
back
to
our
presentation
yeah
there
is,
there
is,
let's
actually
look
at
a
the
life
of
this
curve
request,
as
it
goes
across
clusters.
Well,
we
have
our
client
port
right
and
this
clan
pot
fires.
A
It
lives
on
cluster,
east
and
fires.
Http
requests
to
backend
service
that,
for
the
purpose
of
this
of
this
request,
let's
say
that
this
gets
traffic
split
and
goes
to
cluster
west.
So
once
that
happens
once
this
request
leaves
the
pot
it's
going
to
back
in
service
west,
which
is
the
service
proxy
that
lives
on
our
cluster.
Now
we
do
this.
A
This
back-end
service
has
an
endpoint
associated
with
it,
that
is,
that
specifies
the
public,
ip
and
port
of
the
gateway
on
cluster
west,
and
because
this
request
goes
through
the
proxy,
we
can
be
a
bit
smarter
about
what
we
do
with
it.
So
once
this
request
hits
the
proxy
the
proxy
uses
the
lingerie
destination
component,
which
is
part
of
the
control
plane
and
issues
a
discovery
query
to
it.
Now
the
link
rd
destination
a
service
knows
that
this
is.
This
is
a
proxy
service.
A
So
it's
going
to
return
some
useful
information,
namely
that's
the
fully
qualified
domain
name
of
the
service
on
the
target
cluster
because
it
might
be
different.
You
know
you
might
have
different
cluster
domains,
for
example.
So
when
that
happens,
it's
also
going
to
return
you
the
expected
identity,
that
for
the
gateway,
so
you
need
to
hit
the
gateway
endpoint
on
the
other
side,
with
the
right
tls
identity
in
order
to
in
order
to
establish
mtls.
A
So
once
that
happens,
this
request
the
uri,
the
proxy
rewrites,
the
uri
authority,
with
the
correct
port
and
and
and
fully
qualified
domain
name,
and
when
this
request
leaves
the
pot
or
the
the
cluster.
In
fact,
it
hits
the
external
ip
of
the
gateway
on
the
other
on
the
other
cluster
on
cluster
west,
and
this
gateway
now
knows
which
port,
because
the
uri
authority
is
rewritten,
this
gateway
knows
which
port
and
which
service
this
request
is
destined
for,
so
it
sends
it
there.
A
It
roots
it
to
the
correct
service
and
it's
important
to
because
to
understand
that,
because
there
is,
there
are
proxies
running
in
each
box.
That's
involved
in
this
communication
pattern.
All
of
this
traffic
is
encrypted
even
traffic.
That's
that
that's
going
across
the
clusters,
so
this
is
how
it
pretty
much
works
under
the
hood
and
some
of
the
future
work
that
we're
planning
to
do
on
this
particular
feature
is
that
we've
realized
that
the
service
mirror
controller,
because
it
can
monitor
multiple
clusters
can
get
a
bit
complicated.
A
We
are
also
going
to
introduce
a
crd
to
better
represent
the
target
cluster
information,
because
currently
it's
represented
through
a
bunch
of
annotations
on
both
services
and
kubernetes
secrets.
Now
we're
going
to
consolidate
that
and
use
the
crd,
we
are
also
going
to
support
traffic
policy
and
final
grain
permission
control,
and
one
thing
that
we
are
currently
working
on
is
support
for
pcp
traffic.
So
currently
that
solution
works
with
http
traffic
and
we
want
to
be
able
to
support
justice
arbitration
speed
traffic.
A
A
A
So
yeah,
the
other
question
is
istio,
has
been
very
popular.
How
did
others
typically
choose
to
use
linkedin
instead
of
seo?
Well,
like
I'm
not
particularly
yeah,
I
I
don't
really
know
how
why
other
people
are
considering
linker
d
instead
of
ice
steel
or
how
they
are
using
it
differently
because
yeah,
I
don't
really
directly
work
with
a
lot
of
people,
but
I
would
say:
lingerie
is
quite
simple
and
it's
in
its
ux
and
getting
things
off
the
ground
is
really
simple.
So
maybe
that's
something
that
is
kind
of
attracting.
A
A
So
yeah
there's
another
question
that
the
data
plane
of
linkedin
is
written
in
rust
with
all
the
benefits
of
it.
But
could
this
be
a
reason
that
people
who
don't
have
internal
rust
developers
avoid
linker
d?
I
I
don't
think
that
this
could
actually
be
a
reason
because,
or
rather
I
don't
think
it
should
be
a
reason,
because
the
proxy
is
fairly
transparent
so
and
because
linker
is
like
very
configuration
light,
you
don't
really
need
to
know
a
lot
about
the
internal
workings
of
the
proxy
or
you
know
all
rust
for
that
matter.