►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Upgrading Your Service Mesh to Linkerd 2 - Tilen Faganel, Open Credo ltd.
Service meshes are advancing at an incredible rate. And there’s no sign of it slowing down. Unfortunately, sometimes that means breaking changes occur, which might make updates to the service mesh difficult and potentially disruptive. Particularly if you were an early adaptor. Linkerd 2 was a major update. So big, in fact, most of everything about it changed. Upgrading an existing Linkerd 1 deployment can be pretty daunting, as we need to redesign our service mesh architecture and configuration. All while ensuring the upgrade is performed successfully, reliably and without major disruptions. Come and join me as we will go through a journey of how we helped a client fully upgrade their service mesh from Linkerd 1. And how nobody noticed! We will explore the challenges and pitfalls of doing so, which can be applied to any upgrade of this caliber.
https://sched.co/Nrpp
A
Okay,
I
run
thanks
thanks
for
joining
me
today,
I
will
be
in
the
next
half
hour,
so
I'll
be
taking
taking
you
through
my
experience
with
one
of
our
clients,
of
how
we
basically
were
an
early
adopter
of
service
measures
and
half
an
hour
journey
as
it
went
through
linker,
d1
and
all
the
way
through
linker
d2,
and
what
how
that
looked
like
from
from
from
that
perspective,
so
quick
control.
So
my
name
is
Dylan
I'm,
a
senior
consultant
at
open
Credo.
A
We
are
based
in
London
UK
and
what
we
do
is
basically,
we
are
server
hands-on
consultancy
where
we
help.
You
know
our
clients,
all
sorts
of
sizes,
from
smaller
companies
to
big
enterprises
with
their
cloud
and
if
to
help
design
their
cloud
native
solutions
and
work
on
making
sure
the
designs
are
scalable
and
good
and
future-proof.
You
can
find
me
on
either
on
WeChat
or
on
on
Twitter,
via
the
handles
on
on
on
the
slides
are
shoulders
on
at
the
end
as
well.
So
what
did
he
agenda
for
today?
I'm
gonna.
A
You
know
without
much
issues,
basically
and
I'm,
going
to
talk
a
little
bit
a
few
a
little
bit
about
lessons
learned
on
what
could
we
do
better?
What
do
we
well
what
you
should
avoid
when
doing
so
in
Europe?
Ok,
let's
start
so
just
a
quick
note
of
what
is
a
service
mesh
I
will
find
many
definitions
online.
What
is
the
service
mesh,
but
when
our
life
on
is
that
it
service
is
an
approach
and
a
dedicated
infrastructure
layer
for
operating
and
secure
fast
and
reliable
Microcystis
ecosystem.
A
Now,
that's
a
loaded
statement
as
in
there's
a
lot
in
there
and
it's
sometimes
difficult
to
actually
grasp
what
it
is,
the
service
my
shares
and
what
it
is.
It
would
do
for
you
when
you're
developing
your
cloud
native,
highly
scalable
applications,
so
I've
heard
a
bunch
of
things
from
all
sorts
of
people
about
trans
Bay
to
me
what
it
is-
and
these
are
just
a
few
things
that
I
heard,
for
instance,
it's
a
whole
new
paradigm
of
deploying
stuff
that
cloud.
Ok,
maybe
it
is
a
low
latency
infrastructure
layer.
A
It
is
a
level
7
network
exclusively
for
applications.
For
instance,
it's
just
kubernetes
and
extension
to
kubernetes
I,
even
heard
someone
say
it's
magic
just
does
a
lot
of
stuff,
it's
not
quite
magic,
but
it
is
interesting.
Indeed.
Let
me
show
you
on
an
example
of
what
what
a
service
mesh
does
for
you,
just
a
quick
one
before
we
before
we
deep
dive
in
so
let's
say
we
have
two
services:
now
they
could
be
micro
services,
it
could
be
normal
services
it
for
the
most
part.
It
doesn't
really
matter
what
they
are.
A
For
instance,
let's
say
we
have
a
payment
service
and
we
have
ledger
service
and
that
connects
to
like
a
fourth
system
and
external
fraud
system,
a
third
party,
if
you
may
you
make
a
payment,
it
makes
send
you
the
ledger.
It
sends
the
info
to
the
fault
system
for
analysis.
Very
simple
example:
now
you
might
have
an
Alexa
all
those
are
connected
via
HTTP
or
any
sorts
of
other
network
protocol,
so.
A
I
figured
out
hey
as
ever.
Yes,
sir,
better
is
the
colors,
are
a
bit
off
cool.
So
three
simple
components:
are
you,
sir?
Your
system,
I,
will
typically
have
a
lot
more,
but
for
probably
the
sake,
let's
start
with
these
now
typically,
your
starts.
Some
of
your
components
will
start
scaling
up.
For
instance,
we
wanted
princess
is
more
more
intensive
for
a
ledger
service,
because
we
have
a
lot
more
load
on
it
and
we
needed
to
handle
it.
That
means
that
the
payment
is
still
now
needs
to
do
a
few
things.
A
First
of
all
needs
to
know
that
there
is
more
than
one
system
it
needs
to
load
balance
between
them.
Hey,
maybe
needs
to
handle
if
things
go
wrong,
for
instance,
what
if
one
of
the
instances
in
ledger
is
you
know
foes
in
a
bit
slow
down?
One
of
it
is
dead
completely.
How
do
you
deal
with
that
in
a
fast-paced
production
environment?
Well,
it
turns
out
just
doing
HTTP.
Request
is
a
lot
of
times
not
enough,
because
your
your
availability
will
go
down.
Your
error
rates
will
spike
up.
A
So
we
try
to
mitigate
those
by
doing
a
few
things.
For
instance,
what
we're
trying
to
do
is
well.
First
of
all,
we
want
it
to
be
discovering
instances.
We
want
to
be
dynamic,
so
we'll
implement
service
discovery,
so
we
don't
need
to
statically
set
IP
addresses
or
URLs
for
instances.
We
want
to
load
balance
between
them.
We
could
use
a
little
bouncer,
but
that's
an
extra
hop.
We
might
do
collage
mode
balancing,
for
instance,
it's
a
good
way
of
doing
a
bit
about
it.
A
We
want
the
circuit
backing
if
an
instance
is
dead
or
performs
badly
or
doesn't
perform
at
all.
We
want
to
stop
sending
messages
to
it,
potentially
with
a
fallback
that
is
local
to
the
service
itself.
Of
course,
you
may
want
to
implement
retries,
for
instance,
if
a
call
fails.
Maybe
if
I
try
it
again,
a
different
service,
you'll
succeed-
and
you
know
when
from
the
outside
looking
in
the
call
would
ultimately
be
fine
and
successful.
A
Even
though
there
was
a
retry
that
happened
internally,
how
do
we
do
with
authentication
authorization
between
the
services
a
lot
of
times?
You
just
don't
deal
with
it.
We
assume
that,
because
it's
internal
to
a
network,
it's
fine,
but
that
might
not
be
the
case
over
time,
especially
in
a
cloud
cloud
native
environment,
and
of
course
you
would
like
all
this
to
be
automatic
and
all
of
those
features
are
nowadays
usually
abbreviated
under
the
term
resiliency.
A
So
we
want
this
to
be
resilient,
so
we
want
the
communication
between
all
of
this
between
the
services
to
be
resilient
and
to
have
all
those
features
we
just
mentioned.
Now
we
could
implement
those
ourselves,
there's
a
bunch
of
libraries.
You
can
use
whether
for
Java
for
go
for
c-sharp,
whatever,
whatever
your
services
are
built-in,
you
can
have,
for
instance,
history
cause
a
popular
one
or
resiliency
for
Jay
is
a
new
one
now
and
that
server
supersedes
it.
A
We
could
use
kubernetes
for
service
discovery,
we
could
use
ribbon
for
client-side
load
balancing
and
we
can
use
some
custom
code
to
implement
retries
and
authentications
and
authorization
okay.
That
could
work,
but
you
know
with
doing
it
only
on
the
pavements.
It's
not
enough.
We're
gonna
have
to
do
it
everywhere.
We're
gonna
have
to
down
ledger
we're
gonna
have
to
do
it
in
all
of
our
services
at
the
same
time.
A
Now
what
if
we
don't
own
all
of
our
services?
What
if
our
ledger
is
now
instead
of
a
third
party
product
that
we
bought
from
someone
like
a
core
banking
product?
Well,
we
don't
have
access
the
code.
We
can't
have
anything
in
it.
Well
we're
out
of
luck
here.
We
can't
use
any
of
that
over
here.
So
what
do
we
do
in
this
case?
What
are
we
in
this
case
and
in
the
case
where
we
want
to
potentially
implement
a
new
thing
within
our
resiliency
stack,
so
to
speak?
Well,
we
need
to
go
to
all.
A
We
need
to
go
to
every
single
one
of
our
teams
in
our
organizations
and
put
a
ticket
or
at
or
a
feature
request
on
their
backlog
and
coordinate
everything,
and
it
might
take
months
for
everyone
to
get
on
board
with
new
feature
we
want
to,
and,
of
course
it
takes
away
from
from
developers,
productivity
and
so
on
so
forth.
So
we
want
to
take
that
burden
away
from
the
services
selves
and
instead
rip
it
out
and
put
it
in
a
in
a
proxy.
A
So,
instead
of
having
all
of
that
in
in
in
inside
a
citizen
code,
one
we
have
all
of
that
in
a
single
component,
a
single
proxy
and
the
port
that
next
to
to
our
components
and
just
route,
all
the
traffic
through
it
and
then
let
the
proxy
handle
all
of
the
resiliency
problems.
Let
the
proxy
handle
that
ocation,
let
the
proxy
handle
circuit,
breaking
load,
balancing
service
discovery.
A
Why
do
we
have
a
deputy
decayed
component
that
a
team
develops
or
maintains
that
does
all
of
this
for
everyone
automatically,
but
that
sounds
like
a
lot
of
a
much
better
idea.
Actually,
it
is
another
component
to
another
hop,
but
it
does
give
you
a
lot
of
features
that
you
would
need
to
build
manually
yourself
and
sudden
and
also
a
problem
of
third-party
vendors.
It
goes
away
completely.
A
So,
basically,
what
is
it
service
mesh
if
it
offers
all
of
these
features?
That
would
that
be
that
I
mentioned
before
service
discovery,
load,
balancing
circle
breaking
to
retry,
certification
authorization,
so
on
so
forth,
and
it's
automatic,
and
we
do
that
by
ripping
out
the
functionality
and
putting
in
a
proxy
and
then
creating
a
mesh
of
proxies
that
that
connects
all
the
services
together.
A
So
basically,
if
we
dumb
it
down
a
little
bit
even
more,
it's
just
a
collect
collective
of
small
smart
configurable
autonomous
proxy
and
because
that's
all
there
is
it's
not
any
more
complicated
than
that.
In
reality,
there
are
certain
complexities
to
how
you
implement
it,
but
generally
that
is
what
it
boils
down
to
so
proxy
which
which
one
to
use
did
you
see
nginx,
H,
a
proxy?
Well,
those
are
usually
not
built
to
handle
all
the
scenarios,
so
we'll
probably
go
after
a
few
general.
A
If
you
purposely
built
one
so
you'll
find
nowadays
you'll
find
quite
a
few
products.
Actually
that
solve
the
issues
that
we're
talking
about.
So,
for
instance,
Linkwood
is
a
popular
one.
Sto
is
another
popular
one.
You
could
use
council
for
your
cert
for
yours
as
a
service.
Mesh
envoi
is
usually
used
as
a
proxy
in
sto,
for
instance,
or
in
other
systems.
Even
have
Amazon's
managed,
app
mesh
or
even
kong's,
which
is
primarily
an
API
gateway,
now
moved
into
the
serious
mesh
space
as
well.
A
So
there's
a
lot
of
open
providers
that
are
established
and
a
lot
of
up-and-coming
ones
as
well,
but
let's
go
back
to
when
it
all
started
like
one
of
the
first
one
that
was
linker
D.
So
that's,
let's,
let's
take
a
journey
that
I
took
and
the
journey
that
the
general
community
of
service
meshes
took
of
how
we
came
up
from
where
we
started
to
where
we
are
right
now
so
link
ready.
One
was
one
of
the
first
service
measures,
I
suppose
service
matchboxes
out
there
and
what
it
was.
It
was
basically
a
single
app.
A
It
was
a
single
app
and
an
all-in-one
network.
Boxey
based
on
pinnacle
made
out
of
came
out
of
Twitter.
Basically
vinegar
was
a
part.
I
came
out
of
Twitter.
It
was
a
good
idea,
clients
that
support
a
lot
of
the
features
that
we
need
it
and
got
bundled
into
link
OD
as
a
way
of
dynamically
configuring
it
and
acting
as
a
proxy
on
a
mesh.
Now
it
runs
in
the
JVM,
which
may
or
may
not
be
ok,
but
in
this
case
it
was
okay
as
it
allows
agency.
Now
it
supports
writing
policies.
A
It
supports
most
of
the
resiliency
requirements
that
we
talked
about
before
it
is.
It
has
a
pluggable
design,
which
means
that
it
doesn't
make
any
assumption,
but
later
on,
so
you
can
run
it
on
a
box.
You
can
run
on
a
Raspberry,
Pi
I
ran
it
on
kubernetes,
whatever
you
want
and
you
can
plug
in
different
components
into
it.
For
instance,
service
discovery
into
kubernetes
or
service
discovery
into
console
or
somewhere
or
there's
a
lot
of
plugins.
You
can
take
and
has
a
single,
relatively
simple,
config
and
I'll
see
what
I
mean.
A
It
looks
something
like
this.
Basically
so,
for
instance,
on
the
left
just
an
example:
there's
things
called
D
taps,
which
is
a
bit
of
a
convoluted
way
of
configuring.
Routing,
for
instance,
request
comes
in
and
then
you
can
have
a
survey,
a
kind
of
a
table
of
a
path
of
branching
paths.
The
requests
can
take,
for
instance,
request
comes
in
and
I
need
to
forward
it
to
the
server
service.
A
Ok
sounds
like
a
box
you
were
talking
about.
We
do
need
to
learn
the
language,
so
that's
a
bit
of
a
pain,
but
we'll
do
what
ed
listen,
Ischl,
e
and,
of
course,
on
the
right.
For
instance,
that's
using
llamo.
We
can
configure
all
sorts
of
parameters.
For
instance,
do
we
want
to
have
retries
retries
budget?
Do
you
want
to
have
a
load
balancer?
What
type
of
all
bounds
do
you
want
to
use?
Do
we
want
to
use
latency
based
load,
balancing
or
just
round-robin,
on
balancing
and
so
on
so
forth?
A
Ok,
so
you
know
other
box.
It
sports
a
few
good
future
features.
So
how
do
we
use
it?
How
do
we
use
it?
So
one
of
our
first
use
cases
for
it-
and
this
is
where
really
sort
of
shine
is
where
we
weren't
using
kubernetes.
Yet
so
as
a
few
years
ago,
most
of
the
client
most
of
the
services
were
running
either
on
Prem
on
virtual
machines
and
some
of
them
we
started
moving
to
the
cloud
on,
but
on
normal
ec2
instances.
A
This
is
so
before
the
bigger
enterprises
became
comfortable
with
the
idea
of
running
everything
in
the
cloud
or
the
idea
of
running
everything
kubernetes.
So,
but
still
we
wanted
to
want
to
try
out.
We
wanted
to
bring
some
of
the
resiliency
features
that
sounded
so
promising
and
bring
into
what
we
have
right
now.
Instead
of
waiting
for
the
promised
land
later
down,
the
line
and
Linkwood
e
enables
to
do.
Linker
d1
specifically
enables
to
do
just
that,
because
it
was
just
a
normal
JVM
app.
A
We
just
run
it
on
the
box
next
to
the
service
itself.
Now
it
was
a
VM
or
an
ec2
instance,
or
a
vendor
app
or
any
other
cloud
app.
It
didn't
matter
it.
We
just
installed
it
in
there
and
it
sort
of
worked.
Now
we
had
to
you
know:
work
with
the
configuration
a
little
bit
had
to
make
sure
that
serve
all
them,
see
each
other
and
know
where
to
go,
but
for
the
most
part
this
worked
and
it
was
very
flexible
and
allows
us
to
deploy
it
in
a
hybrid
environment.
A
So
there
was
a
good
take
1.
We
were
very
at
this
point.
We
were
quite
happy
about
it
and
almost
nobody
was
doing
this
at
this
point.
So
it
was
a
big
step
forward
and
allows
us
to
to
bring
back
a
lot
of
that
into
the
mesh
itself
and
basically
release
the
burden
from
the
developers
themselves.
They
didn't
need
to
know
not
know,
but
this
didn't
necessarily
need
to
care
about
doing
all
that
stuff
for
every
one
of
their
micro
services
and
tweaking
every
one
of
their
Microsoft's
individually.
A
You
instead
had
a
single
unified
place
of
doing
that.
Ok
worked
out
for
a
while,
and
then
we
started
looking
at
kubernetes.
Well,
we
actually
start
once
using
it.
We
wanted
to
deploy
some
of
our
work
since
kubernetes.
How
do
we
plug
kubernetes
into
this?
Well,
as
you
probably
can
guess,
we'll?
Probably
an
have
a
kubernetes
cluster,
we're
gonna
have
to
put
a
link
ready
on
it
somehow
and
then
hook
it
up
into
this
mesh.
So
that's
straightforward,
but
it
was
a
problem.
Linka
d
is
not
light
on
resources.
A
No
recommendation
is
a
recommended
amount
of
heat
memories,
at
least
a
gig.
If
not
more,
you
can
easily
listen
to
the
CPU,
so
this
worked
fine
for
ec2
instance,
SL
and
virtual
machines,
because
usually
they
had
enough
overhead
to
handle
this,
but
for
containers
or
for
parts
better
running
it
as
a
sidecar.
It's
probably
not
the
best
idea
to
consume
way
too
many
resources
and
would
blow
to
our
basically
kubernetes
installation
and
make
it
slow
so
step.
A
We
do
it
something
like
this,
which
was
the
recommended
approach
at
that
time
anyway,
was
instead
of
running
it
as
a
sidecar
brand,
as
a
daemon
set
now
effectively
a
single
linker,
the
instance
would
handle
all
of
the
parts
on
a
single
note
and
then
all
the
communications
would
go
through
it
and
then
you
could.
It
would
communicate
between
itself
on
a
node
level
and
then
once
the
request
hits
a
target
note.
A
A
A
Now
again,
this
worked
fine
for
most
of
our
but
introduced
a
lot
of
problems
as
we'll
see
soon
enough,
the
Lister,
which
is
that
what
we
have
nice
and
great
dashboards
for,
for
instance,
our
our
server
or
Lincoln
instance,
like
wit,
services.
It
is
calling
what's
the
traffic
and
so
on.
You
only
got
a
dashboard
per
instance.
A
You
did
not
get
a
collective
dashboard
of
everything
that
was
happening
or
mesh,
which
means
it
was
quite
a
pain
and
actually
figuring
out
what
was
going
on,
because
you
have
to
go
to
every
instance
and
figure
out
where
the
traffic
is
coming
from,
where
it's
going
and
you
had
to,
and
if
you
had
like
hundreds
of
nodes,
it's
just
just
not
possible.
Now,
eventually,
yes,
we
did.
A
We
didn't
move
to
Prometheus
and
start
building
or
fauna
dashboards
to
go
around
the
issues,
but
there
was
a
lot
of
steps
between
us
wanting
to
figure
out
what
was
going
on
on
our
network
and
us
solving
it
and
as
viewing
it
in
a
single
dashboard.
We
had
to
it
Prometheus
Griffin
I,
everyone
had
to
manage
his
design
graphs,
so
on
so
forth
there
was
a
lot
of
all
things
that
happened
there.
So
what
actually?
The
problem?
A
Well,
as
I
mentioned,
solid
instances
in
every
note
because
of
large
uses
consumptions,
which
means
you
had
a
proxy
per
node
instead
of
per
instance,
the
configuration
became
complex
so
were
the
updates.
It
didn't
support.
Dynamic
updates
monitoring
was
very
disjointed,
as
you
saw,
and,
of
course
we
didn't
have
any
proper
MPLS
support
because
we
didn't,
we
couldn't
do
service
to
service
and
the
last
only
note
to
note
which
was
better
than
nothing.
There
is
not
what
we
wanted
at
the
end
and
the
developer
friction
was
quite
high.
A
A
At
the
end
of
the
day,
so
clearly
the
architecture
there
was
designed
for
as
reached
an
answer
or
speak
so
where
to
go
next,
where
did
the
community
all
right?
All
the
different
providers
go
from
there.
How?
How
would
you
design
a
service
mesh
that
works
for
a
modern
cloud
native
kubernetes
based
environment?
A
Well,
you
would
say
how
epoxy,
but
instead
of
having
all
of
those
features
within
a
proxy,
you
would
strip
those
out
and
move
them
into
a
control
plane
and
instead
keep
the
poxy
only
as
a
proxy
and
leave
all
of
the
configuration
and
the
bloatedness
and
the
setup
and
the
management
of
certificate
within
a
control
plane,
which
is
just
a
normal
service
and
noble
deployment
into
your
cluster.
Now,
how
does
that
control
plane?
A
Look
like
well,
it's
just
a
normal
deployment
and
what
it
does
is
it
manages
it
configures,
the
actual
proxies
it's
a
standard,
stateless
deployment.
It
has
a
public
API.
It
collects
the
telemetry
from
all
the
proxies
in
your
clusters
and
exposes
them
with
a
single
interface
and
now
super
attractive
to
us,
because
we
finally
got
a
single
place
where
we
can.
Actually,
we
will
finally
get
a
single
place,
require
actually
put
monitor
our
service
measure
and
can
enforce
policies.
A
A
That's
a
bit
more
appropriate
for
it
that
uses
less
resources
and
can
be
at
with
the
ultimate
goal
of
deploying
it
as
a
sidecar
to
every
instance
to
achieve
the
true
one-to-one
proxy,
to
instance,
ratio
that
we
wanted
to,
and
that
was
basically
the
general
architecture
that
two
of
the
most
popular
service
measures
now
took
both
linker
D
and
sto.
If
you
broke
them
operate
the
same
way,
they're
implemented
differently,
use
different
technologies
under
the
hood,
but
ultimately
they
function
roughly
the
same
way
and
perform
roughly
the
same
functions
now.
A
A
Well,
we
deliberated
a
lot
and-
and
you
you
couldn't,
go
wrong
either
way
to
be
frank
in
the
in
this
case,
but
for
us
we
decided
to
go
to
linker
D
and
one
of
the
deciding
credit
to
in
or
deciding
the
reasons
was
that
well,
we
had
good
experiences
record
d1.
We
wanted
to
continue
that
and
second
of
all,
one
of
the
greatest
features
of
link
32
is
that
it's
a
lot
more
simpler
than
sto
and
one
of
the
biggest
problems
we
had
to
think
ready.
A
One
was
that
it
was
so
daunting
to
to
hundreds
of
developers
and
organizations
that
they
didn't
even
want
to
learn
it.
So
we
wanted
something
more
simple
as
something
that
everyone
can
pick
up
and
they
don't
need
to
fully
understand
it.
They
can,
if
they
want
to,
but
something
they
are
know
how
it
works
and
they
can
sort
of
grasp
I
relatively
quickly,
and
that
was
a
very
attractive
for
us
in
this
particular
case.
So
what
we
did
was
we
started
to
plan
a
migration.
A
How
do
we
get
from
the
architecture
we
had
before
with
linka
d1
to
the
one
that
we
just
showed
you
link
ready
to?
So
that's
so
our
architecture
take
three
basically,
so
that
was
our
third
redesign
of
our
whole
service
match
architecture,
witness
scope
of
two
years
was
it
I
think
and
here's
what
we
needed
to
do.
Of
course,
we
had
to
first
planet,
but
ultimately
here's
what
we
ended
up
doing
so
on
the
left
was
our
previous
architecture
we
mentioned.
A
So
we
had
a
demon
set
per
note
which
handle
our
service
smash
and
we
need
to
move
it
to
something.
Look
like
this,
so
we
had
a
control
plane
which
consists
of
a
few
deployments
and
ports
there's
more
than
one
component,
but
ultimately
it's
just
a
single,
a
single
normal
standard,
community,
stateless
deployment
that
stores
all
of
its
state
in
in
kubernetes
config
maps
and
custom
research,
definitions
and
so
on.
So
it's
fully
cloud
native
and
fully
embraces
the
club.
A
A
Force
walk
without
any
downtime
and
without
the
without
completely
confusing
the
developer.
So
what's
going
on?
Well,
there's
a
few
things:
oh
yeah,
so
this
is.
We
think
that
we'll
get
to
that
in
a
second
and
then
the
second
thing
we
did
was
just
make
sure
we
can
configure
it
correctly.
So
this
is
how
a
configuration
of
a
service,
for
instance,
looks
like
an
included.
A
Next
to
our
deployments,
or
even
help
chart
or
whatever
it
is
it
used
at
the
time
so
yeah
we
couldn't
set
retries.
You
can
set
timeout
stuff
like
that.
As
a
lot
of
things,
we
can
configure
we'll
go
through
all
of
them,
but
it
was
quite
powerful
and
we
could
centrally
central
figure
and
the
developers
DL
develops
when
we
show
them
is
they're
a
lot
more
happy
with
OU
can
just
use
this
just
like
we
can
figure
anything
out
so
kubernetes
to
do
it
so
yeah
what's
positive.
So
what
were
our
goals?
A
So
one
of
the
main
things
we
want
to
do
is
we
wanted
no
required
to
developer
interaction
to
make
the
switch,
and
what
does
that
mean?
So
it
doesn't
mean
we
don't
want
it.
Developers
can
interact
with
it,
but
we
didn't
want
to
require
it.
So
if
somebody
didn't
care
about
it
to
them,
it
would
be
invisible
and
the
same
for
our
users
and
customers
that
use
the
platform.
We
didn't
want
any
disruption
and
we
have
minimal
changes
to
our
current
architecture.
A
Yeah
chose
to
add
append
plumbing
pipeline
so
for
number
one
I
think
number
one
was
a
particular
interesting
one.
How
do
we
not
require
infant
developers?
Well,
if
you
go
back
to
linker
d1
the
way
we
configured
the
parts
themselves
to
use,
the
actual
proxy
was
that
we
set
environment,
variable
HTTP
proxy,
to
point
to
the
load,
to
the
node
that
the
instance
was
running
on
using
a
snippet
like
that
on
the
left.
So
basically
just
got
the
known
name
and
then
set
it
as
the
HTTP
proxy,
and
all
of
the
requests
would
then
be.
A
A
So
that
was
one
thing,
and
the
second
thing
is
we
sort
of
agreed
on
the
default
standard
way
of
how
we
identify
our
services,
which
is
basically
just
service
name,
dot
name
spacing
kubernetes,
and
that
was
it.
Luckily,
the
names
that
the
service
names
ourself
stayed
the
same
in
linker
D,
but
even
if
they
weren't,
we
could
all
configure
them
to
be
the
same,
for
instance,
do
or
even
include
two.
We
can
always
configure
what
the
host
names
actually
are,
so
we
can
migrate
over
without
any
services.
But
how
do
we
solve
the
left
problem?
A
So
we
wanted
to
remove
that
manual
step
of
having
everyone
need
to
do
it
themselves,
so
we
embraced
kubernetes
and
made
use
of
Admissions
web
hooks,
so
we
created
our
own
Duchamp
web
hooks
that
basically
did
that
automatically
on
the
cluster.
So
what
this
does
is
basically
deploy
and
commanders
will
call
the
service
before
each
part
is
scheduled,
and
what
you
can
do
at
that
point
is
you
can
edit
the
depart
the
finishin
itself
to,
for
instance,
add
environment,
variable
or
add
a
container
to
the
pod?
A
Without
you
having
to
do
anything,
it
will
happen
automatically
in
the
background
and
actually
link
ready
to
uses
this
mechanism,
as
well
as
the
as
desist
here
and
all
but
a
bunch
of
other
places
as
well.
So
this
was
a
good,
a
good
search
to
a
problem,
so
what
we
did
was-
and
it's
completely
transparent
to
anyone
using
it.
A
So
what
we
did
was
we
made
sure
that
that
snippet
was
moved
into
initial
controller
and
then
everyone
removed
that
so
yes,
there
was
a
step
that
needed
developer
intervention,
but
once
once
we
move
that
moved
away
from
manual
setup
into
a
like
an
automated
set
up
within
the
cluster.
We
could
then
update
this
in
the
background.
Without
anyone
have
to
update
anything
so
effectively.
What
did
we
do
so
plan
was
actually
sort
of
simple
when
I
look
at
it
back,
so
we
had
all
of
our
infrastructure
definitions
in
energy
depository.
A
We
version
everything
we
serve,
followed
the
infrastructure,
Isco
DevOps
approach
that
everyone
follows,
so
we
build
a
pipeline
to
do
the
migration
and
what
this
pipeline
did
was.
Basically,
we
start
the
migration
by
like
merging
the
changes
into
the
correct
branch
for
the
correct
repository.
Of
course,
we
tested
as
a
bunch
of
times
before
we
moved
to
a
poor
environment.
A
So
what
we
did
was
we
so
what
the
job
did
was
we
wanted
to
deploy
both
service
meshes
at
the
same
time
and
gradually
move
what
service
one
by
one
from
link
ID
1
source
mesh
to
link
ready.
So
the
way
we
did
that
we
first
we
deployed
the
control
plane.
Now
remember:
the
control
plane
is
just
a
normal
deployment.
A
It
doesn't
interfere
with
anything,
there's
no
harm
in
it
being
there
if
nobody
is
using
there,
so
that
was
straightforward.
We
deployed
the
control
plane
and
that
was
in
there
and
ready
to
go.
At
that
point,
we
also
made
sure
to
apply
all
the
configurations
that
the
services
would
need
it
that
with
it
beforehand
now
most
of
it
was
generic
automatic
configurations
that
we
could
tweak
later
on.
But
for
the
initial
migrations
we
didn't
need
to,
so
it
was
fairly
straightforward.
So
we
deployed
the
control
plane
and
then
the
important
step
came
up.
A
How
do
we
actually
now
move
every
service
from
using
the
old
service
smash
into
the
new
service
mesh
without
causing
a
disruption
without
developers
noticing
or
us
noticing,
or
the
customers
are
missing
or
anyone
under
singles?
Like
a
personal
challenge
for
us
to
do
that,
so
because
we
moved
over
to
the
admissions
webhooks
before
to
configure
our
ports,
we
simply
stopped
it,
which
means
any
new
parts
that
were
deployed
would
not
have
the
linker
to
configure
a
d-link
ready,
one
configuration
applied
and
the
same
time
that
we
stopped
it.
A
We
started
up
the
link
ready
to
admission
web
hooks
and
the
way
link
ready
to
admission
epoch
work
is
that
it's
automatically
there
when
they
deploy
the
control
plane.
But
it's
not
activated
and
the
way
you
activate
it
is.
Is
you
basically
put
a
label
on
the
namespace
or
on
the
deployment
themselves
of
where
you
one
to
activate
where
he
wanted
to
be
activated?
A
Now
we
didn't
again,
we
didn't
want
to
edit
the
deployment,
so
we
put
them
all
automatically
in
all
the
namespaces
there
when
in
use
and
and
that
way
we
started
the
admission,
we're
considering,
could
it
so?
What
am
I
wanted
up
happening
else.
Is
that
any
new
part
that
was
gonna
get
deployed
was
gonna?
Have
the
link
ready
to
admission
weapons
applied,
which
would
add
the
sidecut
container
to
the
pot,
which
will
automatically
connect
to
the
service
mesh
to
the
control
plane
of
linker
d2
and
start
using
the
new
service
mesh?
A
So
we
didn't
need
to
worry
about
not
being
able
to
connect
to
sides
of
the
other
sides
of
a
request,
because
one
didn't
migrate
yet,
but
once
ever
migrated
MPLS
will
be
activated
and,
and
everything
would
work
so,
once
we
once
we
flip
the
switch
on
those
two,
we
simply
initiate
a
cluster
right,
rolling,
deploy
one
service
at
a
time
to
be
able
to
pick
up
to
destroy
the
old
parts,
create
the
new
parts
and
pick
up
the
new
pick
up.
The
new
configuration
new
service
mesh
and
basically
connect
to
the
new
service
mesh.
A
We
removed
the
old
admission
web
worker
and
controller
and
remove
the
old
linker
diamond
set
from
the
notes,
and
how
was
it
and
again
the
reason
we
spend
much
more
time
on
on
the
wrong
redeploy.
Is
that
if,
at
any
point
something
went,
went
wrong,
we
could
always
throw
back.
We
could
always
reenable
the
old
one
disabled
new
one
and
then
reapply.
Every
re
redeploy
the
instances
that
we
restore
it
and
it
would
go
back
to
using
the
old
service
mesh.
A
So
we
had
a
back-up
plan
if,
if
things
didn't
go
well,
of
course,
we
tested
a
bunch
of
times
in
production,
so
that
was
a
general
plan
of
how
we
got
things
across
and
how
we
used
Automation
and
the
features
of
kubernetes
to
make
big
architectural
changes
to
our
service
mesh,
and
that's
just
an
example.
What
you
can
do.
You
could
use
the
same
sort
of
concepts
to
make
any
sort
of
changes
in
kubernetes
or
other
content
environments.
A
It
just
leads
to
problems,
try
and
take
the
burden
away
from
them
and
automate
as
much
as
possible
and,
of
course,
to
follow
up
on
that.
Let
the
platform
do
the
heavy
lifting.
That's
why
it's
there
is.
The
reason
kubernetes
is
so
popular
nowadays
is
because
we
don't
need
to
deal
with
where
containers
are
scheduled,
how
they're
used
and
how
they're
how
they
move
around
the
cluster
same
with
stuff.
Like
this,
don't
do
it
yourselves,
don't
need
to
think
about
it.
You
can
just
let
the
platform
how
to
throw
you.
A
You
can
write
the
automation,
so
you
need
to
be
able
to
do
that
as
well,
and
our
last
note
is
I.
Suppose
I
talked
a
lot
about
in
the
beginning
about
what
a
service
mesh
is
and
all
the
times
it's
when
I
talk
to
people,
it's
a
lot.
It
it's
difficult
for
people
to
grasp
or
what
is
it
like?
I
can
explain
three
different
ways,
but
it's
still
difficult
expell
explain
what
it
actually
is,
so
I
prefer
to
say
it
or
what
it
actually
gives
you
and
does
give
you
is.
A
It
provides
you
a
transparent,
reliable,
an
autonomous
network
up
for
any
service
running
or
cluster,
and
the
important
part
is
is
that
you
don't
actually
need
to
know
about
it
as
long
as
you're
aware
that
it's
there
you
can
go
into
is
deep
into
it.
As
you
want
and
configure
it
as
much
as
you
want,
but
you
don't
need
to
it's
there
and
what
it
gives
you
is.
A
You
don't
need
to
think
about
network
anymore,
the
service
mystic.
So
for
you
and
there's
a
lot
of
cool
components,
working
underneath
to
make
that
happen,
cool
right
on
time.
Thank
you
very
much.
I
hope,
I
hope
that
wasn't.
That
was
interesting
for
you
and
yeah.
Let
me
know
if
you
have
any
question
any
questions
about
that.
You
like
that,
if
not
thank
thanks
for
coming
and
hope
you
enjoy
the
conference.