►
From YouTube: Webinar: Canary Deployments with Kubernetes and Istio
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Working with complex systems means that even well written code can behave in unexpected ways, leading to outages and critical issues. Kubernetes allows you to easily rollout new code and quickly revert to safe, stable releases when issues are identified. But often the damage done in the short period between deploy and rollback is significant.
In this session, I’ll show you how to limit the effect of unforeseen issues using canary deploys with Istio. I’ll also cover ways to better monitor your applications in Kubernetes to help you spot potential problems before they affect your entire customer base.
A
Alright,
it
looks
like
our
numbers
have
stabilized,
so
let's
go
ahead
and
get
started,
welcome
everyone
to
today's
CN
CF,
webinar
Canary
deployments
with
kubernetes
and
sto
I'm,
Kaitlin,
Barnard
marketing
manager
for
CNCs
and
I'd
like
to
thank
you
all
for
joining
us
today.
I'd
also
like
to
welcome
today's
presenter
Jason
Yee
who's,
a
Technical
Evangelist
at
data
dawg.
So
just
a
few
housekeeping
items
before
we
get
started
during
the
webinar
you're,
not
able
to
talk
as
an
attendee.
A
So
if
you
have
any
questions
or
comments
throughout
the
webinar,
there
is
a
chat,
a
chat
box
and
a
Q&A
box
at
the
bottom
of
your
screen.
So
please
drop
any
questions
that
you
have
in
there
and
we'll
get
to
as
many
as
we
can
at
the
end
and
just
a
reminder
that
the
session
is
being
recorded
and
will
be
sent
out
afterwards,
along
with
the
slides
within
about
24
hours,
and
with
that
I'll
hand
it
over
to
Jason
to
kick
off
today's
presentation.
B
Thanks
Caitlin
welcome
everyone,
so
hopefully
you're
here
to
hear
about
deployments
so
deployments
right.
What
could
possibly
go
wrong?
That's
a
stupid
question
because
the
answer
is
obviously
a
lot.
That's
why
you're
all
on
this
call
anybody
who's
ever
made.
A
deployment
has
inevitably
like
encountered
a
problem
whenever
I
talk
and
I'm
in
front
of
an
audience,
I
usually
ask
people
that
you
know
who's.
B
A
deploy
issue
today
or
this
week
or
this
month,
and
inevitably
by
the
time
you
get
to
the
point
of
like
this
month,
everybody's
raised
their
hands
for
that
because
it's
it's
really
obvious
that
deployment
as
much
as
we
try
to
make
them
go
flawlessly.
There's
always
problems,
and
so
I
like
to
bring
up
this
meme
that
it's
the
most
interesting
man
in
the
world.
It
was
at
Osakis
campaign,
but
adapted
for
tech.
B
The
meme
is
when
I
do
blank
I,
do
it
in
production
because
oftentimes,
we
do
silly
things
and
we
just
do
them
straight
in
production
without
even
try
trying
things
and
staging
or
things
like
that.
You
know
it's
kind
of
this
bravado,
but
the
problem
when
it
comes
to
deployments
is
that
deploy
to
production.
You
always
do
that
in
production.
B
So
that's
why
it's
basically
impossible
to
guarantee
against
deploying
bad
code
or
code
that
doesn't
work
the
way
we
think
it
should
and
I.
Think
that's
why
canary
deployments
and
using
a
canary
deployment
strategy
should
be
critical
and
should
be
the
strategy
that
we
adopt
for
every
single
deployment
that
we
do
moving
forward
and
I'll
be
showing
you
with
with
kubernetes
and
SEO
how
that
can
be
a
much
easier
process.
B
B
Aside
from
that,
I
am
a
nomad,
so
I
travel
around
so
bones
rolled
in
Montreal,
currently
in
Montreal,
but
I've
been
traveling
around.
It's
been
about
one
city
every
week,
so
I'll
be
in
Portland
Oregon
next
week.
If
you're
in
Portland
hit
me
up,
I'd
love
to
say
hi
but
I'll
be
traveling
around
for
for
the
rest
of
the
year
into
December.
A
B
Then
I
plan
on
settling
gonnigan
also
do
love
whisky.
So
that's
a
photo
of
my
favorite
whiskey,
that's
available
and
also
am
totally
obsessed
with
Pokemon
goes.
So,
if
you
play
that's,
my
friend
code
feel
free
to
friend
me
I
will
send
you
gifts
from
around
the
world
a
little
bit
about
data
dog
data
dog
is
SAS
based
monitoring
platforms,
so
we
do
monitoring
what
you
traditionally
think
of
as
monitoring
with
metrics
automated
alerts,
but
we
also
do
APM
so
we'll
do
distributed
tracing.
B
B
We
are
hiring
so
if
you
are
considering
a
new
opportunity,
if
you'd
like
to
help
build
amazing
tools
that
help
other
engineers
be
better
at
their
jobs,
we
are
hiring
so
take
a
look
at
that
link
and
just
to
give
you
a
caveat
as
we
talk
about
this,
we
do
do
Khmer
u2
plays
within
data
dog.
We
are
running
kubernetes.
Almost
all
of
our
services
are
running
on
kubernetes.
Now
we
have
been
playing
a
lot
with
this
deal
and
working
on
that.
B
B
So
I'd
like
to
update
this
I
haven't
talked
with
Kelsey
about
his
CEO
recently,
but
sto
is
now
1.0.
In
fact,
it's
one
point
0.2
and
one
point:
0.3
is
going
to
be
coming
out
soon.
The
api's
are
mostly
stable,
you'll
notice
in
the
Yama
files
there's
a
couple
spots
where
it
still
says
alpha,
which
means
that
theoretically
they
could
change.
B
But
at
this
point,
sto
is
stable
enough
that
you
should
be
really
considering
whether
you
need
a
service
mesh
and
it
you
do
which
I
think
most
people
are
running
kubernetes
in
large-scale
environments
with
alumni
micro
services.
I
think
those
people
do
need
service
meshes.
So
if
that's
you
it's
something
that
you
should
consider
going
out
and
and
trying
and
playing
with,
because
I
think
it'll
solve
a
lot
of
the
problems
that
you
either
currently
have
or
will
have,
as
you
grow
out,
a
large
micro
services
distributed
architecture.
B
So,
let's
talk
about
how
we're
doing
deploys
currently
blue
green
deploys
very,
very
common
practice
these
days
on
how
to
deploy
software.
If
you're
not
familiar
with
blue
green,
it
really
doesn't
matter
what
color
it
is.
I've
heard
people
call
these
red
black
I've
heard
it
all
all
sorts
of
colors
across
the
entire
color
spectrum.
But
the
concept
is
very
simple:
you
start
with
version
1
of
your
application
and
you
have
version
2
sitting
in
the
sidelines.
B
It's
deployed
on
infrastructure,
but
it's
hidden
behind
some
sort
of
load
balancer,
and
when
you
want
to
deploy
that
in
software,
you
simply
make
a
change
to
that
load.
Balancer
that
load
balancer
could
be
something
as
simple
as
a
DMS
entry
or
it
could
be
even
on
the
software
level.
If
we
think
about
how
Capistrano
deploys
weren't
Capistrano
really
was
just
using
symlinks
here,
so
you
would
sim
link.
B
Really,
you
had
no
downtime
and
it
always
promised
easy
roll
backs
right
if
I
need
a
to
roll
back
from
version.
2
back
to
version
1.
Well,
I
could
just
go
into
my
load,
balancer
settings
or
some
one
of
my
other
settings
and
make
that
flip
back
and
now
all
traffic's
were
getting
directed
to
that
old
version
of
my
software
I.
B
We
have
data,
that's
stored
along
with
them,
so
you
always
have
this
issue
that
you
know.
One
of
your
updates
may
have
made
an
update
to
a
data
store
like
a
database
schema
and
now,
when
you
do
this
rollback,
suddenly
those
queries
that
used
to
work
with
the
old
state
of
your
data
no
longer
works
with
with
the
current
state
of
your
data,
because
it's
been.
B
Date
stores
are
much
harder,
all
back,
and
so
things
inevitably
break
if
you're
using
kubernetes.
Well,
the
nice
thing
is
kubernetes
uses
a
process
called
rolling
deployments,
so
rolling
deployments
are
quite
a
bit
better
than
Bluegreen,
so
for,
if
we're
looking
at
rolling
deployments,
how
does
that
look?
Well,
it's
similar
in
a
sense
to
Bluegreen.
B
You
have
a
bunch
of
now
replicas
replicas
of
one
version
of
your
software,
but
similarly
you
have
a
load
balancer
sitting
there,
and
this
would
be
kubernetes
itself
if
you're
running
that,
and
you
simply
remove
one
of
your
old
versions
and
spin
up
one
of
your
new
versions,
and
you
keep
doing
this
until
all
of
your.
Your
replicas
are
rolled
over
and
one
of
the
nice
things
now
is
that
again,
this
provides
us
with
almost
no
downtime,
but
similarly
because
we're
having
a
support,
version,
1
and
version
2
of
the
software.
B
B
So
then,
what
is
a
canary
deployment?
Well,
a
canary
deployment
is
really
the
idea
that
if
I
have
a
problem,
if
I
do
something
really
stupid,
do
I
want
that
to
be
in
front
of
a
very
large
audience
or
would
I
rather
constrain
that
and
have
it
to
a
very
small
audience.
Similarly,
if
I
were
on
this,
this
webinar
and
I
say
something
really
ridiculous
and
stupid
or
I
say
something
wrong.
B
Well,
I
rather
say
that
if
there's
only
one
of
you
on
this
call,
but
obviously
there's
a
there's,
a
lot
of
people
and
they'd
have
joined
us
today.
So
if
I
make
a
mistake,
the
ramifications
are
much
larger,
so
canary
two
plays
really
do
look
similar
to
rolling
deploys.
You
have
a
bunch
of
replicas
of
one
version
of
your
software.
B
You
have
your
load
balancer,
essentially
redirect
some
of
your
your
users
to
a
new
version,
as
you
tear
down
your
old
version
and
you
spin
up
that
new
version,
but
rather
than
continuing
this,
you
pause
and
you
monitor.
So
this
is
one
of
the
areas
that
Akuma
Nettie's,
unfortunately,
is
really
good
at
rolling
deploys,
because
it's
very
fast
it
doesn't
really
give
you
the
opportunity
to
pause
and
to
monitor,
but
with
carry
employment.
Then
we
would
continue
on.
B
When
we
see
something
that
might
be
erroneous,
another
thing
that
a
lot
of
people
don't
take
advantage
of
when
they're
doing
canary
deploys,
but
I
think
is
a
fantastic
feature.
Is
that
it's
much
more
load
tolerant
when
we
talk
about
things
like
kubernetes
or
going
to
the
cloud,
whether
that's
managed
kubernetes,
something
like
eks
in
amazon
or
or
gks,
and
google,
or
any
of
the
other
managed
providers,
or
whether
that's
just
running
your
own
kubernetes?
On
top
of
cloud
infrastructure,
one
of
the
nice
things
is,
as
we
roll
things
out.
B
If
we
see
that
it's
having
increased
demand
on
our
resources,
it
gives
us
a
chance
to
actually
scale
those
up
right.
So
if
suddenly
I
see
that
whatever
my
new
deployment
is,
maybe
it's
a
fantastic
deployed.
I'm,
seeing
thousands
more
users
hit
my
application,
because
it's
a
much
better
version
of
my
application.
Well
now
it
can
scale
up
those
resources
and
another
thing
that
canary
deployments
provide
is
concurrency.
So
as
you're
able
to
roll
out
multiple
versions
of
your
application,
you
can
start
to
concurrently
roll
those
out.
B
So
that's
the
the
whys
of
canary
to
plays.
Let's
talk
about
how
we
should
think
about
the
strategies
of
how
we
actually
can
implement
this.
So
some
of
the
strategies,
when
you
think
about
Canaria
deployments,
you
really
should
focus
on
your
end
users
right
your
end.
Users
are
the
reason
that
you
have
your
job,
that
you're
doing
what
you
do,
whether
those
are
external
to
your
organization
or
internal,
but
you
should
think
about
how
you
choose.
B
The
sample
sets
those
subsets
where
you
deploy
that
canary
deployment
and
so
number
one
way
that
most
people
choose
to
do
things
is
random
right.
You
just
say:
I'm
gonna
randomly
send
5%
of
my
users
to
my
new
deployment
random.
Unfortunately,
as
we
all
know,
anybody
who's
ever
worked
with
technology.
We
know
that
computers
are
very,
very
bad
at
random
numbers.
B
Right,
there's
always
some
sort
of
issue,
whether
that's
people
forgetting
to
seed
random
or
other
things,
but
random
usually
doesn't
give
us
the
results
that
we
want
so
I
highly
recommend
you
think
about
representative
sets
so
representative
sets
essentially
means
again
keep
your
focus
on
your
end
users.
How
are
your
end
users
going
to
be
using
your
application?
Think
about
their
geography
where
in
the
world
are
they
right,
for
example,
I'm
currently
in
Montreal,
which
is
North
America
or
eastern
North
America
time?
B
But
if
I'm
going
to
roll
out
a
new
feature
and
it
particularly
targets-
or
it
particularly
highlights
uses
that,
for
example,
my
customer
base
in
Australia
might
be
using
well
I
have
to
think
about
where
they're
at
and
what
time
is
it
in
the
world
right?
So
if
I
deploy
this
and
I
have
a
canary
deployment
and
I
look
at
and
say
hey,
it
looks
like
everything's.
Fine
resource
usage
is
perfectly
normal,
but
it
turns
out.
Well,
those
people
are
all
asleep.
B
Then
it's
not
a
useful
canary
deployment
also
consider
the
usage
patterns
for
your
users
when
you
roll
out
a
new
feature.
Who
are
the
users
that
it's
intending
to
target?
Is
it
logged
in
users?
Is
it
administrative
users
versus
end
users,
so
think
about
these
usage
patterns
and
what
you're
rolling
out?
And
this
really
does
force
you
to
think
in
terms
of
DevOps
and
when
I
say
DevOps
I,
don't
mean
automation
or
things
like
that
I
mean
the
actual
developers
talking
with
operations
to
to
work
on
canary
strategy.
B
You
also
have
to
think
about
the
granularity
when
I,
say
granularity
I
mean
those
steps.
So,
in
my
example,
earlier
I
had
four
replicas
and
I
said:
I
was
having
it
pause
in
between
each
of
those
as
replicas
from
version
1
world
over
to
version
2,
but
we
have
to
think
about
the
granularity.
How
are
we
going
to
target
a
small
set
of
users?
B
That's
enough
users
for
us
to
monitor,
and
then
how
are
we
going
to
scale
that
up
to
the
next
step
and
the
step
beyond
that,
because
you
really
don't
want
your
canary
strategy
to
be,
let's
deploy
to
a
small
subset
and
if
it
works,
okay,
we're
just
going
to
go
whole
hog
percent
of
our
users.
Get
this.
You
really
want
to
increment
that
up,
at
least
with
a
few
steps.
B
Grubhub
is
a
customer
of
data
dog
and
they
recently
had
a
talk
about
how
they
do
canary
deployments
at
one
of
our
our
use
of
their
summits,
and
they
have
this
interesting
strategy
that
I
hadn't
even
considered
where
they
do
a
canary
a
deployment
to
small
Sadie's
in
small
towns.
So,
for
example,
in
the
US,
if
you're
us-based,
they
might
roll
out
to
a
city
like
Detroit,
before
rolling
out
to
a
larger
city
like
Chicago
or
New
York
before
rolling
out
countrywide.
B
So
start
to
think
about
about
that.
How
can
you
segment
off
your
users
in
ways
that
will
scale
up
and
provide
you
with
useful
feedback
and,
as
mentioned
Kerry
employees
do
get
that
opportunity
to
scale
up
our
infrastructure
as
well,
so,
along
with
that
think
about
the
resource
mapping
and
do
roll
out
a
new
feature
how's
that
new
feature
going
to
impact
your
infrastructure?
Are
there
additional
calls
to
a
datastore
or
an
API
endpoint?
Well
that
caused
additional
load
or
network
traffic
to
that
particular
resource
that
data
store
or
that
server?
B
That's
providing
that
API
so
start
to
think
about
everything,
that's
involved
from
a
user
perspective
and
from
an
infrastructure
perspective.
All
of
this
does
assume
that
you
are
monitoring,
though
you
can't
do
a
proper
canary
deploy
if
you're,
not
monitoring,
how
you're,
using
your
resources,
how
people
are
using
your
application.
So
some
things
to
think
about
when
you're
actually
doing
a
strategy
around
canary
deployments
for
your
monitoring
first
is
that
you
need
tags,
whether
those
are
tags
and
data
dog,
whether
those
are
labels
and
Prometheus.
B
B
A
B
The
other
thing
that
you
need
to
look
at
is
the
outliers
and
I
like
to
call
outliers
that
one
of
these
things
does
not
look
like
the
other
right.
Outliers
really
is
the
idea
that,
with
a
canary
deployment,
hopefully
you'll
have
some
sort
of
graph
or
some
sort
of
dashboard.
That
shows
you
that
you
can
highlight
where's
my
production
metrics.
What
do
these
look
like
and
then
have
some
sort
of
machine
learning
or
way
to
tell
the
difference
between
what's
an
outlier?
What
isn't
part
of
that
normal
grouping?
B
The
other
thing
that
you
need
to
think
about
is
anomalies
and
I
like
to
say
that
anomalies.
Is
this
what
isn't
like
what
it
was
before?
It's
not
acting
how
it
has
before-
and
this
is
a
key
point-
that
this
is
where
machine
learning
or
if
you've
got
AI
or
you've
got
some
sort
of
algorithmic
anomaly
detection.
This
is
where
it
comes
in
really
handy
because
of
our
search
to
show
you
a
graph
like
this
and
say
well,
how
is
this
canary
deployment
different
right
from
from
how
things
have
worked
in
the
past?
B
B
So
when
you're,
looking
at
your
canary
deployments,
take
a
look
at
changes
in
latency
right
again,
push
all
the
way
out
to
the
p99
how's,
your
latency
changing
because
that'll
be
important
for
your
users.
The
e'en
let
lets
is
for
errors.
So
what
are
the
errors
that
you're,
seeing
not
just
error
rates?
But
what
are
the
the
actual
errors?
Because,
as
you
deploy
new
code,
you're
gonna
see
different
errors
than
you
have
in
the
past.
B
The
T,
unless
is
for
traffic,
so
traffic
is
is
sort
of
a
double-edged
sword.
You
might
see
increased
traffic
and
that
can
be
a
fantastic
thing
could
mean
that
you've
deployed
fantastic
new
features
in
your
users,
just
love
it
and
so
you're,
seeing
much
more
traffic.
But
it
could
also
be
a
bad
thing.
It
could
mean
that
maybe
a
developer
really
some
unoptimized
code,
that's
making
duplicate,
requests
against
the
data
source
or
an
API.
Similarly,
decreased
traffic
could
be
a
good
thing.
B
It
could
be
that
your
developers
optimized
code
and
now
they're
only
making
one
call
to
that
data
store
rather
than
two
or
it
could
mean
that
you
deployed
something
bad
and
users
are
having
problems
actually
using
the
software.
So
take
a
look
at
those
changes
either
way,
and
the
last
is
saturation.
So
take
a
look
at
how
your
infrastructure,
how
your
resources
are
being
used
compared
to
your
deployment.
B
So
what's
all
of
this
have
to
do
with
kubernetes
right
well,
kubernetes,
how
many
people
listening
think
that
this
is
kubernetes?
It's
a
container
Orchestrator
right!
You,
you,
deploy,
containers
on
it
and
manage
desert
containers.
Well,
I
used
to
think
it
was
a
container
Orchestrator,
but
it's
actually
not
it's
a
service
Orchestrator.
B
The
base
unit
within
kubernetes
is
service.
The
containers,
the
pods,
the
rest
of
it
they're
all
ephemeral.
The
thing
that
matters
again
to
hammer
home.
The
point
is,
you
have
to
be
aware
of
your
users
and
your
users
care
about
services.
They
don't
care
about
your
containers
or
your
pots.
So
there's
those
are
just
the
resources
that
provide
the
service.
B
So
services
are
what
we
care
about
and
if
we're
thinking
about
deployments
well,
if
we
want
it
to
deploy
the
one
service
with
kubernetes,
it
wouldn't
make
sense
to
have
two
separate
meshes
of
services
and
just
have
one
giant
load
balance
in
front
of
all
of
them.
So
obviously
kubernetes
makes
that
much
easier
as
a
service
Orchestrator,
something
that
manages
services
it
handles
those
service
deployments
right.
So
we
can
change
individual
services
within
kubernetes
they'll
do
rolling
deployment,
it's
really
fantastic
at
that.
So
why
might
you
need
a
service
mesh
like
Sto?
B
Well,
as
mentioned,
kubernetes
does
is
rolling
deployments
very,
very
well
right.
It
doesn't
so
well
that
it
happens
quickly,
but
it
doesn't
do
canary
deployments
very
well
at
all.
So
how
would
you
do
it
right?
Give
you
an
example:
kubernetes,
let's
set
up
a
service
right,
so
there's
a
basic
web
service
here,
I'm
gonna
have
my
llamo.
It
says
I'm
declaring
service.
B
If
it's
a
metadata,
which
is
my
application,
I'm
going
to
expose
port
80
because
it's
a
web
application,
so
that's
the
service
part,
but
if
I
wanted
to
actually
implement
this
will
I
need
to
deployment,
and
my
deployment
sets
up
my
pods
and
how
many
replicas
I
want.
So
if
I
were
to
do
a
deployment
with
version
1
and
version
2
of
my
application.
Well,
if
I
say
the
same,
replica
is
what
would
this
do?
B
Well
generally,
this
would
just
split
my
traffic
fifty-fifty
right
and
one
service
I've,
two
deployments
so
when
it
comes
to
actually
directing
traffic
or
doing
any
sort
of
routing
kubernetes,
just
sort
of
is
agnostic.
It
says
I,
don't
know
which
one
of
these
really
matters.
So
it's
gonna
split
my
traffic
5050,
mostly
randomly.
So
if
I
wanted
to
do
canary
deployment,
but
you
could
see
how
this
could
be
useful.
B
So
I
can
start
to
scale
this
up
as
a
canary
deployment
right.
Everything
looks
good
with
nine,
and
one
I
can
now
shift
that
tape
to
or
I
can
shift
that
to
seven
and
three
all
the
way
to
the
point
where
I
can
go
from
one
version
of
version,
one
or
one
writer
out
to
comfort
someone
to
nine
replicas
of
version
two
and
then
I
just
delete
version
one
and
shift
version
two
to
ten
replicas,
because
there
is
a
bug
in
kubernetes.
B
On
top
of
the
fact
that
I
gave
you
all
that
great
advice
about
how
you
should
think
about
canary
deployments-
and
you
know
have
representative
sets
and
all
of
this
other
information
were
really
just
locked
down
to
having
percentages
with
kubernetes.
So
this
is
why
kubernetes?
Well,
it's
a
fantastic
tool,
not
so
great
it
at
canary
deployments.
B
So,
what's
the
service
mesh
get
you
well
off
the
bat?
It
gets.
You
routing
a
load
balancing
right,
so
this
is
again
going
to
be
the
important
parts
when
it
comes
to
Canary
deployments.
It's
how
we
we
do
that
routing
and
the
load
balancing,
but
it
also
provides
you
some
some
better
service
discovery,
other
things
that
it
provides
as
timeouts
and
retries.
B
B
You
can
just
automatically
have
your
service
mesh
like
Sto
handle
that
it
also
handles
policy
enforcement,
so,
rather
than
relying
on
your
developers
to
always
implement
a
secure
connection
to
TLS
encrypt
all
of
their
their
calls,
you
can
enforce
that
within
within
a
service
mesh
and
the
other.
The
final
thing
that
provides
is
a
really
great
point
for
monitoring
and
tracing,
because
everything
all
the
network
traffic
is
flowing
through
your
service
mesh.
B
It's
a
great
way
to
start
getting
some
of
that
tracing
down
and
you
can
monitor
how
long
things
take
and
you
can
start
tracer
because,
as
they
make
calls
fyssas
and
you
can
put
timers
on
those
to
understand
which
services
or
which
parts
of
your
application
are
taking
the
most
time.
So
how
does
this
all
work?
Well,
you
have
kubernetes,
it's
monitoring
a
bunch
of
your
services.
In
particular
it's
going
to
be
having
a
bunch
of
pods.
So
the
first
thing
that
you
have
to
do
is
add
a
data
plane.
B
The
data
plane
really
is
here
is
just
going
to
be
envoy
envoys,
a
fantastic
CN,
CF
project
started
by
the
the
folks
at
lyft
and
it's
obviously
open
source,
but
really
when
it
comes
to
implementing
SEO.
It's
relying
on
envoy
for
what
we
call
that
data
plan.
In
order
to
do
that,
really
it's
going
to
modify
your
deployments,
that's
going
to
add
a
and
NIC
container,
that's
going
to
modify
your
IP
tables
and
redirect
all
of
your
container
traffic
through
an
envoy
sidecar.
So
now
that's
the
first
step
in
starting
off.
B
We've
have
all
of
our
network
traffic
going
through
Envoy,
the
next
portion
it
and
the
reason
that
I
think
on
sto
is
so
fantastic.
Is
it
provides
us
this
control
plane?
The
control
plane
is
really
what
I
like
to
say
is
the
guts
of
this.
Do
it's
where
we
have
the
ability
to
create
new
modules,
to
update
our
logic,
if
we
want
to
have
custom
logic,
to
pull
out
monitoring
and
metrics
to
do
a
lot
of
the
things
that
makes
this
deal
so
exciting.
B
Those
are
all
within
the
control
plane,
so
won't
dive
too
deep
into
the
guts
of
ISTE.
Oh,
but
let's
talk
about
how
we
should
do
or
how
we
would
do
a
canary
deployment
with
is
do
what
is
that?
Actually,
they
look
like
well,
we
start
off
again
with
the
service
right.
This
is
again
kubernetes
yeah
Mille.
We
have
a
service,
it's
basically
the
same
thing.
B
Yes,
tio
doesn't
touch
your
services
at
all
when
it
comes
to
our
deployments
again,
it's
just
going
to
be
a
basic
yeah
mol
file
and
for
the
most
part,
it's
gonna
be
the
same
as
well,
at
least
as
far
as
we're
concerned.
As
far
as
operators,
you
don't
have
to
do
anything
special
to
your
deployments.
The
reason
for
that
is,
sto
comes
with
a
fantastic
tool,
called
cube
inject.
So
really
what
this
does?
Is
you
pass
your
llamo
in
it?
B
Does
an
injection
like
I
said
before
that
injection
really
is
the
guts
of
it
is
to
inject
and
emit
containers
about
before
your
service
or
before
your
pod
starts
up
it's
going
to
modify
those
IP
tables.
Now
this
isn't
the
the
standard
way
of
starting
SEO,
but
it's
the
way
that
I
recommend
for
most
people,
because
just
like,
when
someone
tells
you
to
oh,
just
curl
this,
the
shell
script
and
pipe
it
to
bash.
B
You
know:
that's
that's
a
really
a
silly
thing
to
do,
because
you
have
no
idea
what's
in
that
shell
script
and
similarly,
even
though
SDO
is
an
open-source
project
and
it's
totally
safe,
you
should
know.
What's
going
on
so
I
highly
recommend
that,
in
between
this,
in
between
this
first
line
of
Sto
CTL
cube
inject
and
the
cube
CTL
apply,
really,
you
should
run
a
diff
of
your
version
of
that
yeah
Mille
and
the
sto
version.
You
should
see
take
a
look
at
what
changes
are
being
made.
They'll
really
help
you
understand.
B
What's
going
on
with
this
deal
once
you
understand
that
difference,
then
of
course
go
ahead
and
apply
and
you'll
you'll
get
that
rollout
with
issue
the
other
way,
though
the
more
standard
way
of
implementing
sto
is
through
kubernetes
mutating
webhook
admission
controllers.
Really
it's
just
a
fancy
way
of
saying:
there's
a
web
hook
within
kubernetes
that
allows
for
auto
sidecar
injection,
so
you'll
just
namespace
your
service
with
the
with
the
sto
namespace
to
allow
it
to
identify
what
you
want
to
auto
injector
and
then
because
of
the
web
hook.
B
So
just
like
kubernetes
is
a
service
Orchestrator.
It's
service-oriented!
Yes
do
as
a
service
mesh,
and
so
it
is
also
service-oriented.
So
the
first
thing
that
we
would
do
to
do
a
canary
deployment
is
over.
We
set
up
a
virtual
service.
This
is
essentially
similar
to
setting
up
a
service
and
kubernetes.
It
tells
us
here
that
we
have
a
service.
That's
going
to
be
running
this
one's
really
basic.
It's
just
pointing
out
that
I
do
have
a
service.
I've
got
a
basic
route
there.
B
The
second
step
is
to
set
up
a
destination
role,
and
this
is
what
I
like
to
call
the
glue.
This
is
the
glue
between
sto
and
kubernetes.
So,
really,
all
it's
doing
here.
It's
going
to
be
tying
my
virtual
service
and
sto
to
a
deployment
in
kubernetes.
So
here's
just
some
some
really
bare-bones
Yambol
highlighting
the
important
parts,
but
this
is
how
it
connects.
B
So
it's
really
just
going
to
connect
the
name
of
a
subset
and
a
version
to
that
template
metadata
for
my
kubernetes
deployment,
really
pretty
simple,
but
you
can
see
because
this
is
so
simple
with
that
destination
role.
If
I
wanted
to
do
a
canary
deployment
well,
then
I
would
simply
do
a
secondary
deployment
like
I
had
before
with
kubernetes,
but
rather
than
telling
kubernetes
are
letting
kubernetes
randomly
split
that
I'm
going
to
put
that
entry
into
that
destination
rule.
B
So
again,
I'm
going
to
apply
that
clue
and
make
that
a
second
deployment
available
to
test
you
and
then
from
there.
Really.
Everything
again
is
service-oriented
so
I
get
to
control
everything
through
my
service,
so
here
I
can
set
up
some
additional
routing
right.
So
in
this
case,
I
said
that
it
weight
is
80%.
Gonna
go
to
my
version.
1
of
my
application
and
20%
is
gonna,
go
to
version
2,
but
as
I
mentioned
there
were
all
these
great
advice
about.
B
You
should
use
representative
sets,
and
you
should
think
about
various
things
and
that's
the
fantastic
thing
that
issue
provides
us.
It
really
gives
us
that
ability
to
add
additional
layers
so,
rather
than
just
setting
us
static
weight
of
a
percentage,
we
can
actually
do
things
like
inspect
headers
and
look
at
cookies.
Things
like
that,
so
we
can
target
logged
in
users
or
we
can
use
a
Goyt
lookup
service
to
target
users
from
a
particular
part
of
the
world.
B
So
what
else
can
a
CEO
do?
Obviously
it
can
do
tons
of
other
things
that
was
just
the
simple
canary
deployment
or
the
cyclic
routing
rule
for
forests
EO,
but,
as
mentioned
service
meshes
like
his
do,
provide
you
a
lot
timeouts
and
retries
feel
less
policy
enforcement,
metrics
and
monitoring
things
like
that.
So
take
a
look
at
the
documentation
and
play
with
this
view.
So
just
a
quick
recap
service
meshes
give
you
a
lot
more
control
than
what
you'll
get
out
of
the
box
with
kubernetes
and
what
you'll
get
from
kubernetes
itself.
B
Kubernetes
is
a
fantastic
project
and
it's
nicely
limited
to
the
core
things
that
it
does
really
well
but
serve
as
meshes
like
sto
will
give
you
the
control
that
you
need
when
it
comes
to
canary
deployments.
Think
about
representative
sets
think
about
your
end-users.
How
are
they
actually
using
your
product
and
then
think
about
how
you
can
granularly
scale
them
up?
B
I'm
gonna
come
some
monitoring.
You
absolutely
need
monitoring
if
you're
doing
canary
deployments,
if
you're,
not
monitoring
your
Kara
nary
deployments
are
going
to
be
completely
useless,
so
ensure
that
you're
using
tags
or
you're
using
labels.
If
you're
using
prometheus
think
about
outliers,
you
know
the
things
that
don't
look
like
the
others
and
the
anomalies.
Take
a
look
at
your
past
history
of
your
metrics.
B
How
are
you
resources
doing
and,
if
they're
doing
fine,
if
what
they're
creeping
up
gives
you
a
chance
to
scale
up
and
then
finally,
my
number
one
takeaway
point
go
play
with
is:
do
is
see
as
a
fantastic
open
source
project.
It's
making
a
lot
of
headway.
It's
super,
exciting,
I,
think
you'll
really.
A
A
B
B
So
then
you
you
have
to
drill
down
into
what
are
the
resources
that
provide
that
right?
It's
probably
going
to
be
a
web
application
that
web
application
is
composed
of
a
data
store
and
a
web
server,
and
maybe
some
you
know:
application
like
a
Python
script
or
a
ruby
script.
Things
like
that,
so
you
drill
in
and
then
see
check
out.
The
events
that
have
changed
right,
like
has
anything
changed
about
this,
that
we
weren't
expecting
and
they
continue
to
drill
down
right.
So
those
resources,
things
like
the
data
store,
become
your
work
metric
right.
B
So
what
does
a
data
store
need
to
run?
Well,
it
needs
disk
space
and
it
needs
processor
right.
So
you
start
to
drill
down
so
I
highly
recommend
that
you
go
from
a
top-down
approach
right.
What
am
I
users,
seeing?
What
am
users
experiencing
figure
out
the
things
that's
or
that
and
then
drill
it.
Look
at
each
of
those
and
continue
to
drill
down
is
usually
the
best
way
to
do
that.
A
B
Yes,
so
the
nice
thing
is
because
everything
is
you
know,
API
based
really,
you
know
with
both
kubernetes
and
Sto.
You
know
we
generally
think
of
interacting
with
them
as
llamo,
because
llamo
files
are
easy
for
humans
to
read
and
to
edit,
but
at
the
end
of
the
day,
it's
really
an
API,
that's
driving
it.
So
if
you,
if
you
want
it
to
automate
canary
deployments,
you
know
you
could
use
a
tool
like
within
data
dog.
B
We
have
a
web
hooks
integration,
you
know,
so
you
could
use
something
simple
like
web
hook
stuff
when
I
do
a
canary
deployment
wait
for
a
certain
amount
of
time.
Modern
or
my
you
know
certain
metrics.
If
they
all
look
good,
then
you
can
automate,
you
know
going
forward
or
you
can
automate
going
backward
by
invoking
the
web
hooks.
So
you
know
it's
it's
different
with
all
monitoring
products,
but
generally
within
data
dog.
B
B
B
That's
a
great
question
and
I'm
not
too
familiar
with
liquid
base,
so
yeah
since
I'm
not
familiar
with
the
tool,
you
know,
I'm,
assuming
what
you
mean
is
well
I,
can
make
assumptions
about
what
you
mean
by
dynamic
database
but
again
I'm
not
too
familiar
with,
what's
being
referred
to
there.
So
I
think
I
should
probably
pass
on
that.
But
whoever
asked
that
question
happy
to
dig
into
that
further.
So
my
email
and
my
twitter
handle
are
on
screen,
feel
free
to
ping
me
on
one
of
those
those
methods
and
I'd
be
happy
to.
B
B
B
Yeah,
so
within
a
CD
pipeline
you
know
you
could
automate
it
again
because
everything's
API
driven,
so
you
could
automatically
run
a
canary
deployment
but
generally
I'm
very
much
a
fan
of
continuous
integration,
but
not
continuous
deployment.
So
when
it
comes
to,
actually
you
know,
get
committing
your
code
and
pushing
that
I.
B
Don't
think
that
by
default,
everything
should
like
push
out
into
production,
even
though
it
is
so
I
would
say
that
I,
you
know
if
the
idea
is
to
do
a
continuous
deployment,
maybe
not
at
least
to
production.
So
generally,
because
if
you're
doing
a
continuous
deployment
to
a
staging
environment,
I'd
say
just
go
ahead,
go
whole
hog
with
that,
like
there's
generally
no
reason
to
canary
to
a
staging
environment.
B
So
that's
where
I
would
see
the
the
CI
part
going
when
it
comes
to
production.
I'd
say
that
it
should.
You
know
prepare
things
you
might
want
to
have
it
automatically
generate
yeah
mole
for
you,
but
when
it
comes
to
actually
deployment
and
running
the
Canaria
deploys
I
am
very
much
a
fan
of
manually
doing
it
simply
because
I've
seen
automation
run
away
way
too
many
times
so
that
that
might
be
my
PTSD
getting
invoked
there,
but
I
highly
recommend
that
you
just
do
things
manually
when
it
comes
to
deployments
and
and
monitoring.
B
The
process
is
really
pretty
simple.
The
preferred
way
these
days
is
to
use
a
helmet
art.
So
if
you
go
to
sto
IO
and
you
click
on
there's
a
button
that
will
have
you
download
things
and
that
download
really
is
a
bunch
of
fiamma
files
and
charts
and
the
sto
CTL
command
itself.
The
binary,
which
is
the
thing
that
just
modifies
your
yellow
files,
but
when
it
comes
to
actually
deploying
it,
it
really
is,
do
is
just
a
bunch
of
namespaced
services
and
pods
that
are
running
so
there's
nothing
magic
about
it.
B
A
B
B
It
really
comes
down
to
what
I
like
to
say
is
a
democracy
versus
democracy,
democracy
being
that
when
people
start
saying
things
that
things
generally
get
done
or
the
will
of
the
people,
open-source
is
very
much
a
duoc
recei
of
if
you
proposed
it
and
you'd
like
to
see
it
and
you
submit
code
to
the
project.
That's
how
things
get
done
right.
You
have
to
actually
start
to
get
involved.
B
For
those
who
don't
know
who
are
on
the
call
shadow
traffic
essentially
is
when
a
request
comes
in
rather
than
just
routing
that
request,
for
example,
a
canary
deployment
actually
just
like
splitting
that
and
saying
I'm
gonna
send
this
request
to
my
actual
production
version.
That's
gonna
run
in
one
way
and
then
I'm
also
going
to
send
the
request
to
you
know
a
duplicate
version
of
my
infrastructure.
B
My
application,
that's
testing
out
a
new
thing
so
essentially
allows
you
to
Canaria
deploy
without
ever
exposing
that,
canary
to
your
actual
end-users
and
I,
think
that's
a
fantastic
way
to
do
things.
I
think
that
if
it
were
implemented,
then
then,
yes,
essentially,
if
you
know
by
routing
things
to
that
shadow
version,
it
may
negate
some
of
the
need
for
canary
deployments
and
I
say
maybe
cuz
with
shadow
traffic.
Now
you
are
having
to
set
up
duplicate
environment.
A
A
B
Know
the
size
of
your
organization,
what
you're
running
that
may
be
cost
prohibitive
right
if
I
suddenly
have
100
developers
all
working
on
different
features,
multiplying
my
current
infrastructure
times
100,
even
if
just
parts
of
that
infrastructure
could
get
very
expensive,
so
really
depends
on
the
organization.
Your
application,
your
needs,
but
I,
think
shadow
traffic
is
going
to
be
a
very,
very
widely
used
tool
in
the
future.
B
That
is
also
a
fantastic
question,
partly
because
linker
D
is
a
CNC
F
project
and
surprisingly,
sto
isn't,
or
at
least
not
yet
there's
been,
there's
been
talk,
but
nothing
has
been
decided
upon
and
it
hasn't
been
applied
for
you.
So
that
said,
it
really
depends
on
who
you
talk
to
you
know.
If
you
talk
to
the
folks
buoyant
who
have
created
linker
D,
they
would
tell
you
that
that
linker
D
and
s
to
actually
work
really
really
well
together.
So
linker
D
is,
as
I
understand.
It
does
a
lot
of
the
data
plane
layer.
B
B
So
that's
a
fantastic
question:
that's
actually
something
that
data
dogs
been
working
on.
Quite
a
bit,
I
haven't
been
on
so
I'm,
not
on
the
team.
That's
actually
implementing
that.
So
I
don't
have
too
many
of
the
details
for
the
person
who
did
ask
that
feel
free
to
send
me
an
email,
I'm
happy
to
connect
you
with
with
some
of
the
engineers
on
that
team
and
they
can
dive
into
it
a
little
bit
more.
B
But
as
I
understand
it,
you
know
sto
really
with
multi
clusters.
The
the
trick
just
becomes
where
you
want
that
that
control
plane
to
be
managed.
So
that's
something
that
you
know
they're
working
on
with
that
project.
There
is
some
documentation
around
that,
but
again
I'm
gonna
say
hit
me
up
I'm
happy
to
connect
you
with
the
engineers
on
our
team
who
are
actually
doing
that
in
production.
B
Yeah,
so
generally,
you
know,
sto
is
going
to
be
handling
all
of
the
load,
balancing
and
routing
for
you
I'm.
So
if,
if
the
idea
is
you
know,
if
you're
in
public
cloud
and
you've
got,
you
know
a
load
balancer
all
provided
by
by
your
public
cloud,
and
really
you
just
want
to
point
your
traffic
directly
to
to
sto
SEO
has
its
own
ingress
and
egress
components
that
will
handle
all
of
that
for
you.