►
From YouTube: Reliable Application Roll Out and Operations with Istio [B] - Lin Sun, IBM & Mandar Jog, Google
Description
Reliable Application Roll Out and Operations with Istio [B] - Lin Sun, IBM & Mandar Jog, Google
Do you wish for a premium user experience as your team delivers new code to production with speed and agility? Have your users been caught off guard with new feature and enhancement continuously delivered to the cloud? Do you have services go dark while other service can’t handle graceful degradation? Do some of your services get overloaded with too many requests occasionally? If you have experienced some or all of these concerns, we have the answer and it’s service mesh with Istio!
Join us to hear how you can leverage Istio’s intelligent routing feature to roll out newer version of services, apply rate limiting to your services to shed load and gracefully degrade as services fail, and visualize the dependencies and flow of traffic between services to provide optimized user experience to your users as you continuously deliver new updates to your microservice based application.
About Mandar Jog
Mandar Jog is a senior software engineer at Google working on Istio.
Mandar is specifically focussed on Istio configuration and Mixer.
Prior to Google, Mandar has led an engineering team at a startup and worked at Pivotal helping customers deploy Cloud Foundry. Mandar has been a user or a creator of application rollout technologies for years, and believes in user focussed design.
About Lin Sun
Lin is a core contributor and maintainer on Istio. She is passionate about new technologies and love to play with them. She is a master inventor, currently, holds 100+ patents filed or pending with USPTO along with hundreds of articles published at IP.com.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
Hi
good
afternoon,
let's
go
ahead,
get
started.
Sorry
we're
running
a
little
bit
late,
oh
I'm,
so
honored
to
see
many
of
you
on
a
Friday
afternoon.
So
really
appreciate
you
guys
are
still
here.
So
let
me
quickly
introduce
myself,
Lindsey,
Mitchell,
contributor
and
I
started
working
on
is
do
not
so
long
ago
in
August
of
this
year,
so
I'm
also
working
for
IBM
hi.
B
A
Great
yeah,
because
we
want
to
use
that
information
to
help
us.
So
let's
talk
about
comment
about
challenges
right.
So
what
we're
really
trying
to
do
with
this
session
is
to
share
with
you
guys
some
of
the
challenges
were
foresee
and
how
SEO
can
actually
help
you
solve
those
challenges.
So
we
really
try
to
connect
the
dots
for
you.
A
I
got
been
telling
my
dad
booking,
for
there
was
one
thing:
I
really
don't
like
book
info,
because
the
red
star,
ISO,
Chinese
I,
have
an
issue
with
red,
because
red
means
I
hate
you,
and
so
we
recently
did
a
new
version
of
book
info
and
we
change
to
Israel
blue,
and
he
saw
both
of
myself
wearing
is
still
blue.
So
we
want
to
change
that
to
a
different
color,
with
a
different
shape,
with
a
heart
shape.
So
how
do
you
actually
have
a
new
version
rolled
out,
but
you
can
still
continue.
A
Have
your
user
working
on
the
previous
version
and
you
only
not
exposed
the
new
version
to
anybody
else.
So
that's
the
biggest
challenge
number
one
and
the
second
challenging
is,
as
the
new
version
being
rolled
out
to
production
or
tester
stage.
How
do
I
do
a
be
testing
right,
so
I
was
telling
a
Matt
I
wrote
this
newer
version
of
sto
blueheart
for
the
review
service
of
book
info,
but
I
don't
want
to
roll
out
here
every
single
one
of
you.
A
You
know
I
just
want
to
roll
out
you
user
myself,
because
I
want
to
be
the
pilot,
testing
and
test
it
out
right.
So
this
is
actually
not
possible.
If
you
use
Q
Bonetti
by
itself,
you
can't
really
roll
out
a
new
version
of
your
microservice
and
without
really
testing
the
newer
version
and
not
impacting
your
existing
of
anything.
So
that's
the
second
challenging
we're
trying
to
ourselves.
The
third
challenging
we're
trying
to
solve
is
Canaria
testing,
as
you
get
comfortable
with
a
be
testing
as
I
get
comfortable
with
the
new
review
version.
A
I
wrote
about
the
sto
blue
hot
I
want
to
roll
out
to
maybe
a
percentage
of
my
user,
because
you
know
I'm
happy
with
it
and
I
want.
Maybe
10%
of
my
user
started
testing
it
right,
so
I
want
to
make
sure
the
10%
user
can
actually
be
happy
and
in
the
picture
so
that
we
can
actually
roll
out
you
a
much
wider
version
of
the
way,
the
percentage
of
the
user.
So
these
are
the
different
challenges
we
feel
like.
Seo
can
really
step
into
your
help.
A
You
could
do
it
with
cuban
at
ease
with
a
percentage,
but
you
have
to
be
really
really
good
at
a
mess,
and
actually
you
have
to
really
watch
your
system
really
closely,
because,
let's
say,
if
I
want
you
to
say,
90%
of
my
traffic
goes
to
my
existing
version
and
10%
of
my
traffic
goes
to
my
new
version.
Then
I
have
to
really
look
at
my
cuban
and
he
deployments
on
their
count
on
the
replicas
sets
to
say.
A
Okay,
I
need
to
make
sure
my
older
version
has
a
nine
ratio
to
my
ratio
was
my
new
version,
so
you
have
to
count
it
very
closely,
but
with
this,
do
you
don't
really
have
to
do
that?
So
there's
also
some
of
the
other
challenges
right.
What
if
some
of
your
service
goes
down?
Well,
how
do
you
actually
inject
foot
into
your
application
to
simulate
their
foot
exists
in
the
test
or
staging
environment
before
you
even
roll
out
to
the
production
environment?
What,
if
your
team
member,
are
writing
your
micro
services
in
different
language?
A
That's
the
biggest
thing
people
are
advocating
for
micro
services
right
people
have
their
skills
and
language
and
they
could
choose
to
write
it
in
their
preferred
language.
And
how
do
you
want
you?
Have
people
actually
read
you
everything
in
different
language
to
do
the
retry
and
to
do
the
in
check
for
trace?
So
that's
a
lot
of
work
and
they're
not
implementing
a
consistent
way.
The
other
thing
is:
how
do
we,
our
service,
actually
handle
certain
rate?
A
So,
as
the
version
3
it's
putting
some
of
the
load
on
the
system,
it's
not
actually
impact
the
user
behavior
of
the
existing
version
and
also
I
heard
this
word
observability
she's
pretty
hard
to
pronounce
for
me,
but
I
hear
it
throughout
the
conference
this
week.
It's
like
how
do
you
actually
gain
observability
of
your
applications
as
its
deployed
into
your
cluster,
so
you
guys
already
know
it
still
so
really
quick.
A
This
is
a
project
launched
by
IBM,
Google
and
lyft
since
May
this
year,
not
really
so
long
ago,
and
we
really
are
trying
to
bring
sto
to
solve
these
problems.
We
were
just
mentioning
early
on
intelligent
routing
and
load
balancer,
so
user
can
quickly
do
a/b
testing
and
be
able
to
do
canary
testing
before
they
actually
change
their
version
to
the
particular
version
they
liked
you
it
a
pretty.
It
helps
you
to
do
resilience
testing
across
different
language
and
platforms.
A
So,
regardless,
whether
you
are
writing
Java
or
Python
or
Ruby
unreal,
we
have
the
same
envoy,
sidecar
inject
for
you
and
control
the
traffic
for
you.
You
can
also
do
policy
enforcement
on
the
entire
service
match.
In
fact,
we're
going
to
talk
about
rate
limiting
as
an
example
of
the
policy
enforcement
very
soon
and
then
a
tonometry
and
reporting
observability.
This
is
actually
my
favorite,
because
I
feel
like
this
is
the
hot
problem
of
DevOps
right
without
knowing
exactly
what's
going
on
with
the
system.
It
doesn't
matter
really.
A
You
know
how
your
system
is
doing,
because
you
have
no
visibility
into
that
are
components
of
ST
OSIS.
You
guys
already
visit
is
your
sessions.
Many
of
you
are
really
quite
try
to
run
this
through
quickly.
Just
so,
you
guys
know
a
lot
of
this.
The
information
is
already
available
and
it's
still
that
I
also
feel
free
to
visit
there.
A
So
the
way
I'm
trying
to
think
about
envoy
is
actually
we
have
a
charter
envoy
is,
is
the
sidecar
that
we
use
in
Israel
to
inject
into
every
single
of
the
micro
services?
And
the
reason
we
pick
envoy
is
it's
really
better
test
and
lift
it
has.
Lifter
has
100
plus
services,
it
has
like
ten,
nine
10,000,
VMs
really
being
tested
really
well
with
two
million
requests
per
second.
A
So,
on
the
right
side,
you
can
see
what
I'm
worried
already
are
provides
and
also
what
is
still
actually
as
part
of
the
sto
work,
how
the
SEO
team
actually
contribute
to
envoy
at
the
bottom
of
the
right
side,
so
getting
down
to
the
architecture
of
Israel
as
the
traffic
coming
through
the
internet.
As
you
visit
your
micro
services,
that's
deployed
into
the
mesh,
you
can
see.
The
traffic
goes
to
the
sto
ingress,
which
is
the
envoy
site
carcasses
on
the
left
side,
and
as
that's
going
on
it's
going
to
hit
on
to
your
micro-services.
A
In
fact,
I
try
to
think
of
envoy
as
a
dummy
router
with
our
pilot,
because
really
Emily
doesn't
know
how
to
route
your
traffic
without
talking
to
pilot.
That's
why
this
picture
always
having
our
envoy
connected
to
pilot
and
it
always
ask
a
pilot
hey:
what's
how
do
I
route
to
the
next
end
point?
What
is
the
services
endpoint
and
which
part
should
I
reach
out
to
you
and
then
also
other
side?
Kyle
also
talked
to
mixer
as
far
as
tonometry
and
policy
enforcement,
so
they
always
ask
mixer.
A
C
A
When
I
put
its
doors
and
every
security
and
every
single
component
into
your
web
scope,
I
actually
find
out,
it's
geo
security
doesn't
talk
to
you
any
of
the
other
Israel
components,
so
that
took
me
by
surprise
and
I
find
out.
The
inner
main
reason
is
it's.
Your
security
mainly
talk
to
you,
the
Cuban
Airy
API.
To
make
sure
the
service
account
the
certificates
are
generated
properly
and
mounted,
or
properly
traffic
control.
So
traffic
control
really
helped
us
to
solve
our
challenge
in
number
one.
A
So
we
talked
about
how
do
we
roll
out
a
new
version
without
downtime
or
changing
any
of
my
existing
code
right?
So
this
example
gives
you
a
simple
route
rule
where
you
can
specify
I
want
in
my
production,
environment
I
want
100%
traffic
goes
to
this
particular
version.
So
really
simple.
You
don't
have
to
change
any
of
your
application
code.
You
can
just
apply
this
route
route
using
cube,
cuddle
or
it
staccato
and
then
boom
it
would
impact
we'd
be
taking
effect
immediately
on
your
system
traffic
steering
as
an
example.
A
Here
we
were
talking
about
how
do
I
do
a
B
testing
right,
I
wrote
up
this
new
version
of
the
review
system.
I
just
want
you
to
test
it
to
myself
or
maybe
to
my
iPhone
or
maybe
to
all
the
iPhone
users.
So
traffic
steering
is
typically
based
on
the
requests.
The
header
or
cookies
are
to
roll-out
you
a
space
with
a
set
of
user
first
and
then
traffic
splitting.
As
you
can
see,
this
is
another
route.
A
True
we
have
where
we
want
to
say:
okay,
we
want
to
route
90
percent
of
the
route
to
production
continued
and
because
the
new
thing
we
already
did
a
be
testing.
Let's
go
ahead
route,
10%
of
the
new
thing
which
is
alpha,
but
let's
go
ahead:
rock
10%
and
do
some
canary
testing
on
that
resilience.
Is
another
thing.
It's
your
add
out-of-the-box
to
your
micro-services,
so
you
can
add
our
four
tolerance
to
your
application
and,
what's
really
nice,
is
you
don't
have
to
change
your
application
code?
A
So
what
you
do
is
you
can
configure
configuration
to
specify
you
know
what
is
your
destination
and
how
do
you
want
to
configure
circuit
breaker
for
the
example
here,
like
maximum
connection
is
100
and
the
HTTP
max
requests
a
thousand,
and
it's
trying
to
say
you
know
for
every
single,
every
single
five
minutes.
It's
going
to
check
the
hose.
A
It's
going
to
check
the
edges,
you
endpoint
to
see
if
the
still
live
and
if
it
actually
has
arrow
for
seven
times
conservatively
it's
going
to
put
it
on
to
your
sleep
and
give
an
error
code
back
for
15
minutes.
So
this
is
really
effective.
When
you
need
to
inject
a
fort
into
your
system,
you
can
easily
try
it
out
how
your
system
react
to
the
Fords.
A
So
out
of
the
box
is
your
supports
our
timeout
configuration
where
you
can
say
you
know
the
timeout
is
100
milliseconds
and
you
can
also
confer
configure
how
many
times
you
want
to
retry
looking
away
example
here
you
can
configure
it
twice
3
times,
3
3
twice,
so
that
essentially
means
after
300
milliseconds
it's
going
to
reach
the
timeout
and
it
in
kind
written
a
specific
error
code
that
you
actually
pre
specified
in
the
configuration.
So
we
support
HTTP
and
G
RPC
out
of
the
bar
error
code
out
of
the
box.
A
B
Yeah
so
rate
limiting
is
this
kind
of
another
one
of
those
things
that
you
need
in
a
you
know,
microservices
and
environment,
specifically
just
even
within
a
mesh.
You
sometimes
need
to
protect
a
production
service
from
like
a
rogue
or
a
bad
deployment,
so
something
is
deployed
and
it
suddenly
starts
ddossing
kind
of
your
own
service.
So
you
definitely
don't
want
that
even
in
a
canary
situation
and
then
the
normal
API
management
rate
limits
are,
of
course,
kind
of
well-known
those
use
cases.
So
this
is.
B
This
is
just
an
example
of
how
rate
limit
is
configured.
I
won't
go
into
the
details
of
the
format,
because
actually
we
will
see.
Oh
there
is
a
demo
at
the
end,
so
we'll
see
it
some
of
it
a
demo,
but
essentially
we
will
provide
configurable
limits
and
overrides,
is
to
provide
multiple
quota
or
rate
limiting
backends.
So
there
is
one
that
ships
with
Sto
and
you
can
always
use
an
external
kind
of
enterprise-grade
late
rate
limiter.
B
If
you
so
choose
and
then
telemetry,
we
have
spoken
a
few
times,
but
we
have
to
make
sure
that
we
collect
metrics
in
an
in
a
very
systematic
way
and
sto
defines
a
standard
set
of
metrics,
which
are
what
you
would
expect
right.
So
the
mesh
is
monitoring
every
link
and
every
link
has
kind
of
a
set
of
what
you
would
call
like
obvious,
a
reasonable
metrics.
So
you
want
to
know
the
error
rates.
B
You
ought
to
know
that
the
request
rates
the
latencies,
and
then
you
also
want
to
label
them
by
what
service
is
calling?
What
other
service
and
also
service
versions,
so
at
present
kind
of
that's
the
set
of
our
standard
metrics?
We
let
you
look
at
traffic
by
like
these
different
metrics
and
there
dimensioned
by
these
kind
of
four
or
five
parameters,
so
we've
kind
of
taken
you
through
several
features
that
is
Joe,
provides
and
you've
also
attended.
Other
talks
right
in
in
order
to
use
it
to
do
reliable
application.
B
Rollout
right,
you
can
actually
use
many
of
these
primitives
and
kind
of
fit
them
together.
So
here
is
an
example
of
like
how
you
could
do
that,
and-
and
this
is
this-
is
just
a
proposal-
kind
of
you'd
like
your
comments
on
it,
but
it's
also
also
an
example
of
how
you
can
stitch
things
together
that
all
the
primitives
that
it
still
provides
and
make
it
do
kind
of
a
higher
level
task.
So
so
here
we
we
show
that
there
is
a
aniseh
to
deployment
controller
right,
which
is
similar
to
the
kubernetes
deployment
controller.
B
It
is
spawning
its
own
replica
sets
and
attaching
horizontal
port
scalars
to
those
replicas
sets.
So
now,
as
those
get
more
traffic,
they
will
be
automatically
scaled.
That's
that's
kind
of
done
by
the
autoscaler
itself.
The
key
piece
one
of
the
key
pieces
that
is
to
is
adding
here,
is
being
able
to
smoothly
change
the
traffic
percentages.
B
So
you
can
dial
up
from
1%
5%
10%,
and
then
you
can
let
the
horizontal
part
autoscaler
actually
take
care
of
scaling
the
instances
up
or
down
so
that's
kind
of
architectural
II,
separate
as
like
a
traffic
split
controller.
If
you,
if
you
will,
as
I
mentioned
before,
we
collect
a
standard
set
of
metrics,
which
means
as
this
as
this
process
is
going
on,
you
can
watch
those
metrics.
B
You
can
watch
error
rates
that
are
both
to
and
from
whichever
service,
you're
you're
trying
to
deploy
the
new
version
off,
and
then
you
can
make
decisions
regarding
whether
to
proceed,
pause
or
rollback
right.
So
those
are.
Those
are.
The
three
signals
are
kind
of
available
that
the
traffic
split
controller
will
act
on.
If
everything
is
going
well
at
every
step,
you'll
say
okay
proceed
and
then
it
will
gradually
go
to
100%,
at
which
point
we
can
the
whole
replica
and
then
now
this
is
kind
of
kind
of
a
simple
use
case.
B
B
A
This
is
a
project
I
showed
around
this
week,
so
it's
actually
a
research
project
by
the
IBM
research
team.
So
essentially,
I've
been
complaining
to
them.
How
much
I
love
about
zip
can
write,
distribute
tracing
but
I
don't
want
to
be
a
machine
and
look
at
each
single
request
through
typically
that's
just
too
boring.
So
they
come
up
this
project
that
allows
you
to
analyze
a
period
of
aggregated
trace
through
a
configurable
time
and
actually
analyze
for
you.
You
know,
what's
exactly
going
on,
so
you
don't
have
to
analyze
the
100
traces
yourself.
A
B
And
then
kind
of
tying
it
back
back
to
the
previous
slide.
This
can
also
provide
that
signal.
It
looking
at
lots
of
other
things,
it
can
say,
proceed
or
stop
so
last
step
demo
with
kind
of
or
all
standard,
disclaimers
and
prayer
to
the
demo
gods
and
and
and
all
that
and
remember,
if
it
doesn't
work,
it's
actually
live.
A
B
B
So
I
can
I
can
just
talk
well,
yeah,
okay,
so
the
book
info
app
you
you've,
you've,
seen
it
before
I
think
what
we
are
going
to
do
in
and
Lin
set
it
up
before
was
that
there
is
a
review
service
in
the
middle
and
we
are
going.
We
are
going
to
deploy
a
new
version
of
the
review
service
and
kind
of
show
you
some
of
these
pieces
at
work.
B
Okay,
thank
you
very
much
all
right.
So
that's
the
reviews,
v3
service,
it's
whatever
the
red
stars,
and
now
we
want
it
to
so
right
now,
it's
deployed
as
normal,
which
means
that
it's
going
to
round-robin
between
the
various
different,
backends
and
and
you'll
see
that
it.
So
this
is
what
we
want
to
deploy
the
blue
stars
and
then
the
other
two
versions
are
there.
This
is
just
to
show
that
we
deployed
the
third
version.
Yeah.
A
B
Okay,
so
now
what
we
will
do
is
we
will
create
a
drought
rule
which
will
only
which
will
send
all
the
traffic
for
reviews
to
version
v2,
which
means
that
we
should
stop
seeing
the
we
should
stop
seeing
all
the
other
stars,
and
we
should
only
see
the
black
stars.
So
if
everything
works
right,
we
should
see
the
black
stars
and
now
we're
seeing
the
black
stars.
So
this
is
kind
of
what
this
is
the
setup
syrup.
So
now
we
have
our
B
and
we
have
recently
deployed
v3,
but
it's
not
it's
not
visible.
B
B
B
B
B
Okay,
so
now
I've
I've
just
put
some
load
on
it.
This
is
the
rating
service
I'll.
Let
it
load
for
a
little
bit
but
yeah
as
as
you
can
see,
this
traffic
is
all
v3
right,
so
which
means
that
only
v3
is
getting
traffic
and
ratings.
V3
call
straightening,
so
they're,
seeing
all
the
traffic
here
and
no
traffic
from
the
other
side.
That
was
just
manual
traffic
now
once
this,
this
kind
of
goes
on
for
a
little
bit
and
you're
sure
about
yeah
your
application
works
or
seems
to
work.
B
B
Okay,
so
so
now
very
very
soon
we
should
see
both
of
these
please
light
up,
and
here
is
where
I'm
going
to
introduce
the
rake.
The
rate-limiting
part
right
so
now
consider
a
scenario
where
this
new
new
service
that
Lynn
just
updated,
has
like
some
tight
loop
right
has,
has
some
like
crazy,
tight
loop
that
puts
a
lot
of
load
on
the
rating
service.
B
Yeah,
so,
as
you
can
see,
even
even
though
now
traffic
is
evenly
split,
the
amount
of
traffic
that
ratings
is
receiving
from
reviews
is
completely
skewed,
because
there
is
a
title,
apparently
in
the
in
the
code
that
you
wrote:
okay,
so
so
now
so
now
what
we
have
to
do
is
we
have
to
make
sure
that
we
don't.
We
have
to
make
sure
that
that
does
not
happen,
and
the
way
to
do
that
is
to
configure
this
rate
limit
row,
which
is,
which
is
what
what
I
showed
you
earlier.
B
So
what
the
rate,
what
the
rate
limit
rule
says
is
that
if
it's
source
version
is
v3
of
the
service
only
allowed
10
requests
per
second
and
during
when
the
deployment
is
going
on,
it
will
make
sure
that
you're
protecting
your
back-end
services
as
things
are
rolling
forward.
So
we
are
going
to
create
that
rule
by.
B
So
now
here
is
the
quota
rule,
which
was
the
direct
remote
rule
that
was
just
created.
So
now,
if
I
load
the
system
again
I'm
going
to
load
it
for
a
little
bit
more
time
now,
okay,
so
now,
if
I
load
the
system
again,
I
should
see
that
the
the
ratings
service
will
never
get
traffic
more
than
ten
requests
per
second
and
I'm
actually
going
to
do
it
with
the
with
the
cookie
just
to
kind
of
bring
the
point
forward.
B
A
B
Right
so
now,
as
you
can
see,
we
3
2,
V
1,
is
receiving
all
these
429
and
429
means
it's
being
rate
limited.
So
all
the
traffic
is
being
sent
back
from
the
proxy,
it
never
actually
reaches
the
service,
and,
if
your
so,
which
means
your
actual
service,
won't
be
impacted
and
all
their
production
traffic
can
continue
going
through.
So
again,
this
is.
This
is
just
one
of
the
features
and
an
important
feature,
but
but
one
of
the
features
that
it
still
provides-
and
you
can
stitch
together
a
very
compelling
application
rollout
with
the
primitives.
B
A
A
C
A
C
B
Tcp,
okay
and-
and
there
is
so
TCP-
has
a
less
number
of
attributes
than
HTTP,
because
both
HTTP
headers
and
you
can
write
policies
on
headers.
But
yes,
we
we
also
support
TCP,
you'll
you'll,
also
see
TCP
metrics.
In
fact,
if
you
just
scroll
through
yeah,
so
there
are
no
tcp
service
right
now,
so
no
tcp
metrics.
But
if
we
had
TCP
services
in
the
mesh
we
would
have
seen
TCP
metrics
as
well,
and
then
you
can
also
write
policies
on
the
same.