►
From YouTube: Next Generation of CI/CD: Analytics-driven Traffic Management on Kubernetes - Fabio Oliveira, IBM
Description
Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Next Generation of CI/CD: Analytics-driven Traffic Management on Kubernetes - Fabio Oliveira, IBM
We will describe and demonstrate how to architect a CI/CD solution to upgrade Kubernetes services while relying on fully automatic traffic management driven by principled metric and success criteria analysis. Our solution relies on Istio’s support for traffic management and on iter8 (https://iter8.tools), a new open-source Kubernetes controller and analytics engine. Iter8 automatically shifts user traffic to a canary version as it becomes increasingly confident in the canary’s performance and correctness, and it rolls back to the current version if things go bad. By way of example, we will show how IBM Watson Health migrated to our proposed solution and the lessons learned in the process.
https://sched.co/Zep8
A
A
A
What
I
want
to
do
is
to
raise
awareness
of
a
fundamental
problem
that
is,
at
the
core
of
techniques
such
as
canary
releases,
a
b
testing
in
abn
testing,
but
a
problem
that
has
been
largely
ignored
in
the
community.
So
I
want
to
offer
a
solution
to
the
problem,
an
open
solution
to
the
problem
and
engage
the
community
in
that
discussion,
and
yes,
in
this
process,
I'm
going
to
show
you
two
demos.
A
We
want
to
find
ways
to
deliver
code
to
production
as
quickly
as
we
possibly
can
in
order
to
survive
in
a
software
driven
market.
We
want
to
stay
relevant.
We
want
to
retain
the
client,
the
customers
that
we
have.
We
want
to
attract
new
customers,
but
there
are
opposing
forces
that
make
us
think
twice
before
releasing
new
code
into
production.
A
A
A
A
What
users
want
we
have
the
opportunity
to
learn
how
to
find
new
ways
to
increase
our
company's
revenue,
and
how
about
that,
as
we
learn
all
those
things
how
about
maximizing
revenue
for
a
company?
What,
if
I
told
you
that
these
things
can
be
possible
and
you
you're,
probably
mapping
to
your
mind
the
concepts
that
I'm
talking
about
here
to
the
same
techniques
that
we
mentioned
a
few
moments
earlier,
you
know
code,
behavior,
behavior
in
in
production.
You
can
think
about
canary
releases
for
that
performance
test.
A
You
know
test
what
users
want
can
be
mapped
to
a
b
testing
or
abn
testing.
I
like
to
collectively
call
these
techniques
continuous
experimentation,
and
the
work
continues
here
is
important
because
it
has
the
connotation
that
these
things
are
done
continuously
very,
very
frequent
frequently
in
order.
It
allows
us
to
actually
embrace
that
mindset
of
always
learn
about
our
code
about
our
users
about
new
ways
to
thrive
in
in
the
market.
A
However,
in
order
to
embrace
that
mindset
with
peace
of
mind,
there's
an
important
piece-
that's
important
and
that's
analytics
now
make
no
mistake.
Continuous
experimentation
is
an
analytics
problem
at
its
core
and
a
comparative
analytics
problem.
It's
about
comparing
competing
versions
of
the
code
that
are
running
simultaneously.
A
I
have
observed
that
most
of
the
discussion
in
the
community
around
this
topic
tends
to
revolve
around
the
low-level
mechanisms
that
enable
canary
releases
and
ibn
testing
people
say:
oh
here's
a
way
for
us
to
split
the
traffic.
Hence
we
can
do.
We
can
support
canary
releases,
or
here
is
a
way
for
me
to
limit
the
users
that
will
be
able
to
see
an
experimental
version
of
my
code.
Hence
I
can
do
a
b
testing,
but
that's
not
enough.
A
Here's
an
overview
of
how
e3
works.
Imagine
that
you
have
a
number
of
versions
deployed
in
your
in
your
kubernetes
cluster
and
each
of
those
versions
have
some
metrics
associated
with
associated
with
them
performance,
metrics,
correctness,
metrics.
Perhaps
even
business
metrics,
its
rate
analytics,
is
able
to
extract
the
information
about
these
metrics
from
these
versions
and
analyze.
How
these
versions
are
behaving
and
identify
at
each
point
in
time,
which
is
the
most
promising
version
along
with
this
version
of
assessment,
it
analytics
also
provide.
A
Its
rate
controller
is
a
kubernetes
controller
that
periodically
get
assessments
from
e3
analytics
with
the
frequency
that
is
specified
by
the
user
and
applies
that
traffic
control
recommendation
and
we
continue
in
this
feedback.
Loop
was
where
metrics
are
flowing
from
the
versions
into
the
prometeus
at
times
used
database
and
it's
a
controller
gets
assassins
from
analytics
and
so
on
and
so
forth,
and
this
proceeds
until
a
version
can
be
declared
a
winner.
One
to
the
experimental
duration
finishes
with
each
version
assessment.
A
There
is
a
confidence
associated
in
that
decision,
so
it
makes
a
decision,
makes
a
recommendation
and
gives
you
how
confident
it
is
in
that
decision.
It
supports
different
traffic
control
strategies.
Progressive
is
probably
what
users
will
want
for
pretty
much
all
the
use
cases
and
on
top
of
the
smart
traffic
control
users
can
specify
some
saved
filters
to,
for
instance,
cut
off
traffic
on
failure
to
cap.
A
Another
component
of
iterate,
that's
worth
mentioning,
is
what
we
call
iterate
trend,
and
the
idea
of
this
component
is
that
it
analyzes
how
a
particular
service
is
evolving
over
time
as
different
versions
are
rolled
forward
and
it
tries
to
uncover
trends
that
developers
might
not
be
aware
of
now.
Let's
quickly
go
over
the
types
of
experiments
or
continuous
experimentation,
that
iterate
enables
the
simple
simplest
case
is
performance
test.
In
this
case,
there
is
only
one
version
involved.
A
The
idea
is
to
assess
the
behavior
of
that
version
against
success
criteria
that
the
user
decides
to
use
in
this
particular
example.
For,
for
instance,
I'm
saying
that
this
service
in
order
to
be
to
pass
the
performance
test,
it
needs
to
have
a
mean
latency
below
120
milliseconds
and
an
error
rate
of
at
most
0.001,
which
means
like
four
nines
if
availability
right
typically,
this
is
done
in
a
test
or
that
environment,
but
it
can
also
be
done
in
production.
A
A
In
this
particular
case,
relative
criteria
might
be
important.
For
instance,
I
might
want
to
make
a
determination
of
the
acceptable
behavior
of
a
canary
with
respect
to
the
baseline,
so,
for
instance,
I'm
showing
you
here
an
example
that
I
would
tolerate
for
the
canary
an
error
rate
that
is
1.05,
that
of
the
baseline
meaning.
I
am
tolerate
a
five
percent
degradation
in
the
error
rate.
A
In
this
case,
there
are
two
or
more
versions
being
compared:
there's
a
baseline
version
and
one
or
more
candidates-
and
the
idea
here
here
typically
is
to
maximize
some
type
of
reward
metric
that
is
usually
associated
with
the
business
and,
at
the
same
time,
do
so
in
a
way
that
service
level
objectives
that
you
want
to
promise
to
your
users
are
still
being
satisfied
right
so
and
a
reward
metric
is
declared
as
such
in
the
experiment
specification
and
what
it
does
is.
Will
it
will
optimize
for
that
metric?
A
A
So
let's
take
a
look
at
this
demo
now
I
will
show
you
how
to
use
iterate
to
perform
an
automated
abn
test.
You
can
see
it
written
analytics
and
iterate
controller
running
in
my
iterate
name
space
and
my
simple
application
poll
is
running
in
my
current
namespace
and
right
now
there
are
no
ongoing
iterate
experiments
in
that
namespace.
A
We
will
demonstrate
experimenting
with
three
different
versions
of
a
service
named
product
page,
currently
only
version
one
is
running
when
version
one
runs.
You
see
a
red
string
that
I
am
highlighting
on
the
web
page
right
now
version
two
to
change
that
color
to
yellow
in
version
three
change
that
color
to
green.
Now,
let's
check
the
service
metrics,
you
see,
requests
right
on
the
upper
left
mean
request
latency
on
the
upper
right.
A
A
A
Ninety-Five
percent
of
their
requests
must
be
below
500
milliseconds
and
the
winning
version
is
supposed
to
maximize
the
reward
metric
mean
number
of
books
purchased,
while
satisfying
satisfying
the
requirements
on
latency
and
air
for
the
purpose
of
this
demo,
this
abn
test
will
run
for
20
iterations
of
20
seconds
each.
We
are
now
running.
Control
apply
to
create
an
experiment
from
the
ml5
that
you
saw
earlier
and
then
we're
going
to
list
the
experiments
in
this
name
space,
and
you
see
that
that
experiment
was
created,
but
it's
in
pause
state
because
the
candidates
are
missing.
A
You
see
the
traffic
split
at
the
bottom
80
10
10,
and
the
current
best
version
is
product
page
version
2
with
33
percent
of
confidence.
That's
what
it
rate
is
currently
telling
us.
Now
we
have
a
60
20,
20
traffic
split
version.
2
is
still
the
best
version.
Every
time
it
rate
makes
a
assessment,
a
version
assessment
in
a
traffic
update.
It
generates
a
kubernetes
event,
as
you
can
see,
iterate
has
just
reported
that
the
best
version
is
no
longer
version
2,
but
rather
version
3
with
a
confidence
of
76
percent.
A
Let's
quickly
take
a
look
at
the
matrix,
you
see
that
the
mean
latency
of
the
page
version
2,
the
yellow
line
is
higher
than
that
of
both
versions,
1
and
3.
refreshing
the
bookstore's
website
a
few
times.
You
see
the
colors
switching
from
red
to
green
red
to
green
and
a
little
bit
too,
yellow
reflecting
how
the
traffic
is
is
being
split
right
now,
mostly
to
v1
and
v3,
and
not
so
much
to
v2,
which
is
the
yellow
either.
It
has
now
a
confidence
of
89.
A
That
version
three
is
the
best
version
and
you
can
see
the
traffic
gradually
shifting
from
versions
one
and
two
to
version
three.
Now
the
confidence
is
nine.
Nine
percent
and
iterate
is
pretty
much
ready
to
declare
a
winner.
So
if
you
look
at
the
matrix,
you
see
that
version
2
is
violating
our
mean
latency
criterion.
A
A
A
The
experiment
continues
with
almost
no
traffic
to
versions,
1
and
2..
That's
because
we
specify
that
the
duration
of
the
experiment
is
20
iterations.
We
didn't
specify
in
an
early
termination.
In
this
case
now
you
see
again
the
metrics.
You
see
a
reward
for
v3
higher
than
a
reward
for
v1,
and
you
see
that
v2
has
the
highest
reward,
but
it
violates
the
criterion
on
latency.
A
A
Now,
let's
do
another
demo,
so
this
time
we're
going
to
show
what
we
refer
to
as
human
in
the
loop
experimentation,
in
particular
in
this
demo,
human
in
the
loop
abn
testing.
The
scenario
in
this
demo
is
the
same
as
before:
here's
the
difference
in
human
in
the
loop,
the
user,
is
in
control
of
the
experiment.
A
Instead
of
the
trade
controller
automatically
applying
the
recommend
recommended
traffic
split
by
using
the
assessments
coming
from
iterate
analytics
in
this
type
of
experimentation.
That
decision
is
entirely
up
to
the
user
user
sees
the
insights,
sees
the
assessments
and
decides
what
to
do
with
them.
So
that's
the
idea
in
order
to
support
human
loop
experimentation,
we
have
integrated
iterate
with
kui.
A
A
A
A
Now
we
create
the
experiment,
and
once
we
do
that,
we
can
go
to
the
decision
tab
that
will
show
us
what
kinds
of
insights
it
rate
is
giving
us.
You
can
see
that
only
the
baseline
version
is
getting
traffic.
Each
version
has
the
same
probability
of
winning
and
it
has
not
enough
data
to
declare
a
winner
and
it's
giving
now
a
traffic
recommendation
of
50,
25
and
25..
Let's
apply
that
traffic
split
so
that
the
other
versions
can
get
some
traffic.
A
A
A
A
Two
is
higher
than
that
of
versions,
one
and
three,
and
it's
actually
violating
the
criteria
latency,
because
quite
a
few
requests
from
server
version
version
two
are
not
below
500
milliseconds
and
you
see
that
it
has
the
highest
reward
version
two
has
the
highest
reward,
but
it
is
violating
the
criteria
on
latency.
Therefore,
it
has
a
very,
very
small
chance
of
winning
version.
Three
in
turn
has
a
higher
reward
than
version
one
and
satisfies
the
criteria
latency.
A
If
we
get
another
assessment
now
you
see
that
we
can
finally
declare
the
version.
3
is
the
winner
with
99.8
percent
confidence,
and
the
current
traffic
recommendation
is
96
percent
of
version
three,
and
only
two
percent
to
versions
two
and
one
at
this
point
basically
version
three
is:
is
the
winner
with
high
confidence
and
we
just
applied
the
traffic
split,
and
you
see
the
same
trends
on
the
matrix
and
at
this
point
let's
take
a
look
at
the
advanced
statistics.
A
Now
we
are
showing
you,
the
credible
interval
statistic
for
each
metric
of
each
version.
That
is
the
interval
within
which
the
true
value
of
each
metric
lies
the
more
data
we
have
the
closer.
The
two
ends
of
the
interval
become:
let's
end
the
experiment
now
and
let's
choose
to
roll
forward
to
the
winner.
After
doing
that,
we
will
basically
shift
100
of
the
traffic
to
the
winning
version,
which
is,
in
this
case
product
page
version
3..
A
Finally,
I
will
show
you
the
iterate
metrics
command
on
the
kui
visual
terminal.
This
command
can
be
used
to
both
define
metrics
and
visualize
metrics.
You
are
seeing
each
of
the
metrics
that
we
used
in
our
abn
test.
Experiment.
Note
that
knee
latency
iterate
error
count
are
default,
metrics
that
come
out
of
the
box
with
iterate
the
others
are
custom
metrics
we
created
for
this
experiment.
A
All
right,
let's
summarize
what
we
have
discussed,
so
we
showed
you
iterate,
so
raise
awareness
for
a
key
problem
in
continuous
experimentation,
and
the
goal
of
iterate
is
really
to
unleash
the
power
of
continuing
a
continuous
experimentation
in
kubernetes,
and
it
does
that
through
statistically
robust
techniques
driven
by
machine
learning
for
version
assessment
and
traffic
control.
This
is
a
state-of-the-art
techniques
and
we
happen
to
have
some
scientific
work
that
we've
done
to
substantiate
that
it
currently
supports.
A
You
know
kubernetes
issue
for
control
and
and
uses
prometheus
for
metrics
and
in
addition
to
the
integration
with
kui
that
you
show
that
I
showed
you
in
the
previous
demo.
It
is
also
integrated
with
kiali
kiyali
is
the
de
facto
open
source
ui
for
the
easter
service
mesh.
So
we
have
an
iterate
extension
for
kealy
that
can
be
used
to
create
experiments
right
there
and
observe
the
experiments
as
they
run.
A
Here's
where
to
find
us,
you
have
a
link
to
our
iterates
landing
page
and
our
github
organization.
Thank
you
so
much
for
your
time,
hello,
everybody!
I
I
think
you
can
see
me
hear
me
live
now.
So
there's
a
little
bit
of
a
glitch
in
the
pre-recorded
video,
so
I
hope
you
were
able
to
follow
everything.
A
A
A
Does
this
follow
smi's
traffic
split
definitions
of
canary
be
etc?
So
we
use
so
we
have
our
own
crd
that
we
define
that
we
call
of
the
kind
experiment
and
we
use
the
issues,
control
plane
to
control
the
traffic
across
multiple
versions.
So
so
again
we
define
our
crd.
Then
we
rely
on
issue
for
control.
A
So,
let's
see
the
other
question
can
iterate
deal
with
removing
the
unsuccessful
candidates
deployment
from
kubernetes.
So,
yes,
there
is
one
option
in
our
crd
that
allow
you
to
basically
clean
up
after
the
experiment
is
done,
in
which
case
it
would
remove
the
versions
to
which
traffic
is
no
longer
sent
after
the
experiment.
A
A
So
if
you
kind
of
look
at
all
the
details
of
the
of
dcr,
you
should
see
some
some
information,
some
information
there
and
during
the
experiment,
of
course,
the
richer
information
there
and
we,
we
kind
of
admit
every
time.
There's
a
decision
made
there's
a
new
version
assessment
with
with
the
new
confidence
we
emit
the
kubernetes
event
doing
that,
so
we
can
also
see
through
through
that
anyone
is
using
for
production.
Did
we?
We
have
some
proof
of
concept
engagement
with
some
some
users
and
we're
we're
getting
there.
A
What
kinds
of
trends
can
trends
module
detect
so
right
now
we
can.
Basically,
we
rely
on
the
metrics
that
it
comes
out
of
the
box
with
iterate
process
plus
utilization
metrics.
You
can
see
if
there
are
spikes
anomalies
in
those
in
those
metrics
right
now.
So
if
you
see
something
is
deviating
from
the
normal
and
we
do,
we
did
some
some
as
part
of
the
iterate
umbrella
project
umbrella.
A
How
many
experiments
is
possible
to
run
at
the
same
time?
The
limit
is,
it
depends
on
it,
there's
no
limitation,
in
it
rate
per
se,
limiting
the
number
of
experiments
that
can
happen.
At
the
same
time,
it
boils
down
to
yokuberneti's
cluster
in
terms
of
resources,
etc.
So
I
think
we
are
running
out
of
time.
I
I
will
be
engaging
with
you
folks
in
our
in
the
two
dash
cubicle
cicd
icd-slack
channel,
to
answer
further
questions
and
engage
with
you
further
thanks
for
watching.
Thanks
for
engaging.