►
From YouTube: Automating Load Balancing and Fault Tolerance via Predictive Analysis - Steven Rosenberg, Red Hat
Description
Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Automating Load Balancing and Fault Tolerance via Predictive Analysis - Steven Rosenberg, Red Hat
Just imagine, wouldn't it be great if Kubernetes load balancing could predict application traffic in advance and react appropriately? We’ll explain how to improve performance, reduce costs, and increase reliability in order to provide more intelligent workload balancing. The solution is Predictive Analysis which allows us to “predict” the future from historical events. We will discuss how predictive analysis can improve overall system performance while reducing costs and improve the reliability of Kubernetes and Hybrid Cloud based environments. Attendees will come away with a better understanding of cutting edge technology for solving complex problems that are fast becoming the next generation of technological advances.
https://sched.co/Zenp
A
Early
migration
may
also
include
full
tolerance
solutions
of
which
there
is
growing
interest.
The
ability
to
migrate
processes,
whether
they
are
vms
worker
nodes,
managed
by
orchestration
applications,
pods
containers
or
other
types
of
processes
provides
for
many
options
in
conjunction
with
predictive
analysis,
which
we
shall
discuss.
A
So
this
is
a
load
balancing
example
and
you
can
see
you
have
a
load
balancer
and
you
can
see
that
process,
one
which
could
be
vms.
It
could
be
pods,
as
I
said,
worker
nodes
that
runs
on
host
one
host
of
the
process.
Two
is
on
host
two
three
on
three
and
then
because
we
only
have
three
hosts
process.
Four
goes
to
host
one
and
that's
called
a
round
robin
approach,
and
that
gives
us
a
proper
load
balancing.
A
A
A
So
some
solutions
currently
are
high
availability.
Where
you
have
redundancy-
and
you
can
see
here-
you
have
a
host
one
which
is
active.
The
host
two
is
passive.
You
have
a
storage
posts
which
are
redundant,
you
have
redundant
connections
and
this
leads
to
not
only
cost
higher
cost,
but
it
also
leads
to
higher
latency
because
say,
process
two
has
to
wait
until
he
actually
receives
an
indicator
that
the
host
one
is
no
longer
really
active.
A
So
here's
an
example
of
scheduling
to
dispatch
various
processes.
So
you
see
here
you
have
a
process
eight
and
a
process,
nine.
They
could
be
anything
and
basically
there's
an
open
slot
in
host
2
and
it
opens
slot
in
host
3.,
so
8
could
be
scheduled
to
launch
or
migrate.
If
you
will
to
host
2
and
process,
9
could
be
scheduled
to
run
on
host
3..
A
But
what,
if
8,
needs
two
slots?
Well
then,
the
problem
is
that
it
has
to
wait
until
there's
a
free
slot
either
on
host
2
host
3
or
possibly,
two
processes
bring
coming
down
on
host
one.
But
if
you
have
a
more
intelligent
solution,
you
can
move
process
five
on
host
two
to
host
3
and
migrate.
That
and
then
you
can
launch
process
8
onto
host
2..
A
But
if
you
can
predict
that
process
5
is
going
to
exit
within
the
time
it
takes
to
launch
or
if
you
will
migrate
host
process
8,
you
can
start
the
launching
of
migration
of
process
8
at
the
same
time,
launch
or
migrate
process
9
to
host
3,
and
once
the
migrations
are
complete.
Host
5
will
no
longer
run
host,
8
will
be
running
on
process,
8
will
be
running
on
host
2
and
process.
9
will
be
running
on
host
3
and
that's
a
more
intelligent
solution.
A
A
We
can
consider
the
initial
launching
which
we
discussed
migration
for
maintenance.
If
a
host
needs
to
be
brought
down
for
hardware
upgrades,
you
can
migrate
the
processes
from
one
host
to
another,
there's
rebalancing
opportunities
migrating
to
another
host
in
order
to
have
a
better
load
balancing
and
for
fault
recovery
migrating
to
mitigate
the
system
process,
values,
so
policy
units
that
we
need
to
consider
attributes.
A
Then
you
weigh
the
candidate
the
host
candidates
via
scoring
based
upon,
if
it's,
the
balancing,
if
it's
even
distribution,
the
pr
the
host
with
the
most
free
resources,
will
have
a
higher
score.
If
it's
power
saving,
then
those
hosts
that
you
use
the
least
amount
of
energy
will
have
higher
higher
scoring.
A
A
Then,
while
the
process
is
still
running
on
the
source
to
remove
the
deltas,
the
current
differences
of
the
memory
until
you
calculate
the
minimum
pause
time
that
the
process
can
be
brought
down
to
copy
over
the
differences,
then
you
copy
over
the
cpu
state
and
again,
the
goal
is
to
limit
the
pausing
of
the
process,
and
then
you
can
restart
the
process
on
the
destination
host.
Once
you've
stopped
the
process
on
the
source
host
and
then
you
can
clean
up
all
the
data
on
the
source
host.
A
So
we
can
illustrate
that
through
these
transitions.
Currently
the
process
is
running
on
the
source.
We
set
up
a
synchronize,
the
disk
and
we
start
the
memory
transfer.
We
estimate
the
minimum
downtime.
While
we
continue
the
memory
transfer
and
the
deltas
that
are
accumulating
once
we
reach
the
minimum
downtime,
we
pause
the
process
on
the
source.
A
We
activate
the
network,
we
complete
the
memory
transfers.
We
then
run
the
process
on
the
destination
and
then
we
do
the
cleanup
on
the
source
so
for
live
migration
from
host
one
to
host
2
transitioning.
Considering
the
local
disk
data
on
host
1,
you
can
see
that
the
guest
process
already
migrated
to
host
2..
A
It
requests
the
data
from
the
controller
on
host
2,
which
communicates
with
the
controller
on
host
1..
It
accesses
the
storage
that
still
hasn't
migrated
yet
so
that
the
process
can
use
it
currently
and
at
the
same
time,
data
is
continually
being
transferred
from
host
one
to
host
two
via
the
controllers.
Until
all
the
data
is
copied
to
host
two,
in
which
case
you
no
longer
need
the
connection.
A
You
first
start
with
the
historical
data
and
from
that
you
create
a
training
set,
you
develop
an
algorithm
and
the
algorithm
runs
and
creates
what's
called
the
model.
You
then
use
a
test
set
in
order
to
test
the
model,
and
you
look
at
the
results
and
you
compare
the
results
to
the
expected
results.
You
calculate
the
percentage
of
error.
You'll
then
modify
the
algorithm
and
you'll
do
the
process
over
again
until
the
percentage
of
error
is
at
a
lowest
threshold
and
that's
acceptable
and
that's
called
machine
learning.
A
A
So
the
process
for
developing
a
predictive
model.
Again
we
define
the
project,
we
collect
the
data,
we
analyze
the
data,
we
validate
the
data,
we
create
a
model,
but
we
deploy
that
model.
We
monitor
the
model.
We
calculate
the
percentage
of
error
between
what
we
expected.
We
then
redefine
the
project
and
we
run
it
over
again.
So
that's
a
cylindrical
learning,
so
the
types
of
predictive
models
with
examples,
support
vector
machine
models,
our
classification
models
to
predict
a
category
examples
can
be
stock
prices,
increase
or
decrease.
A
It
can
be
yes
or
no
predictive
quantity
regression.
Examples
could
be
predicting
a
person's
age
based
upon
height
weight,
health
and
other
factors,
detecting
an
anomaly
you're,
comparing
normal
behavior
versus
exceptions
such
as
money
withdrawal
anomalies.
If
money
is
missing
from
your
bank
account,
obviously
that's
something
to
be
concerned
about
clustering.
With
this
discover
structure
in
unexplored
data.
Examples
such
as
finding
groups
of
customers
within
similar
behavior,
giving
a
large
database
of
customers
containing
their
demographics
and
past
buying
records.
A
That
is
other
techniques
that
we
can
use
in
predictive,
creating
predictive
models,
applying
predictive
analysis
and
analytics
to
schedulers.
The
criteria
for
data
are
processing,
time
iterations,
and
we
should
adjust
for
resource
capacity
and
priority
to
get
better
results.
Percentage
of
resources
used
adjusting
the
data
for
capacity
and
priority
adjusting
for
anomalies
while
calculating
averages
which
we'll
discuss
later
the
ideas.
Certain
selective
techniques
that
apply
to
other
scheduling
applications
can
be
applied
to
our
types
of
scheduling
applications.
A
A
A
A
Techniques
that
we
can
use
for
anomaly,
error
calculation
methods,
statistical
where
we
can
calculate
the
percentage
of
error
from
the
mean
and
eliminate
results
outside
of
the
threshold
in
order
to
make
sure
that
our
analysis
is
more
accurate
signal
processing
techniques.
If
we
want
to
be
more
fancy
creating
smoothing
filters
to
eliminate
glitches
and
machine
learning,
analysis
of
patterns
and
categories
categorization
between
normal
and
out
of
range
results,
these
can
all
be
used
in
order
to
eliminate
the
outliers
and
to
only
consider
the
actual
valid
data.
A
Someone
asked
if
it's
possible
to
have
slides
so,
yes,
here's
my
email
address,
you
can
send
ping
me
and
send
me
your
any
questions
you
would
like
offline
and
I
can
actually
send
you
the
slides
and
share
it
with
you.
Some
people
ask:
how
do
you
plan
in
improving
accuracy
of
short-term
non-stationary
time
series
analysis,
which
is
typical
of
network
traffic
over
day
week
due
to
sessionality
of
data?
Did
you
consider
using
autoregressive
models
to
perform
the
forecasting
such
as
ar
I
m
a
models?
A
So,
first
of
all,
this
is
a
work
in
progress.
It
hasn't
actually
seen
implementation.
It's
been
through
some
testing
where,
as
I
said,
we
look
for
the
best
solutions
and
time
series
analysis
may
not
be
the
first
phase.
First
phase
mate
for
some
of
the
applications,
such
as
fault,
tolerance,
might
actually
just
be
a
simple,
rule-based
type
algorithm.
A
A
I
hope
I
announced
that
right
and
he
had
a
second
question:
do
you
use
roc,
receive
operation
characteristic,
curves
to
evaluate
the
quality
of
the
prediction
when
using
svms
or
other
binary
predictive
models?
Well,
we
are
open
to
suggestions
and
I
can
look
through
a
lot
of
that,
but,
as
I
said,
we,
I
have
considered
filtering
of
outliers
in
order
to
get
better
predictions
and
to
be
able
to
quantify
the
to
quantify
that
the
that
the
calculations
will
be
accurate.
A
David
stirs
said
thanks
for
it
was
interesting.
Are
there
any
known,
implementations
out
there?
As
I
said,
this
is
a
work
in
progress.
Anyone
who
would
like
is
interested
in.
It
would
like
to
be
involved
in
it.
They
can
ping
me.
This
is
an
open.
This
will
be
an
open
source
project
and
we
welcome
anyone.
That's
interested
in
willing
to
participate.
A
Amer
mukhtar
said
how
do
you
decide
on
which
variables
to
use
in
learning
the
prediction
model?
Well,
as
I
said,
we
need
to
consider
the
many
different
indicators.
If
we're
talking
about.
What's
called
what
I
call
early
migration,
the
variables
were
discussed.
We
need
to
be
able
to
learn
how
long
processes
run
some
processes
are
more
produ
will
be
more
predictable
than
others.
It
depends
upon
dependencies
when
there
are
dependencies,
such
as
database
and
as
the
tables
increase.
A
Obviously
those
dependencies
will
affect
performance
and
we
can
do
calculations
to
kind
of
anticipate
that,
and
we
can
categorize
processes
based
upon
the
level
of
prediction
of
how
easily
or
how
hard
it
is
to
predict
for
fault
tolerance.
We
have
various
types
of
indicators
that
we
can
use.
One
possibility
is
degradation
of
performance,
not
just
in
the
networking
area
as
much
load,
balancers
consider,
but
also
the
cpu
percentage
and
the
memory
usage,
as
well
as
the
storage
usage
and
all
of
those
can
be
considered
for
migrating
processes.
A
A
A
So,
as
I
said,
for
some
scenarios,
simple
rule-based
can
work
and
then,
when
you
want
some
more
robust
approach,
you
can
also
choose
different
machine
learning
approaches.
It
could
be
lambstar
or
neural
network
type
of
approaches,
and
you
need
to
balance,
which
is
the
right
ones
and,
as
I
said,
you
could
use
existing
scheduling,
algorithms
that
are
used
for
human
scheduling,
their
processes
to
also
apply
some
of
that
to
these
types
of
problems
as
well.
A
As
I
said,
it
could
be
a
cpu
usage
or
cpu
percentages.
It
could
be
memory
and
storage,
it
could
be
networking,
it
can
consider
not
only
the
percentage
of
usage
but
the
size.
For
example
storage.
It
would
not
only
be
the
throughput
and
considering
the
latency,
but
also
the
amount
of
disk
space.
That's
left,
so
I
already
covered
some
of
that.
I
hope
that's
more
detailed
disha
ag
agarwal
is
there
any
online
data
sets
for
this
experiment?
A
Then,
once
you
actually
generate
the
data,
then
you
have
the
data.
So,
unlike
a
lot
of
projects
where
you
first
have
to
collect
the
data
and
then
analyze
it
this
project,
you
can
actually
generate
the
scenarios
and
then
have
the
learning
from
the
from
the
data
that
you've
generated.
So
this
is
the
unique
ideas
of
this
particular
project,
so
hermann
gonzalez.
A
Is
there
a
pre-made
tool
in
order
to
help
with
the
predictions?
What
tools
do
you
use?
Well
again,
it's
a
it
is
a
work
in
progress
and,
as
I
said,
the
whole
idea
of
this
project
would
be
you
start
small,
and
then
you
grow
big,
so
we
would
could
start,
as
I
say,
with
full
tolerance.
We
could
start
with
a
rules-based
type
algorithm.
A
So
I
have
many
ideas
for
moving
this
project
forward.
If
the
people
are
interested,
please
ping
me
send
me
contact
me
and
we
can
discuss
any
contributions
that
you
can
give
to
the
project
and
any
contributions
would
be
greatly
appreciated.
Thank
you.
If
there's
any
other
questions,
I
think
I
covered
them
all.
A
Yes,
okay,
yes,
there's
a
second
page
david
stirs.
Would
you
see
implementations
within
a
kubernetes
sig,
which
ones
also,
I
understood
it
has
to
be
combined
with
a
custom,
scheduler
or
normal
one.
Well,
as
I
said
at
the
beginning
for
kubernetes
worker
nodes
are
usually
vms
and
that's
a
very
good
candidate
we
do
have.
A
A
Xavier
sierra
have
you
consider
an
automatic
training
approach
like
auto
ml,
teapot,
etc.
We're
open
to
all
kinds
of
ideas,
and
we
can
certainly
consider
those
options
as
well.
Faizi
gabimi.
Do
you
think
that
a
deep
learning
method
can
predict
anomaly
in
the
cloud
network
and
detect
anomalies
for
different
types
of
applications?
Yes,
I
do.
A
If
you
have
the
proper
models,
you
can
certainly
predict
when
there's
anomaly
in
the
cloud
network,
so
long
as
there's
consistent
patterns
and
that's
the
whole
idea
is
to
be
able
to
detect
the
consistent
patterns
to
be
able
to
train
the
the
application
in
order
to
differentiate
between
the
normal
operation
and
the
anomalies.
So,
yes,
I
believe
so.
A
A
A
Can
you
name
a
few
simulation
software,
perhaps
like
gns3?
Well,
there
are
many
software
solutions
for
creating
stress.
There
are
stress,
testing,
applications
and
things
of
that
nature,
so
that
you
can
generate
the
data
and
then
from
the
data
you
can
also
from
that
data
that
you
generate.
You
can
then
also
use
that
data
in
order
to
train
and
I'm
sure,
there's
many
other
software
or
applications
out
there
for
for
simulating.
A
A
Questions:
okay,
also,
I'm
told
that
I
can
upload
my
sl
slides
into
the
schedulers
and
then
all
the
attendees
can
download
them
there.
So
I'll
also
consider
that
as
well.
Okay,
so
it
isn't
anything
else.
If
there's
any
other
questions,
thank
you
very
much
for
joining
and
I
hope
it
was
informative
and,
as
I
said,
if
anyone
wants
to
get
involved,
please
contact
me
and
we
can
discuss
there's
so
many
good
ideas
in
these
questions
I'll
certainly
consider
a
lot
of
them.
Thank
you.