►
From YouTube: Edge Relocation - Smart Workload Migration Across Multiple Clusters- Grzegorz Panek & Piotr Matysiak
Description
Edge Relocation - Smart Workload Migration Across Multiple Clusters - Grzegorz Panek & Piotr Matysiak, Orange Polska
With the growing development of 5G and its new services, Edge computing is becoming the cornerstone of the ongoing network transformation making Edge application orchestration across multiple kubernetes clusters increasingly important. In this perspective, new operational challenges need to be addressed to deal with the stringent requirements of these applications and their users’ mobility. In this context, applications relocation between Edge clusters is key to maintain their QoS throughout their lifetime while achieving Edge infrastructure load-balancing. However, despite such an analysis, Edge relocation continues to be a gap in the industry. Hence, an open source cloud-native solution to resolve this problem has been implementated. A zero down-time Edge Relocation procedure offers the capability to transparently relocate an Edge application running on a source kubernetes cluster to a target kubernetes Cluster. This solution has enriched* Edge Multi Cluster Orchestrator (EMCO) from Linux Foundation. Additionally, we have implemented an extension mechanism to load-balance MEC clusters, by observing current resource utilization on each Edge Cluster, searching for least loaded Edge Clusters and to relocate Edge Apps respectively. * https://rb.gy/mmnqsb
A
Yeah,
hello,
welcome,
I'm,
really
glad
that
you
decided
to
join
our
session
today.
I
hope
you're,
enjoying
this
thing
at
Amsterdam
and
you're
enjoying
kubecon
this
year.
My
name
is
Gregor
spanak
and
I'm,
a
research
engineer
for
the
orange,
and
today
we've
got
matishak.
We
are
going
to
share
with
you
the
output
of
our
project
called
Edge
relocation.
So
basically
we
are
going
to
talk
today
about
the
seamless
migration
of
the
cloud94
clouds
across
multiple
Edge
clusters.
A
multiple
Edge
servers.
A
So,
first
of
all,
I
will
give
you
some
motivation
from
our
side,
how
we
see
the
edge
and
why
we
are
taking
care
about
the
edge
and
later
on.
I
will
give
the
floor
to
the
pure.
That
will
give
you
a
deeper
view
on
the
technical
stuffs,
how
we
are
managing
multiple
clusters
and
how
we
implemented
a
actual
location
procedure.
And,
finally,
I
will
give
you
some
notes
on
the
future
research
direction
of
our
project.
A
So
currently
orange
is
deploying
its
own
5G
Network
5G
connectivity,
because
we
are
believe
that
the
edge
Computing
is
not
enough
to
achieve
the
low
latency
Communication
in
order
to
take
profit
from
the
low
latency
communication
for
the
S
Services.
We
need
as
well
really
rapid
radio,
Access
Network
to
the
edge
services
and
we
think,
like
integration
of
the
5G
and
Edge
infrastructure,
will
give.
You
will
give
you
and
will
give
us
another
whole
opportunity
to
take
profit
from
the
low
latency
communication
for
the
edge
computing
nowadays.
A
More
and
more
Partners
industrial
partners
are
requesting
are
asking
us.
Are
you
already
able
to
to
deploy
our
own
services
on
your
Edge
infrastructure
and
during
these
exercises
with
industrial
Partners,
we
noticed
that
a
lot
of
the
or
majority
of
the
use
cases
that
the
partners
are
developing
are
characterized
not
only
by
the
low
latency
communication,
but
as
well
about
the
end
user
Mobility.
A
A
However,
as
we
know,
the
users,
the
cars,
are
moving
and
are
approaching
different
geographical
location.
So
it
turns
out
that
once
the
user
will
change
its
position,
it
should
be
hand
off
to
the
another
genome
between
other
Access
Network
station.
However,
the
The
Edge
cluster
that
is
closest
to
the
targeted
Edge
it
in
targets,
its
radio
access
network,
is
not
ready
to
handle
the
end
user
request
because
the
edge
application
is
not
there
yet.
A
That
will
solve
the
problem,
but
we
did
exercises
with
our
operational
team
and
it
turns
out
that
the
deployment
of
the
edge
infrastructure
is
really
costly
and
we
need
somehow
to
optimize
the
utilization
of
the
resources
at
the
whole
topology.
That's
why
we
decided
to
Implement
some
procedure
that,
based
on
the
follow
me
approach?
Okay,
so
if
the
user
is
requesting
to
access
some
application,
we
are
just
on
demand
creating
this
application
there.
A
But
it's
not
only
the
one
reason
why
we
need
to
take
care
about
the
relocation
as
relocation.
We
should
guarantee
as
well,
for
instance,
the
load
balancing
for
our
resources
in
order
to
increase
the
capacity
of
the
system.
We
can
imagine
that
one
of
our
Edge
servers
would
be
off
due
to
I.
Don't
know,
maintenance
reasons,
power
outage
or
something
like
that
or
we
are
just
wanted
to
to
move
our
application
or
move
our
infrastructure
to
the
provider
at
other
cloud
provider.
A
So,
just
before
going
flirts,
the
pl
try
we'll
give
you
a
view
on
the
our
real
objectives
of
the
project
and
the
real
contributions
to
to
solve
this
project.
So
in
fact,
we
implemented
two
procedures,
the
first
one
we
called
Azure
location-
and
this
is
about
the
seamless
migration
of
the
containerized
application
between
the
Clusters
and
the
second
that
we
called
life
cycle
management
of
the
edge
application.
And
but
in
fact
this
is
the
life
cycle,
management
of
edge
application
and
the
end
user
management,
because
this
procedure
is
observing
all
the
time.
A
Thanks
to
the
5G
Network,
it's
observing
all
the
time,
the
user
positioning,
the
radio
conditions
it's
observing
as
well
the
edge
infrastructure
utilization
and,
based
on
this
observability
stuffs,
we
are
taking
a
decision
whether
to
migrate,
migrate,
our
application
or
to
stay
at
the
source,
Edge
cluster.
So
to
implement
that
we
implemented
some
smaller
proof
of
Concepts
that
we
can
mention,
for
instance,
observability
controller.
A
That
is
all
the
time
monitoring
the
resource
utilization
at
kubernetes
clusters
and
we
implemented
as
well
topology
controller
that
is
taking
care
about
where
the
you,
where
the
edge
cluster
currently
is
located,
maybe
not
not
just
is
taking
care
about.
The
edge
infrastructure
is
aware
about
whether
the
cluster
is
overloaded
or
not.
And
finally,
we
implemented
the
placement
controller,
which
implements
some
mathematical
modeling.
Some
mathematical
optimization
on
analysis,
whether
to
migrate,
our
application
or
not
I
will
give
you
more
detail
about
all
the
controllers
in
the
next
slides.
So
now
the
floor
is
yours.
B
Thank
you,
Greg,
and
it's
a
pleasure
to
be
here.
I
would
like
to
talk
about
why
we
are
at
this
event
and
how
we
place
at
this
event.
So
we
talk
about
the
edge
you
can
imagine
that
we
have
different
types,
different
layers
of
edge,
far
Edge
mid
Edge,
close
edge
of.
If
you
will
a
cloud
fork
or
just
Edge
or
you,
you
can
have
different
type
of
layers.
B
You
know
you
call
it
as
as
you
want,
and
on
top
of
that,
you
would
like
to
have
some
Control
Management
plane,
some
kind
of
orchestrator,
so
we
are
at
the
edge
and
to
place
ours
at
the
kubernetes
event.
We
assume
that
each
Edge
cluster
is
a
kubernetes
clusters.
It's
it's
separate
cluster,
with
its
own
management,
with
its
separate
control,
plane
and,
of
course,
separate
worker
nodes,
and
if
you
will
also,
the
cluster
may
be
from
different
providers.
B
Different
kind
of
kind
of
clusters,
number
metal
clusters
managed
clusters
Etc
and
for
the
orchestrator
I,
don't
want
to
go
deep
dive
into
why
we
took
the
MCO
which
stands
for
Edge
multi-cluster
orchestrator.
But
what
we
like
about
Emco
is
that
it's
Intel
based
there
are
a
lot
of
type
of
intents.
The
the
most
important
is
for
us,
the
placement
intent.
So
it's
crucial
to
remember
that
we
have
placement
intents
in
the
MCO
and
also
it's
composed
of
many
different
controllers.
B
Again,
the
placement
controllers
are
the
most
important
for
us
and
Amco
introduced
some
mechanism
to
take
a
lot
of
clusters
from
different
provider
and
different
types
of
clusters
under
under
an
umbrella
called
logical
cloud,
and
then
we
can
deploy
on
those
clusters
application
applications
using
MCO
apis.
You
can
deploy
application
on
single
cluster
on
the
set
of
cluster.
It
depends.
B
What
we
do
not
like
about
Amco
is
the
lack
of
the
functionalities
that
we
would
like
to
have
for
our
use
case.
So
we
extended
the
MCO
with
the
relocation
workflow.
As
being
said,
and
in
fact
the
relocation
workflow
itself,
it's
it
can
be
defined
in
the
MCO
via
endpoints
as
an
intent.
So
in
the
intent
we
can
specify
what
application
should
be
relocated
where
to
which
cluster
and
if
we
deploy.
B
If
we
invoke
such
relocation
workflow
the
following
procedure
will
be
will
be
proceed,
so
we
will
collect
the
information
about
the
placement
intent
based
on
the
relocation
intent.
We
will
update
it.
Of
course
the
changes
will
be
applied
and
the
reconciliation
will
be.
We
will
take
take
turn,
but
we
will
not.
We
will
not
delete
the
old
instance
from
the
previous
clusters
before
that.
B
We
need
to
make
sure
that
the
new
instance
is
ready,
so
we
check
for
the
Readiness,
and
then
we
steer
the
traffic
from
the
user,
which
can
be
external
from
the
from
the
cluster.
So
the
service
mesh
is
not
always
the
solution,
and
only
when
we
steal
the
traffic
we
can
under
a
condition
that
this
is
application
is
not
shared
application.
There
are
no
other
user.
Who
is
using
this
application?
We
can
again
update
the
intent
about
placement
and
remove
this
application
to
I.
Don't
want
to
show
any
demo.
B
We
have
VR
constrained
on
the
time
the
system
makes
bigger
and
bigger,
so
I
will
show
the
abstracted
way
on
the
slides.
So
imagine
that
we
deployed
we
say
to
the
MK
apis.
We
want
to
deploy
the
application
X,
for
example,
on
cloud
one
and
is
deployed,
and
then
we
can
send
the
second
request
to
the
MCO
apis
with
the
relocation
intent
that
we
would
like
to
relocate
the
application
X
to
the,
for
example,
folktu
and
following
the
procedure
I
showed
you
in
the
previous
slides.
B
The
application,
of
course,
will
be
relocated
from
cloud
1,
as
shown
to
the
foc2,
with
almost
as
it
will
be
almost
seamless
migration.
The
traffic
steering
thing
is
work
in
progress.
We
have
a
simple
solution,
but
there
is
a
lot
a
lot
to
do
more,
but,
as
you
can
see
here,
we
have
some
manual
in
invoke
of
the
procedure,
which
is
not
the
case,
for
example,
in
the
scenario
where
we
have
Vehicles,
which
are
moving
because
there
will
be
no
admin
who
will
Define
and
Trigger
such
relocation.
B
B
It's
also
intent
based,
so
we
can
Define
the
intent
for
the
LCM
workflow,
where
we
can
specify
the
application
requirements,
for
example,
for
the
latency,
for
example,
for
the
resource,
how
how
many
resource
the
application
needs,
probably
also
the
type
of
algorithm
to
take
into
account
when
we
will
try
to
select
the
the
best
cluster,
and
this
is
as
follows.
When
we
start
the
LCM
workflow
at
the
first,
we
subscribed
for
the
notifications.
B
In
our
use
case,
tetelco
use
case,
we
had
only
one
Source
from
the
IMF,
which
is
a
network
function
of
the
5G
control
plane,
and
we
received
a
notification
about
the
user
movement.
From
from
that
point,
we
could
subscribe
for
many
type
of
notification
from
many
different
source
of
information.
It
depends
on
our
use
cases.
This
is
for
a
proof
of
concept.
So
when
we
subscribe
for
the
notification,
we
enter
the
loop,
which
is
which
is
running
for
the
entire
lifecycle
of
the
application.
So
we
listen
for
the
notification
for
the
relocation.
B
If
it
occurs,
we
call
the
placement
controller
and
ask
make
it
make
the
decision
where
the
application
should
be
moved.
Where
is
the
optimal
place,
and
we
wait
for
for
such
a
response?
If
this
response
is
received,
we
can
make
use
of
the
relocation
workflow
described
earlier
and
generate
such
relocation
workflow
and
invoke
it.
B
So
this
is
about
the
LCM,
the
placement
controller.
It
requires
some
source
of
knowledge
to
make
the
decision,
so
the
first
source
of
knowledge
is
LCM
workflow,
where
we
Define
requirements,
and
the
second
and
third
are
the
topology
controller
and
the
observability
controller.
We
define
proof
of
concept
versions
of
of
those
and
going
deep
dive.
Placement
controller
receives
also
the
intent
as
a
request.
B
It
receives
the
intent
to
select
the
best
clusters
and
in
the
intent
we
specify
the
information
which
were
provided
in
the
LCM
workflow,
so
application
requirements
the
type
of
algorithm
which
need
to
be
selected
to
to
provide
the
placement
and,
of
course,
we
also
utilize
the
topology
controller
information,
for
example
the
location
of
the
latency
information.
But
it's
up
to
the
some
kind
of
topology
some
kind
of
edge
provider
which
can
provide
such
topology
controller
to
to
Define
what
information
we
can
get.
It's
the
same
for
the
observability
controller.
B
So
to
show
you
an
example,
we
deployed
the
application
manually
as
we
did
and
also
we
defined
the
LCM
workflow,
so
LCM
workflow
at
the
time
of
deployment
is
started
and
it
will
work
until
the
application
is
killed.
So
it
will
listen
for
the
notifications
if
we,
if
it
receives
the
app
notification.
Of
course,
it
will
automatically
make
the
decision
and
Trigger
the
relocation
in
this
case
from
cloud
1
to
Cloud
2..
It
will
continue
and
if
we
receive
another
location
request,
it
will
again
make
the
decision
and
make
them
relocation.
B
We
can
do
it
as
many
times
as
we
want.
I.
Think
you
you,
you
got
the
idea
and
the
LCM
workflow
can
be
started
for
many
applications,
so
we
can
deploy
any
application.
Any
number
of
application,
as
we
as
we
want,
and
they
will
be
separately
decision,
will
be
separately
made,
of
course,
on
the
same
infrastructure,
so
the
different
application.
We
will
make
the
decision
sorry,
so
the
decision
for
one
application
will
somehow
impact
on
the
decision
on
the
second
for
the
second
application,
but
this
is
an
idea.
B
We
also
consider
the
machine
learning
reinforcement,
learning
agent
to
interact
with
from
the
placement
controller
side
and
based
on
that,
we
believe
that
we
can
further
optimize
the
decision
which
cluster
is
the
best,
but
I
think
I
showed
you,
the
general
View
and
our
process,
how
we
received
how
we,
how
we
are
here
and
I,
think
right
now.
I
can
give
a
hand
for
the
Greg
to
continue.
A
I,
don't
know
if
pear
tree
mentioned
that
the
actual
location
procedure
that
we
implemented
is
already
open
source.
So
if
you
are
dealing
with
the
similar
problems
of
the
migration
of
the
application
across
the
Clusters,
so
you
can
just
take
it
from
the
open
source
the
link
was
included
in
the
presentation.
A
So,
as
you
can
see,
we
have
a
5G
cord.
That
is
a
all
the
time
informing
case
about
the
user
position
about
the
radio
conditions
for
the
user.
We
can
still
monitor
what
is
the
status
of
the
Hour
end
user,
whether
it's
a
handoff
or
or
whether
it's
doing
some
movement
or
not,
because,
as
you
can
see,
in
order
to
user
will
access
the
edge
application
it
needs
to
access
through
the
5G
Access
Network.
So
we
implemented
as
well
the
observability
controller.
A
This
constantly
observing
the
resource
consumption
at
the
Deep
different
clusters,
such
clusters
and
finally,
we
implemented
placement
controller
as
George
mentioned.
This
is
the
place
where
we
implemented
heuristic
and
reinforcement
learning
algorithms
in
order
to
decide
whether
to
relocate
or
not
based,
on
the
intents
received
from
the
5G
and
Edge
infrastructure,
okay,
topology
mapping,
how
we
did
that
so
for
the
5G
control
plane,
we
take
profit
from
the
35gc
project.
It
implements
as
well
a
data
plane
for
the
access
network
and
the
simulator
of
the
end
user.
We
took
profit
from
the
uranzim
project.
A
Our
Edge
clusters,
Edge
hosts,
were
implemented
as
a
single
separated,
kubernetes
clusters,
the
service
or
frustrator
that
we
have
in
this
architecture.
We
took
profit
from
the
Amco
project,
which
I
mentioned
before,
and
for
the
observability
controller
we
implemented
like
a
Prometheus
plus
grafana
my
mirror
solution.
A
So
this
is
the
last
Light
I'm
not
going
to
too
deep
in
this
project,
because
this
is
more
mathematical
modeling,
but
as
Pearl
Trend
I
mentioned
before,
we
are
receiving
multiple
notifications
from
the
5G
control
plane
from
The,
Edge
infrastructure,
and
so
first
of
all,
we
implemented
the
heuristic
based
approach
on
whether
to
migrate,
our
application
or
not,
based
on
some
CPU
memory,
measurements
based
on
some
latency
observing
and
currently
our
ongoing
Quirk,
is
implementing
reinforcement,
learning
approach
in
order
to
the
agent.
Let
the
agent
learn
our
Edge
topology.
C
B
Yeah,
in
fact,
we
you
know
right
now
we
do
not
care
about
the
infrastructure
creation,
so
we
assume
that
the
infrastructure
is
here.
We
just
register
the
Clusters
and
for
the
other,
manifests
I
believe
everything
is
in
the
yamla
format,
which
then
is
you
know
passed
in
the
programming
languages
is
yeah,
also
yeah.
We
also
use
current
packages
to
do
describe
the
application,
so
the
home
package
is
the
input
for
the
Amco
project.