►
From YouTube: Live Migration of Production Workloads from Apache Mesos PaaS to Ku... Maria Camacho & Gufran Lutful
Description
Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Live Migration of Production Workloads from Apache Mesos PaaS to Kubernetes - Maria Camacho & Gufran Lutful, Nokia
In 2018 we started the migration of production workloads on our cloud-native big data analytics platform (PaaS) from Apache Mesos and Marathon to K8s. Nokia has multiple production PaaS instances in its datacenters across the globe with various production workloads, including a distributed analytics engine, production databases, data streams and ML applications. These workloads are critical to telecom data scientists requiring high performance, availability and reliability without disrupting continuous delivery of big data services. We want to share the story and lessons learnt when migrating production workloads to K8s while maintaining performance and reliability, and at the same time delivering new services to our demanding users. You will hear how we seamlessly shared cloud resources, network traffic, internal DNS and telemetry systems between two orchestrators in production.
https://sched.co/Zel1
B
A
A
Yes,
and
as
you,
you
all
know
that
nokia,
driven
by
its
award-winning
bell
labs
nokia,
is
a
leader
in
the
development
and
deployment
of
5g
networks.
So
5g
you
know,
is
going
to
be
the
core
of
connecting
people
connecting
world
in
the
future
or
already
happening
actually
so
we're
still
connecting
people.
Yes,
we
do
okay,
this
is
the
story
so
in
in
the
today's
session.
What
actually
we're
gonna
talk
about?
We're
gonna
talk
about
a
migration
story,
our
migration
journey
from
past
to
kubernetes.
A
So
just
imagine,
as
you
can
see
already,
also
on
the
slide.
There
is
a
gif
which
is
showing
that
two
trains
moving
side
by
side.
First,
one
definitely
the
message,
something
that
we
started
working
on
in
the
beginning
of
the
project,
so
message
running,
all
the
workloads
providing
all
the
services
and
so
on,
and
then
kubernetes
comes
in
and
sharing
the
same
platform
resources
right.
A
You
know
moving
side
by
side
and
then
the
workloads
are
migrated
from
one
platform
to
one
train
to
the
other
one,
without
interrupting
the
existing
services
without
interrupting
the
users.
So
a
seamless
migration
on
a
shared
platform,
we're
going
to
talk
about
this
today
in
this
session,
our
journey
of
messages
pass
to
kubernetes.
B
B
We
provide
a
set
of
services
for
end
users
who
bring
network
specific
data
into
our
platform.
They
store
this
data.
They
use
several
tools
that
the
platform
provides
for
analytics.
They
use
a
bi
tool
for
presentations,
so
this
is
a
never
stopping
process
of
different
data
sources.
Coming
in
hundreds
of
users,
hundreds
of
data
analysts
working
on
the
clock
to
provide
analytics
for
our
customers
and
on
the
screen.
You
can
see
the
high
level
architecture
of
of
this
platform.
B
You
can
recognize
probably
the
logos
of
several
open
source
components,
so
we
concentrated
on
using
open
source.
So
this
is
apache
mesos.
We
are
not
using
mesosphere
right,
so
the
the
goal
was
to
switch
keep
the
platform
as
it
is,
keep
our
users
working
as
as
they
were,
but
in
the
in
the
lower
level,
move
slowly
from
mesos
seamlessly.
A
Yeah
and
just
to
mention
few
things,
so
this
is
a
big
data
platform,
so
it's
it's.
We
had
around
five
five
production
instances
across
four
different
continents
and
thousands
of
users.
You
know
this.
This
platform
has
been
also
used
to
do
things
like
such
as
mdt
minimization
of
drive
tests.
You
know
predictive
analysis
and
and
process.
A
A
I
think
we
started
this
project
around
2015
or
that's
correct
time
frame,
but
by
2018
we
noticed
that
you
know:
kubernetes
was
getting
a
lot
of
industry
attention
and
industry
leaders
were
starting
to
use
it.
It
was
starting
to
become
industries
darling,
and
we
also
wanted
to
adopt
it
for
many
different
reasons.
A
Many
different
reasons-
and
you
know
kubernetes-
was
getting
becoming
the
center
of
a
growing
community
quickly
reaching
production
with
maturity
and
there
were
more
and
more
features
being
promised
to
kubernetes,
and
while
we
actually
noticed
at
the
same
time
that
messes
was
getting
less
and
less
work
done
less
and
less
features
coming
to
messages,
but
kubernetes
was
getting
more
and
more
people
working
on
it,
and
also
one
of
the
reasons
could
be.
That
message
was
at
that
time.
A
A
You
know
managing
this
local
persistent
volumes
at
restaurant,
but
the
benefits
were
much
much
more
than
you
know,
some
of
those
things
robux
yeah
and
since
we
did
a
spike
where
we
identified
how
we
would
like
to
migrate.
So
those
were
not
really
any
issue
for
us
because
we
were
planning
to
run
both
of
the
platforms
side
by
side,
seamlessly
migrate
components
one
by
one.
In
a
you
know,
release
driven
manner,
prioritized
and
also
there
were
rollbacks
and
and
features
like
you
know.
A
B
Yeah
a
little
bit
of
a
message
over
a
little
bit
about
mesos
exactly
so,
we
can
take
a
look
at
the
high
level
architecture.
As
you
can
see.
On
the
left
side
of
the
screen,
we
have
the
basic,
highly
available
cluster
three
master
nodes,
two
and
standby
one,
acting
as
leader
zookeeper
in
the
back
end,
keeping
the
leadership
on
and
the
worker
knows
who
actually
hosts
the
workloads
so
how
the
workloads
get
there.
B
B
It
uses
a
schedule
so
selecting
the
available
node
based
on
the
available
resources.
The
schedule
launches
an
executor
and
the
executor
hosts
the
tasks.
Yes,
these
tasks
are
actually
the
the
application.
The
the
if
we
take,
for
example,
spark
apache
spark
as
an
example.
So
apache
spark
as
a
framework,
doesn't
do
anything
when,
when
it's
ready
to
process
data
through
mesos
master,
it
selects
the
appropriate
worker
or
workers
and
launches
the
executors
who
actually
do
the
data
analysis.
A
And,
on
the
other
hand,
for
example,
you
could
mention
the
marathon
framework
that
you
have
been
using
it's
one
of
the
popular
frameworks
for
messes
to
run.
You
know,
containers
you
don't
necessarily
have
to
be
container.
You
can
run
some
tasks
or
even
some
scripts
using
it.
So
in
methods,
this
page
networks
that
you've
got
different
frameworks
and
where
you
start
to
message,
they're
getting
you
know,
resources
from
the
resource,
performances
and
working
based
on
this.
You
know
dynamic
resource
fairness,
scheduling,
policy
yeah.
A
So
that's
the
message
of
review
and
if
we
look
at
this
kubernetes
overview
here,
these
pictures,
as
you
can
see
basically
the
same
they're
doing
this
almost
the
same
things.
Of
course
you
know
both
of
them
have
are
sharing
ancestry
of
googleborg,
but
also
in
kubernetes.
You
can
see
in
the
architecture.
There
is
like
a
three
master
notes,
or
you
know
there
is
even
like
elections
going
on
and
later
and
you've
got
this
api
server,
scheduler
control
measure,
all
this
stuff
and
the
worker
notes.
A
You
can
see
a
keyblade
which
is
basically
you
know,
making
sure
that
the
containers
are
running
a
port
q,
proxy
taking
care
of
the
networking
etc.
But
in
the
messa
side
the
message
slave
or
mrs
agent
actually
processed,
that
was
taking
care
of
similar
things,
so
they
are
very
much
similar
in
terms
of
you
know
how
they
are
working.
The
sketching
on
also,
you
can
see
here
on
the
right
basically
similar
to
messes.
B
B
B
B
Yes,
so
we
will
talk
more
about
the
cubelet
wrapper,
but
this
was
just
one
more
mesos
application
that
was
running
in
the
backend
no
user
interface,
but
it
was
taking
care
that
the
kubernetes
workers
were
up
and
running
and
were
able
to
consume
some
resources
from
the
host
and
hence
host
the
loads.
Yes,.
A
So,
basically,
when
you
run
the
kubernetes
marathon
boom,
you've
got
this
cover
in
this
cluster.
Yes,
you
know
and
interesting
thing
is
that
they
are
not
fighting
with
each
other's
resources.
As
you
can
see
here
in
this
cupid
preparing
marathon
in
this
slide,
you
can
see
how
kibler
paper
is
running
in
marathon.
So
basically
in
marathon
you
do
not
your
application
doesn't
necessarily
need
to
be
have
a
container
it.
Can
you
can
just
write
any
script,
scripts
or
tasks?
A
A
Limitations
of
resources
or
fighting
with
the
resources
and
mesos
also
on
the
other
side,
is
able
to
see
how
much
resources
kubernetes
is
using
from
particular
node
and
so
on.
You
can
actually
even
run
kubelet
with
no
resources
assigned
at
all.
So
basically
that
is
that
could
be
used
for
body
eviction
and
stuff
like
that,
and-
and
so
you
know,
even
a
even
a
node
is
running.
Cubelet
doesn't
necessarily
mean
that
it
is.
It
has
resources,
we
can
control
those
from
this.
So
this
is
really
a
very
nice
magical
script.
A
Okay,
it
looks
very
simple,
but
we
we
try
to
follow.
You
know
this
keys
principle.
Keep
it
simple
stupid.
You
know
systems
work
best
when
they're
kept
simple,
and
we
also
are
a
bit
lazy.
We
try
to
do
less
and
gain
a
little
bit
more,
but,
of
course
all
these
are
coming
from
spike,
which
we
started.
You
know
when
we
started
thinking
to
move
to
kubernetes,
but
now
that
we
have
seen
the
picture
of
a
cube
light
rapper
in
marathon
maria
would
like
to
say
something
how
we
actually
package
it.
B
Major
components
were
actually
packaged
into
a
vm
image,
and
this
vm
image
was
transported
to
the
target
cloud
where
the
platform
was
supposed
to
be
deployed
on
yes,
and
this
gave
us
the
possibility
to
deploy
fairly
on
any
cloud
in
customer
premises
or
in
our
internal
nokia
cloud,
which
was
the
actual
environment,
where
we
had
a
production
clusters
runnings
so
from
the
list
on
the
screen,
you
can
see
all
the
components
you
can
see.
Mesos
you
can
see
marathon,
kubernetes
and
so
on.
B
A
You
know
that
we
would
like
to
let's
say,
carry
offline
in
certain
offline
cases,
which
was
actually
one
of
our
major
requirements
as
well,
that
we
would
be
able
to
deploy
it
offline,
and
so
you,
basically
you
know,
pack
up
everything
in
your
laptop
or
in
a
usb,
stick
and
deploy
it
in
a
custom
premise
or
in
your
environment
case
where
you
don't
have
access
to
internet,
so
packaging,
the
vm
image,
and
now
that
we
have
it
in
the
vm
image.
We
launch
it
from
marathon.
Yes,.
A
You
know
applications
that
are
running
and
messes,
then
you
need
to
make
any
changes,
so
the
existing
way
was
also
working
and
we
also
were
able
to
gradually
move
to
kubernetes.
You
know
based
on
priorities
and
release
cycles,
and
so
on.
So
on
the
bottom
side
you
can
see.
There
is
an
example
of
couch
tv
which
don't
have
this
app
time
help,
so
this
was
actually
a
marathon
application
yeah.
So
that's
how
it's
been
deployed
and,
although
it
leaks
all
these
things,
looks
very
fairly
simple.
B
B
A
lot
more
yes,
little
big
things,
so
we
kind
of
found
we
in
practice
and
while
implementing
the
the
the
side
by
side
deployment
of
the
both
clusters
using
them
side
by
side
and
troubleshooting.
Of
course,
as
as
usual,
you
have
issues
on
the
way,
so
we
found
out
that
part
of
the
of
the
winning
kind
of
conditions
come
from
the
correct
strategy
so,
and
the
points
in
the
achieving
the
best
strategy
is
to
study
first.
A
B
A
Yes,
and
one
immediate
measure
is
because
kubernetes
manages
the
demo
sets,
you
know
very
smoothly,
it's
very
nice
and
also
the
deployment
for
the
helm
applications.
You
know
the
deployment
was
very
nice,
although
in
the
beginning
I
remember
there
was
this
helm
too,
and
there
were
some
issues
with
you
know:
security.
A
Deployment
way
was
really
helpful
for
us
and
cooperatives
and
methods
both
sharing
the
same
resource
pools.
You
know
no
extra
resources
needed
and,
of
course,
we
need
to
imagine
our
cluster
properly,
including
the
system
resources,
because
if
you
are
not
dimensioning
the
cluster
properly,
you
may
have
issues.
Let's
say
you
know,
the
the
application
is
requiring
certain
resource
that
none
of
your
and
nodes
or
what
I
know
is
actually
providing,
and
it
can
happen
even
for
spark
jobs
as
well.
Yes,.
B
A
A
A
B
B
A
Meanwhile,
we
can
maybe
share
what
questions
were
asked.
One
of
the
questions
were:
what
is
marathon?
Is
it
an
app?
Yes,
it's
an
app.
Actually.
Second,
one
was:
how
did
we
handle
the
player
for
load
balancing
where
the
applications
from
message
side
accessible
from
the
kubernetes
side?
Yes,
that
was
actually
the
original
goal
to
be
able
to
share.
You
know
similarities
like
that,
all
the
resources
and
all
the
services
within
the
both
of
the
platforms
to
be
shared,
so,
for
example,
telemetry
systems,
monitoring
and
enlarging
all
these
things.
A
Yeah,
so
as
you
know
that
the
during
the
presentation
we
have
mentioned,
like
message,
is
able
to
know
how
much
is
just
covering
this
is
using
and
kubernetes
also
knows
what
is
its
limit,
and
also
we
have
got
this.
You
know
telemetry
systems,
monitoring
and
alerting
where
we
were
watching
like
at
certain
thresholds.
We
get
a
large
in
terms
of
you,
know,
cpu
or
memory
condition.
A
So
definitely
we
were
getting
a
large
before
issues
happening
and
our
sres
were
taking
care
of
those
but
yeah,
so
messes
and
and
kubernetes
both
the
clusters,
even
though
they
were
sharing
resources
from
the
same
resource,
a
pull
from
the
node.
You
know
they
were
not
getting
like
in
fight
with
each
other
and
before
even
oh
there's
another
question
yeah.
B
B
A
B
A
But
these
are
really
very
nice
questions
and
you
know
the
migration
is
actually
everybody
has
whoever
have
used
some
other
clusters
before
before
kubernetes.
You
know
they
would
have
a
story,
migration
story,
and
probably
there
are
many
different
stories.
People
are
following
different
approaches
for
our
cases.
We
also
wanted
to
save
resources.
We
couldn't
just
you
know,
go,
have
new
set
of
resources
in
a
new
cloud
account
and
you
know,
deploy
the
kubernetes
cluster
separately
over
there
and
then
migrate.
You
know
that
would
be
very
costly
for
us.