►
From YouTube: Moving from Mesos to Kubernetes Without Anyone Noticing [I] - Anubhav Mishra, Hootsuite
Description
Moving from Mesos to Kubernetes Without Anyone Noticing [I] - Anubhav Mishra, Hootsuite
At Hootsuite, we’ve been using Mesos and Marathon as our microservices platform for over two years but last year, we made the decision to bet on Kubernetes as its replacement. Eight months later, a small team of three operations engineers had migrated our first microservice from Mesos to Kubernetes. All without developers making any code changes. This was possible by architecting our applications with the proper set of abstractions. Fast-forward three months later and we have almost 20 microservices running on Kubernetes in production.
In this session, we’ll do a live demo of migrating a service from Mesos to Kubernetes, just like how we did it at Hootsuite! We will cover why architecting your infrastructure with the “right” abstractions helps you do these huge migrations with ease and how Kubernetes already contains these abstractions. We will explore how having a service mesh helps routing between two platforms while doing the migration. Also, how a mature CI/CD pipeline can help you deploy to two platforms with ease. To conclude we will explore the differences in running a service in Mesos and Kubernetes.
About Anubhav Mishra
Anubhav Mishra is a Developer Advocate at HashiCorp. He previously worked at Hootsuite. At Hootsuite he was focused on building cloud infrastructure and distributed systems. His work spans developers and operators. He helped create the next generation microservice delivery platform using Kubernetes. He loves DJing, making music and playing football in his free time. He is also a huge Manchester United supporter.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
Everyone
settled
I'll,
I'll
start
so
before
I
start
I'd
like
to
thank
Kelsey
for
actually
basically
forcing
me
to
submit
this
proposal
and
then
Michelle
for
reviewing
it
before
I
submitted
as
she
gave
some
awesome
feedback.
So
before
we
start
there's
not
a
lot
of
people
in
the
room,
so
we
can
do
a
poll
how
many
of
you
actually
running,
bezos
and
marathon
in
production-
okay,
well
everyone's
here,
so
the
via
the
maze
of
the
marathon
people
already,
okay
great.
A
So
this
talk
is
about
moving
from
mazes
to
kubernetes,
without
anyone
noticing
and
without
anyone
noticing
part
is
in
asterisks,
because
there
are
some
conditions
applied.
My
name
is
Misha.
I
used
to
work
for
a
company
called
HootSuite
I
was
in
the
pair
I
was
in
the
production
operations
and
delivery
team,
which
was
basically
responsible
for
building
our
micro
services
platform
on
mazes
and
marathon,
and
then
we
migrated
it
to
kubernetes.
So
a
bit
of
a
context.
Hootsuite
is
a
social
media
management
tool
that
lets.
A
You
manage
a
different
social
media
accounts
in
one
place
and
we
have
things
like
sentiment,
analysis,
analytics
scheduling
and
things
like
that.
We
have
or
north
of
15
million
customers
with
800
of
the
thousand
14
companies
using
us
where
I
work
now
I'll
talk
about
it
at
the
end,
it's
not
relevant
right
now.
I
also
work
and
maintain
a
project
called
Atlantis,
which
is
basically
a
terraform
workflow
to
my
contribution
to
the
kubernetes
community
has
been
basically
being
part
of
the
community
for
a
year
talking
meetups
in
conferences
about
kubernetes
and
the
cloud
native
tools.
A
A
We
already
know
the
platform,
that's
better
right,
I'm,
just
the
the
agenda
for
this
talk
is
basically
who's
whose
journey
from
moving
from
mazes
to
kubernetes,
it's
very
much
a
user
story
and
then
we're
kind
of
before
we
get
into
actually
like
the
the
implementation
details
will
kind
of
explore
the
micro
services
pipeline,
and
it
turns
out,
like
the
abstractions
that
we
build
in
the
micro
service
pipeline,
actually
helped
us
move
away
from
mazes,
they're
math.
On
very
easily,
then
we'll
kind
of
talk
about
things
that
we
did
to
kind
of
minimize
disruption.
A
Then
I'll
dare
to
do
a
live
demo
for
you,
which
is
actually
moving
a
service
from
mr.
kubernetes,
similar
to
what
we
did
at
HootSuite
and
then
we'll
kind
of
conclude.
Where
the
lessons
learned
and
conclusion
it's
a
bit
of
a
context.
Hootsuite
now
is
hundred
twenty
plus
developers
with
60
plus
micro
services,
written
in
both
Scala
and
golang.
We
have
two
cluster
schedulers,
which
includes
mezzos
and
kubernetes.
A
Major
still
runs
our
Jenkins
pipeline
and
kubernetes
runs
are
stateless
workload
and
we
have
1,500
plus
servers
all
running
on
Amazon,
so
before
we
even
get
to
schedulers.
It's
really
important
to
understand
like
how
things
before
and
will
give
you
a
really
good,
like
kind
of
like
where
we
were
before
and
like
why
we
moved
to
schedulers
so
2014.
You
know
we
was
just
starting
to
do
micro
services.
A
developer
would
ask
for
a
micro
service.
A
Suddenly
an
operator
would
appear
with
like
five
screens
I,
don't
know
why
they
have
so
many
screens
and
and
then
you
had
to
actually
explain
the
operators.
You
know
why
why
we're
going
to
towards
SOA
why
we
want
to
do
a
microscope
for
Microsoft
architecture
and
then
in
the
end
of
the
like,
the
conv
conversation
would
be
like
hey
just
create
me
a
zero
ticket
and
you
get
your
service
so
minutes
later.
A
The
developer
will
just
create
the
ticket,
and
then
this
amazing
conversation
would
start
on
JIRA,
but
like
a
common
thread
like
hey,
how
many
CPUs
do
you
want?
How
many
servers
do
you
want
and
like
how
much
memory
do
you
want
and
things
like
that
and
then
somehow
you'd
end
up
in
confluence,
like
a
developer,
had
to
write
an
ops
run
book
which
is
basically
before
they
actually
build
a
service?
You
have
to
define
how
the
service
would
operate,
so
you
basically
hit
every
other
Atlassian
tool.
A
So
weeks
later,
the
the
servers
would
be
ready,
and
then
the
developer
at
this
at
this
point
is
like
super
sad,
like
it's
already
taken
weeks
and
all
the
wonders
of
micro
service.
In
production,
and
then
on
top
of
that,
we
were
doing
Scala
micro
services
back
then,
so
they
actually
needed
to
install
Java
so
need
to
write.
Some
ansible
also
write
write
some
sensory
checks
to
monitor
the
service
and
then
on
top
of
that,
create
a
Jenkins
pipeline
to
deploy
to
go
servers.
A
So
all
that
work
was
still
to
be
done
and
it's
already
been
weeks.
So,
overall,
this
situation
isn't
good
right,
like
it's
not
happiness
right.
The
whole
promise
of
micro
service
is
not
basically
happening
so
fast
forward.
A
couple
of
years,
2016
2017
still
want
microsomes
I
hope
they
keep
one.
They
love
micro-services
and
then,
in
this
time,
like
five
minutes
later,
they
actually
able
to
deploy
the
micro
service
to
production
without
any
talking
to
anyone.
So
why
the
five
minutes
part
isn't
interesting.
A
The
part
that's
interesting
is
actually
the
cells
of
nature
of
the
platform,
and
that
was
one
of
the
keys
to
how
we
actually
migrated
off
of
the
platform
as
well.
So
the
micro
service
pipeline
for
HootSuite
look,
we've
basically
made
up
of
these
big
things.
One
was
the
project
generator.
One
was
the
idea
of
pipeline
has
code
and
then
the
actual
platform,
which
was
mezzos
and
Mathon,
so
the
project
skeleton
is
basically
the
first
thing
that
a
developer
would
interact
with.
A
It
was
as
a
group
at
HootSuite,
we
came
to
a
consensus
that
we
want
to
build
micro
services
a
certain
way,
so
we
basically
chose
the
languages.
We
chose
the
libraries,
we
chose
the
structure
of
micro
services
and
we
created
a
git
repo
and
committed
that
to
basically
get
so
a
developer
would
clone
this
repo
and
provide
things
like
you
know.
The
the
service
names
and
the
like
the
nice
names
descriptions,
the
maintainer
of
the
service
and
choose
a
language,
and
then
part
of
that
project
was
also
a
go
binary.
You
just
run
that
binary.
A
It
would
actually
generate
you
a
fully
functional
micro
service
in
Cola
or
Scala
and
with
bunch
of
other
stuff.
So
what
was
another
part
of
this?
Like
the
the
repo
was
the
actual
Jenkins
file,
so
you
can
actually
define
your
whole
micro
service
pipeline
in
Jenkins
using
Jenkins
files,
so
it
stages
like
build
test
and
deploy,
and
it
may
be
in
the
future
when
we
were
actually
migrating
off
of
off
of
mazes
to
cover
this.
A
This
would
be
super
useful
and
you
see
how,
once
you
commit
this
Jenkins
file
to
get
it
automatically
like
Jenkins
I
can
scan
your
organization
and
automatically
build
you
a
fully
featured
micro
service
pipeline
with
like
build
test
features
and
things
like
that.
We
also
in
terms
of
like
the
actual
micro
service.
We,
as
I
said
we
both
Scala
and
golang
micro
services.
A
We
also
have
the
deployment
files
checked
in
as
part
of
this
project,
so
it
has
like
things
like
replica
and
resources
and
memory,
and
things
like
that
health
checks
define
already
as
part
of
them,
so
this
is
all
generated
for
you,
and
this
is
again
important
in
the
in
the
next
few
steps,
and
then
we
also
have
the
makefile.
This
is
just
the
ease-of-use
thing.
You
know
to
do
things
like
building
your
service
testing,
it
deploying
it
to
dev
staging
in
production.
A
We
just
have
simple
targets
that
is
just
defined
in
a
make
file
in
terms
of
the
platform
we
run,
both
mesas
math
and
some
of
you
actually
did
not
raise
your
hand
I'll,
just
like
kind
of
mention
what
makes
with
mesas.
These
mesas
is
basically
the
resource
manager
that
pulls
all
these
like
different
notes
together
and
makes
them
feel
like
one
machine
and
then
Marathon
is
the
framework
that
sits
on
top
of
mazes
and
lets.
A
You
schedule
like
long
living
workloads
and
things
like
health
checks
and
stuff
like
that,
so
our
set
up
was
pretty
straightforward.
There's
nothing
special!
Here
we
had
like
three
four
masters
or
whatever,
and
they
were
running
the
mazes
master
process
and
then
math
on
alongside
it,
and
then
we
had
slaves
that
were
running
the
maze
of
slave
and
then
the
docker
container.
So
with
the
way
this
worked
is
Jenkins
would
just
say,
make
deploy
dev,
staging
and
production.
A
This
post
call
would
come
in
and
this
container
will
be
scheduled
somewhere
in
the
menos
cluster
and
that's
how
we
kind
of
did
things
so
now,
if
you're
using
schedulers,
you
probably
have
more
than
one
service
running
in
there,
so
so
to
route
between
these
services
like
how
did
the
actual
mazes
driving
work.
This
is
where
things
get
interesting
and
I'll
kind
of
spend
more
time
in
routing
and
show
you
guys
how
the
actual
migration
happened
to
in
there
in
the
next
few
slides.
A
So
we
kind
of
said
that,
okay,
we're
gonna,
do
this
like
fat,
middle
way
approach
or
the
service
proxy
or
the
service
match.
There's
so
many
names
for
it
now.
But
we
kind
of
decided
on
on
doing
that
really
early
on
and
we
chose
engine
acts
as
a
service
proxy,
and
then
we
chose
console,
which
is
a
2
by
Hoshi
comp
to
do
our
service
discovery
for
us
so
kind
of
connected
all
the
containers
for
us
together.
So,
let's
say
a
service
one
would
come
up
on
some
major
slave.
A
We
use
registrated
to
kind
of
register
that
one
docker
container
in
console
and
that
that
consoles
now,
that
service
is
now
available
for
console
and
you
can
kind
of
interface
with
it
using
a
DNS
interface
or
an
HTTP
interface
and
now
the
way
the
way
the
actual
routing
would
work
is.
Let's
say:
if
this
service
two
was
to
talk
to
service
one
service.
Two
would
just
do
a
simple
localhost
call
on
a
special
board
called
5040
which
expands
to
s
or
a
out,
and
then
you
and
it's
basically
a
path
based
right
like
right.
A
Is
this
engine
deck
started
just
proxy
forwards?
This
call
up
to
the
the
actual
instance
that's
running,
so
all
this
is
great.
You
guys
are
like
okay,
you
actually
had
a
great
platform.
You
know
what
did
you
guys
do
next?
We
spend
a
lot
of
a
lot
of
time
on
actually
getting
this
adoption
like
going
at
HootSuite,
so
spend
a
lot
of
time,
writing
great
documentation,
getting
deep
teams
on
board
and
guess
where
and
so
by
the
way
this
graph
is
the
service
adoption.
A
Basically,
that's
the
number
of
services,
and
that
is
the
actual
actual
timeline.
So
this
is
where
we
did
the
zero
to
five
minutes
thing
you
can
see
like
immediately.
Everyone
wants
to
use
micro
service,
they
want
to
write
it
on
the
scheduler.
So
at
this
point,
sorry
that
was
Nikki
at
this
point.
We
have
great
autonomy
in
the
company.
Our
service
platform
is
doing
really
well,
everyone
loves
us
right.
Developers
are
loving
the
the
whole
new
platform,
five
minutes
thing
because
it
took
weeks.
A
So
there
was
a
lot
at
stake
when
we
said
like,
let's
move
to
kubernetes,
and
now
you
guys
will
ask:
why
did
you
actually
move
to
kubernetes?
We
actually
talked
about
it.
We
wrote
a
pretty
in-depth
blog
about
why
we
compared
all
these
scheduled
the
cluster
managers
and
schedulers
and
decided
on
kubernetes.
So
at
the
end,
I'll
link
this
to
the
two
slides
in
the
end,
so
you
can
kind
of
go
through
it.
A
I'm
not
going
to
go
into
the
exact
reasons
why
we
did,
but
basically
it
came
down
to
kubernetes
had
the
right
attractions
and
primitives
for
us
to
build
basically
a
micro
service
on,
and
we
resonated
that
as
as
as
a
business
and
also
also
in
terms
of
like
the
developer,
tooling
and
the
ecosystem.
It's
amazing
that
you
pretty
much
get
everything
for
free,
so
it
took
us
four
months
and
was
three
of
us.
Luckily,
I
have
both
of
these
guys
here
so
mark
and
Luke
also
here.
A
So,
if
you
have
any
questions
after
the
doc,
you
can
ask
them
as
well.
So
my
whole
team
is
here
is
awesome.
It
took
us
four
months
and
we
got
Cuban
Aires
running
in
for
on
Amazon.
We
told
Kelsey.
He
was
super
happy
I.
Think
you
did
that
like
in
my
mind
he
was
doing
this
basically,
so
it
was
super.
It
was
a
awesome
day
and
it
was
actually
time
to
move
these
books
now
right.
So
now
we
have
all
the
stuff
running
in
mazes.
We
first
need
to
deploy
that
in
kubernetes.
A
Right
second
thing:
we
need
to
figure
out
routing.
You
have
to
route
from
mazes
to
kubernetes
and
also
stuff,
that's
outside
of
mazes
into
communities
as
well,
and
then
the
developers
that
that
want
to
use
a
new
kubernetes
platform
have
to
adopt
that
and
I
kind
of
learned
about
the
platform.
There's
great
kubernetes
documentation.
But
there
are
some
implementation
details
on
how
you
would
roll
that
out
in
your
company.
So
you
know
you
have
to
basically
lay
that
out
for
them
as
well.
So
the
way
we
kind
of
solve
these
problems
is
it's
interesting.
A
So,
like
the
mazes,
the
mazes,
math
or
platform
was
composed
of
these
things
right
project
skeleton
the
pipeline
is
called
docker
containers
and
then
the
dynamic
service
is
covered
using
console
and
nginx
pretty
much
everything
was
used
to
move
off
of
basis
as
well.
So
each
of
these
things
were
actually
important
to
move
off
of
it,
and
then
we
also
wrote
some
great
documentation
to
get
started
on
kubernetes,
and
then
we
wrote
a
tool
called
mazes
to
kubernetes
and
I'll
explain
what
that
was.
A
So
the
first
thing
was
like
once
you
start
looking
at
like
these
mazes
projects,
we
had
to
actually
migrate
them.
We
started
writing
out
these
kubernetes
llaman
files
and
I'm
like
how
do
we
do
that,
like
we
were
doing
that
manually
every
time
so
when
we
would
do
a
pull
request,
like
I,
think
Luke?
Actually
just
pointers
like
we
should
just
automate
this
like
we're
doing
this
every
time
we
know
how
both's
both
schemas
look
like.
So
we
wrote
a
simple
golang
binary
that
you
can
download
and
you
can
just
run
it
on
your
project.
A
It
would
read
all
the
major
marathon
deployment
files
and
write
the
kubernetes
llamó
files
for
you,
because
we
kind
of
knew
how
the
structure
was
in
terms
of
the
actual
deployment
files.
We
also
added
some
new
simple
targets
in
the
make
file
that
you
saw
earlier,
so
we
just
added
targets
for
deploy
here.
It's
dev
staging
so
basically
at
both
clusters
running
at
the
same
time.
So
in
terms
of
the
actual
pipeline,
we
end
up
deploying
to
both
the
platforms.
A
So
we
duplicated
the
workload
we
deployed,
everything
in
in
kubernetes
as
well,
so
once
we
migrated
deploy
everything,
but
the
the
tricky
thing
is
like
we
only
route
it
to
mazes
first,
so
basically,
everything
was
as
it
is
for
all
the
dependent
services
and
the
service
that
was
to
be
moved.
So
it
does.
The
packaging
Scallon
golang
micro
service
pack
is
using
docker
container.
This
is
the
benefit
right
here
that
you
can
actually
move
this
taka-kun
district
kubernetes.
You
don't
have
to
figure
out
the
implementation
details
for
Scala
or
golang,
which
is
really
nice.
A
Both
both
mazes
marathon
and
kubernetes
understand
our
containers.
So
this
is
the
benefit
there
in
terms
of
routing,
as
I
said
that
the
workload
is
duplicated
in
both
places.
You
already
understand
how
the
routing
works
in
mazes
tomatoes.
So
basically
we
spin
down
the
major
side
and
the
Kuban
each
side
is
already
running,
and
it
also
has
that
nginx,
router
and
console
so
console
discovers
the
kubernetes
nodes
and
make
them
available
in
the
backends
for
nginx
and
nginx.
A
Basically,
does
a
routing
call
translation
in
you
see
how
so,
let's
see
if
the
service,
let's
say
that
the
service
to
was
to
be
calling
service,
one
again
say,
would
again
do
that:
localhost
call,
which
is
pretty
much
the
same
for
the
service.
But
this
time
you
see
there's
no
upstream
back
in
available.
So
it's
one
because
it
doesn't
exist
in
in
mazes
anymore,
and
you
would
go
to
this
special
back
and
call
the
bridge,
and
this
is
again
an
implementation
detail.
You
don't
have
to
do
this.
A
We
have
of
something
called
a
bridge
which
is
again
powered
by
nginx
and
console,
and
it
basically
lets
you
bridge
the
blade
literally
bridge
the
gap
between
two
data
centers.
So
with
console
we
kind
of
went
Greenfield.
We
said:
ok,
we're
gonna,
set
up
a
fully
functional
V
PC
and
its
own
network,
its
own
networking
and
then
spin
up
kubernetes
so
be
basically
treated
it
as
a
different
data
center.
So
what
would
happen
with
the
request?
Is
it
would
get
translated
to
the
bridge,
the
bridge?
You
say:
ok,
I'm,
trying
to
find
micro
services.
A
This
name
in
your
data
center
I
couldn't
find
it
and
then,
with
this
this
simple,
dumb
routing,
we
said
everything
else
should
just
go
to
kubernetes.
So
this
is
a
pretty
like
a
decision
that
we
made,
because
we
kind
of
understand
how
a
micro
service
is
work,
and
at
this
point
it
ends
up
in
the
kubernetes
side.
Here
the
engine
extract.
A
It
just
translates
the
request
to
to
actually
say
that
see
the
kubernetes
like
conformant
request,
so
at
that
point
it's
just
Kubb
dns
coop
dns
kicking
in,
but
with
the
service
name
default,
SVC,
the
console,
local
and
then
a
port
8080.
Basically,
so
at
that
point
kubernetes
takes
over,
it
basically
goes
to
the
service,
the
kubernetes
service
and
the
service
forwards,
using
IP
tables
into
the
part.
So
it's
pretty
straightforward.
So,
on
the
other
side,
the
service
actually
didn't.
A
The
service,
too,
did
not
notice
that
this
sir,
was
actually
got
moved
and
it's
still
doing
that
localhost
path
based
call.
It
didn't
realize
that
this
new
service
is
now
there's
nothing
special
on
the
suicide,
and
this
is
where
the
service
meshes
and
service
proxies
actually
shine,
that
the
applications
don't
have
to
can
be
super
dumb
and
the
mesh
can
do
the
the
crazy
routing
for
you
in
terms
of
roll
back.
This
is
interesting.
We
we
basically
had
it
so
that
the
routing
as
I
said
earlier,
would
always
prefer
mazes
first.
A
So
in
this
case,
if
there
was
something
something
went
wrong,
we
would
just
spend
these
workloads
back
up
in
math
on
and
all
that
routing
would
just
still
work.
So
you
would
just
end
up
hitting
the
major
side.
Basically,
so
how
did
the
service
calls
go
outside?
So,
if
gaps,
let's
say
service
one
running
in
kubernetes
and
it
was
to
be
called
some
food
service
which
is
running
ec2
or
mazes
the
way
routing
worked.
Is
we
just
basically
fake
the
community
service?
A
So
we
created
a
community
service
with
the
name
foo,
and
then
we
just
forward
it
all
the
traffic
on
a
special
port,
50
40,
which
would
go
to
the
local
engine,
X
router
and
then,
once
once
that
cold
call
was
made.
This
request
could
get
forward
the
intern
X.
Here
then
we
had
this
amazing
glorified
reg
X,
which
mark
wrote,
which
would
just
translate
the
call
into
the
path
based
call
again,
so
you
would
end
up
with
some
requests.
A
Looking
like
the
path
based
request
again
outside
from
nginx,
but
go
to
the
bridge
bridge
again
knows
how
to
route
this
and
it
would
route
that
to
do
the
actual
service.
This
is
how
we
kind
of
do
things,
and
this
was
kind
of
married,
because
you
had
to
create
a
list
of
all
the
micro-services
dependencies
before
okay.
A
So
in
terms
of
the
project,
skeleton
which
is
like
the
basically
where
a
developer
starts
their
microservice
journey,
what
we
did
is
we
had
a
branch
for
this
like
basically
get
branch
that
was
doing
the
kubernetes
animators,
both
both
the
things
and
maybe
pretty
confident.
We
basically
delayed
deleted
the
major
side.
A
We
wanted
to
make
sure
that
all
the
new
services
that
were
getting
spun
up
would
actually
end
up
in
kubernetes,
so
we
just
merge
that
branch
and
boom,
like
all
the
new
services,
were
now
getting
created
in
kubernetes
I've,
been
really
care
because
we
don't
want
to
create
technical
debt
and
move
those
services
again.
This
is
I
think
the
most
important
part
like
in
the
whole
presentation
the
documentation.
Whenever
you
bring
in
something
new,
you
really
need
to
write
documentation,
that's
readable,
that's
something
that
people
can
follow.
So
we
created
a
dock.
A
That
would
just
basically
give
you
add
option
to
the
new
plot
platform,
so
you'd
say:
okay:
how
to
create
a
service
in
Cuban
Ares.
How
do
you
call
it?
How
does
login
work
in
Cuban
Ares?
How
alerting
work
and
things
like
that?
On
the
other
side,
we
also
created
a
doctor
my
grade,
and
you
see
that
warning
right
there.
It
says
talk
to
us
first,
which
is
basically
our
team
just
so
that
we
can
onboard
them
to
the
new
platform.
Try
to
get
kind
of
give
them
tips
how
to
migrate.
A
The
service
in
terms
of
the
actual
migration
implementation
detail,
we
wrote
everything
down
in
the
doc
and
literally
like
that
bash
commands
that
you
have
to
run
all
the
stuff
that
I
just
talked
about.
They
can
just
follow
through
and
do
it
if
it's
a
simple
service
all
right.
So
now
we
come
to
the
important
part,
which
is
the
actual
live
demo.
Let's
all
bow
our
heads
to
the
demo
God,
which
is
Kelsey
here
right.
Okay,
let's
do
this!
A
Okay,
so
here
I
have
here:
I
have
them
at
three
node
mezzos
marathon
cluster.
It's
very
low
scale
like
this
is
not
how
fish
run
stuff
back
home
and
then
kubernetes
as
just
a
one
node
cluster,
and
then
we
have
console
basically.
So
that's
marathon,
nothing
is
running
there.
That's
console,
there's
only
two
services
that
are
about
console
service
itself
and
then
the
two
services
are
the
kubernetes
services
that
are
being
kind
of
discovered
dynamically.
A
So
we
can
discover
that
kubernetes
node
dynamically
okay,
so
in
terms
of
the
actual
micro
service
today
we're
gonna
migrate,
it's
called
cube.
Contrary
17,
it's
a
simple
like
calling
service.
I'll
just
show
you
the
code,
there's
nothing
exciting
here.
Basically,
what
we're
doing
is
we're
returning
back
response
with
a
random
name
that
we
are
generating
and
then
the
host
name
of
that
instance.
That's
running
it,
so
we
can
kind
of
know
where
it
app
is
running.
So
so
it's
clear
to
us.
A
Okay,
so
let's
go
back
here:
okay,
yeah,
so
I've
already
built
the
docker
image
for
this
for
the
stuff.
So
we
don't
waste
time
and
we
won't
be
using
any
CI
CD
pipeline.
So
if
you
won't
be
going
through
Jenkins
and
stuff
because
it
just
takes
longer
okay,
so
in
terms
of
right
now
we
have
the
maser
disbarment
file.
So
if
you
look
at
that
deploy
folder,
it
has
like
the
major's
mazes
JSON
file.
Let
me
just
get
that
for
you
yeah.
So
it's
a
simple
mazes.
A
You
know
Jason
file,
it
you
probably
know
okay
and
then
we
have
the
make
few
make
targets
here,
so
we're
just
deploying
to
production,
one
environment
and
we'll
just
say:
make
deploy
production
and
that
should
go
to
and
that
should
actually
spin
this
up
in
Marathon
yeah.
This
is
deploying
that
and
thing,
and
we
have
two
healthy
instances
of
this
app
running
and
then
also
consoles
automatically
detected
this
new
container
containers
and
then
they
would
just
register
the
httpservlet
that
attack
HTTP
service
and
would
run
on
mazes
master
one
and
three.
A
So
that's
that's
all
working.
This
is
using
the
register.
Basically,
so,
okay,
that's
all
good
that
that
all
works,
let's
see
if
the
actual
routing
works
inside
of
mazes.
So
let's
go
to
a
mezzos
master
and
like
try
to
hit
the
service
using
that
path
based
thing.
So
if
I
go
up
here,
hopefully
I'm
a
history,
so
here
I'm
doing
a
localhost
call
on
50
40.
As
you
can
see,
slash
service
is
static
here
and
then
the
name
of
the
service
is
cube.
Contouring
17,
so
we're
gonna
do
this.
A
You
can
actually
see
this
routing
is
actually
working.
It's
responding
with
some
random
names
and
their
host
name
and
what's
special
here?
Is
this
header
called
exile,
skyline
driver
ingress
and
egress?
We
just
inject
that
in
engine
X
skyline
is
the
name
of
the
framework
that
we
call
internally
and
here
I'm
we're
just
always
preferring
the
local,
the
local
instance
since
I
own
master
one,
and
this
container
exists
so
I'm
not
going
out
of
the
machine
I'm
just
hitting
the
container.
That's
inside.
A
This
is
the
dependent
service
which
shouldn't
notice
anything,
basically,
okay,
so
now
we've
set
this
up
now
we'll
actually
start
doing
this
migration,
which
is
the
interesting
part,
let's
go
back
here,
okay,
so
the
first
thing
we'll
do
is
translate
the
file
from
from
basically
write
the
cuban
aires
files,
so
we
will
use
the
the
Mazeroski
h2.
This
is
not
this
I'm
just
faking
this.
This
is
not
the
actual
tool.
I
don't
work
at
HootSuite
anymore,
as
I
said.
So
this
is
just
a
simple
go
like
manage
this
output
stuff.
A
This
is
just
gonna
output,
saying
here,
like
you
know,
I
just
generated
the
kubernetes
folder
for
you
I'm
just
moving
a
file
in
the
background,
basically
so
I'm
being
super
transparent
with
you
guys
so,
not
faking
anything
by
the
way.
So,
okay,
here
we
go
so
here's
here's,
the
Cuban
air,
is
Jason
yeah
Mel
file.
So
let's
just
get
that
yeah.
This
is
pretty
simple
here
we
have
a
service
that
we're
creating
and
then
also
the
deployment
and
this
in
in
real
life.
A
A
I'm
just
doing
a
coupe
CDL
apply
in
real
life,
as
I
said,
we
use
ham
charts,
which
basically
translates
the
certain
variables
that
you
define
your
new
service
which
in
Yama
format
and
injects
that
to
the
helm
chart-
and
this
is
nothing
special
here-
you
do
make
you
just
run
this
make
deploy,
hits
so
now
it's
created
the
service
of
kubernetes.
We
do
cube
CTL,
rusty.
A
In
service
yeah,
so
it's
created
the
coupon
service
and
also
the
actual
part
for
those
services.
Again,
two
instances
were
created
at
this
point
to
the
routing,
if
you
see
like
nothing
still
like
nothing's
going
to
kubernetes
yet
right,
like
you're,
still
preferring
mezzos,
because
this
exists
in
Bezos,
oh
yeah.
Now
this
is
the
the
nerve-racking
part.
This
is
where
we
scale
this
thing
down
to
zero
and
wait
and
one
Wi-Fi.
So
please
Wi-Fi,
don't.
A
So
that's
gone
and
then
in
console
we
see
the
service
go
away.
Basically,
so
this
console
local
DC
says:
okay,
we
don't
have
this
service
anymore.
Now,
if
you
go
back
to
to
here
routing
here,
you
see
the
ingress
is
going
to
kubernetes
now
so
seamlessly.
This
is
moved.
You
might
see
some
upstream
errors
based
on
the
way
you
shutdown,
containers
and
things
like
that.
You
can
do
some
smart
free
tries
on
it,
but
for
the
demo
purposes,
I
wasn't
doing
it.
A
A
So
let's
say
if
something
really
bad
happens
as
soon
as
you
see
it
like
this,
the
engineer
see
like
this
error
rates
go
up
and
things
like
that,
if
you
just
scale
this
back
again
right
like
well
just
get
me
back
aware
of
where
I
was
just
scale.
This
back
to
two
again:
okay,
yeah,
so
this
healthy
containers.
Here,
let's
go
here
Yukon
again
this
the
service
is
available
and
you
see
we're
back
on
mazes.
So
basically,
that's
like
the
the
rollback
mechanism,
so
that
is
what
I
have
for
the
demo.
A
Let's
go
back
to
the
presentation
again,
Kelsey
is
still
chilling
yeah.
So
in
terms
of
the
migration
results,
we
we
migrated,
move
20
services
and
one
and
a
half
months
with
three
people
loop
being
like
the
technical
lead
and
then
the
two
interns
that
basically
moved
most
of
our
services,
which
is
not
the
best
but
yeah
I,
wouldn't
advise
on
that.
A
So
we
basically
misspelled
that
or
something
happened
like
that:
I,
don't
exactly
remember,
and
then
we
couldn't
route
from
like
the
service
that
was
inside,
of
kubernetes
outside
like
and
I
was
like
as
simple
as
that
is
just
a
human
error.
Second,
one
is
like
the
classic
security
group.
You
know
a
white
listing
error,
which
is
happens
all
the
time
in
Amazon.
So
let's
say
a
service
in
Bezos
was
using
an
RDS
database
and
we
have
a
whole,
like
security
groups,
white
listing
that
the
white
RDS
database.
A
We
moved
it
to
Cuba
net
he's
forgot
to
whitelist
a
database.
You
know
the
cars
got,
the
call
starts
failing
in
and
we
moved
it
back.
We
rolled
it
back
so
yeah,
two
big
things
that
cause
like
decent
amount
of
impact,
but
overall
it
was
pretty
like
seamless.
So
what
were
the
lessons
that
we
kind
of
learned-
and
this
is
where
like
I,
feel
like
the
technical
implementation
details?
Don't
really
matter
so
much
but
like
the
high-level
stuff
matters,
so
cheese
Jews
always
choose
the
least
important
service
first.
A
This
is
pretty
common,
pretty
simple
to
do.
This
is
important
because
you
want
to
basically
make
sure
that
there's
no
business
impact
when
you
do
these
big
translations
and
kind
of
understand,
like
there's
so
many
things
that
you
won't
realize
you
can't
foresee
this
will
hug
and
of
help
you,
you
know
debug
that
always
have
a
rollback
plan.
You
know
in
our
case,
like
things
went
wrong
bunch
of
times
we
had
to
rollback
bunch
of
times
so
yeah
having
that
mezzos
fallback
was
awesome
in
terms
of
writing
down
like
the
actual
deployment
pipeline.
A
So
sometimes
you
have
to
actually
visualize
how
your
deployment
works
because
there
might
be
different.
Flavors
of
you
know,
services
getting
deployed,
so
it's
nice
to
write
it
down
and
then
kind
of
feel
figure
out
where
you
can
abstract
things.
They
can
inject
think
so
to
make
the
degradation
like
basically
small
in
terms
of
documentation.
It
should
be
readable
for
humans,
don't
do
like
a
brain
dump.
This
is
how
I
did
it,
and
these
are
the
batch
commands
explain
why
you
did
it.
A
This
is
really
really
important,
especially
when
we
were
having
interns
kind
of
go
through
and
do
that
right
in
terms
of
being
pragmatic,
I
think
this
is
another
important
thing.
Nothing
is
like
it's
like.
Usually
you
won't
get
services
that
I
built
exact
same
way.
Sometimes
you
actually
have
to
go
in
and
write
some
code.
So
it's
okay
to
do
that,
there's
going
to
be
some
special
service,
we
all
have
them
and
you
can
just
go
in
and
do
that.
A
It's
totally
okay,
this
is
I
feel
is
like
the
most
important
part
is
like
minimizing
disruption.
I've
said
this
multiple
times,
but
I
can't
emphasize
this
enough.
This
is
for
both
the
services
and
actually
the
people.
The
service
part
is
like
basically
making
sure
that
the
dependence
is
dependent.
Services,
aren't
seeing
anything
any
errors
and
things
like
that,
but
for
people
you
can't
be
asking
like
developers
to
do
these
migrations.
Ask
the
product
manager
and
ask
for
like
bunch
of
story,
points
and
stuff
like
they
won't.
You
know
they
don't
like
that.
A
They
already
have
stuff
to
do
right.
So
you
want
to
make
sure
that
make
sure
that
you
know
this.
The
the
disruption
of
actual
like
the
work
flow
is
also
like
minimize
and
we
focused
on
it
a
lot
like
we
made
it
so
that
it's
actually
similar
to
be
wrote
tools
that
would
make
the
make
the
translation
easy
and
that
actually
helps
with
a
great
adoption
of
a
new
platform.
A
It's
really
easy
to
tarnish
like
an
image
of
a
new
tool
like
you
bring
in
kubernetes,
and
then
you
move
the
service
and
it
causes
outage
everyone's
like
oh
kubernetes
actually
causes
outages
and
it's
like
a
civil.
It
wouldn't
be
kubernetes
fault,
it
would
be
something
else,
and
you
really
want
to.
You
really
want
to
make
sure
that
that
that
image
is
is
maintained,
and
so
that,
with
that
way,
you
can
drive
an
option.
Basically
so
in
terms
of
links.
I
have
a
link
for
that
blog
post
of
like
why
we
moved
to
kubernetes.
A
I
also
have
a
link
to
the
abstraction
that
we
built
on
top
of
marathon
I.
Think
that's
important,
because
if
we
kind
of
kind
of
define
everything
in
Yammer,
as
you
saw
in
the
in
the
slide,
but
in
the
real
life
I,
was
using
JSON
files,
so
we
kind
of
abstracted
that
out
in
an
API
and
then
consoles
link
which
basically
helped
us
link
between
infrastructures
from
both
mesas
and
kubernetes
side.
So
thank
you
really
appreciated,
appreciate
you
guys
coming
out.
A
A
Structure,
side
of
the
migration
right,
how
do
you
went
from
a
mrs.
Kloster
to
equities
right
so
in
terms
of
the
actual
major
side?
Mirror
side
of
the
infrastructure
was
basically
built
using
terraform
and
and
was
the
three
nodes
with
some
slaves,
with
some
console
nodes
and
stuff.
That
was
like
the
backbone
basically
and
we
used
terraform
for,
like
writing,
all
the
defined
things
on
defined
things
on
like
all
amazing
stuff,
and
then
we
use
Packer
to
kind
of
build
these
images
and
use
ansible
to
pretty
much
do
all
I
can
take
management.
A
So
it's
pretty
pretty
much
like
on
in
an
auto
scaling
group
like
the
the
Masters
one
in
the
auto
scaling
group,
the
slaves
were
and
they
were
imaged,
so
you
can
actually
like
and
when-
and
you
know
what
would
come
up,
the
slave
could
just
add
itself
to
the
mazes
master
in
which
is
work.
On
the
kubernetes
side.
We
took,
as
I
said,
a
Greenfield
approach,
so
we
created
all
the
way
down
from
the
V
PC
to
the
actual
nodes.
Again
we
managed
everything
ourselves
using
terraform,
and
then
everything
was
in
auto
scaling
group.
A
So
the
API
server,
the
the
slaves,
the
NC
Dino's,
even
that
was
tricky,
and
hopefully
he
eventually
blog
about
everybody
should
and
then
what
was
interesting
was
like
the
we
also
generate,
like
asserts
for
every
component
of
kubernetes
using
wall.
So
we
generate
basically
dynamic
search
for
every
component,
so
you
spend
a
lot
of
lot.
That's
what
took
us
so
long
took
us
long
time
to
kind
of
get
to
a
production,
ready,
kubernetes
cluster
managed
by
us,
and
we
use
an
overlay
to
do
the
networking
views
planner
to
network
between
in
between
the
kubernetes.