►
From YouTube: Lightning Talk: Multi-Datacenter Strategies for Data Services - Julian Fischer, anynines
Description
Lightning Talk: Multi-Datacenter Strategies for Data Services - Julian Fischer, anynines
As multi-datacenter setups with Cloud Foundry become more popular it’s worth looking at the most common multi-DC and multi-region scenarios. See how these common scenarios can be classified and how the resulting architectures affect the build pipeline, infrastructure platform and most importantly data service automation.
A walkthrough common data services in the context of different multi-DC scenarios will illustrate do’s and don’ts during the automation for the use at large scale. From data services with inherent multi-DC capabilities to those without, an outline of specific and more generic integration strategies will be presented.
At the end of this talk you will consider multi-datacenter automation of data services more differentiated and have gained a basic understanding of problems and potential solutions.
About Julian Fischer
Julian Fischer is CEO of anynines. As an expert for digital transformation he has been attending hundreds of IT conferences and speaking at dozens including several Cloud Foundry events. His passion for Cloud Foundry was the motivation to found anynines, a startup entirely focused on cloud-native software development and operations. anynines has been the first Cloud Foundry based public PaaS and is entirely operated under German jurisdiction. With more than three years of Cloud Foundry operation experience anynines has established in CF related enterprise operations and consulting for central europe. More than that anynines does significant groundwork in the field of highly available, self-provisioning, infrastructure agnostic Cloud Foundry services.
A
A
Also
few
words
about
myself,
I'm
Julianne,
Fisher,
CEO
of
any
nines,
probably
you
might
have
heard
of
us.
We
focus
on
data
service,
automation
for
Cloud,
Foundry
and
other
platforms,
so
a
little
disclaimer
as
I
said,
the
talk
is
actually
meant
to
be
longer
than
that.
So
we
are
going
to
scratch
the
surface
and
at
several
times,
I
will
have
to
over
Frye's
oversimplify
things
to
a
degree
that
I
would
throw
tomatoes
at
myself.
A
Centers
can
be
very,
very
different,
so
the
term,
in
fact,
actually
gives
you
no
information
so
that
you
can
whatever
present
some
fancy,
slides
and
tell
somebody
what
this
means
for
data.
So
this
is
just
there's
no
connection
between
having
multiple
data
centers
and
a
good
design
to
do
it.
So
you
have
to
go
down
the
road
and
make
you
know
yourself
requirements
and
and
define
goals
that
you
want
to
fulfill.
So,
in
those
conversations
you
know
many
different
motivations
may
be
of
relevance.
A
For
example,
you
could
have
an
audience
that
spread
around
around
the
globe
so
that
you
want
to
serve
content
from
a
nearby
data
center
instead
of
having
a
long
distance
connection
with
a
lot
of
latency,
so
lowering
latencies
might
be
an
interesting
motivation
and,
of
course,
availability.
So
in
case
something
happens
so,
for
example,
a
local
disaster
such
a
fire.
Sadly
California
experienced
you
something
like
that
at
the
moment.
A
Could
you
know
put
a
thread
on
your
data
center,
so
there
are
things
in
between,
for
example,
like
do
you
want
to
have
a
load
scale
out,
or
do
you
want
to
have
a
capacity
scale
out?
Do
you
want
to
provide
redundancy
and
load
balancing
like
this?
Things
are
kind
of
interconnected
and
you
have
to
go
through
your
particular
scenario
and
define
what
you
actually
want
to
achieve.
A
So
in
our
platform
strategy,
we
always
had
the
assumption
that
we
would
like
to
have
a
homogeneous
operational
model,
which
means
I
won't
have
the
similar
operational
procedures
for
running
a
Postgres
than
for
running
a
Redis,
and
we
want
to
have
that
across
multiple
data
centers,
without
tying
into
the
technical
details
too
much
so
I've
seen
customers
who
running
on
OpenStack
and
they
are
happy
with
that
and
therefore
they
accept
infrastructure,
specific
ties.
So
this
is
one
of
those
design
goals.
A
A
The
first
one
is
where
you
have
multiple
data
centers
being
in
a
close
range.
I.
Think
Switzerland
is
a
good
example
right,
because
it's
smaller
and
I
know
a
company
who
have
multiple
data
centers
and
they
have
the
luck
to
also
own
the
data
center
connections
between
the
data
centers.
So
they
that's
a
good
example
for
a
close
range
data
center,
because
you
have
a
geographically
close.
You
know
data
center
data,
centers
being
so
close
to
another,
so
I'm
talking
about
less
than
a
few
hundred
kilometers,
ideally
less
than
100
or
50
kilometers
right.
A
So
you
have
a
high
bandwidth.
You
have
a
low
latency
and
therefore
a
reliable
connection.
So
this
makes
you
know
a
few
tricks
more
appealing
than
if
you
were
in
another
scenario,
which
we'll
look
at
a
little
later,
we're
talking
about
the
possibility
to
stretch
the
infrastructure
we've
seen,
customers
do
that
we,
we
are
not
infrastructure
experts
as
such
I
won't,
you
know,
say
anything
against
it.
A
A
A
However,
a
data
center
might
be
troubled
entirely,
so
if
you
lose
data
center
one
in
that
scenario
you
losing
two
or
three
availability
zones
and
for
most
data
service,
this
means
either
losing
the
capability
of
electing
a
leader
or
even
losing
data
depending
on
the
data
service
you're.
Looking
at
so
you
could
add
a
second
a
second
availability
zone,
a
data
center
too,
but
the
problem
here
obviously
still
is
quorum,
because
one
of
the
most
likely
scenario
here
is
that
you
have
a
connection
problem
between
data
center,
one
and
data
center.
A
Two
and
then
you
have
split
lane
our
split-brain
problem,
because
a
cluster
manager
usually
running
on
it.
You
know
on
nodes
and
the
nodes
are
spread
across
the
availability
zones.
It
would
be
split
in
the
middle,
so
there
would
be
two
cluster
nodes
on
the
left
side
to
cluster
node
on
the
right
sides,
and
none
of
these
partial
clusters
would
be
able
to
determine
whether
they
are
the
new
majority,
so
a
leader
election
impossible.
A
So
if
you
add
another
availability
zone
to
one
of
the
data,
centers
you're,
basically
going
back
to
the
first
scenario
where,
if
data
center
one
goes
down,
you
still
lose
a
majority
of
the
notes
and
also
it
really
depends
on
the
data
service
on
how
and
whether
the
data
is
lost
at
that
point.
So
again,
not
in
a
you
know,
I
mean
most
of
the
data
services
you'd
be
able
to
cover
and
survive
to
losing
two
out
of
five
nodes.
So
only
this
scenario
of
losing
data
service
one
would
be
really
really
a
problematic.
A
Still,
what's
the
point
having
two
data
centers
then
so
another
scenario
could
be
to
have
three
data
centers
with
each
having
one
availability
so
on.
Well,
that's
a
way
to
go
and
of
course
you
could
go
for
you
know,
all-in
and
having
three
availability
zones
having
a
local
failover
and
three
data
centers,
but
you
know
who
would
be
able
to
pay
for
that.
Nobody
would
ever
do
that.
A
So,
in
any
case,
you
know
the
stretch.
Infrastructure
scenario
has
the
appealing
part
that
you
can
just
you
know,
declare
the
availability
zones
and
not
tell
Bosh
so
much
about
the
existence
of
different
data
centers
and
let
the
virtual
machines
being
spread
across
availability
zones.
So
in
that
particular
scenario,
you
would
be
distributing
virtual
machines
across
three
data
centers
with
minimal
impact
with
the
data
service
automation.
A
So,
in
our
case,
data
services
will
be
provisioned
with
Bosh,
so
we
will,
just
you
know,
create
a
post
post
cluster
and
the
post
ghost
cluster
would
be
automatically
spot
spread
across
status
data
centers
I'm,
not
advertising
the
solution,
because
you
know
deep
down:
I
I,
don't
trust
the
stretch
data
center,
so
I
would
be
careful
with
that.
But
it's
an
appealing
idea
and
I've
seen
people
looking
at
it.
A
So
let's
look
into
a
scenario
which
is
pretty
common:
two
to
two
countries
such
as
the
US,
where
you
have
East
Coast,
West,
Coast
kind
of
scenarios,
so
your
data
centers
are
widely
spread
and
with
that,
even
if
you
own,
you
know,
connections
I'm,
pretty
sure
that
it's
just
the
physics
that
will
give
you
some
fluctuation
in
bandwidth
and
latency.
So
I'm
not
really
sure
whether
you
want
to
go
with
the
stretch,
infrastructure
and
that's
scenario,
because
it
just
takes
some
time
so.
A
Scenarios
I've
seen
is
that
you
have
multiple
Cloud,
Foundry
instances
or
four
nations.
However,
you
want
to
call
them
and
so,
for
example,
three
data
centers
and
three
Cloud
Foundry
foundations,
but
that
immediately
leads
to
that
question.
Who
is
orchestrating
those
cloud?
Foundries
who's,
synchronizing
those
cloud
founders
because
spoiler
alert,
Clark
family
doesn't
really
support
that
at
least
not
at
the
moment,
pretty
sure
that
they
got
something
on
the
roadmap
and
if
somebody
knows
something
about
that,
maybe
would
be
an
interesting
question
afterward.
A
A
A
So
I
wouldn't
bet
my
life
on
that,
because
it's
really
hard
to
achieve
and
also
similar
to
a
database
you'd
have
the
problem
that,
if
won't
found
is
unavailable,
whatever
changes
are
to
be
committed
to
that
Cloud
Foundry
would
be,
would
have
to
be
stored
in
kind
of
transaction
log
and
then
replayed
once
it
goes
up.
But
then
a
change
could,
you
know,
be
different
depending
on
when
it's
actually
executed.
A
So
not
really
sure.
So
you
have
a
lot
of
trouble
with
that
one,
because
you
have
to
monitor
all
you
have
to
perform
failover
and
you
have
to
think
about
what
what
actually
happens
if
a
cloud
phony
goes
away
and
comes
back
so
another
way
could
be
that
you
have
a
primary
Cloud
Foundry.
You
know
which
is
kind
of
the
master/slave
scenario
where
this
would
be
your
multi
master
scenario,
and
this
would
be
a
kind
of
you
know
primary
secondary
scenario.
A
So
you
have
one
cloud
phone,
you're,
you're,
writing
to
and
and
you're
trying
to
replicate
all
the
changes
to
that
Cloud
Foundry
to
the
secondary
ones.
So,
in
such
a
scenario,
what
you
could
do
is
having
a
database
I'm
using
Postgres
as
an
example
we're
using
Postgres
with
a
synchronous,
replication
and
a
synchronous.
A
This
is
something
that
seems
to
me
kind
of
doable,
so
we
could
have
a
service
broker
that
triggers
a
boss
deployment
either
in
multiple
bosses,
or
you
know,
having
a
boss
that
supports
multiple
infrastructures.
So
you
could
provide
on
the
level
of
behind
the
service
broker
a
automated
automated
awareness
of
different
regions
so
doable.
But
the
problem,
then,
would
be
when
you
look
at
a
synchronous,
replication.
The
inherit
drawback
is
having
a
replication
lag,
so
we
all
look
at
that
a
little
later,
what
it
means
to
a
synchronous
replicating
database.
A
A
Another
commonly
asked
question
was
thus
multi-region
is
multi-region
awareness,
a
matter
of
the
application
or
the
data
service,
layer
and
I
think
the
answer
to
that
is
is
not
clearly
given,
so
it
really
depends.
It
depends
on
the
design
goals
on
your
data
center
topology
on
the
data
service.
You
you
using
and
the
application
architecture
you're
looking
at.
A
But
if
you
force
me
to
give
you,
you
know
some
kind
of
navigational
knowledge,
it
wouldn't
be
much,
but
at
least
I
could
say
that
it
is
the
data
service
matter
if
the
data
service
is
inherently
designed
to
do
it.
So
if
you
look
at
technology
such
as
Cassandra,
where
you
can
configure
the
replication,
behavior
particular
to
your
certain
use
case,
that
is
surely
something
that
well
it's
some
kind
of
in-between,
because
you're
still
still
application
specific,
but
also
it
has
a
data
server
side
to
it
and
it's
data
service,
automation
matter.
A
Looking
at
the
Postgres
example,
where
Postgres
gives
you
the
ability
to
replicate,
but
it
doesn't
give
you
a
cluster
manager,
so
it
doesn't
give
you
a
failure.
Detection
doesn't
give
you
the
automation
that
needs
to
be
executed
once
a
leader
election
takes
place,
so
there
are
still
plans
to
fill
in,
but
at
least
you
have
the
tooling
to
do
that.
A
So
I'm
pretty
sure
there
are
a
lot
of
data
serves
that
are
inherently
not
designed
to
do
multi
TC
scenarios
and,
in
those
cases
you're
most
likely
up
to
do
it
on
the
application
level
or
choose
another
data
service.
But
let's
just
for
a
second
look
into
the
idea
of
having
a
synchronous
replicated
database
in
a
multi
region,
environment.
You
know
you
can
make
that
happen.
You
have
your
master
database
on
US
East
and
your
slave
in
u.s.
A
West
or
your
primary
and
your
secondary,
and
as
long
as
the
right
requests
to
your
primary
IP
to
your
application
on
the
primary
location
is
low
enough.
You
won't
have
problems
with
that
and
I
found
that
picture
to
be
very
suitable,
because
the
replication
lag
always
means
that
you
know
if
you
go
faster
with
your
car
at
some
point,
your
dog
will
start
suffering
in
same
with
databases.
So
if
you
have
databases
being
widely
spread
and
you
you
start
writing
to
the
master
database
too
fast.
A
The
replication
lag,
increases
and
increases,
and
at
some
point
the
replication
is
just
you
know,
to
wait,
who
out
of
seeing
and
may
collapse,
so
it
works
under
certain
circumstances.
But
how
can
you?
How
can
you
you
know,
grasp
these
these
circumstances
and
over
a
wide
read
a
wide
range
of
applications?
So
it's
hard.
So
my
assumption
was
that
in
every
other
case,
it's
a
more
kind
of
application
concern.
But
the
conclusion
is
actually
that
multi-data
cysts
awareness
is
per
se,
not
a
aspect
of
a
platform.
A
However,
it
is.
Why
is
to
design
early
for
motivators
and
awareness
that
I've
seen
customers
who've
been
adding
that
feature
to
their
platform
lately
and
starting
with
one
configuration
of
availability
zones,
a
number
of
data
centers
and
so
on.
So
you
have
to
make
those
changes
and
propagate
them
across
the
stack
which
is
costly
and
time
consuming.
A
So,
in
our
opinion,
the
multi
data
center
awareness
in
general
hits
the
entire
stack
so
from
the
data
center
design
to
the
infrastructure
design,
how
you
configure
your
availability
zones,
the
choice
of
the
data
services
and
possibly
the
automation
when
looking
at
cluster
managers
and
so
on,
and
obviously
also
your
application
design.
Where
you
can
still
you
know,
do
a
lot
of
tricks,
so
you
guys
was
way
longer
than
expected.
So
thanks
a
lot
and
please
feel
free
to
blame
or
ask
questions.