►
From YouTube: Deploy my Application Across Multiple CF with Persistent Data - Vick Kelkar, Redis Labs
Description
Deploy my Application Across Multiple CF with Persistent Data - Vick Kelkar, Redis Labs
We have geo-distributed teams; each using a local [P]CF
We have an internal application that needs to sync persistent data between CFs.
Two ways to solve this problem
We could have setup a GeoDNS based solution for CF, sync blobs, VPN etc
We chose to go with an active-active redis replication solution for our needs
We used a bosh-release of redis-enterprise in each CF
Setup a geo-distributed redis cluster that takes care of replication and consistency using CRDB
So app-developers do a blue-green deploy of their app to both cloud-foundries and the redis-active-active cluster takes care of application persistent data with CRDB.
https://cfseu18.sched.com/event/FRyg/deploy-my-application-across-multiple-cf-with-persistent-data-vick-kelkar-redis-labs
A
Welcome
to
my
talk
folks,
I
am
Vic
Cocker
today,
I'll
be
talking
about
how
to
do
application
development
across
cloud
foundries.
So
just
quick
poll
here.
How
many
are
you
using
cloud
foundry
today
in
dev
staging
prod?
Just
so
I
have
an
idea.
Okay,
that's
great
half
of
you,
so
I
think
hopefully
you'll
find
this
talk
useful.
So
what
I
plan
on
covering
today
is
you
know
what
are
the
challenges
you
are
facing
when
you're
using
cloud
foundry
and
you
have
multiple
instances
of
your
foundations?
A
And
you
know
what
is
the
potential
solution
around
solving
your
data
needs
because
you
can
push
your
applications
very
easily
to
your
local
cloud
foundries.
But
then
how
do
you
synchronize
the
data
across
various
cloud
foundries
that
you
may
have
across
the
geographies,
then
I'll
quickly
introduce
Redis
as
a
potential
solution
and
give
you
a
quick
overview
of
what
Redis
is
and
how
it
can
address
your
data
needs
and
then
finally,
I
will
present
a
walkthrough
of
the
solution
using
cloud.
A
Foundry
and
specifically
I
will
use
the
pivotal
cloud
foundry
distribution,
to
show
you
what
that
solution
looks
like
from
Redis
and
Redis
Enterprise
perspective.
Finally,
I'll
leave
about
a
couple
minutes
or
last
five
minutes
to
open
it
up
for
questions,
and
that
will
give
us
a
good
opportunity
to
ask
any
follow-on
questions
you
may
have.
So
let
me
just
set
the
stage
for
you
on
what
is
required
or
what
is
the
challenge
with
doing
multiple
cloud
foundries
on
a
planet
scale
with
multiple
data,
centers
involved
right.
A
A
And
how
do
you
make
sure
that
your
CF
applications
are
able
to
leverage
the
same
data
set
but
you're
able
to
push
your
Cloud
Foundry
applications
across
multiple
CF
targets
or
multiple
CF
foundries,
so
I
think
that's
one
of
the
bigger
challenges
when
you're
talking
about
data
services
in
Cloud
Foundry,
and
how
do
you
do
data
synchronization
across
regions
across
cloud
foundries
today?
So
some
of
the
reasons
why
you
might
want
to
synchronize
your
data
across
multiple
cloud
foundries
are
some
of
the
use
cases
I'm
highlighting
here.
A
Some
of
those
are
around
scorecards
and
shopping
carts
that
you
may
be
using.
You
may
be
using
this
multiple
Cloud
Foundry
is
to
solve
your
leaderboard
issues
or
leaderboard
solution.
It
could
be
related
to
fraud,
detection
as
well
as
event
tracking
or
you
might
be,
using
the
likes
or
a
recommendation
system.
A
So
for
addressing
all
these
use
cases,
you
want
to
figure
out
a
way
to
synchronize
your
data.
Have
it
be
globally
available
across
multiple
cloud
foundries?
So
there
are
a
couple
challenges
in
the
solution
to
the
potential
in
a
problem
statement
here.
So
one
of
the
key
things
when
you
are
developing
CF
applications
is
you
want
to
focus
on
the
same
data
set
across
multiple
cloud
foundries.
A
You
want
to
ensure
that
you
have
a
very
good
developer
experience
and
a
user
experience,
so
you
want
to
give
the
freedom
for
your
developers
to
port
they're
Cloud
Foundry
applications
locally
quickly
without
having
to
worry
about
synchronization
of
Geo
distributed
teams.
So
one
of
the
ways
you
want
to
do
that
is
you
want
to
keep
very
low
latencies
for
your
applications
by
having
your
applications
pushed
out
to
cloud
foundries
that
are
very
close
to
your
user
base
and
then
one
final
thing
as
a
big
organization
who
has
multiple
cloud
foundries
across
the
planet?
A
One
thing
you
want
to
do
is
you
want
to
keep
the
delta
of
your
application
in
various
regions
and
various
continents
to
be
very
small
right?
You
want
to
use
the
same
code
base
and
maybe
literally
change
the
CF
target
before
you
push
the
application
right.
So
you
don't
want
to
introduce
any
additional
logic,
as
you
are
trying
to
push
your
geo
distributed,
applications
across
multiple
cloud
foundries.
A
Now
there
are
two
potential
possible
solutions
that
you
can
leverage
today
in
public
clouds
or
on
Prem
kind
of
highlighting
couple
of
them.
One
of
them
is
around
spanner
Google
spanner,
which
has
a
two-phase
commit
approach.
So
with
this
approach,
what
you
are
gaining
is
consistency,
but
what
the
trade-off
there
is
in
order
to
achieve
that
you're
looking
at
at
least
200
milliseconds
or
higher
before
your
data
gets
synchronized.
A
Another
approach
is
a
quorum
based
approach.
Cassandra
is
one
of
the
good
examples
of
how
you
would
do
that
at
a
global
scale.
Cassandra
again,
it's
a
quorum
based
system,
so
it
has
to
have
majority
of
the
participating
nodes
to
agree
before
the
commit
is
written
to
all
the
Cassandra
nodes.
So
again,
with
this
approach,
the
quorum
based
approach-
you
are
looking
at
at
least
over
100
milliseconds
of
latency,
so
in
a
typical
web
app.
You
know
you
want
very
quick
response
times.
A
A
So
one
of
the
things
I
want
to
focus
on
is
I.
Don't
want
the
developers,
the
Cloud
Foundry
developers,
who
are
writing
the
applications
to
focus
on
implementing
the
solution
themselves
in
the
code.
What
I
mean
here
is
we
don't
want
the
developers
to
look
at
how
to
do
the
retries?
How
to
do
the
circuit
breaker
pattern
and
write
all
that
in
to
your
application
logic
right,
so
we
want
them
to
focus
on
delivering
value.
A
So
let
me
let
me
propose
and
tell
you
about
a
solution
that
is
being
used
at
Redis
labs
and
what
we
are
doing
at
Redis
labs.
We
have
a
Bosch
based
release
which
is
implementing
this
solution.
This
solution
is
based
on
CR
DTS
CR
DTS
are
conflict-free
replicated
data
types.
This
implementation
of
data
synchronization
is
based
on
a
paper
that
was
written
by
dr.
Carlos
in
2011,
so
dr.
Carlos
is
actually
on
our
technology
board.
So
he's
advising
us
on
how
to
do
this.
A
A
You
know
a
couple
of
the
things
to
highlight
here
is
again
the
data
service
or
the
database
itself
is
doing
all
the
heavy
lifting
of
synchronization
retry
logics
so
that
the
developer,
the
Cloud
Foundry
application
owner
doesn't
have
to
worry
about
those
retry
logics
and
then
the
Redis
is
known
to
be
very
simple
and
fast.
So
it
is
able
to
take
care
of
those
conflicts
that
you
may
have
during
data
replication
using
the
conflict
resolution,
that's
built
into
the
CR
DT
approach.
A
A
A
Like
I
mentioned,
it
makes
it
possible
for
the
developer
to
focus
on
the
business
logic
and
not
necessarily
worry
about
how
to
go
about
doing
the
data
synchronization.
You
know
all
these
normal
patterns
that
you
use
in
micro,
service
developments
like
retry
logic
and
circuit
breakers
patterns,
are
something
that
is
now
offloaded
to
the
data
service
and
the
data
service
will
take
care
of
the
replication,
the
synchronization
and
the
conflict
resolution.
A
So
why
see
our
DTS
and,
what's
so
unique
about
them,
so
see
our
DTS
offer?
You
active,
active
and
I
think
I
want
to
emphasize
this
point
again.
It
is
a
multi
master
approach.
We
call
it
active
active,
so
it's
bi-directional:
it's
not
active
passive.
It
is
master
master
replication,
so
you
are
able
to
write
data
to
both
the
cloud
foundries
at
the
same
time
and
the
conflict
resolution
and
replication
happens
in
the
background.
A
Another
advantage
of
this
approach
is
the
ability
to
create
a
full
mesh
network
of
all
your
Cloud
Foundry
instances
that
may
be
spanning
regions.
Then.
The
couple
other
advantages
to
highlight
here
is
the
Redis.
Implementation
automatically
takes
care
of
the
counter
deletion
for
you.
It
takes
care
of
expiry
as
well,
and
then
the
Redis
CR
DT
offers
up
conflict
resolution
for
these
five
complex
data
types.
A
Some
of
the
easier
ones
may
be
easy
to
explain,
is
the
last
right
wins,
but
as
the
data
structures
get
complex,
the
conflict
resolution
algorithm
also
gets
complex,
but
the
Redis
enterprise
solution
is
already
solving
that
concern
for
you
by
implementing
various
conflict
resolution
strategies
before
I
go
any
further.
Let
me
just
briefly
tell
you
guys
about
Redis.
How
many
of
you
here
are
active
users
of
Redis
today
show
of
hands?
Alright,
we
got
a
few,
so
this
would
be
good
to
go
over
with
the
whole
group
here.
A
So
let
me
tell
you
a
little
bit
about
Redis.
So
Redis
is
an
open
source
in
memory.
No
sequel
database
Redis
labs
is
home
to
the
Redis,
open
source
project,
Salvatori
who's,
the
core
contributor
or
Redis
works
at
Redis
labs,
and
let
me
tell
you
what
are
the
key
advantages
and
differentiators
for
Redis
as
a
technology,
so
we
are
known
for
three
big
differentiators.
One
of
them
is
around
performance.
A
A
How
do
we
get
you
the
highest
throughput
with
the
lowest
latency
possible
for
your
applications,
so
the
graph
on
the
left
is
essentially
showing
you
that
we
are
able
to
achieve
very
high
throughput,
it's
very
low
latencies,
and
this
was
a
survey
done
by
an
independent
benchmarking
firm
to
kind
of
highlight
that
feature.
The
second
advantage
of
Redis
is
around
how
the
footprint
looks
like
for
Redis.
A
So
the
second
graph
on
the
right
is
essentially
highlighting
the
fact
that,
in
order
to
reach
a
million
operations
per
second,
how
many
Redis
instances
and
boss
jobs
will
it
take
for
you
to
achieve
this
outcome.
So,
as
you
can
see
from
the
graph,
Redis
has
a
very
small
footprint
and
is
able
to
deliver
that
performance
and
that
you
may
be
looking
for
as
an
app
developer
on
Cloud
Foundry.
A
The
second
aspect
is
around
simplicity
and
Redis
is
no
sequel
database
running
in
memory,
and
it
essentially
supports
data
structures
like
the
diagrams
on
this
slide.
We
have
kind
of
highlighted
them
as
Lego
blocks.
So
what
these
allows
you
to
do
is
using
these
data
structures
or
Lego
blocks.
You
are
able
to
compose
your
solution
and
address
your
use
case
using
these
ten
basic
data
structures
that
are
available
in
Redis.
A
So
we
have
four
modules
that
we
want
to
maintain
and
highlight
and
publish
from
Redis
labs,
so
ready
search
is
a
capability
that
you
can
introduce
in
your
solution
to
do
basic
recommendation
system
and
full-text
searching
graph
module
is
about
to
hit
GA
in
in
the
near
future.
So
that's
also
a
capability
that
you
can
introduce
in
your
Redis
deployment
on
a
Cloud
Foundry.
A
couple
other
modules
worth
mentioning
is
around
machine
learning
and
JSON
objects,
so
lettis
can
also
do
JSON
object
store
on
your
Cloud
Foundry
service
instances.
A
So
let
me
just
quickly
tell
you
what
it
looks
like
on
a
cloud.
Foundry
I
was
going
to
do
a
live
demo,
but
I.
Think
after
I
saw
what
happened
at
the
keynote
this
morning.
Not
a
lot
of
the
demos
were
working
so
I'm
going
to
go
with
the
static
screenshots,
but
I
will
do
my
best
to
explain
what's
happening
here.
So
I
think
what
you
see
here
is
Redis
Enterprise
deployed
on
a
Cloud
Foundry
foundation
that
is
essentially
running
in
the
u.s.
continent.
A
A
So
I
have
done
so.
The
second
output
here
is
around
when
you
push
that
it's
a
Python
flask
app
that
one
of
our
architects
have
written
so
I've
essentially
pushed
it
out
to
that
US
Cloud
Foundry
did
the
service
binding
so
that
now
you
can
see
that
my
Redis
web
CLI
is
connected
to
the
US
Foundation
in
u.s..
A
Now
I
have
gone
through
and
done.
The
similar
exercise
with
CF
foundation
running
in
the
European
subcontinent.
So
now
I
have
to
cloud
foundries
one
in
the
u.s.
one
in
Europe
both
have
deployed
this
Python
flask
app.
They
are
bound
to
their
local
service
instances
and
the
data
is
getting
synchronized
for
you
in
the
background,
so
I'm
going
to
quickly
walk
you
guys
through
a
couple
examples
of
what
that
looks
like.
So,
let's
start
with
the
data
structure
lists
in
Redis.
A
So
what
I've
done
here
is
kind
of
taken
screenshots
of
my
Python
flask
app
and
I'm,
essentially
walking
you
guys
through
the
four
steps
to
confirm
that
the
data
is
getting
synchronized.
So
if
you
look
at
Box
one,
this
is
in
the
US
data
center
I
input
something
into
my
list.
I
do
the
same
thing
on
the
European
cloud
foundry
in
step.
A
Two
then,
in
step
three
and
step
four
I
am
essentially
pulling
up
that
same
list
and
showcasing
the
fact
that
the
data
has
been
synchronized
for
you
and
by
the
Redis
Enterprise
Service,
in
the
background,
without
you
as
a
developer,
having
to
do
any
additional
steps.
So
that's
a
quick
example
of
how
we
do
it.
In
less
register
data
structure,
we
have
a
similar
concept
of
doing
that.
Data
synchronization
using
another
data
structure
called
sets
same
concept
here.
A
You
know,
I
have
a
four
step
process
in
the
first
one
I'm
in
entering
a
set
in
the
US
Cloud
Foundry
I
do
the
same
on
the
second
step
in
the
European
Cloud,
Foundry,
installation
and
step.
Three
and
four
are
essentially
showing
you
the
fact
that
the
sets
have
been
synchronized
across
both
the
cloud
foundries
and
the
developer
doesn't
have
to
worry
about
the
retry
logic
or
any
of
the
circuit
breaker
that
you
may
have
to
do
on
a
traditional
development
platform.
A
What
I'm
trying
to
highlight
here
is
the
fact
that
you
are
able
to
see
the
synchronization
of
the
Redis
service
instances
across
these
two
cloud
foundries.
So
where
do
you
see
the
small
green
circular?
Icon
towards
the
bottom
of
the
image,
what
you
are
seeing,
there
are
the
two
cloud
foundries
that
I
have
connected
to
each
other,
so
this
allows
you
to
essentially
see
the
synchronization
happening
and
you
can
collect
the
status
of
that
so
that
you
can
focus
on
developing
your
application.
A
Now.
The
web
interface
gives
you
the
clear
indication
that
these
two
cloud
foundries
are
in
sync,
but
we
also
have
a
REST
API.
So
if
you
are
trying
to
automate
the
Status
collection
of
the
synchronization
of
data,
you
can
hit
the
REST
API
endpoint
and
you
can
automate
the
status
update
of
your
data
and
you
know
set
up
any
triggers
or
alerts
for
your
follow-on
sequence
of
Cloud
Foundry
applications.
A
So
in
quick,
summary
I
just
wanted
to
highlight
the
fact
using
CR
DTS.
We
are
able
to
take
care
of
the
data
synchronization
based
on
these
five
data
structures
and
you're
able
to
achieve
the
outcome
here,
of
achieving
strong
eventual
consistency
for
your
data
for
your
Cloud
Foundry
applications
running
on
different
continents
or
across
different
regions
in
Cloud
Foundry
ecosystem.
A
B
Hi
hi,
so
you
mentioned
at
the
beginning
of
a
presentation
that
other
solutions
ensure
consistency
by
two-phase
comments
or
some
quorum
approaches.
How
do
you
ensure
consistency,
because
this
some
probably
somehow
included
in
this
sea
oddity
concept?
But
for
me
it
wasn't
clear
how
you
doing
sure
the
consistency,
because
it
seems
like
it's
a
more
replication
based
approach
to
sync
the
data
right
yeah.
A
No,
that's
a
good
question.
So
actually
the
answer
lies
on
the
fact
that
you
know
you're
talking
about
geographically
distributed
systems,
so
the
data
has
to
traverse
across
the
continent.
So
it's
an
event-based
system,
so
it's
continuously
synchronizing
but
the
the
rate
of
which
it
synchronizes
depends
on
your
architecture
itself.
So
if
they
are
within
the
same
continent,
the
synchronization
will
happen
a
lot
faster.
In
my
example,
it
had
to
go
all
the
way
from
u.s.
west
coast
to
you,
know
Ireland,
so
it
took
a
little
bit
longer.