►
Description
Principles and Strategies in Data Service Automation - Julian Fischer, Anynines
In this talk, Julian Fischer, an automation enthusiast will talk about the exciting challenges you will encounter when automating a large set of data services across several platforms, infrastructures, and operating systems.
You will be guided through the impact a full automation mission has on existing operational models. Learn how the separation of duties, delegation, and automation need to work alongside to accomplish the mission.
See the impact and limitations of the Open Service Broker API and get a sense for false friends and challenging problems to solve when automating a multitude of data services with a small team.
At the end of the talk, you will have a clear picture of how to approach creating a set of production grade, highly scalable new data service for modern platforms including Cloud Foundry and Kubernetes.
About Julian Fischer
Julian Fischer is CEO of anynines a well established platform consultancy with focus on Cloud Foundry. Julian and his anynines team love operating Cloud Foundry environments but their true passion is automating the a9s Data Services.
As a true Cloud Foundry enthusiast, Julian has visited all CF Summits and spoke at most of them.
A
A
So
mission
is
actually
meant
to
give
you
motivation
and
strength,
strength
to
survive,
setbacks
and
also
provide
a
accomplice
effect
enabling
autonomous
decisions
within
your
team,
so
I'm
just
giving
you
one
of
the
examples.
Your
mission
statement
could
look
differently
for
any
nines.
We
decided
to
strive
for
the
four
fully
automating,
the
entire
lifecycle
of
a
wide
range
of
data
services
to
run
on
cloud
native
platforms
across
infrastructures
at
scale.
A
A
So
the
first
and
most
important
principle,
it
also
is
reflected
in
the
title
of
the
talk,
is
that
we
want
to
have
a
mantra
which
is
automate
automate
automate.
So
why
is
automation
of
data
service?
Is
so
important,
I
think
from
the
experience
of
consulting
many
customers
on
the
digital
journey
of
digital
transformation,
we've
seen
that
the
adoption
of
application
platforms
usually
comes
with
ignoring
the
data
service
topic
in
in
the
first
phase
of
adopting
to
the
platform
there
are.
A
There
are
data
stores
such
as
Oracle
databases,
somewhere
in
the
enterprise
and
and
in
the
beginning
they
they
people
are
very
busy
with
learning
how
to
do
agile
development,
how
to
do
micro,
servers,
architectures
and
so
on,
but
over
time,
when
you
realize
that
such
an
application
platform
only
works
when
there
is
a
counterpart
on
the
data
service
side,
that
will
provide
you
with
the
same
user
experience
just
for
data
services.
What
platform
like
cloud
foundry
provides
for
applications?
A
You
will
see
that
automation
is
very
important
and
data
services
need
to
experience
the
same
automation
as
applications,
so
in
modern
application,
development
you're,
looking
at
micro
service
architectures,
where
monolithic
applications
have
been
split
in
to
application
systems,
more
applications
exist
and
subsequently
more
data
services
exist,
as
the
local
autonomy
of
those
teams
will
make
decisions
more
specific
to
a
particular
service.
So
one
service
could
have
a
relational
database.
The
other
could
have
a
document
database
and
so
on.
A
So
as
we
still
aim
for
improving
the
overall
innovation,
the
speed
of
innovation,
we
have
to
reduce
the
operational
friction
that
which
comes
with
that
increasing
number
of
applications
and
data
services.
Therefore,
automation
becomes
a
competitive
advantage
among
platforms
and
therefore
the
automation
of
data
services
also
important.
A
Referring
back
to
the
mission
statement.
There
was
the
statement
fully
automate
the
entire
lifecycle,
and
this
is
what
the
full
lifecycle
management
actually
refers
to.
It
actually
comprises
the
challenge
of
automating
everything.
A
database
administrator
would
do,
and
the
reason
why
this
has
to
be
pointed
out
is
because
there
are
a
lot
of
automation,
solutions
out
there
that
claim
to
be
data,
service,
automation,
solutions,
but
they
ignore
second
day
operations.
So
it's
one
thing
to
have.
A
What's,
for
example,
in
my
sequel
database
up
and
running,
we
have
done
that
with
chef
like
seven
years
ago,
but
in
contrast
to
application
containers,
your
databases
have
state
and
the
lifecycle
of
a
database
usually
is
years.
So
you
have
to
guide
that
database
through
very
different
scenarios,
such
as
a
failing
host
underneath
or
patch
level
releases,
minor
releases
and
even
major
releases.
A
A
So
if
you
start
automate
a
data
service,
it's
wise
to
iterally
increase
the
depth
of
the
automation
to
start
with
low-hanging
fruit,
those
lifecycle
operations
that
are
most
important
and
most
frequently
requested.
So
after
a
while,
this
may
look
like
this.
You
have
covered
the
life
cycle
of
a
data
service
instance.
Creating
a
data
service
may
be
clustered
may
be
single
versions
of
it.
For
example,
with
Postgres
you
could
have
a
single
VM
Postgres.
You
could
have
a
plan
with
cluster
Postgres,
both
of
different
sizes.
A
You
could
have
means
to
go
through
version
updates,
activate
or
deactivate
Postgres
plug-ins
create
backups
and
restore
them.
So
this
as
the
lifecycle,
automation
of
a
service
instance,
it
also
goes.
It
also
covers
very
essential
parts
of
what
we
are
talking
about
here,
but
I
definitely
have
to
point
out
that
this
is
not
everything.
A
But
we
come
to
that
a
bit
later
back
to
that
a
bit
later,
if
you,
if
you
start
automating
it's
it's
kind
of
obvious,
that,
depending
on
the
choice
of
your
data
service,
the
the
efforts
of
automation
will
differ
as
some
of
the
data
services
have
been
created
with
manual
operations
in
mind.
Postgres,
for
example,
is
such
an
is
such
a
data
service
that
has
a
lot
of
legacy
in
a
stay
in
a
because
it
emerged
in
in
a
time
where
automated
data
service
operation
was
not
a
thing.
A
So
let's
say
that
there
are
decision
factors
you
should
be
investigating
when
collecting
candidates
for
your
platform-
and
they
are
numerous-
this
list
is
far
from
being
complete,
but
it
demonstrates
the
complexity
of
such
a
decision
and
in
most
of
the
cases,
you
will
also
look
at
legacy.
Use
case
this
application
that
will
demand
that
you
automate
for
legacy
services.
A
Postgres,
is
a
good
example,
as
every
platform
needs,
a
relational
database
management
system,
you're
looking
into
Postgres
and
my
sequel
for
sure
you
will
have-
and
this
leads
us
to
the
datasource
categories-
you'll
have
to
cover
the
document
database,
the
caches
key
value,
store,
message,
queues
and
so
on.
So
in
in
all
these
services,
we'll
defer
on
how
robust
they
are,
whether
they
are
cluster
Abell
or
how
they,
how
they
handle
replication
and
so
on.
So
once
you've
picked
your
data
services,
it's
it's
somehow
determines
how
much
effort
you
will
spend
on
automation,
form
experience.
A
A
So
when
doing
platform,
consulting
we've
seen
that
most
of
the
organizations
we've
been
talking
to
if
they've
made
multi-million
investments
into
your
building
such
a
platform,
because
it's
just
a
very
much
effort
to
move
a
group
of
people
to
adapt
to
such
a
new
thing
and
change
the
way
they
develop
software
and
so
on.
So
in
an
organization,
it's
very
expensive
to
do
that.
So,
after
a
while
a
platform,
that's
economic,
a
viable
has
hundreds,
if
not
thousands
or
ten
thousands
of
applications.
A
A
Now
it's
a
fact
that
if
you
want
to
do
something
at
scale
that
chain
just
the
optimal
technical
solution
for
it
and
as
I've
been
doing
software
development
for
quite
a
while
and
while
this
seems
to
be
obvious,
people
tend
to
ignore
that
this
is
the
case
for
software
too.
So
I
prepare
that
that
little
funny
comparison.
So
we
have
three
different
technical
solutions
to
do,
one
thing
which
is
barbeque
sausages.
So
it's
just
the
scale
that
the
first
see
the
left
and
the
right
one.
They
they
just
are
different
in
the
scale.
A
They
do
the
same
thing
and
it's
pretty
much
the
same
for
your
data
service
automation.
So
when
I
talk
about
the
scale
of
a
platform,
then
we
are
talking
about
thousands
or
ten
thousands
of
data
service
instances.
So
if
your
automation
would
not
allow
you
to
do
that,
then
you
should
maybe
rethink
your
mission
statement,
because
this
is
basically
what
a
platform
is
designed
to
do
for
applications.
A
Obviously,
when
talking
about
cloud
solutions,
you
also
want
the
vertical
scalability,
so
you
can
stop
an
application
and
just
create
a
bigger
instance
of
it
and
the
same
you
want
to
do
with
your
data
of
the
database,
but
with
the
difference
that
then
you'll
have
to
keep
ensure.
Yet
that
you
have
to
ensure
that
the
data
is
still
there
after
you're
scaling
this
up
and
you
want
to
minimize
the
downtime.
So
this
is
about
service
instant
scalability.
A
So
this
also
requires
that
when
you
automate
data
services,
you
have
to
ensure
that
you
can
automate
a
larger
set
of
data
service
efficiently,
and
once
you
got
those
automation
release,
you
have
to
ensure
that
you
ship
those
into
the
customer
environment
and
just
to
give
you
a
understanding.
For
example,
we
deliver
data
services
in
too
many
different
organizations.
Each
customer
has
it's
very
own
300,
page
handbook
on
on
security,
so
this
delivery
pipeline
is
highly
customer
specific
and
includes
pen,
test
and
security
tests,
vulnerability,
scans
and
so
on.
That
are
highly
customer
specific.
A
A
So
that's
what
I'm
saying
is
you
have
to
get
your
release
management
right,
because
delivering
fast
is
very
important.
So
look
at
that
scenario.
In
the
past
there
was
a
DBA
who
has
access
to
the
upstream
source
code
of,
for
example,
Postgres
and
will
be
able
to
apply
a
patch
to
a
database.
Now
we
create
post
rest
automation,
release
and
ship
it
to
the
customer
environment
and
own,
and
then
it
is
up
to
the
platform
user
to
update
that
service
instance.
So
we
have
three
things
to
do.
The
first
of
all
is
the
automation.
A
Release
should
be
available
shortly
after
the
upstream
change
and
in
our
case,
for
example,
this
is
done
entirely
automatic
automatically.
That
means
that,
for
example,
if
Postgres
nine
point
four
point,
one
nine
point,
four
point:
two
is
released
a
few
minutes
later
we
have
an
automation
release
that
will
trigger
the
execution
of
a
comprehensive
test
suite
to
ensure
that
the
contract
between
the
data
service
and
our
automation
is
still
valid
in
intact.
A
So
then,
you
have
to
ship
that
to
the
customer
as
fast
as
you
can
minimizing,
that
time
requires
to
have
a
framework
here,
a
template
for
such
a
delivery
pipeline,
but
then
specify
and
somehow
customize
it
with
the
specifics
of
a
certain
customer,
and
even
one
customer
usually
has
half
a
dozen
or
more
of
those
Cloud
Foundry
environments,
for
example,
so
that
this
is
already
a
matter
of
scalability.
So
you
have
to
be
quick
with
that
and
obviously
now
the
platform
users
are
have
to
update
their
service
instances.
A
So
once
you
have
installed
the
new
version
of
Postgres
with
your
new
automation
into
the
cloud
environment,
then
you
still
have
hundreds
of
those
database
instances
being
locally
held
by
different,
maybe
customers
of
the
customer
and
and
need
to
be
encouraged
to
update
their
service
instances
and
to
do
that
fast,
because
you
want
to
deliver
security
patches
fast
right
and
to
minimize
that
time
you
have
to
provide
your
users
means
to
do
that
easily.
So,
for
example,
just
perform
a
CF
update
service
and
the
update
will
take
care
will
take
place
automatically.
A
A
Why
don't
we
just
take
out
that
the
entire
code,
which
saves
us
about
50%
of
the
codebase
and
just
throw
kearney
back
a
panic
and
shout
down
the
hole
that
somebody
needs
to
restart
that
server
and
if
you're
running
Linux
today,
he
will
see
kernel
panic.
So
it's
kind
of
a
strategy
that
works
today
as
well.
Coming
back
to
data
service,
it's
not
as
easy.
A
Shirley
has
hardware
redundancies
to
a
certain
degree
and
has
a
very
different
service
level
than
the
unavailability
ask
the
host
machine
where
your
database
runs.
So,
if
you
look
into
that
virtual
machine
that
contains
your
database
and
now
the
virtual
machine
goes
away,
your
persistent
disk
prevails.
It
can
be
a
gracefully
unmounted
and
a
new
virtual
machine
can
be
created
where
you
remount,
that
persistent
disk
and
there
you
go.
This
is
something
that
happens.
A
We
are
using
Bosh
to
automate
the
creation
of
service
instances,
and
this
is
something
that
happens
within
minutes
if
you're
using
kubernetes
the
restart
of
containers
even
faster.
So
this
can
happen
within
seconds
be
aware,
you
shouldn't,
you
shouldn't
place
your
workloads
of
your
data
services
on
a
kubernetes
cluster.
We
are
currently
investigating
deploying
data
services
to
kubernetes,
but
we
wouldn't
recommend
using
that
until
kubernetes
has
solved
the
I/o
isolation
issue
that
it
currently
has.
A
So
now,
all
the
automation
is
a
very
interesting
and
it's
very
nice,
so
we've
been
through
that
creation
of
the
database
and
so
on,
but
look
now
you've
created.
They
have
the
possibility
to
create
a
thousand
Postgres
clusters.
Mongodb
clusters,
rabbitmq
clusters.
Just
like
that.
Imagine
you
have
hundreds
of
customers
who
do
who
are
using
that
in
and
your
infrastructure
has
a
hiccup
and
or
for
some
other
reasons,
a
larger
number
of
clusters
fail
simultaneously.
A
You
still
need
to
ensure
this
on-demand
self-service.
So
how
can
you
do
that,
for
example
rabbitmq?
If
you,
if
you
use
RabbitMQ
wrongly,
you
can
easily
easily
kill
such
a
cluster.
So
how
would
you
recover,
depending
on
the
data
service,
it
will
also
always
be
important
that
you'll
have
such
a
recovery
and
as
a
last
resort.
A
So
by
providing
a
unified
backup
API,
you
can
actually
abstract
from
the
SATA
regina
t
at
some
point
and
then
settle
on
more
generic
resolution.
Logic,
for
example,
recovering
data
service
instances,
which
makes
the
existence
of
such
a
backup
framework,
for
example,
necessary.
Where
you
have
data
service,
specific
plugins
and
also
storage,
specific
products
on
the
right
side
and
in
between
a
filter
chain
that
does
compression
and
encryption.
A
Now,
it's
a
pretty
interesting
time
because
we
have
Cloud
Foundry.
On
the
one
hand,
that
allows
us
to
create
applications,
we
have
underneath
Bosh
that
can
also
be
used
to
provision
data
services
and,
at
the
same
time
we
see
kubernetes
becoming
more
and
more
popular
and
I
would
make
a
bet
that
at
some
point
that
isolation
problem
will
go
away
and
kou
Benitez
will
be
a
fair
choice
to
use
for
data
service
automation.
A
So
we
should
be
prepared
to
ship
in
two
different
automation
formats,
including
boss,
releases,
PCF,
tiles
and
kubernetes
helm,
charts,
for
example,
and
this
is
more
rhetorical
question,
because
at
this
point
you
have
to
somehow
organize
the
answer
to
that
question.
What
is
actually
shared
across
the
automation
of
PCF
tiles
and
kubernetes
ham?
Charts,
and
the
answer
is
your
operational
model,
because
once
you
solved
the
magic
around
organizing,
a
Postgres
cluster,
its
cluster
management,
its
failover,
its
leader
election
and
and
the
propagation
of
a
new
master
node.
A
This
logic
will
basically
be
the
same,
because
the
underlying
principles
are
the
same,
so
whether
you
are
restarting
a
container
or
your
VM
will
be
pretty
much
the
same
thing
and
I
bet.
You
could
use
the
same
test
suite
as
long
as
you
abstract
the
actual
deployment
so
yeah.
This
should
be
done
whenever
you
automate
that
you
should
well.
A
We
use
open
source
databases
and
because
we
believe
that
this
is
an
open
standard
between
the
application
end
and
the
NT
database,
and
you
can
get
it
from
everywhere,
so
preventing
a
login,
the
automation
language,
so
the
language
you
as
a
platform
user
talk
to
trigger
the
creation
of
a
database
should
be
an
open
standard
as
well,
and
the
open
service
broker.
Api
does
a
really
good
job
here,
so
this
avoids
a
bender
login.
A
Looking
at
this
very
at
this
mission
of
having
a
broader
set
of
data
services.
One
of
the
things
that
ensures
fast
release
cadence
is
to
never
touch
the
upstream
source
code,
with
the
exception
of
temporary
hot
fixes
that
are
that
have
to
be
monitored
as
long
as
they
applied.
You
wouldn't
do
that
in
the
long
term,
because
then
the
release
cadence
becomes
slower,
because
you
still
have
to
maintain
the
manual
harmonization
between
your
patch
and
upstream
changes.
A
Another
strategy-
and
this
is
a
major
one-
is
that
you
should
solve
any
issue
on
the
framework
level.
If
possible.
You
shouldn't
approach
the
automation
of
a
broader
set
of
data
service
without
having
a
framework
that
contains
solutions
to
most
common
issues,
so,
for
example,
on-demand
provisioning
of
dedicated
instances,
some
that
we've
put
into
a
framework.
So
whenever
we
automate
a
new
service,
we
can
rely
on
that
functionality,
and
all
we
have
to
do
is
create
a
bas-reliefs
and
a
component
that
manages
credentials
as
well
as
a
backup
and
restore
plugin.
A
That
boils
down
the
creation
of
new
services
to
around
four
to
eight
weeks,
which
is,
which
is
a
pretty
nice
thing
to
have,
and
the
reason
why
this
is
possible
because
we
always
think
about
any
problem
we
have
is.
Is
this
a
framework
level
problem
also
or
data
service,
specific
problem?
So
only
if
that
cannot
be
solved
on
us
on
a
framework
level
or
we
have
great
benefit
to
solve
a
data
service.
A
Specifically,
this
is
what
we
do
so
one
of
the
problems,
one
of
the
issues
that
you
could,
for
example,
solve
at
the
the
framework
level,
is
that
you
should
you
should
protect
your
data
service
instances
against
such
a
trivial
thing
like
like
the
disk
is
full.
You
would
be
surprised
how
many
data
servers
respond
to
that
event,
with
horriffic
failure
and
data
loss,
so
a
parachute
mechanism,
for
example,
that
prevents
that
is
one
of
the
things
you
can
easily
build
for
all
data
services
and
then
reapply.
A
So
obviously,
we've
seen
that
the
release
cadence
among
the
entire
value
chain
is
very
important
to
us.
So
without
release
tests
you,
how
would
you
deal
with
the
uncertainties
of
those
upstream
changes?
How
would
you
detect
that
the
contract
is
broken
between
the
Postgres
version?
You
just
recently
had
the
new
one
and
your
automation,
release,
for
example,
as
trivial
things
as
a
configuration
parameter
that
has
changed
or
a
default
value
that
has
been
introduced.
A
That
will
change
something
about
your
assumption,
especially
when
dealing
with
clusters
there's
a
lot
of
complexity,
so
you
should
be
able
to
test
those
things
and
should
provide
a
comprehensive
test
suite
walking
through
each
release
through
the
lifecycle
of
such
a
database.
So,
for
example,
we
excessively
test
the
reintegration
of
a
failed
master
in
Postgres.
Just
to
give
you
one
example,
this
gives
you
the
confidence
that
a
new
up
stream
automation
release
that
has
been
created
by
you
will
be
working
when
you
deliver
it
more
than
that.
A
So
this
talk
has
been
at
least
three
times
as
long
so
I
had
to
narrow
it
down.
I
will
also
release
a
plot
post
series
around
that
topic
with
a
longer
video
version
of
that
talk.
So
if
you
want
to
dig
deeper
into
that
topic,
just
go
by
the
any
ninth
block
and
you
will
find
more
to
that
to
just
reiterate.
A
I
strongly
believe
that
if
you
change
that
mission
statement
for
example,
and
replace
that
idea
of
having
a
lot
of
data
service
to
be
automated
with
just
one,
that
also
has
a
lot
of
impact
on
the
principles
so
setting
your
mission
in
in
conjunction
to
your
platform
strategy
is
very
important.
It
leads
to
some
principles
that
should
be
known
to
all
your
team
members
and
enables
local
autonomous
decisions.
A
If
you
split
your
automation
team
into
smaller
groups,
the
automation
is
very
important
and
should
rely
on
a
framework
that
has
a
lot
of
those
common
issues
across
data
service
work
into
it
and,
as
you
see,
there
are
technological
and
strategic
shifts
that
make
agnosticism
important.
Where
you
try
to
abstract,
though
so,
for
example,
by
using
technology
such
as
boss,
you
can
protect
against
or
you
can
be
flexible
and
deploy
to
different
infrastructures.
With
the
open
service
broker
API,
you
can
support
multiple
platforms
and
so
on.
A
So
you
try
to
keep
your
investment
safe
by
committing
to
the
right
things.
So
I
hope
you
gained
something
for
your
particular
data
service
mission.
If
you
have
any
questions
feel
free
to
reach
out,
we
have
a
booth
shoot.
Any
questions
if
come
by
and
ask
me
any
question
or
just
tweet
me
or
send
me
a
mail.
There
are
any
questions
left
just
feel
free
to
ask
I
think
we
have
a
few
minutes
left.