►
From YouTube: Containerizing PostgreSQL and Making it Cloud Native Ready - Jeff McCormick, Crunchy Data Solutions
Description
Containerizing PostgreSQL and Making it Cloud Native Ready
In this presentation, I'll show how I leveraged open source cloud technologies like Kubernetes (Openshift) and Prometheus to provide a set of data services around the PostgreSQL database. The Crunchy Containers provides not only a means to run PostgreSQL database clusters but also to administer and monitor them. This basic set of features could apply to any database as well. The design of these containers will be discussed as well as lessons learned along the way.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
Well,
good
day,
today,
I'm
going
to
talk
about
kind
of
a
case,
study
of
something
that
I've
been
working
on
for
about
a
year
and
a
half
crunchy
has
been
working
on
basically
creating
a
containerized
environment
around
the
postgres
database
and
crunchy
data.
A
little
bit
about
us
we're
a
postgresql
company
based
in
the
u.s.
A
It's
what
we
work
on
it's
our
niche.
It's
really
our
focus.
We
offer
secure
support,
there's
several
committers
and
various
postgres
folks
in
the
community
work
for
us.
So
it's
really
what
we
do
and
so
for
me,
my
opportunity
with
them
was
to
figure
out
a
way
to
containerize
the
postgres
and
make
that
experience
good
for
people
that
want
to
deploy
postgres
in
kind
of
a
cloud
environment.
A
A
So
why
would
you
containerize
postgres?
What
are
some
of
the
things?
Why
would
you
do
this?
One
is
to
encapsulate
any
complexities
around
deploying
a
database
installation
configuration
of
a
database
can
oftentimes
be
something
that's
complicated,
at
least
for
a
lot
of
developers
and
places
I've
worked
anyway.
Usually
you
go
to
a
database
group.
They
would
provision
you
a
database
and
data
kind
of
in
consumers
that
developers
really
didn't
know
the
complexities
of
that.
A
A
Originally
I
started
with
a
postgresql
container,
and
that
was
great,
and
then
it
quickly
became
obvious
that
there
was
a
need
for
other
containers,
things
like
being
able
to
do
a
database,
backup
things
like
administering
or
monitoring
containers
of
postgres
that
you
might
have
deployed
so
containers
provide
a
relatively
lightweight
way
to
spin
up
or
deploy
postgres
and
for
some
environments
and
some
customers
being
able
to
maybe
spin
up.
50
databases
really
rapidly
is
what
their
business
requires
them
and
before
that
used
to
be
a
very
timely
or
costly
experience.
A
So
containers
really
enable
that
rapid
deployment
and
most
customers
we're
dealing
with
now
are
putting
their
applications
in
more
of
a
cloud
environment
as
well.
So
we
wanted
the
database
to
go
along
with
say
web
applications
or
end
user
applications.
So
we
want
that
same
kind
of
lightweightness
and
rapid
deployment
model
for
databases,
as
we
did
for
web
applications.
A
We
use
docker
for
this
about
two
and
a
half
years
ago
I
was
working
with
running
postgres
and
open
shift
whenever
it
was
pre-docker
and
then,
when
red
hat
announced
they
were
gonna
need
a
docker.
I
was
kind
of
the
guy
standing
there
going
what's
docker
and
we
kind
of
made
our
bet.
That
was
going
to
be
the
widely
adopted
container
platform,
and
I
think
so
far
that's
turned
out
very
well.
A
So
what
are
some
of
the
design
goals
of
a
postgres
container
suite
early
on?
I
wanted
to
design
the
docker
files
in
such
a
way
that
I
could
support
different
types
of
operating
systems
as
the
base
image,
and
I
wanted
to
provide
this
container
suite
on
a
consistent
base,
image
customers
that
that
we
deal
with
they
would
prefer
to
manage
and
deploy.
A
You
know,
usually
one
kind
of
operating
system,
so
that
applies
with
the
containers
as
well.
So
all
of
the
docker
files
in
crunchy
container
suite
are
either
based
on
cenos7
or
rel7,
and
that
was
by
design
so
that
whenever
you
deploy
the
container
suite
you're
you're
able
to
deal
with
one
consistent
base
image
and
deal
with
security
issues
and
whatever
with
that
one
base
image,
may
support
other
ones
in
the
future.
A
But
those
are
the
ones
that
we
initially
targeted,
the
other
is
we
wanted
to
support
different
deployment
models,
so
the
container
suite
of
early
on
was
designed
to
deploy
on
just
pure
docker,
then
on
openshift
and
then
as
well
on
kubernetes
and
recently
I've
started
a
a
version
that
will
work
on
mesos
marathon.
So
that
is
really,
I
guess,
the
power
of
docker
being
a
standardized
container
suite
is
I'm
able
to
just
through
configuration
files
target
these
different
orchestration
environments
pretty
easily
and
in
a
pretty
seamless
way.
A
The
github
project
that
I'll
have
a
link
to
in
a
second
has
examples
for
all
of
those
environments
deploying
the
container
suite
in
all
of
those
different
environments,
and
there
will
be
probably
others
added
to
that
list.
As
as
I
learn
more
about
some
real,
exciting
new
container
platform,
orchestration
platforms,
that
will
certainly
grow
over
time.
A
All
of
these
basically
containers
postgres
containers
in
particular
all
have
to
address
the
issue
of
persistence,
though
so
people
ask
me
all
the
time
about
postgres
data
and
where
does
it
store?
Well,
all
of
the
data
is
offloaded
onto
outside
of
the
container.
Obviously,
so
you
get
to
the
data
files
for
postgres
through
some
kind
of
volume
mount
and
all
of
those
different
deployment
platforms,
give
you
different
options
for
persistence
and
how
you
mount
volumes
and
how
you
provision
volumes.
A
I
could
probably
talk
for
two
hours
just
alone
on
all
the
various
ways
and
gotchas
and
security
issues.
So
that's
a
big
topic
just
unto
itself,
one
of
the
goals
or
design
goals
with
this
was
to
allow
end
users
to
customize
postgres
configuration
you
can
configure
postgres
in
like
an
infinite
number
of
ways.
It
seems
like
so
things
like
this.
A
The
sql
that
initializes
a
database,
for
instance
is,
is
something
I
wanted
to
be
outside
or
mounted
outside
of
the
container
itself,
so
that
enables
users
to
customize
the
objects
that
populate
the
database
when
it
comes
up.
Things
like
the
postgres,
hba,
conf
and
postgresql.com
files
are
other
examples
of
you
can
use
the
ones
that
are
generated
or
initialized
by
postgres
by
the
container
itself,
or
you
can
mount
those
externally.
A
There's
other
examples
too,
where
maybe
you
don't
want
to
supply
your
own
configuration
files,
but
maybe
you
want
to
tweak
just
some
values
inside
a
standard
configuration
file,
so
those
examples
are
or
those
kind
of
things
are
exposed
with
environment
variables
and
then,
lastly,
if
this
be
with
this
being
open
source
as
well,
you
can
pull
down
the
docker
files
and
completely
customize
the
behavior
as
you
want.
So
all
of
those
things
were
design
goals
really
early
on
and
the
goal
was
to
make
this
container
suite.
A
You
know
as
usable
and
as
extensible
by
as
many
wide
audiences,
we
could
make
it
really
any
database
as
a
service
type
container
environment.
You
would
want
those
kind
of
extensibility
features
in
it,
and
you'd
want
to
look
at
those
from
the
from
kind
of
the
beginning.
If
you
were
going
to
look
at
you
know,
non-postgres
database
container
environments
are
container
suites
as
well.
You
want
those
kinds
of
extension
points.
A
Postgres,
in
particular,
some
of
the
things
that
customers
ask
us
is
they
wanted
the
ability
to
create
a
postgres
streaming
replication
type,
cluster
environment.
They
wanted
to
do
that
pretty
easily,
and
the
reason
is
that
was
something
that
a
lot
of
them
had
struggled
with
and
doing
it
in
a
consistent
way,
and
some
some
shops
found
that
kind
of
complicated,
so
clustering
postgres
was
an
early
design
goal
as
well.
A
Replication
is
part
of
that
and
we
wanted
to
support
either
synchronous
or
asynchronous
replicas.
So
postgres
is
a
single
master
database
today
and
then
you
can
have
multiple
read-only
replicas.
So
that's
another
design
constraint.
Whenever
you
talk
about
building
a
container
suite,
is
you
have
to
keep
that
in
mind
of
the
inherent
way
postgres
replication
works?
A
Multiple
locale
support
is
something
that's
built
into
the
container
suite
for
the
postgres
database
container
and
in
a
cube
environment
or
an
open
shift
environment.
We
support
the
ability
to
encode
your
database
credentials
in
the
secrets
and
that's
a
feature-
that's
just
unique
to
kubernetes
environments.
A
A
So
that
gets
you
into
kind
of
like
more
than
just
a
database
container.
It
started
us
thinking
about
a
suite
or
kind
of
a
database
as
a
service,
and
what
does
that?
Look
like
backup
and
restore
is
a
big
feature
of
that
and
right
today,
the
container
suite
supports
a
full
database,
backup
using
a
postgres
utility
called
pg
based
backup.
A
We
just
want
to
enable
that
use
or
execution
of
that
in
a
container
environment
so
created
a
backup
container,
which
is
called
crunchy
backup
and
when
you
run
that
container
as
like
a
kubernetes
job,
you
just
pass
it
enough
information
about
a
running
database
container
and
it
begins
to
create
a
full
database.
Backup
it'll
persist
it
off
to
something
like
an
nfs
volume
mount
and
all
of
those
nfs
volumes
are
basically
time
snapped
or
so
that
you
can.
A
Then
fire
up
a
brand
new
database
container,
pass
it
in
an
environment
variable
where
its
backup
is,
and
it
will
restore
from
that
backup.
That
is
a
very,
very
small,
thin
container.
But
it's
an
example
of
not
reinventing
the
wheel,
but
just
cloud
enabling
a
already
bulletproof
postgres
feature
or
utility,
typically
kind
of
backtrack,
a
little
bit
typically
that
container
most
of
the
logic
in
it
is
just
a
simple
bash
script
as
well.
A
So
there's
it's
very
easy
for
people
that
if
you
understand
bash
and
a
little
bit
of
linux,
you
can
get
in
and
look
at
the
docker
file
and
tweak
that
docker
file
and
the
bash
script
that
just
merely
wraps
pg
based
backup
and
customize
it
to
fit
your
needs.
So
that's
the
other
goal
is
to
kind
of
keep
the
orchestration
inside
the
containers
to
bare
minimum
and
then
depend
on
orchestration
outside
of
the
containers.
A
So
I've
seen
some
people
they'll
write
a
container
they'll
bake
into
a
million
different
things,
even
orchestration
logic,
and
I've
really
tried
to
resist
that
temptation
and
make
this
just
a
real
lightweight
set
of
docker
or
wrappers.
Essentially
around
some
of
these
things.
High
availability
was
another
thing
that
came
up
as
most
db.
A
So
there's
a
container
called
crunchy
watch
that
does
nothing
but
looks
at
a
postgres
master
container
to
see
if
it's,
if
it's
responding,
if
so
gray,
but
if
not,
it
will
trigger
a
failover
to
one
of
the
slave
replicas
again
a
very
small
amount
of
code,
that's
probably
about
like
100
lines
of
bash
script.
It
leverages
a
existing
postgres
utility
called
pg
is
ready.
A
It
collects
those
and
pushes
them
off
to
prometheus,
and
then
I
used
something
like
grafana
again,
all
very,
very
well
known,
stable,
open
source
projects,
containerized
those
and
now
I've
basically
got
a
custom,
postgres
metrics
collection
and
graphing
dashboard
with
again
very
little
bit
of
code,
and
I
think
that's
really
the
future
again.
A
nice
aspect
of
using
open
source
projects.
A
A
You
basically
now
have
the
ability
to
have
your
own
schedule
of
things
that
you
want
to
do.
There
is
a
one-to-one
relationship
between
the
crunchy
dba
container
instance
and
a
database
container
as
well,
so
that
was
kind
of
the
model
there
is.
If
you
had
50
databases,
you
could
assign
a
unique
dba
container
to
each
one.
So
it's
kind
of
a
virtualized
dba
that
you
you
would
associate
so
again.
I
was
trying
to
cloud
scale
out
automation
or
admin
task
and
monitoring
along
with
the
database
container.
A
A
A
People
use
it
as
a
single
entry
point
into
a
postgres
cluster
took
that
utility
wrapped
it
in
a
docker
container,
add
just
a
little
bit
of
integration
points
so
that
it
can
understand
what
your
master
service
and
your
your
replica
services
are.
How
to
find
them-
and
now
you,
whenever
you
stand,
that
up
in
front
of
a
postgres
container
cluster,
you
essentially
have
a
smart
load.
A
That
is,
that
goal
line
code
that
scrapes
or
pulls
postgres,
metrics,
crunchy
dba
we
mentioned,
which
is
a
simple
cron
scheduler,
with
a
small
amount
of
glue
code
that
leverages
another
feature
of
kubernetes
called
a
service
account
and
service
accounts
enable
you
to
do
things
like
dynamically
talk
to
the
kubernetes
api
and
dynamically.
Therefore
spin
up
containers
on
demand,
so
that's
how
it's
able
to
dynamically
spawn
off
things
like
backup,
jobs
and
vacuum
jobs,
and
then
the
last
one
that
we
created
was
a
containerized
version
of
a.
A
I
have
another
open
source
project
called
pg
admin
4,
which
is
a
brand
new
rewrite
of
the
older
pg
admin
tool.
Pg
admin
4
is
a
web-based
console
for
postgres
so
and
it's
the
build
of
it
is
kind
of
interesting
if
you
try
to
build
it
from
source.
So
this
was
a
good
example
of
when
you
containerize
something
like
that.
A
Orchestration
intersects
with
something
like
this
container
suite
in
a
number
of
different
ways:
replicas
need
to
be
able
to
look
up
their
masters
and
that
ultimately,
usually
turns
into
a
dns
resolution.
So
when
you
run
a
container
suite
in
all
of
these
different
orchestration
environments,
you
need
to
think
about
okay.
How
am
I
going
to
do
resolution
in
a
pure,
docker
environment?
You
may
do
that
just
with
container
links.
A
Okay,
if
you're
not
running
dna,
some
kind
of
dns
service,
but
in
something
like
kubernetes
or
openshift,
you
can
leverage
the
dns
that's
built
in
there.
So
that's
how
some
service
discovery
the
service
discovery
that
we're
making
use
of.
Typically,
when
a
replica
runs,
it
just
needs
to
find
its
master,
and
it
needs
to
be
able
to
look
that
up
in
some
way.
A
Networking
in
general,
multi-host
docker,
is
where
orchestration
frameworks,
I
think,
really
shine
and
make
your
life
easier.
As
somebody
who
early
on
tried
to
set
up
my
own
multi-host
doctor
environment
two
years
ago,
it
was
definitely
a
challenging
thing,
there's
also
another
environment.
I
didn't
mention,
but
we've
ported
this
to
actually
work
with
portworx
to
import
our
source
here
yesterday,
demoing
and
that's
the
basically,
they
offer
a
doctor
volume
driver
mechanism.
A
So
there's
also,
if
you
look
in
the
examples,
a
way
of
running
the
crunchy
container
suite
to
use
the
portworx
environment
and
that's
a
persistence
mechanism
just
in
a
pure
docker
environment
that
you
can
leverage
to
do
replicated
volume
management.
So
that's
an
interesting
one
to
look
at
if
you've
never
looked
at
portworx.
A
A
Ultimately,
though,
in
terms
of
the
postgres
suite
really,
all
you
need
is
a
way
for
replicas
to
look
up
their
master
and
for
you
to
be
able
to
access
databases
through
things
like
host
names
or
container
names,
and
the
examples
that
we
have
out
there
you'll
see
how
I
do
that
if
you
there's
a
document
for
each
environment,
that
the
container
suite
supports,
persistence
in
general
is
an
interesting
topic.
I
could
give
a
separate
discussion
on
that,
but
there
are
some
interesting
things
with
postgres.
A
You
want
to
have
a
a
reliable
right,
so
whatever
file
system
you're
writing
to
if
it's
really
low
or
really
slow
over
a
network,
it's
obviously
going
to
impact
your
postgres
performance.
So
when
you
talk
about
deploying
these
containers
for
real
in
a
production
setting,
the
choice
of
your
persistence
is
really
key,
and
that's
where
the
orchestration
frameworks
really,
I
think,
differentiate
themselves,
and
if
you
were
going
to
deploy
postgres,
for
instance,
especially
clusters
of
postgres.
A
A
Hopefully
I'll
I'm
going
to
give
you
a
demo
of
this,
but
I've
cooked
up
an
example
called
the
kitchen
sink
example.
So
what
can
you
do
with
this
other
than
just
deploy
a
single
database
which
is
sort
of
uninteresting,
so
I
cooked
up
this
example
that
tries
to
use
as
many
of
the
containers
as
possible
to
deploy
a
non-trivial
postgres
deployment.
A
That's
going
to
guarantee
that
my
master
is
always
in
sync
with
something
in
case
the
master
falls
over
then
I'm
going
to
put
a
master
proxy
in
front
of
the
master
container
and
then
I'm
going
to
route
basically
use
that
as
a
basically
a
router
or
a
service
endpoint
into
the
master,
I'm
going
to
do
the
same
for
the
replicas,
the
replicas
that
are
pointed
to
both
the
synchronous
and
the
asynchronous.
So
it's
going
to
do
round-robin
load
balancing
between
those,
then
there's
going
to
be
the
watcher.
A
A
Replicas
can
only
do
reads
so.
Pg
pool
is
the
only
thing
I've
seen
smart
enough
to
direct
sql
traffic
to
the
right
endpoint
based
on
the
type
of
sql
statement.
So
there's
a
service
in
front
of
pg
pool
as
well,
so
that
example
ultimately
is
worked
up
into
lots
of
different
js
kubernetes
json
templates.
A
These
are
all
out
in
the
github
and
you
execute
some
people
never
worked
with
kubernetes.
So
I've
let
this
slide
in
here,
but
it
shows
you
you
execute
these
commands.
It
reads
in
these
templates
which
acts
as
acts
as
reps
recipes
for
the
build
out
of
all
of
that
infrastructure
on
that
diagram,
and
this
example
is
out
there.
It's
called
kitchen
sync
you'll,
see
it
inside
those
json
files
again
as
kind
of
a
recipe
of
exactly
what
container
should
it
start
up
and
what
kind
of
parameters
should
it
pass?
A
That's
the
location
of
this
example.
If
you
really
want
to
see
details
of
a
real
complicated
example,
that's
just
the
dashboard
for
kubernetes,
because
there's
nothing
else
graphical
to
see
and
then
just
to
summarize,
the
crunchy
postgres
container
suite
is
just
one
example
of
a
database
as
a
service.
A
There's
lots
of
other
ones
out
there
that
are
being
built
or
have
already
been
built,
but
I
think
it
is
representative
of
what
people
would
run
into
the
kinds
of
issues
if
they
were
going
to
build
their
own
database
as
a
service.
Suite
networking
and
storage
are
particularly
hard
to
do
when
it
comes
to
building
out
clustered
databases
in
particular,
storage
and
replicating
storage
and
making
making
sure
that
it
meets
your
kind
of
slas.
A
Monitoring.
Administration
need
to
really
mirror
the
the
agility
that
you
get
whenever
you
deploy
databases,
if
you
deploy
a
database
container
in
a
second,
you
really
don't
want
to
go
use
some
really
slow
process
to
deploy
administration
and
monitoring
features.
So
really
what
you
want
to
do,
I
think,
is
try
to
automate
the
administrative
and
monitoring
features
features
along
with
the
database
itself,
using
an
open
source
database
like
postgres
made
this
a
lot
easier.
A
I
could
take
all
of
these
utilities
and
code
build
out
these
containers
without
worrying
about
a
lot
of
close
source
licensing
issues
which
you,
which
would
make
this
a
lot
harder
to
do
so
I'm
freely
able
to
build
out
these
containers
and
distribute
them
like
they
like.
I
do,
and
in
some
cases
I
even
was
able
to
had
to
basically
go
into
the
postgres
c
source
code
to
figure
out
exactly
what
it
was
doing
to
make
it
actually
work
in
things
like
open
shift
dedicated.
A
That's
my
contact
information.
If
you've
got
any
questions,
there's
also
the
github
site
on
here.
You
can
download
all
of
this
and
the
containers
themselves
are
out
on
docker
hub.
I
don't
have
the
docker
have
a
link
there,
but
it's
if
you
do
a
search
for
crunchy
data
on
docker
hub
you'll,
see
all
of
the
the
different
containers.
Now,
if
I've
got
two
minutes
I'll
just
show
you
a
brief
demo
if
or
no
we're
out
of
time,
okay,
so
anyway,
thank
you
very.