►
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Postgres Extensions in Kubernetes - Álvaro Hernández, OnGres
Postgres Extensions are one of the most distinctive and appreciated features of Postgres. Acting like “plugins”, they allow you to extend Postgres functionality. From adding simple data types to turning the database into a sharded cluster with a distributed query planner and executor. Yet they are hard to use in a container environment like Kubernetes. Because container images are immutable! So you either pack all possible extensions in a fat container, which leads to substantial problems; or you load dynamically under demand. Join this talk to explore the extension ecosystem in Kubernetes Postgres Operators; and how to solve this problem by introducing a system for dynamically loading extensions into the containers, and cache them within the cluster to avoid excessive downloads. A new operator pattern, the “pod-local controller” will also be introduced, as the technological solution that powers dynamic extension loading.
A
Hello,
everybody
and
welcome
to
this
talk
on
kubecon
cloud
nativecon
about
postgres
extensions
in
kubernetes.
In
this
talk,
I
would
like
to
introduce
a
novel
mechanism
built
into
an
open
source
project
to
allow
dynamic
loading
of
postgres
extensions
in
kubernetes.
This
is
a
very
noble
way
that
enables
to
run
any
extension
that
you
may
want
to
run
in
again,
kubernetes.
So
first
about
myself,
who
am
I
you
can
just
get
to
the
slide,
but
basically
I
am.
The
ceo
and
founder
of
company
called
ongress.
A
Congress
means
on
postgres
and
that's
pretty
much
everything
that
we
do.
I've
been
myself
a
postgres,
dba,
developer
application
developer
for
more
than
20
years,
and
I'd
like
to
specially
specifically
focus
on
r
d
on
databases
specifically
podcast.
I
also
like
to
speak
at
postgres
and
all
the
types
of
conferences
online.
Actually,
I
have
to
say
that
this
is
my
first
talk
at
cubecon.
I've
tried
four
times
before
I've
been
rejected,
so
I'm
very
happy
to
finally
made
it
finally
very
happy
to
be
here
with
you.
So
let's
look
at
phosphorus
extension.
A
Let
me
do
a
one
on
one
and
pause
for
sectioning
extensions.
In
case
you're
not
familiar
with
them,
so
postgres
extensions,
you
can
think
of
them
as
the
plugins
of
a
browser
right
there,
they're
software,
bundles,
that
you
may
install
that
enhance
the
browser
functionality.
So
postgres
extensions
are
pretty
much
the
same
but
apply
to
a
postgres
database.
They
extend
the
function,
the
database
in
so
many
ways
as
we
shall
see,
they
are
actually
one
of
the
most
often
cited
best
features
of
possible.
A
What
is
also
important
about
extensions
is
that
they
do
not
follow
the
yearly
cuttings
release
cycle
of
postgres.
New
features
come
to
posters
only
once
a
year.
So
if
you
want
to
develop
new
features
and
you
pack
them
as
extensions
you'll,
be
able
to
deliver
these
features
to
your
users
any
moment,
and
so
that's
why
they
are
normally
developed
by
third
parties.
A
The
extensions
are
also
important
for
the
ecosystem
because
they
have
prevented
many
forks
of
phosphorus,
because
if
you
can
run
everything
as
all
your
functionality
as
an
extension,
this
is
often
the
case.
Then
you
don't
need
to
fork
phosphorus
and
it's
just
something
that
built
on
top
of
postgres
and
can
run
on
top
of
any
existing
phosphorus
without
having
to
to
fork
it,
and
most
of
them
are
open
source.
A
So
what
is
the
extent
of
which
you
can
functionality?
You
can
build
on
top
of
extensions.
Well,
first
of
all,
you
can
add
any
database
object.
Database
objects
can
be
data
types
function.
This
includes
aggregate
functions,
operators
or
even
procedural
languages.
Yes,
procedural
languages
are
the
languages
that
are
used
in
phosphorus
to
to
call
functions
or
procedures,
and
they
postures
allows
you
to
bring
your
own
language.
So
you
could
bring
a
procedural
language
as
an
extension.
A
They
could
also
be
what
they're
called
another
database
as
storage
engines,
which
in
postgres
are
table
and
index
access
methods
which
allows
is
an
api
part
of
the
postgres
api
that
allows
you
to
create.
You
can
do
the
same
extension
to
implement
the
different
mechanisms
for
storing
the
data
in
either
tables
or
indexes.
A
You
could
there's
hooks
which
you
can
call
to
implement
extension
functionality,
but
in
reality,
in
an
extension,
you
can
call
any
internal
phosphorus
function.
Everything
is
allowed
from
this
perspective,
so
you
could
also
do
replication
wall
that
affects
durability
of
the
database.
You
can
create
background
workers,
which
are
processes
that
are
managed
by
the
database
to
perform
different
tasks,
and
you
can
even
include
external
files
to
the
database
like
binaries
or
other
kind
of
data.
That
may
also
compact
as
extensions.
A
So
how
do
you
program
an
extension?
Well
if
the
extension
is
just
about
creating
objects
in
the
database,
you
could
use
either
sql
or
plp
sql
or
any
other
procedural
language
to
create
these
objects,
but
for
the
most
part,
extensions
are
programming
c
and
compiled
into
a
shared
library
that
can
be
loaded
into
the
database.
A
These
are
the
extensions
that
actually
allow
you
to
call
all
the
methods
inside
prosperous
code,
but
in
reality
anything
that
compiles
your
shirt.
Library
can
use
with
extensions.
So,
for
example,
you
could
use
the
somebody
extension
that
I
just
mentioned
before
is
programming
rust.
You
could
also
use
c
plus
plus,
or
you
know,
any
new
innovation
is
welcome,
bring
your
own
favorite
language.
If
it
can
compile
to
an
extension,
it
will
run
or
should
run.
A
So
how
do
you
use
the
extensions?
Let's
assume
for
a
moment
that
they're
already
installed
so
if
they're
installed,
they
should
have
some
files
under
typically,
a
user
share
phosphorous
ql
mayor
version
extension
directory
there.
You
may
find
the
control
files
which
are
metadata
about
the
extension
and
sql
files,
which
are
the
commands
required
to
create
the
objects
that
will
represent
the
extension.
If.
A
Now,
once
the
extension
is
installed,
if
it
is
a
c
extension
which
is
going
to
load
the
shared
library
into
the
phosphorus
space,
then
it
probably
needs
to
be
loaded
via
mechanism,
which
is
a
postgresql.conf
configuration
parameter
called
sharepreload
libraries.
This
is
a
list,
a
csv,
a
list
of
extensions
that
you
want
to
preload
when
the
database
starts.
A
So
you
add
the
extension
to
this
parameter
and
then
you
require
database
restart
because
those
are
loaded
into
memory
at
the
start
of
the
database
and
we'll
still
see
where
how
it's
also
important
later,
then
you
connect
to
any
database
where
you
want
this
extension.
This,
let's
say
functions
data
types
whatever
to
become
available
and
just
around
the
sql
command,
create
extension.
Extension.
Name
done.
That's
that's
all!
You
need
to
run
the
extension.
A
But
how
do
we
install
the
extension
because,
before
I
said,
let's
assume
they
are
in
red,
install
how
to
install
them?
Well,
there's
no
such
thing
as
an
extension
store,
so
basically
search
engines
right
in
reality,
some
extensions
come
pack
as
their
packages
or
rpm
packages
and
there's
also
a
repository
called
pgxn,
which
also
comes
with
a
client
tool
that
you
install
and
helps
downloads
the
extensions
the
this
is
a
fantastic
tool,
but
the
problem
is
that
pgx
and
extensions
they
are
in
source
code
form,
so
you
need
to
compile
them.
A
Pgxn
automates
this
process,
but
if-
and
only
if,
you
have
already
installed
all
the
build,
all
the
build
libraries
plus
all
the
build
libraries
and
dependencies
of
all
dependencies
that
the
extension
that
you're
gonna
compile
requires,
which
oftentimes
is,
is
a
hard
task
for
the
user.
So
it's
not
bad,
it's
great,
but
it's
not.
Batteries
included
they're,
providing
this
source
code.
For
so
this
is
how
extensions
are
in
phosphorus
for
the
general
pilot
part.
Now,
let's
look
at
extensions
specifically
in
kubernetes
and
what
problems
come
when
trying
to
bring
all
these
extensions
into
kubernetes.
A
Just
before
doing
that,
let's
have
a
look
at
what
is
the
available
set
of
extensions
on
cloud
managed
offerings?
So
if
we
look
at
postgres,
postgres
basically
includes
76
extensions.
There's
a
small
typo
there
inside
76
extensions,
eight
procedural
language,
extensions
which
are
procedural
languages
for
creating
functions
and
procedures
and
68
so-called
contrib
extensions.
They
they
all
come
with
the
post-resource
code
package,
so
total
of
76.
A
You
will
look
at
the
main
cloud
offerings
like
our
amazon,
rds
and
aurora.
Google
called
sql
or
azure
in.
If
is
different
offerings.
The
number
of
extensions
is
between
50
and
70
in
total,
and
if
you
look
at
the
core
extensions,
this
country,
plus
this
procedural
language,
most
of
them
are
available,
but
definitely
not
all
of
them.
A
In
some
cases,
even
even
32
of
those
four
country,
extensions
are
not
available
to
use,
you
cannot
install
them
and
with
regards
to
third-party
extensions
of
which
there
are
hundreds
or
even
thousands
available
in
the
world,
a
very
few,
a
very
small
number
of
them
are
available.
The
best
case
is
amazon
aurora,
where
38
extensions
are
available.
Now,
what
do
you
do
if
you
want
one
of
the
extensions
that
are
not
present
on
these
environments,
because
you
cannot
load
them?
It's
they're.
A
They
need
to
be
allowed
on
a
one
by
one
basis,
because
this
is
a
managed
offering
right
so
you're
out
of
black.
You
either
need
to
do
without
extension
or
or
manage
database
yourself
now.
Kubernetes
operators
allows
you
to
enjoy
an
experience
similar
to
managed
posters
as
a
service.
So
maybe
kubernetes
operators
is
the
answer
to
this,
because
there,
maybe
you
will
have
more
freedom.
Well,
the
reality
is
that
the
landscape
looks
pretty
much
the
same.
We
look
at.
A
A
Well
that
may
work,
but
it
has
some
problems.
First
of
all,
that
may
result
in
a
gigantic
container
image,
even
though
extensions
are
normally
not
that
large.
Some
of
them
are
like
gis,
which
weighs
in
around
700
megabytes
in
reality,
you're
going
to
end
up
with
a
gigabytes
database
gigabits
container
image.
Well,
that's
not
the
end
of
the
world,
but
it's
certainly
not
ideal,
but
also
from
a
security
perspective.
You're
loading,
hundreds
of
different
projects
which
represent
a
huge
attack
surface,
and
that
is
not
ideal
from
a
security
perspective.
A
But
last
but
not
least
when
you
want
to
introduce
a
new
extension
or
a
new
extension
version
to
to
fix
a
security
problem
on
one
of
the
available
extensions.
As
part
of
this
container
image,
you
need
to
bring
a
new
image
and
this
means
restart
the
pods,
and
this
means
restart
the
database,
which
is
a
quite
disruptive
operation
for
a
database.
So
it's
something
you
don't
want
to
do
when
running
a
database
to
frequently
restart
the
container.
A
A
Then
we
could
think.
Okay,
we
could
look.
We
could
load
the
extensions
into
the
ephemeral
storage
of
the
container
itself.
The
image
is
immutable,
but
once
the
container
is
running,
there's
the
femoral
starch
of
the
container,
and
maybe
we
could
load
the
extensions
there
that
would
work.
However,
it
has
some
problems
if,
for
example,
the
node
dies
or
the
container
is
restarted,
this
has
lost.
You
will
need
to
re-download
the
process,
all
the
time
and
reprocess
extensions
all
the
time,
that's
also
doable,
but
it's
not
ideal.
A
Instead,
what
we
do
is
download
and
install
the
extensions
as
on
the
pv
on
the
persistent
volume
which
is
typically
external
or
distributed
storage,
so
that
it
will
persist
to
survive
and
note
failures
or
or
pod
restarts.
So
once
you
have
installed
the
extension
once
you
will
probably
be
there
all
the
time.
You'll
have
basically
the
same
lifecycle
of
your
database
that
data,
so
the
pg
data
directory.
In
phosphorus,
we
also
use
this
functionality
to
relocate
phosphorus
binaries
into
the
persistent
volume
and
simulink
the
extensions.
A
This
is
also
useful
not
only
for
the
extension
process,
but
also
for
major
venture
upgrades
and
after
this,
the
most
important
and
significant
feature
introduced
by
this
functionality
is
what
we
call
the
cluster
controller.
The
cluster
controller
is
a
is
a
kubernetes
controller
or
operator
more
precisely,
a
controller
that
runs
as
a
sidecar
along
the
pod.
A
This
is
because
this
sidecar
requires
access
to
the
file
system,
is
going
to
download
the
extensions
and
pack
them
and
install
them
into
the
pv,
but
it
is
operates
as
a
small
controller
on
itself
and
I'll
explain
the
functionality.
Actually
I
don't
know
if
this
is
a
this
model
of
having
this
local
controller
or
or
hierarchical.
Controller.
A
So
from
the
architectural
perspective,
we
see
here
that
there
are
two
operators
or
two
controllers,
the
main
stackers
operator,
which
is
the
usual
phosphorus
operator
you
can
think
of
which
is
on
the
top
left
corner
of
the
diagram
and
when
the
user
instantiates
a
scrd
of
a
type
of
sd
cluster
with
a
spec
field
that
basically
specifies
that
you
want
to
install
some
extension.
We're
also
going
to
see
how
this
works.
Then
the
operator
is
going
to
perform
some
actions
and
it's
going
to
say
the
pods
and
so
forth
within
the
pod.
A
This
local
controller,
the
cluster
controller,
is
going
to
read
the
information
on
the
cr
and
it's
going
to
interpret,
which
extension
needs
to
download
it's
going
to
connect
to
this
external
extension
repository
running
on
the
internet,
download
the
extensions
unpack
them
verify.
Digital
signatures
create
siblings
and,
in
general,
make
them
available
to
postgres,
and
they
will
all
be
stored
in
the
in
the
persistent
volume.
A
Let's
see
how
it
really
works
with
a
real
code
example,
so
this
is
the
minimal,
minimal,
cr
minimal,
hd
cluster,
that
you
may
need
in
stackers
to
create
a
cluster
with
two
parts
for
the
possible
latest:
postgres
version,
thinking
about
specific
storage
for
each
node,
and
this
will
also
include
connection
polling
and
amboi
proxy
for
practicing
plus
traffic
and
and
a
few
high
availability.
A
few
other
builds
on
this
list.
Okay,
so
this
is
a
basic
cluster
cr
that
you
may
want
to
create.
A
So,
let's
not
let's
say
that
the
cluster
is
already
created,
let's
add
an
extension
to
it.
So
to
do
so,
we
just
need
to
add
these
two
lines
that
are
involved
here:
extensions
and
the
name
of
the
extension.
We
could
also
add
a
specific
version
for
the
extension,
but
this
is
optional.
If
not
stackers
will
resolve
it
now.
How
do
we
know
which
extensions
are
available,
their
names
or
their
versions?
A
Well,
there's
three
ways:
you
can
do
this:
you
can
either
go
to
the
stackers
web
console
which
allows
you
to
search
for
extensions
and
provide
descriptions
and
see
their
available
versions.
You
can
also
check
the
documentation
or,
if
you're
gonna
go
the
hard
way
you
can
use
jq
to
parse
the
repository
metadata
right.
So
let's
go
back
to
the
extension.
So
when
we
create
these
and
and
we
do
a
quick
control
apply
and
we
have
this
extension's
name
postgres,
what
happens?
A
Is
that
there's
going
to
be
a
mutating
webhook
that
is
going
to
retrieve
the
extension
metadata
from
the
repository?
And
it's
going
to
add
this
to
install
postgres
extensions
section,
which
is
basically
additional
metadata
about
the
extension,
including
the
internal,
build
number,
the
postgresql
version
with
which
is
compatible.
The
detailed
pulse
gis
version
in
this
case
301
the
publisher
or
what
is
the
repository
in
the
extension?
A
If
this
extension
would
not
be
found
a
warning
would
be
issued,
then
there
is
a
validating
webhook
that
checks
that
the
requested
extensions,
because
there
have
been
express
declaratively
that
you
want
this
postgis
extension
to
to
be
present.
That
is
also
present
in
the
to
install
postgres
extensions
resolve
metadata.
If
there's
no
matching,
if
paul's
gis
would
not
be
both
on
the
top
section
and
the
section
below,
then
the
validation
web
hook
would
reject
this
change
so
because
the
extension
would
not
be
available
now.
A
That's
that
will
list
the
extensions
installed
again.
They
need
to
be
installed
for
every
part
and
then
updates
the
status,
bringing
back
information
feedback
to
the
user,
about
which
extensions
have
been
installed
successfully.
Also
following
standard
kubernetes
patterns.
It
creates
a
set
of
conditions
in
the
also
in
this
part
of
the
status
field.
A
B
A
B
A
A
That
will
list
the
available,
not
loaded
just
on
the
file
system,
not
my
day.
They
level
sorry
the
available
extensions
in
in
the
in
the
file
system.
As
you
see,
there's
there's
no
close
gis
actually
there's
a
very
few,
a
small
amount
of
extensions
here.
Normally,
you
would
find
those
of
extensions
in
a
normal
installation.
These
are
just
extensions
that
are
used
internally,
so
they
are
built
by
the
they
are
brought
by
default,
but
they're
also
installed
by
the
same
mechanism.
A
A
A
A
And
we
will
see
here
that,
on
the
request
section,
we
just
said
that
we
want
to
close
gis,
also
the
mutating
webhook
added
the
version
here,
but
then
there's
going
to
be
this
two
installed
postgres
extensions
section
where
the
extensions
are
listed.
In
this
case,
we
can
see
here
that
pulse
gis
has
been
resolved
to
be
version
3001.
That
is
the
publisher.
Is
this
one
for
the
compiled
for
this
major
phosphorus
version,
this
build
number
and
repository,
etc,
etc?
A
Then,
if
we
look
into
the
status
we'll
see
that
the
condition
says
that
it
didn't
fail
so
which
means
that
the
instant
extensions
were
successfully
installed
and
that
it
may
require
a
restart
at
some
point
and
look
for
last
but
not
least,
if
you
look
at
the
statuses
that
we
will
see
that
on
container
cubecon
0,
the
post
gis
version
yes
was
installed
as
well
as
in
the
container
cubicon
one.
So
we
have
postgis
here.
So
yes
looks
like
everything
worked
well.
A
So
if
I
connect
again
to
the
database
and
run
the
same
query
now,
I
see
postgis.
So
it
has
been
indeed
dynamically
loaded
into
the
container
the
pb
and
made
available
to
the
posters.
So
actually,
I
could
just
say,
create
extension
pulse
gis,
oh
my
god,
post
tis
and
now
I
have
this
extension
available
and
I
can,
for
example,
list
the
objects
that
are
brought
by
gis,
which
are
a
lot
right.
A
So
this
is
how
it
looked
like
if
we
actually
look
into
the
web
console-
which
I
also
have
here
handy
you
go
to
the
configuration
tab.
We
will
see
that
it
is
indicated
here
the
list
of
extensions
that
have
been
been
installed
in
the
in
the
cluster.
You
can
also
check
from
the
web
console
which
extensions
are
are
configured
here
and
if,
for
example,
you
may
want
to
add
new
extensions,
you
can
just
edit
the
cluster
go
to
the
extensions
section
and
pick
any
of
the
available
extensions
to
install.
A
Now,
let's
finalize
this
demo
by
looking
into
the
container
image
of
to
see
how
this
looks
like
so,
let's
connect
to
the
controller
cluster
controller,
which
is
the
sidecar
controller
that
is
running
and
I'll
run
a
shell
here
to
see
where
these
extensions
have
been
downloaded.
So
the
pv
is
mounted
on
barley
postgres.
A
So
if
we
go
to
delete
postgres
and
look
here,
we
see
that
this
is
the
data
data
for
the
database.
We
see
the
binaries
for
postgres,
which
have
been
relocated
and
the
extensions.
So,
let's,
let's
look
into
the
extension
directory
and
they
are
yeah
split
by
phosphorus
minor
version
and
then
build
number,
and
here
we
have
the
downloaded
extension.
So
here
we
see
the
post
gis
that
we
requested
with
the
start
package,
which
contains
all
the
code
of
vault
extension,
digital
signature
and
a
file
indicated
whether
it
has
been
installed
or
not.
A
Then
these
extensions
are
unpacked
into
this
directory
and
under
the
user
folder.
So
if
you
get
under
user,
we
can
look
at
the
share
postgres
to
13
major
version
extensions,
and
here
we
see
postgis
with
all
these
control
and
sql
files
that
are
created
when
you
install
the
extension
which
happen
to
be
symbolic
links
to
the
real
directory
where
the
postgres
binaries
are
being
run
and
are
accessible
to
posters.
So,
even
though
it
looks
a
bit
complicated,
the
good
thing
and
the
good
news
is
that
all
this
process
is
absolutely
transparent
to
you.
A
So
it
is
a
mechanism
that
obviously,
as
I
mentioned
before,
allows
loading
dynamically
extensions,
removing
loading
and
loading
upgrading
extensions
they're
not
built
into
the
container
image,
and
this
is
very
good
because
the
container
image
becomes
extension
less.
It
has
two
main
advantages.
One
is
infused
security,
because
the
surface
expo
exposed
surface
attack
surface
is
much
less,
there's
no
dynamic,
this
dynamic
libraries
and
potential
source
code
of
other
extensions
being
built
into
the
container,
but
it
also
decreases
the
container
size.
A
Then
downloading
them
from
the
external
extension
repository
and
bringing
them
to
the
local
file
system
downloading
and
packing
verifying
the
digital
signatures
and
installing
and
making
them
available
to
postgres.
This
is
a
pattern
that
we
believe
is
quite
useful,
potentially
also
for
other
projects,
we're
also
looking
for
feedback.
So
if
you
have
any
idea
or
willingness
to
share
some
some
thoughts
on
this
feel
free
to
do
so,
and
at
the
end,
what
this
allows
you
is
to
run
postgres
on
kubernetes,
which
within
practically
unlimited
number
of
extensions
without
having
to
download
new
images.
A
So
it
allows
you
to
run
as
of
today,
more
than
100
extensions
and
with
a
growing
number
in
the
future.
So
if
you
faced
any
restriction
by
running
on
a
cloud
managed
environment
now
you
can
get
the
same
experience
by
running,
stack,
rest
and
postcards
operators
in
kubernetes,
with
any
extensions
that
you
may
need.
A
So
that's
it
I'm
going
to
open
for
questions.
Thank
you
for
listening.
So
far.
If,
after
this
talk,
you
still
have
some
questions
when
I
when
I
reached
yes,
please
join
the
slack
and
or
discord
stackers
community
services,
so
you
can
join
us
and
continue
the
talk
there
so
open
for
questions,
and
thank
you
very
much
for
listening.