►
Description
Third Party Dynamic Volume Provisioners - Jan Šafránek, Red Hat
"Kubernetes 1.4 introduces an API for dynamic provisioning of persistent volumes by third party provisioners. Any storage vendor, big or small, open or proprietary, can write a standalone plugin that can provision storage for Kubernetes. The plugin can live outside of Kubernetes source tree, running on any platform and be written in any language, as long as it can talk to Kubernetes using REST protocol. In this tutorial, we will write a simple provisioner using a prepared skeleton written in Go language and we will handle all possible corner cases that the plugin needs to deal with in a highly dynamic system such as Kubernetes.
Some user experience with Kubernetes is strongly desired. Golang knowledge is not required, code samples should be universally understandable to anyone with a programming experience."
About Jan Šafránek
Jan is principal software engineer in Red Hat Inc., currently working on storage aspects of Kubernetes. He’s focused on development and maintenance of open source software with focus on system management.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
A
Chef
Eric
I
work
for
a
head,
and
this
presentation
was
supposed
to
be
a
tutorial
about
how
to
write
external
dynamic
provision
when
I
submit
a
talk
next
time
in
the
tutorial,
but
then
I
got
a
40
minute
slot,
so
what
kind
of
tutorial
can
I
do
in
40
minutes?
So
this
will
be
a
presentation
and
you
can
do
the
coding
as
a
homework
so
before
I
start
about
external
dynamic
provisioning
I
have
a
brief
introduction
about
storage
in
kubernetes
and
how
it
works,
because
you
need
to
know
the
core
concept
before
writing
you
code.
A
So
the
all
the
containers
in
kubernetes
are
state
less.
They
don't
have
state,
they
are
started.
Even
you
start.
They
have
lost
their
data
and
cooperatives.
Persistent
volume
framework
is
there
to
resist
mistake.
So
if
a
pot
wants
to
have
some
storage
to
store
some
data,
it
has
to
ask
for
a
person
value.
So
I
start
with
your
with
what
I
call
static.
Pravesh,
storage,
static,
storage,
provisioning
which
is
available
in
comparative
since
forever
and
except
it
expects
that
somebody
in
your
organization
creates
some
kind
of
volumes.
A
It
can
be
NFS,
it
can
be
ice
castle
and
it
can
be
amazing,
EBS
whatever
and
it
creates
them
before
running
kubernetes.
So
let's
say:
I
have
free
some
storage
volumes
somewhere
in
winter
infrastructure
in
my
network
and
I
need
to
know
how
kubernetes
that
there
are
free
volumes
available.
So
in
COBOL
it
is
I,
create
a
persistent
volume
object
each
for
each
storage
volume.
A
Is
this
persistent
volume
object,
it's
just
a
pointer
to
remote
storage
asset,
just
ask
abilities,
hey
Lisa,
and
as
for
him
somewhere
in
my
instruct
infrastructure,
it
has
all
the
information
to
mount
the
volume,
so
it
has
NFS
address
and
the
share
name.
If
you
are
used
using
ice,
cozied
has
IP
address
of
the
portal
and
one
number
or
if
you
are
a
Muslim,
then
it
has
name
of
the
Amazon
EBS.
A
So
now,
if
a
user
wants
to
store
data
somewhere
on
on
one
of
his
volumes,
the
user
creates
another
kubernetes
object,
which
is
persistent
volume
claim
it
tells
kubernetes,
hey
I
want
I,
don't
know
10
gigabytes
of
data.
Please
find
a
volume
for
me
and
what
kubernetes
does
it
looks
at
existing
volumes?
Push
the
volumes
find
some
and
combines
the
claim
and
the
volume
together
so
now.
A
The
cubelet
on
the
node
mounts
it
goes
through
all
these
objects
to
burn
these
objects
and
it
mounts
the
storage
asset.
So
that's
how
storage
works
in
Coburg
notice?
There
is
this
persistent
volume
is
a
pointer
to
some
storage
asset
somewhere
in
your
network.
There
is
a
claim
it.
It
is
a
request
for
storage
and
there's
a
pot
who
uses
the
claim
that
points
to
a
volume
that
points
to
a
storage.
So
let's
what
icon
static
provisioning?
A
We
have
to
provision
this
volume
ahead,
so
that
doesn't
scale
well,
because
you
don't
know
how
many
volumes
you
you
will
need.
You
know
what
size
is
to
create.
So
that's
not
good.
So
in
kubernetes
yeah
and
you
have
big
organization
or
big
faster
than
whisper.
Certain
volumes
are
usually
admins
admin
domain.
There
is
somebody
who
cares
about
this,
and
user
cares
about
disclaims
and
pots.
So
in
when
the
two
we
introduced
dynamic
provisioning.
So
there
is
no
static
volume,
provisioned
anywhere
ahead.
A
User
just
creates
the
claim.
I
want
10
gigabyte
volume
just
do
something
and
give
me
10
gigabyte
volume
and
what
kubernetes
does
it
has
several
very
plugins?
We
have
one
for
Amazon,
GCE
and
cinder
in
kubernetes
1.2
and
it
magically
provisions.
The
volume
in
the
cloud
using
API
calls
to
Amazon,
GC
or
cinder,
and
it
creates
the
constant.
A
B
A
Is
a
brief
history?
How
the
claim
looks
like
the
object
that
the
user
creates
and
the
user
creates
the
claim
and
he
wants
user
space
I
want
something
that's
right
about
and
why
I
want
some
I
won't
do
bigger
bites
of
it
and
in
kubernetes
wanted
to
we
added
a
veggie
connotation
notice
this
alpha
here
the
enable
dynamic
provisioning
if
it
is
enabled
on.
B
A
In
wonder,
this
was
suboptimal
because
you
can't
say
what
do
you
want?
You
do
want
something
fast,
something
cheap,
something
free.
You
can
say
anything
and
we
implemented
just
the
basics
on
the
cloud.
So
on
Amazon
you
get
SSD
volume
on
GCE,
you
get
Hydra
life,
I,
think
and
on
still
on
OpenStack
syndrome.
Yet
whatever
is
default
on
cinder
and
users
were
not
happy
about
that.
A
So
in
1.4
we
introduced
beta,
annotation
and
storage
classes.
Now
the
user
can
say
what
class
of
storage
he
wants.
I
want
something
fast
and
we
introduced
storage
class
object
is
a
new
object
in
component
is
not
for
where
we
say
that
fast
when
the
administrator
says
that
fast
means
I
want
it
on
AWS
I
want
I,
Oh
PS
provision,
volume
and
I
want
50
IO
operations
per
second
per
gigabyte.
So
if
the
user
wants
four
gigabytes,
we
multiply
it
and
we
provision
a
volume
that
has
200
cups
per
seconds
guaranteed
in
Amazon.
A
A
If
I
could
external
dynamic
provisioning
here
is
the
reference
to
PR,
and
there
is
this
document:
where
is
it
which
defines
sort
of
protocol
between
kubernetes
and
the
dynamic
provisioners?
So
you
can
turn
you
can
write
a
piece
of
code
that
lives
outside
of
kubernetes.
It
can
be,
it
can
be
a
demon,
it
can
be
stretched
appliance.
It
can
be
put
inside
the
kubernetes.
Why
not?
It
doesn't
really
matter,
and
here
are
some
rules.
If
you
obey
those,
then
kubernetes
will
accept
volumes
from
you
and.
A
A
So
what
this
proposal
does
this
PR
earth
and
it
has
been
merged
in
205-
is
that
introduced
new
annotation?
So
if
kubernetes
sees
a
new
person,
volume
claim
that
user
wants
some?
Do
you
mean
it
looks
at
the
storage
class
that
the
user
wants
and
if
kubernetes
can't
provision
the
class?
That
means
it's.
A
One
of
these
sorting,
plugins
I,
showed
you
earlier
its
proficiency
and
if
it
doesn't
know
how
to
provision
some
crazy
storage,
then
it
just
puts
this
annotation
on
the
claim
and
it's
up
to
the
external
provision
her
to
watch
kubernetes
api
server
for
these
crimes.
With
this
strange
annotation
and
provisioned
volume,
the
proto
I
said
this
document
defines
the
protocol
is
not
a
network
protocol.
It's
not
like
send
this
request.
You
get
this
response.
Now.
It's
not
Network
protocol.
A
A
A
A
A
A
This
will
be
handled
internally
in
warranties
that
if
a
user
wants
to
submit
a
claim
which
referenced,
which
plus
B
then
which
reference
this
provisional
I
don't
know
a
and
C
I
CMC
guy
from
then
this
emc
provisional
must
detect
that.
For
me,
I
am
the
one
who
is
responsible
for
this
claim
as
provisionally
how
the
provisional
detects
that
it's
written
in
the
proposal
in
the
documentation
they
just
watch
the
further
annotation,
then
the
provisional,
creates
the
storage
asset
creates
the
pv.
The
person
volume
must
point
to
the
there
is.
A
A
It's
pretty
easy
to
write
a
new
provisional
and
we
made
it
even
easier.
We
will
see
later
what
every
provisional
must
do
is
also.
It
must
do
the
deleting
of
modules,
because
when
the
volume
is
not
longer
necessarily
somebody
should
delete
it,
and
only
the
thing
that
created
it
is
no
known,
is
known
to
delete
it
he's
responsible
for
the
deletion.
So
if
you
know
how
communities
works,
when
the
user
decides
that
he
doesn't
want
the
data
anymore,
it
can
be
deleted.
A
Then
the
use
of
TV's,
the
claim
it
says,
I'm
done
with
this
TV
I,
don't
need
a
beta.
Please
do
whatever
you
want
with
that.
Soap
abilities
detects
that
it
marks
the
PBS
released,
and
here
comes
the
external
provisional
coordinator.
The
deleter
must
watch
not
only
claims
but
also
volumes.
It
detects
a
there
is
a
volume
I
need
to
delete,
and
the
external
datura
do
is
the
staunchest
volume
and
that's
it
simple.
A
So
that's
it!
Basically,
if
you
understood
this,
then
you
know
how
to
write
external
permission
and
the
leader
and
to
make
things
even
easier.
We
inherited
it
some
proof
of
concept
which
turned
out
into
NFS
provisional,
so
it's
written
by
Matthew
Vaughn
he's
sitting
over
there
and
with
this
fing
does.
A
A
Provisioning
of
NFS
volumes,
so
if
you
are
running
a
small
development,
cost
faster
on
your
notebook
or
in
I,
don't
know
few
machines,
we
have
in
your
lab
and
you
want
to
test
pad,
set
and
dynamic
provisioning,
and
you
don't
want
to
reach
out
to
Google
or
Amazon,
or
it's
too
complicated
for
you.
You
want
something
you
can
develop
on,
then
you
can
use
this,
which
can
dynamically
provisioned
volumes.
You
just
create
a
claim
and
you
get
NFS
share
somewhere,
and
so
it
also
serves,
as
example,
how
to
write
this
external
provision
herself.
A
There
is
documentation
how
to
run
it,
how
to
create
a
claim
how
to
check
it
out
to
debug
it,
and
it's
also
prepare
to
be
extended.
We
are
not
there
yet.
We
are
thinking
about
splitting
it
into
two
parts:
one
library
for
generic
provisioners
and
the
NFS
part
in
a
different
bit
have
proposed.
So
you
just
use
the
library
plug
the
stuff
that
talks
to
your
stretch,
infrastructure
and
all
this
watching
and
creating
and
deleting
component
of
kubernetes
objects.
A
A
Something
it
just
all
the
watching,
all
the
rest
conditions
and
in
Wendy
finds
out
that
there
is,
you
should
provision
something
that
you
implement.
Just
this
one
function
that
get
some
options
like
capacity
access
modes
whatever,
and
you
return
a
personal
assistant
volume
object
and
again
it
cares
about
this
library.
Will
care
about
is
inserting
into
cooperatives
and
all
the
races
involved,
and
when
there
is
something
to
delete,
then
just
this
function
is
called
and
you
implement
all
this
only
to
this
provision
and
delete
functions,
and
that's
it
so
enough
about
this.
A
It
works
with
kubernetes,
1.4
1.5
is
recommended.
The
reason
is
that
this
magical
notation
was
introduced
in
module
five
and
it
helped
the
provision
error.
It
simplifies
the
provisional
quite
a
lot
and
it
fixes
some
erase
condition
well,
not
fixes
it
prevents
some
various
conditions.
I
will
talk
about
resolutions
later,
I,
don't
worry
about
them,
but
really
1.5
is
recommended.
The.
C
A
Part
is
nice
for
production
use
and
that's
not
because
it's
crappy
this
code,
the
code
is
creepy.
The
code
is
quite
good,
but
NFS
the
the
data
is
stored.
Only
one
one
note.
So,
if
the
note
goes
down,
then
your
data
is
lost
and
you
are
screwed.
So
this
it
is
good
for
development,
for
testing
for
this
kind
of
use,
but
I
wouldn't
put
it
in
the
production.
If
you
want
something
reliable
use,
lustre
self
or
whatever
that
can
survive,
renewed,
being
destroyed
and
finally
time
for
the
Hmong
so
cross.
A
A
The
claim
and
the
claim
is
pending,
nothing
was
provisioned.
That's
why?
Because
the
provisional
is
not
running
yet
and
kubernetes
just
gives
some
hints
why
it
was
not
provisions
because
kubernetes
doesn't
know
anything
about
Matthew,
NFS
and
like
says
I.
Don't
I
won't
mess
with
this
crime
either
do
something
manually
or
run
air
pollution,
so
I
will
run
the
provisioner.
Oh
no
I
have
just
showed
up
people
get
PVC
llamó,
all
the
kubernetes.
A
Of
office
running
cool,
yet
it's
a
pot.
It's
a
pot
that
runs
this
office
provisioner
from
the
dita
people.
It
serves
an
affair,
so
it
has
some
ports
and
it
tells
the
code
inside
that
the
provisional
is
responsible
for
Matt
USB
flash
NFS.
So
this
parameter
for
the
daemon
running
inside
the
container
that
please
provision
everything
works,
formative,
slash,
NFS
and
that's
nothing.
There
is
no
secret
sauce,
it's
just
a
branch
binary.
C
A
A
A
A
A
A
A
A
That
means
that
if
the
user
wants
something,
that's
read-only
readable,
which
is
fine
kind
of
useless
or
if
it
should
be
writable
or
it
could
be
mounted
to
many,
but
it
also
contains
the
capacity
which
says
that
how
much
data
is
going,
the
user
to
store
there,
so
10
gigabytes,
20
right,
handle
events
and
think
that's
not
so
obvious.
This
selector
in
the
claim
the
user
can
say
that
he
wants
the
president
Illume
that
has
certain
labels.
For
example,
zone
equals
e
east
1
or
color
equals
blue.
So
the
provisional.
A
That
something
that
has
these
labels.
That
means
that
the
provision
needs
to
parse
the
selector,
and
if
user
wants
something,
the
test
label
equals
east
one,
then
the
provisional
needs
to
know
what
zone
means
that
it
refers
to
available
Diesel's,
probably
the
and
the
east.
One
is
name
of
the
zone
and
it
has
to
reach
to
the
storage
infrastructure,
to
provision
the
volume
in
the
right
zone.
So
that's
quite
important.
A
You
have
to
parse
the
selector
and
if
the
user
wants
something
like
shape,
equals
rectangle
and
your
provision
doesn't
know
what
shape
means
and
how
to
provision
something
was
rectangular.
Then
it's
perfectly
okay
to
say
no
I
won't
provision
anything
because
I
don't
understand
this
selector,
that's
perfectly
okay,
but
bear
in
mind.
If
you
are
provisioning,
something
then
the
selector
must.
A
A
A
We'll
fix
it
for
you,
you
don't
need
to
care
much
about
it
in
their
external
provisions.
For
example,
what
can
happen
that
user
submits
a
claim?
You
retake
it?
You
start
provisioning,
it
takes
some
time
and
the
user
changes
its
mind
and
do
is
the
claim.
So
we
are
probably
something
that
this
is
not
needed
anymore
and
ok.
So
you
follow
the
protocol,
you
create
a
stretcher
set,
you
create
the
client,
you
create
instant
volume
submitted
through
kubernetes
and
kubernetes
and
they
detect
it.
A
Ok,
I
have
a
personal
volume
and
nobody
is
interested
in
it
and
we
are
real
market
for
deletion.
So
you
follow
the
protocol,
you
see
a
volume,
tradition
related
and
that's
it
so
from
yours.
From
the
extant
provisional
side,
there
is
no
race,
the
race
is
inside
kubernetes
and
kubernetes,
and
was
it
and
if
it
doesn't
place
for
bugs,
because
I
did
it
wrong.
A
A
The
second
one
is
marked
for
division
again,
so
we
follow
the
protocol,
you
create
everything
as
you
should
and
something
is
then
marked
for
deletion
and
you
delete
it
no
race
in
the
external
provisioning
races
in
the
kubernetes,
and
it
should
work
if
not
again,
filled
back
and
also
when
user
submits
a
claim.
We
look
at
the
existing
Peavey's
the
static
one
I
showed
in
the
first
slide.
So
if
there
is
no
static
volume
available,
we
do
design
dynamic
provisioning
again.
A
The
external
provisioning
provisioner
see
that
start
slowly,
preventing
something
and
at
the
same
time,
another
user
creates
a
static
volume
in
kubernetes
sees
hey.
That
is
the
volume
let's
bind
to
it
and
in
the
end,
you,
your
provisional,
creates
another
volume
and
we
have
two
volumes
bound
to
one
crime
and
again
kubernetes
handles
that.
So
this
message
of
this
slide
is
don't
worry
about.
Race
is
too
much
info
box.
If
something
happens,.
D
A
There
could
be
in
the
future
other
options
how
to
do
dynamic,
provisioning
out
of
the
kubernetes
code
on
Monday,
we
had
a
storage
tech
meeting
when
we
discussed
how
to
extend
flex
volumes.
Now
the
flex
volume
can
just
mount
stuff,
and
we
are
thinking
about
extending
it
to
also
provision
stuff.
So
maybe
in
1.6,
maybe
later
we
will
have,
you
could
use
flex
volumes,
they
will
use
different
protocol.
A
So
stay
alert,
but
the
dynamic
provision
I
described
here
will
keep
working.
So
if
you
need
something
now
you
can
use
the
provisioning
I
described.
If
you
can
wait
and
flex
looks
like
interesting
to
you,
then
you
can
write
and
we
have
proposals
for
leap
for
using
clip
storages
volumes.
Web
storage
is
rice
and
libraries
whole
framework.
How
to
use
various
storage
technologies
and
lipst
rich,
abstract
them
and
serves
unified
API
the
kubernetes
to
any
other
thing
that
wants
abstract
storage.
A
B
A
B
A
A
Nfs
share:
if
I
don't
know,
pictures
or
videos
you
need
to
process
in
Cooper
naty's,
then
you
need
to
weigh
how
to
reach
it.
So
that's
typical
brownfield.
You
have
some
application
running
in
your
company
and
you
want
to
use
kubernetes
to
use
the
old
data
you
have
since
decades
then
static
volumes,
that's
it
that
I
would
suggest
everybody
use
dynamic
provision
everywhere.
You
can
because
that's
much
better,
you
don't
waste
resources
with
recreating
volumes
and
stuff.
So
that's
it!
Another
question.
D
A
Question
is
what
happens
if
claim
increases
sighs
and
the
answer
is
claim,
do
not
create
a
claims
are
not
resizable
right
now.
If
you
want
to
resize
it,
you
get
error
that
the
object
is
immutable
in
the
future.
We
may
extend
it
well,
we
will
extend
it.
We
have
couple
of
requests
to
extended
that
I,
don't
know
when
the
future
is.
If
this
one
does
61.7
I,
don't
know
so
right
now,
the
volumes
and
claims
are
fixed
size.
A
How
is
it
with
snapshots
and
back
up
again?
This
is
about
dynamic
provisioning
and
we
don't
recoupment
is
currently
we
don't
have
any
means
to
do
snapshots,
but
again
we
get
requests,
we
know
about
them
and
we
will
extend
it.
There
will
be
snapshots,
but
I
don't
know
again,
I
don't
know
when,
because
these
snapshots
are
pretty
complicated,
because
sometimes
you
know
always,
you
need
the
application
support,
so
the
application
is
to
flash
the
caches
and
the
memory.
So
I
don't
know
how
how
this
is
going
to
happen
and
one
sorry
this
you.