►
From YouTube: Kubernetes Clusters Need Persistent Data
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
Hello
there,
my
name
is
james
spurring
and
welcome
to
this
talk
on
kubernetes
clusters
need
persistent
data,
I'm
the
product,
evangelist
for
storage
os,
and
I
have
a
mixed
background
that
covers
a
variety
of
different
areas,
so
enterprise
storage,
devops
and
software
development.
I
previously
worked
for
nomura
goldman
sachs
and
dell
emc.
A
Okay,
so
jumping
into
this,
how
are
organizations
adopting
and
using
kubernetes
with
our
customers?
We
see
a
mixture
of
different
approaches
and
strategies.
We
have
what
are
the
hybrid
solutions,
ones
that
could
be
deployed
both
on-prem
in
your
own
data
center
or
in
the
cloud.
These
are
clusters
that
you
build
and
manage
yourself
using
the
the
likes
of
cube.
Adm
you've
also
got
those
being
built
using
rancher
where
rancher
deploys
and
manages
the
kubernetes
environment
for
your
target
locations.
A
Next
we
have
the
traditional
cloud-based
offerings
of
kubernetes,
so,
for
example,
your
amazon
eks,
your
azure
aks
and
google
gke,
just
to
name
a
few.
We
also
have
environments
that
may
be
used
by
developers
more
locally
on
their
laptops
or
local
systems.
So
this
could
include
cube
adm,
there's
mini
cube,
there's
kind
and
there's
other
awesome
projects
like
cube
fire.
A
A
If
we
look
at
the
top
two
technologies
running
on
docker
according
to
datadog,
both
nginx
and
redis,
are
first
and
second,
from
a
kubernetes
viewpoint,
the
great
apps
for
running
as
ephemeral
workload
so,
for
example,
nginx.
It's
very
easy
to
run
and
deploy
and
technically
you
could
do
this
without
storage.
A
A
What
we
see
in
the
industry,
whilst
many
organizations
are
adopting
kubernetes
in
production
for
ephemeral
workloads,
they
continue
to
actually
support
legacy
environments
for
non-persistent
workloads,
as
we
know
here,
kubernetes
as
an
operating
system,
provides
many
benefits.
You've
got
your
scheduling
your
resource
utilization
recovery
and,
if
you're
having
to
maintain
a
legacy
environment,
to
support
the
workloads
that
you
can't
run
on
kubernetes,
then
essentially
you're
doubling
the
workload
of
the
team.
A
A
lot
of
the
cases
for
retaining
a
legacy
environment
unfortunately
do
relate
to
persistent
storage
and
they
they
can
actually
be
easily
addressed
with
the
use
of
a
persistent
data
layer
as
well
as
this
applications
may
be
running
in
less
than
optimal
ways.
So
if
we
go
ephemeral
first
then
yes,
we
can
easily
run
both
redis
and
nginx
as
per
the
examples,
but
having
a
better
tool
set
in
your
kubernetes
cluster
gives
you
better
options
as
we
go
through
this.
A
Lastly,
you
don't
need
to
create
scaffolding
to
work
around
the
problems
of
non-persistent
storage
and
really
people
do
this
and
it's
painful
and
ineffective
to
see.
So,
for
example,
this
is
one
I
found
on
the
internet
where
to
actually
work
around
the
problems
of
storage
resistance,
node
affinity
at
the
bottom.
There
is
being
used
to
restrict
that
volume
so
that
it
can
only
ever
run
on
k8
node
1..
Now
it's
easy
to
see
why
this
is
a
bad
idea.
What
happens
if
k8
node
1
fails?
Well,
we
lose
our
data.
A
A
I
encourage
you
as
part
of
your
standard
kubernetes
deployment,
to
look
at
solutions
that
install
a
data
plane
in
the
same
manner:
storage
os,
for
example,
installs
as
an
operator,
and
it's
a
quick
one
line,
installation
script
and
once
installed
this
runs
as
a
daemon
set.
So
why
would
you
want
to
do
this?
Essentially,
an
effective
data
plane
is
like
a
power-up
for
kubernetes.
A
You
can
see
here.
We've
got
mario's
kubernetes,
we
have
the
flower
as
the
data
plane
and
when
these
are
combined
you're,
essentially
running
a
powered
up
cluster.
When
you
increase
the
cluster's
tools
and
functionality,
you
also
increase
productivity.
So
to
understand
this,
let's
look
at
this
a
little
bit
closer.
A
A
storage
class
is
a
standard
kubernetes
component
that
acts
as
the
gateway
for
storage,
os
interaction.
We
use
the
native
kubernetes
csi
driver.
So
when
you're,
using
storage,
os
you're
using
the
declarative
language
as
you'd
expect
to
use
with
kubernetes,
however,
with
storage
classes
acting
as
your
gateway,
you
can
use
this
to
promote
multi-tenancy
and
agility
in
the
environment
that
you're
actually
working
in
so
after
installing
you
may,
for
example,
set
up
different
storage
classes.
A
We
have
a
development
example
here,
so
in
this
case
not
really
concerned
about
the
copies
of
data,
but
wish
for
it
to
be
highly
available.
Therefore,
pod
is
killed
on
one
node
and
it's
starting
on
another.
That
data
is
still
available
for
production.
We
might
have
a
requirement
to
take
this
a
little
bit
further
and
create
two
replicas
taking
this
further
again
top
secret.
You
could
have
your
two
replicas
and,
in
addition
to
this,
you
have
data
encryption
at
rest.
A
Lastly,
you
can
have
something
like
archived
where
we
could
utilize
compression
with
a
single
replica.
By
doing
this,
you've
essentially
created
four
name:
components
that
greatly
simplify
use
your
users
just
make
reference
to
this,
and
ultimately,
they
get
usable
storage
which
most
of
the
time
is
typically
what
they
care
about
the
most.
But
the
specifications
on
the
back
end
meet
the
requirements
that
align
to
your
application
and
organization.
A
Setting
up
a
storage
class
is
very
simple
and
the
following
is
a
template
and
it's
just
some
minor
changes
to
add
parameters.
This
is
the
production
example,
and
you
can
see
here
that
we
have
two
replicas
set
up:
you'd
utilize,
this
in
kubernetes
as
you'd
expect.
So
this
example
here
is
literally
taken
from
the
kubernetes
docs,
and
the
only
change
is
to
the
storage
class
name
and
added
benefit
with
this.
You
can
remove
the
need
to
create
persistent
volumes
yourself.
A
You
just
do
the
claim
for
what
you
need
and
storage
os
would
manage
the
system,
volume
and
its
relationship
to
the
claim,
increasing
productivity
and
kubernetes
usage.
Many
organizations
won't
move
their
applications
to
kubernetes
or
the
cloud
owing
to
compliance
reasons
for
data
at
rest.
So
this
is
especially
prevalent
with
financial
and
health
care
organizations
and
with
storage
os
encrypted.
Data
at
rest
is
a
single
parameter
change
to
the
storage
class.
A
When
we
use
this
all
of
our
data
is
encrypted
and
we
actually
have
a
full
blog
post,
showing
this
functionality
in
detail.
Where
we
actually
look
at
a
low
level
and
prove
that
the
data
is
encrypted
on
the
underlying
node,
so
please
take
a
look
at
that.
If
you're
interested
with
an
effective
data
plane
running
persistent
applications
is
literally
as
you
would
expect
kubernetes
to
do
so,
you
can
see.
In
this
example,
we
have
a
mysql
pod
and
all
we're
doing
is
specifying
the
volume
data
persistence
works
as
you'd
expect
it
to.
A
A
Therefore,
all
nginx
pods
could
access
and
share
the
same
volume
and
an
update
to
the
data
within
that
volume
means
that
all
nginx
containers
are
also
updated,
removing
the
design
of
having
to
destroy
or
recreate
pods
to
initialize
an
update
for
a
website.
Redis
natively
supports
persistent
storage
which
rapidly
improves
recovery
time.
So,
with
a
persistent
data
layer,
you
could
kill
a
container
run
in
redis,
it
could
be
redeployed
elsewhere
and
the
in-memory
keys
are
repopulated
from
the
persistent
storage
and
again
we
have
a
detailed
reference
actually
showing
this
for
those
who
are
actually
interested.
A
Lastly,
other
areas
such
as
githubs
are
easier
to
implement
with
the
correct
framework
so
take.
For
example,
where
we
spoke
about
storage
classes.
I
mentioned
production
development
top
secret
and
archive.
There
is,
however,
nothing
preventing
us
having
a
storage
class
per
application.
So
with
this
approach,
the
application
declaration
stays
exactly
the
same,
but
we
can
vary
the
environment
to
suit
the
needs.
A
So,
let's
say,
for
example,
it's
a
financial
application
that
go
into
compliance
has
to
have
encryption
when
running
in
the
cloud
here,
the
deployment
can
be
managed
with
githubs
and
the
deployment
adapts
accordingly
to
the
environment.
So
in
this
case
we
would
have
the
storage
classes
my
app
one,
both
on-prem
and
in
the
cloud
for
the
on-prem
version.
This
would
actually
be
running
with
two
replicas
for
the
cloud
version.
A
Whilst
it
has
the
same
name,
we
could
have
this
with
two
replicas
and
encryption
and
from
a
declaration
perspective,
the
application
declaration
stays
exactly
the
same,
and
then
you
can
configure
your
ci
cd
execution
jobs
with
it
to
execute
accordingly.
Okay,
so
with
that
covered,
let's
have
a
look
at
a
demo.
Okay.
So
I'm
here
in
my
terminal
and
if
we
take
a
look,
I've
got
a
six
node
kubernetes
cluster.
So
this
is
just
a
standard
kubernetes
cluster
built
with
cube
adm.
A
We
have
the
control
plane
master
as
the
first
node.
Then
we
have
five
worker
nodes
and
if
we
just
take
a
look,
there's
actually
not
that
much
running
here.
This
is
just,
as
I
say,
base
cube,
adm
cluster
and,
as
I
mentioned
earlier,
I've
actually
used
weave
works
there
for
the
network,
and
in
this
I
have
some
examples
that
we're
going
to
actually
look
through.
A
So
the
first
thing
we're
going
to
do
is
we're
going
to
install
xcd,
so
storage
os
uses
an
independent
lcd
for
its
configuration
data,
and
here
I'm
just
using
the
convenient
lcd
operator
when
actually
running
this
in
production.
We
recommend
that
you
configure
a
high
availability
lcd
cluster,
but
for
evaluation
purposes.
A
Okay
and
next
we're
setting
up
the
storage
os
credentials,
so
here
we're
just
using
it's
nothing,
special,
it's
just
a
kubernetes
secret
and
within
this,
we've
actually
got
the
api,
username
api,
password
and
other
settings
that
are
used
by
storage
os.
Now
you'll
notice
there
that
in
this
case,
all
of
the
values
are
the
same,
and
if
we
just
take
a
look
at
this,
all
of
these
are
set
to
the
value
of
storage
os.
A
Next,
we're
setting
up
the
storage
os
cluster
so
for
this
we're
using
the
storage
os
cluster
custom
resource
definition
that
was
actually
set
up,
and
within
this
we
have
a
reference
to
the
version
of
storage
os
that
we're
actually
installing
and
there
in
that
kv
back
end
you
can
see.
The
address
is
the
fcd
endpoint
that
was
actually
set
up
in
the
very
first
set
from
a
resources
perspective.
A
Okay,
so
now
we've
created
our
storage
class,
so
as
per
where
I
actually
showed
there,
as
we
were
going
through
the
presentation.
Much
of
this
is
actually
boilerplate
and
then
within
the
parameters
section,
we
have
the
references
for
the
number
of
replicas
which
we've
got
to
two.
So
that
means
one
primary
copy
of
data
plus
two
additional
replicas,
so
you've
actually
got
free
sources
there
and
encryption
is
set
to
true.
A
If
we
take
a
look
at
the
storage
classes,
we
have
two
listed
there.
We
have
fast,
which
is
the
default
storage
class
set
up
by
storage
os,
so
where
I
was
actually
going
through
those
different
ones,
the
development
production
top
secret
and
archive
this
is
equivalent
to
the
development
storage
class.
That
I
mentioned
then,
underneath
this
we
have
top
secret.
A
Okay,
so
we
create
a
mysql
pvc
and
persistent
volume,
now
you'll
notice
that
I'm
only
actually
specifying
the
persistent
volume
claim
and
for
the
storage
class
name,
I'm
making
reference
to
top
secret
and
again,
this
is
the
example
that
has
just
been
taken
from
the
official
kubernetes
documentation.
There's
nothing
really
to
this.
It
has
a
request
there
at
the
bottom
for
2gig
of
storage.
A
We
have
both
a
pvc
and
the
persistent
volume,
and
this
is
actually
there
it's
connected
to
top
secret
as
desired,
and
the
uuid,
which
is
actually
being
generated
by
storage
os
matches
for
both
the
system,
volume
and
the
system
volume
claims.
So
it's
a
nice
and
convenient
way
of
actually
tallying
those
up.
A
And
this
is
an
encrypted
volume.
So
if
you
actually
take
a
look
at
the
system
volume,
so
here
we're
just
doing
a
cube,
ctl
describe
on
the
system,
volume
and
you'll
notice
in
the
annotation
section
that
we
have
reference
to
a
secret
name
now
this
is
just
a
standard,
kubernetes
secret,
and
if
you
actually
wished
you
could
actually
remove
that
secret
and
that
would
actually
render
access
to
that
data
as
inaccessible.
A
Okay,
so
what
I'm
actually
doing
here,
as
I
mentioned
at
the
start,
this
is
a
six
note
cluster.
The
first
is
the
control
node,
so
that's
not
actually
schedule.
Then
we
have
five
nodes,
five
worker
nodes
and
for
the
example
here.
What
I'm
actually
doing
is
I'm
untainting
one
of
the
nodes
and
I'm
going
to
take
the
others,
so
this
will
actually
force
the
kubernetes
scheduler
to
schedule
any
workload
that
I
actually
do
here
to
kh2
and
I'm
doing
this
implicitly.
So
I
can
show
high
availability
of
data.
A
A
A
A
Okay,
so
there
we're
just
using
the
cube,
ctl
exec
and
we're
passing
in
a
chunk
of
sql
data,
so
we're
creating
a
database
called
shop.
We're
using
that
shop
database
create
a
table
called
fruit
within
that
we
populate
that
with
some
of
my
favorite
fruits
and
right
at
the
end,
there
we're
just
doing
a
query
to
actually
prove
that
that
data
is
there
okay.
So
now
we
delete
that.
A
A
A
So
this
really
highlights
how
simple
it
should
actually
be,
and
this,
whilst
this
is
an
example
for
my
sequel,
you
can
apply
these
same
kind
of
examples
to
legacy
applications
or
other
applications
that
use
operators
and
have
those
storage
class
references
and
what
I'm
going
to
do
now.
We've
spent
a
bit
of
time
here
where
we're
actually
looking
at
the
cli
and
we've
been
working
at
the
kubernetes
level.
A
A
We
use
labels
much
in
the
same
way
that
is
used
in
kubernetes,
so
we
follow
those
standards
there
and
you
can
actually
see
that
we
have
these
labels,
so
encryption
is
actually
true
and
replicas
is
set
to
two
there
and
if
we
actually
click
into
this,
you
can
see
that
again,
this
is
attached
there.
This
is
the
primary
node
for
this.
Then
we
have
our
two
replicas,
so
the
first
replica
is
running
on
k83
and
the
second
replica
is
running
on
k86.
A
As
you
can
see
here,
it
really
makes
life
a
lot
easier
in
different
respects
and
it
will
really
help
your
organization
to
actually
grow
its
kubernetes
usage.
So
some
further
reading
here
we
have
the
performance,
benchmarking
cloud
native
storage
solutions,
so
this
was
a
benchmarking
test
of
ourselves
and
other
persistent
storage
solutions.
A
That's
a
very
good
read.
We
have
the
sibos
lightning
fast,
managed,
kubernetes
development
and
deployment
resource.
So
sivo
are
another
provider
who
actually
offer
a
cloud
managed
kubernetes
solution,
it's
really
really
cool
and
they
actually
use
storage
os
on
mass,
so
some
great
resources
on
on
the
internet,
where
they
actually
show
that
running
on
bmet
or
on
huge
clusters,
definitely
worth
a
look,
some
other
areas
that
we've
actually
got
there.
A
We've
got
the
platform
architecture
overview,
there's
the
documentation
and
if
you
actually
want
to
reach
out
to
us
and
chat
to
us
a
bit
further,
we
have
a
dedicated
slack
group.
So
by
all
means
please
come
and
speak
to
myself
come
and
speak
to
our
other
engineers
and
we'd
be
happy
to
get.
You
started
with
persistent
storage
on.