►
Description
Modern software storage systems are inherently complex. They are composed of numerous distributed components, require careful balancing of resources, and have stringent performance requirements. If you're running your applications in a public cloud you're typically shielded from this complexity and can utilize managed storage services like EBS, S3 and EFS. If you're running on-premise, however, your choices are quite limited and typically result in using traditional big-iron storage systems. In this talk, JJ will walkthrough a deploying and running a production-ready storage cluster using Kubernetes. Storage nodes can run as pods and enumerate the available storage devices within the cluster. JJ will show how some of the features of Kubernetes including controllers/operators, third-party resources, resource management, and rolling upgrades can lead to more powerful and resilient storage clusters.
A
In
the
routine
surprise,
company
behind
limited,
which
not
enough
he's
also
not
there
to
fun
for
anyone
who
have
known
already,
both
the
women
did
us
everyone
here
and
all
the
way
from
Cisco
to
be
here
with
us.
Okay,
you'll
be
talking
about
so
completing
design.
Now
between
work
and
fighting
is
not
a
project.
B
Cool
thank
you
for
having
me
it's
great
to
be
here
in
the
source,
the
offices.
This
is
very
San
Francisco
with
like
the
pizza
area
and
all
the
cool
facades
and
everything.
So
congratulations
on
creating
such
a
cool
space,
so
yeah,
it's
great
to
be
here
in
Madrid
I,
haven't
been
here
in
five
or
six
years.
B
Think
it's
a
good
follow-up
talk
from
from
the
really
great
one
that
we
just
heard
about
all
the
different
storage,
persistent
options
and
all
the
sort
of
paths
of
actually
doing
a
write
in
Linux.
So
just
a
little
bit
about
me.
First
spent
about
13
years
working
for
a
bunch
of
different
startups
over
the
last
five
or
six
years
where
it
worked
for
a
few
companies
in
the
middleware
SOA
kind
of
space
cloud,
and
it
works
at
mesosphere
as
a
consultant
in
2014.
And
then
when
kubernetes
came
out,
it
got
very
excited
about
that
project.
B
B
So
this
is
the
cloud
native
computing
foundation
where
we're
kubernetes
was
donated
to
by
Google
and
started
to
con
we've
donated
coupons
to
the
Linux
Foundation
and
the
in
the
CNC
F,
which
which
actually
manages
and
and
maintains
and
handles
that
event
now
and
with
they
have
professional
staff
and
like
really
awesome
people
that
make
that
that
doesn't
happen.
So
congratulations
to
them
for
making
it
to
grow
and
then
be
a
huge
event.
Next
week
in
Berlin
Germany,
we
have
coop
con
Europe,
that's
happening
this
year.
B
So
pretty
excited
about
that
I
left
a
prenda
about
a
month
ago,
which
is
the
company
that
acquired
his
mattock
and
their
platform
as
a
service
company
they're
doing
a
lot
of
really
cool
work.
I
got
the
itch
to
do
another
startup
and
so
decided
to
take
a
little
bit
of
time
off
and
in
that
time
had
sorted
to
explore
lots
of
different
open-source
projects.
One
of
them
that
stood
out
to
me
that
was
very
exciting.
B
An
interesting
kind
of
solving
one
of
the
big
problems
in
distributed
storage
for
these
sort
of
modern
container
based
applications
that
people
are
building.
It's
a
project
called
rook
and
so
we'll
talk
a
little
about
rook
I'm,
very
active
on
Twitter
I
love
talking
with
people
of
the
world
on
Twitter
and
I.
Think
Twitter's
amazing
just
for
like
staying
in
touch
with
things
and
changes
in
the
world.
B
So
if
you
want
to
follow
me
and
async
I/o
on
Twitter,
so
just
to
kind
of
talk
about
the
outline
we'll
be
introducing
rook
kind
of
explaining
what
the
project
is
all
about,
why
it
exists
and
why
you
might
use
it
why
it
might
be
interesting
to
you.
I'll
talk
about
the
operator
scenarios
and
patterns
that
exist
in
rook,
so
you
can
actually
use
something
that
makes
rook
really
easy
to
deploy
and
life
cycle
on
top
of
kubernetes,
specifically
I
know
ivan
my
friend
talks
about
operators
earlier
in
the
day.
Unfortunately,
mr.
B
talk,
I'm
sure
I'll
watch
the
recording
later
and
then
we'll
talk
a
little
about
other
operators
that
have
been
built.
So
core
OS
is
a
company
doing
awesome
work
and
this
in
the
space
they
a
kind
of
came
up
with
this
idea
of
operators,
which
is
this
sort
of
automation,
methodology
around
sort
of
the
life
cycle
of
different
distributed
services
on
kubernetes
that
are
that
are
typically
stateful
services.
They
can
also
be
really
useful
for
for
stateless
sort
of
services
feel
free
to
interrupt
me
at
any
time.
B
During
the
talk
with
questions
happy
to
take
questions
at
the
end
as
well
so
from
from
myself,
I
actually
have
a
few
questions
so
who
has
heard
of
or
use
uses
kubernetes
today
in
the
audience,
so
almost
everybody
has
either
heard
of
it
or
user
who's
actually
like
using
the
system
today
or
is
pretty
heavily
investing
in
it
in
your
in
your
company
or
in
your
startup
Wow
cool
about
five
or
six
people.
I
know
I
mean
you're
using
it
production
anyone
else
using
kubernetes
in
production.
B
The
audience
just
raise
your
hand,
you'll,
look
really
cool
if
you
just
like
to
say
is
production,
so
it's
still
four
or
five
people.
That's
awesome!
What
about
rook?
Has
anyone
prior
to
this
talk
or
this
sort
of
event?
Has
anyone
heard
of
rook
before
a
couple?
People?
Okay,
awesome!
So
that's
my
census.
So
what
is
rook
rook
is
a
few
things.
B
You
know
X,
it's
an
exabyte
scale,
storage
system,
which
is
pretty
awesome
and
rook
is
really
built
to
solve
a
couple
of
key
problems.
One
stuff
itself
hasn't
really
been
built
over
the
last.
You
know,
15
years
for
the
types
of
distributed
constantly
failing
ephemeral,
multi-core
architectures
that
exist
today
in
the
sort
of
modern
cloud
native
world
where
people
are
actually
distributing
services
and
data
across
multiple
cloud
providers
across
multiple
geographic
regions
across
large
compute
clusters
and
even
though
stuff
was
used
in
that
and
that
kind
of
context.
B
And
in
the
beginning,
since
it's
distributed
a
lot
of
the
the
challenges
of
maintaining
and
operating
and
upgrading
and
patching
and
doing
various
sort
of
care
and
feeding
activities.
Incest
itself
are
really
hard,
and
so
people
have
kind
of
shied
away
from
from
using
sets
for
a
lot
of
the
modern
types
of
purposes.
And
so
what
rook
is
all
about
is
basically
kind
of
raising
the
abstraction
away
from
the
native
staff
interfaces
to
to
provide
much
more
operator
and
application,
developer,
friendly,
sort
of
primitives
and
well
and
we'll
talk
a
little
bit
about.
B
Those
SF
obviously
also
has
a
lot
of
different
moving
parts
and
components
and
agents
and
daemons
that
that
actually
provide
this
sort
of
unified
object,
block
and
file
storage
capability,
and
what
rook
also
does
it
sort
of
simplifies
that
into
a
single
single
binary?
So
you
can
actually
download
rook
and
deploy
that
as
a
container
based
service
in
your
cluster
on
kubernetes.
Today
it
works,
and
you
can
also
run
it
standalone
in
the
future.
You'll
run
it
on
swarm
and
maze,
o
sand,
Cloud,
Foundry
and
other
other
systems.
B
So
it's
designed
to
sort
of
be
universally
deployable
across
all
these
different
frameworks,
but
it's
very
simple
to
deploy
very
simple
to
maintain,
which
is
kind
of
one
of
the
big
advantages
over
just
using
stuff
itself.
Natively
for
this
sort
of
thing,
one
of
the
other
really
huge
benefits.
Is
it's
not
rewriting
the
the
data
path?
So
you
know
what
that
means.
There's
you
know
lots
of
these
newer
systems
that
are
that
are
coming
out,
whether
it's
sort
of
from
a
converged
infrastructure,
vendors.
B
You
know,
selling
acidic
appliance
with
their
software
layer
and
their
control,
plane,
they're,
actually
rewriting
the
way
you
know
bits
actually
get
written
to
disk
in
terms
of
the
the
low-level
software-
and
you
know
things
that
actually
interface
with
with
the
Linux
kernel
drivers
and
the
whole
data
path
that
we
heard
about
earlier
and
to
actually
rewrite
a
data.
Data
path
from
scratch
in
the
beginning
is
incredibly
difficult
and
it
takes
many
many
years
of
hardening
and
many
years
of
production,
sort
of
performance
and
experience
to
make
something
really
stable.
B
And
so
that's
why
I
wrote
gets
sort
of
kind
of
the
best
of
both
worlds.
You
have
something
that
it's
very
proven,
so
you
don't
need
to
really
worry
about
the
data
path,
production
readiness,
but
you
also
have
something
that
works
really
well
in
the
sort
of
modern
application,
architectures
that
are
that
are
kind
of
very
popular
today.
B
B
It
gives
you
this
ability
to
run
local
storage
on
the
instances
or
compute
nodes
that
you're
actually
running
your
applications
on.
It's
actually
make
make
use
of
those
resources
that
sometimes
are
often
underutilized.
So,
for
example,
if
you're
running
on
ec2
instances
that
actually
have
a
fair
bit
of
provision,
local
storage
attached
to
those
to
those
virtual
machines
oftentimes.
If
you're
you
know
starting
to
spin
up
UDS
volumes
or
EFS
or
s3
buckets
or
what-have-you,
you
actually
waste
a
lot
of
the
local
storage
on
those
instances.
B
You
know
without
even
really
knowing
it,
and
so
that's
that's
something
that
a
lot
of
people
don't
think
about
one
of
them.
One
of
the
more
immediate
benefits
on
the
hyper
scale.
Hyper-Converged
thing
is
rook
is
really
great
for
people
just
running
bare
metal
on
physical
data,
centers
I
know
you
know
lots
of
the
hyper
scale.
B
Internet
companies
run
physical,
private
data
centers
still
today,
even
though
a
lot
of
people
are
moving
to
cloud
they're
still,
the
vast
majority
of
compute
is
still
running
in
private
data
centers
that
are
owned
and
managed
by
companies,
and
so
rokh
actually
provides
a
very
nice
alternative
to
the
things
that
you
would
expect
in
a
cloud
provider
like
s3
and
EFS
and
an
EDS
in
a
private
data
center.
So
you
don't
really
need
to
think
too
much
about
you
know.
B
Different
alternatives
are
API
compliant
or
have
a
lot
of
similar
functionality
to
go
and
try
and
emulate
services
behind
your
behind
your
physical
corporate
firewall.
So
that's
kind
of
interesting
rook
is,
is
also
taking
care
of
a
lot
of
the
management
facilities
behind
the
scenes
and
we'll
talk
a
little
bit
about
how
that
happens.
One
of
the
really
cool
things
here
is
you
know
as
an
operator,
you
don't
really
need
to
think
about
managing
and
maintaining
all
the
different
distributed
components
inside
of
Ceph.
B
Obviously,
it's
distributed
its
replicated.
It
gives
you
this
elasticity,
you
simply
add
more
nodes
to
get
more
storage
as
simple
as
that
and
it'll
run
directly
either
on
commodity
machines
or
you
can.
You
know,
purchase
and
build
your
own.
You
know
high-performance
specialized
hardware
and
will
run
fine
on
that
as
well.
The
purpose
is
for
you
to
basically
take
advantage
of
local
storage
running
on
commodity
x86
machines,
which
can
either
be
from
you
know,
white
box,
vendors
or
just
you
know,
from
Delhi
and
see
what
have
you.
B
So
these
are
some
of
the
bigger
sort
of
key
points
behind
rook
the
product
is
very
young.
It's
in
terms
of
the
control
plane,
which
is
written
and
go
not
yet
production
ready,
but
I
think
block
is
the
first
interface.
That's
going
to
be
production
ready
in
the
next
couple
of
months,
as
I
mentioned,
the
data
path
which
is
based
on
SEF
is
fully
production
ready,
but
the
project
is
starting
to
get
a
lot
of
a
lot
of
exciting
popularity
and
the
vanity
metrics
on
github
are
showing
showing
that
for
sure.
B
Since
there's
a
very
tight
feedback,
loop
and
and
people
actually
make
sure
that
they're
using
sefa
natively,
they
actually
benefit
from
a
lot
of
things
that
are
found
in
the
real
community
and
the
rook
users,
which
are
starting
to
grow.
It's
a
pretty
tight
feedback
loop
between
between
people
like
sage,
who's,
the
creator
of
Ceph
and
many
folks
at
Red,
Hat
and
other
people
in
the
community
and
the
rook
development
team,
and
the
rook
engineering
team
is
primarily
based
in
Seattle.
B
But
there's
lots
of
lots
of
different
attributer
'z
that
are
starting
to
generate
PRS
and
submit
issues
and
actually
contribute
to
the
project
which
is
exciting
to
see
so
as
I
mentioned,
kubernetes
is
sort
of
the
first
kind
of
focus
area
for
making
rook
run
very
well
and
reliably
in
a
sort
of
modern
distributed
container,
orchestration,
runtime,
environment
and
I.
Think
that's
one
of
the
huge
gaps
in
the
Korean
communities.
B
It's
one
of
the
things
that
got
me
personally
excited
about
rook,
but
this
this
is
going
to
continue
in
rook
will
be
supported
across
a
variety
of
different
systems,
including
NATO's,
core
OS
and
docker,
one
of
the
one
of
the
really
cool
things
that's
happening
in
the
storage
area
right
now,
which
is
sort
of
the
boring
kind
of
reliable,
saying
that
everyone
expects
to
work.
But
it's
actually
really
hard
and
doesn't
actually
work
until
very
much
later
on.
B
So
talking
a
little
bit
about
the
the
kubernetes
support,
which
is
the
sort
of
as
I
mentioned.
The
sort
of
initial
focus
area
for
four
rook
I
can
talk
a
lot
about
kubernetes,
but
I'll.
Just
talk
briefly
and
then
we'll
talk
about
operators
and
sort
of
how
the
operator
design
was
was
conceived
in
how
it
works
for
for
rook.
So,
as
most
of
you
mentioned,
arrays
Richard
and
you're
familiar
you've
where
you've
been
using
kubernetes,
we'll
talk
too
much
about
it.
B
But
kubernetes
was
basically
designed
for
this
very
ephemeral,
constantly
failing
distributed
world
of
compute
machines
that
obviously
support
distributed,
applications
which
are
becoming
more
and
more
more
more
popular,
and
you
know,
as
the
sort
of
pets
and
cattle
analogy
goes,
kubernetes
was
designed
for
cattle,
and
so
in
this.
In
this
sense
you
know
kubernetes,
hadn't,
really
designed
or
conceived
of
primitives
to
support
stateful
storage
based
apps
from
the
very
beginning
that
is
very,
very
focused
around
web.
B
Kubernetes
has
soap
thanks
to
the
core
OS
folks.
This
idea
of
an
operator
and
operators
are
really
a
really
cool
I.
Think
it's
a
great
idea
that
that
that
was
that
was
popularized
fairly
recently,
actually
just
before
coop
con
Seattle
last
year,
which
I
think
was
around
the
October
time
frame.
So
it's
still
kind
of
a
younger
concept,
but
the
thing
that's
really
awesome
about
operators
is.
It
basically
takes
the
this.
B
Kubernetes
itself
is
completely
random
in
terms
of
the
naming
in
terms
of
the
recovery
semantics
and
a
variety
of
other
things
that
actually
sort
of
making
a
quorum
consistently
reliable
across
failures
is,
is
by
default
impossible
in
kubernetes,
and
so
there's
a
set
of
primitives
over
time
that
have
been
sort
of
introduced.
That
kind
of
allow
you
to
hack
around
this,
but
fundamentally
this
idea
of
an
operator
kind
of
solves
solves
for
this
in
the
most
elegant
way
for
databases.
B
Obviously,
you
want
to
make
sure
that
if
you
have
a
replica
or
a
shard
of
data
on
a
specific
node
with
with
with
the
volume
that
a
containers
is
actually
mount,
mount
'only
don't
want
that
that
container
to
fail
and
come
back
with
a
completely
different
identity
where
it
doesn't
know
where
to
find
that
volume
or
it
doesn't
know
where
to
attach
the
data.
It
doesn't
know
how
to
basically
maintain
a
consistent
identity
between
where
the
containers
and
where
the
volume
is,
and
so
operators
actually
solve
for
that
as
well
in
some
clever
ways.
B
So
we'll
talk
a
little
bit
about
the
operator
pattern
and
sort
of
why
it
was
conceived
and
sort
of
what
it
means
at
a
lower
level.
I
apologize.
If
this
is
a
little
bit
of
a
repeat
from
what
Ivan
presented
on
earlier
I
wasn't
I
wasn't
at
his
talk,
so
I
tried
to
not
actually
a
sort
of
intentional.
He
doesn't
want
to
get
influenced
by
your
genius
so,
but
I
will
watch
it
later.
B
That's
a
very
useful,
useful
pattern
and
then
there's
actually
another
controller
called
now
called
a
stateful
set
used
to
be
called
a
pet
set
that
they
changed
the
name
because
they
don't
want
to
offend
people
who
are
sort
of.
You
know,
thinking
that
pets
should
actually
be
loved
and
not
necessarily
killed
when
the
cluster
dies,
even
though
you
want
identity
there.
So
there
was
a
lot
of
like
a
lot
of
contention
and
argument
of
the
communities
has
actually
changed
from
pets
at
the
stateful
set.
B
What
it
staples
that
does
is
provides
a
stable
name
for
your
Potter
for
your
for
your
application
instance,
along
with
a
along
with
a
stable
Association
to
where
that,
where
that
application
needs
to
run
in
the
cluster,
does
it
run
on
ran
on
a
certain
node
that
has
some
specific
parameter
or
some
specific
property
for
providing
storage
or
providing
some
CPU
type
or
what-have-you?
It
will
always
restart
like
on
that
specific
machine
as
part
of
that
that
sort
of
definition
of
a
stateful
set
and
so
there's
different
types
of
controllers.
B
What
an
operator
gives
you
the
ability
to
do
is
create
a
custom
controller
as
well
as
a
custom
resource
and
so
in.
In
kubernetes.
We
have
this
idea
of
resources
for
a
variety
of
different
variety
of
different
things.
You
have
you
have
an
API
for
for
the
whole
cluster
and
you
have
a
bunch
of
different
resources
for
controlling
how
applications
are
deployed
and
lifecycle
manager
in
the
cluster.
B
What
resources
allow
you
to
do
is
provide
basically
a
common
data
model
for
how
your
application
should
be
represented
and
how
it
should
behave
on
on
sort
of
a
single
single
Association
basis.
The
controller
then
takes
that
resource
manifest.
Basically
a
simple
JSON
llamó
manifest,
and
it
allows
you
to
provide
a
specific
lifecycle
or
controller
mechanism
against
that
resource
and
combined
and
you
and
you
can
have
multiple
custom
resources
for
a
controller.
B
You
can
have
multiple
TP,
RS
or
they're
called
third-party
resources
or
external
resources
that
you
know,
users
create,
and
so
the
operators
basically
a
combination
of
either.
You
know
one
or
more
TP
RS
and
a
controller
that,
basically,
you
know,
dictates
how
your,
how
your
application
or
how
your
service
should
should
be,
should
behave
or
how
it
should
be
life
cycled
in
the
cluster,
and
so
that's
that's
kind
of
one
of
the
one
of
the
the
crucial
components
of
of
operators.
Any
questions
at
all
on
on
that.
B
This
is
a
it's
kind
of
one
of
the
one
of
the
newer
parts
of
kubernetes
that
folks,
who
are
often
curious
about
now,
so
I'll
keep
going
there.
So
we'll
give
you
a
few
examples
of
some
operators
that
exist
today
when,
when
core
OS
came
out
with
this
idea
of
operators,
they
launched
two
operators,
one
for
SCD
and
one
for
Prometheus,
which
is
the
very
popular
sort
of
time
series
based
monitoring,
venting
and
and
visibility
system.
B
That
allows
you
to
get
some
really
interesting,
metrics
and
triggers
out
of
your
distributed
apps,
and
it
gives
you
some
really
nice
monitoring,
visualization
functionality.
Sed
is
obviously
very
popular,
so
we
have
to
talk
too
much
about
that.
What
the
FTD
operator
does
that
the
chorus
books
created
was
it
creates.
B
It
basically
allows
you
to
declaratively
against
that
custom
operator
create
or
stamp
out
that
CD
clusters,
and
so,
if
you
didn't
have
this
operator,
you
would
have
to
go
through
a
lot
of
sort
of
gymnastics
to
figure
out
how
to
best
like
not
not
only
best
represent
and
deploy
a
complete
and
functioning
SD
quorum
in
your
cluster,
but
actually
like
upgrade
it.
How
do
you
make
sure
that
those
nodes
come
back
with
the
same
set
of
registration
and
identity?
How
do
you
make
sure
that
they
can
actually
be
upgraded
properly?
B
How
you
can,
how
can
you
do
backups
and
so
on?
So
the
operator
automates?
All
of
that,
and
it
gives
you
a
custom
controller,
that's
created
by
these
third-party
resources
and
the
custom
controller
there
to
basically
declaratively
say
I
want
a
new
cluster
I
want
to
grow
the
size
of
the
quorum,
I
want
to
backup
the
data,
and
it
gives
you
these,
like
higher-level
controls
to
be
able
to
do
that.
That
declaratively,
which
is
really
awesome
in
terms
of
just
a
quick
snapshot
here.
B
This
is
the
custom
resource
or
a
third
third-party
resource
sort
of
controller
type
that
that
you
would
provide
years.
You'd,
say:
here's
my
example:
I
want
to
get
three.
You
know
three
three
nodes
that
are
desired
and
I
have
my
version,
my
specific
version
that
I
want
to
run
and
through
the
the
kubernetes,
client,
command-line
client,
you
would
just
say,
I
want
to
create
this
replica
replica
set
and
the
pod
the
pod
counts
in
the
environment.
New
labels
that
you
set
are
obviously
defined
in
your
configuration
there
and
then
once
the
once.
B
The
API
server
starts
recognizing
that
those
those
pods
need
to
be
created
based
on
this
manifests
that
you've
passed
to
the
to
the
command
line,
it'll
start
creating
those,
and
you
can
control
the
lifecycle
in
the
same
way
that
you
would
control
replica
sets
or
demon
sets
or
other
other
types
of
controllers.
So
you
now
have
like
a
very
specific
controller
for
a
specific
app
and
you
can
be.
You
can
be
really
certain
about
how
the
sort
of
application
is
managed
over
time.
B
Based
on
the
primitives
that
kubernetes
provides
so
this
is,
this
is
really
great
functionality.
I
think
it's
going
to
solve
a
huge
number
of
problems
for
for
people
looking
to
run
more
complex
applications
on
kubernetes,
without
needing
to
worry
about
all
of
the
all
the
custom
semantics
that
they
need
to
kind
of
wire
together
in
order
I
think
I
kind
of
talked
about
that
one
already,
so
the
rook
operator
is
somewhat
similar
in
terms
of
the
the
design
philosophy
and
the
use
case.
B
As
the
STD
operator,
it's
a
stateful
system,
it
stores
data
SCD
is
a
stateful.
You
know
key
value
store
for
doing
sort
of
cluster
metadata,
synchronization,
and-and-and
and
consensus
and
locking
and
doing
a
bunch
of
other
things
and
it
you
know
it's
similar
in
the
way
that
you
know
you,
don't
you
don't
want
to
lose
data?
You
know
you
want
to
make
sure
it's
reliable.
It's
important
that
the
system
has
certain
identity,
characteristics,
and
so
rook
is,
as
I
mentioned,
a
distributed.
B
Object
lock,
file
storage
system
based
on
staff
that
provides
distribute
storage
to
applications
running
in
your
cluster.
What
this
rook
operator
does
is
it
basically
takes
takes
to
all
the
stuff
components
that
rook
kind
of
abstract
for
you
which
are
running
in
containers?
Now
it
actually
runs
them
as
a
native
sort
of
kubernetes
service
on
the
cluster,
which
is
extremely
useful.
So
this
this
rook
operator
is
relatively
new.
It's
been
it's
been
in
the
works
and
in
development
for
the
last
I
think
three
four
months.
B
These
things
are
kind
of
going
to
disappear
as
the
Linux
kernel,
primitives,
have
sort
of
over
time
disappeared
for
application
developers,
maybe
sometimes
like
not
not
not
so
positively
like.
They
should
know
about
certain
things
and
certain
certain
calls
and
functions
that
they
should
they
should
take
advantage
of,
but
kubernetes
is,
is
providing
a
lot
of
these
really
powerful
primitives.
So
in
the
first
stage
here,
what
this
work
operators
providing
is
basically
the
creation
of
a
deployment
and
what
a
deployment
does
is.
B
The
second
thing
that
it
does
is,
it
starts
and
manages
the
the
set
quorum
itself,
and
so
the
Ceph
monitor
is
pretty
important
for
making
sure
that
you
have
visibility
into
all
the
different
blocks
and
storage
types
that
are
basically
getting
basically
getting
replicated
across
the
cluster
for
all
the
distribution
and
sort
of
finding
and
sort
of
communication
of
of
the
data
in
the
cluster
from
the
client
perspective.
And
then
once
that's
done,
the
OSF
OSDs
are
created
on
each
node
in
the
cluster
through
a
daemon
set.
B
So
when
we
talked
about
these
different
types
of
controllers,
so
those
are
actually
absorbed
and
taken
care
of
and
sort
of
selected
and
defined
for
you
in
advance.
Just
by
deploying
this
operator
on
the
cluster
and
the
daemon
set
for
the
OS,
Keys
is
basically
saying
for
for
the
OSD.
You
know
daemon
that
we
want
to
run
one
per
node,
so
we
have
each
each
OSD
agent
running
on
every
node
in
the
cluster.
B
So
you
can
have,
you
know,
object,
storage
available
to
the
cluster
right
and
you
can
actually
apply
labels
on
those
pod
templates
for
the
OSD
daemons.
So
you
actually
know
where
they're
running
you
can
actually
health
health
check
them
and
manage
them
and
replace
them
and
upgrade
them
and
so
on
through
the
API
server
itself,
which
is
really
which
is
really
useful.
The
the
fourth
step
here
in
the
subset
and
the
rook
operator
is
it's.
B
Normally
without
an
operator,
this
would
be
quite
a
lot
of
work.
You
have
to
figure
out
how
to
write
the
the
configuration
you
know
manifests
for
your
deployments,
for
your
daemon
sets
for
your
controllers,
for
you
know
for
your
for
your
for
your
for
your
daemon
sets
and
deployments,
and
so
on,
and
that's
quite
a
lot
of
work,
but
you
also
have
to
understand
SAS.
B
You
have
to
understand,
like
the
the
deployment
models
like
where
the
agent
should
run,
in
which
sequence
in
which
order
there's
like
a
lot
of
institutional
knowledge,
kind
of
baked
into
that
or
specific
service
knowledge
that
you
know
you
now
really
don't
have
to
think
about
use.
All
you
have
to
do
is
really
just
trust
that
this
operator
is
going
to
sort
of
encapsulate
a
lot
of
that
for
you
reliably,
and
so
this
is.
This
is
pretty.
B
So
the
OSDs
running
on
each
node,
the
the
monitoring
agent
MDS
rgw,
all
of
the
all
the
different
pieces
that
the
restful
api
that
we
mentioned,
and
so
it's
all
running
on
the
cluster
as
a
set
of
containers
as
a
set
of
pods
and
kubernetes
applications
themselves,
basically
directly
mount
those
block
devices
or
file
systems
managed
by
rook
natively,
and
they
can
also
use
native.
So
it's
also
s3
api
compliant
or
swift
api
compliance.
You
can
be
using
something
like
OpenStack
or
something
like
it.
B
Aws
s3
be
familiar
with
those
interfaces
and
API
calls
and
they'll
work
just
just
the
same.
Of
course.
The
rest
api
service
that
we
mentioned
is
available
and
it's
also
exposed
to
a
command
line
tool.
That's
that's!
That's
that's
offered
in
the
service,
the
rook
operator
itself
has
a
single
container
binary,
and
so
you
can
actually
deploy
that
down
as
as
a
single
as
a
single
service
on
a
cluster
and
basically
bootstrap
and
monitor
the
entire
cluster
itself,
and
once
things
are
requested
through
the
API,
you
know
you
basically
start
a
deployment.
B
B
Some
of
this
is
changing,
based
on
the
maturity
based
on
maturity
and
the
and
the
development
of
third-party
resources
in
kubernetes,
which
again
are
fairly
new,
I,
think
I,
think
they're
about
a
year
old,
but
they're,
just
starting
to
get
past
I
think
a
beta
release
and
they're
not
they're,
not
quite
production,
ready,
yet
I
believe
they're
in
beta,
they
might
be
out
of
it,
I
think
they're
in
beta
in
in
kubernetes
1.7.
That
might
be
changing,
which
is
coming
out
in
a
couple
months.
B
B
Obviously,
if
you're
running
on
a
bare
metal
cloud,
you
have
these
sort
of
great
alternatives
to
the
public
cloud
services
that
you
use
very
commonly,
even
if
you
run
on
ec2
frankly,
I
think
there's
a
lot
of
value
for
using
something
like
real
good.
You
don't
actually
have
to
write
natively
to
the
types
of
services
that
would
effectively
lock
you
into
a
cloud
provider
or
maybe
sort
of
robbed
you
of
utilizing
your
local
storage.
That's
running
on
your
instances
where
your
applications
are
so
there's
a
few
things
to
think
about
there.
B
As
far
as
the
roadmap
as
I
mentioned,
this
is
a
fairly
new
early
project.
There's
there's
a
failover
is
getting
implemented
now,
I
think
there's
a
PR
for
that
upgrades
itself
are
are
also
under
under
development
and
there's
there's
also
a
few
other
TP
RS
that
are
getting
developed
for
for
different
parts
of
the
Ceph
cluster
for
describing
topologies
for
selecting
which,
which
notes
to
run
certain
storage
components
on
and
to
control,
which
other
devices
and
directories
are
used.
B
So
the
project
is
very
active,
I
think
there's
about
15
or
so
people
working
on
it
full
time
from
from
quantum.
As
a
few
of
the
folks
that
are
started
to
contribute
to
it
externally
and
the
project
is,
you
know,
getting
a
lot
of
use
in
momentum,
so
lots
of
lots
of
people
contributing
to
it?
Almost
you
know
in
terms
of
just
sort
of
issues
and
variety
of
other
things.
That's
the
main
kind
of
presentation.
B
I
did
have
a
brief
demo
I'm
a
little
worried
about
how
do
I
switch
to
the
allowance
to
see
if
I,
okay,
so
I
was
too
wimpy
to
do
a
live.
Demo
I'm,
not
Kelsey,
Hightower,
a
good
friend
and
very
brave
amazing
person,
but
this
will
loop.
This
is
just
giving
an
example
of
how
easy
it
is
to
deploy
rook
on
a
kubernetes
cluster.
There's
a
couple
of
dependencies
that
you
need
available
in
the
cluster
you
need
to
have.
C
C
C
C
B
The
cluster
manifests
available,
it's
as
simple
as
and
once
your
kubernetes
cluster.
It
has
those
dependencies
in
place,
it's
as
simple
as
creating
a
couple
of
very
simple,
manifest
and,
and
you
have
your
real
cluster
available,
and
so
the
goal
is
to
have
this
very
simple
like
to
command
three
command.
Bootstrapping
deployment
also
possible
on
docker
swarm
on
mezzos
on
Cloud,
Foundry
and
other
systems.
B
So
it's
not
just
specific
to
kubernetes,
but
of
course,
because
kubernetes
is
sort
of
the
you
know,
I
think,
unquestionably,
the
sort
of
dominant
you
know
container
runtime
operating
system
for
distributed
systems.
This
is
sort
of
the
focus
area
initially,
so
that's
kind
of
all
I
had
didn't.
Did
one
say
too
much
more
time
on
this.
Any
any
questions
from
folks
I'm
happy
to
take.
Thank.
A
C
B
Yeah
I
mean
I
think
that
that's
going
to
happen
organically,
it's
an
open
source
project.
People
will
I
think
sort
of
learn
more
about
and
hear
more
about
it
as
the
community
develops
so
the
court.
The
question
was
for
the
video
and
everything
like
how
will
people
who
maybe
don't
know
they
have
a
problem
and
that
rook
might
solve
it
like
how
they
learn
about
rook
and
how
will
they
realize
that
this
will
solve
a
specific
problem?
Is
that
fair,
repeated
your
question?
Yeah
I
mean
I
think
that
that
will
happen
organically.
You
see.
B
The
really
cool
thing
is.
This
is
open
source
software,
100%,
open
source
based
on
go
and
and
SEF,
and
so
you
know
it's
a
really
exciting
and
interesting
area.
I.
Think
storage
is
one
of
the
really
big
unsolved
problem
spaces
for
the
sort
of
container
industry
right
now,
in
particular,
with
kubernetes.
There's
lots
of
offerings
and
lots
of
ways
to
get
around
this,
but
for
sort
of
a
single
system
to
provide
all
these
different
interfaces
and
sort
of
the
native
lifecycle
components
that
we
talked
about
in
operators.
I,
don't
I,
don't
think
anything
quite
exists.