►
From YouTube: CNCF Storage WG 07/18/2017
Description
The CNCF Storage WG meets bi-monthly to explore cloud native storage technology and concepts. Its current initiative is a project that aims to define an industry standard “container storage interface” (CSI). This interface would enable storage vendors to write a driver once and have it work across a number of container orchestration (CO) systems. This video is the WG July 18th meeting where the group discussed ROOK and received a demo of the technology.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
I'm
updating
that
page
right
now,
I,
don't
know
in
terms
of
attendees
and
then
I'd
also
just
like
to
hold
space
for
just
a
second,
so
people
can
share
if
they've
got
topics
that
they
want
to
be
discussed
or
talked
about
in
the
SW
g
and
storage
working
groups.
If
anyone
has
those
now's
a
good
time
to
mention.
D
A
A
Okay,
so
I
think
next
week,
I'm
sorry
next
time
we're
going
to
try
to
have
the
open,
SDS
guys
present
and
talk
about
what
they're
doing
from
the
storage
space
be
good
to
make
sure
we
understand
what's
happening
there
too,
but
I'd
love
to
hear
more
topics
and
if
there's
not
enough
topics,
then
we'll
change
the
Kade
into
the
meeting
primary
two
weeks
to
maybe
every
month
have
topics.
Please
share
them
with
me.
B
A
A
Folks
meet
if
you're
not
talking,
please,
okay,
great,
so
I
wanted
to
bring
it
I
wanted
to
bring
this
project
to
the
storage
working
group.
Allow
do
a
deeper
dive
and
understand
in
more
detail,
and
so
this
is
an
opportunity
for
the
flounder
presents
as
well
as
been
for
us
ask
lots
of
questions.
Will
contour
to
it
bassam.
You
can
go
ahead.
E
C
And
I'm
planning
to
do
a
demo
and
I'm
hoping
that's
cherry-
will
work
for
that
suit,
so
so
high
I
minded,
my
name
is
Bassam
I'm,
the
CTO
of
quantum
we
started
this
project
called
rook
rook.
Essentially,
the
storage
Orchestrator
for
cloud
native
environments
and
I
wanted
to
kind
of
walk
you
through
it
me
and
hopefully,
you'll
get
all
sorts
of
questions
and
feedback
on
the
project.
So
just
a
quick
overview
rook
was
open.
Sourced
in
November
of
2016
is
under
an
Apache
2
license.
C
We
we
started
this
project
essentially
so
that
we
can
get
storage
working
in
a
in
a
cloud
native
way
on
top
of
cloud
native
environments,
so
it
takes
distributed.
Storage
software
think
stuff
for
now
in
the
future
will
be
cluster
or
others
and
then
orchestrate
them
so
that
they
could
run
really
well
on
top
of
cloud
native
environments
essentially
turns
them
into
self-managing,
self-healing
self
scaling
services
properties.
You
expect
out
of
a
cloud
native
service,
and
it
does
this
by
automating
deployments.
Bootstrapping
configuration
provisioning
scaling
all
the
things
that
you
know.
C
If
you're
running
as
a
service,
you
expect
to
be
done
for
you
rook
attempts
to
do
all
of
those
things
in
software
and
by
leveraging
the
cloud
native
container
management
system,
the
scheduler,
the
orchestrator
to
actually
perform
its
duties.
So
one
way
to
look
at
it
is
that
rook
uses
cloud
native
environments
to
run
a
highly
you
know
reliable
and
scalable
storage
services.
C
C
All
right
can
you
so
rook
integrates
deeply
into
cloud
native
environments.
Leveraging
extension
points
it's
the
goal.
There
is
that
we
are
trying
to
provide
a
very
seamless
experience
with
the
container
realm
and
almost
think
of
storage
as
being
a
built
in
an
extension
of
the
cloud
native
environment.
So
things
like
scheduling,
lifecycle,
management,
resource
management,
policies,
security,
including
our
back.
C
The
project
currently
in
alpha
beta
is
planned
for
for
the
Fall
and
we
hope
to
be
production
ready
by
the
end
of
this
year.
So
pretty
excited
schedule
and
I'll
say
part
of
it.
Is
that?
Because
we
are
not
building
a
new
data
path,
we're
not
creating
a
new
storage
system,
we're
orchestrating
battle-tested
existing
storage
systems.
C
C
So
so
you
know
we
started
looking
at
storage
and
running
storage
and
top
of
cloud
native
environments.
You
know
it's
kubernetes
or
docker
basis
or
others,
and
you
know
one
of
just
it's
kind
of
a
starting
point.
We
realized
that
most
of
the
container
native
applied
native
environments
treat
storage
systems
as
an
external
thing,
they're,
not
part
of
the
environment,
they're
not
say
as
we
treat
containers
or
as
we
look
at
networking
where
we
we
make
those
first-class
storage
is
an
external
entity.
C
C
So
if
you
look
at
this
diagram-
and
you
you
know
on
the
left-
is
our
cloud
native
environment
on
the
right
is
the
actual
storage
service
or
the
storage
software
and
and
your
choices
are
if
you're
running
in
a
cloud
environment,
it's
whatever
Amazon
or
as
or
Google,
provide
there
as
the
EBS
and
equivalents
or
it's
open
source
software
that
you
have
to
essentially
run
yourself
or
it's.
You
know
traditional
enterprise,
storage,
think
of
boxes
that
that
provide
some
kind
of
storage
or
NFS,
or
whether
it's
NFS
or
San
or
whatever
the
interface
is.
C
So
we
ask
the
question
why?
Why
is
storage
outside
the
cloud
native
cluster
and-
and
you
know
from
from
that
perspective,
you
know
if
you
look
at
a
use
case,
if
I
wanted
to
run
something
like
get
lab
on,
say:
kubernetes,
let's,
let's
start
with
on-premise
well,
I
mean
there's
a
helmet
for
get
LA,
but
it's
really
great
it
you
know
I
can
I
can
deploy
it
really
quickly.
It
comes
up.
I
get
all
the
amazing
features
of
pod
sale
over
allocation
of
resources
and
resource
management.
C
Networking
kind
of
works
all
of
the
stuff
happens,
but
you
know
good
luck
requires
some
durable,
persistent
storage
for
both
the
Postgres
database,
as
well
as
the
actual
get
data
itself,
and
so
I
essentially
fall
off
a
cliff.
In
that
scenario,
I
have
to
go
figure
out
how
to
provision
storage
outside
of
the
cloud
native
environment
and
the
weather.
It's
a
soft
cluster
or
luster
luster
in
the
open
source
world,
or
it's
some
something
else.
C
On-Premise
I
can't
I,
don't
have
s3
or
EBS
on-premise,
which
means
that
you
know
I
have
to
figure
out
how
to
do
a
security,
scheduling,
availability,
backup
in
the
our
story.
Upgrades
migrations,
monitoring,
all
of
those
things
that
are
required
to
actually
get
storage
to
work
well
to
support.
This
use
case
happen
outside
of
the
cloud
native
environment
n.
C
Even
if
you
think
about
running,
get
love
on,
say
kubernetes
on
on
inside
eight
of
us,
you
know
a
DSS
service,
it
I
get
storage
services,
they
learn
as
a
service,
so
I
get
EBS
and
I
get
s3
and
I
get
even
the
EFS,
almost
storage
standpoint
and
with
dynamic
volume,
provisioning,
saying
kubernetes
and
volume
plugins
and
the
path
SCSI.
The
cloud
native
environment
can
actually
provision
storage
pretty
nicely
in
these
using
violence,
but
I
still
don't
get
the
integrated
security
and
integrated
scheduling,
even
quality
of
service.
C
You
know,
and
then
there
and
there
are
pretty
experienced
in
terms
of
running
a
dynamic
environment
such
such,
like
kubernetes,
in
a
place
like
AWS.
You
start
to
see
kind
of
cracks
in
this
in
the
scenes
here
or
cracks
in
the
hook
and
in
the
story
where
storage
actually
is
not
as
dynamic
as
the
cloud
native
environment
is
I'll.
Show
you
a
couple
of
examples
just
to
kind
of
illustrate
this
so
on
the
Left.
If
I
was
looking
at
our
pod
failing
over
in
and
availabilities
on,
with
EBS
kubernetes
is
a
good
job
at
provision.
C
Ebs
dynamic
volume
provisioning
so
when
a
pod
failover
is
to
take
up
to
an
hour
for
the
EBS
volume
to
detach
and
then
reattach
to
the
pod.
So
while
we
spend
all
this
time
and
energy
trying
to
make
failover
us
pause
and
containers,
be
super
quick
and
super
efficient,
because
storage
is
external,
it
could
take
up
to
an
hour,
for
this
available
would
happen.
So
the
bottom
chart
here
shows
gaps
in
a
prometheus
chart
where
it's
up
to
an
hour
for
it
to
come
back
alive,
because
it's
running
only
dias
and
on
the
right.
C
You
can,
you
know,
think
if
I
was
running
a
cluster
across
available
to
zones.
Well,
if
I
wanted
to
fail
a
pot
over
the
loading
zones
because
I'm
building
it
an
H,
a
app
or
service
and
EBS
William
does
not
fail
over
across
them.
So
I'd
have
to
take
a
manual
snapshot
of
it
or
have
some
automation
outside
of
this
environment.
Send
it
up
to
s3
and
then
do
a
essentially
a
restore
from
that
create
a
new
volume
from
that
snapshot
in
the
other
zone,
and
that's
all
done
outside
of
the
big
cloud
native
environment.
C
F
Song,
this
is
a
steep
walk
from
the
cube
storage
say
that
I
agree.
The
multi
availability
plan
I
think
the
single
availability
sign
is
artifacts.
All
the
bug
series
of
attach
detach
synchronization
issues
in
q1
6
that
I
think
you'll
see
it's
largely
fixed
in
q1.
So
we
hit
this
a
lot
and
I
pictured
online
as
well.
Yeah.
C
Even
outside
of
kubernetes
environment
attach
detach
cycles
take
a
long
time
and
what
EDS?
Even
if
you
don't
introduce
kubernetes
it
really
depends
on
the
size
of
the
volume
or
provision
diets,
and
you
know
what
the
noisy
neighbor
problem
of
what's
currently
happening
in
in
the
environment,
so
so
I
think
I
think
there's
definitely
improvement
on
the
kubernetes
side,
but
I
think
this
problem
is
actually
independent.
E
C
Think
you
know:
storage
services,
themselves,
storage,
software
themselves,
our
distributed
systems
and
cloud
native
environments
are
really
good
at
or
orchestrating
and
managing
distributed
systems,
and
at
some
level
you
could
argue
that
this
is,
you
know,
a
great
marriage
of
some
sorts,
and
so
why
don't?
We
run
the
storage
service
itself
on
top
of
apply
native
environments
using
these
scheduler
using
the
resource
management
facilities
using
the
failover
facilities
to
make
the
storage
system
to
run
more
like
a
service
and
make
it
essentially
and
integrated
deeply
into
the
environment.
C
C
So
that's
my
classes,
so
what
I
have
here
is
a
three
node
kubernetes
cluster
and
I'm
just
going
to
demo
running
work
on
it.
It
has
really
nothing
on
it
at
this
point.
So,
let's
see
so
I
have
a
first
thing.
I'll
do
is
I
will
install
the
rook
operator
so
think
of
the
operator
as
essentially
an
extension
point
off
the
cloud
native
environment.
It's
it's
how
we
add,
essentially
a
controller
that
is
going
to
manage
storage
in
this
cluster.
This
is
a
one-time
operation.
C
C
The
operator
camel
and
you'll
see
that
a
quad
gets
started
with
a
rock
operator
on
it.
So
there
is
the
rock
operator
and
at
this
point,
I
can
go
ahead
and
create
clusters
and
I
create
virtual
pools,
use
them
in
all
sorts
of
interesting
ways.
So,
let's
start
by
creating
a
new
storage
cluster,
so
here's
the
yamo
for
storage
cluster
I
can
specify
the
name
of
the
cluster
and
the
name
space
it
goes
in.
C
I
can
specify
the
actual
version
tag
of
rook
to
use
in
this
case
we're
just
going
out
to
master
and
then
I
can
select.
I
can
specify
a
set
of
properties
that
help
me
identify
nodes,
storage,
nodes
or
you
know,
nodes
that
have
certain
kinds
of
storage
and
my
consultants,
all
sorts
of
interesting
ways
that
make
up
this
storage
cluster.
So
I'll
just
leave
the
defaults
for
now
and
go
ahead
and
create
the
cluster.
C
C
What
happens
is
that
the
rook
operator
notices
that
I
wanted
to
create
a
new
cluster.
The
cluster
itself
is
an
API
object
or
an
extension
object
includes.
The
real
operator
sees
that
there
is
a
desire
to
create
a
new
cluster
and
then
it
actually
goes
ahead
and
starts
the
process
of
orchestrating
a
new
cluster.
So
you'll
see
a
bunch
of
pods
come
up
in
the
namespace
in
this
case
were
orchestrating
stuff,
so
you'll
see
the
SEF
monitors.
Three
of
them
come
up
to
get
bootstrap,
they
have
they
put
in
a
quorum.
C
They
we
wait
for
that
to
happen.
We
check
the
health
of
it.
The
operator
is
doing
all
this
work.
It
starts
the
storage
nodes.
You
know
settlers
deeds,
it
configures
them
appropriately,
essentially
bringing
up
a
cluster
that,
because
I've
specified
all
the
storage
nodes
here,
brings
up
cluster,
given
the
resources
available
and
working
very
closely
with
the
orchestrator,
whether
the
kubernetes
api
and
the
scheduling
and
resource
management
facilities
available
there.
C
So
now
that
I
have
a
cluster,
let's
actually
go
ahead
and
use
it
I'm
going
to
go,
create
a
pool
and
a
pool
is
essentially
a
essentially
a
virtual
pool
that
specifies
the
replication
or
erasure
coding
in
a
redundancy
and
an
other
parameters.
So
I'll
go
and
create.
You
know
we'll
call
it
a
replica
pool.
It
will
be
in
the
rough
namespace
and
we'll
just
for
now
create
no
one.
Do
anything.
C
C
Again,
this
is
inch
dealing
directly
with
cube,
CDL,
so
I'm
doing
all
this
work
inside
kubernetes
and
at
this
point
the
operator
will
go
and
pick
up
my
you
know
the
desire
to
create
a
pool
and
then
go
configure
the
pool
within
the
sub
cluster
itself,
setting
all
the
parameters
doing
all
of
it.
Whatever
is
needed
to
get
that
working
and
we're
essentially
done
I
created
a
pool
and
now
I'm
going
to
I
can
use
the
pool.
I
can
use
it
to
dynamically
provisioned
volumes
within
this
cluster.
So
let's
do
that.
C
Let's
go
fire
up
WordPress,
so
I
have
a
amyl
here
for
deploying
WordPress
and
I'll
show
you.
You
know
from
this
point
on
we're
just
using
the
facilities
for
persistent
volumes
and
precision
are
dynamic,
long
provisioning
inside
kubernetes,
so
as
you're
familiar
with
WordPress
WordPress
uses
MySQL
to
actually
how
the
does
all
the
participants
inside
the
database.
The
workers
also
has
persistent
store
underneath
it,
for
you
know,
content
and
you
know
HTML
content
and
everything
else,
and
so
we'll
see
that
you
know.
C
If
we
look
at
this
that
there
is
a
persistent
volume
claim
for
MySQL-
and
there
is
enough-
and
it's
said
to
be
20
gigabytes
and
the
access
mode
is
really
ones
and
here's
where
we
say
that
we
want
to
use
rook.
So
we
specify
the
storage
class
for
where
we
want
this
longing
to
be
created
and
again
that's
how
we
derive
the
pool
settings
and
gratification
and
or
erase
your
coding
or
everything
else
comes
through
that
and
then
there's
another
volume
down
here
for
for
WordPress
itself.
C
There
is
another
volume
claim,
also
20
gigabytes
and
there's
a
storage
class.
So
I'll
spare
you
the
details,
but
let's
just
go
and
create
WordPress,
so
you'll
see
essentially
the
WordPress
pods
come
up.
You'll
see
that
the
two
claims
have
been
bound,
which
means
that
a
volume
has
actually
been
created,
a
capacity
of
the
volumes,
and
here
we
are
where
press
is
running.
So,
let's
you
know
it's
important
to
know
that
this
is
running
inside
the.
C
C
Call
it
login,
then,
let's
just
go
to
the
default
page,
so
here's
WordPress
running
on
top
of
kubernetes,
where
the
two
volumes
that
it
needs
one
for
MySQL
and
one
for
static
content,
are
a
provision
by
rook
just
for
fun.
Let's
actually
go
ahead
and
kill
one
of
the
pods.
Let's
kill
the
MySQL
pod
and
show
you
a
field
settle
the
case.
C
C
That's
it
that's
it
for
the
demo.
You
know.
The
key
points
here
is
that
storage
is
now
running
inside
the
cluster,
the
storage
service
itself,
it's
self
managing
through
the
operator
itself,
scaling
as
I
add
nodes
and
the
cluster
grows
with
with
my
apps
and
it
integrates
very
deeply
I
didn't
use
any
rook,
API
or
rook
CLI.
This
is
really
just
using
kubernetes
itself
and
treating
storage
as
if
it
was
built
into
kubernetes.
F
Yeah
mic
demo,
the
song
might
had
a
follow-up
question
on
me
when
you
sort
of
deploy.
You
know
the
the
actual
backing
storage
pod.
How
does
it
know
which
block
devices
or
master
spades
to
Hughes,
like?
Presumably,
if
you
get
the
wrong
one,
if
it's
a
block
device
will
pass
in
the
wrong
part
that
could
be
problematic.
How
do
you
guys
wraps
around
there
so.
C
Basically,
when
you
create
a
cluster
II,
you
can
specify
which
storage
nodes
you
want
to
use.
So
you
can
use
the
you
know:
node
selected
facilities
or
inside
kubernetes.
You
can,
you
know,
buy
labels
or
buy
you
know
any
other
kind
of
metadata.
You
can
select
the
nodes
and
then
within
the
storage
node.
You
can
also
tell
within
the
cluster
specification
which
devices
to
use
or
whether
use
the
file
system
and
how
to
configure
these
devices.
All
of
that
is
essentially
part
of
the
cluster
definition,
and
you
can
do
it
both
at
a
say,
I.
C
You
know
we
make
it
easy
to
say
just
use
all
the
devices
or
use
all
the
nodes
for
storage
or
you
could
get
more
selective
or
if
you
want.
You
can
probably
go
down
to
the
level
of
saying
here
exactly
the
storage
note
that
I
want
you
to
use
and
you
can
make
changes
on
the
cluster
definition
and
the
rook
operator
will
actually
pick
those
up
and-
and
you
know,
in
a
reconciliation
to
make
the
changes
as
needed.
Okay,.
C
Yeah,
in
fact,
we'd
a
subset
stuff
is
really
good
at
this
kind
of
topology
we
are
sure
you're
familiar
with
crush
maps
and
all
that,
so
we
use
that
we
use
the
metadata
defined
on
the
cluster
and
a
storage
nerve
and
all
that
stuff
to
feed
into
a
crush
map
so
that
we
can
improve
storage.
But
you
know
both
in
terms
of
durability
and
performance
or
storage
based
on
the
location
of
the
nodes
and
relocation
of
devices.
C
Londoners
are
the
kinds
of
devices
on
the
roads,
and
you
know
things
like
you
know
where,
which
is
going
the
devices
themselves.
Some
of
them
are
SSDs
or
nvme
prices
versus,
say,
spinning
discs,
all
that
factors
also
into
the
selection
criteria
and
how
we
build
a
cluster,
and
you
could
configure
that
as
part
of
the
question.
C
D
C
Good
question
a
number
of
things:
actually,
so
so
the
plugin,
the
assess
plugin
that
we
are
using
in
this
demo
are
the
RBD,
RBD,
plugin
and
also
I.
Think
we
will
support
this
ffs
plugin
two
issues.
Three
things
that
motivate
the
a
new
plugin
one
is
that
the
path
to
actual
disk
storage
node
today
goes
through
the
kernel.
So
you
think
of
a
pod
pod
amounts
of
there's
a
volume
mounted,
and
then
we
do
use
kernel
RBD
going
down
to
the
kernel
and
then
to
the
actual
storage
cluster
itself.
C
There
is
a
desire,
and
in
fact
even
ACEF,
teen
and
others.
You
know
there's
a
desire
to
run
that
body
of
code
in
user
space.
For
numerous
reasons
one
is
you
can
bypass
the
kernel
and
get
our
DMA
and
everything
else.
The
other
is
that
less
less
obvious
one
is
that
the
actual
versions
of
kernel
RVD
in
Colonel,
Seth
FS,
are
quite
lagging
when
it
comes
to
kernel
versions,
because
the
code
gets
checked
into
the
Linux
kernel,
trees
that
are
known
bugs
and
timeouts,
and
issues
in
in
previous
versions.
C
So
a
typical
thing
that
we'd
like
to
do
is
basically
use
things
like
I,
scuzzy
or
NBD
to
mount
the
volume,
but
have
a
demon.
A
demon
set
running,
that's
actually
proxying
am
user
mode
to
the
cluster
and
that
decision
we'd
like
to
be
dynamic,
not-not-not
happened
out,
say
cluster
definition
time,
but
but
actually
one
time
and
actually
base
it
on
to
capabilities
of
the
nodes
itself.
D
C
C
F
Basically,
if
you
win
the
volume
plug-in
mounts
on
a
sip
redirect
once
running,
which
is
an
RB
d
that
that
attach
will
then
lock
that
that
RBD
volume
to
that
host.
So
it's
all
happening
at
the
volume
plug-in
layer,
but
there's
that
same
thing
that
happens
for
the
readwrite
mini
stuff
as
we're
pretty
much
old.
Are
we
drive,
maybe
and.
C
That
does
exactly
REM,
depending
on
the
kernel
version,
CRPD
version
that
you
have
sensing
may
or
my
network
I,
think
for
it
to
work
really
really
well
as
intended.
That
I
have
like
4.11
kernel
on
your,
and
so
you
know,
part
of
the
strategy
ACEF
team
is
doing
is
all
this.
Is
that
they're
they're,
essentially
trying
to
build?
You
know,
do
I
scuzzy?
Are
you
or
NBD
trying
to
move
that
at
all
into
user
space?
C
C
C
Local
storage
arm
and
not
that
they're
really
great
facility
for
this
use
case,
because
it's
or
we
don't
have
to
deal
with
like
you
know,
direct
which
directory
was
it
on
and
we
currently
create
affinity
between
the
roof.
Os
be
the
storage
node
and
the
machine
that
it's
on,
so
that
we
can
come
back
in
the
same
place.
So
a
lot
of
things
get
easier
with
one
set.
Of
course,
we'll
have
to
exact
look
about
a
little
with
one
six,
but
but
where
this
work
is
on
target
afar's
you're,
doing
six
release
cool.
A
C
I
have
a
slide
on
architecture,
I'm
happy
to
walk
through
it.
Let
me
see
if
I
can
bring
that
up,
so
I
think
we're
done
other.
Then
I
can
go
back
to
the
slides
so
currently
there
we
currently
support
two
modes
and
I'll
just
talk
about
one
of
them.
For
now
the
the
kubernetes
implementation,
essentially
is
run
like
kubernetes
controller,
and
so,
if
you're
not
familiar
with
controllers,
it's
essentially
the
there's
the
concept
of
a
desired
state.
What
the
admin
or
cluster
operations
or
cluster
admin
wants
to
happen.
C
It's
a
desire
for
some
something
in
to
happen
in
the
cluster,
then
there's
the
actual
state
of
let's
actually
have
in
the
cluster,
and
the
controller
essentially
sits
in
a
loop
and
tries
to
reconcile
you
know
actual
state
with
desire
state.
So
it's
so
when
you
saw
the
declarative
statements
to
create
a
cluster.
You
know
there
was
a
ya
know:
fire
folder
cluster
and
the
operators
watching
it
watches
the
kubernetes
api.
So
you'll
see
and
it'll
find
that
there's
a
cluster,
a
new
cluster
and
it
will
go,
do
what's
needed
to
bring
the
cluster
up.
C
If
there
are
changes
the
cluster,
then
those
you
know,
the
operators
will
look
probe
the
desired
that
the
actual
state
and
make
appropriate.
You
know
changes
mutations
to
the
cluster
itself,
so
it's
literally
implemented
as
a
controller
where
it
uses
the
kubernetes
api
to
watch
all
these
objects.
Really
these
objects
defined
here
and
it
itself
uses
these
objects
to
create
units
to
actually
deploy
the
cluster
and
when
it
so
for
some
of
them
there
you
can
figure
them
as
input
objects
and
for
others,
it's
actually
using
them.
C
C
It
also
talks
through
management
and
health
API
to
each
of
the
cause
themselves.
So
this
is
at
a
node
now
at
a
node
level,
not
cluster
level,
if
I
need
to
probe
a
given
storage,
node
for
its
health
or
I
need
to
probe
a
given
storage
node
or
ask
it
to
do
something
that
is
at
a
node
level,
operation
that
happens
between
the
operator
and
the
actual
pause
themselves.
Sometimes
that's
done
over.
C
We
follow
an
approach
where
we
now
have
just
a
kind
of
like
hypercube.
We
have
a
hyper
rook
pod.
That
has
you
know
it's
about
a
hundred
megabytes
compressed.
It
has
all
the
logic
needed
to
run
the
entire
storage
cluster
and
it
can
come
up
in
different
roles.
So
you
come
up
as
a
SEF
Lotterer
rook
rook
role,
including
the
operator
itself,
is
deployed
in
this
way
and
then
the
rest
is
really
we.
You
know.
C
With
this
design,
we
there's
a
very
integrated
design
into
kubernetes,
we've
shied
away
from
exposing
any
kind
of
API,
endpoint
Europe
or
at
a
cluster
level
or
any
kind
of
CLI
or
anything
else.
We.
The
goal
here
is
that
to
integrate
deeply
with
the
environment,
so
you'll
see
things
like
you
know:
I
don't
need
a
I,
don't
need
a
credentials
to
connect
to
rip
I'm
we're
we're
leveraging
the
security
model
in
communities,
including
our
back.
So
as
long
as
you
set
up
your
security,
you
know
your
are
back
policies
correctly.
C
You,
the
cluster
admin,
will
have
access
to
to
look.
It
also
means
that
if
you
want
to
enumerate
clusters-
or
you
want
to
enumerate
pools
or
do
all
that
stuff,
they're
all
objects
that
you
can
enumerate,
you
can
list
you
can
you
know
all
your
existing
monitoring
systems
are
all
using
scripts
and
all
us
that
should
work
nicely
with
all
of
it,
because
there's
not
a
new
thing,
it's
just
it's
a
truly
an
extension
opportunities.
C
The
last
thing
I'll
say
is
that
Brooke
itself
keep
state.
It
needs
state
to
you
know
to
manage
the
cluster
and
by
storing
all
of
that
in
kubernetes.
It
also
gets.
You
know.
It's
essentially
gets
stored
in
a
TV,
and
essentially
all
config
and
cluster
management.
State
is
all
in
the
same
place
with
the
rest
of
the
kubernetes
part
of
state,
so
that
that's
a
you
know
from
a
backup
and
be
our
story.
That's
a
that's
a
nice
property
as
well.
F
Yep
Assange
is
a
you
know,
one
of
the
open
questions
I
have
here
just
community
discussion
here.
You
know
like
we
so
the
local
file
and
block
persistent
volumes
and
the
updates
to
the
scheduler.
That
will
look
it
because
of
that
rights
are
basically
hey,
I'm,
going
to
go
into
cube
and
register
all
my
persistent
bodies
that
are
local,
whether
they
be
file
a
block,
so
that
they're,
you
know
discoverable
by
the
scheduler
and
then
basically
have
a
stateful
sentences.
F
Look
I'm
going
to
go,
deploy
some
sort
of
data
store
could
be,
Cassandra
could
be,
you
know,
room
could
be
cluster
or
whatever,
and
then
you
just
use
a
stateful
six
that
basically
consumes
from
a
storage
class
that
is
pointing
to
those
local
file
and
block
TVs
I.
Think
it's
kind
of
interesting
at
that
point.
F
What
beyond
the
monitoring
management
and
health
API
I
get
that's
different,
but
like
from
a
deployment
standpoint,
how
much
we
will
need
operators
going
forward
now,
because
the
stateful
circle
basically
handle
the
deploy
it
also
with
forgiveness,
and
things
I
can't
be
able
to
handle
low
recovery.
Things
like
that
I.
C
Strongly
believe
that
you
need
operators
for
storage
services,
I
mean
we're
building
operators
for
things
that
are
much
easier
services
like
even
you
know
there
is.
There
is
very
specific
business
logic.
That's
related
to
running
the
storage
service,
like
this
I
mean
think
about
doing
an
upgrade
I'll
give
you
that
that's
a
one
that
makes
a
lot
of
sense,
especially
an
upgrade
that
requires
a
disk
format.
Change
right,
you'd
want
a
sequence
that
such
that
you
want
to
minimize
downtime.
C
You
know
data
at
any
time,
because
I
don't
want
to
risk
having
you
know
any
kind
of
vulnerability
where
my
might
you
know,
I
brought
a
bunch
of
notes
down
and
then
I
have
a
failure,
and
then
you
know
I'm
now
looking
at
data
loss,
so
those
kinds
of
scenarios
not
going
to
logic,
just
as
an
example
for
upgrade
require
a
very
you
know,
smart
operator,
that
it
can
sequence
orchestrate
that
kind
of
that
kind
of
you
know
upgrade.
That's
not
something
that's
well
beyond
the
scope
of
say
what
the
stateful
chat
with
you.
C
I
mean
kubernetes
is
great.
It
has
a
bunch
of
controllers
that
try
to
you
know,
make
make
tasks
like
stateful
or
deployment
to
replica
sites
or
all
that
stuff
useful.
But
it
is
you
know
not
not
all
of
those
apply.
You
know
directly
and
are
a
good
fit
at
this
layer.
They
are
useful,
a
lower
layer
and
we,
the
rook
operator,
uses
them.
In
fact
we
use
things
like
recognizing
daemon
sets
and
all
that
stuff,
but
when
it's,
when
you
have
specific
logic,
that's
related
to
this,
your
service
and
I,
don't
see
a
way
around.
C
Writing
that
logic
inside
an
operator
and
I
storage
services
are
probably
the
hardest
ones
trauma.
You
know
if
you
look
at
a
because
they
because
they
are
stateful
and
because
they
require
and
they're
so
finicky
about
upgrade
path.
How
much
of
the
data
you're
touching
at
one
point
and
placement
of
things
in
all
of
that
stuff
is
really
complex
and
requires
custom
logic
beyond
just
primitives
built
into
kubernetes
another
good.
C
If
you
wanna
fall,
why
is
there
a
net
TV
operator
rightly,
and-
and
you
know,
the
reasoning
is
that
you
know
if
you
break
quorum
in
sed
you're
kind
of
screwed.
So
that's
not
something
that
a
stateful
set
understands
something
that
requires
probing
each
of
the
sed
leaders
and
making
sure
that
that
they,
you
know
they
you
maintain
quorum.
While
you
are
doing
an
upgrade,
that
problem
is
amplified
at
storage
service
in
excess,
so.
D
Surface
on
the
thick
one
of
the
things
I'm
thinking
about
here
is
that
there
is
essentially
a
control,
plane,
extraction
above
session
and
other
platforms,
and
it
looks
like
it's
actually,
four,
possibly
more
than
just
control
plane,
some
aspects
of
where
it's
going.
But
you
know
in
terms
of
operators,
wouldn't
it
make
more
sense
to
to
augment
the
existing
operator
interfaces
and
and
to
make
some
of
what
you're
describing
in
terms
of
what
these
like
common
needs,
are
for
for
managing
a
life
cycle.
D
C
That's
a
good
question,
I
think
so
it
will
actually
come
down
to
kubernetes
I
mean
if
there
are
some
patterns
that
emerge.
Where
say
you
know,
stateful
sets
can
do.
If
we
add
one
more
feature
into
staples
has
been.
They
could
do
something
that
a
bunch
of
operators
are
a
bunch
of
controllers,
can
can
leverage,
then
I
absolutely
think
we
should
push
them
down
into
the
kubernetes
layer
and
other
ISM
I.
Think
where
it
starts
to
diverge
is
when
you
have.
You
know
a
logic,
that's
specific
to
a
given
service
or
a
given.
C
You
know
app
if
you
will
and
that's
that
that
becomes
harder
to
do,
I
mean
and
that's
it
having
you
know
if
you,
if
you
think
that
layer
layering
them
there.
Hopefully
there
will
be
enough
primitive
the
lower
layer
to
kind
of
help,
building
help
build
the
layers
above
and
I.
Think
that's
already
the
case
right
now.
There's
definitely
room
for
iteration
and
improvement
here
and.
D
I
think
it's
an
interesting
area.
My
team
did
one
specific
Scalia,
which
is
the
value
of
C
software
platform
similar
to
set,
and
but
I
tend
to
be
how
this
does
start
to
you.
You
build
up
capabilities
and
maybe
build
up
for
a
few
different
platforms,
and
then
you
try
to
convert
it
under
one
project
or
interface
and
figure
out
if
there's
commonality
and
then
I
figure
out
how
to
drive
that
architecture
place
so
yeah.
C
C
Really
man
manpower,
you
know
skilled
folks
that
can
help
us.
Steph
is
actually
one
of
the
harder
ones,
I
think
because
chef,
it's
a
complicated
system,
it's
very
powerful
system,
but
it's
also
very
complicated.
I
I
think
doing
I
mean
we've
certainly
had
people
express
interest
in
in
other
systems.
You
know
even
things
like
HDFS
or
Gloucester
come
up.
There's
a
lot
of
patterns
here.
That
I
think
would
make
make
this
interesting
and
patterns
that
we've
already
addressed
with
stuff.
C
The
other
thing
I'll
say
just
on
that,
like
with
block
storage,
I,
think
you
know,
blog
storage
semantics
are
kind
of
easier
to
normalize.
You
know
you
need
to
take
snapshots
and
you
need
support
also
things
at
the
block
level,
but
when
it
comes
to
file
systems
and
actually
that
the
same
is
true
with
object
storage
as
long
as
it
talks
s3,
then
using
the
object,
storage-
and
you
know
it
has
a
bunch
of
facilities
with
file
systems.
People
are
quite
picky
about
which
costs
and
the
actual
management
and
semantics
are.
C
C
B
E
C
I
think,
if
you
think
about
you
know
the
way
that
rook
is
implemented.
Is
that
we're
not
actually
trying
to
abstract
all
the
different
storage
system
but
we're
trying
to
orchestrate
them
so
there's
not
a
uniform
interface
that
says:
let's
do
you
know
deployment
or
let's
do
that.
This
cluster
definition
applies
to
all
sources
and
in
fact
it's
the
exact
opposite
we're
saying:
let's
expose
what
the
actual
storage
service
wants
to
expose,
but
there's
no
unification
of
them.
C
What
is
interesting
is
using
common-law
patterns
and
common
code
to
help
deal
with
things
like
a
craze
and
versioning
lifecycle
management
and
all
of
that
for
the
different
store
system
that
there
is
commonality
there,
but
it's
not
an
abstraction
on
top
of
a
different
I
think
that
that
comes
across
as
a
difference,
I'll
also
say
we
really
only
focus
at
this
point
on
Seth
and
kubernetes.
So
so
it's
you
know
where
we're
being
asked
are
kind
of
going
in
a
and
support
for
more.
C
Abstraction
Paul
right
now
it's
the
intended
goal,
for
it
is
to
essentially
we're
working
with
whether
six
storage
on
this
essentially
create
a
volume
attachment
object
and
let
the
controller
deal
with
deploying
that
and
we're
hoping
I
know
sad
and
I
need
to
catch
up
on
this,
but
we're
hoping
that
there,
this
pattern
of
you
know
essentially
a
controller
centric
pattern
for
creating
attachments
and
how
it
relates
to
CSI
can
emerge
and
that
we
could
have.
We
can
have
some
good
discussions
around
it.
Okay,
thank
you.
C
I
mean
I,
have
a
couple
slides
left
that
otherwise,
so
this
is
a
community
slide.
I
think
you
guys
can
see
it.
Okay,
we're
we're
very
happy
with
attraction.
It's
getting
a
brook
as
Brooke
is
getting
a
ton
of
traction
right
now
and
we're
happy
with
it
and
we're
hopeful
that
that
you
know
you
had
people
on
this
this
this
sig
could
help
as
well
help
grow
it.
It's
an
open
source
project,
we'd
like
it
to
do
more.
C
So
please
come
help
us,
and
then
you
know
I
think
been
mentioned
earlier,
that
that
rookies
and
the
CNC
fr
are
kind
of
trying
to
figure
out.
This
is
a
good
fit
for
the
CNC
f.
You
know
the
rook
is
actually
quite
well
aligned
against.
It
is
you
know,
container
packages,
micro,
service,
oriented
and
dynamically
manage
to
itself,
but
I
also
had
a
lot
of
it's
very
complementary
to
other
seats
of
CNC
a
project.
C
So
it's
deeply
integrated
into
kubernetes
and
we
expose
per
media
standpoint
works
metrics
for
for
all
of
the
sewer
systems
or
the
destroyed
sets
from
this
case,
and
then
you
know
we
building
integrated
storage
into
the
cloud
native
stack
I.
Think
it's
going
to
unlock
a
bunch
of
powerful
scenarios
where
on
Federation,
Portability
and
multi
cloud.
So
we
think
this
is
a
good
fit,
not
that
I'm
not
going
any
other
questions
are
little
concerns,
thoughts.