►
From YouTube: Rook: Intro And Deep Dive With...- Travis Nielsen & Blaine Gardner, Alexander Trost, Satoru Takeuchi
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe in Amsterdam, The Netherlands from April 17-21, 2023. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Rook: Intro And Deep Dive With Ceph Storage - Travis Nielsen & Blaine Gardner, Red Hat; Alexander Trost, Koor Technologies; Satoru Takeuchi, Cybozu, Inc
Speakers: Alexander Trost, Travis Nielsen, Satoru Takeuchi, Blaine Gardner
The Rook project will be introduced to attendees of all levels and experience. Rook is an open source cloud-native storage operator for Kubernetes, providing the platform, framework, and support for Ceph to natively integrate with Kubernetes. A deep-dive will be presented for the Ceph storage provider to show how Rook provides stable block, shared file system, and object storage for your production data. The recent features in the v1.10 release will be covered to further illustrate how Rook continues to be enhanced for production environments. Rook was accepted as a graduated project by the Cloud Native Computing Foundation in October 2020.
A
B
Hi
everyone
I'm
Alexander
working
for
core
technology,
sync
I'm,
also
maintaining
off
The
Rook
project
and
yeah
we're
gonna
start
diving
into
what
Rook
is
storage
for
kubernetes
and
yeah
I
think
we
don't
necessarily
want
our
customers
data
start
on
like
floppy
disk
anymore.
So
why
don't
we
go
a
step
further
to
the
Future?
B
We
have
hard
drives.
We
have
ssds
either
by
fancy
PCI
connections
or
even
then
NVM
essds,
flash
based
memory
like
it's
all
much
much
faster
than
it
was
some
time
ago,
and
it's
not
just
faster,
but
it's
also
so
much
more
capacity
that
we
have
with
these
well
new
storage
types
and
quotes
that
we
have
nowadays
and
because
of
that
on
how
we
kind
of
already
started
rethinking
how
we
deploy
applications
with
like
containers
with
kubernetes
and
all
the
cool
new
projects
around,
especially
at
the
cncf.
B
B
Give
me
storage,
give
me
a
storage
volume
for
my
application,
so
basically
a
persistence
layer
of
some
sorts
for
your
application,
you're
going
to
kind
of
having
a
bad
time
with
having
storage
for
your
applications.
That's
where
Rook
is
coming
in.
So
if
we
kind
of
see
it
as
like
a
we
have
our
applications.
Our
containers
running
in
well
kubernetes
queuing
an
opener
like
there's
other,
like
just
also
a
bunch
of
distributions
of
kubernetes
out
there,
and
you
have
your
storage
so
far.
B
That's
one
of
the
great
things
that
we
have
to
mention
before
we
further
dive
into
what
Rook
is
now
because
it's
one
of
the
key
aspects
in
the
end
for
you
to
be
able
to
use
storage
in
form
of
kubernetes
native
objects
like
persistent
volume
claims
and
like
storage
classes
and
so
on.
So
what
do
we
have,
thanks
to
the
community
around
kubernetes,
the
great
ecosystem
with
kubernetes,
the
container
storage
interface
also
called
CSI
and
a
nutshell
CSI.
B
It's
basically
one
interface
for
the
storage
backend,
so,
for
example,
for
saf
and
like
the
container
orchestrator,
especially
again
container
orchestrators,
to
implement
that
a
user
can
easily
get
a
volume
of
storage
for
their
application
like
if
I'm
deploying
I
know,
we
can
argue
if
that's
that's
the
best
case,
but
if
you
want
to
deploy
a
mySQL
database
or
like
a
WordPress
as
well.
For
example,
you
need
some
storage.
You
need
some
storage
depending
on
how
you
deploy
it.
Four
like,
for
example.
B
Well,
the
database
files
are
like
the
PHP
files
for
the
WordPress
blog
for
all
that
this
CSI
interface
makes
it
quite
easy
for
applications
containers
to
at
least
from
kubernetes,
I,
just
say:
hey
I
need
50,
gigs
of
persistent
of
storage
of
a
volume.
It's
basically
you're
claiming
From
kubernetes
perspective
you're,
creating
a
persistent
volume
claim,
and
that
part
then
talks
with
the
storage
in
this
case.
B
In
this
example,
we
still
go
on
Seth
to
say:
hey,
please
create
a
volume
and
with
this
and
this
amount
of
storage-
and
that
is
also
one
of
the
points
where
Rook
is
coming
into-
Rook-
is
basically
a
self
operator
for
kubernetes.
It
takes
care
of
automating
the
deployment,
bootstrapping
configuration
and
upgrading
of
a
safe
cluster
and
the
other
components
of
such
a
safe
cluster
will
be
going
into
what
a
self-class
is
in
a
second.
B
So
right
now
just
bear
with
me,
and
your
operator
will
also
take
care
of
creating
everything
needed
for
storage
for
PVCs,
for
persistent
volume
claims
to
be
consumed
or
like
to
be
able
to
be
even
provisioned.
If
we
take
a
look
at
ceph
itself
there
as
a
storage
platform,
we
don't
just
have
one
type
of
storage.
B
B
That
is
always
this.
The
sign
out
State.
Let's
talk
about
a
bit
like
how
Rook
accomplishes
this
in
its
architecture,
how
it
how
it
looks,
for
example,
when
it's
run
on
a
kubernetes
cluster,
this
diagram
is
probably
gonna,
look
quite
overwhelming
for
anyone
that
hasn't
worked
with
Seth,
but
the
main
takeaways
here
on
that
with
these
three
different
types
of
storage
units,
again
block
storage
file,
system
files
and
just
call
the
follow,
prefer
calling
file,
system,
storage
and
object.
Storage
is
all
in
one
safe
cluster.
It's
all
possible
with
just
one
self-cluster.
B
The
main
point
is
that
this
complex
system
from
again
deployment
configuration
upgrading
management
management
and
even
the
components,
for
example,
like
an
object
storage,
if
you
even
want
to
use
objects
that
you
don't
have
to.
If
you
want
to
have
file
system
or
block
storage,
that
that
is
what
Rook,
The
Rook
safe
operator
is
taking
care
of,
observing
the
state
that
you
want
and
taking
care
to
ensure
that
it's
achieved
again
with
CSI
being
mentioned.
So
much
already.
B
We
have
ways
to
easily
consume
the
storage
from
for
Block
storage
using
the
ceph
CSI
RBD
driver
rados
block
device.
Let's
just
keep
it
at
that.
That's
for
Miss
block
storage
for
file
system
storage,
which
is
the
saf
CSI
CFS
driver,
basically,
which
is
well
taking
care
of
managing
the
volumes
for
a
file
system,
the
storage
and
provisioning
them.
So
you
don't
have
to
do
really
too
much.
It's
not
with
CSI.
B
B
If
you
have
a
Sav,
slash,
Rooks,
F
cluster
running.
All
these
components
in
the
end
are
taken
care
of
by
The
Rook
operator
through
the
means
of
the
kubernetes
API,
so
that,
if
anything
happens,
all
the
operator
does
in
the
end,
is,
is
used
to
kubernetes
API
to
make
sure
that
the
end
state
is
achieved
and
The
Rook
operator.
Then,
for
example,
will
also
talk,
obviously,
with
the
set
API
and
create
certain
certain
resources,
pools,
block
storage,
pools
file
systems
and
configure
them
there.
B
Accordingly,
as
an
example
for
like
this
desire
state
that
we
are
some
examples
for
that,
we
have
custom
object.
I've,
hopefully
previously
mentioned
it
already.
A
bit.
Custom
objects
is
also
one
of
the
things
an
operator
in
kubernetes
can
watch
on
and,
for
example,
The
Rook,
safe
operator
brings
a
custom
resource
called
ceph
cluster.
It's
not
that's,
not
all
of
them.
It's
just
the
one
that
we
take
as
an
example
here.
B
Even
with
this
small
example,
we
have
just
a
blob
of
yaml,
basically
which
says:
Hey
I
want
a
self
cluster
with
this,
and
this
version
this
isn't
this
certain
configuration
for
like
bare
data
is
stored,
and
so
on
and
there's
even
more
options
that
can
be
said
to,
for
example,
say
which
disks
should
be
used
in
which
server
or
even
if
all
servers
should
be
used.
All
all
devices
all
disks
in
the
servers-
and
this
is
exactly
the
desired
state
which
we
are
talking
about.
B
B
It's
even
without
companies.
This
great
wrong
set
folks,
but
even
combined
with
kubernetes,
it's
making
everything
even
greater
because
with
kubernetes
kind
of
being
this
abstraction
to
some
degree
of
a
hardware
or
even
your
Cloud.
This
whole
API
ecosystem.
That
kubernetes
has
enables
The
Rook
operator
to
easily
run
a
complex
storage
system
like
Seth,
and
now
Travis
will
walk
you
through
some
more
Self
Storage
related
features
and
what
features
we
have
worked
on
in
the
latest
releases.
Yeah
thanks.
A
Alexander
for
taking
us
through
the
introduction
to
Rook
what
it
is,
why
we're
doing
it
and
how
it's
useful
to
you
yeah.
We
we
love
Rook,
we
love
providing
this
this
platform,
so
you
can
have
storage
in
your
kubernetes
clusters.
So
let's
talk
in
more
details
now
dive
into
where
is
Rook
more
useful?
What
features
do
we
have?
Why
would
you
use
it?
First
of
all,
how
do
you
deploy
it
so
you've
seen
what
Alexander's
example
is
already
that
you
know
there
are
some
Yamaha
files
you
can
go,
create
those
specify
your
desired
state.
A
I'll
also
point
out
that
there
are
two
Helm
charts
that
we
have
one
chart
will
install
the
operator
and
all
the
related
resources
and
the
second
chart
will
install
all
the
Rooks
CRS.
So
you
can
create
a
cluster.
You
can
create
the
block,
pools
the
set
file
systems,
the
set
object,
storage,
all
of
those
types
of
resources
for
ceph
have
these
different
custom
resources.
So
that's
how
you
basically
deploy
the
cluster
now,
there's
lots
of
ways
to
configure
Rook.
A
Just
an
Alexander
show
just
a
few
of
those
different
options,
but
really
I
want
to
point
out
you
can
you
can
deploy
across
multiple
cluster
topologies?
You
can
deploy
it
where
there's
only
hosts
or
multiple
hosts.
You
can
deploy
in
a
Zone,
topology
or
there's
a
number
of
other
topologies
where
you
can
Define
racks
or
data
centers
or,
however,
your
topology
is
laid
out
and
then
ceph
will
make
sure
that
the
data
is
replicated
across
the
cluster
in
a
way
that
keeps
the
data
safe,
reduces
the
risk
of
of
data
loss.
A
You
know,
as
one
zone
goes
down.
The
other
zones,
for
example,
would
still
hold
the
data,
even
if
one
zone
goes
down,
and
this
helps
Seth
keep
the
data
highly
available
and
again
you
can
customize
it.
Whatever
your
topology
may
be.
Yeah
most
people
do
run
stuff,
I
would
say
in
a
bare
metal
environment
where
that's
where
the
main
scenario
was
when
we
started
on
the
Rook
project,
because
there
was
no
other
solution
for
on-prem
storage
to
really
interact
or
be
installed
with
kubernetes.
A
There
are
also
scenarios
that
are
common
for
running
stuff
in
a
cloud
environment
where,
first
of
all,
you
want
a
consistent
storage
platform
wherever
kubernetes
runs.
Why
should
you
have
to
use
one
platform
in
one
Cloud,
environment
or
in
bare
metal
like?
Let's
just
have
one
storage
platform
that
works
consistently?
A
Second,
there
are
shortcomings
that
cloud
providers
have
had
from
the
beginning
and
they
still
have
like
being
able
to
access
storage
across
azs
or
they
have
slow
failover
times,
and
because
you
want
seconds
of
failover
time
instead
of
minutes,
you
want
to
be
able
to
have
a
large
number
of
PVS.
Perhaps
if
you
have
many
small
volumes
or
need
small
volumes
reapplications,
you
know
some
storage
providers
may
have
a
limit
of
30
per
node
where,
with
with
Rook,
basically
there,
there
is
no
limit.
A
There's
a
you
can
have,
let's
say
a
thousand
PVS
per
node,
but
there
is
no
real
limit.
It's
just.
It
comes
down
to
resources.
There
are
also
maybe
characteristics
of
large
volumes
that
you
want,
where
you
provision
large
volumes
in
the
cloud
provider
and
then
Seth
allows
you
to
have
smaller
volumes
on
top
of
those.
Those
larger
volumes
and
your
applications
don't
know
the
difference,
and
finally,
you
know
Seth
monitors
and
osds.
A
They
will
run
in
a
cloud
environment
backed
by
PVCs,
so
there's
no
need
for
direct
access
to
local
devices,
so
it
just
works
seamlessly
in
those
Cloud
environments,
next
scenario
commonly
used.
So
some
people
have
CEF
clusters
that
are
already
running
outside
of
kubernetes
and
they
want
to
connect
their
kubernetes
applications
to
those
to
that
stuff
cluster
and
that's
perfectly
common
and
valid
scenario.
A
Where
you
take
the
ceph
information
from
that
existing
ceph
cluster,
the
Seth
mons,
the
set
key
ring
cluster
fsid,
a
few
other
items,
and
then
you
import
those
Into,
The,
Rook,
cluster
and
Rook
will
then
configure
basically
Rook
configures
the
CSI
driver
to
connect
to
that
sap
storage
to
just
make
it
seamlessly
work
with
your
kubernetes
applications
and
that
CSI
driver
already
mentioned
again.
It
has.
You
know
it
has
many
features
of
standard
CSI
drivers
that
you're
familiar
with
it
is
very
flexible
and
has
many
features.
A
So
you
can
support,
read
write
once
volumes
with
step.
Rdb
RBD
sffs
supports
read,
write
many
volumes,
it
does
snapshots
and
clones
volume,
encryption,
volume,
expansion,
ephemeral
volumes
and
all
the
the
features
that
you
would
really
expect
from
a
storage
platform,
and
this
is
just
a
few
of
the
features
that
it
has
next
feature.
Alexander's
already
touched
on
this,
so
if
you
want
object,
storage
with
an
S3
endpoint
except
provides
that
with
frgw
and
the
way
you
do
this,
is
you
define
a
storage
class
for
your
object?
Storage
and
you
say:
wait.
A
Storage
class
is
usually
just
for
volumes
that
you
attach
to
your
pod.
Well,
the
way
Rick
has
implemented
this.
The
storage
class
allows
you
to
provision
a
bucket
with
an
object
bucket
claim.
The
operator
creates
this
bucket
when
you
create
this
claim,
which
is
a
very
similar
pattern
to
PVCs.
But
it's
for
object,
storage
that
lets.
You
have
access
to
a
bucket,
a
KMS
encryption.
So
you
want
your
data.
If
you
want
your
data
encrypted
at
rest,
okay,
Mass
encryption
allows
you
to
do
this.
A
We
we
support
multiple
backends
for
encrypting,
storing
the
encrypted
keys,
KMS,
hashicorp,
IBM
key,
protect
kmib
and
we're
working
on
adding
support
for
others,
so
that
you
can
ensure
that
your
data
is
is
secure
and
encrypted
at
rest
stretch
cluster.
So
in
some
scenarios
you
really
only
have
two
different
data,
centers
or
two
zones
where
you
have
storage
available
in
this
configuration
staff
still
needs
a
third
node
or
a
self-monitor
that
acts
as
the
tiebreaker
between
the
two.
So
this
this
third
node
is
needed
somewhere
else
to
work
with
the
other
two
zones
But.
A
Ultimately,
the
storage
is
only
found
in
two
of
the
zones.
You've
got
replicas
in
each
of
the
two
zones
to
provide
greater
resiliency
data
mirroring
for
any
real
application
that
needs
to
replicate
its
data.
It
needs
to
ensure
that
that
data
is
also
available
for
applications
that
need
to
fail
over.
When
you
know
some
cluster
goes
down
say
a
whole
kubernetes
cluster
goes
down.
If
you've
mirrored
your
data
to
another
kubernetes
cluster
with
Seth
as
long
as
you've
implemented
Dr
for
your
application.
At
that
level,
you
can
switch
it
over.
A
You
can
make
your
application
switch
over
its
active
configuration
and
run
completely
from
the
other
cluster,
because
the
data
is
mirrored
over
between
the
the
Clusters.
All
three
types
of
data
can
be
married
with
Seth
we've
got
the
block
and
the
file
system
and
object
with
rgw
multi-site.
All
of
those
can
be
mirrored
using
asynchronous
asynchronous
replication.
Again
it's
for
enabling
Dr
scenarios.
You
really
want
resiliency
in
your
applications
for
disaster
recovery.
A
So
let's
talk
about
some
of
the
recent
features
that
we've
added
really
in
the
last
few
months:
1.9
and
1.10,
but
we've
put
some
effort
into
our
crew
plugin
there.
So
this
is
a
tool
it
runs
outside
the
operator.
You
may
be
familiar
with
other
crew
plugins
that
are
available
in
the
community,
so
this
is
one
that
will
help
you
administer
these
one-off
scenarios
and
Rook
that
the
operator
just
doesn't
handle
directly.
So
you
can
do
things
like
print
out
the
status
of
your
cluster.
You
can
show
the
help
of
the
cluster.
A
If
you
need
to
perform
really
Advanced
operations
on
the
ceph
Monza
osds,
you
can
start
what's
called
a
debug
pod.
If
you
need
to
get
rid
of
an
OSD,
you
can
Purge
it
and
then
other
simple
things
like
just
restart
the
operator.
So
we're
continuing
to
improve
upon
this
plugin
and
add
new
features.
It's
really
going
to
help
with
Disaster
Recovery
scenarios
again
and
things
where
the
operator
really
just
doesn't
cover
the
operator
covers
scenarios
where
it's
constantly
running
and
monitoring
desired
state,
but
sometimes
disasters
happening.
A
A
We
did
a
documentation
facelift,
so
we're
just
happy
to
have
an
updated
framework
with
make
docs
I
need
to
look
and
feel
it's
searchable,
we'll
continue
to
add
content,
and
if
you
have
any
feedback,
we'd
love
to
hear
your
feedback
on
the
documentation
as
well
as
always
Seth
is
making
updates
to
the
data
layer,
and
our
goal
is
to
continually
add
support
for
what
Seth
is
doing.
So.
We've
added
support
this
year
for
Seth
Quincy,
which
came
out
earlier
this
year
and
then
support
for
staff.
Octopus
was
also
dropped
by
stuff.
A
It
reached
end
of
life,
and
so
now
we
dropped
support
for
that
and
Rook.
So
the
latest
releases
of
Staff,
Pacific
and
Quincy
are
supported
in
the
latest
work
releases.
Some
of
the
CSI
improvements
that
we've
made
in
the
1.10
release
with
the
latest
CSI
driver.
There's
k-map
integration
for
you
know:
encrypting
your
volumes,
RBD
volumes
for
Block
storage,
there's
NFS
support
for
snapshots,
restoring
clone
and
volume
expansion,
there's
PB
and
snapshot
metadata.
A
So
if
you
want
to
know
more
about
your
PBS
that
the
metadata
is
attached
to
the
PVS,
there
is
shallow
read-only
support
without
cloning.
The
underlying
snapshot.
Nfs
has
been
a
an
area
of
focus
for
the
team
as
well.
Adding
more
support
for
Enterprise
NFS
features,
so
you
can
connect
generally
from
the
outside
of
of
the
kubernetes
cluster
to
the
Self
Storage
inside
kubernetes,
for
example,
adding
client
ID
management
via
sssd,
with
ldap
Kerberos
client
authentication,
support
again,
yeah
really
supported
for
external
clients
for
now
and
we're
yeah.
A
So
a
brief
thought
on
road
map
and
where
we're
going
with
Rook
again,
our
goals
are
always
to
really
support
new
kubernetes
features
when
they
come
out
to
make
sure
ceph
is
always
managed
in
a
very
robust
and
and
thorough
way.
So
we
need
to
support
set
features
when
new
set
features
come
out,
we
make
sure
they're
exposed
and
configurable
through
rook
in
a
simple
way
with
crds.
Finally,
we
it's
always
a
goal
of
ours
to
respond
to
community
input.
A
We
want
to
know
what
storage
needs
you
have
and
how
we
can
improve
and
just
make
sure
your
your
needs
are
heard
or
as
a
cncf
graduated
project.
We
really
want
to
make
sure
we're
doing
what
the
community
needs
needs
to
be
done,
we're
here
to
to
respond
and
hope.
You
feel
that
response
as
you
come
to
our
our
slack
and
GitHub
issues
and
in
other
places
for
feedback.
Some
of
the
features
areas
we're
working
on.
You
may
have
heard
of
qazi
the
object,
storage
interface
for
containers.
A
It's
going
to
kind
of
take
the
place
of
obc's,
eventually
it'll
be
a
while
out,
because
Kazi
is
still
so
new.
It's
just
an
alpha.
It
kubernetes
1.25,
so
we'll
have
an
implementation
of
that
soon
for
Rook
other
areas
encryption.
So
we
do
have
encryption
across
the
wire,
for
example,
but
also
other
types
of
encryption
and
other
places
to
store
the
encryption
Keys.
Depending
on
the
the
KMS.
We've
got
Disaster
Recovery
scenarios
and
lots
more
there's
always
more
to
do
in
the
storage
room.
A
I
have
to
get
involved.
We
hope
you
can
visit
The
Rook
booth
at
kubecon,
while
you're
here
and
if
you
know,
visit
our
Rook
GitHub
visit.
Our
book
slack
Twitter
follow
us
on
Twitter
and
if
you'd
like
to
talk
to
us
and
don't
find
us
at
The,
Rook
Booth,
you
can
join
us
in
our
community
meetings
on
a
bi-weekly
basis.
We've
got
a
link
to
that
on
the
main
GitHub,
so
we
hope
to
hope
to
meet.
You
hope
to
hear
from
you
and
thank
you.