►
From YouTube: Rook: Intro and Deep Dive with Ce... Travis Nielsen, Sebastien Han & Blaine Gardner, Satoru Takeuchi
Description
Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Rook: Intro and Deep Dive with Ceph Storage - Travis Nielsen, Sebastien Han & Blaine Gardner, Red Hat; Satoru Takeuchi, Cybozu, Inc.
The Rook project will be introduced to attendees of all levels and experience. Rook is an open source cloud-native storage operator for Kubernetes, providing the platform, framework, and support for Ceph to natively integrate with Kubernetes. A deep-dive will be presented for the Ceph storage provider to show how Rook provides stable block, shared file system, and object storage for your production data. The recent features in the v1.9 release will be covered to further illustrate how Rook continues to be enhanced for production environments. Rook was accepted as a graduated project by the Cloud Native Computing Foundation in October 2020.
A
Hello,
welcome
to
the
rook
intro
nsf
deep
dive.
I
am
travis
nielsen
one
of
the
rook
maintainers
and
I
work
for
red
hat.
Let's
get
going
all
right
for
our
agenda
today,
we're
going
to
start
off
by
talking
about
what
are
the
storage
challenges
that
you
might
have
with
kubernetes,
then
what
is
rook?
What
is
sef
just
cover
some
basics
and
then
we'll
get
into
the
key
features
of
what
rook
provides.
A
What
new
features
are
in
our
latest
1.9
release
that
just
came
out
in
april
and
we'll
get
into
a
demo
that
blaine
will
give.
If
you
have
questions
we'll
should
have
some
time
at
the
end
to
answer
some
of
those
okay.
So
what
are
your
storage
challenges?
Kubernetes
really
was
a
platform
built
to
manage
sort
distributed,
applications,
ideally
stateless,
and
if
they
need
storage.
A
Well,
storage
is
kind
of
an
afterthought.
There's
a
way
to
plug
in
storage
to
kubernetes.
But
storage
is
not
a
native
component
of
kubernetes.
So
you,
if
you
rely
on
external
storage,
it's
either
not
portable.
Maybe
it's
a
burden
to
deploy
it.
If
you're
in
a
cloud
provider,
you
know
you're
locked
into
that
that
vendor,
so
how
do
you
get
out
of
that
vendor
locking?
So
some
of
those
questions
is
where
we
started
with
rook
and
we
wanted
to
really
bring
storage
to
kubernetes
in
a
native
way.
So
that
brings
us
to
the
question.
A
What
really
basically
does
provide
so
rook
brings
storage
inside
your
kubernetes
cluster
and
manages
it
for
you.
It
makes
it
so.
Your
applications
can
consume
storage,
just
like
any
other
kubernetes
storage
with
storage
classes
and
persistent
volume
claims.
The
way
we
do.
This
is
with
kubernetes
recovering
operator
and
crds.
So
you
tell
rook
how
you
want
to
manage
steph
and
the
storage
layer,
and
then
rook
will
go
automate
that
for
you,
the
work
will
deploy
configure
upgrade
step
for
you.
So
you
don't
have
to
worry
about
all
the
details
of
managing
that
store,
storage.
A
Layer.
Rookie
is
open,
source
apache
2.0
license
and
we
just
try
to
have
an
open
community.
We
want
to
do
what's
best
for
the
community
as
you're
getting
started
with
rook.
We
want
to
make
sure
you
know
about
all
the
resources
that
we
have
on
our
website.
Documentation
join
our
slack
for
questions
at
the
bottom
of
the
slide.
What
we
want
to
point
out
is
we're
just
starting
to
release
some
new
training
videos,
I'm
getting
started
with
some
basic
concepts
with
brooke
on
cubebuyexample.com,
so
check
it
out,
and
I
hope
it's
useful
for
you
now.
A
A
It
provides
three
types
of
storage,
so
block
storage,
a
shared
file
system,
where
you
need
to
share
storage
among
pods
and
or
an
object,
storage,
that's
s3,
compliant
seth,
favors
consistency,
and
it
was
first
released
in
july
2012.
So
we're
coming
up
on
the
10th
anniversary.
Since
seth
has
been
running
in
production
storage
clusters,
we're
really
excited
about
that
anniversary.
A
So
what
basic
architecture
layers
does
rook
have
so
rook
again
really
deploys
and
manages
everything
you
need
for
storage
with
seth,
so
the
ceph
csi
driver.
Now
it
will
provide
that
plug-in
layer
with
kubernetes,
so
you
can
provision
and
mount
the
stuff
storage
into
your
application.
Pods,
just
like
you
do
with
any
other
csi
driver
and
then
ceph
itself
provides
the
data
layer
really
so
when
you're
writing
reading
and
writing
data,
it's
going
to
go
straight
to
that
data
layer
for
optimal
performance.
A
So
here's
a
view
of
what
management
looks
like
from
work's
perspective
and
kind
of
looking
at
what
pods
are
created.
So
there
are
a
lot
of
pods
that
need
to
be
created.
But
again
you
don't
have
to
worry
about
this
recreates
all
of
them.
So
you
start
with
the
rook
operator
and
you
tell
rook.
I
want
to
deploy
seth,
and
this
is
how
I
want
it
configured
so
then
rook
goes
and
starts
the
you
know
sf
bond
pod
on
each
of
three
nodes.
A
Then
it
will
go.
Look
for
all
of
the
devices.
So
say
you
have
devices
on
your
your
nodes
that
you
just
want
to
consume
as
storage
in
your
data
center
rook
will
create
osds
on
each
of
those
devices.
Osd
is
the
fundamental
storage
component
of
sev.
So
all
these
red
pods
basically
are
ceph
demons
and
the
green
pods
are
really
csi
driver
pods
that
help
you
then
provision
and
attach
the
storage.
A
So
that's
in
a
nutshell,
kind
of
idea
of
what
pods
you're
going
to
get
with
rook,
and
so
now,
when
your
application's
ready
to
go,
consume
provision
that
storage
we've
got
this
picture
here
of
three
different
types
of
storage.
So
here
on
the
left
we
start
with.
If
your
app
needs
block
storage,
you're
going
to
create
a
read,
write
once
volume
claim,
you
define
a
storage
class
which
uses
ceph
rbd
and
then
the
csi
driver
for
ceph
is
going
to
create
one
of
these
ceph
rbd
volumes
and
attach
it
to
your
apple,
your
application,
pod.
A
A
It
uses
the
staff
ffs
storage
class,
which
allows
for
the
sharing
of
the
pod
or
the
volume,
and
then
it
mounts
it
with
the
sepsiet
seppfest
driver
all
right,
so
that's
file,
storage
and
then,
if
you
have
an
application
that
needs
to
use
the
object,
storage
rest
endpoint
with
an
s3
api.
Well,
we
have
the
we
kind
of
follow
the
same
pattern
that
you
see
with
pbc's,
so
you
can
create
a
bucket
claim
to
say
I
want
a
bucket
to
read
and
write
data
to.
A
A
A
Just
like
you
write
to
any
other
stored,
local
storage
that
write
or
read
will
go
through
the
ceph
rbd
kernel
driver,
which
will
then
talk
to
the
ceph
daemons
that
are
running
in
your
cluster,
and
you
don't
need
to
know
anything
about
where
they're
running
how
they're
running
seth
just
takes
care
of
that
for
you,
no
matter
what
type
of
storage,
whether
it's
the
block
file
or
object,
storage
with
the
s3
client.
Those
clients
know
how
to
connect
to
this
storage.
A
Okay,
let's
get
into
some
of
the
basic
features
that
rook
brings
to
the
table
with
seth.
So
first
of
all,
installing
stuff
in
kubernetes
is
is
simple.
We've
made
it
as
simple.
That's
been
one
of
our
goals
from
the
start
of
the
project,
how
to
make
ceph
deployable
for
the
majority
of
clusters
without
having
too
much
management
overhead
we've
got
some
sample
yaml
files,
some
manifests
that
you
just
say
go
create
all
of
those
and
we'll
create
and
configure
ceph
as
desired.
The
last
one
is
cluster.yaml
is
the
one
that's
pictured
here
on
the
right?
A
That's
where
you
tell
rook
what
settings
you
want
to
deploy
with
like
how
many
mons
and
how
to
consume
the
storage
that
rooks
finds
on
those
nodes
to
create
that
storage,
cluster.
Okay,
we
also
have
helm
charts
that
make
this
deployment
even
simpler
for
those
who
are
using
helm
all
right,
so
the
csi
driver
brings
a
number
of
features,
standard,
csi
features,
we've
got
dynamic,
provisioning
for
flock
and
file
storage,
we've
got
volume,
expansion,
snapshots
and
clones
and
there's
this
enables
all
sorts
of
applications
to
do.
A
Failover
work
across
data
centers
work
across
clusters
mirroring
and
a
number
of
scenarios.
So
what
environments
can
you
deploy?
Rook
in
so
first
of
all,
primarily
where
we
started
was
in
bare
metal.
You've
got
your
own
hardware
and
on
bare
metal
you
don't
even
have
storage
options
unless
you
plug
in
some
external
appliance.
A
So,
let's
talk
a
little
bit
more
about
that.
The
rook
in
the
cloud
environment
can
overcome
shortcomings.
So
some
of
the
shortcomings
include
the
cloud
provider.
Might
only
limit
storage
to
inside
a
single
availability
zone.
Well,
rook
can
span
az's
bailover
times
can
be
long
in
the
in
cloud
providers,
so
you
can
fall
over
in
seconds.
Instead
of
minutes,
you
can
have
more
basically
unlimited
number
of
pbs
per
node.
A
B
A
A
Even
with
ceph
major
upgrades
that
update
is
automatic
automatic,
our
upgrade
guide
can
look
a
little
scary
kind
of
a
long
document,
but
really
it's
just
trying
to
be
extra,
careful
make
sure
you
feel
confident
knowing
your
cluster
is
healthy
before
during
and
after
you're
upgrading
another
key
feature
of
rook
is
that
the
csi
driver
allows
you
to
connect
to
ceph.
That
is
running
external
to
your
kubernetes
cluster.
A
Maybe
you
already
have
a
step
cluster
running,
or
maybe
you
just
really
don't
want
to
run
your
storage
on
the
same
hardware,
with
kubernetes
or
in
the
same
environment,
so
you
can
run
ceph
independently,
deploy
usually
with
sep
adm
and
then
connect
your
cluster.
To
that
I
did
mention
briefly
already
using
buckets
with
object,
storage,
the
obcs,
the
object
bucket
claims
allow
you
to
provision
buckets
easily
and
you
know
we're
looking
forward
to
the
quasi
the
container
object,
storage
interface
that
will
be
coming
with
the
kubernetes
enhancement
soon.
A
Alright,
so
rook
1.9
was
just
released
in
april.
Let's
talk
about
some
of
those
new
features
that
are
just
out.
First
of
all,
seth
quincy
is
the
latest
major
release
of
seth
b17
that
was
just
released
in
april
as
well
on
their
annual
release
cycle.
So
with
quincy
comes
the
latest
and
greatest
storage
layer.
So
we
don't
get
into
what
those
features
all
are.
What
rook
does
support
that
latest
major
release?
A
A
Now,
with
the
csi
driver,
a
csi
driver
will
provision
them
and
then
the
kubernetes
nfs
will
well
mount
them
for
you
and
that
community
nfs
driver
is
available
today
and
our
documentation
and
brooke
explains
how
to
work
through
that
and
get
that
working
in
this
release.
We
do
have
a
new
crd
for
creating
rados
name
spaces,
so
a
rados
namespace
is
a
concept
concept
that
really
gives
you
isolation
and
multi-tenancy
so
that
you
don't
need
to
create
separate
pools.
A
A
pool
is
kind
of
a
large
entity
inside
ceph,
and
so
this
just
gives
you
isolation
within
pools.
Some
network
features
so
on
what
happens
on
the
wire
with
set
communication,
so
you
can
have
encryption
on
the
wire
now
and
compression
on
the
wire
as
well.
They
do
require
a
recent
kernel,
5.11
and
then
there's
much
more.
Of
course,
lots
of
fixes
with
each
each
update
to
rook
emission
controller,
for
example,
is
enabled
by
default.
A
If
we
find
the
cert
manager
is
available,
we
support
multis
networking
now
with
our
latest
release
and
then
updated
prometheus
alerts
and
much
more
and
those
alerts
will
really
help
you
make
sure
staffed
on
your
storage.
Cluster
is
healthy.
All
right
now,
we'll
turn
the
time
over
to
blaine
for
a
demo.
B
This
is
on
amazon
web
services,
I'm
going
to
have
three
control
nodes
for
kubernetes
and
three
worker
nodes.
I've
chosen
to
use
m5.8x
large
nodes
here.
This
will
allow
us
to
run
the
final
size
of
the
stuff
cluster
we're
going
to
get
to
as
well
as
having
about
50
of
the
node
left
over
for
user
applications,
which
is
just
kind
of
a
rough
estimate,
I'm
going
to
be
using
slow,
but
pretty
cost
effective,
gp2
volumes,
and
this
is
using
what
is
at
the
time
I'm
recording
this.
B
B
These
might
be
dynamic
from
like
gp2
today,
or
these
might
be
local
persistent
volumes
that
you've
created
yourself
that
represent
physical
disks
on
the
hardware
but
are
claimed
via
kubernetes
native
mechanisms
for
the
host-based
cluster.
This
is
suitable
for
a
simple
cluster,
especially
for
proof
of
concept
clusters.
B
The
pvc
based
cluster
on
the
surface
seems
a
little
more
complicated,
but
we
don't
need
to
describe
hardware
configuration
and
it
becomes
pretty
easy
to
expand.
We
can
increase
the
count
of
disks
that
we
use
for
stuff
osds
or
we
can
increase
the
size
of
those
disks
by
increasing
the
resources.size
parameter
here
in
the
storage
class
device
set.
B
B
B
Once
that's
done,
we
can
create
the
operator
itself
because
I'm
running
an
open
shift.
I'm
running
the
openshift
flavor
of
this
operator,
although,
if
you're
running
on
normal
kubernetes,
just
operator.yaml,
is
the
one
you
want
and
we'll
see
that
it
gets
scheduled
and
it
starts
running
pretty
soon
after
here
on
the
left
side
from
here,
I'm
going
to
go
ahead
and
start
cluster
installation.
B
B
All
right,
a
little
over
10
minutes
has
passed.
Most
of
this
time
was
spent
setting
up
the
rick
monitors.
This
seems
to
be
a
result
of
the
pvc
driver
for
gp2
being
a
little
slow
so
from
here.
Let's
look
at
the
crew
plug-in
and
how
that
gets
used,
the
first
thing
being
to
install
the
rucksack
crew
plugin.
So
that's
crew,
install
rxf.
B
B
B
B
Each
of
our
osd's
is
10
gigabytes.
So
this
is
really
what
we
expect.
I
also
want
to
talk
about
a
ceph
command.
That's
a
little
bit
advanced
called
set
osd
tree,
and
this
is
going
to
show
us
the
hierarchy
of
ceph.
Osb
says
ceph
understands
it,
so
we
can
see
that
we
have
osds
running
in
one
region.
This
is
u.s
west
one
and
within
this
region
we
have
two
zones
being
west,
one
b
and
west
one
c.
B
B
At
some
point,
we
will
reach
the
iops
limit
of
these
gp2
volumes
with
scale
up
increasing
them
in
size.
More
won't
really
get
us
more
iops,
so
we
may
want
to
scale
out
the
cluster
instead
to
create
more
osds,
and
this
will
effectively
allow
us
to
get
better
performance
for
the
size,
we're
increasing
again
we're
going
to
want
to
edit
a
cluster
manifest
and
here,
instead
of
changing
the
size,
we're
going
to
be
changing
the
count.
B
So
I'm
going
to
change
this
from
3
to
6,
which
is
doubling
the
number
of
osds
and
should
double
the
available
capacity
to
600
gigabytes.
Once
I
apply
this
to
look
at
this
visually,
we
are
adding
three
new
osd's
here
in
blue
and
rook
is
going
to
create
these
and
try
to
spread
them
evenly
across
the
node.
So
we
should
see
one
extra
osd
on
each
node
running
skipping
forward
a
few
minutes
until
when
that's
done.
B
I
do
want
to
make
one
small
note
with
this
scale
out.
I
talked
about
potentially
doing
this
for
performance
reasons.
Another
option
is
to
create
a
new
storage
glass
device
set
rather
than
expanding
an
existing
one
with
a
new
storage
class
device
set.
I
might
use
a
different
and
faster
storage
like
io2
instead
of
gp2,
and
I
could
provide
this
as
a
backing
pool
for
faster
storage
for
some
user
applications.
A
All
right,
thanks
blaine
for
that
demo.
Again,
we
refer
you
back
to
all
these
resources.
We
have
for
getting
started
with
rook
the
website.
The
docs.
Please
join
our
slack,
it's
a
great
place
for
asking
questions
or
go
to
our
github
join
our
community
meeting.
If
you
have,
if
you
want
to
talk
to
us
over
the
call
and
again
check
out
our
training
videos
to
get
started
thanks
for.