►
From YouTube: Rook: Intro and Ceph Deep Dive - Blaine Gardner, Alexander Trost, & Travis Nielsen, Sébastien Han
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2021 Virtual from May 4–7, 2021. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Rook: Intro and Ceph Deep Dive - Blaine Gardner, SUSE, Alexander Trost, Cloudical, & Travis Nielsen, Sébastien Han, Red Hat
The Rook project will be introduced to attendees of all levels and experience. Rook is an open source cloud-native storage orchestrator for Kubernetes, providing the platform, framework, and support for a diverse set of storage solutions to natively integrate with cloud-native environments. Rook turns storage software into self-managing, self-scaling, and self-healing storage services. The benefits and use cases of Rook will be explored along with an overview of each of the Rook storage providers: Ceph, Cassandra, NFS, EdgeFS, YugabyteDB, and CockroachDB. In the latter half of the talk, a deep-dive will be presented for the Ceph storage provider. Rook is run today in many production environments, providing a stable storage platform for your data. The architecture and recent improvements will be show how Rook provides the Ceph management layer for production environments. Rook was accepted as the first storage project hosted by the Cloud Native Computing Foundation in January 2018.
https://sched.co/ekIQ
A
A
A
B
A
Here
so
rook
did
get
initially
inducted
as
a
sandbox
project.
Back
almost
three
years
ago
in
january
2018
we
progressed
to
incubation
in
september
of
that
same
year
and
then
graduated
now
in
october,
just
just
this
last
month.
So
thank
you,
a
big
thank
you
to
the
rook
community,
cncf
committee,
rook
maintainers,
and
everyone
involved.
It
has
been
a
wonderful
journey
and
we're
grateful
your
support
for
your
support
and
look
forward
to
continued
growth
in
the
project
with
that
support.
B
A
Lot
of
large-scale
production
deployments
out
there,
we
know
people
are
using
it
to
store
their
their
critical
data
in
production
environments,
and
this
is
that's
a
great
evidence
of
that.
We
went
through
a
security
review
by
the
third
party
that
cncf
helped
us
coordinate,
and
maybe
the
third
point
is
that
really
brooke
is
an
open
project.
It's
it's
not
only
open
source,
but
the
governance
around
the
project
says
that
hey
this
is
this:
is
vendor
neutral?
A
We
have
multiple
companies
contributing
and
expected
to
contribute
so
that
it
stays
remains
a
project
that
is
useful
to
the
whole
community
and
and
not
vendor,
driven
so
again,
thanks
to
for
all
your
support,
we've
had
a
great
journey
to
get
here.
All
right.
Let's
get
started
on,
you
know
what
what
are
we
talking
about?
What
is
rook?
What
does
it
address?
What
what
are
issues
with
storage
and
kubernetes?
A
A
A
Okay,
you
have
to
you
know,
day
two
operations
at
once:
you've
set
it
up,
you've
got
to
upgrade
it
you've
got
to
manage
it,
make
sure
it
stays
reliable.
So
all
of
those
are
challenges
that
that
everyone
has
with
any
stateful
application
in
kubernetes.
So
here's
where
rook
comes
in.
We
saw
this
as
a
real
challenge
with
kubernetes
and
we
wanted
to
address
it
and
kind
of
solve
this
problem
for
the
community.
So
rook
really
does
have
the
goal
of
making
storage
available
inside
your
kubernetes
cluster.
A
It's
not
something
external.
It's
something!
That's
deployed
as
any
other
application
in
your
kubernetes
cluster.
It's
managed
with
kubernetes
operators
and
crds,
just
like
any
other
application,
and
because
it's
driven
with
operators,
the
deployment
of
of
the
storage
platform
is
automated,
so
deploying
it
configuring
it
upgrading
it.
You
you
get
to
decide
how
it's
deployed
through
the
crds
and
other
settings,
but
ultimately
rook
manages
all
of
the
headaches
of
of
that
day.
A
A
A
A
These
are
still
in
alpha
and
we're
always
looking
for
community
involvement
to
help
grow.
Those
we
do
have
one
storage
provider
that
is
deprecated
so
edge
of
s.
The
the
owners
of
edge
of
fest
are
working
on
a
replacement
for
it.
The
timeline
of
that
I
I'm
not
aware
of,
but
that's
why
it's
deprecated,
since
something
new
will
be
coming
all
right,
so
kind
of
overall
project.
A
A
You
know,
as
far
as
how
popular
project
is,
you
know
all
the
github
stars
downloads,
going
on
200
million
downloads
now
for
our
containers
about
300
contributors.
Now
on
on
the
github
project,
and
as
mentioned
we
are
a
graduated
project,
so
thanks
again
to
the
the
community,
these
are
just
a
few
of
the
stats
that
show
how
how
much
it
is
growing,
okay,
so
rook
and
sef.
If
we're
getting
into
the
ceph
portion
of
this.
What
does
that
mean
to
bring
ceph
into
kubernetes?
A
That's
what
rick
will
do
rook
will
bring
the
ceph
storage
layer
into
kubernetes
and
make
it
work
well.
So
what
is
steph
if
you're
not
familiar
with
it?
Seth
is
another
open
source
project,
but
it
is
a
software
defined
storage
solution.
It
provides
block
shared
file
system
and
object.
Storage,
that
is
s3
compliant.
A
So
these
are
the
really
the
three
building
blocks
of
any
storage
system
that
you'll
find
out
there
and
ceph
has
them
all
in
one
in
one
system,
no
need
for
deploying
one
solution
for
block
and
another
one
for
object
or
file
system.
The
ceph
provides
all
of
those
types
of
storage
together
so
early
on
the
project.
We
really
saw
ceph
as
being
a
reliable
platform.
It's
been
around
for
a
long
time.
We
just
wanted
to
bring
it
to
kubernetes.
So
how
did
we
bring
it
to
kubernetes?
A
So
rook
takes
a
lot
of
that
complexity
out
of
it
and
manages
it
for
you
second
layer
now,
we've
got
the
ceph
csi
driver,
so
this
csi
driver,
just
like
any
other
storage
platform
kubernetes,
will
provision
and
then
mount
the
storage
into
your
app
pod.
So
this
is
how
you
consume
this:
the
ceph
storage,
once
you
have
rook
installed
and
then
finally,
the
third
layer
is
the
data
layer.
So
when
you
are
reading
and
writing
data
to
the
cluster
ceph
is
purely
the
one
that
is
responding
to
that.
B
A
There
is
no
rook
management
code
in
the
data
path.
It's
just
ceph
at
the
data
layer,
so
you
really
get
performance,
high
performance
storage
at
that
layer
all
right.
So
let's
look
at
some
pictures
of
what
these
layers
look
like
so
for
the
first
layer
with
rook.
So
this
this
is
a
picture
trying
to
show
that
okay,
we've
got
a
single
cluster
with
three
nodes,
so
each
of
these
black
boxes
is
is
a
node
in
your
kubernetes
cluster,
and
each
of
these
nodes
is
running
pods
that
run
different
types
of
demons.
A
So
in
the
center
center
node
here
we've
got
the
rook
operator
pod
and,
of
course,
that
that's
the
management
layer
that's
going
to
manage
everything
else,
so
this
operator
pod,
is,
is
going
to
deploy,
ceph
and
and
the
csi
driver
depending
on
what
settings
and
how,
how
you
tell
it
to
be
configured
okay,
so
the
these
blue
pods
are
really
core
rook
pods
that
are
providing
the
management
layer.
The
discovery
pod
is
discovering
what
storage
is
available
on
each
node,
so
you
so
rook
knows
how
to
deploy
the
sep
demons.
A
The
green
pods
are
csi
drivers,
so
on
each
node
you
need
a
csi
driver
that
will
mount
mount
the
storage
and
then
the
red
pods
are
all
the
ceph
demons.
So
there
are
quite
a
number
of
seth
demons
that
provide
that
data
layer,
the
mons
and
the
osds
are
backed
by
local
storage
on
the
node.
So
it's
very
important
for
sef
to
have
that
local
storage
anyway
rook
deploys
all
these
pods
for
you
and
does
that
management
at
the
kubernetes
resource
layer,
creating
deployments
pod
services?
Okay.
A
A
Okay,
so
you
create
a
persistent
volume
claim,
usually
with
read,
write
once
access.
Okay,
so
that
when
you
create
that
claim
the
the
request
goes
to
the
storage
class,
which
is
which
uses
cephrbd.
That
rook
sets
up
for
you
and
then
the
csi
driver
will
mount
that
that's
rbd
storage
into
your
application
pod,
something
very
similar
happens
for
shared
file
system.
A
Where
you've
got
two
applications
who
need
to
share
the
file
system,
so
they
both
have
their
persistent
volume
claims
and
the
request
goes
to
the
storage
class,
and
then
this
csr
driver
for
the
shared
file
system
mounts
that
storage.
Okay,
now
the
third
type
of
storage
we've
got
with
ceph
is
object
with
the
s3
input.
Now
this
this
is
different,
but
we
have
exposed
it
in
a
way
that
will
be
consistent
with
as
consistent
as
it
can
be
with
block
and
file
storage.
A
Okay.
So
the
if
you
want
object,
storage,
really
what
you
want
is
a
bucket,
so
you
can
rewrite
objects
to
the
bucket,
and
so
we
create
a
bucket
claim
which
requests
that
storage
from
the
storage
class
crafted
for
object,
storage
and
we
have
a
bucket
provisioner,
then
that
actually
creates
a
bucket
in
enceph
the
sept
optic
storage
and
provides
that
back
to
the
applications
all
right.
So
that
gives
us
all
three
types
of
storage
at
the
provisioning.
Now,
what
does
it
look
like
at
the
data
path?
A
You've
installed,
rook
at
layer,
one
you've
got
the
csi
driver
with
your
storage
mounted
layer
two.
Now
your
application
just
needs
to
write
and
read
data
to
the
to
the
cluster.
So
again
your
application.
It's
already
got
the
volume
mounted
and
there
is
the
sef
rbd
kernel
driver.
Then
that
will
take
care
of
writing
to
the
ceph
cluster
for
you
and
that
driver
will.
It
knows
how
to
connect
to
the
different
ceph
demons,
the
mons
and
osds
and
manager.
A
You
know
to
make
that
that
write,
that
data
to
the
cluster
again
at
the
file
system
file
system
layer,
connect
to
the
mds
daemon
in
ceph,
which
manages
the
you
know,
the
locks
and
the
file
system,
semantics
and
the
s3
client
then
connects
to
the
rdw
endpoint,
which
then
writes
the
objects
into
the
ceph
cluster.
Hopefully
we're
good
there
after
layer,
three
that
you've
set
up
rook
and
now
you
can
write
to
the
cluster.
So
what
does
it
take
to
get
started?
A
This
is
you
know,
I'm
sure
it
sounds
like
a
complex
system,
but
really
to
get
started
with
rook.
We've
tried
to
make
it
as
simple
as
possible
and
installing
ceph
has
never
been
this
simple.
Really
there
are
three
steps
for
for
installing
sap
through
okay.
The
first
thing
is
well,
you
need
to
set
up
authorization
or
rbac
in
kubernetes,
so
you
say:
cuddle
create
common.yaml.
A
Okay.
Next,
you
need
to
run
the
operator,
and
so
you
create
the
operator
that
gets
the
rook
operator
running,
but
now
the
operator
you
have
to
tell
it
how
to
deploy
ceph.
So
now
we've
got
what
we
call
the
cluster
that
cluster
cr
and
you
can
see
a
picture
of
that
on
the
right
here
where
you
tell
rook
how
to
deploy
stuff.
You
tell
it
well
what
version
of
stuff
do
you
want
to
deploy
now?
A
We
all
and
you
can
choose,
what
do
you
want
the
latest
version
or
a
previous
version
of
stuff,
so
you
can
choose
when
to
update
the
the
ceph
version
and
then
some
other.
There
are
many
many
settings
here,
we're
just
showing
a
few
of
them,
how
many
mods,
how
to
deploy
the
storage?
That
is
the
basic
cluster
once
you've
created
these
three,
these
three
manifests
you'll
have
a
basic
ceph
cluster
up
and
running
in
rook.
As
a
side
note,
there's
also
a
helm
chart.
A
A
Just
like
any
other
storage
platform?
First
of
all,
the
admin
has
to
create
the
storage
class
after
they
have
increased
the
storage
class.
The
application
request
that
storage
was
with
the
pvc,
and
then
you
create
your
application
pod
that
that
will
mount
that
that
volume,
so
here
on
the
right,
we've
just
got
an
example
pod
that
shows
this.
B
B
B
So
basically
it's
all
about
consistency,
because
kubernetes
has
been
adopted
by
all
the
major
cloud
providers
today,
then,
it
is
very
easy
to
run
any
cloud
native
application
as
part
of
any
cloud
providers
available
out
there,
and
also
there
are
a
bunch
of
limitations,
shortcomings
coming
from
the
cloud
providers,
for
instance
the
limitation
of
pvs
you
can
attach
to
a
given
node.
Let's
assume
that
we
are
in
a
cloud
environment,
then.
B
Obviously
we
have
hypervisors
and
those
hypervisors
typically
have
a
limitation
in
a
number
of
disks
that
can
be
attached
to
a
given
virtual
machine
and
typically
that
limit
is
around
30,
which
means
that
if
you're
looking
at
providing
dynamic
provisioning
to
applications,
then
you
might
be
limited
by
the
number
of
vms
times
the
number
of
pvs
you
can
attach
to
it.
If
you
do
run
rook
in
the
cloud,
though,
rook
will
be
running
inside
virtual
machines
and
because
we
will
be
attaching
devices,
then
we
use
our
own
technology
to
provide
dynamic
provisioning.
B
Then
we
can
scale
up
to
thousands
and
thousands
of
of
pvs
instead
of
that
30
limitation
by
virtual
machine,
and
not
only
we
bypass
that
limitation,
but
also
because
we
are
aggregating
storage,
then
we're
providing
much
much
better
performance.
Overall,
if
you
were
to
use
a
single
disk
which
is
typically
attached
to
a
storage
class
which
essentially
represents
the
flavor
of
storage,
then
that
flavor
would
have
iops
as
well
as
throughput
limitations.
B
B
B
So
if
you
run
ceph
in
rook
step
inside
a
data
center,
and
then
you
have
multiple
rooms,
multiple
racks
and
multiple
nodes,
then
we
can
easily
build
such
topology
by
assigning
labels
to
nodes,
and
all
of
that
will
be
reflected
as
part
of
step.
By
doing
so,
you're
increasing
data
availability
and
resiliency
because
you're
spreading
it
across
different
field
domains
and
again
those
domains
can
be
nodes,
racks
rooms
and
even
across
zones
between
different
data
centers.
B
If
you
want
to
stretch
your
cluster
safe,
csi
drivers,
so
from
the
very
beginning
rook
once
csi
was
released
and
the
spec
was
released,
we
started
to
collaborate,
the
cfcsi
team,
to
integrate
as
much
as
possible.
So
just
like
travis
mentioned,
we
support
a
large
variety
of
dynamic
provisioning
modes,
rwx
and
rocks,
and
this
for
both
block
and
file
system
interfaces.
B
B
Typically
upgrades
are
some
kind
of
a
tedious
process
and
it
is
really
making
administrators
nervous,
especially
when
it
comes
to
storage.
You
always
be
wondering
okay.
If
something
goes
wrong,
will
I
be
losing
data
first?
This
cannot
really
happen
with
sev,
and
also
one
of
the
good
things
about
surf
is
from
the
very
beginning.
It
has
proven
to
be
super
robust
at
performing
upgrades
and
it's
one
of
the
few
storage
solution,
not
to
say
software,
most
of
the
time
that
can
upgrade
in
a
rolling
fashion
with
no
downtime
and
yeah,
while
providing
a
service.
B
A
B
Notes
by
nodes
and
making
sure
they're
all
recovering
and
being
upgraded
and
backfilling
properly
and
then
go
to
the
other
one.
So
upgrades
made
easy,
basically
x,
no
cluster
connection.
So
this
is
a
really
popular
scenario
and
we
have
introduced
it
in
the
1.3
release
cycle
and
the
the
use
case
is
quite
simple,
because
not
everything
is
about
green
field
environments.
B
You
already
have
clusters
out
there
up
and
running,
which
are
maybe
standalone
clusters
providing
object,
storage
or
you
might
have
subclusters
providing
block
for,
let's
say
openstack,
for
example,
you're
ready
to
run
applications
on
kubernetes,
but
you're
not
you're,
not
really
ready
to
move
all
of
that
storage
because
it
is
difficult
and
it's
challenging
to
do
to
two
kubernetes.
So
you
want
to
keep
your
cluster,
which
was
deployed
by
whatever
tool
can
be
can
be.
Defensible
can
be
rook
as
well.
B
If
you
want
the
print,
the
goal
is
to
from
your
kubernetes
cluster,
you
deploy
rook
and
then
with
a
few
details
you
connect
to
the
external
cluster
and
then
once
the
connection
is
established,
we're
really
into
this
consumer
producer
relationship
where
rook
is,
at
this
point,
only
consuming
the
external
storage,
getting
all
the
connection,
details
and
passing
them
to
csi,
so
that
we
can
provide
persistent
storage
to
containers.
But
that
is
the
only
thing
we
do.
B
We
mentioned
the
object,
bucket
claim
and
again
the
this
is
quite
simple
as
a
user,
the
only
thing
I
care
about
is
because
I'm
developing
an
s3
compliant
application.
I
just
want
to
have
a
bucket
with
a
few
policies
on
that
bucket
and
I
want
to
be
able
to
consume
that
bucket,
storing
data
and
retrieving
data,
and
that's
all
I
care
about
so
in
a
similar
fashion
of
claiming
for
block
or
file.
B
Then
I
want
to
claim
for
a
bucket
and
for
that
we
have
embedded
for
quite
a
while
now
the
lead
bucket
provisioner,
but
because
we
we
really
wanted
to
have
it
in
a
more
native,
slash
native
kubernetes
way.
There
was
enough
upstream
to
implement
a
cozy
interface
which
is
similar
to
csi,
but
for
container
object,
and
it
is
really
the
csi
equivalent
of
object.
B
So
the
capped,
kubernetes
announcement
proposal
was
merged
upstream
and
now
we
will
be
able
to
vendors
will
be
able
to
integrate
their
object
solution
through
a
native
kubernetes
interface,
which
is
really
great.
So
we
just
released
rook
one
five
and
we're
super
excited
because
we
have
a
bunch
of
features
and
it's
probably
one
of
the
most
feature-rich
release
we
have
ever
delivered.
B
So
first
is
encryption
with
kms
support
during
the
wonderful
cycle.
We
introduced
encryption
for
osds
on
pvcs
and
we
were
storing
encryption
keys
into
kubernetes
secrets.
So
we
all
know
that
kubernetes
secrets
are
not
so
secret
and
secure,
because
they're
just
merely
a
a
base.
64
hash
value
of
of
your
value.
To
move
that
to
the
next
level
we
have
introduced.
B
B
Obviously,
this
works,
along
with
with
steps,
csi
and
and
work
is,
is
still
still
moving
ahead
to
to
improve
that
support.
So
now
we
do
support
automating
automatic
configuration
of
of
peers
and
where
pr
is
basically
a
site.
Last
but
not
least,
the
the
stretch
cluster
accumulates
the
stretch
kubernetes
cluster,
which
is
essentially
one
cluster.
B
That
is
stretched.
One
kubernetes
crystal,
is
stretched
across
different
zones.
So
the
use
case
typically
is
that
I
have
two
data
centers,
but
I
only
have
two.
I
don't
have
three
data
centers,
which
will
be
really
ideal,
but
I
only
have
two
and
because
ceph
is
a
quorum
based
election
based.
We
need
to
have
some
quorum
resolution
when
we
do
certain
operations
in
order
to
maintain
the
stability
of
the
cluster.
B
B
We
have
to
add
a
solution
for
sev
to
work
a
bit
smoothly
with
latencies
with
higher
latencies.
Basically-
and
we
implemented
this
new
mode
in
rook.
So
now
you
can
have
a
stretched
kubernetes
cluster
in
an
orbital
zone,
which
will
only
contain
one
month
to
resolve
quorum
and
act
as
a
member
of
the
quran
and
perform
elections
and
pursue
decision
making.
B
All
of
the
other
zones
will
have
storage
available,
and
I
guess
that's
it
for
today,
so
it
is
really
easy
to
get
involved
as
soon
as
you
reach
rio,
then
you
will
have
links
to
everywhere,
basically
the
doc,
the
slack
channel
and
how
to
contribute
obviously
on
on
github.
Thank
you
for
your
attention,
thanks
for
being
here
today,
it
was
a
pleasure
talking
with
everybody,
and
hopefully
we
will
see
everybody
in
person
next
time
have
a
good
day
and
stay
safe.
Bye-Bye.