►
Description
Cephalocon APAC 2018
March 22-23, 2018 - Beijing, China
Bassam Tabbara, Upbound Founder and CEO
A
So
I'm
I'm,
so
my
name
is
Bassam.
I
am
gonna,
be
talking
today
about
a
project
called
rook
and
how
it
helps
SEF
run
on
kubernetes.
Can
you
hear
me?
Okay?
There
we
go.
That's
that's
a
lot
better.
Okay,
so
I
wanted
to
start
by
talking
about
kubernetes
who's
been
playing
with
kubernetes
here
in
the
crowd.
A
Okay,
that's
that's
a
about
the
rate
I
expected,
so
so,
what's
kubernetes,
there
are
a
lot
of
people
to
describe
it
very
in
very
different
ways.
You
hear
container
orchestration.
You
hear
resource
management,
you
hear
an
abstraction
layer,
many
different
uses
of
kubernetes
I
like
to
think
of
it
as
it's
a
cluster
operating
system
essentially
a
way
to
run
modern
cloud
native
applications
on
multiple
nodes
and
treat
all
those
nodes
as
if
they
were
one.
A
A
You
can
declaratively
define
how,
where
they
run,
kubernetes
provides
all
the
resource,
scheduling,
lifecycle
management,
essentially
allowing
you
to
horizontal
scale,
your
apps
automate
things
like
rollout
and
rollback
service
discovery
and
a
lot
of
other
services
that
kubernetes
provides,
and
so
so
so
again,
if
you
think
of
it,
as
this
is
a
modern
cluster
wide
operating
system,
writing
applications
that
run
in
this
operating
system
is
a
desirable
thing
and
kubernetes
has
actually
been
getting
a
ton
of
momentum.
It's
not
literally
everywhere.
A
All
the
public
cloud
providers
support
kubernetes
have
some
form
of
a
managed
kubernetes
service.
All
the
distros,
even
docker
is
now
running
kubernetes,
so
it's
definitely
gained
a
ton
of
momentum
and
it
seems
like
it's
here
to
stay,
and
so,
if
we
were
to
look
at
how,
if
you
had
stateful
applications
that
are
running
on
kubernetes
and
they
wanted
to
consume
storage
today,
the
story
is
kubernetes
runs.
A
You
know
thus
mostly
stateless
applications
if
they
use
storage,
kubernetes
reaches
out
using
a
common
interface,
whether
it's
CSI
or
dynamic
volume,
provisioning
to
a
storage
provider,
a
cluster
of
some
sort,
so
SEF
is
obviously
one
of
those.
If
you
were
running
a
seft
cluster,
you
know
in
whatever
way
you've
deployed
a
manage
stuff.
Then
kubernetes
can
actually
consume
through
CSI
or
through
the
dynamic
volume
provision
it
can
consume.
Storage.
That's
provided
by
that
cluster.
What's
interesting
about
this,
is
that
if
you
look
at
SEF
itself
SEF,
it
is
a
distributed
application.
A
Yet
most
common
deployments
of
SEF
today
in
service
of
kubernetes,
essentially
don't
are
completely
independent,
so
you're
running
a
soft
cluster
using
whatever
tools
you're,
you
know
things
like
ansible
or
you
know,
relying
on
system,
D
or
SEF
deploy,
or
you
know,
classic
SSH
and
bash
scripts,
or
even
things
like
salt
stack
or
you
know
it
many
different
ways
that
stuff
can
be
deployed
today.
So
one
of
the
things
that
motivated
the
rook
project
is
why
not
run
SEF
on
kubernetes
stuff
is
a
you
know
dynamically
manage
this
is
a
horizontally
scalable
application
itself?
A
A
So
that's
the
motivation
for
the
project
called
rook,
so
I
rook
essentially
makes
SEF
awesome
on
kubernetes.
That's
that's
the
premise
of
a
verb.
So
what
is
rook
rook
is
a
cloud
native
storage
Orchestrator.
So
it's
not.
It
doesn't.
Actually
it's
not
a
storage
engine
itself.
It's
not
software-defined
storage.
It
is
an
orchestration
layer
for
storage
systems,
so
it
extends
kubernetes.
It
adds
essentially
new
types
and
controllers
that
help
run
storage
systems
on
kubernetes
right
now.
Sef
is
the
only
storage
system
that
rook
orchestrates.
There
will
be
more
coming
and
and
rook
automates
deployment.
A
Bootstrapping
configuration
provisioning,
essentially
all
the
things
that
typically
either
a
person
is
doing
or
you've
tried
to
encode
in
scripts
or
in
you
know,
whatever
play
books
or
whatever
you're
doing
those
things
are
now
captured
in
code
as
part
of
controllers.
And
now
these
controllers
are
not
a
one-time
action.
You
just
don't
it's
not
about
a
deployment
they're
actually
sitting
and
watching
the
system
and
continuously
making
changes
to
the
system
to
make
it
run.
A
Well,
so
it's
it
definitely
has
a
runtime
aspect
and
we'll
cover
we'll
cover
what
that
means
in
a
little
bit,
rooks
open
source
and
it's
currently
hosted
by
the
cloud
native
foundation.
So
it's
a
it's
a
pretty
much
an
open
project
and
and
part
of
this
whole
cloud
native
movement.
That's
that's
underway!
So
I
didn't
venture
to
do
demo
today,
but
I
want
to
show
you
kind
of
what
Brooke
can
do
so
this
is
the
closest
I
can
get
to
a
demo.
A
A
The
disks
essentially
get
a
basic
cluster
up
and
running
with
SEF,
and
it
does
that
and
it's
continuously
act
actively
monitoring
and
watching
the
cluster
itself.
If
you
were
to
add
another
node,
given
the
the
criteria
here
of
using
all
nodes,
several
automatically
provisioned
admins
rook
will
automatically
provision
that
node
to
you
know
essentially
extend
the
cluster.
If
one
of
the
Mons
was
to
go
down,
rook
will
automatically
provision
a
third
mond
and
re-establish
quorum.
A
It
has
a
control
loop
that
is
actively
watching
and
managing
the
system,
which
makes
it
a
little
different
from
most
other
approaches
to
deployment
and
management
of
such
systems.
So
it's
actively
sitting
in
a
loop
and
that's
the
term
we
use
as
an
operator,
but
it's
also
called
a
kubernetes
controller.
A
So
if
you
have
a
radish
cluster
up
and
running,
then
you
can
do
things
like
create
a
file
system
and
that's
also
a
very,
very
simple
act.
You
to
find
declaratively
what
you
want
as
part
of
the
file
system,
and
then
you
post
that
and
then
rook
actually
crew
goes
and
creates
the
MDS
servers.
And
now
you
have
CF
FS
available
and
the
same
is
true.
If
you
were
to
create,
if
you
wanted
to
create
an
object
store
using
rgw,
you
essentially
define
the
object
store.
A
So
as
s
talking,
rook
is
essentially
an
operator.
The
other
term
is
controller.
People
use
them
interchangeably,
so
the
the
yamo
that
you
saw
is
the
declaration
of
desired
state.
It
defines
what
you
want
to
happen
in
the
system,
not
what
is
actually
out
there.
So
it's
a
high-level
declaration
of
the
the
intent
of
the
operator
or
the
intent
of
the
cluster
admin
on
what
they
want
in
their
cluster,
and
you
do
that
with
objects
like
storage,
cluster,
a
pool,
object,
store
file
system
and
others,
and
then
what
the
rook
operator
does
is.
A
It
sits
in
a
reconciliation,
loop,
essentially
looking
observing
what
is
out
there
and
dipping
what's
out
there
with
the
desired
state,
and
if
there
are
differences,
it
will
act
on
them
and
that
is
running
continuously.
It's
fairly
efficient,
but
it's
actively
watching
the
system
and
reconciling
desired
state
with
actual
state
in
this
Brook
is
not
in
the
data
path.
It's
essentially,
you
know
on
the
control
path.
It's
doing
lifecycle
management
off
the
demons.
It's
not
anywhere
in
the
data
path.
It's
the
data
path
is
classic
SEF.
A
Your
clients
are
talking
directly
to
the
Deo
SDS
and
the
MDS
a--'s,
and
there
are
GWS.
There
is
no
change
to
the
data
path
at
all
and,
as
I
mentioned,
rook
itself
is
implemented
as
a
kubernetes
extension.
It
has
no
persistent
state
itself,
all
its
state
lives
in
kubernetes,
which
in
turn
lives
in
that
city.
So
it's
not
it's
not
another
component.
You
have
to
manage
itself.
It
is
truly
built
as
an
extension
of
kubernetes.
A
If
you're
already
running
a
kubernetes
cluster
you're,
you
can
get
rook
up
and
running
with
no
additional
changes
and
on
the
you
know,
on
the
brooke
has
a
essentially
a
volume
plugin.
It's
a
it's
based
on
flex
volumes,
we're
switching
to
CSI
going
forwards,
and
the
idea
there
is
that
this
is
a
client-side
components
where
it'll
help
you
consume.
The
storage
bye-bye
essentially
mounting
the
right
volumes
on
the
client-side
so
fairly,
straightforward,
kubernetes,
centric
architecture.
A
And
so
one
of
the
things
that
comes
up
is
the
division
of
labor
between,
say
what
rook
is
doing
and
what
SEF
demons
are
doing
so
I
try
to
kind
of
explain
what
I
think
is
the
division
of
labor
here.
So,
if
you
think
about
data
management
as
one
direction,
things
like
Mon
and
OSD
and
even
SEF
manager
are
managing
data
right,
so
Mons
and
OS
DS
are
doing
recovery
and
backfill
and
rebalancing
and
everything
else,
and
actually
the
manager
is
now
doing
a
rebalancing.
All
of
that
stuff
is
around
data
management.
A
A
A
And
so,
let's
go
back
to
the
the
use
case.
If
you're
running
a
kubernetes
cluster
and
you
wanted
storage,
you
could
go
create
a
dedicated
storage
cluster
now
on
kubernetes,
which
means
that
the
surface
area
for
management,
the
tools
you're
using
for
management,
the
the
training
that
you
have
to
do
is
essentially
all
the
same.
You're
running
kubernetes
on
both
sides
just
happens
that
you
have
an
application
on
the
right-hand
side.
That's
safe,
that's
providing
storage
or
probably
a
more
interesting
scenario.
A
A
You
have
to
be
careful
in
these
scenarios.
Resource
management
and
and
and
essential
policies
have
to
be
carefully
managed
here,
but
it
is
truly
possible
for
for
you
to
actually
run
completely
converged
on
something
like
kubernetes
today
and
sorry.
I
want
to
leave
with
essentially
an
aspirational
goal.
I
say
I'd
like
chart
challenges
to
say
going
forward.
We
should
always
be
running
stuff
in
kubernetes,
and
that
should
become
the
default
deployment
of
Ceph
that
we
would
normalize
as
a
community
on
SEF
on
kubernetes
and
stop
thinking
about
the
all
the
different
ways
of
running
stuff.
A
And
so,
if
you're,
if
this
sounds
interesting
and
you
want
to
get
involved,
Brooke
is
an
open
project.
We're
actively
looking
for
contributors.
Please
come
help.
We're
a
very
happy
healthy
community.
Here
are
all
the
links.
Advise
you
to
you
know,
come
join
the
project.
Slack
is
a
good
place
to
find
a
lot
of
people
that
are
building
around
this
as
forums
and
there's
community
meetings
every
every
couple
of
weeks
on
Tuesdays
and
that's
it.
Thank
you
and
I
have
I
brought
some
stickers
if
you
guys
want
some
rug
stickers
put
them
here.
B
Thank
you,
sir
I'm
very
interested
in
raqqa,
predicta
I
have
tried
to
record
in
my
kubernetes
cluster
and
so
I
have
a
question
about
the
performance
losses.
In
fact,
I
haven't
I,
haven't
seen
any
and
data,
or
yesterday
her
about
rocker
rocker
in
performance
loss
before
we.
If
we
use
the
can
continue
to
continue
endure
self.
So
if
you
could
you
could
you?
Could
you
even
do
some
about
that?
Yeah
I
know.
A
That
they're
really
have
not
been
a
lot
of
performance
testing
of
stuff
on
kubernetes.
That
I'm
aware
of
I
think
that's
about
to
start
the
one
that
comes
up
a
lot
is
networking,
because
kubernetes
has
this
funky
networking
overlay
networking
rook
allows
you
to
turn
that
off.
You
could
run
a
cluster
on
the
host
network
and
avoid
all
this
networking,
but
I
I
think
it's
still.
B
Another
question
that
I
conserve
armature
is
that
is
it
has
step
stable
in
a
flow
of
rocket
use
de
producer
environment
use,
you
know,
as
I
see,
is
that
rocker
now
ring
zero
point
estimate:
oh,
not
not
a
1.0
version,
so
that
and
so
so
how
do
you
think
it's
a
it's
a
driver
to
use
a
de
in
producer
environment
an
hour?
It's.
A
Stuff,
it's
still
an
alpha.
There
are
some
I'm
aware
of
tens
of
production
deployments,
I'd
advise.
If
you
are
going
to
put
in
production,
you
should
not
only
be
comfortable
with
self
and
doing
using
self
and
going
stepping
outside
of
rook
and
using
stuff
directly,
but
also
be
comfortable
with
making
code
changes
in
rook.
If
you're
going
to
do
that
today,
you
know
Saif
is
production-ready,
has
been
for
many
years
so
and
as
I
mentioned,
Brooke
is
not
on
the
control
path
is
not
on
the
data
path.
So
from
that
perspective
it
seems
you
know.
A
Saif
is
very
much
ready,
but
there
are
always
cases,
and
we
just
ran
into
a
couple
one
case
where
I
Brooke
was
doing
something
nasty
to
the
volumes
and,
and
so
so
I
expect
rook
to
be
production
ready
in
the
next
six
months,
so
we're
working
towards
that
goal
there.
You
know
it'll
probably
go
to
beta
on
some
of
the
types
in
the
next
few
months
and
then
over
over
that
period,
we'll
we'll
get
it
to
production
ready.
Thank.
C
A
So
certainly
you
can
deploy
SEF
on
docker.
You
self
can
go
into
containers.
I.
Think
the
primary
difference
is
that
on
kubernetes
you
have
an
active
controller
that
is
managing
SEF,
so
SEF
can
be
a
lot
more
dynamic
and
actually
a
man
it's.
It
starts
to
look
like
a
managed
SEF
on
kubernetes
and
that's
the
primary
difference
if
you
run
it
on
docker
you're,
essentially
having
to
do
a
lot
of
the
things
manually
yourself
still,
even
though
it's
running
in
containers.
A
A
Are
very
interesting
scenarios,
so
so
one
of
the
things
that
kubernetes
will
allow
you
to
do
is
do
a
horizontal
scaling,
yep
and
that
could
be
based
on
a
policy.
So
you
could
do
something
like,
as
the
number
of
connections
go
up
on,
say
a
ratas
gateway
or
if
there
are
health
metrics
that
are
exposed,
and
that
could
be
a
feedback
loop
to
growing
the
cluster
itself,
that
those
are
those
are
yet
more
examples
of
what
I,
why
I
think
running
stuff
in
a
dynamically
managed
environment
like
kubernetes,
is
going
to
help
the
project.
Thank.