►
From YouTube: Rook Project Intro – Bassam Tabbara, Tony Allen & Jared Watts, Upbound (Any Skill Level)
Description
Want to view more sessions and keep the conversations going? Join us for KubeCon + CloudNativeCon North America in Seattle, December 11 - 13, 2018 (http://bit.ly/KCCNCNA18) or in Shanghai, November 14-15 (http://bit.ly/kccncchina18)
Rook Project Intro – Bassam Tabbara, Tony Allen & Jared Watts, Upbound (Any Skill Level)
About Bassam
Bassam Tabbara is the founder and CEO of Upbound, a new stealth-stage startup in the cloud-native computing space. He created the Rook project which provides Storage Orchestration for Kubernetes. Prior to Upbound Bassam was the CTO @ Quantum, co-founder and CTO @ Symform (a P2P storage startup acquired by Quantum), and Partner Architect @ Microsoft.
About Jared
Jared Watts is a Founding Engineer at Upbound, where he is working on advancing cloud-native computing by ""freeing the cloud"" and is also a maintainer for the open source Rook project (https://rook.io/). Before Upbound, he worked on innovative storage projects at Quantum Corp. and helped found Rook. Prior to Quantum, Jared worked on P2P storage distributed across the public internet at Symform, a startup later acquired by Quantum. Before that, Jared began his career at Microsoft working on applications for Windows Server setup and management.
About Tony
Tony Allen is a software engineer at Upbound, where he is working on advancing cloud-native computing. He is currently a contributor to the Rook project (rook.io). Prior to Upbound, Tony worked on the storage engine at Nutanix as part of their file systems and core data path group.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
All
right
that's
been
go
ahead
and
get
started.
Here's
people
filling
in
hey
there,
yan
alright.
So
this
is
the
the
Brooke
project
intro.
My
name
is
Jared
watts
I'm
one
of
the
maintainer
zhan,
the
Brooke
project
and
Brooke
was
recently
accepted
a
couple
of
months
ago
as
a
hosted
project
by
the
cloud
native
computing
foundation,
the
CNC
F.
So
that's.
Why
we're
talking
about
it
today?
A
So
just
a
quick
show
of
hands
who
is
actually
used
rook,
but
so
far
good
cool
fare
money.
You
guys.
So
let's
talk
about
what
rook
is
so
rook
is
a
cloud
native
storage,
Orchestrator,
and
so
what
that
means
is
that
in
cloud
native
environments
Brooke,
you
can
use
rook
to
deploy,
manage
setup,
configure
all
different
types
of
storage
solutions.
A
So
you
know
inside
kubernetes
you
can
use
it
to
deploy,
let's
say,
SEF
or
or
mini,
or
some
other
storage
solutions
in
the
cloud
native
environments,
so
how
it
does
that
is
by
extending
kubernetes
with
custom
types
and
controllers
that
we
call
operators
and
we'll
get
into
that
more
and
some
of
the
interesting
things
that
it
does,
the
you
know
kind
of
the
value
that
it's
bringing
here.
So
your
storage
needs
are
a
bunch
of
automation,
for
you
know,
deploying
and
bootstrapping
configuring
scaling
it
up
scaling
it
down.
A
Upgrading
your
cluster
disaster
recovery,
all
that
sort
of
stuff.
You
know
complicated
management
tasks
that
you
need
to
complete
in
your
cluster
it'll
perform
those
for
you
and,
as
we
kind
of
alluded
to
earlier,
it's
now
a
framework
as
well
for
other
storage
providers
and
solutions
to
integrate
in
with
kubernetes
as
well.
A
So
Brook
is
an
open
source
project
with
the
Apache
2.0
license
and
as
I
mentioned,
it's
hosted
by
the
cloud
native
computing
foundation
now,
as
of
the
end
of
January
I,
believe
so,
let's
talk
about
storage
in
kubernetes
and
what
a
normal
solution
or
deployment
looks
like
there.
So
oftentimes
with
kubernetes,
you
have
your
applique
Asians
that
are
running
inside
the
cluster,
but
you
have
for
those
applications
storage.
What
time?
What
happens
often
is
that
you
will
have
an
external
storage
set
up
that
could
be
running.
A
You
know
something
like
an
Amazon
with
the
EBS
or
Google
persistent
disk,
or
it
could
be
an
appliance.
You
know
in
Iraq,
somewhere
in
your
data
center,
either
way,
though,
that
storage
is
coming
from
somewhere
external
to
the
cluster.
So
there's
a
couple
of
limitations
with
that
approach.
One
thing
your
the
big
kind
of
overarching
aspect
is
that
it's
not
portable.
A
So
when
you
you
have
your
application,
then
it's
got
storage
needs
if
you're,
relying
on
storage,
that's
kind
of
external
in
the
cluster
somewhere,
you
need
those
to
be
accessible
when
you're,
when
you're
deploying
your
clusters,
so
that
creates
a
burden
for
deployment.
If
you
want
to
pick
up
your
application,
take
it
somewhere
else
and
put
it
into
another
environment,
say
it
to
move
from
Amazon
to
Google.
You're
gonna
have
to
also
worry
about
how
your
the
external
source
storage
gets
moved
over
there
as
well.
A
Another
thing
is:
if
you're,
relying
on
storage
from
let's
say
a
cloud
provider,
you're
gonna
often
end
up
worrying
about
lock-in
with
being
in
you
know,
a
cloud
providers
service
and
relying
on
something
like
Amazon's
relational
database
service
RDS.
You
know
you
take
a
dependency
on
that
from
your
application
and
it
makes
it
more
difficult
to
move
your
applications
somewhere
else.
A
So
one
approach
to
this
solving
this
is
making
storage
run
on
top
of
kubernetes
also,
so
we
already
have
a
kubernetes
cluster
up
and
running.
You've
got
your
applications
running
in
kubernetes,
so
we
could
also
use
kubernetes
to
manage
your
storage
as
well
in
addition
to
your
application.
So
this
then
allows
us
to
have
highly
portable
applications,
because
we
are
not
only
describing
the
application
itself
when
we're
deploying
it
we're
also
describing
its
storage
dependencies
as
well.
So
you
have
a
situation
where
you
know
your
application
itself
can
be
described.
A
You
know,
by
way,
in
a
set
of
kubernetes
resources.
You
know
pods
services,
things
like
that,
but
you
can
also
now
start
describing
your
storage
as
well
and
hand-in-hand.
Those
can
get
deployed
out
to
any
environment.
The
kubernetes
runs
in
you
know,
I'd
be
very
portable.
That
way,
so
you
can
use
me
you
can
deploy
it
to
Microsoft
Azure
or
to
Amazon,
or
it's
a
you
know,
digital
ocean
wherever
it
may
be
very
very
easily,
and
this
is
also
not
the
only
architecture
that
you
can.
You
know
enable
this
with.
A
You
can
also
have
a
dedicated
kubernetes
storage
cluster
by
itself.
That's
running
your
storage,
just
kind
of
separate
your
compute
from
your
storage.
So
let's
talk
about
abstractions,
real
quick.
So
that's
one
of
the
ways
that
kubernetes
accomplishes
being
able
to
run
everywhere
is
by
having
a
strong
set
of
EPs
abstractions
that
are
applicable
to
all
environments
that
they
run
in.
A
So,
for
instance,
you
know
a
pod.
A
pod
is
a
pod,
no
matter
where
it's
running
right.
It
can
be
running
on
premise:
it
could
be
running
and
Amazon
doesn't
matter.
A
pod
captures
the
concept
of
an
application
running
in
an
environment.
So
similarly
we
need
some
portable
abstractions
that
also
describe
data
needs.
You
know
your
debt,
your
application
needs
a
database
or
it
needs
a
message
queue
whatever
it
may
be.
Your
application
has
needs
of
data
if
it's
going
to
be
stable.
A
If
it's
gonna
be
useful
right,
so
we
want
to
make
some
abstraction
--zz
for
those
as
well
for
data.
That's
gonna
work
across
environments
across
providers.
You
know
hive
hybrid
scenarios,
on-premise
cloud
providers,
all
that
sort
of
stuff
and
then
we're
getting
into
this
example.
Here
of
when
your
your
app
needs
a
database.
You
know
if
it's
the.
If
we
have
an
abstract
concept
for
a
database,
then
when
you
go
deploy
your
app
inside
Amazon,
it
can
use
RDS,
and
if
you
deploy
your
app
within
Google,
then
it
will
use
cloud
sequel
instead.
A
So,
let's
talk
about
how
rook
does
this?
It
works
on
a
couple
different
ways
to
extend
kubernetes
and
leverage
the
power
of
kubernetes.
One
is
the
operator
pattern
we
know
of
a
number
of
operators
out
there
now
Prometheus
has
one
@cd
has
one
and
Brooke
also
has
an
operator
or
set
of
operators?
Actually
so
real,
quick,
we'll
just
talk
about
what
that
is,
so
operators,
basically
codify
their
expertise
or
subject
matter
expertise
for
how
to
run
a
complicated
piece
of
software.
So
you
know
running
software
especially
distributed
systems.
They
have
a
lot
of
complexity.
A
They've
got
a
lot
of
you
know,
components
to
them
and
there's
a
lot
of
tasks
that
you
need
to
do
in
order
to
keep
that
running,
healthy
and
running.
You
know
up
so
an
operator
is
kubernetes
piece
of
software.
That
automates,
that
for
you
and
the
way
it
does,
that
is
by
running
a
control
loop,
just
like
any
of
the
other
kubernetes
controllers
here,
the
node
controller,
pod
controller
admission
controller,
all
those
things
they
basically
run
in
a
loop.
That
does
three
things.
A
So
these
are
the
way
that
you
teach
kubernetes
about
new
objects.
So
in
the
kubernetes
api
you've
got
the
built-in
objects,
you
know
pods
and
services
and
config
maps
and
things
like
that.
But
with
the
CRD,
you
can
declare
arbitrary
types
types,
any
type
that
you
want
and
then
once
you've
defined
that
type.
It
looks
just
like
any
other
object
in
in
the
kubernetes
api
and
you
can
even
use
cube
cuddle
to
manage
it
as
well.
A
So
in
a
CR
D,
that's
a
you
know
you
can
have
all
these
different
properties
defined
for
how
a
user
might
want
their
system
to
act.
So
the
user
can
create
one
of
your
custom
resource
definitions
and
set
the
different
properties
that
they
want
in
order
to
get
their
system
configured
the
way
they
want
it
to
all
right.
So,
as
we
talked
about,
Brooke
has
a
number
of
operators
that
are
implemented
to
to
enable
distributed
storage
systems
and
other
types
of
storage
to
now
run
inside
of
kubernetes.
A
They
capture
the
desired
state
like
we
were
talking
about
for
a
storage
cluster,
so
for,
for
example,
SEF
is
a
distributed
storage
system
and
it's
got
a
lot
of
moving
parts.
It's
got
a
lot
of
complexity,
but
then
the
Brooke
operator
and
the
rook
custom
types
have
captured
some
of
that
in
some
more
abstract
concepts
like
a
cluster
or
a
pool
or
object
store,
and
so
the
user
can
create
those
and
the
rook
operator
will
be
watching.
For
those.
You
know
see
what
the
user
wants
the
desired
state.
A
So,
as
we
talked
about
also
some
of
the
really
you
know
more
interesting
things
that
an
operator
can
do
is
some
of
those
complicated
things
that
a
human
normally
has
to
do,
such
as
handling
upgrades
or
rebalancing
the
cluster.
Those
are
complicated
tasks.
There's
you
know
a
lot
of
steps
along
the
way.
There's
a
lot
of
checks.
You
need
to
make
to
make
sure
that
everything's
still
healthy,
possibly
even
rollback
the
clustered.
If
something's
gone
wrong
and
that's
you
know
that
that
could
be
complicated
but
having
a
piece
of
software
to
do
that.
A
These
operators
are
not
on
the
data
path
and
that
ends
up
being
pretty
good,
because
if
you
look
at
a
piece
of
software
like
SEF,
which
has
been
you
know
in
development
for
years
and
years
to
reach
the
level
of
maturity
and
reliability
that
it's
obtained
now
you
know
it's.
We
don't
want
to
be
in
the
business
of
writing
a
new
data
path.
Basically,
we're
gonna
take
advantage
of
some
of
those
existing
storage
systems
that
have
been
out
there
for
a
while.
A
This
is
where
the
operators
are
watching
they're,
getting
an
event
from
the
kubernetes
api
that
says
that
that
new
cluster
was
created
and
then
it
goes
out
and
does
the
right
things.
Excuse
me
in
the
cluster
to
make
sure
that
the
request
from
the
user
for
new
storage
or
a
new
storage
cluster
has
been
created
and
it's
persisted
and
it's
healthy
and
all
those
sorts
of
things.
A
If
we
look
at
the
bottom
of
bottom
right
of
the
of
the
image
here,
we
see
that
there's
also
a
volume
plugin
for
that
talks
to
the
cubelet
when
there's
a
dynamic
request
for
first
storage.
So,
for
instance,
if
if
a
new
pod
comes
up,
it
needs
a
block
device
that
goes
through
a
dynamic
provisioning
flow.
A
Just
a
quick
example
to
kind
of
tie
a
few
things
together
here.
So
when
we
were
talking
about
SEF
running
on
kubernetes,
you
know
what
the
user
does
is,
creates
an
instance
of
one
of
those
cluster
CR
DS
right,
the
cluster
type.
So
we
could
see
here
on
the
left
that
this
will
be
a
llamo
file,
that
a
user
will
use,
cube,
cuddle,
create
dash
F
and
pointing
to
that
file
there
that
specifies
how
they
want
their
Ceph
storage
cluster
to
get
created.
They
can
specify
things
like
what
devices
to
use
where
metadata
should
go.
A
So
that
would
that's
safe,
but
we
also
have
support
new
support
for
other
types
of
storage
within
kubernetes.
So
from
the
work
that
we
did
with
SEF,
we've
learned
a
lot
of
lessons
and
saw
that
there's
a
lot
of
patterns
they
can
be
applied
to
other
types
of
storage
systems
as
well.
So
now
you
can
think
of
rook
is
not
just
a
collection
of
operators
and
CR
DS
and
your
custom
types.
It's
also
a
framework
that
can
be
used
for
other
storage
providers
that
want
to
integrate
into
kubernetes
as
well.
A
Some
of
the
patterns
there
that
you
see
are
you
know
normalization
of
how
you
declare
and
define
storage
resources
of.
Oh
I
want
this
disk
or
this
directory,
or
this
is
how
I
want
you
know,
data
to
be
provisioned
or
I
want
compute
and
storage
to
be
separate.
You
know
those
types
of
decisions
that
you
need
to
make
when
you
are
defining
how
your
storage
clusters
should
be
created
is
now
a
normalized
consistent
way
across.
You
know
more
than
just
one
storage
provider
more
than
just
SEF.
A
You
know
operators
they're,
they're,
complicated
pieces
of
machinery,
but
not
that
complicated,
and
you
know
the
Brooke
framework
here
has
a
number
of
patterns
and
the
plumbing
for
a
storage
provider
that
wants
to
get
into
kubernetes
to
do
that
very
easily.
You
know
policies
like
backup
and
restoration
and
networking
policies.
All
these
sort
of
you
know,
policies
and
logic
are
reusable
across
different
storage
providers.
A
So,
just
last
week
we
started
creating
where
we
have
up
and
running
operators
for
cockroach,
DB
and
Mineo
as
well.
In
addition
to
SEF-
and
this
is
using
that
rook
framework,
where
the
the
coding
job
wasn't
wasn't
that
hard
anymore,
because
we
have
these
tools,
we
have
these
constructs
to
bring
more
types
of
storage
into
kubernetes.
In
a
you
know,
reliable,
consistent
way,
there
will
be
support
for
next
sentence.
Well
soon,
all
right!
A
Is
that
somewhat
readable,
but
back
there?
Okay,
it's
the
same
here!
Okay!
So
if
we're
over
here
in
this
window,
I've
just
got
a
little
watch
going.
That's
going
to
show
us
what
we
have
running
in
the
cluster,
your
name
spaces,
pod
services,
persistent
volume
claims
things
like
that.
We
don't
have
much
running
right
now.
We
just
have
you
know
the
namespaces
that
come
with.
You
know
the
kubernetes
system
I
have
up
here
running,
that's
it!
So
let's
get
a
set
closer
running
and
then
we're
also
on
top
of
that
stuff
cluster.
A
We're
going
to
run
a
stateful
application
and
I'll
show
you
how
easy
this
is.
So,
let's
go
ahead
and
create
I'm
gonna
use
some
aliases
to
KS
cube
cuddle.
I
can
only
type
so
much
so
it's
two
faces
here.
The
first
is
that
we
need
to
get
the
operator
itself
running,
so
we
deploy
out
a
am
will
manifest
that
describes
the
Ceph
operator
for
rook
and
it
goes
ahead
and
declares
some
custom
resources
that
we
can
see
here
as
well,
such
as
clusters
or
file
system
objects
or
pools.
A
A
Creating
a
cluster
so
the
custom
type,
a
scepter
that
we've
defined.
We
created
an
instance
of
it.
The
operator
is
watching.
It
gets
that
event
that
says:
hey
the
user
wants
a
a
self
cluster,
so
the
operator
is
going
to
go
ahead
and
make
that
happen.
I
should
be
on
this
window
here
now
sorry.
So
what
is
the
operator
doing
right
now?
It's
you
know,
it's
very
simple
to
say:
oh,
create
a
cluster
and
okay.
I
will
create
a
question
for
you,
but
there's
more
detail
than
that.
A
So
I,
don't
know
how
much
you
all
know
about
SEF,
but
it
has
a
number
of
components
such
as
monitors,
OS,
DS
managers,
and
so
what
the
operator
is
doing
right
now
is
that
we
asked
it
to
use
every
node
in
the
system
and
and
what
else
did
we
ask
I?
Think
I
would
like
that
in
a
second,
but
we've
basically
told
it
to
create
a
cluster
that
can
give
me
file,
storage,
object,
storage,
block
storage
and
the
operators
acting
on
that
behalf
on
our
behalf,
so
it's
created
a
service
you
can
see
down
here.
A
Yeah
we
got
to
see
some
more
services
and
things
like
that
all
right.
So
then,
that's
great
chef
cluster
is
running
in
kubernetes.
It's
ready
for
holding
data.
You
know,
that's
only
half
the
story,
let's
actually
use
that
data.
That's
actually
the
interesting
part,
so
I'm
going
to
do
a
couple
of
things
here
of
defining
a
storage
class
and
that
storage
class
is
going
to
be
able
to
that's
the
link
from
a
application
that
says:
I
want
block
storage
and
then,
on
the
other
side
of
that
for
rook.
Fulfilling
that
storage
request.
A
All
right,
so,
let's
go
ahead
and
create
a
wordpress
and
also
my
sequel,
underneath
it
and
let's
watch
over
here
as
those
start
coming
up
so
WordPress
is
by
itself.
You
know,
isn't
a
stateful
app,
it's
just
you
know
a
front-end,
but
it
needs
to
store
store
data
somewhere
in
my
sequel
and
that's
where
it
gets
interesting
right.
A
Where
you
can
have
blog
posts,
you
can
have
comments,
you
can
have
you
know
all
sorts
of
interesting
state
state
right
so
WordPress
by
its
needs,
a
backing
store,
a
data
base
and
that
data
base
needs
to
be
stored
or
persisted
somewhere,
and
so
what
we're
seeing
here
is
when
my
sequel
gets
created
it.
It
needs
block
storage.
A
So
it
asks
the
rook
operator
for
a
block
device,
and
that
goes
through
the
process
of
creating
a
volume
in
of
the
cluster
and
then
attaching
that
volume
to
the
correct
node
that
were
that
my
sequel
is
running
on
and
mounting
it
and
making
sure
that
the
my
sequel
pod
is
able
to
then
start
consuming
it.
So
let's
go
ahead
and
look
at
WordPress
real,
quick.
A
Okay,
let's
look
at
the
different
instances
we
have
here,
so
we
see
that
these
persistent
volume
claims
are
representing
the
data
needs
of
WordPress
in
of
my
sequel,
so
we
can
see
that
they
both
asked
for
20
gigabytes
and
those
were
bound
and
fulfilled
by
the
rook
operator
in
order
to
make
sure
that
that
storage
was
available
for
WordPress.
Here
we
go
so
then
that's
so
that's
interesting
right
that
you
know
we
have
a
stateful
application.
It
has
data
needs.
A
A
So
what
I'm
gonna
do
here
is
I'm
going
to
kill
the
database
instance.
So
when
we
kill
the
database
instance
because
we're
running
in
kubernetes
we're
gonna
get
some.
Some
very
awesome
management
constructs
here,
so
I
killed
it
and
we
should
see
my
sleep
while
terminating
right
here
and
so
when
mice
that
database
dies.
You
know
all
these
kubernetes
amazing
management
capabilities.
We
have
here
we'll
bring
up
another.
A
So
if
everything
worked
there
with
the
new,
my
sequel
instance
running,
we
should
be
able
to.
You,
know,
refresh
and
see
the
see
the
page
here
and
you
to
still
be
able
to
use
the
site,
and
you
know
the
data
is
still
there,
even
though
the
my
sequel
instance
maybe
removing
around
the
cluster.
You
know
the
data
is
going
along
with
it
right.
A
A
And
so
you
know,
as
I
mentioned,
we're
an
open
source
project,
so
we
you
know
we
would
love
to
have
new
contributors.
We've
got
a
nice
healthy
community
going,
but
we
can
always
have
more
contributors,
so
one
way
to
get
involved
is
to
come,
but
to
our
github
contributes
open
issues.
You
know
if,
when
you
use
Rooke
and
run
into
a
problem
open
an
issue
come
talk
to
us
on
slack,
we
actually
have
a.
A
We
actually
have
a
conferences
channel
on
our
slack
right
now
to
to
keep
in
touch
with
each
other
as
we're
going
through
this
Brook
underscore
IO
is
our
Twitter.
We
also
have
community
meetings
every
couple
of
weeks,
so
we've
got
a
community,
we
want,
you
know,
to
meet
more
people,
talk
to
more
people
and
get
more
people
interested
in
Brook,
and
this
week
we
actually
have
some
more
sessions
as
well
as
well,
since
we
are
a
CN
CF
project.
B
A
The
state
of
support
is
in
like
support
when
there's
a
problem
or
supporting
those
data
types.
Okay.
So
the
question
was:
what's
the
you
know,
state
of
support
for
different
data
types
comparable
to
what's
in
Amazon
like
EBS,
and
so
that's
that
depends
on
what
the
storage
is
provider
that
rook
is
deployed
is
so
we
need
to
deploy
SEF
you
get.
A
C
A
That's
a
good
question,
so
the
question
is
about
what
can
be
down
in
rook
for
data
to
still
be
usable
for
data
to
still
be
accessible.
As
we
talked
about
the
operator
it
can
is
not
on
the
data
path,
so
that
can
be
down
in
the
rook
agent.
Part,
as
you
noticed,
is
that's
only
on
the
data
setup
path,
so
only
at
attaching
in
mounting
time
and
then
also
unmounting
and
detaching.
A
D
A
E
A
So
that
that's
a
good
question
this
over
the
performance,
benchmarking
that
we've
done
so
far
has
been
focused
on
you
know,
read
and
write
throughput
and
latency
things
like
that.
I,
don't
have
any
information
about.
You
know
how
many
PBC's
can
be
running
or
sort
of
the
you
know
number
of
concurrent
accessors
to
the
data.
I.
Don't
actually
have
a
figure
for
that.
E
We
were
running
on
glass
traces,
but
we
had
issues
when
when
we
rebooted
a
node,
the
volumes
could
not
be
mounted
or
because
this
was
not
there.
So
we
are.
We
are
doing
the
same
thing
you're
in
ruk,
right
so
more
more
towards
that,
where
all
the
volumes
could
be
attached
or
reattached
when
there
is
a
node
failure,
sure.
A
Yeah
right
yeah,
and
that's
that's
something.
You
know
that
it's
really
interesting
and
you
know
we
kind
of
tried
to
simulate
that
for
you
today
of
when
something
goes
wrong.
You
know
to
move
data
around
to
another
node
in
the
cluster,
not
even
the
data
itself,
because
if
you
think
about
it,
distributed
storage,
Questor
you've
got
all
the
data
already
spread
around
and
when
one
node
goes
down,
you're
losing
some
of
the
redundancy.
The
durability
of
the
data
is
getting
affected,
but
it
should
absolutely
still
be
available,
even
though
you've
lost
the
node.
A
The
interesting
part
for
the
immediate
thing
you
have
to
worry
about
when
you
lose
a
node,
is
more
so
any
of
the
local
applications
or
instances
like
pods
that
we're
using
the
data
on
that
node
right
and
so
as
soon
as
you
know,
the
node
is
is
noticed
to
be
lost.
Then
the
rook
agent
starts
doing
work.
You
know
if
it
can,
because
god
knows
what
might
be
wrong
with
the
node
may
not
be
able
to.
A
It
might
be
completely
powered
off
right
and
it
won't
be
able
to
unmount
it
or
detach,
but
as
soon
as
the
note
is
lost
in
kubernetes,
it'll
start
doing
that
work
and
move
the
volume
somewhere
else
as
quickly
as
I
can
and
we've
with
some
some
of
the
tests
that
we've
done
in,
for
instance,
in
the
Google
compute
engine
gke.
You
know,
we've
done
that
and
you're
within
30
seconds,
or
so
it
shows
up
on
another
node
and
written
that
pod
is
already
scheduled
and
using
the
data
again
on
another
node.
B
A
The
thing
I
remember
about
that
Alex
might
remember
more
about
that
as
well,
but
the
the
blue
store
is
made.
It's
designed
for
devices
specifically
running
on
raw
devices
so
running
on
an
existing
file
system.
You're.
You
don't
have
good
performance
on
that
anyway,
since
you're
going
through
multiple
layers,
but
you
also
don't
have
that
that
ongoing
management
of
you
know
freeing
blocks
and
I.
Don't
know
if
that
was
a
design
flaw,
and
it
necessarily,
but
blue
story
is
definitely
intended
to
be
used
on
raw
devices.
Do
anything
else
to
add
to
that
Alex
yeah.
D
A
It
is
on-premise,
yes
really
well
it
also.
Actually
is
we
gotta
drink
water,
so
we
were
running
a
set
of
bare
metal
nodes
in
our
office
for
quite
a
while,
and
you
know
deploying
it
you
know,
is
the
important
part
is
having
kubernetes
there.
So
once
you
have
kubernetes
that
abstracts
away
the
underlying
environment,
so
it
doesn't
matter
if
it's
a
cloud
provider
or
if
it's
an
on-premise
scenario
right,
the
storage
cluster
can
be
deployed,
configured
and
managed
it's
basically
the
same.
It
doesn't
really
realize
that
it's
on-premise
or
in
a
cloud
provider.
A
So
that's
one
of
the
interesting
things
about
the
you
know:
power
of
abstractions,
where
the
environment
becomes
much
less
prominent
in
in
the
management
of
it,
because
kubernetes
and
other
abstractions
have
kind
of
dealt
with
that
for
us
at
its
time.
I
think
we
got
one
more
minute
here.
What
would
you
say
that
is
the
the
biggest
issue
you're
facing
right
now
with
the
brook?
That's
a
really
good
question.
A
I
think
you
know
the
so
Brook
is
an
alpha
project
right
now
we're
working
on
getting
to
beta
and
the
the
aspect
of
you
know
getting
the
management,
and
you
know
edge
case
scenarios
you
know
being
able
to
deal
with
with
failure.
Unexpected
failures,
really
gracefully,
that's
kind
of
the
the
next.
You
know
that
last
part
there
of
getting
from
an
alpha
project
to
a
beta
project
and
those
are
the
elusive
ones
that
are
difficult
to
track
down
difficult
to
get
Reaper.
You
know,
reproducible
and
stuff
to
figure
out
what
the
real
cause
is.
A
So
I
think
those
are
the
those
final
issues
about
getting
it
to
full
and
ready
production.
Reliability
is
the
biggest.
You
know
thing
that
I
am
concerned
with
personally
right
now,
and
that
is
the
end
of
the
session
I
believe.
So,
if
you
wonder,
if
you
have
more
questions
to
come
talk
to
me,
thank
you
guys.