►
Description
Presentatin by Bassam Tabbara, Founder and CEO, Upbound at 2018 GeekWire Cloud Tech Summit: geekwire.com/cloudsummit
Kubernetes has become the de facto container orchestrator and now runs everywhere from public clouds to private clouds. One of the most powerful features of Kubernetes is the ability to deploy your applications consistently no matter what the environment is, but the same should be true for your storage if you want to realize the full potential of Kubernetes.
In this talk, we will explore options and tradeoffs for running stateful workloads on Kubernetes. We will also introduce Rook, a CNCF-hosted storage orchestrator for Kubernetes, and show how it can help run portable stateful workloads on Kubernetes.
A
I
think
I'll
try
this
one
first,
so
hi
I'm,
Bassam
I'm,
the
CEO
and
founder
of
a
company
called
upbound
new
startup
in
Seattle
in
this
space
in
the
infrastructure
space,
and
we
are
the
sponsors
of
a
CNC,
a
project
called
rook
which
is
really
about
dealing
with
stateful
workloads
on
kubernetes.
So
what
I
hope
to
do
today
is
to
kind
of
walk
through
some
of
the
choices
you
have
when
you're
running
staple
workloads
or
interacting
with
staple
workloads
on
kubernetes.
A
Some
of
these
are
not
so
obvious
when
people
think
about
running
state
and
stateful
and
kubernetes
so
I
hope
to
kind
of
walk
through
some
choices
and
talk
through
some
of
the
trade-offs,
but
feel
free
to
ask
questions
and
happier
happy
to
have
this
a
little
bit
of
a
library
live
discussion,
so
so
look
show
of
hands
how
many
of
you
guys
have.
Well,
let's
start
with
the
easement
heard
of
kubernetes,
okay,
great
you're
in
the
right
room.
A
A
So
just
the
level
said
you
know,
I
think
people
describe
kubernetes
in
a
lot
of
different
ways,
I'd
like
to
think
of
it,
as
the
cluster
operating
system.
Just
like
Linux
is
the
an
operating
system
for
a
single
machine
or
node
I.
Think
of
kubernetes
as
the
operating
system
for
the
cluster
and
as
an
operating
system.
It
does
a
number
of
things,
most
people
kind
of
refer
to
kubernetes
as
the
container
orchestration
which
it
does,
but
it
does
a
bunch
of
other
stuff
that
people
don't
talk
about
or
don't
highlight
as
much
as
container
registration.
A
A
The
thing
that
I
think
gets
probably
the
least
amount
of
attention
when
people
talk
about
kubernetes
is
the
fact
that
it
actually
exposes
a
portable
cloud
API,
and
that
is
an
important
strategic
piece,
because
it
it
is,
you
know,
try
to
deliver
on
the
promise
of
writing
ones
and
running
everywhere,
and
if
you
were
to
write,
put
your
stuff
in
containers
and
run
in
kubernetes
for
the
most
part,
that's
true.
Kubernetes
today
is
supported
in
all
the
different
public
clouds,
including
Amazon
now
and
by
most
platform.
A
So
if
you
were
to
put
yours
code
in
containers-
and
you
don't
really
need
anything
outside
of
the
kubernetes
space,
the
write
once
run
everywhere
is
actually
fairly
true
and
you'll.
Seeing
we're
seeing
a
lot
a
lot
more
companies
converging
on
kubernetes
and
talking
about
how
that's
become
a
productivity
enhancement
and
how
they've
changed
their
entire
operation
to
be
kubernetes
based
and
how
they've
seen
productivity
wins
as
a
result
of
doing
that,
the
story
falls
short
when
it
comes
to
storage,
and
so
let
me
let
me
first
level
set
on
how
we
think
about
storage.
A
A
Any
kind
of
state
that
needs
to
survive
beyond
the
lifecycle
of
a
container
a
pot
or
a
function
is
probably
in
the
definition
of
storage
and
so
you'll
hear
different
terms
of
this.
Usually
you'll
hear
stateful
stateful,
stateless.
None
of
them
are
perfect.
Persistent
storage
you'll
hear
durable
long-lived.
Those
are
all
examples
of
how
people
describe
what
what
we're
talking
about
here
and
then
there
are
different
forms
of
storage
and
so
the
most
obvious
ones
that
people
think
of
when
you
think
of
storage
as
blocks
and
files.
A
A
So
let
let's
bring
the
two
together
if
we
think
about
kubernetes
as
being
this
platform,
abstraction
and
cluster
operating
system,
and
if
we
think
about
storage
as
being
an
important
piece
of
every
application,
every
service,
how
do
they
come
together
when
I
keep
hitting
the
mic
so
well?
Kubernetes
defines
an
abstraction,
a
portable
abstraction
for
one
form
of
storage
volumes.
So
that's
block
storage
and
file
storage.
A
So
if
you
wanted
the
size
or
the
quality
or
any
kind
of
annotations,
you
could
describe
that
on
the
volume
itself
and
on
the
right
here
we
have
a
pod,
that's
using
this
volume
and
if
you're,
using
this
abstraction,
you're,
actually
able
to
run
your
service
and
consume
volumes
block
our
file
anywhere.
That
kubernetes
runs
for
the
most
part.
It
still
fits
the
right
ones,
run
everywhere,
mantra,
and
so,
and
unfortunately
this
is
the
only
abstraction
that
kubernetes
has
related
to
stateful
workloads.
A
It
stops
here
I
with
with
volumes
almost
everything
else.
That's
exposed
like
how
to
run
and
scale
services
are
all
mostly
on
the
computation
or
on
the
networking
side,
not
on
the
storage
stateful
side,
and
so,
if
you
are
using,
if
you're,
using
kubernetes
putting
your
stuff
in
containers,
even
using
the
volume,
abstraction
you're
able
to
run
new
containers
and
you're
able
to
interact
with
storage
services
that,
for
the
most
part,
live
outside
of
your
kubernetes
cluster.
A
So
rook
is
a
cloud
native
storage,
Orchestrator,
think
of
it
as
an
extension
to
kubernetes
that
adds
one
more
abstractions
beyond
volumes
and
to
the
ability
to
run
stateful
storage
services
inside
the
community
inside
the
kubernetes
cluster.
So
that's
those
are
the
two
things
that
work
provides.
It's
open
source,
it's
apache2,
Oh
license
and
it's
hosted
by
the
cloud
native
computer
foundation.
So
it's
it's!
A
You
know
a
project
program.
My
company
upbound
is
a
sponsor
off
the
project
and
that's
why
we're
we
like
to
kind
of
walk
around
talk
talk
about
it.
So
let
me
let
me
walk
you
through
what
that
means.
So
what
rook
does
is
it
helps
storage
systems,
things
like
Ceph,
cockroach,
TB,
mini
Oh
next
sentence.
There's
a
bunch
more
coming
essentially
run
like
a
service
on
top
of
a
kubernetes
cluster.
So
and
it
does
that
by
automating.
A
All
of
you
know
things
that
people
usually
do
by
scripts
or
others
into
logic
that
helps
these
services
run
within
the
cluster
and
and
then
extends
kubernetes
to
be
able
to
essentially
provide
these
abstractions
that
one
run
on
these
services
and
we'll
walk
through
this.
So
so,
for
example,
with
it's
something
like
Brooke
you
now,
if
you
wanted
to,
if
that
makes
sense,
for
your
deployment,
can
run
storage
systems
as
if
they
were
managed
services
inside
your
kubernetes
cluster,
so
think
a
typical
deployment
say
a
bare-metal
deployment,
if
you're
doing
bare
metal.
A
Where
are
you
running
outside
of
a
cloud
environment
where
you
don't
have
these
hosted
managed
services
and
you
still
want
to
run
the
database?
So
you
still
want
some
block.
Storage
Rooke
makes
it
easy
to
run
that
on
the
same
cluster
as
your
kubernetes
cluster,
so
you
don't
and
it
uses
kubernetes
itself
to
help
manage
these
services
and
scale
them
and
manage,
and
and
do
essentially
what
you
know
what
most
services
would
do,
what
the
cloud
vendor
would
do
outside
of
this
environment.
A
Another
option
is
to
say
well
great
if
I
wanted
to
do
that.
Well,
I
could,
if
I
wanted
to
I,
can
have
my
compute
cluster
be
separate
from
a
storage
cluster,
so
you're
able
to
say,
run
your
compute
on
one
side
of
the
house
and
still
use
kubernetes
to
manage
your
stateful
storage
services
on
the
other
side
and
scale
them
independently.
So
that's
another
option,
that's
available
to
you.
If
you
were
to
use
some
something
like
this.
So
to
give
you
an
example
of
what
Europe
does
say,
you
wanted
to
run
yours.
A
The
storage
nodes,
the
Mon,
build
quorums,
deploy
and
run
storage
nodes
to
expose
your
own
EBS
or
your
own
s3
service
within
your
kubernetes
cluster.
So
that's,
that's
a
that's.
You
know,
that's
a
pattern
it
in
in
this
model.
You
are
now
running
your
storage
services
inside
your
kubernetes
cluster
and
rook.
Does
this
by
essentially
are
following
the
operator
pattern?
A
How
many
here
have
heard
of
the
kubernetes
operators
are
great,
and
so
essentially
it's
a
it's
continuously
reconciling
desired
state,
which
is
what
you
want
to
happen
with
actual
state
which
is
what's
actually
happening
in
the
cluster.
So
if
you
are
to
what's
cool
about
this
is
if
you're
a
new
storage
node
shows
up
and
it
matches
the
desired
state,
then
you'll
see
it
deploy,
become
a
new
storage
node
and
you
know
so
become
affiliated
with
the
storage
cluster
itself
or,
if
something
fails,
then
it's
remedied
as
the
system
heals
automatically.
A
So
we
talked
about
how
volumes
exist
in
kubernetes
within
the
rook
project.
We
are
actually
adding
buckets
and
databases
as
a
new
abstraction
in
kubernetes
and
enabling
many
stateful
services
that
sit
behind
them
to
essentially,
you
know,
extend
the
reach
of
the
write
once
run
everywhere
to
things
like
data
bases
and
buckets
so
you'll
see
us,
you
know,
there's,
please,
you
know
I'll
talk
about
how
you
can
get
involved,
but
definitely,
if
you
this
is
interesting
to
you.
You
should
definitely
get
involved
in
this
project.
A
So
as
an
example,
here's
a
you
know
a
persistent
bucket
claim,
which
is
a
new
API
that
enables
you
to
provision
and
manage
and
use
buckets
within
your
pods
and
your
deployments
without
really
knowing
where
who
the
provider
of
the
object
storage
is
just
as
with
volumes.
Kubernetes
gives
you
that
promise
today
and
then.
Similarly,
if
you
wanted
to
my
sequel
database
or
Postgres
database,
then
you
should
be
able
to
do.
A
The
same
thing
should
be
able
to
say
I'd,
like
my
sequel
database
as
part
of
my
deployment,
and
that
you
know
you
could
write
your
app
and
use
a
my
switch
every
favorite.
My
sequel
connector,
you
need
within
your
app,
but
the
provider
of
the
my
sequel
service
could
be
RDS
in
one
cluster
and
it
could
be.
You
know
my
sequel
running
in
kubernetes
in
another
write
that
level
abstraction
needs
to
exist
and
now
level
are
separated.
A
And
so,
if
you
had
that
you'd
be
able
to
say
you
know,
you
have
a
choice,
you
could
say:
I
can
ride
my
pods,
they
use
a
my
sequel,
a
post
grass
or
a
bucket
and
volumes
and
they're
now,
stateful,
apps,
very
stateful
will
use
different
kinds
of
storage
and
the
choice
could
be
whether
the
provider
of
the
storage
could
be.
You
know
a
public
cloud
managed
service
if
that's
where
you
deployed
or
it
could
be
in
cluster
or
it
could
be
in
a
separate
cluster.
A
A
A
If
you're,
also
interested
in
up
at
come
talk
to
me
or
Steve
here,
I
was
also
come
up
and-
and
this
looks
interesting-
there's
lots
of
opportunities
that
continue
to
do
similar
stuff
in
in
hearings
in
Seattle.
So
thank
you.
Thank
you.
Papi
they'll
take
questions,
so
we
have
any
questions
from
the
room.
Yes
of
the
Bob.
A
Yes,
so
so
there
is
a
there's,
a
project
called
open
serviceworker,
which
I
think
is
stated
goal,
is
to
help
broker
access
to
manage
services
that
are
running
in
public
cloud
environments
or
other
environments,
and
the
abstractions
provided
by
the
open
service
broker
are
essentially
how
to
provision
a
binding
to
a
service
and
an
instance
of
a
service
that
is
running,
and
so
that
model
works.
If
you,
you
know,
the
more
everything
looks
like
a
service
endpoint,
and
you
know
a
binding
to
it.
That
model
is
definitely
fine.
A
In
fact,
you
can
argue.
Rook
should
use
that
when
it's
talking
to
things
like
RDS
or
others
right
well,
we
were
attempting
to
do.
Is
we
are
with
the
rook
we're
actually
trying
to
provide?
You
know:
high-level
abstractions,
like
a
bucket
which
could
have
commonality
across
the
different
providers
of
it,
and
then
attempting
to
do
a
separation
of
concern.
We're
just
like
volumes.
There
is
a
concept
of
a
bucket
in
a
claim
or,
and
that
is
vendor-neutral,
and
you
know,
and
also
implementation
independent
right.
A
But
we
are,
you
know
this
is
all
over
in
the
open
community,
we'd
love
to
figure
out
how
we're
actually
talking
to
folks
at
the
open
source
worker
team
and
trying
to
figure
out
how
do
these
things
you
know
come
together,
but
the
end
goal
is
I
think
it
is
the
same
we'd
like
to
get
the
goal
of
write
once
run
everywhere
to
actually
address
the
majority
of
what
people
run
in
terms
of
apps
and
services
and
not
just
the
compute
and
networking
piece.
That's
that's
our
that's
our
goal
here
to
question.
A
I
mean
so
that
that's
the
question
is:
do
you
think
that
the
bucket
implementation
and
the
objects
towards
providing
it
could
provide
the
same
SLA
as
a
a
managed
service
provider?
Would
I
think
that
berries
I
mean
a
lot
of?
It
is
a
function
of
who's,
running
the
object
store
and
what
they're
doing
to
run
the
object
store
in
some
cases
of
some
customers?
Don't
have
a
choice,
they'd
liked
for
certain
workloads.
They
need
to
have
say
on-premise
and
others
and
that's
what's
available
to
them,
and
they
work
with
a
vendor
right
to
do
that.
A
I
think
what
we're
seeing
is
essentially
there's
a
fair
amount
of
automation
that
has
been
built
to
run
things
like
object,
stores,
reliably.
A
lot
of
that
automation
today
is
in
public
cloud
vendors
and
they
obviously
give
you
an
SLA
and
they
carry
majors
and
they
do
the
whole
thing.
So
one
of
the
things
to
think
about
is
if
the
automation
increases
the
point
where
you
know
through
things
like
operators
and
others,
it
becomes
say
it
gets
you
80%
of
the
way.
Would
it
be
efficient
for
an
enterprise
to
run
with
a
the
20%.
A
Just
handle
the
20%
and
I
think
the
folks
that
are
running
any
kind
of
stateful
storage
services
in-house
are
are
doing
that
today.
Is
it
the
same
SLA?
Well,
it's
hard
to
say,
because
they
don't
there's
no
SLA
contract
between
them
and
themselves.
You
know
orally,
but
but
that's
not
it
I
guess.
The
thing
that
that's
more
interesting
to
us
is
that
the
choice
should
be
available
as
opposed
to
it's
one
or
the
other
right,
and
certainly
the
choice.
A
A
So
the
question
is:
how
does
data
persistence
and
recovery
work
in
this
in
this
tool?
A
lot
of
that
is
a
function
off
the
actual
storage
provider
itself.
So,
if
you're
looking
at
something
like
Stef,
Romania
or
cockroach,
DB
or
others
that
run
on
on
something
like
this,
they
all
have
provisions
for
recovering
data
and
resilience
around
failures
and
dealing
with
failure
cases.
A
So
what
this
provides
is
the
ability
to
kind
of
marry
that
with
kubernetes,
so
that
if
a
node
fails
in
kubernetes
that
you
know,
the
storage
provider
can
actually
trigger
its
resilience
facilities
or
healing
facilities
or
if
things
grow
and
scale
that
you
know,
as
you
add
nodes,
that
the
cluster
itself
grows
and
becomes
stronger
and
faster,
but
each
of
the
storage
systems
have
in
very
different
ways
of
dealing
with
that
SEF.
Does
it
one
way
cockroach?
Does
it
a
different
way?
Some
use
erasure
coding,
some
use
redundancy
replicas
and
it
varies
by
storage
system.