►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Intro to the CNCF SIG Storage - Quinton Hoole, Futurewei & Alex Chircop, StorageOS
The CNCF SIG Storage collaborates to explore and understand how different storage technologies are used in cloud-native environments. Topics include block stores, file systems, object stores, key-value stores and databases, amongst others. Different architectural approaches (centralized, distributed, sharded etc) are compared in terms of key attributes like availability, scalability, performance, data consistency, durability, fault tolerance, ease of development and operational complexity. In the intro, we'll provide an overview of the work currently underway and on the roadmap. You'll get the meet the people leading these efforts, and find out how best to get involved and contribute. We'll present our work thus far, and host discussions around the findings of our recent end-user surveys covering how cloud native storage is approached in the real world, including some representative successes and failures. Our hope is that others can learn from, and expand upon those experiences.
https://sched.co/W87B
B
I'm
Quinton,
who
all
some
of
you
might
have
seen
me
in
previous
lives
in
different
roles,
work
for
future
way
in
the
US
and
yeah
welcome
everyone.
This
can
be
a
very
interactive
session.
So
then
be
shy
to
interrupt
me.
If
you
have
questions
or
comments,
those
will
be
most
welcome.
Alex
and
I
will
be
doing
the
presentation.
We're
co-chairs
of
the
CN
CF
sig
storage,
as
together
with
Erin
who's
sitting
right
over
there
and-
and
we
also
have
we'll,
go
through
some
other
details.
We
have
some
tech,
leads
etc
as
well.
B
So,
first
of
all,
yeah
I'd
like
to
just
give
you
an
overview
of
what
is
a
what
is
a
CN,
CF,
sig
I'm
sure
many
of
you
might
have
come
across
kubernetes
SIG's
in
the
past.
Sig
stands
for
special
interest
group,
of
course,
which
is
common
with
many
other
competing
organizations
around
the
world.
The
CN
CF
SIG's
are
different
from
the
kubernetes
SIG's,
in
the
sense
that
community
SIG's
tend
to
be
very
focused
on
kubernetes,
specific
implementations,
projects,
etc.
Cn
CF
SIG's
have
a
slightly
different
role
to
play
as
you
familiar
with.
B
B
Let's
just
say
that,
so
as
this
this
is
grown.
Obviously
the
number
of
projects
has
grown
as
well.
The
first
you
know
the
first
cubic
on.
We
probably
had
I,
don't
remember
off
the
top
of
my
head
six
or
seven
projects,
I
believe
we
have
forty
now
you
know
at
least
close
to
ten
of
which
are
graduated,
so
the
amount
of
bandwidth
required
of
the
TRC,
the
TRC,
is
still
the
same
size
as
it
was
on
day
one.
B
So,
in
order
to
allow
it
to
scale,
we
decided
to
form
these
special
interest
groups
partition
the
the
technical
space
up
into
about
six
or
seven
areas
and
then
enable
the
community
to
get
involved
in
the
in
their
areas
of
speciality
and
storage
is
obviously
one
of
those.
You
could
look
for
details,
there's
some
links
on
the
screen
there.
B
If
you
want
to
find
out
some
more
about
what
the
SIG's
are
and
what
they
do,
the
main
objectives
are
to
strengthen
the
project
ecosystem,
so
that
involves
taking
existing
CN
CF
projects,
making
sure
they're
healthy,
making
sure
that
they're
not
running
into
any
problems.
I've
been
kind
of
parachuted
into
one
or
two
of
them
that
ran
into
some
some
issues
and
we
managed
to
sort
those
out.
We
also
need
to
identify
gaps
in
the
ecosystem.
Clearly,
we
we
know
we're
over
over
the
years.
B
We've
we've
built
up
a
reasonable
portfolio
of
projects,
but
there's
still
some
fairly
glaring
holes
in
that
in
that
ecosystem
storage,
I
think
is
actually
fairly
well
covered,
but
we'll
go
into
that
a
little
later.
We
also
want
to
educate
and
inform
users
about
the
particular
area
of
that
that
is
covered
by
that
sig.
So,
to
give
you
an
example,
I
don't
know
if
we
have
them
in
the
slides
here,
maybe
we'll
carry
them,
carry
them
over
later
or
mention
them
later.
B
But
there's
you
know,
security
storage,
networking,
the
the
fairly
obvious
areas
that
you
would
imagine
and
a
lot
of
these
areas
are
non-trivial
storage.
In
particular,
you
know
there
are
many
different
layers
of
storage
block
up
to
objects
and
databases
etc,
and
there
are
different.
You
know,
aspects
of
all
of
those
things.
B
The
API
is
the
management,
the
properties
of
the
storage,
so
there's
quite
a
lot
to
to
kind
of
wrap
your
head
around
to
be
able
to
make
sense
of,
and
so
we
try
to
do
a
lot
of
that
hard
work
on
behalf
of
all
of
the
users.
So
that
they
can
consume
some
of
our
educational
material,
fostering
project
maturities
make
sure
the
project's
move
forward,
eventually,
hopefully
either
graduate
or
in
some
cases
we
actually
retire
projects
and
archive
them.
B
There
are,
you
know
many
different
projects
in
the
cnc
of
both
with
in
storage
and
elsewhere,
and
we
need
to
make
sure
we
understand
the
relationships
between
them.
How
do
they
integrate
together,
work
together,
etc,
new
TRC
contributions
and
recognition
for
people
who
want
to
you
know
you
may
not
yet
be
sufficiently
experienced
and
in
this
kind
of
environment,
to
go
and
stand
as
a
TRC
member?
B
These
are
very
you
know,
seasoned
individuals,
typically
who's
been
around
for
a
very
long
time,
but
even
if
you're
not
feel,
if
you
don't
feel
comfortable
at
that
level,
yet
you
can
start
by
contributing
to
a
sig.
You
can
graduate
to
a
technical
lead
or
a
chair
of
a
sig,
and
then
you
can
eventually,
you
know,
perhaps
work
your
way
up
to
being
on
the
technical,
Oversight,
Committee
and
yeah.
One
of
the
important
aspects
of
all
of
this
stuff
is
to
be
vendor-neutral.
You
know
there
are
a
lot
of
competing
vendors
in
this
space.
B
We
like
that
fact.
We
think
that
that
it's
very
good
that
we
have
this
very
healthy
ecosystem
of
commercial
vendors,
but
you
know
one
does
also
have
to
be
mindful
of
the
fact
that
sometimes
these
they're
there
are
competitive
situations
that
can
arise
that
are
not
in
the
interests
of
the
community
or
in
the
interests
of
this
CNC,
if
in
general,
so
that
that's
another
one
of
our
roles
being
vendor
neutral,
making
sure
that
we
disseminate
accurate
objective
information
as
best
we
can
I
think
I'm
going
to
skip
over
some
of
these
details.
B
We
have
you
know
all
of
this
stuff
is
documented
on
the
CNC
F
website.
If
you
want
to
find
out
more
about
the
SIG's,
what
their
functions
are
you
can
you
can
find
out
some
of
the
detail
there
and
you
user
education
I,
think
we've
we've
covered
many
of
the
cigs,
so
of
the
several
that
we've
actually
specified.
Not
all
of
them
are
actually
operational.
B
B
We
do
various
community
enablement
kinds
of
exercises
and
we
form
or
we
perform
a
function
as
trusted
expert
advisors
to
the
TRC,
so
that
the
TRC
members
are
typically,
you
know
very
many
of
them
are
CEOs
of
companies,
etc.
They
don't
necessarily
have
the
bandwidth.
Many
of
them
have
the
technical
acumen
and
the
experience,
but
they
don't
necessarily
have
the
time
to
do
a
lot
of
the
hard
work
that
gets
done.
For
example,
in
doing
technical,
due
diligence
azan
projects
that
apply
to
be
part
of
the
CNC
F.
B
We
want
to
make
sure
that
they
get
included
at
the
correct
the
appropriate
level.
We
don't
want
to
have
you
know
relatively
young
projects
with
with
inappropriate
levels
of
stability
or
community
involvement,
being
you
know,
as
promoted
to
two
levels
that
indicate
that
they're
something
other
than
that
and
we
have
a
logo.
This
is
the
new
SIG's
storage
logo,
I
think
approved.
Probably
a
week
ago,
Alex
we're
very
proud
of
our
new
logo.
We
even
have
stickers
if
anyone
wants
a
cig
storage
sticker
for
your
laptop
or
otherwise
stick
around
afterwards.
B
Hopefully,
alex
has
a
pocket
full
of
them.
Why
is
storage
so
critical?
So
now
we're
moving
on
to
the
you
know,
most
storage,
specific
stuff
and
less
about
tier
C,
rather
cigs
in
general.
So
there's
a
lot
of
talk
about
stateless
applications.
People,
you
know
say:
hey.
Everybody's
applications
are
stateless.
There's
no,
there's
really
genuinely
no
such
thing
as
a
stateless
application.
All
applications
have
some
other
state,
they
store
it
somewhere.
Sometimes
they
delegate
that
storage
to
you
know
a
public
cloud
provider
service
or
something
else,
but
very
few
real
world
applications
are
truly
stateless.
B
B
Cloud
native
storage,
in
particular,
is
is
a
little
different
than
the
traditional
storage
systems
that
were
perhaps
and
use
before
the
cloud
came
along.
Many
of
them
tended
to
be
fairly
centralized,
often-
and
you
know,
very
expensive
commercial
kinds
of
storage
there's
been
a
shift
as
you're,
probably
aware
of
the
last
decade
and
a
half
towards
more
software
oriented
more
distributed
kinds
of
storage.
They
provide
scalability
properties
that
are,
in
some
cases,
more
suitable
to
certain
kinds
of
applications
than
more
monolithic
storage
solutions.
B
They
often
more
portable
as
well,
because
you
can
you
know
you
can
run
these.
These
storage
solutions
on
different
public
clouds
or
on-premise
and
there's
a
lot
of
integration
between
the
various
different
storage
provided,
the
storage
systems
that
are
provided,
so
they
they
they
work
well
together.
What
is
our
mission
statement?
B
We
cover
you
know
everything
you
could
reasonably
consider
to
be
storage,
we
consider
to
be
in
scope.
So
this
includes
block
stores,
file
systems,
object,
stores,
databases,
key
value
stores
and
all
the
related
caching
mechanisms
that
are
typically
found
around
these
things
and
what
we
want
to
really
focus
on
is
is
understanding,
and
this
is
an
area
where
people
get
some
sometimes
a
little
confused.
I
just
had
a
look
at
the
time,
did
I
just
overshoot.
My
time
dramatic.
B
We
we
want
to
really
make
sure
that
people
understand
the
fundamental
characteristics
of
the
different
kinds
of
storage,
so
they're
things
like
availability,
scalability,
performance,
durability,
consistency.
These
are
all
you
know,
pretty
words,
but
there's
there's
quite
a
lot
of
detailed
content
underneath
those
words
and
it's
important
when
you're
making
it
trying
to
make
a
choice
between
different
storage
solutions
and
Cebu's.
Here,
one
of
our
techniques,
it's
important
that
you,
you
have
a
you
know
fairly
clear
understanding
of
what
these
things
mean
and
what
the
various
trade-offs
are.
There
is
really
genuinely.
B
B
This
is
a
list
of
the
people
that
I've,
some
of
whom
I've
mentioned
already
Aaron
is
sitting
in
the
third
row.
Sugu
is
sitting
in
the
first
row
and
Ying
is
here
as
well.
Singing
sorry
and
I.
Think
some
of
the
others
are
missing.
Saad
I'm
not
sure
if
he's
around
but
they're,
all
at
the
conference
for
the
most
part,
didn't
necessarily
make
it
today
and
then
we
have,
you
know
a
bunch
of
interactions
with
other
related
groups.
B
A
So
we've
had
a
number
of
different
project
presentations
covering
everything
from
databases,
as
well
as
things
like
CSI
different
storage
products,
which
cover
everything
from
block
two
to
file
system,
and
recently
we
were,
we
were
also
reviewing
dragonfly,
which
is
a
a
container
registry.
So,
when
we're
looking
at
the
different
projects,
where
we're
looking
to
help
raise
awareness
of
the
project,
but
primarily,
were
there
to
gather
and
to
gather
feedback
for
presentation
to
the
TOC
so
so
effectively
part
of
this.
A
This
scaling
process
with
the
TOC
means
that
we
can
provide
them
with
the
information
needed
and
a
bit
more
of
a
technical
review
of
the
project
before
they
have
a
TRC
project
presentation.
So,
over
the
last
over
the
perhaps
last
18
months,
there
have
been
a
number
of
CN
CF
storage
projects
that
have
gone
through
the
process.
A
Rook
was
accepted
as
a
sandbox
project
and
is
now
a
very
successful
incubation
project.
Mineo
is
in
the
process
to
Vitesse,
as
you've,
probably
seen
from
sugru
on
the
keynote
stage.
Yesterday
I
just
graduated,
which
is,
which
is
an
amazing
milestone
and
achievement
at
CD,
which
is,
which
is
the
the
key
value
store
behind
most
kubernetes
deployments
is,
is
as
part
of
that
project
similarily
RTI
kV,
which
is
which
is
a
sandbox
project,
and
now
an
incubation
project,
which
is
another
key
value
store
and
also
the
basis
for
the
TI
DB
database.
A
So
the
nice
thing
is
that
there
are
there's
quite
a
wide
storage
landscape
available
and
covering
everything
from
you
know:
commercial
implementations
and
open
source,
implementations
and
client
services.
So
so
you
know
to
be
deployed
on
pram
or
or
in
client,
and
each
of
those
systems
have
have
a
bunch
of
different
attributes.
So
we
trying
to
we
tried
to
define
the
attributes
that
of
these
different
storage
systems
so
that
you
can
match
them
with
what
your
application
requires.
A
We
also
made
made
a
decision
to
to
make
sure
that
storage
wasn't
just
interpreted
as
a
volume,
so
we
kind
of
grouped
storage
into
two
main
categories,
one
in
a
volume
bucket
and
one
and
in
that
half
application.
Api
is
right.
So
so
the
volumes
are,
are
things
like
block
stores
and
file
systems
and
shared
file
systems,
and
the
application?
A
Api's
includes
things
like
databases
and
key
value
stores
and
object
stores,
and
we
also
worked
quite
hard
to
define
a
management
interfaces
which
are
obviously
really
important
to
make
storage
a
first-class
declarative
resource
in
in
kubernetes
right
because
in
in
much
the
same
way,
that's
your
pod
definition
can
now
define
the
CPU
requirements
and
the
memory
requirements
and
the
network
requirements.
You
should
also
just
be
able
to
define
your
storage
requirements
and
the
attributes
of
that
storage.
A
So
we
we
split
the
the
storage
attributes
into
into
five
main
groups,
availability,
scalability,
performance,
consensus,
tea
and
durability,
and
in
reality
you
know.
Different
storage
systems
have
pros
and
cons
in
each
one's
of
those
groups
so,
and,
and
and
it's
and
it's
very
often
that
it's
work
very
often
fans
that
different
different
attributes
in
one
area
that
are
that
that
excel
in
one
particular
storage
system,
meaning
that
it
has
you
know
consistent
cons
or
or
issues
in
another
area.
A
So
so
we
look
at
you
know
things
like
availability,
which
include
things
like
failover,
but
also
the
ability
to
move
data
between
nodes
and
enable
that
that
portability,
scalability
can
often
be
measured.
In
terms
of
you
know,
the
number
of
clients
or
the
operations
per
second
or
throughputs
that
that's
going
through
a
system
and
performance
is,
is
often
measured
in
terms
of
latency
operations
or
through
boots,
and
different
different
systems
have
different
requirements.
A
So,
for
example,
a
database
might
have
a
very
latency
focused
requirement,
whereas
perhaps
an
analytics
or
a
machine
learning
use
case
might
have
a
very
throughput
based
focus
and
obviously,
in
all
of
these
distributed
systems.
Consistency
and
durability
are
also
key
right.
So
many
systems
are
engineered
with
a
fine
balance
between
performance
and
consistency
in
order
to
enable
that
scale.
A
The
other
thing
that
we
defined
in
the
in
the
landscape
white
paper
is
the
different
storage
layers
that
you
typically
find
within
a
storage
system.
So
when
we
first
started
talking
about
this,
we
tried
to
categorize
some
of
the
attributes
of
storage
systems
based
on
how
those
storage
systems
are
consumed.
So
perhaps
you
know,
a
block
store
might
be
known
for
low
latency
and
shared
file
might
be
particularly
good
for
scaling,
for
example,
but
the
reality
is
that
the
the
different
layers
in
a
storage
system
are
really
important.
A
Right,
where
you
know
you
might
have
a
file
system
which
is
layered
on
top
of
an
object
store.
So
it
might
have
the
shared
file
properties
of
a
shared
file
system,
but
the
latency
properties
of
an
object
store,
for
example,
and
and
those
are
important
things
to
understand
the
some
of
the
new
things
that
we're
working
on.
So
the
database
is
addition
to
the
white
paper.
This
helps
this.
This
adds
in
a
database
section
to
the
white
paper
which,
which
we
missed
out
previously
just
due
to
lack
of
time.
A
So
when
we
first
put
together
the
white
paper,
we
said:
okay,
let's,
let's
document
the
landscape,
let's
document
all
the
different
layers,
let's
document
the
difference,
apologies
and
now
the
use
cases
are
are
building
on
top
of
that,
so
that
we're
able
to
say,
for
example,
look
for
the
specific
type
of
use
cases.
These
are
the
types
of
storage
systems
that
you
should
be
looking
at.
So,
for
example
apart,
you
know
this
use
case
might
be
suitable
for
a
shared
file
system.
A
So
the
the
the
key
takeaway
here
is
that
you
know
don't
look
at
published
results
for
making
comparisons
when
selecting
when
selecting
products
do
run
your
own,
your
own
benchmarks,
because
it's
it's
without
the
deep
understanding
of
the
exact
environments
that
the
benchmark
is,
is
running.
It's
really
hard
to
make
an
apples-to-apples
comparison,
and
in
that
respect
then
we're.
A
The
document
goes
on
to
actually
define
different
benchmarking
tools
for
benchmarking
volumes
and
databases,
along
with
a
number
of
tips
on
how
to
level
sets
the
the
environments
that
you're
using
for
the
benchmarking
and
finally,
please
do
get
involved.
All
of
the
six
storage
is
meetings
were
done
in
public.
We
meet
twice
a
month
on
the
second
and
fourth
Wednesday.
A
The
the
repo
is
part
of
the
CN
CF
github
repository
all
of
our
meeting
minutes
and
under
mailing
lists
are
public
and
what
we
specifically
would
like
help
with
is
we'd
like
you
to
provide
any
information
you
can
and
fill
in
the
CNC
F
storage
survey.
This
really
helps
provides
the
focus
and
the
information
that
we
before
the
the
next
set
of
projects
and
the
next
set
of
activities
that
we're
looking
at.
A
So
if
you're
running
a
project-
and
you
know
you're,
you're-
fed
up
with
users
asking
you
whether
what
the
best
source
solution
is
contributes
to
the
repo
and
we
can
start
pointing
the
users
that
way,
obviously,
if
we're
also
very
keen
to
to
have
any
help
in
boats
reviewing
or
contributing
contents
to
to
our
performance
and
database
white
papers
which
are
in
progress
and
we're
also
looking
for
we're,
always
looking
for
techniques
that
can
help
with
project
reviews.
And
that
mean
you
know
that
includes
reviewing
existing
projects
during
part
of
their.
A
You
know
annual
or
periodic
reviews,
as
well
as
reviewing
new
projects
which
are
being
considered
for
for
for
sandbox,
for
example,
and
finally,
we'd
really
love
to
know.
If
you
see
gaps
in
the
landscape
or
gaps
in
the
in
the
in
the
set
of
projects
where
you
think
there
are
particular
projects
that
would
make
really
good
additions
to
the
CSF
portfolio,
so
anything
anything
any
sort
of
feedback.
C
A
D
B
Documentation,
I
think
it
would
be
super
useful
to
you
know
what
the
use
cases
aim
to
do
is
say
you
know,
here's
a
fairly
concrete
problem,
and
this
is
how
one
might
solve
it,
and
these
are
the
pros
and
cons
and
maybe
there's
two
or
three
ways
of
solving
that
and
I
think
the
disaster
recovery
use
case
is
a
very
useful
one.
So
if
you're
I
don't
know,
if
you're
feeling
are
you
wanting
to
be
a
consumer
of
such
a
such
a
guidance
or
or
you
feel
equipped
to
provide
such
guidance?
B
C
A
E
A
B
Everything
here
is
done
in
the
open.
There's,
no,
you
know
back
room
where
things
get
produced.
Sometimes
you
might
have
to
dig
a
bit
and
the
best
way
to
do
it
is
you
know,
email,
the
cig
and
say:
where
are
you
guys
working
on
this?
You
know
document
and
you'll
be
able
to
get
a
link
and
and
comment
and
complain
and
contribute.
A
F
F
B
Believe
and
correct
me
if
I'm
wrong,
I
believe
the
procedure
is
you
just
join
the
mailing
list?
You
will
automatically
get
an
invitation
to
all
the
meetings
which
has
the
link
to
the
zoom
room
in
it
and
I
believe
you
will
get
permission
to
all
the
documents
as
well.
Typically-
and
you
know
in
theory,
if
somebody
sends
obscenities
to
the
mailing
list,
they
will
get
removed.
But
as
long
as
you
don't
do
that
you
should
be
fine.