►
From YouTube: Cloud Native Storage - a View of the Landscape, Projects... Alex Chircop, Xing Yang & Quinton Hoole
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Cloud Native Storage - a View of the Landscape, Projects and Technology - Alex Chircop, StorageOS; Xing Yang, VMware & Quinton Hoole, Facebook
This talk will discuss how the CNCF storage SIG in the CNCF operates, identifies projects for Cloud Native admission and where we see the future of storage in the Cloud Native Ecosystem heading.
During this session we will cover:
- Overview of the SIG, how to join and how to help
- Overview of storage projects in the CNCF
- Projects that are currently being being reviewed
We will also share updates of our latest work including:
- the CNCF Storage Landscape document
- the Performance and Benchmarking document
- the Cloud Native Disaster Recovery document
A
Good
morning,
good
afternoon
or
good
evening,
welcome
to
this
session,
where
we're
going
to
talk
about
some
of
the
work
that
we're
doing
in
the
cncf
storage
sig,
covering
a
view
of
the
landscape,
the
projects,
the
technology.
That's
that
we're
working
on
and
some
of
the
interesting
things
that
are
happening
in
the
cncf
relating
to
storage.
A
B
A
A
Share
some
information
on
how
to
do
that
in
a
second.
So
who
are
we
there's
a
diverse
group
of
people
who
form
the
sig?
They
some
are
independent
contributors.
Some
are
working
for
different
vendors
or
or
organizations,
but
in
general,
we're
all
leaders
and
early
adopters
in
the
in
the
storage
space.
A
A
The
the
important
thing
here
is
we
we
provide
that
storage
experience
and
and
advisory
services
to
to
the
toc,
as
well
as
providing
a
number
of
a
number
of
documents
and
other
educational
materials
to
the
end
user
community.
At
the
end
of
the
day,
we
we
want.
Our
mission
is,
is
to
be
able
to
improve
the
number
of
projects
in
the
storage
space
and
improve
the
adoption
of
cloud-native
storage
in
in
environments
for
the
cncf.
A
So
a
large
part
of
the
work
we
do
is
to
provide
easy
to
read,
documentation
and
materials
that
help
end
users
understand
all
the
different
technologies.
The
the
landscape
is
is
a
of
either
a
fast
and
varied
projects
and
technologies,
and
so
you
know
it's
really
important
to
try
and
document
that
and
cover
all
of
the
different
products
and
practices
and
trends
and
technologies,
etc,
that
we
have
that
we
have
in
the
environment.
A
Two
of
the
important
documents
that
we
have
provided
that
we
have
provided
are
the
cncf
storage
white
paper
and
jing
will
go
through
that
in
a
shortly
as
well
as
some
new
documents
like
the
performance
and
benchmarking
white
paper,
which
are
developments
and
following
feedback
from
from
our
original
storage
white
paper
as
much
as
possible.
The
information
that
we're
providing
is
is
based
on
on
on
research
and
the
collective
experience
of
of
the
tech
leads
and
the
community
within
the
sig.
A
A
Sandbox
projects
are
there
to
provide
an
experimental
zone
within
the
cncf
to
allow
collaboration
and
allow
projects
to
to
build
their
community.
The
idea
is
that
sandbox
projects
have
a
low
barrier
to
entry
and
they
help
and
they
help
the
projects
and
build
and
perhaps
ratify
things
like
their
licensing
and
ip
policies
to
be
compatible
with
the
cncf
incubation
projects
are
where
we
do
a
lot
of
due
diligence.
A
So
as
a
sig,
we
help
the
toc
review
and
and
document
the
the
different
projects
as
they
go
through
this.
This
life
cycle
within
the
environment,
when,
when
a
project
goes
through
incubation,
for
example,
we
we
usually
have
the
project
presents
to
the
to
the
sick
and,
following
that
presentation
and
a
review
with
the
sig
will
provide
a
recommendation
to
the
toc
to
move
forward
with
due
diligence.
A
One
important
point
to
note
here
is
that
the
sig
is
also
very
keen
to
learn
about
new
projects
and
new
open
source
projects
in
the
storage
space.
So
we
often
have
presentations
from
from
projects
who
are
interested
in
joining
the
cncf,
either
at
incubation
or
or
at
sandbox,
and
those
and
those
presentations
are
provides.
You
know
valuable
material
and
valuable
input
to
the
to
the
toc
in
in
in
their
decision
making
process.
A
A
One
thing
to
bear
in
mind
is
that,
whilst
we're
considered
advisors
to
to
the
toc-
and
we
help
with
the
technical
due
diligence
as
well
as
in
project
reviews
during
their
life
cycle
within
the
cncf
we're
we
are
advisors
and
and
and
that
is
the
key-
and
that
is
the
key
point
here.
So
so,
ultimately,
in
all
cases
the
toc
are
responsible
for
making
the
final
decisions
in
any
in
any
in
any
project.
Adoption
or
project
review
process.
A
So
how
can
you
get
involved?
Well,
we
have.
We
have
a
fantastic
community.
We
cover
a
huge
range
of
different
topics,
everything
from
management
frameworks
and
and
tooling
block
stores,
file
systems,
subject,
stores,
key
value
stores,
databases,
storage
in
cloud
length
of
storage
is,
is
coverage
covers
a
wide
variety
of
topics
and
would
be
very
interested
to
to
hear
from
you.
Please
do
consider
joining
our
meeting
and
signing
up
to
to
our
mailing
list.
B
Thanks
alex
I'm
going
to
talk
about
graduated
and
incubating
story
projects,
juke
is
a
graduated
project.
It
provides
storage
operators
for
kubernetes,
zhuk
turns
distributed
storage
systems
into
self-managing,
soft
scaling
and
self-healing
storage
services,
rook
orchestrates,
multiple
storage
solutions,
each
with
a
specialized
kubernetes
operator
to
automate
management.
It
currently
supports
saf,
cassandra
and
nfs.
B
B
B
Etcd
is
a
graduated
project.
It
is
a
distributed
key
battery
store
that
provides
a
reliable
way
to
store
data
across
a
cluster
of
machines.
All
kubernetes
clusters
use
etcd
as
their
primary
data
store.
It
handles
storing
and
replicating
data
for
kubernetes
cluster
state
and
uses
the
raft
consensus
algorithm
to
recover
from
hardware
failure
and
network
partitions
tech.
Tv
is
a
graduated
project.
B
B
B
We
also
have
a
few
sandbox
storage
projects.
Triple
fs
is
a
cloud
native
third
platform
that
provides
both
posix
compliant
and
s3
compatible
interfaces.
Project
longhorn
implements
distributed
block
storage,
using
containers
and
micro
services.
Lancorn
creates
a
dedicated
storage
controller
for
each
block
device.
Volume
and
synchronously
replicate
the
volume
across
multiple
replicas
stored
on
multiple
nodes
project.
B
Project
priors
is
the
cloud
native
search
system
that
enables
kubernetes
local
precision
volumes
with
dynamic
provisioning
resource
management
and
high
availability
and
project.
Proviga
is
an
open
source,
distributed
storage
service
implementing
streams
it
offers
stream
as
the
main
primitive
for
the
foundation
of
storage
systems.
B
B
B
B
Similarly,
the
performance
of
a
storage
system
can
measure
against
different
criteria
latency
to
perform
a
storage
operation.
The
number
of
search
operations
that
are
possible
per
second
and
the
throughput
of
data
that
can
be
stored
or
retrieved
per
second
consistency.
Attributes
of
a
student
system
refer
to
the
ability
to
access
newly
created
data
or
updates
to
the
same
after
it
has
been
committed.
B
B
B
There
are
several
storage
layers
that
can
impact
the
storage
attributes,
for
example,
rather
than
directly
access
resources,
a
hypervisor
can
provide
access
to
resources
which
could
add
excess
overhead
storage.
Topology
describes
the
arrangement
of
storage
and
compute
resources
and
the
data
link
between
them.
This
includes
centralized
distributed.
Sharded
or
hyper-converged.
Topologies
search
systems
usually
have
data
protection
layer
which
adds
redundancy.
B
B
There
are
two
categories
of
data
access
interfaces
here
we
call
them
volumes
and
api
co
has
interfaces
for
volumes
which
supports
both
block
and
file
system
in
kubernetes.
There
are
two
body
modes
file
system
and
block
processing
mode
allows
workloads
to
consume
file
system
directly,
underneath
it
can
be
either
file
system
or
block
block
mode
allows
roblox
device
to
be
consumed
directly
by
the
workload
on
the
api.
We
have
object,
store
api,
that
stores
or
retrieves
objects
or
blocks.
B
B
It
could
also
have
a
high
latency
and
high
throughput
note
that
there
is
a
kubernetes,
sig
storage,
subproject
called
cozy,
which
introduces
kubernetes
apis
to
support
oxidation
of
object,
store
operations
for
kubernetes
workloads,
therefore
bring
in
object
storage
as
the
first
class
citizen
in
kubernetes.
Just
like
file
and
block
storage.
B
It
also
introduces
container
object,
storage
interface
or
cosy
as
a
set
of
grpc
interfaces
for
provisioning
object
stores.
It
is
targeting
alpha
in
kubernetes
1.22
release
under
api.
We
also
have
key
value
stores,
which
use
an
api
to
store
retrieval
values
from
source
based
on
a
key.
It
supports
strong
consistency.
B
It's
typically
used
to
store
state
or
configuration
for
distributed
systems
on
the
api.
We
have
databases.
Databases
are
typically
accessed
through
an
api.
In
the
past,
the
term
database
used
to
be
synonymous
to
a
relational
database.
However,
there
are
now
many
new
sql
systems,
as
well
as
specialized
ones
like
graph
databases
that
get
categorized
as
databases.
B
B
B
B
B
The
orange
box
here
is
called
frameworks
and
tools.
This
is
an
extension
of
control
plane
interfaces
in
addition
to
provisioning.
These
interfaces
could
provide
functionalities
such
as
discovery,
automation,
data
services
such
as
data
protection,
disaster,
recovery
data,
migration
and
data
lifecycle,
management
and
so
on
for
application
api,
including
keyword,
restore
and
databases.
B
B
B
A
B
A
Of
storage
in
the
future,
one
of
the
one
of
the
things
that
we
realized
as
we
as
we
were
talking
about
the
different
options
and
tools
that
that
could
be
used
to
to
measure
performance
and
benchmarking
within
cloud
native
environments.
Was
there
actually
is
a
number
of
of
problematic
common
pitfalls
that
that
a
lot
of
people
fall
into.
A
Of
different
systems,
of
course
it's
important
to
understand.
You
know
some
of
the
basics.
When
jing
was
describing
those
attributes.
You
know
we
could
see
that
the
attributes
can
can
be
measured
across
multiple
axises,
perhaps
operations
per
second
and
also
throughput,
and
it's
probably
fair
to
say
that
you
need
to
have
an
understanding
of
your
use
case
to
be
able
to
understand
which
of
operations
and
throughput
are
more
important
to
to
to
your
environment.
A
A
Data
protection
having
having
commonality
on
data
protection
is,
is
another
key
aspect,
so
whether
the
data
is
whether
the
storage
system
is
implementing
some
sort
of
replication
or
perhaps
erasure
coding,
has
a
big
factor
in
the
performance.
Data
reduction
can
also
can
also
impact
performance
in
in
many
different
ways.
A
A
You
know
that
can
be
measured
in
in
microseconds
or
or
milliseconds
typically,
and
that
gives
you
an
idea
of
the
number
of
operations
per
second,
but
very
often,
if
you're,
if
you're
thinking
about
operations
on
a
website
or
the
number
of
database
transactions
that
can
be
run
through
a
system,
latency
is
often
is
often
the
the
key
measurement
that
you'll
want
to
focus
on.
As
that,
the
low
latency
has
a
direct
relation
to
the
number
of
transactions
per
second
that
the
system
can
implement.
A
So
if
the
application
is
is
able
to
utilize
the
concurrency
within
the
storage
environment,
then
you
can
maximize
number
of
operations
or
throughput,
but
if
the
application
is
is
very
serialized,
for
example,
and
then
it
can't
make
use
of
that
concurrency
one
other
big
factor
is
is
caching.
This
can
happen
at
multiple
layers
within
the
storage
within
the
storage
topology.
A
Starting
from
you
know,
the
operating
system,
layer
and
file
system
caches.
All
the
way
down
to
you
know
different
storage
layers
and
different
storage.
Subsystem
layers
and
caching
is-
is
one
of
those
things
that
often
catches
people
out
when
they're
doing
when
they're
doing
benchmarks,
and
that's
because
for
practicality
reasons
they
might
be
testing
volumes
or
databases
of
a
small
size
to
allow
to
allow
the
benchmark
to
to
happen
easily.
A
But,
of
course,
if
the
data
set
that
you're
testing
is
significantly
smaller
than
the
the
memory
of
the
nodes
that
you're
that
you're
running
that
load
on,
then
the
reality
is
you're
you're,
probably
making
a
lot
of
benefit
you're,
getting
a
lot
of
benefit
from
the
cache
and
you're,
probably
testing
the
speed
of
the
cache,
as
opposed
to
the
speed
of
the
storage
system.
So
that's
that's.
A
The
white
paper
that
we're
developing
is
is
open
for
comments.
We
really
really
appreciate
any
contributions
or
or
comments
to
to
to
the
document
we're
also
working
on
another
document
relating
to
availability
and
disaster
recovery.
This
is
this
is
document.
That's
very
close
to
one
of
our
tech
leads
hearts.
Rafaeli
who's
been
focusing
on
this.
For
us
we
talk
about.
A
We
talk
about
availability
and
consistency,
for
example
in
in
the
storage
white
paper,
but
but
this
document
aims
to
to
go
down
a
bit
further
and
and
look
at
some
of
the
architecture,
considerations
for
implementing
disaster
recovery
in
cloud
native
environments.
So
when
we
talk
about
high
availability,
we're
often
asking
the
question,
you
know
what
happens
when
a
component
in
in
a
failure
domain
is
is
lost.
A
When
we
talk
about
failure
domain,
it's
you
know
we're
looking
at
more
than
one
individual
component,
you
know
so
we're
looking
at
multiple
nodes
or
a
cluster
or
or
perhaps
an
entire
data
center.
And
so
when
we
talk
about
disaster
recovery,
we're
now
we're
now
elevating
availability
to
the
failure
of
or
the
loss
of,
an
entire
failure
domain.
So
what
happens
if
you
lose
an
entire
cluster?
A
But
the
idea
here
is-
and
this
is
just
you
know-
a
sample
reference
architecture.
The
idea
here
is
that
some
sort
of
low
balancing
capability
will
give
you
access
to
to
to
multiple
ingresses
and
front
ends
that
will
be
able
to
see
a
stateful
that
will
be
able
to
implement
a
stateful
workload
and
persist
the
data
in
a
consistent
way
across.
You
know
multiple
failure.
B
A
Without
that
coordination
and
storage
layer,
you
also,
you
will
often
be
able
to
reduce
the
recovery
time
and
recovery
point
objectives,
and
but
of
course,
that
depends
on
the
entire
architecture,
from
top
to
bottom
being
implemented
to
to
facilitate
this
this
disaster
recovery.
So
it's
not
just
the
storage
layer,
but
also
the
the
networking
and
the
east-west
communication
and
load
balancing
capabilities
for
that
ingress.