►
From YouTube: Cloud Native Storage: The CNCF Storage TAG, Projects... Alex Chircop, Xing Yang, Raffaele Spazzoli
Description
Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Cloud Native Storage: The CNCF Storage TAG, Projects, Technology & Landscape - Alex Chircop, Ondat; Xing Yang, VMware; Raffaele Spazzoli, RedHat
This talk will introduce the CNCF Storage TAG and discuss how the TAG operates, how we work with CNCF Storage projects, and the work we have done to build guidance and write whitepapers for the ecosystem. During this session we will cover an overview of storage projects in the CNCF, including the broader ecosystem, as well as projects that are currently being reviewed. We will also share updates of our latest work including the CNCF Storage Whitepaper, Performance and Benchmarking whitepaper and the Cloud Native Disaster Recovery whitepaper. Join us to find out how to contribute and participate in the CNCF storage community and discover practical guidance on how to use cloud native storage in your environments.
A
A
A
A
But
then
we
realized
that
there
was
too
much
confusion
between
kubernetes,
cigs
and
cncf
sig,
so
we
renamed
the
cncf
sig
to
tags,
we're
we're.
We
meet
every
two
weeks
or
twice
a
month
on
the
at
eight
a.m:
pacific,
so
five
pm
european
and
we
we
have
all
of
our
calls
online
and
everything
is-
is
public.
So
please
feel
free
to
join.
A
The
tag
is
a
group
of
diverse
people.
We
have
individual
contributors,
we
have
representatives
from
different
storage,
vendors,
but
fundamentally
they're
all
expert
have
provide
expertise
in
our
storage
space.
We
have
a
number
of
coaches
tech,
leads
and
liaise
with
talk,
which
is
the
technical
operations
committee.
A
A
All
applications
everywhere
store
states
somewhere,
whether
it's
a
database,
a
file,
a
key
value,
store,
an
object,
store
or
a
file
system.
So
this
is
an
integral
part
of
of
each
parts
of
the
environment
and
and
whilst
in
some
of
the
cloud
native
spaces,
stateless
was
the
focus
for
a
number
of
years.
Stateful
applications
are
a
thing
and
all
the
database
providers,
all
the
key
value
stores
all
the
objects
etc
provide
a
plethora
of
solutions
for
this.
A
So
a
little
bit
about
what
cloud
native
storage
is
a
few
of
the
key
things
are
that
it's
declarative
and
composable.
So
in
much
the
same
way,
that
developers
should
be
able
to
define
cpu
memory,
networking
load,
balancers,
etc
in
in
for
their
environment
and
for
their
application.
They
should
also
be
able
to
define
the
requirements
that
they
need
from
storage
and
the
data
services
for
high
availability
and
disaster
recovery
and
we'll
touch
on
a
little
bit
of
that
later.
A
The
other
key
important
thing
is
that
cloud
data
storage
is
application-centric,
and
what
do
I
mean
by
that
today?
A
lot
of
storage
solutions
are
server-centric
and
we
think
of
of
storage
in
a
very
server
or
operating
system.
Way.
Where
say,
volumes
are
presented
to
a
particular
server
or
databases
are
installed
on
us
on
a
specific
server,
but
what
we?
A
For
example,
you
want
the
storage
to
be
portable
and
move
with
the
application
and,
as
as
your
application
scales,
and
fails,
overs
and
and
and
gets
automated
by
things
like
kubernetes
and
I'll
touch
on
this
last
point
here
as
well,
which
is
it's
a
bit
cliche,
then
some
people
roll
their
eyes,
but
but
I
also
think
that
cloud
native
storage
should
be
agile,
and
what
do
I
mean
by
that
in
cloud
native
environments?
You
have
lots
of
moving
parts,
nodes
come
and
go.
A
So
in
the
in
the
cloud
native
in
the
cncf
storage
tag,
one
of
the
things
we
did
to
help
end
users
work
is
we
created
a
storage
white
paper
which
describes
some
of
the
attributes
that
we
would
like
to
look
at
in
terms
of
in
terms
of
cloud
native
storage?
This
was
supposed
to
be
presented
by
zhing,
so
I'm
going
to
play
a
quick
five
minute.
Video
of
of
the
presentation
here.
C
Storage
systems
have
several
storage,
attributes,
availability,
scalability,
performance,
consistency
and
durability.
Availability
defines
the
ability
to
access
the
data
during
failure.
Conditions
scalability
can
be
measured
by
the
ability
to
scale
the
number
of
clients,
throughput
or
number
of
operations.
C
The
capacity
and
the
number
of
components
performance
can
be
measured
against
latency
the
number
of
operations
per
second
and
the
throughput.
Consistency
refers
to
the
ability
to
access
newly
created
data
or
updates
after
it
has
been
committed.
A
system
can
either
be
eventually
consistent
or
actually
consistent.
C
There
are
several
smudge
layers
that
can
impact
the
storage
attributes,
for
example,
rather
than
directly
access
resources,
a
hypervisor
can
provide
access
to
resources
which
could
add
excess
overhead
storage.
Topology
describes
the
arrangement
of
storage
and
compute
resources
and
the
data
link
between
them.
This
includes
centralized
distributed.
Sharded
and
hyper-converged.
Topologies
storage
systems
usually
have
data
production
layer
which
adds
redundancy.
C
C
C
C
Note
that
there
is
a
kubernetes
six
storage,
sub
project
called
cozy
container
object,
storage
interface,
which
introduces
kubernetes
apis
to
support
orchestration
of
object,
store
operations
for
kubernetes
workloads.
It
also
introduces
cozy
as
a
set
of
grp's
interfaces
so
that
a
object
story
vendor
can
write
a
driver
for
provisioning
accessing
object
stores.
C
C
C
A
Thank
you
virtually
so
we'll
move
on.
So
after
we,
after
we
put
together
the
storage
white
paper
which
which
effectively
covers
the
attributes
and
the
different
layers
of
the
storage
system
and
how
they
interact
with
each
other,
which
is
which
is
so
important
to
to
to
understand
nowadays,
because
so
many
of
the
systems
that
we
use
are
formed
of
composite
layers.
A
So
we
put
together
a
white
paper
on
performance
and
what
we're
looking
to
build
in
here
and
we're
still
open
to
to
drafts
by
the
way.
So
anybody
who's
who's
open
to
contributing
is
very
welcome.
We're
covering
a
number
of
different
concepts.
So
we
we're
looking
at
how
to
benchmark
databases
primarily
and
volumes
as
as
two
of
the
first
main
things
and
what
we're
looking
to
to
and
what
we're
covering
is
the
basics.
A
So
one
of
the
one
of
the
things
we
we
want
to
measure
is
how
applications
scale
in
these
environments-
and
often
concurrency,
is
one
of
those
is
one
of
those
key
factors
there.
Where
we
we're
talking
about.
You
know
how
many
clients
can
connect
in
parallel,
how
many
parallel
threads
and
how
many
parallel
cues
can
can
can
operate
and,
of
course,
in
all
of
these
systems.
A
Caching
happens
at
multiple
layers.
You
know
at
the
operating
system
layer
at
volume,
manager
layers,
that's
the
file
system,
layers,
page
caches
device
layers.
So
it's
important
it's
important
to
really
understand
what
you're
measuring
and
what
and
how
much
of
an
impact
the
cache
is
having
to
those
environments
to
actually
get
a
real
understanding
of
the
of
the
system.
A
One
of
the
key
takeaways,
though,
is
that
it's
really
important
to
to
focus
on
testing
your
your
own
applications
in
your
own
environments.
It's
really
really
hard
to
compare
published
results
without
a
deep
understanding
of
the
test
conditions,
and
in
fact
so
many
so
so
so
much
of
the
paper
is
actually
dedicated
to
to
the
pitfalls
and
the
common
issues
that
that
people
encounter
when
when
doing
performance
benchmarking,
I
can't
tell
you
how
many
benchmarks
I've
seen
where
somebody
has
published,
for
example.
A
Oh,
I
just
got
two
gigabytes
per
second
on
my
file
system,
and
then
you
ask
them
what
they're
running
on
and
they're
running
on
a
hard
drive
that
can
only
give
you
200
megabytes
per
second,
so
they
were
really
only
testing
the
speed
of
their
cache
rather
than
the
speed
of
their
file
system,
for
example.
So
so
those
those
are
the
things
that
that
we
need
to
to
look
at.
The
next
thing
that
we
covered
was
disaster
recovery,
which
roughly
is
going
to
is
going
to
take
us
through.
B
We
want
to
do
active,
active
deployments,
the
trigger
for
the
the
detection
and
the
trigger
for
the
disaster,
so
the
decision
that
we
are
experiencing
a
disaster
situation
and
we
have
to
start
moving
the
workload
or
reacting
to
it
in
traditional
disaster
recovery.
It's
typically
a
human-based
decision.
B
Something
goes
wrong,
all
the
alerts
fire
people
meet,
and
then
they
decide.
Okay,
we
cannot
recover.
We
need
to
trigger
the
disaster
recovery
procedure
in
cloud
native
disaster
recovery.
We
want
autonomous
decisions
made
by
the
system,
so
autonomous
decision
and
then
also,
as
we
see
in
the
next
line,
automated
response
to
the
disaster,
the
procedure
itself.
Normally,
what
we
see
in
a
large
organization
is
a
mix
of
automated
and
manual
tasks.
The
better
the
organization
is
the
more
automation
there
is,
but
you
know
they
need.
B
B
It's
again
zero,
two
hours,
depending
on
what
what
approach
I
use
to
to
persist
data,
but
in
clarinet
disaster
recovery.
It's
exactly
zero,
so
I
never
lose
data
if
I'm
using
a
strongly
consistent,
consistent
approach
and
it's
theoretically
unbounded,
but
for
you
know,
practical
deployment
close
to
zero.
B
If
I'm
using
events
for
consistent
deployment
and
then
looking
at
more
at
the
organization
side,
the
owner
of
the
disaster
recovery
procedure
in
traditional,
dr,
it's
actually
the
storage
team,
it
formally,
the
application
teams
have
to
build
their
own
business
continuity
document
process,
but
what
they
usually
do
is
they
turn
around
to
the
storage
team
and
they
ask
what
is
your?
You
know,
disaster
recovery
procedure,
what
are
your
sla
and
then
they
adopt
whatever
the
storage
team
is
doing
in
that
organization.
B
So
it's
really.
The
storage
steam
is
driving
the
disaster
recovery
for
the
entire
company,
but
in
cloud
native,
dr,
it's,
the
the
applica,
the
application
teams
are
the
owner
of
the
disaster
recovery
procedure
and
because
the
dr
is
really
a
responsibility
of
the
middleware
in
in
clarinet
vr.
So
it's
going
to
be
a
responsibility
of
the
database
of
the
of
the
queue
system
of
the
cache,
and
so
those
are.
The
middleware
are
now
owned
by
middle
products,
are
now
owned
by
the
application
team
and
then,
in
terms
of
capability.
B
This
is
a
finding
that
we
did
that
we
discovered
by
actually
implementing
these
architectures
in
in
terms
of
capability
for
traditional,
dr.
We
usually
rely
on
storage
capabilities
in
order
to
implement
these
architectures
and
and
the
disaster
recovery
procedures.
So
we
use
things
like
backup,
restore
or
volume,
replication,
synchronous
or
asynchronous,
but
in
cloud
native
vr,
what
we
really
need
are
capabilities
from
networking
more
than
storage.
B
We
covered
the
cap
theorem
a
little
bit
and
then,
of
course,
if
you're
interested,
you
should
go
and
read
about
it
much
much
more
in
depth,
but
all
this
new
generation
of
middleware
that
can
be
deployed.
The
way
I
described
is
really
built
around
the
concepts
of
the
cap
theorem.
So
it's
something
that
we
should
know,
and
then
we
talk
about
the
anatomy
of
this
distributed
state
for
workload
with
shards
replica,
and
I'm
going
to
show
a
little
bit
about
that.
B
So
I'm
gonna
pick
some
of
these
things.
Just
to
give
you
a
little
bit
of
overview
what
you
can
find
in
in
the
in
in
this
white
paper-
and
I
find
this
one
interesting.
This
is
the
anatomy
of
a
stateful
application.
So
if
you
abstract
this
stateful
middleware
enough,
you
will
find
that
they're.
They
all
look
the
same,
whether
it's
a
cache,
whether
it's
a
queue
system,
whether
it's
a
database
or
nosql
database.
They
always
have
a
similar
structure.
They
have
replicas.
B
B
They
have
a
storage
layer,
they
have
a
chord,
which
is
the
thing
that
communicates
with
the
disc
with
the
actual
volumes,
and
then
they
have
a
coordination
layer,
which
is
what
helps
coordinating
with
with
the
rest
of
the
instances,
and
then
they
have
an
api
layer
which
really
defines
the
identity
of
the
of
the
type
of
workload,
so
that
a
queue
system
has
a
different
api
than
a
sql
system.
There
are
no
sql
system
and
then
we
have
in
the
coordination
layer.
We
can
identify
two
kinds
of
coordination.
B
So
you
can
see
we
have
three
here.
This
is
a
strong,
consistent
deployment,
so
we
we
need
three
failure
domain.
At
least
this
is
because
of
the
cap
theorem,
and
so
we
have
the
state
workload
that
is
deployed
in
each
of
this
failure
domain
you
from
there
the
yellow
arrows
indicate
the
fact
that
they
need
to
be
able
to
talk
to
each
other
even
across
failure
domain.
That's
that's
the
east-west
network
capability
that
I
was
mentioning
before.
B
So
if
you're
in
kubernetes,
you
need
a
way
to
have
inter-cluster
communication,
and
there
are,
you
know
several
ways
to
do
it,
and
then
we
have
probably
a
front
end
in
front
of
our
database
or
stateful
workload.
And
then
we
have
a
global
balancer.
That
is
supposed
to
do
some
health
checking
and
decides
to
swing
the
traffic.
B
A
B
A
Okay,
great
so
so
we
talked
about,
we
talked
about
the
disaster
recovery,
so
thank
you
rafael.
We
talked
about
performance
and
the
white
paper.
One
of
the
common
things
that
we
get
asked
in
the
tag
is,
is
how
do
we
work
with
the
toc
in
terms
of
the
the
projects
that
get
approved
into
into
the
tac
and
then
to
join
the
foundation?
A
C
Cncm
projects
have
three
stages:
sandbox
is
the
earliest
stage,
meaning
the
project
is
still
experimental.
Donating
to
cscf
will
help
build
a
stronger
community
around
the
project.
Incubation
is
the
second
stage.
It
means
the
project
has
been
used
successfully
in
production
and
has
a
healthy
number
of
commuters.
C
C
C
C
C
C
D
A
D
More
to
do
with
the
automated
dr
parts
of
the
recovery
process
that
you
showed
us,
but
a
lot
of
the
workloads
that
are
being
run
on
keywords
are
now
databases,
and
in
that
case,
just
a
csi
snapshot
wouldn't
guarantee
a
consistent
backup
of
the
database
itself.
This
specific
database
tooling
needed
to
get
that
sort
of
an
automated
rpo
going.
Do
you
feel
there's
some
synergies
there?
That
could
happen
between
databases
and
basic
storage
projects.
B
A
E
F
E
A
I
think
what
we're
seeing
is,
and
certainly
what
I'm,
what
I'm
seeing
with
with
sort
of
about
the
community
that
we
work
with
and
customers
that
I
work
with,
is
that
the
latency
comes
down
to
two
factors
primarily
which
is
the
physical
media
and
and
the
overall
latency
or
through
to
our
storage
stack,
but
also
the
networking
right.
So
some
of
the
some
of
the
some
of
the
discussions
that
we
were
having
earlier
around
the
different
attributes
and
and
where
you
make
compromises,
are
exactly
the
sort
of
things
that
affect
latency.
A
So,
for
example,
if
you
employ
replicas,
you
tend
to
get
lower
latency.
But
if
you
employ
erasure
coding,
you
get
higher
latency.
If
you
do
replication
across
regions
like
rafaela
was
discussing,
you
could
get
high
latency,
but
you
could
also
have
eventual
consistency
in
some
of
those
in
some
of
the
scenarios
to
to
improve
latency.
But
we've
seen
we've
seen,
I've
certainly
seen
environments,
doing
extremely
low
latency.
A
You
know
much
lower
than
sub
millisecond
for
for
for
database
transactions
and
and
it
is
definitely
possible
to
achieve
extremely
low
latency
in
in
kubernetes
environments
and
we're
seeing
both
in
environments
and
then
on-prem.
You
know
the
availability
of
nvme
disks,
for
example,
that
that
supports
hundreds
of
thousands
of
iops
and
10
40
and
100
gig
connections
on
on
on
cloud
instances,
which
is
which
is
which
can
which
can
help
it
with
all
of
those
situations.
B
Yeah,
no,
it's
not
done
yet.
I
think
there
was
a
edge
component
in
the
question
right
yeah.
We
we
don't
explicitly
talk
about
edge
in
this.
I
don't
think
we
have.
We
have
analyzed
that
scenario
much,
but
I
I
think,
and
I'm
not
an
expert
but
the
little
that
I've
done
for
edge
deployments.
I
think
you
are
willing
in
those
cases
to
sacrifice
you
know.
B
G
Hi,
please
correct
me
if
I
am
wrong,
but
in
your
table
of
comparison
between
traditional
deployment
and
kubernetes
deployment,
it's
there
is
some
kind
of
mismatching
between
h,
a
and
dr,
for
example,
in
traditional
deployment.
If
I
depl
want
to
provide,
I
deploy
several
instances
from
the
same
data
center
on
different
data
centers
and
provide
east-west
communication
between
them.
It
can
be
active,
active
active
passive,
so
it's
very
close
to
your
example
in
kubernetes
on
the
same
hand,
in
kubernetes,
maybe
we
need
to
provide
some
backup
as
well.
A
When
we
were
putting
that
table
together,
we
had
quite
a
lot
of
debate
as
to
what
each
of
the
terms
should
mean,
and
you
know
what
we,
what
we
decided
on
was
it's
impossible
to
count
all
of
the
different
possible
scenarios
and
all
of
the
different
edge
cases.
So
we
decided
to
to
put
together
something
that
kind
of
represented
the
the
most
informed.
I
guess
scenario
and
the
most
common
scenarios
that
that
we
were
coming
around
with
in
in
the
actual
document
itself.