►
From YouTube: Cloud Native Storage: Storage TAG Intro, Projects, Landscape &... Alex Chircop & Raffaele Spazzoli
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Cloud Native Storage: Storage TAG Intro, Projects, Landscape & Technology - Alex Chircop, StorageOS & Raffaele Spazzoli, RedHat
This talk will introduce the CNCF Storage TAG and discuss how the TAG operates, how we work with Storage CNCF projects as well as the projects to build guidance and write whitepapers for the ecosystem. During this session we will cover: - Overview of the TAG, how to join and how to help - Overview of storage projects in the CNCF - Projects that are currently being being reviewed We will also share updates of our latest work including: - the CNCF Storage Landscape whitepaper - the Performance and Benchmarking whitepaper - the Cloud Native Disaster Recovery whitepaper
A
Good
morning,
good
afternoon
and
good
evening,
depending
on
where
you're
watching
this
we'd
like
to
introduce
you
to
the
cncf
tag
storage
session,
where
we're
gonna
cover
an
intro
to
the
tag
some
of
the
projects
we're
working
on
and
the
landscape
and
technology
in
our
space,
my
name
is
alex
kirkoff,
I'm
the
ceo
of
storage
os
and
I'm
also
a
co-chair
of
the
cncf
storage
tag,
and
my
co-presenters
are
shane
and
rafael
and
I'll.
Let
them
introduce
themselves.
B
C
A
Brilliant,
so,
let's
dive
right
in
so
very
quickly
today
we're
going
to
cover
an
overview
of
the
tag
and
some
of
the
storage
projects
that
are
being
worked
on
in
the
cncf
and
that
are
being
reviewed
by
the
tag.
We'll
also
cover
some
of
the
documents
and
content
that
we've
been
working
on,
including
the
storage
landscape
documents,
the
performance
and
benchmarking
and
the
disaster
recovery.
A
As
always,
we're
aiming
to
have
a
few
minutes
at
the
end
for
for
q,
a
so
feel
free
to
post
questions
as
we
go
along
so
very
quickly.
The
the
cncf
tags
are
technical
advisory
groups.
They
used
to
be
called
sigs,
but
they
were
renamed
because
of
some
of
the
confusion
with
with
between
the
cncf
six
and
the
cumulative
sigs.
All
of
our
goals
are
open.
A
Fundamentally,
we're
a
mixed
bunch
of
of
people
verging
from
you
know,
technologists
and
and
business
users
and
vendors,
but
all
specializing
in
in
storage.
We
have
a
number
of
coaches
that
that
manage
and
plan
with
the
tag
and
the
number
of
tech
leads,
who
are
technical
authorities
in
their
space,
and
we
work
with
the
toc,
specifically
sad
ali,
who
also
used
to
be
in
the
in
the
sag
and
an
aaron
boyd,
who
also
used
to
be
an
attack
before
they
joined
the
the
toc.
A
But
of
of
course,
what
we're
trying
to
do
here
is
is
lots
of
research
and
that
kind
of
ties
in
then
to
the
second
thing
we
do,
which
is
which
is
projects
today.
The
cncf
has
three
types
of
projects:
there
are
sandbox
incubation
and
graduation
sandbox
projects
is,
is
the
starting
point
for
for
a
lot
of
the
projects
journey
into
the
cncf
and
it
allows
them.
A
Whereas
incubation
projects
are
projects
which
happen
after
a
due
diligence
step
and
those
incubation
projects
are
effectively
used
successfully
in
in
production
and
have
a
healthy
number
of
committers,
and
also
a
number
of
users
that
that
that
can
vouch
for
for
the
project.
Graduation
is
is
for
projects
that
have
gone
mainstream,
and
you
know
things
like
kubernetes.
A
B
B
B
B
B
B
It
handles
storing
and
replicating
data
for
kubernetes
cluster
state
and
uses
the
raft
consensus
algorithm
to
recover
from
hardware
failure
and
network
partitions
thai
kv
is
a
graduated
project.
Tech
kv
is
an
open
source
distributed
transactional
key
value
database
using
rust.
It
provides
transactional
key
value
apis
with
acid
guarantees.
B
B
Dragonfly
is
an
incubating
project.
It's
also
a
project
under
sig
runtime
dragonfly
is
an
open
source.
P2P
based
cloud
native
image
and
file
distribution
system.
It
was
originally
created
to
improve
the
user
experience
with
application.
Cache
log
image
distributions
at
very
large
scales
for
dragonfly,
no
matter
how
many
clients
start
the
downloading.
B
B
B
B
Thanks,
okay
storage
systems
have
several
storage,
attributes,
availability,
scalability
performance,
consistency
and
durability.
Availability
defines
the
ability
to
access
the
data
during
failure.
Conditions
scalability
can
be
measured
by
the
ability
to
scale
the
number
of
clients,
throughput
or
number
of
operations
per.
Second,
the
capacity
and
the
number
of
components
performance
can
be
measured
against
latency
the
number
of
operations
per
seconds
and
the
throughput
consistency
refers
to
the
ability
to
access
newly
created
data
or
updates
after
it
has
been
committed.
B
B
There
are
several
storage
layers
that
can
impact
the
storage
attributes,
for
example,
rather
than
directly
access
resources,
a
hypervisor
can
provide
access
to
resources
which
could
add
excess
overhead
storage.
Topology
describes
the
arrangement
of
storage
and
compute
resources
and
the
data
link
between
them.
This
includes
centralized
distributed
sharded
and
hyper-converged
topologies
storage
systems,
uoa
have
data
production
layer
which
adds
redundancy.
B
B
B
There
are
two
categories
of
data
access
interfaces
here
we
call
them
volumes
and
api
container
orchestration
system
has
interfaces
for
volumes
which
supports
both
block
and
file
systems
in
kubernetes.
There
are
two
volume
modes
file
system
and
block
cloud
system
mode
allows
workloads
to
consumer
file
system
directly,
underneath
it
can
be
either
file
system
or
block
interface.
B
B
Note
that
there
is
a
kubernetes,
sig
storage,
sub
project
called
cozy
container
object,
storage
interface,
which
introduces
kubernetes
apis
to
support
observation
of
object,
storage
operations
for
kubernetes
workloads.
It
also
introduces
cozy
as
a
set
of
grpc
interfaces
so
that
an
object,
storage
vendor
can
write
a
driver
for
provisioning
accessing
object
source.
B
It
is
targeting
alpha
in
kubernetes
1.23
release
on
the
api.
We
also
have
key
value
source
which
use
an
api
to
store,
retrieve
values
from
source
based
on
a
key.
It's
typically
used
to
store
state
configuration
for
distributed
systems
and
on
the
api
we
also
have
databases.
Databases
are
typically
accessed
through
an
api.
B
B
B
This
orange
box
is
called
framework
and
tools.
This
is
an
extension
of
control,
plane
interfaces
for
application
api,
including
key
value,
store
and
databases.
Ceos
currently
don't
have
direct
interfaces
for
it
yet,
but
some
frameworks
and
tools
have
support
for
them.
For
example,
juke
supports
cassandra.
A
So
so
we
look
at
you
know
some
of
the
basics
where
we,
where
we
look
at
you
know,
operations
and
and
throughput.
So
things
like
you
know,
ios
per
second
or
perhaps
transactions
per
second
in
a
database
or
or
perhaps
you
know,
megabytes
per
second
or
gigabytes
per
second,
even
when
it
comes
to
throughput
and
a
lot
of
those
things
are
very
much
influenced
by
a
number
of
different
factors.
A
Because
all
of
those
services
add
overhead,
you
know
topology
whether
you're
accessing
data
locally
or
you're
accessing
data
over
a
storage
network,
for
example,
data
protection
when
it
comes
to
comparing
the
different
ways
that
the
data
is
protected.
You
know
whether
it's
mirroring
or
replicas
or
or
or
some
sort
of
raid
or
even
erasure
coding.
A
For
example,
data
reduction
like
like
compression
and
detube
can
also
have
a
huge
effect
on
compression
on
sorry
on
on
your
on
your
performance
and
on
the
benchmarking
capabilities,
especially
if
the
data
that
your
work,
the
data
set
that
you're
working
with
is,
is
highly
compressible
or
highly
duplicable.
And
of
course
you
know.
Encryption
for
security
is
a
very
common
factor
nowadays
and,
and
obviously
adds
some
overheads,
but
in
many
systems,
encryption
is
is,
is
highly
accelerated
because
it's
already
capable
in
many
frameworks
now.
A
The
two
other
things
then
which
which
come
into
play
is
the
concurrency.
So
things
like
the
number
of
queues
and
the
number
of
clients
that
are
used
to
to
generate
the
workload
you
know
you
have
to
make
sure
that
when,
when
you're
looking
at
the
performance,
if
you
don't
have
artificial
bottlenecks,
that
might
be
you
know,
cpu
bound
or
or
network
band,
and
and
using
multiple
clients
and
multiple
backends
can
help
with
that.
A
And,
of
course,
you
know,
the
the
probably
the
single
biggest
thing
that
that
catches,
so
many
people
out
is,
is
understanding
the
caching
at
multiple
layers
and
and
making
sure
that
if
you
want
to
test
the
storage
performance
of
your
of
your
system,
that
you
need
to
to
at
least
use
a
data
set.
That's
that's
that's
a
number
of
orders
of
magnitude
bigger
than
the
the
cache
size
available.
A
C
C
This
is
a
this
is
not
a
new
approach,
but
it's
an
approach
that
we
think
with
cloud
native
technologies
and
and
solutions
is
going
to
be
easier
to
implement
and
also
more
affordable.
That's
not
to
say
that
you
have
to
use
it.
It's
something
that,
but
we
think
it's
something
worth
considering
when
you
design
your
disaster
recovery
architecture
and
to
define
it,
we
compare
and
contrast
it
with
what
we
call
transition.
C
Traditional
disaster
recovery
approaches.
We
we
debated
a
lot
about
using
this
term
traditional
and
but
just
to
be
clear.
By
that
we
mean
what
you
normally
find
in
in
enterprises
in
in
the
average
enterprise
or
company.
So
not
what
the
web
scalers
are
using,
not
what
startups
that
don't
have
technical
that
are
using,
but
on
average
what
you
can
find,
and
so,
let's,
let's
take
a
look
at
some
of
these
dimensions
that
we
use
to
define
the
cloud
native
disaster
recovery.
C
For
example.
What
is
the
trigger
of
the
disaster
recovery
procedure?
Traditionally,
it's
a
human
decision
with
cloud
native.
We
want
the
decision
to
be
autonomous,
so
the
system
knows
when
it's
experiences
a
disaster
and
can
react
and
then
what
happens
when
the
disaster
recovery
procedure
is
triggered?
C
What
we
see
in
traditional
enterprises
is
mix
of
automation
and
human
action,
okay,
but
the
best
ones,
maybe
do
more
automation
other
do
more
human
actions
with
cloud
native.
We
want
everything
to
be
fully
automated,
so
the
system
sees
that
there
is
a
problem
and
self
readjust
to
the
new
situation
and
then
coming
to
the
main
metrics,
by
which
you
mean
your
disaster:
recovery
slas,
which
are
rto
and
rpo
a
recovery
time
objective
and
recovery
point
objective
in
transitional
disaster
recovery.
C
We
see
rto
so
the
time
that
the
system
is
down
between
minutes
and
hours,
depending
how
long
it
takes
to
make
the
decision
and
how
long
it
takes
for
the
disaster
recovery
procedure
to
recover
in
cloud
native
disaster
recovery.
We
want
it
to
be
near
zero,
it's
going
to
be
in
the
order
of
seconds
and
because
the
health
sects
automatically
need
to
realize.
C
What's
going
on
and
it
needs
they
need
to
redirect
the
traffic
to
the
lc
locations,
but
it's
going
to
be
order
of
seconds
and
rpo,
which
is
the
amount
of
data
that
is
lost
because
of
the
disaster
in
traditional
disaster.
Recovery
can
be
from
zero
to
hours,
depending
on
how
you
implement
your
volume,
replications
or
your
backup
system
in
cloud
native
disaster
recovery.
C
Normally,
what
we
see
in
traditional
enterprises
is
that
the
the
storage
team
owns
the
disaster
recovery
process
and
technically,
it's
really
owned
by
the
applications
team
which
have
to
publish
their
own
business
continuity
document.
But
what
happens
in
reality
is
that
the
application
teams
normally
turn
to
the
storage
team
and
ask
what
is
your
rto?
What
is
your
rpo
and
they
adopt
whatever
is
available
from
the
storage
team
in
cloud
native
disaster
recovery?
Instead,
we
think
that
the
application
team
should
own.
C
It
should
completely
own
that
process
and
finally
doing
these
experiments
and
building
this
architecture.
What
we
notice
is
that,
from
for
traditional
disaster
recovery,
usually
the
technical
capabilities
that
enable
the
disaster
recovery
architectures
are
coming
from
storage,
so
things
like
backups,
restores
and
volume
sinks,
but
in
cloud
native
disaster
recovery
what
we
need.
Instead,
our
capabilities
from
networking
so
is
twice
communication
between
between
regions
or
between
the
our
failure
domains
and
a
global
load
balancer
in
front
of
everything
to
direct
the
traffic
where
to
the
lt
locations.
C
So
what
can
you
find
inside
this
document
very
quickly?
You
there
is
this
definition
that
we
just
covered,
and
then
we
have
some
definition
on
concepts
that
are
needed
in
the
rest
of
the
document.
Like
failure
domain
h,
a
d
r,
we
we're
not
going
in
detail
on
these
things.
It's
just
enough
to
understand
the
rest
of
the
document.
There
are
books
around
the
about
availability
and
disaster
recovery.
C
Where
these
definitions
are
given
in
much
more
details,
then
we
cover
the
cap
theorem
and
everything
that
comes
with
it
right,
like
consistency,
availability,
network
partitioning,
the
cab
theorem
is
the
base
of
why
it's.
It
is
possible
to
realize
architecture
like
we
described
here
with
the
cloud
native
disaster
recovery.
C
So
it's
something
that
we
need
to
explain
in
this
document
and
and
understand
to
continue
to
to
understand
why
those
things
are
possible.
Then
we
define
we
again
look
at
the
stateful
workloads
and
their
anatomy,
similar
to
what
we
do
in
the
white
paper
in
the
in
the
storage
white
paper.
But
here
we
we
focus
in
on
the
characteristics
of
the
anatomy
that
provide
availability
and
scalability,
which
are
the
concerns
for
for
disaster
recovery.
C
Here
and
then
we
look
at
consensus
protocols,
which
is,
are
the
protocols
and
algorithms
that
are
needed
to
keep
all
of
the
instances
of
this
distributed
workload
in
sync,
so
paxos
and
raft
for
consensus
between
instances
that
have
to
do
the
same
action
and
then
through
phase
commit
and
three-phase
commit
for
synchronization
between
instances
that
don't
have
to
do
the
same.
Don't
have
to
do
the
same
action
so
they
are
working
on
different
partitions
or
different
data
structures.
C
C
C
These
are
modern
storage
products
or
stateful
workloads,
and
these
are
the
choices
that
they
make
in
terms
of
consensus
protocol
for
keeping
the
replicas
in
sync,
so
replicas
able
they
have
to
do
the
same
thing,
so
they
can
use
rafter
boxes
or
some
derivative
of
it
of
those
two,
and
then
this
the
sync
protocol,
or
between
the
shards
or
the
partitions,
which
are
doing
different
things.
So
in
that
case
you
use
two-phase,
commit
or
three-phase
commit,
or
some
other
protocol
of
that
kind.
C
C
Okay,
so
here
looking
at
the
picture
on
the
left,
we
have
three
kubernetes
clusters
in
different
fader
domains:
disaster
recovery
failure
domain,
so
that
could
be
data
centers
or
could
be
cloud
regions
right,
depending
on
your
company's
definition
of
risk
and
disaster.
You
you
will
have
maybe
a
distance
that
these
regions
or
data
center
have
to
be
spaced
or
have
to
be
far
apart.
C
So
you,
you
need
to
set
up
this
kind
of
deployment,
and
then
you
will
have
a
global
balancer
in
front
of
it
to
direct
the
traffic,
probably
a
stateless
front
end,
and
then
our
stateful
workload
notice
that
the
workload
needs
east-west
communication
capability
between
these
clusters,
so
that
all
the
instances
can
find
each
other
and
can
execute
the
replica
and
the
sink.
And
then
at
the
bottom.
C
C
I'm
not
going
to
go
further
in
detail
on
this.
Let's,
let's
look
at
the
next
slide
for
for
time's
sake
and
the
next
one
is
a
event,
a
reference
architecture
for
cloud
native
disaster
recovery,
but
in
this
case
for
eventual
consistency.
So
in
this
case
we
don't
need
three
regions.
We
only
need
two
of
them
in
in
the
case,
it's
a
other
than
that
the
architecture
is
similar,
but
in
the
case
we
lose
a
region
here.
C
The
the
remaining
region
or
the
main
data
center
can
keep
working,
but
there
is
the
possibility
that
the
two
states
diverge
and
then,
when,
when
they
come
back,
when
the
disaster
is
recovered,
they
will
have
to
converge
to
an
agreed
state.
And
that's
you
know,
that's
that's
the
characteristic
of
event
or
consistency.
C
We
go
in
in
detail
in
more
detail
on
this
in
the
document
and
we
talk
about
the
fact
that
eventual
consistency
does
not
necessarily
mean
eventual
correctness,
so
the
resulting
reconciliation
may
not
be
correct
from
the
point
of
view
of
the
application,
but
you
know
the
only
thing
that
this
this
eventually
consistent
state
of
workload
will
guarantee.
Is
that
eventually
there
will
be
a
they
will
share,
they
will
have
a
common
view
on
the
state.