►
Description
Want to view more sessions and keep the conversations going? Join us for KubeCon + CloudNativeCon North America in Seattle, December 11 - 13, 2018 (http://bit.ly/KCCNCNA18) or in Shanghai, November 14-15 (http://bit.ly/kccncchina18).
SIG Storage (K8s) Intro - Saad Ali, Google (Any Skill Level)
Join Kubernetes SIG Storage to learn about the areas of our focus, what we are working on currently, and how you can get involved. Veteran SIG Storage members will be on hand to help answer questions.
About Saad
Saad Ali is a senior software engineer at Google where he works on the open-source Kubernetes project. He joined the project in December 2014, and has led the development of the Kubernetes storage and volume subsystem. He serves as a lead of the Kubernetes Storage SIG, and is co-author and maintainer of the Container Storage Interface initiative. Prior to Google, he worked at Microsoft where he led the development of the IMAP protocol for Outlook.com.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
All
right,
hello,
everyone
can
everyone
hear
me?
Okay,
all
right,
my
name
is
Sally.
I
am
one
of
the
co
leads
of
six
storage.
I
work
at
Google,
mostly
on
the
storage
system
for
kubernetes
I,
see
a
lot
of
familiar
faces
here.
Patrick
back
there
I
see
Sergey
who's
helped
a
lot
with
the
CSI
stuff,
and
we've
got
a
panel
of
folks
up
here.
That
will
pull
up
after
the
presentation
to
answer
some
of
your
questions
by
a
show
of
hands.
A
A
Little
technical
difficulty:
oh
there,
we
go
so
on
the
agenda
today.
We're
going
to
talk
about
who
the
storage
cig
is.
Who
are
the
folks
that
make
up
the
storage
thing?
What
it
is
that
we
do
what
we've
been
working
on
and
how
you
can
get
involved
and
then,
finally,
we'll
close
it
off
with
a
panel
discussion
we'll
bring
up
some
of
the
folks
who've
been
involved
with
the
storage
sake
deeply
for
a
long
time
and
have
them
answer
some
questions
from
you.
A
All
right
so,
first
up,
who
is
the
storage
cig?
The
storage
cig
is
a
group
of
contributors
to
kubernetes
who
specialized
in
the
storage
volume
subsystem
specifically
we're
responsible
for
ensuring
that
file
and
block
storage
is
available
in
a
container,
regardless
of
where
that
container
is
scheduled.
A
We're
responsible
for
provisioning
that
volume
we're
responsible
for
attaching
and
mounting
it,
and
then,
of
course,
unmounting
it
detaching
and
deleting
it
that's
basically
the
core
of
what
the
volume
subsystem
is
responsible
for
beyond
that
we're
also
responsible
for
influencing
the
workload
so
we're
the
pods
are
actually
scheduled
based
on
storage.
So,
if
your
pod
has
a
storage
dependency,
it's
our
responsibility
to
ensure
that
where
it's
scheduled,
the
storage
is
actually
accessible.
And
finally,
we
are
responsible
for
storage
capacity
management.
A
So
some
notable
examples
of
the
features
that
are
owned
by
our
sig
include
persistent
volume.
Persistent
volume
claims
so
if
these
are
API
objects
and
in
kubernetes
that
you've
heard
of
our
team
is
responsible
for
that
and
if
you're
interested
in
helping,
we
are
the
team
that
you
should
be
working
with,
so
storage
classes
and
dynamic
provisioning
are
also
under
our
purview.
A
The
entire
kubernetes
volume
plug-in
system
is
part
of
the
Storage
sig,
and
now
we
have
a
new
initiative
called
the
container
storage
and
interface,
which
is
trying
to
move
the
volume
plugins
out
of
tree.
I'll
talk
a
little
bit
about
that
later
and,
of
course,
if
you've
heard
of
secret
config
map
or
down
word
API
volumes.
These
are
a
few
more
ephemeral
volumes.
A
They
are
also
a
part
of
the
Storage
sig.
We've
got
a
lot
under
under
our
control
and,
if
you're
interested
in
helping
the
first
place
to
start
is
the
team
page
for
cig
storage.
What
does
sig
mean?
That's
a
very
good
question.
A
special
interest
group,
so
kubernetes
is
a
very,
very
large
project
in
order
to
try
to
have
some
resemblance
of
being
able
to
understand.
A
What's
going
on,
we
decided
to
break
kubernetes
into
smaller
sub
areas,
special
interests
so
that
folks
can
focus
and
work
on
sub
components
of
kubernetes
together,
and
so
the
storage
sig
is
responsible
for
the
things
that
I
just
mentioned.
There
are
lots
of
other
sakes,
I
think
we're
approaching
40,
including
cig
node,
which
is
responsible
for
everything
on
the
node
sig
net
working,
which
is
responsible
for
networking
related
things
and
so
on
and
so
forth.
A
A
So
ultimately,
what
we
do
is
write
new
features,
right
tests
for
those
features
and
fix
bugs.
That
is
the
core
of
what
we
do.
How
do
we
get
that
done?
We
meet
virtually
every
two
weeks.
We
have
a
zoom
hangout,
and
then
we
try
to
meet
face
to
face
every
now
and
then
so,
every
one
or
two
quarters
will
find
some
locations.
A
Now,
I
want
to
talk
about
some
of
the
work
that
we've
been
working
on.
So
1.10
was
a
very
big
release
for
us.
This
was
last
quarter.
In
particular,
we
moved
the
container
storage
interface
to
beta.
The
container
storage
interface
is
an
attempt
to
externalize
the
volume
plug-in
system.
The
plug-in
system
that
kubernetes
has
today
is
very
powerful.
A
Basically
the
permissions
that
we
grant
to
those
components,
end
up
being
granted
to
the
volume
plugins,
which
is
not
ideal,
and
it
also
means
that
any
bugs
in
these
volume
plugins
could
crash
these
critical
kubernetes
components
like
the
cube
litter,
cube
controller
manager,
so
we'd
like
to
get
away
from
that
design.
But
it's
not
just
beneficial
for
the
the
kubernetes
developers.
It's
also
beneficial
for
vol,
plug-in
developers,
volume
plug-in
developers,
don't
want
to
be
attached
to
the
kubernetes
release
cycle.
We
have
releases
every
three
months.
A
This
is
great
in
kubernetes,
because
kubernetes
has
this
idea
of
pet
versus
cattle
right.
Your
workloads
can
be
terminated
on
any
one
node
and
moved
somewhere
else,
and
as
long
as
your
storage
is
remote,
that
storage
can
move
around
with
your
workload.
But
what
we
started
to
realize
was
that,
if
you
type
in
my
password
what
we
started
to
realize,
that
was
that
there
are
a
set
of
use
cases,
especially
on
pram,
where
it
makes
sense
to
be
able
to
utilize.
A
The
storage
from
the
local
host
machine
in
a
persistent
manner
and
Michelle
can
probably
expand
on
that.
More
and
she'll
have
a
presentation
on
that
later
this
week,
if
you're
interested
in
it
mount
propagation
is
a
feature
that
allows
volume
mounts
that
happen
inside
of
the
container
to
be
propagated
out
to
the
host
machine
or
to
another
container,
this
feature
moved
to
beta
and
it's
a
dependency
for
CSI
in
order
to
launch
volume
plugins
that
are
containerized
in
kubernetes.
This
is
a
required
feature.
A
Ephemeral,
storage,
request,
limit
API
is
something
that
jingshu
worked
on.
The
idea
here
is
that
volume
plug-in
or
sorry
containers
consume
disk
space
from
the
local
host
machine.
Even
if
they
don't
have
any
volumes
attached
to
them,
they
will
consume
disk
space
in
the
form
of
the
overlay
file
system
inside
the
container
logs
a
standard
in
standard
out
in
a
bunch
of
different
ways.
The
containers
themselves
are
generating
content,
that's
being
pushed
down
to
the
host
machines
disk.
A
Until
this
feature,
we
had
no
way
to
be
able
to
put
limits
on
that.
That
meant,
if
a
container
was
spewing
out
logs,
it
could
potentially
take
up
all
the
disk
space
on
your
host
machine.
Now,
with
the
ephemeral
storage
request
limit
API,
you
have
the
ability
to
set
limits
on
on
that
usage
and
then,
finally,
it's
a
nice
little
feature
that
prevents
the
deletion
of
PvP,
VC
and
pod
objects
out
of
order
in
the
past.
A
A
So
moving
forward
we're
hoping
to
drive
these
features
to
GA
over
the
next
couple
quarters
for
CSI
moving
the
interface
as
well
as
trying
to
figure
out
how
we
can
move
some
of
these
entry
volume
plug-ins
to
CSI.
That's
going
to
be
a
big
effort
over
the
next
couple
quarters
finishing
up
local
storage,
as
well
as
block
volume
support.
Another
really
interesting
feature
is
going
to
be
a
generic
topology
aware
handling.
A
The
idea
here
is
that
volumes
are
not
necessarily
equally
available
throughout
the
cluster
and
an
example
of
this
is
AWS
and
GC.
Persistent
disks,
AWS
CBS,
where
the
volumes
are
actually
zonal
and
you
could
have
clusters
that
are
multi,
zonal,
so
they're
only
available
in
a
sub
section
of
your
cluster.
Today
we
have
essentially
hacks
in
place
for
AWS
and
GCE
to
allow
these
volumes
to
be
scheduled
appropriately.
A
But
what
we're
working
on
is
a
generic
API
that
will
allow
any
volume
plug-in
to
be
able
to
express
that
it
is
not
equally
available
throughout
the
cluster
and
have
either
end
users
or
the
scheduler
be
able
to
influence
both
where
it's
scheduled
and
understand
that
it
is
not
equally
available
and
scheduled
to
work
loads
appropriately.
So
that's
a
big
effort
that
Michelle
is
working
on
as
well,
and
then
a
snapshot
and
restore
API.
So
as
we
grow
beyond
just
making
storage
available,
we're
looking
at
the
set
of
storage
features
that
we
can
abstract
away.
A
The
idea
of
kubernetes
is
that
we
try
to
become
the
abstraction
layer
between
cluster
implementation
and
the
end
user.
So
one
of
the
operations
that
were
starting
to
find
that
is
very
common,
is
snapshotting
and
restoring
volumes,
and
this
is
something
that
Shing
is
working
on
and
we're
going
to
introduce
that
in
CSI,
as
well
as
the
core
kubernetes
api,
beyond
the
features
that
we're
working
on
we're,
also
working
on
a
set
of
videos
to
help
onboard
new
contributors.
Aaron
Boyd
has
launched
this
effort.
A
We
taped
the
first
intro
video
to
this
a
couple
weeks
ago.
We're
going
to
have
some
of
the
members,
give
presentations
record
that
and
make
that
available
to
everyone
to
help
make
it
easier
to
onboard
and
then
finally,
I
mentioned
the
face
to
face
meeting.
We
have
one
scheduled
in
two
weeks
in
California
on
May,
15
and
16.
A
If
you're
interested
in
joining,
please
either
reach
out
to
me
or
someone
in
the
sig,
or
take
a
look
at
the
signaling
list
for
details
on
how
you
can
do
that
see
how
we're
doing
on
time,
okay,
so
next
up
is
how
do
you
get
involved
with
the
Storage
sig
go
to
the
home
page?
That's
the
place
that
you
start.
It
has
information
about
all
of
this
next
up
start
attending
the
bi-weekly
meetings.
Bi-Weekly
here
means
every
two
weeks
not
twice
a
week.
A
A
Next
up,
familiarize
yourself
with
a
code
I
find
it
best
to
just
start
walking
through
the
main
of
any
program
that
I'm
trying
to
familiarize
myself
with
beyond
that.
Take
a
look
at
two
big
components:
one
is
the
volume
manager
inside
cubelet
and
the
second
is
the
controllers
that
exist
in
cube,
cube
controller
manager
for
storage.
The
primary
one
is
the
persistent
volume
controller
there's
a
couple
others,
but
that's
the
one
that
you
should
start
with
once
you
have
some
idea.
What's
going
on,
take
a
look
at
the
bugs.
We
have
a
lot
of
bugs.
A
So
as
of
last
month,
we
had
238
bugs
against
our
sig.
We
need
help
fixing
bugs
all
the
time
and
more
than
that,
we
need
help
writing
tests.
We
have
a
huge,
huge
surface
that
we
expose
and
of
utmost
importance
to
us
is
making
sure
that
we
don't
introduce
regressions.
We
have
to
realize
that
the
features
that
we're
developing
are
being
used
by
enterprise
companies
by
folks
that
are
running
you
know,
healthcare
software
and
all
sorts
of
important
things
on
there
and
data
is
probably
the
most
important
aspect
of
alpha
system.
A
A
The
process
is
pretty
straightforward:
we
have
one
kubernetes
release
every
quarter
at
the
beginning
of
the
quarter
in
our
weekly
meeting
or
in
our
bi-weekly
meeting,
we
will
host
a
quarterly
planning
session
for
the
next
release,
and
then
we
have
a
spreadsheet
where
we
keep
track
of
the
features
that
the
sig
is
working
on.
For
that
Corder,
we
spend
about
two
meetings
trying
to
finalize
on
what
we're
going
to
be
working
on
for
that
quarter
and
who's
going
to
be
working
on
it.
A
So
please
take
a
look
at
that
and,
as
a
reminder,
every
feature
that
we
work
on
must
have
a
feature
repo
issue.
This
is
the
way
that
the
kubernetes
larger
organization
keeps
track
of
the
features
that
are
going
into
kubernetes.
They
use
this
to
keep
track
of
things
like
documentation,
release
notes,
etc,
etc.
So
we
have
to
make
sure
that
the
appropriate
feature
repo
issue
is
opened.
B
C
D
E
A
So
the
question
is
about
CSI
in
the
transition
to
CSI.
How
long
are
the
current
volume
plugins
the
entry
volume
plugins
going
to
be
supported,
so
I
can
take
that
the
plan
is
not
to
deprecate
the
existing
entry
volume
plugins.
The
deprecation
policy
for
the
kubernetes
api
is
very,
very
strict.
The
volume
plugins
that
we
have
entry
expose
api
objects
and
those
objects
are
not
going
anywhere.
What
we
do
plan
to
do
is
change
the
internal
business
logic
and
proxy
those
api's
to
CSI.
A
A
E
D
A
Yeah
good
question,
so
the
question
is
about
multi
cluster,
potentially
migrating
sharing
data
between
clusters.
Anybody
interested
in
taking
that
yeah
no
I
mean
it's
a
very
good
question
and
if
you
look
at
it,
the
cool
thing
kubernetes
is
done
is
that
within
a
cluster
we
have
the
ability
to
dynamically
provisioned
volumes
and
with
snapshots
we're
going
to
have
to
be
have
the
ability
to
be
able
to
pre
populate
those
volumes.
With
some
data
we
haven't
really
addressed
the
multi
cluster
storage
story.
A
Yet
there's
a
couple
of
different
interesting
areas:
there
one
is
migrating
data
offline
and
the
second
is
replication.
Live
replication,
live
replication
has
probably
can't
forget
to
happen.
Data
migration,
on
the
other
hand,
is
something
that
we've
been
looking
at.
Jessie
has
been
spending
a
lot
of
time
looking
at
it
keep
an
eye
out
in
the
next
few
quarters
for
interesting
solutions.
A
C
B
The
question
yeah,
the
question
is
about
how
to
provide
data
gravity
to
reduce
network
traffic
and
people
the
schedule
parts
directly
to
nodes
where
the
data
resides,
so
this
can
be
accomplished
by
the
volume
topology
work
that
is
currently
in
development.
The
main
premise
there
is
to
be
able
to
take
the
same
way
that
you
can
express
pot
apology
today
using
node
affinity
and
take
that
over
to
persistent
volumes,
and
so
as
of
1:10
its
beta
now,
and
what
you
can
do
is
specify
a
node
affinity
on
a
persistent
volume.
B
A
D
A
That
is
more
challenging
Eltham
Utley.
It
goes
down
to
back
to
your
specific
storage
system.
You're
gonna
have
to
see
what
tools
you
have
available
to
actually
make
the
data
available
within
your
cluster.
As
far
as
the
snapshot,
backup
and
restore
capabilities
are
concerned,
there's
speaking
directly
to
the
storage
system,
that's
available
in
that
cluster.
D
B
One
of
the
main
main
challenges
there
is
that
we
don't
there's
no
such
like
standard
for
a
snapshotting
format.
So
the
way
that
snapshots
work
is
that
the
snapshot
source
has
to
be
the
same
volume
type
as
the
volume
you're
trying
to
create
from
the
snapshot
right.
If
you
wanted
to
combine
like
two
different
sources
like
go
from
NFS
to
block
storage
or
something
right,
you
we
would.
Someone
needs
to
develop
some
sort
of
like
translation
layer
to
be
able
to
translate
NFS
from
you
know,
from
NFS
to
file.
A
Sure
yeah,
so
the
question
here
is:
what
is
the
difference
between
the
problems
that
rook
is
trying
to
solve
and
the
problems
that
this
Storage
sig
is
trying
to
solve?
My
thoughts
on
that
is
that
the
difference
is
between
making
storage
available
and
consuming
storage.
The
sake
here,
for
the
most
part,
except
for
local
storage,
is
responsible
for
making
sure
that,
regardless
of
what
type
of
storage
you
want
to
use,
we
can
make
it
available
to
your
workload.
So
we're
about
consuming
the
storage
that
is
already
available
in
your
cluster.
A
The
rook
project
is
more
about.
How
do
you
make
a
distributed
network
attached
storage
available
on
your
cluster,
and
you
know:
you've
got
the
software-defined
storage
systems
that
exists,
F,
cluster,
etc.
Rook
is
basically
trying
to
take
those
and
make
them
easy
to
deploy
through
kubernetes,
so
you
could
deploy
them
outside
of
kubernetes
today,
rook
takes
it
and
makes
it
very
easy
to
deploy
on
kubernetes
and
makes
make
storage
available.
So,
if
you're
running
on
pram,
you
have
you
know,
storage
that
is
available
to
each
one
of
these
host
machines.
C
A
Yeah
so
yeah
sure
the
questions
about
Rex
Rey
and
how
that
relates
to
CSI
and
kind
of
what
it
is
so
Rex
tre
was
a
project
that
was
started
by
the
Delhi
MC
code
team
a
while
ago.
In
fact,
it
predated
CSI
and
the
original
intention
of
Rex
Wray
was
to
be
similar
to
CSI
in
a
certain
way.
It
was
intended
to
be
an
interface
between
arbitrary
storage,
vendors
and
arbitrary
container
orchestration
systems.
A
Csi
came
along
and
took
captured
a
lot
of
that
functionality,
and
so
my
understanding
is
that
Rex
ray
pivoted
and
is
now
basically
building
libraries
around
CSI.
So
if
you
want
to
build
a
storage
plug-in
that
is
CSI
compatible,
what
rex
ray
does
is
give
you
a
framework
that
says
just
write
this
bit
of
code
and
you
will
be
able
to
create
a
CSI,
compatible
volume,
plugin
and
in
fact,
LED
used
to
work
on
that
team.
So
maybe
he
can
add
more
color
to
that
yeah.
C
And
it's
it's
I
mean
support,
for
it
has
kind
of
slowed
down,
but
I
think
it's
going
ramped
up
again
and
to
that
to
that
point
that
the
there's
a
lot
of
work
toward
earlier
this
year,
yeah
earlier
this
year,
had
gone
into
making
it
easy
to
to
build
CSI.
If
you're
a
CSI
plug-in
developer
a
lot
went
into
it
to
make
it
easier
to
to
create
CSI
plugin
with
minimal
code.
C
A
A
A
The
big
difference
with
CSI
is
that
it
was
an
exec
based
model
where
we
had
either
scripts
or
binaries
that
were
executed
for
operations
like
attach
mount
and
the
biggest
problem
with
that
approach
was
that
it's
actually
very
difficult
to
deploy
volume
plugins,
because
you
actually
have
to
have
physical
access
to
the
machines
to
be
able
to
copy
these
files
to
a
very
specific
directory
on
the
machine.
The
other
big
challenge
is
that
for
master
operations,
which
includes
provisioning
volumes
and
attaching
a
lot
of
clusters,
don't
actually
give
you
access
to
the
master.
A
So
if
you
look
at
the
google
kubernetes
engine,
you
don't
have
access
to
the
master
to
be
able
to
actually
copy
the
file,
so
flex
didn't
really
take
off
and
CSI
eventually
came
along
and
addressed
a
lot
of
those
issues.
So
as
far
as
the
sig
is
concerned
concerned,
CSI
is
going
to
be
where
we're
going
to
continue
to
invest
most
of
our
time.
A
That
said,
we
realized
that
flex,
a
lot
of
folks
have
written
production
drivers
against
flex
and
we
plan
to
maintain
the
Flex
API
as
is
going
forward
so
that
if
you
have
flex
volumes,
you
don't
have
to
worry
about
it
becoming
deprecated
or
anything
like
that.
That
said,
we're
going
to
try
not
to
expand
the
API
too
much
further,
so
that
we
don't
have
to
massive
projects
to
maintain
the
idea
would
be
to
maintain
flex
as
is,
and
continue
to
expand,
CSI
sure.
A
Okay,
all
right
yeah,
also
a
good
question.
So
the
question
is
they've
started
migrating
their
plugins
to
CSI.
They
are
coding
against
the
CSI
spec
version
0.2,
which
was
released
in
February.
Is
that
going
to
be
stable
or
not?
And
that's
a
great
question
because
we
launched
0.1
of
the
CSI
spec
in
December
and
when
in
February
we
launched
0.2,
we
made
a
bunch
of
breaking
changes,
so
any
volume
plug-in
that
was
0.1
compatible
broke
with
0.2,
and
so
the
history
behind
this
is
that
with
CSI
once
the
interface
was
where
we
wanted
it.
A
We
specifically
did
not
want
to
make
it
1.0,
because
what
we
realized
is
that
when
you
actually
go
and
implement
these
things,
you're
going
to
find
tons
of
issues,
and
we
wanted
to
give
ourselves
the
flexibility
to
be
able
to
modify
it
as
we
go.
So
we
kept
it
as
a
zero
point,
X
release.
So
that
explains
the
big
change
between
0.1
to
0.2.
A
We're
planning
a
0.3
release
around
the
May
time
frame.
I
am
going
to
try
to
minimize
the
number
of
breaking
changes
between
0.2
to
0.3,
and
that
said,
it's
not
completely
out
of
the
picture.
Once
1.0
is
cut
which
I'm
hoping
is
going
to
be
around
q3
timeframe,
then
we
have
to
to
have
rock-solid
backwards.
Compatibility
like
kubernetes.