►
From YouTube: Velero CSI Integration Proposal meeting
Description
Here is the recording of the meeting for our discussion of the Velero/CSI integration. Thanks to everyone who could attend.
The written proposal is at:
https://github.com/vmware-tanzu/velero/pull/1661
The summary slides are at:
https://docs.google.com/presentation/d/1QG0gu5Y9xiF6rjkqFgsOMOWMjSkj6Il18dNWKWRjipo/edit?usp=sharing.
The plan currently is to get the proof of concept plugins up to date with the v1beta types, get the design document merged, then ensure that the Velero core code base is updated to reflect necessary changes behind the `EnableCSI` feature flag - these changes will be communicated in release notes and blog posts when it's released. We'll also be giving regular updates during our community meetings.
A
A
This
is
meant
to
be
a
summary
of
a
github
PR.
That's
been
open
for
a
few
months
now.
This
is
being
recorded
and
it'll
be
available
for
review.
The
PR
link
is
here:
it's
number
1661,
there's
code
for
a
proof
of
concept
that
Steve
Chris
and
I
worked
on.
Do
you
do
these
plugins
and
I'm
gonna
give
some
background
on
how
those
work
and
I
also
have
a
place
for
collaborative
notes,
but
the
main
agenda
is
going
to
be
giving
some
background
just
so.
A
So,
first
we're
gonna
go
over
the
background
of
kind
of
Valero
and
CSI
box
how
falero
works
and
then
CSI
volume
snapshots
and
we're
going
to
go
into
the
current
plugin
implementation.
Then
the
challenges
around
that
implementation
and
then
our
next
kind
of
next
steps
and
any
discussion
in
Q&A
so
to
jump
in
gonna,
try
to
go
through
this
kind
of
quickly.
I'm
not
gonna,
go
super
deep,
so
we're
gonna
go
into
full
arrow,
plugins
and
the
CSI
data
types,
and
to
do
that.
I'm
gonna
be
kind
of
clear
on
what
we're
talking
about
snapshots.
A
So
when
we're
talking
about
snapshots
here,
we're
talking
about
things
that
are
like
standalone
that
are
replicated
within
a
system
like
a
cloud
provider,
so
there
they're
not
directly
tied
to
a
volume
and
they're,
not
incremental
so
they're
more
like
an
e
BS
or
GCP
snapshot.
They're,
not
like
a
snapshot.
You
might
we're
not
thinking
about
them
like
a
snapshot,
you
might
take
in
a
more
traditional
system
and
they're,
not
necessarily
compatible
across
story
systems.
A
Okay.
So
if
you're
familiar
with
velaro,
we
have
these
plugins
that
have
that.
Do
the
volume
snapshot
for
you.
They
call
the
volume
providers
API
directly
and
Valera
was
originally
called
EBS
snapshot.
Er
existed
before
CSI
was
a
thing
and
Valero
was
meant
the
whole
for
the
whole
time
to
snapshot
volumes.
A
It's
specific
to
Valero's
implementation
and
the
output
of
these
plugins
is
very
similar
to
what
the
CSI
volume
snapshot
API
does.
It
creates
a
snapshot
on
the
backup
side
and
it
creates
a
persistent
volume
at
the
end
of
verse,
store
and
use.
The
volume
snapshot,
location
object
to
define
information
like
regions
and
zones.
It
doesn't
necessarily
rely
on
cloud
metadata
or
anything
like
that.
A
Metadata
services
valera
also
has
item
action
plugins,
and
these
are
plugins
that
falero
uses
on
kubernetes
resources
as
they're
getting
backed
up
or
restored
so
like
as
a
backup
is
processing,
velaro
will
say:
hey.
Does
this
resource
get
matched
these
plugins
and
if
so,
then
it
will
run
the
execute
method.
So
this
is
used
for
like
taking
a
pod
and
backing
up
any
relevant
persistent
volume
claims
and
then,
from
those
persistent
volume
claims
it
would
back
up
relevant
for
assistant
volumes.
A
Moving
on
the
CSI
going
to
find
what
I'm
going
to
call
CSI
common
controllers,
so
these
are
controllers
that
run
in
the
cluster
that
kind
of
reconcile
state
for
the
CR
DS
that
I'm
going
to
talk
about,
and
these
are
maintained
by
the
community,
they're
not
specific
to
a
vendor.
The
vendor
stuff
are
like
the
Amazon
or
Brooke
or
SEF
Rob's
F,
open
EBS,
those
kinds
of
implementations,
so
these
would
be
the
external
snapshot
er
its
current
provision
or
external
attached
or
those
kinds
of
things.
A
There's
the
volume
snapshot
contents,
which
is
cluster
scopes.
It's
not
in
a
namespace,
represents
the
on
discs,
contents
of
a
snapshot
and
it
points
to
that
on
disk
snapshot,
and
it
has
a
two-way
link
to
the
volume
snapshot
so
think
of
it.
Like
a
PVC
points
to
a
pv,
the
volume
snapshot.
Content
is
the
pv.
In
that
kind
of
comparison,
then
there's
volume
snapshot
class,
which
is
kind
of.
If
again,
if
we're
using
that
comparison
to
the
existing
storage
types,
this
is
like
a
storage
class.
A
It
defines
parameters
around
how
those
snapshots
are
treated
and
those
will
be
vendor
specific.
The
big
ones
that
aren't
vendor
specific
are
the
retain
retention
policies
which
would
be
retaining
this
now
shots
beyond
the
life
of
the
Sierra
DS
or
deleting
them
when
the
CRTs
are
deleted
and
then,
finally,
with
the
addition
of
the
CR
DS,
persistent
volume
claims
have
a
new
field
data
source,
which
means
that
you
can
create
a
new
persistent
volume
from
a
volume
snapshot
using
this
data
source
and
just
to
illustrate
how
all
the
new
CR
DS
relate.
A
I've
got
a
volume
snapshot
that
has
a
bi-directional
relationship
with
volume,
static
content
and
they
both
point
to
the
volume
snapshot
class.
And
then,
if
we
look
at
how
Valero's
proof-of-concept
works,
we've
implemented
it
as
a
set
of
four
plugins
on
the
backup
side.
Velaro
actually
just
has
one
nap
raha
that
processes
persistent
volume
claims,
and
then
we
have
a
set
of
three
plugins
that
act
on
the
different
objects
when
they
come
into
the
cluster
and
then
for
convenience.
We
put
them
all
and
register
them
all
in
one
binary.
A
So
if
we
look
at
our
cluster,
the
PVC
is
pointing
to
a
PV
which
points
to
a
volume
on
disk
and
the
Valero.
Csi
plug-in
applies
to
PVCs
and
it's
going
to
create
a
volume
snapshot
CRD
based
on
the
PVC
and
from
there
the
CSI
controllers
are
actually
going
to
create
the
volume
snapshot
content.
So
all
the
Valero
does
is
creates
that
volume
snapshot,
content,
I'm,
sorry,
volume
snapshot
so
on
the
back
up
side
Valero
actually
doesn't
do
a
whole
lot.
A
It
creates
the
CR,
D
and
kind
of
moves
on
and
then
inside
the
back
up.
You've
got
all
these
objects
pointing
to
each
other
and
they
point
out
to
the
snapshot
on
disk
and
then
for
the
restore
the
volume
snapshot
restore
item
action
plug-in
all
it
really
does.
Is
it
breaks
the
binding
to
the
original
PVC
so
that
the
CSI
controllers
will
know
that
it's
a
quote,
unquote
import
case
so
you'll
see
up
here
it
was
the
PVC
I'm.
Sorry,
the
volume
snapshot
should
have
been
pointing
up
to
the
PVC.
A
The
volume
snapshot,
content,
object,
restore
item
action
plug-in
will
update
the
bi-directional
relationship
between
these
volume.
Snapshot
and
volume.
Snapshot
content
objects
so
that
the
newly
restored
objects
in
the
cluster
are
the
actual
objects,
because
the
UUID
changes
when
we
do
a
restore
and
then
the
persistent
volume
claim
restored
item
action,
updates
the
data
source
field
so
that
a
new
PVC
will
be
pointing
to
the
volume
snapshot
and
can
make
a
new
volume
from
the
snapshot.
A
So
if
people
have
been
attending
some
of
our
other
meetings.
One
of
the
questions
we've
had
has
been
for
us,
the
Valero
maintainer
z'
has
been
is
the
the
is.
The
item,
action,
plug-in
format,
the
right
format
for
us
to
use
the
prototype
was
a
set
of
item
action
plugins
because
it
seemed
like
the
path
of
least
resistance.
A
The
volume
snapshot
or
approach
was
tried
by
a
few
people,
but
there's
a
bunch
of
pretty
much
irrelevant
parameters
on
that
and
doing
restores
actually
competes
without
the
CSI
comic
controllers
work.
So
we're
not
sure
that
that's
a
great
path
to
take,
and
also
a
long
term
I'm,
not
sure
that
that's
a
useful
way
to
go
because
assuming
CSI
volume
snapshotting
becomes
the
way
to
go
and
as
it
matures
it
grows
more
methods
for
more
complex
scenarios.
The
velaro
volumes
snapshot
or
api
is
probably
not
gonna.
Keep
up
so
I'm.
A
A
There's
this
Valero
plugin
for
AWS,
and
this
Valero
plugin
for
AWS
will
call
the
AWS
API
for
doing
snapshots
and
that
overlaps
with
AWS
is
own
driver.
At
this
point,
ideally
long
term.
We
would
like
to
retire
this
this
API,
but
for
the
short
term,
it
might
make
sense
for
Valero
to
say
the
CSI
drivers
fit
into
a
volume,
snapshot
or
plugin.
A
But
I
don't
know
so
we
want
to
use
the
CSI
drivers
I.
Think
that
that's
the
kind
of
the
actual
goal
for
I
think
the
question
is:
do
we
make
it
fit
into
a
Valero
snapshot
or
plug-in?
That
then
does
basically
the
same
work
as
what
I
outlined
up
here,
of
making
the
volume
snapshot
that
gets
submitted
to
the
API
server.
B
A
Right
right
so
I
guess
the
the
goal
so
this
list
we
know
this-
this
actual
list
is
longer
where
actually
we
would
like
this
list
to
kind
of
condense
down
to
just
this
one
so
that
Valero
plug-in
for
CSI
is
just
kind
of
any
any
any
volume
that
is
CSI
supported.
This
one
would
work
for.
I
guess
is
the
end
goal.
A
So
this
questions
is
probably
like
Valero
specific,
it's
it's
an
internal
implementation
detail,
and
maybe
we
even
something
we
on
the
Valerio
teaming
to
asked
even
was
do
maybe
we
even
treated
specially
inputted
in
core
I.
Don't
know
that
we'll
do
that
immediately,
but
it's
it's
certainly
an
option.
C
C
So
if
we
assume
we're
not
doing
it
through
a
volume
snapshot
or
plugin,
then
we
can
kind
of
either
take
the
path
of
implementing
it
as
item
action
plugins
or
we
can
art
cut
a
bunch
of
logic
in
bolero
and
I
do
think
there
that
using
the
item,
action
plugins
makes
sense.
You
know
part
of
the
intent
of
having
those
item.
C
Actions
was
to
be
able
to
kind
of
encapsulate
the
custom
logic
for
different
resources
into
their
own
little
snippets
of
code,
essentially
so
that
it's
not
all
just
munched
together
in
the
main,
you
know,
backup
and
restore
code
within
bolero
brain,
so
I
think
you
know
packaging.
Each
of
those
bits
of
logic
for
the
different
resource
types
up
into
their
own
item
actions
makes
sense.
C
A
We
have
code
implemented
what,
as
these
item
actions
so
that
they're
logically
separated
not
baked
into
the
core
path,
and
once
once
CSI
volume
snapshots
themselves
get
promoted
to
GA
and
the
Valero
support
gets
promoted,
then
we
can
maybe
start
thinking
about.
Does
it
make
sense
to
promote
this
in
entry?
A
A
C
A
C
Yeah
I
just
just
wanted
to
go
back
to
the
packaging
discussion
for
a
second,
so
in
I.
Think
in
my
mind,
I
think
we
should
plan
to
move
these
CSI
item
action
plugins
into
Balearic
or
at
least
for
GA
yep,
because
I
I
don't
see
any
reason
why
we
can't
keep
them
in
their
own
repository
for
now,
given
that
this
is
gonna,
be
behind
a
feature
flag
and
essentially
a
beta
feature,
but
I
do
think
eventually
will
want
to
do
that,
just
because
they
will
be
a
you
know:
they'll
be
core
functionality.
C
A
A
So
the
next
one,
so
this
this
one
came
up.
This
is
a
question
that
it
kind
of
goes
to
Valero's
operating
model,
so
so
Valero
for
its
own
quote-unquote
native
snapshots.
It
keeps
a
list
of
the
snapshot.
Web
CS
I
call
snapshot
handles
in
a
JSON
file
in
object
storage.
So
it
does
track
volume
snapshots,
but
it
doesn't
put
them
in
kubernetes.
It
puts
them
in
a
list
in
object
store,
so
it's
not
putting
them
in
the
backup
tarball.
A
A
So
this
is
this
is
gonna,
be
something
we're
going
to
have
to
excuse
me
consider
this.
May
be
fixed
on
an
import
case,
so
when
you
restore
from
that
Tuesday
backup,
it's
gonna
create
a
volume,
snapshot,
content
and
a
volume
snapshot
object,
but
they'll
be
pointing
to
a
volume,
snapshot.
I'm.
Sorry,
a
snapshot
handle
on
disk
that
doesn't
exist
and
I,
don't
know
exactly
what
that.
What
will
happen
there
and
I
Jing
I,
don't
know
if
you
can
clarify
what
will
occur
there.
Will
those
just
sit
there
not
able
to
be
used.
B
A
B
Valid
okay
yeah,
so
for
the
new
for
the
beta
code,
this
actually
depends
on
whether
the
CSI
driver
implements
these
snapshots.
If
it
didn't
lists
didn't
implement
that
we
just
assume
it
is
valid,
so
it's
going
to
treat
it
as
a
valid
snapshot,
but
if
they
implemented
that
were
going
to
check,
if
it
really
exists
or
not.
A
C
B
B
A
B
B
C
I'm
sorry
I
just
dumped
a
whole
bunch
of
words
here
so
I
couldn't
try
and
summarize
the
questions
a
little
bit,
the
first
one
so
I'm
just
thinking
through
how
CSI
snapshots
will
interact
with
Valero's,
pre
and
post
hooks,
and
so
as
a
quick
summary
for
how
this
works
today,
valera
lets
you
configure
pre
or
post
hooks,
which
are
commands
that
are
run
within
a
pod
before
or
after
backing
up
that
pod.
So,
typically,
if
you
have
a
pod,
that's
using
a
PVC.
What
you
can
do
is
on
that
pod.
C
You
configure
a
pre
hook
to
acquiesce
your
application,
then,
as
part
of
the
pod
being
backed
up
at
the
PVC
that
it's
using
is
also
backed
up.
So
you
can
essentially
get
a
snapshot
while
the
pod
is
being
backed
up
and
then
after
the
pod
is
finished,
being
backed
up,
the
post
hook
will
be
run,
so
you
can
there
UNCHR.
C
A
A
A
Like
I
think,
I
think
the
order
of
operations
here
we'll
be
moving
our
code
to
the
beta
and
then
testing
that
with
the
hooks
and
doing
experiments
with
with
the
various
states,
because
each
of
the
plugins,
like
with
the
slides
the
most
complicated
plug-in,
is
the
backup
one
and
I
don't
think
off
the
top.
My
head
I
didn't
look
at
the
at
that
one,
but
I
don't
think
it
reaches
into
the
volume
snapshot
content
right
now.
If
it
does
it
just
like
sets
an
annotation
for
the
backup
it's
associated
with
yeah.
C
B
A
B
A
B
Case
yeah,
so
Chris
snapshot
will
be
caught
multiple
times
the
first
time
we
should
get
a
handle
and
then
we're
going
to
call
out
again
to
check
the
status
until
it
is
ready.
But
as
soon
as
you
as
soon
as
it
returns,
you
should
get
a
handle,
and
at
that
time
you
should
be
able
to
resume
you
application.
Okay,.
A
B
B
B
B
B
A
Okay,
yeah:
okay,
that's
what
I
thought
was
the
case,
but
I
mean
if
we,
if
we
check
the
volume
snapshot,
content
for
the
snapshot,
handle
that's
like
a
way
to
be
super
extra
sure,
but
but
yeah
it
sounds
like
we
don't
want
to
wait
for
ready
to
use
true
unless
we're
doing
like
a
restore
immediately.
Yes,.
B
A
Whatever
reason,
but
on
the
restore
side,
when
we're
doing
the
static
provisioning
like
if
we're
reassembling
all
the
pieces,
the
ready
to
use
the
the
controllers
should
update
ready
to
use
to
true
because
we're
gonna
be
yeah,
resetting
the
status
and
everything
when
we
put
all
the
pieces
back
together.
So
actually
now
that
I
think
about
it.
That
status
probably
doesn't
matter
as
much
I.
A
But
it
should
be,
there
was
that
status
should
be
updated
when
we
so
at
restore
time.
Velaro
takes
all
the
backed
up
resources,
so
let's
say
we're.
Rebuilding
a
cluster
in
a
new
kubernetes
cluster
in
the
same
availability
zone
will
put
the
kubernetes
manifests
in
so
in
this
case,
valerio's
calling
it
a
restore,
but
CSI
would
call
it
static.
Provisioning
or
a
static,
binding,
so
flare
would
be
basically
deleting
the
status
fields,
oh,
but
the,
but
the
CSI
controller.
B
B
B
A
A
C
Just
just
one
other
thing
on
that,
which
is
that
you
know
so
velaro
already
for
its
existing
snapshots
has
the
limitation
where
it
doesn't
have.
It
doesn't
model
waiting
for
the
snapshot
to
be
ready
right.
So
if
you
go
to
do
a
restore
of
a
backup
and
the
snapshot
hasn't
actually
completed
replicating,
you
won't
be
able
to
restore
it.
This
is
the
issue
scott
ran
into,
so
I
think
a
long
term
like
we
want
to
model
that
probably
both
for
our
existing
plugins,
as
well
as
for
CSI
and
with
CSI.
C
C
A
To
your
next
question,
so
you're
asking
about
volume
snapshot
locations
which
in
Valero
this
the
this
defines
things
like
available
as
the
availability
zone
region.
Basically,
any
settings
we
would
hand
off
to
the
storage
provider
Valera
needed
that
because
Valero
isn't
deeply
integrated
with
the
storage
vendor
implementation,
and
I
am
I
representing
that
question
correctly.
C
Yeah
I
mean
there
were
you
know
there
are
various
settings
that
we
stored
in
here.
Basically,
any
opaque
parameters
for
our
plugins,
but
I
think
they're
not
relevant
when
we're
using
CSI
right,
like
with
basically
you'll,
have
a
volume
snapshot
class
which
essentially
takes
the
place
of
this,
and
so
as
a
Valero
user.
You
just
you,
set
up
your
volume
snapshot
class
in
your
cluster.
You
make
sure
that
your
your
providers,
CSI
plug-in,
is
installed,
and
then
you
don't
have
to
define
a
volume
snapshot
location
in
Valero
to
be
able
to
get
snapshots.
C
B
A
So
I
was
gonna
say
I
was
I
was
looking
at
the
AWS
implementation
today
to
see.
If
I
could
answer
this
and
the
the
settings
don't
map
directly,
so
the
volume
like
the
there,
it's
not
a
one-to-one
mapping
of
yeah.
All
the
settings
that
were
in
the
Valero
volume
snapshot
locations
go
directly
into
the
volume
snapshot
class
but
I
think
in
depending
on
your
vendor.
A
You
may
have
less
configuration
to
do
so
if
you're
on
AWS,
for
example,
and
you're
using
the
CSI
driver,
because
the
CSI
driver
is
now
more
like
it's
it's
your
entire
storage
solution.
It
knows
more
about
your
topology
and
you
don't
have
to
do
anything
specifically
for
AWS
they're,
using
their
metadata
servers
and
it's
getting
the
region
and
zone
all
that
information
dynamically,
so
I
think
the
answer
is
kinda.
So,
yes,
the
volume
snapshot
locations
are
not
needed
when
using
CSI,
but
no
they're
not
directly
replaced
by
volume
snapshot
class.
C
I
mean
if
you
take
a
sure
as
an
example,
like
you
know,
in
Valero
today
you
can
define
a
volume
snapshot,
location
for
a
sure
and
one
of
the
configuration
options.
There
is
the
resource
group
to
put
the
snapshots
into
which
doesn't
have
to
be
the
same
resource
group
as
where
your
cluster
is
or
where
your
volumes
are,
and
so
I
would
think
that
with
CSI
you
know
the
azure
CSI
plug-in
could
could
expose
the
option
to
define
a
resource
group
as
a
as
a
config
parameter
on
the
volume
snapshot.
C
Class
I,
don't
know
if
it
does
or
not,
but
right
theoretically,
it
could,
but
that's
like
for
CSI
snapshots
through
Valero.
That's
where
we
should
be
directing
folks
for
any
configuration
for
their
provider
exactly
because
we
wouldn't
have
any
way
to
pass
it
in
to
CSI
anyway,
even
if
you
did
define
a
volume
snapshot,
location
right.
A
It's
it's
gonna
end
up
being
either
a
volume
snapshot,
class
or
the
storage
class
or
whatever
wherever
they
end
up.
The
the
storage
provider
ends
up
defining
that
parameter
like
the
topologies
looking
at
the
Amazon,
one
I
think
it's
gonna
end
up
where
they
put
volumes
like
I,
don't
know
exactly
how
the
implementations
will
work,
but
I
think
they're
gonna
end
up
like
going
to
the
node,
where
the
volume
is
in
some
cases.
A
But
yeah
that
and
and
you
actually
bring
up
a
good
point
there,
like
in
Azure,
we
can
do
stuff
like
send
to
a
different
resource
group.
I,
don't
know
if
they'll
allow
that
in
the
in
their
implementation
and
if
they
don't
that
might
be
a
place
we'll
have
to
help
them
with.
Because
I
know.
People
like
that
right,
yeah,
just
having
a
quick
look
at
the
edge
of
disks
ESI
driver,
and
it
may
only
have
one
entry
for
resource
group
in
config
and
it
seemed
to
be
a
way
to
specify
one
specifically
for
snapshot.
C
That's
a
shame:
yeah
I
mean
we
could
certainly
raise.
You
know
Friedan,
like
each
other
providers
that
we
have.
We
could
raise
the
the
configuration
you
know
basically
tell
the
maintainer
z'
of
each
of
these
providers
like
here
the
configuration
options
that
we've
had
users
ask
for
and
that
we've
implemented,
and
so
you
know
these
potentially
could
be
good
things
to
add
to
the
CSI
drivers.
Yep.
C
Any
way,
though,
like
the
main
reason
I
raise,
this
is
because
it
means
a
different
UX,
depending
on
whether
you're
using
CSI
snapshots
or
not,
and
it
means
different
documentation.
So
we're
just
gonna
have
to
keep
that
in
mind
as
we
go
through.
This
is
like
we're
gonna
kind
of
need
to
fork
the
documentation,
depending
on
whether
you're,
using
CSI
or
not
for
snapshots.
Yes,.
A
C
It's
sort
of
an
extension
of
the
the
thing
we
were
talking
about
on
the
previous
slide
like
there's.
There
are
existing
ones
that
were
created
by
previous
Valero
backups,
but
there
are
also
existing
ones
that
were
created
completely
outside
of
Valero
right
I'm,
just
trying
to
think
through.
Like
does
it
make
sense
to
pack
these
things
up,
because
you
certainly
have
the
same
scenario
where,
when
you
go
to
restore
them,
you
know.
Potentially
the
actual
snapshot
itself
has
has
actually
been
deleted.
A
But
I
don't
know
what
that
looks
like
because
I
haven't
given
it
a
ton
of
thought,
I
think
it's
something
on
Don
and
I
mentioned
in
our
talkin
cube
con,
because
I
think
it
makes
sense
to
make
full
ROP
a
like
a
good
citizen
in
the
community
to
be
like,
hey,
you
played
around
with
CSI
snapshots
here.
Let
Valero
grab
them,
but
I
think
it
that
may
end
up
being
like
some
sort
of
extra
tool
in
Valero
to
be
like
Valero
CSI,
adopt
and
goes
and
scans
for
volume,
snapshots
and
but
I
I.
A
A
C
It's
not
a
it's,
not
a
special
volume
snapshot,
it's
not
one
that
was
triggered
by
Valero,
true
for
the
purposes
of
backing
up
of
PVC.
It's
just
another
kubernetes
resource
that
happens
to
exist
in
the
namespace,
so
I
think
it's!
You
know
it's
probably
fine
to
back
it
up,
there's
no
harm
in
in
storing
the
Y
ammo
for
it
in
our
tarball
mm-hmm.
B
C
A
True
it'll
be
treated
the
same
way
as
if
it
was
a
boy
I'll
back
up
one
and
and
it
got
deleted
yeah
we
didn't
do
bookkeeping
on
it,
yeah
because
I
was
gonna
say
even
if,
even
if
the,
even
if
the
snapshot
was
retained,
it
should
be
treated
the
same
unless
we
do
more.
More
granular
applies
to
in
the
restore
plugins
to
say
like
it
only
only
applies
to
stuff
that
has
of
Lauro
backup
label
orientation.
A
A
Got
it
yeah
yeah,
it's
it's!
It's
a
balance
like
on
the
one,
on
the
one
hand
like
putting
on
my
Valero
developer
hat,
it's
not
surprising,
it's
just
it's
just
a
thing
in
the
API
based
on
user
questions,
I've
seen
and
people
just
thinking
about
they're
like
well,
it's
a
CSI
snapshot.
Of
course
falero
has
CSI
support.
I
could
see
somebody
being
like
well,
why
didn't?
Okay,
everything
up.
C
A
Know
including
these
from
Buller,
the
including
these
things
in
the
velaro
backup,
does
get
weird
I
agree.
So
that's
that's,
maybe
something
we
can
pull
apart
as
we
go,
but
I
think
for
now,
I
I,
don't
think
we
have
to
assault
everything
but
they're.
Certainly
big.
There
was
big
questions.
I
would
like
to
at
least
if
we
put
them
in
front
of
ourselves
and
think
about
and
I
actually
have
a
hard
stop
on
the
hour,
so
so
for
the
next
one.
C
A
So
for
the
next
steps,
I
think
the
the
next
thing
to
happen
will
be
for
us
to
update
our
CSI
plugins
to
use
the
beta
CR,
DS
and
code,
so
that
were
kind
of
current
with
the
upstream
I
also
would
like
to
help
the
CSI
team.
Get
their
controllers
and
code
moved
forward
and
get
get.
You
know,
help
them
move
along
to
get
to
GI
status
so
that
more
of
the
kubernetes
community
can
get
onto
CSI
I.
Think
Steve,
you
added
we
can.
We
need
to
get
to
Valero
design
doc
merged.
A
A
C
A
Cool
all
right
well,
thank
you.
Everyone
for
coming
once
I
get
their
link
to
the
recording
I
will
send
that
out
to
the
Google
group
and
I
hope
we
can
do
more
of
these
open
design,
discussions,
rustic
and
any
other
larger
topics.
Sorry
for
the
short
notice
on
this
one.
Hopefully
we
have
a
little
bit
more
lead
time
on
ones
in
the
future,
but
yeah
and
have
a
great
week
go
ahead.
Steve
I
was.
C
Just
gonna
say:
I'd
love
to
you
know,
since
this
is
a
pretty
big
new
feature,
I'd
love
to
spend
some
time
covering
just
general
updates
on
it
kind
of
on
each
weekly
community
meeting
yeah.
There
sure
you
know
everyone's
kind
of
kept
in
the
loop
on
what's
happening
as
we
go,
because
this
is
a
you
know.
It's
a
big,
exciting
new
feature.
Yeah.
A
Absolutely
so
plan
to
keep
that
rolling
and
keep
people
up
to
date
and
I
know.
This
is
also
something
that
it's
gonna
evolve
outside
of
Valero.
So
it's
not
just
something
we're
doing
like
at
Jing.
Also
thank
you
to
Jing
who's
on
the
call.
Cuz
she's
pushed
a
lot
of
this
upstream.
There
really
wouldn't
be
volume
snapshots
without
her
efforts.
So
thank
you
and
thanks
everybody
for
coming.