►
Description
March 10, 2020
Agenda
Status Updates
ashish
CSI Integration
Use Velero to backup and restore using the v1beta1 Snapshot APIs
Backup and restores- in-cluster restores and cluster migration. Live demo of this later.
Created some issues:
Upstream external snapshotter:
VolumeSnapshotRef.UID should be cleared on volumesnapshot deletion
Relation between VolumeSnapshotContent and VolumeSnapshot should be 1:many
Lifecycle of VolumeSnapshot Kubernetes objects should be decoupled from that in the storage system
Velero:
Ensure restored workloads can be backed up
Ensure backup and restore of volumesnapshots created outside of velero work
Repeated backup of the same CSI workload should not leave orphan resources
I’ll add more details into the issues in the Velero repo.
See our progress in the velero-plugin-for-csi repo and feel free to jump in and contribute.
carlisia
Was out last week
On support rotation this week
Catching up on PR reviews
Pushed an update to the CLI PR containing yaml and kustomize files that properly create/update a velero deployment (https://github.com/vmware-tanzu/velero/pull/2202/commits/bda5099153c588477c80494f1782147108031b51)
jonas
Plugins page is live!
https://velero.io/plugins/
nolan
PR #2232, fixing CRD int/float conversions on backup.
Thank you to guyguy333 and NissesSenap for the reports, data, and testing!
Tracked down nodePort validation error (issue #2308), will require some more thinking
Started working on uploading CSI artifacts to object storage
Need to review Unprioritized backlog to help build roadmap, missed that last week
steve
finalized refactoring of restore/discovery code (https://github.com/vmware-tanzu/velero/pull/2248)
working on refactoring to move logic out of controllers
merged https://github.com/vmware-tanzu/velero/pull/2299
WIP https://github.com/vmware-tanzu/velero/pull/2317
worked on implementing blob chunking in azure plugin: https://github.com/vmware-tanzu/velero-plugin-for-microsoft-azure/pull/30
lots of reviews
Discussion Topics
v1.3.1 patch release
https://github.com/vmware-tanzu/velero/pull/2322
nrb and skriss will try to get v1.3.1 out today
Important for users of v1.3.1 to upgrade, but it’s largely based on what CRDs users have; if there are numeric fields, they will see this bug
Update from Dylan Murray - actively working on https://github.com/vmware-tanzu/velero/issues/2332 before tackling client & install issues
Contributor Shoutouts
guyguy333 and NissesSenap for reports, data, and testing for issue 2319/PR 2322
betta1 for the detailed reproduction case on issue 2308
A
Hi
everyone
and
welcome
to
another
episode
of
the
Valero
community
meeting,
slash
open
discussion.
My
name
is
Janice.
Roslyn
I'll
be
your
moderator
here
today
and
with
us.
Of
course
we
have
the
lever
team
and
the
Valero
community
members,
so
we
get
a
bunch
of
stuff
that
we
want
to
talk
about
today
on
the
agenda
here.
A
B
I've
been
working
on
like
the
CSI
integration
with
leather
or
basically
using
velaro
to
perform
backup
and
restores
of
applications
that
use
CSI
volumes.
The
backup
and
restore
will
use
like
the
even
beta
one
start
api's
to
do
the
backup
and
store
stuff
right
now,
the
current
state.
You
know
this
is
all
using
the
valero
plugin
for
plugin
for
CSI
right
now.
I
am
able
to
put
for
use
the
plug-in
to
perform
like
backup
and
the
stores
in
cluster
and
also
like
cross
cluster
migration
scenarios
been
running
tests
using
the
host
particle
again.
B
C
D
D
B
D
D
B
D
A
C
So
last
week
we
got
a
report
on
version
1.3
point
0
that
one
of
the
bugs
that
Scott's
ego
had
reported
in
the
beta,
I
believe
or
videos
in
1.2
that
the
int
float
conversion
on
CRTs
actually
popped
up
in
another
location,
so
I
fixed
that
last
week.
Thank
you
to
guy
guy
3
3
3,
and
this
is
enough
for
the
reports
and
the
data
turned
out.
C
This
issue
to
3:08
basically
looked
like
the
expectation,
was
that
the
error
should
come
back
as
already
exists
from
kubernetes
server
and
therefore
valerio
should
report
it
as
the
service
is
a
duplicate.
So
therefore
it
should
continue
with
the
store
just
not
port
this
error,
but
it
turns
out
kubernetes
with
node
ports.
If
you
have
the
same
port
number
allocated,
it
returns.
C
Validation
error,
not
a
in
already
exists
error,
and
this
is
due
to
the
way
ports
and
IP
addresses
are
allocated
in
kubernetes,
so
there's
no
fix
for
that,
but
I
spent
a
few
hours
digging
through
the
Knights
code
to
figure
out
why
it
was
returning
a
different
here.
So
this
is
in
contributor
shoutouts,
but
thank
you
and
abet
for
tracking
that
down
the
reproduction
case.
C
So
I
could
go
in
and
find
that
we
had
a
discussion
about
some
approaches
that
we
can
do
there
and
then
I
started
working
on
uploading,
CSI
artifacts
in
the
core
Valero
repo.
This
is
to
get
some
of
our
client
code
working
on
par
with
our
other
snapshot
types.
So,
like
our
rustic
snapshots
and
our
Valero
native
snapshots
can,
if
you
do
arrow,
backup
or
restore
describe
details,
you
can
get
the
types
of
snapshots
in
there.
This
will
facilitate
that
and
also
some
garbage
collection
logic.
C
E
That's
a
huge,
miss
Nolan,
but
by
the
way,
so
Steve
and
myself
and
and
and
and
team,
did
some
of
that
work
and
we
added
already
some
items
in
there
the
backlog.
Obviously
you
guys
work
in
the
weeds
every
day,
so
we
wanna
bottoms
up
putting
up
some
cases
and
issues
that
you
guys
feel
that
important
that
will
enable
our
users.
A
F
F
So
it's
basically
customized
allows
you
to
just
patch
the
river
of
deployments
and
I
tested
that
with
two
different
plan,
I
mean
two
plugins
being
patched
into
the
valor
deployment
and
I
only
used
the
backup,
storage
location.
I
didn't
bother
with
the
volume
because
it's
just
to
get
an
idea
the
flow
and
then
then
I'm
going
to
start
updating
the
the
actual
text.
In
that
sound
document.
A
F
D
Yeah
a
few
different
things
so
last
week,
I
mentioned
I've
mentioned
a
couple
of
these
just
updates
on
them.
So
I
had
been
working
on
some
refactoring
of
our
restore
code,
around
kind
of
the
discovery
API
and
how
we
deal
with
CR,
DS
and
and
all
that
and
so
I
got
that
PR
finalized.
So
it
finished
up
with
some
testing
and
a
little
bit
of
changes
to
the
code.
So
it's
up
there
ready
for
review
I,
think
everyone
has
already
taken
a
look
at
it
and
then
just
I
think
I
addressed
the
last
comment.
D
The
next
thing
I
was
working
on
was
some
refactoring
to
extract
a
lot
of
the
logic,
that's
currently
in
the
controllers
out
into
packages,
and
particularly
starting
with
the
backup
and
restore
code,
so
the
first
PR,
which
was
just
sort
of
a
prereq
for
getting
this
done
around
the
informer
cache
lifecycle
has
been
merged
and
then
I've
been
working
on
the
extracting
the
backup
code.
That
does
a
lot
of
the
validation
and
default
thing
for
the
API
objects,
as
they're
coming
in
moved
that
into
the
backup
package.
So
that's
so
work
in
progress.
D
D
Sounds
good
and
what
else
to
have
on
here?
Yeah!
It's
been
a
couple
days:
user
reported
a
bug
in
the
or
an
issue
with
the
Azure
plug-in,
where
backups
that
were
larger
than
256
Meg's
failed
to
upload.
So
it
turns
out
that
Azure
has
restrictions
and
their
blob
storage.
So
you
can
only
basically,
if
you
have
objects
or
blobs,
that
you're
trying
to
upload
that
are
larger
than
256
Meg's.
D
You
need
to
break
it
into
chunks
of
no
larger
than
100
Meg's,
each
and
upload
individual
chunks
and
then
upload
a
list
of
the
IDS
of
the
chunks
to
make
up
the
overall
blob.
So
this
user
was
running
into
it.
They're
backing
up
very
large
clusters,
so
I
was
working
on
this
I
think
I
got
it
implemented.
I
was
able
to
push
out
an
image
that
this
person
could
test
and
verified,
fix
their
issue,
and
so
yeah
I
think
this
one's
ready
for
review
as
well.
All
my
testing
looked
good.
A
C
D
I
I,
don't
know
yeah
I,
don't
know
if
rustic
has
code
to
do
chunking
or,
if
rustic
actually
uploads
any
objects
that
are
that
big
I
think
their
packs
tend
to
be
smaller
than
that,
but
I'm,
not
not
totally
sure
might
be
worth
looking
into
a
little
bit.
But
you
know
if
it's:
if
there
was
an
issue,
it
would
require
changes
in
the
rustic
good
base.
It's
probably
not
something
we
could
work
around
in
the
lair
right
right.
We
could.
We.
C
D
D
A
slack
thread
is
that
by
default
events
get
backed
up
kubernetes
events
because
they're
just
another
API
resource
and
this
user,
given
that
they
had
a
very
large
cluster
had
a
huge
number
of
events,
and
so
a
large
percentage
of
the
backup
size
was
due
to
those
events
which
probably
don't
need
to
be
backed
up
because
they're
sort
of
ephemeral
things
anyway.
So
you
know
excluding
events,
is
one
possibility
to
bring
down
the
backup
size,
but
still
this
is
something
that
we
we
should
support.
Shouldn't
shouldn't
have
an
arbitrary
limit
on
backup
size.
D
That's
a
good
question.
I
know
AWS
supports
like
multi-part
upload,
so
you
can
chunk
things
and
upload
individual
chunks,
but
I
don't
believe
they
have
a
requirement
that
you
do
that,
but
it's
probably
worth
looking
into
and
do
at
least
for
AWS
and
GCP
for
the
other
tier
plugins.
We
maintain
and
making
sure
we
wouldn't
have
similar
issues
there.
Yeah.
D
A
D
Yeah,
so
just
following
up
from
Nolan's
discussion
on
this
PR
around
the
conversion,
issues
with
CR
DS,
so
yeah
I
think
we
can.
We
can
and
should
push
out
a
1.3.1
I
think
it's
fine
to
go
ahead
and
do
it
today.
I
mean
I
think
this
will
be
the
only
issue
and
then
I
took
a
quick
look
through
the
commit
history
and
I.
D
Don't
think,
there's
really
anything
else
that
we
would
want
to
cherry-pick
into
this,
so
we'll
have
to
Nolan
will
have
to
create
a
1.3
branch
off
of
the
1.3
tag
and
then
cherry-pick
this
commit
into
there
and
then
probably
make
a
bunch
of
Doc's
updates,
because
we
have
you
know
anywhere
and
like
the
upgrade
instructions
or
anything
like
that.
We're
gonna
need
to
increment
the
version
number
yeah.
C
I
look
and
I,
don't
think
you
would
mention
in
slack
we
haven't,
we
haven't
really
done
the
patch
release
process
in
a
while,
like
I,
don't
I,
don't
think
we've
done
hatch
release
since
changing
our
release.
Notes
process
with
the
unreleased
stuff
shouldn't
be
a
big
deal
since
it's
just
one.
One
release
note
this
time,
but
yeah
yeah,
so.
D
C
Would
say
everyone
on
one-30
is
probably
upgrade,
because
what
this
does
is.
Basically,
if,
if
you
try
to
back
up
a
CR
D
that
has
a
whole
number,
no
I
wasn't
talking
series
you
try
to
back
up
in
a
CRD
with
a
whole
number
in
one
of
the
fields
that
is
defined
as
a
float,
which
is
usually
like
a
maximum
a
minimum
or
what
the
third
one
is,
but
that
could
be
in
the
top
level
or
any
number
of
subfields
I
tried
to
back
that
up.
C
D
C
It's
kind
of
hard
to
know
if
an
application
you're
using
has
CR
DS
that
use
it
until
you
run
into
it
and
from
what
I'm
hearing
upstream.
So
they
from
what
I'm.
What
I'm
understanding
upstream
is
I
think
there
may
be
a
miscommunication,
but
what
I'm
understanding
upstream,
they
don't
seem
to
think
the
from
unstructured.
C
Function
that
we
were
using
should
be
doing
this
integer
to
float
conversion,
but
I
think
that
that
may
just
be
a
misunderstanding
because,
like
in
my
test
cases,
the
number
5.0
or
any
other
whole
number,
so
2.0
3.0
those
ran
into
the
issue.
Their
floats
their
boat
floats,
but
it
still
caused
an
error.
So
we
can
work
around
it,
but
it's
still
kind
of
annoying
helm
like
Scott,
had
reported
helm,
ran
into
this.
G
Yeah
and
why
to
it
on
the
restore
was
also
pretty
160.
The
reason
I
was
asking
you
who
was
only
116
was
I
only
I
didn't
hit
it
on
the
back
of
I
only
needed
on
the
restore,
and
so
that's
why,
when
I
reported
it
when
I
tested
it
out,
we
found
the
restore
case,
but
we
didn't
find
back
updates
but
that
that
may
have
been
because
I
wasn't
testing
with
some
recent
code
that
you
were
adding
and
later
a
mesh
or
yeah.
So.
C
I
wrote
this
remapping
plugin
that
basically
inspected
CRD
to
see
if
it
was
a
structural
schema
and
what
we
needed
to
do
with
that,
and
excuse
me
what
I
did
not
really
thinking
about.
It
was
I
used
from
unstructured
again
and
so
I
reintroduce
the
bug
in
a
different
location.
So,
instead
of
being
on
restore,
it
was
now
on
backup
in
1.3.
C
So
this
is
fixing
the
book
again,
but
it
was
basically
playing
playing
whack-a-mole
so
for
CR
DS
to
go
into
the
go
type.
If
you
got
an
unstructured
from
the
kubernetes
server.
Unfortunately,
you
can't
use
the
runtime
default.
Unstructured
can
convert
or
die
from
unstructured
to
go
straight
into
the
go
type,
not
right
now,
anyway,.
D
And
that's
our
that's
our
standard
way
of
going
from
unstructured
types
which
we
get
back
from
the
dynamic
line
into
structured
types
in
the
Valero
code,
Basin,
and
so
it's
like
every
time
we
do
that
if
we're
doing
it
for
a
CR
D.
We
need
to
remember
that
we
can't
do
that
and
we
need
to
go
through
JSON
until
potentially
this
gets
changed
upstream,
but
there's
definitely
more
discussion
that
needs
to.
G
H
D
C
Right
right
in
and
I
was
even
having
a
hard
time
replicating
it,
because
if
I
did,
if
I
did
the
JSON
structure
and
did
two
unstructured
and
that
all
worked
fine,
it's
if
it
came
from
the
kubernetes
server
into
unstructured
and
then
I
did
from
unstructured,
then
it
would
fail.
If
I
take
the
unstructured
put
it
in
JSON,
then
unmarshal
the
JSON
back
into
the
go
type
directly.
Then
that
worked.
C
So
it's
that
specific
code
path
which
somebody
on
the
helm
chart
has
a
fix
for
it's,
it's
a
fairly
small
fix,
but
the
the
code
in
the
kubernetes,
repos,
fairly
old,
so
I
think
it's
going
to
just
require
some
more
communication
with
a
upstream
community
of
stream,
maintainer
'z
to
clarify.
What's
going
on
yeah.
C
I
think
it's
gonna
be
fairly
prevalent,
so
some
of
the
CR
DS
that
folks
were
sending
me
were
like
cert
manager,
one
of
them
that
really
confused
me
was
they
had
the
CSI
sierra
DS,
which
were
failing,
which
that
one
really
confused
me
because
they
weren't
the
same
definitions.
That
occasion
I
have
been
working
with,
so
I'm
not
sure
exactly
where
they
got
those,
but
it's
really
gonna
come
down
to
the
CR
DS
that
you
use.
C
I
C
C
They
offered
to
test
an
early
image
on
the
CRTs.
You
and
also
I.
Think
this,
the
node
port
stuff
on
services
with
two
3:08
I
think
we've
been
chasing
that
for
a
little
while
I
know
in
that
issue.
There's
another
one:
rep
there's
another
issue
referenced,
and
there
wasn't
a
reproduction
case
on
that
previous
issue.
But
thank
you.
C
C
Yeah
that
one
it's
just
there's,
there's
an
error
with
validating
the
the
allocating
ports
is
basically
just
a
wrapper
around
a
struct
that
counts
up
and
if
they
can't
count
up
anymore,
it
just
returns
error
that
says:
I
can't
allocate
anymore
and
that
gets
bubbled
up
all
the
way
through
the
rest
api
and
that
comes
back
through
Kate's
as
invalid.
Instead
of
oh
it,
that
port
already
exists
so
yeah
we
need
to.
We
need
to
revisit
that
one
way
or
another
in
bolero,
yeah.
J
Also,
it
seems
for
the
note
put
the
day
place
of
actually
validates
fast.
You
know
checks
if
the
pots,
if
any
of
the
note
post
I've
been
allocated
and
then
from
there.
It
now
goes
to
check
if
there,
for
example,
a
service
within
that
names,
namespace
exists
with
the
same
the
same
name,
so
it
looks
like
the
the
validation
is
fast
for
the
note,
Porter
location
and
then
next
is
the
the
name
of
the
service
object.
That.
E
C
J
I
was
taking
for
that
one
because
you
know
if
there's
a
service
in
a
different
name
space
that
is
using
that
not
put,
then
you
know
that's
it's
a
it's
a
it's,
an
actual
error
and
I
think
the
restore
shall
fail.
You
know
rightly
so,
but
if
it's
the
same
namespace
then
I
think
in
that
case
we
can.
You
can
scope
it
to
their
namespace.
Otherwise
you
know.
Definitely
we
don't
want
to
scan
through
all
the
namespaces
to
validate
that
I.
J
C
Actually
make
sense
why
they
will
check
that
first
yeah
in
that
case
and
for
context
on
this.
This
is
where
you
have
a
service
for
those
that
aren't
familiar
with
the
case.
It's
you
have
a
service
that
you're
trying
to
restore
in.
In
the
reproduction
case.
The
service
already
exists
in
the
namespace
and
when
you
try
to
restore
it
with
the
exact
same
port
instead
of
saying
it
already
exists,
it'll
say
that
ports
already
in
use.
C
So
it's
a
validation
error,
but
there's
a
bunch
of
edge
cases,
so
you
can
say
well
what,
if
the
port
is
already
in
use,
but
by
a
service
in
a
different
namespace
or
what,
if
there's
a
bunch
of
different
ports
on
the
no
ports
on
the
service,
or
so
we
need
to
look
at
some
avoiders
logic
for
restoring
restoring
objects.
That's
not
just
giving
objects
to
the
kubernetes
server
and
keying
off
an
already
exists.
Error,
so
I
think
there's
more
discussion.
It
needs
to
happen
there,
but
the
reproduction
case
helped
move
this
along.
A
B
B
Okay,
can
you
all
see
my
screen?
You
want
picking
up
yeah,
okay,
so
in
this
demo,
I
wanted
to
show
how
we
can
use
a
little
to
perform
backup
and
restores
of
applications
that
are
using
volume
start
by
CSI,
as
I
mentioned
before.
During
my
status
update,
we
will
be
using
the
v1
beta
one
snapshot
api's
to
perform
the
backup
and
restore
the
snapshot.
Api
is
were
recently
promoted
to
beta
and
they
are
compatible
kubernetes
117
onwards.
B
B
Kubernetes
117,
for
the
purposes
of
this
demo
and
in
the
interest
of
time
I
already
did
some
setup.
I
have
deployed
the
CSI
components
which
is
like
this:
yes
I,
provisional,
CSI
attached
and
all
the
other
comp,
along
with
the
CSI
driver
on
this
cluster.
All
of
them
are
running
as
individual
parts,
just
that
make
kind
of
makes
it
easier
for
me
to
show
what's
running
as
instead
as
against
running
them
as
side
cars.
B
So,
as
you
can
see,
I
have
all
the
CSI
components
running,
and
this
is
the
driver
that
we
will
be
using.
The
extra
containers
here
here
are
like
the
liveness
probes
and
one
other
contingent
that
I
can't
remember
so.
I
have
that
stuff
running
I
have
storage
class
corresponding
to
this
driver
all
set
up
and
ready
to
go
that
I
will
be
using
that
that
will
be
performing
the
provisioning
of
CSI
volumes
and
performing
snapshots.
B
So
that
is
what
is
in
my
cluster
I
also
have
like
a
sample
application
that
I
deployed
to
use
the
this
is
an
CSI
application
using
CSI
back
volumes.
So
I
have
this
running
in
this
namespace
called
CSI
app
clear.
My
screen
so
now
I
have
this
part
which
is
running
using
this
volume
claim
and
which
is
backed
by
this
volume
provisioned
by
the
host
path,
CSI
driver.
B
B
What
what
I
didn't
show
before
was
I
will
show
this
later,
but
this
part
mounts
this
volume
at
/data,
so
we,
as
you
can
see
there
is
like
nothing
in
this
data.
Now
we
will
kind
of
generate
some
data
here.
So
I
have
this
handy
stuff
to
generate
some
data.
I
will
let
this
run
for
like
three
seconds
and
then
this
should
have
some
data
I'm
not
going
to
cut
this
file,
because
this
is
going
to
be
a
huge
file,
as
you
can
see,
so
I
have
generated
some
data.
Now.
B
B
Now
we
have
this
application
that
is
running.
That's
generated
some
data.
Now
we
will
use
velaro
to
perform
a
backup
using
a
command
that
most
of
you,
if
not
all
of
you
know
or
familiar,
are
familiar
with
I
will
gel
I'm
doing
a
bolero,
create
backup
I'm,
going
to
call
this
backup,
CSI
app,
backup,
demo
and
I'm
going
to
include
just
this
name
space.
B
B
Yeah,
so,
as
you
can
see,
it
is
now
as
part
of
creating
the
backup
bolero
will
create
the
volume
snapshots
and
the
values
volume
snapshot,
API
objects
which
will
instruct
the
CSI
driver
to
perform
the
stock
short
operation
on
the
volumes.
So
so
the
the
CSI
plugin
for
Valero
is
what
is
taking
care
of
all
this
and,
as
you
can
see,
the
backup
is
now
completed
and
we
should
be
able
to
see
the
volume
snapshot
objects
that
Valero
created
for
us
to
perform
the
backup.
So.
H
B
B
B
B
You
can
say
so.
This
policy
basically
says:
should
this
volume
snapshot
content
stick
around
if
the
associated
volume
snapshot
goes
away,
so
I've
set
it
up
to
be
retained
in
case
the
volumes
not
shot
gets
deleted.
Now,
once
this
name
space
goes
away,
we
will
go
back
to
verify
that
the
volume
snapshot
contents
X
exists.
My
session
here
cut
them
in
hates.
The
part
got
killed
so
waiting
for
this
okay,
the
name
space
is
now
deleted,
and
then
we
will
verify
that
this
still
exists,
but
this
doesn't
okay.
B
Now
we
are
ready
to
restore
well
not
quite.
This
is
like
the
workaround
that
I
mentioned
for
the
bulk.
This
is
a
workaround
for
the
bug
that
I
mentioned
during
my
update.
We
will
have
to
so
what
just?
What
happened
right
now
is
the
volume
snapshot,
object
or
deleted,
but
the
reference
to
that
object
in
the
volume
snapshot
content
still
exists,
so
that
should
have
been
cleared
up
and
that's
the
bug
that
has
been
open.
B
B
What
I
want
to
edit-
and
this
is
the
reference-
this
is
the
dangling
reference
to
the
volume
snapshot
that
got
left
behind,
so
we
will
delete
these
two
lines
and
save
and
exit
now
to
perform
a
restore.
We
will
again
use
a
command
that
is
bolero
creator
store
and
we
will
be
referencing
the
backup
that
we
created
before
so
this
is
we.
This
is
the
command
that
we
are
running
well,
arrow
creator
store.
B
B
Okay,
the
restore
is
complete
as
part
of
the
restore
well
or
what
the
what
Valero
did
is
it
used
the
volume
snapshot
as
the
data
source
for
the
persistent
one
you
claim
to
populate
the
data
back
onto
the
volume
that
was
going
to
be
provisioned
for
that
claim.
So
we
will
take
a
look
at
that
later,
but
before
before
that,
we
will
make
sure
that
our
name
space
is
back.
Our
application
is
back
and
running.
B
So,
as
you
can
see,
the
name
space
is
back,
CSI
get
pards,
I
want
to
see
PVC
and
then
I
also
want
to
see
the
PB.
So
we
see
the
my.
Our
application
is
happily
running
with
this
volume
claim
that
was
created
65
seconds
ago
and
then
this
persistent
volume
that
was
provisioned
sixty
five
seconds
ago.
So
this
all
guarantees
that
our
our
pod
came
back,
but
did
our
volume
the
data
that
we
wrote
here
into
the
volume?
Did
this
come
back?
B
B
C
K
K
B
K
I
B
K
J
B
B
So
when
Valero
performs
the
restore,
it
is
going
to
see
that
this
volume
snapshot,
content,
object,
already
exists,
and
it's
going
to
say:
hey
I'm
not
going
to
touch
this,
because
this
is
a
resource.
That's
already
there
so
and
I'm
going
to
proceed
with
the
other
parts
of
the
restore
that
I
have
to
do.
But
then
this
reference
will
will
kind
of
no
bills
kind
of
stick
around
and
that
is
that
won't
let
the
namespace
mapping
to
go
through.
B
B
B
But
the
workaround
there
is
like
you,
go
in
and
clean
up
the
namespace
as
well,
so
that
the
namespace
mapping
will
kind
of
go
through
the.
The
other
thing
that
you
could
proper
possibly
do
is
not
have
a
retain
policy
on
the
volume,
such
short
content
and
let
the
volume
snapshot
content
also
go
away.
But
then
there
is
another
problem
there,
which
is
the
lifecycle.
Management
of
the
kubernetes
objects
should
not
be
tied
to
the
lifecycle
of
T
snapshot
in
the
storage
provider.
B
A
Awesome,
well,
we
don't
have
any
any
extra
comments
here.
I
think
you
so
much
to
see
for
the
demo.
Thank
you,
everyone
for
the
input
and
comments
as
well
and
and
yeah
looking
forward
to
the
next
demo
or
cluster
migration.
Maybe
next
week,
yeah
we're
so
that'd
be
awesome
with
that
have
a
fantastic
week.
Everyone
and
yeah
hope
to
see
you
during
the
next
meeting
have
a
good
one,
see
ya
bye.
Thank.