►
Description
April 21, 2020
Status Updates
[stephanie] Introducing Velero’s new PM!
[nrb]
Outlined our approach to versioning the backup file format https://velero.io/docs/master/output-file-format/#output-file-format-versioning (PR)
Finished up PR 2323, Velero master will now upload CSI snapshot information for use by client-side commands (such as velero backup --details)
Started in on 2448, which will add the client-side info.
Investigated issue 2383, where CRDs are reporting errors about fields being duplicated in versions yesterday.
This is a result of the “fix” for v1beta1 to v1 CRD restores in Velero v1.3.0. More in discussion time.
Will be on the Brett Fisher show next week talking about Velero
[steve]
was on community support
CSI reviews & testing
draft PR & testing for backup progress
[carlisia]
wip: bsl/vsl/velero readiness: https://github.com/vmware-tanzu/velero/issues/1967
presented Velero to Brazilian meetup
[ashish]
CSI bug fixes and testing
Community support this week
Sharing a link to the survey we sent out 3w ago. If you’ve not had a chance to look at this, please do. Survey is available here.
Discussion Topics
[marccampbell] Willingness / timeline for custom restore hooks? https://github.com/vmware-tanzu/velero/issues/2116
[mayank] patch release for plugin velero.io/change-pvc-node-selector
I had a discussion about this on slack with Steve. It won’t be a good idea to have a patch release for the above plugin since it’s not a bug-fix.
[nrb] Possible fixes for 2383
if the object is truly a v1beta1 CRD, re-fetch it using a v1beta1 client in the plugin
Use the API server’s internal representation & conversion functions to get the correct version on restore. If we keep this in the backup tarball, this has ramifications for the backup file format
Essentially 3 versions of CRDs in Go code:
v1beta1.CustomResourceDefinition
v1.CustomResourceDefinition
apiextensions.CustomResourceDefintion - this one is internal to the API server, and is version-independent.
Merge #2373 and try v1 CRD, if it fails, try applying v1beta1
Action items:
Get #2373 merged
Try 1 & 2 for v1.4
investigate 2 for long term implementation. Problem with it is that it may trigger a Velero v2 as the internal CRD representation isn’t directly restorable
To prevent this kind of thing in the future: document regression test scenarios, stand up e2e kind tests, and begin automating.
[carlisia] - Tilt
Demo of running Tilt with Velero
Walkthrough of the setup
Contributor Shoutouts
@hasheddan for pointing out the CRD conversion functions!
@TomaszKlosinski for providing the elasticsearch & kibana CRD issues
Helm
@tareqhs - added quotes to extraEnvVars to support special chars
@yurinnick:
for making our chart Helm 3 compatible!
allowing multiple backup/snapshot providers
fixed timeout chart-testing parameter in the chart
prioritize location provider over global provider
@ahmadhajali - enable specifying docker image by digest in addition to tag
@limed fixing toYaml generation
A
Hi
everyone
and
welcome
to
this
week's
episode
of
the
Balearic
community
meeting
slash
open
discussion
per
usual.
We
have
a
bunch
of
things
to
go
through
and
I'm
gonna
link
the
agenda
in
the
chat
here.
So
if
you
want
to
add
anything
to
the
agenda,
please
do
so
in
the
discussion
topics.
We
have
a
bunch
of
discussion
topics
that
we'll
try
to
get
to
those
as
quickly
as
possible.
Today
we
do
have
a
few
carryovers
from
from
last
week
as
well.
A
B
A
D
That's
mainly
mainly
for
Rafael
burritos,
current
PR,
with
adding
all
versions
of
resources
into
the
PR
or
into
the
backup
realizing
now,
I
probably
should
have
incremented
that,
for
the
next
I
know
might
have,
but
this
this
kind
of
outlines
the
rules
that
we
should
follow
for
incrementing
the
file
format
version
on
on
Valero.
The
main
thing
is
we're
gonna
try
to
follow
semantic
versioning,
where,
if
we
add
add
new
directories
or
files,
we
will
increment
the
minor
and
patch
versions.
D
If
we
move
stuff
around
such
that
it
is
not
restorable
by
previous
versions
of
valerio's
of
Valero.
We
will
increment
that
with
a
major
version
of
Valero
and
it
will
be
a
major
version
for
final
format,
but
just
because
we
increment
a
major
version
of
Valero
doesn't
necessarily
mean
that
file
format
is
changing
the
version.
D
D
D
D
D
There
are
a
few
issues:
2383
being
one
of
them
where
CR
DS
reporting
errors
about
fields
being
duplicated,
they
would
say
something
like
all.
The
versions
had
the
same
field
in
this
particular
case,
additional
printer
columns,
they're
all
the
same
version,
all
the
same
value
and
you
should
put
them
in
the
top
level.
Instead,
after
doing
about
two
research
yesterday,
it
turns
out
this
was
a
result
of
fix
that
we
did
in
1.30
it's
more
research
and
that
fix
was
not
enough.
D
E
A
About
Valero,
so
yeah
this
is
happening
on
Thursday
next
week.
I
believe
it
is
at
1:00
p.m.
Eastern
I'll
make
sure
to
update
the
notes
here
with
a
when
the
lake
goes
live
as
well
as
we
can
have
that
there
and
then,
of
course,
we'll
add
this
to
the
YouTube
channel
that
we
have.
So.
Thank
you
so
much
Nolan
for
stepping
up
and
doing
this
yep
yeah.
F
Yeah
relatively
quick
update
from
me
this
week,
I
was
on
community
support
last
week,
so
did
my
best
to
keep
up
with
incoming
github
issues
and
slack
stuff
I.
Think
that's
in
a
reasonable
place.
Beyond
that,
I
was
working
on
continuing
to
do
some
reviews
and
testing
on
the
CSI
snapshot
work.
So
I
made
it
through
a
little
bit
more
testing
there
filed
a
couple
of
issues.
I
think
the
like
the
core
backup
and
restore
flows
are
looking
really
good
here.
F
How
many
total
items
we're
gonna
back
up
so
that
we
can
report
progress
as
we're
going
and
so
right
now
that
PR
is
just
outputting.
The
progress
information
to
the
backup
log,
which
is
also
visible
in
the
server
pod
log,
so
I
think
this.
This
PR
I
probably
won't
add
more
to
this
PR,
but
there
will
be
at
least
one
more
PR
to
also
update
on
backup
progress.
F
We
talked
about
doing
it
in
the
status
of
the
backup
custom
resource,
so
periodically
updating
some
fields
there
that
indicate
what
what
number
of
objects
we've
processed
and
then
we
we're
also
having
some
discussion
on
whether
we
should
use
kubernetes
events
to
report
information
like
this.
So
if
you
need
to
do
a
little
bit
more
research
there,
but
that'll
probably
come
in
or
PR
I
think
this
one.
That's
in
your
can
more
or
less
stand
as
it
is.
F
A
This
we
we
talked
about
this
last
week
and
previously
as
well.
I
remember.
There
were
some
issues,
of
course,
getting
how
getting,
how
big
the
backup
would
be
and
and
having
the
process
or
progress
shown
there.
So
I
think
this,
showing
how
many
items
are
being
backed
up.
That's
a
really
smart
way
of
showing
progress
to
you.
F
Yeah
and
it's
it's
a
little
bit
tricky,
because
when
we
scrape
the
API,
we
get
a
set
number
of
objects,
but
then,
as
we
actually
go
to
to
process
those
objects,
we
may
have
plugins
that
pull
in
additional
related
items.
So,
for
example,
if
we're
you
know,
if
we're
backing
up
PVC
from
a
namespace,
we
don't
know
until
we
go
to
actually
run
the
plugins
for
that
item,
that
we're
gonna
pull
in
the
PD,
the
PV
that
it
uses,
which
is
a
cluster
scoped
resource.
F
C
Yes,
not
many
items
from
me
either
I
presented
Valero
for
a
Brazilian
beat
up
last
week,
which
was
great
and
I'm
working
on
the
the
filler
or
startup
behavior
and
I
have
to
clean
up
the
description
of
that
tickets,
because
I
had
misunderstood
a
couple
of
the
things.
Basically,
besides
the
happy
path.
C
C
One
of
them
is
invalid,
but
it's
not
the
fault
that
the
default
is
invalid
and
that's
the
case
where
we
need
to
handle
another
user.
No.
Well,
we
you
could
make
a
backup
if
you
lose
points
you
so
a
different
bill,
Sayle
to
create
your
backup,
so
in
making
it
the
server
having
status
of
waiting.
So
we
come.
It
I
think
that
sums
up
the
the
change
for
that
circuits
and
yeah
I
was
supposed
to
be
updated,
but
I
haven't
yet
that's
it.
E
A
C
A
A
A
E
So
pretty
light
update
from
me
as
well.
I
was
out
for
a
couple
of
days
last
week
and
I've
been
doing
mostly
the
saw
the
CSI
testing
with
Steve
and
fixing
some
of
the
bugs
we
found
and
then
I'm
on
community
support
this
week,
and
then
that
should
kind
of
take
up
my
whole
week.
I
have
a
few
questions
that
have
not
been
answered
on
slack
that
came
in
earlier
this
morning,
so
I'm
like
trying
to
look
at
that
and
figure
out.
E
What's
going
on,
I,
don't
know
if
all
the
words
on
the
call
here
to
talk
about
something
he
posted
his
channel
thought
I
I
would
reach
out
to
him
and
it's
in
the
channel,
if
he's
not
in
the
call
right
now.
I
also
wanted
to
share
the
link
to
the
survey
again
that
we
put
out
like
two
weeks
ago
three
weeks
ago.
Maybe
so,
if
folks
haven't
had
a
chance
to
take
a
look
at
it,
please
do
so.
That's
that's
pretty
much.
A
F
This
was
this:
one
was
a
carryover
from
last
week.
I
know
he
was
on
the
call
last
week,
but
I
think
we
ran
out
of
time
to
cover
this
I
know
he's
interested
in
in
pursuing
this.
Adding
restore
hooks
to
be
able
to
execute
custom
commands
on
restore,
and
he
did
interest
in
driving
design
for
this.
So
I
can
I
can
just
reach
out
on
this
issue
and
let
him
know
we'd
be
happy
for
him
to
take
the
lead
on
the
design
and
and
see
what
we
can
do
to
help
yeah.
D
D
D
D
He
provided
a
small
CRD
to
start
looking
at
this,
and
I
was
taking
some
notes,
so
I
started
with
this
and
the
issues
that
we
saw
were
we
couldn't
restore
sub-resources
because
they
all
had
the
same
status.
But
if
you
look
at
this
there's
a
lien
there's
no
sub
resources
and
their
status
is
blank.
So
if
we
scroll
down
some
more.
D
More
so,
if
I
just
remove
those
at
worked,
if
you
scroll
down
some
more
I
found
that
after
I
applied
the
elastic
search,
the
elastic
search,
Sarah
D
I
got
the
additional
printer
columns
was
moved
from
the
top
level
spec
into
the
version.
It
was
triplicated
and
then
the
next
comment
that
21
hours
I
found
it
was
actually
coming
back
through
the
v1
end
point.
So
you'll
notice
that
cube
CTL
get
raw
was
going
through
the
V
1
beta
1
custom
resource
definitions
end
point.
D
D
D
The
other
approach
that
is
more
involved
was
to
go
with
some
of
Dan's
suggestion
and
use
the
API
servers
internal
representation
of
CR
DS,
which
I
need
to
explore
this
a
little
bit
more,
but
essentially
there's
three
versions
of
CDs.
In
go
code.
There's
V
1
beta
1
customers,
rows
definition,
there's
the
V
1
customer
resource
definition
and
then
there's
this
bottom
one.
That's
unburdened!
It's
the
API
extensions
dot,
custom
resource
definition.
D
This
is
the
one
that
the
API
server
uses
and
there's
a
whole
bunch
of
conversion
functions
that
were
that
are
auto-generated
that
you
can
in
dan
links
to
this
in
the
in
the
issue,
he's
got
yeah
I've
got
links
to
it
too.
Sorry
that
comment
20
hours
ago,
yeah
yep,
that
one
so
we've
got
these
auto-generated
functions
that
help
us
convert
back
and
forth
between
this
internal
representation
and
the
version
representations.
D
Is
a
little
easier
but
I
think
doing
this.
Internal
representation
might
be
more
correct
long
term,
but
I,
don't
I,
don't
know
what
it
means
for
the
backup
format
yet
so
at
this
point,
I
think
I'd
open
it
up
for
questions
because
I
don't
know
if
I'm
describing
it
correctly
and
and
just
see
what
people
think.
Oh
and
I
guess
one
more
thing,
because
this
CRD
issue
has
been
a
long,
long
standing
thing.
F
F
D
D
Basically,
the
issue
is
arising
because
we
changed
the
version
string.
It
came
back
from
the
version
from
the
API
server
as
the
preferred
version
V
1.
We
changed
the
version
string
on
it,
but
the
kubernetes
server
had
already
done
all
the
conversions
that
made
it
v1
valid,
but
Valera
broke
it
because
so
using
the
elastic
start.
Example,
additional
printer
columns
is
not
it's
not
valid.
D
So
if
it
comes
back
as
a
v1
beta
one,
that's
actually
incorrect
and
so
yeah
it,
the
heuristic
appears
to
be
applied.
Just
fine.
It's
it's
the
fact
that
the
version
strings
getting
changed
and
I
think
the
problem
was.
The
data
set
I
was
using
when
testing
this
out.
Some
Sierra
T's
work
just
fine,
but
others
didn't
so
like
for
these
regression
tests
that
I'm
proposing
we
would
have
a
much
larger
data
set
applying
to
whatever
latest
version
of
kubernetes
is.
F
D
Found
that
yet
the
closest
I
found
is
fetch
it
from
the
v1
beta
one
endpoint,
which
is
what
I
that
raw
command
was,
and
that's
very
similar
to
like
what
Raphael's
work
does,
which,
which
may
or
may
not
be
what
our
plugin
could
do
like
that
plug-in
work
may
overlap
with
his
PR
in
terms
of
when
V
1
beta
1
goes
away.
I
don't
know,
I'll
have
to
find
out
in
terms
of
reapplying
it.
The
the
guidance
has
been
get
it
from.
If
you
made
it
V
1
beta
1,
get
it
from
that.
Endpoint.
F
Okay,
yeah
I'm
not
and
just
quick
comment
on
number
2
I
mean
I
I.
Definitely
you
know
we
can.
We
can
look
at
this
whether
we
want
to
store
the
the
storage
versions
of
API
objects,
but
I
agree.
It's
definitely
a
you
know.
It's
a
major
change
to
what
we're
storing
in
a
backup
tarball
like
these
things
are
not
things
that
can
be
applied
directly
to
an
API
endpoint,
so
I
I
wouldn't
want
to
go
down
that
path
without
kind
of
carefully
considering
it
yeah.
D
Kind
of
going
back
to
my
first
status,
update
honestly
I
think
that
might
trigger
valerio
2.0
just
because
it
would
be
a
major
format
update
to
the
file
format,
because
it
would
then
it
would
then
break
the
expectation
that
you
could
just
take
whatever's
in
the
tarball
and
directly
apply
it.
That's
my
suspicion
I!
Don't
know
that,
but
that's
kind
of
I
think
it's
more
long-term
sustainable,
but
I
also
kind
of
want
to
deliver
this
fix
in
1.4
and
I.
G
But
question
is
what,
if
maybe
gonna,
think
with
you
offline,
but
what,
if
we
use
the
PR
prototype
that
take
backups
of
all
versions
right
and
then
doing
the
restore
we
manually
use
that
version
to
see.
If
what
happens,
then
at
least,
if
we
release
the
backup
of
all
versions,
at
least
is
operational,
is
something
that
we
can
provide
without
the
logic
of
restoring.
You
know
what
I
mean
yeah.
D
D
F
D
Yeah
I
think
I,
like
I,
wanted
to
get
that
that
that
backup
plug-in
in
this
soon
this
week
like.
But
that
is
that's
a
third
option
for
now
to
get
1.4
out.
I
think
we
do
need
to
revisit
a
handling
of
CR
DS,
longer-term
to
be
more
in
line
with.
How
upstream
does
it
but
yeah
I
kind
of
I
kind
of
want
to
put
this
post
CRD
saw
go
to
bed
and
then
for15
get
our
testing
better,
because
I
I
don't
like
leaving
our
users
in
this.
D
In
this
scenario,
in
getting
getting
our
end-to-end
tests
and
having
better
signal
around
this
and
having
this
seems
like
a
good
candidate
to
start
our
end-to-end
tests,
because
it
doesn't
rely
on
volumes,
it's
just
API
objects,
so
it,
and
that
would
give
us
signal
like
we
can
just
start
throwing
CDs
at
it.
Like
people
give
us
their
DS,
we
can
throw
things
at
it.
D
But
it
sounds
like
we
have
three
options,
two
of
which
the
the
backup
backing
up
all
versions
and
then
trying
the
to
see
our
DS
does.
If
you
want
beta
1
that
v1
or
trying
the
client,
which
honestly
is
duplicating
work
with
the
backup.
Those
two
seem
relatively
low
effort
in
terms
of
getting
out
for
thief
v1
for.
C
C
C
C
F
C
C
C
C
C
C
C
A
C
A
F
C
There
was
addition
to
quotes
for
the
extra
and
fours
you
know.
So
that
mean
that
way.
It
supports
special
characters
and
Yuri
Nick.
He
has
four
items.
First,
he
made
the
home
chart
home
three
compatible,
which
was
great,
and
he
added
a
PR
to
allow
multiple
backup
and
snap
shopped
providers
jumping
to
the
last
one
here.
Actually
he
also
had
another
pair
to
make
it
so
the
what
would
be
prioritized
would
be
the
configured
location
provider
in
order
global
default
one
and
he
also
added
a
pair
to
fix
the
timeouts
testing
parameter
in
the
charts.
C
We
had
a
contribution
from
ahmed
hajjali
to
enable
specifying
a
docker
image
by
digest.
In
addition,
in
addition
to
the
way
we
did
originally
with
the
tag
and
lime
d
for
fixing
the
Toyama
generation,
which
got
broken
by
one
of
these
one
of
the
one
of
the
PR,
some
non
naming
names,
but
because
also
was
my
failure
to
test-
and
it
was
great
so
now
he
put
defects
in
and
that's
it.