►
Description
Oct 13, 2020
Status Updates
carlisia
DownloadRequest migration to kubebuilder/runtime-controller: https://github.com/vmware-tanzu/velero/pull/3004
Testing https://github.com/vmware-tanzu/velero/issues/2991 (BSL controller loop always invokes init() regardless of backup location’s ValidationFrequency)
PR reviews
nrb
Code for https://github.com/vmware-tanzu/velero/issues/2570, but need to test against clusters now.
Working on community issue support process with Carlisia to help improve response time on GitHub issues.
Need to get the Azure and CSI plugin updated and out in the next day or so.
Noticed PRs that go from draft to open aren’t getting processed by the review Action; may look into it, or just file an issue.
bridget
PR for deleting backups when cloud resources have been deleted (https://github.com/vmware-tanzu/velero/pull/2993). Need to follow up on comments from @nrb.
Discussion Topics
Phuong
enhance BackupItemActionPlugin with timeout https://github.com/vmware-tanzu/velero/issues/2922
Aspects to this are in the wait for AdditionalItems to be ready
[nrb] Can we make timeouts work for both?
[nrb] Are the timeouts per plugin, per plugin type, or universal?
sseago
https://github.com/vmware-tanzu/velero/pull/2867 was for a specific CR in OpenShift
[nrb] there’s a use case for the TAS Postgres operator now too
dymurray
Supporting restore progress reporting https://github.com/vmware-tanzu/velero/issues/21
Tackle restores similar to the backups right now as a first phase
Current issue is on restore we don’t have a fully accurate number of items. For example, when using --include-namespaces, we don’t know the number of to-be-restored resources ahead of time.
Getting a 2 pass restore system would help immensely here, since we would pre-process and match resources before sending them to the API server. This would give us an overall count to report for progress.
Antony
https://github.com/vmware-tanzu/velero/issues/2991:
BSL controller initializes backup storage locations almost every second
Significant issue for object stores where init has expensive calls (e.g API calls)
Any chance we could get a hotfix? When does v1.5.2 ship?
Contributor Shoutouts
@mickkael: Allow Timezone change in the container
@jenting: [velero] Add helm annotation for velero.io/v1 schedule CRD
A
B
B
B
It
could
be
wrong,
but
anyway,
trying
to
make
that
work
and
the
other
thing
is
there.
There
was
a
report
of
the
the
way
the
bs,
the
backup
storage
location,
validates
is
now
really
taking
into
account
the
free,
the
validation
frequency
setting
and
a
wonderful
soul.
I
also
opened
the
pr
to
fix
it,
so
I'm
testing
that
to
get-
and
hopefully
maybe
we
can
merge
that
today.
B
B
D
There
appears
to
be
a
race
condition
where
sometimes
valero,
as
a
result
of
how
valero
restores
pvs
and
pvcs
a
pv
can
get
claimed
by
another
pvc
in
a
cluster
and
basically
the
way
valero
volume
restore
works,
is
it
it
restores
a
persistent
volume
and
then
the
persistent
volume
claim
and
what
was
as
a
summary.
What
was
happening
here
is
the
persistent
volume
was
being
restored
and
then,
in
some
other
name
space
that
was
not
the
target
namespace
the
kubernetes
pvc
controller.
D
D
To
make
sure
this,
this
doesn't
change
anything
in
actual
clusters,
but
this
appears
to
be
mostly
an
issue
in
really
busy
clusters
and
it's
a
race
condition.
So
it's
not
always
something
that
happens,
but
it
took
us
a
while
to
understand
this
issue,
so
we're
gonna
try
to
get
this
fixed
into
152
and
there
can
be
security
implications
if
you're
using
namespaces
as
a
security
construct
to
separate
privileged
access.
D
So
yeah
excuse
me
this.
This
took
a
little
while
to
understand
exactly
what
was
going
on,
but
it
boils
down
to
valero
needs
to
keep
keep
the
claim
ref
on
persistent
volumes
when
it's
restoring
and
if
you're
using
the
namespace
remap
feature
it
needs
to
update
the
the
namespace
on
the
claim
ref
so
that
everything's
linked
up
correctly.
D
D
D
D
Yeah,
so
the
next
one,
as
carlisa
mentioned,
we
did
some
brainstorming
yesterday
on
ways.
We
can
improve
our
responsiveness
to
github
issues
that
are
raised,
so
we
want.
We
want
to
make
sure
we
get
to
people
when
they
they
raise
support
issues
in
a
timely
manner
and
we're
also
looking
at
how
we
can
make
sure
we
have
a
framework
for
scheduling,
actual
fixes
and
giving
people
reliable
timelines
for
that
for
those
fixes.
D
D
D
D
B
B
D
Because
I
think
we
should
definitely
use
the
draft
feature,
it's
it's
great.
I
just
noticed
I
was
like
hey
this.
This
one
was
ready
for
review
and
people
weren't
tagged
on
it,
and
so
I'm
thinking
that
the
action
wasn't
firing
and-
and
I
think
we
only
have
it
specified
for
two
events
like
if
you
open
a
brand
new
pr
that
doesn't
go
through
draft,
and
I
forget
what
the
other
one
is.
E
Hi
everyone,
so
I
have
a
huge
amount
to
report,
as
I
had
a
few
days
of
pto
this
week,
so
I'm
just
back
today.
So
last
week
I
opened
the
pr4
issue
that
we
discussed
during
last
week's
community
meeting
around
attempting
to
download
the
charabal
to
process,
delete
item
action,
plugins
and
the
error
that
happens
if
the
associated
cloud
resources
have
already
been
deleted
and
nolan's
already
commented
on
this,
given
he's
already
looked
at
the
pr,
so
I
am
need
to
follow
up
on
those
comments.
E
I
did
take
a
quick
look.
I
just
wasn't,
I
think,
maybe
just
I'll
follow
up
in
the
pr
just
whether
how
far
we
need
to
go
with
either
retry
logic
or
trying
to
introduce
new
error
types
and
whether
that
was
something
that
we
would
do
now
or
get
an
initial
fix
in
first
and
then
come
back
to
this
with
a
better
design
later.
But
I'll
follow
up
with
you
after
this
or
on
the
pr.
E
D
If
we,
if
we
included
any
retry
logic
in
this
pr,
I
would
think
it
would
be
simple,
like
one
or
two
like
literally
hard-coded,
one
or
two
tries
to
see
if
it
was
if
it
was
transient
and
then
just
log
that
we
couldn't
get
it
without
getting
super
deep
into
that
that
that's
my
thinking,
okay,
but
yeah.
I
think
anything
that
that
dives
all
the
way
down
into
the
plug-in
level
stuff.
I
think
we
need
to
not
address
it
here
and
address
that
directly
in
the
with
a
separate
design
document.
E
D
Yeah,
I
I
think
each
I
think,
there's
a
404
type
or
a
404
error
that
we
bubble
up
somewhere,
okay,
but
I'd
have
to
look.
If
it's
like
it's
404,
then
we
know.
But
yes,
I'd
have
to
look
at
the
the
persistence
types
again,
because
some
of
that
is
something
we
have
to
translate
between
the
object,
storage
platforms,
which
is
a
little
tricky.
E
F
Basically,
we
want
to
able
to
time
out
when
we
run
our
plugin
the
background
information
of
that
would
be
when
we
run
a
snapshot
on
the
vsphere
and
it
it
takes
a
long
time
because
of
some
we
try
or
something
like
that,
and
it
it
take
a
long
time
and
then
when
we,
because
it
is
at
our
consistent,
so
we
don't
want
to
it
takes
too
long.
We
have
to
be
timeout
within
a
few
seconds.
F
F
D
D
G
D
H
Yeah,
I
was
going
to
say
I'm
in
my
case.
The
the
timeout
part
is
more
a
question
that
was
kind
of
an
open
question
as
to
because
one
of
the
things
that
I'm
that
that
proposal
is
to
add
a
way
of
when,
when
a
plugin
sends
an
additional
items
saying
we
need
this,
there's
no
code
right
now
that
waits
for
that
item
to
actually
exist
and
be
ready,
and
you
know
we
don't
want
to
wait
forever
for
that.
H
H
Guess
I
guess
the
difference
is
in
in
in
this
case
that
what
you're
talking
about
the
timeout
itself
is
the
point
of
being
added
and
for
my
proposal
a
time
out
is
kind
of
an
aspect
of
this.
We
need
a
timeout
in
place
in
order
for
my
plug-in
enhancement
to
make
sense,
because
otherwise
things
could
block
forever.
But
you
know
the
timeout
is
kind
of
secondary
to
the
the
intent
which
is
to
prevent,
restores
from
failing.
You
know
if
the
item
isn't
there
yet
right.
H
F
D
F
A
H
Is
that
a
quick
question?
I
know
I
missed
the
meeting
last
week
and
no
one
had
brought
up
my
design.
Pr
is
something
to
talk
about.
I
didn't
know
if
that
was
something
that
we
needed
to
talk
about
today
on
here
with
their
additional
concerns.
Beyond
that
timeout
question
or
or
whether
that
was
a
different
different,
you
know
concern.
D
H
From
when
I
heard
from
dylan
I
had
some,
I
got
the
impression
there
was
some
concern
as
to
whether
we
were
you
know
whether
this
was
really
needed
or
whether
there
was
some
other
way.
That
made
more
sense.
I
mean,
I
don't
know
it's
worth
mentioning
again
the
kind
of
the
context
as
to
where
I
saw
that
we
needed
this,
and
that
was
specific
to
one
of
one
of
the
open
shift.
H
Resources
is
the
image
image
stream
tag
where,
basically,
if
you
have
a
one
image
stream,
that's
an
alias
referencing
another
tag.
If
you
restore
them
in
the
wrong
order,
the
the
the
inner
stream
tag
that
references,
the
other,
will
fail,
restore
you.
Actually,
it's
not
a
valero
issue.
You
know,
you
know.
H
Even
a
you
know,
just
a
regular
create
will
fail
because
it
fails
validation
because
it
references
something
that
doesn't
exist
and
if
you
know
reverse
the
order-
and
you
do
the
the
thing
that's
pointed
to
first,
then
it
will
succeed.
But
we
have
this
race
condition
where
you
know,
if
you're
submitting
one
restore
followed
by
the
other
immediately
following
and
there's
not
enough
time
in
between,
it
may
still
fail
right.
D
And
I
I
also
spoke
to
someone
on
our
working
on
a
post
rest
operator
here
within
vmware,
and
this
proposal
actually
would
be
helpful
for
them
as
well.
Okay,
so
they
they
are.
They
are
running
into
some
order
of
operations
that
this
the
kubernetes
kind
of
eventually
consistent
model
is
unfortunately,
not
not
working
out
there.
So
right
having
having
this
forced
weight
would
help
them
yeah.
So
I
think
my
question
was
I
I
I
wasn't.
D
H
You
know
one
restored
before
the
other,
and
if
you
want
that,
then
you
also
want
the
ability
to
wait
to
make
sure
it's
ready
in
cases
where
you
know
like
in
our
image
stream
tag
where
the
the
actual
you
know
restore
will
fail,
whereas
you
know
in
another
case,
you
know
if
we
have
a
crd
which
has
an
item
reference,
and
so
you
want
to
make
sure
it's
there.
But
you
know
it
follows
that
kubernetes
model
of
you
know
eventually
ready
it's
fine.
Then
then
you
don't
need
the
weight
because
it
won't
fail
to
create.
D
H
Weight
is
definitely
the
exception,
which
is
again
one
of
the
reason
I
propose
doing
it
as
an
optional
function,
pointer
rather
than
you
know,
something
that
everything
we
need
to
implement
because
of
all
the
plug-ins
that
we
have.
We
only
have
one
example
where
we
need
this,
and
it
sounds
like
the
vmware
you
know,
so
the
in
the
postgres
again
is
a
sim,
similar
case
where
you
know
you
have
one
plug-in
where
you
have
this
issue
and
then
the
rest
of
them.
It's
not
a
concern.
D
Right
yeah,
I
I
think
in
the
cases
where
it's
useful,
it's
super
useful,
but
most
of
the
time
like
you
said,
if
most
things,
hopefully
you
don't
need
it
if
you're
following
kubernetes
patterns
and
can
be
eventually
consistent,
but
the
reality
is
not.
Everything
can
be
that
way
and
I
think
there's
there's
some
stuff
inside
of
vmware
that
could
probably
take
advantage
of
it
as
well
like
we're
talking
about
some
via
like
restoring
vms.
I
think
we
may
be
able
to
use
that
there
so
yeah.
I
I'm
not
against
the
proposal.
D
H
And
I
think
the
the
one
thing
in
the
proposal
that
again
it
still
sounds
like
it's
a
little
bit,
not
clear
is
that
that
timeout
part,
but
even
that's
something.
You
know
if
we
need
to
put
the
timeout
in
using
that
kind
of
static
value
initially
and
then
a
follow-up
change
later.
That's
fine,
too
flexible.
D
I
Hey
so
I
I
just
wanted
to
really
briefly
bring
up,
so
we
have
folks
on
our
team.
I
think
pranav
is
on
the
call
so
for
not
jumping
if
I'm
misrepresenting
things,
but
we
are
interested
in
tackling
this
issue
has
been
around
for
a
while
around
supporting
restore
progress,
and
I
know
that
with
the
recent
valero
releases
we've
tackled.
I
kind
of
do
this
problem
as
multi-layered
right
you,
the
backup
approach.
I
Today,
we
don't
have
a
fully
accurate
representation
of
progress
reporting
because
it's
dynamically
changing
at
backup
time-
and
I
know
the
restore
process-
is
going
to
work
the
same
way,
but
I
wanted
to
get
some
thoughts
and
I
don't
remember
who
actually
contributed
the
backup
progress
reporting,
but
what?
What
are
the
thoughts
on
taking
the
same
approach
with
restore
where
we
we
start
with
a
first
phase
of
saying?
I
I
D
That
no
one
started
on
that
to
me.
That
makes
sense,
and
I
think
that
was
like
the
last
thing
steve
worked
on
before
he
he
moved
over
to
contour
and
the
I
yeah
the
the
idea
with
the
backup,
especially
I
think
the
restore
can
be
a
little
bit
more
accurate
because
well
we
don't.
We
don't
have
a
two
pass
system
with
a
two
pass
system.
We
could
be
100
accurate,
but
we
have
the
full
scope
of
everything
that's
going
to
be
restored.
D
D
I
Well,
actually,
I
just
had
this
idea
kind
of
right
now,
since
we
do
have
that
total
number
at
the
end
of
the
backup.
Could
we
also
potentially
just
write
an
extra
metadata
file
into
object
storage
that
contains
the
total
number
of
items
that
we've
processed
to
backup
time?
I
know
it'll
it
it's
not
going
to
be
fully
accurate
again
because
restore
plugins.
Could
you
know
subtract
or
change
or
add
to
that
number,
but
it
would
be
a
more
accurate
representation
I
think,
than
what
we
would
get
initially
by
just
saying.
D
Well,
the
reason
I'm
thinking
the
number
isn't
the
number
isn't
necessarily
the
total
number
in
the
backup,
because
if
you
do
a
restore
with
dash
dash
include
namespaces,
for
example,
then
it
might
be
just
the
things
inside
those
namespaces
or
based
on
additional
item
actions.
It
might
be
pvs
attached
to
those
namespaces
or
you
know
whatever
plug-ins
might
pull
in.
D
D
I
it
may
require
some
updates
to
how
we
do
restore
just
because,
like
we've,
we've
still,
I
know
steve
and
I
talk
a
lot
about
doing
a
two-phase
restore.
So
we
can
get
this
kind
of
information,
so
we
can
say
what
what?
What
is
everything
that
matches
the
selectors
you
specified
it
like?
Okay,
take
your
include
namespaces,
your
exclude
namespaces,
your
your
label
selectors.
I
D
I
get
one
thing
we
might
be
able
to
do
for
namespaces
without
going
too
deep
into
implementation
for
spaces.
If
we
download
the
tarball
and
look
inside
the
directory
for
the
name
space,
we
could
maybe
do
the
equivalent
of
like
ls
sure
nwc.
You
know
ls
pipe
wc-l,
but
but
that
doesn't
help
with
like
label
selection.
I
Okay,
well,
you
know
you
know
we
were.
I
was
basically
asked
to
so
I
put
off
on
the
call
we
have
another
person,
who's
done
a
couple
of
contributions.
I
think
to
the
plugins
jada
kumar
gabbani
and
I
think
he's
probably
the
one
tackling
this
and
I
was
just
hoping
to
get
someone
that
he
could
probably
bounce
ideas
off,
but
he
was
going
to
try
to
follow
a
similar
pattern
that
was
taken
in
the
back
of
progress
reporting,
but
it
sounded
like
it
wasn't
a
one-to-one
thing
you
can.
G
So
one
more
idea
that
I
wanted
to
bounce
here,
it's
the
same.
That
dylan
was
saying
that
somehow,
if
we
get
the
number
of
items
in
from
the
backup
and
then
so,
we
would
have
this
two
two
status
information.
One
is
the
number
we
got
from
backup
so
that
we
can
display
the
the
resources
from
backup
that
we
have
processed
serially
and
the
other
piece
of
information
is
the
actual
resources
created
against
api
server.
That
can
be
that
can
be
taken
at
runtime.
G
G
D
D
I
think
that
was
added
in
one
three
or
maybe
one
two
that
lists
all
of
the
resources
based
on
type
and
it
lists
every
single
item
and
that
file
might
be
able
to
be
used,
but
it
if
you're,
using
a
restore
with
selectors,
it
becomes
tougher
to
get
an
accurate
number
like
you
can.
You
could
just
always
use
the
total
number
from
the
backup
and
say
this
might
be
less
based
on
selectors.
D
But
valero
processes
selectors
on
a
in
an
individual
item
at
restore
time,
so
it
so
it
basically
goes
through
the
tarball
and
says.
Should
I
restore
this
item
no
moves
on.
Should
I
restore
this
item?
No
move
on
it?
Doesn't
it
doesn't
go
gather
them
all
up
first,
which
is
why
getting
getting
the
count
of
total
things
to
restore
is
tough.
G
Yeah
that
that
makes
sense,
so
so
what
I
am
envisioning
as
progress
reporting
is
kind
of
two-phased
right.
So
right
one
is
this:
this
valero
taking
the
backup,
turbo
and
processing
each
item.
That
is
one
progress
report
that
we
can
generate,
and
the
second
is
after
valero
processes,
each
backup
item,
whether
or
not
it
has
passed
all
the
selectors,
all
the
condition
and
created
resources
against
the
api
server
at
restore
time.
That
is
another
information
we
can
process
and-
and
I
think,
there's
value
in
displaying
both
of
them
right
right.
G
D
D
D
D
Because
it
basically
involves
pulling
the
because
we
don't
have
any
help
from
the
cube
api
server
here,
basically
have
to
pull
the
ammo
out
and
look
at
it
look
at
each
one.
I
I
think
it's
possible
it
just
it
more
or
less
I
mean
we
may
end
up
by
doing
this
may
just
be
building
that
two-phase
run
two-phase
restore
system.
F
So
this
kind
of
two-phase
to
restore
two-phase
restore
approach
was
able
to
enable
scenario
like
that
to
be
executed
in
the
value
itself.
Instead
of
right
now,
I
would
have
to
run
the
evaluation
time
because
restore
two
times
the
first
time
I
restore
some
part
of
available
namespace
I
want,
then
I
restore
the
data.
Then
I
restore
the
entire
website
again.
D
Yeah,
I
I
guess
what
I'm
trying
to
figure
out
is:
if
we
can
accommodate
getting
this
status
without
having
to
do
it,
but
we
may
end
up
doing
the
whole
two-phase
thing
in
order
to
get
the
status,
I
certainly
wouldn't
say
no.
I
I
just
like
if,
if
folks
at
red
hat
are
volunteering
to
go
through
it,
I
would
just
love
to
see
design
for
it
and
and
discuss
it.
Yeah
yeah,
because
I
absolutely
we
we
absolutely
want
this
restore
progress,
reporting.
It's
we.
D
I
C
C
Is
we
open
a
connection
to
the
backup
target
as
part
of
initializing
the
the
plugin,
and
so
with
this?
Because
of
this
issue?
Valero
was
opening
a
connection
to
the
backup
target
every
second
and
it
seems
it's
it's
because
of
the
the
the
bsl
controller.
So
I
looked,
I
took
a
look
at
it
looks
like
it's
a
generic
generic
problem
that
will
affect
pretty
much
all
objects,
object,
store,
plugins
and
even
for
the
azure
one.