►
From YouTube: Kubernetes Data Protection Workgroup Meeting 20210407
Description
Kubernetes Data Protection Workgroup Meeting - 07 April 2021
Meeting Notes/Agenda: -
Find out more about the Data Protection WG here:
https://github.com/kubernetes/community/tree/master/wg-data-protection
Moderator: Xing Yang (VMware)
A
A
Okay,
so
I
see
that
you
have
a
dog
talked
about
motivation.
B
A
A
Yeah
yeah:
that's
why
I'm
thinking,
maybe
that
should
be
in
the
appendix
instead
so
like.
Let's
take
a
look
of
this
right,
so
we
actually
it's
like
at
the
at
the
end,
okay
yeah.
So
this
is
very
very
long.
A
This
is
still
here.
We
still
here
yeah.
So
I'm
thinking
how
about
we
just
keep
this
whole
section
in
the
appendix
otherwise.
This
it's
just
for
each
type
of
database-
and
we
have
a
you-
know-
have
a
lot.
B
Definitely
yeah
does
that
work.
I
think
then
we
still
want
to
have
probably
a
small
section
that
gives
the
overview
yeah.
A
Sure
sure
yeah,
I
think
this
is
still
need
to.
I
think
this
one.
You
still
need
to
clean
up
a
little
bit,
so
is
it
for
each
because
I
think
for
each
type
of
database
we
have
different
people
working
on
it,
so
I
think
the
style
may
be
a
little
different.
At
least
you
need
to
make
them
consistent.
A
A
C
It's
probably
easier,
if
you
drive
than
if
I
try
to
take
over
right
so
so
so
basically
here
broke
it
down
into
first,
and
I
think
this
might
overlap
with
something
as
I
read
later.
But
but
this
notion
of
again
the
the
rules
and
scopes
of
the
the
fact
that
there's
an
application
owner
or,
however,
you
want
to
fight
yeah.
A
C
C
You
can
protect
a
namespace,
you
could
protect
a
cluster
which
is
which
is
kind
of
the
discussion
we
had
had
and
the
vast
majority
focuses
on
application,
because
that
was
also
where
I
think
we
had
the
vast
majority
of
discussion
and
and
focus
when
we
went
through
the
notes,
the
first
time
through
and
so
talked
a
little
bit,
and
this
tom
may
may
then
link
in
with
some
of
what
you're
also
doing
with
things
like
quiesce
the
notion
of
what
defines
an
application
and
yeah
I'm
happy
to
take
flexfall
out.
A
I
think
this
section
may
be
overlapping
with
the.
I
think
there
is
a
section
defines
applications
yeah,
but
we
can
sort
out
that
later.
A
A
C
Exactly
so
so,
we've
got
customers
that
basically
use
rds
databases
in
aws
and
then
and-
and
this
is
where
kind
of
aws
is
pushing
the
whole
ack
kind
of
thing
right.
Where
I
could
provision
manage
and
maybe
someday
even
trigger
backups
of
rds
databases
that
are
not
managed
by
kubernetes
they're.
Not
oh.
C
But
but,
and-
and
I
think
I
I
know-
I
know
we
get
quite
a
bit
of
this-
that
as
much
as
so
this
is
like
in
the
environment,
you
have
mixed
quantities,
yeah
exactly
it's
a
mixed
environment
where
I
might
be
running,
let's
say
a
inside
the
cluster,
but
I
might
also
have
dependency
on
an
oracle
outside
the
cluster
and
they
don't
want
to
run
oracle
in
the
cluster.
C
So
that
remains
external,
but
from
the
point
of
view
of
the
backup
they
they
they
want
that
to
be
coordinated,
because
there
is
a
relationship
and
a
dependency
in
their
mind.
A
C
S3
store
would
be
another
one
yeah,
so
so
so
that
was
so.
That
was,
that
was
kind
of
the
first
part
is
and-
and
this
ties
into
I
think
and
there's
a
bit
later-
that
needs
to
be
filled
in
the
section
right
below
this
one
that
talks
about
application
crds,
because
we
don't
have
an
application
object
inside
kubernetes.
C
Is
you
know
it's
not
as
easy
as
just
hey
snapshot
of
csi
volume?
Now,
when
you
talk
about
databases,
it
can
get
arbitrarily
complicated,
it's
not
just
quiesce,
and
then
you
know
snapshot
a
database.
C
If
I'm
doing,
let's
say
more
of
a
data
transfer
style
database,
backup,
I've
got
to
start
worrying
about
how
I
set
up
policies
for
things
like
log,
truncation
and
incremental
backup
management
and
those
sorts
of
things
again.
Consistency
groups,
volume
groups.
However,
we
want
to
talk
about,
and
those
are
a
big
deal
and
then
the
whole
decision
to
snapshot
versus
backup
and
again.
C
I
know
a
lot
about
us
on
this
caller
going
through
these
same
sorts
of
discussions
with
our
customers
around
needing
to
give
them
these
options
and
then,
like
we
were
talking
about
before
the
external
data
store.
What's
the
mechanism
by
which
they
specify
that
we
can
connect
to
that
to
get
you
know,
do
they
do
they
put
in
authentication
information?
How
do
you
get
there
and
then
the
last
part
is
again
within
an
application
there
may
be,
you
know.
C
Not
only
do
I
have
individual
databases
that
need
to
be
protected,
but
I
may
need
to
specify
the
order
of
protection
of
elements
in
the
in
this
application,
or
I
may
need
to
run
broader
hooks
that
quiesce,
the
entire
application
or
those
sorts
of
things.
So
so
so
we
again
this
is
this
is
again
right
now.
I
think
all
of
us
are
doing
it
in
a
custom
fashion.
It's
a
company
by
company
implementation
decision,
but
these
are
the
pieces.
C
Your
traditional
gold,
silver
bronze
kind
of
thing
they
may
want
to
be
able
to
again
almost
select
or
ask
for
or
request
a
certain
service
so
that
again
they're
not
having
to
talk
to
their
backup
administrators,
and
they
can
simply
say
this.
Application
is
really
important
and
I'd
like
this
to
to
get
the
platinum
level
protection.
C
These
these
become
important
to
customers
who
are
trying
to
manage
both
defining
the
application,
defining
the
recipe
for
how
you
back
up
the
application
and
then
selecting
the
almost
the
the
service
level
of
of
what
their
their
backup
policies
should
be,
so
that
was
that
was
kind
of
the
focus
of
these
first
sections
and,
like
I
said,
I
think
it
ties
into
some
of
what
tom's
writing
with
quest
hooks,
and
you
know
this
is
the.
Why
do
we
need
those
quiesce
hooks?
C
A
C
Yeah
because
because
again
and
until
and
unless
that
happens
like
I
said,
I
think
every
vendor
will
come
up
with
their
own
way
of
specifying
it.
I'm
I'm
sure
yeah.
D
C
Casting
obviously
they
have,
they
have
their
their
mechanism.
We
have
ours,
I'm
sure
trullo
and
every
I'm
sure
all
of
us
are
doing
it
because
there's
there's
not
another
way
to
get
around
it
right
now.
So
if
we
don't
standardize,
then
we'll
just
stay
at
the
wild
wild
west.
C
And
then,
and
then
we
get
into
the
actual
use
cases
proper,
which
are
things
like
I've
got
to
be
able
to
recover
an
application
in
the
event
that
it's
been
either
it's
been
destroyed
or
the
cluster's
been
destroyed,
or
the
name
space,
but
is
destroyed
or
the
region
right
and
then-
and
so
that's
the
first
set
there.
Then
the
second
is
the
rollback
which
is
more
of
the
you
know.
C
Probably
you
know
carl
carl
corrupted
my
application
and
I
it's
not
entirely
gone,
but
I
do
need
to
be
able
to
you
know,
sort
of
pull
it
back
to
a
previous
point
in
time
when
things
were
happy,
so
you
know
different
than
setting
up
a
new
application.
This
is
this
is
just
you
know,
going
back
to
a
previous
point
in
time
and
one
of
the
key
things
that
we
get
a
lot
there
is
giving
me
the
option
to
overwrite
or
not
overwrite.
C
And
so
so
that's
the
and
then
the
migration
case.
This
is
again
I'm
moving
to
a
new
cluster,
a
new
region,
a
new
geography,
a
new
vendor.
You
know
whatever
it's
going
to
be,
and
so
so
here
you
know
generally,
what
you
get
is:
either
a
I'm
going
to
take
down
time
and
do
a
one-time
cut
over
or
the
ultimate
approach
for
a
higher
end
application.
C
Is
I'm
going
to
set
these
up
side
by
side
for
a
while
keep
the
target
offline
or
invisible
until
I
do
a
final
cut
over,
so
that
the
down
time
is
less
but
but
either
way
the
focus?
There
is
the
same
that
I
want
to
move
that
over
they.
There
is
a
good
chance
that
they
may
need
to
modify
the
resources
as
they
as
they
migrate.
So,
for
example,
you
know
if
I'm
migrating
from,
I
don't
know
eks
to
aks
my
storage
class
definitions
may
change.
I
may
change
how
I
define
my
containers.
C
And
then
cloning
is
like
an
easier
version
of
disaster
recovery
in
some
given
that
least
the
the
source
is
still
up,
but
here
you
you,
you've
got
to
be
able
to
obviously
run
concurrently
because
it's
a
clone,
which
means,
obviously
resources
have
to
be
renamed.
So
there's
no
conflict
data
would
have
to
be.
You
know,
copied
or
cloned,
so
that
you're
not
trying
to
mess
with
the
same
data
set
and
and
then
a
request
that
we're
we
get
increasingly
often
is
this.
C
This
whole
golden
copy
push
out
push
out
updates
to
the
golden
copy
kind
of
kind
of
approach.
So
so
think
of
this,
as
I
cloned
off
20
versions
for
let's
say
a
training
exercise,
and
then
a
week
from
now,
I've
updated
the
primary
data
set.
I'd
like
to
not
reseed
everything,
but
just
push
the
updates
to
those
cloned
copies.
C
And
then
that
takes
us
into
the
application
retrieval
so
think
of
this
more
as
there's
a
legal
case.
There's
you
know,
or
I
just
need
to
get
data
back
for
this
is
this
is
more
from
a
longer
term
retention
point
of
view
and
in
particular
here
what
we're
hearing
more
of
from
from
our
customers
again.
Is
it's
not
just
about
getting
the
data
back
now?
C
So
so
there
is
an
interest
in
being
able
to
recreate
the
whole
flow,
and
so
this
opens
up
questions
about
again
backward
compatibility
of
resource
versions
and
how
we
map
those
forward,
and-
and
I
don't
know
if
we're
the
only
ones
hearing
this
or
if
others
hear
it.
But
there's
been
more
interest
now
around
protecting
the
actual
container,
because
the
backup
teams
are
somewhat
skeptical
that
the
container
repository
will
have
the
version
they
need
three
years
from
now,
and
they
don't
want
to
be
beholden
to
that.
C
C
So
the
retrieval
on
the
whole
is
a
major
use
case,
because
that
is
the
longer
term
retention.
I
need
to
pull
stuff
back
the
container
protection
piece.
I'd
say:
that's,
that's
that's
coming
up
as
discussion,
but
not
buying
criteria.
Yet
so
that's
why
I
kind
of
threw
it
in
there
partially
to
see
if
other
people
on
the
call
were
starting
to
hear
this,
or,
like
I
said,
is
this
just
we've
had
a
few
one-offs.
I
always
want
to
be
careful
about
that,
but
you
know
the
three
three
anecdotes
does
not
make
data.
F
Exactly
yeah
yeah
we're
hearing
similar
things
from
time
to
time,
mostly
on
the
use
case
where
the
registry
is
not
available,
and
you
know
you
still
want
to
make
sure
that
your
backup
can
succeed.
You
know
with
some
images
I
mean,
even
if
the
image
is
not
present
at
the
destination
site
or
wherever
you're
restoring
to
so.
We
do
hear
that
from
time
to
time
as
well.
E
Yet
the
the
interesting
thing
is
that
this
is
not
only
application
container
images
but
also
kubernetes
version
right.
The
kubernetes
will
release
the
new
version
like
in
a
quarterly
basis.
After
three
years,
there
might
well
be
backward
incompatible
changes
there.
C
A
So
there
is
actually
there
was
a
presentation
done
by
someone
a
while
back.
I
think
it's
from
the
vlar
team
talking
about
the
the
different
versions
of
a
kubernetes
cluster.
A
You
may
want
to
take
a
look
at
that.
Maybe
I
can
refer
to
that
doc.
E
E
They
need
the
ability
to
go
back
far
in
time
like
five
years
or
something
like
that
to
meet
this
purpose.
But
I
agree
with
this
is
a
very
complex
and
hard
problem
to
sew
no
money,
backups
the
restore
restoration
from
backups.
E
And
yeah
yeah,
probably
you
want
to
add
the
kubernetes
version
over
there
as.
B
C
B
E
F
E
Yeah,
this
is
slightly
different
than
that
because,
because
they
are,
they
are
customers
or
users,
using
this
kind
of
model
to
manage
the
images
and
by
packaging.
The
image
registry
directly
into
the
distribution
of
kubernetes
that'll,
make
this
problem
slightly
easier,
but
not
completely
solve
the
problem
like,
for
example,
kubernetes,
problem,
etc,
etc
and
yeah,
and
for
those
who
are
using
public
registry,
then
that's
the
real
thing.
G
G
C
C
But
but
you
know
there
is
an
increasing,
especially
in
the
financial
industry,
because
there's
so
many
ai
things
and
particularly
hedge
funds,
have
have
come
to
us
and
said
that
actually
head
funds
and
governments
both
that
that
they
need
to
be
able
to
recreate
results
from
their
ai
algorithms,
so
that
they
can
demonstrate
that
you
know
whatever
choices
were
made.
You
know
how
how
those
choices
were
made.
So
so
that
is
an
increasing
thing.
That's
coming
as
well,
so
it's
it's.
C
I
need
the
old
database
to
interpret,
but
I
may
also
need
to
rerun
the
result
and
show
how
we
got
to
where
we
got
to
for
for
a
legal
case.
Be
it
you
know,
because
of
in
government
and
and
police
cases,
that
you
weren't,
you
know
discriminated
when
you
were
just
there,
wasn't
any
discrimination
in
terms
of
the
algorithms
and
in
terms
of
the
hedge
funds
that
you
weren't
cooking
the
books
to
to
you
know
gamestop
people.
C
Okay
and
then,
and
then
the
next
one
resource
recovery.
So
this
is
just
the
I
might
have
backed
up
an
application,
but
I
don't
always
need
to
recover
the
whole
application.
I
might
just
want
a
database
out
of
it.
C
I
might
just
want
one
file
system
out
of
it,
so
so
there
obviously
has
to
be
a
way
to
to
to
to
be
able
to
filter
out
effectively
a
resource
or
set
of
resources
from
a
backup,
so
so
that
people
can
just
restore
the
subset
that
they
need
so
granular
recovery,
always
always
a
big
deal.
C
And
then,
and
then
the
next
two
covered
a
little
bit
more
lightly
only
because
from
a
namespace
perspective-
and
again
I
don't
know
how
other
people
see
this.
But
you
know,
protecting
namespaces
in
in
one
sense
is,
is
a
requirement
we
get,
because
we
have
customers
whose
app
owners
don't
want
to
define
applications
and
they
just
go.
Oh
just
back
up
the
name
space
at
least
you'll
get
everything
so
it's,
but
it
is
a
cruder
way
of
doing
things
because
then
we're
not
getting
the
you
know.
C
These
things
need
to
be
half
consistent
and
here's
the
relationships,
and
so
it's
it's
a
bit
of
a
sort
of
a
heavy
hammer,
just
snapshot
a
bunch
of
stuff
and
close
your
eyes
and
hope
it's
all
going
to
be
okay,
but
it
is.
It
is
a
useful,
I
guess,
is
a
brute
force
kind
of
mechanism
and
we'll
get
people
who
say
back
up
all
the
applications
in
a
namespace
and
then,
if
there's
stuff
left
over
just
stick
that
in
a
bag,
because
maybe
I'll
need
it
later.
C
So
so
I
tried
to
capture
that
sense.
I
am
not
in
love
with
backing
up
namespaces,
but
again
because
we
don't
have
apple,
you
know,
well-defined
application
objects
and
kubernetes.
It's
a
thing
that
comes
up
for
us
again,
I'm
happy
to
morph
this.
If
other
people
are
like
super
passionate
that
namespace
backup's
the
greatest
thing
since
sliced
bread,
but
it's
sort
of
I
sort
of
hold
my
nose
and
do
it.
H
But
I
think
you're
right,
we
see
it
a
lot
too,
but
you
know
namespace
is
the
most
popular
way
of
segregating
kubernetes
applications
at
the
moment,
so
so
people
just
create
name
spaces
when
they
have
to
create
a
new
application.
I've
seen
cube
admins
tell
I've
heard
cube.
Admins.
Tell
me
that
whenever
a
new
application
thing
comes
to
me
and
asks
for
you
know
like
space
in
the
cluster,
I
create
a
namespace
and
give
it
to
them,
so
they
either
run
more
than
one
application
or
or
they
carve
out
namespaces
per
application.
H
F
See
look
at
the
cncf
data
as
well.
Exactly
it's
like
55
to
60,
using
name
spaces
to
segregate
my
take
is
that
you
know
starting
off.
You
know.
Customers
and
prospects
are
using
namespaces
to
segregate,
but
you
know,
as
the
maturity
builds.
I
wouldn't
be
surprised
if
the
granular
scope,
you
know,
goes
on
to
more
of
the
sub
items
within
the
namespaces.
H
I
think
we
have
to
do
both.
I
mean
we
see
both
right
and
I
have
also
seen
namespace
as
a
service
being
rolled
out
by
customers.
So
so,
when
you
you
get
a
namespace
and
that's
where
you
play
you.
You
are
totally
isolated
from
the
physical
clusters
and
if
you
go
and
look
at
apple's,
you
know
apple
did,
I
think,
keynote
speech
in
the
last
kubecon.
They
explicitly
mentioned,
that's
how
either
apple
or
adobe.
I
forget,
that's
how
they
manage
it.
I
I
C
So
yeah,
so
so
I
I
think
we
all
see
the
same
things
I'm
happy
to
flesh
this
out.
I
didn't
want
to
basically
just
cut
and
paste
all
the
things
from
above,
though,
given
that
the
document
is
pretty
heavy.
So
that's
why
I
kind
of
did
the
you
know.
Sort
of
this
is
another
level
and
the
the
flows.
The
flows
on
recovery
are
basically
the
same.
C
And
then,
and
then
on
cluster-
and
this
was
the
one
that
I
think
when
we
when
we
had
talked
before
was
as
much
about
what
we
didn't
want
to
cover
is
what
we
did
and
so
in
particular,
I
I
think
from
the
notes
I
had
had.
You
know
this
group
was
not
in
the
you
know.
We
recreate
clusters
right
and
we
also
said
look.
You
know,
there's
value
in
potentially
vendors
backing
up
and
restoring
etcd,
but
that
was
outside
the
scope
of
sort
of
what
this
group
was
interested
in.
C
So
so
I
sort
of
explicitly
called
those
out
as
potentially
useful
and
interesting,
but
not
not
in
our
scope
and
then
also
sort
of
called
out
that
you
know
again.
We
certainly
have
some
user
interface
feedback
that
says.
Couldn't
I
just
select
a
cluster
and
selecting
cluster
just
means
either
selecting
all
name,
spaces
or
selecting
all
all
all
applications
which
is
again
to
me
more
a
vendor
specific
ui
hack.
C
I
think
when
we
talked
about
cluster
protection,
what
we
were
really
looking
at
was
the
ability
to
protect
cluster-scoped
objects
that
are
not
tied
either
to
a
namespace
or
an
application.
So
crds
and
I
think
unattached
pvs
were
the
two
that
came
up
and
some
customers,
I
think,
want
a
mechanism
to
do
that
and
if
I
recall
I
thought
it
was
the
valero
team
that
actually
said
they
had
facilities
for
doing
that
and
then
and
and
then
on.
Recovery
are
fairly.
C
You
know
restrictive
or
had
checks
in
place
to
make
sure
that
customers,
don't
you
know,
sort
of
shoot
themselves
in
the
foot.
But
I
I
I'll
be
honest
that
one
I
didn't
fill
in
as
well,
because
I
didn't
remember
exactly
all
my
notes
had
more:
they
don't
cover
this
stuff.
C
I
I
was
not
sure
in
terms
of
the
the
recovery
piece,
and
so
I
was
hoping
we
could
have
a
brief
discussion
on
that
and,
like
I
said,
I
thought
it
had
been
the
valero
team
that
had
sort
of
taken
the
the
first
steps
there,
but.
E
Yes,
steve,
I
yeah,
I
kind
of
fully
agree
with
you
on
that.
I
think
you
know
in
general,
restoration
is
not
it's
a
much
riskier
operation
compared
to
backup
itself
right
worst
case
scenario
for
backup
is
your
application
might
be
stuck
over
there
because
of
the
execution
hook
requires
you
or
you
need
some
manual
information,
but
for
restoration
it
can
literally
ruin
your
original
application
right.
If
you
don't
do
this
carefully,
but
so
putting
restrictions
on
restoration
piece
is
probably
most
many.
E
Many
backup
vendors
will
follow
right,
including
the
permissioning,
including
the
careful
selection
of
resources,
etc,
etc,
or
implicitly
restore
some
of
this
stuff
into
the
kubernetes
cluster.
For
example,
if
you
choose
to
restore
rcr
the
cr,
what
about
the
crv
right,
the
the
the
news
might
not
necessarily
need
to
select
the
crds
et
cetera,
but
put
putting
all
this
aside.
E
I
I
think
we
should
add
a
little
bit
cover
to
this
piece
as
well
is,
but
to
talk
about
some
of
the
resources
that
our
group
will
not
be
covering
for
restoration
or
we
don't
recommend
for
covering,
for
example,
cube
system
right.
This
is
critical,
because
some
of
those
thoughts
are
simply
it's
not
simple:
it
could
cut
or
apply
or
delete
and
apply.
It
really
needs
it.
Really.
E
A
Maybe
I
think
for
crds
maybe
also
talk
about
what
is
the
problem
of
restoring
cr,
because
it
could
trigger
something
to
be
created.
A
You
know
which
could
cause
some
conflict
right,
so
maybe
just
briefly
explain
that.
So
that's
why
we
really
have
to
handle
each
crd
but
yeah.
This
is
different.
It's
it's
not
like!
We.
We
can
just
have
this
general
solution
for
all
yeah.
E
The
idea
there
are
there
are
many
implications
over
there
right.
I've
seen
people
delete
the
crd
first
and
then
recreate
it,
and
that
causes
a
huge
issue,
because
crds
are
non-namespaced
right.
You
have
no
idea
whether
there
are
other
namespaces
using
these
crs
of
this
crd
deletion
of
the
crd
will
be
a
trigger
deletion
for
all
the
crs
right,
and
this
is
problematic
in
many
cases.
So
I
fully
agree
with
you.
Restoration
should
be
much
more
restricted
than
backup.
B
Yeah
sean
that
seems
pretty
bad
if
you're
deleting
the
crds
on
restore
we've,
actually
also
seen
issues
with
crd
versions
for
people
who
put
the
you
know,
some
people
will
update
a
lot
of
the
open
api
schema
in
crds,
and
so
do
you
update
it?
Does
that
invalidate
existing
ccrs?
It's
pretty
it's
pretty
problematic.
Actually,
that's
right!.
B
You
know
we
already
talked
about
the
kubernetes
version
problem,
this
problem's
even
worse,
because
people
can
arbitrarily
update
their
crds
with
or
without
bumping
diversion.
J
So
when
we
say
unexpected
behavior,
I
guess
the
question
I
would
have
is:
wouldn't
this
be
incumbent
on
the
given
operator,
slash
controller
to
do
the
right
thing
with
either
a
out
of
date,
resource
or
you
know,
be
able
to
convert
using
conversion
web
hooks
or
something
along
those
lines.
E
Okay
yeah,
I
guess
this
is
individual,
backup
vendors
choice
right.
It's
hard
for
this
group
to
come
up
with
an
opinion
in
its
way
of
how
should
backup
be
done
per
se.
E
In
some
cases,
I
know
where
I
did
backup
crds
right.
Those
are
non-namespace
resources,
you
many
because
in
many
cases,
even
though
you
back,
if
you
only
backup
namespace
resources
and
at
restoration
time,
it
happens
to
be
a
brand
new
cluster
with
no
crds
definition.
There's
no.
The
restoration
cannot
happen
because
you
cannot
refer.
J
I
100
agree.
I
guess
the
question
that
I
was
asking
is:
isn't
that
a
new
use
case
then
about
backing
up
like
cluster
configuration,
and
so
you
can
restore
your
secondary
cluster
to
the
same
state
as
your
primary
cluster
when
it
goes
down
right.
Is
that
a
second
use
case
that
we
haven't
discussed.
J
E
C
C
Yeah,
I
think
I
think
you
were
you
were
asking
about
cluster
config
right,
so
so
yeah,
so
so
I'll
confess
in
my
head.
I
generally
thought
about
it.
Is
customers
set
up
their
cluster
and
then
we
restore
applications
into
it
and
if
something's
missing,
like
a
storage
class
definition
or
something
we
either
a
give
them
a
default
b,
give
them
a
way
to
to
map
or
transform
or
see.
We
fail
and
say
you
know
this.
These
these
things
are
missing.
It
is
an
interesting
question,
though
you
know.
C
J
100
agree
and
that's
the
case
that
I
was
going
to
make
of
why
we
should
not
include
crds
or
we
should
advocate
for
not
including
crds,
because
it
should
be
incumbent
on
the
cluster
setup
to
get
your
crds
and
your
cluster
in
the
right
state
to
be
able
to
restore.
That
was
going
to
be
my
my
argument
for
why
we
should.
We
should
potentially
advocate
to
not
attempting
to
solve
the
crds
problems.
E
E
A
E
You
know
realizing
that
the
crd
is
just
one
of
the
example
right.
There
are
the
examples
as
well,
for
example,
cluster
row
binding
and
this
kind
of
thing
you
we
can
claim
that
those
are
cluster
creation.
Time
set
up,
or
we
can
say
this
is
actually
a
letter
on
when
a
namespaces
is
created
called.
Then
they,
the
corresponding
application,
is
deployed.
Then
the
cluster
row
binding
is
created
to
grant
the
permissions
right.
In
this
case,
do
you
want
to
include
those?
E
Those
are
non-name
specific
resources
as
well
right
so
again
having
them
in
the
backup
probably
is
not
going
to
hurt
that
much,
but
during
the
restoration
time
it
needs
to
be
a
careful
decision
either
made
by
users
or
backup
vendors,
so
yeah,
I
fully
agree.
So
we
should
not
get
too
much
into
this
business.
A
B
So,
for
example,
if
I'm
writing
an
operator,
should
the
operator
control
itself
need
to
deploy
the
crd,
or
should
it
be
done
out
of
band,
because
I
think
if
an
operator
itself
is
deploying
the
crdm
managing
it
through
through
the
controller,
then
you
know
the
restoration.
Behavior
will
be
very
different
than
if
you're
doing
something
out
of
band,
and
if
we
have
a,
if
there's
one
way,
that's
easier
for
us
to
do
backups
we
can.
A
Coordinating
right,
yeah,
so
the
backup
software
and
the
control
arduino
controller,
they
have
to
coordinate
with
each
other.
H
Here's
the
here's.
The
thing,
though
I
understand
where
you
guys
are
coming
from,
but
the
operator
paradigm
is
so
popular
for
deploying
applications.
Customers
expect
that
you
know
a
backup
vendor
to
be
able
to
discover
apps
that
are,
you
know,
deployed
using
operators
and
do
the
need
for
which
is,
you
know,
backing
up
crs.
You
know
crds
and
the
loops
etc
to
make
sure
that
when
they
restore
their
app,
actually
comes
up
and
and
can
function
instantly
right.
So
it's
something
you
know
people
deal
with.
H
D
D
An
app
that
depends
on
a
particular
crd.
It
presumably
depends
on
the
whole
other,
app
or
operator
or
whatever
it
is
that
created
that
crd,
and
so
you,
you
kind
of
need
to
back
up
the
application
and
its
whole
dependency
chain,
which
might
not
be
easy
to
find.
Just
by
looking
at
the
kubernetes
cluster,
you
kind
of
have
to
have
an
understanding
of
what
is
it
depending
on,
so
you
can
make
sure
that
the
right
dependencies
are
there.
On
the
other
side.
F
E
A
D
But
I
think
you
could
I
mean
there's
certain
places
you
have
to
stop
that
recursion
to
say.
Okay,
like
this
thing,
I'm
just
going
to
assume
is
there
on
the
other
side,
because
there's
no
way
I
can,
you
know
auto
reliably
back
it
up
in
a
way
that
I
can
restore
it
onto
an
arbitrary
cluster.
So
at
some
point
you
have
to
say:
okay,
you
know
snapshot
the
snapshot.
Controller
is
part
of
the
cluster.
D
J
To
this
problem
is
aggregated
api
servers
yeah
like
are
we
also
going
to
be
backing
up
then
aggregated
api
servers
and
following
those
and
and
doing
that,
those
have
the
exact
same
edges
that
crds
have.
J
A
Yeah
yeah,
I
think
we
can
we
can.
My
thought
is,
I
think
we
can
come
up
with
a
like
a
small
example
just
to
explain
what
is
what
are
the
problems,
but
we're
not
saying
that
we
are
giving
out
a
complete
solution
just
to
like
point
out
what
other
issues
we
need
to
pay
attention
to
when
we're
doing
this,
but
still
you
know
discussing,
because
because
I
think
I
would
assume
a
lot
of
backup
vendors
have
to
deal
with
this
anyway.
E
Yes,
yeah,
I
I'm
a
little
hesitant
to
provide
guidance.
That's
what
I
was
thinking,
because,
because
those
are
really,
the
complexities
lies
in
the
backup
systems
right.
So
the
people
do
this
in
a
different
way
and
people.
We
treat
this
pro
solving
this
problem
in
a
different
way
as
well,
so
us
as
a
community.
C
Mean
I
I
I
think,
there's
a
degree
of
for
sure,
but
I
I
mean
I
do
like
I
do
like
you
know.
I
think
the
idea
of
look
this
would
be
a
best
practice
now
as
a
vendor.
I
can't
assume
people
will
always
follow
best
practices,
but
what
I
can
do
is,
I
can
say
if
you
follow
the
best
practices,
you'll
get
the
best
behavior.
If
you
don't,
then
you
know
here's
the
messiness
that
you
go
through,
so
I
don't
I
mean
I'd,
be
willing.
I
mean
if
there's
a
couple
other
people
interested.
C
A
Yeah,
I
think
we
can
talk
about
this.
We
don't
have
to
make
decisions
saying
whether
we
want
to
include
that
in
the
white
people
or
not,
but
I
think
I
think
that's
a
good,
very
interesting
topic
for
this
working
group
at
least
right.
So
maybe
we
don't,
even
if
we
don't
all
agree
on
it.
We
don't
even
include
that
in
white
paper,
but
it's
we're
discussing
that
anyway
in
the
swelling
group.
How
about
that.
C
J
A
Okay,
I
think
I
think
we're
done
with
the
use
case
doc,
thanks
steve,
oh.
A
That's
actually
very.
H
Yeah,
it's
very
good
actually
was
this.
Is
this
draft
like
generally
available
sure.
A
Okay,
let
me
actually
include
this,
but
this
one
is
still
a
little
messy,
so
that's
why
I
didn't
put
it
in
there.
I
was
wondering
if
I
should
like
include
like
small
links,
because
this
is
this
big
dog
is
a
little
messy.
How
about
you
I
do.
That
is
that
maybe
that
will
be
better.
I
was
just
thinking
this
dog
is
kind
of
very
long
right.
What
do
you
guys
think,
or
should
I
just
yeah.
H
I
think
that
it
has
good
information.
It
could
be
long,
but.
E
Yes,
we're
constantly
change
the
content
of
this
document,
so
with
those
sections,
maybe
we
can
market
to
do
there
if,
if,
if
you
guys
are
okay
with
that,
I
you
know
sharing
with
that.
There's
no
problem
at
all.
H
A
Yeah,
so
I
add
a
link
here:
this
is
a
work
in
progress.
Talk,
okay,
let's
see,
we
talked
about
that
use
cases,
and
then
we
have
a
few
minutes
phone.
Do
you
want
to
go
over
different
snapshots,
doc
or
you
think
you're
ready
or
you
need
more
time.
G
G
Yeah,
so
basically
we
we
come
up
with
the
a
small
section
at
the
section
of
the
white
space
on
the
white
paper,
I
mean
for
the
different
sort
of
snapshots.
G
We,
I
still
prefer
to
use
the
word
differential
map
shots
because
there's
different
certain
snapshots,
including
most
the
let's
use
it
for
the
block
and
the
files
system
volumes
right
for
if
it
was
usability
that
it's
just
for
the
block,
and
so
we
want
to
say
you
should
work
like
different
snapshots
and
in
the
paper
we
also
specify
what
is
the
difference
between
the
reference
of
snapshot
and
incremental
snapshot.
There
I
bought
some.
I
will
post
the
link
to
the
paper
here
because
we're
still
editing
it.
G
I
just
received
some
feedback
from
from
zingy
and
app
yesterday,
and
I'm
working
on,
you
know,
incorporate
the
feedback,
so
we,
but
hopefully
that
in
the
next
few
days
we
can
finalize
this
this
segment
of
the
differential
snapshot
and
integrate
it
into
the
white
white
table
of
the
data
protection.
A
Okay,
thank
you.
Okay.
We
have
four
minutes
all
right,
so
we
have
this
so
chanting
and
I
have
put
together
this
diagram
just
to
show
like
what
other
missing
building
blocks
that
we
identified
right.
So
so
we
want
to
get
you
guys,
opinion
and
see.
Does
it
make
sense
to
add
this
diagram
in
the
in
the
dock
or
not
so?
But
this
has
a
different
color
coding,
green,
meaning
existing
kubernetes
components.
So
we
already
have
one
in
snapshot.
A
We
have
workload
apis,
there's
some
applications
crd
there
that
those
are
existing,
oh
and
then
the
blue
one
shows
the
process
for
how
you
do
backup
as
a
process
and
yellow
meaning
those
are
working
progress
component.
So
we
have
a
container
notifier
there.
We
have
cpt
there
and
orange
meaning
missing
building
blocks,
so
we
have
backup
and
then
we
have
a
one
for
restore.
A
B
My
high
level
feedback
is
that
I
think
it
it
doesn't
include
the
relationship
between
backup
vendors
as
well.
You
know,
if
you
say,
something's
missing,
that
that
implies
that
it's
missing,
that
we
want
to
build
that
within
kubernetes,
but
it
may
be.
B
That
we,
we
don't
and
then
it
it's
kind
of
in
the
domain
of
the
back
of
vendors.
A
K
A
Like
content
notifier,
I
know
back
vendors
already
doing
the
sequoias
yourselves.
A
So
yeah
cpt,
you
know
some
black
men
will
do
that
as
well
right.
So
some
of
those
I
may
not
you
know-
maybe
some
people
do
this,
not
that
everybody
do
this.
Like
a
consistency,
group
cozy.
This
is
basically
like
a
backup
repository
type
of
thing,
so
I
think
every
backup
vendor
will
have
to
do
some
something
for
the
backup
repository,
not
necessarily
cozy,
but
so
yeah.
We.
A
E
Instead
of
going
deep
into
the
heavy
text
in
detail
just
putting
some
diagram,
there
helps
understanding
the
workflow
we'll
be
more
than
happy
to
add
them.
If
you
feel
if
anyone
feels
there's
something
missing
there
in
this
picture,.
B
Yeah,
I
think
if
you
tweet
a
little
bit,
you
know
you
know,
for
example,
saying
components
to
me
kind.
B
That
these
are
these
are
things
that
ship
with
kubernetes,
but
actually,
if
you
think
about
a
lot
of
it
is
kind
of
interfaces,
you
know
like
the
even
container
notifier,
for
example.
That's
it's
really
kind
of
an
interface
and
a
you
know
more
of
a
framework
component.
E
A
B
A
A
Okay,
so
maybe
think
about
that-
I
think
we
are
running
all
the
time
think
about
this.
We
can
go
back
to
this
next
time.
If
you
have
any
suggestions
on
how
to
improve
this
diagram
or
if
you
think
it
doesn't
make
sense
to
keep
this
in
the
white
people.
That's
fine,
too,
just
want
to
get
some
suggestions.