►
From YouTube: Kubernetes Data Protection WG Bi-Weekly Meeting 20210616
Description
Kubernetes Data Protection WG Bi-Weekly Meeting - 16 June 2021
Meeting Notes/Agenda: -
Find out more about the Data Protection WG here: https://github.com/kubernetes/community/tree/master/wg-data-protection
Moderator: Xing Yang (VMware)
A
A
A
So
so
tom,
I
think
I
made
you
a
co-host.
Let
me
actually
stop
sharing,
so
you
can
share.
B
So
I
can
ever
see
my
screen.
Okay,
yes
great,
so
we
got
a
few
questions
on
how
to
do
data
protection
for
managed
services,
and
so
I
put
together
a
few
slides
just
to
kind
of
spark
a
discussion
here.
B
So
I
first
wanted
to
kind
of
compare
to
how
you
can
do
volume
backups-
and
I
think
I
think
ben's
talked
about
this
a
little
bit,
but
it's
possible
even
with
just
csi,
to
take
backups
of
volumes,
because
you
can
do
a
csi
snap
and
a
csi
restore
and
extract
data
from
that
restored
volume.
So
this
works
even
in
cases
where
you,
your
snapshots
on
your
storage
system
are
not
that
persistent
and
you
can't
rely
on
them
for
a
backup.
B
B
And
finally,
the
last
approach
that
I
think
is
kind
of
a
good
default
is
to
consider
snapshots
durable.
So
a
lot
of
cloud
providers
you
can
imagine
aws
edf,
for
example,
a
storage
snapshot
of
a
block
snapshot
is
actually
pretty
durable.
It's
stored
in
s3,
and
if
you
want
to
change
your
failure
domains,
you
can
even
copy
that
snapshot
between
different
providers.
B
This
make
sense
because
I
think
I'm
going
to
make
a
parallel
of
this
to
to
data
services
as
well.
B
B
Up
getting
exported
to
s3
automatically
under
the
covers,
and
you
maybe
don't
go
through
csi
or
any
type
of
you
know,
kubernetes
api
to
take
those
snapshots,
and
if
you
do
that,
you
can
actually
control
the
the
snapshot
completely
through
the
apis.
I.
E
A
C
That's
something
we've
prototyped,
where
you
actually
use
the
kubernetes
snapshot
api,
but
you
establish
variables
in
snapshot
class.
That
say
this
snapshot
should
should
go
to
a
safe
place
such
that
it
is,
could
be
used
as
a
backup
and
not
just
a
local
copy
right
and
then
and
then
you
can
leverage
all
the
existing
api
stuff
and
get
something
that
resembles
a
backup.
Just
you
don't
have
a
lot
of
leverage
to
control
it
other
than
the
snapshot
class.
B
That's
that's
a
great
point.
I
think
you're
absolutely
right.
The
implementation
that
we
use
here
does
use
if
you,
if
you
go
provider
specific,
we
kind
of
do
provider
for
everything.
A
A
That's
not
yeah,
but
I
mean
yeah.
It's
actually
not
necessary.
I'm
just
saying
that
you
can
actually
go
through
csi
route
and
achieve
the
same
result
yeah
for
the
durable.
If
they
support
that.
Yes,
that.
E
A
G
Side,
what
tom's
trying
to
point
out
here
is
that
there's
there's
basically
a
fundamental
disconnect
between
on-prem
vendors
and
the
cloud
and
the
cloud
vendors
went
ahead
and
implemented
durable
snapshots
under
the
covers
behind
their
snapshot.
Apis
so
like,
for
example,
in
valeria
valero
came
out
of
the
cloud.
Originally,
it
expects
that
these
snapshots
are
durable
and
ching,
and
I
went
through
all
kinds
of
backflips
in
order
to
make
this
work
on
vsphere.
G
C
But
but
I
feel
like
there's
there's
almost
a
fourth
case:
that's
a
blend
between
one
and
three
here,
that
is,
that
is
significantly
different
enough
from
three
that
it
is
worth
spelling
out,
which
is,
it
is
a
csi
snapshot.
It
is
a
csi
restore,
but
instead
of
extracting
the
data
from
the
volume,
the
snapshot
itself
is
just
durable.
A
What
I
thought
number
three,
not
not
what
that
mean:
okay,
yeah,
maybe
you
need
to
add,
like
a
yeah,
maybe
three
a
or
b
or
something
or
even
four
yeah.
So
maybe
the
you
still
keep
your
original
provider
snapshot.
That
does
not
involve
csi
and
then
and
then
the
second
case
is
actually
basically
c.
Is
a
driver
just
to
handles
everything
or
when
you
take
a
c
the
snapshot
it
automatically
does
the
app
upload.
A
E
H
A
C
A
Or
durable
snapshot
well,
but
I
think
people
most
people
on
this
corporate
know
this
anyway,
because
if
you
know
the
css
back,
we
all
know
that
that
actually
supports
two
type
of
snapshot.
C
E
I
So
also,
this
is
also
a
use
case
thing
like
you,
take
regular
snapshots
which
are
stored
in
the
storage
system
and
you
use
it
if
you
have
to
revert
or
you
have
to.
Basically
you
know
you
corrupted
your
database
somewhere
and
revert
and
and
a
snapshot
that
gets
copied
out
to
object,
storage
somewhere,
which
is
how
the
cloud
providers
do
that
you
use
when
you
have
to
restore
from
failure,
because
your
store
local
storage
system
is
done,
you
know
so
so
they
they
have
two
different
sets
of
use
cases.
A
I
think
I
think
yeah,
I
think
tom,
I
think,
yeah.
I
I
think
you'll
not
see
that
yeah
the
configuration
problem
people
are
having,
so
I
think
most
people
knows
the
differences,
but
then,
when
you're,
comparing
this,
this
looks
like
the
like
the
cloud
provider
snapshots
are
better,
that's
not
what
we
want
to
say
and
that's
definitely.
A
I
don't
think
that
is
the
intention
of
the
csi
spec
when
we
introduce
it
when
we
added
that
list
snapshots
to
make
people
you
know
so
make
sure
that
when
you're
taking
a
snapshot,
that's
actually
get
uploaded.
You
know
you
have
to
wait
for
that
to
be
uploaded,
but
I
mean
maybe
so
the
approach
should
not
be
approach-wise.
Csi
right,
actually,
csi
can
be
either
a
local
snapshot
or
a
remote
snapshot.
So
this
is
weird
when
we
are
named
that,
like
this.
B
Yeah,
I
think
that's
a
good
point,
so
yeah
there's
tons
of
points
here.
It's.
A
Not
it's
not
different
approach
like
you're,
basically
talking
about
different
sort
of
systems.
Now
it's
not
like
you're,
comparing
different
approach.
Right
approach
is
always
you
you
take
a
either
you
take
a
kubernetes
volume,
snapshot,
api
or
you're
using
vendors.
You
know
whatever
you
say
you
call
it
provider
api
right
so
like
under
c
is
that
okay,
maybe
I
think,
approach
one
css
snapshot.
You
should
have
like
two
one
is
in
case
you're
doing
low
code.
A
Then
you
you
have
you
know,
then
you
actually
have
three
three
ways
or
something
like
that
right
either.
If
it's
local,
then
you
have
to
do
this
two
step
or
if
it's
local,
you
can
also
yeah.
You
have
to
do
that
two
step,
but
also
you
can
actually
do
this.
One
inside
your
css
drive.
You
can
do
upload
or
something
like
that.
G
G
F
A
Actually,
not
not
the
yeah.
This
is
actually
not
the
goal
of
this,
because
I
I
would
assume
most
who
knows
about
all
of
this
already,
maybe
not
everybody,
but
most
people
should
know
the
difference
already.
So
maybe
tom,
maybe
you
can
update
this
later.
We
can
actually
add
comments
to
your
slide
deck
afterwards.
G
Yeah
it
sounds
like
this
is
a
good
topic
for
a
future
meeting
is
snapshots
durability
and
how
do
we
start
referring
to
them
because
we've
been
calling
them
durable?
I
don't
know
where
that
started
from
I.
I
know
we
started
inside
vmware
and
if
there's
alternate
terms,
we
can
certainly
discuss
those.
A
B
E
A
A
Move
forward
with
the
you
know
the
core
of
what
you're
trying
to
explain.
I
mean
rather
than
you
know,
comparing
yeah.
I
know
I
get
it.
I
think
it's
almost
like
we're
talking
about
who's,
the
snapshot
and
kind
of
is
better.
That's
not
what
we
want
to
focus
on.
B
Yeah-
and
I
I
think
for
nomenclature
when
I
say
backups,
I
don't
I'm
talking
about
what
you
do
with
local
snapshots
and
not
local
snapshots
only,
although
there
are
obviously
very
valid
use
cases
for
local
snapshots
only
as
well
as
people
pointed
out,
but
yeah
this.
I
love
this
conversation.
I
think
everyone
had
really
some
really
good
points,
but
maybe
I'll
move
on.
We
can
beat
up
my
next
slides
as
well
and
we're
deaf.
J
Tom,
I
have
a
quick
question
yeah.
This
is
not
that
we
got
into
mckinney
then,
but
under
what
cases
you
would
rather
call
provider
provided
snapshot
interfaces
instead
of
using
csi
snapshot.
Is
that
because
the
css
snapshot
do
not
have
sufficient
flexibility
for
you
to
be
able
to
create
remote
snapshots
in
cloud
systems.
B
Yes,
so
the
most
common
requirement
in
I
know,
google
personal
disk
actually
doesn't
have
this
problem
because
most
of
their
disks
are
not
region.
They're
global,
but
in
aws
the
the
snapshots
are
actually
tied
to
a
region
because
they're
they're,
an
s3
which
is
an
s3
bucket
in
a
region,
although
I
think
they
might
have
released
some
global
snapshots
as
well.
But
so
a
common
use
case
for
us
is
actually
taking
a
csi,
taking
a
snapshot
of
an
ebs
volume
and
then
copying
that
to
a
different
region
for
extra
durability.
J
B
E
E
G
Right,
we
started
a
lot
of
this
stuff
before
the
csi
snapshots
are
available,
so
we
have
cloud
provider
specific
plugins
that
we're
looking
to
move
towards
csi
and
then
like.
We
have
the
on-prem
cases
where
we
need
to
be
able
to
take
a
local
snapshot,
but
then
we
need
to
be
able
to
move
the
data
some
place
else
and
that's
not
really
available
via
csi.
Yet
so
I've
been
kind
of
dragging
my
feed.
A
A
A
A
A
All
right,
let's
yeah,
we'll,
do
the
next
slide.
B
No,
I
definitely
love
that
because
I
think
we're
in
the
same
boat
as
as
valero,
where
we
have
some
legacy
implementations
that
maybe
could
be
updated
to
use
csi,
but
I'm
not
sure
all
providers
support
what
we
need.
So
we
have
to
kind
of
dig
it
dig
to
the
second
level
of
detail.
There.
A
B
So
for
data
services,
you
have
actually
this
pretty
similar
problem,
and
so
I
want
to
frame
kind
of
the
primitives
that
that
we
have
obviously
the
ability
to
take
snapshots
through
apis
in
the
provider
which
you
know
could
be
similar
to
the
provider.
Snapshots
that
I
talked
about
for
volumes
and
these
apis
are
kind
of
across
all
the
different
cloud
providers
it
may
you
may
or
may
not
have
them
if
you
have,
if
you
have
kind
of
managed
services
on-prem.
B
So
if
you're
using,
if
you
have
kind
of
a
team
running
these
things,
you
may
or
may
not
have
snapshots,
and
they
may,
they
may
have
different
levels
of
consistency
as
well,
but
something
that
most
of
these
data
servers
have
in
common
is
logical
snapshots.
So
you
can
always
take
a
logical
dump
of
a
database
and
the
model
that
we
we
use,
which
isn't
isn't
always.
The
only
model
is
that
you
have
a
reference
to
the
data
service
inside
your
cluster.
B
So
when
I
go
back
up,
for
example,
in
namespace,
maybe
I
have
config
maps
and
secrets
that
tell
me
how
to
use
that
that
data
service
and
how
to
connect
to
it-
and
this
is
actually
a
pattern
you
see
not
just
for
for
data
protection,
but
you
see
this
even
for
normal
application
development.
You
can
imagine.
I
want
to
consume
just
a
database
and
at
all,
and
the
way
I
can
do
that
is
by
writing.
B
B
Csi
right
yeah,
so
something
outside
the
cluster.
You
know-
and
this
is
pretty
common
in
cloud
providers
where
you
have
things
like
managed
databases,
but
we're
actually
seeing
this
in
on-prem
customers
right.
So
if.
E
B
F
F
No
no,
this
is
this,
is
this
is
sort
of
their
their
attempt
to
try
to
just
connect
in
things
like
rds
into
into
the
cluster,
so
that
theoretically
inside
the
cluster,
you
could
allocate
external
resources
and
whatnot.
It's
still
super
early,
but
they've
been
talking
about
it
for
the
last
couple
of
months.
B
F
And
I'm
not
sure
they've
gotten
apis,
yet
that
allow
the
backup
of
the
snapshot
stuff
to
happen.
They've
they've
been
trying
to
do
more
provisioning
and
whatnot,
but
it's
an
interesting
path
but
yeah,
I
think
we're
the
only
at
least
the
way
they
act,
we're
the
only
ones
that
have
been
talking
to
them
about.
Why
isn't
backup
in
there?
So
I
was,
I
was
maybe
hoping
to
get
some
other
people
to
just
sort
of
make
that
feature
request
see
if
we
can
get
them
to
expand
it.
B
Yeah,
that's
great.
Let
me
I'll
I'll
take
a
note
to
reach
often
because
that
is
very
interesting.
Certainly.
K
B
And
so,
combining
these
two
things
you
can
actually
kind
of
get
a
similar
set
of
approaches
that
we
talked
about
earlier.
Maybe
it's
only
three
instead
of
four
or
maybe
I'm
missing
a
few
here
as
well,
but
for
any
managed
database,
you
can
imagine
just
taking
a
logical
dump,
so
you
can
always
with
connection
information,
go
and
do
essentially
a
mysql
damper,
a
postgres
dump,
dump,
etc.
B
B
It
can
have
a
really
big
impact
on
performance,
and
so
something
that
we've
gotten
several
requests
for
is
actually
to
integrate
both
logical
dumps
and
provider
snapshots.
Because
many
of
these
you
know
rds,
for
example,
has
direct
snapshots.
You
can
take
and
combine
the
two
approaches,
so
you
can
imagine
doing
something
similar
like
that
that
I
talked
about
doing
for
volumes,
which
is
you
take
provider
snapshot
in
this
case,
it'll
be
an
rds
snapshot.
B
You
restore
that
that
database
and
you
have
essentially
a
temporary
copyright,
a
copy
of
that,
and
then
you
can
do
a
logical
dump
from
that
connection.
So
you
can
not
only
can
the
benefit
of
making
it
logical.
Is
that
not
only
can
you
store
it
in
rds
or
the
same
service,
but
it
actually,
it
should
be
kind
of
portable.
B
You
should
be
able
to
instantiate
a
new
instance
of
mysql
postgres
in
your
cluster
or
in
a
different
provider
or
a
different
cluster
and
restore
that
module
dump
into
that
database
and
again
just
like
with
volumes
you
can.
Actually.
I
should
probably
change
this.
Maybe
like
this
cloud.
You
can
do
something
similar
with
volumes
where
you
can
take
just
snapshots
and
then
use
apis
inside
the
provider
to
manage
it,
so
you
can
copy
it
between
different
regions.
You
can
you
know
kind
of
change.
B
The
failure
domains
with
that
and
those
are
those
are
really
kind
of
the
main
ways
that
we
we
integrate
with
these
managed
services.
I
think
it's
not.
I
wouldn't
say
it's
a
a
huge
percentage
of
kind
of
what
we
do
most
of
the
things
we
back
up.
Usually
are
in
kubernetes,
but
we
definitely
do
see
some
use
cases
for
managed
services
here.
M
Comment
on
the
the
service,
the
application
backup
in
general-
is
that
that's
one
of
the
things
that
these
the
we
lack
is
seem
to
be
the
relationship
between
component
of
the
services.
For
example,
you
know
when
we
have
an
application
that
use
a
multiple
part
and
then
in
backup
I
think
it
might
be
okay,
but
when
we
go
to
the
restore
time
that
if
this
component
is
not
up,
the
other
one
will
not
be
even
functional.
M
So
when
we
restore,
we
have
to
restore
them
in
a
specific
order
for
it
to,
for
the
whole
service
to
be
to
be
functional
so,
and
that
is
it
does
it
doesn't
seem.
I
haven't
seen
this
capture
in
this
document
or
in
many
of
the
restore
service
that
that
we
we
saw
here
is
because
without
those
functioning
then,
for
example,
one
component
can
go
up
and
it
it.
M
It
doesn't
see
the
other
components
on
up
and
it
will
time
out
and
it
will
shut
down
that
component,
although
of
course
there
are
many
ways
to
get
around
with
it,
but
that
just
shows
the
the
important
of
dependency
between
these
components
of
a
service
of
of
an
application
when
we
do
the
restore.
This
is
my
two
cent
on
this.
B
I
think
I
think
you're
absolutely
right.
I
would
say
that's
a
general
problem,
though
not
specific,
to
managed
services.
You
know
the
easy
answer
is
what
we
do
is.
Essentially
we
treat
this
connection
configuration
as
the
object
that
we
want
to
back
up
because
that's
what
we
have
the
reference
to
in
kubernetes,
and
so,
if
you
have
an
ordering
constraint,
then
you
have
to
include
that
constraint
when
you
restore
these.
B
So
if
I
go,
you
know,
the
workflow
is
essentially
that
I
go
and
restore
objects
and
kubernetes
in
some
order
with
dependencies
between
them.
In
this
case
I
would
restore
you
know
it
would
be
like
I'm
restoring
these
things,
but
really
what
I'm
restoring
is
both
the
configuration
as
well
as
whatever
hooks
I
need
to
go
and
restore
the
managed
services.
G
Yeah,
I
think,
there's
also
kind
of
the
in
between
case,
which
is
the
operator
driven
applications
running
in
the
cluster,
but
the
operator
is
responsible
for
standing
things
up
and
even
has
like
the
zolando
postgres
operator
have
been
messing
with
and
they've
got
a
backup
in
there.
That's
based
on,
I
think,
wall
the
right
ahead.
Log
shipping,
among
other
things,.
B
Yeah,
that's
that's.
You
know
that's
more
complex
even
than
this,
because
you
you
first
kind
of
have
to
restore
the
operator
and
well.
This
is
one
approach.
When
you
first
restore
the
operator,
then
you
restore
the
instances,
but
in
that
case
you're
also
having
to
deal
with
replication
with
wall.
So
you
have
to
go
and
figure
out
how
you
want
to
apply
them,
which
means
you
probably
need
some
kind
of
custom
scripting
and
hopes
to
do
that.
G
B
Okay,
cool
this
slide
was
a
lot
less
controversial
than
this
one,
so
you
know
feel
free
to
the.
E
J
The
goal
here
is
for
tom
is
one
weren't
here
to
share
the
experience
of
doing
this.
I
see
how
external
externally
hosted
applications
like
cloud
sql
already
in
aws
this
kind
of
applications
protected
within
the
kubernetes
context.
N
E
A
Yeah,
so
we
basically
got
a
question
during
our
session
at
coupon,
so
shantiana
actually
got
this
question
and
then
I
think
the
do
we
have
a
I'm,
not
sure
if
the
person
who
brought
this
up
is
on
this
quality.
Yes,
I'm
here,
oh
hi
yeah,
maybe
you
want
to
yeah.
Maybe
you
want
to
talk
about
what?
What
was
your
original
ask
so
that
everybody
is
clear.
O
E
So
the
the
part
that
is
a
little
wake
to
me
is
that
you
know
whether
we're
talking
about
different
cloud
services
or
whether
we're
talking
about
different
databases.
You
know
each
one
is
done
differently.
There
are
different
steps
involved.
There
are
different
ordering
of
things
and
you
know
whether
we
want
to
formalize.
E
E
You
know,
services
or
different
across
different
databases
or
whether
it's
really
the
job
of
the
operator
for
those
applications
or
those
services
to
do
the
right
thing
based
on
what
needs
to
be
involved.
I
guess
I
guess
that's
the
part
that
is
unclear
to
me
whether
we
want
to
come
up
with
a
unified
api
or
whether
we
want
to
just
leave
it
to
the
operators
to
do
the
right
thing.
G
So
the
the
trick
that
I've
been
seeing
is
that,
as
when
you
have
like
one
service,
it's
not
a
problem,
you
go
into
like
rds,
you
say
back
this
up
every
day
and
that's
the
only
service
that
your
cluster
uses.
Fine,
not
a
problem,
but
you
start
proliferating
services
and
you
start
proliferating
clusters
using
services
using
instances
and
now
you've
got
this
big
management
problem
of
is
my
stuff
being
backed
up.
Is
it
being
scheduled
properly
did
I
you
know,
I
stood
up
a
new
cluster.
G
Did
I
go
to
rds
and
set
up
a
schedule
to
back
up
the
database
there?
So
that's
something
where,
like
from
the
valero
side-
and
I
think
the
cast
inside
as
well
really
looking
at
being
the
orchestrator
of
a
lot
of
stuff
and
sometimes
like
you
know
in
these
cases,
even
like
csi
snapshots,
where
evs
is
really
doing
the
data
movement
or
these
rds
snapshots
are
very
analogous
we're
not
really
a
backup
utility,
because
we
don't
actually
like
copy
the
data,
but
we
are
orchestrating
the
overall
backup.
G
B
B
I
can
go
and
query
aws
for
all
the
rds
instances
that
I
can
see
in
some
security
group
and
then
I
can
generate
the
config
mapping
connection
information,
but
then
the
downside
is
that
most
of
those
services
probably
aren't
being
used
from
inside
this
cluster.
D
B
Of
hope
is
that,
in
fact,
if
they're,
using
even
applications
that
would
connect
to
those
databases,
they
would
actually
use
kubernetes
objects
to
represent
that
that
configuration
anyway
but
yeah
I
mean
it's
all
customer
driven.
We
don't
have
a
discovery
mechanism.
G
Yeah,
I
think,
from
our
side
you
know:
we've
got
tanzu,
which
also
has
tanzania
data
services,
and
that
covers
you
know
we
have
like
green
plum,
which
is
distributed
postgres.
We
have
a
postgres
operator,
a
few
other
things
we're
looking
at
how
to
get
those
services
when
they're
in
use
via
kubernetes
to
be
exposed
out,
so
we
can
handle
them
with
data
protection
and
yeah,
so
we're
kind
of
in
the
early
stages.
Some
of
that
work.
B
Yeah,
that's
great
there's
a
project
from
apps
code
that
actually
was
pretty
cool,
that
put
put
all
the
stuff
in
a
crd,
and
the
idea
was
that
if,
if
that
was
standardized,
you
could
kind
of
build
tools
on
top
of
it.
I
didn't
see
it
gain
too
much
fraction,
but
I
really
liked
the
idea.
B
You
know
I
if
we,
if
we're
going
to
propose
a
standard.
Actually
this
might
be
a
good
thing
to
kind
of
get
this
group
on
board
on
around,
but
for
now
I
think
you
know
if
you
can
use
config
amounts
of
secrets,
that's
kind
of
our
prescribed
approach
right
now,.
G
Yeah,
certainly
it's
a
it's
a
it's
a
available
approach.
Personally,
I'm
really
pushing
us.
I
really
want
to
push
towards
operator
driven
stuff.
For
this
I
mean
you
can
easily
see
an
operator
living
in
front
of
rds,
even
where
you
can
create
a
resource
like
rds
database.
The
operator
goes
and
talks
to
rds
and
sets
it
up
and
that
we
get
a
lot
of
these
details
of
how
do
I
backup
and
restore
this
object,
gets
pushed
inside
the
the
object
itself
into
the
operator.
If
you
will.
B
Have
you
have
you
seen
crossplane?
Have
you
dug
into
that
at
all
sounds
familiar,
but
I
don't
remember
it
it's
it's
from
upbound.
That's
not
it's
kind
of
the
some
people
worked
on
rook.
Actually,
you
know
a
new
company
bound
to
work
on
it.
I
think
they're
trying
to
do
that.
Essentially,
you
know
they
kind
of
integrate
with
like
terraform,
and
you
can
essentially
provision
resources
based
on
crs
that
you
create.
G
Yeah,
so
if
we
can
get
them
to,
for
example,
put
in
paths
where
we
can
trigger
their
snapshot
mechanisms,
then
we
don't
have
to
run
around
behind
them
and
figure
out
how
to
do
it.
E
J
Definitely
in
in
this
case
is
a
possible
database
where
what
tom
is
showing
right
now
in
in
you
know,
for
example,
I
don't
know
other
cash
cases,
radius
cases
or
cassandra
cases,
they
might
need
a
completely
different
set
of
input
generating.
Those
input
is
not
straightforward.
E
Sure,
but
but
the
nice
thing
about
you
know
what
we're
discussing
now
is
that
all
of
that
gets
handled
by
the
operator
you're
not
exposed
to
it.
You
know,
as
far
as
like
valerie's
concern
casting
is
concerned,
you
just
define
a
schedule,
you
just
say,
take
a
snapshot
and
then
whatever
information
they
need,
whether
it's
configmap
cr
or
you
know
secret.
Whatever
that
that's
taken
care
of
by
the
operator.
G
Yes,
and
so
that
we're
not
like
trying
to
figure
out
how
to
take
a
snapshot,
the
operator
should
already
have
all
this.
You
know
it's
got
a
resource
that
told
it
what
it's
set
up.
It
should
have
any
secrets,
config
maps
etc
in
there
like,
for
example,
talking
to
the
service,
or
even
you
know,
which
pods
are
involved
with
this,
which
persistent
disks
are
involved
with
this
and
and
really
push
that
complexity
behind
it,
because
I
think
everything
well,
almost
everything
can
be
boiled
down
to
like
a
snapshot.
G
J
G
J
G
The
working
plan
I've
been
doing
is
to
basically
have
snapshot,
crds
that
are
analogous
to
the
volume
snapshot,
the
csi
ones,
but
probably
a
little
slim
down,
and
so
you
can
find
a
resource.
For
example,
like
the
postgres
operator
exposes
postgres
crs,
and
you
should
be
able
to
write
a
snapshot
resource
that
references,
the
cr
and
then
the
operator
picks
it
up
and
does
the
intervals.
G
J
In
order
to
support
those
that
that
needs
the
existing
work
workloads
to
basically
change
the
way
of
managing
things,
a
lot
well.
G
Well,
like,
like
you
know,
like
tom
and
casten
they've,
got
a
solution
right
now,
where
they
say
hey.
If
you
use
this
external
service,
go
ahead
and
write
a
script
and
read
your
config
map
that
you
wrote
and
go
ahead
and
use
it
and
trigger
that
external
stuff.
So
essentially
I
mean,
if
you,
if
you
step
back
a
little
bit,
that's
really
very
much.
This
snapshot
pattern,
except
that
it
was
built
by
the
customer
as
a
one-off
for
each
installation,
because
each
installation
is
unique.
B
Yeah
there's
some
there's.
Some
commonality,
obviously
like
for
rds
taking
a
snapshot
in
rds,
actually
has
relatively
few
parameters
the
a
lot
of
the
parameters
that
we
need
are
actually,
if
you
want
to
also
take
a
logical
dump.
For
example,
you
need
the
connection
information
to
the
specific
database.
B
So
hopefully
you
know.
My
hope
is
that
you
can
kind
of
get
this
for
this.
You
don't
have
to
do
the
cross
product
of
everything
you
can
implement
the
logical
part
and
then
implement.
You
know
the
interaction
with
just
rds
or
the
you
know,
cloud
sql
or
the
other
azir.
You
know
sql
server
and
all
that.
P
Is
the
idea
here
that
it
looks
something
this
snapshot?
Light
instance
rcr
looks
something
like
an
ingress
resource
or
a
gateway
ins
resource
where,
like
different
people,
can
implement
their
own
things
and
using
like
so,
you
can
have
more
than
one
of
these
ideas
on
the
cluster
like.
Is
that
what
you're
yeah?
Are
you
proposing
a
shared
api.
G
A
common
api
but
different
implementations
so
that
we
should
be
able
to
find
a
resource
that
says
snapshot,
type
resource
type
x,
so
like
postgres
database
resources
have
an
associated
snapshot.
Cr
and
the
implementation
of
the
snapshot.
Cr
is
not
done
in
a
common
place
necessarily
but
like
the
postgres
operator
should
be
exposing
that
api
or
responding
to
those
resources.
G
But
in
the
way
that
we
tell
it
so
that
you
know
we
don't
have
to
rewrite
for
each
individual
operator
like
right
now.
This
orlando
postgres
operator
has
got
apis
for
backing
up
and
restoring,
but
they're
unique
to
the
postgres
operator.
So
I'll
have
to
write
a
plug-in
for
valero.
That
knows
how
to
talk
to
the
lando
and
then
the
next
one
and
the
next
one
or
the.
I
G
One
and
I'd
rather,
and
the
other
option
is
to
get
all
the
vendors
to
write
a
plugin
for
valero,
but
then
that
leaves
caster
now,
because
you
know
you
go,
you
can't
use
our
plug-ins.
So
I
want
to.
I
want
to
try
and
drive
us
towards
a
standard,
or
at
least
a
concept
where
there's
a
relatively
simple
volume
snapshot
cr.
G
You
know
either
the
the
backup
companies
right
and
plugins
or
the
database
developers
right
and
plugins,
and
that
you
know
that
that
seems
like
madness.
A
Hey
dave,
I
think
one
challenge
I
think,
with
this
approach
is:
how
do
you
define
this
common
snapshot?
Crd?
What
is
in
the
stack?
What
is
in
the
status,
because
we
are
backing
up
different
things
right,
yeah,
but
I
don't
know.
A
I
mean
normally
I'm
just
saying
normally
like
let's
say:
if
you
take
a
you,
should
look
at
one
snapshot
right,
you
have
slack
and
you
have
status.
You
have.
There
are
objects
that
you
want
to
observe
like
what's
the
status
of
those,
but
if
it's
just
like
a
general
one
for
all,
what
will
be
returned
like
what?
How
are
you
watching
that
you
know.
M
G
A
I
was
just
wondering
like,
for
example,
you
could
have
different
resources
there
or
you're
saying
just
kind
of
just
say:
save
them.
E
Just
to
be
more
specific
dave
are
you,
let's
say
you
know.
Obviously
you
know
four
different
databases:
we're
going
to
have
different
crds.
You
know,
for
example,
let's
say
one
for
postgres
snapshot,
one
for
let's
say
mysql,
so
these
are
different
crds.
So
there's
no
way
we
can
come
up
with
the
same
crd
for
both.
Unless
you
know
we
just
call
it
something.
Generic
which.
E
G
And
also
that
the
crd-
basically,
even
though
it
so
so
so
we
have
like
volume,
snapshot
crd
right
now
and
the
postgres
snapshot
crd
should
look
a
lot
like
the
volume
snapshot.
Crd,
the
mysql
crd
should
pretty
much
be
identical
to
the
postgresql.
Maybe
there's
room
for
extra
parameters,
but
we
should
be.
C
G
I
don't
know
at
some
point:
there's
there's,
definitely
application
specific
stuff.
The
model
I
have
defined
right
now
like
we're
using
this
inside
the
vsphere
plug-in.
We
can
snapshot
something
and
then
it
exposes
out
a
data
stream
with
the
snapshot
and
we
can
copy
that
someplace
else.
So
we
can
leave
it
sitting
there,
but
the
magic
of
volume.
I
G
G
Finish
english,
let
me
finish
please,
if
you're
looking
at
this
from
the
point
of
view
of
a
backup
provider,
even
if
your
snapshots
are
durable,
customers
are
going
to
say
things
like
hey.
I
want
you
to
actually
copy
that
over
to
the
veritas
fault
and
put
it
into
the
disk
array,
and
so
even
with
csi
volume
snapshots,
I
need
this
data
extraction
api
to
get
stuff.
C
You
know
you
kind
of
want
a
common
way
to
do
this
where
you're
like
this
is.
This
is
the
place
where
all
my
backups
are
going
to
go
and
if
you
have
three
different
apps,
they
can
use
the
same
information
to
decide
to
shove
the
three
different
backups
into
that
place,
so
they
don't
have
to
specify
it
three
times.
E
Here,
as
you
know,
dave
mentioned
like
we
can
have
a
similar
construct,
just
like
snapchat
class
for
these
application
level
databases.
You
know
to
capture
that
information
and
we
are
doing
the
same
thing.
Netapp
is
doing
the
same
thing.
E
You
know
what
we
can
talk
about
offline,
but
we
are
doing
the
same
thing
outside
of
the
csi
mechanism,
where
you
can
specify
how
to
back
up
a
volume
to
an
object
store
and
without
even
invoking
csi
apis
by
just
dealing
with
crs.
So
we're
already
doing
this
today.
I
C
Develop
is
a
more
open
way
to
say
you
know
for
someone
who
knows
how
to
generate
backups,
and
you
know-
and
I
I
need
a
place
to
shove
them
that
that
I
can
in
principle,
expect
to
exist
anywhere
right
that
that's
the
beauty
of
this
design
is
that
it
doesn't
depend
on
any
particular
underlying
technology.
It
just
says
I
know
it's
a
postgres
database.
I
want
to
back
it
up.
You
know
it.
C
G
And
and
we'll
see
that
some
things
may
not
expose
it
so
that
we
can
get
access
to
the
data
like
like.
Take
a
look
at
like
evs
snapshots
prior
to
I
think.
Last
year
there
was
no
way
to
access
the
data
in
an
evs
snapshot
except
by
cloning,
the
volume
and
attaching
it
to
something,
and
that's
true
for
like
rds
snapshots,
you
can't
get
into
the
yeah
rds
so
like
in
the
middle
of
rds
postgres.
You
can
create
a
snapshot.
P
Yeah,
I
kind
of
wanted
to
jump
in,
but
I
didn't
want
to
like
talk
over
anybody,
so
I
I
the
only
question
that
I
wanted
to
bring
up
was.
It
seems
like
there's
two
problems,
there's
like
the
problem
with
like:
where
does
the
data
go
to
and
and
this
stuff
from
the
csi,
and
we
want
to
like
give
options
for
that,
but
then
there's
the
second
problem
that
seems
to
be
like
there
are
some
application,
specific
things
that
need
to
happen
for
a
given
backup.
P
J
I
I
think
you,
I
think,
you're
right,
you
know
shin
and
I
or
we
we.
We
went
through
this
discussion
for
multiple
times.
J
Unlike
what
dave
was
mentioning
using
the
operator
way,
the
thought
was
using
application
constructs
and
that
actually
it
was
kind
of
in
the
original
thoughts
of
defining
applications,
because
if
you
really
think
about
whether
it's
externally
managed
databases
or
in
a
stateful
workload
deployed
directly
in
the
cluster,
if
there
is
a
way
we
can
group
the
resources
and
the
different
ways
of
taking
a
snapshot
of
that
those
applications
within
the
application
construct,
then
all
these
problems
should
be
solved.
E
J
J
It's
sure
the
the
reason
why
we
didn't
go
too
much
on
that
in
the
beginning
is
the
variety
of
the
applications
makes
the
application
cr
definition
pretty
quick,
pretty
hard
right,
that's
one
with
regarding
to
the
destination
the
backup
target,
I
think
there's
also
a
discussion
right.
One
one
is
the
causey
or
something
like
that
is
to
define
my
my
thoughts
was
to
define
a
target
in
within
the
kubernetes
construct
right.
This
target
could
well
be
a
file.
J
Mining
could
well
be
a
object,
storage
or
something
like
that,
and
that
target,
or
even
a
I
don't
know,
maybe
a
multi-region
bucket
in
the
cloud
right
in
this
case
all
when
we
do
a
backup
orchestration
when
backup
orchestration
system
comes
in,
they
can
refer
to
that
destination.
As
a
as
a
single
point,
a
single
point
of
truth,
everything
just
goes
there
in
an
ideal
world,
but
in
reality
it's
really
challenging,
especially
with
this
operating
model.
B
I'm
sorry,
oh,
go
ahead.
John,
do
you
think
I'm
having
data
path
apis
in
that
model.
J
Well,
it
depends
on
which,
right,
if
it's
it's
a
cloud
hosting
services
unlikely,
you
will
likely.
You
will
not
be
on
the
data
path
right,
because
yeah,
the
snapshot
will
be
taken
care
of
by
the
cloud
providers
or
external
stateful
workload
providers,
whatever
it
is,
but
if
it
is
a
construct
sitting
within
the
kubernetes,
yes,
then
you
might
be
on
the
data
path,
but
not
your
not.
The
orchestrator
could
well
be
the
application
operator,
as
as
there
was
mentioning.
B
J
G
I
want
both
because
no
seriously
because
there
should
be
a
fallback
position
where
it's
the
same
as
with
like
the
existing
csi
design,
the
fallback
position
to
get
the
data
out
of
the
snapshot
is
clone,
attach
extract
and
it's
slow
potentially,
but
it
should
always
work.
So
that's
like
your
fallback
position.
A
Hey,
I
think
we
only
have
a
couple
minutes
left
yeah.
I
think
this
is
a
great
discussions
I
want
to
get
to
the
remaining
of
the
identity.
Maybe
we
should.
J
A
Yeah
sure
yeah.
We
should
actually
continue
that.
Yes,
so
let's
sync
up
offline
and
maybe
we
could
do
follow
up
in
the
next
meeting.
B
For
that
discussion
dave,
could
you
bring
your?
Could
you
present
the
what
you
have
right
now
for
the
crs
that
you're
thinking
of
I'd
love
to
dig
into
those.
G
A
Okay,
so
let
me
just
share
the
I
okay,
so
we
basically
just
want
to
look
at
the
white
paper
and
see
how
we
are
doing.
Oh
there's
such
a
whole
issue
here
so
white
paper.
I
think
I
see
font
says
he
hasn't
got
time
to
update
that.
Yet
tom,
do
you
have
any
update
about
this,
or
are
you
you're
waiting
for
people
to
give
feedback.
B
No
update,
I
think
the
last
thing
we
talked
about
was
steven
actually
put
together.
I
think
a
lot
of
the
a
lot
of
things
that
would
go
in
this
section,
so
I
think
the
next
step
here
is
probably
organizing
things
a
bit
and
figuring
out
what
we
want
to
pull
off
from
his
section
and
put
it
in
here
or
if
we
want
to
extend
this
at
all.
You
know
I
can
write
some
text
for
this
as
well.
A
Oh
so
he
added
in
this
in
the
stock.
I
can
take
a
look
if
it's
already
added
there.
B
No
steven,
maybe
like
a
month
and
a
half
ago,
presented
a
section
that
had
he
I
mean,
there's
a
lot
of
detail.
There.
B
Cases
right,
it
was
use
cases,
but
I
think
there
was
a
lot
of
discussion
of
motivation
that
included
hook
hook.
Information,
oh.
F
Duplicative
and
and
yeah,
because
you're
right,
I
I
sort
of
wrote
mine,
totally,
not
understanding
what
else
was
around
it,
so
it
might
make
sense.
That'd
be
great.
A
All
right
so
far,
you
basically
just
need
to
you're
not
waiting
for
anyone
else
right.
You
just
need
to
update
it
yourself
right.
I
think,
last
time
we.
K
Yeah
I
just
I
need
to
look
a
little
bit
more
detail
to
see
if
we
can
extract
an
example
flow
out
of
the
netpass
snap
it
or
if
we
can,
I
can
review
the
example
that
we
currently
have
okay
to
do
to
illustrate
that
it
can
only
be
also
used.
Similarly
in
snap
another
one
that
I
have
I
haven't
have
a
chance
to
look
at
is
how
we
can
do
something
like
you
know,
elastics
to
to
take
advantage
of
this
snapshot.
Q
A
Okay,
thank
you
all
right.
I
think
we
are
at
the
top
of
the
hour
yeah
great
discussion.
Today
we
will
have
a
follow-up
discussions.