►
From YouTube: Kubernetes Data Protection WG Intro and Deep Dive - Xing Yang, VMware & Xiangqian Yu, Google
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Kubernetes Data Protection WG Intro and Deep Dive - Xing Yang, VMware & Xiangqian Yu, Google
Data Protection WG is dedicated to promoting data protection support in Kubernetes. The Working Group is working on identifying missing functionalities and collaborating across multiple SIGs to design features to enable data protection in Kubernetes. In this session, the co-chairs of this WG will discuss what is the current state of data protection in Kubernetes and where it is heading in the future. They will also talk about how interested parties (including backup and storage vendors, application developers, and end users, etc.) can join this WG and contribute to this effort. Details of the WG can be found here: https://github.com/kubernetes/community/tree/master/wg-data-protection.
A
B
A
A
A
A
A
There
are
volume
operations
either
through
csi
driver
or
entry
drivers,
and
they
are
working
apis
deployment,
stateful,
set,
etc.
Just
a
product
with
more
and
more
stateful
workload.
Moving
to
kubernetes
the
desire
of
providing
fundamental
building
blocks
to
allow
users
protect
the
data.
It's
getting
hit
hit
it
git
ops
could
potentially
be
used
for
rolling
back
of
stateless
workloads.
A
That's
to
support
the
data
operations
for
data
protection
specifically,
and
this
working
group
has
two
major
stakeholders:
sick
apps,
and
seek
storage
and
potentially
seek
note
next
slice
piece
so
who
are
oregon
world?
We
have
a
bi-weekly
meeting
and
those
are
just
probably
incomplete
list
of
companies
that
are
supporting
this
initiative.
A
A
The
main
purpose
of
data
prediction
is
to
make
sure
that
an
application
and
its
data
can
be
recovered
to
a
previously
preserved
state.
We
call
it
backup
for
the
state,
after
any
corruption
or
losses
that
can
include
badly
wrote
out.
Software
cost
data
corruption.
It
can
be
disaster
recovery
when
you
lost
the
whole
data
center,
etc,
etc.
A
A
At
the
volume
level,
the
volume
snapshot
features
and
we're
looking
to
into
one
backup
features
within
restore
in
the
application
level,
we're
looking
to
define
what
consisted
in
your
application
and
how
to
use
choirs
and
inquires
and
applications
such
that
application.
Consistent
snapshot
can
be
achieved.
A
A
Here
this
diagram
shows
you
a
pretty
typical
stateful
application,
backup
workflow.
So
a
user
generally
starts
a
backup
either
with
your
coi
or
something
it
has.
Two
missions
to
do.
One
is
to
back
up
the
kubernetes
resources
and
get
them
archived
or
exported
to
an
external
externally,
managed
storage,
typically
object,
storage
systems
and
then
the
other
piece,
the
data
backup
as
you're
made
aware
of
some
of
the
stateful
workloads
like
mysql,
postgres,
radius,
etc,
etc.
A
They
have
this
so-called
application
native
data
dump.
So
what
they're
capable
of
is
ncri
tool
which
allows
you
to
download
the
current
state
of
the
application
into
a
local
file,
and
this
local
file
can
be
later
unused
for
restoration
in
this
case,
if
the,
if
the
such
application
has
a
narrow
data
down,
a
user
can
choose
to
use
the
linear
dump
and
export
the
dump
file
into
the
remote
bucket
object
bucket.
A
A
A
A
A
A
Next
slides,
please!
So
how
does
this
work?
How
does
this
building
blocks
fit
into
the
workflow,
as
you
might
see
the
green
box
over
there?
What
color
the
apis
and
seek
application
seek?
Apps
application
crds
can
help
the
backup
system
understand
what
other
kubernetes
resources.
This
backup
should
contain
an
exercise.
A
Please
worm
snapshot,
feature
majorly,
sits
in
when
the
con
in
the
controller
or
coordinate
workflow,
where
the
controller
can
literally
create
a
one
snapshot
object,
and
this
will
trigger
a
snapshot
over
the
system
of
the
quiz
cook
is
called
and
during
the
next
slide
space
and
the
installation
in
the
restoration,
workflow
the
same
thing,
a
volume
snapshot
can
be
directly
used
to
rehydrate
pvc
from
from
it
next
slides.
Please
we
talk
about
existing.
So
what
I,
what
I'm
missing?
A
This
is
just
a
short
list
of
missing
features:
we're
looking
to
design
a
build,
william
bacca
worm
snapshots
in
some
of
the
vendors
they
are.
They
they're
stored
externally
in
some
of
the
windows
actually
they're
locally
stored
in
the
primary
storage
system
and
backup
repository.
Where
do
we
put
or
where
do
we
put?
Our
backup
manifests
choice
and
quest
hooks?
A
A
So
the
worm
backup,
just
like
williams
snapshot,
fits
into
the
picture
of
controller,
coordinated
backup
process
and
the
backup
repository
serves
as
the
as
the
the
storage
place
location
for
when
you
try
to
expose
the
backup,
an
application
backup.
It's
a
high
level
construct
organize
all
this
and
coordinating
all
this
process.
B
B
We've
already
got
a
volume
snapshot
api,
but
there's
no
explicit
definition
made
in
the
design
to
have
snapshots
stored
on
different
backup
device,
separate
from
the
primary
storage
for
some
cloud
providers.
A
snapshot
is
actually
a
backup
that
is
uploaded
to
an
object
store
in
the
cloud.
However,
for
most
other
search
vendors,
a
snapshot
is
locally
stored
alongside
the
volume
on
the
primary
storage.
B
B
B
B
Another
use
case
for
ct
is
snapshot-based
replication,
where
you
take
snapshots
periodically
and
replicate
it
to
another
site
for
disaster
recovery
purpose.
So
what
other
alternatives?
Without
cbt?
We
can
either
do
full
backups
or
we
can
call
each
storage
api
individually
to
retrieve
cvt,
which
is
highly
inefficient.
B
B
The
third
missing
building
block
is
the
backup
repository
backup
repository
is
a
location
or
repo
to
store
data.
This
can
be
an
object
store
in
cloud,
on-prem
storage,
location
or
a
nss
based
solution.
There
are
two
types
of
data
to
be
backed
up
that
we
need
at
the
restore
time.
The
first
one
is
kubernetes
cluster
metadata.
B
B
B
It
also
introduces
container
object,
source
interface
or
cosy
as
a
set
of
grp
interfaces
for
provisioning
object.
Buckets
kubernetes
cozy
is
already
a
sub
project
in
6th
storage
and
it
is
a
pretty
active
project.
It
has
weekly
design
meetings
and
standard
meetings.
It
is
targeting
afar
in
1.22
release.
B
B
B
The
volume
populator
feature
allows
us
to
provision
a
pvc
from
an
external
data
source,
such
as
a
backup
repository.
In
addition,
it
allows
us
to
dynamically
provision
a
pvc
having
data
populated
from
the
backup
repository
and
honor
the
wait
for
first
consumer
volume
binding
mode
during
restore
to
ensure
that
volume
is
placed
at
the
right
node.
Where
part
is
scheduled
near
then,
any
volume
data
source
alpha
feature
gate
which
was
introduced
in
1.18.
B
B
B
B
B
B
B
B
In
phase
2,
we
also
propose
to
add
an
inline
product
definition
for
signals
and
allow
the
api
object
to
send
a
request
to
trigger
delivering
of
those
signals
and
also
move
logic
in
the
power
notification
controller
into
kubelet,
so
that
kuberlead
will
be
watching
part
of
notification
objects
runs
the
commands
and
update
statuses
of
pod
notifications
in
phase
3.
A
probe
may
be
added,
if
needed,
as
an
inline,
4
definition
to
verify
the
signal
is
delivered
or
whether
the
command
is
run
and
the
results
in
desired
outcome.
B
What,
if
you
can't
request
the
application
or
if
the
application
class
is
too
expensive,
so
you
want
to
do
it
less
frequently,
but
still
want
to
be
able
to
do
a
take.
A
crash,
consistent
snapshot
more
frequently
also
an
application
may
require
the
snapshots
from
multiple
volumes
to
be
taken
at
the
same
point
in
time.
B
That's
when
consistent
group
snapshot
comes
into
the
picture.
Here's
a
cap
on
one
group
and
a
group
snapshot.
It
proposes
to
introduce
a
new
voting
group
crd,
that
groups,
multiple
warnings
together
and
a
new
group
snapshot
crd
that
supports
taking
a
snapshot
of
all
volumes
in
the
group
to
ensure
right
order,
consistency.
B
B
We
have
snapshot
apis
for
individual
volumes,
but
what
about
protecting
a
stateful
application?
There
is
a
cup
submitted
that
proposes
the
kubernetes
api
that
defines
the
notion
of
state
publications
and
defines
how
to
run
operations
on
those
stateful
applications
such
as
snapshot,
backup
and
restore.
This
is
still
in
a
very
early
design
stage.