►
From YouTube: Introduction to Data Protection WG in Kubernetes - Xing Yang, VMware & Xiangqian Yu, Google
Description
Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Introduction to Data Protection WG in Kubernetes - Xing Yang, VMware & Xiangqian Yu, Google
Data Protection WG in Kubernetes was formed following discussions at KubeCon in San Diego. This is a Working Group dedicated to promoting data protection support in Kubernetes, identifying missing functionality and working together across multiple SIGs to design features to enable data protection in Kubernetes. In this session, the co-chairs of this WG will explain the motivation behind the formation of this WG, the charter of this WG, who are involved, what is the current state of data protection in Kubernetes and where it is heading in the future. They will also talk about how interested parties (including backup and storage vendors, application developers, and end users, etc.) can join this WG and contribute to this effort. Details of the WG can be found here: https://github.com/kubernetes/community/tree/master/wg-data-protection.
https://sched.co/ZeuM
A
B
A
So
here
is
today's
agenda.
First,
we
will
talk
about
why
we
are
forming
this
data
protection
work
group
who
are
involved.
We
will
talk
about
the
charter,
the
data
protection
definition,
what
other
existing
building
blocks
in
kubernetes
and
what
are
still
missing,
what
we
have
been
working
on
and
finally,
we'll
talk
about
how
to
get
involved.
A
And
a
persistent
volume
has
an
independent
life
cycle
from
the
pod
that
is
consuming
it,
so
that
data
can
be
preserved
under
an
underlying
solar
system,
even
if
the
part
goes
away.
However,
what,
if
the,
what?
If
the
underlying
story
system
is
stricken
by
a
disaster,
what
if
the
volume
on
this
resistance
gets
corrupted?
For
some
reason,
when
that
happens,
even
data
stored
on
the
precision
volumes
will
be
gone
so
to
prevent
data
loss
from
happening.
A
A
A
A
A
We
reached
a
consensus
that
a
dedicated
work
group
should
be
formed
to
focus
on
this
area.
We
got
support
from
sixth
floor
chair
saad
ali
and
sick
abs,
chair
adnan.
We
sent
out
an
email
in
kubernetes
down
main
list
and
got
overwhelming
responses,
so
we
formally
submitted
this
pr,
as
shown
here,
to
form
this
main
group.
It
was
approved
by
the
steering
committee
in
january,
so
the
group
was
formally
established.
A
And
this
is
the
charter
of
the
data
protection
work
group.
We
have
this
listed
on
our
wing
groups.
Community
page
this
one
group
is
formed
so
that
we
can
have
a
cross
seek
collaboration
to
figure
out
what
are
the
missing
functionalities
and
work
together
to
design
features
in
order
to
support
data
protection
in
kubernetes
sponsoring
six,
for
this
green
group
are
seek
apps
and
seek
storage.
B
Thank
you
here,
xin
in
the
following
10
to
15
minutes.
I
walked
through
the
definition
of
data
protection
in
this
working
group
and
what
are
the
problems
we
are
looking
to
solve
in
this
working
group?
What
are
the
kubernetes
native
constructs?
We
are
this
working
group.
It's
exploring
or
designing,
etc.
B
Data
protection
in
the
industry
can
either
refer
to
protecting
data
from
compromise
and
ensure
data
privacy
or
from
time
to
time.
It
also
refers
to
ensuring
that
data
and
that
application
are
protected
such
that
they
could
be
quickly
restored
and
recovered
into
any
previously
saved
state.
Whenever
there's
a
data,
loss
or
corruption
happens
in
this
working
group,
we
focus
on
the
latter.
In
the
kubernetes
environment,
there
are
two
major
processes
in
data
protection,
backup
and
recovery.
B
The
backup
process
produces
durable,
backups
of
a
system.
In
many
cases,
backup
processes
are
scheduled
to
comply
with
certain
policies.
For
example,
an
organization
might
want
to
enforce
weekly
backup
of
all
its
production
database
for
business
continuity
reasons
and
the
recovery
process
takes
a
backup
and
restore
resources.
Indeed,
particu
it
can
be
partially
restoration
or
can
be
full
restoration.
B
B
B
B
At
persistent
volume
level,
we
need
building
blocks
to
support
one
snapshot,
volume,
backup
and
restoration
from
snapshot.
Backup
snapshot
is
already
a
beta
feature,
supported
by
many
csi
drivers
in
kubernetes.
Since
1.17
recreating
a
pvc
from
a
volume
snapshot
has
been
released
as
part
of
the
feature
compared
to
volume,
backup
traditionally,
worm
snapshot
is
in
general,
relatively
cheap
and
fast
as
it
is,
as
it
normally
sits
on
the
same
storage
media
as
the
source
warning.
B
However,
many
cloud
providers
implemented
indifferently
in
today's
kubernetes
csi
drivers
I'll
go
a
little
bit
deeper
later
at
application
level.
We
need
to
building
blocks
to
first
of
all
group
api
resources,
which
belongs
to
an
application
that
basically
defines
all
the
api
resources
that
constructs
this
particular
application.
B
B
B
In
the
persistent
volume
level,
one
of
snapshot
is
already
better,
as
mentioned
I'll,
try
to
cover
more
details
in
the
foreign
slides
they
plan
to
move
this
feature
to
ga
in
the
latter
version.
In
the
application
level,
they
are
rich,
fundamental
workload,
apis
such
as
stat4set
replica
set
deployment,
etc.
B
On
top
of
that,
you
from
time
to
time
also
need
a
service.
If
you
take
a
look
at
the
definition
of
the
application
crd,
it
is
defined
by
a
group
of
group
kinds,
as
well
as
a
selecting
labels
which
selects
the
resources
of
those
corresponding
group
kinds
to
construct
the
application
look
at
in
the
middle.
The
application
can
basically
theoretically
take
any
group
kinds
of
existing
kubernetes
constructs,
stateful
deployment,
service,
etc.
B
You
can
take
a
look
at
the
component
kind
section,
there's
a
service
which
exposes
interfaces
to
external
kinds
and
there's
also
a
application,
which
is
a
kind
of
stateful
set
which
holds
the
real
workload
for
this
mongodb
application.
Crd
gives
a
potential
way
of
data
protection.
Working
group
can
use
to
define
an
application
and
group
the
api
resources
that
belongs
to
the
application
for
snapshot
and
backup.
B
B
B
We
still
are
missing
backup
repositories.
This
is
more
about
storing
backups
into
a
different
storage
media
than
your
primary
storage,
such
that
in
cases
like
a
disaster
recovery.
Where
you
don't
have
your
data
center
or
your
cluster
around.
You
still
have
a
backup
seat
somewhere,
maybe
in
the
cloud
or
maybe
in
somewhere
place
some
other
data
center.
That
can
restore
your
original
data
center
into
the
next
into
a
different
place.
B
A
A
This
allows
the
pvc
to
be
provisioned
for
an
external
data
source,
not
just
a
warning
snapshot
or
another
existing
pvc,
but
it
could
be
an
object,
store,
a
github,
repo
or
image
or
something
else.
This
is
very
important
in
the
backup
and
resolve
path.
It
allows
us
to
dynamically
provision
a
pvc
based
on
a
storage
class
create
an
underlying
volume
and
allow
data
on
that
volume
to
be
restored
from
someplace
else,
such
as
a
backup
repository
create
a
pv
to
point
to
that
volume
and
then
that
kubernetes
pve
controller
to
bind
about
pv
and
pvc.
A
A
A
A
A
A
A
A
It's
it
refers
to
a
physical
bucket
in
a
cloud
or
on
a
pram
and
contains
connection
information
and
metadata
to
communicate
with
a
physical
bucket.
For
this
feature,
there
have
been
weekly
design
meetings.
It
has
become
a
sub
project
under
six
storage
and
repos
have
been
created
under
kubernetes
6
for
prototyping.
A
A
So
the
motivation
for
this
feature
is
that
the
existing
volume
snapshot
feature
was
not
explicitly
designed
for
backups,
because
there's
no
explicit
definition
in
that
design
to
have
snapshots
stored
on
a
different
backup
device
separate
from
the
primary
storage
for
most
cloud
providers.
A
snapshot
is
actually
a
backup
that
is
uploaded
to
an
object
store
in
cloud.
A
A
A
A
This
refers
to
applications
that
maintain
persistent
state
and
defines
how
directly
how
to
run
operations
on
those
applications
to
do
snapshot,
backup
and
restore
a
snapshot
of
a
safe
application
is
defined
as
a
point
in
time.
Capture
of
the
state
of
the
application
ensuring
application
consistency.
A
So
we
are
also
working
on
other
things.
There
will
be
some
potential
topics
that
we
may
discuss
in
the
future,
as
shown
here.
The
first
one
is
the
diffs
between
two
snapshots,
thus
can
be
change
blocks
for
block
volumes
or
change
files
for
file
volumes.
This
is
just
for
incremental
snapshots
or
backups.
A
We
actually
have
not
started
the
discussion
on
this
topic
yet,
and
second
one
listed
here
is
the
data
production
policy.
This
usually
refers
to
how
we
can
set
up
schedule
to
do
periodic
backups
and
set
up
a
backup
retention
policy
to
automatically
clean
up
old
backups
or
set
up
a
topology
to
specify
a
backup
location.
A
A
So
if
you
are
interested
in
getting
involved
in
this
working
group,
take
a
look
of
the
working
groups-
community
page
community
page
first-
it
has
a
lot
of
information
to
get.
You
started
join
the
bi-weekly
meeting
on
wednesdays
at
9am,
pacific
time.
Previous
meetings
are
recorded
and
available
on
youtube,
join
the
mailing
list
and
the
slack
channel.
A
B
A
A
Okay
same
question
is:
does
this
working
group
collaborate
with
like
sneer
or
specific
vendors
to
benefit
of
their
years
of
experience
yeah?
So
in
this
learning
group
I
think
there
is
one
page
that
was
shown
earlier
in
the
presentation.
There
are
multiple
vendors
supporting
us.
For
example,
let's
see,
let's
start
trying
to
see
the
page
like
some
of
their
traditional
sweetie
vendors
like
dell
emc,
you
know,
netapp,
there
is
a
ibm
yeah,
so
there
are
actually
multiple
vendors
already
supporting
us
and
then
they
they
are
in
this
working
group.
B
A
A
Okay
next
question:
how
would
you
judge
the
need
for
kubernetes
building
backup,
restore
capabilities
versus
the
application
itself,
just
handling
it
at
a
higher
level?
That
was
question
that
has
always
been
a
discussion
point
for
traditional
applications
yeah.
So
we
think
that
we
should
still
have
this
capability
built
into
kuwaitis.
A
As
you
can
see
that
we
actually
got
a
lot
of
support
from
welcome
vendors
and
story
vendors,
they
would
like
to
have
this
capability
in
kubernetes.
There
are
many
missing
pieces,
so
if
we
can
provide
those
missing
pieces,
then
that
would
enable
the
studio,
the
backup
vendors,
to
have
a
full
solution
for
data
protection
communities.
A
B
So
when
we
look
into
solving
the
problems
in
the
data
protection
problems
in
kubernetes
con
context,
one
of
the
things
is
that
how
can
we
us
as
a
community
supply,
essential
building
blocks
to
solve
applications,
problems
right,
for
example,
worm,
snapshot,
execution,
hooks
wooden
grouping,
application
level,
resource
grouping,
etc,
etc?
Those
are
all
fundamental
building
blocks
such
that
applications.
They
themselves
can
effectively
achieve
application,
consistency,
snapshot
or
backup.
B
A
Thanks
next
question:
how
much
does
the
current
work
on
data
protection
support
or
type
of
solutions
like
velara?
So
we
are
basically
providing
some
tools
for
vlara
to
use
right,
like
chanting,
nation,
that
we're
working
on
the
choirs
hooks,
so
they
can
actually
leverage
this
in
valara
right
now.
Without
this,
they
will
be
just
directly
using
the
the
exact
sub
resource
to
to
do
this
requires,
but
with
this
tool
it
will
be
much
easier.
The
lara
is
also
actually
part
of
this
working
group,
so
yeah.
A
B
A
A
Yet
so
right
now,
if
you,
if
you
are
doing
a,
if
you
want
to
do
a
backup,
then
it's
really
depending
on
the
specific
backup
application
right,
so
it
can
go
automatically
check
if
it's
the,
if
the
first
one
is
for
backup
and
the
second
one
could
be
automatically
a
incremental
backup
yeah.
So
we
don't
so
right
now
we
don't
really
have
a
timeline
yet,
but
this
is
definitely
in
our
chatter.
A
A
Any
thought
on
the
ability
to
question
api
kick
back
up
if
taking
snapshot.
The
edc
is
not
always
feasible
due
to
access
issues
yeah,
so
it
is.
We
are
not
tackling
the
epcd
itself
at
this
point,
but
I
think
that
that
could
be.
When
we
look
at
it
like
backing
up
the
whole
cluster,
then
we
can
look
at
that
piece,
but
that's
already
tackled
by
some
other
tools.
We
are
really
trying
to
figure
out
how
to
do
choice
before
taking
snapshots.