►
From YouTube: Day 2 with Stateful Applications - Implementing a Data Protection... - Deepika Dixit & Vaibhav Kamra
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Day 2 with Stateful Applications - Implementing a Data Protection Strategy - Deepika Dixit & Vaibhav Kamra, Kasten
As teams start to onboard mission-critical applications into production, there’s a need to address day-2 concerns. Dealing with regulatory requirements, user error, ransomware and cluster upgrades - requires safeguarding of data and state. For this reason, Volume snapshots have been one of the most requested features from the storage team and now show up in the CSI spec. Snapshots alone, however, are often not enough. Application consistency, local storage, archival, retirement and compliance are just some of the other concerns. In this session, the speakers describe the challenges associated with implementing a robust data protection strategy in a containerized environment, open source tools available to help build this and a framework to evaluate what your environment needs. This talk targets DevOps teams looking to improve the reliability of their production environments.
To learn more: https://sched.co/GrYk
A
B
A
I'm
bap
of
camera,
I'm,
the
co-founder
and
CTO
at
Caston
I'm,
also
a
contributor
on
a
project
called
canister.
So
we'll
talk
a
little
bit
about
that
today,
as
well.
Previously
I've
done
databases
at
Microsoft
file
systems
and
storage
at
magnetics,
and
most
recently
I
was
doing
data
protection
at
EMC.
So
very
happy
to
talk
to
you
about
this
topic,
it's
very
near
and
dear
to
us.
At
Caston
we
do
data
management
for
cloud
native
applications,
so
data
protection
is
one
of
those
things
so
I
hope
to
make
this
relevant
for
you
guys.
A
So
what
are
we
going
to
talk
about
today?
So
the
first
half
of
the
talk
will
talk
about
where
the
data
is
when
it
comes
to
applications
and
kubernetes.
What
do
we
really
mean
by
stateful
applications?
Everybody
has
their
own
definition,
we'll
talk
about
what
data
protection
is.
You
know
just
set
the
problem
up
for
everyone
and
the
second
half.
We
will
talk
about
what
it
takes
to
get
this
right
in
kubernetes,
hopefully
give
a
blueprint
for
all
of
us
to
use
when
we're
thinking
about
it
in
our
environments.
A
So
quick
show
of
hands
who
here
is
running
stateful
applications
in
kubernetes
all
right,
and
so
by
this,
what
I
mean
is
you're
using
persistent
volumes,
you're
deploying
databases
in
kubernetes
you're,
using
data
services,
elasticsearch
Redis
a
whole
bunch
of
things
right,
and
let
me
ask
you
a
little
slightly
different
question:
who's
running
applications
that
store
data
outside
of
kubernetes,
so
you're
using
a
managed
service
cloud,
SQL,
Amazon
RDS,
something
like
that
right!
Thank
you.
So
that
is
a
good
segue
in
at
this
slide.
A
So
when
it
comes
to
applications
in
kubernetes
they're,
all
there
are
different
adoption
patterns
that
we
see
the
first
one
is
what
we
just
talked
about
so
they're
empty
hexagons
there.
Those
are
pods
right,
so
you've
got
pods
which
have
no
state
and
you've
got
pods
that
have
stayed
so.
The
first
pattern
is
that
all
the
data
services,
all
the
data,
is
in
kubernetes
you're,
either
using
volumes
or
using
a
database
like
Postgres
or
something
else.
A
The
second
one
is
you're
still
all
in
kubernetes,
but
your
data
services
and
your
application
are
logically
separate.
They're
deployed
managed
independently.
So
if
you
think
about
the
database
as
a
service
pattern,
the
pattern
where
people
are
using
operators
to
deploy
databases-
this
is
what
we
see
and
then
you've
got
applications
that
will
communicate
with
the
databases
or
the
data
services.
A
I
should
say
using
either
control
or
data
api's,
and
then
there's
the
third
one,
which
we
also
talked
about
where
your
your
all
your
data
service
is
outside
and
your
again
you're
using
control
api
is
to
provision
you
using
data.
Api
is
to
talk
to
those,
and
so
what
we're
going
to
talk
about
today
covers
all
of
this,
because
I
think
these
are
all
valid
patterns
when
you're
building
your
applications,
there
are
pros
and
cons
to
each
of
them,
but
what
we
will
talk
about
will
apply
to
everything.
A
So
what
is
data
protection?
Let's,
let's
set
that
up
at
a
very
high
level.
It
means
that
you
have
systems
in
place
to
recover
applications
and
the
data.
If
things
go
bad
and
there
are
a
bunch
of
reasons
or
you
would
need
this,
dealing
with
accidental
or
malicious
data
loss,
so
think
bugs
in
your
code
that
we
need
to
recover
from
you
got
hit
with
ransomware
infrastructure
failure
to
deal
with
application,
miss
configuration
this
is
the
I
have
a
typo
in
my
config
file,
and
suddenly
a
production
was
pointing
to
dev.
A
It
does
happen
and
then
regulatory
reasons
in
certain
environments
having
systems
in
place
like
this
is
a
requirement
and
when
you
think
about
when
we
think
about
data
protection
strategy,
there's
certain
elements,
certain
things
that
are
table
stakes
and
especially
in
these
environments
at
scale,
automation
around
our
backup
and
recovery
processes,
policies
in
place
that
will
take
care
of
scheduling
and
retirement
security.
So
you
want
to
make
sure
you
have
the
right
controls
in
place.
Who
has
access
to?
What
who
can
restore
Encryption
SLA
is
around
recovery.
A
So
how
long
this
thing
are
things
gonna
take
how
much
data
and
me
I,
I
might
lose.
This
is
all
important,
but
most
important
is
doing
all
of
this
at
scale,
so
we're
in
a
day
to
talk
over
here
day.
Two
in
kubernetes
is
all
about
doing
things
at
scale,
doing
things
in
a
repeatable
way,
not
having
manual
processes
and
also
handling
all
sorts
of
applications
that
are
going
to
show
up
in
our
cluster,
not
just
the
the
one
application
or
the
two
application
right
now.
A
Everything
I
think
I
said
right
now.
I
think
this
is
a
pretty
standard
part
of
any
operational
table,
I'm
sure
we're
all
aware
of
it
if
we're
running
our
environments
in
production,
but
when
it
comes
to
kubernetes
application,
applications
and
kubernetes
I
think
there's
some
misconceptions
out
there
that
we
should
just
talk
about,
because
this
is
something
that
keeps
coming
up
over
and
over
again.
So
the
first
one
is
I:
don't
have
any
stateful
applications
and
kubernetes.
So
this
is
obviously
changing.
A
I
mean
the
show
of
hands
shows
that,
but
also
when,
when
people
make
this
statement
or
when
we,
when
we
hear
this,
it's
really
a
description
of
where
the
data
lives
rather
than
the
application
itself.
The
application
has
stayed.
What
we're
saying
here
is
that
this
state
may
be
outside
of
kubernetes
and
that's
okay.
You
still
have
data,
that's
valuable
and
you
still
need
a
strategy
in
place.
A
Second,
one
is
my
data
stores
are
replicated
right.
They
I'm
using
a
cloud
native
database
I'm
using
X
or
Y,
and
so
why
do
I
need
something
like
this
now
replication
is,
is
great
for
resiliency.
It
can
help
help
you
with
infrastructure
failures,
but
when
it
comes
to
dealing
with
accidental
data
loss,
malicious
data
loss
configuration
issues.
You
still
need
a
complete
data
protection
strategy.
You
need
points
in
time,
backup
and
recovery.
A
So
that's
that's
what
what's
important
here
and
the
third
one
which
I
think
we'll
spend
a
little
bit
more
time
on,
is
my
underlying
infrastructure
I'm
using
a
cloud
service
I'm
using
a
storage
system
I'm
using
a
database.
It
already
has
things
like
backup
and
recovery.
So
why
do
I
need
something
else
so
for
this?
Let's,
let's
make
it
a
little
bit
visual,
because
I
think
that'll
help
us
and
we'll
keep
coming
back
to
this
one
later
as
well.
A
So,
let's
look
at
what
a
typical
cloud
native
application
looks
like
you
might
have
a
service
I'm
gonna
see
if
this
shows
up
I,
think
it's
big
enough,
so
you
might
have
let's
say
a
front-end
service,
that's
behind
an
ingress
over
there.
Let's
see
it
has
a
couple
of
other
back-end
micro
services
micro
service.
Architectures
are
what
you'll
see
a
lot
in
these
environments?
Let's
say
they're
stateful,
so
the
first
one
is
using
a
volume,
it's
putting
unstructured
data
on
there
and
you've
got
another
one.
A
That's
using
it's
using
a
cloud
database
for
some
structured
data,
so
this
is
a
pretty
typical
pattern
for
cloud
native
applications.
Now,
when
you
think
about
the
operational
unit
here,
what
do
you
care
about?
What
do
we
deploy?
What
do
we
want
to
manage?
What
do
we
want
to
backup
and
eventually,
if
we
have
an
issue,
what
do
we
want
to
recover?
It
is
this
entire
thing:
it
is
the
application
doing
that
at
the
infrastructure
level
becomes
really
hard.
A
So
when
I'm
sitting
at,
for
example,
the
storage
layer
over
there
or
within
Amazon
RDS
I've
got
a
backup
policy
running
over
there.
It
has
no
awareness
about
the
entire
application.
It
has
no
awareness
of
what
the
dependencies
are.
Are
there
micro
services
that
I
need
ordering
against
across
do
I
need
to
collect
backup
all
of
this
at
one
time
together,
there's
kubernetes
I
mean
how
is
all
of
this
gonna
tie
into
kubernetes
there's
no
real
awareness
of
that
layer,
so
it
becomes
really
hard
to
do
this
at
just
the
infrastructure
layer.
A
So
now
that
we've
set
the
problem
up
a
bit,
let's
talk
about
implementation
and
so
implementation.
There
are
a
variety
of
things
to
talk
about
what
I'm,
trying
to
what
I'm
gonna
try
and
do
is
offer
a
blueprint
of
things
to
think
about
maybe
four
or
five,
whenever
you're
thinking
about
data
protection,
so
I
would
I
would
hope
that
people
can
take
this
back,
look
at
their
own
environments,
their
own
applications
and
consider
each
topic
in
here
and
say:
okay,
what
am
I
gonna
do
about
a
B
and
C,
so
we're
gonna
go
through
those.
A
The
first
one
is
again:
I've
got
that
picture
of
that
application
of
there.
So
what
are
we
gonna
do
about
the
kubernetes
resources?
The
application
configuration
I
might
have
other
information
I
need
to
recover.
This
application,
I
might
have
shared
secrets
in
my
environment.
I
might
I
might
need
pipeline
information.
How
was
this
deployed?
What
environment
was
it
running
in?
So
we
need
to
think
about
how
we're
gonna
capture
that
information
when
we're
trying
to
back
up
an
application
when
it
comes
to
the
kubernetes
resources.
A
There
are
a
few
techniques
you
can
go
to
the
API
server
and
that
really
gives
you
what
the
runtime
state
is.
If
it's
all
infrastructures
code,
you
can
go
and
you
can
get
a
source
code
reference.
You
could
go
refer
to
your
repository.
If
you're
using
something
like
helm,
you
can
protect
the
helm
chart,
you
can
bring
that
in
or
even
just
store
a
reference,
but
that's
something
we
do
need
to
worry
about,
because
applications
are
so
dynamic
that
configurations
change
quite
quite
often
in
these
environments.
A
So
if
you're
backing
up
data,
you
want
to
make
sure
that
you
get
the
configuration
that
actually
matches
that
data.
If
you
ever
want
to
recover
this,
the
next
one
is
what
we
think
about
most
often
is:
what
are
we
going
to
do
about
the
persistent
data,
and
this
really
depends
on
where
the
data
is
so
if
the
data
is
in
volumes,
we've
got
a
few
options
over
there.
We
can.
If
the
storage
system
supports
it,
we
can
leverage
volume
snapshots.
A
A
A
If
not,
you
can
use
application
level
tools
so
think
of
using
this
would
be,
for
example,
using
my
sequel
dump,
with
my
sequel
or
a
Postgres
tool,
or
a
tool
using
both
of
these
in
a
combination
is
also
very
powerful,
especially
when
you
start
to
think
of
large
data
sets
and
then,
if
you're,
dealing
with
managed
services.
Similarly,
you're
going
to
you
can
use
the
application
level
tools.
I
can
use
my
sequel
dump
to
extract
it
out
of
RDS
if
I
want
to
the
managed
service
may
also
offer
API.
A
I
might
want
to
order
the
see
the
steps,
how
steps
run
across
the
application
I
might
need
to
quiet
the
application
I
might
have
pre
and
post
steps
to
run
you're
dealing
with
kubernetes
over
here.
Remember
we
talked
about
bringing
back
the
entire
application.
You
need
to
worry
about.
How
are
you
going
to
orchestrate
this
in
kubernetes?
What
resources
need
to
get
created
before?
How
do
you
get
volumes
from
snapshots?
A
Stateful
sets
from
those
volumes,
so
let's
use
an
example
to
actually
illustrate
what
I
mean
by
workflow
orchestration
and
why
you
need
to
think
about
it.
So
if
you
look
at
the
standard
recovery
playbook
for
Postgres,
for
example,
if
I'm
getting
Postgres
back
from
physical
backups,
if
one
stress
was
running
in
a
VM,
I
would
go
in
there,
I
would
shut
down
the
post
curse,
server,
I
would
restore
the
database
and
the
the
log
files
I
had
run
a
recovery
step
and
then,
finally,
when
I'm
done,
I
start
post
quest
and
that
works
great.
A
What
happens
in
a
containerized
environment
when
I'm
running
in
coburn
Eddy's
as
soon
as
you,
shutdown
Postgres?
If
you
get
to
that
container,
the
container
is
going
to
shut
down
when
the
container
shuts
down
the
pod
goes
away.
You
lose
access
to
the
database,
the
files,
all
the
other
steps
in
the
workflow.
You
can't
do
anymore,
so
you
need
to
worry
about
now.
Okay,
what
am
I
gonna
do
with
my
recovery
playbook.
What?
A
What
is
the
workflow
gonna
look
like
in
kubernetes,
so
one
option
to
do
to
use
here
is
using
init
containers,
but
in
containers
it
becomes
really
brittle.
Because
what
happens?
Is
you
end
up
embedding
script
embedded
scripts
in
your
application
in
a
sidecar
container?
A
better
approach,
for
example,
is
to
do
this,
and
this
is
how
you
would
orchestrate
this
on
kubernetes.
You
can
scale
down
the
original
Postgres
instance.
You
can
start
another
pod,
and
this
is
a
container
that
has
all
your
tools
has
Postgres
in
there,
but
you
can
control
the
lifetime
of
it.
A
You
can
attach
those
volumes
that
have
all
the
data.
Then
you
can
go
and
run
the
rest
of
the
steps
of
the
workflow
restore
the
database
files
Postgres
recovery.
Finally,
when
you're
done
you
shut
down
and
then
bring
scale
up
the
original
Postgres
application
again,
those
volumes
go,
get
attached,
you're
good
to
go,
so
this
is
kind
of
what
I
mean
by
thinking
about
orchestration
in
these
environments.
A
Finally,
I
think
considering,
where
you're
gonna
put
your
backups,
backup,
storage
becomes
really
important
and
what
format
you're
gonna
put
in
there.
That's
something
to
take
up
think
about
object.
Storage
tends
to
be
a
really
good
choice.
It's
scalable,
it's
low
cost,
has
many
tiering
options.
Has
the
right
controls
for
security,
so
you
can
say
who
has
access
to
what
encryption
has
really
good
integration
over
there?
So
I
think
this
is
a
great
option
to
use
if
you're
leveraging
data
service
snapshot.
A
So
if
you're
leveraging
volume
snapshots,
if
you're
leveraging
RDS
snapshots,
then
one
of
the
things
to
think
about
is
water.
What
are
the
requirements
or
what
are
the
guarantees?
The
underlying
system
is
going
to
give
me
think
about
a
storage
appliance
that
you
take.
A
snapshot
on
those
snapshots
are
not
really
backups,
because
when
the
appliance
goes
away,
the
snapshots
are
gone.
If
you've
taken
snapshots
in
the
cloud,
you
might
get
better
guarantees.
So
this
is
something
to
consider.
Portability
is
another
one,
so
think
about
what
environment
you're
gonna
have
to
recover.
A
These
applications
in
is
the
same
infrastructure
gonna
be
there.
If
it
is,
then
you
might
be
able
to
use
a
snapshot.
That's
tied
to
that
infrastructure,
if
not,
you
might
have
to
make
things
portable,
for
example,
take
a
volume
snapshot
and
then
run
a
file
system
backup
out
of
that
to
make
the
extracted
data.
A
Lastly,
I
mean
obvious
stuff:
I.
Think
encryption
is
something
that,
at
this
point
everybody
does
think
about,
but
what
we
find
is
that
security
in
terms
of
who
has
access
to
what
especially
you're
talking
about
lots
of
applications
getting
backed
up
it's
as
good
as
having
Apple
access
to
the
application.
So
you
have
to
worry
about
who
has
access
to
the
data
who
is
allowed
to
restore,
so
we
need
to
write,
have
make
sure
you
have
the
right
controls
in
place.
A
B
Thank
You
Abu
so
now
that
we've
seen
that
what
all
goes
into
implementing
or
even
thinking
about
a
data
protection
strategy,
let's
look
at
a
demo
app
and
see
how
we
would
go
about
with
the
same
so
your
if
you
can
see
we
have
a
simple
application.
It's
a
stateful
application.
It
is
deployment
with
one
replica
set
with
one
replicas,
so
it
has
one
pod
and
we
have
given
it
three
persistent
volumes.
B
So
it's
using
my
sequel
for
one
of
the
volumes
and
then
it
has
some
config
data
in
the
other
one
and
it
has
an
unstructured
file
data.
So
it
is
a
simple,
typical
stateful
application
that
anyone
could
run
in
the
aquaponics
cluster
or
let's
go
to
the
demo
and
see
what
this
would
look
like
in
our
cluster.
So
I
think
everybody
can
see
that
we've
pre-recorded
it
to
speed
things
up.
So
we
can
get
the
cluster
and
see
what
it
looks
like
actually
in
production.
B
Oh
okay,
I
just
did
a
get
on
deployment
pods
in
service
and
the
PVC.
So
if
you
look,
there
is
one
deployment
picture
gallery
and
we
have
one
pod
running
with
it,
and
apart
from
that,
we
have
three
volumes.
So
we
have
this
my
sequel
and
then
we
have
the
config,
and
then
we
have
the
pictures.
We
can
also
go
and
look
at
what
the
UI
looks
like
for
this.
That
will
give
us
a
better
idea
of
what
kind
of
an
app
we
are
dealing
with.
B
So,
hey,
look,
it's
Pippy
and
we
have
12
pictures
over
here.
It's
a
simple
picture
gallery:
VP's
crying
he'll
get
back
to
it
later,
and
so
it's
a
typical
application,
which
is
using
some
persistent
volumes,
and
now
we
need
to
backup
and
restore
an
application
like
this.
So
for
that
we're
going
to
introduce
a
tool,
our
tool,
urs
canister,
it's
an
open
source
project.
What
it
does
is
it's
going
to
help
you
plan
your
data
protection
workflows
and
it's
going
to
use
kubernetes
custom
resources
for
that.
So
that
gives
you
a
single
pane
of
control.
B
B
Have
your
local
file
system,
my
sequel,
Postgres
canister
has
a
lot
of
primitives
for
data
capture
and
the
most
important
thing
that
even
mentioned
before
is
we
all
have
a
data
protection
plan,
but
we
need
somebody
to
orchestrate
all
the
workflow
for
us
like
we
saw
in
the
Postgres
example,
there's
a
bunch
of
extra
steps
that
are
needed
to
have
a
complete
data
protection
story.
So
canister
will
help
you
orchestrate
that,
so
you
just
tell
it
the
tell
canister
what
workflow
needs
to
be
done.
It
will
execute
all
the
steps,
all
the
queuing.
B
Everything
will
be
taken
care
of
for
you.
So
let's
get
back
to
the
app
and
see
what
a
backup
boxer
would
look
like.
So
if
you
remember
our
app
has
a
single
pod
and
it
has
a
couple
of
volumes.
So
a
typical
backup
workflow
would
be
something
like
this.
We
will
first
go
and
discover
the
volumes.
Then
we
need
to
snapshot.
The
PVC
is
on
the
underlying
volumes
and
then,
lastly,
we
need
to
store
the
snapshot
information
somewhere.
So
it
could
be
some
backup
storage,
which
is
local
or
it
could
be
in
cloud
I
know.
B
How
does
canister
help
you
do
this,
so
in
canister
we
define
something
called
as
the
blueprint
which
is
the
kubernetes
custom
resource.
So
let's
start
writing
the
blueprint.
Since
it's
a
kubernetes
resource,
we
have
the
typical
API
version
and
kind
in
there,
and
next
up
we
are
going
to
define
something
called
as
actions,
so
we're
going
to
tell
it
that
you
are
going
to
this
blueprint
has
a
bunch
of
actions.
The
first
one
we
are
going
to
define
is
backup,
so
for
backup
is
an
action
and
it's
working
on
type
deployment.
B
So
this
blueprint
is
telling
what
needs
to
be
done
for
this
action
and
it's
working
on
type
deployment
say
instead
of
deployment
we
could
have
stateful
sets
or
any
other
kubernetes
resource.
This
particular
blueprint
is
working
on
our
picture
gallery,
which
is
a
deployment,
which
is
why
the
type
here
is
deployment.
So
next
up
is
output,
artifacts
or
if
you
see
the
third
step,
is
you
need
to
push
the
backup
in
for
somewhere
into
backup
storage?
So
the
backup
info
in
there
is
location
details
where
you
want.
B
They
have
to
go
and
the
last,
but
the
most
important
is
phases.
So
these
are
the
steps
in
the
workflow
that
needs
to
be
captured
for
a
backup
action.
So
here
we
are
using
an
inbuilt
canister
function,
called
create
volume,
snapshot,
it's
a
canister
primitive
which
will
help
you
discover
the
PVCs,
then
it
will
snapshot
the
underlying
volumes
and,
lastly,
it
will
push
the
backup
in
for
using
the
backup
info
at
effects
that
we
have
mentioned
before.
B
So
this
is
how
a
backup
blueprint
would
look
like
now.
Let's
continue
writing
this
blueprint,
but
first,
let's
look
at
restore
workflow,
so
assume
we
have
a
backup
everything
in
place.
Now
something
went
wrong,
so
we
wanna
do
a
restore
so
for
our
application.
The
restore
steps
would
probably
be
look
like.
First,
we
need
to
scale
down
the
application.
Then
we
should
delete
the
existing
PVCs.
B
Then
we
should
create
the
new
PVCs
from
the
snapshot
that
we
have
taken
already
and
then
finally
scale
the
application
back
up.
So
let's
continue
writing
the
blueprint,
and
this
time
we
will
define
a
new
action
restore.
So
again,
the
action
is,
restore
it's
working
on
type
deployment
and
here
instead
of
output
artifacts
we
have
input
artifacts.
So
since
the
restore
needs
information
about
the
snapshot
where
it's
coming
from
where
it's
stored,
all
the
information
goes
in
over
here,
and
the
next
step
is
phases
now
back
up
a
simple
in
restore.
B
We
have
a
couple
more
phases.
So,
first
up
we
are
using
the
cannister
and
will
function
to
scale
the
workload
down.
Next
up,
we
are
doing
create
volumes
from
snapshot,
which
is
another
cannister
inbuilt
function.
You
are,
as
you
can
see,
it's
using
the
backup
info
to
get
information
for
where
the
volume
snapshot
lies
in
and,
lastly,
we
are
going
to
scale
the
workload
back
up
so
I'm
going
to
pause
here
for
a
minute
for
you
to
take
in
all
the
information.
It's
a
blue,
it's
a
blue
print
is
again
a
custom
resource
in
kubernetes.
B
B
So,
let's
go
back
to
the
demo
I'm
using
something
called,
as
can
cuddle,
which
is
the
canister
tool
which
is
helping
us,
create
a
custom
resource,
called
action
set
and
action
set.
Helps
you
execute
the
blueprint
that
we
just
defined
so
again,
it's
a
kubernetes
custom
resource.
You
could
have
all
this
information
in
a
Yaman
if
you're
more
comfortable
with
that,
and
you
scoop
cuddle
applied
to
do
that.
We're
just
using
can't
cuddle
because
it's
easier
to
create
for
clear
to
CR
through
this,
so
we
are
doing
a
create
on
action
set.
B
The
action
type
here
is
backup,
since
we
want
to
execute
the
backup
part
first
and
then
we
are
telling
that
we
only
use
the
blueprint
that
we
just
defined.
So
we
are
telling
it
blueprint
as
picture
gallery,
and
we
are
telling
that
do
this
on
the
deployment
picture
gallery,
which
is
in
the
canister
namespace.
So
we
are
giving
can
cuddle
all
the
information
it
needs
to
start
executing.
B
The
blueprint
last
up
is
profile,
which
is
the
backup
information
that
remain
so
it
has
all
the
location
details
that
it
needs
and
as
simple
as
that,
an
action
set
was
created
by
can
cuddle.
So
what
can
cuddle
did?
Is
it
created
a
custom
resource
it
validated
it
and
then
it
applied
to
to
kubernetes
if
you
are
como
comfortable
riding
a
Yamaha
also
perfectly
fine
again
action
set
is
a
kubernetes
custom
resource,
so
we
can
do
a
cube,
cuddle
describe
and
get
the
information
for
the
action
set.
So
let's
do
that.
B
So
I'm
gonna
pause.
If
you
see
at
the
bottom
of
the
page,
you
can
see
a
bunch
of
events
that
this
action
said
went
through.
So
first
stop
was
started.
It
started
to
execute
the
backup
or
action
next
up
since
the
backup
workflow,
if
you
remember
just
had
one
phase
so
that
was
the
backing
up
the
volume,
so
it
started
executing
that
phase
and
then
once
that
was
completed,
marked
the
status
as
complete
for
this
particular
action
set.
B
B
B
So
let's
go
back
to
our
demo
in
the
terminal
and
again
we
are
using
canned
cuddles
same
thing.
It
will
create
an
action
set,
but
this
time
the
action
that
we
are
asking
it
to
perform
is
restore
and
the
input
parameters
are
pretty
simple.
Since
we
already
took
a
backup,
we
are
going
to
tell
it
that
create
this
action
set,
restore
from
and
the
backup
that
we
just
created.
So
as
I
mentioned
before,
the
backup
has
all
the
information
we
need.
B
So
the
restore
action
is
pretty
simple:
it
just
gets
the
information
from
the
backup
and
the
action
set
is
created
and
again
can
Cardo
created
it.
For
us,
we
didn't
have
to
do
anything,
but
if
you're
comfortable
with
the
amyl,
you
can
do
that
as
well.
Let's
do
a
cube
car
will
describe
on
this
as
well
and
see
if
the
execution
of
this
restore
has
begun.
B
So,
as
you
can
see,
the
execution
is
still
in
progress.
It
had
a
couple
of
phases.
The
first
phase
is
done,
which
is
shutting
down
the
board
and
then
now
it's
in
the
volumes
face,
so
we
are
going
to
do
a
weight
on
the
volumes
to
see
if
they
are
bound,
so
the
my
sequel
and
the
config
file,
if
you
can
see
they
are
new
and
the
pics
is
coming
up
so
once
that's
bound
I,
the
action
set
will
go
into
the
next
phase,
which
were
scaling
up
the
workload
back
up.
So
we're
gonna.
B
Look
at
the
action
set
again
to
see
where
it's
at,
as
you
can
see
the
first
and
the
second
phase
is
completed.
We
are
in
the
third
phase
of
bringing
the
pods
up.
So
let's
just
look
at
the
pod
status
and
see
if
it's
up
and
the
pod
seems
to
be
running.
So
we
can
just
go
back
and
see
check
on
our
deployments,
see
if
everything
is
in
place.
The
way
we
expected
it
to
be
like
the
deployment,
the
pods
and
the
volumes.
B
B
So
back
to
the
blueprint,
so
that
is
how
a
canister
helps
you
execute
your
workflows
for
you,
that's
what
we
mean
by
orchestration.
So
we
define
the
blueprint
it
works
on
a
type
deployment.
We
gave
it
all
the
phases
it
needed,
and
then
we
use
canister
to
help
orchestrate
that
now
this
blueprint
is
pretty
generic.
As
you
can
see,
it
is
working
on
PVCs.
So
the
backup
phase
is
pretty
simple.
The
restore
phase
is
comparatively
simple.
You
can
customize
this
blueprint
as
per
your
need,
since
this
is
working
on
PVCs
on
a
deployment.
B
I
can
actually
reuse
it
for
another
example,
but
I
can
also
customize
it
to
my
application.
So
maybe
my
application
needs
some
pre,
pre
and
post
hooks
or
maybe
I
want
to
use
a
typical
third-party
tool,
like
my
sequel
dump
or
something
in
the
middle
I
can
add
that
as
well,
a
blueprints
are
meant
to
be
like
shareable
and
you
can
customize
them
as
per
your
need.
You
can
keep
them
as
simple
as
you
want,
or
you
can
keep
them
as
a
kind
of
detail
as
you
want,
and
we
have
a
few
on
the
website.
B
So
you
can
go
check
them
out
as
well.
I'll
get
I'll
show
the
link
later,
so
here's
the
link
for
canister
the
first
one,
which
we
just
discussed,
it's
an
open
source
project.
So
the
picture
gallery
app
that
I
just
discussed
is
on
there
with
all
the
steps
that
you
need
to
execute.
It
there's
also
a
bunch
of
other
examples.
Like
my
sequel
and
Postgres,
you
can
go,
go
and
look
the
next.
If
you
want
an
enterprise
version
of
this,
we
have
a
website
on
cashton.
You
can
go
check
that
out.
B
Apart
from
that,
as
I
can
see
that
the
community
is
being
very
active,
especially
for
stateful
applications
and
backup
and
restore
there's
a
bunch
of
other
projects
that
are
worthy
of
mention
like
art,
then
there's
reshift
or
there's
also
kubernetes
snapshot
this,
and
if
you
look
at
the
last
link,
the
community
has
created
an
exhaustive
list
of
projects
that
are
there.
So
all
things
stateful
is
there
at
that
link
and
we
have
uploaded
the
slides.
So
you
can
go
and
have
a
look
at
them
at
your
ease.
B
Now
what
we
just
discussed
was
definitely
important,
but
there
are
some
other
topics
that
we
couldn't
get
to
today,
which
are
equally
important
when
you're
thinking
about
a
data
protection
strategy.
So
you
need
to
think
about
cataloguing
your
snapshots
and
your
backup.
You
also
need
to
think
about
scheduling
and
retirement
of
these
as
well
as
you
need
to
be
able
to
validate
your
snapshots,
because
a
snapshot
is
not
useful
until
you
have
restored
from
it.
So
you
need
to
have
a
plan
for
validation
and
staying
of
these
snapshots.
B
And,
lastly,
all
this
you
need
to
be
able
to
integrate
into
CI
CD.
We
had
a
talk
yesterday
on
this.
Hopefully
you
guys
made
to
it.
But
if
you
didn't
look
for
the
slides,
they
will
be
up
and
you
will
get
a
more
detailed
version
of
that.
That's
it
from
us
for
now
I
think
we
will
take
questions,
but
if
you
want
to
discuss
something
offline
or
in
detail,
we
have
a
booth.
So
please
come
check
us
out
over
there
and
I.
Think
I'll
open
the
floor
for
questions.
Thank.
A
A
Are
you
talking
about
in
this
particular
example
right?
So
the
question
was:
where
does
we
talked
about
scheduling?
Where
does
the
schedule
live
when
it
comes
to
canister
blueprint,
so
canister
itself
does
not
have
scheduling.
You
can
build
a
schedule
around
it,
whether
you
want
to
use
cron
jobs
or
or
any
other
tool
yeah
exactly
and
the
other
one
I
was
gonna
say.
Is
that
we're
also
seeing
a
lot
of
people
integrate
this
into
pipeline,
so
CI
CD
so
use
that
as
a
scheduler
as
well?
If
that,
if
that's
a
better
option.
B
A
Something
we're
doing
as
well.
I
mean
we,
you
know
you
can
ping
us
on
slack
over
there
if
you
want,
if
you're
looking
for
something
specific,
what
we
are
planning
to
do
with
canister
is
really
and
what
I
should
have
mentioned
all
year
is
the
community
is
doing
a
lot
of
work
with
volume,
snapshots
and
CSI,
and
so
our
goal
is
to
make
sure
we
support
that
right.
Now,
that's
not
there's
still
alpha.
A
Okay,
so
thank
you.
So
the
question
was:
what's
the
feasibility
of
orchestrating
a
cluster
restore
with
a
tool
like
canister,
so
we've
definitely
actually
done
this
in
practice.
Deepika
mentioned
k10,
something
that
we
work
on,
which
is
more
enterprise,
but
that
actually
uses
canister
under
the
covers
and
is
something
that
we
do
use
in
practice.
A
A
The
question
was
on
the
back
end:
you
should
be
able
to
integrate
with
any
storage
provider
if
it's
a
PVC.
So
that's
yes
and
that's
correct
at
this
point
of
time.
We
do
leverage
volume
snapshot
functionality,
so
it
really
and
since
that's
not
surfaced
as
a
standard
API.
Yet
the
API
is
in
alpha
and
the
CSI
providers
drivers
are
getting
to
one.
Oh.
A
So
what
that
means
right
now
is
that
you,
every
provider
you
want
to
support,
you
need
to
go
and
understand
the
infrastructure,
go
call
those
provider
api's,
so
we're
adding
support
for
we
already
have
AWS
they're,
Google,
IBM
safe
and
those
should
start
to
show
up
soon,
but
really
our
goal.
A
roadmap
is
to
really
align
with
the
CSI
and
the
volume
snapshot.
Api
is
from
kubernetes.
A
Question
was:
does
open
EBS
have
the
features
that
you
need
for
this
I
believe
it
does
I
believe
it
does
have
a
snapshot.
Api
I
do
not
know
if
they
have
a
CSI
provider,
but
I
know.
There's
someone
here
who
can
comment
on
that
and
tell
us:
do
you
have
a
CSI
provider
for
open
EBS,
it's
coming
with
snapshots
all
right
so
soon.
A
So
the
question
was:
do
you
support
incremental
backups
to
answer
that
there,
a
variety
of
cases
where
we
do
support
incremental
backups,
so
canister
allows
you
to
hook
in
at
the
application
level
when
it
comes
to
databases?
And
if
so,
if
you
look
at
the
examples
on
the
website
when
it
comes
to
something
like
Postgres
it,
we
do
support
incremental
backups
over
there
and
in
fact
we
offer
point
in
time
recovery
as
well,
because
we're
using
physical,
backups
and
walls.
A
When
you're
dealing
with
volume
snapshots,
often
the
volume
snapshot,
the
storage
provider
will
do
incremental
backups.
So
we
get
some
of
it
over
there
and
then.
Finally,
when
we're
dealing
with
storage
providers
that
don't
have
volume
incremental
support,
we
use
filesystem.
We
use
a
tool
called
Rustique
which
gives
us
deduplication,
and
so
you
can
build
an
incremental
story
over
there.