►
From YouTube: Webinar: Application Snapshots Using Consistency Groups
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Running data-heavy applications such as SQL, NoSQL and BigData is a challenging task. What is even more challenging is the lifecycle management of these applications. One of the important life-cycle operations is taking consistent snapshots of data-heavy applications for data protection and backing up these application snapshots up to a different media, eg. Cloud. What does it really take to snapshot an entire application? In this webinar, Ravikumar Alluboyina will introduce the concept of Volume Consistency Groups. We will take a look at why a Volume Consistency Group is a must-have for retaining data consistency.
A
Is
CN
CF,
webinar
applications
snapshots
using
consistency,
groups
on
Jorge,
Castro,
Community,
Manager,
VMware
and
a
cloud
native
ambassador
I'll
be
moderating
today's
webinar.
We
would
like
to
welcome
our
guest
presenter
today,
Ravi
a
Lubo
ena
senior
architect
at
Robin
dot,
IO
a
few
housekeeping
items
before
we
get
started
during
the
webinar
you're,
not
able
to
talk
as
an
attendee
there's
a
Q&A
box
at
the
bottom
of
your
screen
and
the
zoom
UI.
A
Please
feel
free
to
drop
your
questions
in
there
and
we'll
get
to
as
many
as
we
can
at
the
end
of
the
webinar.
This
is
an
official
webinar
of
the
CN
CF
and
I
such
a
subject
to
the
CN
CF
code
of
conduct.
Please
do
not
add
anything
to
the
chat
or
questions
that
would
be
in
violation
of
that
code
of
conduct.
Basically,
please
be
respectful
of
all
of
your
fellow
participants
and
presenters.
With
that
I'll
hand
it
over
to
Ravi
to
kick
off
today's
presentation.
B
B
With
that,
let's
talk
about
kuben
breeze
and
the
application
landscape
cover.
Today's
has
become
the
de
facto
standard
for
running
stateless
applications.
It's
pushed
aside
every
other
Orchestrator
out
there.
If
you
want
to
run
an
engine
xform,
the
default
option
is
kubernetes
today
and
all
of
these
applications
beat
nodejs
of
java
or
engineers
wordpress.
All
of
these
applications
roughly
qualified
as
web
applications.
B
B
B
B
So
this
is
a
classification
from
the
applications
side.
But
if
you
are
building
a
platform,
if
you're
open,
it
is
orchestrated,
there
is
a
different
division,
so
we
call
them
stateless
applications.
We
call
all
of
these
stateful
applications.
What
we
are
going
to
dive
into
is
deploying
and
orchestrating
stateful
applications,
especially
taking
snapshots.
B
Let's
go
briefly
over
these
two
segments
of
sequel
databases
and
no
sequel
databases.
Some
of
the
attributes
of
sequel
databases
are
primarily
acid
compliance,
and
these
are
monolithic
architectures,
primarily
because
these
are
built
like
decades
ago,
where
bare
metal
is
the
predominant
platform
where
this
his
applications
were
deployed,
and
these
are
row
oriented
databases,
they
primarily
work.
Loaded
is
transactional
workload
and
they
have
a
standardized
sequence,
so
very,
very
well
known
datastore,
whereas
the
no
sequel
databases
have
made
some
compromises
to
address
certain
workloads.
There
is
a
tunable
consistency.
There
is
stable
durability,
guarantees.
B
These
are
scalar
architectures
to
average,
to
leverage
commodity
hardware
and,
like
I,
said
they
target
different
workloads,
some
of
them
target,
even
least
evaluate
some
of
them
in
data
stores.
They
serve
both
well
OLAP
and
OLTP.
They
sell
both
transactional
load
as
well
as
analytical
lower
and
they
have
non-standard
client
interfaces.
Some
of
them
do
give
out
sequel
semantics,
but
they
have
non-standard
client
interfaces
like
sequel,
Modi
decline
or
various
like.
B
So
this
is
a
very,
very
high
level
over
on
the
distinction
between
sequel
and
Oracle
databases.
So
this
is
important
in
understanding
or
designing.
The
parameters
that
Lenovo
luster
on
these
applications
and
commodities-
let's
talk
about
the
standard
database
deployment
models,
so
this
is
our
code.
This
application
assume
that
it
is
running
in
a
container.
We
are
talking
about
who-ville
easier,
so
what
is
required
for
running
Postgres
on
kubernetes?
Obviously,
you
need
a
data
volume
which
is
provisioned
through
a
CSI,
and
that
is
coming
from
a
storage
tab.
Let's
call
it
data
volume.
B
B
B
So
these
are
the
recommendations
for
performance
say
if
you
want
better
performance,
you
you
follow
these
recommendations
of
having
two
volumes
and
on
top
of
that,
Cassandra
is
a
little
complicated
because
it's
a
distributed
architecture,
so
we
have
more
than
one
pods
running
and
each
part
consuming
more
than
one
volume
of
different
iya
patterns.
Of
course,
this
is
a
consistent
hashing
scheme
for
Sandra.
So
what
we
have
established
here
is
on
a
standard
sequel
database.
B
It's
not
just
a
container
and
storage.
There
is
more
to
it,
so
you
really
have
to
understand
what
the
application
is
doing
and
allocate
storage.
Accordingly,
there
is
a
data
volume
on
there's
a
while,
but
right
ahead
long
when
it
comes
to
Cassandra,
it's
even
more
complicated
because
there
are
multiple
containers
in
play
and
multiple
volumes
in
so
with
this
understanding,
let's
see
the
actual
transaction
flow.
B
What
happens
when
a
user
issues,
a
sequence
statement
assume
that
this
sees
the
data
volume
so
which
is
where
the
actual
final
data
resides,
which
is
one
volume,
and
this
is
the
wall.
This
is
the
right
ahead.
Log
value.
These
are
two
separate
volumes
coming
from
two
different,
let's
say
disks
now,
let's
say
user
were
to
issue
a
sequel
statement.
A
sequel
statement
could
be
auto
commit
or
it
could
be
a
series
of
statements
with
a
star
transaction
and
I've.
B
B
B
Let's
say,
for
example,
there
is
an
update
on
a
key
text.
We
are
changing
a
value
to
20
so
that
X
equal
to
20
is
written
in
the
bar,
not
exactly
in
that
format.
But
roughly
the
idea
is
to
capture
the
changes
in
the
transaction
and
we
make
more
more
changes
to
the
other
to
the
database,
and
then
we
do
our
commit
here.
B
Essentially,
what
it
means
is
we
take
all
the
changes
and
apply
to
the
actual
data
blocks,
which
means
we
are
writing
the
I/os.
You
take
me
I
use
from
one
volume,
compacting
them
and
writing
to
our
different
volume.
That's,
that
is
where
the
value
of
x
equals
20
is
applied
to
a
date
ago.
So
this
is
the
standard
flow
in
a
in
a
sequel
database
in
a
no
sequel
database.
B
It's
slightly
more
challenging
because
there's
a
star
for
the
transaction
because
they
some
of
the
you
know,
sequel
databases
or
distributed
databases,
do
one-off
or
transactional
semantics.
They
offer
sewer
transactions.
Matters
same
thing
apply
the
transaction,
the
primary
records
it
admits,
commit
log
and
the
commit
log.
The
transfer
to
the
secondary
sources
or
the
replicas
replicas
are
registered
least
changes
and
they
emit
there's
a
flush
that
happens.
B
So
miss
we
have
seen
that
the
changes
are
applied
to
the
transaction
log
first
and
then
periodically.
They
get
rush
to
the
date
of
loss,
and
why
is
that?
Fundamentally,
there
are
some
design
assumptions
made
either
for
performance
or
durability
of
consistency,
and
what
of
us
design
isn't
on
the
infrastructure
side.
The
design
assumption
is
that
the
transaction
log
and
the
data
files
will
go
to
separate
discs,
and
why
is
that?
B
B
Wall
is
an
append-only
workload
to
leverage
the
sequential
axis
of
a
swindle
and
crash.
Consistent
C
is
assumed
and
design
into
databases
at
any
point
in
time.
If
you
abruptly
shut
down
the
server
on
which
a
database
is
running,
it
will
come
back
up
online,
so
that
is
assumed
in
the
design,
and
that
is
a
that
is
one
of
the
reasons
why
we
have
read
on
undo
logs
in
databases
and
no
sequel,
databases
which
are
distributed.
Databases
assume
that
the
partitions,
which
are
those
independent,
individual
containers
they
go
on
different
physical,
hard
walls.
B
So
these
are
some
of
the
design
assumptions
on
the
infrastructure
side
of
it.
So
in
a
database
designer
regime
that
I
will
have
this
set
from
the
infrastructure,
and
then
I
will
do
some
ie
optimizations,
which
is
Val,
try
to
hide
log,
which
is
an
appallingly
worked,
or
we
call
it
undo.
/
redo
log
in
different
database
schemes.
It's
an
append-only
workload,
it's
a
persistent
circularbuffer.
It
could
be
one
or
more
buffers
residing
on
the
disk.
B
B
B
So
what
are
we
coming
in?
So
we
talked
about
the
assumptions
made
on
the
infrastructure
side.
We
talked
about
some
I
of
what
optimizations
or
the
sequencer.
This
is.
We
are
barely
scratching
the
surface
here
with
the
iPod
optimizations
I
am
talking
in
terms
or
my
goal
is
to
is
to
make
you
understand
what
is
the
infrastructure
requirement
just
for
this
X
log?
B
B
B
It
is
a
volume
that
is
provided
to
your
database
and
all
of
these
volume
management
options
beats
an
or
LVM
they
have
some
sort
of
snapshot
abilities,
let's
say
for
to
use
those
built-in
snapshotting
capabilities
and
snapshot
these
volumes,
the
data
volume
above
so
first
our
we
take
a
snapshot
which
is
we
create
a
snapshot
of
these
two
volumes.
Second,
our
weekly
and
third
of
you
can
snapshot
what
is
wrong
with
this
approach
or
more
than
what
is
wrong.
How
are
we
taking
these
snapshots?
B
The
loop
one
of
the
simple
strategies
to
take
the
snapshot
when
multiple
volumes
are
involved
is
to
write
up
in
yes,
like
that
rule,
nothing
about
three
lines:
go
for
each
one.
You
take
its
natural
one,
what's
wrong
with
this,
nothing
so
far,
but
when
it
comes
to
no
sequel,
databases,
so
again,
the
same
loop,
loop
or
loop
for
each
container
snapshot
each
volume
in
that
container.
B
B
B
B
B
Depending
on
how
the
loop
is
implemented.
It
could
be
other
way
as
one
we
take.
The
snapshot
of
the
data
and
the
process
gets
scheduled
down
and
we
go
on.
Take
the
snapshot
right.
Everyone
anything
is
possible.
I
am
just
giving
you
an
example
of
a
standard
database,
but
imagine
a
Cassandra
application
with
50
partitions,
which
will
translate
to
maybe
hundred
volumes,
and
if
you
follow
the
strategy
of
for
each
container
for
each
volume
snapshot,
you
could
be
getting
snapshots
of
these
volumes
and
different
points.
B
In
time
question
we
should
be
asking
now
is:
it
is
inevitable
that
you
get
snapshots
of
these
volumes
at
different
points
in
time.
You
could
argue
that
the
delays
in
milliseconds
or
microseconds,
but
it
is
inevitable
that
you
get
snapshots
of
these
individual,
is
a
different
ones
in
them.
The
big
question
now
is:
are
they
consistent?
B
So
this
is
our
time
line.
Somebody
runs
a
commit,
it's
the
starter,
a
transaction.
They
shared
a
bunch
of
updates
and
there's
a
commit
statement.
So
this
is
how
right
after
Shane
the
commit.
This
is
how
our
right
ahead
log
will
look
like,
and
this
is
how
the
data
blossom
network
x,
equals.
20
is
written
to
the
transaction
log
and
commit
entries
written
to
the
transaction
log.
There
is
nothing
written
so
far
to
the
data
box,
because
that
is
a
periodic
trend.
B
B
B
Now
we
now
are
Orchestrator
comes
in
the
snapshot
Orchestrator,
it's
trying
to
finish
the
snapshot,
which
means
it
will
reschedule
to
do
in
the
loop.
The
next
volume
is
a
transaction
log
and
we
take
a
snapshot
of
the
transaction
log
so
that
this,
the
picture
that
you
see
on
the
far
right
is
our
final
snapshot.
B
B
So
you
can
clearly
see
the
difference
here.
We
ran
a
snapshot.
The
snapshot
was
triggered
after
we
issued
a
commit
with
the
acid
guarantees
of
the
database.
X
equals
20
should
be
present
in
the
database
in
the
in
the
snapshot,
but
unfortunately
we
don't
have
it.
So
there
is
a
race
between
the
for
loop
and
the
DB
flush.
There
are
two
processes
which
are
racing
here
and
that
will
cause
an
invalid
snapshot
to
be
created.
This
is
a
big
big
problem,
and
this
is
where
our
consistency
groups
come.
B
B
Own
or
extending
blooms
and
group
that
maintains
a
right
order,
say
we
write
a
transactional
and
then
the
rights
make
today
devalue
the
right
order
is
maintained
here
and
the
group
that
fits
was
a
crash
consistent
systematics,
because
there
is
an
assumption
that
the
transaction
log
is
written
first
and
the
rights
have
taken
unwritten
to
data
volumes.
We
maintain
the
crash
consistency
at
any
point
in
time.
If
the
database
were
to
go
down,
we
know
for
sure
that
the
data
is
in
the
transaction
log.
B
Let's
design
a
brighter
data
protection
strategy
now
now
that
we
know
about
the
consistency
groups,
let's
design
it
better
data
protection
strategy,
let's
combine
these
volumes
and
call
it
a
consistency
group.
So
we
take
a
snapshot
again
on
the
first
R
we
get
a
consistent
snapshot
of
these
two
volumes
same
goes
for
three
picots
natural
sedative.
B
B
B
B
We
also
capture
the
topology
of
it.
If
the
Cassandra
has
50
partitions,
we
capture
the
fact
that
there
are
50
partitions
in
the
snapshot
and
after
the
snapshot,
if
you're
a
scale
out
200
partitions
and
move
back
to
52.
That
snapshot
will
get
50
partitions
back.
So
we
capture
the
configuration
topology
and
the
data.
So
that's
a
much
much
better
approach
to
snapshotting
than
just
natural
volumes.
B
B
Yeah
we
do
this,
so
I'll
go
oral,
Viva
architecture,
overview
of
Robin,
so
we
have
built
on
top
of
kubernetes.
We
have
not
changed
any
qualities.
You
have
taken
a
CNC
of
certified
kubernetes
deployment,
I
mean
they
build
a
distributed.
Storage
stack
with
all
the
enterprise,
great
features
like
snapshots,
clones,
qsr,
replication,
backup.
We
also
have
a
pluggable
networking
stack.
What
we
realized
is
for
applications
like
Oracle.
We
need
the
container
identity,
persistence,
which
means
the
IP
address
have
to
be
reading.
B
So
we
have
storage,
we
have
compute
in
terms
of
kubernetes
and
we
have
networking
and
we
built
of
a
flow
manager
which
will
orchestrate
the
snapshots
clones,
which
is
responsible
for
the
consistency
groups,
backups
cetera.
With
this
package.
We
can
manage
the
end-to-end
lifecycle
of
these
applications.
B
B
And,
like
I
said
we
can
consume,
you
cannot
on
any
cloud
any
VM.
We
can
run
as
a
storage,
only
provider
on
top
of
any
star
keys,
which
is
uber,
Nettie's
engines
running
in
cloud
and
without
you
can
run
any
of
the
distributed
applications
and
we
allow
for
the
lifecycle
management
of
these
applications.
B
B
Deployment
of
400
Oracle
RAC
databases
around
and
if
you
are
interested
in
product
demos,
please
go
to
Robin
REO
and
check
out
how
we
do
snapshots
and
rollbacks.
And
how
can
we
create
clones
of
these
applications
out
of
snapshots?
How
can
you
back
up
to
cloud
and
instantiate
an
application
in
cloud.
B
A
Awesome
thanks
Ravi
for
the
great
presentation
we
now
have
some
time
for
questions.
If
you
have
a
question
that
you
would
like
to
ask,
just
drop
it
in
the
Q&A
tab
at
the
bottom
of
your
screen
and
we'll
get
to
as
many
as
we
have
time,
for
we
have
one
question
so
far
from
Peter
who
asks
hello
is
Robin
ready
for
oracle
cloud
infrastructure.
B
While
the
questions
are
coming
and
I'd
like
to
show
a
quick
demo-
oh
sure,
Oh
Robin,
like
I,
said
Robin
new
software
platform.
We
use
you
get,
you
can
install
it
on
a
bunch
of
VMs
or
once
you
install
you,
get
a
dashboard
like
this
app
store
experience
where
you
will
have
all
these
data
heavy
applications,
yeah
saundra.
Well,
there
are
Couchbase
after
Horton
works.
Here
is
Oracle
RAC,
EBS
V.
So
all
of
these
are
my
charities.
B
B
So
Oracle
RAC
is
an
act
systems,
which
means
there
are
two
containers
at
play
here
and
if
you
can
see
this,
there
are
lots
of
volumes
here
same
thing.
That
I
was
trying
to
explain
that
there
are
multiple
data
volumes.
There
are
multiple
media
volumes.
There
is
splash.
There
is
great,
although
this
is
a
small
deployment
in
terms
of
size,
but
I
want
to
show
you
the
the
scale
in
terms
of
number
of
volume
that
needs
to
be
managed.
B
B
And
if
you
have
noticed
there
is
an
option
to
create
an
application,
consistence
natural.
What
databases
give
out
of
the
box
is
a
crash,
consistent
snapshot.
We
can
also
allow
you
to
plug
in
hooks
where
you
can
freeze
the
application.
If
you
choose
to
and
then
take
a
snapshot.
So
now
we
have
an
entire
application
snapshot
available
for
us.
B
B
The
same
flow
works
for
MongoDB
or
Cassandra.
It's
exactly
the
same
view
here.
I
have
a
MongoDB
application,
which
is
a
MongoDB
distributed
applications.
You
have
a
lot
more
container
simply
and-
and
there
are
a
lot
more
volumes
in
play
here
and
if
you
want
a
snapshot,
this
MongoDB
cluster
I
can
just
say
Snatcher.
B
B
A
A
B
So
what
I
want
to
show
here
is
you
can
set
the
sizing
for
your
computer,
which
means
goes
in
memory.
You
can
also
define
the
storage,
you
can
say:
I
want,
replicated
copies,
spread
across
racks
and
I
incent,
the
drop
size
or
I
can
set
compression
and
I
can
said
Packard.
This
is
a
layout
of
the
data
on
the
disk,
but
it
said
encryption
and
also
define
what
row
type
this
is.
These
settings
are
more
useful
when
you
are
defining
something
like
a
data
hold
I'd
say
you
created
almost
you
know
know
how
to
cluster.
B
Here
is
where
an
interesting
thing
comes
in,
where
prevent
placing
more
than
one
day
turn
out
container
on
the
same
node.
So
what
this
will
do
is
it
will
place
the
three
containers
on
three
different
physical
cells
and
we
can
also
enforce
storage
and
compute
for
the
original
to
be
coming
from
the
same
road.
These
two
policies
are
must-have
for
any
distributed
application
that
is
doing
its
own
replication
and
there
are
many
of
them.
This
Holger
Cassandra
is
more
de
this
couchdb
influx.