►
From YouTube: Cloud Native Social Hour - May 24, 2019
Description
This time on the Cloud Native Social Hour we pick up the pieces. We talk about Disaster Recovery, what it means and what it looks like in a Kubernetes environment. Carlisia Campos joins us to talk about Project Velero and to give a cool demo!
A
A
We
were
just
talking
right
before
word
starting
recording,
but
this
time
we
decided
to
dr
instead
of
going
over
cube
con,
because
this
is
the
we
could
keep
con
because
most
of
us
who
didn't
attend,
Q
Khan,
didn't
have
enough
time
to
watch
any
of
the
videos.
Yet
so
we
couldn't
do
a
good
job,
doing
a
review
of
cube
con
and
those
who
attended
aren't
here
right,
we're
still
flying
back
or
they're
on
vacation,
which
they
should
so
we're
gonna.
A
A
So
it's
just
a
way
to
spread
the
love
a
little
bit
and
allow
us
to
you
know,
meet
each
other
and
see
what
everyone's
talking
about,
because
this
is
a
social
hour
and
we
want
to
be
social
right,
although
it's
not
actually
usually
an
hour,
it's
usually
two
hours
but
whatever
so
far.
No
one
else
has
called
me
out
on
that.
Only
I'm,
the
only
ones
ever
brought
that
up
so
the
cloud
and
the
cloud
you
know
social
are
basically
what
we're
doing
is
just.
A
A
B
A
I'm,
like
oh
crap,
it
was
one
of
those
things
where
I'm
like
I'm,
confident
that
it's
Memorial
Day
and
then
I
questioned
myself
immediately
and
I
was
like.
Oh
no,
is
it
yeah,
so
it's
Memorial
Day,
it's
a
Friday
before
Memorial
Day
weekend.
So,
as
you
can
imagine,
most
people
who
aren't
me
and
have
a
brain
took
today
off
for
a
longer
weekend
for.
B
A
A
That's
better
something
that's
kind
of
interesting
for
me
in
from
a
cloud
native
perspective,
I'm
having
more
of
a
chance
to
play
with
eks
right
now.
These
people
that
I'm
working
with
they're
trying
of
eks
and
something
that
I
haven't
had
a
chance
to
so
it's
Amazon's
managed
kubernetes
service
and
some
parts
are
so
far
so
good,
like
I've
had
a
few
problems,
but
that
could
be
there.
A
We're
discovering
that
with
the
eks
container,
Network
and
four
main
interface,
there
are
some
different
ways
you
have
to
think
about
how
you
split
up
your
subnets,
then
you
would
with
something
with
an
overlay
network
like
final
or
NSX
or
with
something
like
BGP
that
comes
from
calico.
So
in
a
kubernetes
perspective
like
architecting
kubernetes
cluster
in
eks,
there's
a
bit
more
overhead
that
you
need
to
think
about,
and
so
that's
been
kind
of
interesting
for
me
to
like
play
around
to
this.
B
A
So
the
reason
why
that's
it
breaks
up
that
ways
of
the
eks
CNI
or
the
vista
amazon
CNI
assigns
IP
ibises
per
network
interfaces,
elastic
network
interface
of
Eni,
so
each
ni
gets
like
well
a
chunk.
You
like
it
can
support
this.
Many
IP
addresses,
like
X
number
of
IP
addresses
essentially
and
Ike
has
like
so
many
sub
interfaces
it
can
associate
with,
and
each
one
gets
an
IP
address.
So
each
instance
in
Amazon
gets
like
X
number
of
en
eyes,
and
each
Ani
has
X
number
of
idea.
A
This
is
you
can
allocate
to
it
so
like
a
m5
large
instance
and
Amazon
can
support
3
en
eyes
and
10
IP
addresses
giving
you
30
IP
addresses
total
for
that
node.
Now,
as
you
scale
up
the
size
of
instances.
So
to
do
you
get
bigger
and
better
en
iso,
you
can
support
something
more
robust,
so
you
can
get
up
to
like
244
IP
addresses,
which
is
basically
comparable
with
what
you
get
in
a
standard
Calico
deployment,
which
is
around
254,
I,
think
per
node.
So.
A
B
My
encounter
with
KS
is
trying
to
set
it
up
because
I
don't
have
to
really
set
up
and
manage
clusters
in
production.
I
only
need
to
set
them
up,
so
I
can
develop
full
hour
against
them,
and
somebody
only
counter
with
the
KS
was
trying
to
set
it
up.
Giving
up
and
using
this
really
nifty
tool
called
EAS
control.
Yeah.
A
B
A
A
A
I
believe
yeah
I
believe
that's
the
case.
I
know
the
people
who
I'm
working
with
are
lured
leveraging
more
direct,
tooling,
so
I
think
they're,
using
like
either
care
reform
or
something
else
again.
I
can't
recall
exactly
but
they're
building
up
their
own
automation
for
it.
I'm
like
that's
pretty
dope
too,
and
so
yeah.
So
I
had
a
chance
blow.
Dk
SME
talk
about
cloud
native,
it's
it's
in
the
cloud
right,
it's
built
by
one
of
the
cloud
companies.
A
So
so
far
it's
been
interesting
to
play
with
I'm
still
trying
to
figure
out
all
the
e
like
bits
and
pieces
to
it.
But
it's
been
pretty
neat
I
think
it's
also
cool
to
see
how
its
mature,
like
when
I
tried
it
out
when
it
first
got
announced
and
it
wasn't
eks,
wasn't
really
in
a
ready
state.
Yet
I
think
it
is
far
closer
now
to
being
like
a
production,
writing
type
thing.
But
that's
not
that's
neither
here
nor
there.
A
It's
just
been
fun
to
play
with
I
think
we
actually
might
do
a
review
of
eks
versus
aks
versus
GK
I
mean
one
of
the
other
men
services
as
well,
and
kind
of
like
have
a
bunch
of
people
like
chime
in
on
what
they
think,
one
versus
the
other.
You
know
pros
and
cons
of
using
this
system,
not
necessarily
to
say
like
this
we're
not
trying
to
be
like
this
is
the
one
you
did.
A
A
One
thing
that
to
continue
the
tradition
from
the
last
time
we
did
this
so
it
we
did
take
a
break
two
weeks
ago,
because
I
was
on
vacation.
Let
me
tell
you
I
needed
to
go
on
vacation
I
needed
to
realize
her
a
little
bit,
but
last
time
I
announce
this
thing
are
not
necessary.
This
thing
where
I
recommend
a
song,
and
so
this
is
Nick's
song
time
corner
this
time,
I'm
recommending,
if
you
haven't,
checked
it
out,
the
bleachers
song
I
miss
those
days.
A
It's
a
really
like
they're,
simply
chooses
like
a
very
poppy
alternative
music
kind
of
band
that
they're
the
kind
of
band
that
like
they.
They
don't
have
a
steady
number
of
members
in
the
band
that
people
jump
in
people
jump
out,
whatever
their
needs
are,
and
so
this
song
is
really
nostalgic
right.
It's
very
much
like
I
missed
those
days
like
yeah.
A
Well,
like
one
of
the
lines
is
yes,
I
was
a
lost,
but
I
miss
those
days
right,
and
so,
let's
say
even
though
there
were
times
you
know
like
that,
we're
like
bad
or
that
you
were
like
what
was
I
doing.
Sometimes
it's
nice
to
like
reflect
on
like
maybe
how
simple
things
were,
or
you
know
what
it
was
like
before
so
I
was
like
it's
just
it's
a
very
happy
sounding
song
with
a
very
kind
of
bittersweet
nostalgia
mixed
in
I,
really
like
it
so
I.
A
A
B
A
Yeah,
so
it's
funny
like
there's
one
line
that
the
exit,
the
next
line
it
goes
like
and
everything
is
changing.
The
storefronts
rearranging
yes,
I
was
lost,
but
I
miss
those
days,
I
totally
get
that
feeling
because
I'm
being
from
Seattle
and
if
you've
never
been
to
get
him
into
Seattle.
Recently,
it's
a
totally
different
city
than
it
used
to
be
like
back
when
I
lived
in
downtown
Seattle,
it
looks
almost
fundamentally
different,
and
that
was
only
like
six
years
ago,
like
it,
this
city
has
changed
so
much
and
so
yeah.
C
A
Well,
yeah,
it's
like
oh
there's
things
like
Bill
parts
or
like
for
Seattle.
For
instance,
there's
an
area
called
South,
Lake
Union
and
before
Amazon,
Google
and
Facebook
changed
their
headquarters
on
headquarters
for
most
of
those,
but
Amazon
moved
its
headquarters
down
in
South
Lake
Union.
There
was
nothing
but
warehouses,
it
was
a
large
stretches
of
land,
but
there
was
like
nothing
there
essentially,
and
so
they
just
like
took
down
all
these
old
buildings
and
put
up
these
giant
skyscrapers
and
it
just
looks
wild
now.
It's
totally
different
yeah.
B
I
mean
you
know,
other
cases
in
Boston
there
were
upgrades,
but
some
instances
there
were
also
I.
Don't
know
just
changes,
that's
debatable
if
they
were
good
or
bad,
for
example,
habit
square.
There's.
There
are
a
lot
of
big
stores
in
the
little
shops.
You
know
they
were
driven
away
because
the
rent
is
skyrocketed.
Yeah.
C
A
That
kind
of
made
up
the
community
or
like
pushed
out
in
favor
of
like
bigger
and
more
like
costly
things,
so
yep,
so
bleachers
I
miss
those
days
totally
got
a
sound
this
this
path,
but
super
recommendation
I
really
like
that.
I
like
that
band
it.
Basically
everything
they've
done
I've
liked,
which
is
actually
kind
of
hard.
For
me
to
say
these
days,
I'm
gonna
jump
in
a
new
band
I'm
like
oh
I,
love
every
single
song
with
bleachers.
That
is
demoing
the
case.
That's
yeah
high
recommendation.
A
So
this
time
we're
talking
about
disaster
recovery,
which
kind
of
dovetails
nicely
with
our
last
topic,
which
was
chaos
engineering.
So
let's
say
you,
chaos
yourself
into
a
state
where
you
can't
work
anymore,
like
your
cluster
is
now
functionally
destroyed.
How
do
you
recover
from
that
so
I'm
sure?
Most
of
us
who
have
worked
in
the
tech
industry
long
enough
are
pretty
familiar
with
the
concept
of
disaster
recovery
right.
You
have
a
disaster
and
you
need
to
recover
from
it.
That's
basically
it
so
how?
A
What
does
that
look
like
right
for
most
place
for
most
companies
and
for
most
like
tech
disaster
recovery
is
backups
snapshots
having
your
data
somewhere,
so
that,
if
you
need
to
you
can
get
it
back
right.
There
are
a
lot
of
like
mechanisms
place
to
handle
this,
but
I
would
like
lots
of
different
tools
to
handle
this,
but
it
was
fundamentally
different
from
a
kubernetes
and
from
a
container
standpoint.
How
do
you
like?
What
does
it
mean
to
do
a
disaster
recovery
for
something
like
kubernetes
right?
A
You
have
like
I
said
in
the
old
days
in
the
infrastructure
days,
you
had
just
data
right,
you
just
needed
anything.
That's
on
this
file
system,
I
need
it
backed
up
and
put
somewhere,
so
I
can
get
it
back
what
I
need
to,
but
with
the
ephemeral
nature
of
containers,
we've
been
coming
up
and
coming
down
all
the
time.
What
does
that
even
mean?
What
does
that
look
like
right?
A
One
of
those
solutions
is
something
that
we're
going
to
be
talking
about
today
and
that's
project
Valero
nei,
hep,
Tio's,
Ark
and
when
I
saw
it
the
first
time
arc
I
was
just
like
this
is
it.
This
is
perfect.
This
is
exactly
what
we
need
from
a
disaster
recovery
standpoint:
khaleesi,
do
you
mind
talking
to
us
a
little
bit
about
Valero.
B
Of
course,
not
I'd
love
to
and
wanted
to
wheel
back
a
little
bit
and
just
such
based
a
little
bit
on
the
fact
that
when
we
are
talking
about
infrastructure
and
clusters,
there
are
different
types
of
backup
state
that
you
want.
A
little
address
is
one
part
of
the
your
backup
needs,
but
you
should
be
setting
up
we.
This
is
more
like
an
infrastructure
subject.
You
should
be
setting
up
in
the
land
of
cognitive
and
kubernetes.
You
should
be
setting
up
your
infrastructure.
B
You
know
in
an
automated
way,
so
that
if
things
go
wrong,
you
can
quickly
bring
it
up
again,
and
so
that's
your
actual
clusters
and
your
kubernetes
environment.
Now,
when
it
comes
time
to
back
things,
are
then
one
thing
to
look
at
is
how
much
the
need
to
back
up.
Do
you
need
to
back
up?
Do
you
want
to
back
up
everything
together
package,
everything
together
in
the
backup
which
could
be
useful?
But
if
you
doing
that
it
won't
be
useful
all
the
time,
because
sometimes
you
only
need
a
partial
restore.
B
C
B
Pick
up
pet
CD
completely
and
which
would
also
not
be
possible
if
you
were
using
a
managed
service.
So
if
you
were
using
that
public
cloud,
you
wouldn't
have
access
to
a
CD,
so
you
couldn't
even
do
it
so
Valera
like
uses
the
kubernetes
api
and
allows
it
to
do
a
full
backup
which
is
recommended.
But
you
can
also
do
selective,
backup,
just
backup
certain
resources
and
we
need
to
ideally
you
to
be
doing
full
backups
and
when
you
do
a
restore
you
select.
B
A
Some
of
the
things
that
bolero
does
and
they're,
like
the
one
of
the
reasons
why
I
liked
it
so
much
is
the
fact
that
it
didn't
try
to
do
anything
particularly
complicated
like
try
to
access
it.
You
need
to
figure
out
the
current
state
of
things
and
then
do
a
backup.
Based
on
that
there
is.
We
all
talked
about
that
in
just
a
moment,
doing
that
sort
of
backup.
A
One
thing
else
that
it
does
is
persistent
volume
snapshots,
which
I
think
a
really
dope
people
oftentimes
they're,
like
whoa,
okay,
well,
I
have
this
data
that
I'm
need
backed
up
in
kubernetes
as
well.
Does
Valera
do
that
and
the
answer
is
yes.
As
long
as
your
infrastructure
supports
it,
Valera
can't
just
magically
make
a
snapshot
for
you,
but
it
can
tell
like
it,
let's
say:
you're
using
an
s3
bucket.
It
can
kick
off
in
this
three
snapshot
or
some
other
tool
who's
that
I
can't
think
of
that
functionality
right
now.
B
A
Any
of
these
other,
like
the
clouds
that
you
run
I,
think
that's
really
cool
right,
I
and
I
know
that
it's
not
something
that
works
like
100
percent,
all
the
way
that
people
expect
necessarily
like.
We
see
you
like
things
that
mentions
in
the
community
and
she's
that
pop
up
around
that,
but
it
does
a
good
job.
It
does
as
good
of
a
job
as
your
underlying
infrastructure
and
that's
kind
of
all
you
can
ask
for
from
a
cloud
native
application.
The
infrastructure
should
support
it.
A
If
it
doesn't
sorry
so
I
think
that's
very
valuable
like
it's
not
just
these
objects
and
I
think
that's
a
different
case.
It's
from
what
you
were
saying
before,
which
is
the
automation
right.
If
it
was
just
the
objects
like
these
deployments
and
they're
all
stateless
things
you
who
cares,
if
you
have
a
backup,
just
read,
just
create
a
new
cluster
and
redeploy
yeah.
C
A
A
lot
of
these
a
lot
of
people,
a
lot
of
users
have
more
robust
needs
than
that
and
falero
and
projects
like
Calero,
which
I
will
talk
about
just
a
moment,
do
a
good
job
to
like,
like
help
with
people
out
in
this.
In
this
way,
so
besides
whole
arrow
I
was
looking
up.
Another
tool
called
cube,
D
curly,
say
if
you
had
a
chance
to
look
at
this
one,
hey.
A
A
I
haven't
had
a
chance
to
look
at
it
too
much
myself.
I
haven't
gotten
like
hands
on
like
I
said
it
was
pretty
busy
this
week
and
trying
to
think.
Oh
I
want
to
go
this,
but
basically,
like
I,
think
the
difference
between
Valero
and
cube
D
is
cube.
D
is
a
cluster
operator,
so
it's
using
the
operator
framework
that
core
OS
created
not
created
I
mean
just
codified,
essentially
the
operator
framework.
A
So
in
a
way
to
like
manage
your
objects
and
all
the
things
instead
of
kubernetes,
in
a
way
that
you
can
like
back
it
up
into
a
CRV
and
manage
your
backups.
That
way,
I
think
that's
how
it
works.
I
again
haven't
had
enough
time
to
look
at
it,
but
it's
another
option
for
our
our
friends
and
the
cloud
native
community.
I
would
say:
look
at
both
of
these
tools.
A
C
A
A
I
bolero
works,
it
works
well,
I,
love
it
and
it's
nice.
The
the
nice
thing
about
it
and
tools
like
it
like
QT
is
it
gives
you
the
the
safety
in
mind,
like
you,
give
it
a
peace
of
mind
to
know
that
if
you're
doing
your
backups
regularly,
if
something
goes
down,
if
something
happens
you
can
recover,
but
nowadays
kubernetes
is
in
such
a
state
that
it
is
a
more
stable
environment.
A
It's
less
likely
for
your
cluster
to
completely
brick
itself
such
that
you
can't
use
it
anymore,
but
there
are
still
scenarios
where
that
could
happen.
For
instance,
we
still
don't
have
a
good,
a
particularly
good
story
around
certificate
rotations
if
your
certificate
CA
for
kubernetes
or
for
NC
d
expires,
which
it
will
after
a
year
because
that's
the
default
in
all
kubernetes
installations
unless
it's
been
particularly
set
by
a
provider
which
I'm
pretty
sure,
no
one
ever
thinks
about
to
do
that.
So
all
CA
is
incriminate.
Ease
are
a
year
old.
A
So
if
you
have
a
cluster
that
lives
longer
than
a
year,
you
can
experience
it
experience
a
catastrophic
outage.
So
what
do
you
do?
Well,
replacing
a
cert
like
replacing
the
root
CA
of
a
cluster
is
a
huge,
a
monumental
task.
It
is
far
easier
just
to
create
a
new
cluster
and
then
keep
you
know,
have
a
backup
of
your
things
and
deploy
to
the
new
cluster
Bingbing
Bosch.
Everyone
who
need
enough
to
spend
your
weekend
fixing
your
root.
A
Ca
problem,
you
just
spin
up
a
new
cluster
and
there
you
go
Blair
Blair,
orchid
bees
save
the
day,
a
fun,
a
funny
thing
that
came
out
with
bolero
and
other
disaster
recovery
tools
on
the
way
is
it's
not
just
disaster
recovery,
there's
extra
fringe
functionality
associated
with
it
a
lot
of
times.
People
use
these
things
for
migration
or
for
duplication.
I
could
say
you
in
Amazon
region,
and
you
want
to
stand
up
a
new
region.
A
You
just
create
a
new
cluster,
but
you
wanted
to
look
similar
to
the
old
cluster,
my
backup
one
restore
to
the
other
boom.
You
now
have
duplicate
clusters
or
if
you
want
to
migrate
regions,
you
take
a
backup
here.
Destroy
this
cluster,
create
this
cluster
spin
it
up.
Hopefully
you
do
this
more
in
tandem.
A
Otherwise,
you
just
establish
an
outage,
and
that
would
be
bad,
so
this
is
a
bad
example,
but
whatever
I'm
doing
this,
like
from
the
top
of
my
head,
send
up
a
new
cluster
and
then
push
all
your
objects
to
the
cluster
with
your
snapshots
and
there
you
go.
You've
got
a
fresh
migration.
It's
kind
of
an
interesting
use
case
that
I
don't
think
that
we
were
thinking
of
initially
when
project
Ark
was
created
and
a
bolero
but
I'm
happy
to
see
that
there's
like
extra
functionality
that
people
are
happy
with.
A
B
B
A
B
Sure
Nick
I'm
gonna
start
with
my
favorite
change,
which
was
the
install
feature
before
for
us
to
set
up
a
Valero
on
the
carbonate
uncovered
eighties.
We
were
applying
a
series,
and
now
we
have
we,
you
don't
have
to
do
that.
It
was
a
little
bit
cumbersome
because
you
had
to.
We
had
some
examples,
for
example,
but
even
examples
we
had
to
edit
specific
things
like
the
bucket,
and
now
we
just
run
an
install
command.
You
know
we
have.
We
pass
in
some
information
and
so
much
easier
to
use.
B
I,
really
love
that
we
added
that
we
did
May
overall
overhaul
of
our
plug-in
system.
We
cut
down
on
a
lot
of
the
dependencies
that
we
had
before
so
now.
If
you
are
developing
a
plug-in
for
snapshot,
support
for
provider
that
we
don't
support
or
for
for
provider
for
which
there
is
no
plugin.
Yet
you
when
you
import
when
you
import
the
plug-in
framework,
you
get
oh,
like
tiny
set
of
dependencies.
B
B
B
B
We
did
run
up
upgrades
to
the
port
works
plug-in,
which
is
a
very
specific
use
case.
We
made
a
little
bit
of
improvements
to
the
rustic
plug-in
that
supports
the
or
astok
is
it
takes.
It
makes
backup
of
file
system
and
that's
useful
way,
and
so
we
support
the
three
major
service
providers.
So
we
we
have
primitive
snapshot,
support
for
as
your
AWS
in
Google
cloud
platform
in.
In
addition
to
that,
we
have
plugins
that
other
providers
have
developed.
B
B
B
B
A
C
B
Don't
back
up
s3
there
correction?
What
we
do
is
we
backup
the
kubernetes
resources
and
we
backup
the
snapshots
or
file
system
using
drastic,
and
we
put
all
that
in
a
file
and
we
dump
it
to
an
s3
buckets
that
you
specify
when
it's
setting
up
falero
so
that
that's
the
direction
of
the
backup
we
back
things
up
and
dump
dump
the
back
up
in
the
nest,
three
buckets
so
an
s3
bucket
or
any
anything,
that's
s3
compatible.
So
if
you
are
using
Amazon
there'll
be
an
s3
bucket
anything
any
other.
B
Three
are
the
two
other
ones.
We
also
have
nice
to
be
compatible
storage.
Now,
if
you
don't
have
access
to
anything,
that's
s3
compatible.
You
can
stop
menu
and
point
the
backup
to
that
minium,
installation
and
because
Mineo
is
s3
comparable
and
it's
also
very
easy
to
set
up.
So
you
that's
where
your
backup
or
go,
and
likewise
the
restore,
will
fetch
from
that
bucket
in
order
to
restore
it
to
wherever
you
need
to.
C
B
A
C
B
A
A
B
C
B
B
C
B
C
B
C
B
C
And
one
other
question
is
so:
let's
say
there
is
a
CNI
change
with
respect
to
cluster
which
fee
bagged
up
and
if
you
want
to
back
up
the
data
with
the
new
network,
configuration
so
do.
Is
there
any
limitation
with
respect
to
that?
Let's
say
we
are
using
cube
net
networking
in
one
of
the
cluster
and
we
want
to
back
it
up
to
see
a
SS
CNI.
A
I
can't
think
of
a
reason
why
the
network
would
have
be
like
how's
that
issue
okay:
it
should
be
independent,
so
like
Valero,
it
backs
up
the
objects,
so
things
like
pods
appointment
status.
Daemon
sets
all
these
things,
so
the
API
objects
themselves
get
backed
up
and
then
any
data
is
backed
up
as
well
and
stored
some.
Neither
yes
on
all
these
things.
It
should
be
network
independent.
A
A
Yeah
network
network
and
network
changes
are
always
kind
of
a
funny
thing.
So
good
luck
with
that.
Good
luck
with
your
segmentation
remember
to
allocate
more
IP
addresses
than
you
expect
to
use
when
using
AWS
c,
and
I
EK
SC,
and
I
just
because
it
has
the
cooling
down
period
right.
So
you
have
your
pool
of
active
IPS
and
a
pool
of
cooling
down
IPs.
Hopefully
you
have
more
active
IPS
that
this
doesn't
really
matter,
but
just
keep
that
in
mind.
Okay,.
A
C
B
B
A
If
but
I
think
that
there
is
some
value
to
having
like
there
is
context
associated
with
data,
and
so
having
that
context
with
the
data
that
you're
backing
up
at
the
same
time,
I
think
there
is
some
value
there,
but
that
is
really
up
to
the
end-user.
A
lot
of
these
questions
do
come
in
like
well.
It
depends
on
the
scenario
right,
yeah.
B
It
depends
on
your
use
case.
Definitely
if
you
don't
have
a
use
case,
if
our
use
cases
you
know
I
just
need
backup
of
my
backup
of
my
snapshot.
Yeah.
Definitely
you
can
use
the
service
provider,
but
other
things
I
can
think
to
is.
Let's
say
you
want
to
backup
a
snapshot
from
one
provider
to
another
provider.
You
can
use
Rasik
to
do
that
pretty
well
and
in
the
future
we
might
have
native
support
for,
for
that,
don't
know,
but
it's
something
we
talked
about.
B
A
A
A
B
C
A
A
So,
basically
like
the
way
that
this
functions
is
you
go
into
your
ed
city
cluster,
either
from
a
management
service
or
just
jump
like
SS
aging,
into
a
node
issuing
an
sed
at
City,
CTL,
back
up
command
and
then
taking
that
data
and
storing
it
summer-like
s3
summer.
So
that
and
doing
this
on
a
regular
basis
either
through
automation
or.
A
However,
you
want
to
do
it
so
that,
if
there's
a
problem
with
your
at
city,
if
you
need
to
do
it,
be
close
to
migration,
and
you
want
to
make
sure
that
all
of
that
data
comes
with
you,
you
can
do
a
restore.
There
is
a
caveat,
though
EDD
does
not
handle,
restores
very
well
that
data
is
actually
kind
of
like
a
CD
as
powerful
as
that
city
is
and
useful
as
it
city
is,
it
can
also
be
incredibly
fragile
and
so
doing
a
full
backup
and
restore.
A
A
B
A
B
C
C
B
To
basically
we're
hilarious,
we
talked
a
bunch
already,
not
gonna
spend
a
lot
of
time.
You
can
do
I'm
just
going
to
like
scan
and
see
things
that
maybe
we
didn't
talk
about
so
relevant
here
is
you
can
do
full
in
selective
backup
and
restore
because
we
use
the
kubernetes
api.
You
can
backup
a
cluster
and
duplicate
that
cluster
into
another
cluster,
which
is
very
handy.
If
you
want
to
duplicate
environments,
for
example,
you
have
a
dev
environment,
a
test
environment,
production
environments.
B
B
It
he
has
a
scheduling,
feature
super
handsy,
the
selective
backup
and
restore
works
on
with
namespaces,
like
I
said,
can
have
different
environments.
For
example,
if
you
delineate
the
environments
by
namespaces,
you
can
just
say
back
this
name
space
into
another
cluster
resources,
so
you
can
specify
the
resources
you
want
to
backup
or
you
can
use
labels
lectures.
You
can
label
things
parts
and
say
you
know
everything
with
this
label
gets
restored.
B
B
This
is
for
people
like
someone
from
digitalocean
wants
to
have
a
snapshot
again,
but
the
will
are
worse
with
they
would
do
that
and
if
you
don't
have
anything
that
fits
those
two,
you
can
always
use
rustic,
but
we
recommend
that
you
only
use
rustic
if
you
don't
have
a
plug-in
or
you're,
not
using
one
of
those
three
cloud
providers.
We
have
pre-imposed
hooks
for
customization,
free
backup,
Rebecca,
private,
post,
backup,
free,
restore
forests,
or
you
can
freeze
them
for
annotate
things
or
do
other
things.
They
think
you
might
need
to
do.
B
B
So,
basically,
on
the
left,
you're
singing
back
backup
everything
create
a
backup
called
my
backup
only
for
the
namespace
called
as
an
example
and
include
my
snapshots,
my
volumes,
so
the
blue
rectangle
is
the
coronaries,
the
oranges
velaro.
So
for
the
kubernetes
api,
we'll
pull
those
objects
from
the
variety
from
the
kubernetes
side
of
things,
and
then
we
will
dump
that
in
the
format
of
a
backup
into
an
s3
buckets
anything
just
s3
compatible
and
when
you
need
to
do
a
restore,
you
say,
create
a
restore
from
backup,
my
backup,
which
is
the
same
name.
B
B
C
B
C
A
B
B
B
B
So
basically,
there
is
a
section
that
says:
install
Valero
in
another
section
in
separate
sections
that
that
says
what
you
have
to
do
in
specific
for
either
AWS
edger
or
Google
cloud
platform
and
as
far
as
those
providers
go,
it
doesn't
matter
if
you're
doing
the
the
managed
service
or
the
full
kubernetes
like
your
own
kubernetes
running
on
the
service
providers.
It
makes
them
as
far
as
the
letter
goes,
makes
absolutely
no
difference.
B
So
imagine
you
have
your
cluster
up.
You
set
up
your
buckets.
You
have
the
right
policy,
you
have
to
add
a
vero
auerbach
things,
so
you
have
a
developer.
Has
access
to
that
bucket?
It's
all
another
documentation,
so
I
already
have
that.
So,
when
I
set
up
the
bucket
I,
give
it
a
name
in
it's
a
see,
apparently
it's
C
aw
eyes
forever
buckets.
B
B
That
there
whoa
I
know
sorry
so
I'm
gonna
just
make
to
make
it
easy
for
me
setup
the
couple
variables
for
buckets
in
region
and
then
I'm
going
to
copy
the
the
install
command
I
want
to
go
over
this
a
little
bit.
So
we
are
the
first
line:
driller
install
second
line
specifying
which
provider
we
are
using.
Then
the
backpack
buckets
variable
here
that
I
just
specified
up
there,
the
backup
location
config.
B
This
is
just
a
default
in
this
called
us
East,
that's
the
region
for
my
things
and
the
snapshot,
location,
config
I
was
the
same
same
region
and
I
have
a
credential
file
here.
That's
another
thing
that
I
need
so
when
I
create
my
cluster
I
get
credentials
to
access
that
cluster
and
the
wave
the
valerian
style
works
is
that
you
have
to
pass
the
the
name
of
the
file
to
this
flag
and
I'm.
Saying
here:
use
rustic
because
I'm
going
to
use
rustic,
because
why
not.
B
B
C
A
A
If
it
just
worked
yesterday
like
what's
going
on,
I
realized
that
I
hadn't
changed
my
path
to
point
to
the
new
cube
CTL
version,
because
I
run
a
like
pretty
poor
versioning
control
on
my
local
laptop
because
I'm
kind
of
lazy,
but
it
runs
it
iron.
The
problems
like
this,
such
as
I,
can't
get
all
right,
can't
make
sure
that
I
have
keeps
detailed
versions.
Products
that
problem
so
approach
tip.
Make
sure
that
you're
running
your
client
at
the
same
or
higher
version
or
same
version
as
the
kubernetes
server
you're,
trying
to
connect
to
all.
B
C
A
B
A
C
A
A
B
C
A
C
C
A
So
something
that's
kind
of
nice
about
there
being
a
community
around
disaster
recovery
in
kubernetes
and
that
I
see
it
growing
is
that
there
isn't
just
like
the
one
tool.
As
we
said,
it's
the
fact
that
there
are
tools
at
all
I
remember
when
I
first
started
using
kubernetes
back
on
like
three
OpenShift,
three,
oh
and
three
one.
There
was
nothing.
It
was
a
barren
wasteland
at
the
time
when
we
needed
some
kind
of
disaster
recovery
at
all.
A
D
D
It
seems
a
little
hairy
seems
like
this
is
a
better
path,
but
another
thing
we're
looking
at
sort
of
contained
them,
because
we're
also
moving
a
few
things
off
of
some
older
1.8
cops
clusters
and
for
that
move
where
she's
doing
more
of
a
manual
migration,
because
we
want
to
switch
to
using
them,
customized,
mm-hm
or
managing
some
of
the,
like.
You
know,
per
environment
differences
or
is
up
till
now,
we've
been
managing
stuff
by
having,
like
you
know,
a
folder
per
cluster
and
then
sort
of
duplicate,
but
slightly
different.
D
You
know,
kubernetes
ya
know
manifests
in
in
each
folder.
So
of
course
that
does
mean
right
that
some
of
what
ends
up
as
like
the
actual
resources
inside
the
kubernetes
cluster,
look
a
little
different
from
what
you
have
on
the
file
system
or
what
you
you
know,
get
back
from
the
API.
So
I'm
sort
of
curious
how
like
something
like
that,
where
someone's
using
customized
or
maybe
JSON,
that
are
some
other
templating
kind
of
emerging
tool?
D
So
yes,
like
in
the
case
of
customize,
are
kind
of
have
this
idea
of
like
overlays,
so
you've
got
sort
of
your
base,
j-mo
and
then
other
fragments
of
the
mo
and
they're
not
like
you
know,
merges
them
together
and
spits
that
out
and
then
you
pipe
that
into
like
you,
cuddle
apply
and
it
sends
it
up.
So
so
there-there
is
sort
of
a
workflow
right
of
where
you
have
get
repo
as
a
source
of
truth
for
your
your
stuff.
For
your
your.
C
C
A
I
I
think
this
is
a
case
where,
if
you're
doing
something
stateless
lis,
it
might
make
more
sense
to
use
the
tool
itself
to
deploy
to
redeploy,
because
I
don't
believe
that
there
is
any
mutation
that
Valero
does
to
handle
scenarios
like
this.
It
just
takes
whatever's
in
the
API
and
back
that
up
and
then
redeploys
like
things
like
the
UID
or
specific,
like
you
anytime
creation
time.
All
these
things
that
are
created
by
the
API
server
itself
or
actually
like
is
the
scheduler
controller
manager.
No.
C
A
A
There's
also
monitoring
it's
kind
of
something
we
talked
about
last
time
on
the
chaos
engineering
episode,
but
monitoring
you're
in
is
vital
to
know
when
something
is
wrong
and
to
know
when
you
need
to
restore
something
because
it
you
could
be
in
a
case
where
you,
you
don't
know
that
your
machines
are
there,
your
clusters
down
like
your
control
plane,
could
be
down,
but
you're
the
pods
are
running
and
you
wouldn't
maybe
necessarily
know
about
it.
Unless
you
were
observing
your
cluster
and
alerting
on
it
Brendon.
A
A
D
That's
actually
interesting
question.
You
know
our
infrastructure
itself
is
mostly
I'm
still
like
instance-based
our
crew.
Vinay's
usage
is
more
of
sort
of
infrastructure,
like
you
know,
artifactory
or
like
other
services
that
are
sort
of
to
the
to
the
side
of
a
boyfriend's
itself.
If
you
will
okay,
so
I
mean
you
know
in
our
case,
like
terms
of
monitoring,
it's
more
about,
like
you
know,
like
being
able
to
get
like
note,
state
and
and
things
like
that,
it
helped
on
the
in
our
same
group
on
the
symphony
side.
D
C
A
A
B
C
C
A
C
C
B
What
is
it
hold
on,
or
here
the
very
last
night
got
backups
from
backup
stores,
backup
count
22
how's,
that
possible
did
not
just
boot
up
the
lower
layers
when
you
delete
the
Lera
Butler
doesn't
delete
your
backups,
which
is
a
good
thing,
and
so,
since
I'm,
putting
up
a
little
in
the
same
cluster
I
was
using
that's
running
against
the
same
bucket.
I
was
using
before
he
recognized
that
I
have
22
backups
in
that
bucket
and
obviously
I
can
still
restore
from
any
of
those.
So.
B
Valera
Qin
has
a
controller
that
continuous
continuously
checks
for
backups
for
the
backups
that
we
have
in
the
buckets.
For
example,
if
you
delete,
if
you
manually,
go
there
in
the
least,
let's
say
you
have
one
backup,
you
delete
that
backup,
so
they're
all
check
for
that
backup
and
garbage
collecting
from
the
system
if
step
back
up
no
longer
exists,
Oh
phew
deleted.
But
why
would
you
do
that?
Yeah.
A
B
B
So
what
we
so
here
we
have
filler
up
and
running
we
against
the
buckets
that
had
to
be
set
up
so
now.
What
we're
going
to
do
is
we're
going
to
deploy
engine
into
next
app
and
that's
going
to
run
on
the
nodes
and,
basically,
then
we're
going
to
use
rustic
to
back
that
up.
I
think
now,
I
can
turn
back
to
fish
and
so
basically
I'm
going
to
run
this
commands
here
to
control
I
and
that
is
I'm
at
the
roots
of
the
world
project.
C
B
C
B
A
The
answer
off
my
honest
answer,
like
the
actual
and
leave,
if
I'm
being
like
a
tech
guru,
the
extras
like
be
aware,
this,
the
series
that
you
follow
and
then
watch
to
see
if
there's
any
changes,
the
actual
answer
is
wait
until
you
get
bitten
like
this
and
then
figure
out
why
and
then
make
the
appropriate
changes,
because
this
is
I
mean
honestly.
How
are
you
going
to
know
like
it?
A
C
A
C
A
B
A
B
C
A
B
C
B
B
C
B
B
B
B
A
A
B
B
A
B
B
A
B
A
C
C
C
A
B
A
B
C
B
No,
my
last
somebody
wants
to
see
if
something
anything
specific
I
mean
viler
is
very
simple
to
use.
As
you
can
see,
it's
nothing
to
it
from
the
user
ten
points,
but
you
can
get
more
complicated
once
you're
dealing
with
TVs
and
you
want,
you
start
wanting
to
select
different
things,
combine
different
things.
B
Schedule
e
works
like
same
way,
so,
let's
see
you
do
of
the
Lyra,
create
schedule
and
because
I
don't
know,
I'm
gonna
see
what's
available
so
very
schedule
name
just
you
see
up
here.
So
in
any
place
you
have
flags,
and
these
are
the
flags
just
like
just
like
a
backup
or
like
a
restorer
like
a
backup.
You
have
the
same,
pretty
much
the
same
flags
and
the
thing
here
that's
going
to
be
relevant
is
this:
can
you
see
it?
My
highlights
yeah.
B
A
D
C
A
B
A
C
B
C
A
To
get
back
and
watch
the
cube
con
videos
there's
a
couple
that
I'm
very
excited
to
see:
Amy
our
art,
friends,
amy
codes
and
aaron,
apparently
put
on
a
good
a
good
show
and
I'm
gonna
watch
that
I'll
live.
Did
the
Lightning
talk
about.
If
so,
our
friend
and
colleague,
olive
did
a
lightning
talk
on
what
to
do
to
prep
for
your
cka
exam,
so
I
highly
recommend
checking
that
out.
She's
great
and
yeah.
A
A
A
A
B
C
B
Thank
sky
for
suggesting
the
conch
you
confi,
config
creds,
so
I
easily
even
say
mention
what
my
problem
was
because
Nick
was
talking
and
I
was
following
through
when
when
is
Nick
not
talking
exactly
so.
My
problem
was
that
when
I
pasted
in
I
pasted
an
additional
return
carriage
in
the
file
that
screwed
it
up,
so
I
opened
it
again
because
I
I've
been
using.
This
should
be
no
problem,
so
I
opened
it
up
again
saw
the
extra
character
removed.
It's
fine
awesome.