►
From YouTube: Discussion about Gitaly Disaster Recovery plans
A
A
Okay,
so
I
have
the
first
item.
This
is
the
issue
that
we're
checking
for
this
discussion,
which
is
for
this
quarter
to
come
up
with
a
plan
to
see
like
what
we
can
do
for
Italy
Dr.
For
a
little
bit
of
background,
the
current
Dr
solution
for
giddly
is
to
use
snapshots.
We
have
established
that
the
RPO
for
snapshot
recovery
is
six
hours.
Rto
is
about
30
minutes.
A
Rpo
is
the
amount
of
potential
data
loss
and
the
reason
for
that
is
due
to
the
frequency
that
we
create
disk
snapshots
which
can
be
up
to
six
hours.
More
typically,
you
know
I
think
snapshots
are
more
typically
as
old
as
four
hours
for
our
test.
We
had
a
snapshot
that
was
around
I
think
between
one
and
two
hours
it
takes
about
30
minutes
to
restore
from
snapshot.
That's
the
time
to
create
the
disk
and
reconfigure
giddly
foreign.
A
B
Yeah,
so
by
next
item,
do
you
mean
b
I
or
b2a
2A,
okay,
yeah,
yeah
I?
Think
you
are
the
one
true.
That's.
B
Of
git
data,
which
is
six
hours
30
minutes-
that's
that
makes
sense.
Do
you
want
to
go.
A
A
Yeah
yeah,
we
can
go
through
each
of
them,
so
we
have
the
PCP
backup
Appliance.
This
is
a
relatively
new
feature
that
gcp
offers
it's
an
agent,
that's
installed
on
each
server
and
a
backup
appliance.
That
is
that
Google
runs
on
your
behalf.
A
Like
I
said
it's
fairly
new.
We
had
a
discussion
where
I
I
had
a
discussion
with
Google
about
this,
an
asynchronous
discussion
through
a
Google
doc.
To
answer
some
questions
at
this
time.
I
don't
think
it's
going
to
be
suitable.
A
The
main
one
of
the
main
reasons:
why
is
that?
There's
no
G
Cloud
terraform
support
yet,
which
makes
a
little
bit
difficult
to
automate,
there's
also
an
additional
cost
of
I
approximated
around
500
a
month
per
storage
node,
that's
in
in
addition
to
the
storage
cost,
so
you
would
have
the
storage
cost
in
object,
storage
for
the
data
that
we're
backing
up
plus
500
per
month.
For
you
know,
each
storage,
node
and.
B
A
And
maybe
there's
some
like,
maybe
there's
some
flexibility
there,
but
from
my
discussions
with
them
they
they
charge
on
the
source.
You
know
per
per
Source
disk,
so.
A
One
of
the
things
that
attracted
me
to
the
solution
was
with
gcp
snapshots.
We
don't
have
any
control
over
when
full
or
incremental
snapshots
are
taken.
We
had
a
conversation
with
Google
about
that
and
they
said,
there's
really
nothing
that
we
can
do
to
control
when
when
that
happens,
and
that's
the
reason
why
our
RPO
can
be
so
long,
because
these
full
snapshots
take
a
significant
longer
amount
of
time
compared
to
the
incrementals
with
the
pack
of
Appliance.
We
have
a
lot
more
control
over
that.
So
that's
one
benefit.
B
Yeah
and
so
so,
it's
500
per
Storage,
Note,
okay,
I'm
thinking
about
cost
reasons
as
well,
because
I.
A
See
yeah
I
think
like
like
yeah,
we
could
do
the
cost
calculation.
If
we
had
tiered
you
know
tiered
storage.
We
could
probably
compare
the
cost
of
this
versus
the
cost
of
regional
right.
B
A
video,
tiered
storage:
does
this
mean
that
we're
gonna
have
an
RPO
and
RTO
different,
depending
on
the
tier
that
you
are
on
yeah.
B
I
think,
because
that
is
a
big,
a
big
caveat
right
with
regional
disks
like,
and
it's
going
to
be
way
too
expensive
to
roll
it
out
everywhere
and
if
we
only
roll
it
out
on
a
license
basis,
it's
gonna
be
different.
Rp
ortos
per.
A
Yeah
I
mean
I
I'm,
going
to
I'm
kind
of
going
in.
With
that
assumption
that
we
would
advertise,
you
know
different
RPO
RTO
I
mean
I
yeah
different
RPO
RTO,
depending
on
the
the
plan,
and
that
would
probably
be
okay,
but
maybe
we
need
to
explore
that
a
bit
further
yeah
so
for
B.
So
for
B2
yeah.
You
had
some
comments
here.
Steve.
B
Yeah
I
guess
we
do
need
to
Benchmark
if
there's
like
any
difference
between
reach
and
not
PD
balanced
and
be
debased
like
performance
hits
or
anything
like
that,
since
they're
gonna,
be
you
know,
different
systems
right,
so
we're
gonna
need
to
look
at
how
electricity
cluster
reacts
to
a
regional
disk,
and
these
two
proposals
at
the
moment
seem
very
hard
to
say,
high
level
right
proposals
which
makes
us
but
I
feel
like
before
we
take
a
decision.
A
B
More
is
this
going
to
cost
us
with
the
current
setup
that
we
have
and
how
much
is
the
backup
apply?
Appliance
gonna
cost
us
because,
like
if
it's
something
that
we
have
to
do
a
few
clicks
once
it's
not
going
to
be
that
big
of
a
deal
until
gcp
kind
of
automates
that,
through
the
API.
B
Yeah
so
I
I
guess
we'll
have
to
do
a
full,
detailed
cost
analysis
of
the
current
setup.
If
that
makes
sense
before
we
can
do
a
decision
on
this
as
well
and
have
like
a
benchmarking
for
the
regional
PD
balanced
and
see
like
okay,
if
we
deploy
a
gitile
node
with
regional
BD
balance
like
how
quick
is
it
going
to
be
to
recover
if
something
goes
wrong
there
and
things
like
that
right.
A
And
the
backup,
Appliance
and
the
regional
storage
isn't
I,
don't
think
is
a
good
like
I
mean
obviously
the
the
backup
Appliance
has
still
has
an
RPO
issue.
When
I
asked
gcp
about
like
how
long
these
incrementals
will
take,
they
said:
okay,
they're
gonna
probably
take
around
the
same
amount
of
time
as
snapshots,
so
we're
still
looking
at.
A
Probably
you
know,
I,
you
know
I
guess
like
we
need
to
get
an
exact
number
for
that,
but
we're
still
looking
at
some
data
loss,
I
assume
for
the
regional,
but
you
know
balance
provision
storage
that
it's
going
to
be
much
less.
B
Yeah
a
little
loss
so
and
and
the
action
items
I'm
writing
down
something.
So
if
you
scroll
down
from
your
screen,
so
maybe
we
need
to
create
some
creative
criteria
for
the
best
option
right,
which
would
be
costs,
of
course,
for
for
current
utility
Fleet.
So
let's
say:
okay,
let's,
for
example,
the
backup
storage,
the
backup
Appliance
is
going
to
cost
us
x
amount
of
dollars
per
month,
but
the
89
hours
that
we.
B
Rpo
RTO
ignoring
the
story
like
user's
license,
so
let's
say:
okay,
if
we
deploy
Regional
balance
disks
to
everything
right,
we
ignore
the
what
license
the
user
is
on.
How
much
is
that
gonna
cost
us
and.
B
Is
there
anything
else
that
we
need
to
look
at
here,
or
is
it
mostly
cost
and
RPO
that
we
want.
B
A
Right
yeah
I
mean
I,
guess,
I
guess
performance
impact
is
needs
to
be
taken
into
consideration.
A
I
wonder
if
there's
going
to
be
like
yeah
I,
think
probably
probably
the
best
option
here
would
be
to
use
the
the
Dr
testing
environment
and
we
can.
We
can
just
start
testing
out
the
different
different
options.
B
Yeah,
because
to
me,
they
both
like
I'm,
only
focusing
on
the
first
two,
because
the
last
two
are
pretty
much
but
non-starters
right,
increasing
snapshot,
frequency,
that's
not
possible
and
get
the
clustery
architecture.
That's
way
ahead
of
our.
B
At
the
moment,
so
focusing
on
those
two
to
me
anyway,
so
I'm
not
sure
about
you,
but
both
of
them
don't
seem
like
a
decent
option,
but
we're
still
not
clear
which
one
is
the
best
option
right
yeah.
B
So
maybe
we
can
get
some
more
data
on
that
like
get
like
an
Excel
sheet
or
whatever
like.
This
is
how
much
it's
gonna
cost
us
on
all
of
them
and
then,
if
we
want
to
make
regional
balance
disk
storage
even
more
attractive,
we
can
say,
like
here's
gonna,
be
the
third
storage
with
caveat
that
we
need
to
implement
all
that
logic,
which
is
also
a
bunch
of
work.
That
needs
to
be
done
and
it's
more
of
a
cost
optimization
than
anything
else.
I
guess.
B
A
A
Yeah
no
I
think
that
sounds.
That
sounds
good.
That
sounds
good
to
me
and
I.
Think
those
three,
those
three
things
make
make
sense,
so
I
think
the
the
first,
the
first
two
might
be
a
little
bit
more
straightforward
than
the
performance
impact
we're
going
to
have
to.
B
B
Yeah,
yeah
and
I'm
not
sure
how
to
test
that
right,
because
benchmarking
is
really
expensive,
and
this
sounds
like
not
just
money-wise
but
time
wise,
like
getting
a
basement.
A
B
A
B
Will
you
open
up
issues
to
start
like
running
like
create
a
cluster
with
PG
Regional
and
see
how
that
goes,
and
things
like.
A
B
Okay,
so
and
okay,
so
let's
say
so
the
Epic
that
you
have
now
the
issue
that
you
have.
A
B
With
a
plan
to
reduce
RPO
So,
the
plan
is
for
the
original
goal
was
to
put
it
as
a
queue
for
like
close
this
on
Q4
and
come
up
with
a
plan
on
Q4
right.
A
Yeah
I
was
I
was
thinking
that
I
guess
the
output
of
this
issue
would
be
this
new
epic,
with
a
plan
to
I
mean
more
like
it's
a
plan
to
come
up
with
a
plan,
but
it's
I
think
I
think.
That's
probably
all
we
want
to
do
in
this
quarter.
We
can
probably
start
in
January
but
I'm,
not
seeing
us
getting.
B
Yeah,
so
so
what
should
we
create
yeah?
What
if
we
create
an
epic
yeah
to
Benchmark
the
proposed
options
right
so
right.
A
B
Have
to
to
propose
options
using
the
backup,
Appliance
and
the
regional
disks,
and
then
we
can
go
through
that.
Another
item
I
have
in
the
agenda
is
I'm,
not
sure.
Where
is
that,
where
I
put
it
yeah,
have
we
gone
through
with
sessions
with
gcp
on
this
because
I
know
in
the
past?
B
A
Yeah
the
the
the
sessions
we've
had
with
them
have
all
been
async
and
are
captured
in
the
docks,
and
this
was
them
going
to
the
product
teams
to
answer
questions
that
we
had
with
regard
to
snapshot,
frequency
and
the
the
appliance,
the
backup
Appliance.
So
yeah,
that's
pretty
much.
All
we've
done
so
far.
I
think
we
could
maybe
maybe
in
this
epic
we
can
once
we
have
like
a
very
high
level
idea
of
like
cost,
and
you
know
calls
for
the
two
options
and
the
I
guess
the
impact.
B
I
would
say:
maybe
we
can
go
to
a
Solutions
architects
now.
Is
that
like
tell
them
like
Hey
we're
looking
into
the
backup
plans
we're
looking
into
the
region
of
provision
storage?
Our
main
goal
is
to
improve
the
RTO
and
our
po.
Maybe
they
can
provide
us
any
other
hints
that
they
can
give.
A
Us
yeah
yeah
I
guess
we
could
I
I
I
mean
our
conversations
with
them.
So
far
have
like
led
us
to
this
back
of
Appliance
and
you
know
using
snapshots
and
yeah.
That's
pretty
much
it
but
I
don't
know
if
we've
talked
to
really
like
a
Solutions
architect.
That's
you.
A
Know
where
this,
where
this
is
like
someone
who
is
knowledgeable
in
this
sort
of
thing,
so
maybe.
B
B
Yeah
I
feel
like
that
is
very
very
far
ahead
and
yeah,
like
my
problem
with
regional,
balanced
disk
storage,
I
feel
like
if
we
want
to
go
with
that.
We're
gonna
end
up,
deploying
it
Fleet
wide,
not
dirt
storage,
because
dirt
storage
just
makes
it
a
lot
more
complicated
right.
A
Okay
sounds
good,
so
I'm
gonna
take
the
action
to
create
the
epic
and
add
issues.
I'll
also
add
an
issue
for
this
meeting
with
the
solutions
architect,
so
we
can
just
track
it
and
I'm
expecting
like
maybe
some
of
this.
We
can
start
in
January,
but
it's
going
to
definitely
bleed
into
the
next
quarter.
Yeah.
A
A
I
I
think
that's
kind
of
going
to
be
up
to
whether
you
have
bandwidth
to
help
you
know
like
then
you
I,
I,
probably
don't,
but
at
least
probably
not
in
this
quarter,
but
maybe
maybe
we
can.
We
can
work
together
on
some
of
this
stuff
yeah
and
from
the
giddly
team.
I
think
what
I
would
like
to
understand
is
you
know
where
we
are
with
the
rav3
architecture
and
where
we
are
with
tiered
storage.
If
that's
in
the
road
map,
but
like.
B
B
A
B
You
know
what
that
means,
and
it
means
like
it's
another
few
more
quarters
to
deploy
it
on
github.com,
so
I
would
say
it's
I
would
say
like
a
year
or
two
away
to
be
honest
to
for
it
to
be
for
the
mature
of
our.com
and
then
that
implements
storage
as
well
right,
so
yeah
and
I
feel
like
that
is
very
far
away.
Unfortunately,.