►
From YouTube: Day 3 - Pairing on spiking out a developer framework for adding data types to be replicated by Geo
Description
Working on issue titled "Geo: Spike Self-Service Framework with Package Files": https://gitlab.com/gitlab-org/gitlab/issues/197319
B
A
A
A
D
D
When
it's
consuming
the
data,
so
I
talked
it.
I
talked
with
Tom
today
and
I
think
we
both
agree
that,
ideally
somewhere
in
the
code
base,
you
will
just
say
in
a
more
top-down
level
like
I
need
to
copy
a
repository
from
A
to
B
or
I
need
to
copy
a
file
from
A
to
B,
and
this
should
be
like
the
high
level
things
that
a
developer
using
our
framework
will
do.
D
D
The
jela
cursor
reads:
events
from
the
queue
then,
instead
of
it
actually
calling
and
the
process
method
and
the
process
method,
scheduling
something
and
psychic
I
think
it
makes
sense
that,
as
I
step
before,
we
always
schedule
a
sidekick
job,
and
this
like
this,
generic
sidekick
job
will
always
call
the
correct
process
method.
So
the
process
method
inside
the
replicator
will
actually
work
as
a
process
method
inside
a
psychic
job
like.
A
D
It
as
a
summary,
and
that
doesn't
prevent
us
from
finding
other
psychic
jobs
from
that
process.
If
that
makes
sense,
I
think
this
first
step
is
important
because
we
are
not
holding
the
cursor
and
I
think
this
is
not
only
important
for
how
we
have
it
right
now,
but
as
future
possible
interactions.
If
we
have
something
like
a
message
queue,
the
actual
message
queue
you
don't
want
you
to
like
to
hold
the
the
whole
thing
is
about
an
acting
as
you're
processing
and
code
on
the
same
train,
yeah,
so
yeah.
A
A
You
I'd
rather
write
the
entire
cue
system
to
be
basically
optimistic.
It
is,
it
is
going
to
accomplish
whatever
we
intend
to
accomplish
and
anything
that
is
so
important
that
we
need
to
guarantee
that
it
is
run.
We
have
to
have
some
other
means
of
doing
that.
Okay
of
making
that
guarantee
like
backfill
and
verification.
E
D
So,
with
the
current
state
of
things,
there
is
one
thing
that
we
yeah
well
for
the
jobs
that
we
process
now:
the
pollsters.
We
have
verification
and
we
have
McFeely.
We
have
that
feeling
for
for
all
of
them
right
now
and
for
the
new
ones.
I
think
we
expect
you
to
have
the
same
mechanisms,
so
I
think
it's
safe
to
go.
That
way
like
we
can
existing
behavior
as
it
is
today
and
as
soon
as
we
remove
them
to
the
new
framework
and
make
sure
that
they
all
comply
with
it.
So.
A
C
A
The
job
is
lost
that
performs
this.
You
know,
then
it's
that's
it
there's
no
failsafe
and
then
I
think.
Similarly
for
these
hashed
storage,
migration
and
stuff,
like
that,
maybe.
D
Like,
let's
say
that
by
default
they
are
always
a
sink,
but
you
can
specify
that
one
is
not
a
sink
and
it
should
be
recessive
by
the
cursor
like
by
the
main
train.
But
this
is
let's
see
so
we
have
problems
today
because
sidekicks
not
a
hundred
percent
reliable
right
and
we
we
want
to
be
able
to
actually
rely
that
we
are
not
losing
anything
for
caching
motivation.
D
If
we
don't
replicates
the
migration,
what
will
happen
is
that
the
repository
verification
will
be
triggered
at
the
endpoint
and
we
might
find
the
new
upholstery
on
the
new
location
and
it
will
fix
that
lacking
of
data
there.
We
also
has
to
have
it
on
the
old
director,
so
you're
not
losing
data.
That's
fine!
It's
recoverable!
A
C
D
I
know
what
what
you're
talking
about
if
the
legacy
storage
it
that
there
is
a
possibility
that
let's
say
you
moved
something
to
another
location
and
someone
tries
to
create
a
new
project
or
new
something
with
the
same
name.
Let's
say
that
you're
in
the
same
namespace,
then
they
K
that
they
can
have
access
to
that
old
project
content.
But
that's
not
the
case
with
I
should
start
because
it's
always
on
this.
It's
always
a
new,
a
new
thing
and
it
will
never
collide.
A
The
event
contains
information
about
where
the
thing
is
that
we
need
to
delete
are
same
with
all
of
the
deleted
events,
and
if
we
fire
off
or
we
Inc
you
a
job
and
we
delete
the
event.
You
know
because
it's
just
in
a
queue,
then
the
the
job
data
is
the
only
place
where
we
know
delete
something
and,
and
that
can
be
lost
well.
D
D
Instead
of
removing
it
completely
from
from
memory
it
it
kinds
of
reserve
that
thing
for
you
like,
say:
okay,
it's
taken
by
someone
and
there
is
a
TTL
where
you
have
to
at
the
end.
You
have
to
say:
okay,
it's
done
or
you
need
to
renew
that.
So
if,
for
example,
either
your
job
crashes
or
takes
too
long
to
renew
it
goes
back
to
the
queue
to
the
main
queue,
and
this
is
a
guarantee
of
the
architecture.
D
A
It
yeah
anyways,
there's
the
there
isn't
actually
redundancy
right
like
it's,
it's
more
reliable
of
a
job,
but
if
something
does
go
wrong,
which
is
always
possible,
there
is
no
redundancy
and
so
I'm
gonna.
A
Idea
out
there
about
this
to
add
a
redundancy
which
is
that
maybe,
like
maybe
the
structure
of
Geo,
all
along
should
have
been
in
the
in
like
when
we
want
to
replicate
something
whatever
like
we
store,
we
store
where
we
store
where
it
is
supposed
to
be
and
on
the
secondary.
It
also
has
it's
an
independent
store
of
where
things
are
supposed
to
be,
because
they
can
be
out
of
sync
right,
and
if
the
time
comes
to
delete
something,
then
we
always
have
that
data
like
say
on
the
registry,
for
example,
mm-hmm.
E
D
Yeah,
let's
suppose
that
we
are
going
to
change
the
cue
system
like
in
the
near
future,
one
one
possible
way
to
to
make
it
more
reliable
or
to
behave
as
this
wrapped
MQ
example
would
be
to
have
actual
cue
use
it,
as
is
example
in
RabbitMQ.
So
let's
say
that
we
are
using
something
that
does
support
TTL.
D
A
A
D
A
D
A
D
A
So
so
part
of
why
I
think
it
would
be
good
for
geo
to
store
where
something
is
is
because
currently,
we
kind
of
like
for
every
model
that
we
replicate,
we
kind
of
like
derive
where
its
location
is,
and
we
do
the
same
thing
on
the
secondary
and
I.
Don't
know
it
just
like.
If
it
does,
if
it
changes,
then
we
I.
D
Let's
say
that
there
are
some
jobs
that
are
fine
to
lose
like,
for
example,
the
replication
of
the
poster
like
it's
not
like
the
end
of
the
world.
If
we
miss
one
because
it
will
be
eventually
recovered,
but
for
the
other
things,
maybe
what
we
need
is
to
actually
do
it
in
in
two
steps:
confirmation
things
so,
for
example,
the
let's
say
that
we
absolutely
want
to
make
sure
cache
is
invalidated.
D
E
D
We're
patching
sidekick
off
of
it,
so
we
don't
have
to
rely
that
sidekick
will
magically
work
forever
and
there
being
no
bugs
and
whatsoever
so
wait.
We
like
we
fire
that,
but
then
later
on,
we
check.
If
it
was
done
it
will.
It
works
kind
of
like
the
back
she
named
for
the
requesters,
but
she
for
things
that
and
there
is
no
way
for
us
to
actually
confirm
or
to
their
shine.
D
A
A
Sometimes
people
sometimes
customers
do
come
to
us
with
like.
Oh
you
know,
why
is
the
secondary
using
like
three
terabytes
and
the
primary
use
just
one
you
know,
or
something
like
that
mm-hmm
but
I
guess
like
it,
wouldn't
be
that
bad
to
just
automate
some
kind
of
like
one
of
one
of
the
things,
one
of
the
rake
tasks
that
are
or
all
of
the
rake
tasks
that
we
have,
that.
A
A
D
Think
that
the
main
difference
is
that
we
don't
have
to
test
that
the
psychic
worker
was
triggered
as
part
of
the
so
I
was
trying
to
let's
go
back
a
little
bit.
I
was
trying
to
have
the
main
logic
be
available
in
one
location.
So
if
today
we
want
to
like
simulate
the
replication
of
a
repository,
you
have
probably
to
open
10
different
files
to
figure
out
what
are
the
parameters
that
you're
passing
from
here
into
there
and
what
is
like
the
whole
flow.
D
If
we
understand
that
that
job
is
receives
these
parameters
and
it's
part
of
the
framework
that
it
always
be
executed
inside
a
sidekick
job,
that
is
generic
again,
we
know
how
how
it
works
and
we
can
like
provide
standard,
vlogging
and
metrics,
etc.
Then
best
thing
that
something
is
being
executed
means
you
have
to
call
only
one
function
that
you
know
how
it
works,
so
you
don't
have
to
test
the
same
thing
every
time
this
is,
it
should
be
tested
as
part
of
the
framework,
but
not
that's
part
of
the
job.
That
makes
sense.
Yeah.
D
D
D
A
A
A
A
A
D
When
the
cursor
goes
by,
it's
super
fast
and
start
scheduling
jobs,
but
at
some
point
the
one
job
you
want
to
execute
it
right
away.
I
think
that's
the
intention
of
the
synchronous
ones.
What
we
can
do
instead
is
maybe
allowing
you
to
specify
a
different
view
or
a
priority
or
low
priority
queue,
something
like
that
as
part
of
this
job
consumption.
D
D
A
D
A
D
A
D
A
E
D
B
D
A
B
A
C
D
A
F
A
D
E
D
Think
what
what
is
weird
is
that
we
are
you're
handling
these,
as
has
another
event,
if
that
makes
sense,
and
it's
just
where
the
code
is.
That
is
a
little
bit
weird,
but
maybe
can
be
moved
like
two
to
the
main
loop
and
like
if
otherwise
it's
this
type
of
event
does
whatever
it's
doing
now.
Otherwise
do
this
new
way
of
handling,
because
the
idea
is
that
we
are
going
to
remove
code
from
from
the
cursor
you're
like
oh,
how
this
event
files
will
be
removed
as
we
migrates
them.
D
D
D
A
This
our
names
face.
Oh
these
are
namespaced,
no
eggs.
A
E
A
B
E
E
E
A
D
D
A
E
A
A
B
D
D
A
B
C
B
D
E
A
F
A
F
F
F
F
F
A
E
A
F
A
F
E
A
E
A
C
A
E
C
F
F
A
B
C
C
C
C
C
C
C
C
F
A
A
B
A
A
F
So
what
one
one
thing
that
last
time
that
I
saw,
how
like
we
are
using
the
API
for
like
this
kind
of
stuff,
is
that
we
we
ended
up
building
a
kind
of
a
controller
but
with
a
different
name.
F
E
F
E
F
Also
combat
a
little
bit
and
the
retriever
the
babysitter
yeah,
so
it
also
again
concerned
it
about
like
how
it's
going
to
hurt
things
back
to
the
request
and
I.
Don't
think
that
the
retriever
should
be
doing
that
like
it
should
only
pass
the
time
either
it
gets
the
file
or
doesn't
get
the
file
on
how
we
are
going
to
present
this
and
it
shouldn't
be
here.
Maybe
we
can
actually
use
a
presenter
to
do
exactly
that
or
something
like.
A
F
A
Do
you
think
what
we're
doing
like
is
the
changes
we're
talking
about
are
safe
enough.
You
know
what
I
mean
because,
like
we're
gonna
feature
flag,
what
we're
doing,
but
if
we
make
a
lot
of
changes
to
how
this
works-
and
maybe
we
you
know
I,
don't
know.