►
From YouTube: 2020 04 27 Database Sharding Working Group
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
C
B
A
B
A
A
A
There
was
some
concerns
that
the
consensus
was
service
extraction,
Sid
had
some
concerns
for
sure,
and
we
had
a
meeting
there's
a
a
YouTube
livestream
of
it
there
and
the
big
big
driver
has
called
out
at
that
1440
times,
so
the
big
driver
we
have
behind
sales
and
iacv
is
that
we
are
a
single
application.
Single
datastore
and
the
concern
with
doing
moving
forward
on
service
extraction
was
that
we
can
no
longer
say
that
right
we
lose
a
big
market
if
we
start
having
multiple
services,
multiple
databases
so
continuing
to
focus.
A
The
request
was
to
continue
to
focus
on
database
sharding
and
to
do
some
prototypes
and
after
been
a
lot
of
conversations
over
the
last
couple
days.
So
we
talked
further
about
what
that
means
to
do
prototyping.
There
is
a
story
in
the
handbook
and
the
database
team
that
I
need
to
link
that
I
did
not
do
where
Andreas
has
done
some
work
on
what
partitioning
looks
like
at
the
issue
level
and
what
performance
gains
that
we
see
and
what
some
of
the
concerns
are
on
cross
partition
join.
A
So
I
will
add
that
in
here,
so
we
do
have
some
data
on
prototypes
that
we've
done
in
the
past.
There's
still
some
concerns
about
I,
guess
migrations
for
having
everything
in
a
single
database.
What
the
what
we
can
shard,
what
we
can't
shard
and
what
we
talked
about
just
literally
a
half
hour
ago,
is
continuing
to
talk
about
what
were
some
of
the
goals
of
service
extraction.
A
How
we
can
bring
that
into
the
single
application
and
single
database
and
the
term
that
andreas
came
up
with
was
identifying
the
isolating,
identifying
and
isolating
the
access
pattern.
So
we
can
understand
where,
where
we
would
be
inefficient
with
sharding
right
now,
based
on
the
access
patterns
and
namespace
was
a
good
example,
because
we
can
we'll
see
some
advantages
to
charting
at
the
namespace
level.
But
things
like
users
will
not
work
very
well
with
charting,
so
how
do
we
isolate
users
and
make
it
char
double
within
the
confines
of
single
application
single
database?
A
So
ondrea's
quickly
put
this
issue
together
here
under
what's
happening.
Next,
we
could
start
brainstorming
and
thinking
about
the
various
isolation
patterns
that
we
would
need
to
implement
to
be
able
to
properly
shard
the
database
and
have
that
horizontal
scalability
that
we're
looking
for
as
the
ultimate
outcome
of
this
working
group
and
database
sharding.
So
I'm
sure
there
are
a
ton
of
questions
and
a
lot
of
my
answers
will
be
I,
don't
know
yet,
because
a
lot
of
this
just
happened
like
within
the
last
eight
working
hours.
A
D
We
looked
at
the
group
managed
accounts
as
saying
that
whether
it
will
won't
be
effective
and
helping
a
shard
or
partition
at
the
namespace
level.
It's
a
new
feature
that
the
manage
team
is
building
for
me.
The
access
team.
Well,
they
just
manage
access
that
is
intended
to
provide
over
an
isolated
user
account
system
for
namespaces,
I'm
ghetto,
calm,
I.
A
A
D
If
that
makes
sense,
but
basically
that
you
can
think
of
one
promise
trying
to
solve,
which
is
you
have
a
you're,
a
part
of
a
company
account
say:
gitlab,
you
have
a
global
use
account
on
get
live.com,
you
take
a
private
repository.
You
fork
it
it's
now
in
your
personal
namespace
and
you
can
then
potentially
make
it
public
or
do
whatever
you
want
with
it.
And
so
or
you
know,
you
don't
float
something
else
to
your
personalized
base
or
something
like
that,
and
so
previously
I.
D
Don't
it's
self
managed
instance.
The
admins
would
have
full
control
over
all
personal
namespaces
on
the
instance
and
with
comments,
not
true,
and
so,
as
you
fork
things
in
your
personal
name
space.
They
sense
you
lose
control
over
them
and
their
only
recourse
is
like
DMCA
takedown
requests.
It's
like
not
good
so
so
this
provides
more
of
a
fully
managed
corporate
count
on
get
lab
comm
to
help
solve
some
of
these
challenges.
D
D
E
Now,
I
guess
one
of
the
problems
that
we
have
is
that
there
is
no
notion
of
an
organization.
Yet
in
the
application.
So
you
know
we
have.
We
have
many
users
and
they
can
all
participate
and
basically
all
the
names
places
they
have
access
to,
including
the
public
ones
and
that
sort
of
makes
it
hard
to
shard
by
an
organization,
because
we
don't
have
that.
E
But
if
we
had
the
opportunity
or
the
possibility
to
create
users
that
only
have
access
to
their
organization
to
the
repositories,
containers
issues
much
requests
of
their
organization,
and
it
would
be
much
easier
to
charge
by
that
by
that
I
mentioned,
and
that's
that's
also.
What
we
talk
about
when
we
talk
about
access
patterns
to
the
data,
so
I
think
it
makes
sense
to
figure
out
what
kind
of
access
patterns
we
have
today.
E
One
of
these
is
obviously
by
namespace,
so
because
a
lot
of
features
and
data
is
inside
a
namespace
and
that
lines
naturally,
but
there
are
other
data
access
patterns
where
you
come
in
with
your
user.
You
want
to
see
that
all
your
assignments,
for
example
and
I,
think
we
should
figure
out
those
those
access
patterns
and
figure
out
how
to
isolate
them,
because
that
enables
us
to
actually
apply
a
charting
technique
later.
D
Yeah,
no
it's
so.
Basically
it's
from
looking
at
it.
I
linked
under,
what's
been
done,
like
I,
said,
play
not
the
best
place
to
linking
what's
coming
next,
but
check
it
out.
It's
already
added
and
for
a
little
bit
of
time,
it
actually
looks
like
told
that
one
seems
old,
but
there's
more
coming
here.
I'll
try
and
find
a
better
epoch,
but
the
idea
is
that
it.
D
It
essentially
prevents
them
from
sharing
things
outside
the
top
of
the
namespace
and
so
I
think
they.
So
they
can't
share
anything
outside
of
it,
but
they
can't
access
other
public
projects
across
the
database
across
the
instance,
as
anyone
would
be
able
to,
but
I
would
think
with
namespace
starting
though
I
mean
even
though
it's
sharted,
we
would
still
need
to
be
able
to
act.
I
cross
access
things
right,
like.
D
A
A
We'll
get
some
more
concrete,
next
steps
in
this
issue
below
on
what
we're
trying
to
identify,
how
we're
gonna
go
about
identifying
that
what
we
think
is
chargeable
what
we
think
we
need
to
change
to
be
more
shareable,
so
they're
just
there's
a
lot
of
work
that
happens
with
a
slightly
directional
change,
so
I
wish
I
had
more
information,
but
there
was
just
a
lot
that
came
down
over
the
last
couple
days.
So
that's
where
we
are
do.
F
E
Perhaps
that
makes
sense
to
talk
about
an
example.
We
use
issues
quite
a
lot
in
recent
examples,
and
so
we
have
issue
data.
You
know
descriptions,
you
know
with
comments
on
that,
and
this
is
always
inside
a
namespace
right,
an
issue
currently
outside
a
namespace,
so
you
could
think
that
drew
we
can.
E
So
in
this
case
you
don't
have
a
namespace
available
and
that
that
doesn't
align
with
with
the
namespace
sharding
very
well.
So
that's
what
we
mean
when
we
say
that
some
things
are
not
shard
of
all:
it's
not
that,
because
the
data
is
not
char
double,
but
because
we
have
these
conflicting
access
patterns
where
you
want
to
have
one
perspective,
and
you
also
want
to
have
the
other,
and
we
need
to
figure
out
how
to
isolate
that,
and
one
thing
that
is
Aspen
brought
up
was
service
extraction.
E
You
could
think
that
you
extract
that
into
into
a
service
and
isolated
that
that
way,
but
it
could
also
mean
that
we
we
only
isolated
on
the
codebase
so
doesn't
have
to
be.
A
service,
could
also
just
be
a
module
and
the
companies
with
the
proper
interface
to
to
hide
that
from
you,
then,
ultimately,
we
may
be
able
to
apply
a
namespace
sharding
to
all
the
things,
because
this
is
for
actually
the
majority
of
the
data
lives
once
we
have
sort
of
removed
those
those
conflicting
access
patterns.
F
E
Totally,
for
example,
we
also
have
global
search
and
and
CIA
has
a
global
scope
as
well
as
look
if
I
understand
it
correctly
to
do
so,
our
user
base.
So
it's
not
that
everything
results
in
a
namespace.
What,
if
there's
more
more
of
these
I.
A
Think
the
term
not
chargeable
is
probably
a
bit
of
a
misnomer
right.
We
could
shard
the
database
in
various
and
spectacularly
bad
ways.
It's
just
not
performant,
not
scalable,
to
shard
it
in
some
of
the
ways
that
that
we're
talking
about
so
the
goal
of
the
access
isolation
is
to
make
it
to
where
it's
actually
performant
and
provides
value
by
scaling
horizontally
and
avoids
a
lot
of
these
known
pitfalls
like
the
cross.
Shard
joins.
C
Just
idea
from
my
set
thanks
Josh
for
sharing
the
curve
search
partitioning
are
the
I'm.
Sorry
that
group
managed
accounts.
One
piece
of
advice
I
can
give
is
if,
if
we
need
to
just
pick
on
one
thing
to
shard
on,
just
as
for
the
prototyping
for
the
proof
of
concept
right,
like
just
know,
the
team
has
that
flexibility
to
use
that
is
an
iteration
know
that
there's
a
lot
of
complexities
know.
There's
gonna
probably
be
some
way
that
we're
gonna
change
it
but
like
if
it's
just
to
get
kind
of
the
vet,
that
it's
saying.
C
Okay,
here's
how
the
applications
gonna
have
to
change.
You
know,
or
here's
where
we're
gonna
have
to
go,
implement
new
code,
because
we're
gonna
have
to
work
across
drugs.
That's
the
end
point:
that's
the
intent
of
the
perfect
concept
right
is
to
uncover
that
that
level
of
work
associated
with
it,
which
I
don't
want
the
team
to
get
hung
up
on
too
much
is,
is
exactly
which
formed
a
Charlaine.
Unless,
like
you
know
the
first
one,
they
tried
is
really
bad
in.
A
Yeah
and
I
probably
didn't
advertise
it
enough,
but
Andres
did
some
great
work
on
the
issue
group
search
as
an
example
for
partitioning
and
there's
some
really
good
data
there
on
what
worked
well
for
partitioning,
which
would
be
the
logical
progression,
the
sharding,
with
foreign
data
wrappers
and
Postgres.
So
we
do
have
some
prototyping
information
there
and
we
can
iterate
on
that
or
take
a
different
example
for
an
iteration,
and
I
guess
that
we'll
discuss
further
in
this
issue
here.
C
C
C
D
G
G
G
A
G
Most
based
on
our
based
on
our
current
knowledge,
there
are
no
customers
that
will
ever
get
to
that
level,
because
basically,
our
biggest
customer
fits
in
with
every
other
gitlab
user
on
a
single
database
right
now.
So
it's
basically
there's
basically
no
use
case.
Besides
get
lab
comm,
unless
somebody
was
to
create
a
competitor
to
get
lab
comm
and
they
can
go
like
there
is
literally
no
customer.
That
is
big
enough.
That
could
justify
a
sharding
solution.
D
C
It's
just
the
opposite
of
the
service
distraction.
It's
that's
my
understanding,
so
I'm
just
trying
to
channel
the
team
to
make
sure
I
understand
quickly.
But
you
know
service
abstraction
causes
you
to
have
latency,
because
you
got
to
go
to
several
services
to
do
it.
If
you
shard
it
improperly,
you're
gonna
have
to
go
to
so-called
DPS
to
basically
do
the
same
operation.
So
you
can
get
super
cumbersome.
If
you
don't,
if
you
don't
do
it
correctly,.
D
D
Yeah
be
good
to
know
sort
of
where
we
think
the
scaling
problems
are
like.
It
sounds
like
the
user.
Stable
is
one
that's
like
cause
for
concern,
I'd,
be
curious
to
know
if
it's
five
years,
ten
years
out
from
now
and
and
then
what
everything
starting
would
would
be
a
helpful
solution
to
that
problem.
I.
A
A
G
I
have
a
scheduled
meeting
tomorrow
with
the
cockroach
lads
to
talk
about
how
far
they've
progressed
with
their
rails
adapter
and
what
what
they've
been
actually
independently
looking
into
how
to
run
get
lab
on
top
of
cockroach
TB.
So
they
should
be
it's
just
a
quick
sync
up
to
see
how
they're
doing
with
that
and
and
and
see,
we'll
see
how
viable
running
on
get
lab
on
cockroach
TB
is
I.
Think.
G
This
this
is
just
to
see
in
their
opinion,
how
viable
it
is
because
they
they've
been
working
on
this
independent.
They
they
have
better
knowledge
of
how
viable
it
is
as
a
database,
and
so
it's
just
and
and
what
they've,
what
things
they've
run
into
when
trying
to
run
get
lab
on
a
cockroach,
TV
doctor,
okay,.
A
We
are
coming
up
on
time
any
other
topics
we
should
cover
before
we're
done.
Any
questions
folks
have
I'm,
not
sure
about
container
registry
dan
was
in
on
the
call
where
we
talked
about
having
prototypes
for
that
and
what
they
are
going
to
implement
and
the
different
options
they
have
on
the
database
side.
A
But
I
would
hesitate
to
go
into
further
detail
because
I,
don't
know
enough
about
that
to
intelligently
describe
what
they
are
gonna
work
on
and
where
their
database
lives
and
does
not
live
so
I
will
just
leave
it.
They
are
going
to
work
on
some
various
prototypes
for
the
container
registry
and
how
it
interacts
with
our
database.