►
From YouTube: 2020 06 08 Database Sharding Working Group
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
Thanks
everybody
here
for
the
database,
sharding
working
group
and
actually
thought
it'd
be
a
good
time.
To
recap
where
we
were
from
last
meeting
in
actually
probably
the
last
couple
weeks,
there's
been
a
lot
of
important
developments
and
clarifications
that
have
happened
over
the
last
that
last
period
so
run
through
them
real,
quick
and
feel
free
to
interrupt
if
I'm,
going
too
fast
or
if
I.
If
we
need
some
clarification
on
something.
A
When
the
working
group
first
started,
we
thought
the
wall
was
much
closer
and
we
had
a
critical
scalability
issue,
but
it
seems
as
though
that's
been
pushed
back
in
12
months,
and
that
does
not
mean
at
month
13
we're
going
to
hit
a
wall,
and
this
is
a
metric
we're
going
to
re-evaluate
every
month,
though
next
point
so
concerns
about
auditing.
Our
website,
partitioning
by
time
on
on
on
events,
has
been
alleviated.
A
It's
not
something
that
will
block
us
if
we
choose
a
different
partitioning
or
sharding
strategy,
it's
it's
if
they
are
in
conflict,
it's
a
migration
event.
So
if
we
need
to
repartition
audit
events
on
namespace,
if
that's
possible,
then
we
would
just
migrate
from
the
time
partitioning
to
a
namespace
partitioning
and
the
time
that
takes
to
do
that.
Work
is
equal
to
hit
it
will
take.
We
can
reuse
the
migration
tools
that
we've
created
for
audit
events
for
the
new
partitioning
strategy.
If
that
should
happen,
but
it's
not
a
blocker.
B
C
D
A
Good
I'm
glad
we're
having
the
recap
so
all
valid
comments
and
concerns
and
they're,
not
all
recaptures.
This.
This
concern
came
from
Sid
that
he
thought
that
going
down
this
was
orthogonal
to
the
overall
strategy
of
namespace
partitioning
and
the
the
reason
we
talked
about
this
last
week
was
to
allay
those
concerns
were
said
that
this
would
not
stop
us
from
namespace
partitioning
and
that
to
Alberto's
point
we
could
have
both
I'm
sorry
namespace
sharding.
We
could
have
both
we
could
still
namespace
yard
and
then
on
the
individual
shards.
A
We
could
still
partition
by
this
strategy
as
well
and
yes,
Josh.
That
was
another
concern
about
the
number
of
partitions
that
were
created
and
those
questions
are
answered
deep
in
some
of
the
related
issues
within
audit
events.
Partitioning
epoch
so
think
we're
okay
with
the
number
of
partitions
that
we've
identified,
we're
looking
at
96
and
the
question
becomes
around
100
to
200,
if
you're
starting
to
hit
a
point
where
you're
going
to
hit
some
performance
problems.
So
those
questions
have
been
answered
in
this
in
within
the
epoch
itself.
A
So
the
tenant
sharding
discussion
Sid
requested
a
meeting
to
talk
further
about
details
on
that
I
believe
it
was
originally
scheduled
for
today,
but
that
got
pushed
back
to
the
22nd.
In
the
meantime,
I
think,
there's
probably
a
couple
of
conversations
that
we
should
have
Andrew
Josh
and
whoever
else
wants
to
be
involved
to
kind
of
hammer
out
some
of
the
details
and
maybe
get
some
ideas
on
what
an
MVC
first
iteration
would
look
like
for
that.
So
we
can
compare
to
some
of
the
research.
A
That's
already
been
done
on
namespace
sharding
and
some
of
the
changes
that
we
wouldn't
debate,
because
there
are
entries
in
the
handbook
on
the
database
section
about
what
changes
would
need
to
take
place.
If
we
did
namespace
charting
on
issues
what
application
changes
would
need
to
take
place.
What
would
also
need
to
take
place
in
the
database
so
making
sure
that
both
the
development
part
and
the
product
changes
are
covered?
I
think
we
should
probably
have
a
couple
sinks
on
that
before
we
meet
up
with
Sid
on
June,
22nd
and
I
will
setting
those
up.
A
There's
some
parallel
work,
going
mostly
on
the
database
team
right
now
to
improve
scalability.
These
are
I,
keep
calling
them
incremental.
They'll
get
us,
probably
NX,
not
the
100x
overhead
improvements,
but
it
will
get
us
some
improvements
on
database
scalability
side
right
now.
The
main
focus
is
partitioning
of
audit
events,
I'm,
pretty
optimistic
that
we
can
have
the
database
implementation
in
production
next
week.
I'm
sorry
next
milestone
and
what
that
looks
like
is
we
would
have.
A
A
On
the
partitioning
keys,
because
we're
going
by
created
at
it's
going
to
be
monthly,
if
you
as
an
example,
if
you
pick
the
last
seven
days,
there's
a
pretty
good
chance,
you're
going
to
span
two
different
partitions,
which
isn't
terrible
but
we're
gonna,
try
and
drive
behavior
more
towards
the
optimistic
path
on
date
pickers
or
at
least
that's
what
we're
talking
about
product
management
about.
So
the
TLDR
tables
will
be
in
the
production
database,
we'll
be
able
to
test
side
by
side
just
using
console
instead
of
actually
going
through
the
application.
A
And
if
you
take
a
look
at
the
epoch,
we
are
making
sure
that
the
health
status
represents
where
we
are
for
the
milestone.
All
these,
so
we're
on
track
for
pretty
much
everything
we
have
for
13:1
this
one
specific
issue
about
partition.
Audit
events
table
that's
kind
of
just
a
summary
of
the
work.
That's
being
done.
I
did
not
author.
This
issue,
I,
would
actually
probably
recommend
closing
it,
but
I
think
other
teams
are
using
that
to
track
so
we'll
circle
back
on
that
one,
but
making
good
progress
on
partitioning
all
that
events.
A
For
now,
and
then
there
are
some
other
incremental
scalability
improvements
that
we
have
listed
under
database
girl,
database
growth
and
the
epic
is
listed
under
the
EMR.
There
are
a
bunch
of
changes
that
I've
added
to
the
mr4.
This
handbook
entry
for
this
working
group,
but
I,
would
recommend
going
taking
a
look
and
sending
me
some
feedback.
A
One
thing
that
I'm
I
recommend
is
we
change
the
name
of
this
working
group
from
database
sharding
to
charting
the
focus
is
no
longer
just
on
the
database
side.
A
significant
application
changes
that
need
to
be
made
need
to
make
sure
that
product
management
is
fully
involved
on
making
sure
those
application
changes
are
accounted
for,
and
we,
you
know
we
might
have
to
get
some
more
dedicated
development
teams
on
this
as
well.
F
Put
a
note
in
there
happy
to
review
it
I
started
trial
and
said
it
Craig,
but
you
may
want
to
push
me
on
it
this
week
to
make
sure
merged.
Ok
makes
this
to
me
yep.
That
was
a
lot
of
information
that
changing
yeah,
like
the
name
change
makes
sense
to
me.
I
should
say
I
mean
the
details.
Stuff
I
saw
was
changing,
so
I
want
to
just
make
sure
I
understand
that
yep.
A
And
I
fixed
josh's
title
in
there
I
think
he's
listed
as
a
Geo
product
manager
as
well,
so
lots
of
lots
of
things
changed
alright
and
then
I
kind
of
covered.
What
we're
talking
about
for
partitioning
on
the
database
team
side?
These
will.
These
are
all
in
support
of
getting
the
partition
tables
into
a
testable
state
and
then
benchmarking,
so
I'm
actually
finishing
up
the
migration.
Benchmarking
are
up
next.
F
G
G
D
So
I
think
I'm
not
aware
of
a
separate
efforts
around
like
that,
but
I
do
think
depending
on
maybe
the
outcome
of
the
meeting
with
Sid
I'm
intimate
charting
that
Jeremy
should
potentially
be
involved
there,
but
also,
if
you
said,
gofer
rapport
with
that,
we
should
probably
make
sure
Jeremy
gets
decided
here
as
this
he
sort
of
is
the
product
manager
for
that
area.
That's
that
how
about
your
question
or
Mavis
understanding
yeah.
G
D
Well,
I
think,
I'm.
Sorry,
it's
replace!
Okay.
If
we
go
namespace
the
application
user
experience
doesn't
really
change
right
like
it,
it's
the
same
function.
It's
the
same,
look
and
feel
it's
kind
of
a
multi
experience.
Where
you
can
span
groups
you
can
span
to
stuff,
like
that's
all.
That's
all
the
same
right,
I
think.
A
When
you
say,
if
we
go
namespace,
we
talking
about
namespace
sharding,
yes,
sorry,
okay,
I'm,
not
necessarily
true,
there
will
be
some
product
decisions
that
need
to
be
made
and
I
can
link
you
back
to
the
handbook.
Entry
we're
under
s
did
an
example
of
searching
by
issues.
So
if
we
there's
some
product
decisions
that
he
calls
out
in
there
like
it'll
work,
just
fine
if
we
narrow
the
scope
of
search
to
the
namespace
that
the
user
is
in
or
in
depending
on
the
functionality
within
get
lab.
A
But
if
you
look
at
like
the
issue
board
for
a
user
that
will
most
likely
break
if,
if
we
go
with
namespace
sharding,
because
the
issue
board,
they
can
look
at
every
namespace,
the
users
involved
in
and
then
the
query
would
have
to
span
all
of
those
shards.
So
there
will
be
some
product
changes
that
would
need
to
happen.
Okay,.
D
A
G
D
For
Brendan
lieutenant
model,
that's
going
to
basically
have
a
more
isolated
experience
for
tenants
and
get
lab
and
get
away
from
the
github
model
which
is
sort
of
where
it's
all.
You
know,
it's
all
kind
of
one
big
common
area
right
and
shifting
more
towards
in
a
slack
model
I,
you
know,
then
Jeremy
should
be
involved,
I
think
because
the
t
owns
this
product
area
as
far
as
the
managed
product
manager
right,
which
is
isolation
and
he's
already
driving
teachers
in
this
area,
like
managed
accounts
and
things
like
that.
So.
A
G
A
That
sounds
like
some
pretty
good
overlap.
There
I
mean
we
have
the
goals
for
this
working
group
for
scalability
and
customer
isolation.
So
getting
Jeremy
involved
makes
sense
and
the
details
of
how
he
will
be
involved
in
the
world
clarify,
as
we
figure
out
name,
space
versus
tenant,
know,
I'll,
make
sure
he's
involved.
I.
B
Agreed
that
in
principle,
something
like
named
space
sharding
may
require
some
application
changes,
but
just
due
to
the
optimizations
needed
when
some
queries
that
are
gonna
hit,
service
charts
need
to
be
performed,
but
not
necessarily
from
a
change
or
reduction
reduction
in
functionality.
Even
if
the
users
board
is
going
to
be
shown,
you
can
I,
learn
it
through
the
throw
the
query
through
that
or
in
a
door
paying
some
latency
penalty,
or
you
can
do
that
very
manually
in
parallel
to
all
the
charts
from
the
application
side.
B
Yet
still
you
can
keep
the
same
functionality,
but
maybe
a
hidden
performance
may
need
some
work,
but
in
theory,
from
a
general
perspective,
it
may
remain
with
the
current
functionality,
whereas
the
older
model,
it
is
quite
simple,
simpler
I,
would
say
to
implement,
but
in
which
person
changing
on
the
product
functionality
so
that
that
I
think
is
the
balance.
But,
theoretically
again
the
charting
by
namespace
model
may
not
require
to
change.
You
should
proceed
functionally.
D
That's
why
I
think
that
to
a
business,
if
is
more
about
database
sharding?
And
it's
not
going
to
impact
the
broader
user
experience
significantly,
then
you
know
I'm,
not
sure
Jeremy
really
Minds
too
much,
but
if
we're
gonna
go
to
attend
isolation
model
over
to
make
dramatic
changes
to
just
isolation
in
general,
we
should
definitely
be
involved.
A
F
Yeah,
it
was
more
of
thanks
for
putting
the
table
together
and
it
kind
of
listed
out
that
mr
request,
if
files
tables
the
largest
table
but
like
based
on
the
proposal,
it's
gonna
be
much
smaller
like
you
know,
basically
an
order
of
magnitude
change
which,
to
me
is
a
30
times
multiplier.
So,
like
I
looked
at
that
was
excited
about
that
I
guess.
The
question
was,
though,
is
when
I
looked
at
the
issue.
A
Actually,
Alberto
and
I
were
just
talking
about
that,
so
the
in
the
background,
the
external
storage
was
happening,
but
something
happened
recently
to
cause
it
to
break
and
it
was
incredibly
slow.
I
think
we're
looking
at
multiple
months
before
this
was
completed
so
I
haven't
looked
since
Alberto
and
I
talked
about
this
and
late
last
week,
so
is
ongoing
and
there
might
need
to
be
a
gentle
nudge
to
make
sure
that
it
started
back
up.
If
it
has
not
continued.
Does
anybody
know
if
that
migration
has
resumed?
A
F
E
E
So
yeah,
the
other
sorry
Craig,
just
I,
think
the
other
thing
we
got
to
discuss
with
that
is
it's
taking
or
it's
gonna.
If
you
look
at
the
the
rate
of
which
is
it's
moving
over
to
it's
just
really
really
slow,
and
so
even
if
we
fix
all
the
bugs
you
know,
we
can't
have
this
thing
taking
two
years
or
even
a
year
to
do
this
migration,
and
so
we
need
to
have
a
discussion
around
that
as
well
at
some
stage
just
to
raise
that
act
as
a
point.
Okay,.
F
E
Is
there
who's
the
DRI
we're
running
that
migration,
because
it
kind
of
feels
like
we've,
just
sort
of
decided
that
it's
actually
an
infrastructure
is
cool
but
I,
don't
know
who's
driving.
This
so
I
mean
that's
sort
of
outside
the
scope
of
this
working
group,
but
I
don't
know
who
it
is,
but
someone
should
be
running
as
it
feels
like
it's
kind
of
not
owned
by
anyone.
You
know.
F
That's
a
that's
a
fair
origin.
Let
me
put
that
on
my
schedule
to
talk
to
Steve
about
that
kind
of
like
from
that
perspective,
the
reason
why
I
said
and
prefer
to
because
I
just
saw
the
namespace
for
infrastructure.
So
then
I
just
was
like
okay.
You
know
it's.
My
first
lecture
Simon
telling
me
that's
my
first
indicator
for
which
purpose
responsible,
yeah.
F
A
So,
and
to
address
your
point
earlier,
Christopher
about
you
know
shrinking
this
table
gives
us
30x
overhead,
yes,
and
no.
So
yes
on
database
storage,
but
no
now
on
the
overall
picture
right
because
it
doesn't
include
like
CPU
memory,
I/o,
all
the
other
things
that
we
consider
when
we're
looking
at
scalability.
So
it
is
one
improvement,
but
not
the.
F
A
Along
those
lines,
like
the
top
three
tables,
take
up
well
over
70%
of
our
database
storage
and
that
is
covered
under
the
database
growth
epoch
that
we
have
just
looking
into
retention
strategies.
How
we
can
shrink
that
how
we
can
make
it
more
manageable.
So
all
considerations
we're
looking
at
and
then
mek.
C
A
Don't
know
I
would
hope
by
then
we
have
settled
on
like
the
big
questions.
Now
our
name
space
versus
tenant,
which
approach
are
we
going
to
take
and
having
some
goals
set
by
then
I?
Don't
I'm,
not
confident
we
would
have
something
delivered
by
then
so
yeah.
We
should
probably
consider
changing
it
or
clarifying
the
goals
that
we
have.
You
know
we've
chosen
a
strategy.
We
have
an
MVC
identified
and
first
implementation
by
a
certain
date
format
with
data
from
those
decisions.
So,
yes,
the
data
is
very
questionable.