►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
How to Migrate a MySQL Database to Vitess - Sugu Sougoumarane, PlanetScale & Morgan Tocker, PlanetScale
Vitess is a cloud-native storage solution that can scale indefinitely. This session will cover a high level overview of all the Vitess features, the architecture, and what database workloads are a good fit. We will then walk through a demo of live-migrating an existing MySQL installation into Vitess. Because Vitess also speaks the MySQL protocol, it is easy to retrofit scaling into your existing database systems.
https://sched.co/UaiN
A
So
today
we're
gonna
be
doing
a
presentation
and
to
talk
about
migrating
from
MySQL
to
MySQL,
with
the
tests
I
had
intended
to
do
a
little
demo
at
the
end,
I
made
some
assumptions
about
the
networking
that
turned
out
to
be
not
true,
so
I'm,
not
sure
of
that'll
work,
but
I'll
try
the
best
I'll
talk
through
the
examples
and
sort
of
like
talking
about
using
a
canary
to
make
sure
that
our
risk
is
low.
Really
I
want
to
take
the
focus
here.
If
you
know,
you've
got
an
application.
This
is
the
application.
A
A
I'll
sort
of
preface
that
by
giving
a
very
brief
introduction
about
the
test
for
those
who
are
not
familiar
and
I
will
ask
for
a
show
of
hands
who's
not
familiar
with
the
tests,
all
right,
so
I
was
as
justified
in
adding
this
slide.
That's
about
50%,
so
the
tests
in
in
summary
is
a
cloud
native
database.
So
the
test
allows
you
to
operate
essentially
MySQL
and
we'll
get
to
that
in
a
second
in
a
very
kubernetes
like
way.
A
So
traditionally,
with
with
monolithic
databases,
you
treat
them
like
monoliths
and
you
kind
of
tend
to
them
individually
like
they're
pets,
rather
than
like
they're
cattle,
and
so
the
tests
allows
you
to
not
have
to
have
those
databases
to
the
side
as
you
move
to
to
kubernetes.
So
you
can
bring
them
with
you
and
you
can
manage
your
MySQL
in
a
very
similar
way.
A
The
tests
it
project
origin
came
out
of
YouTube,
actually
and
being
part
of
Google,
also
where
kubernetes
came
from
that's
sort
of
where
some
of
that
synergy
comes
from,
but
one
of
the
goals
of
the
tests
was
to
allow
you
to
us
or
horizontally
scale,
MySQL,
so
that
you
can
achieve
massive
scalability
as
you
need
to
add
capacity
to
a
MySQL
service.
You
know
in
a
traditional
monolithic
way,
you
would
have
scaled
up
now.
You
can
scale
out
as
well
by
horizontally,
adding
nodes.
A
It
also
provides
a
level
of
orchestration
around
MySQL
so
that
you
can
have
high
availability.
So
it
uses
what
what
we
recommend.
Semi
synchronous
replication,
see
how,
if
you
as
you
make
a
modification
through
your
application,
make
sure
that
at
least
couple
of
the
nodes
have
copies
of
the
data
so
that
you
can
have
recovery.
A
And
finally,
the
last
point
is
that
it's
fairly
easy
to
migrate
into
the
tests
because
to
your
applications,
it
looks
like
it's
MySQL
and
it
is
based
on
MySQL
and
we'll
describe
that
in
just
a
couple
of
slides
forward
when
we
go
into
architecture
of
a
test
recently
graduated
as
a
CNC
F
project,
it's
the
8th
project
to
be
able
to
do
that.
So
that's
something
that
we're
very
excited
on
this
there's.
A
couple
of
maintain
is
in
the
room.
A
A
A
This
is
the
architecture
of
a
tests
and
I
think
this
slide
will
be
really
interesting
as
we
go
through
the
examples
today
on
the
on
the
very
far
side
you
can
see.
This
is
your
application,
and
your
application
could
be
anything
anything
that
has
a
MySQL,
connector
or
MySQL
driver
and
wants
to
be
able
to
speak
to
MySQL.
It
can
speak
to
the
tests
because
the
test
speaks
the
MySQL
protocol,
or
rather
the
component
of
the
tests
that
that
presents
this
MySQL
view
of
the
world
to
your
application
is
something
called
VT
gate.
A
So
VT
gate
is
a
stateless
proxy
that
can
do
routing
to
make
sure
that
you're
connected
to
the
correct
MySQL
server
to
be
able
to
retrieve
that
data
that
you
needed
to
acces
I
can
also
do
other
features
such
as,
if
it
had
to
access
more
than
one
MySQL
server,
underneath
it
can
scatter
gather
and
it
can
join
those
results
together.
So
many,
but
not
all
of
the
my
SQL
queries
that
you
could
execute
on
regular
MySQL,
you
can
execute
in
the
test
as
well,
so
things
like
joins
and
transactions
are
supported.
A
So
beneath
bt
gate
we
have
our
tablets.
Tablets
is
the
Vitesse
terminology
to
say
this
is
a
combination
of
a
V
T
tablet
process
and
a
MySQL
server.
The
test
actually
does
use
my
skill
onto
the
hood
to
be
able
to
store
data,
which
is
a
good
thing.
Mysql,
you
know
it's
it's
a
very
stable
product
with
a
lot
of
history
and
actually
its
performance
on
a
single
node
basis
is
very
good.
A
It's
just
that
it
doesn't
allow
you
to
scale
horizontally,
and
this
is
where
the
test
comes
in
and
helps
you
be
able
to
do
that.
So
you
can
have
many
tablet
servers
many
VT
Gate
servers
and
your
application
just
sees
a
single
view
of
the
world
where
it
sees
a
single
database
that
it's
used
to
connecting
to.
It
doesn't
have
to
know
this
detail
it's
sort
of
under
the
under
the
covers,
which
is
why
we
have
this
line
that
delineates
between
your
application
and
with
the
tests.
A
A
In
the
YouTube
case,
they
had
many
data
centers
around
the
world
and
they
used
the
tests
to
be
able
to
manage
this
at
a
simple
level.
I,
don't
want
you
to
think
of
the
test
as
as
VT
gate,
you
know,
routing
and
being
a
proxy
is
part
of
it,
but
the
test
is
really
an
orchestration
system.
It
helps
manage
these
MySQL
service
and,
as
I
said,
allows
you
to
not
treat
them
like
pets.
A
A
Yeah,
that's
a
great
question,
so
I'll
repeat
the
question:
do
we
have
any
plans
to
support
other
databases?
You
know
in
the
case
of
YouTube
they
use
MySQL.
So
that's
how
they
test
started,
but
the
test
is
fairly
agnostic.
So
we
do
have.
We
do
have
ideas
and
we
do
have
plans,
but
but
we're
not
currently
implementing
it.
It'll
it'll
come
in
the
future,
we'd
love
to
support
Postgres.
We
think
it's
a
great
idea.
A
Does
the
VT
tablet
make
sure
those
shards
are
in
sync,
with
the
master
I
will
throw
in
a
level
of
detail
that
I
didn't
show,
but
is
on
my
slide.
The
metadata
is
stored
in
something
we
call
a
topology
server
and
both
the
tablets
and
the
VT
gates
can
connect
to
the
the
topology
server,
which
is
your
choice
of
@cd,
which
is
the
most
common
these
days
or
zookeeper,
or
console
to
be
able
to
store
that
state
information.
It's
not
actually
connected
to
as
part
of
the
active
query
serving
path.
A
Does
VT
gate
ever
connect
directly
to
my
school
dunno?
It
accesses
via
the
tablet
and
actually
a
detail.
We've
not
shown
here
is
that
bt
gates,
VT
tablet
is
a
g
RPC
protocol
and
then
it
converts
it
back
to
MySQL
D.
Here
your
application.
You
know
in
most
common
usage,
these
eight
people
do
connect
by
the
MySQL
protocol,
but
there's
also
a
G
RPC
protocol
directly
to
VC
gate
as
well.
A
A
Don't
know
if
I
caught
your
question
but
I'll,
try
and
tell
me
if
I'm
wrong,
could
you
connect
directly
to
to
the
VT
tablet
or
connect
directly
to
the
VT
gate
with
G
RPC
yeah,
you
can
connect
directly
with
G
RPC
to
the
VT
gate.
That's
officially
supported
yeah.
That
was
the
original
way
that
will
actually
operate
inside
YouTube
and
there
are
drivers
for
Java,
PHP
and
Python
that
a
gr
pc-based
to
the
VT
gate.
A
A
Yeah,
so
how
would
this
interact
with
something
like
Google
Cloud
seek?
Oh,
that
is
a
great
question.
I
in
my
next
slide.
As
we
get
into
the
migration,
it
is
possible
to
be
able
to
do
a
Google,
Cloud,
sequel
migration
using
the
same
methodology
that
I'm
describing
today
the
VT
tablet
of
detail,
I'm
not
showing
that
I'll
show
in
that.
A
I
did
say
last
question:
okay,
one
more
all
right
yeah,
we
actually
released
a
benchmark
at
the
company
I
work
for
a
planet
scale
that
showed
that
if
you
put
the
tests
on
top
of
Aurora,
you
can
horizontally
scale
or,
and
that's
the
real
value
add
as
it
doesn't
horizontally
scale
itself,
and
we
had
just
massive
numbers:
I
think
16
million
TV
CC
Trent
transactions
per
second
per
minute
per
minute.
Thank
you.
As
a
result
of
that,
yeah
I
would
recommend
that
you
use
both
pots,
but
you
could
just
use
one
pot.
A
A
Monolith
is
a
bad
word,
ulcer,
okay,
from
a
MySQL
server
to
the
test,
and
we
want
to
make
sure
that
we
reduce
our
risk
as
much
as
possible
right.
We
want
to
make
sure
that
you
know
we
know
our
queries
are
compatible
with
the
tests.
A
latency
don't
go
up
and
we
want
to
be
able
to
use
a
really
responsible,
ops
process,
which
is
to
be
able
to
use
a
canary
to
be
able
to
get
there,
and
you
can
do
this
with
the
test.
A
You
can't
do
this
with
all
database
migrations,
it's
very
hard
to
be
able
to
do
that,
but
you
can't
do
this
with
the
test.
So
let's
talk
about
these
three
three
problems.
First,
how
we
test
our
queries
to
know
that
they're
compatible
with
the
tests
I
think
it's
obviously
best
to
start
this
with
dev
and
QA,
maybe
have
a
test
suite.
Maybe
you
can
validate
this,
but
I've
always
still
cautious
when
we
get
to
production
that
we
have
to
validate
this
in
production.
A
Sometimes
you
have
applications
where
users
can
click
on
buttons
in
a
certain
sequence
and
that's
not
covered
by
your
test
suite
or
that
generates
a
query.
That's
really
special
and
so
I
want
to
be
able
to
validate
that
in
production
and
I
want
to
know
that
it's
able
to
execute
in
the
test
as
it
did
in
MySQL.
A
There's
one
thing
to
note
about
the
the
architecture
that
we
have
for
the
test,
that
it's
got,
that
VT
gate
layer.
It
adds
an
extra
network,
hop
and
actually
frequently
you'll,
even
put
a
load
balance
in
front
of
those
VT
gates,
so
you
can
balance
between
them
and
that
would
be
two
extra
network
hops.
So
you
have
to
make
sure
that
this
doesn't
impact
our
application
in
a
negative
way
right.
A
So
an
easy
back
of
napkin
math
way
of
doing
this
is
project
about
a
milli
second
or
two
per
per
query
that
is
going
to
be
added
by
the
tests
and
so
your
application.
You
know
in
the
specific
case
that
it
uses
like
row
at
a
time:
processing
with
n
plus
one
I
think
it
could
see
that
it
gets
a
regression.
A
It's
not
that
easy
to
say
yes
or
no.
Some
applications
actually
see
an
improvement,
because
in
the
MySQL
case
they
were
just
so
overloaded
that
queries
were
taking
longer
when
they
moved
to
the
tests.
Vt
gate
has
some
other
nice
features
like
connection
pooling
that
could
actually
lead
to
an
improvement.
But
anyway,
if
you
want
to
be
responsible,
we
want
to
be
able
to
test
this
and
make
sure
that
this
doesn't
impact
our
app,
and
you
want
to
fix
these
in
your
app
anyway.
A
And
so
let's
talk
about
it.
Let's
talk
about
it
through
the
architecture,
so
I
revisit
this
slide.
For
a
second
and
say
this
is
a
fully
deployed
Vitesse,
where
we've
got
our
bt8
and
we've
got
our
tablets
that
are
managed
by
private
tests
and
then
off
to
the
side.
We've
got
our
topology
server
and
out
VT
CTL,
which
is
the
D,
which
is
the
management.
I
didn't
mention
that
this
is
our
canary
deployment
model.
A
A
This
works
great
and
it
works
great
because
VT
gate
itself
is
stateless.
It's
not
storing
any
of
the
data.
You
can
insert
an
update
and
select
data
from
each
one
of
these
access
paths
and
you'll
be
able
to
see
it
in
the
other
location.
This
is
the
first
step
and
this
this
confirms
for
us
in
you
know.
You
know
things
that
we
want
to
validate
that.
The
query
latency
is
going
to
be
okay
for
our
application
and
the
queries
that
we're
hoping
to
support
are
supported
as
well.
A
A
Okay,
so
this
is
where
we
want
to
get
to.
This
is
our
phase
two
of
our
migration,
where
we
want
to
say
that
there's
no
direct
access
to
my
school
everything
goes
through
a
VT
gate
and
the
VT
gate
is
the
router.
So
as
we
move
into
two
shotted
workloads
or
as
we
have
multiple
key
spaces
to
use
Vitesse
terminology,
we
have
to
make
sure
that
everyone
is
accessing
a
VT
gate
so
that
it
gets
routed.
A
A
But
we
could
either
start
creating
new
tables
in
that
location
or
we
could
copy
the
data
from
the
existing
MySQL
server
to
out
of
a
test
tablet,
and
this
is
really
cool,
because
this
it's
a
feature,
we
call
table
migration.
It
can
actually
happen
online,
so
it
starts
copying
from
the
existing
MySQL
server
without
blocking
reads
without
blocking
writes
it
just
happens
in
the
background
and
then
when
it's
finished
copying
you
know
it
could
be
a
several
gigabyte
table.
A
A
Let's
talk
a
little
bit
about
table
migration,
what
it
means
it's
also
currently
called
vertical
split
clone
inside
some
of
the
documentation
for
the
tests.
We
do
plan
to
entirely
rename
this
to
call
it
our
table
migration.
It
allows
you
to
move.
You
know,
as
I
said,
online
online
means
that
it
doesn't
block,
reads
or
a
dozen
book
writes
it's
just
going
to
block
for
a
couple
of
seconds
at
the
very
last
stage
when
you
want
to
do
that
final
transition,
it
does
have
some
requirements
to
be
able
to
do
this
on
the
MySQL
server.
A
Most
of
these
requirements
are
on
the
MySQL
side,
not
on
the
Vitesse
side.
These
are
not
the
defaults,
but
actually
in
the
case
of
Google
Cloud,
SQL
I
think
they
are
the
defaults,
so
it
should
be
able
to
do
for
that
question.
It
had
earlier
a
migration
from
cloud
SQL
to
Vitesse.
Using
this
methodology.
In
most
cases,
I
think
you'd
be
pretty
safe
to
change
these
defaults
on
a
MySQL
server.
A
A
Okay,
so,
let's,
let's
talk
about
completion
so
as
we
get
to
the
end
of
the
copy,
what
happens
now
we
have
everything
pointing
to
to
the
VT
gate,
which
means
that
it
will
effectively
route
to
know
that
it
should
go
to
the
to
the
existing
MySQL
server.
If
we
have
a
legacy
table
or
we
can
go
to
the
new
question.
A
B
A
Okay,
so
we
add
we
get
to
the
completion
and
then
we
end
up
I'm
just
going
to
sort
of
shuffle.
This
is
to
the
side,
so
you
can
kind
of
see
what
it
it
looks
like
closer
to
that
original
architecture,
diagram
that
we
showed
we
had
you
mysql
or
your
applications,
your
VT
gate,
and
then
your
tablets.
Now
we're
kind
of
moving
into
that
area,
where
we
have
VT
gate,
pointing
to
the
to
the
new
tablets
or
pointing
to
the
old
tablet
that
points
to
the
existing
MySQL
server.
A
So
we
end
up
in
this
position
where
we
can
have
just
the
old
tables
in
MySQL
and
any
new
table
if
we
want
to,
we
can
create
it
in
this,
this
new
location,
so
I'll
set
up.
That's
that's
to
run
some
details
to
be
able
to,
you
know,
make
it
referenceable
legacy.
Mysql
is
using
one
key
space
and
we
use
the
terminology.
That's
externally
managed
key
space
by
the
tests.
A
This
is
the
same
technology
that
allows
you
to
use
the
Aurora
case
where
it's
managed
by
Aurora,
but
we
put
the
test
on
the
top
of
it
to
horizontally
scale
it,
and
then
we
have
a
single
unsoldered
key
space.
That
is
our
new
infrastructure.
The
test
supports,
suppose
sharded
an
uncharted
key
spaces
just
to
throw
in
a
detail.
A
Now,
if
you
do
move
in
a
Charlotte,
that's
just
a
few
less
queries
that
is
supported
as
sometimes
it's
difficult
to
be
able
to
figure
out
where
to
access
the
data
from
so
for
the
simple
case
we
went
with
the
uncharted
key
space
and
then,
from
from
there
we're
able
to
sort
of
deploy
additional
key
spaces
additional
tables.
It's
recommended
to
keep
when
you
kind
of
shod
systems,
it's
recommended
to
keep
your
updates
to
be
within
an
individual
key
space
in
an
individual
shot,
rather
than
having
to
update
things
that
are
in
many
places.
A
A
A
C
A
So
that's
a
great
question,
so
does
your
application
have
three
typically
not
showing
my
my
diagram?
People
put
a
load
balancer
in
front
of
the
the
three
as
an
easy
way
to
do
it
yeah,
and
that
way
you
only
have
to
have
one
point
to
be
able
to
do
that:
yeah,
they
weren't
sort
of
reroute
traffic
to
the
other
nodes
as
they
get
busy
I
mean
they
just
speak
the
MySQL
protocol,
so
it
doesn't
support
that
native
natively.
A
B
So
the
question
is:
what
kind
of
support
do
we
have
for
change
data
capture?
That's
the
feature
that
I've
been
working
on
for
the
last
one
year.
It's
called
V
replication
and
it's
about
a
couple
of
weeks
away
from
completion.
It
helps
you.
It
does
a
whole
bunch
of
things,
including
what
we
call
materialization.
You
can
basically
point
at
a
table
and
say
I
want
to
materialize
this
table
using
the
Select
query,
and
then
it
will
just
materialize
the
entire
thing
into
another
key
space,
basically
project
it
anywhere.
B
C
A
B
A
B
One
very
specific
thing
that
we
have
about
Vitas
is:
there
is
no
requirement
about
that
is
there
is
there
is
no
restriction
about
which
sharding
key
you
use
for
your
table
versus
what
primary
key
the
table
has.
As
a
matter
of
fact,
most
workloads
are
work
better.
That
way,
because
enforcing
that
the
sharding
key
is
part
of
the
primary
key,
you
know
is
a
huge
restriction
in
terms
of
your
ability
to
group
related
data
that
has
to
be
together.
So
that's
I,
don't
know.
If
that's
that
was
your
underlying
question
in
that
case.
A
C
B
Yes,
is
it
theoretically
possible
to
do
that?
Yes,
but
if
you
ask
me
my
sequel,
replication
as
a
master
replicas
situation
is
pretty
good
and
pretty
efficient,
so
there's
no
need
to
get
in
between
there,
but
what
it
does
not
do
is
if
you
want
to
apply
transformations
and
if
you
actually
wanted
the
target
to
be
slightly
different,
like
for
example,
you
want
to
filter
only
some
rows
or
you
want
to
do
aggregation
on-the-fly.
So
those
kinds
of
things
my
sequel
replication
will
not
do.
In
that
case,
we
use
with
tests
for
it.
B
A
A
B
A
B
So,
and
also
in
the
long
run
as
in
that's
actually
our
focus
for
with
tests
until
we
reach
this
goal
is
to
make
it
fully
compatible.
With
my
sequel,
I
mean
our
lofty
goal
is
to
make
it
a
drop-in
replacement,
yeah,
so
queries
that
don't
work
today
will
most
likely
start
working
in
a
few
months.
From
now
and.
A
B
B
So
the
a
bunch
of
people
actually
came
with
the
impression
that
pagination
works
one
shot
at
a
time
right.
So
if
your
data
is
ordered
in
such
a
way
that
you
scan
within
a
shard
and
then
go
to
the
next
chart
and
then
go
to
the
next
chart,
so
that
is
actually
not
the
recommended
way,
because
organizing
your
data
is
a
guaranteed
way
to
end
up
with
hot
shots,
because
popular
items
tend
to
be
within
one.
So
the
way
that
we
recommend
you
do
it
with
Vitas
is
actually
spread.
B
It
out
say
my
pseudo
randomly
into
all
shards
and
then,
when
you
paginate,
what
we
test
will
do
is,
let's
say,
select
limit
hundred
right,
so
you
will
actually
fetch
100
from
each
shard
and
then
cut
off
as
needed.
So
it
feels
a
little
inefficient,
but
it
is
still
acceptable
because
the
work
is
distributed.
B
But
pagination
has
a
problem,
especially
if
you
use
a
limit
query
with
offset
and
count.
Is
it
degrades
over
time
like
if
your,
if
your
paginating
10,000
pages,
you
are
going
to
be
in
trouble
and
that's
a
my
sequel
limitation,
so
what
people
have
done
in
that
case
is
actually
the
token
idea
that
you
talked
about,
which
is
actually
remember
the
last
value
that
you
fetched
and
use
a
greater
than
that
value
and
just
use
a
limit
class.
So
that
is
what
people
have
used
so
far.
I.
A
A
Yeah,
so
if
you
are,
how
does
it
work
with
asset
transactions?
So
if
your
update
is
within
a
Schad,
then
it's
basically
just
routing
to
mysql
that
it
is
a
transaction.
If
it
is
a
cross
shot
boundaries,
then
you
can
enable
two-phase
commit
to
be
able
to
end
the
tests
will
kind
of
maintain
that
log
of
the
operations
to
be
able
to
survive
with
recovery.
We
typically
recommend
that
you
try
and
model
things
so
that
you're
in
the
first
camp,
not
the
second
campus.
It's
a
performance
degradation
to
do
that.
B
A
A
B
A
A
It's
so
how
do
we
do?
Backups
is
a
question
and
it's
very
similar
to
how
you
did
backups
with
regular
MySQL,
only
the
test
kind
of
manages
that
for
you
with
a
plug-in
system,
so
some
of
our
users
use
extra
backup
is
the
plug-in
and
some
of
them
use
the
shutdown
replicas
and
then
backup
as
a
method
and
in
most
work
fairly
well,
and
it's
it's
kind
of
like
on
a
on
a
per
tablet
basis
that
it
will
back
those
up
for
you,
I.
B
A
Believe
we
currently
do
know,
but
it's
it's
technically
possible
that
we
might
in
the
future,
just
just
from
a
I
done
this
far
too
much
is
from
a
philosophical
perspective.
When
I
was
operating
at
YouTube,
often
I'd
use
local
storage
and
I
just
had
many
replicas,
and
so
that
works
well.
When
you've
got
a
lot
of
read
traffic,
we
know
a
lot
of
users
do
use
the
the
cloud
provided:
network
storage,
but
I.
B
No
I
don't
think
so,
I
it
just
it
just
stores
it
to
whatever
there's
an
interface.
You
can
either
say
mounted
volume
or
s3
or
I.
Don't
know,
there's
one
more
one
other
adapter
that
we
have
for
it,
saves
it
there
and
then
it
knows
how
to
go
back
to
it.
It's
not
kubernetes,
dependent,
I
think
the
the
answer
why
it's
not
cooking
it
it's
dependent
is
because
there
are
people
who
run
with
us
outside
of
kubernetes,
so
the
Vita's
code
itself
has
nothing
that
that
depends
on
kubernetes
by
itself.
A
B
B
So,
as
you
do
that
scale
out,
your
traffic
is
growing,
so
you
deploy
more
VD
gates
and
then
afterwards,
what
you
will
do
is
you'll
start
recharging
your
data
and,
as
you
reach
hard,
you
will
scale
out
even
more
vita
gates,
and
then
you
start
adding
cells.
Essentially
like
youtube.
We
start.
We
had
one
database
that
we
call
the
main
DB
and
by
the
time
I
left.
We
had
tens
of
thousands
of
nodes,
so
it
was
all
slowly
grown
over
time,
adding
more
data,
centers,
more
VD
gates
and
yeah
I.