GitLab Sharding Group, 5 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Rails 6.1 horizontal sharding - findings

Description

1st part of video

https://gitlab.com/gitlab-org/gitlab/-/issues/328516

A

Hi, my name is tony and I will be giving a demo on rails.

B

6.1, horizontal shouting.

A

So the first thing that we need to do to.

B

Have multiple databases is to modify the config database here.

A

And it changes from the normal to it here into three tiers.

B

Where you basically have a section for each database, so the primary is the default database that voice expects and then you can have multiple additional data databases. So here we have additional one called shaft one and if you do.

A

B

We will have some additional database.

B

Reg tasks for each database, so here we have migrations for each chart. Migration status for.

B

So and if you run rick davey, my good deadlines it for all databases.

A

So it wasn't unfortunate, it wasn't possible to create dynamic uh databases infinitely, and I think that's a good thing anyway, because you don't want to suddenly.

B

Lose your database connection because some dynamic thing has changed.

B

What I managed to do was to dynamically, create new model connections, model database connections using by reading from this configuration, so I've created a new abstract class where we set up the connects to shards, which is the way to achieve horizontal sharding in rails.

B

So basically, what we do is that we connect the shards and then we have this thing here that reads: it constructs all the different uh shots for each darkness.

B

Let's see what other things.

A

B

Each of the different databases.

A

A

B

A different database other than the default database. uh You need to use the namespace.connector and put put recorder inside the block.

A

So if you do that, then so, for example,.

B

We want to call you the sharp one database for the space shot. So if you.

A

Do that, then it will come up with this namespace count. Here is the.

B

Number of namespaces for the sharp one notice.

A

B

User is not shouted, so it will always return the same. So if I go user account.

B

For comparison, if we do line space software, which will default to the primary, if it's different, it's much larger.

A

So the horizontal sharding support is still pretty basic so far, and some of it is down to the fact that if you don't supply equal shot, it will default to primary. So, where places where it's not implemented, then you run into issues so for.

B

Example, if we try to get the main space, the first lady space out, which was.

A

Sorry and assign that to that um you can see that it's actually got the wrong.

B

Path, unfortunately, which is not right.

A

Because if you, you know, if you cannot compare the actual path.

B

uh It's actually lost and found so.

A

So that's one of the traps so far and the way.

B

A

Is to load everything that you need within the block so that so that and as you can see, it kind of gets very tedious and it's quite a trap. If you, if you don't preload it and you try and get the association in the main block, then it's going to run quickly against the default.