GitLab Geo Group, 9 Oct 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Geo HA upgrade from 12.1.9 to 12.2.8 going wrong

Description

In this video we're attempting to perform an upgrade from GitLab 12.1.9 to 12.2.8 on a Geo HA setup. We're evaluating the new documentation, which is not merged yet.

During the upgrade, Toon decided to be smart and use his shell-fu to be quick, and you won't believe what happened next.

Issue: https://gitlab.com/gitlab-org/gitlab/issues/32435
MR under review: https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/3577

A

Okay for a little bit context, so for those who don't know, you want to improve documentation on geo upgrades, especially for a change installation and we're going to do the demo to upgrade from twelve point. One point: nine to twelve point: two point: eight, let me share my screen and there we get can get started.

A

So I stole the setup from Formica guest, so I've made a terminal window for each each machine, so on the left side, all the primary machines on the right side, all the secondary machines are check their status and they're all healthy. Now, they're both running the same exact version. The only warning we have is that we are not using HTTPS, it's quite hard to do that with a load, balancer I think, but it should not block us on anyway.

A

Okay, let me check the the list between beverage. Okay, let's check hit the record button and now we can go and can get going with the documentation, so I guess for so here's the documentation. It's the last commit from the merge request. We were working on not sure if it was updated very recently, but I just opened it a few minutes ago. So it should be fine.

A

So we multi note h8 deployment with Geo I did not prepare the ferry. Well, sir, want to be surprised by it and not be biased and in advance. So then I'm sure the documentation is what I are inspecting it. It would be.

A

Updating the Geo primary multi not deployed on all nodes, create an empty file to skip all to reconfigure. Okay, let's do that.

A

Let's meet on the primary note, all other nodes.

A

Okay, this is computing.

A

Make sense so we want to skip other migrate on all machines, except for they point out, I will be using app or one a deployment note show. Let me set this.

A

A

Okay, so that's what we did and we have to reconfigure them. I.

A

Think most ashamed already had this because it was in the installation, instructions for h8 setups, but.

B

Yeah follow the instructions on the EMR that Fabian is working instead of the readme, but this is from the merge request. I did find America's. It should be.

A

So this is the merge request. Okay,.

C

A

Is this should be the same.

C

D

C

So same commit okay, great.

A

C

A

Thanks checking, so we did reconfigure.

A

Apparently it's taken quite some time. A few machines I.

A

Don't know why it's happy, because I already reconfigured right before doing this starting to demo so I didn't expect anything like huge to happen, but well, let's just wait a bit until it completes okay, those are regular machine, so we can move on to the deploying nodes already. So all the flowing out, we can update the kidnap package.

E

A

Dispersion, so here right, it's like the twelve point, two point: eight and it's from the command.

A

This better and in turn also the other nodes have completed their.

A

A

Shall await further.

A

Shouldn't should continue to complain.

A

Okay, this is interesting, I didn't do anything yet, and we already have a server error.

A

So is it on the app one or go to our boat.

A

Whoa create a password for your new account.

F

There's no root user, whatever it's.

B

Not headed back into the primary mode, the user to signing.

G

A

Yeah, it's weird to be accessing the second, the second application, no directly, but still it should be aware. That's, but you should be shoot using the same database, but reconfigure is cool Slate's.

A

This is odd, I.

A

Was checking out the load balancers there's a little bit of monitoring there, although I don't fully understand how it works or what it shows hasn't been done at all that our PS stands for what this number your ease.

A

Yeah, this was not a lot about.

F

Requests per second.

A

A

Okay, what can we do about this.

B

Can I check the start, something.

F

Right arrow right, I.

A

A

Gonna copy paste is.

F

Thank you for sufficiently scaring me into never attempting to echo something into a file.

A

And I I told myself if I do this through this ride and I was.

A

There something specific for this machine I, don't things all right holes the same for finally and secondary.

A

So what should you have here.

A

How does this look don't wanna pee? This will take a while. I have to prepare up like this against Roberta.

F

I think we might want to reconvene later man, that's terrible. Oh I was stupid of me.

F

We had three more people watching.