GitLab Geo Group, 12 Nov 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: GitLab 13.7 Kickoff - Enablement:Geo

Description

Geo's PM walks through the team's upcoming 13.7 release plans. For more information, check out Geo's 13.7 planning issue: https://gitlab.com/gitlab-org/geo-team/discussions/-/issues/4983

A

Hello, everyone all geo users around the globe. This is fabian simmer the product manager for gitlab geo, as you can see in our new geoswag first, I would like to thank nick for covering, while I was out on parental leave. So thank you so much and also thanks to syndring for designing these wonderful logos.

A

So this is the kickoff video for 13.7 gitlab's. Next release scheduled to arrive shortly before christmas on december 22nd, our present to the community and our users.

A

So let's see what geo has in store for this release all right before I go into the details, just to take a step back, gitlab geo is gitlab's solution for distributed teams and as part of a disaster recovery solution. So there are really two use cases, folks that have several offices several gitlab installations and they would like to bring their data closer to a remote office.

A

That's far away from their main gitlab installation, that's the geo-replication angle, and then there is also the the option to have a geosecondary that replicates all of the data from agio primary and you can fail over to this secondary in case of a disaster.

A

So these are the two main use cases and I'm going to explain to you what we are going to work on in 13.7 and how these things relate to our overarching goals for geo.

A

So the first thing I want to talk about is that we're working on improving our usage ping data, so we really try to understand how our product is used. Specifically, we are currently interested in understanding the usage of secondaries, and this is really to help. You have a better experience with our product and so in 13.6, we've started working on understanding how users interact with a secondary node, how they log in and then in 13.7 we're going to instrument this and provide summary data, anonymous summary data that actually allows us to understand usage patterns.

A

We are also interested in tracking some other operations, for example, how many git clone operations are there that helps us understand usage overall, it can make us help us, be more data driven make better decisions so that in the end we build a better product.

A

So that's number one number two is to do with the geo self-service framework and a first implementation for package file, verification and there to provide a little bit of context. You may have heard me speak about the self service framework before, but what this is really about is that geo is a very fast moving product.

A

We add new features every month we add new things that are valuable to our customers to you, and so in some instances these features would really benefit from being replicated to a geosecondary either because of disaster recovery use cases or because you want that data closer to your users and the over arching goal for the self-service framework is to make it a lot easier for our developers to ship geo support for their features. So when something is released, it supports geo.

A

That's the goal that we're working towards, but there are also some improvements that we can make by standardizing how geo actually works under the hood and one of the feature requests that has been around for a while and that we're trying to work towards here is to actually be able to verify the replication of files blobs so rather than git traffic, where we support verification already verify that the data that is transferred replicated to a secondary is also correct.

A

It's not corrupted either in flight or at rest, and so first step towards that here is to implement verification for package files. This is ongoing. It's a complex effort, so you can see. There are many issues that have to do with this and many that we already closed so we're working on this in 13.6, but this will carry over into 13.7.

A

It's the first implementation of it, but having it in the standardized way will allow it to ship is really fast to other data types as well as, for example, external merge, request, divs or terraform state files or any other file types that we will support.

A

Secondly, we are working on alpha support for petroni. This is a really interesting endeavor. So petroni is a high availability template for postgres the database that gitlab uses under the hood and so for customers that run scaled environments of gitlab. There may be business requirements for availability so rather than having a single database node, they may have several in case. One of the database node fails and petroni allows you to manage this.

A

We have experimental support for petroni already in omnibus the package that we ship, but so far we also only have experimental support for geo, so we're working here on improving our support for petronio on a secondary.

A

So you can imagine a primary, let's say, has three database nodes, and this will be exactly mirrored on the secondary with also three nodes on the secondary as a standby cluster, and this is something where there are a few things that we still need to document and test, for example, in 13.6 and 13.7, we're going to finish documenting the failover process to that standby cluster and we're also going to document how to upgrade a single node from a non-patrony instance to a patrony instance. So this is really something that we are we're pushing forward.

A

It's quite exciting because it is highly relevant to some of our customers and overall should allow us to get closer to a situation where a secondary has the same infrastructure deployment pretty much compared to the primary. So when you fail over, there are not many differences.

A

The next thing I would like to talk about is maintenance mode and so maintenance mode or a read-only mode, which is, I think, the better way to think about. It will allow you to put your primary into a state where users can still read from it. They can go to the web interface, look at things they can clone, but they are no longer allowed to make changes and for jio. The main use case here is that, in case of a planned failover to a secondary, we constantly replicate data in the background in their cues.

A

But when you do a planned failover, you are trying to avoid any kind of data loss from your primary. So before you want to fail over, you need to disable any new operations on the on the primary itself.

A

Currently you can do this by, for example, taking out gitlab out of the load balancer or blocking access out of the application, but this is not very user-friendly and it would be a lot nicer if folks were still able to actually look, at least at the instance for a few minutes before you do the failover operations.

A

So this is what we're working on in 13.7. The testing will really kick in, so we're working on the implementation. Many of these things are going to hopefully finish in the next few weeks, but then we want to test it in various environments- scaled environments, small environments, to really ensure that, once this ships, the quality assurance, is very high.

A

So that's maintenance mode, quite excited for that one, and then I think something that I'm very, very proud of and really happy to see is that the geo administrator interface, so what cis admins actually see when they're trying to understand the state of geo on our home page, it really could benefit from some improvements and we've shipped a lot of like user experience, improvements and ui improvements in the past and we've worked towards a redesign of the geo node page.

A

This is the new design. Here, we've spoken to quite a few sysadmins. We've asked what the pain points are. We've then gone back to the drawing board. Came with a design, spoke to the systems administrators again and asked them. Is this something that you actually like? Does it solve your problems and we got good feedback so we're now at a point where the design is done and we will work towards implementation of this. So this is something we're thinking about in 13.7.

A

It may not finish, but I wanted to point it out, and some of the really interesting benefits for users are essentially more details for specific replication situations in the secondary, but also a lot more compact views for folks that have more than one secondary.

A

So, for example, here with three secondary sites, you can essentially compress them and it's no longer so overwhelming, so we're really trying to provide high level summary information to quickly understand are things okay and when they're not okay, be able to zoom back into the details. Say: okay, you know something is wrong here. What do I need to do and quite excited for for that change? I'm looking forward to the release when this ships.

A

And lastly, I just want to talk very briefly about a stretch goal, so we have many things that we're doing in parallel already, but the team is very productive, so we're always trying to be ambitious. So one of the things that we may be able to do is actually move one of the let's say old data types. The self-service framework that I spoke about earlier, we'll likely do lfs files we'll see how it goes.

A

It depends a little bit on the bandwidth, but if that happens, that may be quite interesting because once lfs files are in the self-service framework and we've done verification for files, lfs files may support that immediately. I think that's really one of the things that people are looking forward to so those those are the main things a lot going on in geo.

A

Some of the bits that xinjing the product design- and I are going to also investigate further- is we're going to do some work, moving geo into a jobs to be done framework to make it more user focused to really understand specific areas where we can increase the usability, and I'm also going to work with the quality team to improve some of our quality landscape.

A

We have some amazing efforts on automation that allow us to spin up geo deployments complex, invoking problems very quickly, so that will come in handy so yeah it's going to be an exciting release. I hope you look forward to 13.7 and have a wonderful rest of your month. Thank you for listening, bye.