GitLab Geo Group, 18 Jan 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: GitLab 13.9 Kickoff - Enablement:Geo

Description

Geo's PM walks through the team's upcoming 13.9 release plans. For more information, check out Geo's 13.9 planning issue: https://gitlab.com/gitlab-org/geo-team/discussions/-/issues/4994

A

Hello, everyone, my name is fabian zimmer and I am the product manager for github geo and today I'm going to walk you through our work for gitlab 13.9, the upcoming gitlab release. Let me quickly share my screen.

A

Okay, so before I go into the details for 13.9 planning, I wanted to give you a high level overview of what we are up to in general.

A

So, as I've said before gitlab, this git of geo is essentially responsible for two different use: cases geo-replication so making gitlab usable and fast for distributed teams and disaster recovery, so essentially creating a complete secondary site that follows the primary site very closely that you can fail over to in case of a disaster or as part of your business continuity planning and so for the first quarter of the new financial year.

A

We are going to work on improving the maturity of the disaster recovery category to complete we've finished the implementation work for viable there's, some validation for usability scorecards still worked being worked on, but from an engineering side. We are now firmly in the work for complete maturity, and so everything that I actually talk about specifically for 13.9 is part of this epic here and you can see, there's a bunch of sub epics for specific features and functionalities.

A

There's a roadmap for 13.9 and beyond. We use a more continuous process, even though we release every month so take this with a grain of salt. It's a forward-looking um sort of out outlook for the next few months, but this is what we what we are doing at the moment, and so please keep that in mind, but also all of this is publicly available. Go check out. This epic um you'll see a lot of the work that I'm going to discuss here.

A

Okay for 13.9, I've talked about this for 13.8, so this is an ongoing focus. Essentially, we are working on adding verification capabilities to our self-service framework, both for file verification and for git data, and so we're focusing on file verification first for package files.

A

One of our team members has been working on this for a little while and we're making some good progress. So some of these items are going to roll over into 13.9, but we hope that we can ship package file verification as the first implementation for file verification in 13.9.

A

Secondly, we're going to continue working on geopatroni support. We shipped alpha support in 13.8, so we're now moving to beta support um alpha means is not ready for production. Yet there's some instability in beta. We are going to address some of these issues and stabilizing the overall offering to remind you. What is this about?

A

Many of our customers would like to be able to mirror the overall architecture of the primary site to the secondary side, so when they fail over, they don't have to make any modifications to the architecture on the secondary, and the reason for this is that you know when you actually fail over in form of like in case of disaster or as a planned failover.

A

You want to be able to do this quickly with minimal downtime and interruption to your users, and so one thing that we can do right now is provide a highly available postgres cluster on the secondary side. So a postgres database cluster with more than one node and um if one node would topple over for whatever reason the other nodes could fully fill in.

A

So petroni is a technology originally developed by salando that we also use a production for gitlab.com that is now being shipped as the high availability solution for postgres, aha in gitlab, and we are working on enabling this on a secondary site. So this would be a standby cluster, so something that could be used and transformed from a standby cluster into a live cluster upon failover.

A

And so we have quite a few items to work through here to enable to move forward in our maturity with petroni, for example, documenting how you can upgrade from a single node to a full petroni cluster, how we can automatically recover from a leader change. So there's a lot of technical work that needs to happen, but we are making good progress and I'll keep you updated.

A

Then um maintenance mode I've talked about this before you can see, there's a whole number of issues that we intend to deliver in 13.9, where we try to iterate it very quickly, and these are small issues that came out of testing, so the functionality of a read-only maintenance mode is essentially implemented.

A

This is valuable because it allows systems administrators to completely sync up a secondary and a primary site, so users are still able to log into a primary site.

A

You know they can browse the web interface, they can perform, read operations like git pools or clone operations or via the api, but no changes to the database or other data allowed, and that allows the secondary to really replicate everything.

A

So when you do failover, there is no replication lag, no data loss at all and, as I said, we are we're deep down in the testing of this.

A

um We have a testing on our staging environment for gitlab.com, scheduled for the 25th of january and, as you can see, many of these items are going to roll over into 13.9, but they're being worked on is many small fixes, in collaboration with our quality department, to ensure first nvc here works as intended for everybody who wants to use it, and so we hope to ship the first iteration of maintenance mode in 13.9.

A

Next lfs files, so I've talked about the self-service framework before it's now easier than ever before, for teams feature teams within gitlab or the wider gitlab community to add geode support to their features, but we also still have some features that we implemented in the past that we would like to move over to the framework.

A

So this may have some. uh It has actually several benefits for lfs files.

A

There may be some performance benefits, because the self-service framework has some interesting opportunities there, but in the long run it actually means that, for example, verification that we're implementing for the self-service framework right now will then out of the box also work for lfs files, so we're moving it over and this is taking a little bit longer because we have to migrate the existing feature as well. But it's well on the way and again fingers crossed we're intending to ship that in 13.9 we've also have a new design for the geo-node page.

A

We've done some initial work in 13.8 and we continue to iterate in this we're a little bit resource constrained at the moment and many things going on. um So I'm we may be able to do some work here, but I'll keep you posted.

A

Finally, um we've done a little bit of research on how to actually enable usage ping on geosecondary, specifically for git operations.

A

So we we can already measure operations via the web interface, but we can via git, and arguably many developers rely on geosecondaries for git operations, and so we would really like to measure that, and in order to do so, we need to enable usage ping on secondaries and then hook it up to our monitoring stack, namely prometheus. So that's also, hopefully going to happen in 13.9.

A

That's it lots of interesting stuff happening. As I said, this is just a snapshot of ongoing work in this area. We try to iterate as quickly as we can, but this is a this. Is my forecast for 13.9 so stay tuned thanks for listening and talk to you next time.