GitLab UX Team, 9 Aug 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: MR design reviews internal survey Aug 2021 - Insights summary

Description

To better understand the current state of merge request (MR) design reviews at GitLab, particularly their pain points, we ran an internal survey between August 9–13, 2021, aimed at Product Designers.

Pedro Moreira da Silva, Staff Product Designer at GitLab, shares the main insights and recommendations that came out of this internal research.

- Research issue: https://gitlab.com/gitlab-org/gitlab-design/-/issues/1547
- FY22-KR issue: https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/12127

A

Hi, my name is pierre meredith silva and I'm a staff product designer working at kit lab and recently we did an internal survey to understand how designers were reviewing merge requests and what were the main pain points in that experience. So let me quickly walk you through what we found out and what the recommendations are as a result of this. So this research is tied to one of our key results for q3 to improve the quality of design, merge, request, reviews and we ran this survey.

A

In the second week of august, we had 22 participants, so that gives us roughly 95 percent of confidence on the responses with a 10 confidence interval, and here are the main insights that came out of this.

A

So the first one top pain points that designers highlighted is git, lab development kits or gdk, and git pod issues, test environment, setup, vague incomplete descriptions and merge requests that miss requirements, and these were very clearly highlighted, not only in the um in the responses when we asked people to rank them uh all of these problems, but also in the open text responses.

A

The setup issue was clearly emphasized, but it was also interesting that other than these top issues and top pain points. There were also the awareness and knowledge issue and the planning context, switching and workload issue, and so, as a result of this, and I invite you to look at the insights yourself, but as a result of this, we're recommending that we improve discoverability and guidance for troubleshooting with the review tools, especially when you have problems with your local gdk and git bot.

A

How do you find help, uh and how do you troubleshoot things, um also some guidance on how to quickly set up and populate a test environment, because that's taking a long time and designers are um that is consuming a lot of designers time? um Also, the fact that sometimes merge request descriptions are not completely uh filled out by the merge request. Authors so we'll have to find out how we can encourage mr authors to do that.

A

More often and better and finally related to the workload is, how do we add guidance on how to balance merge, request reviews with the other responsibilities that designers have the other second uh insight that we had was related to the frequency, and we found out that, although this is all self-reported that designers estimate of how many merge request reviews they do per milestone is substantially different and there's a lot of variation.

A

So, as you can see here, there's not a consensus on how many merchandise quest reviews are done per designer they're all evenly spread out, and you can see a large accumulation here between 0 and 9, merge, request, reviews per milestone, and this, I think, suggests a discrepancy of how stage groups operate, which could be valid or not, and also potential differences in the throughput of those groups or the designers. Involvement in word request reviews.

A

So this was interesting and, and it surprised us- and I think we should look more closely to understand if there's something reasonable here in the substantial variation or if it's acceptable and valid.

A

The third insight has to do with the requirement to review merge requests. How do designers decide if a merge request is worth reviewing from a design perspective or not, or do they review all of them? So there's a tendency to only review, merge requests with user interface changes and even so designers have different criteria on how they decide that requirements.

A

So, as you can see here- and this was intentionally an open text response so that we wouldn't bias the respondents that most of them clearly said. Yes, we need to review, merge, requests that have ui changes, but then others also talked about. Oh, we need to also to review merge requests that have backend changes, because there's always something useful that designers can add to the conversation and some constructive feedback. There are very interesting comments here that people left in the survey.

A

I invite you to take a look at them if you're interested as a result of this, we feel the need to clarify this requirement, which kind of changes should designers review or are required to review and in what capacity, and so that there's a clear guideline and consistency for everyone in terms of confidence, it was very good to see that most designers feel fairly or very confident about reviewing merge requests, and this was, I think, sets perhaps a baseline- that we can then retest and see what designers feel after we make these improvements to understand if they feel more or less confident after them.

A

Basis of review so which tools supports, do and materials do designers rely on when they're reviewing merch requests, and we were surprised to see that there's a big tendency to rely on screenshots videos more than we would expect. Git pod is almost as popular as the local gdk and review apps. um Perhaps, unsurprisingly, are largely unused as we can see in the inside, so our recommendations is to add guidance on how to choose between all these different supports.

A

So when should you choose a local gdk over gitbot or review apps, or for screenshots and videos, or even as one of the respondents suggested syncing with the author, when the merge request is quite complex or the setup process is quite time consuming, um we also suggest promoting git pod and review apps benefits and improving the getting started documentation. How can designers use them, as some of them had questions about their?

A

How should I access them? Where do I click? Which passwords do I use to login into the instance and so on um and and again many? We expect, or we suspect that many designers are still very much used to the local gdk, because that's what they've been using for perhaps years and haven't yet picked up on geekpod or review apps, which can be much faster and cleaner and reliable than your local gdk.

A

Even so, there are two main cons to using git body review apps. One of them has to do with the ability to save the environment setup so, for example, data the license or feature flags that you have enabled when you're reviewing a merge request. um It would be good for us to look into those things and see if there's any possibility for us to um to salvage that and to improve that experience so that designers can use git, pod and review apps more and specifically about review apps.

A

There's there are concerns about reliability and uh we will have to look into that reliability today, if it's better than what it was in the past and if it's already an appropriate and production ready tool to review, merge requests, uh at least in the gitlab project, because responses show that designers are very much used to review apps in the context of documentation, changes, so user documentation or handbook changes or even in the pajamas website.

A

The other thing that we looked into was the aspects of review. What the designers look at when they're reviewing, and mostly that comes down to visual design, content and flow, not surprisingly, and there's a clear lack of acceptance criteria. That was something that was mentioned a lot also accessibility. Should we consider it or not um dark mode cross, browser compatibility responsiveness?

A

Should it look on, should it work on mobile or not so there are many questions around the acceptance criteria, although the top three that everyone agrees on has to do with visual design, content and flow. So, as a result of this, we are recommending that we add clear acceptance, criteria to aid designers with reviews so that they have almost like a checklist that is easy to reference and go through, as you review a merge request that ensures that designers are covering all aspects that are needed in a consistent way across stage groups.

A

And, lastly, we have review familiar unfamiliar, mrs and scope changes within mrs and for for both of these we, the recommendation is quite similar. So let's look into what this means.

A

So how often do merge requests reviews for unfamiliar issues are requested of designers and sometimes designers experience this, which is, I think, fair, but a good portion of designers experiences reviewing unfamiliar mrs half or most of the time- and this was quite surprising um and although the count itself is not a lot, we have six respondents that said about half the time or most the time.

A

I think it's worth looking into those cases to understand if there's something going on there and how we can mitigate that, because it again adds to context switching efforts and to unplanned work and finally related again to review affiliate. Mrs scope changes within merge requests so when something is added removed or changed from the scope, when development is happening inside of a merge request or as a result of a merge request, review and sometimes designers experience this, which is fair, but then we also have a good portion of designers around 37 or 36 percent.

A

We're experiencing this about half the time right and I I these cases could be valid or not, um but anyway we should look into those to make sure that we're not creating a scope creep inside of merge requests and that velocity is not impacted by this, and this is it. This is a quick review of the insights of this survey to improve the quality of design, merge, request, reviews at gitlab. Thank you so much for your time and let me know what you think in the comments see ya.