GitLab Create Code Review Group, 25 Feb 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Code review process problems surveys — Jan–Mar 2021

Description

Pedro Moreira da Silva, Staff Product Designer in the Create:Code Review area of GitLab shares the insights from the code review process problems surveys done in the beginning of 2021.

- Insights: https://dovetailapp.com/projects/4c107a06-a291-4dc0-9340-b232fb833788/insights/present
- Internal survey (January 27 – February 25, 2021): https://gitlab.com/gitlab-org/ux-research/-/issues/1258
- External survey (March 16 – 30, 2021): https://gitlab.com/gitlab-org/ux-research/-/issues/1346

A

Hi, my name is peeler de silva and I'm the staff product designer working in the create code review area of gitlab, and today I'm going to walk you through the insights of one of the recent research that we did on the code review process problems and how they affect the efficiency of primarily engineers that use gitlab for code review.

A

So let's take a look, so this uh research is a result of two surveys that ran in the beginning of 2021, one internal and another external survey, so that we could compare the usage and the problems of code review with our internal audience of gitlab team members.

A

But we also wanted to hear from other users and customers with the external survey, and the focus was to understand how engineers that use gitlab prioritize problems that affect their efficiency when they're doing code review, with the aim of helping define our roadmap in this group that we call the code review group of git lab.

A

So we had one research question and two hypothesis.

A

So the research question was what is the most frustrating part of the code review process for engineers using gitlab, and that was without a doubt, dealing with large, mrs, both internally and externally, and it was no surprise because this reinforces the feedback that we continuously receive in issues in social media from customer conversations.

A

But we have to be aware that this dealing with large, mrs, is a problem that is quite broad. It has many different sub problems inside of it. However, in the open text responses, it was clear that when users were and the survey respondents were saying that dealing with larger mars was the number one problem for them, that the most visible and impactful issue was the pure product performance so response times.

A

The first hypothesis that we got had was there isn't a significant difference in how internal and external users prioritize code review problems or how they define large merge requests, and there were only a few differences and we think those do have to do with the different work practices of um internal or internal teams and the external users.

A

One of them was related to the problem of tracking. My progress when looking at a merge request, it had some difference, as you can see here internally it ranked in the 11th position, but externally it was ranked in the fourth position, so it was much higher in the prioritization for external users, which is interesting and something that we can look at separately in. Another research reviewing commit messages which is often asked as one of very top feature requests that we have ranked last internally and externally. It didn't have as much difference as we would expect.

A

We thought it would rank higher in people's uh needs, but um yeah this one ranking last internally. It's just, as I said, has to do with the work practices. We usually don't review the commit messages and the gitlab project internally. So it's not something that our engineers are concerned about in terms of the size of large merge requests. How do they define? Learn, merge, requests? The participants mainly indicated two aspects, so the number of change lines and the number of files.

A

However, the specific size was not clear. There was no consensus about. Where is the threshold for a large, mr, and this is in line with previous research that we did last year?

A

Our final hypothesis was that external users struggle with reviewer assignments more than internal users, given gitlab's internal use of the danger bot for review recommendations. So we have this bot this automated process that recommends reviewers for us internally when we're contributing to the gitlab project in the gitlab company and external users.

A

Don't have access to that feature because it's something that we created internally, and so we thought that external users would rank that problem higher, that they need have a desperate need to find more appropriate reviewers for merge requests, but, interestingly, finding appropriate reviews for merge requests ranked higher internally than externally, so 9th place internally and 14th the last position externally. So it was the least important problem that people had have externally. So that was very interesting um and we're not sure exactly why this is.

A

But one of our assumptions is that this difference could be because of the uh that the average external respondent was an engineer in a team of up to 10 people, whereas gitlab engineers often work across teams, and we have close to 300 engineers at gitlab at the moment, so the team size could have had an effect here now.

A

Let's talk about the efficiency problems um in a very clear way, so the respondents had 14 different problems that they were asked to rank in order of severity from the most to the least impactful on their own team's efficiency. So the top three problems from mostly severe were again dealing with larger mars.

A

The first position second position, knowing what happened since I last visited mr and then in third position, understanding the broader impact of changes which, in a way, can be related to this first problem of dealing with a large merge request and at the bottom, the least, important problems that affect their efficiency is communicating or understanding the intention of comments, reviewing commit messages and finding appropriate reviewers for emerge requests, as we saw before now. What about a large merge request?

A

As I said before, the number of changed lines or files is the main characteristic when defining a large merge request. But the specific size is not clear. We actually received praise for recently shipped features and those were praised in the open text responses and were really happy about this. So multi-line comments, marking a file is viewed. Suggestions and reviewers were some of the features that were highlighted and we're very happy about that.

A

And finally, what what is our recommendation after uh analyzing all of this and looking at these insights?

A

So our number one recommendation is doubling down on improvements to product performance, because it was the frequently mentioned challenge uh in the open text responses and it maps to what we just saw as a number one problem dealing with large merge requests product performance.

A

Again, it includes things like response times: loading changes, loading comments, navigating the user interface and fortunately, the code review group as part of an okr for our q2, our second quarter of the year, we're prioritizing and exploring those improvements uh so that we can reduce the number of complaints and make sure that the credit review experience is as effortless and frictionless as it can be, and- and that's that, thank you so much for your time and if you have any questions or comments, please check the links at the in the video description for more information.

A