Home
Contribute Contact Us Browse all meetings
Home Contribute Contact Us Browse all meetings
GitLab / System Usability Scale - Key Themes / 8 Jul 2020
GitLab / System Usability Scale - Key Themes / 8 Jul 2020
Previous Meeting
Next Meeting
⏯
Add meeting Rate page Subscribe
youtube image
►
From YouTube: Measuring usability with the System Usability Scale

Description

The System Usability Scale is a quarterly initiative that helps us understand the needs, challenges, and pain points of GitLab users.

In this series, Katherine Okpara (UX Researcher, Create) will discuss the top themes we’ve gathered from the SUS surveys conducted thus far and highlight opportunities for improvement.

Learn more about the SUS here: https://gitlab.com/groups/gitlab-org/-/epics/3831

A

Hi everyone, my name, is catherine OCH, para and I am a UX researcher for the create stage of get lab, and today, I'd like to talk to you about the UX Department's ongoing work to improve the usability of --get lab the system. Usability scale is a quarterly initiative that helps us understand the needs, challenges and pain points of gitlab users.

A

So what is the system usability scale? So this survey involves 10 Likert scale, statements that are designed to answer the following questions about a system. So first can users successfully achieve their objectives. Second, how much effort is expended in achieving those objectives? And, third, how satisfactory was the experience? Every quarter? We send the survey to members of the wider gitlab community. So far we have conducted this survey every quarter from q1 F, Y, 22, Q and FY 21, which is five quarters in total.

A

So, let's take a look at our scores over time, so on the screen here we have all the scores for the past five surveys that we've conducted and you can see the breakdown of the number of responses we've received and the grade, and so just to give you some context. The industry average score for the sus is 68 across various companies and scores above 68 are considered good, but in need of improvement. So the differences between gitlab scores.

A

It's not considered significant as they fall within similar grade and percentile ranges, but we're able to see that there are areas of the product that likely needs to be improved in order to increase the score over time and a quick note about the margin of error down here.

A

So the margin of error highlights the challenges we face in recruiting enough participants for the survey, so the larger the sample size, the smaller the margin of error, will be and the smaller the margin of error, the more confidence we can have that our results reflect the experience of get Webb's entire use, basse the way our working to iterate on the way we run the survey in order to reduce the margin of error over time.

A

So.

A

Here we have the charts that show the breakdown of responses to the Likert statements, so I just wanted to give you an overview of kind of how the data is distributed.

A

So the scale goes from strongly agree to strongly disagree and, as you can see where we have a pretty good trend where, for the questions that kind of rate gitlab more positively such as I would like to use the it lab frequently we we Trent, we tend to have more users agreeing and for questions that have get elaborated a bit more negatively, such as finding gitlab unnecessarily complex.

A

We tend to have more users disagreeing, so this is pointing to us that, even though you know we like to get a higher score, at least over the, at least over the quarters, we're seeing some consistency in how users are proceeding the product. So you can take a look at these examples, and this also continues for the rest of the questions in the scale.

A

So how can we improve usability over time when we first started running the survey? The overall goal is just to have a consistent way to measure usability of gitlab and track that as a department KPI and since the 10 scale, questions assess the product as a whole. The sus itself is not diagnostic and cannot be used to pinpoint specific problems. However, we realized that we need to be able to identify those critical areas and issues that users are having in order to actually make changes that will improve usability over time.

A

So the solution to that challenge has been gathering more granular feedback on user needs and challenges through the open text, questions that are included in the survey every quarter. So these questions give users the space to share additional thoughts around usability and optimizing get lab for their workflow, and since we gather such a large amount of feedback from this quarterly initiative, it has become increasingly important to find streamlined ways to share and prioritize that feedback.

A

So, in order to make these findings more actionable and easy to share throughout the company, we highlight themes across supporters to help us understand the frequency and severity of the usability issues. Users report in their responses. So in this series, I will discuss the top themes. We've gathered from the sus surveys conducted thus far and highlight opportunities for improvement, but also shed light on themes that our department is already doing, such as UX scorecards and our okrs. That support improvements to usability.

A

You.