GitLab UX Team, 10 May 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Vision for the AI UX framework

Description

Pedro Moreira da Silva, Staff Product Designer, shares a vision for the AI UX framework at GitLab, with its patterns and dimensions.

• Slides: https://docs.google.com/presentation/d/1rO2BpI2WZC9Dxhv7oVR6XHk8GMb77AswESYcDANnQhA/edit?usp=sharing
• Epic: https://gitlab.com/groups/gitlab-org/-/epics/10334
• Figma: https://www.figma.com/file/s4TP1i2Akd1VTh4jhbg234/AI-prioritized-prototypes?type=design&node-id=2195-41323

A

Hi, my name is Silva and I'm. The ux functional lead for the AI integration working group and also a staff product designer at gitlab, and today, I'm going to share with you the vision for our ux framework for the interfacial intelligence efforts that we're putting together. So what is the problem that we're facing? We have more and more teams working on AI driven features. We have a fast space to ship experiments. Immature features teams are working on very similar problems which ultimately is leading to inconsistent ux throughout our AI explorations.

A

Ultimately, what we want is one AI experience for the devsecops platform and the ux framework helps us achieve that by helping teams apply the right patterns and help maintain a consistent, AI experience. So what are some of the dimensions that we can use to help us evaluate? What is the most appropriate ux pattern to apply to the AI experience I'm going to show some patterns that we have identified later in the presentation, but let's first look at these Dimensions.

A

So as an example, this is the dimension of the technical solution, so these three levels were shared early on in the integration efforts where the first one ad hoc task is an immediate API response and is the easiest technical solution and from there we start increasing the difficulty with pre-process and fine-tune and then finally a custom technical solution.

A

So what about dimensions for ux patterns? The easiest one is about interactivity. So do we want the AI to engage with users proactively or reactively, based on user interaction, where users are aware that they're going to trigger an AI engagement, then we have the approach you can automate or you can augment. Automates tasks is about increasing the efficiency of humans by replacing the human decision making and actions, always with the awareness and consent of users.

A

We didn't have augmenting augmenting capabilities is focused on increasing the effectiveness of humans by supporting and improving their decision making and actions, then we have the task. There are many ways that we can slice and dice these tasks. That AI helps us with, but these are just three possible ways to categorize them, so classification categorize suggest rank match generation.

A

What everyone's talking about with generative AI summarize explain, create and also prediction or regression, which is about forecasting continuous non-categorical data like numerical values and finally, we have the mode so I think this is would be probably the most helpful to understand. All of the patterns that we've identified so far um and uh the three levels here are focused supportive and integrated. Where focus is where the AI is.

A

The main context with the dedicated Focus supportive mode is where AI complements the main context and accompanies users along their journey to help them achieve their goals and integrate. It is where AI is Blended into the existing experience into specific moments of users flow to help them complete small discrete tasks.

A

So, let's take a look at some of the patterns. Let's first look at supportive, the one in the middle. This is also called the chat mode and that's to remind you. This is the the mode where AI complements the main context and accompanies users along their journey to help them achieve their goals, and we can see this here with a chat interface on the right side, complementing the main context.

A

It is a single thread version of what we're look going to look a bit uh later on. When we look at the focus mode, it's ideally context aware, and it will possibly allow you to navigate to and create other threads, and on the left side we can see how the dimensions help us choose the most appropriate pattern. So here for supportive, the approach is primarily about augmenting human capabilities. The interactivity is reactive. It only appears when users intentionally want to interact with it task.

A

It is able to help us with any kind of tasks can transition to focus. We can see that on the right side here, a small thumbnail of what the focus mode could look like and in this pattern AI can also suggest automations and navigation.

A

Here's an example of supportive mode, complementing the main contacts which is filling out a form or series of forms as part of wizard, and so users can interact with AI to help them fill out. All of this in the easiest way possible.

A

Now, let's take a look at the integrated patterns, so in integrated mode. This is an example of search, so it acts almost like a mini version of what we've just seen with supportive and something like this can be possibly integrated into the doc site. So here we trigger the global search in gitlab and users are able to ask a question and have ai answer that question. Furthermore, we can suggest other questions below related to the answer and even have a button here that may allow users to trigger the supportive mode.

A

So there's always this connecting thread between all of the different modes, integrated supportive and focused as we're going to see later.

A

Another example of integrated is help a small addition here to The Help menu that allows users to trigger the supportive mode to ask questions and get help.

A

Another pattern text area or in our rich or markdown text, editors, the ability to generate content with AI. So how can we blend this into the existing experience and again, possibly triggering the supportive mode? In case users? Have follow-up questions or requests the same thing that we were just seeing with text areas, but in a much smaller scale for text inputs. So in this case an example of how we can help users build complex queries to filter the lists using AI in the same list.

A

Possibly we can use AI for ranking and have a column just for the score, and when users hover the score, we can provide explanation of the criteria that AI use and what how it got to that result and again, we can trigger the supportive mode in case users want to ask for help in, for example, interpreting this data and how to make the best use of it.

A

Similarly, we can use the pattern with a popover for inline explanations of terms and other words that people see in the UI integrated. We also have the task of summarization, and this is an example of how we could provide the activity summary in issues epics and merger quests, or any other object in gitlab that has activity.

A

So AI would summarize everything that has happens and again allow users to transition to the supportive mode in case they want to request certain actions from AI or help interpret the summary and the same thing could be applied to description summaries that keep a block of Co of content that is always up to date with the issue activity or the merge request, changes or whatever we feel is valuable to keep as a AI generated summary in the description.

A

And finally, this is the task of prediction. So, in this chart pattern we're forecasting, continuous non-categorical data, like numerical values and AI, showing this forecast allows us to transition to supportive mode in case again, users want help to interpret the forecast and how they can make the best use of it, and so we've seen supportive mode. We've just seen some examples for integrated.

A

What about Focus so focus is something that we're still not sure if it will be developed or not or exactly how it would look like, but this is an example of how it could be so and specific entry in the navigation for focused that lists all of the previous threads that and interactions that the user has had with AI, and it's almost like a maximized version of what we've seen with supportive and it allows you to browse and navigate and create threads and also the most interesting part is possibly navigating back to the original context where users triggered the interaction with AI.

A

So if you had the, uh if you were interacting with AI in an issue, uh maybe going back to that issue, if you're in a file and you've asked AI to explain a block of code going back to that file that you looked at some days ago and again in Focus, the AI is the main context with the dedicated focus and that's why we're explaining it here.

A

So ultimately, we want one AI experience, and here is a potential example where users ask a question about how they can use a specific feature or set up a specific feature. Ai provides the answer and possibly even suggests that AI helps us configure that feature.

A

So we go all the way to the form where AI helps us with natural language in the supportive mode to fill out the the form and configure the feature and then finally, we can jump to focus mode in case we want to go back to other interactions, we've had with AI, so this spectrum of focused, supportive and integrated is what allows us to have a unique AI experience and a single thread that connects all of this and back from um supportive, sorry from integrated to supportive to Dedicated.

A

Thank you so much for watching. This is not necessarily a prescription of what we're going to do. It's just Explorations and setting a vision for what we want and what we're looking to explore in our AI integration feel free to comment with your thoughts. Ideas, suggestions, concerns in the Epic. That is a link from this slide, which is also linked from the description of the video below. Thank you so much and until next time.