GitLab Testing Group UX, 22 Nov 2019

Previous Meeting

⏯

youtube image

►

From YouTube: Testing UX:PM:Research review 11.22.2019

Description

Review of design work for upcoming milestones 12.6 and 12.7 and review of Solution Validation work the Testing group is doing.

A

Alright, alright.

B

So this is the testing UX research product meeting so on the agenda today. I wanted to talk through if there's anything remaining in 12:6 that needs design with one and we opted in on how the twelve seven items are coming along with the purpose. There is to make sure that the testing team can view this async or view the agenda and get updates and will carry those over into the issues or ask questions on our testing meeting next week and then we'll dig in on the item.

B

That's in solution, validation and talk through the things that we have in problem validation. So.

A

Screen awesome.

B

We go ahead and share my screen. I.

B

Try to share my screen.

B

So in twelve six it looks like the show a graph for code coverage is it has a deliverable tag, but it's still in the workflow design, State yeah.

A

That one's coming along I think it's that we have a good baseline based on that issue, that we have been discussing things on the issue. I shared all these signs with the CIC designers yesterday and they had some feedback and I, probably I, can sure. I can probably put that on the issues like people can see it, but the the main feedback that I got from it is like. Yes, it looks good, but there were two concerns.

A

One was what happens when there's a mono repo that might have like a lot of languages so like if we're gonna stack, the graphs I couldn't answer that question because I don't know exactly how many languages can like live in a mono, repo and I. Don't know how testing works like when he when it comes to code coverage, how does it work and I'm gonna report or naive, like when someone deploys to a mono repo he tests?

A

Absolutely everything I, don't know how how that's configured I probably wanted to discuss that with you, but like yeah. So the concern was, if there's like more de and let's say like 10 languages, why? If you're gonna stack ten of those graphs, then it's gonna look pretty bad. Probably it's not gonna be usable at all right. So that was one concern and then the other concern was also about the stacked rough.

A

Apparently there's an issue somewhere and I have the link like I can put it in the in the issue as well. There's kind of some accessibility concerns about stacking graphs.

A

Apparently, when you do that, like people with you know like like site issues, they probably because I have like trouble consuming those graphs. So all that kinda like made me think that maybe we should go back to the model that I had at the beginning, which is I, could drop down where you select the language. So if you have a let's say ten languages, you can just like kind of like, like waka, like trouble to them right likes people likes like Java screaming.

A

You know like blue, whatever you have on your tests, reading on, like whatever, like whatever you're testing, so that that made me think that that that that could be the solution. But yeah I wanted to hear your opinions on that. I.

B

Mean it seems, like anybody, who's, spitting out code coverage on ten languages in a modern repo. It feels like it's gonna, be an edge case okay and for a minimum. Viable change. I feel like we'll be okay, like we'll hit a nice 80/20, I'm kind of split with this has gotta work for 80% of users and yeah.

B

We could keep spinning on it before we released the MVC I'm sure we're gonna have a folks who it does not work for, but I'm enough to just refresh myself on the issue, but I thought we'd tackle to this problem already in talking through it like there's one code coverage metric that we were gonna target and use that, but I could be mistaken.

B

So let me refresh on the issue async and then I'll comment back. If you can put those concerns in to the design and just make sure that we haven't documented in the issue. Yeah.

A

B

We can document how we made the decision why you made the decision for the NBC right.

A

Okay, I'll do that. I was fine to do that today anyway, so and then you can like try mean.

A

All right, y'all.

A

All right, so the other thing was in the visual review up, which you seen in development right now. There was one decision, but I think it was never taken. So the sheet is basically. This happened, like a month ago.

A

I suggested that, like yes, you should be able to meet like as an anonymous person, but we shouldn't, like explicitly, can I have an option personally, basically, what you should be able to do is like you can pick whatever name you want and then, like the you know, address right, so you can like picking out, for instance, rights like it will be, like the only thing that it's mandatory is the name the email address.

A

It's optional keeps like the, and then you can, just if you wanna say like John, Doe or like an animals that will be fine, cuz, that's what I found. But for me it's going to capture. That's opposed to the likes just designs before where there was like an explicit decision to say, like submit external newsperson I, think one ever say anything about that. So, basically, like the way that I took it is like silence, means agreement.

A

That's good policy, so I can I wanted to touch back on that and I can I can try to find a comment where I captured that think it's fun I, don't think it's on the main issue say.

B

Which, which issue are you talking about.

A

It's in the I, don't think it's the main before review up issue, I. Think there's another related issue.

B

Is it an issue were working in twelve six? Is it already closed? It's.

A

Probably an issue that it's already closed, but it's relevant to these.

A

You know what let's not get stuck on these I'm gonna I'm gonna, find a comment and then I'm gonna bring it to the main issue: the eleven four seven, four okay. We can actually have like some culture yeah, but do you have thoughts about that? That was kind of my main question. Like do you think.

B

um I mean I'm, okay, with people being able to spit the anonymous feedback. We have. This thing feature flagged off right now, young and if you are running self hosted, you could always feature flag adopt yourself and it's only gonna be available if somebody starts up to review app, okay, but.

A

My main question was like you: have like a preference between like explicitly to meeting my ceremony moves like saying: I want to be anonymous and then we say like okay. Each person needs to me in the time.

B

I, don't think we need to add that feature on point I think we can. We can leave a text form and require you to put something there. Okay and the full code quality report in 12:6 is in an open state. It's a sign to Daniel still, but he should be getting it assigned. You have Fred ends yet today, hopefully, but that had some UX questions in it from drew on the backend side. Okay,.

A

Yeah, for sure I mean like I, can I.

A

Can look into it I remember this issue. This is the issue like showing, like the whole full quality before in the mr, we jib and like how like there's probably some concerns about how like we're gonna display it yeah yeah I'll, look into it but yeah. If that means design, it should be very easy to do. Okay, yes,.

B

A my guidance initially was use what what was done for the security report, where it's paginate it, and then we just sort by sort by points we remediation points with w stuff. On top, it seems like it's gonna hit. Most of our use. Cases of, let me knock out the big stuff and improve my code quality pretty quickly. Okay,.

A

Yeah, I think that I think that make sense I'll look into it anyway, to see by I mean and.

B

Let's dig into 12:7, then a little bit yeah so.

A

Working on I'm working on the CI view for selenium spines and when I was working on me so first of all, I'm going to share those designs early next week in like put them on education, so people can look at them and like give feedback like perhaps like, we can't thing that people like sure these early designs to get their feedback as well.

A

The the question that I had for you is that so, since this is gonna live in the CI view, the CIA view has the the runner logger right there at the top and these designs, as they are shown here, apparently they're, kinda like assuming that that's not gonna, be there. So my question for you is like I, don't think that's a correct assumption. I think that Runner lock should stay there and we probably need to do is collapse it and then show these right. Yeah.

B

So you would have within within a job you're looking at a specific output yeah, that's where you have.

A

A

So what I'm, assuming they were think, is like because it's a slanting job partner, actually right yep, because it's a slanting job like the probably thing. Okay, we, instead of showing these random logger, we can probably just show like the raw logs coming from selenium and like reach view with the screenshot. Then what I'm thinking is that we cannot really get rid of this, because this is also like printing things from the runner itself. Oh, you know it's. So what I was thinking is like the solution.

A

The way that I see it is probably the best thing that we can do is that we, we add, like collapsible functionality to this thing, and then this thing is collapse and then like below that we have the tabs. You know selenium whatever I, don't know. What are your thoughts on that? That.

B

Makes sense I mean just that already collapse? What's that? Second, no doesn't like actually I, just like scroll back and forth in the trip yeah, okay,.

A

B

If we can just minimize that window, yeah.

A

And I didn't eat should be pretty easy, like from like a like a front-end perspective. I'm gonna see what else you just on the issue and then I'm gonna talk to me tree to see if he has any thoughts on that, but I don't think I think collapsing. These makes sense, because if we're gonna keep like enriching this review, then we're gonna. We need to that yeah right, yeah.

B

That makes sense: let's share that, with what we'll share it via this video and get some feedback from our front-end folks to see if they have some input on. What that would feel like all right would.

A

Be yeah, so that's coming along I'm, very confident about like things that I'm gonna do I think that the only kinda like other question that I have is that there's this thicket for one pipe to which is in solution. Validation as well and I was wondering if that. How is that different from what we already did? Solution validation, wise on selenium yeah,.

B

So this was just a result of you and I both opened a solution, validation, item, okay,.

A

B

So I'm, usually we talked about, we were just gonna, leave them both open for now and then figure out what to do when we got done with the interviews. So that would be a great time to have that conversation of whatever we had to do it. These two issues.

A

Well, I read through these one and I think we kind of already answered those questions through the interviews. Yeah so I wouldn't mind, closing these one and then like basically like linking this issue to the other issue easily.

B

A

And say, like we I, don't know whatever you want to do, but I think like which looked close, probably close this one.

B

So Lori love your input here, what's the normal workflow for something that goes from solution validation, and is that you X item or is that just an issue in the gitlab project and how does that get into then the development workflow? What's? What's your experience been with that yeah.

C

So what I know is that from doing the solution, validation, you should have a research issue, open that says, somebody's doing research on the solution, validation for this particular process or feature, and then you conduct the research and then I've sent one some instructions on how to do this.

C

Once you look at the issues, the findings from the research for the issue, there's a process to create a in epic in the insights repo, and then you put each insights associated with that epic in as an issue and you link them and then you tag the heck out of everything, and this allows people to go back to the insight repo later in search for CI, selenium and, and you can find whatever was dealt with- that. What did we find what what projects after that's done?

C

There also should be a discussion between whoever did the research and product and development to say this is what we've learned. This is what I suggest that we do with what we've learned. What would you like to do with what we've learned so that process is different with everybody, because everybody kind of has their own little situation? How they want to handle it, but typically the product manager will say: ok, we will. We will want to change the feature in this way or take on this ADIZ additional pieces now development.

C

How much long, how much time will it take? Can you get it done in X milestone? You tell me like and then that's that negotiation process of trying to put it on the roadmap. After that? Ok, that's.

B

My understand after we've validated the solution, mm-hmm.

C

B

Cool yeah because.

C

Validate and solution might also uncover other issues that you may have to pull it out of the roadmap, for because you might need to design additional solutions and invalidate those too okay cool.

B

So it sounds like we'll need an insights item for insights.

B

Yeah and then yeah one of these two, but my inkling is take the old issue. Yeah we closed the one that we have. We work some of the feedback into the which one is it the one? That's in twelve, seven now, okay to see I view for selenium NBC. Ideally, what would have happened is instead of that being an NBC item.

B

It would have been initially just an item with a problem: validation, workflow tag on it or works a little label on it rather and then it became one with the solution, validation label, and then it turned into an issue itself, I think as I'm reading the product development flow handbook page that looks like what the intent is. I think we could always spawn new issues, though, if our solution validation has turned like that issue has grown so much or our problem validation turned into. We have to validate for solutions like that.

B

Initial item isn't going to be one thing that we keep iterating on over and over instead it'll spawn out a bunch of new solution, validation items, so I would yeah, let's close the one that I have opens, we'll keep the one you have opened and create an insights item over in over in that insights project, so sound reasonable as next steps. Yeah.

A

So I think yeah so basically refreshing that so it in baby aged to close.

A

Close this one right this then we I'm gonna, add all the insights to the UX research insights and we linked all those issues to this one right, yeah.

B

Well, there we'll put a link to the inside stuff in there.

A

B

C

A

No I I think we.

A

Research I'm gonna pray be apt. I'm gonna do all that. It's searchable I'm, just saying like for our internal processing testing I, probably should put the insights.

A

C

What they, what release had me do and I'm doing this for everybody else that I'm finishing up projects for is in the epic in the insights repo to also link the issue, the initial issues, so where all of these discussions started as well as the research asked issue, because that happened outside of that insights, repo, that's in the research repo, so I'm Lee. If I, they told me if I link those to that epoch, it will pull through to those issues. So.

A

C

Yeah, so I don't think that's documented anywhere. Yet it's on my list of things to suggest in in Mar 4, but yeah that might make it easier. I, don't know.

A

All right, so the other thing that it's on open on 12 7 was wondering like this um I am not aware of this. Yet.

B

Accessibility scanning yeah.

A

B

We put that into 12 7 to start the discussion with development of what it would take, but I think we might be premature since we're just wrapping they're still into I mean really problem. Validation of that, like I, need to put together the opportunity, canvas and review it. I, don't think we're going to get that wrapped and yeah the solution. Validation done in the 12:6 epic yeah.

A

Okay, so these seems that these could be like an actual. Actually, this could be workflow design item for full seven, maybe yeah.

B

We might end up doing some like I, think we'll get done with the problem validation cycle. We validated that a problem is out there and then we can work on design user story math and use that again as I'm reading this product flow.

B

We use those artifacts as part of our solution, validation to go back to either those initial users or a set of users who we've identified, who want accessibility, continuous accessibility scanning in the is part of their flow and show them that solution describe that workflow for them make sure that we understand it and that it solves their problem and gets their outcome right got.

A

It okay that make sense cool. Then these changes we did in visual review function. Oh yeah.

B

That's we're on the testing dreaming call earlier this week when we.

A

B

A

A

A CIC vu which review calls I, couldn't make it so.

B

What we initially thought was that this was going to be a great big thing and then, as we started digging into it, we realized that basically, that drop-down on view app yeah.

A

B

Want to add that to review. Oh that's literally, the change like all of the backend code exists. This is just a way to solve the problem like for me for release post I just want to go. Look at the thing. I just changed, yeah that.

A

Makes a lot of sense, yeah.

B

A

A beaut opens the app like in the first like in the entry point, so you got like find whatever changed. Yeah.

B

A

B

An interesting opportunity here with a tool tip or some help text or something about if you're, building a static site like the handbook or like about get calm, you can make use of the route map. So that pages that are like markdown pages, like the release post that gets mapped to a static page and, if you're, using that map and leverage that in the drop-down.

B

Okay, um so that you can get to that page itself, which is a case here that we would we would want to use, but that that feature and documentation of how to do that is like super hidden and not intuitive, and, like half of the team figured out what route maps were most of the team figured out what route maps were on that call, yeah yeah like granted, a bunch of us are new to get lab, but like it was still like one or two people knew that that existed, and that we can make you seven.

B

So that's something! We think that we can add to the discoverability urged I. Do awareness for users if they're using this, they might need to use that route map. So thinking about how we're using a tooltip there or helper text would be, it's super interesting as part of the UX cycle on this okay.

A

No that's a great idea, so I'm gonna look into these to watch record on grooming.

B

If we did I, don't know if it made it to unfiltered, you like to ask Elliott for a recording, yeah.

A

All right, yeah, all right so I think the last thing was checking on problem valuation items. So yeah.

B

So I haven't had a chance to go in and look at the insights all of the insights yet from the accessibility stuff. It's on my list for today or next week as meetings kind of RIA, Aidid.

A

And there's a lot of like first of all, things Laurie that was pretty great I, think there's so much useful stuff. There I think yeah I think we we probably gotta, do like James you and I. We probably should do Kanaka, maybe like get together like what for like a three minute so I try to like analyze.

A

What's the most important thing that we got from that yeah, we we can already have an idea like based on like what I'm seeing on the on the like on the twelve seven I, okay, whatever that twelve seven are in, like ultimately, accessibility to like to the review ups I. Think that's not a good first move from these, but I think it's a good idea to just like analyze from those insights.

A

Yeah yeah steps are yeah.

B

You and I do a deep dive on the results that makes its way into the opportunity canvas and then we go from there, because if we get approval on the opportunity canvas, we may we may even start doing like some story. Mapping and design taking yes starting to do some design and bring that with us as potential solutions or potential workflow items that we could then validate as part of a solution. Validation cycle.

A

B

We are at time and speaking of validation, the my next thing is actually an interview for load, testing, problem validation that I need to go jump on yeah.

A

All right, yeah I'm, we're not doing that one too, so great all right. Thank.

C

You guys bye guys.