GitLab Testing Group UX, 5 Aug 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Testing UX / PM / Research Sync 2020-08-05

Description

Today we discussed our current research and design projects, next steps for the Category Maturity Scorecard for the Code Testing and Coverage category and Juan and James talked about how the team can integrate the work being done by the Certify Group to capture tests and requirements for releases to extend the Test History MVC.

Test History Solution Validation: https://gitlab.com/gitlab-org/ux-research/-/issues/951
Code Coverage Reports Problem Validation: https://gitlab.com/gitlab-org/ux-research/-/issues/989

Quality Management MVC issue: https://gitlab.com/groups/gitlab-org/-/epics/3852

A

Sounds good: this is the verified testing, ux pm research, sync for august 5th. um It looks like I have the first point I just wanted to check in with juan. Since we were doing this synchronously, we got a pile of design issues in the 13-3 milestone. Is there anything I can do to help with those.

B

uh I don't think so. I think I'm good. uh I I think I'll, let you know later, if there's any blockers, but I think I'm yeah, I'm good. Okay, great yeah.

A

um And then the next one up was uh looking forward at the maturity chart, um we're slated to move maturity for code testing and coverage from minimal to viable by october.

A

I think the first step there is um doing a umex ubux measure um to get the core job to be done, measured and see just where it's at um and if we do that externally or if we do that internally, uh just validating uh where we're at to move from minimal to viable for that uh and the defined core job to be done so kind of just wanted. To put it out. There of this is the first step, just opening a new recruiting request and a new research item.

A

To do that, I just wasn't sure what the process was.

B

uh Yeah, that makes sense, I mean it's, uh I think it's just doing the recruiting and then doing what we did before right.

A

Yeah, so I think yeah I figured we just mirrored the process for accessibility, yeah, okay, yeah right! Well, I will get that kicked off then um today or this week, uh so that we can make sure we're moving forward there and see where we're at so.

C

You are spot on on on a process james and you link to the page uh the right page. I also like kind of like highlighted it here. um We have the description of the new uh cms process there. uh The first step is to define, recruit the users and there you go step by step.

C

um I'm sure if there is any questions that we're gonna have or anything that we will need help with worries uh she's with us and she would be super happy to help or let me know, uh I would also be happy to solve um any roadblocks that are coming in, but it's it's exactly it's uh to define who we want to recruit and then open up recruitment, issue, etc and uh yeah move move with the standard process.

A

So all right, that's the whole agenda for the day. um Anything else we want to talk through um about our current, like the things in flight, are solution. Validation on the test history um report, uh that's going to be the the first one is the one that's plugged into um the project level for the manager uh we'll follow that up with test history back into the test, summary widget and the junit report.

A

But I think we made great progress on that in the last week with all the solution, validation, interviews and customer conversations that we've had and reviewed the design with those folks.

A

So that's in flight and then the other one is uh I'm going to be writing the discussion guide today for code coverage report, problem validation, I'm trying to understand where there's opportunities and what the problem is um with code coverage reports that are generated, but not viewable within gitlab, and what is the data that helps a customer solve their problems that they would go to their code coverage report for see what that opportunity is.

B

uh I'll be showing the the test history designs tomorrow in our uh design review session. So that's gonna. Give us like further. You know like input, I mean we're very close. I think we're circling the solution, but that's gonna give us like more polish. At the end, yeah.

A

um Sorry I yeah after we got off that call yesterday with mark, um so we talked with mark about the test test, quality quality management um and the way that they're gonna start storing history kind of rolling it up to a release started to think about.

A

Can we use whatever data framework they put into place, especially on that report for managers and say here are the individual pipelines that ran back to a release where you cut a release and then just start reporting release by release on the test history? Right? Because that's really all you care about at some point when you release this, this test was passing.

A

What happened between those releases doesn't really matter potentially, um and so we might want to validate a solution like that same thing, then, when we look at um test history for the cus or for the junit and the test summary report and looking at here's the test, as it's run on your branch back, you know, however many iterations or pipelines you've run and then here's previous ones from where you branched from where you started um back to the branch that you're going to merge into right.

A

So you can see, is it? Has it been flaky in your target branch or did the flakiness start in your source branch.

B

Yeah, I think that makes sense I still kind of like. After that meeting. I was wondering like a little bit of how that's gonna work on their site, since this is a new issue of all type that you create.

B

So I assume, there's gonna be like some manual process when you do that, you know like some, you gotta pick the tests or you gotta.

B

I don't even kind of fully understood if there's going to be like some automation being done on their behalf, like on the behalf of the user, yeah.

A

So yeah I mean the mvc sounded like it's just you can create an issue and call it a test yeah exactly.

B

Development yeah yeah yeah. I know I got that it's almost like another type of uh yeah. Our requirement is like a new type of issue right. So like it's the same thing, but I was wondering I mean what, if that's gonna be the case, there must be some kind of data that it's being exposed through the issue right and that's where I was like. I mean like what what are you gonna show there. You know.

A

Yeah, I I think that what I interpreted from mark's vision um is that eventually it'll be here: we've identified tests and these tests roll up to a requirement, so you can validate that passing test and it's probably more of an integration test or an end-to-end test has passed and it then this requirement has been met, and so that's where I think we can lean and understand hey this requirement. That is shown as passing, um contains this test. So we can say that this test has been passing in this release or this milestone. Oh bought it.

A

Okay, that's how I'm thinking about how we stitch that back without having to store every single test result.

B

A

Like how do it gets us out of having to figure out? How do we parse and aggregate data as well.

B

Yeah, I think that makes sense actually.

B

Yeah, I think that makes sense. I think we should just uh uh I'm gonna meet with nick uh next week. He said he was gonna drop, something on my calendar so based on that, I think when I have way more context that I can bring back to you yeah.

A

I'll try to draw something up just to diagram it and to better explain what I'm talking about. As far as like here's my pipeline, here's, the pipeline, here's my.

C

A

Here's the branch I'm merging into and then also the here's, your default branch for your project or any branch for your project and then back to releases to kind of stitch those two things together. I think that that just gets us out of the. um I think it helps solve our potential performance problem of storing every single test front forever, which is just not a place. We want to get to.

B

Yeah that makes perfect sense all right, yeah I'll have that in.

A

Mind uh nadia, and whoever else is watching this sorry for the lack of context there. It was a discussion that uh mark wood and I had started previously brought the designers into we're gonna get the engineers into. I will link to their direction page in the youtube um in the comments, so that everyone has that context of the discussion that we had in the vision and vc that we looked at yesterday.

C

Awesome thanks a lot thanks for adding yeah.

A

Sorry about that, we just jumped right into the end of the conversation that we had yesterday and forgot to add that all right.

C

Is there is anything that I can help with or with the solution validations or with cms process, or with any of the design issues juan that you have on your list like? If there is anything that I could.

B

uh No, I think, I'm good. I mean I have plenty of work to do, but also uh yeah. I just basically need to like a sink uh asynchronously ask james about certain priorities, but I mean looking good. I don't think I have any blockers on my side. I was actually looking at them right now, there's kind of like so it's one, two three there's three of these that already have work off like I have like plenty of things.

C

B

Oh by the way, there's these- I actually talked with maddie about that um on monday, and I added, like a label type this, that it's called like jj and it's active and standby. You that's mainly for me, but if you see it with active, it means that I'm like really looking at it. You know yeah.

B

I saw that show up on one of those issues.

A

That we, I pinged you on, so I will uh look for that to get status asynchronous.

B

Yeah yeah, that's kind of like another signal for you to like be like. Oh okay, like this is being totally like he's paying attention to that.

A

Oh yeah, I trust that you're always paying attention to the issues. Usually after I ask you is I realize that the design is already there that you might get in the comments and we're discussing it so yeah. I just don't dig far enough. Yeah.

B

I know for sure, uh but I'll try to like uh do that more with the labels too. Well so like I can like you can like, so you can block me like hey this one seems happy, but I don't see these signs. Do you have designs for that? So I can't of course yeah. I think I'm good uh I'll. Probably thank you later james. I have a couple of questions very specific things that we have here on.

B

uh We have a. We have one on 13.4, one on 13.4, one on one on backlog and two on 13.3, so I want to just make sure that we uh I mean I'm guiding myself based on those uh milestones, but I also want to make sure that that's the priority you know.

A

Yeah sounds good yeah we're gonna all right.

B

Right, uh yeah, I don't have anything else. How do you do you have something else.

C

All looks like we're set great, don't forget to reach out to if you, if any help from her needed on the cms, because uh yeah we've been discussing just a recent one that was uh run in uh create in depth.

C

So the process is still a bit like yeah, it's easier, but it's a bit uh well complicated, maybe maybe not yeah. So yeah just wanted to say that.

B

All right, well, yeah I'll, definitely make sure to reach out to her, like I did the first one with the old process so like it will be again everything just like learning the brain again coming from scratch right, so uh I'll definitely reach out to her and ask about that too.

B

Awesome all right have a good one. Bye.

A