GitLab Pipeline Insights Group, 30 Jun 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Verify:Testing Category walk through June 2021

Description

A walkthrough of the current capabilities of the Testing Categories, what tiers they are available at and the near term roadmap for the Testing group as of June 2021.

You can find previous versions of this recording in the Testing Group Playlist: https://www.youtube.com/playlist?list=PL05JrBw4t0Kq53VUOvTk3VdXN79PA0SXT

Testing Group Milestone Kickoff Playlist: https://www.youtube.com/playlist?list=PL05JrBw4t0KradvCrf8fL2pggEn5D3w2k

A

Hi, I'm james heimbach, the product manager for the testing group in the verify stage of give on uh today, I wanted to record an update to our category walkthrough that I last published in april of this year, so go ahead and share my screen.

A

I do want to start this with talking about this. Is information related to upcoming products, features functionality? So all those standard warnings apply.

A

uh So, just to remind you, the mission for the testing group is really. We want software teams to feel confident that the changes they're introducing into their code are safe and conformant, and our vision long term is that we want to enable developers, development teams to have production-ready code within an hour of the first commit to an mr.

A

Talk a little bit about our jobs to be done I'll, add a link to our jobs to be done page in the handbook, so you can dig into those jobs to be done as you want, as well as our current capabilities.

A

I did want to highlight, because in june we released the 14-0 milestone or we hit the 14-0 milestone. The team has done a lot of great work since 13-0 introducing things like failure. History, visualizing, test coverage tracking test coverage over time at both the project and the group level, along with many other changes across the other categories. That testing is responsible for and I'll include a list to show all of the 26 things that we've introduced at the bottom of this video.

A

So let's talk about our tier value. Testing doesn't just provide value at the upper tiers.

A

We provide value across all of the chairs at gitlab, in the free tier developers can see how accessibility and test results are changing in their merge request, as compared to their target branch and team leads, can track test coverage within their project, so they can see how test coverage is changing over time by the keeping it at the level that they want in the premium, tier developers can see how load and browser performance has changed in their merge request as compared to the target branch as well as utilize, our custom reporting or custom metrics reports to track things that we don't necessarily track.

A

This is really leveraged by iot teams. You could use it. Our git lab team uses it to cap or track how big the ruby gems are, that they're generating. So that is something that you can do and then directors can track that test coverage in all of their projects, see it at the group level. So they can dig in into individual projects as they see a need to and see how they're changing over time and then in the ultimate tier.

A

A feature that we've just introduced is being able to see code quality changes as part of the mrdiff, so not just seeing within the widget that something has changed as part of the mr, but in the changes tab itself, seeing the line that that is changed on. So you can more quickly leave feedback as part of a code review or address things as a developer.

A

Just to reiterate, the accessibility changes seeing those in the mr seeing how test coverage is changing over time as well as then within the within the mr itself.

A

All of the jobs that make up that test coverage and visualizing within the changes tab is there test coverage for this or not in that premium tier, seeing that same kind of merge, request, widget for browser performance and load performance testing to see how that is, changing or would change in the target branch and then tracking across all of the projects as part of a group and seeing how is test coverage changing and then being able to dig into one of those projects as you go and in the ultimate to your brand new feature being able to see as part of your code review in that changes.

A

Tab is this line, something that is flagged or a new code, quality violation. What is going on here? Can I leave a comment as a reviewer? Is it something that I can fix easily as the author of the mr? So that's where we're at today, let's look a little bit at our near-term roadmap.

A

What we're working on now just finished up the code quality violations in that mr diffu and we'll be enforcing test coverage in mrs by utilizing a merge request, approval rule so using the existing user experience that teams are used to with security approvals, they'll be able to set a code coverage, merge, request, approval rule and ensure that that merge request can't make code coverage go down, but through approval, if you're in an emergency patch kind of situation that can be approved so that you can get that merge, request merged and that shipped.

A

What we'll be working on next is helping team leads and directors. Those personas see trends in their test data. Things like the test coverage over time, test, execution, history and the code quality violations at that project level. So a single page we're at a high level. You can see.

A

Where are those things at today and how have they changed over the last 30 days, currently doing solution, validation against this and love to talk to customers about how they're feeling about our solution and are we solving the problem the right way and later later on in this year? And into next year we're going to be starting to dig into flaky tests. We think that our first step there is seeing a project history report or understanding how a test is potentially flapping over time.

A

This is an extension of an mvc that we introduced between 1300 and 140. That'll show you if a test has failed. Is it already been failing on my default branch? How many times has that failed over the last 14 days we're getting good feedback from customers that that's a good indicator, that a test might be flaky, but there are next steps that we can take there, so that wraps up just the quick overview that I wanted to do today. As of june 2021, thanks.