GitLab Pipeline Insights Group Open Meetings

24 Jul 2023

Sync discussion to decide the direction with an ongoing MR https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121750

4 participants
15 minutes

project

proposal

merge

allows

concerns

share

adding

complicate

grouper

ci

24 Jul 2023

Feedback on the design proposal for secrets manager POC.

3 participants
50 minutes

gitlab

secrets

tooling

maintainers

storing

private

project

accessing

sharing

details

22 Jun 2023

No description provided.

1 participant
13 minutes

project

documentation

enforcing

decisions

concerns

purpose

variables

workflows

insights

deployments

5 Jun 2023

No description provided.

2 participants
9 minutes

redesigning

important

insights

issue

validation

having

discussion

vault

project

gitlab

30 May 2023

1 participant
13 minutes

uploading

rails

backup

storage

pipeline

workhorse

copy

aws

slow

optimized

30 Mar 2023

Brain dump on variables from Dov as part of the category transition.

2 participants
57 minutes

variables

variable

inject

configuration

manipulate

predefined

complicate

users

gitlab

commit

7 Feb 2023

No description provided.

5 participants
28 minutes

testing

capabilities

validation

intelligence

process

gitlab

pipelines

considering

development

pass

31 Jan 2023

This quick video highlights the Pipeline Insights plans for FY24.

1 participant
5 minutes

insights

testers

pipeline

plans

detecting

features

milestone

gitlab

deprecating

operationally

26 Jan 2023

This is a sneak peak of the new Artifacts page we are rolling out in milestone 15.9.

1 participant
5 minutes

artifacts

artifact

manage

maintainer

deletion

users

insights

project

pipeline

browse

10 Nov 2022

Theme: Build Artifact improvements: Usability features and Performance bugs.

2 participants
4 minutes

milestones

testing

debug

insights

pipeline

bulk

documentation

planning

features

issue

10 Nov 2022

Reboot of our internal stakeholder monthly sync.

5 participants
31 minutes

insights

pipeline

analysts

review

process

updated

project

capability

concerned

ongoing

2 Nov 2022

No description provided.

1 participant
8 minutes

functionality

gitlab

artifacts

insights

features

pipelines

ui

improvements

testing

iterating

Jackie Porter (Director, Product - Verify & Package), Jocelyn Eillis (Product Manager, Pipeline Insights), and Gina Doyle (Product Designer, Runner & Pipeline Insights) review the survey to validate Artifacts JTBDs, discuss the plan, and finalize next steps.

JTBD issue: https://gitlab.com/gitlab-com/Product/-/issues/3954

3 participants
16 minutes

tasks

survey

project

review

thinking

completed

making

job

plan

organization

14 Apr 2022

Gina Doyle (Product Designer: Pipeline Insights & Runner) walks through the design proposal to improve the Enable Review App modal today in the Environments page.

Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/352528
Docs referenced: https://docs.gitlab.com/ee/ci/review_apps/

1 participant
6 minutes

git

reviews

apps

kubernetes

manually

project

gitlab

deprecated

documentation

preview

7 Feb 2022

No description provided.

2 participants
25 minutes

testing

expired

overview

junit

artifact

workflow

debugging

indication

capabilities

issue

19 Jan 2022

Feature Change Lock (FCL) Closing Ceremony for the Verify:Testing group.

1 participant
4 minutes

fcl

investigation

mitigation

review

issue

corrective

testing

considered

retrospective

scl

9 Nov 2021

Today we dug deep into what's next for test history at the project level and how it can help the engineering productivity run fewer tests and move closer to their goals of shorter pipelines.

* https://gitlab.com/groups/gitlab-org/-/epics/3129#note_718919880

5 participants
30 minutes

testing

onboarding

workflow

reports

enhancements

tentatively

review

delivery

tooling

contribute

28 Oct 2021

Gina and James talk through getting the Project Quality Summary MVC issues to be more up to date, what's really in the MVC and how we got where we are about the data being presented.

2 participants
19 minutes

testing

finalizing

authentication

tasks

sync

improvements

validation

taking

discussion

ahead

28 Oct 2021

Gina Doyle (Product Designer, Verify:Testing and Runner) walks through a proposed update to test coverage report badges (found on project overview pages), which will ultimately be carried through to the Project Quality Summary that the testing group is working on developing.

Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/344231

1 participant
3 minutes

testing

badges

colors

coverage

indication

ruby

issue

pajamas

documentation

icons

26 Oct 2021

Walkthrough of the 14.6 planning issue: https://gitlab.com/gitlab-org/ci-cd/testing-group/-/issues/72

The big focus will be on resolving performance issues of slow build artifact endpoints previously identified.

1 participant
3 minutes

refinement

performance

planning

endpoints

features

slow

ahead

milestone

project

refined

25 Oct 2021

James and Michael spent some time talking through an issue to add trace collection to GitLab CI/CD Pipelines, the tech behind it, what's happening in the industry and possible future vision for this area of functionality.

https://gitlab.com/gitlab-org/gitlab/-/issues/338943

2 participants
50 minutes

cicd

workflow

threading

pipelines

technical

monitoring

tracing

cumbersome

discussion

commit

21 Oct 2021

Gina and James cover how to wrap up current research around JTBD and Opportunity validation and how to hand those items off at the end of the quarter.

3 participants
18 minutes

testing

scheduled

profile

performance

task

interview

review

users

ready

recruiting

13 Oct 2021

James and Gina discussed the iterations on the Project Quality Summary MVC (https://gitlab.com/groups/gitlab-org/-/epics/5430) and what next steps would be now that the initial Jobs To Be Done (https://about.gitlab.com/handbook/engineering/ux/jobs-to-be-done/) research is done for Usability Testing.

2 participants
16 minutes

milestones

discussion

gitlab

testing

project

features

taking

gina

sync

week

7 Oct 2021

Gina and James talk priorities, the new iteration plan for the Project Quality Summary, progress on the MR Widget WG, how we delete job logs and how that compares to job artifacts, and why the dogs in James' neighborhood always bark during this call.

3 participants
22 minutes

milestones

reviewing

prioritize

process

completed

updates

planning

performance

ahead

project

30 Sep 2021

Gina and James did a bit of a deep dive on what's next for JTBD research in prep for upcoming maturity changes (covered in a previous video: https://www.youtube.com/watch?v=4dRkVVS-0s4).

2 participants
24 minutes

testing

sync

maintenance

tasks

reviewing

prioritize

performance

ready

screener

timeline

30 Sep 2021

Today we talked about some current research around Usability JTBD and insights uncovered related to Environment Management. We also did quick overview of how GitLab is using Metrics Reports (https://docs.gitlab.com/ee/ci/metrics_reports.html) uncovering a possible degradation. Lastly we talked about the future of test reports and how filtering by pipeline is an important use case for our internal users.

3 participants
15 minutes

testing

reports

gitlab

users

review

validation

process

verify

important

taking

28 Sep 2021

How we use GitLab to display our roadmap to different stakeholders internal and external in the Verify:Testing team.

All the team category direction pages can be found on our team page: https://about.gitlab.com/handbook/engineering/development/ops/verify/testing/#mission

The Quarterly View: https://gitlab.com/groups/gitlab-org/-/boards/2350419?scope=all&label_name[]=group%3A%3Atesting&not[label_name][]=priority%3A%3A1

Epic Roadmap: https://gitlab.com/groups/gitlab-org/-/roadmap?state=opened&sort=end_date_asc&layout=MONTHS&timeframe_range_type=CURRENT_YEAR&label_name%5B%5D=group%3A%3Atesting

All the refinement kickoff and milestone kickoff videos are posted on the Verity:Testing team's playlist: https://www.youtube.com/playlist?list=PL05JrBw4t0Kq53VUOvTk3VdXN79PA0SXT

1 participant
8 minutes

stakeholders

gitlab

roadmap

testing

informational

milestones

thinking

refinement

current

validation

23 Sep 2021

Gina and James discuss how to analyze the data coming from the Jobs To Be Done interviews for the Usability category, when we lock down the planning issues and the research James and Scott are starting to schedule.

2 participants
11 minutes

testing

edits

brainstorming

analyzing

refinement

project

findings

review

users

sync

22 Sep 2021

Walkthrough of the team's 14.5 planning issue. https://gitlab.com/gitlab-org/ci-cd/testing-group/-/issues/70

The big focus of deliverables is on the theme of GitLab-hosted First: https://about.gitlab.com/direction/#gitlab-hosted-first

1 participant
4 minutes

milestones

gitlab

repository

refinement

planning

capacity

deliverables

endpoint

maintenance

priority

15 Sep 2021

Gina and James discuss priority on a MR page UX improvement, current research initiatives and priority vs. design iterations.

2 participants
10 minutes

reviewing

workflow

milestones

issue

updates

interview

priority

project

testing

product

9 Sep 2021

Gina and James remember they have to record a 14.4 kickoff and talk about the design issue and research issues that will be worked on during that milestone.

Project Quality Summary Iterations: https://gitlab.com/gitlab-org/gitlab/-/issues/337307
Usability JTBD Research: https://www.google.com/url?q=https://gitlab.com/gitlab-org/ux-research/-/issues/1557
User Journey Research: https://gitlab.com/gitlab-org/ux-research/-/issues/1552

3 participants
8 minutes

scheduled

discussion

interview

workflow

gina

week

priorities

sync

testing

feature

2 Sep 2021

Gina and James talk about how the changes to Maturity dates (https://www.youtube.com/watch?v=4dRkVVS-0s4) impact the UX KRs and Gina's research in FY22Q3.

2 participants
18 minutes

testing

2021

workflow

ready

review

validating

updates

ux

performance

process

31 Aug 2021

Walk through the updates made to Maturity Plans for the Testing Categories from Aug 2021.

Maturity Page: https://about.gitlab.com/direction/maturity/#verify

Category Maturity Scorecard info: https://about.gitlab.com/handbook/engineering/ux/category-maturity-scorecards/

1 participant
8 minutes

gitlab

testing

updates

maturity

validated

features

delay

2023

process

reviewing

18 Aug 2021

Walkthrough of the 14.4 planning issue focusing on the theme of the Milestone (Build Artifacts), Design issues being worked on and Bugs to address: https://gitlab.com/gitlab-org/ci-cd/testing-group/-/issues/68

We also go over the high level of what the team is working on in the second half of the year.

1 participant
6 minutes

milestone

planning

project

testing

features

dashboard

performance

artifact

ongoing

refinement

5 Aug 2021

Today's the first recording with our new Product Designer Gina. We recapped what she's working on and refined an upcoming change to the Browser Performance Widget display to show percentage of changes as an MVC and future iteration ideas.

Performance changes as a % issue: https://gitlab.com/gitlab-org/gitlab/-/issues/226987

3 participants
18 minutes

milestones

browser

performance

updates

figma

prioritize

gina

discussion

plan

analysis

3 Aug 2021

No description provided.

4 participants
27 minutes

gina

onboarding

usability

initiatives

transitioning

testing

users

enhancements

discussion

tasks

21 Jul 2021

Walkthrough of the 14.3 planning issue focusing on the theme of the milestone (artifacts!) and how we'll be improving some MR Widget functionality in design during the milestone. The planning issue can be found here: https://gitlab.com/gitlab-org/ci-cd/testing-group/-/issues/63

1 participant
4 minutes

artifacts

maintenance

planning

capacity

milestone

project

validation

issue

recalculating

coverture

15 Jul 2021

James (Product Manager) and Erick (BE Engineer) Walk through setup and sue of the new coverage check approval rule: https://docs.gitlab.com/ee/ci/pipelines/settings.html#coverage-check-approval-rule.

2 participants
23 minutes

approvals

workflow

canary

coverage

developers

testing

issue

functionality

approver

editing

14 Jul 2021

Kickoff Page: https://about.gitlab.com/direction/kickoff/#verify
Ops Direction: https://about.gitlab.com/direction/ops/#verify

Issues Discussed:
* https://gitlab.com/gitlab-org/gitlab/-/issues?scope=all&state=opened&label_name%5B%5D=Category%3ABuild%20Artifacts&milestone_title=14.2
* https://gitlab.com/gitlab-org/gitlab/-/issues/322203

We love feedback and you can reach me at jheimbuck@gitlab.com or mention me on GitLab.com using @jheimbuck_gl

2 participants
4 minutes

milestone

gitlab

testing

refreshed

capabilities

gina

feature

verify

degradations

review

8 Jul 2021

The Verify:Testing group documented (https://about.gitlab.com/handbook/engineering/development/ops/verify/testing/#error-budget-issues) how they handle prioritizing issues that may come from review of the Error Budget dashboard and how this works with prioritization of Technical Debt and features.

2 participants
3 minutes

budgeting

dashboard

workflow

maintenance

debt

issue

technical

verified

prioritizing

allotted

30 Jun 2021

A walkthrough of the current capabilities of the Testing Categories, what tiers they are available at and the near term roadmap for the Testing group as of June 2021.

You can find previous versions of this recording in the Testing Group Playlist: https://www.youtube.com/playlist?list=PL05JrBw4t0Kq53VUOvTk3VdXN79PA0SXT

Testing Group Milestone Kickoff Playlist: https://www.youtube.com/playlist?list=PL05JrBw4t0KradvCrf8fL2pggEn5D3w2k

1 participant
6 minutes

testing

gitlab

capabilities

ensure

validation

reviewer

execution

project

warnings

tiers

21 Jun 2021

Walkthrough of the 14.2 planning issue focusing on the theme of the milestone and some issues that need attention during the refinement phase: https://gitlab.com/gitlab-org/ci-cd/testing-group/-/issues/62

1 participant
3 minutes

refinement

milestone

project

usertesting

endpoint

vitica

execution

quality

improving

review

16 Jun 2021

We (PM and Designer) discussed the business goal and opportunity for the project and the assumptions we will test as part of solution validation.

This was the in depth discussion of the epic overview in this video: https://www.youtube.com/watch?v=bWUyo4k8uvc

3 participants
23 minutes

validations

views

dashboard

usertesting

workflow

monitoring

purposes

mvc

endpoint

data

8 Jun 2021

Walk through the high level problem to solve/goals of the Project Quality Dash MVC epic and our initial thinking on how to implement.

Epic: https://gitlab.com/groups/gitlab-org/-/epics/5430
Design issue: https://gitlab.com/gitlab-org/gitlab/-/issues/322203

1 participant
6 minutes

milestones

validation

project

dashboard

execution

quality

endpoints

version

mvc

v2k

27 May 2021

Today we talked about the impact of Code Quality moving to the Static Analysis group, what the Engineering Productivity group is working on next and the best way to reduce time between build failure and start of the next pipeline.

Agenda: https://docs.google.com/document/d/1ijjjvLeIY6a82dS2_8H-G0bhvpaZ5zxPgYpU08EYaHc/edit#heading=h.b4f8595pqvm6

5 participants
42 minutes

mrdiffs

maintainers

climate

delivered

milestones

vetted

assessment

updates

quality

summary

21 May 2021

Walkthrough of the 14.1 planning issue focusing on the theme of the milestone and some issues that need attention during the refinement phase: https://gitlab.com/gitlab-org/ci-cd/testing-group/-/issues/58

1 participant
3 minutes

monitoring

mvc

refinement

functionality

milestone

quality

docker

dashboard

resolution

endpoints

18 May 2021

Quick run through of the new feature to display Code Quality violation data in the Merge Request Changes Tab. For information about how to setup Code Quality please reference this video (https://www.youtube.com/watch?v=4pJcrvZfGQU) or the documentation link below.

Documentation: https://docs.gitlab.com/ee/user/project/merge_requests/code_quality.html
Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/267612

1 participant
2 minutes

git

problems

app

reviewer

project

verified

degradation

demo

mrdiff

feature

10 May 2021

Taylor, Thomas, Becka and James talk through what's left for Testing to complete before handing the category off to the SAST group.

4 participants
30 minutes

discussions

handoff

milestones

proprietary

validating

workflow

ux

review

transitioning

ready

10 May 2021

Developers and testers let us know that while it was great to be able to upload a screenshot of a failed test into GitLab they were hard to find and match up to the test results in the Test Reports. We've implemented an issue that now includes a direct link to a screenshot captured that's associated with a failed test in the Test Report Details.

Documentation: https://docs.gitlab.com/ee/ci/unit_test_reports.html#viewing-junit-screenshots-on-gitlab
Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/216979

1 participant
2 minutes

gitlab

testing

screenshot

grabbed

fail

git

uploaded

demo

functionality

bug

29 Apr 2021

Today we caught up on how additional information about about tests like failure count and duration would be helpful for the Engineering Productivity team.

Kyle shared how the EP team is using reference tags (https://docs.gitlab.com/ee/ci/yaml/#reference-tags) to use the standard Code Quality template again and use the rules defined to get speedy pipelines.

Agenda: https://docs.google.com/document/d/1ijjjvLeIY6a82dS2_8H-G0bhvpaZ5zxPgYpU08EYaHc/edit#heading=h.b4f8595pqvm6

5 participants
24 minutes

testing

screenshots

reports

takes

noting

prepping

review

triage

showed

verify

22 Apr 2021

Think Small today about open metrics and the custom metrics reports available in GitLab.

Think Big portion: https://www.youtube.com/watch?v=bWqoHtS1Mco

4 participants
23 minutes

milestones

monitoring

metrics

thinking

refinement

small

data

project

dashboards

2021

20 Apr 2021

Walkthrough of the 14.0 planning issue focusing on the theme of the milestone and some issues that need attention during the refinement phase: https://gitlab.com/gitlab-org/ci-cd/testing-group/-/issues/49

1 participant
4 minutes

milestones

testing

refinement

issue

approvals

docker

maintenance

vetting

verified

capacity

13 Apr 2021

Today's ThinkBig was about how GitLab can better make use of OpenMetrics in the Custom Metrics Report to solve problems customers have. We had great insights from Kevin Chu the Group Product Manager for Monitor, Configure and Release about what he's heard from customers wanting to track data as part of their CI/CD Pipelines.

You can see an overview of why and how Testing does ThinkBig sessions here: https://www.youtube.com/watch?v=UAaOnnbIud0

4 participants
17 minutes

openmetrics

gitlab

monitoring

metrics

observability

gamify

operationalize

data

dashboarding

features

12 Apr 2021

Update to the original speed run (https://www.youtube.com/watch?v=B32LxtJKo9M) with new features introduced since original recorded in January 2020.

A new MR was created to show the features but using the original project: https://gitlab.com/jheimbuck_gl/jh_java_example_project/-/merge_requests/26

1 participant
3 minutes

gitlab

report

updates

rewatching

speed

quality

pipeline

testing

merge

project

12 Apr 2021

A walkthrough of some of the current capabilities of the Testing Categories, what tiers they are available at and the near term roadmap for the Testing Features as of April 2021.

1 participant
7 minutes

testing

gitlab

verify

workflow

capabilities

approvals

project

execution

ensure

reviewing

18 Mar 2021

Walkthrough of the 13.12 Verify:Testing planning issue: https://gitlab.com/gitlab-org/ci-cd/testing-group/-/issues/44

1 participant
3 minutes

wanderfalke

kiwi

app

seas

surprise

beın

people

live

iphone

cars

16 Mar 2021

Speed run explaining the Group Code Coverage Data feature.

Documentation: https://docs.gitlab.com/ee/user/group/repositories_analytics/#current-group-code-coverage

Feedback issue: https://gitlab.com/gitlab-org/gitlab/-/issues/231515

1 participant
2 minutes

gitlab

analytics

project

annotating

repositories

coverages

updated

testing

graph

repository

16 Mar 2021

No description provided.

3 participants
22 minutes

validation

issue

design

considering

finalized

confirm

diff

version

sigma

review

9 Mar 2021

No description provided.

2 participants
22 minutes

augmenting

dashboard

versions

implementation

ahead

discussion

issue

figma

milestone

validating

9 Mar 2021

Today's think small continued our discussion from thinking big about how to enable Testing features by default for customers or make them easier to use.

Think Big: https://www.youtube.com/watch?v=HmNosq4MsK0&list=PL05JrBw4t0Kq53VUOvTk3VdXN79PA0SXT&index=2&t=267s

4 participants
24 minutes

testing

enablement

milestones

tasks

implicit

introduce

automating

functionality

verify

incremental

2 Mar 2021

No description provided.

4 participants
24 minutes

vitica

issue

diff

sync

usability

testing

approvals

views

ui

updates

2 Mar 2021

Today's ThinkBig came out of a recent customer conversation. The customer is a GitLab advocate and user but wishes they were doing more "testing" or that it was just happening for them in their GitLab projects. This aligns with our On By Default Product Principle and the team wanted to discuss more ways we can get existing GitLab users introduced to testing features sooner.

4 participants
27 minutes

opinionated

testing

warnings

gitlab

mandating

big

verified

thinking

maintainers

prompted

18 Feb 2021

Walking through the process for how we create, curate and then debate (it had to rhyme) ThinkBig issues in Verify:Testing.

ThinkBig board: https://gitlab.com/gitlab-org/ci-cd/testing-group/-/issues?label_name%5B%5D=ThinkBIG%21

1 participant
2 minutes

think

big

expectations

comments

checklist

ideas

review

discussion

contribution

process

16 Feb 2021

Today we re-visited the various Think Big topics we've covered as a group to talk about how the very small thing turned out, what might be next and if we're still working towards the right big vision.

3 participants
30 minutes

milestones

testing

small

big

slow

processing

concerns

measures

tests

performance

29 Jan 2021

Speed run of the repeat failed test counter introduced in 13.8 showing how to see if a failed test has failed and how often on the default branch in the past.

Release Post: https://about.gitlab.com/releases/2021/01/22/gitlab-13-8-released/#repeat-failed-test-counter

1 participant
2 minutes

testing

gitlab

failures

tests

test

rspec

repeat

milestone

verified

runs

29 Jan 2021

Update walking through OKR alignment now that they are close to finalized for the quarter and how they map to work in this milestone for Testing.

Testing 13.10 planning issue: https://gitlab.com/gitlab-org/ci-cd/testing-group/-/issues/34
CEO OKRs: https://about.gitlab.com/company/okrs/fy22-q1/

1 participant
3 minutes

verify

capacity

milestone

updates

review

help

finalized

tests

reporting

75

27 Jan 2021

Today Kyle joined us to catch us up on what Engineering Productivity has been up to, how they might use some existing test features and what the new repeat failed test counter.

Testing group playlist: https://www.youtube.com/playlist?list=PL05JrBw4t0Kq53VUOvTk3VdXN79PA0SXT

Last ThinkBig discussion: https://www.youtube.com/watch?v=mV9jCk5Znhw&list=PL05JrBw4t0Kq53VUOvTk3VdXN79PA0SXT&index=3

3 participants
42 minutes

testing

coverage

monorepo

assess

2021

performance

reports

version

concerned

failures

22 Jan 2021

An update on progress since the last update in Oct 2020. Walk through some high level themes for the next 12 months for the Verify:Testing team specific to Web Performance Testing.

Oct 2020 update: https://www.youtube.com/watch?v=Mf6r_Infg8w&list=PL05JrBw4t0Kq53VUOvTk3VdXN79PA0SXT&index=12&t=2s

Web Performance Testing Direction: https://about.gitlab.com/direction/verify/web_performance/

1 participant
4 minutes

testing

gitlab

performance

dashboard

thinking

review

functionality

long

reminder

updates

22 Jan 2021

Walkthrough of the 13.10 planning issue: https://gitlab.com/gitlab-org/ci-cd/testing-group/-/issues/34

1 participant
4 minutes

functionality

refactoring

validation

milestone

quality

planning

deliverables

projects

repository

1310

11 Jan 2021

Today we thought small about what we could deliver to move us closer to the vision we discussed in the Think Big portion of the discussion last week. https://www.youtube.com/watch?v=ZAAzBViRhF4

The very next step will be to create an artifact from what is already created specific to the testing group and then socialize that with Engineering Managers to validate a similar solution with their data would be an outcome that's valuable.

5 participants
30 minutes

thinking

gitlab

testing

responsibility

prototyping

hypothesis

chunks

repo

discussion

concern

6 Jan 2021

Darren (Runner PM), James (Testing PM) and Rayana (Designer) talk about next major research initiatives on the last joint call.

4 participants
20 minutes

runners

workflow

discussion

planning

epic

initiative

tracking

prototyping

technical

enterprise

5 Jan 2021

Continuing our series of discussions of test coverage, code quality, etc. this week we talked about how to organize data a team may care about within a monorepo. This came out of a discussion we had with the Package Group who wanted to know test coverage for just their area of the monorepo.

4 participants
29 minutes

gitlab

monorepo

stakeholders

testing

repository

project

coverage

problems

monorego

monitoring

11 Dec 2020

Kickoff Page: https://about.gitlab.com/direction/kickoff/#verify

Ops Direction: https://about.gitlab.com/direction/ops/#verify

Code Quality Notices in MR Diffs Epic: https://gitlab.com/groups/gitlab-org/-/epics/4609

We love feedback and you can reach me at jheimbuck@gitlab.com or mention me on GitLab.com using @jheimbuck_gl

1 participant
2 minutes

testing

milestone

gitlab

functionality

reports

verified

quality

features

metrics

review

24 Nov 2020

Continuing our async refinement here's a walk through of the 13.8 planning issue as of November 24, 2020.

Issue: https://gitlab.com/gitlab-org/ci-cd/testing-group/-/issues/22

1 participant
5 minutes

capacity

docker

milestones

refinement

testing

maintenance

quality

1398

planning

deliverables

18 Nov 2020

Darren (Runner PM), James (Testing PM) and Rayana (Designer) do introductions and talk about current projects.

4 participants
29 minutes

onboarding

hi

collaboration

gitlab

interview

ci

experience

finishing

introduce

technical

9 Nov 2020

Today we think small about code coverage reports and what a minimum viable change would be to make it easier for team leads to identify code files with low test coverage within GitLab.

This is a follow-up of our Think Big #9: https://www.youtube.com/watch?v=dZLPqeWG-tw&list=PL05JrBw4t0Kq53VUOvTk3VdXN79PA0SXT&index=1

3 participants
35 minutes

gitlab

testing

coverage

workflow

simplistically

concerns

implementation

thinking

analyze

spaghetti

2 Nov 2020

Today we think big about code coverage reports, who uses them, getting data out of them for developer and team lead workflows and how we can provide a great experience for users leveraging multiple GitLab stages.

5 participants
29 minutes

coverage

testing

developer

codecov

review

dangerbot

monitoring

validation

annotation

workflow

29 Oct 2020

Today was an AMA style discussion.

We talked about the next epic the team will focus on (Code Quality: https://gitlab.com/groups/gitlab-org/-/epics/3686) and Kyle provided an update on the next steps Engineering Productivity is taking to run fewer tests.

Link to last months call: https://youtu.be/7Y8CCoeh-pQ

3 participants
25 minutes

milestones

testing

refresher

updates

report

workflow

dashboard

2020

reviewing

roadmap

27 Oct 2020

The Verify:Testing team is going to do more async refinement for the 13.7 milestone. This is a walk through of the planning issue on Oct. 27, 2020 to identify key deliverable issues and bugs, items that need refinement and next steps for the team. This is just a point in time and the issue board is always our Single Source of Truth: https://gitlab.com/groups/gitlab-org/-/issues?scope=all&utf8=%E2%9C%93&state=opened&label_name[]=group%3A%3Atesting&milestone_title=13.7

Planning Issue: https://gitlab.com/gitlab-org/ci-cd/testing-group/-/issues/20

1 participant
6 minutes

milestones

gitlab

capacity

dashboards

testing

tasks

performance

backend

priority

managed

13 Oct 2020

Walking through the themes for the next 12 months the Verify:Testing team specific to the Code Quality and Code Testing and Coverage categories.

Code Quality: https://about.gitlab.com/direction/verify/code_quality/
Code Testing and Coverage: https://about.gitlab.com/direction/verify/code_testing/

1 participant
6 minutes

gitlab

testing

maturity

features

developer

improving

problems

dashboard

review

updates

7 Oct 2020

Today Juan, Nadia, Lorrie and James reviewed the current state of the Research Projects the Testing group is doing and the needs for design on Code Quality visualization in the MR / Diff view.

5 participants
10 minutes

testing

tasks

validated

updates

surveys

concerns

2020

research

sync

verify

24 Sep 2020

Today we had a great update from Kyle on how their use of the the Fail Fast Template and the TFF gem (https://gitlab.com/gitlab-org/ci-cd/test_file_finder/). We also dug into the problems we are looking to solve for Code Quality to move the category to Viable (https://gitlab.com/groups/gitlab-org/-/epics/3686).

6 participants
38 minutes

validation

report

concerns

iterating

reviewing

workflows

testing

implementation

2020

standards

23 Sep 2020

Today we had an in depth discussion about the current state of the Find Failures Fast project and next steps to help Engineering get to their goal for Cost / pipeline (https://about.gitlab.com/handbook/engineering/quality/performance-indicators/#average-cost-per-merge-request-pipeline-for-gitlab) by running fewer tests in MR Pipelines. We also touched on the priority balance for Verify:Testing between Internal Customer needs and our Primary Performance Indicator of PGMAU (https://about.gitlab.com/handbook/product/ops-section-performance-indicators/#verifytesting---paid-gmau---count-of-active-paid-testing-feature-users).

5 participants
30 minutes

testing

junit

enhancements

important

gitlab

documentation

tams

milestones

functionality

reports

23 Sep 2020

Today we think big about Code Quality, who has a problem this category could solve and how we could differentiate from competitors.

4 participants
35 minutes

brainstorming

problem

contributions

code

discussion

quality

thinking

technical

critical

project

22 Sep 2020

In this session we talked about the 1 and 3 year vision for the group, problems we want to solve in that time and some early design ideas that will help as a north star towards the vision.

We then did a quick overview of our current epics to ensure they are contributing to that vision.

3 participants
25 minutes

testing

stakeholders

thinking

review

problems

trending

discussion

monitoring

small

validation

16 Sep 2020

Today Juan, Nadia and James talked about upcoming design issues in 13.5 and 13.6, current state of research and why James got yelled at by his daughter's teacher during the call.

3 participants
20 minutes

sync

anticipate

project

verify

brainstorming

taking

scheduled

concern

roadmaps

challenges

16 Sep 2020

Discussion between James (Product) and Juan (UX) about wireframes for 1 year vision for the Testing category and some of the problems that could be solved in 3 years alongside an early wireframe.

2 participants
29 minutes

idea

rethink

design

views

decisions

thinking

problems

ahead

discussion

project

14 Sep 2020

Quick walk through of the use case, setup and use of the Visual Reviews feature.

Visual Reviews documentation: https://docs.gitlab.com/ee/ci/review_apps/#visual-reviews

Review Apps Walk through: https://www.youtube.com/watch?v=h2pv_syqO24&t=515s

Direction Page for the Usability category: https://about.gitlab.com/direction/verify/usability_testing/

1 participant
5 minutes

gitlab

reviews

debugging

staging

testing

project

usability

documentation

app

toolbar

2 Sep 2020

Juan J. Ramirez shares designs for the vision for testing categories.

1 participant
16 minutes

testing

thinking

gitlab

initiatives

showing

tests

tested

test

benchmarking

planning

1 Sep 2020

Today we think small about Code Quality, what some simple fixes to the experience might be and pros/cons to displaying an opinionated score of what "quality" means.

The Think Big session: https://youtu.be/aJPp32Dhazk

5 participants
33 minutes

risky

deployments

thinking

quality

milestone

testing

codebase

actionable

tweak

maintainability

26 Aug 2020

Today Juan, Nadia and James talked about:

* Current Design items being done in 13.4 and the priority of them https://gitlab.com/groups/gitlab-org/-/issues?scope=all&utf8=%E2%9C%93&state=opened&label_name[]=group%3A%3Atesting&label_name[]=workflow%3A%3Adesign&milestone_title=13.4
* Which personas to target for the current research of JTBD for Cod Testing and Coverage - https://about.gitlab.com/handbook/engineering/development/ci-cd/verify/testing/JTBD/
* Ways to research low or declining usage of features - https://app.periscopedata.com/app/gitlab/633395/Testing-Category-Metrics

3 participants
25 minutes

testing

prioritizing

milestones

discussed

workflow

validating

worries

reviews

showing

taking

24 Aug 2020

Brainstorming session for JTBD for the Code Testing and Coverage category.

Issue for JTBD: https://gitlab.com/gitlab-org/ci-cd/testing-group/-/issues/19
Testing JTBD page: https://about.gitlab.com/handbook/engineering/development/ci-cd/verify/testing/JTBD/

5 participants
21 minutes

job

testing

brainstorming

process

project

doing

junit

working

useful

implement

12 Aug 2020

Today Juan, Nadia and James reviewed some of the current items in design in 13.3 and coming in 13.4 and clarified the expectations and feedback loop timing for the designs for Testing Vision and how that works into upcoming Vision work for the Verify stage. There was a long discussion about the current state of the JUnit Report page and all the improvements/data being added to the page and how to bring some order to those . . . . changes.

4 participants
19 minutes

validation

ahead

designing

concern

milestone

workflow

testing

2020

review

updates

5 Aug 2020

Today we discussed our current research and design projects, next steps for the Category Maturity Scorecard for the Code Testing and Coverage category and Juan and James talked about how the team can integrate the work being done by the Certify Group to capture tests and requirements for releases to extend the Test History MVC.

Test History Solution Validation: https://gitlab.com/gitlab-org/ux-research/-/issues/951
Code Coverage Reports Problem Validation: https://gitlab.com/gitlab-org/ux-research/-/issues/989

Quality Management MVC issue: https://gitlab.com/groups/gitlab-org/-/epics/3852

3 participants
13 minutes

milestones

validations

process

ux

testing

minimal

sync

verified

review

release

4 Aug 2020

This is a follow-up to our Think Big about Custom MR widgets; https://youtu.be/mFkPiVb-9PE.

Issues created/found from the sessions:

- Blog/video about how to use existing functionality to get data into an MR: https://gitlab.com/gitlab-org/gitlab/-/issues/233543
- Don't show Testing MR Widgets if green - https://gitlab.com/gitlab-org/gitlab/-/issues/233540
- Minimal Custom MR Widget - https://gitlab.com/gitlab-org/gitlab/-/issues/26533

4 participants
27 minutes

thinking

blocker

validation

small

dangerbot

widgets

problems

users

milestone

process

30 Jul 2020

An in depth discussion on the next steps for TestFileFinder and design for Test History.

Links:

https://docs.gitlab.com/ee/user/project/merge_requests/fail_fast_testing.html

Test History Design: https://gitlab.com/gitlab-org/gitlab/-/issues/223737/

6 participants
43 minutes

testing

slow

failures

performance

triage

reviewing

2020

reports

concern

process

28 Jul 2020

Today we think big about Custom MR Widgets, who they solve a problem for and how we could differentiate a solution to content on the MR page within GitLab.

6 participants
28 minutes

discussion

workflows

users

initiative

problems

merge

thinking

interface

suggestions

big

22 Jul 2020

In today's session Juan and James deep dive on why not to fix the confusing Visual Review Tools button right away (https://gitlab.com/gitlab-org/gitlab/-/issues/227049) to drive additional adoption of the feature. Hopefully.

3 participants
22 minutes

maintainer

testing

diesel

verified

issue

review

tooltip

users

developer

p80

14 Jul 2020

Today we think small about slow tests. What is the fastest way to test if users value knowing the test that is the slowest.

6 participants
28 minutes

testing

big

slow

thinking

exploratory

problematic

actionable

small

insights

clarify

8 Jul 2020

In today's session we talk about next steps for Test Execution History, 13.2 and 13.3 designs and Juan's progress on making the MR Widgets easier to understand. I'm looking at you "Source branch only".

5 participants
17 minutes

testing

discussion

planning

refinement

emily

milestone

ahead

workflow

review

project

7 Jul 2020

Today we think big about Test History. Who would use it, the ideal experience for them and how this could be a differentiator for GitLab.

5 participants
31 minutes

testing

think

functionality

discussion

tester

analyze

big

debugging

projects

tests

25 Jun 2020

A review of the epics in active development, check in on the recent release of the Identify Failures Fast template and talk about what testing features we should next focus on increasing internal use of through dog fooding.

Links:

- Identify Failures Fast Docs: https://docs.gitlab.com/ee/user/project/merge_requests/fail_fast_testing.html
- Identify Failures Fast Feedback Item: https://gitlab.com/gitlab-org/gitlab/-/issues/222706
- Accessibility testing documentation: https://docs.gitlab.com/ee/user/project/merge_requests/accessibility_testing.html#accessibility-testing

6 participants
28 minutes

review

concerns

report

testing

quality

actively

enhancements

actionable

coverage

capability

24 Jun 2020

In today's session we talk about the scorecard process to move "Code Quality" and "Code Testing and Coverage" categories from Minimal to Viable, the design work happening in 13.2 and 13.3 and how the team can help Lorie in interviews for the MR performance research.

3 participants
18 minutes

maturity

validating

discussed

viability

finalized

scorecard

testing

reviewing

tasks

forerunners

17 Jun 2020

Today's session was a discussion on how the MR beautification aligns with a Product Key Result, discussion on how to proceed with researching where/how to display test history and how Sahara sand is coming for Texas.

4 participants
14 minutes

prioritizing

enhancements

milestones

discussed

usability

product

merge

clarify

tasks

issue

5 Jun 2020

Today's session was a discussion on an MR Beautification project, an ask to create some mocks to "show" the vision for Code Testing and Coverage for an upcoming analyst review, using the Web Performance and Accessibility features for an internal epic (yeah dog-fooding!) and an update from Juan on some inflight design and implementation items.

Agenda: https://docs.google.com/document/d/1lBo6x1hoZigWL7kcS6uFo-ncxcu7SE8_WDUsewk4Kbc/edit#

MR Beautification Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/217568

4 participants
22 minutes

enhancements

testing

proposal

milestones

discussed

achieving

concerns

review

merge

process

29 May 2020

Today we talked about how we'll record the kickoff video for 13.2 and had a follow-up discussion on where the Accessibility category ended up from the maturity scorecard review.

4 participants
21 minutes

milestones

initiative

ux

product

addressing

kickoffs

stages

workflow

2020

managed

27 Mar 2020

No description provided.

4 participants
22 minutes

followup

initiative

research

task

currently

testing

participants

important

discussions

plans

29 Dec 2019

This is a demo to show how to configure a review app through the .gitlab-ci.yml file

1 participant
11 minutes

staging

lab

reviewing

configure

app

demo

repository

production

project

deploying

GitLab / Pipeline Insights Group

24 Jul 2023

24 Jul 2023

22 Jun 2023

5 Jun 2023

30 May 2023

30 Mar 2023

7 Feb 2023

31 Jan 2023

26 Jan 2023

10 Nov 2022

10 Nov 2022

2 Nov 2022

26 May 2022

14 Apr 2022

7 Feb 2022

19 Jan 2022

9 Nov 2021

28 Oct 2021

28 Oct 2021

26 Oct 2021

25 Oct 2021

21 Oct 2021

13 Oct 2021

7 Oct 2021

30 Sep 2021

30 Sep 2021

28 Sep 2021

23 Sep 2021

22 Sep 2021

15 Sep 2021

9 Sep 2021

2 Sep 2021

31 Aug 2021

18 Aug 2021

5 Aug 2021

3 Aug 2021

21 Jul 2021

15 Jul 2021

14 Jul 2021

8 Jul 2021

30 Jun 2021

21 Jun 2021

16 Jun 2021

8 Jun 2021

27 May 2021

21 May 2021

18 May 2021

10 May 2021

10 May 2021

29 Apr 2021

22 Apr 2021

20 Apr 2021

13 Apr 2021

12 Apr 2021

12 Apr 2021

18 Mar 2021

16 Mar 2021

16 Mar 2021

9 Mar 2021

9 Mar 2021

2 Mar 2021

2 Mar 2021

18 Feb 2021

16 Feb 2021

29 Jan 2021

29 Jan 2021

27 Jan 2021

22 Jan 2021

22 Jan 2021

11 Jan 2021

6 Jan 2021

5 Jan 2021

11 Dec 2020

24 Nov 2020

18 Nov 2020

9 Nov 2020

2 Nov 2020

29 Oct 2020

27 Oct 2020