Understand the evidence behind your score.

Include all projects in your chosen scope. Consider whether people must open each project or ask teams separately to build the summary.

0. We cannot reliably find the results needed for the decision. Level 0
1. We ask people and search across messages, files, or pipeline pages. Level 1
2. We use a known list of sources, then assemble the results for each decision. Level 2
3. We have a shared summary of the required sources, updated before the decision. Level 3
4. Our shared summary updates as results arrive and clearly shows its scope. Level 4

Required sources are the suites or pipelines you expect to review. Can someone move from the summary to the exact failed test and its available diagnostic evidence?

0. We have no usable automated test results. Level 0
1. We mostly see whether a pipeline passed or failed. Level 1
2. We can find individual test outcomes for some of the required sources. Level 2
3. We can find individual test outcomes for all required sources. Level 3
4. Those outcomes also link directly to the available failure details and diagnostic evidence. Level 4

Expected scope means the sources and tests you agreed should report. It is different from code coverage or requirements coverage.

0. We have no defined check for missing tests or reports. Level 0
1. Someone may notice an unusual count or a missing report by chance. Level 1
2. We compare the received results with a maintained checklist before the decision. Level 2
3. Automated checks flag missing sources or reports against a defined expected scope. Level 3
4. Those checks also flag unexpected reductions in test scope, with changes explicitly reviewed. Level 4

If no tests were skipped recently, answer for the process that would detect and handle them.

0. They are not distinguished from passed tests, or are left out of the summary. Level 0
1. They can be found in individual reports, but are not reviewed routinely. Level 1
2. They are visible in the release summary and reviewed before the decision. Level 2
3. Excluded tests also have recorded reasons and named owners. Level 3
4. Exclusions are time-limited, reviewed, and closed with evidence of restored testing. Level 4

Use an exact commit, artifact digest, package version, or equivalent identifier. A branch name alone can contain several versions.

0. We usually cannot link a result to a specific version. Level 0
1. We mostly rely on a branch name or whichever result is latest. Level 1
2. We can manually trace the exact run, version, and test environment. Level 2
3. These identifiers are included in the release summary for every required source. Level 3
4. A mismatch with the intended release version or environment is explicitly flagged. Level 4

Consider the period your team needs. Keeping evidence indefinitely is not required.

0. We do not keep a record of the evidence used. Level 0
1. We keep links to changing latest results or rely on people's memory. Level 1
2. We record the exact runs, but do not check how long their evidence remains available. Level 2
3. We retain the selected evidence for a defined period that meets our needs. Level 3
4. We also check that the retained evidence stays accessible and identifiable throughout that period. Level 4

A named team can be an owner. Consider whether the next action remains clear when work changes hands.

0. There is no clear person or team responsible for the next action. Level 0
1. We ask around in messages to find someone to investigate. Level 1
2. A person or team is assigned, but progress is tracked inconsistently. Level 2
3. The owner, current status, and next action are visible to the people involved. Level 3
4. Unresolved failures are also reviewed against an agreed response time and escalation path. Level 4

Answer for your agreed practice even if this has not happened recently. If reruns do not apply, choose Not applicable.

0. The passing rerun replaces or hides the original failure. Level 0
1. Both attempts remain available, but we usually move on without review. Level 1
2. We review both attempts before deciding whether the failure is resolved. Level 2
3. Repeated instability also gets a tracked issue and owner. Level 3
4. Unstable tests have reviewed, time-limited handling and evidence is required to close the issue. Level 4

Consider the written criteria, not just whether a pipeline has a green status.

0. There are no agreed criteria. Level 0
1. The criteria depend mainly on who is available or how urgent the release is. Level 1
2. We use a checklist, but it leaves required evidence or exceptions unclear. Level 2
3. Criteria define the required evidence, blockers, and who can approve an exception. Level 3
4. We also review and update those criteria using findings from previous releases. Level 4

A checkpoint can be an enforced pipeline rule or a required, recorded human review.

0. Release can proceed without a defined check or recorded exception. Level 0
1. Someone may stop the release, but the check is informal. Level 1
2. A required checkpoint flags the problem, but exceptions are not consistently recorded. Level 2
3. Release is held until criteria are met or an authorized exception is recorded. Level 3
4. The checkpoint also catches missing or outdated evidence, and exceptions have follow-up owners. Level 4