A release dashboard should help a team decide what needs attention. It should not hide uncertainty behind a reassuring percentage or turn a collection of green CI jobs into an unqualified release approval.

The useful question is not simply, “Did the pipelines finish?” It is:

Do we have enough complete, current, and understood test evidence to make the next release decision?

Answering that question requires several signals that are related but not interchangeable.

Separate execution from test evidence

A provider workflow can complete while its report is missing. A report can arrive while part of the expected test scope is absent. Tests can fail even though the runner and workflow behaved correctly.

A dependable readiness view therefore keeps at least these dimensions separate:

  • Pipeline status: did the configured workflow or pipeline start and finish?
  • Result intake: did QaCockpit receive a usable report from the exact run?
  • Test outcome: what passed, failed, or was skipped in the reported scope?
  • Scope completeness: did the expected tests and required sources report?
  • Freshness: is the evidence recent enough for the current decision?
  • Failure understanding: are current failures new and unclassified, or already owned and understood?

Collapsing those dimensions into one green or red icon makes a dashboard simpler to draw, but less useful to operate.

Make the reason visible beside the status

High risk, Attention, Ready, Stale, Setup needed, and No data are useful only when the user can see why the label was selected.

A high-risk advisory might be supported by one or more exact facts:

  • a required source pipeline did not start;
  • the latest workflow completed but no result was imported;
  • current failed tests are not linked to triaged failure work;
  • a source reported only part of its established Test Scope;
  • the latest usable evidence is older than the selected analysis window.

The view should also expose the next action. Open executions, Review current failures, or Triage unreviewed is more operational than a score with no path to the evidence.

QaCockpit deliberately calls this a release readiness advisory. It organizes current evidence and attention signals; it does not claim to approve a release on behalf of the team.

Keep the organization view project-weighted

An organization may have one project with 10,000 tests and another with 100. If a portfolio metric simply adds test cases, the largest suite dominates the apparent health of the organization.

QaCockpit rolls up project states instead. This keeps a smaller but high-risk product visible rather than allowing a large healthy suite to dilute its signal.

QaCockpit Organization Cockpit showing portfolio health, project-level risk states, evidence gaps, and a Project Health Matrix with direct actions.

Project-weighted portfolio health keeps each product decision visible while preserving the exact reason and next action in the matrix.

The organization view is a navigation and prioritization layer. The project view remains the place where the user sees the exact executions, failures, completeness signals, and freshness behind the roll-up.

Use the campaign report as release evidence

Project readiness explains the current state. A Campaign Run Report provides an immutable view of one specific regression or release collection.

The report separates:

  • required source reports;
  • finalized campaign items;
  • imported reports with complete expected scope;
  • reported test outcomes;
  • awaiting, missing, or partial evidence;
  • per-source workflow, result-intake, and test-result state.

QaCockpit Campaign Run Report showing 3 of 3 required source reports, partial expected scope, failed tests, and a per-source evidence breakdown.

A completed collection can still require attention when one source reports partial scope or failed tests.

This distinction matters when a stakeholder asks why a release is considered risky. The answer should lead to an exact source, execution, missing test, failure cluster, or artifact—not a private spreadsheet formula.

When a dedicated readiness dashboard is unnecessary

One repository, one small workflow, and one team that already understands every result may not need another quality layer. A clear CI job summary and retained artifacts can be enough.

A dedicated readiness view becomes more valuable when:

  • releases depend on several repositories or test workflows;
  • results come from different frameworks;
  • someone repeatedly assembles the status by hand;
  • missing evidence is easy to mistake for passing evidence;
  • QA, engineering, product, and release stakeholders need the same conclusion at different levels of detail.

The goal is not a more colorful dashboard. It is a shorter path from a release question to evidence that the team can inspect and act on.

For a closer look at incomplete evidence, read A green pipeline is not enough. If your release status is still assembled manually, request a quality-landscape review.