CI/CD reports are usually organized around executions. A JUnit file explains what happened in one run, and the next run produces another file.

Quality teams also need the inverse view:

What do we know about this test across its executions, sources, failures, and changes over time?

A Test Catalog turns imported run evidence into a persistent inventory without asking the team to maintain a second copy of its test code.

Keep source code and CI as the technical source of truth

QaCockpit does not require teams to recreate automated test cases manually. The first catalog record is derived from imported result evidence.

For JUnit, the technical identity can include:

  • Test Source;
  • exact testsuite.name;
  • testcase name;
  • class and file when available;
  • parameters;
  • an optional stable ID supplied by the framework or source-controlled metadata.

The exact testcase name remains visible. Suite, class, file, source, and technical key stay separate rather than being concatenated into an unreadable display name.

This model allows Playwright, pytest, Rest Assured, xUnit, and other frameworks to share one catalog when they produce structurally compatible JUnit XML. It does not pretend they are the same framework or silently reinterpret native report formats as JUnit.

Separate the latest result from long-term health

The latest result answers what happened most recently. A health label summarizes a wider pattern.

Useful distinctions include:

  • Passed versus Stable;
  • Failed versus Currently failing;
  • mixed recent evidence versus Flaky candidate;
  • two or more consecutive failures versus Repeated failure;
  • a test not observed for more than 30 days versus Inactive candidate;
  • no imported evidence versus No data.

QaCockpit presents technical JUnit Failed and Error results as one product-facing Failed outcome while preserving the raw status for diagnosis.

Candidate labels remain evidence signals. They are not automatic confirmation that a test is flaky, obsolete, or safe to ignore.

Make ownership gaps measurable

A large catalog often exposes an organizational problem before a testing problem: failed or unstable tests have no clear owner.

The catalog can show ownership coverage and allow permitted users to assign an owner to one or several tests. Project access is validated, and the change is audited.

Other useful enrichment includes component, team, criticality, tags, and suite metadata. Those fields should improve routing and analysis without blocking result import or campaign execution when they are absent.

Connect the test to scope and failures

A catalog row becomes more useful when it links to the evidence around it:

  • implementations in different Test Sources;
  • recent result history and pass rate;
  • average duration and last-seen time;
  • linked Failure Clusters;
  • exact execution results and artifacts;
  • active Test Scope and identity changes.

QaCockpit Test Scope comparison showing the active baseline and the exact tests added to the latest reported scope.

Persistent identity makes it possible to review scope changes as exact test sets rather than comparing only aggregate counts.

When a test has several implementations, a targeted rerun must choose the correct Test Source. If the underlying workflow has no safe selector capability, the product should explain why a single-test rerun is unavailable rather than forwarding an arbitrary command.

Use filters that reflect real maintenance work

A useful catalog should answer questions such as:

  • Which tests are currently failing?
  • Which failures are already classified as known issues?
  • Which tests have repeated-failure or flaky-candidate evidence?
  • Which tests have no owner?
  • Which suites or frameworks contribute the most current problems?
  • Which tests stopped appearing in imported results?

Project, environment, analysis window, suite, source, framework, status, health, owner, criticality, and tags create a practical operating scope without requiring a free-form analytics builder.

A catalog is not manual test management

QaCockpit’s catalog is automation-first. Test descriptions, steps, labels, and descriptive attachments should come from source-controlled metadata when available. Manually maintaining a second authoritative test specification inside the portal would create the same synchronization problem the catalog is meant to remove.

The product is most useful when a team has several automation sources and needs shared identity, history, ownership, and maintenance signals. A small suite that is already easy to understand directly in CI may not need this layer.

For the missing-scope side of the catalog, read Test scope drift: know when tests disappear from CI. If your automation inventory currently lives in several tools, tell us how it is organized.