Skip to content

CI collects coverage but enforces no threshold: _mstest-coverage.yml never converts to Cobertura or compares a floor #561

Description

@drmoisan

Summary

.github/workflows/_mstest-coverage.yml collects code coverage but never gates on it. The job runs
vstest.console.exe ... /EnableCodeCoverage and uploads the resulting .trx and .coverage
artifacts. It never converts the .coverage output to Cobertura and never compares any figure
against a floor.

Impact

A coverage regression cannot fail CI on main. The 80% Cobertura line-coverage gate reconciled by
#494 is enforced only by local tooling (Assert-CoberturaLineCoverageThreshold in
scripts/vscode/Invoke-MSTestWithCoverage.Helpers.ps1), which developers invoke through the VS Code
task. It is not one of the five required status checks in the main ruleset.

This is the same class of defect the build-ci-coverage-gate-fidelity epic set out to remove: a
gate that appears to measure something and does not. The epic corrected the arithmetic the gate
computes without wiring that gate into CI.

Evidence

  • .github/workflows/_mstest-coverage.yml lines 54-96: coverage is collected and uploaded; no
    threshold step exists.
  • main branch ruleset required contexts: actionlint / actionlint,
    format-check / Verify formatting, build-analyzers / Build with analyzers and code style enforcement,
    build-nullable / Build with nullable warnings treated as errors,
    mstest-coverage / Run MSTest suite with coverage. None asserts a coverage percentage.

Suggested resolution

Add a step to _mstest-coverage.yml that converts the .coverage artifact to Cobertura and invokes
the same threshold assertion the local tooling uses, so the CI gate and the developer gate enforce
one number. Resolve #TBD-threshold-contradiction first, so the number being enforced is settled.

Found during the build-ci-coverage-gate-fidelity epic fan-in review.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions