From 22837bef7f2817afd14782711218193a96adba91 Mon Sep 17 00:00:00 2001 From: fadidurah Date: Sun, 2 Aug 2026 21:49:09 -0400 Subject: [PATCH] Add code-coverage-onboarding skill Portable skill to set up end-to-end code-coverage tracking and to raise coverage. Core is scripts/coverage_report.py (stdlib-only) with four subcommands: parse (JaCoCo/Cobertura/LCOV -> normalized NDJSON), report (NDJSON -> Markdown), gaps (ranked worklist of least-covered classes/files from a coverage report, for closing coverage gaps), and wow (Kusto -> week-over-week HTML trend). Includes references for coverage generation, Kusto setup, pipeline integration, report/email integration, and the find-gaps -> write-tests -> gate loop for increasing coverage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6bbcda86-3ea3-48c4-80da-45481bc4967a --- .../skills/code-coverage-onboarding/SKILL.md | 106 ++ .../references/coverage-generation.md | 134 ++ .../references/increasing-coverage.md | 98 ++ .../references/kusto-setup.md | 112 ++ .../references/pipeline-integration.md | 139 ++ .../references/report-integration.md | 88 ++ .../scripts/coverage_report.py | 1168 +++++++++++++++++ 7 files changed, 1845 insertions(+) create mode 100644 .github/skills/code-coverage-onboarding/SKILL.md create mode 100644 .github/skills/code-coverage-onboarding/references/coverage-generation.md create mode 100644 .github/skills/code-coverage-onboarding/references/increasing-coverage.md create mode 100644 .github/skills/code-coverage-onboarding/references/kusto-setup.md create mode 100644 .github/skills/code-coverage-onboarding/references/pipeline-integration.md create mode 100644 .github/skills/code-coverage-onboarding/references/report-integration.md create mode 100644 .github/skills/code-coverage-onboarding/scripts/coverage_report.py diff --git a/.github/skills/code-coverage-onboarding/SKILL.md b/.github/skills/code-coverage-onboarding/SKILL.md new file mode 100644 index 00000000..51eb31b2 --- /dev/null +++ b/.github/skills/code-coverage-onboarding/SKILL.md @@ -0,0 +1,106 @@ +--- +name: code-coverage-onboarding +description: >- + Onboard a team/repo to automated code-coverage tracking and reporting. Use when a user wants + to set up code coverage measurement, add coverage to CI/CD, track coverage over time + (week-over-week), ingest coverage into Kusto/Azure Data Explorer, generate a coverage report + or email, wire coverage into a weekly/scheduled pipeline, port an existing coverage setup to + another team, or raise/increase coverage. Handles JaCoCo (Gradle/Maven/Android), + Cobertura (.NET/Python/JS/Go), and LCOV (JS/TS/C++) reports, Azure DevOps and GitHub Actions + pipelines, Kusto ingestion with cost control, report/email integration (including Azure + Communication Services), and ranking uncovered classes/files to boost coverage. Triggers + include "set up code coverage", "add coverage tracking", "coverage + report", "coverage in the pipeline", "track coverage in Kusto", "onboard to code coverage", + "weekly coverage email", "increase/raise code coverage", "where should I add tests", + "code coverage skill". +--- + +# Code Coverage Onboarding + +Set up end-to-end code-coverage tracking for a repo: generate coverage → parse to a normalized +schema → publish a summary → (scheduled) ingest into Kusto → surface a week-over-week trend in +a report/email. Portable across CI systems and languages. The core is `scripts/coverage_report.py` +plus a Kusto table; the pipeline wiring is adaptable to Azure DevOps or GitHub Actions. + +## Bundled resources +- `scripts/coverage_report.py` — the portable engine. Four subcommands: `parse` (coverage + report → normalized NDJSON), `report` (NDJSON → Markdown summary), `gaps` (report → ranked + worklist of the least-covered classes/files, for raising coverage), `wow` (Kusto → HTML trend + fragment with grouped tables + Overall rows). Parses JaCoCo XML, Cobertura XML, and LCOV. + Stdlib only; runs on any image with Python 3. +- `references/coverage-generation.md` — make the build emit JaCoCo/Cobertura/LCOV (Gradle, + Android AGP, Maven, .NET, Python, Node, C/C++, Go). +- `references/kusto-setup.md` — table schema, WIF/MI auth, ingestor grants, inline ingest, and + the Sunday-only cost-control gate. +- `references/pipeline-integration.md` — Azure DevOps and GitHub Actions patterns, plus how to + create a scheduled build if none exists. +- `references/report-integration.md` — inject the trend into an existing report, a Kusto + dashboard, or a new ACS email; grouped-table configuration. +- `references/increasing-coverage.md` — the find-gaps → write-tests → gate loop for *raising* + coverage. Read when the user wants to boost numbers, not just track them. + +## Workflow + +This skill has five capabilities. Confirm scope with the user, then implement in order. Steps +1–4 are the tracking pipeline; step 5 (raising coverage) is optional and builds on them. + +### Step 0 — Scope the setup +Ask (only what's not already clear): +- **Build tool & language** → determines report format (JaCoCo / Cobertura / LCOV). +- **CI system** (Azure DevOps / GitHub Actions / other). +- **Is there a recurring/scheduled build?** If not, one is needed (see step 2). +- **Track history in Kusto?** Default **yes**; a team can opt for summary-artifact-only. +- **Existing reporting** to inject into, or start fresh? + +### Step 1 — Generate coverage +Get the build to emit JaCoCo, Cobertura, or LCOV output. See `references/coverage-generation.md`. +**Reuse an existing test run** rather than adding a second one — re-running tests just for +coverage doubles CI time. Gate coverage behind a flag so normal builds aren't slowed. + +### Step 2 — Parse & summarize in CI +Add steps that run `coverage_report.py parse` (per module → shared NDJSON) then `report` +(NDJSON → Markdown summary), and publish both as an artifact. These run every build and are +**non-fatal** (warn, don't fail). See `references/pipeline-integration.md`. If the team has no +scheduled build, create a minimal weekly one whose job is: test + parse + ingest. + +### Step 3 — Ingest into Kusto (default on, cost-gated) +Create the `CodeCoverageData` table, grant the CI identity Ingestor, and add an ingest step +that converts NDJSON → CSV and POSTs an inline `.ingest`. **Only ingest on the scheduled +reporting run** (e.g. Sunday) with an operator override parameter — see the gate in +`references/kusto-setup.md`. Ingestion **should fail loudly** if it can't write (unlike the +reporting steps). Skip this whole step only if the team declined history. + +### Step 4 — Surface the trend +Render the week-over-week trend with `coverage_report.py wow` and get it in front of the team. +Priority: (1) inject the HTML fragment into an existing recurring report, (2) a Kusto +dashboard, (3) a new ACS email. See `references/report-integration.md`. Use `--group-file` to +split modules into labeled tables, each with an Overall row. + +### Step 5 — Increasing coverage (integrated, optional) +When the user wants to *raise* coverage (not just track it), run the find-gaps loop: use +`coverage_report.py gaps` on the latest report to rank the least-covered classes/files, write +tests for the highest-value targets, re-run to confirm, then lock gains in with a no-regression +gate. See `references/increasing-coverage.md`. This is a separate effort from steps 1–4 — do it +once a baseline is visible so progress can be verified. + +## Key conventions (carry these when porting) +- **Normalized schema** is fixed: `Date, Repo, Module, Metric, Covered, Missed, Percentage, + CommitId, BuildId, Branch`. The KQL and ingest CSV depend on these names. +- **Reporting is non-fatal; ingestion is fatal.** The report/email must still ship on a + coverage hiccup, but a silent ingest failure that greens the run is a bug. +- **WoW uses calendar semantics**, not a rolling 7-day window: the baseline is last week's + start-of-week run (`startofweek()`), so same-week manual re-runs don't skew the delta. +- **Overall rows sum Covered/Total** and derive the delta from summed prior Covered/Missed — + never average per-module percentages. +- **Ingest on a schedule only** to control Kusto cost; keep cheap parse/publish per build. + +## Validating the script +`coverage_report.py` is stdlib-only. Sanity-check after any edit: +```bash +python3 -m py_compile scripts/coverage_report.py +python3 scripts/coverage_report.py parse --input sample.xml --repo r --module m --out rows.ndjson +python3 scripts/coverage_report.py report --input rows.ndjson --metric LINE --goal 75 +python3 scripts/coverage_report.py gaps --input sample.xml --top 10 +``` +The `wow` subcommand needs a live Kusto table + token; test rendering logic with mock rows if +no cluster is available. diff --git a/.github/skills/code-coverage-onboarding/references/coverage-generation.md b/.github/skills/code-coverage-onboarding/references/coverage-generation.md new file mode 100644 index 00000000..7933a7f0 --- /dev/null +++ b/.github/skills/code-coverage-onboarding/references/coverage-generation.md @@ -0,0 +1,134 @@ +# Coverage Report Generation + +How to make a build emit a machine-readable coverage report that +`scripts/coverage_report.py` can parse. The parser auto-detects **JaCoCo XML**, +**Cobertura XML**, and **LCOV** tracefiles, so the only job here is to get the build +tool to produce one of those. + +## Table of contents +- [Which format do I have?](#which-format-do-i-have) +- [Gradle + JaCoCo (Java/Kotlin/Android)](#gradle--jacoco-javakotlinandroid) +- [Android Gradle Plugin specifics](#android-gradle-plugin-specifics) +- [Maven + JaCoCo](#maven--jacoco) +- [.NET (Coverlet / VSTest → Cobertura)](#net-coverlet--vstest--cobertura) +- [Python (coverage.py → Cobertura)](#python-coveragepy--cobertura) +- [Node/JS (nyc/jest → Cobertura or LCOV)](#nodejs-nycjest--cobertura-or-lcov) +- [C/C++ (lcov / gcovr)](#cc-lcov--gcovr) +- [Go (gocover-cobertura)](#go-gocover-cobertura) +- [Finding the report in CI](#finding-the-report-in-ci) + +## Which format do I have? +- Root element `` → **JaCoCo** (`--format jacoco`, or let auto-detect handle it). +- Root element `` → **Cobertura** (`--format cobertura`). +- Text lines like `SF:`, `DA:`, `LF:`, `end_of_record` (usually `lcov.info` / `*.info`) → + **LCOV** (`--format lcov`). + +The parser reads the top-level aggregate `` totals (JaCoCo), the +`lines-covered`/`lines-valid`/`branches-*` attributes (Cobertura), or the summed +`LF`/`LH` + `BRF`/`BRH` records (LCOV; falls back to counting `DA:` records when the +summaries are absent). It does **not** need per-class/per-file detail, so any +correctly-formed report works. + +## Gradle + JaCoCo (Java/Kotlin/Android) +Apply the plugin and ensure a report task emits XML: + +```groovy +plugins { id 'jacoco' } +jacoco { toolVersion = "0.8.10" } + +tasks.named('jacocoTestReport') { + dependsOn test // or the flavor-specific unit test task + reports { + xml.required = true // REQUIRED - this is what the parser reads + html.required = true // optional, for humans + } +} +``` + +Run: `./gradlew jacocoTestReport` → XML at +`build/reports/jacoco/jacocoTestReport/jacocoTestReport.xml`. + +**Gate coverage behind a flag** so normal builds aren't slowed down: +```groovy +def enableCodeCoverage = project.hasProperty("codeCoverageEnabled") + ? codeCoverageEnabled.toBoolean() : false +tasks.withType(Test) { jacoco { enabled = enableCodeCoverage } } +``` +Then in CI: `./gradlew jacocoTestReport -PcodeCoverageEnabled=true`. + +**Reuse existing test runs.** If the build already runs a coverage task (many Android +setups have `UnitTestCoverageReport`), do NOT add a second test run — point the +parser at the XML that task already produces. Re-running tests just to get coverage +doubles CI time. + +## Android Gradle Plugin specifics +- AGP unit-test coverage is per **build variant**: `testDebugUnitTest` → JaCoCo exec → + `createUnitTestCoverageReport`. Use the variant your CI actually builds + (e.g. `dist`/`release`), not `localDebug`, or the `.exec` won't exist and the report + will be empty. +- `includeNoLocationClasses = true` is required for **Robolectric** tests. +- Modules with no `src/test` sources produce no report — skip them (don't fail the build). +- Robolectric/instrumented mixes: only unit-test coverage flows through JaCoCo XML here; + instrumented (Espresso) coverage needs a connected device and is out of scope for the + weekly trend. + +## Maven + JaCoCo +```xml + + org.jacoco + jacoco-maven-plugin + 0.8.10 + + prepare-agent + reporttestreport + + +``` +Run `mvn test` → XML at `target/site/jacoco/jacoco.xml`. + +## .NET (Coverlet / VSTest → Cobertura) +```bash +dotnet test --collect:"XPlat Code Coverage" -- \ + DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura +``` +Emits `**/TestResults/**/coverage.cobertura.xml`. Parse with `--format cobertura`. + +## Python (coverage.py → Cobertura) +```bash +coverage run -m pytest +coverage xml # -> coverage.xml (Cobertura) +``` + +## Node/JS (nyc/jest → Cobertura or LCOV) +Either format works — the parser reads both natively. +```bash +# jest — Cobertura +jest --coverage --coverageReporters=cobertura # -> coverage/cobertura-coverage.xml +# jest — LCOV (often already the default) +jest --coverage --coverageReporters=lcov # -> coverage/lcov.info +# nyc +nyc --reporter=cobertura npm test # or --reporter=lcovonly -> coverage/lcov.info +``` + +## C/C++ (lcov / gcovr) +```bash +# lcov -> lcov.info (parse with --format lcov / auto-detect) +lcov --capture --directory . --output-file coverage.info +# or gcovr -> Cobertura +gcovr --cobertura -o coverage.xml +``` + +## Go (gocover-cobertura) +```bash +go test -coverprofile=cover.out ./... +go run github.com/boumenot/gocover-cobertura < cover.out > coverage.xml +``` + +## Finding the report in CI +When report paths vary per module, locate JaCoCo reports by content rather than a fixed path: +```bash +grep -rlZ --include='*.xml' 'JACOCO//DTD' "$SOURCES_DIR" | tr '\0' '\n' +``` +For Cobertura, match on the ` +## Finding gaps with `coverage_report.py gaps` +The `gaps` subcommand turns a raw coverage report into a prioritized, per-class/per-file +worklist. It reads the **same** JaCoCo / Cobertura / LCOV reports the build already produces — +no extra tooling. + +```bash +# Biggest absolute wins first (default): classes with the most uncovered lines. +python3 coverage_report.py gaps --input '**/build/reports/**/*.xml' --metric LINE --top 25 + +# Lowest-coverage classes first, ignoring anything already >60% or tiny (<5 missed lines). +python3 coverage_report.py gaps --input coverage.cobertura.xml \ + --sort pct --max-pct 60 --min-missed 5 + +# Branch gaps for a JS module from LCOV, saved as a checklist artifact. +python3 coverage_report.py gaps --input coverage/lcov.info --metric BRANCH \ + --out-md gaps.md --out-json gaps.json +``` + +Key flags: +- `--metric LINE|BRANCH` — line gaps are the usual target; branch gaps expose untested + conditionals/error paths. +- `--sort missed` (default) ranks by **most uncovered units** = biggest number bump per test + written. `--sort pct` ranks by **lowest coverage** = worst-tested code first. +- `--max-pct N` hides units already well covered; `--min-missed N` hides trivial ones. Together + they focus attention on "big and poorly covered". +- `--top N` caps the list (0 = all). `--out-md` / `--out-json` persist it as a work artifact. + +Format is auto-detected; pass `--format jacoco|cobertura|lcov` to force it. The output columns +are `Class/File | Coverage | Missed | Covered/Total`. + + +## Choosing targets +The ranked list tells you *where the uncovered code is*; combine it with judgment on *what is +worth covering*: +- **Prefer big + low-coverage + high-churn** modules — best return on effort. Cross-reference + the `gaps` list against `git log`/churn if available. +- **Cover behavior that matters**: core logic, error/exception paths, boundary conditions, + regression tests for recently-fixed bugs, and any newly-added code. +- **Skip / exclude the denominator noise**: generated code, DTOs/`data class`es, `toString`, + builders, and test code. Exclude these in the build tool so the number reflects meaningful + coverage instead of chasing 100% on trivial code. +- Use `--sort pct` for the "worst offenders" view when you want to eliminate near-zero classes, + and `--sort missed` when you want the fastest overall percentage gain. + + +## Writing tests that count +- Assert on observable outcomes (return values, state changes, thrown exceptions, emitted + events) — never write tests that execute code without asserting just to move the number. +- One behavior per test; name tests for the behavior, not the method. +- For each `gaps` target, open the class and cover its untested branches first (constructors and + simple getters are low value even if uncovered). +- Re-run tests + `gaps` after each batch so progress is visible and you don't over-invest in one + class. + + +## Locking in gains with a gate +Prevent backsliding once a module improves. **Prefer a no-regression gate over a fixed absolute +threshold** — it's fair to modules that start low and doesn't block unrelated PRs. +- Compare a PR's coverage to the baseline/dev branch; **fail only if it *lowers* coverage**. +- Make the gate flip via a variable (e.g. `ENFORCE_COVERAGE_GATE`) between report-only and + enforcing, with an emergency off-switch. +- Introduce it gradually: report-only first, then enforce once numbers are stable. +- Exclude generated/test/DTO code from the denominator where the build tool supports it. + + +## Anti-patterns +- Tests that assert nothing (executing code just to raise the number). +- Chasing a global percentage instead of covering risky code paths. +- Hard-gating an absolute threshold on day one — blocks unrelated work and breeds resentment. +- Padding coverage with generated/DTO code left in the denominator. diff --git a/.github/skills/code-coverage-onboarding/references/kusto-setup.md b/.github/skills/code-coverage-onboarding/references/kusto-setup.md new file mode 100644 index 00000000..03ffb691 --- /dev/null +++ b/.github/skills/code-coverage-onboarding/references/kusto-setup.md @@ -0,0 +1,112 @@ +# Kusto Ingestion & Long-Term Tracking + +Store normalized coverage rows in a Kusto (Azure Data Explorer) table so coverage can be +tracked week-over-week. This is **on by default** in the recommended setup, but is optional — +a team that only wants the per-run summary artifact can skip it. + +## Table of contents +- [Table schema](#table-schema) +- [Creating the table](#creating-the-table) +- [Authentication (WIF / Managed Identity)](#authentication-wif--managed-identity) +- [Granting ingestor permission](#granting-ingestor-permission) +- [Ingesting rows](#ingesting-rows) +- [Cost control: only ingest on a schedule](#cost-control-only-ingest-on-a-schedule) +- [Querying the data](#querying-the-data) + +## Table schema +One row per module + metric + run. Matches the NDJSON emitted by `coverage_report.py parse`: + +```kusto +.create table CodeCoverageData ( + Date: datetime, // run date (UTC) + Repo: string, // owning repository + Module: string, // module / project name + Metric: string, // LINE or BRANCH + Covered: long, // covered units + Missed: long, // missed units + Percentage: real, // Covered / (Covered + Missed) * 100 + CommitId: string, + BuildId: string, + Branch: string +) +``` + +Keep the column names exactly as above — the `wow` KQL and the ingest CSV ordering depend on them. + +## Creating the table +Run the `.create table` command above in the target database (via the Kusto MCP +`kusto_command`, the Azure Data Explorer web UI, or `az kusto`). Pick a database your team +already owns; a shared telemetry cluster is fine. + +## Authentication (WIF / Managed Identity) +CI should authenticate with a **workload-identity federation (WIF) service connection** or a +managed identity — never a stored secret. In Azure DevOps this is an `AzureCLI@2` task with an +`azureSubscription` that points at a WIF service connection; `az account get-access-token` +then yields a bearer token scoped to Kusto. + +```bash +token=$(az account get-access-token --resource "https://kusto.kusto.windows.net" \ + --query accessToken -o tsv) +``` + +In GitHub Actions, use `azure/login@v2` with OIDC (`permissions: id-token: write`) and the +same `az account get-access-token` call. + +## Granting ingestor permission +The identity behind the service connection needs **Database Ingestor** (and **Viewer** to +query) on the target database. Run once, as a database admin: + +```kusto +.add database ingestors ('aadapp=;') 'Code coverage ingest' +.add database viewers ('aadapp=;') 'Code coverage read' +``` + +If ingestion returns 403/Forbidden, this grant is almost always the missing step. + +## Ingesting rows +Convert NDJSON → CSV (fixed column order) and POST an inline `.ingest` management command: + +```bash +csv=$(python3 -c 'import json,sys; cols=["Date","Repo","Module","Metric","Covered","Missed","Percentage","CommitId","BuildId","Branch"]; print("\n".join(",".join(str(json.loads(l)[c]) for c in cols) for l in open(sys.argv[1]) if l.strip()))' coverage-all.ndjson) +token=$(az account get-access-token --resource "https://kusto.kusto.windows.net" --query accessToken -o tsv) +body=$(python3 -c 'import json,sys; print(json.dumps({"db":sys.argv[1],"csl":".ingest inline into table "+sys.argv[2]+" <|\n"+sys.argv[3]}))' "" "CodeCoverageData" "$csv") +curl -sS -X POST "https://.kusto.windows.net/v1/rest/mgmt" \ + -H "Authorization: Bearer $token" -H "Content-Type: application/json" -d "$body" +``` + +Inline ingest is fine for the small row counts here (a handful of modules per run). For large +volumes use queued ingestion instead. + +**Fail the stage on ingest failure** if the data matters for tracking — don't mark the run +green when nothing was ingested. (Reporting/rendering steps, by contrast, should be non-fatal +so the email still sends.) + +## Cost control: only ingest on a schedule +Ingesting on every CI run wastes storage and money. Gate ingestion so it only runs on the +**scheduled** reporting cadence (e.g. weekly on Sunday), with an operator override for manual +validation. Since a cron may fire daily, check the day at runtime: + +```bash +FORCE="${FORCE_INGEST:-false}" # pipeline parameter, default false +if [ "$FORCE" != "true" ]; then + if [ "$BUILD_REASON" != "Schedule" ] || [ "$(date -u +%u)" != "7" ]; then + echo "Skipping Kusto ingestion (not a Sunday scheduled run). Set FORCE_INGEST=true to override." + exit 0 + fi +fi +# ... proceed with ingest ... +``` +`date -u +%u` returns 1=Mon … 7=Sun (UTC). Adjust to your reporting day. The parse/publish +steps can still run every build (they're cheap and give per-PR visibility); only the **ingest** +is gated. + +## Querying the data +Latest coverage per module: +```kusto +CodeCoverageData +| where Metric == "LINE" +| summarize arg_max(Date, Percentage, Covered, Missed) by Repo, Module +| order by Repo asc, Module asc +``` +The `wow` subcommand runs a richer version of this (see `scripts/coverage_report.py` +`_wow_kql`) that also computes the week-over-week delta with calendar (start-of-week) semantics. diff --git a/.github/skills/code-coverage-onboarding/references/pipeline-integration.md b/.github/skills/code-coverage-onboarding/references/pipeline-integration.md new file mode 100644 index 00000000..cf6efc9d --- /dev/null +++ b/.github/skills/code-coverage-onboarding/references/pipeline-integration.md @@ -0,0 +1,139 @@ +# CI/CD Pipeline Integration + +Wire coverage generation, parsing, and (scheduled) Kusto ingestion into a pipeline. The +coverage logic is CI-agnostic — anything that can run tests and execute a Python script works. +This file gives concrete patterns for **Azure DevOps** and **GitHub Actions**. + +## Table of contents +- [The four steps](#the-four-steps) +- [If no scheduled build exists yet](#if-no-scheduled-build-exists-yet) +- [Azure DevOps](#azure-devops) +- [GitHub Actions](#github-actions) +- [Multi-module / multi-repo aggregation](#multi-module--multi-repo-aggregation) + +## The four steps +Every integration is the same shape, regardless of CI system: +1. **Run tests with coverage** so the build emits JaCoCo/Cobertura XML (see + `references/coverage-generation.md`). Prefer reusing an existing test run. +2. **Parse** each XML → append to a shared NDJSON: `coverage_report.py parse ...`. +3. **Publish** the NDJSON + a Markdown summary as a build artifact (cheap; every run). +4. **Ingest** the NDJSON into Kusto — **only on the scheduled reporting run** (see the + cost-control gate in `references/kusto-setup.md`). + +Reporting/parse steps should be **non-fatal** (warn, don't fail) so a coverage hiccup never +breaks the build. Ingestion, when it runs, **should** fail loudly if it can't write the data. + +## If no scheduled build exists yet +The trend needs a recurring run that executes the unit tests. If the team has no weekly/nightly +pipeline, create a minimal scheduled one whose only job is: build + run unit tests with +coverage + parse + ingest. Reuse it as the home for the coverage trend. + +- **Azure DevOps**: add a `schedules:` block (cron) to a YAML pipeline, or configure the + schedule in the pipeline's UI (Edit → Triggers) — UI schedules can't pass parameters, so + rely on the runtime day/`Build.Reason` gate for Sunday-only ingestion. +- **GitHub Actions**: add `on: schedule: - cron:` to the workflow. + +Pick a low-traffic time (e.g. Sunday early UTC) so the scheduled coverage run doesn't compete +with weekday CI. + +## Azure DevOps +Minimal per-module steps (bash), assuming the coverage XML already exists: + +```yaml +- bash: | + nd="$(Build.ArtifactStagingDirectory)/coverage-all.ndjson" + xml=$(find "$(Build.SourcesDirectory)" -type f -name '*.xml' | xargs grep -l 'JACOCO//DTD' | head -1) + python3 scripts/coverage_report.py parse --input "$xml" --format jacoco \ + --repo "$(Build.Repository.Name)" --module "mymodule" \ + --branch "$(Build.SourceBranchName)" --commit "$(Build.SourceVersion)" \ + --build-id "$(Build.BuildNumber)" --out "$nd" + displayName: Parse coverage + continueOnError: true + +- bash: | + python3 scripts/coverage_report.py report \ + --input "$(Build.ArtifactStagingDirectory)/coverage-all.ndjson" \ + --metric LINE --goal 75 \ + --out-md "$(Build.ArtifactStagingDirectory)/coverage-summary.md" + echo "##vso[task.uploadsummary]$(Build.ArtifactStagingDirectory)/coverage-summary.md" + displayName: Build coverage summary + continueOnError: true + +- publish: $(Build.ArtifactStagingDirectory) + artifact: coverage-report +``` + +Scheduled, cost-gated ingest (add a `forceIngestCoverage` boolean pipeline parameter, +default false): + +```yaml +- task: AzureCLI@2 + displayName: Ingest coverage into Kusto + inputs: + azureSubscription: '' + scriptType: bash + scriptLocation: inlineScript + inlineScript: | + FORCE="${{ parameters.forceIngestCoverage }}" + if [ "$FORCE" != "True" ] && [ "$FORCE" != "true" ]; then + if [ "$(Build.Reason)" != "Schedule" ] || [ "$(date -u +%u)" != "7" ]; then + echo "Skipping ingest (not a Sunday scheduled run)."; exit 0 + fi + fi + # ... NDJSON -> CSV -> POST .ingest (see references/kusto-setup.md) ... +``` + +## GitHub Actions +```yaml +on: + schedule: + - cron: '0 6 * * 0' # 06:00 UTC every Sunday + workflow_dispatch: + inputs: + forceIngestCoverage: + type: boolean + default: false + +permissions: + id-token: write # for azure/login OIDC + contents: read + +jobs: + coverage: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: { python-version: '3.x' } + - name: Run tests with coverage + run: ./gradlew jacocoTestReport -PcodeCoverageEnabled=true + - name: Parse coverage + run: | + xml=$(find . -name '*.xml' | xargs grep -l 'JACOCO//DTD' | head -1) + python3 scripts/coverage_report.py parse --input "$xml" --format jacoco \ + --repo "${{ github.repository }}" --module mymodule \ + --branch "${{ github.ref_name }}" --commit "${{ github.sha }}" \ + --build-id "${{ github.run_number }}" --out coverage-all.ndjson + - name: Build summary + run: | + python3 scripts/coverage_report.py report --input coverage-all.ndjson \ + --metric LINE --goal 75 --out-md coverage-summary.md + cat coverage-summary.md >> "$GITHUB_STEP_SUMMARY" + - uses: actions/upload-artifact@v4 + with: { name: coverage-report, path: coverage-all.ndjson } + - name: Ingest into Kusto (scheduled/forced only) + if: github.event_name == 'schedule' || inputs.forceIngestCoverage + run: | + # azure/login (OIDC) then NDJSON -> CSV -> POST .ingest + # see references/kusto-setup.md +``` +GitHub `on: schedule` only fires Sunday here, so the `if:` is enough — no runtime day check +needed (unlike a daily ADO cron). + +## Multi-module / multi-repo aggregation +- Append every module's rows to **one** NDJSON, then `report`/ingest once. Use `--repo` / + `--module` per parse call so rows stay attributable. +- When each module publishes its own coverage artifact, download them all and locate each + module's report by exact filename (handle re-runs by taking the newest by timestamp). +- Keep the coverage-artifact publish **gated by a variable** so only the pipelines that opt in + (weekly/nightly) publish — release/hotfix pipelines shouldn't pay the cost. diff --git a/.github/skills/code-coverage-onboarding/references/report-integration.md b/.github/skills/code-coverage-onboarding/references/report-integration.md new file mode 100644 index 00000000..8ef09d20 --- /dev/null +++ b/.github/skills/code-coverage-onboarding/references/report-integration.md @@ -0,0 +1,88 @@ +# Report Integration + +How to surface the coverage trend to the team. Three cases, in priority order: +1. The team already has a recurring report → **inject** the coverage section into it. +2. The team has a dashboard tool → point it at the Kusto table. +3. The team has nothing → **suggest the email approach** below. + +## Table of contents +- [The `wow` fragment](#the-wow-fragment) +- [Case 1: Inject into an existing report](#case-1-inject-into-an-existing-report) +- [Case 2: Dashboard on Kusto](#case-2-dashboard-on-kusto) +- [Case 3: Email via Azure Communication Services](#case-3-email-via-azure-communication-services) +- [Splitting into grouped tables](#splitting-into-grouped-tables) + +## The `wow` fragment +`coverage_report.py wow` queries the Kusto table and prints an **HTML fragment** (a heading + +one or more ``s, each with an Overall row and week-over-week deltas). It writes the same +HTML to `--out-html` when given. It is intentionally self-contained HTML so it can be appended +to almost any report body. + +```bash +export KUSTO_TOKEN=$(az account get-access-token --resource https://kusto.kusto.windows.net --query accessToken -o tsv) +python3 scripts/coverage_report.py wow \ + --cluster "https://.kusto.windows.net" \ + --db "" \ + --table CodeCoverageData \ + --out-html coverage-wow.html +``` + +The command is **non-fatal**: if the token is missing or the query fails, it emits a short +"coverage unavailable" note instead of failing, so the surrounding report still ships. + +## Case 1: Inject into an existing report +If a scheduled job already builds a report body (text/HTML file), append the fragment before +the send step: + +```bash +python3 scripts/coverage_report.py wow --cluster ... --db ... --out-html cov.html +if [ -f cov.html ] && [ -f report.html ]; then + cat cov.html >> report.html +fi +``` +This is exactly how the Android Auth weekly report does it — the coverage section is generated +after the main report is assembled and concatenated onto the report body that gets emailed. + +## Case 2: Dashboard on Kusto +Once rows are in Kusto, any of these can render the trend without the email path: +- **Azure Data Explorer dashboards** (native, fastest to stand up). +- **Grafana** with the ADX data source. +- **Power BI** with the Kusto/ADX connector. + +Pin a tile using the "latest per module" or the `_wow_kql` query. This is the lowest- +maintenance option for teams that live in dashboards. + +## Case 3: Email via Azure Communication Services +For teams with no existing report, recommend **Azure Communication Services (ACS) Email** — it +sends from a verified domain with just a REST call and an access key, no mail server. + +Setup: +1. Create an **ACS resource** and an **Email Communication Service**; verify a sender domain + (Azure-managed subdomain is fastest for internal use). +2. Connect the domain to the ACS resource; note the **ACS endpoint** and an **access key** + (store the key in a pipeline secret / Key Vault — never in YAML). +3. Send the assembled HTML body (with the coverage fragment appended) via the ACS Email REST + API `POST {endpoint}/emails:send`, or the `Azure.Communication.Email` SDK. + +Keep the send step separate from coverage generation so a coverage failure never blocks the +email. Recipients should be a team distribution list. + +## Splitting into grouped tables +By default `wow` renders one table ("All Modules"). To split modules into labeled tables +(each with its own Overall row) — e.g. separating two product areas — pass a `--group-file` +JSON mapping table title → module list: + +```json +{ + "Android Broker Modules": ["msal", "common", "common4j", "broker4j", "AADAuthenticator"], + "Authenticator Modules": ["AuthApp", "MfaLibrary"] +} +``` +```bash +python3 scripts/coverage_report.py wow --cluster ... --db ... \ + --group-file groups.json --out-html coverage-wow.html +``` +Modules not listed in any group fall into a trailing catch-all table (rename via +`--other-title`, default "Other Modules"). Each table's **Overall** row sums its own +Covered/Total and shows an aggregate WoW delta computed from summed prior Covered/Missed +(not an average of per-module percentages). diff --git a/.github/skills/code-coverage-onboarding/scripts/coverage_report.py b/.github/skills/code-coverage-onboarding/scripts/coverage_report.py new file mode 100644 index 00000000..89714667 --- /dev/null +++ b/.github/skills/code-coverage-onboarding/scripts/coverage_report.py @@ -0,0 +1,1168 @@ +#!/usr/bin/env python3 +# Copyright (c) Microsoft Corporation. All rights reserved. +# Licensed under the MIT License. +""" +Portable code-coverage parser, reporter, and week-over-week (WoW) renderer. + +Designed to be dropped into any repo's CI (Azure DevOps, GitHub Actions, or any +runner that can execute a Python script after a test run). Stdlib only - no +third-party dependencies - so it runs on any pipeline image. + +Reads JaCoCo XML (Gradle/Maven/Android), Cobertura XML (Coverlet/VSTest, +coverage.py, gcovr, etc.), OR LCOV tracefiles (lcov.info; the default for many +JS/TS and C/C++ setups) and normalizes to a single row schema that matches a +Kusto tracking table: + + Date, Repo, Module, Metric, Covered, Missed, Percentage, CommitId, BuildId, Branch + +Subcommands: + parse - parse one or more coverage files -> append normalized NDJSON rows. + report - aggregate NDJSON rows -> Markdown summary (+ optional combined JSON). + gaps - rank the biggest uncovered classes/files -> a test-writing worklist + for raising coverage (per-class/file, not aggregate). + compare - diff a base report vs a PR/branch report per-class -> Markdown+JSON + showing where coverage regressed; exits non-zero on a drop beyond + --tolerance so it can gate a PR check. + gate - gate a build on a coverage drop vs the last-ingested Kusto baseline + for a branch (module-level; no rebuild of the target branch needed). + Exits non-zero on a drop and lists the modules that regressed. + wow - query a Kusto table for week-over-week coverage -> HTML fragment for + an email/report. Supports splitting modules into named tables, each + with an "Overall" summary row. + +Typical CI flow: + 1) Run tests so the build tool emits a JaCoCo/Cobertura XML report. + 2) `parse` each XML into a shared NDJSON file (one row per module+metric). + 3) `report` the NDJSON into a Markdown summary artifact. + 4) Ingest the NDJSON into Kusto (see references/kusto-setup.md) for history. + 5) On the scheduled reporting run, `wow` renders the trend into the email. +""" + +import argparse +import glob +import json +import os +import sys +import urllib.error +import urllib.request +import xml.etree.ElementTree as ET +from datetime import datetime, timezone + +# Metrics normalized across both report formats. LINE is the primary tracking +# metric; BRANCH is captured too when the report provides it. +METRICS = ("LINE", "BRANCH") + + +def _detect_format(root): + """Return 'jacoco' or 'cobertura' based on the XML root element.""" + tag = root.tag.lower() + if tag == "report": + return "jacoco" + if tag == "coverage": + return "cobertura" + raise ValueError(f"Unrecognized coverage XML root element: <{root.tag}>") + + +def _parse_jacoco(root): + """Extract {metric: (covered, missed)} from a JaCoCo report root. + + The top-level children of hold the aggregate totals. + """ + result = {} + for counter in root.findall("./counter"): + ctype = counter.get("type", "").upper() + if ctype in METRICS: + covered = int(counter.get("covered", 0)) + missed = int(counter.get("missed", 0)) + result[ctype] = (covered, missed) + return result + + +def _parse_cobertura(root): + """Extract {metric: (covered, missed)} from a Cobertura coverage root. + + Coverlet/VSTest 'Format=Cobertura', coverage.py, and gcovr emit + lines-covered / lines-valid / branches-covered / branches-valid attributes. + Fall back to counting elements when those attributes are absent. + """ + result = {} + + lines_covered = root.get("lines-covered") + lines_valid = root.get("lines-valid") + if lines_covered is not None and lines_valid is not None: + covered = int(lines_covered) + valid = int(lines_valid) + result["LINE"] = (covered, max(valid - covered, 0)) + else: + covered = missed = 0 + for line in root.iter("line"): + hits = int(line.get("hits", 0)) + if hits > 0: + covered += 1 + else: + missed += 1 + result["LINE"] = (covered, missed) + + br_covered = root.get("branches-covered") + br_valid = root.get("branches-valid") + if br_covered is not None and br_valid is not None: + covered = int(br_covered) + valid = int(br_valid) + result["BRANCH"] = (covered, max(valid - covered, 0)) + + return result + + +def _parse_lcov(text): + """Extract {metric: (covered, missed)} from an LCOV tracefile (lcov.info). + + LCOV is line-oriented text with one block per source file. Aggregate over all + files using the per-file summary records: + LF:/LH: = lines found / lines hit -> LINE + BRF:/BRH:= branches found / branches hit -> BRANCH + Fall back to counting DA:, records when LF/LH are absent (some + tools omit the summaries). FNF/FNH (functions) are ignored - we track LINE + and BRANCH to stay consistent with the JaCoCo/Cobertura output. + """ + lines_found = lines_hit = 0 + br_found = br_hit = 0 + have_line_summary = have_branch = False + da_covered = da_missed = 0 + + for raw in text.splitlines(): + line = raw.strip() + if line.startswith("LF:"): + lines_found += int(line[3:] or 0) + have_line_summary = True + elif line.startswith("LH:"): + lines_hit += int(line[3:] or 0) + elif line.startswith("BRF:"): + br_found += int(line[4:] or 0) + have_branch = True + elif line.startswith("BRH:"): + br_hit += int(line[4:] or 0) + elif line.startswith("DA:"): + parts = line[3:].split(",") + if len(parts) >= 2: + try: + hits = int(parts[1]) + except ValueError: + hits = 0 + if hits > 0: + da_covered += 1 + else: + da_missed += 1 + + result = {} + if have_line_summary: + result["LINE"] = (lines_hit, max(lines_found - lines_hit, 0)) + else: + result["LINE"] = (da_covered, da_missed) + if have_branch: + result["BRANCH"] = (br_hit, max(br_found - br_hit, 0)) + return result + + +def _looks_like_lcov(path): + """Sniff whether a file is an LCOV tracefile (text, not XML).""" + try: + with open(path, encoding="utf-8", errors="replace") as handle: + for _ in range(50): + line = handle.readline() + if not line: + break + stripped = line.strip() + if not stripped: + continue + if stripped.startswith("<"): + return False # XML + if stripped.startswith(("TN:", "SF:", "DA:", "LF:", "LH:", + "BRF:", "BRH:", "FNF:", "FNH:")) \ + or stripped == "end_of_record": + return True + except OSError: + return False + return False + + +def parse_file(path, fmt="auto"): + """Parse one coverage file -> {metric: (covered, missed)}. + + Supports JaCoCo XML, Cobertura XML, and LCOV tracefiles. With fmt='auto', + LCOV is detected by content sniffing and XML by its root element. + """ + if fmt == "lcov" or (fmt == "auto" and _looks_like_lcov(path)): + with open(path, encoding="utf-8", errors="replace") as handle: + return _parse_lcov(handle.read()) + + tree = ET.parse(path) + root = tree.getroot() + detected = _detect_format(root) if fmt == "auto" else fmt + if detected == "jacoco": + return _parse_jacoco(root) + if detected == "cobertura": + return _parse_cobertura(root) + raise ValueError(f"Unsupported coverage format: {detected}") + + +def _gaps_jacoco(root, metric): + """Per-class (covered, missed) for a metric from a JaCoCo report.""" + units = [] + for cls in root.iter("class"): + name = (cls.get("name") or "").replace("/", ".") + for counter in cls.findall("counter"): + if counter.get("type", "").upper() == metric: + units.append({ + "Unit": name, + "Covered": int(counter.get("covered", 0)), + "Missed": int(counter.get("missed", 0)), + }) + break + return units + + +def _gaps_cobertura(root, metric): + """Per-class (covered, missed) for a metric from a Cobertura report. + + LINE counts elements. BRANCH reads the + 'condition-coverage="P% (c/t)"' attribute Coverlet/coverage.py emit. + """ + units = [] + for cls in root.iter("class"): + name = cls.get("filename") or cls.get("name") or "" + covered = missed = 0 + if metric == "LINE": + for line in cls.iter("line"): + if int(line.get("hits", 0)) > 0: + covered += 1 + else: + missed += 1 + elif metric == "BRANCH": + for line in cls.iter("line"): + cond = line.get("condition-coverage") or "" + if "(" in cond and "/" in cond: + frac = cond[cond.find("(") + 1:cond.find(")")] + try: + cov, tot = (int(x) for x in frac.split("/")) + except ValueError: + continue + covered += cov + missed += max(tot - cov, 0) + if covered + missed > 0: + units.append({"Unit": name, "Covered": covered, "Missed": missed}) + return units + + +def _gaps_lcov(text, metric): + """Per-source-file (covered, missed) for a metric from an LCOV tracefile.""" + units = [] + cur = None + for raw in text.splitlines(): + line = raw.strip() + if line.startswith("SF:"): + cur = {"Unit": line[3:], "LF": 0, "LH": 0, "BRF": 0, "BRH": 0, + "da_c": 0, "da_m": 0, "has_line": False, "has_br": False} + elif cur is None: + continue + elif line.startswith("LF:"): + cur["LF"] = int(line[3:] or 0) + cur["has_line"] = True + elif line.startswith("LH:"): + cur["LH"] = int(line[3:] or 0) + elif line.startswith("BRF:"): + cur["BRF"] = int(line[4:] or 0) + cur["has_br"] = True + elif line.startswith("BRH:"): + cur["BRH"] = int(line[4:] or 0) + elif line.startswith("DA:"): + parts = line[3:].split(",") + if len(parts) >= 2: + try: + hits = int(parts[1]) + except ValueError: + hits = 0 + if hits > 0: + cur["da_c"] += 1 + else: + cur["da_m"] += 1 + elif line == "end_of_record": + if metric == "LINE": + if cur["has_line"]: + covered, missed = cur["LH"], max(cur["LF"] - cur["LH"], 0) + else: + covered, missed = cur["da_c"], cur["da_m"] + units.append({"Unit": cur["Unit"], "Covered": covered, "Missed": missed}) + elif metric == "BRANCH" and cur["has_br"]: + units.append({"Unit": cur["Unit"], "Covered": cur["BRH"], + "Missed": max(cur["BRF"] - cur["BRH"], 0)}) + cur = None + return units + + +def gap_units(path, metric, fmt="auto"): + """Parse one coverage file -> per-class/file [{Unit, Covered, Missed}] for a metric.""" + if fmt == "lcov" or (fmt == "auto" and _looks_like_lcov(path)): + with open(path, encoding="utf-8", errors="replace") as handle: + return _gaps_lcov(handle.read(), metric) + + tree = ET.parse(path) + root = tree.getroot() + detected = _detect_format(root) if fmt == "auto" else fmt + if detected == "jacoco": + return _gaps_jacoco(root, metric) + if detected == "cobertura": + return _gaps_cobertura(root, metric) + raise ValueError(f"Unsupported coverage format: {detected}") + + +def _pct(covered, missed): + total = covered + missed + return round(100.0 * covered / total, 2) if total else 0.0 + + +def cmd_parse(args): + paths = [] + for pattern in args.input: + paths.extend(glob.glob(pattern, recursive=True)) + if not paths: + sys.stderr.write(f"ERROR: no coverage files matched: {', '.join(args.input)}\n") + return 2 + + date = args.date or datetime.now(timezone.utc).strftime("%Y-%m-%d") + rows = [] + for path in sorted(set(paths)): + try: + metrics = parse_file(path, args.format) + except (ET.ParseError, ValueError, OSError) as exc: + sys.stderr.write(f"WARNING: skipping {path}: {exc}\n") + continue + for metric, (covered, missed) in metrics.items(): + rows.append( + { + "Date": date, + "Repo": args.repo, + "Module": args.module, + "Metric": metric, + "Covered": covered, + "Missed": missed, + "Percentage": _pct(covered, missed), + "CommitId": args.commit, + "BuildId": args.build_id, + "Branch": args.branch, + } + ) + + if not rows: + sys.stderr.write(f"ERROR: no usable coverage data parsed from {paths}\n") + return 2 + + with open(args.out, "a", encoding="utf-8") as handle: + for row in rows: + handle.write(json.dumps(row) + "\n") + + for row in rows: + print(f"{row['Repo']}/{row['Module']} {row['Metric']}: {row['Percentage']}% " + f"({row['Covered']}/{row['Covered'] + row['Missed']})") + return 0 + + +def _load_rows(patterns): + rows = [] + for pattern in patterns: + for path in glob.glob(pattern, recursive=True): + with open(path, encoding="utf-8") as handle: + for line in handle: + line = line.strip() + if line: + rows.append(json.loads(line)) + return rows + + +def cmd_report(args): + rows = _load_rows(args.input) + if not rows: + sys.stderr.write("ERROR: no coverage rows found to report on\n") + return 2 + + metric = args.metric.upper() + filtered = [r for r in rows if r["Metric"] == metric] + + # Aggregate covered/missed to a per-repo total across its modules. + per_repo = {} + for r in filtered: + agg = per_repo.setdefault(r["Repo"], {"Covered": 0, "Missed": 0}) + agg["Covered"] += r["Covered"] + agg["Missed"] += r["Missed"] + + lines = [ + f"# Weekly Code Coverage ({metric})", + "", + f"_Generated {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M UTC')} " + f"- goal: {args.goal}%_", + "", + "| Repo | Coverage | Covered | Total | Goal gap |", + "| --- | --- | --- | --- | --- |", + ] + for repo in sorted(per_repo): + covered = per_repo[repo]["Covered"] + missed = per_repo[repo]["Missed"] + total = covered + missed + pct = _pct(covered, missed) + gap = round(args.goal - pct, 2) + gap_txt = "met" if gap <= 0 else f"+{gap}%" + lines.append(f"| {repo} | {pct}% | {covered} | {total} | {gap_txt} |") + + markdown = "\n".join(lines) + "\n" + + if args.out_md: + with open(args.out_md, "w", encoding="utf-8") as handle: + handle.write(markdown) + if args.out_json: + with open(args.out_json, "w", encoding="utf-8") as handle: + json.dump(filtered, handle, indent=2) + + print(markdown) + return 0 + + +def _kusto_query(cluster, database, kql, token): + """POST a KQL query to the Kusto v1 REST endpoint; return list-of-dict rows. + + Stdlib only. The bearer token must already be scoped to the Kusto cluster + resource (https://kusto.kusto.windows.net or the cluster URL). + """ + url = cluster.rstrip("/") + "/v1/rest/query" + payload = json.dumps({"db": database, "csl": kql}).encode("utf-8") + request = urllib.request.Request( + url, + data=payload, + headers={ + "Authorization": "Bearer " + token, + "Content-Type": "application/json", + "Accept": "application/json", + }, + method="POST", + ) + with urllib.request.urlopen(request, timeout=60) as resp: + data = json.loads(resp.read().decode("utf-8")) + + tables = data.get("Tables") or [] + if not tables: + return [] + primary = tables[0] + columns = [c.get("ColumnName") for c in primary.get("Columns", [])] + rows = [] + for raw in primary.get("Rows", []): + rows.append(dict(zip(columns, raw))) + return rows + + +def _wow_kql(table, metric): + """KQL comparing each module's latest coverage row to last week's run. + + Calendar semantics (NOT a rolling 7-day span): 'curr' is the most recent row + per module; 'prior' is the most recent row from BEFORE the current week + started. startofweek() anchors to Sunday 00:00, so same-week manual re-runs + never count as the WoW baseline. If two rows exist for a module on the same + day, arg_max(Date, ...) keeps the latest run of that day. + + Emits PrevCovered/PrevMissed so an aggregate "Overall" row can compute a true + summed delta rather than averaging per-module percentages. + """ + return ( + 'let m = "{metric}";\n' + "let curr = {table}\n" + "| where Metric == m\n" + "| summarize arg_max(Date, Covered, Missed, Percentage) by Repo, Module;\n" + "let prior = {table}\n" + "| where Metric == m\n" + "| join kind=inner (curr | project Repo, Module, LatestDate = Date) " + "on Repo, Module\n" + "| where Date < startofweek(LatestDate)\n" + "| summarize arg_max(Date, Percentage, Covered, Missed) by Repo, Module;\n" + "curr\n" + "| join kind=leftouter (prior | project Repo, Module, " + "PrevPercentage = Percentage, PrevDate = Date, " + "PrevCovered = Covered, PrevMissed = Missed) on Repo, Module\n" + "| extend PrevFromLastWeek = iff(isnull(PrevDate), bool(null), " + "PrevDate >= startofweek(Date) - 7d)\n" + "| project Repo, Module, CurrentPct = Percentage, Covered, Missed, " + "PrevPct = PrevPercentage, PrevDate, PrevFromLastWeek, " + "PrevCovered, PrevMissed\n" + "| order by Repo asc, Module asc" + ).format(table=table, metric=metric) + + +def _coverage_table(title, rows): + """Render one coverage table (with an Overall summary row) as HTML lines. + + Returns [] when there are no rows so the caller can omit the section. The + Overall row sums Covered/Total across the table's modules; its WoW delta is + computed from summed prior Covered/Missed (a true aggregate, not an average + of per-module percentages). + """ + if not rows: + return [] + + out = [ + f"{title}", + "
", + "" + "", + ] + stale_baseline = False + sum_covered = sum_missed = 0 + sum_prev_covered = sum_prev_missed = 0 + have_prev = False + + for r in rows: + module = r.get("Module", "") + cur = r.get("CurrentPct") + prev = r.get("PrevPct") + covered = int(r.get("Covered") or 0) + missed = int(r.get("Missed") or 0) + total = covered + missed + cur_val = round(float(cur), 2) if cur is not None else 0.0 + + sum_covered += covered + sum_missed += missed + + if prev is None: + delta_cell = "" + else: + have_prev = True + sum_prev_covered += int(r.get("PrevCovered") or 0) + sum_prev_missed += int(r.get("PrevMissed") or 0) + delta = round(cur_val - float(prev), 2) + color = "green" if delta >= 0 else "red" + sign = "+" if delta >= 0 else "" + marker = "" + if r.get("PrevFromLastWeek") is False: + marker = " *" + stale_baseline = True + delta_cell = f'' + + out.append( + f"" + f"{delta_cell}" + ) + + overall_total = sum_covered + sum_missed + overall_pct = _pct(sum_covered, sum_missed) + if have_prev: + overall_prev_pct = _pct(sum_prev_covered, sum_prev_missed) + overall_delta = round(overall_pct - overall_prev_pct, 2) + color = "green" if overall_delta >= 0 else "red" + sign = "+" if overall_delta >= 0 else "" + overall_delta_cell = ( + f'' + ) + else: + overall_delta_cell = "" + out.append( + f"" + f"{overall_delta_cell}" + ) + + out.append("
ModuleCoverageΔ WoWCovered/Total
n/a{sign}{delta}%{marker}
{module}{cur_val}%{covered}/{total}
{sign}{overall_delta}%n/a
Overall{overall_pct}%{sum_covered}/{overall_total}
") + if stale_baseline: + out.append( + "* last week's run had no coverage data for this module; " + "Δ WoW is measured against an earlier week's run." + ) + return out + + +def _load_groups(args): + """Return an ordered list of (title, module_predicate) for wow tables. + + --group-file is a JSON object mapping table title -> list of module names + (case-insensitive). Modules not listed in any group fall into a trailing + catch-all table (title from --other-title). With no --group-file, all + modules render in a single table titled --single-title. + """ + if not args.group_file: + return [(args.single_title, lambda r: True)], None + + with open(args.group_file, encoding="utf-8") as handle: + mapping = json.load(handle) + + groups = [] + assigned = set() + for title, modules in mapping.items(): + wanted = {str(m).strip().lower() for m in modules} + assigned |= wanted + groups.append( + (title, (lambda w: (lambda r: (r.get("Module") or "").strip().lower() in w))(wanted)) + ) + other_pred = (lambda r: (r.get("Module") or "").strip().lower() not in assigned) + return groups, (args.other_title, other_pred) + + +def _wow_html(rows, metric, groups, other): + """Render WoW coverage rows as an HTML fragment, split into grouped tables.""" + out = ["
", f"Weekly Code Coverage ({metric})"] + tables = [] + for title, pred in groups: + tables.append(_coverage_table(title, [r for r in rows if pred(r)])) + if other: + other_title, other_pred = other + tables.append(_coverage_table(other_title, [r for r in rows if other_pred(r)])) + + rendered = [t for t in tables if t] + for i, table in enumerate(rendered): + out.extend(table) + if i < len(rendered) - 1: + out.append("
") + return "\n".join(out) + "\n" + + +def cmd_wow(args): + """Query Kusto for WoW coverage and emit an HTML fragment. + + Non-fatal by design: the report must still send when the coverage section + can't be produced, so failures emit a short note instead of a non-zero exit. + """ + metric = args.metric.upper() + groups, other = _load_groups(args) + + def _emit(html): + if args.out_html: + with open(args.out_html, "w", encoding="utf-8") as handle: + handle.write(html) + print(html) + return 0 + + token = os.environ.get(args.token_env, "").strip() + if not token: + sys.stderr.write( + f"WARNING: no Kusto token in ${args.token_env}; skipping coverage section\n" + ) + return _emit( + f"
Weekly Code Coverage ({metric})
" + "Coverage data unavailable this run (no Kusto access token).\n" + ) + + try: + rows = _kusto_query(args.cluster, args.db, _wow_kql(args.table, metric), token) + except (urllib.error.URLError, ValueError, KeyError) as exc: + sys.stderr.write(f"WARNING: Kusto coverage query failed: {exc}\n") + return _emit( + f"
Weekly Code Coverage ({metric})
" + "Coverage data unavailable this run (Kusto query failed).\n" + ) + + if not rows: + return _emit( + f"
Weekly Code Coverage ({metric})
" + "No coverage rows found in Kusto yet.\n" + ) + + return _emit(_wow_html(rows, metric, groups, other)) + + +def cmd_gaps(args): + """Rank the biggest uncovered classes/files as a test-writing worklist.""" + paths = [] + for pattern in args.input: + paths.extend(glob.glob(pattern, recursive=True)) + if not paths: + sys.stderr.write(f"ERROR: no coverage files matched: {', '.join(args.input)}\n") + return 2 + + metric = args.metric.upper() + agg = {} + for path in sorted(set(paths)): + try: + units = gap_units(path, metric, args.format) + except (ET.ParseError, ValueError, OSError) as exc: + sys.stderr.write(f"WARNING: skipping {path}: {exc}\n") + continue + for u in units: + entry = agg.setdefault(u["Unit"], {"Covered": 0, "Missed": 0}) + entry["Covered"] += u["Covered"] + entry["Missed"] += u["Missed"] + + rows = [] + for unit, cm in agg.items(): + covered, missed = cm["Covered"], cm["Missed"] + if missed < args.min_missed: + continue + pct = _pct(covered, missed) + if pct > args.max_pct: + continue + rows.append({"Unit": unit, "Covered": covered, "Missed": missed, + "Total": covered + missed, "Percentage": pct}) + + if not rows: + sys.stderr.write("No coverage gaps matched the filters " + f"(metric={metric}, min-missed={args.min_missed}, " + f"max-pct={args.max_pct}).\n") + return 0 + + if args.sort == "pct": + rows.sort(key=lambda r: (r["Percentage"], -r["Missed"])) + else: # "missed" - biggest absolute win first + rows.sort(key=lambda r: (-r["Missed"], r["Percentage"])) + + top = rows[:args.top] if args.top > 0 else rows + + lines = [ + f"# Coverage Gaps ({metric}) - top {len(top)} targets", + "", + f"_Sorted by {'lowest coverage' if args.sort == 'pct' else 'most missed lines'}; " + f"{sum(r['Missed'] for r in rows)} total {metric.lower()} units uncovered " + f"across {len(rows)} classes/files._", + "", + "| Class / File | Coverage | Missed | Covered/Total |", + "| --- | --- | --- | --- |", + ] + for r in top: + lines.append(f"| {r['Unit']} | {r['Percentage']}% | {r['Missed']} | " + f"{r['Covered']}/{r['Total']} |") + markdown = "\n".join(lines) + "\n" + + if args.out_md: + with open(args.out_md, "w", encoding="utf-8") as handle: + handle.write(markdown) + if args.out_json: + with open(args.out_json, "w", encoding="utf-8") as handle: + json.dump(top, handle, indent=2) + + print(markdown) + return 0 + + +def _aggregate_units(paths, metric, fmt): + """Aggregate per-class/file {Covered, Missed} across one or more reports.""" + agg = {} + for path in sorted(set(paths)): + try: + units = gap_units(path, metric, fmt) + except (ET.ParseError, ValueError, OSError) as exc: + sys.stderr.write(f"WARNING: skipping {path}: {exc}\n") + continue + for u in units: + entry = agg.setdefault(u["Unit"], {"Covered": 0, "Missed": 0}) + entry["Covered"] += u["Covered"] + entry["Missed"] += u["Missed"] + return agg + + +def cmd_compare(args): + """Diff a base coverage report against a PR/branch report and gate on regressions. + + Produces an overall pass/fail verdict plus a per-class breakdown of WHERE coverage + changed, so a PR author knows exactly which classes to add tests for. Exits non-zero + when overall coverage drops by more than --tolerance percentage points (unless + --no-fail-on-drop), so it can be used as a blocking PR check. + """ + metric = args.metric.upper() + base_paths, pr_paths = [], [] + for pattern in args.base: + base_paths.extend(glob.glob(pattern, recursive=True)) + for pattern in args.pr: + pr_paths.extend(glob.glob(pattern, recursive=True)) + if not pr_paths: + sys.stderr.write(f"ERROR: no PR coverage files matched: {', '.join(args.pr)}\n") + return 2 + if not base_paths: + # No baseline to compare against (e.g. first run / new module). Don't block the PR. + sys.stderr.write("WARNING: no base coverage files matched: " + f"{', '.join(args.base)}; skipping comparison (not gating).\n") + return 0 + + base = _aggregate_units(base_paths, metric, args.format) + pr = _aggregate_units(pr_paths, metric, args.format) + + base_cov = sum(v["Covered"] for v in base.values()) + base_miss = sum(v["Missed"] for v in base.values()) + pr_cov = sum(v["Covered"] for v in pr.values()) + pr_miss = sum(v["Missed"] for v in pr.values()) + base_pct = _pct(base_cov, base_miss) + pr_pct = _pct(pr_cov, pr_miss) + delta = round(pr_pct - base_pct, 2) + + # Per-class deltas. Classes present only in the PR with missed units are "new" untested + # code; classes whose percentage fell are "regressed". + regressed, new_gaps = [], [] + for unit in sorted(set(base) | set(pr)): + b = base.get(unit) + p = pr.get(unit, {"Covered": 0, "Missed": 0}) + p_pct = _pct(p["Covered"], p["Missed"]) + if b is None: + if p["Missed"] >= args.min_missed: + new_gaps.append({"Unit": unit, "Percentage": p_pct, + "Missed": p["Missed"], + "Total": p["Covered"] + p["Missed"]}) + continue + b_pct = _pct(b["Covered"], b["Missed"]) + d = round(p_pct - b_pct, 2) + if d < -args.unit_tolerance: + regressed.append({"Unit": unit, "BasePct": b_pct, "PrPct": p_pct, + "Delta": d, "Missed": p["Missed"]}) + + regressed.sort(key=lambda r: (r["Delta"], -r["Missed"])) # biggest drop first + new_gaps.sort(key=lambda r: (-r["Missed"], r["Percentage"])) + + failed = args.fail_on_drop and (pr_pct < base_pct - args.tolerance - 1e-9) + verdict = "FAIL" if failed else "PASS" + sign = "+" if delta >= 0 else "" + + lines = [ + f"# Code Coverage Comparison ({metric}) - {verdict}", + "", + f"| | Base | PR | Delta |", + f"| --- | --- | --- | --- |", + f"| **{metric} coverage** | {base_pct}% | {pr_pct}% | {sign}{delta} pp |", + "", + ] + if args.tolerance: + lines.append(f"_Allowed drop (tolerance): {args.tolerance} pp._") + lines.append("") + if failed: + lines.append(f"**Coverage dropped by {abs(delta)} pp** (base {base_pct}% -> PR " + f"{pr_pct}%), exceeding the allowed {args.tolerance} pp. " + "Add tests for the classes below to restore coverage.") + lines.append("") + + if regressed: + top_reg = regressed[:args.top] if args.top > 0 else regressed + lines += [ + f"## Classes with reduced coverage ({len(regressed)})", + "", + "| Class / File | Base | PR | Delta | Missed (PR) |", + "| --- | --- | --- | --- | --- |", + ] + for r in top_reg: + lines.append(f"| {r['Unit']} | {r['BasePct']}% | {r['PrPct']}% | " + f"{r['Delta']} pp | {r['Missed']} |") + lines.append("") + + if new_gaps: + top_new = new_gaps[:args.top] if args.top > 0 else new_gaps + lines += [ + f"## New/changed classes lacking coverage ({len(new_gaps)})", + "", + "| Class / File | Coverage | Missed | Covered/Total |", + "| --- | --- | --- | --- |", + ] + for r in top_new: + covered = r["Total"] - r["Missed"] + lines.append(f"| {r['Unit']} | {r['Percentage']}% | {r['Missed']} | " + f"{covered}/{r['Total']} |") + lines.append("") + + if not regressed and not new_gaps: + lines.append("_No per-class coverage regressions detected._") + lines.append("") + + markdown = "\n".join(lines) + "\n" + + if args.out_md: + with open(args.out_md, "w", encoding="utf-8") as handle: + handle.write(markdown) + if args.out_json: + payload = { + "metric": metric, "basePercentage": base_pct, "prPercentage": pr_pct, + "deltaPp": delta, "tolerancePp": args.tolerance, "failed": failed, + "regressed": regressed, "newGaps": new_gaps, + } + with open(args.out_json, "w", encoding="utf-8") as handle: + json.dump(payload, handle, indent=2) + + print(markdown) + return 1 if failed else 0 + + +def _modules_from_rows(rows, metric): + """Aggregate NDJSON rows into {module: {"Covered": c, "Missed": m}} for one metric.""" + agg = {} + for r in rows: + if str(r.get("Metric", "")).upper() != metric: + continue + entry = agg.setdefault(r.get("Module", ""), {"Covered": 0, "Missed": 0}) + entry["Covered"] += int(r.get("Covered", 0) or 0) + entry["Missed"] += int(r.get("Missed", 0) or 0) + return agg + + +def _read_ndjson(path): + rows = [] + with open(path, encoding="utf-8") as handle: + for line in handle: + line = line.strip() + if line: + rows.append(json.loads(line)) + return rows + + +def _kusto_baseline_modules(args, metric, token): + """Latest ingested per-module coverage for the baseline branch, as a dict. + + This is the module-level baseline a PR is gated against - the last coverage + numbers recorded for the target branch. Kusto only stores per-module rows, so + the gate is module-level (which module regressed); per-class 'where' comes from + running `gaps` on the regressed modules' current reports. + """ + branch_filter = ' and Branch == "%s"' % args.branch if args.branch else "" + kql = ( + "{table}\n" + '| where Metric == "{metric}" and Repo == "{repo}"{branch}\n' + "| summarize arg_max(Date, Covered, Missed) by Module\n" + "| project Module, Covered, Missed" + ).format(table=args.table, metric=metric, repo=args.repo, branch=branch_filter) + agg = {} + for r in _kusto_query(args.cluster, args.db, kql, token): + agg[r.get("Module", "")] = {"Covered": int(r.get("Covered", 0) or 0), + "Missed": int(r.get("Missed", 0) or 0)} + return agg + + +def cmd_gate(args): + """Gate a build on a coverage drop vs the last-ingested baseline for a branch. + + Unlike `compare` (which diffs two XML report sets per-class and needs the base + branch rebuilt), `gate` compares this build's per-module coverage against the + baseline stored in Kusto - so no second build of the target branch is needed. + Emits an overall verdict plus the list of modules whose coverage fell (the + 'where'), and exits non-zero on a drop beyond --tolerance unless + --no-fail-on-drop. The regressed-module list is written to --out-json so a + pipeline can run `gaps` on exactly those modules. + """ + metric = args.metric.upper() + current = _modules_from_rows(_read_ndjson(args.current), metric) + if not current: + sys.stderr.write("ERROR: no current %s rows found in %s\n" % (metric, args.current)) + return 2 + + if args.baseline: + baseline = _modules_from_rows(_read_ndjson(args.baseline), metric) + else: + token = os.environ.get(args.token_env, "") + if not token: + sys.stderr.write("ERROR: no Kusto bearer token in $%s\n" % args.token_env) + return 2 + baseline = _kusto_baseline_modules(args, metric, token) + + def _overall(mods): + c = sum(v["Covered"] for v in mods.values()) + m = sum(v["Missed"] for v in mods.values()) + return c, m, _pct(c, m) + + cur_c, cur_m, cur_pct = _overall(current) + + if not baseline: + # No baseline for this branch yet (first run / new branch / nothing ingested). + # Report but do not gate - blocking here would fail every PR until the first + # scheduled ingest lands. + markdown = ("# Code Coverage Gate (%s) - SKIPPED\n\n" + "_No baseline coverage found for repo `%s` branch `%s`; not gating. " + "Current overall %s coverage: %s%% (%d/%d)._\n" + % (metric, args.repo, args.branch or "(any)", metric, cur_pct, + cur_c, cur_c + cur_m)) + if args.out_md: + open(args.out_md, "w", encoding="utf-8").write(markdown) + if args.out_json: + json.dump({"metric": metric, "gated": False, "reason": "no-baseline", + "currentPercentage": cur_pct, "regressed": []}, + open(args.out_json, "w", encoding="utf-8"), indent=2) + print(markdown) + return 0 + + base_c, base_m, base_pct = _overall(baseline) + delta = round(cur_pct - base_pct, 2) + + regressed, new_mods = [], [] + for mod in sorted(set(baseline) | set(current)): + b = baseline.get(mod) + p = current.get(mod, {"Covered": 0, "Missed": 0}) + p_pct = _pct(p["Covered"], p["Missed"]) + if b is None: + new_mods.append({"Module": mod, "Percentage": p_pct, "Missed": p["Missed"]}) + continue + b_pct = _pct(b["Covered"], b["Missed"]) + d = round(p_pct - b_pct, 2) + if d < -args.unit_tolerance: + regressed.append({"Module": mod, "BasePct": b_pct, "CurrentPct": p_pct, + "Delta": d, "Missed": p["Missed"]}) + regressed.sort(key=lambda r: (r["Delta"], -r["Missed"])) + new_mods.sort(key=lambda r: (-r["Missed"], r["Percentage"])) + + failed = args.fail_on_drop and (cur_pct < base_pct - args.tolerance - 1e-9) + verdict = "FAIL" if failed else "PASS" + sign = "+" if delta >= 0 else "" + + lines = [ + "# Code Coverage Gate (%s) - %s" % (metric, verdict), + "", + "| | Baseline | This build | Delta |", + "| --- | --- | --- | --- |", + "| **%s coverage** | %s%% | %s%% | %s%s pp |" % (metric, base_pct, cur_pct, sign, delta), + "", + "_Baseline = last ingested coverage for `%s` branch `%s`._" % (args.repo, args.branch or "(any)"), + "", + ] + if args.tolerance: + lines += ["_Allowed drop (tolerance): %s pp._" % args.tolerance, ""] + if failed: + lines += ["**Overall coverage dropped by %s pp** (baseline %s%% -> %s%%), " + "exceeding the allowed %s pp. Add tests to the modules below " + "(run the per-class gaps worklist to see which classes)." + % (abs(delta), base_pct, cur_pct, args.tolerance), ""] + + if regressed: + top_reg = regressed[:args.top] if args.top > 0 else regressed + lines += ["## Modules with reduced coverage (%d)" % len(regressed), "", + "| Module | Baseline | This build | Delta | Missed |", + "| --- | --- | --- | --- | --- |"] + for r in top_reg: + lines.append("| %s | %s%% | %s%% | %s pp | %d |" + % (r["Module"], r["BasePct"], r["CurrentPct"], r["Delta"], r["Missed"])) + lines.append("") + if new_mods: + lines += ["## New modules (no baseline, informational) (%d)" % len(new_mods), "", + "| Module | Coverage | Missed |", "| --- | --- | --- |"] + for r in new_mods[:args.top] if args.top > 0 else new_mods: + lines.append("| %s | %s%% | %d |" % (r["Module"], r["Percentage"], r["Missed"])) + lines.append("") + if not regressed and not new_mods: + lines += ["_No module coverage regressions detected._", ""] + + markdown = "\n".join(lines) + "\n" + if args.out_md: + open(args.out_md, "w", encoding="utf-8").write(markdown) + if args.out_json: + json.dump({"metric": metric, "gated": args.fail_on_drop, "failed": failed, + "basePercentage": base_pct, "currentPercentage": cur_pct, + "deltaPp": delta, "tolerancePp": args.tolerance, + "regressed": regressed, "newModules": new_mods}, + open(args.out_json, "w", encoding="utf-8"), indent=2) + print(markdown) + return 1 if failed else 0 + + +def build_parser(): + parser = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + sub = parser.add_subparsers(dest="command", required=True) + + p = sub.add_parser("parse", help="Parse coverage XML into normalized NDJSON rows.") + p.add_argument("--input", nargs="+", required=True, + help="Coverage XML file(s) or glob(s).") + p.add_argument("--format", choices=["auto", "jacoco", "cobertura", "lcov"], default="auto") + p.add_argument("--repo", required=True) + p.add_argument("--module", required=True) + p.add_argument("--branch", default="") + p.add_argument("--commit", default="") + p.add_argument("--build-id", default="", dest="build_id") + p.add_argument("--date", default="", help="UTC date (YYYY-MM-DD); defaults to today.") + p.add_argument("--out", required=True, help="NDJSON file to append rows to.") + p.set_defaults(func=cmd_parse) + + r = sub.add_parser("report", help="Aggregate NDJSON rows into Markdown + JSON.") + r.add_argument("--input", nargs="+", required=True, + help="NDJSON file(s) or glob(s) produced by 'parse'.") + r.add_argument("--metric", default="LINE") + r.add_argument("--goal", type=float, default=75.0) + r.add_argument("--out-md", default="", dest="out_md") + r.add_argument("--out-json", default="", dest="out_json") + r.set_defaults(func=cmd_report) + + g = sub.add_parser("gaps", help="Rank the biggest uncovered classes/files (worklist " + "for raising coverage).") + g.add_argument("--input", nargs="+", required=True, + help="Coverage report file(s) or glob(s) - JaCoCo/Cobertura/LCOV.") + g.add_argument("--format", choices=["auto", "jacoco", "cobertura", "lcov"], default="auto") + g.add_argument("--metric", default="LINE", help="LINE (default) or BRANCH.") + g.add_argument("--top", type=int, default=25, + help="Show only the top N targets (0 = all).") + g.add_argument("--min-missed", type=int, default=1, dest="min_missed", + help="Ignore classes/files with fewer missed units than this.") + g.add_argument("--max-pct", type=float, default=100.0, dest="max_pct", + help="Ignore units already above this coverage %%.") + g.add_argument("--sort", choices=["missed", "pct"], default="missed", + help="'missed' = biggest absolute gain first; 'pct' = lowest coverage first.") + g.add_argument("--out-md", default="", dest="out_md") + g.add_argument("--out-json", default="", dest="out_json") + g.set_defaults(func=cmd_gaps) + + c = sub.add_parser("compare", help="Diff base vs PR coverage; show regressions and " + "gate (non-zero exit) on a drop.") + c.add_argument("--base", nargs="+", required=True, + help="Baseline coverage report file(s) or glob(s) (target/dev branch).") + c.add_argument("--pr", nargs="+", required=True, + help="PR/branch coverage report file(s) or glob(s).") + c.add_argument("--format", choices=["auto", "jacoco", "cobertura", "lcov"], default="auto") + c.add_argument("--metric", default="LINE", help="LINE (default) or BRANCH.") + c.add_argument("--tolerance", type=float, default=0.0, + help="Allowed overall drop in percentage points before failing " + "(default 0.0 = any drop fails).") + c.add_argument("--unit-tolerance", type=float, default=0.0, dest="unit_tolerance", + help="Per-class drop (pp) below which a class is listed as regressed.") + c.add_argument("--min-missed", type=int, default=1, dest="min_missed", + help="Ignore new classes with fewer missed units than this.") + c.add_argument("--top", type=int, default=25, + help="Show only the top N rows per table (0 = all).") + c.add_argument("--no-fail-on-drop", action="store_false", dest="fail_on_drop", + help="Report the diff but never exit non-zero (non-gating).") + c.add_argument("--out-md", default="", dest="out_md") + c.add_argument("--out-json", default="", dest="out_json") + c.set_defaults(func=cmd_compare, fail_on_drop=True) + + ga = sub.add_parser("gate", help="Gate a build on a coverage drop vs the last " + "ingested Kusto baseline for a branch (module-level; " + "no rebuild of the target branch needed).") + ga.add_argument("--current", required=True, + help="NDJSON produced by 'parse' for THIS build.") + ga.add_argument("--baseline", default="", + help="Optional NDJSON baseline (offline/testing). If omitted, the " + "baseline is queried from Kusto.") + ga.add_argument("--metric", default="LINE", help="LINE (default) or BRANCH.") + ga.add_argument("--repo", default="", help="Repo value to match in Kusto/NDJSON.") + ga.add_argument("--branch", default="", + help="Baseline branch (e.g. the PR target branch) to look up in Kusto.") + ga.add_argument("--cluster", default="", help="Kusto cluster URL (when no --baseline).") + ga.add_argument("--db", default="", help="Kusto database (when no --baseline).") + ga.add_argument("--table", default="CodeCoverageData") + ga.add_argument("--token-env", default="KUSTO_TOKEN", dest="token_env", + help="Env var holding the Kusto bearer token.") + ga.add_argument("--tolerance", type=float, default=0.0, + help="Allowed overall drop in pp before failing (default 0 = any drop fails).") + ga.add_argument("--unit-tolerance", type=float, default=0.0, dest="unit_tolerance", + help="Per-module drop (pp) below which a module is listed as regressed.") + ga.add_argument("--top", type=int, default=25, help="Show only the top N rows (0 = all).") + ga.add_argument("--no-fail-on-drop", action="store_false", dest="fail_on_drop", + help="Report the gate but never exit non-zero (non-gating).") + ga.add_argument("--out-md", default="", dest="out_md") + ga.add_argument("--out-json", default="", dest="out_json") + ga.set_defaults(func=cmd_gate, fail_on_drop=True) + + w = sub.add_parser("wow", help="Query Kusto for WoW coverage; emit HTML fragment.") + w.add_argument("--cluster", required=True, help="Kusto cluster URL.") + w.add_argument("--db", required=True, help="Kusto database name/alias.") + w.add_argument("--table", default="CodeCoverageData") + w.add_argument("--metric", default="LINE") + w.add_argument("--out-html", default="", dest="out_html", + help="File to write the HTML fragment to.") + w.add_argument("--token-env", default="KUSTO_TOKEN", dest="token_env", + help="Env var holding the Kusto bearer token.") + w.add_argument("--group-file", default="", dest="group_file", + help="JSON mapping {table title: [module,...]} to split into " + "multiple tables. Omit for a single table.") + w.add_argument("--single-title", default="All Modules", dest="single_title", + help="Table title when --group-file is not used.") + w.add_argument("--other-title", default="Other Modules", dest="other_title", + help="Title for the catch-all table of ungrouped modules.") + w.set_defaults(func=cmd_wow) + + return parser + + +def main(argv=None): + args = build_parser().parse_args(argv) + return args.func(args) + + +if __name__ == "__main__": + sys.exit(main())