Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,9 @@ repository still gets a decision, never by following the link; no release carrie
- :memo: docs(changelog): regenerate CHANGELOG.md for the EX-S10 lane
- :memo: docs(changelog): regenerate CHANGELOG.md for the EX-S10 F2/F5 fixes
- :memo: docs(changelog): regenerate CHANGELOG.md for the EX-S10 F6 fix
- :memo: docs(changelog): regenerate CHANGELOG.md for the AUD-S18 CHECK_STAGES fix
- :memo: docs(changelog): regenerate CHANGELOG.md for the check_coverage_bar narrowing fix
- :memo: docs(changelog): regenerate CHANGELOG.md for the stale-'15' cosmetic fix

### Features
- :sparkles: feat(docs): gate example pack and format claims against dogfood
Expand All @@ -152,9 +155,12 @@ repository still gets a decision, never by following the link; no release carrie
- :bug: fix(docs): pin case+decision pairs, not just case names, in the walkthrough (F1)
- :bug: fix(examples): scope format extraction to match: lines, not the whole classes: block (F2)
- :bug: fix(examples): close the same-line trailing-comment gap in format extraction (F6)
- :bug: fix(audit): sync AUD-S18 CHECK_STAGES pin with Taskfile check task
- :bug: fix(audit): narrow check_coverage_bar to avoid dogfood-examples' per-pack coverage lines

### Refactoring
- :recycle: refactor(examples): discover dogfood packs instead of a hardcoded 3-name loop
- :art: style(audit): de-stale hardcoded '15 stages' literals in AUD-S18 gate

### Testing
- :white_check_mark: test(release): anchor the D-120 note check on its header sentence, not the bare token
Expand Down
49 changes: 38 additions & 11 deletions hack/audit/exitgate_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
#
# (1) REL-07 — the AUD-S01 fail-closed cassettes PASS by name (not "go test
# exited 0", which a `-run` regex matching nothing also does).
# (2) RELSE-01/TEST-03 — `task check` is green with all 15 pinned stages
# (2) RELSE-01/TEST-03 — `task check` is green with all pinned stages
# actually executed (incl. `changelog-verify`) and the measured
# aggregate coverage at or above the epic's 91.0% bar.
# (3) RELSE-05 — the release job still runs the verify-green-on-tag-SHA gate,
Expand Down Expand Up @@ -122,9 +122,9 @@ S01_TESTS_CMD=(
TestFoldSnapshotPathsIncompleteEnumeration
)

# (2) The 15 stages of `task check`. This list is the authority: the Taskfile's
# (2) The stages of `task check`. This list is the authority: the Taskfile's
# `check:` list must equal it, and a real `task check` transcript must show all
# 15 having run. Grading on the exit code alone is what "aborts at the first
# of them having run. Grading on the exit code alone is what "aborts at the first
# failure" makes meaningless.
CHECK_STAGES=(
fmt
Expand All @@ -133,6 +133,7 @@ CHECK_STAGES=(
test
coverage
build
dogfood-examples
dogfood-comparison
compare-exitgate-test
changelog-verify
Expand All @@ -141,6 +142,7 @@ CHECK_STAGES=(
docs-gates
lint-depguard-test
lint-workflow-pins-test
dogfood-wiring-test
ci-audit-test
)

Expand Down Expand Up @@ -455,7 +457,7 @@ check_s01_cassettes() { # <transcript>
# (2) RELSE-01 / TEST-03 — `task check` green at the new bar
# ============================================================================

# The Taskfile's `check:` list must be exactly the 15 pinned stages. Without
# The Taskfile's `check:` list must be exactly the pinned stages. Without
# this, deleting `- task: changelog-verify` reopens RELSE-01 with `task check`
# still exiting 0 — the regression the whole story exists to prevent.
check_check_wiring() { # <taskfile>
Expand Down Expand Up @@ -512,7 +514,7 @@ check_check_wiring() { # <taskfile>
# Every pinned stage must have actually RUN. go-task prints `task: [<name>] <cmd>`
# per stage; that transcript is the only evidence a stage executed, because
# `task check` aborts at the first failure and the exit code cannot distinguish
# "15 stages green" from "stage list truncated to 2".
# "all stages green" from "stage list truncated to 2".
check_check_stages() { # <transcript>
local tr="$1" rc=0
[[ -f "$tr" ]] || {
Expand Down Expand Up @@ -557,12 +559,18 @@ check_coverage_bar() { # <transcript>
local plain="$WORK/cov.plain"
strip_ansi "$tr" >"$plain"

# Anchored at column 1: `go test -coverprofile` prints a per-PACKAGE
# `… coverage: 92.3% of statements` on every ok line, and grading on the
# minimum of those would compare the worst package against the aggregate bar.
# Only the gate's own `echo "coverage: ${pct}% (required: …)"` starts the line.
# Anchored at column 1 AND on a digit right after the colon-space: `go test
# -coverprofile` prints a per-PACKAGE `… coverage: 92.3% of statements` on
# every ok line (not at column 1, already excluded by the anchor), and
# EX-S08's dogfood-examples stage echoes ONE "coverage: OK — N rule(s),
# every rule tested in both polarities" line per example pack (at column 1,
# excluded only by the digit check — "OK" is not `[0-9]`). Grading on either
# impostor would either compare the worst package against the aggregate bar
# or count N-pack lines as N spurious "aggregate" measurements. Only the
# gate's own `echo "coverage: ${pct}% (required: …)"` (Taskfile.yml
# coverage:) starts the line with a digit.
local hits="$WORK/hits.coverage"
awk 'index($0, "coverage: ") == 1 { print }' "$plain" >"$hits"
awk '/^coverage: [0-9]/ { print }' "$plain" >"$hits"
local n
n="$(wc -l <"$hits" | tr -d '[:space:]')"
if ((n != 1)); then
Expand Down Expand Up @@ -1551,7 +1559,7 @@ expect_red check_s01_cassettes "the -run pattern matched NO test (go test exits
echo

# ---------------------------------------------------------------------------
echo "== (2) RELSE-01 / TEST-03 — task check green, 15 stages, coverage >= ${COVERAGE_BAR}% =="
echo "== (2) RELSE-01 / TEST-03 — task check green, ${#CHECK_STAGES[@]} stages, coverage >= ${COVERAGE_BAR}% =="

expect_green check_check_wiring "Taskfile check: runs exactly the ${#CHECK_STAGES[@]} pinned stages" "$TASKFILE"
expect_green check_coverage_floor "the single-sourced D-010 floor (Taskfile COVERAGE_MIN) is >= ${COVERAGE_BAR}" "$TASKFILE"
Expand Down Expand Up @@ -1700,6 +1708,25 @@ mutate_awk "$m" \
'coverage: 62.5% of statements'
expect_green check_coverage_bar "a per-package coverage line does not displace the aggregate one" "$m"

# EX-S08's dogfood-examples stage echoes its own "coverage: OK — N rule(s),
# every rule tested in both polarities" line per example pack — ALSO anchored
# at column 1, unlike the per-package control above. Without the digit anchor
# these three impostors plus the real aggregate line make `n == 4`, the exact
# regression this control pins (the bug this fix closes: reachable in
# practice as soon as dogfood-examples ran more than zero packs, and reached
# main unnoticed because check_check_wiring failed first and aborted the
# script before this check ever ran).
m="$WORK/check.synth.dogfoodcov"
cp "$SYNTH_CHECK" "$m"
mutate_awk "$m" \
'{ print } /^task: \[dogfood-examples\]/ {
print "coverage: OK — 6 rule(s), every rule tested in both polarities"
print "coverage: OK — 8 rule(s), every rule tested in both polarities"
print "coverage: OK — 9 rule(s), every rule tested in both polarities"
}' \
'coverage: OK — 6 rule(s), every rule tested in both polarities'
expect_green check_coverage_bar "dogfood-examples' per-pack 'coverage: OK — N rule(s)' lines do not displace the aggregate one" "$m"

echo

# ---------------------------------------------------------------------------
Expand Down