fix(ci): reconstruct the paths cross-package tests really read, instead of trusting quoted prose - #9826
Conversation
…ad of trusting quoted prose `check-cross-package-test-inputs` lets a declared radius be NARROW, and says in its own prose why that is safe: a glob is only allowed to be narrow while it still covers every path the tests actually name. The roster behind that claim came from one flat regex, which sees a path only when the WHOLE repo-relative path sits inside ONE quoted string starting at a known top-level directory. Three live spellings do not, so the reads they express were absent from the coverage check -- silently, exit 0, in the fails-toward-FALSE-GREEN half of #9747's meta-shape. Measured on 06f9848: for `create-objectstack`, dropping the declared glob AND unquoting two header COMMENTS made this gate print `OK ... exit 0`, while the two tests that genuinely load `scripts/sync-template-versions.mjs` went on loading it. Prose held the radius. The fix is a reconstruction, not a wider regex. `walkLiteral` already resolved these expressions to compute the DEPTH that decides the escape verdict; it now carries the segment NAMES through the same walk, so a path split across `join('scripts', 'x.mjs')` arguments and an ascent-relative `new URL('../../../scripts/x.mjs', import.meta.url)` both come out as the repo-relative string an author would have quoted. No parser, no dependency -- the gate stays un-mutable in CI. The third spelling was a data defect rather than a collector one: `skills/` was simply missing from the flat alternation. An unreadable argument costs the NAME and keeps the depth, so the escape verdict is unchanged and the roster never gains an entry pointing at a file nobody reads. A directory counts only when a directory-listing read consumed it, and is judged by `coversDirectory` against the real entries -- `packages/lint/src/**` covers that listing while not matching the bare string `packages/lint/src`. Measured: +8 (package, path) pairs, 0 lost, 6 packages gaining. Six of the eight are radii the declarations already covered and prose was holding. Two are reads that were never declared at all and this pass found on its first run: `content/docs/api/error-catalog.mdx` and the strictness-ledger audit file, both read ascent-relative by `@objectstack/spec`. Their globs and the matching turbo.json inputs are added here; no declaration was narrowed. `--self-test` grows 33 -> 52 cases, one per added spelling as #8995 requires, each pinning the repo-relative name produced rather than merely that some path came out. Refs #9763 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XqDQYVU5smx29ts9pAErja
…eral-collector-spellings
✅ PM ACCEPT — #9763 / PR #9826Verified independently: 2 files ⭐ H3's price does not apply, and the reason is the best partI asked whether the collector could reconstruct a split-segment path at all, warned that AST-ifying a regex-based checker would be a much bigger change than the card implied, and said a partial fix was acceptable if you named what it still missed. The answer made the question moot:
The machinery was already there. It was computing the path and throwing the names away. That is a materially cheaper fix than either option I offered, and it was found by reading what the checker already does rather than by accepting my framing of what it would need. Ruling 2 answered too: split-segment and ascent-relative are one mechanism, ⭐ The directory subtlety is one I did not anticipate
The obvious implementation would roster any path that looks like a directory prefix, which would have manufactured false coverage everywhere. Gating on the call that actually consumed it, and judging the glob against real entries so H1 — both halves of the ladder, before and after
That last row is the half I said the fix must produce or it has not closed the gap. Both directions, reverse-verified from the committed state with a byte-clean restore. ⭐ No-false-leads, and a change you measured and DECLINED46 → 54 pairs: +8, 0 lost, 6 packages gaining — against #9639's rejected +139,084 and #9700's +4,751 / 0 lost / 1 gaining. All 8 hand-checked as real reads, 6 already inside declared globs. A gain that small, individually verified, is the strongest possible answer to ruling 3. And:
Declining a change that measures as a no-op — rather than shipping it because it looks symmetrical — is exactly right. A prefix that buys nothing is future maintenance with no present benefit, and most people would have added it for tidiness. ⭐ And you found two radii that were never declared at all
That is the fix finding real gaps on its first run, which is the best evidence it works. And per ruling 1, no declaration was narrowed or reshaped to fit the collector — the movement was all in the other direction. The prose is updated, and the distinction is correctI asked you to update #9765's rationale, since your fix makes it obsolete. You did: And you correctly left the On the mechanics
Verdict: ACCEPT. Arming once the two running gates converge. Generated by Claude Code |
|
…eral-collector-spellings # Conflicts: # scripts/check-cross-package-test-inputs.mjs
Conflict resolved —
|
| region | marker? | what landed |
|---|---|---|
create-objectstack rationale + globs |
<<<<<<< |
the only thing git asked about |
@objectstack/spec entry — main's serve.ts glob + its 11-line rationale |
none | main's hunk 1, +11 lines |
turbo.json — both sides' new inputs |
none | auto-merged file, no stages at all |
So I ran the thing rather than trusting the merge:
$ node --check scripts/check-cross-package-test-inputs.mjs
PARSE_EXIT=0
and swept for #9712's actual failure mode — a binding left bound twice by an unmarked auto-merge:
duplicate top-level bindings: (none)
duplicate package keys in CROSS_PACKAGE_TEST_INPUTS: (none)
turbo.json never conflicted, so it has no merge stages to read — I diffed it against both parents by key instead:
turbo.json inputs rostered by a parent but LOST in the merge: 0
task keys lost: (none)
Ruling 3 — the gate's own verdict lines, verbatim, on 1e60ca74ec
cross-package-selftest exit=0 All 52 self-test cases passed.
cross-package-verify exit=0 OK: 12 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob.
examples-live-selftest exit=0 All 31 self-test cases passed.
examples-live-imports exit=0 examples/** live-import inventory OK -- 0 invisible (declared), 6 inputs-declared, 69 graph-visible.
ratchet-remedy-authority exit=0 OK check-ratchet-remedy-authority: 97 scripts swept (scripts/*.{mjs,mts}); 6 mark the expanding remedy ⛔ MAINTAINER-ONLY, 3 turn it down outright, 88 hand out no ratchet-expanding remedy. Control corpus: 18 hand-classified scripts, set-equality audited both ways.
nul-bytes exit=0 check-nul-bytes: OK (scanned 6322 text file(s) -- 6322 tracked, 0 untracked-not-ignored; skipped 5 binary; no raw ASCII control bytes).
dispatch-gates-selftest exit=0 ✓ dispatch-gates self-test: 310 cases pass.
Every exit code captured before any pipe (cmd > file 2>&1; ex=$?; tail file), never $? after a | tail. dispatch-gates re-derived on the new head still names 2 families, both the same gate — unchanged by main's arrival.
I re-ran this whole union after the last commit: I reflowed one comment paragraph and amended, which moved the head from c0eee64340 to 1e60ca74ec, so the numbers above are from the tree that is actually pushed.
H1 — two ablations, both with landing confirmed independently of the editor
A. Is the branch's guard still alive?
The card's ladder, third rung: drop the scripts/sync-template-versions.mjs glob and unquote both flat-visible mentions, leaving only the split-segment read.
Landing confirmed by asserted match count, then by git diff --stat and by grepping for the removed and injected text:
glob removed x1 mention A x1 mention B x1
packages/create-objectstack/src/template-consistency.test.ts | 2 +-
packages/create-objectstack/src/template-version-stamps.test.ts | 2 +-
scripts/check-cross-package-test-inputs.mjs | 1 -
removed text now absent — glob: 0 · backticked mentions: 0, 0
injected text now present — :79 // scaffolded project. scripts/sync-template-versions.mjs re-stamps all three
:3 // The declaration surface of scripts/sync-template-versions.mjs (#9554).
Merged tree — RED, naming the file that actually reads:
ABLATION_A_EXIT=1
FAIL: cross-package test inputs are not declared consistently.
- create-objectstack names path(s) no declared glob covers, so a change to them would not
re-run its tests:
scripts/sync-template-versions.mjs (named in packages/create-objectstack/src/template-version-stamps.test.ts)
And the control, so the ablation is a discriminator rather than red for some unrelated reason — the identical ablation applied at merge-base 9ff11921a:
PREFIX_EXIT=0
OK: 12 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob.
Pre-fix: the exit-0 false green. Post-merge: red, naming the reader. The branch's guard survived.
B. Is main's side still load-bearing?
Removed all five globs main's two commits added (serve.ts ×2 entries, scaffold-e2e.yml, gen-sdui-manifest.sh, publish-smoke.sh). Landing confirmed the same way — asserted counts x2/x1/x1/x1, then 1 file changed, 5 deletions(-), then each glob grepped to 0:
ABLATION_B_EXIT=1
- @objectstack/spec names path(s) no declared glob covers…
packages/cli/src/commands/serve.ts (named in packages/spec/scripts/publish-smoke-port-collision.test.ts)
- create-objectstack names path(s) no declared glob covers…
scripts/gen-sdui-manifest.sh (named in packages/create-objectstack/src/scaffold-e2e-boot-probe.test.ts)
scripts/publish-smoke.sh (named in …/scaffold-e2e-boot-probe.test.ts)
packages/cli/src/commands/serve.ts (named in …/scaffold-e2e-boot-probe.test.ts)
.github/workflows/scaffold-e2e.yml (named in …/scaffold-e2e-boot-probe.test.ts)
All five fire. Main's side survived. Both ablations restored byte-clean from the committed state (git diff HEAD → 0 lines, gate green again).
⭐ H3 — the population, and a result I did not expect
| tree | packages | pairs |
|---|---|---|
merge-base 9ff11921a |
12 | 46 |
origin/main 56656aa1aa |
12 | 51 (+5) |
branch head c74b2b9aa |
12 | 54 (+8) |
merged 1e60ca74ec |
12 | 60 |
Zero pairs lost — every pair either parent rostered is still rostered.
But 46 + 5 + 8 = 59, and the merge rosters 60. The extra pair is real, and it is the best evidence the resolution is right:
NEW: rostered by the merge but by NEITHER parent:
create-objectstack | .github/workflows/scaffold-e2e.yml
Main's scaffold-e2e-boot-probe.test.ts spells the workflow split-segment:
const WORKFLOW = path.join(REPO_ROOT, '.github', 'workflows', 'scaffold-e2e.yml');On main alone that read is invisible — .github is not even in the flat alternation. On the branch alone the test does not exist. Only the two together see it. So I ablated that, on origin/main by itself:
=== drop the .github glob on origin/main ALONE ===
MAIN_ALONE_EXIT=0
OK: 12 package(s) read outside themselves, all declared…
On main, main's own newest declaration is held by nothing machine-visible — dropping it is a silent exit 0. After this merge, dropping it goes red. The branch's reconstruction is what makes #9779's declaration load-bearing for the first time. The two sides do not merely coexist; they compose.
H2 — assertion counts, by the script's own counter
Counted by running --self-test at each rev and reading the total the harness prints from cases.length, not by grep (fixture strings in this file contain the call text, exactly the undercount you warned about):
| rev | cases |
|---|---|
merge-base 9ff11921a |
33 |
origin/main 56656aa1aa |
33 |
branch head c74b2b9aa |
52 |
| merged | 52 |
33 + 0 + 19 = 52. Main's two commits added no self-test cases — they only touched the declaration table — so the sum is exact and no case was dropped.
H4 — does main still merge clean?
Re-checked after pushing. origin/main did move while I worked (56656aa1aa → 8fa4e2a76d, one commit, #9940, AGENTS.md only — touches neither of my two files):
$ git merge-tree --write-tree --messages origin/claude/issue-9763-literal-collector-spellings origin/main
bcc9ecae4cdc3f40d247074aedd94b1d9f202a32
MERGE_TREE_EXIT=0 CONFLICT lines: 0
Clean.
Two disclosures
1. I edited four words of main's prose, inside the conflict block. Main's new rationale cited sync-template-versions.mjs as an example of the "NAMED rather than read" shape. This PR makes that path read-held, and the paragraph immediately above it now says so — merging the two sides verbatim would have shipped a comment block contradicting itself two paragraphs apart. I dropped that half of the cross-reference and kept check-nul-bytes.mjs, which is still true ("the one entry no test READS"). Nothing else of main's text changed; all six globs kept.
- // shape as `check-nul-bytes.mjs` and `sync-template-versions.mjs` above and
+ // shape as `check-nul-bytes.mjs` above and settled the same way: …Revert it if you would rather the merge be literal — the gate is green either way.
2. The identical stale clause exists a second time, and I did not touch it. Main's other hunk — the one git auto-merged with no marker, in the @objectstack/spec entry — carries the same cross-reference. That one is outside the conflict, so per ruling 4 I filed it rather than fixing it: #9977, with the one-line patch ready. Note it is the second occurrence of a clause #9651 already retired once, eight days ago.
Narrowing declared: no eslint locally — this worktree has no node_modules and none exists anywhere in the container, so an install was the only route. The change is comment text plus a string array in a file whose parse (node --check) and execution I verified directly; CI runs the farm regardless.
Generated by Claude Code
Generated by Claude Code
PM review of the conflict resolution — ACCEPT. Open question → A. Re-arming.Verified at ⭐ Ruling 1 confirmed, and worse than I warnedI said one reported conflict is not one conflict. Measured:
And you checked for the exact #9712 failure mode by name: ⭐ H3 — the finding of the round: the merge ACTIVATES a guard neither parent hadPopulation is non-additive by one: 46 + 5 + 8 = 59, merged = 60.
Then you proved it rather than inferring it: dropping that glob on ⇒ H1 — both ablations, and the control that makes A mean something
Every mutation's landing confirmed independently of the editor — asserted match counts, then H2 — 33 / 33 / 52 / 52
Open question → A. Keep the four-word edit.Ruling 4 said do not fix defects in the accepted content. That rule is about inherited text. The conflict block is text you had to author — and authoring a comment that contradicts the paragraph two above it is a defect you would be introducing, not inheriting. Concretely: main's prose cited Leaving the identical stale clause in the cleanly auto-merged region untouched, and filing it as #9977 instead, is the right line — that text you did not have to author. And it being the second occurrence of a clause #9651 already retired eight days ago makes it a recurrence, not a straggler. Labelled H4 and the declared narrowing
Narrowing declared rather than hidden: no eslint locally (no Generated by Claude Code |
Fixes #9763. Sub-issue of #9747 (the meta-card), in its fails toward FALSE GREEN half.
check-cross-package-test-inputslets a declared radius be NARROW, and says in its own prose why that is safe: "a glob is only allowed to be narrow while it still covers every path the tests actually name, and the moment someone adds a probe outside the declared radius the gate fails naming the file." The roster behind that promise came from one flat regex, which sees a path only when the WHOLE repo-relative path sits inside ONE quoted string starting at a known top-level directory. Three live spellings do not — so the reads they express were absent from the coverage check, silently, at exit 0.H1 — the three-step ladder, before and after
The card's acceptance test. Ablating
create-objectstack's declaredscripts/sync-template-versions.mjsglob and progressively unquoting the two header-comment mentions of that path:06f9848f9)c74b2b9aa)template-consistency.test.ts(a mention)template-consistency.test.tstemplate-version-stamps.test.ts(the other mention)template-version-stamps.test.tsOK: 12 package(s) ..., exit 0template-version-stamps.test.ts, the file that actually READSAnd the other half the card demanded: with the glob kept and both mentions unquoted, the gate is green — the glob is now justified by the read, not by prose. Both mentions are ordinary prose again, free to reword.
The two mentions were markdown code spans inside comments (
`scripts/sync-template-versions.mjs`), which the flat regex cannot tell from code. Reverse-verified from the committed state: restoring the old collector restores the exit-0 false green.H3 — AST or regex? Neither. The resolver was already there
The checker is not a flat regex over source. It already carries a small recursive expression resolver (
pathExpression) that peelsfileURLToPath, recognises the seeds, splitsjoin/resolveargument lists on top-level commas, and folds constants across arguments — to compute the depth that decides the escape verdict. It then threw the segment names away.So this is not an AST-ification, and it is not priced like one:
walkLiteralnow carries the segment names through the same walk it already performed, seeded with the scanned file's own repo-relative path. A path split acrossjoin('scripts', 'x.mjs')arguments and an ascent-relativenew URL('../../../scripts/x.mjs', import.meta.url)both come out as the repo-relative string an author would have quoted. The gate stays dependency-free — the property that keeps it un-mutable in CI.Per the card's ruling 2, the three named spellings are not one change:
join/resolve— reconstruction (mechanism)..)skills/prefix — a one-line data fix, correctly diagnosed in the card.skills/was simply missing from the flat alternation.What it still does not see, stated rather than discovered later: a path built by template literal, one whose segments come from a variable the scan cannot fold, and a directory read whose path is only a loop variable. Each yields no name — never a wrong one. An unreadable argument costs the name and keeps the depth, so the escape verdict is unaffected and the roster never gains an entry pointing at a file nobody reads. Reads reaching another package through Node's resolver rather than through
fsare outside this gate entirely (that is whydownstream-contract'spackages/spec/src/**stays unheld here).I measured one alternation candidate beyond the card and did not ship it: adding
docsalongsideskillschanged the pair count by exactly zero (54 both ways), because the onedocs/read in the tree is ascent-relative and the reconstruction already resolves it.No false leads (ruling 3)
Matching #9700's bar, against #9639's +139,084:
Every one of the eight, hand-checked:
@objectstack/clipackages/connectors/connector-mcp/src/connector-mcp-plugin.ts@objectstack/downstream-contractpackages/spec/package.json@objectstack/formulaskills/objectstack-formula/SKILL.mdskills/case@objectstack/metadata-protocolscripts/check-durability-degradation-log-level.mjsnew URL@objectstack/plugin-securitypackages/spec/src/security/high-privilege.ts@objectstack/specpackages/lint/src@objectstack/speccontent/docs/api/error-catalog.mdx@objectstack/specdocs/audits/2026-07-unknown-key-strictness-ledger.mdNothing outside the intended radii moved: zero pairs lost, and six of the eight fall inside globs the declarations already carried.
Two radii that were never declared at all
The reconstruction found these on its first run — they are not radii
@objectstack/specgrew here:packages/spec/src/api/error-catalog-docs.test.tsreads the error-catalog page viaresolve(__dirname, '../../../../content/docs/api/error-catalog.mdx')and asserts it documents everyStandardErrorCode. spec declaredcontent/docs/references/**, not this page.packages/spec/scripts/strictness-ledger.test.tsreads the audit ledger viaresolve(SPEC, '../../docs/audits/...')and ratchets it against the schema files it inventories.Both are
readFileSyncon real files. Their globs and the matchingturbo.jsoninputs are added — declared per-page rather than as subtrees, for the reason the@objectstack/clientry already gives. Per ruling 1, no declaration was narrowed or reshaped to make the gate pass; this is the opposite direction, and it is what the gate exists to force.H2 — the full sweep, all 12 entries / 53 declared globs
The card's "four of twelve" was found while investigating one entry. Sweeping every glob, and splitting the roster by which half produced it:
Seven declared globs moved into read-held, including all four the card named:
create-objectstackscripts/sync-template-versions.mjs@objectstack/specpackages/lint/src/**@objectstack/metadata-protocolscripts/check-durability-degradation-log-level.mjs@objectstack/formulaskills/objectstack-formula/**@objectstack/clipackages/connectors/.../connector-mcp-plugin.ts@objectstack/plugin-securitypackages/spec/src/security/**@objectstack/downstream-contractpackages/spec/package.jsonAn important correction to the card's framing. "Flat-literal-held" is not the same as "prose-held". The flat collector cannot distinguish a genuine probe roster in code from a comment — and most of the remaining 23 are the former.
@objectstack/dogfood's conformance tests, for instance, iterate a literal table offile: 'packages/client/src/realtime-api.ts'entries and read each one; that IS the designed spelling, not prose. Telling the two apart is explicitly out of this gate's design ("declaring one rarely-touched file is cheaper than teaching the scanner to tell prose from code"), so the honest number is 23 flat-held, of unmeasured prose/code split, not 23 at risk. The 11 held by nothing are mostly live ESM imports held bycheck-examples-live-importsinstead, plusgit ls-fileswalks and module resolution — reads no source scan offscalls can see.Directories
Two of the card's three split-segment examples resolve to directories, so the file-only roster filter would have dropped them. A directory now counts when a directory-listing read (
readdirSync/opendirSync) consumed it — never when it is only a prefix used to build a path, which is the case the existing filter comment warns about and which@objectstack/downstream-contractspells exactly.statSync/existsSyncare deliberately excluded (either kind), andglobSynctakes a pattern.Coverage for a directory is a different question from
matchesAny, and the difference is load-bearing: subtree globs are written to match FILES, sopackages/lint/src/**does not match the bare stringpackages/lint/srcwhile turbo hashing that glob does re-run the test.coversDirectoryanswers it against the directory's real entries rather than inferring from the glob's shape —packages/lint/src/**andpackages/lint/src/**/*.object.tsare the same shape and only one of them re-runs when an ordinary.tsfile appears.Tests
--self-testgrows 33 to 52 cases, one per added spelling as #8995 requires. Each pins the repo-relative name produced, not merely that some path came out — a case asserting "a path appeared" would pass just as happily on a wrong one. Three of my own new cases failed on first run by asserting more than the design promises; they were tightened to pin the real guarantee (an unreadable argument yields no name for the path it builds, while intermediate bindings that do resolve still yield theirs).H4 — gate union, re-derived from the real diff
dispatch-gatesnames only 2 families for this file, both the same gate — the shortfall #9651's dev hit. Re-derived after mergingmain(no new families appeared) and ran the implicated ones by hand regardless. All green onc74b2b9aa:check-ratchet-remedy-authorityclassifies this file by name andcheck-examples-live-importsmirrors itsglobToRegExp(unchanged here) and reads its globs;dispatch-gates' own self-test reads this file's watch hints and asserts itsscripts/**andcontent/**declarations still reach specific paths.mainwas merged (not rebased) atc74b2b9aaand the whole union re-run on that head, including the H1 ladder. Prettier is not a gate on these paths —origin/main's copies of both files failprettier --checktoo, andlintis eslint-only — so no reformat was taken.No changeset: a CI gate script and
turbo.jsonpublish nothing.Generated by Claude Code