Skip to content

fix(ci): gate cross-package conformance scans by their real input set (#7802) - #7928

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-7802-cross-package-scan-ci-scope
Aug 12, 2026
Merged

fix(ci): gate cross-package conformance scans by their real input set (#7802)#7928
os-zhuang merged 1 commit into
mainfrom
claude/issue-7802-cross-package-scan-ci-scope

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes #7802

The card's surviving half: an affected-subset CI keyed on the package a cross-package conformance scan lives in cannot gate that scan. (The spec half — SINGLE_RECORD_WRITE_ONLY['sys_api_key'] — shipped in #7818; nothing here touches packages/spec or packages/platform-objects.)

What I measured, and where the PM's model was wrong

The dispatch's hypothesis was that there are two filtering layers and only the first opens up on merge_group. Measured at origin/main, that is not the escape:

Layer Behaviour on a platform-objects-only diff Verdict
dorny/paths-filter (filter job) core matches packages/**, so it was true all along never the leak
turbo ls --affected (test job, PR path) 51 packages affected, @objectstack/spec not among them real hole, PRs only
turbo's task cache @objectstack/spec#test hashes $TURBO_DEFAULT$ — package-local this is why it reached main

The merge queue takes the else branch and partitions the full package list, so it did schedule spec's suite. It just replayed a cached green: turbo run test --filter=@objectstack/spec after the violating edit printed >>> FULL TURBO in 42ms, replaying the previous run's log, while --force on the identical tree failed the scan. A third layer the card and the dispatch both missed.

packages/spec declares no dependency on packages/platform-objects and should not — the scan reads source text precisely to avoid inverting the spec -> * direction. So this is structural: no tuning of a dependency-graph affected set can reach it.

The fix

Packages whose tests read outside themselves declare that radius once (scripts/check-cross-package-test-inputs.mjs), and both layers are driven from it:

Why this is not just another list someone must remember

It is a list, and a list you must remember to update is exactly what produced #7802. So the list does not rely on memory. The gate finds escaping tests itself, statically, and fails on:

  • a package with an escaping test and no declaration,
  • a declaration gone stale,
  • a declared glob turbo.json does not hash,
  • a radius too narrow to cover the repo-relative paths its own tests name.

The default for an unregistered cross-package scan is a red gate naming the file and the globs to write — not a silent skip. Demonstrated below.

The last check is what lets radii stay narrow safely, which is how the affected-subset optimisation survives: packages/**/*.object.ts keeps spec's 5-minute suite off every PR that does not touch an object, where an always-run carve-out would have put it on all of them. It also caught four real gaps in globs I had authored by hand from a careful census (plugin-auth reads core/src/security/api-key.ts; dogfood pins spec's zod schemas; …).

Census: 20 such tests in 8 packagesspec (8), dogfood (4), lint (3), cli, core, plugin-auth, plugin-security, create-objectstack. Corroborating the card's "at least the third instance": packages/core/src/security/operation-private-keys.pin.test.ts carries a #7706 header about its own scan surface. A name-based grep for REPO_ROOT misses cli's (it uses PACKAGES_DIR) — which is why the detector is structural rather than name-based.

Positive demonstration (a green CI run would not prove this)

The failure mode is a scan that silently does not run, so I constructed a diff touching only a package the scan judges but does not live in — sys_user's apiMethods losing 'bulk' in packages/platform-objects — and ran it through the exact pipeline:

  • Layer A, before: turbo ls --affected → 50 packages, partition-test-shards puts spec on no shard (0/3).
  • Layer A, after: the union step reports + @objectstack/spec (declared glob matched packages/platform-objects/src/identity/sys-user.object.ts); spec lands on shard 1.
  • Layer B, before: >>> FULL TURBO, 42ms, cached green — the scan never executes.
  • Layer B, after: cache misses, the suite runs, and it fails with sys_user: [get, list, update] grants single-record writes but not 'bulk' (1 failed | 379 passed).
  • Control — the optimisation survives: on an unchanged tree the same command still cache-hits in 69ms >>> FULL TURBO. This does not disable caching; it makes the cache key honest.
  • Anti-rot: dropping a brand-new repo-walking test into packages/objectql (registered nowhere) fails the gate with @objectstack/objectql has test(s) that read outside the package but declares no input radius, naming the file.

The demo commit was removed; the branch contains only the fix.

Gates

check:cross-package-test-inputs (12 self-test cases), check:changeset-gate-self-tests, check:node-version, check:nul-bytes, check:required-contexts, check:shard-attestation, check:workflow-status-functions, check-changeset-no-major — all pass locally, re-run after rebasing onto 0e79785. eslint --no-inline-config on the new script: clean.

Notes for review

  • No changeset — CI config, a root gate script and turbo.json ship no published package. Needs the skip-changeset label.
  • File-surface deviation: the dispatch scoped this to .github/workflows/ci.yml (+ scripts/). I also touch turbo.json, .github/workflows/lint.yml and root package.json. turbo.json is unavoidable — Layer B cannot be fixed anywhere else, and it is the layer that actually let fix(platform-objects,plugin-auth): give API-key revoke/restore a working product route #7769 through; the other two register the gate so it runs. No new required context: the gate is a step in the existing lint job.
  • Cost: the eight declaring packages' test caches now invalidate when their declared globs change, and they join the affected set on PRs that touch those globs. That is the intended behaviour and the reason the radii are narrow rather than repo-wide.
  • Residual gap I did not close: spec's liveness ledgers (packages/spec/liveness/*.json) cite repo paths as data, and the literal-coverage check reads test sources only. A ledger citing a file outside spec's declared globs would not be covered. The ledger itself is in-package (so editing it does re-run the suite); only a deletion of a cited file elsewhere could slip. Flagged rather than fixed — closing it means either a repo-wide radius for spec or teaching the gate to read ledger data.

Generated by Claude Code

…#7802)

A test that resolves the repo root and walks it has an input set far wider
than the package it lives in, and both of CI's scoping layers judged such a
test by its home package:

  - `turbo ls --affected` (ci.yml `test`, the PR path) reaches packages
    through the dependency GRAPH. `packages/spec` declares no dependency on
    `packages/platform-objects` — and should not; the api-methods batch
    conformance scan reads source text precisely to avoid inverting the
    spec -> * direction. Measured on turbo 2.10.7: a platform-objects-only
    diff makes 51 packages affected, `@objectstack/spec` not among them.

  - turbo's task cache hashes `$TURBO_DEFAULT$`, which is package-local, so
    `@objectstack/spec#test` hashed identically before and after the change.
    Even the merge-queue and push builds — which deliberately partition the
    FULL package list — replayed a cached green. Measured: `>>> FULL TURBO`
    in 42ms replaying the prior log, while `--force` on the same tree failed
    the scan. This layer, not the paths-filter, is why #7769 reached `main`.

Packages whose tests read outside themselves now declare that radius once,
and both layers are driven from it: ci.yml unions a declaring package back
into the shard's package set when the diff touches its globs, and turbo.json
carries matching `$TURBO_ROOT$` inputs so the cache hash moves with them.

The declaration list cannot rot, which is the point: the gate finds escaping
tests itself, statically, and fails on a package that has one and no
declaration, on a stale declaration, on a glob turbo.json does not hash, and
on a radius too narrow to cover the paths its own tests name. A new
cross-package scan that nobody registers produces a red gate naming it and
the globs to write — not a silent skip.

Census: 20 such tests in 8 packages. Radii are kept narrow so the
affected-subset optimisation the 3-way shard exists for survives; the
literal-coverage check is what makes narrowing safe.
@vercel

vercel Bot commented Aug 12, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 12, 2026 4:37am

Request Review

@github-actions github-actions Bot added size/l ci/cd dependencies Pull requests that update a dependency file labels Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

Check Changeset red — expected, and it needs a label rather than a commit.

PR Automation / Check Changeset (job 94011572625) fails with "This PR adds no changeset." That is the correct verdict on this diff, not a defect in it: the PR touches .github/workflows/{ci,lint}.yml, turbo.json, root package.json and a new scripts/ gate. None of those ships in a published package, so there is nothing to version.

Resolution is the skip-changeset label. I'm not applying it myself — the dispatch for #7802 assigns it to the PM at review, and it's a review judgement about what this PR releases, not something the author should self-certify.

Explicitly not taking the two routes that would turn this green wrongly:

No other check has reported a failure on 4c8cdb2 — the rest of the suite is still running. Nothing to fix on the branch for this one.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd dependencies Pull requests that update a dependency file size/l skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

1 participant