Skip to content

feat(spec): declare the settings visible grammar the evaluator actually implements (#7327) - #7387

Merged
os-zhuang merged 2 commits into
mainfrom
claude/issue-7327-settings-visible-grammar
Aug 10, 2026
Merged

feat(spec): declare the settings visible grammar the evaluator actually implements (#7327)#7387
os-zhuang merged 2 commits into
mainfrom
claude/issue-7327-settings-visible-grammar

Conversation

@os-zhuang

@os-zhuang os-zhuang commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Closes #7327. Part of #7169 — the alignment half, direction (b) as ruled by the maintainer on 2026-08-10 and dispatched by the spec-lane PM.

Both visible slots on a settings manifest were typed ExpressionInputSchema, whose bare-string arm normalises to dialect: 'cel'. Nothing has ever evaluated them as CEL. This narrows the declaration to the grammar that is actually evaluated; it does not touch the evaluator, and it does not touch ExpressionInputSchema.


Premise check (verified before writing code)

Premise How verified Result
PR #7310 is merged on main git log --grep "#7310" on origin/main d538647fix(service-settings): fail closed on an unevaluable visible predicate (#7169) (#7310)
Its "every bundled manifest predicate parses" pin exists grep in service-settings settings-service.test.ts:2568
The evaluator's grammar is what the card states read packages/services/service-settings/src/visibility-eval.ts post-#7310 ✅ tokenizer + recursive descent match the card verbatim, relational operators included
The card's line anchors (:234, :491) still point at the two slots grep -n visible on the target file ✅ both exact — re-anchored by content anyway
No non-bundled manifest in this repo ships a visible predicate swept packages/, apps/, examples/ for SettingsManifest producers ✅ only objectql's lifecycleSettingsManifest, which declares none

node scripts/check-adr-0087-registration.mjs --base origin/mainno docs/adr/** edit demanded (0 declared-breaking changesets). No ADR or release-notes file is touched by this PR.

Corpus verification — the guard that decided the direction

The #7169 measurement re-run against the narrowed declaration (not just the evaluator):

Direction (a): evaluate as declared CEL Direction (b): declare as evaluated — this PR
Bundled predicates broken 93 of 94 (===/!== are not CEL) 0 of 94
Manifests stored outside the repo all, unmigratable only those already failing at save

Measured, not asserted: settings-visibility-declaration.pin.test.ts parses all 10 bundled manifests through SettingsManifestSchema and collects refusals — the array is empty. It also re-measures the corpus size (10 manifests / 94 predicates) so a manifest gaining or losing a predicate sends the next reader back to the measurement instead of trusting a stale number.

Grammar as declared vs grammar as evaluated

Left: visibility-eval.ts (save-time, #7310). Right: settings-manifest.zod.ts (publish-time, this PR). Every row is asserted equal by the pin test, not by inspection.

Construct Evaluated Declared Note
Root data only data only current_user, bare identifiers → refused
Member access one level, [A-Za-z_][A-Za-z0-9_]* same data.a.b → refused
Boolean ops || && ! same
Comparison === !== == != >= <= > < same longest-first tokenization, !== beats !
Literals string (' / ", backslash escapes), number [0-9]+(.[0-9]+)?, true, false, null same no negative literals — -5 is refused on both sides
Grouping ( … ) same
Wrapper ${…} stripped by visibilitySource() identical unwrap, deliberately mirrored so "${a} && ${b}" refuses on both sides rather than one side repairing it
Envelopes bare string, { dialect, source } same ast-only envelope passes through (opaque at this layer)
Empty source "${}" → evaluates true, no parse accepted, no walk mirrored rather than "improved" — over-narrowing is what direction (a) was rejected for

The wire shape does not move. Bare string still normalises to { dialect: 'cel', source }; the reference table's type column is byte-identical. Only the accepted source strings narrow — the description column now carries the grammar, because a type cell cannot express it.

The refusal is self-prescribing:

Unsupported `visible` predicate "data.provider in ['smtp', 'resend']": unsupported
identifier "in" — the only root is `data`. A settings `visible` predicate is not CEL:
it is read by the save-time evaluator in `@objectstack/service-settings`, whose grammar
is closed — a single root `data` with one-level member access (`data.some_key`), the
operators `||` `&&` `!` and `===` `!==` `==` `!=` `>=` `<=` `>` `<`, parentheses, and
string / number / `true` / `false` / `null` literals. The whole predicate may be wrapped
in `${...}`, and both a bare string and a `{ dialect, source }` envelope are accepted.
Rewrite CEL membership as an `||` chain (`${data.x === 'a' || data.x === 'b'}`); function
calls, macros and member paths deeper than one level have no equivalent here.

Two statements of one grammar is the drift that caused #7169, so they are pinned to each other rather than left free. settings-visibility-declaration.pin.test.ts lives on the consumer side (spec cannot import a service — Prime Directive #2) and asserts "the schema accepts it" and "the evaluator can parse it" are the same bit, over a 15-case in-grammar table, an 11-case out-of-grammar table, and the real corpus.

Found in CI: the surface was about to leave the ADR-0058 ratchet

The expression-surface conformance ledger (ADR-0058 D7 / ADR-0060) re-discovers its surfaces by matching <key>: ExpressionInputSchema textually. Narrowing the two slots onto their own schema therefore dropped them out of discovery and turned their ledger entry stale — a live predicate surface silently leaving the ledger, which is precisely the #1887 class the ledger exists to catch. Caught by Dogfood Regression Gate (3/3) on the first push, fixed in 60b3120:

  • Discovery now reads a registered list of expression-declaring schema names instead of one hardcoded name, with the failure mode written down: a slot narrowed onto its own schema must register that schema on the same commit.
  • The classification is corrected while it is being moved. system/settings-manifest.zod.ts:visible sat under cel-uidialect: 'cel', enforced by "SchemaRenderer + celEngine" — and is evaluated by neither. It gets its own settings-visibility row naming evaluateVisibility, its closed grammar and its fail-closed policy, proved by the pin test. ExprDialect gains a member for it: that type is the ledger's own vocabulary (it already carries js, retired from the spec enum in formula: retire the js expression dialect — redundant with L2 ScriptBody; hasDialect also mis-reports the stub as real #3278), and spelling this surface cel would restate in the ledger the exact claim this PR removes from the schema.

Reverse-checked: dropping SettingsVisibilityInputSchema back out of the discovery list reproduces the identical STALE covers failure, so the registration — not a deleted assertion — is what keeps the surface watched.

Reverse verification — predictions written before running

Both mutations were applied alone, measured, and reverted. Predictions were committed to a scratch file before either run.

Mutation Predicted Measured
R1 — un-narrow: ExpressionInputSchema back on both slots 12 failures in spec/settings-manifest.test.ts (5 refuses CEL, 5 refuses malformed, the prescription case, the manifest-level case) + 13 in the service pin (11 refuses … on both sides, prescription, manifest-level) = 25 12 + 13 = 25, the exact named cases
R2 — drift the declaration behind the evaluator: drop >= <= > < from the spec-side operator list only 3 failures in spec (> 0, >= 0.5, <= 10 && …) + 6 in the service pin (corpus, both-sides, and 4 accepts) = 9; the corpus failure names exactly one refusal, namespace auth 3 + 6 = 9; corpus refusal: auth: specifiers.16.visible — … "data.lockout_threshold > 0": unexpected character ">"
R3 — un-register the narrowed schema from the ratchet's discovery list the ADR-0058 ledger test fails with STALE covers — surface no longer in source: system/settings-manifest.zod.ts:visible exactly that, 1 failed / 2 passed

R2 is the load-bearing one: it is #7169's failure mode with the sides swapped, and it reproduces on the exact predicate PR #7310 extended the evaluator for. ${-5 > data.x} was predicted to keep passing under R2 (the tokenizer hits - before it ever reaches >) and did.

Gates

Gate Result
pnpm --filter @objectstack/spec build ✅ (run first, before any regen)
check:generated (all 11) ✅ — 1 stale, exactly as predicted: check:docs. --fix regenerated only that one
check:authorable-surface ✅ not stale — key names unchanged (system/SettingsManifest:visible, system/Specifier:visible)
check:api-surface / check:export-origins ✅ not stale — the narrowed schema is module-private, no export added/removed/renamed, so the dual-snapshot rule does not fire
pnpm --filter @objectstack/spec test ✅ 362 files / 9492 tests
pnpm --filter @objectstack/service-settings test ✅ 18 files / 385 tests
dogfood expression-conformance (ADR-0058 ledger + ratchet) ✅ 3/3 after 60b3120
pnpm --filter @objectstack/spec typecheck (+ check:scripts-typecheck, check:test-typecheck)
eslint --no-inline-config on all touched files
check:adr-0087-registration --base origin/main ✅ no disposition demanded
check:empty-changeset, check:changeset-no-major, check:doc-authoring

Changeset level: '@objectstack/spec': minor, no ! / **BREAKING**

Judged from precedent, not by feel:

Files

File
packages/spec/src/system/settings-manifest.zod.ts the narrowed schema + a parse-only mirror of the evaluator's tokenizer/grammar; both slots re-typed and re-described
packages/spec/src/system/settings-manifest.test.ts +24 cases: accepted shapes, envelope forms, normalisation unchanged, CEL refusals with their reasons, malformed refusals, the prescription, the manifest-level slot
packages/services/service-settings/src/settings-visibility-declaration.pin.test.ts new — the producer/consumer pin and the corpus re-measurement
packages/qa/dogfood/test/expression-conformance.test.ts discovery reads a registered schema list, so a narrowed slot stays in the ratchet
packages/qa/dogfood/test/expression-conformance.ledger.ts new settings-visibility row; the surface leaves cel-ui, which never described it
content/docs/references/system/settings-manifest.mdx generated (gen:docs)
.changeset/settings-visible-grammar-declared.md new

No content/docs/releases/ or docs/adr/** file is touched.

…ally implements (#7327)

Both settings-manifest `visible` slots — specifier-level and manifest-level —
were typed `ExpressionInputSchema`, whose bare-string arm normalises to
`dialect: 'cel'`. Nothing has ever evaluated them as CEL: their only readers
are the console's client-side `new Function(...)` and, since #7310, the
server-side `evaluateVisibility`, which implements a small closed grammar.
`===` / `!==` — used throughout the bundled manifests — are not CEL at all.

#7169 measured which side should move: routing the declared CEL into
evaluation breaks 93 of the 94 bundled predicates, narrowing the declaration
breaks 1, and #7310's relational-operator extension had already taken that 1
to 0. Per the maintainer's 2026-08-10 ruling (and #7071's "each protocol keeps
its own spelling"), the declaration moves.

Both slots now accept exactly the evaluated grammar: single root `data`, one
level of member access, `|| && !`, `=== !== == != >= <= > <`, parentheses and
string/number/bool/null literals, optionally `${...}`-wrapped. Bare string and
`{ dialect, source }` envelope are both still accepted and a bare string still
normalises to the canonical envelope, so the wire shape does not move — only
the accepted `source` strings narrow. Real CEL (`data.x in [...]`,
`size(data.y) > 0`, `data.a.b == 1`) is refused at publish/parse with a message
naming the offending source, the reason, and the grammar that would work.
#7310's save-time refusal stays as defense in depth.

A second statement of one grammar is the drift that caused #7169, so the two
are pinned to each other: `settings-visibility-declaration.pin.test.ts` asserts
"the schema accepts it" and "the evaluator can parse it" are the same bit, over
an in/out-of-grammar table and over the real corpus — re-measured at 10
manifests / 94 predicates, 0 refused.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdPj3S347aPWapzTuHCb4N
@vercel

vercel Bot commented Aug 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 10, 2026 8:56am

Request Review

@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 2 package(s): @objectstack/dogfood, @objectstack/spec.

107 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/agents.mdx (via @objectstack/spec)
  • content/docs/ai/skills-reference.mdx (via @objectstack/spec)
  • content/docs/ai/skills.mdx (via @objectstack/spec)
  • content/docs/api/client-sdk.mdx (via @objectstack/spec)
  • content/docs/api/environment-routing.mdx (via @objectstack/spec)
  • content/docs/api/error-catalog.mdx (via @objectstack/spec)
  • content/docs/api/error-handling-client.mdx (via @objectstack/spec)
  • content/docs/api/error-handling-server.mdx (via @objectstack/spec)
  • content/docs/api/index.mdx (via @objectstack/spec)
  • content/docs/automation/approvals.mdx (via @objectstack/spec)
  • content/docs/automation/connectors.mdx (via @objectstack/spec)
  • content/docs/automation/flows.mdx (via @objectstack/spec)
  • content/docs/automation/hook-bodies.mdx (via packages/spec)
  • content/docs/automation/hooks.mdx (via @objectstack/spec)
  • content/docs/automation/index.mdx (via @objectstack/spec)
  • content/docs/automation/webhooks.mdx (via @objectstack/spec)
  • content/docs/automation/workflows.mdx (via @objectstack/spec)
  • content/docs/concepts/architecture.mdx (via @objectstack/spec)
  • content/docs/concepts/design-principles.mdx (via packages/spec)
  • content/docs/concepts/index.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-driven.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-lifecycle.mdx (via packages/spec)
  • content/docs/concepts/north-star.mdx (via @objectstack/spec)
  • content/docs/data-modeling/analytics.mdx (via @objectstack/spec)
  • content/docs/data-modeling/drivers.mdx (via @objectstack/spec)
  • content/docs/data-modeling/external-datasources.mdx (via @objectstack/spec)
  • content/docs/data-modeling/field-types.mdx (via @objectstack/spec)
  • content/docs/data-modeling/fields.mdx (via @objectstack/spec)
  • content/docs/data-modeling/formulas.mdx (via @objectstack/spec)
  • content/docs/data-modeling/index.mdx (via @objectstack/spec)
  • content/docs/data-modeling/objects.mdx (via @objectstack/spec)
  • content/docs/data-modeling/queries.mdx (via @objectstack/spec)
  • content/docs/data-modeling/schema-design.mdx (via @objectstack/spec)
  • content/docs/data-modeling/seed-data.mdx (via @objectstack/spec)
  • content/docs/data-modeling/validation-rules.mdx (via @objectstack/spec)
  • content/docs/data-modeling/validation.mdx (via @objectstack/spec)
  • content/docs/deployment/cli.mdx (via @objectstack/spec)
  • content/docs/deployment/tenancy-modes.mdx (via @objectstack/spec)
  • content/docs/deployment/troubleshooting.mdx (via @objectstack/spec)
  • content/docs/deployment/validating-metadata.mdx (via @objectstack/spec)
  • content/docs/getting-started/build-with-claude-code.mdx (via @objectstack/spec)
  • content/docs/getting-started/common-patterns.mdx (via @objectstack/spec)
  • content/docs/getting-started/examples.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-reference.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-start.mdx (via @objectstack/spec)
  • content/docs/getting-started/your-first-project.mdx (via @objectstack/spec)
  • content/docs/kernel/cluster.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/auth-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/cache-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/data-engine.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/index.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/metadata-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/storage-service.mdx (via @objectstack/spec)
  • content/docs/kernel/index.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/data-service.mdx (via @objectstack/spec)
  • content/docs/kernel/runtime-services/email-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/examples.mdx (via @objectstack/spec)
  • content/docs/kernel/runtime-services/index.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/queue-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/sharing-service.mdx (via @objectstack/spec)
  • content/docs/kernel/runtime-services/sms-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/storage-service.mdx (via @objectstack/spec)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/spec)
  • content/docs/kernel/services.mdx (via @objectstack/spec)
  • content/docs/permissions/authorization.mdx (via packages/qa/dogfood, @objectstack/spec)
  • content/docs/permissions/delegated-administration.mdx (via packages/qa/dogfood)
  • content/docs/permissions/permission-sets.mdx (via @objectstack/spec)
  • content/docs/permissions/permissions-matrix.mdx (via @objectstack/spec)
  • content/docs/permissions/positions.mdx (via @objectstack/spec)
  • content/docs/permissions/rls.mdx (via @objectstack/spec)
  • content/docs/permissions/sharing-rules.mdx (via @objectstack/spec)
  • content/docs/permissions/system-context.mdx (via packages/spec)
  • content/docs/plugins/adding-a-metadata-type.mdx (via @objectstack/spec)
  • content/docs/plugins/development.mdx (via @objectstack/spec)
  • content/docs/plugins/index.mdx (via @objectstack/spec)
  • content/docs/plugins/packages.mdx (via @objectstack/spec)
  • content/docs/protocol/backward-compatibility.mdx (via @objectstack/spec)
  • content/docs/protocol/diagram.mdx (via packages/spec)
  • content/docs/protocol/kernel/config-resolution.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/http-protocol.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/i18n-standard.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/index.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/lifecycle.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/plugin-spec.mdx (via @objectstack/spec)
  • content/docs/protocol/knowledge.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/schema.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/security.mdx (via packages/spec)
  • content/docs/protocol/objectql/state-machine.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/actions.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/concept.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/layout-dsl.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/record-alert.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/widget-contract.mdx (via @objectstack/spec)
  • content/docs/ui/actions.mdx (via @objectstack/spec)
  • content/docs/ui/apps.mdx (via @objectstack/spec)
  • content/docs/ui/create-vs-edit-form.mdx (via @objectstack/spec)
  • content/docs/ui/dashboards.mdx (via @objectstack/spec)
  • content/docs/ui/field-grouping-and-order.mdx (via @objectstack/spec)
  • content/docs/ui/forms.mdx (via @objectstack/spec)
  • content/docs/ui/index.mdx (via @objectstack/spec)
  • content/docs/ui/public-data-collection.mdx (via @objectstack/spec)
  • content/docs/ui/setup-app.mdx (via @objectstack/spec)
  • content/docs/ui/translations.mdx (via @objectstack/spec)
  • content/docs/ui/views.mdx (via @objectstack/spec)

7 release-owned page(s) also reference the affected code. These are read-only:

  • content/docs/releases/implementation-status.mdx (via @objectstack/spec)
  • content/docs/releases/index.mdx (via @objectstack/spec)
  • content/docs/releases/v12.mdx (via @objectstack/spec)
  • content/docs/releases/v13.mdx (via @objectstack/spec)
  • content/docs/releases/v16.mdx (via @objectstack/spec)
  • content/docs/releases/v17.mdx (via @objectstack/spec)
  • content/docs/releases/v9.mdx (via @objectstack/spec)

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@github-actions github-actions Bot added documentation Improvements or additions to documentation protocol:system tests tooling labels Aug 10, 2026
…ratchet, classified honestly (#7327)

The expression-surface conformance ratchet discovers surfaces by matching
`<key>: ExpressionInputSchema` textually, so narrowing the two settings
`visible` slots onto their own schema dropped them out of the scan and turned
their ledger entry stale — a live predicate surface silently leaving the
ledger, which is the #1887 class the ledger exists to catch.

Discovery now reads a registered list of expression-declaring schema names
rather than one hardcoded name, with the failure mode written down: a slot
narrowed onto its own schema must register that schema on the same commit.

The classification is corrected while it is being moved. `settings-manifest
visible` sat under `cel-ui` — `dialect: 'cel'`, enforced by the SchemaRenderer
and celEngine — and is evaluated by neither. It gets its own `settings-visibility`
row naming `evaluateVisibility`, its closed grammar, and its fail-closed policy
(#7310), proved by the producer/consumer pin. `ExprDialect` gains a member for
it: the ledger records what a surface IS, and spelling this one `cel` would
restate in the ledger the exact claim #7327 removes from the schema.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdPj3S347aPWapzTuHCb4N
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation protocol:system size/l tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Narrow the settings-manifest visible declaration to the grammar the save-time evaluator actually implements (#7169 alignment half, measured 1-vs-93)

2 participants