Skip to content

Commit 52c9cab

Browse files
os-helpclaude
andauthored
docs(kernel): align data-engine contract page with the shipped engine seam (#7057) (#7173)
Three measured divergences on content/docs/kernel/contracts/data-engine.mdx, all verified against origin/main rather than the issue's line anchors. 1. `EngineQueryOptions.cursor` and 2. query-level `distinct` were retired by #4286 (ADR-0049 / ADR-0078) but were still listed as live members of the interface. They are not merely absent from the schema: `retiredKey()` tombstones REJECT them by name, so a reader copying this block wrote a query the engine refuses. Both lines are removed from the code block and replaced by a "Removed in protocol 17" subsection carrying each tombstone's own migration prescription (keyset as a `where` predicate on the sort key; `distinct(object, field)` / `groupBy` / `count_distinct`). 3. The `WriteObservabilityOptions` section stopped at #3407 and never learned #5126's `strictReadonlyWrites` — absent from both prose and the code block — so "The write still succeeds" read unconditionally where strict refuses the write with ERR_READONLY_FIELD_REJECTED. The strip enumeration also still named only the two author-declared strips, missing the runtime-owned strip (#5503, the one that also runs on INSERT) and the primary-key strip (#6437), even though #7125 had already repaired the `reason` enum in the code block. The section is rewritten against packages/spec/src/contracts/data-engine.ts: a strip table (strip / reason / verbs / writers it skips), the two options as alternative outputs of one seam (`onFieldsDropped` does NOT fire on a refused write), the INSERT rule and its two exempt writers, and the engine-seam vs DataProtocol-ingress layering note (#3043; `preserveAudit` is UPDATE-only at the ingress, #6640) — that layering verified in metadata-protocol's `stripReadonlyForInsert`, not taken on trust. The in-process-only Callout now covers the whole bag, since a client toggling write-refusal is the specific thing #5126 ruled out. Docs-only; no package behaviour changes. Claude-Session: https://claude.ai/code/session_01KJATVrh6V2ysutYUJigh3B Co-authored-by: Claude <noreply@anthropic.com>
1 parent edb4af0 commit 52c9cab

1 file changed

Lines changed: 109 additions & 14 deletions

File tree

content/docs/kernel/contracts/data-engine.mdx

Lines changed: 109 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -110,10 +110,8 @@ interface EngineQueryOptions {
110110
limit?: number; // LIMIT
111111
offset?: number; // OFFSET
112112
top?: number; // Alias for limit (OData compat)
113-
cursor?: Record<string, unknown>; // Keyset pagination
114113
search?: FullTextSearch; // Full-text search
115114
expand?: Record<string, QueryAST>; // Recursive relation loading
116-
distinct?: boolean; // SELECT DISTINCT
117115
context?: ExecutionContext; // Identity, tenant, transaction — any subset
118116
}
119117
```
@@ -123,6 +121,30 @@ with defaults applied, is `ExecutionContextParsed`). Supply what you have, the e
123121
`{ isSystem: true }`; an automation run with no resolvable identity passes only
124122
its run id (`{ flowRunId }`), a context that deliberately carries no principal.
125123

124+
#### Removed in protocol 17: `cursor` and `distinct`
125+
126+
Both keys were **removed** from `EngineQueryOptions` in protocol 17 (#4286,
127+
ADR-0049), alongside the identically-named keys on `Query`. They are not merely
128+
absent: the schema keeps a tombstone for each, so a query still carrying one is
129+
**rejected by name** with the migration prose below rather than silently ignored.
130+
131+
- `cursor?: Record<string, unknown>` (keyset pagination) — no driver ever
132+
implemented it, so the cursor was accepted and ignored and every page came back
133+
identical (a caller looping "until `hasMore` is false" never terminates).
134+
`QueryBuilder.cursor()` was removed with it. Express the keyset as an ordinary
135+
`where` predicate on your sort key — `where: { created_at: { $gt: last.created_at } }`
136+
with the matching `orderBy` — which every driver executes with canonicalised
137+
comparands. A first-class cursor, if ever built, will be a response-minted
138+
opaque token, not this caller-built record.
139+
- `distinct?: boolean` (SELECT DISTINCT) — no driver ever rendered it, and the
140+
flag's only observable effect was MIS-WIRED: the REST list path treated a
141+
distinct query as not countable and silently degraded `total`/`hasMore` to a
142+
page-local estimate while still returning duplicate rows. `QueryBuilder.distinct()`
143+
was removed with it, and the count suppression is gone (`total` is truthful
144+
again). For unique values of one column use the SQL/memory drivers'
145+
`distinct(object, field)` door; for unique combinations, `groupBy`; for a
146+
deduplicated count, the `count_distinct` aggregation.
147+
126148
### FilterCondition (where)
127149

128150
Filters use the canonical **`where` + MongoDB-style `$op` object syntax** from `FilterConditionSchema`:
@@ -267,18 +289,28 @@ interface EngineUpdateOptions {
267289

268290
### WriteObservabilityOptions
269291

270-
The write methods (`insert` / `update`) additionally accept an **in-process**
271-
`onFieldsDropped` listener. The engine invokes it when caller-supplied write
272-
fields are legally stripped from the payload before the driver write — static
273-
`readonly` fields or a TRUE `readonlyWhen` predicate. The write still succeeds;
274-
the listener exists so callers that report per-field success (e.g. the flow
275-
engine's `update_record` step) can surface a warning instead of a silent
276-
success.
292+
The write methods (`insert` / `update`) additionally accept two **in-process**
293+
options that govern what happens when caller-supplied write fields are legally
294+
stripped from the payload before the driver write: observe the strip
295+
(`onFieldsDropped`) or refuse the write outright (`strictReadonlyWrites`).
296+
297+
The strips these two options cover are the engine's legal ones:
298+
299+
| Strip | `reason` | Verbs | Writers it skips |
300+
|:---|:---|:---|:---|
301+
| Static `readonly: true` (#2948) | `readonly` | `update` | `isSystem` |
302+
| A TRUE `readonlyWhen` predicate (#3042) | `readonly_when` | `update` | none — every caller, `isSystem` included |
303+
| Implicitly-readonly runtime-owned type (#5503`RUNTIME_OWNED_FIELD_TYPES`, today `autonumber`) | `readonly` | `insert` **and** `update` | `isSystem`, `preserveAudit` (#3493) |
304+
| Primary-key strip of a payload `id` the update dispatch already ruled is not an identifier (#6437) | `primary_key` | `update` | none |
305+
306+
The two AUTHOR-DECLARED strips are insert-exempt at this seam by design (#3413) —
307+
see **On `insert`** below.
277308

278309
{/* os:check */}
279310
```typescript
280311
interface WriteObservabilityOptions {
281312
onFieldsDropped?: (event: DroppedFieldsEvent) => void;
313+
strictReadonlyWrites?: boolean; // refuse instead of stripping. Default: false
282314
}
283315

284316
interface DroppedFieldsEvent {
@@ -290,12 +322,75 @@ interface DroppedFieldsEvent {
290322
}
291323
```
292324

325+
#### `onFieldsDropped` — quiet and observable
326+
327+
The engine invokes the listener once per strip pass that dropped at least one
328+
caller-supplied field. **The write still succeeds** and commits without those
329+
fields; the listener exists so callers that report per-field success (e.g. the
330+
flow engine's `update_record` step) can surface a warning instead of a silent
331+
success. Branch on `reason` exhaustively — it is an OPEN vocabulary that grows
332+
with the write path's legal strips, never a binary test.
333+
334+
#### `strictReadonlyWrites` — loud instead (#5126)
335+
336+
Default `false`. When `true`, a write whose payload WOULD have caller-supplied
337+
fields stripped **throws before the driver is touched** instead of committing the
338+
remainder. Nothing is written: not the stripped fields, not the fields that would
339+
have survived. The strip passes still run — that is how the engine learns WHICH
340+
fields would go — but their result is discarded.
341+
342+
Its coverage is DERIVED from what `onFieldsDropped` reports, not an enumeration
343+
frozen at #5126: every strip in the table above is refused, and a new `reason`
344+
adds a new refusal by construction. Covering only the static arm would leave a
345+
trusted caller — the very caller this option exists for, one that already passes
346+
`{ context: { isSystem: true } }` and is therefore exempt from the static strip —
347+
still losing `readonlyWhen` fields in silence. The flag's NAME is narrower than
348+
its coverage and stays that way on purpose; the coverage sentence, not the name,
349+
is the contract.
350+
351+
The refusal is `ReadonlyFieldRejectedError`, code `ERR_READONLY_FIELD_REJECTED`
352+
(registered in `ERROR_CODE_LEDGER` under `@objectstack/objectql`), carrying the
353+
FULL list of rejected fields accumulated across every strip pass the operation
354+
runs — one error naming everything, so a caller fixes its payload once instead of
355+
one round-trip per field. Catch it by `code`, not `instanceof`, and read `drops`
356+
for the per-reason breakdown; the code is stable across reasons deliberately, so
357+
adding a reason never adds an error code.
358+
359+
`onFieldsDropped` does **not** fire on a write this option refuses. The two are
360+
alternative outputs of one seam, not a sequence: `DroppedFieldsEvent` means
361+
"fields dropped and the write completed without them", and under strict the write
362+
does not complete. Quiet-and-observable or loud — pick one per call.
363+
364+
**On `insert`.** The two AUTHOR-DECLARED strips are deliberately insert-exempt at
365+
this seam (#3413: an in-process create may seed a `readonly: true` field's initial
366+
value, and `readonlyWhen` cannot lock anything on a create at all), so an insert
367+
refusal can only ever be about a runtime-owned value — a caller-supplied record
368+
number. With the option `true` that insert throws (`operation: 'insert'`) and
369+
nothing is written; without it the value is stripped, the write completes, and
370+
`onFieldsDropped` fires with `reason: 'readonly'`. The engine-level writers exempt
371+
from that strip — and therefore never refused — are the two the error message
372+
itself names: `isSystem`, and the `preserveAudit` historical import reinstating
373+
legacy record numbers (#3493).
374+
375+
<Callout type="warn">
376+
**Layering — this is the engine seam.** The exemption pair above is *this*
377+
in-process seam's. The DataProtocol ingress enforces its own author-declared
378+
`readonly` policy on create (#3043), where `preserveAudit` is UPDATE-only (#6640)
379+
and runtime-owned types are left to the engine strip — see `FieldSchema.readonly`.
380+
Nothing on this page widens or narrows that ingress policy.
381+
</Callout>
382+
293383
<Callout type="warn">
294-
`onFieldsDropped` is a **TS-contract-level, in-process-only** channel. It is
295-
deliberately not part of the serializable Zod options schemas: a function is
296-
unrepresentable in JSON Schema and cannot cross the RPC (Virtual Data Engine)
297-
boundary, so remote callers never receive these events. A listener that throws
298-
never breaks the write — the engine catches and logs.
384+
`WriteObservabilityOptions` is a **TS-contract-level, in-process-only** bag —
385+
both members. It is deliberately not part of the serializable Zod options schemas:
386+
a function is unrepresentable in JSON Schema and cannot cross the RPC (Virtual
387+
Data Engine) boundary, so remote callers never receive these events; and putting
388+
`strictReadonlyWrites` in the serializable bag would let any client toggle
389+
write-refusal on a security-adjacent path (#5126 ruling). A remote caller can set
390+
neither and gets NEITHER behaviour: its write is stripped and committed, silently
391+
from its side — a 200 whose read-only columns kept their stored values. Widening
392+
strict to the wire is a SEPARATE decision. A listener that throws never breaks the
393+
write — the engine catches and logs.
299394
</Callout>
300395

301396
### delete

0 commit comments

Comments
 (0)