Skip to content

Commit 97b6658

Browse files
os-helpclaude
andauthored
docs(spec): fix QA field/capture path convention — no body. prefix (#7365) (#7565)
TestAssertionSchema.field's describe() text and TestStepSchema.capture's sibling describe() both taught a `body.*` path convention that never matched the runtime: TestRunner resolves both against `result` directly, the value HttpTestAdapter.handleResponse returns (the parsed response body itself, no `body` wrapper). Executes the 2026-08-11 maintainer ruling on #7365 (comment 5248467805): docs follow the adapter, root- relative paths, no stored-suite compatibility to preserve since body.* never worked. Acceptance face unchanged - field and capture keep their original Zod types, only description text moves. Regenerates content/docs/references/qa/testing.mdx via `pnpm --filter @objectstack/spec gen:docs`; check:docs reports all 231 files in sync. Adds a patch changeset for @objectstack/spec, following the #7444 precedent for describe/TSDoc-only spec docs fixes. Claude-Session: https://claude.ai/code/session_016R9de1FqP7NvwKvqXi92Gh Co-authored-by: Claude <noreply@anthropic.com>
1 parent 34c01a5 commit 97b6658

3 files changed

Lines changed: 45 additions & 4 deletions

File tree

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
---
2+
"@objectstack/spec": patch
3+
---
4+
5+
docs(spec): fix the QA `field`/`capture` path convention — no `body.` prefix (#7365)
6+
7+
`TestAssertionSchema.field`'s `.describe()` text read `'Field path in the
8+
result to check (e.g. "body.data.0.status")'`, and `TestStepSchema.capture`'s
9+
sibling `.describe()` repeated the same `body.*` example. Neither convention
10+
ever matched the runtime: `TestRunner.assert`/`runStep` resolve both paths
11+
against `result` directly — the value `HttpTestAdapter.handleResponse`
12+
returns, which is the parsed response body itself with no `body` wrapper (nor
13+
does the platform's own response envelope, `data`/`meta`, ever nest under a
14+
`body` key). A suite written to the documented convention resolved every path
15+
to `undefined`.
16+
17+
Filed as #7365 (observation-class finding from #7256's blast radius): with
18+
`equals`-class operators a `body.*` path already failed loudly, so an author
19+
worked the real convention out by trial; with `contains`, `undefined` fell out
20+
of the switch and the assertion silently passed, so a `body.*` `contains`
21+
reported green forever. #7256 (PR #7348) turned that silent pass into a loud
22+
failure, which is correct, but it meant an author following the schema's own
23+
example now hits a error that never says the *documentation* is wrong.
24+
25+
Maintainer ruling, 2026-08-11 (issue comment 5248467805): "docs follow the
26+
adapter" — fix the `describe()` text (and the regenerated reference) to the
27+
real convention, root-relative, no `body.` wrapper. `body.*` never worked, so
28+
there is no stored-suite compatibility to preserve. The acceptance face is
29+
unchanged — `field` and `capture` both stay their original Zod types; only the
30+
description text moves.
31+
32+
Both faces now read, in the file's existing one-sentence-plus-example style:
33+
34+
- `field`: `'Field path in the result to check, resolved against the parsed
35+
response body root — no "body." prefix (e.g. "data.0.status")'`
36+
- `capture`: `'Map result fields to context variables, paths resolved against
37+
the response body root (e.g. { "newId": "data.id" })'`
38+
39+
`content/docs/references/qa/testing.mdx` is regenerated to match
40+
(`pnpm --filter @objectstack/spec gen:docs`); `check:docs` reports all 231
41+
generated files in sync.

content/docs/references/qa/testing.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ A test assertion that validates the result of a test action
6363

6464
| Property | Type | Required | Description |
6565
| :--- | :--- | :--- | :--- |
66-
| **field** | `string` || Field path in the result to check (e.g. "body.data.0.status") |
66+
| **field** | `string` || Field path in the result to check, resolved against the parsed response body root — no "body." prefix (e.g. "data.0.status") |
6767
| **operator** | `Enum<'equals' \| 'not_equals' \| 'contains' \| 'not_contains' \| 'is_null' \| 'not_null' \| 'gt' \| 'gte' \| 'lt' \| 'lte' \| 'error'>` || Comparison operator to use |
6868
| **expectedValue** | `any` || Expected value to compare against |
6969

@@ -126,7 +126,7 @@ A single step in a test scenario, consisting of an action and optional assertion
126126
| **description** | `string` | optional | Human-readable description of what this step tests |
127127
| **action** | `{ type: Enum<'create_record' \| 'update_record' \| 'delete_record' \| 'read_record' \| … +4 more>; target: string; payload?: Record<string, any>; user?: string }` || The action to execute in this step |
128128
| **assertions** | `{ field: string; operator: Enum<'equals' \| 'not_equals' \| 'contains' \| 'not_contains' \| 'is_null' \| 'not_null' \| … +5 more>; expectedValue: any }[]` | optional | Assertions to validate after the action completes |
129-
| **capture** | `Record<string, string>` | optional | Map result fields to context variables: `{ "newId": "body.id" }` |
129+
| **capture** | `Record<string, string>` | optional | Map result fields to context variables, paths resolved against the response body root (e.g. `{ "newId": "data.id" }`) |
130130

131131

132132
---

packages/spec/src/qa/testing.zod.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ export const TestAssertionTypeSchema = lazySchema(() => z.enum([
4242
]).describe('Comparison operator for test assertions'));
4343

4444
export const TestAssertionSchema = lazySchema(() => z.object({
45-
field: z.string().describe('Field path in the result to check (e.g. "body.data.0.status")'),
45+
field: z.string().describe('Field path in the result to check, resolved against the parsed response body root — no "body." prefix (e.g. "data.0.status")'),
4646
operator: TestAssertionTypeSchema.describe('Comparison operator to use'),
4747
expectedValue: z.unknown().describe('Expected value to compare against')
4848
}).describe('A test assertion that validates the result of a test action'));
@@ -55,7 +55,7 @@ export const TestStepSchema = lazySchema(() => z.object({
5555
action: TestActionSchema.describe('The action to execute in this step'),
5656
assertions: z.array(TestAssertionSchema).optional().describe('Assertions to validate after the action completes'),
5757
// Capture outputs to variables for subsequent steps
58-
capture: z.record(z.string(), z.string()).optional().describe('Map result fields to context variables: { "newId": "body.id" }')
58+
capture: z.record(z.string(), z.string()).optional().describe('Map result fields to context variables, paths resolved against the response body root (e.g. { "newId": "data.id" })')
5959
}).describe('A single step in a test scenario, consisting of an action and optional assertions'));
6060

6161
export const TestScenarioSchema = lazySchema(() => z.object({

0 commit comments

Comments
 (0)