Skip to content

Commit f5434b0

Browse files
hotlongclaude
andauthored
feat(verify): --rls runs one probe persona per declared POSITION, so app-authored narrowing is exercised (#7978) (#8067)
* feat(verify): per-declared-position RLS probe personas (#7978) The base probe persona holds no positions by construction, so an app policy carrying `positions: [...]` is never applicable to it — the app's own narrowing went unexercised while only the platform gate underneath it was proven. That is the authoring shape the real #7665 defect wore. `objectstack verify --rls` now fans out: one persona per position the app DECLARES (derived from `config.positions`, never a transcribed list), each holding that position and nothing else. Probe targets are established once and shared, each persona writes a distinct short marker, and a position that yielded no verdict is reported rather than dropped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B3Kurx8qufrDzNjk4rag7V * docs(changeset): verify --rls position personas (#7978) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B3Kurx8qufrDzNjk4rag7V * chore(gate): classify provisionRlsPositionPersona as NOT_A_STAND_IN (#7978) `check:verify-stand-in` discovers every exported `packages/verify` function whose first parameter is annotated with an interface the package declares and publishes, and fails until each is classified. `provisionRlsPositionPersona(stack: VerifyStack)` is the same class as its sibling `provisionRlsProbePersona`: it takes the handle `bootStack` returned and MINTS a persona through it — sign-up via that stack's auth route, one `sys_user_position` row through `stack.kernel`'s ObjectQL service. It checks nothing, so the parameter type is not the compile-time half of any conformance, and reaching the kernel is what a minimal structural surface could never carry. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B3Kurx8qufrDzNjk4rag7V --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 6ad13bb commit f5434b0

7 files changed

Lines changed: 800 additions & 67 deletions

File tree

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
---
2+
"@objectstack/verify": minor
3+
"@objectstack/cli": minor
4+
---
5+
6+
feat(verify): `--rls` runs one probe persona per DECLARED POSITION, so app-authored narrowing is exercised (#7978)
7+
8+
`objectstack verify --rls` proves one invariant — **you cannot mutate what you
9+
cannot see** (#1994). Since #7685 its probe persona authors its own capability
10+
(object read+edit, owner-scoped `select` only), which makes the *platform's*
11+
by-id write gate reachable. That persona holds **no positions** by construction,
12+
and a policy carrying `positions: [...]` is never applicable to a caller who does
13+
not hold one — so an app's own position-gated narrowing was never exercised, only
14+
the platform gate underneath it. That is exactly the authoring shape the real
15+
#7665 defect wore: an ordinary `contributor` against
16+
`positions: ['contributor']` rules.
17+
18+
## What changed
19+
20+
- **One persona per position the app DECLARES.** The set is derived from
21+
`config.positions` (`declaredPositionNames`), never a list kept in the
22+
verifier — a position added to an app is covered without touching this
23+
package. Each persona holds that position and nothing else, so its whole
24+
capability is what the app itself binds to it (`provisionRlsPositionPersona`
25+
writes one `sys_user_position` row; the built-in `everyone` / `guest` anchors
26+
are excluded, since no app declares them).
27+
- **Probe targets are established once and shared** by every persona, so a
28+
position costs 4 HTTP calls per object rather than re-deriving and re-creating
29+
a record per persona. Each persona mutates with a **distinct short marker**, so
30+
"did the row change" stays attributable — and short, because a probe field's
31+
`maxLength` would truncate a long marker into a false negative.
32+
- **Coverage is reported per position, never rolled into one number.**
33+
`RlsReport` gains `positionRuns[]` (one summary per position), `totals` (every
34+
persona's verdicts summed; the unit is one *object × persona* probe) and
35+
`positionCoverage` (`declared` vs `ran`, plus `notRun` for a declared position
36+
whose persona could not be provisioned, and a `note` when the app declares no
37+
positions at all — "nothing to run" must not read like "nothing to find").
38+
`summary` / `results` / `unproven` still describe the base persona exactly as
39+
before.
40+
- **`verify` counts a position persona's holes.** The exit contract reads
41+
`totals.holes`, and an unprovisionable declared position is a hard failure for
42+
the same reason a degraded base persona is: the run covered less than its
43+
numbers read.
44+
45+
## Measured, before → after
46+
47+
| | showcase | crm |
48+
|:--|:--|:--|
49+
| before | 23 probes: 20 proven (20 consistent, **0 holes**), 3 unproven (0 probe-blocked, 3 skipped) — exit 0 | 6 probes: 6 proven, 0 holes, 0 unproven — exit 0 |
50+
| after | 230 probes (base + 9 positions): 35 proven (33 consistent, **2 HOLES**), 195 unproven (54 member-visible, 111 probe-blocked, 30 skipped) — exit 1 | 24 probes (base + 3 positions): 6 proven, 0 holes, 18 unproven (18 probe-blocked) — exit 0 |
51+
52+
Cost: showcase 22s → 50s, crm 10s → 12s (`dogfood-verify` budget is 20 min).
53+
54+
**The two showcase holes are real and are NOT fixed here** — filed as #8059. A
55+
`contributor` reads `GET 404` on a `showcase_invoice` and still PATCHes it by id;
56+
the app's check-only `update` policy suppresses #7665's write-scope derivation,
57+
and the post-image `check` that should have caught it is dropped for
58+
position-scoped callers. Tuning the probe to keep the run green is the one thing
59+
this verifier must never do, so `verify --rls` on the showcase now exits 1 until
60+
#8059 lands.
61+
62+
The new personas are falsifiable, not decorative: ablating the #7665 write-scope
63+
derivation flips the `contributor` persona's `showcase_task` from
64+
`rls-consistent` to `rls-hole` (and the base persona's 16, unchanged from #7685's
65+
measurement).
66+
67+
## No behaviour change outside the verifier
68+
69+
Tooling only: no runtime, spec or enforcement path is touched. `runRlsProofs`'
70+
existing call shape still works, and consumers that only read the report gain
71+
fields rather than losing any.

packages/cli/src/commands/verify.ts

Lines changed: 37 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,10 +11,13 @@ import {
1111
runRlsProofs,
1212
formatRlsReport,
1313
provisionRlsProbePersona,
14+
provisionRlsPositionPersona,
15+
declaredPositionNames,
1416
rlsProbeSecurity,
1517
type VerifyReport,
1618
type RlsReport,
1719
type RlsProbeDescriptor,
20+
type RlsPositionPersonaInput,
1821
} from '@objectstack/verify';
1922
import { loadConfig } from '../utils/config.js';
2023

@@ -138,7 +141,31 @@ export default class Verify extends Command {
138141
degraded: `probe persona provisioning failed: ${(e as Error).message}`,
139142
};
140143
}
141-
rls = await runRlsProofs(rlsStack, adminToken, probeToken, config, { probe });
144+
// [#7978] The base persona holds no positions by construction, so an
145+
// app policy carrying `positions: [...]` is never applicable to it and
146+
// the app's OWN narrowing goes unexercised. Mint one persona per
147+
// DECLARED position — derived from the config, never a list written
148+
// here — so the position-gated half is probed too. Provisioning lives
149+
// on this side because it needs the live stack; the runner re-derives
150+
// the intended reach from the config, so a position missing from this
151+
// loop reports as `positionCoverage.notRun` instead of quietly
152+
// shrinking the run.
153+
const positionPersonas: RlsPositionPersonaInput[] = [];
154+
const positionFailures: Array<{ position: string; error: string }> = [];
155+
for (const position of declaredPositionNames(config)) {
156+
try {
157+
const persona = await provisionRlsPositionPersona(rlsStack, position);
158+
positionPersonas.push({ position, token: persona.token, label: persona.email });
159+
} catch (e) {
160+
positionFailures.push({ position, error: (e as Error).message });
161+
}
162+
}
163+
164+
rls = await runRlsProofs(rlsStack, adminToken, probeToken, config, {
165+
probe,
166+
positionPersonas,
167+
positionFailures,
168+
});
142169
} finally {
143170
await rlsStack.stop();
144171
}
@@ -147,12 +174,19 @@ export default class Verify extends Command {
147174
// Failure contract: a "real" runtime break the app's author must see.
148175
// A degraded RLS probe counts: the run reported verdicts it could not have
149176
// established, which is worse than no verifier at all.
177+
//
178+
// [#7978] `totals`, not `summary`: a hole a POSITION persona found is
179+
// exactly as real as one the base persona found — reading `summary` here
180+
// would run the fan-out and then throw its findings away. A declared
181+
// position that could not be provisioned counts for the same reason a
182+
// degraded base persona does: the run covered less than its numbers read.
150183
const hardFailures =
151184
crud.summary.createFailed +
152185
crud.summary.readFailed +
153186
crud.summary.fidelityGaps +
154-
(rls?.summary.holes ?? 0) +
155-
(rls?.probe.degraded ? 1 : 0);
187+
(rls?.totals.holes ?? 0) +
188+
(rls?.probe.degraded ? 1 : 0) +
189+
(rls?.positionCoverage.notRun.length ?? 0);
156190

157191
if (flags.json) {
158192
this.log(JSON.stringify({ app: crud.app, config: absolutePath, multiTenant, crud, rls, hardFailures }, null, 2));

packages/qa/dogfood/test/rls-runner.test.ts

Lines changed: 174 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -20,16 +20,35 @@
2020
// verdicts, i.e. green that no platform change could have turned red.
2121
// • target adoption — an admin create the app's own validation rejects no
2222
// longer cascades its dependents out of the run.
23+
//
24+
// [#7978] And by the POSITION fan-out, which this file is the liveness oracle
25+
// for in exactly the same sense: the live apps report 0 holes for every position
26+
// persona too, so only a scripted stack can answer "can a position persona still
27+
// SAY `rls-hole`". A fan-out that cannot fail is decoration, and this file is
28+
// what stops it becoming that.
2329

2430
import { describe, it, expect } from 'vitest';
25-
import { runRlsProofs } from '@objectstack/verify';
31+
import { runRlsProofs, declaredPositionNames } from '@objectstack/verify';
2632
import type { VerifyStack } from '@objectstack/verify';
2733

2834
const CONFIG = {
2935
manifest: { id: 'fixture' },
3036
objects: [{ name: 'note', fields: { name: { type: 'text', required: true } } }],
3137
};
3238

39+
/** [#7978] The same app, declaring one position — the fan-out's input. */
40+
const CONFIG_WITH_POSITION = {
41+
...CONFIG,
42+
positions: [{ name: 'contributor', label: 'Contributor' }],
43+
};
44+
45+
/** How one persona behaves. The member token uses the top-level scenario fields. */
46+
interface PersonaScript {
47+
canRead: boolean;
48+
writeMutates: boolean;
49+
objectGateDenies?: boolean;
50+
}
51+
3352
interface FakeOpts {
3453
memberCanRead: boolean;
3554
memberWriteMutates: boolean; // does member's PATCH actually change the row?
@@ -39,6 +58,10 @@ interface FakeOpts {
3958
adminCreateStatus?: number;
4059
/** Rows that already exist on the object, as seed data would. */
4160
seeded?: Array<Record<string, unknown>>;
61+
/** [#7978] Per-token scripts — one per position persona. */
62+
personas?: Record<string, PersonaScript>;
63+
/** [#7978] Call log, so cost and marker-distinctness are assertable. */
64+
calls?: { posts: number; patches: Array<{ token: string; value: unknown }> };
4265
}
4366

4467
/** A fake stack: admin always sees/owns; member behaviour is scripted per scenario. */
@@ -47,18 +70,26 @@ function fakeStack(opts: FakeOpts): VerifyStack {
4770
for (const row of opts.seeded ?? []) store[String(row.id)] = { ...row };
4871
const json = (body: unknown, status = 200) =>
4972
new Response(JSON.stringify(body), { status, headers: { 'Content-Type': 'application/json' } });
73+
const scriptFor = (token: string): PersonaScript =>
74+
opts.personas?.[token] ?? {
75+
canRead: opts.memberCanRead,
76+
writeMutates: opts.memberWriteMutates,
77+
objectGateDenies: opts.objectGateDenies,
78+
};
5079

5180
const apiAs: VerifyStack['apiAs'] = async (token, method, path, body) => {
5281
const isAdmin = token === 'admin';
82+
const script = scriptFor(token);
5383
// `/data/<object>[?query]` (list) or `/data/<object>/<id>` (by id)
5484
const [, , objectSegment, id] = path.split('/');
5585
const object = String(objectSegment).split('?')[0];
5686

5787
// The object-level gate answers first, for every verb — exactly what made
5888
// the grant-less persona's verdicts meaningless (#7685).
59-
if (!isAdmin && opts.objectGateDenies) return json({ code: 'PERMISSION_DENIED' }, 403);
89+
if (!isAdmin && script.objectGateDenies) return json({ code: 'PERMISSION_DENIED' }, 403);
6090

6191
if (method === 'POST') {
92+
if (opts.calls) opts.calls.posts += 1;
6293
if (opts.adminCreateStatus) return json({ error: 'VALIDATION_FAILED' }, opts.adminCreateStatus);
6394
const newId = 'rec1';
6495
store[newId] = { id: newId, ...(body as object) };
@@ -68,17 +99,18 @@ function fakeStack(opts: FakeOpts): VerifyStack {
6899
if (id === undefined) {
69100
// LIST — the runner's reachability probe, and the admin-side source the
70101
// cascade-stopper adopts a target from.
71-
if (!isAdmin && !opts.memberCanRead) return json({ records: [] });
102+
if (!isAdmin && !script.canRead) return json({ records: [] });
72103
return json({ records: Object.values(store) });
73104
}
74-
if (!isAdmin && !opts.memberCanRead) return json({ error: 'not found' }, 404);
105+
if (!isAdmin && !script.canRead) return json({ error: 'not found' }, 404);
75106
return json({ object, id, record: store[id] ?? null });
76107
}
77108
if (method === 'PATCH') {
109+
if (opts.calls && !isAdmin) opts.calls.patches.push({ token, value: (body as any)?.name });
78110
// Admin always writes. Member writes only "land" when the scenario says so
79111
// (i.e. RLS failed to scope the by-id write — the #1994 bug).
80-
if (isAdmin || opts.memberWriteMutates) Object.assign(store[id], body as object);
81-
return json({ object, id, record: store[id] }, isAdmin || opts.memberWriteMutates ? 200 : 403);
112+
if (isAdmin || script.writeMutates) Object.assign(store[id], body as object);
113+
return json({ object, id, record: store[id] }, isAdmin || script.writeMutates ? 200 : 403);
82114
}
83115
return json({}, 405);
84116
};
@@ -176,3 +208,139 @@ describe('[#7685] an unsatisfiable admin create does not cascade objects out of
176208
expect(report.summary.proven).toBe(0);
177209
});
178210
});
211+
212+
describe('[#7978] declaredPositionNames — the fan-out reads the app, never a list', () => {
213+
it('derives the position names from the config, in order, deduplicated', () => {
214+
expect(
215+
declaredPositionNames({
216+
positions: [{ name: 'contributor' }, { name: 'manager' }, { name: 'contributor' }],
217+
}),
218+
).toEqual(['contributor', 'manager']);
219+
});
220+
221+
it('covers a position added to the app WITHOUT any change here — the point of deriving', () => {
222+
const tomorrow = { positions: [{ name: 'contributor' }, { name: 'field_ops_delegate' }] };
223+
expect(declaredPositionNames(tomorrow)).toContain('field_ops_delegate');
224+
});
225+
226+
it('excludes the built-in audience anchors and tolerates an app with no positions', () => {
227+
// No app declares `everyone`/`guest` (ADR-0090 D5/D9): every authenticated
228+
// member already holds `everyone` — the base persona's own baseline — and
229+
// `guest` is the anonymous audience no signed-up persona can hold.
230+
expect(declaredPositionNames({ positions: [{ name: 'everyone' }, { name: 'guest' }, { name: 'ops' }] }))
231+
.toEqual(['ops']);
232+
expect(declaredPositionNames({})).toEqual([]);
233+
expect(declaredPositionNames(undefined)).toEqual([]);
234+
});
235+
});
236+
237+
describe('[#7978] a POSITION persona can still say `rls-hole` — fan-out detector liveness', () => {
238+
it('flags a HOLE found by a position persona, and rolls it into `totals` where the CLI reads it', async () => {
239+
// The base persona is clean; only the persona holding the app's position
240+
// can see the defect, because only its policies are position-gated. This is
241+
// the whole class #7978 exists to reach — and the assertion that keeps the
242+
// fan-out falsifiable rather than decorative.
243+
const stack = fakeStack({
244+
memberCanRead: false,
245+
memberWriteMutates: false,
246+
personas: { 'tok-contributor': { canRead: false, writeMutates: true } },
247+
});
248+
const report = await runRlsProofs(stack, 'admin', 'member', CONFIG_WITH_POSITION, {
249+
positionPersonas: [{ position: 'contributor', token: 'tok-contributor', label: 'pos@test' }],
250+
});
251+
252+
expect(report.summary.holes).toBe(0); // base persona: consistent
253+
expect(report.positionRuns).toHaveLength(1);
254+
expect(report.positionRuns[0].position).toBe('contributor');
255+
expect(report.positionRuns[0].results[0].status).toBe('rls-hole');
256+
expect(report.positionRuns[0].summary.holes).toBe(1);
257+
// `totals` is what `objectstack verify` counts as a hard failure: a hole a
258+
// position persona found is exactly as real as one the base persona found.
259+
expect(report.totals.holes).toBe(1);
260+
expect(report.totals.proven).toBe(2); // base consistent + position hole
261+
expect(report.positionCoverage).toMatchObject({ declared: ['contributor'], ran: ['contributor'], notRun: [] });
262+
});
263+
264+
it('reports probe-blocked — NOT a pass — when the object gate refuses a position persona', async () => {
265+
// A declared position the app binds no object grants to. Honest: the
266+
// by-id-write class was not exercised for it, and it must not read as reach.
267+
const stack = fakeStack({
268+
memberCanRead: false,
269+
memberWriteMutates: false,
270+
personas: { 'tok-finance': { canRead: false, writeMutates: true, objectGateDenies: true } },
271+
});
272+
const report = await runRlsProofs(stack, 'admin', 'member', CONFIG_WITH_POSITION, {
273+
positionPersonas: [{ position: 'contributor', token: 'tok-finance', label: 'pos@test' }],
274+
});
275+
expect(report.positionRuns[0].results[0].status).toBe('probe-blocked');
276+
expect(report.positionRuns[0].summary).toMatchObject({ consistent: 0, proven: 0, probeBlocked: 1, unproven: 1 });
277+
expect(report.positionRuns[0].unproven.map((u) => u.object)).toEqual(['note']);
278+
expect(report.totals.holes).toBe(0);
279+
expect(report.totals.proven).toBe(1); // the base persona's verdict, and only that
280+
});
281+
282+
it('gives each persona a DISTINCT marker and creates the probe target only once', async () => {
283+
// Two properties in one run. Shared targets are what keeps the fan-out
284+
// affordable (one admin create for N personas, not N). Distinct markers are
285+
// what keeps it correct: with one shared marker, persona 2's refused write
286+
// would re-read persona 1's successful mutation as its own — a fabricated
287+
// hole on a platform that is behaving.
288+
const calls = { posts: 0, patches: [] as Array<{ token: string; value: unknown }> };
289+
const stack = fakeStack({
290+
memberCanRead: false,
291+
memberWriteMutates: false,
292+
calls,
293+
personas: {
294+
'tok-a': { canRead: false, writeMutates: true },
295+
'tok-b': { canRead: false, writeMutates: false },
296+
},
297+
});
298+
const report = await runRlsProofs(stack, 'admin', 'member', CONFIG_WITH_POSITION, {
299+
positionPersonas: [
300+
{ position: 'contributor', token: 'tok-a', label: 'a@test' },
301+
{ position: 'manager', token: 'tok-b', label: 'b@test' },
302+
],
303+
});
304+
305+
expect(calls.posts).toBe(1);
306+
const markers = calls.patches.map((p) => p.value);
307+
expect(new Set(markers).size).toBe(markers.length);
308+
// `tok-b` writes after `tok-a` mutated the row, and is still judged correctly.
309+
expect(report.positionRuns[0].results[0].status).toBe('rls-hole');
310+
expect(report.positionRuns[1].results[0].status).toBe('rls-consistent');
311+
});
312+
});
313+
314+
describe('[#7978] a position that produced no verdict never reads as a pass', () => {
315+
it('says so distinctly when the app declares NO positions — and adds nothing to `proven`', async () => {
316+
const stack = fakeStack({ memberCanRead: false, memberWriteMutates: false });
317+
const report = await runRlsProofs(stack, 'admin', 'member', CONFIG);
318+
expect(report.positionRuns).toEqual([]);
319+
expect(report.positionCoverage.declared).toEqual([]);
320+
expect(report.positionCoverage.note).toMatch(/no position personas to run/);
321+
// No silent contribution: the totals are exactly the base persona's.
322+
expect(report.totals).toEqual(report.summary);
323+
});
324+
325+
it('records a declared position whose persona could not be provisioned, with the reason', async () => {
326+
const stack = fakeStack({ memberCanRead: false, memberWriteMutates: false });
327+
const report = await runRlsProofs(stack, 'admin', 'member', CONFIG_WITH_POSITION, {
328+
positionFailures: [{ position: 'contributor', error: 'no sys_user row for pos@test' }],
329+
});
330+
expect(report.positionRuns).toEqual([]);
331+
expect(report.positionCoverage.notRun).toEqual([
332+
{ position: 'contributor', reason: 'no sys_user row for pos@test' },
333+
]);
334+
// No `note` here: the app DOES declare positions, so "none ran" is a gap,
335+
// not the app-declares-nothing case.
336+
expect(report.positionCoverage.note).toBeUndefined();
337+
});
338+
339+
it('reports a declared position the caller simply never provisioned a persona for', async () => {
340+
const stack = fakeStack({ memberCanRead: false, memberWriteMutates: false });
341+
const report = await runRlsProofs(stack, 'admin', 'member', CONFIG_WITH_POSITION);
342+
expect(report.positionCoverage.notRun).toHaveLength(1);
343+
expect(report.positionCoverage.notRun[0]).toMatchObject({ position: 'contributor' });
344+
expect(report.positionCoverage.notRun[0].reason).toMatch(/NOT exercised/);
345+
});
346+
});

0 commit comments

Comments
 (0)