Skip to content

Commit dc61def

Browse files
os-zhuangclaude
andauthored
feat(spec): a skill trigger condition's value must have the shape its operator reads (#7113) (#7212)
`SkillTriggerConditionSchema.operator` and `.value` were declared independently, so every operator accepted every shape: `{ operator: 'in', value: 'admin' }` — a membership test whose list is not a list — was spec-valid. The dormant twin of #6227 on `ViewFilterRuleSchema`; the fix mirrors that one (PR #7114) key for key. Dormant is the point: the sole consumer (`SkillRegistry.evaluateCondition`, cloud agent runtime) coerces the scalar itself, so nothing ever failed. What is closed is a second dialect — a consumer-side lenient coercion standing in for a contract the producer never declared. That coercion becomes a no-op here; removing it is a producer-first follow-up in the cloud repo. - `in` / `not_in` (SKILL_TRIGGER_LIST_VALUE_OPERATORS) require an array. - `eq` / `neq` (SKILL_TRIGGER_SCALAR_VALUE_OPERATORS) require a string — `===` on an array is reference identity, so an array comparand is a dead predicate. - `contains` is deliberately unchanged: it has two live branches (string substring, array subset), and #5685 rules against a schema stricter than its runtime. Both vocabularies are exported so producers enumerate from the contract. Authoring impact censused first across this repo and the cloud repo: no real skill authors the scalar-on-set-operator form. Claude-Session: https://claude.ai/code/session_01HCb6mPxnEjvhKnnka1RNxw Co-authored-by: Claude <noreply@anthropic.com>
1 parent 424c510 commit dc61def

6 files changed

Lines changed: 449 additions & 4 deletions

File tree

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
---
2+
"@objectstack/spec": minor
3+
---
4+
5+
feat(spec): a skill trigger condition's `value` must have the shape its OPERATOR reads (#7113)
6+
7+
`SkillTriggerConditionSchema.operator` and `.value` were declared independently
8+
`z.enum(['eq','neq','in','not_in','contains'])` beside
9+
`z.union([z.string(), z.array(z.string())])` — so every operator accepted every
10+
shape. `{ field: 'userRole', operator: 'in', value: 'admin' }` was a spec-valid
11+
skill trigger: a membership test whose list is not a list.
12+
13+
This is the **dormant twin** of #6227 on `ViewFilterRuleSchema`, and the fix
14+
mirrors that one (PR #7114) key for key — the exported operator vocabularies,
15+
the `superRefine`, the single issue at path `['value']`.
16+
17+
**Why "dormant" is the whole point.** #6227's shape genuinely failed at query
18+
time (`assertListComparandShapes`, 400 `INVALID_FILTER`), which made it a
19+
two-stage failure. This one never failed at all: the sole consumer,
20+
`SkillRegistry.evaluateCondition` in the cloud agent runtime, coerces the scalar
21+
itself with `Array.isArray(expected) ? expected : [expected]`. Nothing 400s and
22+
the predicate evaluates the way the author meant. What is being closed is
23+
therefore not a break but a **second dialect** — a consumer-side lenient
24+
coercion standing in for a contract the producer never declared, on a surface
25+
whose authors are increasingly AI-generated, where "declared = enforced" is what
26+
keeps generated metadata honest. That coercion becomes a no-op once this ships;
27+
removing it is a follow-up in the cloud repo, producer-first.
28+
29+
**The constraint, and its deliberate limit:**
30+
31+
| operator | `value` must be | why |
32+
|---|---|---|
33+
| `in` / `not_in` (`SKILL_TRIGGER_LIST_VALUE_OPERATORS`) | an array, any length | the consumer answers them with `list.includes(fieldValue)` — the authored value IS the list |
34+
| `eq` / `neq` (`SKILL_TRIGGER_SCALAR_VALUE_OPERATORS`) | a string | `===` / `!==` on an array is reference identity, so an array comparand is a DEAD predicate: `eq` never fires, `neq` always does |
35+
| `contains` | **unchanged — either shape** | it has two live branches: string∈string substring, and array⊆array subset (`expected.every(v => fieldValue.includes(v))`) |
36+
37+
`contains` is left alone on purpose. #5685 ruled on the opposite error — a
38+
schema stricter than its runtime in ways the runtime deliberately allows — and
39+
`SkillContext` is indexed `[extraField: string]: unknown`, so an array-valued
40+
context field is a shape the consumer is written for. Refusing it here would
41+
un-declare a working capability, which is an ADR-0049 retirement decision and
42+
not a rider on a shape fix.
43+
44+
Both vocabularies are **exported** so a producer — a condition editor, a
45+
generator, a test — asks the question the schema asks instead of keeping its own
46+
copy of the list, the same reason `VIEW_FILTER_LIST_VALUE_OPERATORS` is exported
47+
one module over.
48+
49+
**Authoring impact: measured, not assumed.** Censused before landing across this
50+
repo (`packages/`, `examples/`, `content/`, `docs/`) and the cloud repo
51+
(`packages/service-ai` skill definitions, seeds, fixtures, docs corpora): no
52+
real (non-test) skill authors `triggerConditions` in the scalar-on-set-operator
53+
form. The framework's six built-in skills declare no `triggerConditions` at all,
54+
and both authored examples already use the array form on `in`. The one in-repo
55+
test that handed a scalar to all five operators was asserting the decoupling
56+
itself and is updated to enumerate the shape each operator reads.

packages/spec/api-surface/ai.json

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,8 @@
126126
"PromptVariable (type)",
127127
"PromptVariableParsed (type)",
128128
"PromptVariableSchema (const)",
129+
"SKILL_TRIGGER_LIST_VALUE_OPERATORS (const)",
130+
"SKILL_TRIGGER_SCALAR_VALUE_OPERATORS (const)",
129131
"Skill (type)",
130132
"SkillParsed (type)",
131133
"SkillSchema (const)",

packages/spec/export-origins/ai.json

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,8 @@
126126
"PromptVariable": "src/ai/model-registry.zod.ts#PromptVariable (type)",
127127
"PromptVariableParsed": "src/ai/model-registry.zod.ts#PromptVariableParsed (type)",
128128
"PromptVariableSchema": "src/ai/model-registry.zod.ts#PromptVariableSchema (const)",
129+
"SKILL_TRIGGER_LIST_VALUE_OPERATORS": "src/ai/skill.zod.ts#SKILL_TRIGGER_LIST_VALUE_OPERATORS (const)",
130+
"SKILL_TRIGGER_SCALAR_VALUE_OPERATORS": "src/ai/skill.zod.ts#SKILL_TRIGGER_SCALAR_VALUE_OPERATORS (const)",
129131
"Skill": "src/ai/skill.zod.ts#Skill (type)",
130132
"SkillParsed": "src/ai/skill.zod.ts#SkillParsed (type)",
131133
"SkillSchema": "src/ai/skill.zod.ts#SkillSchema (const)",
Lines changed: 205 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,205 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [#7113] `SkillTriggerConditionSchema.value` is shaped by the condition's
5+
* OPERATOR — the dormant twin of #6227 (`ViewFilterRuleSchema`, PR #7114).
6+
*
7+
* "Dormant" is the whole difference and these pins are written around it. The
8+
* #6227 shape genuinely failed at query time; this one never failed at all —
9+
* the sole consumer (`SkillRegistry.evaluateCondition`, cloud
10+
* `packages/service-ai/src/skill-registry.ts`) coerces the scalar with
11+
* `Array.isArray(expected) ? expected : [expected]`. So what these pins hold is
12+
* not a break-fix but the contract-first property: the producer declares the
13+
* one spelling instead of letting a consumer quietly accept two.
14+
*
15+
* Every rejection pin asserts the issue CODE and PATH, not merely that a throw
16+
* happened: a bare `.toThrow()` cannot tell "refused for the right reason at
17+
* the right key" from "refused because the value union rejected the type", and
18+
* those are different defects (#6142).
19+
*
20+
* The accept pins matter as much as the reject pins. `contains` keeps BOTH
21+
* spellings on purpose — the consumer has a live array⊆array branch for it —
22+
* and #5685 rules that a schema stricter than its runtime is the wrong side of
23+
* the fix. A pin that only checked rejections would let that regress silently.
24+
*/
25+
26+
import { describe, expect, it } from 'vitest';
27+
import {
28+
SKILL_TRIGGER_LIST_VALUE_OPERATORS,
29+
SKILL_TRIGGER_SCALAR_VALUE_OPERATORS,
30+
SkillSchema,
31+
SkillTriggerConditionSchema,
32+
} from './skill.zod';
33+
34+
/** Parse helper — the authored object form, exactly as a skill carries it. */
35+
const parse = (condition: Record<string, unknown>) =>
36+
SkillTriggerConditionSchema.safeParse(condition);
37+
38+
/** The single `value`-path issue a shape refusal must produce. */
39+
function valueIssue(result: ReturnType<typeof parse>) {
40+
expect(result.success).toBe(false);
41+
if (result.success) throw new Error('unreachable');
42+
const issues = result.error.issues.filter((i) => i.path.join('.') === 'value');
43+
expect(issues).toHaveLength(1);
44+
return issues[0]!;
45+
}
46+
47+
describe('#7113 — the reported shape is refused at authoring time', () => {
48+
it('refuses the card example: a set operator carrying a scalar', () => {
49+
const result = parse({ field: 'userRole', operator: 'in', value: 'admin' });
50+
const issue = valueIssue(result);
51+
52+
expect(issue.code).toBe('custom');
53+
expect(issue.path).toEqual(['value']);
54+
expect(issue.message).toContain(
55+
'Operator "in" on field "userRole" requires an ARRAY of values.',
56+
);
57+
// The refusal carries what the author has to DO, not just what is wrong.
58+
expect(issue.message).toContain('Received a string ("admin")');
59+
expect(issue.message).toContain('write ["admin"] for a single value');
60+
expect(issue.message).toContain('or use "eq" to compare against it');
61+
// And it says the empty list is NOT what is being refused.
62+
expect(issue.message).toContain('An empty list [] is allowed');
63+
});
64+
65+
it('names the consumer-side coercion as the thing being replaced', () => {
66+
const issue = valueIssue(parse({ field: 'userRole', operator: 'not_in', value: 'admin' }));
67+
expect(issue.message).toContain('coerces the scalar today');
68+
expect(issue.message).toContain('#7113');
69+
});
70+
});
71+
72+
describe('#7113 — list operators require an array', () => {
73+
it.each(SKILL_TRIGGER_LIST_VALUE_OPERATORS)('%s refuses a scalar', (operator) => {
74+
const issue = valueIssue(parse({ field: 'objectName', operator, value: 'lead' }));
75+
expect(issue.code).toBe('custom');
76+
expect(issue.path).toEqual(['value']);
77+
expect(issue.message).toContain(`Operator "${operator}"`);
78+
expect(issue.message).toContain('requires an ARRAY of values');
79+
});
80+
81+
it.each(SKILL_TRIGGER_LIST_VALUE_OPERATORS)('%s accepts an array', (operator) => {
82+
const result = parse({ field: 'objectName', operator, value: ['lead', 'opportunity'] });
83+
expect(result.success).toBe(true);
84+
});
85+
86+
it.each(SKILL_TRIGGER_LIST_VALUE_OPERATORS)(
87+
'%s accepts an EMPTY array — it is a real predicate, not the defect',
88+
(operator) => {
89+
expect(parse({ field: 'objectName', operator, value: [] }).success).toBe(true);
90+
},
91+
);
92+
93+
it('refuses a missing value with ONE issue — the required check, not two', () => {
94+
// Measured, not assumed: Zod 4 skips a `superRefine` when the object's own
95+
// shape already failed, so an omitted `value` reports only the required
96+
// issue. Pinned because the refinement's "no value" wording exists for the
97+
// case where a future carrier makes `value` optional — this records that
98+
// today it is unreachable, rather than leaving a reader to guess that a
99+
// missing value produces two competing complaints at one key.
100+
const result = parse({ field: 'objectName', operator: 'in' });
101+
expect(result.success).toBe(false);
102+
if (result.success) throw new Error('unreachable');
103+
const atValue = result.error.issues.filter((i) => i.path.join('.') === 'value');
104+
expect(atValue).toHaveLength(1);
105+
expect(atValue[0]!.code).not.toBe('custom');
106+
});
107+
});
108+
109+
describe('#7113 — identity operators require a string', () => {
110+
it.each(SKILL_TRIGGER_SCALAR_VALUE_OPERATORS)('%s refuses an array', (operator) => {
111+
const issue = valueIssue(parse({ field: 'objectName', operator, value: ['lead'] }));
112+
expect(issue.code).toBe('custom');
113+
expect(issue.path).toEqual(['value']);
114+
expect(issue.message).toContain(`Operator "${operator}"`);
115+
expect(issue.message).toContain('requires a single STRING value');
116+
// The message must explain the DEAD-predicate mechanism, since nothing
117+
// errors today — an author has no runtime symptom to reason from.
118+
expect(issue.message).toContain(operator === 'eq' ? 'never fire' : 'always fire');
119+
expect(issue.message).toContain(operator === 'eq' ? 'use "in"' : 'use "not_in"');
120+
});
121+
122+
it.each(SKILL_TRIGGER_SCALAR_VALUE_OPERATORS)('%s accepts a string', (operator) => {
123+
expect(parse({ field: 'objectName', operator, value: 'lead' }).success).toBe(true);
124+
});
125+
});
126+
127+
describe('#7113 — `contains` keeps BOTH shapes (#5685: no stricter than the runtime)', () => {
128+
it('accepts a string comparand — the substring branch', () => {
129+
expect(parse({ field: 'viewName', operator: 'contains', value: 'kanban' }).success).toBe(true);
130+
});
131+
132+
it('accepts an array comparand — the live array⊆array subset branch', () => {
133+
// `evaluateCondition`: `expected.every(v => fieldValue.includes(v))` when the
134+
// context field is an array. `SkillContext` is indexed `[k: string]: unknown`,
135+
// so that is a shape the cloud runtime is deliberately written for.
136+
// Refusing it here would un-declare a working capability (an ADR-0049
137+
// retirement decision), not tighten a contract.
138+
expect(parse({ field: 'tags', operator: 'contains', value: ['a', 'b'] }).success).toBe(true);
139+
});
140+
141+
it('is in neither constrained vocabulary', () => {
142+
expect(SKILL_TRIGGER_LIST_VALUE_OPERATORS).not.toContain('contains');
143+
expect(SKILL_TRIGGER_SCALAR_VALUE_OPERATORS).not.toContain('contains');
144+
});
145+
});
146+
147+
describe('#7113 — the exported vocabularies are the contract, not a copy', () => {
148+
it('the two vocabularies are disjoint and both subsets of the operator enum', () => {
149+
const all = [
150+
...SKILL_TRIGGER_LIST_VALUE_OPERATORS,
151+
...SKILL_TRIGGER_SCALAR_VALUE_OPERATORS,
152+
];
153+
expect(new Set(all).size).toBe(all.length);
154+
for (const operator of all) {
155+
// Every declared member must actually be an operator the schema accepts.
156+
expect(parse({
157+
field: 'f',
158+
operator,
159+
value: (SKILL_TRIGGER_LIST_VALUE_OPERATORS as readonly string[]).includes(operator)
160+
? ['x']
161+
: 'x',
162+
}).success).toBe(true);
163+
}
164+
});
165+
166+
it('pins the membership so a future operator has to be classified', () => {
167+
expect([...SKILL_TRIGGER_LIST_VALUE_OPERATORS]).toEqual(['in', 'not_in']);
168+
expect([...SKILL_TRIGGER_SCALAR_VALUE_OPERATORS]).toEqual(['eq', 'neq']);
169+
});
170+
});
171+
172+
describe('#7113 — the refinement does not disturb the carrier', () => {
173+
it('an unrelated operator/value pair still parses through Skill.triggerConditions', () => {
174+
const skill = SkillSchema.parse({
175+
name: 'order_management',
176+
label: 'Order Management',
177+
instructions: 'Manage orders.',
178+
tools: ['create_order'],
179+
triggerConditions: [
180+
{ field: 'objectName', operator: 'eq', value: 'order' },
181+
{ field: 'userRole', operator: 'in', value: ['sales', 'support'] },
182+
],
183+
});
184+
expect(skill.triggerConditions).toHaveLength(2);
185+
});
186+
187+
it('a bad condition inside a skill reports at the nested value path', () => {
188+
// The path prefix proves the refinement travels with the carrier rather
189+
// than only firing on a standalone parse.
190+
const result = SkillSchema.safeParse({
191+
name: 'order_management',
192+
label: 'Order Management',
193+
instructions: 'Manage orders.',
194+
tools: ['create_order'],
195+
triggerConditions: [{ field: 'userRole', operator: 'in', value: 'admin' }],
196+
});
197+
expect(result.success).toBe(false);
198+
if (result.success) throw new Error('unreachable');
199+
const issue = result.error.issues.find(
200+
(i) => i.path.join('.') === 'triggerConditions.0.value',
201+
);
202+
expect(issue).toBeDefined();
203+
expect(issue!.code).toBe('custom');
204+
});
205+
});

packages/spec/src/ai/skill.test.ts

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,27 @@
11
import { describe, it, expect } from 'vitest';
22
import {
3+
SKILL_TRIGGER_LIST_VALUE_OPERATORS,
34
SkillSchema,
45
SkillTriggerConditionSchema,
56
defineSkill,
67
type Skill,
78
} from './skill.zod';
89

910
describe('SkillTriggerConditionSchema', () => {
10-
it('should accept all operators', () => {
11+
it('should accept all operators — each with the value shape it reads', () => {
12+
// #7113: `value` is coupled to `operator`. This used to hand a scalar to
13+
// ALL FIVE, which is precisely the shape the tightening refuses — `in` /
14+
// `not_in` are membership tests and take the list. The list vocabulary is
15+
// read from the schema's own export so a future operator cannot be added
16+
// without being classified there.
1117
const operators = ['eq', 'neq', 'in', 'not_in', 'contains'] as const;
18+
const listOperators = SKILL_TRIGGER_LIST_VALUE_OPERATORS as readonly string[];
1219

1320
operators.forEach(operator => {
1421
expect(() => SkillTriggerConditionSchema.parse({
1522
field: 'objectName',
1623
operator,
17-
value: 'support_case',
24+
value: listOperators.includes(operator) ? ['support_case'] : 'support_case',
1825
})).not.toThrow();
1926
});
2027
});

0 commit comments

Comments
 (0)