Skip to content

Commit 465a0fa

Browse files
claude[bot]claude
andauthored
fix(driver-sql): refuse scalar-comparison operators on JSON/multi-value columns instead of answering silently wrong (#7415)
* fix(driver-sql): refuse scalar-comparison operators on JSON/multi-value columns A `multiple: true` field — and every other `JSON_COLUMN_TYPES` field — is stored as a JSON TEXT column, and the equality family lowered straight to SQL against that text with no column-type consultation. The result was a wrong answer with a 200: {members:{$in:[U1]}} -> 0 rows (fail-closed) {members: U1} -> 0 rows (fail-closed) {members:{$nin:[U1]}} -> the excluded row (fail-OPEN) {members:{$lte:U1}} -> 1 row, lexicographic on the leading '[' `members not in ('U1')` is TRUE — the stored text genuinely is not equal to that id — so "exclude these" compiled to "return everything". An exclusion that silently stops excluding widens a result set, and a 200 with [] is byte-identical to a query that legitimately matched nothing, so nothing existed for a caller to key on. Gate the three lowering entries on the column type, ahead of every rewrite and both comparison emitters: the operator-object branch and the bare-value branch of applyFilterCondition, and the plain-map loop of applyFilters. Placing it before applyNormalizedComparison matters — a `multiple: true` datetime column on an external object is served by the normalised whereRaw arms rather than the plain whereIn arms, and showed the identical defect. The refusal names the operator, the field, why the column cannot answer it, states the filter was not applied, and prescribes $contains (or an $or of $contains for any-of). ADR-0112 class 1 — INVALID_FILTER / 400, the same envelope as the unknown-operator refusal, on every face that lowers a filter. $contains / $notContains / $startsWith / $endsWith / $icontains and the null predicates are untouched: the LIKE family matches the serialization as text and is the only working membership spelling, and column presence is a well-formed question whatever the column holds. Fixes #7398 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011GCuQuqxKvWLYUGss7CXdc * test(driver-sql): type the aggregate face's query bag instead of casting it `check:query-options-erasure` counts an `as any` at the options position of find/findOne/count/aggregate, and the new refusal sweep raised the test surface 249 -> 250. The cast was gratuitous: `aggregations` is on `DriverQuery`, so the call types as written once the entry carries its `field`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011GCuQuqxKvWLYUGss7CXdc --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 1d0d7a2 commit 465a0fa

3 files changed

Lines changed: 697 additions & 0 deletions

File tree

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
---
2+
'@objectstack/driver-sql': patch
3+
---
4+
5+
fix(driver-sql): refuse scalar-comparison operators on JSON/multi-value columns with 400 `INVALID_FILTER` instead of answering a silently wrong result
6+
7+
A `multiple: true` field — and every other `JSON_COLUMN_TYPES` field — is stored by this driver as a **JSON TEXT** column. The equality family lowered straight to SQL against that text with no column-type consultation, so a filter naming such a column compiled, ran, and returned a wrong answer with a `200`.
8+
9+
**Behaviour change (user-visible).** On a row whose `members` holds `["U1","U2"]`:
10+
11+
| filter | before | after |
12+
|---|---|---|
13+
| `{members:{$in:[U1]}}` | `200`, **0 rows** | `400 INVALID_FILTER` |
14+
| `{members:{$eq:U1}}` | `200`, **0 rows** | `400 INVALID_FILTER` |
15+
| `{members: U1}` (bare equality) | `200`, **0 rows** | `400 INVALID_FILTER` |
16+
| `{members:{$nin:[U1]}}` | `200`, **the row it was asked to EXCLUDE** ⚠️ | `400 INVALID_FILTER` |
17+
| `{members:{$ne:U1}}` | `200`, **the row it was asked to exclude** ⚠️ | `400 INVALID_FILTER` |
18+
| `{members:{$lte:U1}}` | `200`, **1 row** (lexicographic, on the leading `[`) | `400 INVALID_FILTER` |
19+
| `{members:{$contains:U1}}` | `200`, 1 row | **unchanged** |
20+
21+
**`$nin` is why this is a fix and not a documented footgun.** `members not in ('U1')` is TRUE — the stored text genuinely is not equal to that id — so "exclude these" compiled to "return everything". `$in` fails **closed** (fewer rows than exist, bad but narrowing); `$nin` and `$ne` fail **OPEN**, so any exclusion built on them silently stops filtering and the failure direction is *widening*. A downstream delete-guard written as `plans.find({ where: { assignees: { $in: memberIds } } })` therefore never fired once since it shipped, threw nothing, logged nothing, and type-checked — and a `200` with `[]` is byte-identical to a query that legitimately matched nothing, so no caller had anything to key on.
22+
23+
**What is refused:** `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$in`, `$nin`, `$between`, the bare `{ field: value }` spelling, and the infix spellings the normalised emitter also answers (`=`, `<>`, `in`, `nin`, `not_in`, `notin`, …) — on any column this driver stores as JSON, i.e. `field.multiple` arrays **and** the structured-JSON types (`address`, `location`, `composite`, the file-metadata and multi-option types). The structured-JSON half is included because the mechanism is the JSON-text storage rather than the array-ness: `{address:{$nin:['Beijing']}}` showed the identical fail-open inversion.
24+
25+
The refusal names the operator, the field, why the column cannot answer it, states that the filter **was not applied**, and prescribes the working spelling. It carries the same ADR-0112 envelope as the unknown-operator refusal (`INVALID_FILTER` / 400), on every face that lowers a filter: `find`, `findOne`, `count`, `aggregate`, `distinct`, and the where-clauses of `updateMany` / `deleteMany`.
26+
27+
**What does NOT change:** `$contains`, `$notContains`, `$startsWith`, `$endsWith`, `$icontains` — the `LIKE` family matches the serialization as text, and `$contains` (or an `$or` of `$contains` for any-of) is the working membership spelling this refusal points at. `$null` / `$exists` also keep working: the column's presence is a well-formed question whatever it holds. Filters on scalar columns are untouched, and a table this driver was never told about (no registered field types) is unaffected — the gate fires only where the column is KNOWN to be JSON.
28+
29+
Giving array columns a real membership operator (`$overlaps` / `$containsAny`) is a separate question about the closed `FILTER_OPERATORS` set and is deliberately not answered here.

0 commit comments

Comments
 (0)