Skip to content

Commit 3db8a3e

Browse files
committed
fix(driver-sql): refuse scalar-comparison operators on JSON/multi-value columns
A `multiple: true` field — and every other `JSON_COLUMN_TYPES` field — is stored as a JSON TEXT column, and the equality family lowered straight to SQL against that text with no column-type consultation. The result was a wrong answer with a 200: {members:{$in:[U1]}} -> 0 rows (fail-closed) {members: U1} -> 0 rows (fail-closed) {members:{$nin:[U1]}} -> the excluded row (fail-OPEN) {members:{$lte:U1}} -> 1 row, lexicographic on the leading '[' `members not in ('U1')` is TRUE — the stored text genuinely is not equal to that id — so "exclude these" compiled to "return everything". An exclusion that silently stops excluding widens a result set, and a 200 with [] is byte-identical to a query that legitimately matched nothing, so nothing existed for a caller to key on. Gate the three lowering entries on the column type, ahead of every rewrite and both comparison emitters: the operator-object branch and the bare-value branch of applyFilterCondition, and the plain-map loop of applyFilters. Placing it before applyNormalizedComparison matters — a `multiple: true` datetime column on an external object is served by the normalised whereRaw arms rather than the plain whereIn arms, and showed the identical defect. The refusal names the operator, the field, why the column cannot answer it, states the filter was not applied, and prescribes $contains (or an $or of $contains for any-of). ADR-0112 class 1 — INVALID_FILTER / 400, the same envelope as the unknown-operator refusal, on every face that lowers a filter. $contains / $notContains / $startsWith / $endsWith / $icontains and the null predicates are untouched: the LIKE family matches the serialization as text and is the only working membership spelling, and column presence is a well-formed question whatever the column holds. Fixes #7398 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011GCuQuqxKvWLYUGss7CXdc
1 parent 8a857a2 commit 3db8a3e

3 files changed

Lines changed: 694 additions & 0 deletions

File tree

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
---
2+
'@objectstack/driver-sql': patch
3+
---
4+
5+
fix(driver-sql): refuse scalar-comparison operators on JSON/multi-value columns with 400 `INVALID_FILTER` instead of answering a silently wrong result
6+
7+
A `multiple: true` field — and every other `JSON_COLUMN_TYPES` field — is stored by this driver as a **JSON TEXT** column. The equality family lowered straight to SQL against that text with no column-type consultation, so a filter naming such a column compiled, ran, and returned a wrong answer with a `200`.
8+
9+
**Behaviour change (user-visible).** On a row whose `members` holds `["U1","U2"]`:
10+
11+
| filter | before | after |
12+
|---|---|---|
13+
| `{members:{$in:[U1]}}` | `200`, **0 rows** | `400 INVALID_FILTER` |
14+
| `{members:{$eq:U1}}` | `200`, **0 rows** | `400 INVALID_FILTER` |
15+
| `{members: U1}` (bare equality) | `200`, **0 rows** | `400 INVALID_FILTER` |
16+
| `{members:{$nin:[U1]}}` | `200`, **the row it was asked to EXCLUDE** ⚠️ | `400 INVALID_FILTER` |
17+
| `{members:{$ne:U1}}` | `200`, **the row it was asked to exclude** ⚠️ | `400 INVALID_FILTER` |
18+
| `{members:{$lte:U1}}` | `200`, **1 row** (lexicographic, on the leading `[`) | `400 INVALID_FILTER` |
19+
| `{members:{$contains:U1}}` | `200`, 1 row | **unchanged** |
20+
21+
**`$nin` is why this is a fix and not a documented footgun.** `members not in ('U1')` is TRUE — the stored text genuinely is not equal to that id — so "exclude these" compiled to "return everything". `$in` fails **closed** (fewer rows than exist, bad but narrowing); `$nin` and `$ne` fail **OPEN**, so any exclusion built on them silently stops filtering and the failure direction is *widening*. A downstream delete-guard written as `plans.find({ where: { assignees: { $in: memberIds } } })` therefore never fired once since it shipped, threw nothing, logged nothing, and type-checked — and a `200` with `[]` is byte-identical to a query that legitimately matched nothing, so no caller had anything to key on.
22+
23+
**What is refused:** `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$in`, `$nin`, `$between`, the bare `{ field: value }` spelling, and the infix spellings the normalised emitter also answers (`=`, `<>`, `in`, `nin`, `not_in`, `notin`, …) — on any column this driver stores as JSON, i.e. `field.multiple` arrays **and** the structured-JSON types (`address`, `location`, `composite`, the file-metadata and multi-option types). The structured-JSON half is included because the mechanism is the JSON-text storage rather than the array-ness: `{address:{$nin:['Beijing']}}` showed the identical fail-open inversion.
24+
25+
The refusal names the operator, the field, why the column cannot answer it, states that the filter **was not applied**, and prescribes the working spelling. It carries the same ADR-0112 envelope as the unknown-operator refusal (`INVALID_FILTER` / 400), on every face that lowers a filter: `find`, `findOne`, `count`, `aggregate`, `distinct`, and the where-clauses of `updateMany` / `deleteMany`.
26+
27+
**What does NOT change:** `$contains`, `$notContains`, `$startsWith`, `$endsWith`, `$icontains` — the `LIKE` family matches the serialization as text, and `$contains` (or an `$or` of `$contains` for any-of) is the working membership spelling this refusal points at. `$null` / `$exists` also keep working: the column's presence is a well-formed question whatever it holds. Filters on scalar columns are untouched, and a table this driver was never told about (no registered field types) is unaffected — the gate fires only where the column is KNOWN to be JSON.
28+
29+
Giving array columns a real membership operator (`$overlaps` / `$containsAny`) is a separate question about the closed `FILTER_OPERATORS` set and is deliberately not answered here.

0 commit comments

Comments
 (0)