Skip to content

feat(driver-mongodb): native dateGranularity lowering ($dateToString → engine bucket labels) + publish supports.queryDateGranularity - #7590

Merged
huangyiirene merged 1 commit into
mainfrom
claude/issue-7580-mongodb-date-granularity
Aug 11, 2026
Merged

feat(driver-mongodb): native dateGranularity lowering ($dateToString → engine bucket labels) + publish supports.queryDateGranularity#7590
huangyiirene merged 1 commit into
mainfrom
claude/issue-7580-mongodb-date-granularity

Conversation

@huangyiirene

Copy link
Copy Markdown
Collaborator

Fixes #7580

Maintainer-ordered card (direct route, 2026-08-11 — verbatim in the issue's Provenance). driver-mongodb is unfrozen per #5499 comment 5249019855; driver-memory stays frozen and is untouched, and the driver-sql bucket suites were read as precedent only.

What changes — and what deliberately does not

driver-mongodb now lowers a dateGranularity-bearing groupBy into the aggregation pipeline and publishes the capability, so engine.aggregate pushes a bucketed aggregate down instead of fetching every matching row and bucketing in JS.

The answers were already right. Measured, not assumed: MongoDBDriver.supports published no queryDateGranularity, and engine.ts's aggregate dispatch pushes a bucketed node down only for granularities a driver advertises — so the engine bucketed every granularity in memory and never reached #7550's refusal. This card moves the work server-side; it does not fix a wrong answer. What it buys is that a year-over-year rollup stops shipping the whole result set to the client first.

Decisions worth reviewing

1. No $dateTrunc, despite the card's title. The contract is not "truncate the instant", it is "produce the string bucketDateValue produces" — the engine's fallback emits LABELS and the two paths are chosen per query, so a drill-down crossing that seam must see the same spelling. $dateTrunc answers a truncated date that still needs formatting afterwards; $dateToString answers the label directly, out of one operator. It also keeps the server floor at MongoDB 3.6 instead of 5.0 and avoids binSize/startOfWeek — semantics this fleet cannot observe. Fewer unobserved operators is the whole argument: every operator emitted is one more documentation-derived claim (#5517), so the lowering needing the fewest wins. The route was offered as optional and this is the measurement for declining it.

Granularity Reference (bucketDateValue) Emitted
year '2024' %Y
month '2024-01' %Y-%m
day '2024-01-15' %Y-%m-%d
week '2025-W01' %G-W%V
quarter '2024-Q1' %Y + -Q + a $switch over %m

2. All five granularities are advertised — week included, where driver-sql on SQLite cannot. $dateToString carries both halves of the ISO-8601 week date: %G (week-numbering YEAR) and %V (zero-padded week). Those are exactly the two quantities bucketDateValue computes off the Thursday of the week, so 2024-12-30 labels '2025-W01' on both sides. SQLite has neither specifier, which is why the two records legitimately differ.

3. quarter is derived with a $switch over the zero-padded month STRING, not with arithmetic. MongoDB has no quarter specifier. Deriving the digit as $ceil($divide($month, 3)) would make the label depend on how $toString formats a double1 vs 1.0 is the difference between '2024-Q1' and a label nothing else in the repo produces, and it is precisely the kind of claim this environment cannot check. '01' <= '03' is lexicographic and exact for fixed-width numerals, so every value in the expression stays a literal string. Null propagation then falls out of $concat, which the manual defines as returning null if any argument is null — no $cond guard, one fewer operator to model.

4. Storage forms (ADR-0053): all three declared kinds are SERVED, none is refused. The capability record is per-granularity only — it has no field-kind axis — so advertising month means the engine will push a month bucket down onto a date or time column too. Refusing those at the builder would turn a query the engine used to answer into a 501. Measured instead, per kind, against bucketDateValue's new Date(...):

Stored form in-memory reference this lowering
BSON Date (datetime) the instant $convert of a date is the date
'2024-01-15' (date) new Date(...) = midnight UTC $dateFromString semantics: midnight UTC
'14:30:00' (time) Invalid Date → the empty bucket not a date → onError → the empty bucket
epoch-ms number new Date(ms) numeric read as epoch ms
null / missing / junk the empty bucket onNull / onError → the empty bucket

$convert … onError/onNull: null makes the read total, which is not a nicety: bucketDateValue is total, so a non-total expression would fail the whole aggregation on one junk row where the engine's fallback answers. A Field.time column is a wall clock and not an instant (#2004) — both paths agree it has no bucket rather than one of them inventing a day, which is the measurement that says it needs no separate refusal.

⚠️ The residual divergence, stated rather than papered over. JS's parser accepts legacy spellings ISO-8601 does not ('2024-01-15 10:00:00' with a space, '2024/01/15'), so a column holding one buckets in memory and empties here. Those are not forms this driver writes — every write goes through coerceTemporalValue — so the exposure is pre-#4047 legacy rows, and it is the same exposure every SQL face already carries. It is asserted as a live difference in the suite, not hidden.

5. Timezones stay engine-side, and nothing needed narrowing. engine.aggregate forces the in-memory path whenever a non-UTC reference zone meets a date bucket (ADR-0053 Phase 2 D2, pinned by engine-aggregate-timezone.test.ts), and the AST it hands a driver carries no timezone key at all. So UTC is not a narrowing this PR chose — it is the only thing this lowering could mean. Pinned by showing the zoned label for a straddling instant is a different label, i.e. that a driver silently answering the UTC one would be answering the wrong question.

6. Declared = enforced is a mechanism, not a promise. The capability record and the builder's refusal read one constant (MONGODB_DATE_GRANULARITIES), and the suite asserts identity rather than equality — two literals that happen to agree today are exactly how a record drifts from the lowering it advertises. That drift is the failure worse than advertising nothing: the engine stops bucketing in memory on the strength of the bit, so a working query starts answering 501.

7. #7550's refusal is kept and its pins were consciously rewritten. They used to assert that every declared granularity is refused — true then, false now. Rather than delete the coverage, they now assert the new substance: an advertised granularity must lower, and an unadvertised one must still refuse with NOT_IMPLEMENTED/501 and code+status asserted (ADR-0112). Since the advertised set is all five, that second population is reachable only from outside the declared enum — a direct caller handing the exported builder an unknown granularity, which is the caller the refusal was always written for. Tested with one rather than deleted as unreachable, because "unreachable today" is how a refusal quietly becomes a default: that answers. The message now names what is bucketed here, driver-sql's wording.

The parity evidence, and one dependency decision

driver-sql, driver-turso and driver-sqlite-wasm each hand-copy bucketDateValue into their bucket suites and say what it costs: a copy that stops tracking its original leaves the copy and the driver agreeing while both are wrong. The repo's real answer is checkDateBucketParity (@objectstack/verify), which imports the actual applyInMemoryAggregation — but it drives a LIVE driver, so driver-mongodb cannot join it here.

Measured rather than inherited: objectql depends on no driver, so driver-mongodb → objectql is acyclic, and as a devDependency it never enters the published package. So mongodb-date-bucket-parity.test.ts imports the real applyInMemoryAggregation and bucketDateValue. The reference side is the engine's fallback — the drift those ⚠️ Keep in sync comments admit they cannot detect does not exist for this driver. This is the one thing here outside the driver's own source; it is a devDependency line, and turbo's test → ^build already orders the build.

The suite runs the same rows through both paths across every advertised granularity × storage form, using checkDateBucketParity's own instants (year/quarter/month/ISO-week boundaries, midnight in both directions) plus three rows a live fixture cannot easily carry: explicit null, the columns MISSING entirely, and unparseable junk. It also runs the cross-column pass — at and on name the same calendar day, so a granularity that labels them differently is a storage-form leak (#3773's shape).

Two guards against the failure a two-implementation comparison invites — both sides sharing one wrong idea:

  1. literal label pins, stating the required spellings out loud rather than deriving them from either side ('2025-W01' for 2024-12-30, '2026-W53' for 2027-01-01, '2024-Q3' for 2024-07-01);
  2. a discrimination block, replaying a plausible-but-wrong lowering through the evaluator and requiring it to FAIL.

The evaluator moved to mongodb-pipeline-evaluator.testkit.ts (the legacy-datetime-storage.testkit.ts convention) so both suites share one strict reader — a second, laxer copy would be the easiest way to bless a broken lowering. It moved verbatim; only the date operators were added, so nothing #7550 asserts changed meaning. Critically it models the documentation independently: its ISO-8601 grammar is written out rather than delegated to new Date(...), and %G/%V come from the standard rather than transcribed from bucketDateValue — otherwise the two sides would agree by construction and the comparison would mean nothing.

Reverse-verified — each deliberate break was executed and had to fail:

Injected defect Result
%G-W%V%Y-W%V (ISO week-year dropped) 4 failed — both week cells + both year-boundary label pins
$convert removed (raw field into the date operator) 14 failed — every cell on all three storage forms
quarter boundary off by one month 3 failed — both quarter cells + the Q3 label pin
capability record re-spelled as a second, value-equal literal 1 failed — the identity assertion

What this does NOT claim

⚠️ Whether a real mongod agrees. This fleet cannot fetch a mongod binary (proxy 403 — #5517), so $convert / $dateToString / $concat / $switch / $lte are modelled from the manual, not observed. What is proven is that the lowering agrees with the engine under those documented semantics; what is not is that a server implements them as documented. Same bound #7550 carried for $cond/$ifNull/$addToSet, and it is written in three places rather than one: the suite header, the published supports doc where the capability is enrolled, and beside the lowering.

Gates run locally

Gate Result
pnpm --filter @objectstack/driver-mongodb test 341 passed, 143 skipped (was 266 before this PR; the opt-in real-mongod halves stay skipped)
pnpm --filter @objectstack/driver-mongodb typecheck clean
pnpm check:driver-conformance OK — 37 covered cells, 3 DEBT, 0 exempt (unchanged)
eslint --no-inline-config on every changed file clean
build closure (--filter @objectstack/driver-mongodb..., now incl. objectql) clean

The rest of the lint farm is left to CI. packages/objectql source is untouched, and no engine-side defect was found to file.

Deliberately untouched

  • packages/drivers/driver-memory — frozen under [裁决] driver-memory / driver-mongodb 投入冻结 —— 维护者 2026-08-05 口径(跨单锚点) #5499.
  • packages/objectql — read for the label contract and the dispatch gate; not edited.
  • packages/qa/dogfood, packages/verifycheckDateBucketParity needs a live driver, so enrolling driver-mongodb there is not available in this fleet. Recorded as the open question, not worked around.
  • packages/spec/src/data/aggregation-conformance.ts — its scope note already excludes date bucketing by design ("a per-dialect capability … folding it in would make the table unpassable for a face that legitimately buckets nothing"), so there was nothing to update.

🤖 Generated with Claude Code

https://claude.ai/code/session_019QkW7hhVvhTxh4V9Wmnqdy


Generated by Claude Code

…bucket labels, and publish supports.queryDateGranularity (#7580)

`driver-mongodb` published no `queryDateGranularity`, so `engine.aggregate`
bucketed every granularity in memory: correct answers, but the whole result set
shipped to the client before the rollup. #7550 refused a bucketed node at the
builder rather than implement or silently drop it, and sized the native lowering
as a card of its own. This is that card.

The bucket LABELS are the engine's own spellings, because the engine picks
between the pushed-down and in-memory paths per query and a drill-down can cross
that seam. All five declared granularities are advertised — `week` included,
where driver-sql on SQLite cannot, because `$dateToString` has both halves of
the ISO-8601 week date (`%G`/`%V`).

No `$dateTrunc`: it answers a truncated DATE that still needs formatting, raises
the server floor to MongoDB 5.0, and adds binSize/startOfWeek semantics this
fleet cannot observe. `$dateToString` answers the label directly out of one
operator, and fewer unobserved operators is the whole argument.

All three ADR-0053 storage forms are served through one total expression:
`$convert … onError/onNull: null` mirrors `bucketDateValue`'s totality, so a
BSON Date, `YYYY-MM-DD` text, `HH:MM:SS` text, null, missing and unparseable
junk all bucket the way the engine buckets them.

The capability record and the builder's refusal read ONE constant, so an
advertised granularity the builder would refuse cannot exist — the failure worse
than advertising nothing, since the engine stops bucketing in memory on the
strength of the bit.

Parity is proven by running the SAME rows through the real
`applyInMemoryAggregation` and through the emitted pipeline. driver-mongodb can
depend on objectql where driver-sql cannot (objectql depends on no driver), so
this is a devDependency rather than the hand-copied `bucketDateValue` the three
SQL bucket suites carry, and the drift their `⚠️ Keep in sync` comments admit
they cannot detect does not exist here.

The strict in-process evaluator moved to `mongodb-pipeline-evaluator.testkit.ts`
so both suites share one reader; only the date operators were added. It models
the manual independently — its own ISO-8601 grammar, `%G`/`%V` from the standard
— so the two sides CAN disagree and the comparison means something.

⚠️ Nothing here has met a real mongod (#5517: proxy 403 on the binary). The
bound is written into the suite header, onto the published capability, and
beside the lowering.

Fixes #7580

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019QkW7hhVvhTxh4V9Wmnqdy
@vercel

vercel Bot commented Aug 11, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 11, 2026 6:36am

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/driver-mongodb.

5 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/data-modeling/drivers.mdx (via @objectstack/driver-mongodb)
  • content/docs/getting-started/glossary.mdx (via @objectstack/driver-mongodb)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/driver-mongodb)
  • content/docs/plugins/packages.mdx (via @objectstack/driver-mongodb)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/driver-mongodb)

1 release-owned page(s) also reference the affected code. These are read-only:

  • content/docs/releases/implementation-status.mdx (via @objectstack/driver-mongodb)

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@github-actions github-actions Bot added documentation Improvements or additions to documentation dependencies Pull requests that update a dependency file tests tooling labels Aug 11, 2026
@huangyiirene
huangyiirene marked this pull request as ready for review August 11, 2026 07:10
@huangyiirene
huangyiirene added this pull request to the merge queue Aug 11, 2026
Merged via the queue into main with commit d17a222 Aug 11, 2026
27 checks passed
@huangyiirene
huangyiirene deleted the claude/issue-7580-mongodb-date-granularity branch August 11, 2026 07:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation size/xl tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

driver-mongodb: native dateGranularity lowering ($dateTrunc → engine bucket labels) + publish supports.queryDateGranularity

2 participants