fix(query): close the memory-budget blind spot on the non-aggregate scan path

[aaj3f]

Forest map — where this PR sits

This is PR #2 of 4 in the Tier-0/1/2 implementation of the 2026-07 big-iceberg audit (audit-2026-07/), the bundle composition decided in decisions/DEC-001-pr-bundling.md. Base = the integration branch perf/audit-tier012 (perf tip 10e073fe9 + a clean merge of origin/main ≥ 7581f0a, the #1508 resurrection fix). Siblings: PR-SAFE-MOR (on main), PR-COVERAGE, PR-HARNESS (the terminal re-bless leaf).

This PR closes	Finding	Receipts
Memory-budget blind spot on the non-aggregate scan/crawl path	F-AUD-3 (CONFIRMED, V2)	00-MASTER-AUDIT.md §2 F-AUD-3; V2-membudget-verification.md §1–§7

Four V2 fix sites, all here except V2 site B (per-file buffer accounting — excluded; see Residuals):

Site	What	Where
D	BINDING_EST_BYTES 64 → size_of::<Binding>() (=88) + compile-time floor guard	fluree-db-query/src/context.rs
C	Per-query budget division at the runner attach point	fluree-db-query/src/execute/runner.rs
A1	record_alloc + checkpoint on the materialized scan window, released on hand-off (see below)	fluree-db-query/src/r2rml/operator.rs (advance_one_window, pull-loop poll)
A2	record_alloc + checkpoint inside the fact-as-parent build loop (stays cumulative — persistent map)	fluree-db-query/src/r2rml/operator.rs (build_parent_lookup)
release	saturating release decrement primitive on the budget counter (pairs the A1 window charge)	fluree-db-core/src/cancellation.rs + context.rs wrapper

A1 window-scoped release (q038 fix). The first cut of A1 charged each window cumulatively with no decrement — the documented V2 conservative edge. The live re-bless proved it bites a mainstream shape: q038 (a 36M-row un-fused COUNT on the per-row materialize path) false-aborted typed at 38 s (~8.61 GB > 8.59 GB) while completing fine at 52.5 s with the accounting off, because ~70 sequentially-freed 512K-row windows summed past the budget though only one is ever resident. Fix: charge + checkpoint() before emit (an oversized single window, or this window atop a retained A2 build / upstream fold, still aborts typed), then release() the charge once the window is handed off (produced drops). A streaming scan now accounts only its resident window. A2 and the fold/join/fused builds are not released — they genuinely persist, so their cumulative charge is correct.

Why — the V2 worst-case arithmetic

The R3-B memory budget instruments the aggregating/joining operators (record_alloc at 7 sites in hash_join.rs / group_aggregate.rs / fused_aggregate.rs). The non-aggregate R2rmlScanOperator path had zero — six check_cancelled() and no record_alloc/checkpoint. So a single wide crawl (default switches) carries, entirely unaccounted:

Component	Bound	Resident	Accounted before?
Overflow window self.pending	≤512K rows	~1.19 GB	NO
Scan-replay cache (correlated)	≤512K rows	~1.19 GB	NO
In-flight file buffers	≤32 files	~2 GB (up to ~8 GB broad)	NO
Transient fact-as-parent build	fact rows (uncapped)	up to ~7 GB (36M entries)	NO

≈ 4.4 GB unaccounted for a modest wide crawl, ~11 GB with one fact-as-parent lookup — past both the 8 GiB budget and the 10 GiB Lambda ceiling, with record_alloc seeing zero of it. With no per-query division (site C), N×4.4 GB OOMs a 10 GiB node at N≈2. This is not hypothetical: forensic specimen 071cd59f — the point-lookup crawl ?ol ex:orderLineKey "1" . ?ol ?p ?o — hard-OOM'd at 10237 MB on exactly this path (R2rmlScanOperator wildcard crawl + build_parent_lookup over a 36M-row fact). It is now the regression test r3b_scan_window_budget_aborts_typed.

After this PR the scan path trips a typed QueryError::MemoryBudgetExceeded (507, distinct from a 408 timeout) instead of OOMing — A1 catches accumulated bounded windows at the pull-loop checkpoint; A2 aborts the unbounded parent build before the whole map is resident.

Design note — the per-query division (site C)

V2 §3: set_memory_limit had no production caller, so checkpoint() always compared each query's own counter against the full process budget — N concurrent queries each read "under 8 GB" while the node sat at 10 GB.

I shipped the sound minimal static form: pin budget / FLUREE_QUERY_BUDGET_SHARE_DIV at the runner attach (execute_prepared_into, the with_cancellation seam). Default div=1 pins nothing — checkpoint falls back to the full process budget, byte-for-byte today's behavior; an embedder's explicit ceiling is never clobbered. Operators set div to their deployment's max query concurrency (e.g. a Lambda's reserved concurrency) to opt into sharing.

I deliberately did not ship the dynamic divisor (divide by live concurrency so a lone query keeps the full budget), even though the audit recommends default-on sharing: a correct live count must count only top-level queries, and the runner attach point is also re-entered by nested policy f:query / reasoning / sub-queries — an in-engine counter there would over-divide and abort legitimate queries. V2 §7(C) reaches the same conclusion ("the authoritative source is the server that spawns the watchdog and holds the handle, fluree-db-server/src/query_control.rs"). So the dynamic count is deferred to the server request boundary; the static form is present, neutral by default, and sound. A non-1 default is intentionally NOT set here — it would shrink every query's budget and risk aborting today-passing queries in a safety PR.

Residuals (deferred, documented)

• V2 site B — per-file in-flight buffer accounting (r2rml.rs fan-out, ≤32 files × up to 256 MB) is excluded: releasing on batch flush needs a decrement primitive the current monotonic AtomicUsize counter lacks. Adding one is a larger change to fluree-db-core::cancellation; deferred. The file buffers remain invisible to the budget (bounded O(32) by buffer_unordered backpressure, so not unbounded — but GB-scale).
• Heap under-count beyond the constant — BINDING_EST_BYTES is now the true 88-byte stack size but still omits the Arc<str> IRI heap (~50–70 B) an Iceberg row carries, so a wide IRI-crawl is counted at ~1/2.2 of true resident bytes. Deliberate floor (over-count only ever aborts a query already near OOM); a heap-aware estimate is follow-up. GROUP_EST_BYTES is likewise a flat estimate.
• Cumulative conservatism now applies only to PERSISTENT allocations — after the q038 fix, scan windows are charged then released on hand-off, so a long streaming scan accounts only its resident window rather than the all-time sum (that sum was the q038 false-abort). The remaining non-released charges are the genuinely persistent ones — the A2 fact-parent map and the fold/join/fused builds — where cumulative == resident and the conservatism is correct. The retained scan overflow (self.pending, ≤ one window, drained before the next pull) is left untracked (minimal). Per-file buffers (site B) still need their own release pairing (deferred).
• SWITCHES.md — the two new switches below are documented here and in code but the registry regeneration is PR-HARNESS's job per DEC-001 (item 4). Flagged for that leaf.

Kill-switch ledger (house rule: no unswitched mechanism)

Switch	Default	Mechanism	Revert
FLUREE_SCAN_MEM_ACCOUNTING	on	A1 + A2 scan/crawl record_alloc + checkpoint	=off → scan path records nothing; checkpoint() degrades to a pure cancellation poll (prior behavior)
FLUREE_QUERY_BUDGET_SHARE_DIV	1	C per-query budget = full / div	=1 (default) → full budget per query, exactly today

Both read via the established idioms (env_switch_enabled OnceLock for the on/off; parsed env for the divisor). FLUREE_QUERY_MEMORY_BUDGET_BYTES=0 still disables the whole budget guard, unchanged.

Local gate record — verification of record

CI does not fire on this PR: ci.yml gates only when base == main, and this PR's base is perf/audit-tier012 (verified in DEC-001 Adjudication B.3). The local record below is the verification of record. Reproduced in a worktree at branch head cfd773d75 (base = integration tip 9c6739c3b, the PR-SAFE-MOR guard + its cache-hit re-apply arm, all in fluree-db-iceberg/* + fluree-db-api/iceberg_catalog.rs, zero overlap with this PR's four files). This PR touches four files — fluree-db-core/src/cancellation.rs (the release primitive) plus fluree-db-query/src/{context.rs, execute/runner.rs, r2rml/operator.rs}:

• cargo fmt --check — the four files this PR changes are clean (0 diffs each). The workspace check reports 2 pre-existing drift spots in fluree-db-query/src/hash_join.rs:1070,1120 that this PR does not touch — present at the base (git show origin/perf/audit-tier012:fluree-db-query/src/hash_join.rs | rustfmt --check shows them), part of the perf line's known fmt debt, left for the hygiene pass (db-verify-gotchas: don't fix pre-existing debt in a scoped PR).
• cargo clippy -p fluree-db-core --all-targets --no-deps — clean re: this PR (1 pre-existing unused import in commit.rs:943, untouched by this PR). cargo clippy -p fluree-db-query --all-targets --no-deps — clean, 0 warnings (clippy 1.97.0).
• cargo test -p fluree-db-core — 704 lib + bins pass, 0 failed (incl. the 2 new release tests).
• cargo test -p fluree-db-query — 1296 lib + all integration bins pass, 0 failed (incl. the 6 new hermetics).
• cargo test -p fluree-db-api (grp_* bins, full package) — all bins pass, 0 failed (99 ignored = live-credential Snowflake/Iceberg suites).

No fluree-db-server changes (site C lives in the query runner, not the server), so that gate is not applicable. The release primitive lands in fluree-db-core (the budget counter's home) — minimal (a single saturating decrement), per the design guardrail not to introduce a general decrement surface beyond the window pairing.

Hermetic tests (extend the r3b_*_budget_aborts_typed pattern)

• r3b_scan_window_budget_aborts_typed — specimen 071cd59f regression: a ?s ?p ?o wildcard crawl with a 1-byte ceiling aborts typed (A1; still aborts after the release fix — the checkpoint fires while the window is charged, before it is released).
• r3b_scan_windows_release_no_false_abort — q038 regression: 64 one-row windows under an 8000-byte ceiling COMPLETE (verified to fail pre-fix at window ~16 with MemoryBudgetExceeded 8448 > 8000) — the released per-window charge never accumulates.
• release_subtracts_and_saturates_at_zero + disabled_handle_release_is_a_noop (fluree-db-core) — the release primitive decrements, is shared across clones, and saturates.
• r3b_parent_build_budget_aborts_typed — a fact-as-parent build_parent_lookup with a 1-byte ceiling aborts typed on the first batch (A2, still cumulative).
• shared_ceiling_trips_each_query_at_its_divided_budget + per_query_ceiling_divides_and_floors — two queries under a divided ceiling each trip at budget/N, not the full budget (C).
• binding_est_bytes_is_at_least_binding_stack_size — the 88-byte canary; the >= size_of invariant is a compile-time const _ next to the constant (D).

[aaj3f]

Consolidating at maintainer request. #1521 (the memory-budget guard on the non-aggregate scan path) is carried forward in #1528 (the big-Iceberg-audit implementation), which brings the audit program forward as a single reviewable unit. Its verification of record is recorded under R-1521 in pr-reviews-impl.md (verdict SHIP; every residual bounded, documented, and over-count-safe), alongside the whole-stack live gate of 77 records / 0 hash mismatches / 0 perf violations. The pre-consolidation branch tip is preserved at tag archive/pre-refactor-2026-07-21/perf_audit-mem-guards — no history was rewritten. Closing in favor of #1528; discussion continues there.