perf(r2rml): PR-4d batched-OPTIONAL seed coalescing (F14) — one inner scan for the whole driving side

[aaj3f]

What

The batched correlated-OPTIONAL hash-left-join (optional.rs) seeded and scanned its inner once per OUTER (driving) batch. When the driving side emits many small windowed batches (a fact scan under a LIMIT budget), the inner table is re-scanned per window — the F14 residual left after PR-4b/4c collapsed the per-row rebuild. PR-4d coalesces the whole (bounded) driving side into one seed so the inner is scanned once.

Mechanism

OptionalOperator::next_batch called build_batch once per required batch. New pull_coalesced_required concatenates required batches (column-wise, via Batch::schema_arc()) up to a cap into one combined batch, so build_batch seeds the inner over the whole distinct correlation set at once. Gated to the batched hash-join-safe path via OptionalBuilder::supports_seed_coalescing() (default false; PlanTreeOptionalBuilder = true iff every inner pattern is hash-join-safe) — so only the batched-OPTIONAL family (q016/q050) changes; every other OPTIONAL keeps the streaming single-batch pull.

• Kill switch: FLUREE_R2RML_OPTIONAL_SEED_COALESCE (default on). Off ⇒ single-batch pull, byte-identical.
• Cap: FLUREE_R2RML_OPTIONAL_SEED_COALESCE_CAP (default 512K rows). A LIMIT-bounded driving side exhausts below the cap (… LIMIT k coalesces exactly the ≤k rows into one scan); beyond the cap it degrades to cap-sized windows (still ≫ the pre-PR-4d one-per-outer-batch).

Gate — REQUIRED per-head full-corpus baseline (priming + 3-rep, cache-thrashed, per-query manifest timeout)

Oracle correctness: 54 records, 0 hash mismatches, 0 perf violations. (The prior mini-gate compared nothing to the blessed oracles; this is the real check.)

query	metric	pre / OFF	post / ON (3-rep median, in-protocol)
q016	scan_table	182	3
q016	wall	43.1 s	17.47 s (2.5×)
q016	rows / oracle	5000	5000, hash-clean
q050	scan_table	63	5
q050	wall (3-rep median)	1.24 s (F9 head)	1.11 s (marginally faster)
q050	result_hash	373c16256315	373c16256315 (byte-identical)
q015 (sentinel)	wall	—	1.66 s (healthy)
q028 (sentinel)	wall	—	3.81 s (unchanged — rides slate items 4/5)

• q050 anomaly resolved: the earlier single-rep mini-gate showed 7.4 s ON vs 5.4 s OFF — that was isolated-cold noise (the mini-gate didn't pre-warm q050's dims). At the 3-rep in-protocol median q050 is 1.11 s ON, marginally faster than the pre-4d 1.24 s, hash byte-identical.
• q016 wall is REPORTED, not claimed-done. 43.1 s → 17.47 s is a real 2.5× from the scan collapse, but ≤3 s is not reached and is not in this PR's scope. Decomposition (spans nest — fetch/decode are inside parquet_read): load_table 1.55 s (n=1) + parquet_read 4.77 s (n=23,010 = 3× full 7,670-file fact reads, ~1.6 s/scan) + ~11 s hash-left-join partition/materialize (a full 180 K-row FACT_SHIPMENT inner scan + 5000-bucket combine). q016's ≤3 s path is a future follow-on (PR-2a decode-wall for the fact scans + an IN-set-probe materialization lever), tracked in the audit ROADMAP — not the loadTable-cache alone.

Tests

• pr4d_pull_coalesced_required_concatenates_all_batches_in_order — 3 batches (2+1+3) → one 6-row seed, order preserved.
• pr4d_multi_batch_driver_collapses_to_one_build_batch — 3 driving batches ⇒ exactly ONE build_batch (one inner scan); all 6 left-join rows survive.
• Full fluree-db-query lib suite 1237 passed, 0 failed (incl. the existing batched-vs-per-row differentials).

Design: docs/audit/2026-07-virtual-dataset-perf/19-pr4d-optional-seed-coalesce.md (F14).

[aaj3f]

Consolidating at maintainer request. #1501 (PR-4d — batched-OPTIONAL seed coalescing) is carried forward in #1507 (the R2RML/Iceberg virtual-dataset performance program), which brings the #1475–#1507 lineage forward as a single reviewable unit. Its verification of record is the program's corpus benchmark, published as C2-bench-wave1.md (native == virtual, 0 hash mismatches). The pre-consolidation branch tip is preserved at tag archive/pre-refactor-2026-07-21/perf_r2rml-pr4d — no history was rewritten. Closing in favor of #1507; discussion continues there.