Skip to content

perf(r2rml): F20 RefObjectMap-target resolution prune — q031 72s→5.2s (kill the shared-predicate fan-out) - #1502

Closed
aaj3f wants to merge 4 commits into
perf/r2rml-pr4dfrom
perf/r2rml-q031-refprune
Closed

perf(r2rml): F20 RefObjectMap-target resolution prune — q031 72s→5.2s (kill the shared-predicate fan-out)#1502
aaj3f wants to merge 4 commits into
perf/r2rml-pr4dfrom
perf/r2rml-q031-refprune

Conversation

@aaj3f

@aaj3f aaj3f commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

F20 — RefObjectMap-target resolution prune (kill the shared-predicate fan-out)

Stacked on perf/r2rml-pr4d (#1501). Kills q031's over-scan: a star subject ?p bound solely as one RefObjectMap object (parent class C) has its downstream single-predicate star's TriplesMap resolution constrained to C, so ?p edw:name ?pn no longer fans out to all six edw:name-bearing dims.

q031 — honest numbers (cache-thrashed, full-corpus 3-rep protocol)

load_table.n scan_table.n rows wall
OFF (pre-F20 fan-out) 7 1448 5000 154.0 s
ON (F20 prune) 2 243 5000 5.21 s (3-rep median, in-protocol)

72 s → 5.21 s (~14×), a deterministic fan-out kill (7→2 loads, 1448→243 scans) — the load/scan collapse is cache-independent; the wall is the measured cache-thrashed 3-rep median. rows_only parity (row count ON==OFF; hashes differ by unordered-LIMIT nondeterminism, expected).

Residual (measured, not inferred — doc 20 §8)

The 243 surviving scans decompose (per-scan trace, reverted) as 2 fact + 1 deduped DIM_PRODUCT parent lookup + 241 budgeted DIM_PRODUCT MAIN re-scans of ?p edw:name (row_budget=Some(5000) alone disqualifies the scan_cache; filters=0, not parent lookups). An A/B differential collapsing those 241→3 measured Δ −0.5 s — they cost nothing. A query-scoped small-dim main-scan cache is therefore measured-and-rejected. q031's residual is the fact-scan + materialize + loadTable floor — the same floor as q016-post-#1501; the isolated first-ask still pays ~2–3 s of loadTable (amortized to load=0 in corpus order). Named levers to close ≤3 s (not in this PR): the loadTable-METADATA cache + PR-2a / materialization.

The verdict chain (six evidence-forced flips — preserved in the register)

q031 cost six flips: PR-5-suspected → bisect-refuted → memo×limit-pushdown → cold-floor loadTable-pin → measurement: RefObjectMap-target fan-out (F20) → the residual measurement above (small-dim cache rejected). Along the way F19 (the "query-scoped parent-memo doesn't engage" theory) was refuted — the memo already shares on q031's path (with_active_graph clones the Arc; confirm-experiment 1× shared vs 5× fresh-ctx control); its one sound residual fix (with_graph_ref memo-clone) is filed and deferred, affecting no corpus query.

Soundness (the load-bearing part)

The prune reuses the existing class_prune_hint verbatim (zero operator change). It fires only when BOTH:

  • Invariant A (join-var provenance): ?p's only binding in scope is exactly one RefObjectMap member (unique parent class C). Computed via Pattern::produced_vars (not referenced_vars) — so a BIND target or a path endpoint (which rebind ?p) decline, while FILTER/MINUS/(NOT) EXISTS (which only constrain) do not. One principled choice; future IR pattern types inherit correct treatment. Declines on: a second object binding, a UNION/VALUES-bound ?p, a second RefObjectMap with a different parent, a mixed ref/non-ref predicate, an ambiguous parent class, or a standalone class assertion on ?p.
  • Invariant B (template-disjointness): wildcard_class_fusion_is_safe(C) — the F10 vertical-partition guard, verbatim; a map sharing C's subject template that also maps the predicate declines.

Cross-scope pre-bound ?p (seeded outside the scope) is sound by conjunction: the in-scope ref member restricts ?p to class-C FK objects regardless of the prune (engine-tested via CountingProvider: valid product 1→1 row; non-product Store IRI → 0 rows, identically pruned or not).

Kill switch: FLUREE_R2RML_REF_TARGET_PRUNE (default on) — off is byte-identical to the pre-F20 fan-out, as is every DECLINE path.

Tests + gate

12 tests: 10 hermetic rewrite (positive + BIND/UNION/VALUES/2nd-binder/different-parent/template-share/class-assert declines + FILTER-no-decline) + 2 engine-level (cross-scope conjunction). Full fluree-db-query lib green (1247). Full-corpus baseline at this head: 54 records, 0 hash mismatches, 0 perf violations; the 3 expected-error queries unaffected; 44/51 ok-queries ≤3 s.

CI note: this PR's base is perf/r2rml-pr4d, not main, so repo CI does not fire on it (stacked-PR convention).

aaj3f added 4 commits July 15, 2026 13:14
…ed-predicate fan-out

When a star subject ?p is bound SOLELY as one RefObjectMap object (parent class
C), constrain its downstream single-predicate star's TriplesMap resolution to C
via the existing class_prune_hint (reused verbatim; zero operator change). Kills
q031's ?p edw:name fan-out to all 6 name-bearing dims (load_table 7->2).

Soundness: invariant A (join-var provenance) via produced_vars — DECLINE if ?p is
produced by any other pattern (BIND target / path endpoint / VALUES / UNION /
OPTIONAL / second ref of a different parent / a second object binding) or carries
a class assertion; FILTER/MINUS/EXISTS constrain-not-bind so do NOT decline.
Invariant B (template-disjointness) = wildcard_class_fusion_is_safe verbatim (the
F10 vertical-partition guard). Cross-scope pre-bound ?p is sound by conjunction
(the in-scope ref member restricts ?p to class-C FK objects regardless of the
prune). V1: single-predicate star only. Switch FLUREE_R2RML_REF_TARGET_PRUNE
(default on). 10 hermetic tests (positive + BIND/UNION/VALUES/2nd-binder/
diff-parent/template-share/class-assert declines + FILTER-no-decline).
The static in-scope analysis can't see a ?p seeded from outside the graph scope
(outer VALUES / cross-graph join). Two engine tests via CountingProvider prove
the prune stays sound by conjunction: a pre-bound VALID product IRI still yields
its name (self-check — join works + prune keeps the row), and a pre-bound
NON-product (Store) IRI dies (0 rows) because the in-scope ref member has no
matching fact row — identically pruned or not.
…D (6th) — small-dim cache rejected

F19: the base-ctor ::default()-vs-clone() suspected cause is refuted — the named
sites are root constructors (no parent Arc to clone) and q031's real per-batch
derivation (with_active_graph) already shares the memo Arc (confirm-experiment:
1x shared vs 5x fresh-ctx control). The only true resetter is with_graph_ref
(deliberate for const_sid_cache's store switch, not on any corpus path); cloning
the memo THERE is sound (key is store-disambiguated) and is filed as the one
residual fix, DEFERRED to the next PR touching SERVICE/multi-source R2RML. No
corpus query is affected. F19 closes.

q031 residual (doc 20 §8): the 243 surviving scan_table are 241 budgeted
DIM_PRODUCT MAIN re-scans of ?p edw:name (row_budget disqualifies scan_cache;
filters=0, not parent lookups). A/B differential collapsing them 241->3:
delta -0.5s (worthless). hyp-A (~5-7s) refuted, hyp-B (fact-floor) wins. A
query-scoped small-dim main-scan cache is REJECTED. q031's residual is the
fact-scan+materialize+loadTable floor, shared with q016; ledger to loadTable-cache
+ PR-2a/materialization.
Clears this PR's own fmt hunks (#1502 review). Restacked over the fix/f18 docs
fold-in: doc-20 merged to preserve BOTH the design-resolved 'Open questions
RESOLVED' record and this PR's Implemented note + section (8) measured-residual
attribution.
@aaj3f

aaj3f commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

Consolidating at maintainer request. #1502 (F20 — the RefObjectMap-target resolution prune) is carried forward in #1507 (the R2RML/Iceberg virtual-dataset performance program), which brings the #1475#1507 lineage forward as a single reviewable unit. Its verification of record is the program's corpus benchmark, published as C2-bench-wave1.md (native == virtual, 0 hash mismatches). The pre-consolidation branch tip is preserved at tag archive/pre-refactor-2026-07-21/perf_r2rml-q031-refprune — no history was rewritten. Closing in favor of #1507; discussion continues there.

@aaj3f aaj3f closed this Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant