perf(r2rml): F20 RefObjectMap-target resolution prune — q031 72s→5.2s (kill the shared-predicate fan-out) - #1502
Closed
aaj3f wants to merge 4 commits into
Closed
perf(r2rml): F20 RefObjectMap-target resolution prune — q031 72s→5.2s (kill the shared-predicate fan-out)#1502aaj3f wants to merge 4 commits into
aaj3f wants to merge 4 commits into
Conversation
This was referenced Jul 15, 2026
…ed-predicate fan-out When a star subject ?p is bound SOLELY as one RefObjectMap object (parent class C), constrain its downstream single-predicate star's TriplesMap resolution to C via the existing class_prune_hint (reused verbatim; zero operator change). Kills q031's ?p edw:name fan-out to all 6 name-bearing dims (load_table 7->2). Soundness: invariant A (join-var provenance) via produced_vars — DECLINE if ?p is produced by any other pattern (BIND target / path endpoint / VALUES / UNION / OPTIONAL / second ref of a different parent / a second object binding) or carries a class assertion; FILTER/MINUS/EXISTS constrain-not-bind so do NOT decline. Invariant B (template-disjointness) = wildcard_class_fusion_is_safe verbatim (the F10 vertical-partition guard). Cross-scope pre-bound ?p is sound by conjunction (the in-scope ref member restricts ?p to class-C FK objects regardless of the prune). V1: single-predicate star only. Switch FLUREE_R2RML_REF_TARGET_PRUNE (default on). 10 hermetic tests (positive + BIND/UNION/VALUES/2nd-binder/ diff-parent/template-share/class-assert declines + FILTER-no-decline).
The static in-scope analysis can't see a ?p seeded from outside the graph scope (outer VALUES / cross-graph join). Two engine tests via CountingProvider prove the prune stays sound by conjunction: a pre-bound VALID product IRI still yields its name (self-check — join works + prune keeps the row), and a pre-bound NON-product (Store) IRI dies (0 rows) because the in-scope ref member has no matching fact row — identically pruned or not.
…D (6th) — small-dim cache rejected F19: the base-ctor ::default()-vs-clone() suspected cause is refuted — the named sites are root constructors (no parent Arc to clone) and q031's real per-batch derivation (with_active_graph) already shares the memo Arc (confirm-experiment: 1x shared vs 5x fresh-ctx control). The only true resetter is with_graph_ref (deliberate for const_sid_cache's store switch, not on any corpus path); cloning the memo THERE is sound (key is store-disambiguated) and is filed as the one residual fix, DEFERRED to the next PR touching SERVICE/multi-source R2RML. No corpus query is affected. F19 closes. q031 residual (doc 20 §8): the 243 surviving scan_table are 241 budgeted DIM_PRODUCT MAIN re-scans of ?p edw:name (row_budget disqualifies scan_cache; filters=0, not parent lookups). A/B differential collapsing them 241->3: delta -0.5s (worthless). hyp-A (~5-7s) refuted, hyp-B (fact-floor) wins. A query-scoped small-dim main-scan cache is REJECTED. q031's residual is the fact-scan+materialize+loadTable floor, shared with q016; ledger to loadTable-cache + PR-2a/materialization.
Clears this PR's own fmt hunks (#1502 review). Restacked over the fix/f18 docs fold-in: doc-20 merged to preserve BOTH the design-resolved 'Open questions RESOLVED' record and this PR's Implemented note + section (8) measured-residual attribution.
aaj3f
force-pushed
the
perf/r2rml-q031-refprune
branch
from
July 15, 2026 17:17
eee6435 to
c38d412
Compare
Contributor
Author
|
Consolidating at maintainer request. #1502 (F20 — the RefObjectMap-target resolution prune) is carried forward in #1507 (the R2RML/Iceberg virtual-dataset performance program), which brings the #1475–#1507 lineage forward as a single reviewable unit. Its verification of record is the program's corpus benchmark, published as C2-bench-wave1.md (native == virtual, 0 hash mismatches). The pre-consolidation branch tip is preserved at tag |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
F20 — RefObjectMap-target resolution prune (kill the shared-predicate fan-out)
Stacked on
perf/r2rml-pr4d(#1501). Kills q031's over-scan: a star subject?pbound solely as oneRefObjectMapobject (parent class C) has its downstream single-predicate star's TriplesMap resolution constrained to C, so?p edw:name ?pnno longer fans out to all sixedw:name-bearing dims.q031 — honest numbers (cache-thrashed, full-corpus 3-rep protocol)
72 s → 5.21 s (~14×), a deterministic fan-out kill (7→2 loads, 1448→243 scans) — the load/scan collapse is cache-independent; the wall is the measured cache-thrashed 3-rep median.
rows_onlyparity (row count ON==OFF; hashes differ by unordered-LIMIT nondeterminism, expected).Residual (measured, not inferred — doc 20 §8)
The 243 surviving scans decompose (per-scan trace, reverted) as 2 fact + 1 deduped DIM_PRODUCT parent lookup + 241 budgeted DIM_PRODUCT MAIN re-scans of
?p edw:name(row_budget=Some(5000)alone disqualifies thescan_cache;filters=0, not parent lookups). An A/B differential collapsing those 241→3 measured Δ −0.5 s — they cost nothing. A query-scoped small-dim main-scan cache is therefore measured-and-rejected. q031's residual is the fact-scan + materialize +loadTablefloor — the same floor as q016-post-#1501; the isolated first-ask still pays ~2–3 s ofloadTable(amortized to load=0 in corpus order). Named levers to close ≤3 s (not in this PR): the loadTable-METADATA cache + PR-2a / materialization.The verdict chain (six evidence-forced flips — preserved in the register)
q031 cost six flips: PR-5-suspected → bisect-refuted → memo×limit-pushdown → cold-floor loadTable-pin → measurement: RefObjectMap-target fan-out (F20) → the residual measurement above (small-dim cache rejected). Along the way F19 (the "query-scoped parent-memo doesn't engage" theory) was refuted — the memo already shares on q031's path (
with_active_graphclones the Arc; confirm-experiment 1× shared vs 5× fresh-ctx control); its one sound residual fix (with_graph_refmemo-clone) is filed and deferred, affecting no corpus query.Soundness (the load-bearing part)
The prune reuses the existing
class_prune_hintverbatim (zero operator change). It fires only when BOTH:?p's only binding in scope is exactly oneRefObjectMapmember (unique parent class C). Computed viaPattern::produced_vars(notreferenced_vars) — so aBINDtarget or a path endpoint (which rebind?p) decline, whileFILTER/MINUS/(NOT) EXISTS(which only constrain) do not. One principled choice; future IR pattern types inherit correct treatment. Declines on: a second object binding, a UNION/VALUES-bound?p, a secondRefObjectMapwith a different parent, a mixed ref/non-ref predicate, an ambiguous parent class, or a standalone class assertion on?p.wildcard_class_fusion_is_safe(C)— the F10 vertical-partition guard, verbatim; a map sharing C's subject template that also maps the predicate declines.Cross-scope pre-bound
?p(seeded outside the scope) is sound by conjunction: the in-scope ref member restricts?pto class-C FK objects regardless of the prune (engine-tested via CountingProvider: valid product 1→1 row; non-product Store IRI → 0 rows, identically pruned or not).Kill switch:
FLUREE_R2RML_REF_TARGET_PRUNE(default on) — off is byte-identical to the pre-F20 fan-out, as is every DECLINE path.Tests + gate
12 tests: 10 hermetic rewrite (positive + BIND/UNION/VALUES/2nd-binder/different-parent/template-share/class-assert declines + FILTER-no-decline) + 2 engine-level (cross-scope conjunction). Full
fluree-db-querylib green (1247). Full-corpus baseline at this head: 54 records, 0 hash mismatches, 0 perf violations; the 3 expected-error queries unaffected; 44/51 ok-queries ≤3 s.CI note: this PR's base is
perf/r2rml-pr4d, notmain, so repo CI does not fire on it (stacked-PR convention).