docs(audit): F18→F20 investigation record + PR-4d/loadTable slate reservations - #1509
Closed
aaj3f wants to merge 12 commits into
Closed
docs(audit): F18→F20 investigation record + PR-4d/loadTable slate reservations#1509aaj3f wants to merge 12 commits into
aaj3f wants to merge 12 commits into
Conversation
… red herring) Sketch investigation A/B: PARENT_MEMO on/off is IDENTICAL for q031 in both limit-pushdown states (1448 with budget, 8 without) — so the memo is a no-op; and LIMIT_PUSHDOWN=0 is ALSO ~76-81s (full FACT_INVENTORY_SNAPSHOT materialize, no LIMIT cut). Real mechanism = LIMIT-pushdown row-budget windowing x correlated-inner-join driving re-invokes build_progress ~724x (each re-scanning fact+dim). Two fix lanes (B: decouple LIMIT cut from driving granularity in the budget/window lifecycle, PR-4a; A: hoist parent-lookup / fix memo survival, PR-8b), pending a targeted trace of the memo-miss sub-question. Covers all 5 sketch points + DoD. STOP for review.
…or, not memo/limit Trace done. Split locked (1 fact + 1447 single-file dim re-scans, fact does NOT restart) but the wall is NOT the dim re-scans: q031 = 72s w/ budget vs 76s w/o (dim re-scans ~free, load_table cached at 7), and 188ms fully warm. Span breakdown: load_table 21.2s (cold catalog, biggest) + cold parquet/decode/fetch + ~18s materialization. A memo/dim fix (Lane A) buys only ~10-20s (72->~55s, misses <=3s). The FILTER(oh<rp) is un-prunable+selective so the LIMIT can't cut the fact scan. So q031's <=3s is a COLD-FLOOR/cache-residency problem (PR-8 family: kill the 21s cold load_table), not the memo x limit-pushdown premise. Recommend reclassifying F18 (re-scope to cold-floor / mark cache-residency exception / ship dim-memo cleanup as ~10-20s partial). Memo-miss cause found: correlated sub-ctx re-creates r2rml_parent_memo (::default) per batch. STOP for re-frame decision.
… routing row to cold-floor
… 1) — loadTable-pin + residency
…cing (one inner scan)
…north-star burndown (F18 cold-floor re-scope, q029/q016 corrections)
…ObjectMap-target resolution fan-out (F8/PR-3 family), not a leak; fact is 7670-file decode-wall
…q031 root) + q031 resolution-prune sketch; ROADMAP slate→F20, shared decode-floor note
…ound ~3-5s, measured via q032 proxy; ≤3s needs a creds-cache follow-up); rulings folded (single-ref, mandatory template-disjointness, switch name)
…he (the named <=3s closer) with AJ's no-creds-on-disk hard constraint; correct q031 slate figure to measured ~3-5s loadTable-bound
…(scan 182->3; ~11s join/materialize residual); revise shared-floor (PR-2a IS on q016's path, not 'stays closed'); oracle 0-mismatch
…ass_prune_hint; parent class via rom.parent_triples_map; invariant B = wildcard_class_fusion_is_safe); precise rewrite.rs plan + pollution-scan for invariant A
Contributor
Author
|
Consolidating at maintainer request. #1509 (the F18→F20 investigation record and slate reservations — a docs-only branch opened so the trail is reviewed rather than riding to |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Docs-only branch (verified: no engine code — audit docs + one bench chore) that had no PR of its own; opening it so the F18→F20 investigation trail is reviewed rather than riding to main unreviewed, and so the stacked-merge sequence has no orphan (closes the #1501/#1502 review finding).
Contents: the F18 cold-floor re-scope, the measurement that refuted the L1 pin-leak reading (7 loads = a RefObjectMap-target resolution fan-out, not a leak), the filing of F20 (the real q031 root) with its resolution-prune sketch, the PR-4d (F14) seed-coalescing sketch + measured decomposition (q016 43.1s→17.47s), the loadTable-metadata-cache slate reservation with AJ's no-creds-on-disk hard constraint, and the north-star bar ruling stamped into the burndown. The branch name predates the F18→F20 re-scope (F18 proved to be resolution-fan-out, not memo/limit) — kept for git history; the record itself is corrected on its own line.
Base is fix/f9-virtual-curie (#1499); the PR-4d code (#1501) and everything above it stacks on this branch's tip.