Skip to content

feat(storage): CompactStore v5 mapped graph structures (G-EM0.2) - #8

Open
jarmen423 wants to merge 2 commits into
feature/g-em0-1-direct-mapped-containerfrom
feature/g-em0-2-mapped-graph-structures
Open

feat(storage): CompactStore v5 mapped graph structures (G-EM0.2)#8
jarmen423 wants to merge 2 commits into
feature/g-em0-1-direct-mapped-containerfrom
feature/g-em0-2-mapped-graph-structures

Conversation

@jarmen423

Copy link
Copy Markdown

Summary

Implements G-EM0.2 — bounded graph metadata and mapped proportional structures — on top of G-EM0.1 container mmap ownership.

  • CompactStore payload v5 with checked 64-byte header + 48-byte segment directory (R0 contract)
  • Mapped views for CSR arrays, string dictionaries, sorted ID lookups (O(log N)/O(log E)), and column bodies
  • Split memory accounting (mapped_payload_index_bytes, anonymous_owner_schema_bytes, proportional must be 0)
  • mapped_graph_tests unit suite + compact_store_readonly_graph_parity at two graph sizes
  • v1–v4 compatibility readers preserved; default writer emits v5

Verification

export TMPDIR=/data/tmp
export CARGO_TARGET_DIR=/data/cargo-targets/jfrie-grafeo-em0-2/target
export RUSTC_WRAPPER=sccache SCCACHE_DIR=/data/sccache

cargo test -p grafeo-core --lib --features compact-store mapped_graph_tests -- --nocapture
cargo test -p grafeo-core --lib --features compact-store graph::compact::section:: -- --nocapture
cargo test -p grafeo-engine --features compact-store \
  --test compact_store_direct_mmap \
  --test compact_store_readonly_graph_parity \
  --test compact_store_allocation_inventory \
  --test compact_store_integration \
  --test compact_store_large_string_persistence -- --nocapture

All of the above PASS in this leaf session.

Notes for parent

  • Packet base is G-EM0.1 head 7ae5f217… (feature/g-em0-1-direct-mapped-container).
  • Do not merge to main from this leaf; parent owns integration after review.
  • Zone-map table/block segments (kinds 18–19) are optional and not yet fully populated on the v5 writer; property/CSR/ID paths are the G-EM0.2 exit-gate surface.
  • Concurrent writable open of the same path can invalidate an existing RO mapping (file-backed strings); the direct_mmap test now uses a byte-identical copy for the LegacyEager check.

Implement the R0-accepted v5 segment-directory payload so proportional graph
structures stay file-backed on read-only mapped open: CSR arrays, string
dictionaries, sorted ID lookups, and column bodies are checked mapped views.
Add split memory accounting, mapped_graph_tests, and readonly graph parity at
two sizes. Preserve v1–v4 compatibility readers; default writer emits v5.
…xtures

Emit and restore CompactStore v5 kinds 18–20 (table/block zone maps and
dictionary code index) so mapped reopen keeps prune and O(log D) string
equality. Add sparse-ID, high-fanout, many-block, and 96 KiB fixtures plus a
strict ≥4× mapped-payload size proof for parent H-EVID.RO dispatch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant