perf(cpp): tune NUMA mmap policy for CPU inference

[Jackson57279]

Summary

• Added plans/oxidize-cpp-cpu-performance.md with the scoped CPU performance spec.
• Added autotune mmap_advice so dense single-node loads use sequential_prefetch, while huge or near-RAM-limit interleaved loads use random mmap advice.
• Capped ai-box-style dense dual-socket CPU autotune at single/16, and made split GGUF fingerprints sum all shards.
• Applied the selected mmap advice in GgufModel::load() / split GGUF loading, and exposed it in --print-plan --json.
• Fixed batched-forward KV cache layer indexing to use the capped runtime kv_context_ instead of advertised GGUF context size.
• Updated the remote ai-box benchmark helper for >5B, >35B, and >500B-class model scenarios.

Validation

Local:

• git diff --check
• cmake --build oxidize-cpp/build --target oxidize-cpp autotune_test gguf_mmap_advice_test kernels_test parity_test grad_check -j$(nproc)
• ctest --test-dir oxidize-cpp/build --output-on-failure -> 5/5 passed
• scripts/test_bench_ai_box_matrix.sh
• scripts/test_cpp_kv_context_stride.sh

Remote ai@192.168.1.132:

• Plan small Llama 3.1 8B: single, threads=16, mmap_advice=sequential_prefetch
• Plan medium Mixtral 8x22B: single, threads=16, mmap_advice=sequential_prefetch
• Plan >500B-class Kimi/K2 file: interleave, threads=48, mmap_advice=random
• Small >5B run completed: prefill_tps=1.626562, decode_tps=1.410998, total_s=56.694285
• Medium >35B run completed: prefill_tps=0.185467, decode_tps=0.208863, total_s=131.529042

• 500B-class Kimi/K2 run reached a clean current-limit failure under interleave/48: LlamaModel: MLA / lightning-attn / shortconv architecture is unsupported (BIG_RC=1)

The >500B-class file is present and exercises planning/loading, but generation is not available until oxidize-cpp supports that MLA/lightning-attn/shortconv architecture.

---

Summary by cubic

Tune CPU inference by auto-selecting mmap advice based on NUMA mode and model size, and apply it during GGUF loading. Also cap dense dual-socket plans to 16 threads and fix a KV-cache stride bug.

• New Features

  • Add mmap_advice to autotune: sequential_prefetch for single-node dense loads; random for huge or near-RAM-limit interleaved loads.
  • Apply advice in GgufModel::load() (including split GGUF); CLI passes the advice; --print-plan --json now shows it.
  • Cap dense dual-socket CPU plans at single/16; fingerprinting sums split GGUF shard sizes; CLI validates model existence before planning.
  • Add tests for advice mapping and split fingerprinting; update ai-box benchmarks for small/medium/500B-class scenarios.

• Bug Fixes

  • Use runtime kv_context_ for KV-cache layer indexing in batched forward to avoid stride errors.

^{Written for commit 7a32d74. Summary will update on new commits.}

[Jackson57279]

Closing as superseded by the #28 → #29 → #31 stack: #28's MmapPolicy generalizes the mmap_advice here (same madvise mechanism, four policies vs two), the kv_context_ stride fix is already included verbatim in that stack, and #31's benchmarks reversed the single/16 thread cap in favor of interleave + all cores for large dense models. Unique bits worth lifting from codex/cpp-perf-ulw if wanted: the plans/oxidize-cpp-cpu-performance.md spec, split-GGUF shard-sum fingerprinting in autotune, and the bench-ai-box matrix test scripts.