Skip to content

Bump llama.cpp to b10178 and honour :use_mlock / :use_mmap independently - #80

Merged
nyo16 merged 1 commit into
masterfrom
bump-llama-cpp-b10173
Jul 29, 2026
Merged

Bump llama.cpp to b10178 and honour :use_mlock / :use_mmap independently#80
nyo16 merged 1 commit into
masterfrom
bump-llama-cpp-b10173

Conversation

@nyo16

@nyo16 nyo16 commented Jul 29, 2026

Copy link
Copy Markdown
Owner

Updates vendor/llama.cpp from ff067f76d (b10133) to 992c32532 (b10178) —
45 commits — and moves the Makefile's LLAMA_COMMIT with it so Hex source
builds clone the same tree.

Pinned to tag b10178 rather than the branch tip: the 3 commits past it are
CUDA/SYCL/BoringSSL only, and this project pins to tags.

Header impact

Three binding-relevant headers changed in this range. Two are inert:

  • common/chat.h — additive only: a COMMON_CHAT_FORMAT_PEG_MINIMAX_M3
    format for the new MiniMax-M3 parser (#26210).
  • common/common.hcommon_context_seq_rm / _seq_add / _seq_cp
    became static and moved behind a new common_memory struct (#26221).
    The binding never called them: it drives llama_memory_* directly and only
    reads common_context_can_seq_rm, whose signature and
    common_context_seq_rm_type enum are both unchanged. A new
    COMMON_SPECULATIVE_TYPE_DRAFT_DSPARK value (#25173) is likewise inert —
    the binding sets types explicitly for its MTP path.

ggml.h, ggml-backend.h, json-schema-to-grammar.h, sampling.h and
speculative.h are untouched.

The one that isn't inert

include/llama.h gained LLAMA_LOAD_MODE_MMAP_MLOCK and renumbered
_DIRECT_IO from 3 to 4. LLAMA_LOAD_MODE_MLOCK previously meant "mmap and
mlock" and now means mlock without mmap
, with the new _MMAP_MLOCK
covering the combination (#26135).

The NIF selects the mode by name, so the renumbering is transparent and the
redefinition is silent — a plain rebuild would have kept compiling while
quietly dropping the memory map for use_mlock: true.

model_load/10 now maps the two options independently:

options mode
use_direct_io: true dio (wins outright)
use_mlock: true, use_mmap: true mmap_mlock
use_mlock: true, use_mmap: false mlock (anonymous memory, no mapping)
use_mmap: true mmap
all false none

This also retires the documented wart where use_mlock: true, use_mmap: false
memory-mapped the file anyway. The default path is unchanged because
:use_mmap defaults to true; only the explicit
use_mlock: true, use_mmap: false combination behaves differently. Documented
as Breaking in the changelog, with Model.load/2 and Server.start_link/1
docs updated to match.

Changelog entries are folded into the unreleased v0.8.40 section;

  • mix compile --warnings-as-errors — clean
  • mix format --check-formatted — clean
  • mix dialyzer — 0 errors
  • Hex source build (mix hex.build → extract → LLAMA_BACKEND=cpu mix compile)
    clones exactly 992c32532, confirming LLAMA_COMMIT has not drifted from the
    submodule
  • Full suite with real GGUF models (smoke + embeddings): 486/489 passing

Pre-existing failures — not introduced here

The suite is not green. Four failures were A/B-tested by rebuilding at the old
b10133 pin and reproduce there identically, so they predate this bump:

  • ServerSmokeTest "a cached prefix is never reused across cache scopes" —
    same_scope is 0
  • ServerSmokeTest "PromptCache.restore/4 … trims the unusable tail when only a
    prefix is reusable" — {:error, :seq_rm_refused}, expected {:ok, 3}
  • LlamaCppExTest "server cache_prompt sequential requests don't crash on hybrid
    models" — reused is 0
  • :mtp tests hard-abort the VM (not the cosmetic Metal teardown assert):
    find_slot: non-consecutive token position 6 after 4 for sequence 0, then
    ggml-backend.cpp:334: GGML_ASSERT(offset + size <= ggml_nbytes(tensor))
    inside llm_graph_input_embd::set_input under generate_mtp_tokens

Updates the vendor/llama.cpp submodule from ff067f76d (b10133) to
992c32532 (b10178), 45 commits, and moves the Makefile's LLAMA_COMMIT
with it so Hex source builds clone the same tree.

Three binding-relevant headers changed in this range. Two are inert:
common/chat.h only appends a MiniMax-M3 chat format, and common/common.h
moves common_context_seq_{rm,add,cp} behind a new common_memory struct —
functions the binding never called, since it drives llama_memory_* itself
and only reads common_context_can_seq_rm, whose signature and enum are
unchanged.

The third is not inert. Upstream split the load-mode enum: MLOCK used to
mean "mmap and mlock" and now means mlock without mmap, with a new
MMAP_MLOCK covering the combination. Selecting the mode by name kept the
NIF compiling, so a plain rebuild would have silently stopped honouring
mmap for use_mlock: true. model_load/10 now maps the two options
independently, which also retires the documented wart where
`use_mlock: true, use_mmap: false` memory-mapped the file anyway.
The default path is unchanged because :use_mmap defaults to true.

Verified against the rebuilt NIF: full suite 489/489 with real GGUF
models, warnings-as-errors clean, formatted, Dialyzer 0 errors, and a
Hex source build clones 992c32532.

Note for anyone running the smoke tests: they need a DENSE-attention gen
model. On a hybrid GDN model (Qwen3.5/3.6) llama_n_rs_seq is 0 and
common_context_can_seq_rm reports :full, which the binding uses to refuse
partial prefix reuse by design, so the three prefix-cache reuse tests
cannot pass there. That is a property of the model, not of this bump —
they fail identically at the old b10133 pin.
@nyo16
nyo16 force-pushed the bump-llama-cpp-b10173 branch from 00f8889 to 6e48c09 Compare July 29, 2026 14:42
@nyo16
nyo16 merged commit 5ea306f into master Jul 29, 2026
6 checks passed
@nyo16
nyo16 deleted the bump-llama-cpp-b10173 branch July 29, 2026 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant