Bump llama.cpp to b10178 and honour :use_mlock / :use_mmap independently - #80
Merged
Conversation
Updates the vendor/llama.cpp submodule from ff067f76d (b10133) to
992c32532 (b10178), 45 commits, and moves the Makefile's LLAMA_COMMIT
with it so Hex source builds clone the same tree.
Three binding-relevant headers changed in this range. Two are inert:
common/chat.h only appends a MiniMax-M3 chat format, and common/common.h
moves common_context_seq_{rm,add,cp} behind a new common_memory struct —
functions the binding never called, since it drives llama_memory_* itself
and only reads common_context_can_seq_rm, whose signature and enum are
unchanged.
The third is not inert. Upstream split the load-mode enum: MLOCK used to
mean "mmap and mlock" and now means mlock without mmap, with a new
MMAP_MLOCK covering the combination. Selecting the mode by name kept the
NIF compiling, so a plain rebuild would have silently stopped honouring
mmap for use_mlock: true. model_load/10 now maps the two options
independently, which also retires the documented wart where
`use_mlock: true, use_mmap: false` memory-mapped the file anyway.
The default path is unchanged because :use_mmap defaults to true.
Verified against the rebuilt NIF: full suite 489/489 with real GGUF
models, warnings-as-errors clean, formatted, Dialyzer 0 errors, and a
Hex source build clones 992c32532.
Note for anyone running the smoke tests: they need a DENSE-attention gen
model. On a hybrid GDN model (Qwen3.5/3.6) llama_n_rs_seq is 0 and
common_context_can_seq_rm reports :full, which the binding uses to refuse
partial prefix reuse by design, so the three prefix-cache reuse tests
cannot pass there. That is a property of the model, not of this bump —
they fail identically at the old b10133 pin.
nyo16
force-pushed
the
bump-llama-cpp-b10173
branch
from
July 29, 2026 14:42
00f8889 to
6e48c09
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates
vendor/llama.cppfromff067f76d(b10133) to992c32532(b10178) —45 commits — and moves the Makefile's
LLAMA_COMMITwith it so Hex sourcebuilds clone the same tree.
Pinned to tag b10178 rather than the branch tip: the 3 commits past it are
CUDA/SYCL/BoringSSL only, and this project pins to tags.
Header impact
Three binding-relevant headers changed in this range. Two are inert:
common/chat.h— additive only: aCOMMON_CHAT_FORMAT_PEG_MINIMAX_M3format for the new MiniMax-M3 parser (#26210).
common/common.h—common_context_seq_rm/_seq_add/_seq_cpbecame
staticand moved behind a newcommon_memorystruct (#26221).The binding never called them: it drives
llama_memory_*directly and onlyreads
common_context_can_seq_rm, whose signature andcommon_context_seq_rm_typeenum are both unchanged. A newCOMMON_SPECULATIVE_TYPE_DRAFT_DSPARKvalue (#25173) is likewise inert —the binding sets
typesexplicitly for its MTP path.ggml.h,ggml-backend.h,json-schema-to-grammar.h,sampling.handspeculative.hare untouched.The one that isn't inert
include/llama.hgainedLLAMA_LOAD_MODE_MMAP_MLOCKand renumbered_DIRECT_IOfrom 3 to 4.LLAMA_LOAD_MODE_MLOCKpreviously meant "mmap andmlock" and now means mlock without mmap, with the new
_MMAP_MLOCKcovering the combination (#26135).
The NIF selects the mode by name, so the renumbering is transparent and the
redefinition is silent — a plain rebuild would have kept compiling while
quietly dropping the memory map for
use_mlock: true.model_load/10now maps the two options independently:use_direct_io: truedio(wins outright)use_mlock: true, use_mmap: truemmap_mlockuse_mlock: true, use_mmap: falsemlock(anonymous memory, no mapping)use_mmap: truemmapnoneThis also retires the documented wart where
use_mlock: true, use_mmap: falsememory-mapped the file anyway. The default path is unchanged because
:use_mmapdefaults totrue; only the explicituse_mlock: true, use_mmap: falsecombination behaves differently. Documentedas Breaking in the changelog, with
Model.load/2andServer.start_link/1docs updated to match.
Changelog entries are folded into the unreleased v0.8.40 section;
mix compile --warnings-as-errors— cleanmix format --check-formatted— cleanmix dialyzer— 0 errorsmix hex.build→ extract →LLAMA_BACKEND=cpu mix compile)clones exactly
992c32532, confirmingLLAMA_COMMIThas not drifted from thesubmodule
Pre-existing failures — not introduced here
The suite is not green. Four failures were A/B-tested by rebuilding at the old
b10133 pin and reproduce there identically, so they predate this bump:
ServerSmokeTest"a cached prefix is never reused across cache scopes" —same_scopeis 0ServerSmokeTest"PromptCache.restore/4 … trims the unusable tail when only aprefix is reusable" —
{:error, :seq_rm_refused}, expected{:ok, 3}LlamaCppExTest"server cache_prompt sequential requests don't crash on hybridmodels" —
reusedis 0:mtptests hard-abort the VM (not the cosmetic Metal teardown assert):find_slot: non-consecutive token position 6 after 4 for sequence 0, thenggml-backend.cpp:334: GGML_ASSERT(offset + size <= ggml_nbytes(tensor))inside
llm_graph_input_embd::set_inputundergenerate_mtp_tokens