Fix option forwarding

CHANGELOG.md

@@ -1,5 +1,36 @@
 # Changelog
 
+## v0.8.39
+
+Maintenance release: llama.cpp bump to b10133, on top of b10075 from v0.8.38.
+Unlike recent bumps this range **breaks the upstream C API**, so one NIF change
+was required — see the `model_load/10` entry below. The public Elixir API is
+unchanged. Full suite against the rebuilt NIF with real GGUF models (smoke, slow
+and MTP speculative-decoding tests all included): 252 passed, 0 failures.
+
+### Changed
+
+- **llama.cpp submodule** — Updated from 76f46ad29 to ff067f76d (58 commits, tag b10133). Two binding-relevant headers changed, one of them breaking:
+  - `include/llama.h` — **breaking**: `llama_model_params` loses the `use_mmap`, `use_direct_io` and `use_mlock` booleans; they are replaced by a single `enum llama_load_mode load_mode` field with values `LLAMA_LOAD_MODE_NONE` / `_MMAP` / `_MLOCK` / `_DIRECT_IO` (`none`/`mmap`/`mlock`/`dio`), plus new `llama_load_mode_name` / `llama_load_mode_from_str` helpers (#20834). Note `LLAMA_LOAD_MODE_MLOCK` means "mmap **and** mlock" — the two are no longer independent.
+  - `common/chat.h` — `common_chat_params::thinking_end_tag` (`std::string`) became `thinking_end_tags` (`std::vector<std::string>`) so the reasoning-budget sampler can accept multiple end sequences (#25544). The binding never read that field, so no change was needed.
+  - `common/common.h` changes (`common_params::load_mode`, `reasoning_budget_end` widened to `std::vector<llama_tokens>`, new MCP server config fields) do not affect the binding, which does not use `common_params`. `common/json-schema-to-grammar.h`, `common/speculative.h`, `common/sampling.h` and every `ggml/include/` header are untouched in this range.
+  - **llama core / models**: add GLM 5.2 Indexer support (#25407); add support for Laguna XS.2 & M.1 (#25165); assorted llama bug fixes (#26051); fix DeepSeek4 APE tensor op in llama-arch (#25945); fix the crafted DeepSeek4 template (#25414); fix the reasoning-preserve variable for DS4 (#25999); cohere2 MoE template parser enforces the JSON schema for text responses when a response schema is provided (#26018); synchronize save-load-state generation in the tests (#26056).
+  - **common**: support multiple end sequences in the reasoning budget sampler (#25544); fix a use-after-free when loading a LoRA adapter fails (#25611); skip the empty implicit default preset (#25643); infer the speculative type from draft-repo sidecars (#25989) and resolve a draft repo to its requested sidecar (#25955).
+  - **ggml**: declare `gguf_writer_base`'s destructor virtual (#25867); enable PowerPC backend variants on AIX (#25983); add the `GGML_BACKEND_DL_IMPL` invocation for the OpenVINO backend (#25795).
+  - **Metal**: add f16 type support to leaky relu (#25981).
+  - **CUDA**: `GET_ROWS` for quantized types (#25962); vectorize same-type `get_rows` with an int4 copy (#25929); improve NVFP4 W4A4 activation quantization (#25730); add `sqrt_softplus` in topk-moe for dsv4 (#25896); fix external compilation of q1_0 MMQ (#25778).
+  - **Vulkan**: refactor `vk_queue` to use per-instance mutexes and unique handles (#23570).
+  - **HIP**: remove rocWMMA FlashAttention (#26046).
+  - **WebGPU**: add a CONV_2D_DW (depthwise conv2d) kernel (#25847); fix WASM compilation with OpenMP (#25943).
+  - **OpenCL**: cache compiled `cl_program` binaries on disk (#26050); do not treat NULL-mask flash attention as causal (#25771).
+  - **hexagon**: further pipeline improvements to the core bits (L2, DMA, MM, FA) (#26049); partial im2col support (#26007); activation ops update (#25974); check tensor type when reusing descriptors (#25968); fix a Windows crash when `op_poll` is enabled (#26029).
+  - **kleidiai**: warn once when a weight type has no KleidiAI kernel (#25701).
+  - **mtmd**: use RAII for setting and resetting non-causal attention (#25723); use `align_corners` for qwen3vl vision position embedding interpolation (#25781).
+  - **convert**: fix the non-MoE NomicBert GGUF conversion error (#25996); handle the HunyuanVL XD-RoPE config (#25514).
+  - **tools/server/ui** (not linked into the binding): MCP stdio support (#26062) and MCP display-name conflict fix (#26011); `"reasoning_effort": "none"` in the OAI API (#26045); a `format` arg on the datetime tool (#26117); missing `adaptive_target`/`adaptive_decay` task parameters in `generation_settings` (#25830); return 400 instead of 500 on validation errors with `X-Conversation-Id` (#25760); properly handle a null `llama_context` (#25868); reduced per-token render cost while streaming (#26053); assorted web UI fixes.
+  - **vendor / ci**: update cpp-httplib to 0.51.0 (#26067) and `subprocess.h` (#26061); fix the SYCL package shared-library lookup (#25987).
+- **NIF `model_load/10`** — Now maps the existing `:use_mmap` / `:use_mlock` / `:use_direct_io` options onto the new `llama_load_mode` enum instead of setting the three removed booleans. The documented precedence is preserved (direct I/O takes precedence over mmap, and mlock implies mmap): `dio` > `mlock` > `mmap` > `none`. The Elixir API and its defaults are unchanged, so no caller updates are needed; all four resolved modes were verified against a real model load.
+
 ## v0.8.38
 
 Maintenance release: llama.cpp bump to b10075. Full suite against the rebuilt

c_src/llama_cpp_ex/llama_nif.cpp

@@ -143,11 +143,15 @@ model_load(ErlNifEnv* env, std::string path, int64_t n_gpu_layers, bool use_mmap
            bool use_mlock, bool use_direct_io, bool vocab_only, bool check_tensors) {
     auto params = llama_model_default_params();
     params.n_gpu_layers = static_cast<int32_t>(n_gpu_layers);
-    params.use_mmap = use_mmap;
     params.main_gpu = static_cast<int32_t>(main_gpu);
     params.split_mode = static_cast<enum llama_split_mode>(split_mode);
-    params.use_mlock = use_mlock;
-    params.use_direct_io = use_direct_io;
+    // Upstream collapsed the use_mmap/use_mlock/use_direct_io booleans into a
+    // single llama_load_mode enum. Preserve the documented precedence of the
+    // Elixir options: direct I/O wins over mmap, and mlock implies mmap.
+    params.load_mode = use_direct_io ? LLAMA_LOAD_MODE_DIRECT_IO
+                     : use_mlock     ? LLAMA_LOAD_MODE_MLOCK
+                     : use_mmap      ? LLAMA_LOAD_MODE_MMAP
+                                     : LLAMA_LOAD_MODE_NONE;
     params.vocab_only = vocab_only;
     params.check_tensors = check_tensors;
 

docs/performance.md

@@ -275,12 +275,17 @@ Additional options when loading models:
 {:ok, model} = LlamaCppEx.load_model("model.gguf",
   n_gpu_layers: -1,       # Offload all layers to GPU
   use_mmap: true,          # Memory-map file (default, faster loading)
-  use_mlock: true,         # Pin in RAM (prevent swapping)
+  use_mlock: true,         # Pin in RAM (prevent swapping); implies use_mmap
   use_direct_io: false,    # Bypass page cache
   check_tensors: true      # Validate tensor data (debugging)
 )
 ```
 
+These three flags resolve to llama.cpp's single `load_mode` enum, with
+`use_direct_io` > `use_mlock` > `use_mmap` precedence (`dio`, `mlock`, `mmap`,
+or `none` when all are false). Only one mode is ever active, so combining them
+is not additive.
+
 ## Complete Optimization Example
 
 Here's a production-ready server configuration combining multiple optimizations:

lib/llama_cpp_ex.ex

@@ -42,43 +42,9 @@ defmodule LlamaCppEx do
     UTF8Stream
   }
 
-  @context_opt_keys [
-    :n_threads,
-    :n_threads_batch,
-    :n_batch,
-    :n_ubatch,
-    :type_k,
-    :type_v,
-    :flash_attn,
-    :offload_kqv,
-    :op_offload,
-    :rope_scaling_type,
-    :rope_freq_base,
-    :rope_freq_scale,
-    :yarn_ext_factor,
-    :yarn_attn_factor,
-    :yarn_beta_fast,
-    :yarn_beta_slow,
-    :yarn_orig_ctx,
-    :attention_type,
-    :no_perf,
-    :swa_full
-  ]
-
-  # Sampling options forwarded to Sampler.create/2 by the generation entry
-  # points. Keep in sync with the options documented on generate/3.
-  @sampler_opt_keys [
-    :seed,
-    :temp,
-    :top_k,
-    :top_p,
-    :min_p,
-    :penalty_repeat,
-    :penalty_freq,
-    :penalty_present,
-    :grammar,
-    :grammar_root
-  ]
+  # Context and sampling options are owned by the modules that consume them —
+  # see Context.tuning_option_keys/0 and Sampler.option_keys/0. Do not copy the
+  # lists here; three copies had already drifted apart.
 
   # Chat-templating options split off before the rest flows to generation.
   @chat_opt_keys [:add_assistant, :enable_thinking, :chat_template_kwargs]
@@ -266,8 +232,8 @@ defmodule LlamaCppEx do
       max_tokens: Keyword.get(opts, :max_tokens, 256),
       n_ctx: Keyword.get(opts, :n_ctx, 2048),
       timeout: Keyword.get(opts, :timeout, 60_000),
-      sampler_opts: Keyword.take(opts, @sampler_opt_keys),
-      ctx_opts: Keyword.take(opts, @context_opt_keys)
+      sampler_opts: Keyword.take(opts, Sampler.option_keys()),
+      ctx_opts: Keyword.take(opts, Context.tuning_option_keys())
     }
   end
 

lib/llama_cpp_ex/context.ex

@@ -1,13 +1,83 @@
 defmodule LlamaCppEx.Context do
   @moduledoc """
   Inference context with KV cache.
+
+  ## Option ownership
+
+  This module is the single source of truth for the options `create/2` accepts.
+  Callers that forward user options into a context (`LlamaCppEx`,
+  `LlamaCppEx.Server`, `LlamaCppEx.MTP`) must select them with
+  `tuning_option_keys/0` rather than keeping their own copy of the list — three
+  hand-maintained copies had already drifted, silently dropping `:n_threads`,
+  `:n_threads_batch` and `:n_ubatch` on `LlamaCppEx.Server`.
+
+  The keys are split by kind:
+
+    * `tuning_option_keys/0` — performance knobs that are safe to forward from
+      any caller. They never change what the context *is*.
+    * `structural_option_keys/0` — options that decide the context's purpose or
+      size (`:embeddings`, `:pooling_type`, `:ctx_type`, `:n_ctx`, ...). Each
+      caller sets these explicitly; forwarding them blindly would let, say,
+      `embeddings: true` turn a generation server into an embedding context.
+
+  Callers pass their own values as `[n_ctx: computed] ++ forwarded_opts`, which
+  wins because `Keyword.get/3` returns the first match.
   """
 
   @enforce_keys [:ref, :model]
   defstruct [:ref, :model]
 
   @type t :: %__MODULE__{ref: reference(), model: LlamaCppEx.Model.t()}
 
+  @tuning_option_keys [
+    :n_threads,
+    :n_threads_batch,
+    :n_batch,
+    :n_ubatch,
+    :type_k,
+    :type_v,
+    :flash_attn,
+    :offload_kqv,
+    :op_offload,
+    :rope_scaling_type,
+    :rope_freq_base,
+    :rope_freq_scale,
+    :yarn_ext_factor,
+    :yarn_attn_factor,
+    :yarn_beta_fast,
+    :yarn_beta_slow,
+    :yarn_orig_ctx,
+    :attention_type,
+    :no_perf,
+    :swa_full
+  ]
+
+  @structural_option_keys [
+    :n_ctx,
+    :n_seq_max,
+    :kv_unified,
+    :embeddings,
+    :pooling_type,
+    :ctx_type,
+    :n_rs_seq
+  ]
+
+  @doc """
+  Options that are safe for a caller to forward from user-supplied opts.
+
+  See the "Option ownership" section in the module doc.
+  """
+  @spec tuning_option_keys() :: [atom()]
+  def tuning_option_keys, do: @tuning_option_keys
+
+  @doc """
+  Options a caller must set explicitly rather than forward blindly.
+
+  See the "Option ownership" section in the module doc.
+  """
+  @spec structural_option_keys() :: [atom()]
+  def structural_option_keys, do: @structural_option_keys
+
   @doc """
   Creates a new inference context for the given model.
 

lib/llama_cpp_ex/model.ex

@@ -8,6 +8,36 @@ defmodule LlamaCppEx.Model do
 
   @type t :: %__MODULE__{ref: reference()}
 
+  @tuning_option_keys [
+    :main_gpu,
+    :split_mode,
+    :tensor_split,
+    :use_mmap,
+    :use_mlock,
+    :use_direct_io,
+    :check_tensors
+  ]
+
+  @structural_option_keys [:n_gpu_layers, :vocab_only]
+
+  @doc """
+  Options that are safe for a caller to forward from user-supplied opts.
+
+  `LlamaCppEx.Server` selects its model options with this function rather than
+  keeping its own copy of the list.
+  """
+  @spec tuning_option_keys() :: [atom()]
+  def tuning_option_keys, do: @tuning_option_keys
+
+  @doc """
+  Options a caller must set explicitly rather than forward blindly.
+
+  `:vocab_only` in particular must never be forwarded into a server — it would
+  load a model with no weights.
+  """
+  @spec structural_option_keys() :: [atom()]
+  def structural_option_keys, do: @structural_option_keys
+
   @doc """
   Loads a GGUF model from the given file path.
 
@@ -21,12 +51,22 @@ defmodule LlamaCppEx.Model do
       Defaults to `:none`.
     * `:tensor_split` - List of floats specifying the proportion of work per GPU
       (e.g. `[0.5, 0.5]` for two GPUs). Defaults to `[]`.
-    * `:use_mlock` - Pin model memory in RAM to prevent swapping. Defaults to `false`.
+    * `:use_mlock` - Pin model memory in RAM to prevent swapping. Implies `:use_mmap`.
+      Defaults to `false`.
     * `:use_direct_io` - Bypass page cache when loading (takes precedence over mmap).
       Defaults to `false`.
     * `:vocab_only` - Load vocabulary and metadata only, skip weights. Defaults to `false`.
     * `:check_tensors` - Validate model tensor data on load. Defaults to `false`.
 
+  > #### Load mode {: .info}
+  >
+  > llama.cpp collapsed its three loading booleans into one `load_mode` enum, so
+  > these options resolve to a single mode with `:use_direct_io` > `:use_mlock` >
+  > `:use_mmap` precedence — respectively `dio`, `mlock`, `mmap`, and `none` when
+  > all are false. Because `mlock` now implies mmap upstream, passing
+  > `use_mlock: true, use_mmap: false` memory-maps the file rather than reading
+  > it into anonymous memory.
+
   ## Examples
 
       {:ok, model} = LlamaCppEx.Model.load("path/to/model.gguf", n_gpu_layers: -1)

lib/llama_cpp_ex/mtp.ex

@@ -52,29 +52,13 @@ defmodule LlamaCppEx.MTP do
           n_draft: pos_integer()
         }
 
-  @context_opt_keys [
-    :n_threads,
-    :n_threads_batch,
-    :n_batch,
-    :n_ubatch,
-    :type_k,
-    :type_v,
-    :flash_attn,
-    :offload_kqv,
-    :op_offload,
-    :rope_scaling_type,
-    :rope_freq_base,
-    :rope_freq_scale,
-    :yarn_ext_factor,
-    :yarn_attn_factor,
-    :yarn_beta_fast,
-    :yarn_beta_slow,
-    :yarn_orig_ctx,
-    :attention_type,
-    :no_perf,
-    :swa_full,
-    :n_ctx
-  ]
+  # Context options forwarded to both the target and draft contexts. The list is
+  # owned by Context (tuning_option_keys/0); MTP additionally lets the caller set
+  # :n_ctx, which is structural everywhere else because each caller normally
+  # computes it.
+  defp forwardable_context_opts(opts) do
+    Keyword.take(opts, [:n_ctx | Context.tuning_option_keys()])
+  end
 
   @doc """
   Initializes an MTP speculative session: builds the target context, the MTP
@@ -98,7 +82,7 @@ defmodule LlamaCppEx.MTP do
     n_draft = Keyword.get(opts, :n_draft, 3)
 
     if is_integer(n_draft) and n_draft > 0 do
-      base_ctx_opts = Keyword.take(opts, @context_opt_keys)
+      base_ctx_opts = forwardable_context_opts(opts)
       main_opts = Keyword.merge(base_ctx_opts, ctx_type: :default)
       # Match upstream server: MTP draft context is created with n_rs_seq=0.
       # The MTP impl handles state rollback internally via cached hidden
@@ -166,19 +150,7 @@ defmodule LlamaCppEx.MTP do
     emit_stats_every = Keyword.get(opts, :emit_stats_every, 0)
     timeout = Keyword.get(opts, :timeout, 60_000)
 
-    sampler_opts =
-      Keyword.take(opts, [
-        :seed,
-        :temp,
-        :top_k,
-        :top_p,
-        :min_p,
-        :penalty_repeat,
-        :penalty_freq,
-        :penalty_present,
-        :grammar,
-        :grammar_root
-      ])
+    sampler_opts = Keyword.take(opts, Sampler.option_keys())
 
     Stream.resource(
       fn ->

lib/llama_cpp_ex/sampler.ex

@@ -11,6 +11,30 @@ defmodule LlamaCppEx.Sampler do
 
   @type t :: %__MODULE__{ref: reference()}
 
+  @option_keys [
+    :seed,
+    :temp,
+    :top_k,
+    :top_p,
+    :min_p,
+    :penalty_repeat,
+    :penalty_freq,
+    :penalty_present,
+    :grammar,
+    :grammar_root
+  ]
+
+  @doc """
+  The options `create/2` accepts.
+
+  This module is the single source of truth: callers that forward user sampling
+  options (`LlamaCppEx`, `LlamaCppEx.Server`) select them with this function
+  instead of keeping their own copy of the list. Every sampling option is safe
+  to forward, so there is no tuning/structural split here.
+  """
+  @spec option_keys() :: [atom()]
+  def option_keys, do: @option_keys
+
   @doc """
   Creates a new sampler chain.