diff --git a/CHANGELOG.md b/CHANGELOG.md index c47ab1e..2a11561 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,8 @@ Maintenance release: llama.cpp bump to b10133, on top of b10075 from v0.8.38. Unlike recent bumps this range **breaks the upstream C API**, so one NIF change was required — see the `model_load/10` entry below. The public Elixir API is unchanged. Full suite against the rebuilt NIF with real GGUF models (smoke, slow -and MTP speculative-decoding tests all included): 252 passed, 0 failures. +and MTP speculative-decoding tests all included): 263 passed, 0 failures, over 12 +consecutive runs. ### Changed @@ -31,6 +32,10 @@ and MTP speculative-decoding tests all included): 252 passed, 0 failures. - **vendor / ci**: update cpp-httplib to 0.51.0 (#26067) and `subprocess.h` (#26061); fix the SYCL package shared-library lookup (#25987). - **NIF `model_load/10`** — Now maps the existing `:use_mmap` / `:use_mlock` / `:use_direct_io` options onto the new `llama_load_mode` enum instead of setting the three removed booleans. The documented precedence is preserved (direct I/O takes precedence over mmap, and mlock implies mmap): `dio` > `mlock` > `mmap` > `none`. The Elixir API and its defaults are unchanged, so no caller updates are needed; all four resolved modes were verified against a real model load. +### Fixed + +- **Documented the Metal teardown abort in the smoke-test instructions (exit 134)** — On Metal, a fully green suite could still abort while the VM shut down, printing `263 passed, 0 failures` and then exiting 134 with `ggml-metal-device.m:622: GGML_ASSERT([rsets->data count] == 0) failed`. llama.cpp's Metal device is owned by a function-local `static std::vector`, so it is destroyed by `__cxa_finalize_ranges` *after* the BEAM calls `exit(3)`, and its destructor asserts that the global `MTLResidencySet` collection is empty. The BEAM makes no promise that NIF resource destructors have run by then, so a model or context still holding Metal buffers trips the assert; it reproduced on 3 of 12 all-green full-suite runs. `test/test_helper.exs` now explains the mechanism and the documented smoke-test commands set `GGML_METAL_NO_RESIDENCY=1`, which stops the collection from being allocated at all so `ggml_metal_rsets_free` returns early — removing the assert rather than racing it (12 of 12 clean runs). A residency set is only an OS memory-residency hint, so buffer allocation and compute are unchanged. The variable has to come from the shell: `System.put_env/2` does not reach the C `getenv` ggml reads. Nothing in the library changed — production keeps the upstream default, and CI is unaffected either way because it runs on Linux with `LLAMA_BACKEND=cpu` and loads no models. The same exit-ordering race applies to any node that halts while a model or context is still referenced; `backend_free/0` does not help, as it only reaches `ggml_quantize_free()`. + ## v0.8.38 Maintenance release: llama.cpp bump to b10075. Full suite against the rebuilt diff --git a/test/server_smoke_test.exs b/test/server_smoke_test.exs index f5f5021..c6beb3d 100644 --- a/test/server_smoke_test.exs +++ b/test/server_smoke_test.exs @@ -1,8 +1,12 @@ defmodule LlamaCppEx.ServerSmokeTest do # Integration tests for Server behaviors that need a real model. Run with: # + # GGML_METAL_NO_RESIDENCY=1 \ # LLAMA_SMOKE_GEN_MODEL=/path/to/chat-model.gguf mix test --include smoke # + # On Metal, GGML_METAL_NO_RESIDENCY=1 keeps a passing run from aborting with + # exit 134 while the VM tears down; test/test_helper.exs explains why. + # # async: false — each test starts its own server against the GPU. use ExUnit.Case, async: false diff --git a/test/smoke_test.exs b/test/smoke_test.exs index 3cf5b3f..0fefaca 100644 --- a/test/smoke_test.exs +++ b/test/smoke_test.exs @@ -10,10 +10,14 @@ defmodule LlamaCppEx.SmokeTest do is slow. Run it explicitly after bumping the `vendor/llama.cpp` submodule or rebuilding the NIF: + GGML_METAL_NO_RESIDENCY=1 \\ LLAMA_SMOKE_GEN_MODEL=/path/to/chat-model.gguf \\ LLAMA_SMOKE_EMB_MODEL=/path/to/embedding-model.gguf \\ mix test test/smoke_test.exs --include smoke + On Metal, `GGML_METAL_NO_RESIDENCY=1` keeps a passing run from aborting with + exit 134 while the VM tears down; `test/test_helper.exs` explains why. + `LLAMA_SMOKE_GEN_MODEL` is required for the generation/chat/grammar tests; `LLAMA_SMOKE_EMB_MODEL` is optional and only enables the embedding tests. Any small instruct model works for generation (e.g. a 0.5B–3B Q4 GGUF). diff --git a/test/test_helper.exs b/test/test_helper.exs index 1fe67fe..99aa029 100644 --- a/test/test_helper.exs +++ b/test/test_helper.exs @@ -1,8 +1,27 @@ # Smoke tests load real GGUF models and run inference; they are excluded by # default. Run them explicitly with model paths set, e.g.: # +# GGML_METAL_NO_RESIDENCY=1 \ # LLAMA_SMOKE_GEN_MODEL=/path/to/chat-model.gguf \ # LLAMA_SMOKE_EMB_MODEL=/path/to/embedding-model.gguf \ # mix test --include smoke # +# `GGML_METAL_NO_RESIDENCY=1` is only needed on Metal, and only to keep the VM +# from aborting *after* the suite has passed: +# +# ggml-metal-device.m:622: GGML_ASSERT([rsets->data count] == 0) failed +# +# llama.cpp's Metal device is owned by a function-local `static std::vector`, so +# it is destroyed by `__cxa_finalize_ranges` after the BEAM calls `exit(3)`, and +# its destructor asserts that the global `MTLResidencySet` collection is empty. +# The BEAM does not promise that NIF resource destructors have run by then, so a +# model or context still holding Metal buffers trips the assert and the run exits +# 134 with a green result already printed (measured at 3 of 12 full-suite runs). +# Setting the variable stops the collection from being allocated at all, which +# removes the assert instead of racing it; residency sets are only an OS +# memory-residency hint, so nothing under test changes. +# +# It has to come from the shell: `System.put_env/2` does not reach the C +# `getenv` that ggml reads, and the library deliberately does not set it either +# — production keeps the upstream default. ExUnit.start(exclude: [:smoke])