Skip to content

Explain the Metal teardown abort and disable residency sets in smoke runs - #78

Merged
nyo16 merged 1 commit into
masterfrom
fix-metal-teardown-abort
Jul 26, 2026
Merged

Explain the Metal teardown abort and disable residency sets in smoke runs#78
nyo16 merged 1 commit into
masterfrom
fix-metal-teardown-abort

Conversation

@nyo16

@nyo16 nyo16 commented Jul 26, 2026

Copy link
Copy Markdown
Owner

On Metal, a full suite could print 263 passed, 0 failures and then abort the VM
with exit 134:

ggml-metal-device.m:622: GGML_ASSERT([rsets->data count] == 0) failed

Root cause

The backtrace puts the abort after the suite is over, in static teardown:

exit(3) → __cxa_finalize_ranges
  → ~vector<unique_ptr<ggml_metal_device>>   (ggml-metal-device.cpp)
    → ggml_metal_device_free → ggml_metal_rsets_free → ggml_abort

ggml_metal_device_get keeps its devices in a function-local static std::vector,
so the device outlives the run and is destroyed by __cxa_finalize_ranges after
the BEAM calls exit(3). Its destructor asserts that the process-global
MTLResidencySet collection is empty. A residency set is added when a Metal
buffer is created and removed in ggml_metal_buffer_free, so a non-empty
collection means a llama_model / llama_context NIF resource has not been
destructed — and the BEAM makes no promise that resource destructors run before
halt.

Hence the race: 3 of 12 all-green full-suite runs aborted (3B model + 9B MTP,
real GGUF models, smoke and slow included), and it fires on a 6-test run as
readily as on the whole suite — it is not accumulation, and not a failing test.

Fix

Set GGML_METAL_NO_RESIDENCY=1 in the documented smoke-test commands, and write
the mechanism down in test/test_helper.exs where the next person will hit it.
With the variable set, dev->rsets is never allocated and
ggml_metal_rsets_free returns early, so the assert is unreachable rather than
merely unlikely. Residency sets are an OS memory-residency hint, so buffer
allocation and compute are unchanged.

What this deliberately is not

  • Not backend_free/0 in an after_suite hook. llama_backend_free() is
    just ggml_quantize_free() — it frees no models, contexts or buffers and
    cannot empty the collection.
  • Not System.put_env/2 in test_helper.exs. Tried and verified
    ineffective: the device still logged use residency sets = true and a run
    still aborted. os:putenv does not reach the C getenv ggml reads, so the
    variable has to come from the shell.

Verification

aborts result wall clock
before 3 / 12 runs (exit 134) 263 passed, 0 failures ~30 s
after 0 / 12 runs 263 passed, 0 failures ~30 s

Every run after the change logs use residency sets = false. Default
mix test is green (189 passed, 4 skipped, 8 excluded) and mix format --check-formatted is clean.

Scope

The library is untouched — production keeps the upstream default. CI is
unaffected either way: it runs on Linux with LLAMA_BACKEND=cpu and loads no
models, so no Metal device is ever created. The same exit-ordering race does
apply to any node that halts while a model or context is still referenced, which
the CHANGELOG entry notes; fixing that would need explicit release of those
resources before halt.

The v0.8.39 CHANGELOG entry is corrected in place (its full-suite figure now
reads 263 passed over 12 consecutive runs) rather than annotated retroactively,
because v0.8.39 is not tagged yet — remote tags stop at v0.8.36.

…runs

On Metal, a full suite could print `263 passed, 0 failures` and then abort
the VM with exit 134:

  ggml-metal-device.m:622: GGML_ASSERT([rsets->data count] == 0) failed

The backtrace puts the abort after the suite is over, in static teardown:

  exit(3) -> __cxa_finalize_ranges
    -> ~vector<unique_ptr<ggml_metal_device>> (ggml-metal-device.cpp)
      -> ggml_metal_device_free -> ggml_metal_rsets_free -> ggml_abort

ggml_metal_device_get keeps its devices in a function-local
`static std::vector`, so the device outlives the run and is destroyed by
__cxa_finalize_ranges after the BEAM calls exit(3). Its destructor asserts
the process-global MTLResidencySet collection is empty. A residency set is
added when a Metal buffer is created and removed in ggml_metal_buffer_free,
so a non-empty collection means a llama model or context resource has not
been destructed — and the BEAM makes no promise that NIF resource
destructors run before halt. Hence the race: 3 of 12 all-green full-suite
runs aborted (3B model + 9B MTP, real GGUF models, smoke and slow
included), and it fires on a 6-test run as readily as on the whole suite.

Set GGML_METAL_NO_RESIDENCY=1 in the documented smoke-test commands.
With it, dev->rsets is never allocated and ggml_metal_rsets_free returns
early, so the assert is unreachable rather than merely unlikely: 12 of 12
runs clean, same wall clock (~30s), 263 passed each. Residency sets are an
OS memory-residency hint, so buffer allocation and compute are unchanged.

Two things it deliberately is not:

  * not `backend_free/0` in an after_suite hook — llama_backend_free() is
    just ggml_quantize_free(); it frees no models, contexts or buffers and
    cannot empty the collection.
  * not System.put_env/2 in test_helper.exs — verified ineffective: the
    device still logged `use residency sets = true` and a run still
    aborted. os:putenv does not reach the C getenv ggml reads, so the
    variable has to come from the shell.

The library is untouched: production keeps the upstream default. CI is
unaffected either way, since it runs on Linux with LLAMA_BACKEND=cpu and
loads no models, so no Metal device is ever created.
@nyo16
nyo16 merged commit 31d675f into master Jul 26, 2026
4 checks passed
@nyo16
nyo16 deleted the fix-metal-teardown-abort branch July 26, 2026 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant