stablediffusion-ggml backend segfaults instead of returning an error when a Metal pipeline fails to compile, and its stderr never reaches the LocalAI log

[akboogie83]

Summary

When a Metal pipeline fails to compile, the backend logs the failure and then
uses the pipeline anyway. The null dereference takes SIGSEGV inside cgo and the
process dies. LocalAI sees only exitCode=2 and rpc error: EOF.

Compounding it, the backend's stderr is not attached to anything LocalAI logs, so
the underlying Metal error is invisible from the server side. Between the two,
a precise and well-reported error from Metal reaches the operator as an opaque
crash.

Environment

• LocalAI 4.7.1
• Backend metal-stablediffusion-ggml, digest
  sha256:9cafbf0956c55791cac472f20c73f55f425eeaefeb9d9a9bb22c9c5b8e8d7258
• macOS 26.5.2 (build 25F84), Apple M3 Ultra, 256 GB unified memory

What the operator sees

Nothing useful. A /video request fails, and the LocalAI log contains only:

exitCode=2
rpc error: code = Unavailable desc = error reading from server: EOF

No Metal error, no kernel name, no indication that anything compiled or failed to
compile. Two earlier attempts against this backend produced exactly this and were
abandoned undiagnosed, because there was nothing to go on.

What is actually happening

Metal reports the failure correctly and precisely:

compiling pipeline: kernel_mul_mv_ext_bf16_f32_r1_5
[ERROR] MTLLibraryErrorDomain Code=5 "Function kernel_mul_mv_ext_bf16_f32_r1_5
        was not found in the library"
SIGSEGV: segmentation violation, signal arrived during cgo execution

The [ERROR] line shows the failure is detected and logged. The SIGSEGV
immediately after shows the result is then used without a nil check. A returned
error at that point would have made this a one-line diagnosis.

How to see it at all, which is the second half of the report

The backend's stderr does not reach LocalAI's log. To capture it we had to edit
the backend's own launcher, appending a redirect to the exec line in
run.sh:

exec "$CURDIR"/stablediffusion-ggml "$@" 2>>/tmp/sd-ggml-stderr.log

then reproduce, then restore the file. That is not a reasonable diagnostic path
for an operator, and it is the only reason the missing kernel in the companion
report was ever identified (companion issue: #AAA).

What would help

1. Check the pipeline before using it. A failed newComputePipelineState (or
   equivalent) should propagate an error rather than being dereferenced. This is
   the fix that matters; it converts an opaque crash into an actionable message
   for every future missing-kernel case, not just this one.
2. Surface backend stderr in the LocalAI log, or document how to capture it.
   A backend that dies with exitCode=2 and no output leaves the operator with
   nothing.
3. Optionally, include the Metal error text in the gRPC error returned to
   LocalAI, so rpc error: EOF carries a cause.

Why this is worth fixing independently of the missing kernel

The companion report is one absent kernel in one build, and a rebuild will fix
it. This defect turns any future pipeline-compile failure, from any cause,
into an uninformative segfault with no log output. The next occurrence will cost
the next person the same days.

Notes

Happy to run diagnostics or test a patched build on request. The model and
hardware stay available specifically to re-test.

[localai-org-maint-bot]

I’m picking this up. I’ll trace both the Metal pipeline failure path and LocalAI’s backend stderr handling from current master, add regression coverage where practical, and open a focused fix.