Exception Containment + CPU fallback on accelerated backend failure

AGENTS.md

@@ -39,6 +39,25 @@ scripts/ci/clang-format.sh --check    # verify, no changes
   formatted. CI gates our C/C++ in
   `.github/workflows/clang-format.yml`.
 
+## C ABI Exception Discipline
+
+No C++ exception may escape a public entry point.
+
+- A new public entry point must either route through an `api_guard_*`
+  wrapper (`src/transcribe.cpp`) or be nothrow by construction. Device and
+  registry queries are not pure reads; guard them.
+- Entry points with ownership out-params enforce "non-OK => `*out == NULL`,
+  nothing leaked" in their forwarders on every error return.
+- Teardown never uses raw `ggml_backend_free` / `ggml_backend_buffer_free` /
+  `ggml_backend_sched_free` in library code: use `transcribe::safe_*` from
+  `src/transcribe-backend.h`. `tests/lint_teardown.cmake` fails CI on
+  violations.
+- Host log callbacks are contained at the emission site
+  (`transcribe_log_invoke`).
+- The `TRANSCRIBE_TEST_*` fault hooks (`_DEV_INIT_THROW`, `_TEARDOWN_THROW`)
+  intentionally ship in release artifacts for wheel clean-install CI.
+  Present-but-empty values are inert.
+
 ## Verification
 
 - End-to-end numerical checks:

CMakeLists.txt

@@ -387,6 +387,15 @@ if(TRANSCRIBE_GGML_BACKEND_DL)
     set(GGML_NATIVE OFF CACHE BOOL "" FORCE)
 endif()
 
+# MSVC: use /EHs so exceptions thrown by ggml's C-ABI frames unwind to
+# transcribe.cpp's public-entry guards. Apply before ggml and src are added.
+if(MSVC)
+    string(REPLACE "/EHsc" "/EHs" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
+    if(NOT CMAKE_CXX_FLAGS MATCHES "/EHs")
+        set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /EHs")
+    endif()
+endif()
+
 # ggml's CMake declares its own warning flags; let it.
 add_subdirectory(ggml)
 

docs/environment-variables.md

@@ -15,6 +15,13 @@ copies and writes raw activations to disk, so it carries I/O cost and a
 data-leak surface. Leave it unset in production unless you are deliberately
 collecting dumps.
 
+The two `TRANSCRIBE_TEST_*_THROW` fault-injection hooks also ship in release
+binaries on purpose: wheel clean-install CI proves exception containment
+against the exact artifact users install (see `AGENTS.md`, "C ABI Exception
+Discipline"). Unlike the boolean convention above, **any** non-empty value
+arms them (including `0`); unset or empty is inert. Leave them unset outside
+of tests.
+
 | Variable | Effect |
 | --- | --- |
 | `TRANSCRIBE_NO_FLASH` | Disable flash attention on encoder and decoder (forces the manual F32 path). |
@@ -24,6 +31,8 @@ collecting dumps.
 | `TRANSCRIBE_DUMP_DIR=<dir>` | Enable the per-stage tensor dumper; writes `<name>.f32` + `<name>.json` per dumped tensor into `<dir>`. The basis for the numerical-comparison harness (`scripts/compare_tensors.py`). |
 | `TRANSCRIBE_PERF_DEBUG` | Print a per-stage timing breakdown to stderr (DEBUG log) on the families that profile (`cohere`, `granite`, `canary`, `canary_qwen`, `moonshine`, `moonshine_streaming`, `qwen3_asr`, `whisper`). For whisper, a value containing `cpu` or `all` additionally prints the CPU sub-section breakdown. |
 | `TRANSCRIBE_VOXTRAL_REALTIME_STREAM_TIMING` | Print a per-component streaming wall-time breakdown at stream finalize (voxtral_realtime). |
+| `TRANSCRIBE_TEST_DEV_INIT_THROW=<match>` | Fault injection: backend device init (`ggml_backend_dev_init`) throws for devices whose name contains `<match>` (`*` matches every device). Exercises throw → skip → CPU-fallback in backend probing; an explicit backend request fails with `TRANSCRIBE_ERR_BACKEND`. Used by `backend_init_throw_unit` and `scripts/ci/vulkan_degradation_check.py`. |
+| `TRANSCRIBE_TEST_TEARDOWN_THROW` | Fault injection: any non-empty value injects a throw after each real free inside the `transcribe::safe_*` teardown wrappers, proving containment without leaking the handle. Used by `teardown_safety_unit`. |
 
 ## Tier 2 — validation hooks
 

include/transcribe.h

@@ -52,6 +52,12 @@
  *   releases. Consumers should rebuild against matching headers and
  *   should not assume any layout, enum value, or symbol set is frozen.
  *
+ * Exception safety (C ABI):
+ * - No C++ exception escapes a public entry point. Allocation failure maps
+ *   to TRANSCRIBE_ERR_OOM, backend/driver failure to TRANSCRIBE_ERR_BACKEND,
+ *   and free/teardown functions never fail. The log callback receives
+ *   exception detail when available.
+ *
  * Params (size-aware structs):
  * - Every caller-owned public struct crossing the ABI carries `struct_size`
  *   as field 0, both for inputs (params, family extensions) and outputs
@@ -407,6 +413,10 @@ typedef void (*transcribe_log_callback)(transcribe_log_level level, const char *
  *   cb == NULL            logging explicitly disabled: library and ggml
  *                         messages are dropped, nothing goes to stderr.
  *
+ * The callback must not throw. If it does anyway, the exception is contained
+ * at the emission site, the message is dropped, and a note is written to
+ * stderr.
+ *
  * Call once at process startup (see the threading contract above).
  */
 TRANSCRIBE_API void transcribe_log_set(transcribe_log_callback cb, void * userdata);
@@ -1433,6 +1443,10 @@ TRANSCRIBE_API const struct transcribe_model * transcribe_get_model(const struct
  * On success, results are populated on the session and may be read via
  * the accessors below. Calling transcribe_run() again replaces the
  * previous result on the same session.
+ *
+ * TRANSCRIBE_ERR_BACKEND from this function (and batch/streaming run calls)
+ * is recoverable in-process: free the session and model, reload with
+ * transcribe_model_load_params::backend = TRANSCRIBE_BACKEND_CPU, and retry.
  */
 TRANSCRIBE_API transcribe_status transcribe_run(struct transcribe_session *          session,
                                                 const float *                        pcm,

scripts/ci/vulkan_degradation_check.py

@@ -15,7 +15,10 @@
 
   loader      A loader (+ driver, e.g. lavapipe or a real GPU) is present.
               The SAME artifacts must discover a Vulkan device. With
-              --model: a real transcription runs with backend="vulkan".
+              --model: a real transcription runs with backend="vulkan";
+              then, with TRANSCRIBE_TEST_DEV_INIT_THROW=* injecting a
+              device-init throw, explicit vulkan fails cleanly and default
+              backend falls back to CPU.
 
 Used by: provider-dl-vulkan (python-bindings.yml, hand-assembled provider
 dir), clean-install (python-wheels.yml, the shipped wheel — ships in the
@@ -29,6 +32,7 @@
 
 import argparse
 import array
+import os
 import sys
 import wave
 
@@ -97,6 +101,28 @@ def main() -> int:
             text = transcribe(t, args.model, args.audio, backend="vulkan")
             print("text (vulkan):", text.strip())
             assert "country" in text.lower(), text
+
+            # Same shipped artifact, injected device-init failure.
+            os.environ["TRANSCRIBE_TEST_DEV_INIT_THROW"] = "*"
+            try:
+                try:
+                    t.Model(args.model, backend="vulkan")
+                except t.TranscribeError as exc:
+                    print("throwing vulkan device correctly rejected:", exc)
+                else:
+                    raise AssertionError(
+                        "backend='vulkan' must fail when device init throws")
+                with t.Model(args.model) as m:
+                    # m.backend is the ggml backend name ("CPU", "Vulkan0"),
+                    # not a lowercase kind string.
+                    assert m.backend.lower() == "cpu", m.backend
+                    with m.session() as s:
+                        text = s.run(load_pcm(args.audio)).text
+                print("text (throwing devices, default backend):",
+                      text.strip())
+                assert "country" in text.lower(), text
+            finally:
+                del os.environ["TRANSCRIBE_TEST_DEV_INIT_THROW"]
         print("ok: loader present — the same artifacts discovered Vulkan")
     return 0
 

src/arch/canary/canary.h

@@ -60,7 +60,7 @@ struct CanaryKvCache {
 
     void free() {
         if (buffer != nullptr) {
-            ggml_backend_buffer_free(buffer);
+            safe_buffer_free(buffer);
             buffer = nullptr;
         }
         if (ctx != nullptr) {

src/arch/canary/model.cpp

@@ -46,7 +46,7 @@ static_assert(std::is_base_of_v<transcribe_session, CanarySession>);
 CanarySession::~CanarySession() {
     kv_cache.free();
     if (sched != nullptr) {
-        ggml_backend_sched_free(sched);
+        safe_sched_free(sched);
         sched = nullptr;
     }
     if (compute_ctx != nullptr) {
@@ -183,27 +183,27 @@ CanaryModel::~CanaryModel() {
         bn_fused_ctx = nullptr;
     }
     if (bn_fused_buffer != nullptr) {
-        ggml_backend_buffer_free(bn_fused_buffer);
+        safe_buffer_free(bn_fused_buffer);
         bn_fused_buffer = nullptr;
     }
     if (conv_pw_f32_ctx != nullptr) {
         ggml_free(conv_pw_f32_ctx);
         conv_pw_f32_ctx = nullptr;
     }
     if (conv_pw_f32_buffer != nullptr) {
-        ggml_backend_buffer_free(conv_pw_f32_buffer);
+        safe_buffer_free(conv_pw_f32_buffer);
         conv_pw_f32_buffer = nullptr;
     }
     if (ctx_meta != nullptr) {
         ggml_free(ctx_meta);
         ctx_meta = nullptr;
     }
     if (backend_buffer != nullptr) {
-        ggml_backend_buffer_free(backend_buffer);
+        safe_buffer_free(backend_buffer);
         backend_buffer = nullptr;
     }
     for (auto it = plan.scheduler_list.rbegin(); it != plan.scheduler_list.rend(); ++it) {
-        ggml_backend_free(*it);
+        safe_backend_free(*it);
     }
     plan.scheduler_list.clear();
     plan.primary      = nullptr;

src/arch/canary_qwen/model.cpp

@@ -61,7 +61,7 @@ CanaryQwenSession::~CanaryQwenSession() {
     kv_cache.free();
     kv_cache_batch.free();
     if (sched != nullptr) {
-        ggml_backend_sched_free(sched);
+        safe_sched_free(sched);
         sched = nullptr;
     }
     if (compute_ctx != nullptr) {
@@ -76,27 +76,27 @@ CanaryQwenModel::~CanaryQwenModel() {
         ctx_meta = nullptr;
     }
     if (backend_buffer != nullptr) {
-        ggml_backend_buffer_free(backend_buffer);
+        safe_buffer_free(backend_buffer);
         backend_buffer = nullptr;
     }
     if (bn_fused_buffer != nullptr) {
-        ggml_backend_buffer_free(bn_fused_buffer);
+        safe_buffer_free(bn_fused_buffer);
         bn_fused_buffer = nullptr;
     }
     if (bn_fused_ctx != nullptr) {
         ggml_free(bn_fused_ctx);
         bn_fused_ctx = nullptr;
     }
     if (conv_pw_f32_buffer != nullptr) {
-        ggml_backend_buffer_free(conv_pw_f32_buffer);
+        safe_buffer_free(conv_pw_f32_buffer);
         conv_pw_f32_buffer = nullptr;
     }
     if (conv_pw_f32_ctx != nullptr) {
         ggml_free(conv_pw_f32_ctx);
         conv_pw_f32_ctx = nullptr;
     }
     if (linear_f32_buffer != nullptr) {
-        ggml_backend_buffer_free(linear_f32_buffer);
+        safe_buffer_free(linear_f32_buffer);
         linear_f32_buffer = nullptr;
     }
     if (linear_f32_ctx != nullptr) {
@@ -105,7 +105,7 @@ CanaryQwenModel::~CanaryQwenModel() {
     }
     packed_gate_up.free();
     for (auto it = plan.scheduler_list.rbegin(); it != plan.scheduler_list.rend(); ++it) {
-        ggml_backend_free(*it);
+        safe_backend_free(*it);
     }
     plan.scheduler_list.clear();
     plan.primary      = nullptr;
@@ -357,7 +357,7 @@ transcribe_status promote_conv_pw_to_f32_on_cpu(CanaryQwenModel & m) {
     const auto * f16_traits = ggml_get_type_traits(GGML_TYPE_F16);
     if (f16_traits == nullptr || f16_traits->to_float == nullptr) {
         log_msg(TRANSCRIBE_LOG_LEVEL_WARN, "canary_qwen: no f16 to_float trait — skipping conv promotion");
-        ggml_backend_buffer_free(m.conv_pw_f32_buffer);
+        safe_buffer_free(m.conv_pw_f32_buffer);
         m.conv_pw_f32_buffer = nullptr;
         ggml_free(m.conv_pw_f32_ctx);
         m.conv_pw_f32_ctx = nullptr;
@@ -483,7 +483,7 @@ transcribe_status promote_linears_bf16_to_f32_on_cpu(CanaryQwenModel & m) {
     const auto * bf16_traits = ggml_get_type_traits(GGML_TYPE_BF16);
     if (bf16_traits == nullptr || bf16_traits->to_float == nullptr) {
         log_msg(TRANSCRIBE_LOG_LEVEL_WARN, "canary_qwen: no bf16 to_float trait — skipping linear promotion");
-        ggml_backend_buffer_free(m.linear_f32_buffer);
+        safe_buffer_free(m.linear_f32_buffer);
         m.linear_f32_buffer = nullptr;
         ggml_free(m.linear_f32_ctx);
         m.linear_f32_ctx = nullptr;

src/arch/cohere/cohere.h

@@ -78,7 +78,7 @@ struct CohereKvCache {
 
     void free() {
         if (buffer != nullptr) {
-            ggml_backend_buffer_free(buffer);
+            safe_buffer_free(buffer);
             buffer = nullptr;
         }
         if (ctx != nullptr) {

src/arch/cohere/model.cpp

@@ -49,7 +49,7 @@ static_assert(std::is_base_of_v<transcribe_session, CohereSession>);
 CohereSession::~CohereSession() {
     kv_cache.free();
     if (sched != nullptr) {
-        ggml_backend_sched_free(sched);
+        safe_sched_free(sched);
         sched = nullptr;
     }
     if (compute_ctx != nullptr) {
@@ -193,27 +193,27 @@ CohereModel::~CohereModel() {
         bn_fused_ctx = nullptr;
     }
     if (bn_fused_buffer != nullptr) {
-        ggml_backend_buffer_free(bn_fused_buffer);
+        safe_buffer_free(bn_fused_buffer);
         bn_fused_buffer = nullptr;
     }
     if (conv_pw_f32_ctx != nullptr) {
         ggml_free(conv_pw_f32_ctx);
         conv_pw_f32_ctx = nullptr;
     }
     if (conv_pw_f32_buffer != nullptr) {
-        ggml_backend_buffer_free(conv_pw_f32_buffer);
+        safe_buffer_free(conv_pw_f32_buffer);
         conv_pw_f32_buffer = nullptr;
     }
     if (ctx_meta != nullptr) {
         ggml_free(ctx_meta);
         ctx_meta = nullptr;
     }
     if (backend_buffer != nullptr) {
-        ggml_backend_buffer_free(backend_buffer);
+        safe_buffer_free(backend_buffer);
         backend_buffer = nullptr;
     }
     for (auto it = plan.scheduler_list.rbegin(); it != plan.scheduler_list.rend(); ++it) {
-        ggml_backend_free(*it);
+        safe_backend_free(*it);
     }
     plan.scheduler_list.clear();
     plan.primary      = nullptr;

src/arch/funasr_nano/model.cpp

@@ -47,7 +47,7 @@ FunAsrNanoSession::~FunAsrNanoSession() {
     kv_cache.free();
     kv_cache_batch.free();
     if (sched != nullptr) {
-        ggml_backend_sched_free(sched);
+        safe_sched_free(sched);
         sched = nullptr;
     }
     if (compute_ctx != nullptr) {
@@ -62,12 +62,12 @@ FunAsrNanoModel::~FunAsrNanoModel() {
         ctx_meta = nullptr;
     }
     if (backend_buffer != nullptr) {
-        ggml_backend_buffer_free(backend_buffer);
+        safe_buffer_free(backend_buffer);
         backend_buffer = nullptr;
     }
     packed_gate_up.free();
     for (auto it = plan.scheduler_list.rbegin(); it != plan.scheduler_list.rend(); ++it) {
-        ggml_backend_free(*it);
+        safe_backend_free(*it);
     }
     plan.scheduler_list.clear();
     plan.primary      = nullptr;

src/arch/gigaam/model.cpp

@@ -43,7 +43,7 @@ static_assert(std::is_base_of_v<transcribe_session, GigaamSession>);
 
 GigaamSession::~GigaamSession() {
     if (sched != nullptr) {
-        ggml_backend_sched_free(sched);
+        safe_sched_free(sched);
         sched = nullptr;
     }
     if (compute_ctx != nullptr) {
@@ -59,11 +59,11 @@ GigaamModel::~GigaamModel() {
         ctx_meta = nullptr;
     }
     if (backend_buffer != nullptr) {
-        ggml_backend_buffer_free(backend_buffer);
+        safe_buffer_free(backend_buffer);
         backend_buffer = nullptr;
     }
     for (auto it = plan.scheduler_list.rbegin(); it != plan.scheduler_list.rend(); ++it) {
-        ggml_backend_free(*it);
+        safe_backend_free(*it);
     }
     plan.scheduler_list.clear();
     plan.primary      = nullptr;

src/arch/granite/model.cpp

@@ -41,7 +41,7 @@ GraniteSession::~GraniteSession() {
     kv.free();
     kv_batch.free();
     if (sched != nullptr) {
-        ggml_backend_sched_free(sched);
+        safe_sched_free(sched);
         sched = nullptr;
     }
     if (compute_ctx != nullptr) {
@@ -56,7 +56,7 @@ GraniteModel::~GraniteModel() {
         bn_fused_ctx = nullptr;
     }
     if (bn_fused_buffer != nullptr) {
-        ggml_backend_buffer_free(bn_fused_buffer);
+        safe_buffer_free(bn_fused_buffer);
         bn_fused_buffer = nullptr;
     }
     packed_gate_up.free();
@@ -65,11 +65,11 @@ GraniteModel::~GraniteModel() {
         ctx_meta = nullptr;
     }
     if (backend_buffer != nullptr) {
-        ggml_backend_buffer_free(backend_buffer);
+        safe_buffer_free(backend_buffer);
         backend_buffer = nullptr;
     }
     for (auto it = plan.scheduler_list.rbegin(); it != plan.scheduler_list.rend(); ++it) {
-        ggml_backend_free(*it);
+        safe_backend_free(*it);
     }
     plan.scheduler_list.clear();
     plan.primary      = nullptr;