handy-computer · cjpais · Jun 29, 2026 · Jun 29, 2026 · Jun 29, 2026 · Jun 29, 2026
@@ -12,9 +12,10 @@ final-layer hidden states (doubling the projector K/V input from 1024 to
 
 Offline multilingual speech-to-text with word-level timestamps. Covers
 English plus French, German, Spanish, and Portuguese (no Japanese on this
-variant). Takes a 16 kHz mono WAV and produces a transcript, optionally
-interleaved with `[SS:N]` word-timestamp markers (centiseconds since
-segment start).
+variant). Takes a 16 kHz mono WAV and produces a transcript; with
+`--timestamps word` it returns per-word start/end times. Internally the model
+emits `[T:N]` end-of-word centisecond markers; the runtime parses them into
+structured word timestamps and returns a clean transcript.
 
 This variant is transcription-only. Unlike the base
 [`granite-speech-4.1-2b`](granite-speech-4.1-2b.md), it does not perform
@@ -74,11 +75,20 @@ build/bin/transcribe-cli \
   samples/jfk.wav
 ```
 
-Output includes `[SS:N]` markers (centiseconds since segment start)
-interleaved with words:
+The runtime returns a clean transcript plus structured per-word start/end
+times. (Internally the model emits `[T:N]` markers giving each word's end time
+in centiseconds, modulo 1000 with a 10 s rollover that the runtime unwraps; the
+markers are stripped from the returned text.)
 
 ```
-[SS:23] And [SS:52] so [SS:92] my [SS:121] fellow [SS:154] Americans ...
+text: and so my fellow americans ask not what your country can do for you ask what you can do for your country
+words: 22
+  [   0.30 ->    0.57] and
+  [   0.57 ->    0.95] so
+  [   0.95 ->    1.25] my
+  [   1.25 ->    1.60] fellow
+  [   1.60 ->    2.13] americans
+  ...
 ```
 
 ## Performance
@@ -143,7 +153,7 @@ Linux 6.18 (Fedora 43), transcribe.cpp `dbe5814`.
 | Transcribe (English)        | Yes    |
 | Transcribe (fr/de/es/pt)    | Yes    |
 | Translate                   | No (ASR-only variant; use the base [granite-speech-4.1-2b](granite-speech-4.1-2b.md) for translation) |
-| Word-level timestamps       | Yes (`--timestamps word`, `[SS:N]` markers) |
+| Word-level timestamps       | Yes (`--timestamps word`, structured per-word t0/t1 parsed from the model's `[T:N]` markers) |
 | Speaker diarization         | No (upstream supports via prompt; not exposed in v1 of transcribe.cpp) |
 
 ## Numerical Validation

@@ -22,7 +22,7 @@ perspective they are one family of GGUFs.
   LLM head, autoregressive; 1.31% BF16 / 1.32% Q8_0 WER on LibriSpeech
   test-clean. Improved punctuation and casing over 4.0-1b.
 - **Most features.** `granite-speech-4.1-2b-plus` — same 4.1 base plus
-  word-level timestamps (`[SS:N]` centisecond markers). Encoder
+  word-level timestamps (parsed from the model's `[T:N]` centisecond markers). Encoder
   concatenates a mid-layer and the final layer (`cat_hidden_layers=[3]`),
   doubling projector K/V width.
 - **Fastest, ASR-only.** `granite-speech-4.1-2b-nar` — non-autoregressive
@@ -101,8 +101,8 @@ Translation (`granite-4.0-1b-speech`, `granite-speech-4.1-2b`):
   from the audio. The `-plus` variant is ASR-only and does not translate.
 
 Plus only (`granite-speech-4.1-2b-plus`):
-- **Word-level timestamps** as `[SS:N]` centisecond markers interleaved
-  with words (`--timestamps word`).
+- **Word-level timestamps** via `--timestamps word` — structured per-word
+  start/end times, parsed from the model's `[T:N]` centisecond markers.
 
 NAR only (`granite-speech-4.1-2b-nar`):
 - **Single-pass non-autoregressive decode** — fastest of the four.

@@ -74,15 +74,15 @@ context, NVIDIA's self-reported FLEURS en-US WER is **7.91%** (and an
 | --- | ---: | ---: |
 | Reference (NeMo) | 7.99 | 3.03 |
 | F32    | 7.97 | 3.04 |
-| F16    | 7.97 | 3.04 |
-| Q8_0   | 7.88 | 3.05 |
-| Q6_K   | 8.02 | 3.08 |
+| F16    | 7.97 | 3.03 |
+| Q8_0   | 7.88 | 3.06 |
+| Q6_K   | 8.02 | 3.07 |
 | Q5_K_M | 8.15 | 3.10 |
-| Q4_K_M | 8.49 | 3.30 |
+| Q4_K_M | 8.49 | 3.28 |
 
 The F32 reference dtype meets the measured-Oracle gate on both datasets.
 F16/Q8_0/Q6_K/Q5_K_M land inside the reference 95% CI; Q4_K_M carries the
-largest quantization loss (+0.50 on FLEURS, +0.27 on LibriSpeech) but is
+largest quantization loss (+0.50 on FLEURS, +0.25 on LibriSpeech) but is
 accepted for shipping.
 
 ## Quick Start

@@ -155,7 +155,7 @@ Allowed statuses: `PASS`, `SKIP - not exposed by runtime`, `ACCEPTED GAP - <reas
 | Transcribe | granite-speech-4.1-2b-plus | explicit language hint | `build/bin/transcribe-cli -m models/granite-speech-4.1-2b-plus/granite-speech-4.1-2b-plus-BF16.gguf --language en samples/jfk.wav` | non-empty plausible English transcript | PASS (granite-4 system-role template; transcript matches reference within one comma) |
 | Transcribe | granite-speech-4.1-2b-plus | auto / no language hint | `build/bin/transcribe-cli -m models/granite-speech-4.1-2b-plus/granite-speech-4.1-2b-plus-BF16.gguf samples/jfk.wav` | non-empty plausible transcript | PASS |
 | Translate | granite-speech-4.1-2b-plus | n/a | n/a | runtime rejects `--translate` (`supports_translate=false`) | SKIP - not exposed by runtime (upstream model card lists -plus as ASR-only; the base granite-speech-4.1-2b is the AST variant. The fused LLM can be prompt-coerced to translate — the smoke produced "Ich bitte meine Mitbürger, ..." — but the converter advertises `stt.capability.translate=false`, so the dispatcher rejects the task.) |
-| Word timestamps | granite-speech-4.1-2b-plus | text-protocol output | `build/bin/transcribe-cli -m models/granite-speech-4.1-2b-plus/granite-speech-4.1-2b-plus-BF16.gguf --timestamps word samples/jfk.wav` | `[T:N]` centisecond markers interleaved with the transcript text | PASS (model emits `[SS:N]` markers: e.g. "[SS:23] And [SS:52] so [SS:92] my [SS:121] fellow [SS:154] Americans ..."). Spec says `[T:N]`; granite uses `[SS:N]` literally. |
+| Word timestamps | granite-speech-4.1-2b-plus | structured per-word timestamps | `build/bin/transcribe-cli -m models/granite-speech-4.1-2b-plus/granite-speech-4.1-2b-plus-BF16.gguf --timestamps word samples/jfk.wav` | clean transcript + per-word t0/t1, parsed from the model's `[T:N]` end-of-word centisecond markers | PASS (model emits `[T:N]` markers, e.g. "_ [T:30] and [T:57] so [T:95] my [T:125] fellow [T:160] americans ..."; the runtime drops the `_` placeholders, unwraps the mod-1000 / 10 s rollover, and returns structured words. Verified against IBM's transformers reference with the model-card timestamps prompt; the earlier `[SS:N]` reading was an artifact of an incorrect prompt string.) |
 | Speaker diarization | granite-speech-4.1-2b-plus | multi-speaker audio | n/a | `[Speaker N]:` tags preceding speaker turns in the text | SKIP - not exposed by runtime (user-deferred to later) |
 | Incremental decoding | granite-speech-4.1-2b-plus | prefix_text continuation | n/a | transcript continues from supplied prefix | SKIP - not exposed by runtime |
 

@@ -43,7 +43,11 @@ transcribe_cpp:
 
 The `<dataset>` suffix is the spec's `wer.metadata_key` (default
 `librispeech_test_clean`); the `<machine>` suffix is the `perf:` rig key with `-`
-mapped to `_`. Standard HF keys (`license`, `language`, `pipeline_tag`,
+mapped to `_`. A spec can also publish secondary benchmarks inline: any dataset-named key
+under `wer:` whose value is a `{quant: wer%}` map (e.g. `librispeech_test_clean:`)
+is emitted as its own `wer_<dataset>` block alongside the headline one — so a
+model whose per-quant column is e.g. FLEURS can still expose LibriSpeech
+machine-readably. Add a dataset by adding a key; no wrapper needed. Standard HF keys (`license`, `language`, `pipeline_tag`,
 `base_model`, `tags`, …) are emitted alongside and unchanged by this block.
 
 ## Reading it

@@ -50,20 +50,28 @@ def build_transcribe_cpp_block(spec: dict) -> str:
         return ""
 
     caps = spec.get("capabilities", {})
-    dataset_key = spec["wer"].get("metadata_key", "librispeech_test_clean")
-    block: dict = {
-        f"wer_{dataset_key}": {
-            q["name"].lower(): float(str(q["wer"]).rstrip("%")) for q in spec["quants"]
-        },
-        **{
-            f"rtf_{machine.replace('-', '_')}": backends
-            for machine, backends in spec["perf"].items()
-        },
-        "streaming": bool(caps.get("streaming", False)),
-        "translate": bool(caps.get("translate", False)),
-        "lang_detect": bool(caps.get("lang_detect", False)),
-        "timestamps": caps.get("timestamps", "none"),
+    wer = spec["wer"]
+    dataset_key = wer.get("metadata_key", "librispeech_test_clean")
+    block: dict = {}
+    # Headline dataset: per-quant WER taken from the `quants:` column.
+    block[f"wer_{dataset_key}"] = {
+        q["name"].lower(): float(str(q["wer"]).rstrip("%")) for q in spec["quants"]
     }
+    # Any additional per-quant WER maps listed inline under `wer:` (keyed by
+    # dataset name, e.g. `librispeech_test_clean:`) are emitted as their own
+    # `wer_<dataset>` blocks. Only dict values count as datasets; scalar keys
+    # (metadata_key, source, notes) are skipped.
+    for key, per_quant in wer.items():
+        if isinstance(per_quant, dict):
+            block[f"wer_{key}"] = {
+                str(q).lower(): float(str(v).rstrip("%")) for q, v in per_quant.items()
+            }
+    for machine, backends in spec["perf"].items():
+        block[f"rtf_{machine.replace('-', '_')}"] = backends
+    block["streaming"] = bool(caps.get("streaming", False))
+    block["translate"] = bool(caps.get("translate", False))
+    block["lang_detect"] = bool(caps.get("lang_detect", False))
+    block["timestamps"] = caps.get("timestamps", "none")
     dumped = yaml.safe_dump(
         {"transcribe_cpp": block}, sort_keys=False, default_flow_style=False
     )

@@ -50,7 +50,7 @@ capabilities:
   streaming: false
   translate: true
   lang_detect: false
-  timestamps: word  # none | segment | word | token
+  timestamps: none  # none | segment | word | token
 
 # Speedup-over-realtime (×RT) per rig/backend, from docs/models; published
 # raw as rtf_<machine> in the metadata block.
@@ -62,6 +62,7 @@ perf:
     metal: 11
     cpu: 5
   ryzen-4750u:
+    vulkan: 2.95
     cpu: 1.6
 
 wer:

@@ -63,6 +63,7 @@ perf:
     metal: 18
     cpu: 4
   ryzen-4750u:
+    vulkan: 3.65
     cpu: 1.5
 
 wer:

@@ -41,8 +41,9 @@ summary: |
   decoder) with two changes: the encoder concatenates mid-layer (idx 3) and
   final-layer hidden states (`cat_hidden_layers=[3]`, doubling the projector
   K/V input from 1024 to 2048), and the LM token embeddings are tied with
-  the lm_head. Takes a 16 kHz mono WAV and produces a transcript, optionally
-  interleaved with `[SS:N]` word-timestamp markers. Transcribes English,
+  the lm_head. Takes a 16 kHz mono WAV and produces a transcript, with
+  `--timestamps word` returning structured per-word timestamps (parsed from the
+  model's `[T:N]` centisecond markers). Transcribes English,
   French, German, Spanish, and Portuguese (no Japanese on this variant).
   This variant is transcription-only: unlike the base granite-speech-4.1-2b,
   it does not perform speech translation.
@@ -66,6 +67,7 @@ perf:
     metal: 11
     cpu: 5
   ryzen-4750u:
+    vulkan: 2.75
     cpu: 1.4
 
 wer:

@@ -51,7 +51,7 @@ capabilities:
   streaming: false
   translate: true
   lang_detect: false
-  timestamps: word  # none | segment | word | token
+  timestamps: none  # none | segment | word | token
 
 # Speedup-over-realtime (×RT) per rig/backend, from docs/models; published
 # raw as rtf_<machine> in the metadata block.
@@ -63,6 +63,7 @@ perf:
     metal: 12
     cpu: 4
   ryzen-4750u:
+    vulkan: 2.80
     cpu: 1.4
 
 wer:

@@ -90,7 +90,14 @@ wer:
   metadata_key: fleurs_en
   source: FLEURS test en (en-US), offline att_context_size=[56, 13]
   notes: |
-    WER measured on FLEURS test en (647 utterances), greedy RNN-T, --language en-US, whisper-normalizer scoring; the per-quant column is FLEURS en. NeMo reference baseline on the same manifest: 7.99% (NVIDIA self-reports 7.91% en-US). On LibriSpeech test-clean (2620 utterances) the same presets score F32 3.04 / F16 3.04 / Q8_0 3.05 / Q6_K 3.08 / Q5_K_M 3.10 / Q4_K_M 3.30, against a 3.03% NeMo reference.
+    WER measured on FLEURS test en (647 utterances), greedy RNN-T, --language en-US, whisper-normalizer scoring; the per-quant column is FLEURS en. NeMo reference baseline on the same manifest: 7.99% (NVIDIA self-reports 7.91% en-US). On LibriSpeech test-clean (2620 utterances) the same presets score F32 3.04 / F16 3.03 / Q8_0 3.06 / Q6_K 3.07 / Q5_K_M 3.10 / Q4_K_M 3.28, against a 3.03% NeMo reference.
+  librispeech_test_clean:
+    f32: 3.04
+    f16: 3.03
+    q8_0: 3.06
+    q6_k: 3.07
+    q5_k_m: 3.10
+    q4_k_m: 3.28
 
 quants:
   - name: F32

@@ -9,14 +9,9 @@ void apply_family_invariants(transcribe_model & model) {
 
     caps.native_sample_rate = 16000;
 
-    // Word timestamps are advertised on the -plus variant (the
-    // timestamps prompt template emits inline `<|n.nn|> word` markers).
-    // The 1b / 2b variants only emit transcript text. The runtime
-    // distinguishes via a per-variant capability KV (the converter sets
-    // stt.capability.word_timestamps appropriately); apply_family_invariants
-    // sets the default upper bound here so the loader's
-    // read_capability_kv step can lower it for variants that do not
-    // expose timestamps.
+    // Only -plus exposes word timestamps ("[T:N]" markers). WORD is the family
+    // upper bound; granite::load() lowers it to NONE per variant from the GGUF's
+    // stt.capability.word_timestamps.
     caps.max_timestamp_kind = TRANSCRIBE_TIMESTAMPS_WORD;
 
     // Translation is advertised on the 1b / 2b variants via a separate