Parakeet TDT v3 (offline): multi-second span silently dropped from transcript

[jonhoo]

I have a 70s window of a lecture recording where running Parakeet TDT 0.6b v3 inference silently drops a ~15s span of clearly-audible speech in the middle of the window. The same audio transcribes correctly when the window is shifted or shortened, and the failure is bit-identical across quantizations, so does not seem to be an audio or precision problem.

Observed via the Rust bindings (transcribe-cpp crate v0.1.1), CPU backend (x86-64, Zen 3), model handy-computer/parakeet-tdt-0.6b-v3-gguf.

Reproduction

https://jon.thesquareplanet.com/share/dropped-span-repro.wav is the audio snippet encoded as 16 kHz mono s16 WAV. When transcribed, the transcript is empty for the statement at 0:37–0:52: "There's a sort of a quote from one of the famous people behind the C programming language, Brian Kernigan, …").

To reproduce, set up a Cargo repository with dependencies on hound (for WAV decoding) + transcribe-cpp 0.1.1, then:

use transcribe_cpp::{init_backends_default, Model, RunOptions, TimestampKind};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut args = std::env::args().skip(1);
    let usage = "usage: repro <model.gguf> <audio.wav>";
    let model_path = args.next().expect(usage);
    let wav_path = args.next().expect(usage);

    // Decode the WAV to f32 mono in [-1, 1] (it is already 16 kHz mono s16).
    let mut reader = hound::WavReader::open(&wav_path)?;
    let spec = reader.spec();
    assert_eq!(spec.sample_rate, 16000, "repro audio is 16 kHz");
    assert_eq!(spec.channels, 1, "repro audio is mono");
    assert_eq!(spec.sample_format, hound::SampleFormat::Int, "repro audio is s16");
    let scale = (1i64 << (spec.bits_per_sample - 1)) as f32;
    let samples = reader
        .samples::<i32>()
        .map(|s| s.map(|s| s as f32 / scale))
        .collect::<Result<Vec<f32>, _>>()?;
    eprintln!("audio: {:.1}s", samples.len() as f64 / 16000.0);

    init_backends_default()?;
    let model = Model::load(&model_path)?;
    eprintln!("backend: {}", model.backend());
    let mut session = model.session()?;
    let transcript = session.run(
        &samples,
        &RunOptions {
            timestamps: TimestampKind::Word,
            ..Default::default()
        },
    )?;

    let mut max_gap = 0i64;
    let mut gap_at = 0i64;
    for pair in transcript.words.windows(2) {
        let gap = pair[1].t0_ms - pair[0].t1_ms;
        if gap > max_gap {
            max_gap = gap;
            gap_at = pair[0].t1_ms;
        }
    }
    println!(
        "{} words; largest inter-word gap: {}ms starting at t={}ms",
        transcript.words.len(),
        max_gap,
        gap_at
    );
    if max_gap > 5_000 {
        println!(
            "BUG REPRODUCED: ~{}s of audible speech dropped after t={:.1}s \
             (expected the \"There's a sort of a quote ...\" sentence there)",
            max_gap / 1000,
            gap_at as f64 / 1000.0
        );
        std::process::exit(1);
    }
    println!("no drop detected (transcript covers the full window)");
    Ok(())
}

RunOptions: defaults + timestamps = Word (drop reproduces with pure defaults as well). Word rows around the hole:

t0_ms   t1_ms
35840   36400   "it."     <-- speech before the hole
52000   52160   "And"     <-- next emitted word; 15.6s hole

I also tried some window experiments with the same audio to see if the same gap appears:

window	result
0–70s	15.6s hole (36.4→52.0)
0–55s	same hole
0–52s	no hole (quote fully transcribed)
0–45s	no hole
10–70s	no hole (!)
20–50s	no hole
34–64s	no hole

Additional data points:

• Prepending 10s of digital silence to the 0–70s window makes it worse: the hole grows to 31.2s (36.4→67.6 in the original timeline).
• Q8_0 and F16 GGUFs produce bit-identical word rows (same 180 words, same hole), so this doesn't seem like a quantization artifact.
• CPU and Vulkan backends both reproduce the hole at the same position, so it's not a backend numerics issue.
• spec_k_drafts = 0 (speculative decode disabled): no change.
• The int8 ONNX export of the same model (via onnxruntime and transcribe-rs) transcribes the full 195s window correctly, including the quote.

[jonhoo]

I don't know how much of this is transcribe.cpp's fault, and how much of it really comes down to the underlying model though 😬