Filing this as requested by @cjpais in cjpais/Handy#1601.
Context
Voxtral Small 24B (Voxtral-Small-24B-2507-gguf) advertises Feature::InitialPrompt — transcribe_model_supports(model, TRANSCRIBE_FEATURE_INITIAL_PROMPT) returns true. However, the only way to actually pass a prompt today is the whisper-kind run extension (transcribe_whisper_run_ext), and Voxtral rejects that ext kind: transcribe_model_accepts_ext_kind(model, TRANSCRIBE_EXT_SLOT_RUN, TRANSCRIBE_EXT_KIND_WHISPER_RUN) is false, and attaching it anyway fails the run with INVALID_ARG.
So there is currently a gap: the feature flag says "this model takes a prompt", but no arch-appropriate mechanism exists to deliver one. Downstream apps (Handy) use the prompt for custom vocabulary / proper nouns, which is very valuable for dictation (proper nouns and technical terms get corrected at decode time).
Request
Expose a prompting path for non-whisper models that support it — e.g. a Voxtral run extension, or a generic prompt field in run params that each arch maps to its native mechanism (for Voxtral, presumably the LLM decoder's context/system prompt).
Environment
- transcribe.cpp via Handy 0.9.0 (transcribe-cpp crate 0.1.1), also reproduced with transcribe-cli built from main
- macOS 26, Apple M5 Pro, Metal backend
- Model: Voxtral-Small-24B-2507-Q5_K_M.gguf
Happy to test any branch — I have a working setup with real Spanish dictation audio. Thanks!
🤖 Generated with Claude Code
Filing this as requested by @cjpais in cjpais/Handy#1601.
Context
Voxtral Small 24B (
Voxtral-Small-24B-2507-gguf) advertisesFeature::InitialPrompt—transcribe_model_supports(model, TRANSCRIBE_FEATURE_INITIAL_PROMPT)returns true. However, the only way to actually pass a prompt today is the whisper-kind run extension (transcribe_whisper_run_ext), and Voxtral rejects that ext kind:transcribe_model_accepts_ext_kind(model, TRANSCRIBE_EXT_SLOT_RUN, TRANSCRIBE_EXT_KIND_WHISPER_RUN)is false, and attaching it anyway fails the run withINVALID_ARG.So there is currently a gap: the feature flag says "this model takes a prompt", but no arch-appropriate mechanism exists to deliver one. Downstream apps (Handy) use the prompt for custom vocabulary / proper nouns, which is very valuable for dictation (proper nouns and technical terms get corrected at decode time).
Request
Expose a prompting path for non-whisper models that support it — e.g. a Voxtral run extension, or a generic
promptfield in run params that each arch maps to its native mechanism (for Voxtral, presumably the LLM decoder's context/system prompt).Environment
Happy to test any branch — I have a working setup with real Spanish dictation audio. Thanks!
🤖 Generated with Claude Code