Skip to content

fix(cloud): azure/edge silent zero-audio — bare <speak> envelopes, <mark>, and zero-audio-as-success - #23

Merged
willwade merged 1 commit into
mainfrom
fix/ssml-envelope-and-zero-audio
Aug 18, 2026
Merged

fix(cloud): azure/edge silent zero-audio — bare <speak> envelopes, <mark>, and zero-audio-as-success#23
willwade merged 1 commit into
mainfrom
fix/ssml-envelope-and-zero-audio

Conversation

@willwade

Copy link
Copy Markdown
Contributor

Found while debugging AACTools/VoiceGarden-SPD#1 (speech-dispatcher SSML produced no audio, reported as success). Two Azure/Edge failure modes complete the turn normally (turn.end / HTTP 200) but synthesise zero audio, with no error anywhere:

1. Bare <speak> envelope → zero audio

Azure/Edge accept an envelope missing version/xmlns/xml:lang — the turn completes cleanly and nothing is synthesised. xml:lang is the required missing piece. That is exactly what speech-dispatcher's index-marking wrapper sends, so every SSIP client using SSML mode hits this.

Fix: normalize_ssml_envelope() completes the envelope before the request goes out (WS + REST paths), deriving xml:lang from the voice name (en-GB-SoniaNeuralen-GB), same heuristic as build_azure_ssml. Present attributes, custom namespaces, tag-name case and the rest of the document pass through verbatim; plain text and envelope-less fragments are untouched.

2. SSML <mark> element → zero audio

Azure/Edge don't support <mark>; one anywhere in the document silently zeroes the whole utterance. speech-dispatcher injects <mark name="__spd_N"/> around every pause, so this is the same repro from the other direction.

Fix: strip_unsupported_marks() drops the (empty, unspoken) <mark>/</mark> tags on the azure/edge SSML path. Consumers needing positions should use word-boundary events, which already exist.

3. Zero audio reported as Ok(()) (all cloud engines)

The wrapper was the only layer that knew a turn produced nothing, yet returned success — every consumer had to detect "no bytes" independently. Now:

  • WS path (edge + azure-with-boundaries): wire audio frames counted; clean turn.end with zero bytes → Err("… synthesis completed with no audio").
  • All REST paths (streaming MP3/PCM, Google audioContent, ElevenLabs audio_base64, raw body): decoded/delivered bytes counted; zero → Err("… synthesis returned no audio").

This is what surfaced failure #2 during verification — the error path works.

Verification

  • New unit tests: envelope completion (order, gaps-only, case, self-closing, fragments untouched, composition with inject_voice_if_missing), voice_lang, mark stripping (speechd shape, paired tags, <market> untouched, unterminated left alone).
  • Live against the Edge endpoint via cargo run --example edge-bare-envelope: plain text, bare <speak>, and <speak>+<mark> all synthesise audio with word boundaries (previously: 0 bytes, Ok(())).
  • cargo fmt, cargo clippy -D warnings (cloud and system,cloud feature sets), full test suite green.

Consumers: VoiceGarden-SPD currently works around #1/#2 module-side (AACTools/VoiceGarden-SPD#3); once this merges, the module's ensure_ssml_document becomes a harmless no-op and can be simplified.

…nd <mark>

Two Azure/Edge failure modes complete the turn normally (turn.end /
HTTP 200) but synthesise zero audio, with no error anywhere —
consumers saw a silent success:

- A bare <speak> envelope (no version/xmlns/xml:lang) — exactly what
  speech-dispatcher's index-marking wrapper sends. normalize_ssml_envelope()
  now completes the envelope before the request goes out (xml:lang
  derived from the voice name, like build_azure_ssml).
- An SSML <mark> element, which Azure/Edge don't support —
  speech-dispatcher injects <mark name="__spd_N"/> around every pause.
  strip_unsupported_marks() drops the (empty, unspoken) elements.

Defense in depth: a turn that finishes cleanly with zero audio now
returns Err("synthesis completed with no audio") on both the WS path
(azure/edge) and every REST response path, so any future silent
zero-audio failure surfaces as an error.

Verified against the live Edge endpoint: plain text, bare <speak>,
and <mark> documents all synthesise (examples/edge-bare-envelope.rs).
@willwade
willwade merged commit 055c923 into main Aug 18, 2026
9 checks passed
willwade added a commit that referenced this pull request Aug 18, 2026
…, keep text

Live testing of SpeechMarkdown #[style] sections through Edge (which
#24 routed to the Azure platform, so they now emit
<mstts:express-as style=…>) showed the free Edge endpoint synthesises
zero audio for express-as just like <bookmark>. The zero-audio Err
from #23 surfaced it loudly instead of silently.

In Edge mode the strip now drops the express-as wrapper tags and keeps
the spoken content. Azure keeps express-as (documented, accepted).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant