Skip to content

Lead program audio by the jitter buffer's own depth - #4

Merged
ustacode merged 1 commit into
mainfrom
feature/program-audio-timing
Aug 2, 2026
Merged

Lead program audio by the jitter buffer's own depth#4
ustacode merged 1 commit into
mainfrom
feature/program-audio-timing

Conversation

@ustacode

@ustacode ustacode commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

The bug

Program audio reached viewers as a fraction of the real audio interleaved with digital silence — "robotic and stuttery" in the user's words.

BufferedAudio.getSampleBuffer selects by timestamp: it walks the queue consuming every buffer whose PTS is at or behind the output clock, returns only the newest one consumed, and when it finds the queue empty it re-emits the previous buffer with its samples zeroed.

We were stamping program audio at capture time. That is by definition always behind "now", so:

  • every tick consumed the entire queue and discarded all but one buffer
  • the queue could never accumulate the 200 ms it was configured to hold
  • the gaps came back as zeroed buffers

The fix

Shift the presentation timestamp forward by the source's own target latency before queueing — which is exactly what the built-in path already does:

// AudioUnit.appendBufferedBuiltinAudio, existing vendored code
let presentationTimeStamp = presentationTimeStamp + CMTime(seconds: bufferedBuiltinAudio.latency)

Program audio simply never got the same treatment.

Applied inside the engine rather than pushed onto the caller: the caller stamps at capture time, which is the honest thing for it to know, and the engine owns its own jitter buffer.

Verification

  • 38 tests in 9 suites pass; new test pins the lead (appendLeadsByTargetLatency)
  • FakeStreamEngine models the shift and exposes lastProgramAudioPresentationTime — a fake that skipped it would let a consumer pass against audio the real engine throws away
  • Scripts/check-containment.sh clean

🤖 Generated with Claude Code

Program audio was stamped at capture time, which made the feed unusable.
The buffered-audio source picks buffers by comparing their timestamps to
its output clock: it consumes everything already in the past and returns
only the newest. Audio stamped at capture time is ALWAYS behind "now", so
every tick drained the whole queue and threw away all but one buffer of
each burst — and emitted digital silence whenever it then found the queue
empty. What reached viewers was a fraction of the audio interleaved with
zeros, which is what "robotic and stuttery" sounds like.

The built-in buffered-audio path already does this (`AudioUnit
.appendBufferedBuiltinAudio` adds `latency` before queueing); program
audio just never got the same treatment. Shift is applied inside the
engine rather than asked of the caller — the caller stamps at capture
time, which is the honest thing for it to know, and the engine owns its
own jitter buffer.

FakeStreamEngine models the shift too, and exposes the resulting
timestamp: a fake that skipped it would let a consumer pass its tests
against audio the real engine discards.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ustacode
ustacode merged commit 27ffdf8 into main Aug 2, 2026
1 check passed
@ustacode
ustacode deleted the feature/program-audio-timing branch August 2, 2026 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant