Facade: let the app supply program audio (ProgramAudioSink) - #2
Merged
Conversation
The vendored audio path SELECTS a source, it never mixes: AudioUnit's captureOutput takes one buffer straight to the encoder, and selecting a buffered source makes the microphone path bail out entirely. So "the streamer's voice AND the alert/TTS audio" cannot be produced inside the engine — there is nowhere for a second stream to join. Rather than fork the vendor to add a mixing stage, hand the job to the app, which is where the other audio already lives (speech synthesis, alert sounds, the captions tap): it owns one graph, mixes what belongs on the broadcast, and pushes the result through the new ProgramAudioSink seam. The engine treats it as the program source and encodes it. Kept off the StreamEngine protocol on purpose — it trades in CMSampleBuffer, and that protocol stays free of media types so a fake needs zero hardware. Same reasoning as CameraPreviewSource. The takeover is re-established after every setNetStream rebuild, alongside the mute re-apply: without that, go-live silently reverts the broadcast to bare mic. Releasing it re-attaches the default audio device rather than passing a nil buffered id, which would attach nothing and leave the stream silent. Tests: fake conformance records the takeover and buffer counts; 37/9 green on the simulator, containment clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Foundation for mixing alert/TTS audio into the broadcast.
Why the app has to do the mixing. The vendored audio path selects a source and never mixes —
AudioUnit.captureOutputhands one buffer to the encoder, and selecting a buffered source makes the mic pathreturnoutright. There is no seam inside the engine where a second audio stream could join, so "streamer's voice and TTS" is not expressible there. Rather than fork the vendor for a mixing stage, the app — which already owns speech synthesis, alert sounds and the captions tap — mixes and pushes the result in.Surface:
ProgramAudioSink(startProgramAudio(targetLatency:)/appendProgramAudio(_:)/stopProgramAudio()/isProgramAudioActive), adopted byIRLStreamEngineandFakeStreamEngine. Deliberately not onStreamEngine: it trades inCMSampleBuffer, and that protocol stays media-type-free so a fake needs no hardware — same reasoning asCameraPreviewSource.Two traps handled:
setNetStreamrebuild (alongside the mute re-apply) — otherwise go-live silently reverts the broadcast to bare mic;Tests: 37 / 9 suites green on iPhone 17 Pro sim;
check-containment.shclean.🤖 Generated with Claude Code