Custom audio tracks: push PCM from JS into a published WebRTC track

[MiloszFilimowski]

What

Adds createCustomAudioTrack() — the audio counterpart of the existing custom video track support. JS pushes PCM (any chunk size, Float32Array or Int16Array) into a WebRTC audio track that publishes through the normal Fishjam path, e.g. audio produced with react-native-audio-api.

const { stream, track } = await createCustomAudioTrack({ sampleRateHz: 48000 });
await setStream(stream);                    // publishes as `customAudio`
pushAudioSamples(track, samples);           // worklet-safe

How

• src/createCustomAudioTrack.ts — mirrors createCustomVideoTrack.ts idiom-for-idiom: same ensureInstalled() (10 s timeout, E_NO_JSI → "requires the New Architecture"), same by-reference JSI host-object sink (__fishjamWebrtcGetCustomAudioSink), same MediaStream+handle result, disposal via the normal track.stop() path. pushAudioSamples is 'worklet'-tagged.
• common/cpp/fishjam-audio/FJAudioFrameScheduler.{h,cpp} — the piece that makes this usable. The fork's source demands exactly one 10 ms int16 frame per call, in real time; JS produces audio in whatever size it likes, whenever it likes. A per-track FIFO + feeder thread re-paces pushes onto an absolute-deadline clock, inserting silence when the buffer runs dry so the track behaves like a live microphone, and dropping oldest whole frames past maxBufferedDurationMs. This is why callers can hand over a whole buffer at once and hear a steady tone — pacing from a JS timer instead produces audible beating.
• common/cpp/fishjam-audio/FJAudioPushJSI.{h,cpp} — structural twin of FJVideoPushJSI, with one deliberate difference: instead of routing each push to a platform deliver callback, it owns the per-track schedulers (registerTrack/unregisterTrack). Float32→Int16 conversion happens in C++; malformed input is dropped, never thrown, on the hot path.
• iOS (WebRTCModule+CustomAudio.mm, CustomAudioSourceController) — RTCExternalAudioSource → RTCAudioTrack, landed in localTracks/localStreams so the existing publish path is untouched. The scheduler feeder pushes straight into the ObjC source. The controller is pure lifecycle: the release path runs an idempotent teardown that unregisters the scheduler (joining the feeder) before the source is released. startCapture/stopCapture stay no-ops — a disabled audio track is muted by the sender itself, matching microphone semantics.
• Android (FJAudioPushInstaller.{java,cpp,h}, WebRTCModule, GetUserMediaImpl, new webrtc-custom-audio-track CMake lib) — the emit direction differs: the source is a Java object and this module's C++ doesn't link webrtc, so the feeder calls up into Java under a ThreadScope (it isn't JVM-attached) with a direct ByteBuffer wrapping the scheduler scratch, consumed synchronously. The push direction stays hop-free in C++. customAudioSources is concurrent so the feeder never races the executor mutating tracks; disposeTrack drops the source first (in-flight emits become no-ops), then joins the feeder, then disposes.
• Pins → FishjamWebRTC ~> 124.0.2.3 / com.github.fishjam-cloud:webrtc:v124.0.2.3, which carry the required ExternalAudioSource (Add ExternalAudioSource: application-pushed PCM into local audio tracks webrtc#1).

No changes needed in react-client: useCustomSourceManager already publishes stream.getAudioTracks()[0] as { type: 'customAudio' }.

Validation

End-to-end against a Fishjam staging room, both platforms, with a second (web) client decoding the received track via WebAudio FFT:

• Steady tone — 440 Hz pushed from JS decoded at RMS 0.178 vs 0.177 expected (amplitude 0.25/√2): bit-accurate through Float32 → JSI → int16 → 10 ms re-pacing → Opus → SFU.
• A real chord, natively mixed — three OscillatorNodes rendered in one OfflineAudioContext pass and pushed as a single buffer decoded as 264 / 328 / 393 Hz simultaneously (C4+E4+G4, RMS ~0.22) on both iOS and Android. Mixing belongs in the audio graph; this API is the transport (it appends to the FIFO, it does not mix).
• Microphone and custom tracks publish simultaneously and stay independent; the external stream never registers with AudioState, so it neither engages the mic nor mixes mic audio in.
• Clean teardown: stopping removes the track at the receiver.
• Pins verified against the published artifacts: CocoaPods installs FishjamWebRTC (124.0.2.3) (symbols present in both slices), Gradle pulls the JitPack AAR with sha256 identical to the released asset.

Note for consumers using react-native-audio-api

Call AudioManager.disableSessionManagement(). audio-api otherwise takes over the iOS audio session and forces AVAudioSessionCategoryPlayback, which starves WebRTC's AUVoiceIO unit — AURemoteIO::Initialize then times out and aborts the app. Worth documenting when the consumer-facing hook lands.

Follow-ups (not in this PR)

• useCustomAudioSource hook in mobile-client, in the useCamera/useCustomSource family.
• A productized demo screen (validation used a temporary synth pad in fishjam-chat).

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.