atmos renderer#82
Open
tryptz wants to merge 12 commits into
Open
Conversation
Land the first self-contained, verifiable layer of the E-AC-3 JOC object
renderer described in the renderer plan, clean-room from the public ETSI
specs and dependency-free so it builds under the NDK and unit-tests on a
host toolchain.
Native (cpp/atmos):
- bit_reader.h MSB-first bit reader + ETSI variable_bits escape
- emdf.h EMDF container framing walk; locates OAMD/JOC payloads
- oamd.h OAMD position coordinate math + unit-cube -> render-space map
- vbap.h VBAPBedPanner: direction -> 5.1/7.1/7.1.4 speaker gains
(2-D adjacent-pair + 3-D tightest-triplet VBAP)
- atmos.cpp TU so the NDK type-checks the headers for every shipped ABI
- tests/atmos_tests.cpp 489 host checks, all green (round-trips, coordinate
boundaries, VBAP source-at-speaker/energy/non-negativity)
- monochrome_atmos static CMake target + README documenting scope
Domain infra (JVM-unit-tested):
- TrackSource, RendererMode (Direct/Object/Bed+HRTF), ChannelLayout
- isDolbyAtmos / isAtmosCapableCodec detection mirroring isThxSpatialAudio
- RendererProfile with safe passthrough-stereo defaults
Scope is honest: the QMF filterbank, JOC decoder, HRTF back-end and the JNI/
Media3 wiring need reference Atmos content to validate and are documented as
TODO in atmos/README.md, not stubbed to look finished.
Connect the Atmos domain infra to the persistence layer, mirroring the existing THX Spatial Audio path: - LocalTrackEntity gains an isDolbyAtmos column; DB v9->v10 migration adds it to local_tracks and backfills rows whose title/album names "Dolby Atmos" (local-only, since sideloaded Atmos files land in the scan). Real migration, registered alongside MIGRATION_8_9. - TagReader detects Atmos at scan time from the MIME / metadata via isDolbyAtmos() and carries it on AudioTags; MediaScanner persists it. Extension-only .ec3/.eac3 stays 'capable' not 'Atmos' until the native demux confirms JOC. - PreferencesManager persists a RendererProfile (mode / layout / HRTF id) as JSON, same shape as the player-glass blobs. Kept device-local (not in the cloud-sync allow-list) since layout tracks the DAC and HRTF is a local measurement. The channel layout needs no new column: ChannelLayout.fromChannelCount derives it from the existing channels field.
Settings > Audio > Spatial Audio > Atmos Renderer Configuration: a full renderer config screen plus a top-down speaker map that lights the channels the current mix drives. Domain (RendererProfile expanded, JVM-tested): - StereoDownmixMode (Binaural / Lo-Ro / Lt-Rt) and DrcMode (Off..Heavy) - typical renderer fields: mode, auto-detect layout, target layout, stereo downmix, HRTF profile, binaural strength, height virtualization, bass management + crossover + LFE gain, DRC, dialogue normalization - clamped() bounds continuous fields; effectiveLayout() resolves the auto-detect flag; ChannelLayout.speakers() is the shared speaker model (angles mirror cpp/atmos/vbap.h). Persisted device-local, clamped on I/O. UI: - AtmosRendererScreen + AtmosRendererViewModel (debounced persistence) - SpeakerLayoutMap: Canvas top-down room, each active channel drawn on its azimuth with a soft pulsing radial bloom; height speakers on an inner ring, LFE below the listener - chip selectors, labeled sliders, reset-to-defaults - route registered; entry added under Audio > Spatial Audio Tests cover the new defaults, clamping (incl. NaN), effectiveLayout, and the speaker map (one speaker per channel, exactly one LFE, height only on 7.1.4).
Faithful C# -> C++ port of Cavern's 64-band complex QMF analysis/synthesis filterbank (QuadratureMirrorFilterBank.cs/.Process.cs/.Amp.cs) plus the QMath multiply/accumulate primitives it builds on. This is the QMF domain the JOC upmix operates in (plan A5). - quadrature_mirror_filterbank.h: 640-tap prototype filter reproduced verbatim (extracted programmatically, not retyped), exact cos/sin modulation and delay-line indexing - qmath.h: the multiply-and-set/add + dot primitives - host reconstruction test: analysis -> synthesis reconstructs a broadband multi-sine at correlation 1.00000 with the expected 577-sample delay - included from atmos.cpp so the NDK type-checks it on every ABI LICENSING: Cavern is NOT MIT. It ships under a custom non-commercial, no-ads licence that requires attribution + a source link and the creator's permission for public/commercial use, and applies virally to including projects. cavern/NOTICE.md reproduces the licence verbatim and records the obligations; per-file headers link the upstream source. The cavern/ subtree is therefore NOT clean-room -- the rest of cpp/atmos still is. Obtaining the creator's permission before any public/monetized release is an owner action, flagged in NOTICE.md.
|
This pull request has been ignored for the connected project Preview Branches by Supabase. |
Faithful C# -> C++ port of the Joint Object Coding decode path — the step that reconstructs Atmos objects from the E-AC-3 core bed + JOC side data (plan A6), building on the already-ported QMF filterbank. - joc_tables.h: Huffman code tables (nodes extracted verbatim, not retyped), joc_num_bands, and the parameter-band mapping expanded once into a subband->parameter-band LUT (SetupStaticCache) - joint_object_coding.h: frame decode (header/info/data + Huffman), coarse dequantization, and per-timeslot mixing-matrix interpolation (1/2 data points, steep/smooth). Reuses the existing MSB-first BitReader, which is bit-identical to Cavern's BitExtractor - joint_object_coding_applier.h: QMF-domain channel->object mix (object = sum_ch channelQMF[ch] * mixMatrix[ch]) + inverse QMF + gain Deviations, behaviour-preserving: Cavern's ThreadPool fan-out is sequential; exceptions become a valid() flag (NDK is -fno-exceptions); the sparse WIP path matches upstream (dequantize with gainStep 0). Host tests (87 checks, all green): Huffman encode/decode round-trip over the real tables, parameter-band mapping, the coarse dequant recurrence, and an end-to-end decode -> mixing matrices -> applier run producing finite output. Numeric bit-exactness still needs A/B vs reference Atmos content. LICENSING unchanged: this is under Cavern's non-commercial/no-ads licence (attribution + source link, creator's permission for public/commercial use). cavern/NOTICE.md lists every ported file and the obligations.
Faithful C# -> C++ port of ObjectInfoBlock.cs + the OAMD enums — the per-object Object Audio Metadata decode that gives each object its position (plan A4), to pair with the JOC-reconstructed object PCM. - object_info_block.h: absolute/differential position, gain (with the 3 dB anti-clip), size, and anchor (bed/room/screen) decode. Scales match the clean-room oamd.h (xy 1/62, z 1/15). NonStandardBedChannel / ObjectAnchor enums included. read_signed reproduces Cavern's BitExtractor.ReadSigned faithfully (it collapses to 0 — kept for A/B parity, with the C# shift-mask applied to avoid C++ shift UB). - qmath.h: add db_to_gain (QMath.DbToGain). - bitstream decode is reproduced exactly (bit alignment preserved); Cavern's UpdateSource() render integration (Listener/Source) is NOT ported — resolved_position() applies the differential + render-space map instead. Host-tested (16 checks): absolute position -> render-space centre, gain, bed->speaker anchor, and the degenerate ReadSigned. Compiles under the NDK production flags; wired into atmos.cpp for per-ABI coverage. Still to port: the OAMD framing (ObjectAudioElementMetadata, program assignment) and the EMDF container that drive these blocks. Licensing unchanged — see cavern/NOTICE.md.
Completes the Atmos side-data decode path (plan A3): the framing that ties OAMD object positions and JOC together. - object_metadata.h: OAElementMD (element read, update timing, per-block ramps) + ObjectAudioMetadata (program/bed assignment, object count). The EMDF VariableBits helpers and ReadBits (reversed fill) are reproduced verbatim. - extensible_metadata_decoder.h: ExtensibleMetadataDecoder — scans the EMDF syncword (0x5838) and dispatches OAMD (id 11) / JOC (id 14) to their decoders, holding both. - bit_reader.h: add set_position/back_position (the EMDF walk seeks past payloads by absolute bit position) — clean-room generic accessors. Scope: bitstream decode reproduced exactly (bit alignment preserved); Cavern's UpdateSources() render integration (Listener/Source) is not ported, decoded state is exposed via accessors for a future renderer. With this the full EMDF -> OAMD + JOC -> QMF upmix chain is ported. Host tests (49 checks): EMDF VariableBits round-trip, ReadBits, and program assignment (dynamic-only + LFE, bed LFE-only). Compiles under NDK production flags; wired into atmos.cpp. Licensing unchanged (cavern/NOTICE.md). Still to build: E-AC-3 core audio decode (bed PCM), container demux, HRTF back-end, and the JNI/Media3 wiring.
Expose the ported EMDF -> OAMD + JOC decode chain to the app, off the
player's critical path.
- atmos_jni.cpp: JNI bridge — nativeSelfCheck() and
nativeDecodeEmdfObjectCount() (walk an EMDF buffer, return JOC object
count, exercising the full decode path natively). Host-compiles clean
against a real jni.h.
- monochrome_atmos CMake target: STATIC -> SHARED so
System.loadLibrary("monochrome_atmos") works; still type-checked for
every ABI on each CI run.
- AtmosEngine.kt: lazy, failure-tolerant loader (ensureLoaded returns false
instead of throwing if the .so is missing) + the external-fun surface.
Nothing loads the library at runtime yet, so playback is unaffected. This is
the boundary the render/output wiring grows into once a core E-AC-3 bed
decoder is chosen (Cavern port vs FFmpeg eac3, plan Option 1).
Rationale: the E-AC-3 core audio decode is a large, multi-file, framework-
entangled Cavern codec that can't be host-validated without reference AC-3
content; this JNI step is bounded, CI-verifiable, and no-regret under either
core-decoder choice.
… logos Reworks the Streaming onboarding step to present the catalog sources (TIDAL, Qobuz, Apple Music) plus the Spotify radio connector, each with its brand logo. Apple Music is flagged Dolby Atmos capable and, like the others, routes through the self-hosted TrypT HiFi instance. Adds vector drawables logo_tidal/logo_qobuz/logo_spotify/logo_apple_music; ServiceCard renders a drawable (tinted for the monochrome TIDAL mark, brand colour otherwise).
…stance Mirrors the Qobuz catalog wiring end to end: SourceType.APPLE, SourceMode.APPLE_ONLY, an Apple Music instance URL (falls back to the Qobuz URL since it's the same TrypT HiFi server), InstanceManager.appleInstanceOrNull, HiFiApiClient.searchApple hitting /api/apple/get-music (reuses the Qobuz-shaped envelope + mappers), MusicRepository.searchApple, Track.toAppleUnifiedTrack (SourceType.APPLE, reuses QobuzCached playback), a third parallel fan-out in SearchViewModel (runs under BOTH or APPLE_ONLY), a SearchSourceFilter.APPLE chip, and Settings: an 'Apple only' catalog toggle + an Apple Music URL field. NOTE: not compile-tested (no local Android SDK); build to verify. Apple pagination is first-page only for now; playback reuses the Qobuz cached path (both Qobuz and Apple can stream).
…+ cloud cache) Adds PlaybackSource.AppleCached and routes Apple tracks off the Qobuz path onto the instance's /api/apple/download-music, which returns the wrapper-resolved manifest. StreamResolver.resolveAppleCached reads manifest.delivery.streamUrl (the cloud-cached decrypted ALAC/Atmos file, Range-capable) and streams it directly. Atmos-flagged tracks request the atmos variant so decryption flows through the Tailscale wrapper. Adds AppleDownloadEnvelope model, HiFiApiClient.getAppleStreamUrl, MusicRepository.appleStreamUrl; repoints toAppleUnifiedTrack to AppleCached. NOT compile-tested (no local Android SDK). Offline Download-to-device (DownloadWorker) still uses the Qobuz path and needs a source flag - follow-up.
DownloadWorker now routes Apple tracks to the wrapper-backed Apple download endpoint instead of the Qobuz one, saving the cloud-cached decrypted file as .m4a (audio/mp4), skipping FLAC header-sniffing/tagging. Apple track ids are tracked in QobuzIdRegistry (registerAppleTrack/isAppleTrack, persisted), populated by searchApple; the worker detects Apple via the registry so no call-site changes are needed. NOT compile-tested.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.