[FEATURE] Fused NEON WaveNet engine for AArch64 (Apple Silicon)

[rikkus]

Fused NEON WaveNet engine for AArch64 (Apple Silicon)

Disclaimer

Human input here was restricted to requests for:

• An attempt to optimise for Apple Silicon - the initial prompt
• Defence of approach, such as, "Why is the A2 model creation for the benchmark a reasonable thing to do?"
• Explanations as to why other avenues should - or should not - be attempted
• Attempts at other potentially-worthwhile approaches
  • Tweaks to the fused implementation
  • SME2 (tried on iPhone 17, with good results, but original fused implementation still much faster)
• Creation of REAPER projects with multiple duplicate tracks, to allow listening for glitches
• Clear and honest benchmark result presentation

The rest of the work was done by Claude Code and Cursor, using Fable and Opus models.

Summary

Adds NAM/wavenet/fused.{h,cpp}: a fused, register-tiled NEON fast path for
standard-shape WaveNet models on AArch64. Per layer, the dilated-conv taps +
bias + input mixin, the activation, and the head accumulation + layer1x1 +
residual run as three passes over an L1-resident buffer, replacing ~8 buffer
passes and 4 dynamic-size Eigen GEMM calls per layer (which dominated the
profile at ~65% and heap-allocate on the audio thread). Conv history lives in
power-of-two ring buffers with a lazily-refreshed mirrored tail, so tap reads
are contiguous and per-block cost is constant.

It is strictly additive: a shape detector routes only known-supported models
here, everything else runs the existing engines unchanged. Gated behind
NAM_ENABLE_FUSED (CMake option, default ON); a no-op stub on non-ARM builds.

Speedup vs. what main runs today

main already has an ARM fast path for the A2 architecture (a2_fast, #251),
so the honest baseline differs per model family. Measured on Apple M2, 48 kHz,
buffer 64, per 2 s of audio (full reports and scripts under
benchmark_reports/):

Model	Engine on main	main	fused	vs main
A1 standard (16/8)	generic (Eigen)	~86 ms (23× RT)	~39 ms (51× RT)	~2.2× (~2.7× at buffer 16)
A2 standard (8 ch)	a2_fast	~68 ms (29× RT)	~40 ms (50× RT)	~1.7×
A2 nano (3 ch)	a2_fast	~6 µs/block (218× RT)	—	unchanged — fused declines it; a2_fast is already excellent there
Everything else (2-ch arrays, gated, FiLM, grouped…)	generic	—	—	unchanged — detector declines

Notes on the A2 standard row: fused-vs-generic is ~2×, but generic is not
what main actually runs for that model, so the number that matters is
fused-vs-a2_fast: ~1.7× per block (45.2 µs → 26.8 µs at p50). At 8 channels
a2_fast gains ~1.17× over generic; its large win (~7×) is the 3-channel
nano, which this PR deliberately leaves on a2_fast untouched.

As an end-to-end sanity check in a DAW (REAPER, M2 MacBook Air, 128-sample
buffer, a real trained A2-architecture capture, matched offline renders): the
fused build carried ~1.27× lower per-instance load than a2_fast — the
difference between 100 simultaneous plugin instances playing cleanly and
crackling on that machine.

Supported shapes and dispatch

The strict detector (is_fused_shape) accepts a model only when: mono in, no
condition DSP, no post-stack head; every layer array has
bottleneck == channels, channels a multiple of 4 (≤ 32), groups == 1, gating
none, no FiLM, head1x1 inactive. Kernel sizes (≤ 64), dilations,
activations, head kernel size/bias, and the number of layer arrays are
arbitrary. In practice this covers the A1 standard/lite family and the A2
standard model.

Dispatch order in wavenet::create_config is slimmable → fused → a2_fast →
generic, so nothing that previously matched a2_fast or slimmable loses its
fast path except A2 standard (8 ch), which moves to the faster engine.

Activations use NEON kernels only for known implementations (fast-tanh, ReLU,
LeakyReLU, Hardtanh, Softsign); anything else calls the exact same
Activation object the generic path would, so semantics (including
enable_fast_tanh() and LUTs) never diverge.

Benchmark methodology — how the A2 models were made

The A2 benchmark models are produced by the committed script
benchmark_reports/generate_a2_models.py: the exact A2 architecture — 23
layers, the fixed kernel-size pattern (14×6, 15, 15, 7×6), the fixed dilation
pattern, LeakyReLU(0.01), head kernel 16, at 3 and 8 channels — filled with
uniform random weights in ±0.3 from a fixed seed, so the files are
byte-reproducible by anyone.

Why synthetic weights are valid for a performance comparison:

1. The architecture is shape-exact by construction, and machine-checked.
   The generator uses the same layer constants a2_fast.h hard-codes, and the
   generated files pass a2_fast's own strict is_a2_shape detector — if
   they didn't, bench_a2_fast could not route them to the fast path at all.
2. Inference cost does not depend on weight values. Both engines execute a
   fixed sequence of convolutions/GEMVs whose FLOP count and memory-access
   pattern are functions of the architecture alone; there is no
   data-dependent branching. Bounded ±0.3 weights also keep activations well
   inside normal float range (no subnormal slowdowns), and both engines see
   the same input signal regardless.
3. It is the convention this repo already uses: tools/test/test_a2_fast.cpp
   builds its models the same way (std::uniform_real_distribution(-0.3, 0.3),
   fixed seed).
4. Accuracy claims never rest on the synthetic models. Correctness is
   validated on real trained models — see below.

Correctness

• tools/test/test_fused.cpp (wired into run_tests) compares fused vs
  generic within 5e-5 — the same tolerance as the existing a2_fast tests —
  across: A1 standard 16/8 with exact tanh and fast tanh, 8/4, channels
  12/20/32, ReLU / LeakyReLU / Hardtanh / Softsign / Sigmoid, A2-like mixed
  kernel sizes with head kernel 16, layer1x1-inactive arrays, and block sizes
  64/256/23 (the odd size exercises remainder tiles and ring-wrap paths).
  Plus detector negative tests.
• End-to-end render of real trained models through both engines: max sample
  difference ~1e-6, RMS error −127 dB relative to the signal. The only
  deviation source is FMA contraction/ordering inside the conv sums.

Real-time safety

Everything is preallocated in SetMaxBufferSize; process() performs no heap
allocation, verified by an allocation-tracking test.

Design notes

docs/fused-engine.md documents the profile that motivated the design, the
kernel layout, and the alternatives that were measured and rejected —
including Accelerate/AMX (cblas_sgemm only ties the NEON kernel at these
tiny per-tap matrix sizes), fp16/bf16 (no compute win on Apple cores; we are
compute-bound), and multithreading. The conv kernel runs at ~105 GFLOP/s fp32,
~93% of a single M2 P-core's theoretical FMA peak.

Reproducing

mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
cmake --build . --target run_tests benchmodel benchmodel_bufsize bench_a2_fast render -j
./tools/run_tests
python3 ../benchmark_reports/generate_a2_models.py
../benchmark_reports/run_benchmarks.sh

To A/B against the previous behavior, configure a second build directory with
-DNAM_ENABLE_FUSED=OFF. benchmodel gains a --seconds N flag for longer
profiling runs.