diff --git a/CHANGELOG.md b/CHANGELOG.md index 2220261..47ec393 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,19 @@ version line is kept in lock-step with the underlying SKaiNET engine The format roughly follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.34.1] — 2026-07-05 + +Patch on **0.34.0** (same SKaiNET engine 0.34.0). Fixes Moonshine encoder parameter naming. + +### Fixed + +- **Layer-qualified Moonshine encoder parameter names.** The encoder's attention and LayerNorm + parameters were not prefixed with the layer (`attn.q_proj.weight`, `attn_norm.weight` repeated + identically every layer), while the FFN parameters were (`enc.$layer.ffn_*`). By-name weight + loading could therefore not distinguish the layers. All parameter names are now unique and + layer-qualified (`enc.$layer.attn.*`, `enc.$layer.attn_norm.*`, `enc.$layer.ffn_norm.*`), + matching the FFN convention. No public API change — `moonshineEncoder()` is unchanged. + ## [0.34.0] — 2026-07-05 Ships against **SKaiNET engine 0.34.0**. Headline: the first **Moonshine** speech-to-text encoder diff --git a/README.md b/README.md index ccd8b73..e7757b2 100644 --- a/README.md +++ b/README.md @@ -103,7 +103,9 @@ Honest status — see the project-status note at the top of this README. ## Current release -The current release is **0.34.0** (against **SKaiNET 0.34.0**). It adds the first **Moonshine** +The current release is **0.34.1** (against **SKaiNET 0.34.0**) — a patch that layer-qualifies the +Moonshine encoder's attention/LayerNorm parameter names so by-name weight loading can tell the +layers apart (no public API change). It builds on **0.34.0**, which adds the first **Moonshine** speech-to-text encoder authored entirely in the SKaiNET NN DSL (`skainet-transformers-inference-moonshine`, bf16-native) — it emits portable StableHLO and transcribes correctly on both CPU and the Synaptics Torq NPU. Supporting this, `transformer-core` RoPE now computes its rotation and `cos`/`sin` tables in **f32** diff --git a/docs/modules/ROOT/nav.adoc b/docs/modules/ROOT/nav.adoc index 4262b66..e7b6c2c 100644 --- a/docs/modules/ROOT/nav.adoc +++ b/docs/modules/ROOT/nav.adoc @@ -16,6 +16,7 @@ .Reference * xref:reference/architecture.adoc[Architecture Overview] +* xref:reference/moonshine-encoder.adoc[Moonshine Encoder (Speech-to-Text)] * xref:reference/pipeline.adoc[Inference Pipeline] * xref:reference/tokenizer-api.adoc[Tokenizer API] * xref:reference/chat-session-api.adoc[ChatSession API] diff --git a/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc b/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc new file mode 100644 index 0000000..f80d8e6 --- /dev/null +++ b/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc @@ -0,0 +1,120 @@ += Moonshine Encoder (Speech-to-Text) +:description: The Moonshine-tiny audio encoder authored in the SKaiNET NN DSL — portable StableHLO that runs on CPU and NPU targets. + +`skainet-transformers-inference-moonshine` provides the **Moonshine-tiny** speech-to-text +*encoder* built entirely in the SKaiNET NN DSL. Unlike the GGUF/eager LLM runtimes, this model +is authored as a graph you *compile*: DSL → StableHLO → IREE, so the same definition runs on a +CPU backend or a hardware NPU. It was first published in **0.34.0**; **0.34.1** layer-qualifies +its parameter names (see below). + +[NOTE] +==== +**Encoder only — for now.** Moonshine is an *encoder-decoder* (seq2seq) model, like Whisper: an +encoder turns audio into a memory tensor, and an autoregressive decoder emits text using +self-attention (KV-cached) plus cross-attention into that memory. This module currently +implements **only the encoder** — the compute-heavy half and the one that carried the hard NPU +compilation work (bidirectional attention, partial RoPE, bf16 LayerNorm). The decoder is not yet +authored in the DSL: in the SL2610 demo it runs from prebuilt vendor vmfbs +(`decoder.vmfb` + `decoder_with_past.vmfb`). `MoonshineConfig.decoderLayers` reserves the shape; +a DSL decoder is future work. +==== + +== Coordinates + +[source,kotlin] +---- +dependencies { + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.34.1")) + implementation("sk.ainet.transformers:skainet-transformers-inference-moonshine") +} +---- + +== API + +[source,kotlin] +---- +public fun moonshineEncoder( + cfg: MoonshineConfig, + dtype: KClass, +): Module +---- + +Build it with **`BF16`** so the DSL→StableHLO export keeps bf16 weights at the matmul — the +Torq NPU requirement (fp32 weights crash the Torq compiler's `getWeightMemoryFormat`). The +input is the conv-frontend output `[batch, frames, dim]`; the output is the encoder memory +`[batch, frames, dim]` for a decoder's cross-attention. + +=== `MoonshineConfig` (moonshine-tiny defaults) + +[cols="2,1,4"] +|=== +| Field | Default | Notes + +| `dim` | `288` | model width (`nHeads × headDim` = 8 × 36) +| `encoderLayers` | `6` | pre-norm transformer layers +| `nHeads` | `8` | attention heads +| `headDim` | `36` | per-head width +| `ffnDim` | `1152` | GELU MLP hidden (4 × dim) +| `maxFrames` | `165` | encoder sequence length after the conv frontend +| `partialRotaryFactor` | `0.9` | rotate 32 of 36 head dims; the trailing 4 pass through +| `ropeBase` | `10000` | RoPE base +|=== + +== Architecture + +Each layer is pre-norm: `x + Attn(LN(x))` then `x + MLP(LN(x))`, with + +* **non-causal RoPE self-attention** (bidirectional, no bias). Moonshine uses *interleaved* + (adjacent-pair) RoPE with **partial rotary** — only `headDim × partialRotaryFactor = 32` of + the 36 head dims are rotated, and `inv_freq` is computed over the rotary dim (32), not the + full head dim. +* a plain **GELU MLP** (`fc1 → GELU → fc2`), both projections biased. + +A final LayerNorm produces the encoder memory. + +== Compilation path + +The emitted StableHLO is **portable** — it names no backend. Hardware-specific optimizations +(e.g. the Synaptics Torq attention/FFN tiling and compile flags) plug in from *outside* core via +the `TargetOptimizer` registry, so the model definition stays target-agnostic: + +[source] +---- +moonshineEncoder() → tape → ComputeGraph → DtypeForwardPropagation(bf16) → StableHLO + ↓ + (vendor plugin adds target passes) → iree-compile → vmfb +---- + +See the *SKaiNET Embedded* docs (`sl2610-function-calling`) for an end-to-end example that +self-compiles this encoder from the DSL and runs it on the Synaptics SL2610 NPU. + +[TIP] +==== +**Validation status.** This encoder is proven end-to-end, not just structurally emitted: on +`test.wav` the 6-layer DSL encoder → decoder transcribes *"One, two, three."* correctly on the +SL2610 Torq NPU, matching the reference (encoder cosine **1.0** on f32 CPU, **~0.9998** bf16 CPU; +the full 6-layer graph compiles and runs on the NPU). The *decoder* in that run is an external +one (HF `MoonshineForConditionalGeneration` in the validation harness; prebuilt vendor vmfbs on +the board) — authoring the decoder in the DSL is the next milestone. +==== + +== Parameter names + +As of **0.34.1** every parameter is uniquely **layer-qualified**, so by-name weight loading can +tell the layers apart: + +[source] +---- +enc.$layer.attn_norm.{weight,bias} +enc.$layer.attn.{q,k,v,o}_proj.weight +enc.$layer.ffn_norm.{weight,bias} +enc.$layer.ffn_up.{weight,bias} # fc1 +enc.$layer.ffn_down.{weight,bias} # fc2 +enc_out_norm.{weight,bias} # final norm +---- + +[NOTE] +==== +In 0.34.0 the attention/LayerNorm names were *not* layer-qualified (`attn.q_proj.weight` +repeated every layer); 0.34.1 fixes this with no public-API change. +==== diff --git a/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc b/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc index 786839d..a976c33 100644 --- a/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc +++ b/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc @@ -25,7 +25,7 @@ In your `build.gradle.kts`: [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.32.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.34.1")) implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-agent") @@ -41,7 +41,7 @@ Or in Maven (Maven needs the `-jvm` classifier suffix on platform artifacts): sk.ainet.transformers skainet-transformers-bom - 0.32.1 + 0.34.1 pom import diff --git a/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc b/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc index 498d4e0..cb67e9d 100644 --- a/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc +++ b/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc @@ -52,7 +52,7 @@ The pieces you need live in three modules: [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.32.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.34.1")) implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-agent") diff --git a/gradle.properties b/gradle.properties index dee7bc8..71889ea 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.transformers -VERSION_NAME=0.34.0 +VERSION_NAME=0.34.1 POM_DESCRIPTION=SKaiNET-transformers diff --git a/llm-inference/moonshine/src/commonMain/kotlin/sk/ainet/models/moonshine/MoonshineEncoder.kt b/llm-inference/moonshine/src/commonMain/kotlin/sk/ainet/models/moonshine/MoonshineEncoder.kt index ae7b80e..ccbd88c 100644 --- a/llm-inference/moonshine/src/commonMain/kotlin/sk/ainet/models/moonshine/MoonshineEncoder.kt +++ b/llm-inference/moonshine/src/commonMain/kotlin/sk/ainet/models/moonshine/MoonshineEncoder.kt @@ -40,15 +40,18 @@ public fun moonshineEncoder( for (layer in 0 until cfg.encoderLayers) { val stage = StageImpl(nnCtx, "enc.$layer", dtype) - // x + Attn(LN(x)) - stage.layerNorm(intArrayOf(dim), eps.toDouble(), id = "attn_norm") + // x + Attn(LN(x)). Layer-qualify every id ("enc.$layer.*") so the attention / + // LayerNorm parameter names are UNIQUE across layers — matching the FFN naming below. + // Without the prefix, `attn.q_proj.weight` / `attn_norm.weight` repeat every layer and + // by-name weight loading can't tell the layers apart. + stage.layerNorm(intArrayOf(dim), eps.toDouble(), id = "enc.$layer.attn_norm") stage.multiHeadAttention( dim = dim, nHeads = cfg.nHeads, nKVHeads = cfg.nHeads, causal = false, // encoder = bidirectional self-attention bias = false, - id = "attn", + id = "enc.$layer.attn", ) { rope( headDim = cfg.headDim, @@ -64,7 +67,7 @@ public fun moonshineEncoder( // x + MLP(LN(x)) — plain GELU MLP (not gated). VoidDense carries explicit // in/out dims (no eager placeholder alloc, and no reliance on DSL // dimension-tracking through the attention/residual sub-blocks). - stage.layerNorm(intArrayOf(dim), eps.toDouble(), id = "ffn_norm") + stage.layerNorm(intArrayOf(dim), eps.toDouble(), id = "enc.$layer.ffn_norm") // Moonshine MLP is a biased fc1 -> GELU -> biased fc2 (the reference model // carries `mlp.fc1.bias`/`mlp.fc2.bias`). addBias=true keeps the trace faithful. stage.modules += VoidDense("enc.$layer.ffn_up", cfg.ffnDim, dim, dtype, addBias = true)