compile: bf16-native DSL -> StableHLO export path#788
Merged
Conversation
Enables authoring a bf16 model in the NN DSL and exporting StableHLO where weights reach the matmul as bf16 (required by the Torq NPU: fp32 weights crash its getWeightMemoryFormat). Four fixes, each only exercised by bf16-native export (all-fp32 models never hit them): - DtypeForwardPropagationPass (new, skainet-compile-opt): the trace records reductions/norms and the primary input as FP32 even for a BF16 model. This pass walks the graph topologically, sets each node's input-spec dtype from its producer's output dtype, inherits it on dtype-preserving op outputs, and (when targetFloatDtype is set) coerces float source nodes to it so the whole model unifies end-to-end. No-op for uniformly-typed graphs. - TypeMapper.negInfBits(): width-matched -inf bit pattern for the softmax / attention max-reduce identity (bf16 -> 0xFF80; the 32-bit 0xFF800000 is out of range for a bf16 constant). Applied in the Activation and Attention converters. - BasicMathConverter: emit stablehlo.convert as `(A) -> B`, not the invalid `A to B` short form iree-compile rejects. - DenseTensorDataFactory.zeros/placeholder: support BF16 (float-backed, bf16-tagged, mirrors FP16) so the dense builder can place bf16 weights. Verified: a DSL-authored bf16 Moonshine encoder now traces to all-bf16 StableHLO (matmuls bf16 x bf16) and compiles to an aarch64 llvm-cpu vmfb. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
📖 Documentation Preview The documentation has been built successfully for this PR. Generated Files:
Artifacts:
This comment will be updated automatically when the PR is updated. |
michalharakal
added a commit
that referenced
this pull request
Jul 5, 2026
…nt 2.07× Bumps VERSION_NAME 0.33.0 -> 0.34.0. Bundles the develop changes since 0.33.0: the new skainet-data-source module (URI-backed sources, HF auth, raw format parsers, suspend data pipeline DSL) + dataset operation views and richer batches (#784/#785), the bf16-native DSL -> StableHLO export path and the pluggable per-phase/per-target compile-optimization seam (#788/#791), NEON K-quant matmul perf (block-outer order + fused Q8 int8 dot, 2.07x Q4_K on Cortex-A55) with aarch64 board verification (#786/#787), LayerNorm f32 normalization + rank-0 tensor-type emission fixes, macOS host build fix (#789), Code of Conduct (#790), and the offline markup-antora docs image (#781). Minor bump (not patch): new published module skainet-data-source; all data-api additions are default-bearing (no source-incompatible changes). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Enables authoring a bf16 model in the NN DSL and exporting StableHLO where weights reach the matmul as bf16 (required by the Torq NPU: fp32 weights crash its getWeightMemoryFormat). Four fixes, each only exercised by bf16-native export (all-fp32 models never hit them):
(A) -> B, not the invalidA to Bshort form iree-compile rejects.Verified: a DSL-authored bf16 Moonshine encoder now traces to all-bf16 StableHLO (matmuls bf16 x bf16) and compiles to an aarch64 llvm-cpu vmfb.