Skip to content

Fold DyT alpha scalar into tanh LUT - #21952

Open
cgreenberg wants to merge 1 commit into
pytorch:mainfrom
cgreenberg:export-D116560649
Open

Fold DyT alpha scalar into tanh LUT#21952
cgreenberg wants to merge 1 commit into
pytorch:mainfrom
cgreenberg:export-D116560649

Conversation

@cgreenberg

@cgreenberg cgreenberg commented Aug 19, 2026

Copy link
Copy Markdown

Summary:
Dynamic Tanh (DyT) normalization computes tanh(alpha * x) with a learned scalar
alpha. Quantized, that lowers to a full-tensor integer Mul followed by a tanh
TABLE. On Ethos-U the TABLE is free but the Mul is a real per-element cost, so
the Mul is pure overhead.

This adds FoldDyTAlphaIntoLUTPass in backends/arm/_passes/, which folds the
alpha multiply into the tanh lookup table and deletes the Mul and its
surrounding rescales.

Folding alpha in floating point before quantization would change rounding, so the
pass does it in the integer domain instead: it replays the exact TOSA
SINGLE_ROUND RESCALE and Mul arithmetic over all 256 int8 input codes, feeds the
results through the existing tanh quantization mapping, and materializes the
result as one 256-entry TABLE. The rewrite is therefore byte-exact rather than
approximate. It fails closed on anything it cannot prove: non-scalar alpha,
activation-side rank views, and narrowed tanh ranges are all handled explicitly.

Also adds register_pass_factories_before() to the Arm pass manager, a small
hook for inserting a pass that needs access to the ExportedProgram ahead of a
named target pass. FoldDyTAlphaIntoLUTPass needs it to read constant tensors.

Note on one import: the pass imports register_pass_factories_before inside
register_fold_dyt_alpha_into_lut_pass() rather than at module scope.
_passes/__init__.py imports arm_pass_manager last and arm_pass_manager
imports back from the package, so a module-scope import from a pass module that
__init__.py re-exports is circular. Only the register helper needs it.

The pass is inert until a model registers it, so this diff changes no behaviour
on its own.

Differential Revision: D116560649

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani

@pytorch-bot

pytorch-bot Bot commented Aug 19, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21952

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 3 Unrelated Failures

As of commit 54b33a0 with merge base 1b2838b (image):

NEW FAILURES - The following jobs have failed:

  • Cadence Build & Test / hifi-build / hifi4 (gh)
    ##[error]Refusing to check out fork pull request code from a 'pull_request_target' workflow. This workflow runs with the base repository's GITHUB_TOKEN, secrets, default-branch cache scope, and runner access. Fetching and executing a fork's code in that trusted context commonly leads to "pwn request" vulnerabilities. To opt in, review the risks at https://gh.io/securely-using-pull_request_target and set 'allow-unsafe-pr-checkout: true' on the actions/checkout step.
  • Cadence Build & Test / vision-build / vision (gh)
    ##[error]Refusing to check out fork pull request code from a 'pull_request_target' workflow. This workflow runs with the base repository's GITHUB_TOKEN, secrets, default-branch cache scope, and runner access. Fetching and executing a fork's code in that trusted context commonly leads to "pwn request" vulnerabilities. To opt in, review the risks at https://gh.io/securely-using-pull_request_target and set 'allow-unsafe-pr-checkout: true' on the actions/checkout step.

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 19, 2026
@meta-codesync

meta-codesync Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

@cgreenberg has exported this pull request. If you are a Meta employee, you can view the originating Diff in D116560649.

@linux-foundation-easycla

linux-foundation-easycla Bot commented Aug 19, 2026

Copy link
Copy Markdown

CLA Missing ID

  • ❌ The email address for the commit (54b33a0) is not linked to the GitHub account, preventing the EasyCLA check. Consult this Help Article and GitHub Help to resolve. (To view the commit's email address, add .patch at the end of this PR page's URL.) For further assistance with EasyCLA, please visit our EasyCLA portal and chat with our support bot.

@meta-codesync meta-codesync Bot changed the title Fold DyT alpha scalar into tanh LUT Fold DyT alpha scalar into tanh LUT (#21952) Aug 19, 2026
@github-actions github-actions Bot added ciflow/trunk module: arm Issues related to arm backend labels Aug 19, 2026
@pytorch-bot

pytorch-bot Bot commented Aug 19, 2026

Copy link
Copy Markdown

Workflows were awaiting approval. CI has now been triggered for the ciflow labels on this PR.

@@ -0,0 +1,341 @@
# Copyright 2025-2026 Arm Limited and/or its affiliates.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update copyright to Meta

Summary:
Dynamic Tanh (DyT) normalization computes `tanh(alpha * x)` with a learned scalar
alpha. Quantized, that lowers to a full-tensor integer Mul followed by a tanh
TABLE. On Ethos-U the TABLE is free but the Mul is a real per-element cost, so
the Mul is pure overhead.

This adds `FoldDyTAlphaIntoLUTPass` in `backends/arm/_passes/`, which folds the
alpha multiply into the tanh lookup table and deletes the Mul and its
surrounding rescales.

Folding alpha in floating point before quantization would change rounding, so the
pass does it in the integer domain instead: it replays the exact TOSA
SINGLE_ROUND RESCALE and Mul arithmetic over all 256 int8 input codes, feeds the
results through the existing tanh quantization mapping, and materializes the
result as one 256-entry TABLE. The rewrite is therefore byte-exact rather than
approximate. It fails closed on anything it cannot prove: non-scalar alpha,
activation-side rank views, and narrowed tanh ranges are all handled explicitly.

Also adds `register_pass_factories_before()` to the Arm pass manager, a small
hook for inserting a pass that needs access to the `ExportedProgram` ahead of a
named target pass. `FoldDyTAlphaIntoLUTPass` needs it to read constant tensors.

Note on one import: the pass imports `register_pass_factories_before` inside
`register_fold_dyt_alpha_into_lut_pass()` rather than at module scope.
`_passes/__init__.py` imports `arm_pass_manager` last and `arm_pass_manager`
imports back from the package, so a module-scope import from a pass module that
`__init__.py` re-exports is circular. Only the register helper needs it.

The pass is inert until a model registers it, so this diff changes no behaviour
on its own.

Differential Revision: D116560649
@meta-codesync meta-codesync Bot changed the title Fold DyT alpha scalar into tanh LUT (#21952) Fold DyT alpha scalar into tanh LUT Aug 20, 2026
@zingo zingo added the partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm label Aug 20, 2026
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. meta-exported module: arm Issues related to arm backend partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants