Experimental FlyDSL GEMM backend for TE PyTorch (BF16/FP16/FP32/FP8/MXFP8) - #676

Open

aris134 wants to merge 69 commits into

devfrom

amartin/flydsl-gemm-integration

Commits on Jul 22, 2026

Add optional FlyDSL dependency for ROCm PyTorch builds via NVTE_USE_FLYDSL
aris134
committed
cf54ce6
Wire FlyDSL into the TransformerEngine MXFP8 GEMM dispatch path
aris134
committed
2ed8285
Add initial support for TN BF16 FlyDSL GEMM backend
aris134
committed
17b9b74
Add initial support for TN FP16 FlyDSL GEMM backend
aris134
committed
155e81e
Add initial support for TN FP8 FlyDSL GEMM backend
aris134
committed
492a29c
Add initial support for TN FP32 FlyDSL GEMM backend
aris134
committed
6ef76b3
add layout support for NN/NT FlyDSL GEMM
aris134
committed
9a9462e
add support for bf16/fp32 output types for flydsl mxfp8 gemm
aris134
committed
4c9543f
add support for bf16/fp32 output types for flydsl fp8 gemm
aris134
committed
b9eaa8a
add broad output dtype support for flydsl fp8/fp16/bf16 gemms
aris134
committed
71c4ef4
add mixed e4m3/e5m2 fp8 dtype support for flydsl fp8 GEMM
aris134
committed
aa19610
add mixed e4m3/e5m2 fp8 dtype support for flydsl mxfp8 GEMM
aris134
committed
2ee10d3
add pytorch flydsl gemm tests
aris134
committed
979f38c
add the actual pytorch flydsl gemm tests
aris134
committed
e1896cd
add e2e flydsl test in test_numerics
aris134
committed
838e64f
add flydsl gemm fallback support
aris134
committed
9670650

Commits on Jul 23, 2026

Add direct FP8 NN/NT FlyDSL GEMM specializations
aris134
committed
ede7ace

Commits on Jul 28, 2026

feat(flydsl): add FP8 NN and NT GEMM specializations
aris134
committed
842bfa2
Add direct MXFP8 NN/NT FlyDSL GEMM specializations
aris134
committed
1ba15ed
gemm wrappers patch
aris134
committed
2524a08
correct FP8 NT storage contract
aris134
committed
de6d22a
Unify FP8 TN, NN, and NT GEMM paths
aris134
committed
38ccfb2
unify mxfp8 gemm shape variants
aris134
committed
f1a5213
FlyDSL MXFP8: fuse scale prepacking into dedicated GPU kernels
aris134
committed
d36c6c2

Commits on Jul 29, 2026

Normalize MXFP8 TN to direct rowwise storage
aris134
committed
fbc6862
improve backward bf16 gemms with shape specialization
aris134
committed
4d7d273
add shape specialization for flydsl fp16 gemm backend
aris134
committed
e928833
add todo comments
aris134
committed
9d6cbfb
add missing EOLs
aris134
committed
e5f4a0c
Merge remote-tracking branch 'origin/dev' into amartin/flydsl-gemm-integration
aris134
committed
ef46d52
remove calls to old utility get_tolerances and use dtype_tols instead for flydsl test
aris134
committed
0948d9a
add gpu arch gating to flyDSL GEMM backend enablement
aris134
committed
63c5c4c

Commits on Aug 3, 2026

Pin FlyDSL below 0.3 for buffer_ops compatibility
aris134
committed
12d06c7

Commits on Aug 6, 2026

Merge remote-tracking branch 'origin/dev' into amartin/flydsl-gemm-integration
aris134
committed
eb72e5f

Commits on Aug 7, 2026

Migrate FlyDSL GEMM kernels off removed buffer_ops API

aris134
and
claude
committed
10a0813
Prefer system ROCm tree for ROCM_PATH in _rocm_init

aris134
and
claude
committed
fe3fe6b

Commits on Aug 10, 2026

Deduplicate FlyDSL GEMM helpers into gemm_common_utils

aris134
and
claude
committed
9aa4621
Extract shared M/N/K block-tiling guard into require_block_tiling

aris134
and
claude
committed
66633b9
Fuse FP16 and BF16 GEMM kernels into half_prec_gemm

aris134
and
claude
committed
91412fd
Add fused forward BIAS epilogue across all FlyDSL GEMM backends

aris134
and
claude
committed
1e206ca
Add fused GELU_AUX / GELU_AUX_BIAS epilogue to MXFP8 GEMM

aris134
and
claude
committed
b760621

Commits on Aug 11, 2026

Extend fused GELU_AUX / GELU_AUX_BIAS epilogue to all FlyDSL GEMM backends

aris134
and
claude
committed
87b1bb0
Raise FlyDSLUnsupportedError so unsupported GEMM configs fall back

aris134
and
claude
committed
3b7bdaf
Reject comm+GEMM overlap instead of silently dropping it

aris134
and
claude
committed
c131f20
Reject oversized GEMM operands that overflow the int32 launch signature

aris134
and
claude
committed
343b8c3
Address FlyDSL GEMM test review: copyright, CI enrollment, drop smoke block

aris134
and
claude
committed
4402b56
Address review: __init__ header, DSL divmod, import fallback, formatting

aris134
and
claude
committed
1a0ab36
Make test_linear_accuracy_flydsl detect silent FlyDSL fallback

aris134
and
claude
committed
77a069d
Gate flydsl_kernels/test_gemm on gfx950 and detect silent fallback

aris134
and
claude
committed
86b0e2a
Fix flydsl test get_shapes to reference N for NN/NT + add M!=N shape

aris134
and
claude
committed
b4612f0
Fix rank mismatch in multidim FP8 test reference

aris134
and
claude
committed
5c14856
Add AMD copyright + provenance to adapted FlyDSL GEMM utils

aris134
and
claude
committed
68d7b17
Remove dead symbols from FlyDSL GEMM kernels

aris134
and
claude
committed
4d0448a
Remove redundant FP32 double-transpose; core is TN-only

aris134
and
claude
committed
064c1e0
Make MXFP8 shape rejections fall back instead of crashing

aris134
and
claude
committed
6f78bb7
Harden FlyDSL GEMM fallback, test isolation, and lint

aris134
and
claude
committed
0711978
Add MXFP8 fallback regression test

aris134
and
claude
committed
0b89f0a

Commits on Aug 12, 2026

Gate FlyDSL CI suite on gfx950, detected once

aris134
and
claude
committed
d8481b6
Simplify FP8 payload reinterpret to gfx950 OCP-only

aris134
and
claude
committed
b391dca
Drop FlyDSL from install_requires; make it user-installed

aris134
and
claude
committed
9d549bd
Warn when NVTE_USE_FLYDSL=1 but flydsl is not installed

aris134
and
claude
committed
3d276b7
Add AMD provenance header to gemm_common_utils

aris134
and
claude
committed
3296331
Probe system ROCm roots in canonical order in _rocm_init

aris134
and
claude
committed
563f6f7
Require flydsl >= 0.3.0 and document the FlyDSL GEMM backend

aris134
and
claude
committed
f10a416
Drop redundant CI arch gate; rely on test self-skip

aris134
and
claude
committed
e118c96
Add GELU_AUX_BIAS test coverage for regular and fp8 backends

aris134
and
claude
committed
777a53d
Add epilogue-inactivity guard to GELU_AUX_BIAS tests

aris134
and
claude
committed
fd65884

Commits on Aug 13, 2026

Merge remote-tracking branch 'origin/dev' into amartin/flydsl-gemm-integration
aris134
committed
3c4be5d
Test FlyDSL Linear with fp8-initialized weights

aris134
and
claude
committed
3ce2e31