Skip to content

[SM100] Select SiTU through the activation argument - #78

Merged
Fridge003 merged 2 commits into
sgl-project:devfrom
Fridge003:codex/explicit-situ-activation
Aug 14, 2026
Merged

[SM100] Select SiTU through the activation argument#78
Fridge003 merged 2 commits into
sgl-project:devfrom
Fridge003:codex/explicit-situ-activation

Conversation

@Fridge003

@Fridge003 Fridge003 commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • accept activation="situ" at the FP8xFP4 MegaMoE host/JIT kernel boundary
  • thread an explicit compile-time SiTU flag into the SM100 kernel instead of inferring it from activation_clamp == 0.03125f
  • preserve activation_clamp as a SwiGLU-only option and reject combining it with SiTU
  • add a GB300 JIT/launch regression that distinguishes explicit SiTU from tightly clamped SwiGLU

This replaces the sentinel selection introduced by #67 while preserving the Kimi-K3 SiTU constants (beta=4.0, linear beta=25.0).

Testing

  • ./build_sgl_deep_gemm.sh
  • python sgl_deep_gemm/tests/test_mega_moe_situ.py --num-processes 1 (GB300)
  • python sgl_deep_gemm/tests/test_mega_moe_l1_sentinel.py --num-processes 1 --num-tokens 32 (GB300, rel-RMSE 1.1648)
  • Python compile checks, shell syntax check, Ruff, and diff checks

@Fridge003
Fridge003 marked this pull request as ready for review August 14, 2026 23:29
@Fridge003
Fridge003 merged commit 80b2c44 into sgl-project:dev Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant