Skip to content

Replace block_ptr in HSTU attn kernel - #4736

Open
scxiao wants to merge 8 commits into
mainfrom
scxiao/hstu_replace_block_ptr
Open

Replace block_ptr in HSTU attn kernel#4736
scxiao wants to merge 8 commits into
mainfrom
scxiao/hstu_replace_block_ptr

Conversation

@scxiao

@scxiao scxiao commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Motivation

block pointer was deprecated from Triton 3.8, this PR is to replace the block pointer in the HSTU attention kernel with normal pointers.

Technical Details

None

Test Plan

Existing correctness test can pass and no performance regression

Test Result

Correctness test passed, saw perf uplift with some tuning parameter changes, see the following table:

  ┌────────────┬─────────────┬──────────┬───────┬──────────┬────────────┬─────────────┬────────────┬────────────┐ 
  │ batch_size │ max_seq_len │ sparsity │ heads │ attn_dim │ hidden_dim │ Triton 3.7  │ Triton 3.7 │ Triton 3.8 │
  │            │             │          │       │          │            │(noPR)TFLOPS │ (PR)TFLOPS │ (PR)TFLOPS │
  ├────────────┼─────────────┼──────────┼───────┼──────────┼────────────┼─────────────┼────────────┼────────────┤ 
  │         32 │        1024 │    0.500 │     4 │      128 │        128 │     130.309 │    156.241 │    160.427 │
  ├────────────┼─────────────┼──────────┼───────┼──────────┼────────────┼─────────────┼────────────┼────────────┤
  │         32 │        2048 │    0.500 │     4 │      128 │        128 │     218.606 │    252.746 │    261.425 │ 
  ├────────────┼─────────────┼──────────┼───────┼──────────┼────────────┼─────────────┼────────────┼────────────┤
  │         32 │        4096 │    0.500 │     4 │      128 │        128 │     344.016 │    382.364 │    393.997 │ 
  ├────────────┼─────────────┼──────────┼───────┼──────────┼────────────┼─────────────┼────────────┼────────────┤ 
  │         32 │        8192 │    0.500 │     4 │      128 │        128 │     427.342 │    462.575 │    478.822 │
  ├────────────┼─────────────┼──────────┼───────┼──────────┼────────────┼─────────────┼────────────┼────────────┤ 
  │         32 │       16384 │    0.500 │     4 │      128 │        128 │     450.008 │    493.805 │    512.873 │
  ├────────────┼─────────────┼──────────┼───────┼──────────┼────────────┼─────────────┼────────────┼────────────┤ 
  │        512 │         512 │    0.970 │     4 │      128 │        128 │     200.882 │    253.690 │    252.419 │
  ├────────────┼─────────────┼──────────┼───────┼──────────┼────────────┼─────────────┼────────────┼────────────┤ 
  │        512 │        3072 │    0.366 │     4 │      128 │        128 │     393.988 │    450.191 │    447.854 │
  ├────────────┼─────────────┼──────────┼───────┼──────────┼────────────┼─────────────┼────────────┼────────────┤ 
  │       1024 │        1024 │    0.768 │     4 │      128 │        128 │     200.937 │    258.980 │    257.411 │
  └────────────┴─────────────┴──────────┴───────┴──────────┴────────────┴─────────────┴────────────┴────────────┘ 

Submission Checklist

@scxiao
scxiao requested a review from a team August 13, 2026 16:20
@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:gfx1250-ffm-triton Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4736 --add-label <label>

@scxiao
scxiao force-pushed the scxiao/hstu_replace_block_ptr branch from 0fd7739 to d20e125 Compare August 13, 2026 16:21
@zufayu
zufayu requested a review from Dewei-Wang-sh August 14, 2026 01:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant