Skip to content

Upstream sync 81/N: merge 92643d68f5 (28 commits, conflict-free) - #1189

Open
roberteg16 wants to merge 35 commits into
rogarcia.merge-upstream-80from
rogarcia.merge-upstream-81
Open

Upstream sync 81/N: merge 92643d68f5 (28 commits, conflict-free)#1189
roberteg16 wants to merge 35 commits into
rogarcia.merge-upstream-80from
rogarcia.merge-upstream-81

Conversation

@roberteg16

@roberteg16 roberteg16 commented Aug 15, 2026

Copy link
Copy Markdown

Context

Eighty-first step of the batched upstream catch-up. Stacked on #1188.

Conflict-free step.

Merged upstream commit 92643d68f5 "K3 DSpark AR fusion (vllm-project#50242)"
Landed on upstream main 2026-07-31 11:12 UTC
Commits in this PR 28

Conflict-free upstream batch: the 28 commits between 4f1da84 and
92643d6 (2026-07-31 01:41 UTC .. 2026-07-31 11:12 UTC). git merge reported
no conflicts; nothing in this merge is a manual resolution.

164 files, +2123/-719. csrc/rocm/, CMakeLists.txt, vllm/platforms/rocm.py,
_aiter_ops.py and the fla tree are all untouched, so no SKINNY=1. No new
pre-commit hook.

One file deleted, vllm/kernels/xpu_ops.py. The surviving "xpu_ops" hits in
fa_utils.py, paged_attn.py, mamba_ssm.py and mixed_precision/xpu.py all import
from vllm._xpu_ops, which is the compiled extension namespace and a different
module; nothing references vllm.kernels.xpu_ops. deepseek_v32/common/fused_ops.py
moves to common/ops/fused_allreduce_rms_norm.py as a 91% rename with no stale
path reference left.

14 definitions are removed and four have no surviving definition under vllm/.
Three - _get_group_hash, is_xpu_kernels_found, prepare_int8_moe_layer_for_cpu -
have zero references tree-wide and are XPU/CPU code.

The fourth looked bad and was worth the trace: FusedMoE, with 15 surviving
references. It is not dangling. Upstream renames the factory,

def FusedMoE(...) -> FusedMoE   ->   def FusedMoEFactory(...) -> MoERunner

and updates both call sites together with their imports. FusedMoEFactory is at
fused_moe/layer.py:99 and MoERunner at fused_moe/runner/moe_runner.py:221, and
both are re-exported from fused_moe/init.py. All 15 remaining mentions of
the old name are in comments and docstrings - there is no import, no
assignment, no class, and no string literal "FusedMoE" anywhere in any file
type, so no lookup can miss.

That last check matters because a renamed symbol surviving only in prose is a
new false-positive shape for this series: a reference count alone reads it as
15 live uses of a deleted definition.

Fork instrumentation intact: create_attention_profiler_scope counts unchanged
in the ROCm/Triton attention backends (8/3/3).

py_compile passes over all 143 changed Python files; ruff check and ruff format
pass on the same set.

Merge commit only — do not squash or rebase.

AI assistance was used to prepare this merge.

Test plan

  • New VLLM_ROCM_USE_AITER_MOE_SITUV2_A8W4 confirmed declared in envs.py — import-time AttributeError otherwise
  • FusedMoE traced: renamed to FusedMoEFactory/MoERunner, all 15 survivors are comments and docstrings, no import/assignment/string literal
  • DummyFusedMoE / DummyCapturer traced to their own test files
  • py_compile over all 52 changed Python files; ruff check and ruff format clean
  • Build + 5-model benchmark sweep vs the Strix Halo dashboard (queued)

AI assistance was used to prepare this merge.

chaeminlim-mb and others added 30 commits July 30, 2026 20:50
…eal qlen/dtype (vllm-project#45227)

Signed-off-by: Edwin Lim <edwin.lim@mangoboost.io>
Signed-off-by: Chaemin Lim <chaemin.lim@mangoboost.io>
Signed-off-by: QinPR <1905873179@qq.com>
Co-authored-by: Edwin Lim <edwin.lim@mangoboost.io>
Co-authored-by: limeward <32970461+edwinlim0919@users.noreply.github.com>
Co-authored-by: QinPR <1905873179@qq.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tion-level uniqueness (vllm-project#50420)

Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Signed-off-by: khluu <khluu000@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
…ject#50467)

Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
…vllm-project#50450)

Signed-off-by: Dino Music <Dino.Music@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
…ct MI325X (vllm-project#50006)

Signed-off-by: vanshbhatia-amd <vansh.bhatia@amd.com>
Signed-off-by: Ryan Rock <ryan.rock@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: depthfirst-app[bot] <184448029+depthfirst-app[bot]@users.noreply.github.com>
Co-authored-by: root <root@smci355-ccs-aus-m11-05.cs-aus.dcgpu>
…50349)

Signed-off-by: Ma Jian <jian1.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
…vllm-project#50434)

Signed-off-by: Wu, Xiaochang <xiaochang.wu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
…m-project#50301)

Signed-off-by: Change72 <changg@nvidia.com>
Co-authored-by: Cursor Agent <cursor-agent@cursor.com>
…6 (gfx950) (vllm-project#49309)

Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
…-project#48047)

Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
…s with a clear error (fixes vllm-project#50337) (vllm-project#50352)

Signed-off-by: seewoo <seewoo@ucsc.edu>
Co-authored-by: Claude <noreply@anthropic.com>
…0293)

Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
…y (4 GPUs) (vllm-project#50373)

Signed-off-by: zengxian <xiangdong.zeng@intel.com>
Co-authored-by: Jiazhi Mi <jiazhimi@tencent.com>
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
…ecific implementation (vllm-project#46981)

Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
…0x (vllm-project#50219)

Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
…t#50517)

Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: bnellnm <49004751+bnellnm@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai>
Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>
Conflict-free upstream batch: the 28 commits between 4f1da84 and
92643d6 (2026-07-31 01:41 UTC .. 2026-07-31 11:12 UTC). git merge reported
no conflicts; nothing in this merge is a manual resolution.

164 files, +2123/-719. csrc/rocm/, CMakeLists.txt, vllm/platforms/rocm.py,
_aiter_ops.py and the fla tree are all untouched, so no SKINNY=1. No new
pre-commit hook.

One file deleted, vllm/kernels/xpu_ops.py. The surviving "xpu_ops" hits in
fa_utils.py, paged_attn.py, mamba_ssm.py and mixed_precision/xpu.py all import
from vllm._xpu_ops, which is the compiled extension namespace and a different
module; nothing references vllm.kernels.xpu_ops. deepseek_v32/common/fused_ops.py
moves to common/ops/fused_allreduce_rms_norm.py as a 91% rename with no stale
path reference left.

14 definitions are removed and four have no surviving definition under vllm/.
Three - _get_group_hash, is_xpu_kernels_found, prepare_int8_moe_layer_for_cpu -
have zero references tree-wide and are XPU/CPU code.

The fourth looked bad and was worth the trace: FusedMoE, with 15 surviving
references. It is not dangling. Upstream renames the factory,

    def FusedMoE(...) -> FusedMoE   ->   def FusedMoEFactory(...) -> MoERunner

and updates both call sites together with their imports. FusedMoEFactory is at
fused_moe/layer.py:99 and MoERunner at fused_moe/runner/moe_runner.py:221, and
both are re-exported from fused_moe/__init__.py. All 15 remaining mentions of
the old name are in comments and docstrings - there is no import, no
assignment, no class, and no string literal "FusedMoE" anywhere in any file
type, so no lookup can miss.

That last check matters because a renamed symbol surviving only in prose is a
new false-positive shape for this series: a reference count alone reads it as
15 live uses of a deleted definition.

Fork instrumentation intact: create_attention_profiler_scope counts unchanged
in the ROCm/Triton attention backends (8/3/3).

py_compile passes over all 143 changed Python files; ruff check and ruff format
pass on the same set.

Signed-off-by: Robert Esclapez Garcia <robert.garcia@amd.com>
@roberteg16

Copy link
Copy Markdown
Author

Verification

Build clean. Correctness runs in CI (test-kernels-correctness); this is the five model benchmarks. All five sanity checks passed.

Test TTFT (ms) dashboard Δ Decode (tok/s) dashboard Δ
Qwen3-30B-A3B-Instruct-2507-AWQ-4bit_128 188.1 189.6 -0.8% 80.5 80.1 +0.5%
Qwen3.6-35B-A3B-W4A16 VLM +int8 lm-head 264.5 283.0 -6.5% 83.9 83.7 +0.2%
Gemma-3-4B-IT_VLM_w4a16 472.5 465.0 +1.6% 61.8 63.6 -2.8%
Qwen3-Omni-30B-A3B-Instruct_VLM_AWQ-4bit 690.7 689.8 +0.1% 76.0 75.7 +0.4%
Qwen2.5-0.5B-Instruct-AWQ_128 (AWQ canary) 17.3 357.1 354.8 +0.6%

Decode is within ±1% except: Gemma-3-4B-IT_VLM_w4a16. If that is the Gemma-3 row it is the #1169 regression (batch 61) inherited through the stack, not introduced here; anything else needs a second build before being called a finding.

The canary's TTFT baseline is stale (predates c4fd9794e9, which removed AsyncMicrobatchTokenizer), so no delta is shown — ~16–17 ms is the current normal.

@roberteg16
roberteg16 marked this pull request as ready for review August 18, 2026 10:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.