[Blog] Keeping vLLM Fast Under CPU Pressure: An sched_ext Scheduler for GPU Inference - #300
Open
ianchen0119 wants to merge 4 commits into
Open
[Blog] Keeping vLLM Fast Under CPU Pressure: An sched_ext Scheduler for GPU Inference#300ianchen0119 wants to merge 4 commits into
ianchen0119 wants to merge 4 commits into
Conversation
Author
|
Would you please help to review the PR? |
There was a problem hiding this comment.
Pull request overview
Adds a new ecosystem/community blog post documenting how CPU scheduling contention can bottleneck vLLM GPU inference, and how the Gthulhu sched_ext (SCX) scheduler + tiered policies can recover decode throughput under CPU pressure.
Changes:
- Introduces a new long-form post with a reproducible MicroK8s + GB10 walkthrough and benchmark methodology.
- Documents tiered Gthulhu policy setup (node-level + pod/thread-level) targeting GPU IRQ handling and vLLM’s
EngineCore. - Presents benchmark results comparing EEVDF baseline vs Gthulhu with/without policies.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…or GPU Inference Signed-off-by: Ian Chen <ychen.desl@gmail.com>
…, MicroK8s casing, and align decode drop wording Signed-off-by: Ian Chen <ychen.desl@gmail.com>
ianchen0119
force-pushed
the
post_gthulhu
branch
from
August 7, 2026 05:20
1f13242 to
b88eb9c
Compare
Author
|
cc @vadiklyutiy @ZJY0516 @NickLucche @ZhanqiuHu @njhill @llx-08 @jiahanc for vis since you are mentioned in the blog post. |
- Generate chart_decode.svg and chart_ttft.svg with matplotlib - Replace Chart.js runtime charts with static SVG images - Fix TTFT averages in overall comparison table (A: ~336ms, B: ~126ms, C: ~169ms) - Correct misleading summary that implied C had lower TTFT than B Signed-off-by: Ian Chen <ychen.desl@gmail.com>
ianchen0119
force-pushed
the
post_gthulhu
branch
from
August 11, 2026 15:02
c9030b3 to
52246af
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a new community/ecosystem blog post about improving vLLM inference performance under CPU contention using Gthulhu, a Linux sched_ext (SCX) scheduler.
The post walks through a reproducible experiment on a DGX Spark (GB10, arm64) running vLLM (Qwen2.5-0.5B-Instruct) on MicroK8s, and shows that CPU scheduling — not GPU capacity — is the bottleneck when an inference node is under CPU pressure from noisy neighbors.