From-scratch, heavily-annotated CUDA inference runtime for Qwen2.5-Coder-7B on H100 (sm_90). Custom INT4 packer, fused GEMV, paged KV, split-KV attention, CUDA graph decode — every hot path commented for the why. Educational, not a llama.cpp replacement.

fable-5's improvements to the CIFAR-10 speedrun over the hiverge baseline, measured

Optimized CSM-1B TTS pipeline for RTX 5090 (Blackwell sm_120). CUDA graph replay via patched HF Transformers. ~0.46x RTF. Topics (tags): csm text-to-speech rtx-5090 blackwell cuda-graphs torch-compile sesame streaming pytorch

Measuring what makes a VLA fast enough to run on the robot: a 5.9x CUDA-graph win, four experiments on why low-bit doesn't, a budget-driven deploy-compiler, and a runtime safety supervisor. Live demo: hf.co/spaces/LaelaZ/embodied-efficiency

Reproducible vLLM serving scheduler benchmark with 144-run cross-layer performance analysis.

CUDA-graph-safe per-request activation steering plugin for vLLM. pip install hotwire-vllm

Silicon phonons with a MACE foundation model, benchmarked across four machines — phono3py displacement force sets as the textbook CUDA-graph workload (capture once, replay 111x; x12 single, x28 batched). Sister project to phosbench.

Technical note + runnable demo: cudaMemcpyPeerAsync isn't capturable into a CUDA graph; capture a peer-to-peer copy as a DeviceToDevice UVA memcpy or a peer-access kernel. Confirmed identical on WSL2 and native Linux.

Adaptive CUDA Graph runtime for faster, validated AI inference.

GB10 inference port; see fork.md

From-scratch C++/CUDA LLM inference engine: paged KV cache, continuous batching, CUDA-graph decode. 4,748 tok/s on an RTX 4090 - benchmarked against vLLM and llama.cpp with byte-identical-output gating and fully committed raw data.

Microsecond-scale limit order book inference with custom CUDA/cuTile kernels, TensorRT Plugin V3, CUDA Graphs, and p99 latency benchmarking.

vLLM v0.21 + Qwen 3.6 DFlash + real thinking_token_budget enforcement on Blackwell (sm_120 / sm_121a)

Three measured notebooks on GPU inference optimization: roofline analysis, KV-cache decode optimization (4.21x), and CUDA-graph launch-overhead elimination (5.38x). Pure PyTorch.

PyTorch GPU inference performance exercises: roofline analysis, decode-loop profiling, KV-cache optimization, torch.compile, and CUDA graphs.

DiSpec — a from-scratch LLM inference engine: paged attention, continuous batching, CUDA-graph decode, speculative decoding, and prefill/decode disaggregation

vLLM v0.21 + DFlash + thinking_token_budget for Gemma 4 & Qwen 3.6 on Blackwell GB10 (sm_121a / sm_120)

Prefill performance study on Qwen2.5-7B using vLLM. Compares static vs mixed (bucketed) prefill under eager execution and CUDA Graphs, with controlled concurrency and real-world latency/throughput metrics.

Profile-first CUDA lab for killing GPU pipeline bubbles: stream overlap, CUDA Graphs, torch.compile, spec-decode overlap — Nsight-verified