Skip to content

[DEMO] WrenchComposer zero-range regression (perf-gate demo) - #5

Open
Neil4561 wants to merge 2 commits into
unified-POCfrom
neil/perf-demo-wrench-regression
Open

[DEMO] WrenchComposer zero-range regression (perf-gate demo)#5
Neil4561 wants to merge 2 commits into
unified-POCfrom
neil/perf-demo-wrench-regression

Conversation

@Neil4561

Copy link
Copy Markdown

Demonstration PR — not for merge

This PR exists to demonstrate the Performance Regression Gate catching a
real, documented performance regression end-to-end.

What it does

It deliberately reproduces the per-step regression that upstream
isaac-sim/IsaacLab#5265
(the WrenchComposer dual-buffer port) caused on locomotion tasks. When an
apply_external_force_torque event is configured with all-zero
force_range/torque_range (the default for velocity tasks such as H1, G1,
Anymal-C), the zero wrenches were still routed through the dual-buffer composer
every step, paying the full compose-and-apply cost in write_data_to_sim for a
semantic no-op (~47% throughput hit). The upstream fix added an early return
when both ranges are exactly zero; this PR removes that guard to recreate the
regression.

Expected gate verdict

  • Isaac-Velocity-Flat-G1-v0 → RED (regressed; exercises the zero-range event)
  • Isaac-Cartpole-Direct → GREEN (unaffected control)

The gate builds the image from this PR's own source (production-faithful path),
selects baselines by git merge-base ancestry, and posts a sticky verdict table.

The matrix is scoped to these two tasks for a fast, clean demo run.

Neil4561 added 2 commits June 22, 2026 19:04
…m#5265

This is a DEMONSTRATION pull request for the performance regression gate.
It is NOT intended to merge.

It deliberately reproduces the real, documented per-step performance
regression that upstream PR isaac-sim#5265 (the WrenchComposer dual-buffer port)
caused on locomotion tasks: when an `apply_external_force_torque` event
is configured with all-zero `force_range`/`torque_range` (the default for
velocity tasks like H1, G1, Anymal-C), the zero wrenches were still
routed through the dual-buffer composer every step, paying the full
compose-and-apply cost in `write_data_to_sim` for a semantic no-op
(~47% throughput hit).

The upstream fix added an early return when both ranges are exactly zero.
This commit removes that guard to recreate the regression so the gate can
catch it end-to-end on a real PR (mirrored to a pull-request/<N> branch),
benchmarked from this PR's own source, and report a RED verdict on
Isaac-Velocity-Flat-G1-v0 while untouched tasks stay green.

Reference: upstream isaac-sim/IsaacLab PR isaac-sim#5265.
Demo-only change: restrict tasks.json to the two tasks that have
sufficient ancestry-selectable baselines for this demonstration run
(Cartpole as the unaffected GREEN control, Velocity-Flat-G1 as the
regressed RED case). Factory-GearMesh and Repose-Cube-Vision are dropped
to keep the build-from-source gate run fast and free of insufficient-
baseline noise. Not intended to merge.
@github-actions

github-actions Bot commented Jun 22, 2026

Copy link
Copy Markdown

Performance Regression Gate

Overall result

🚫 One or more blocking-level performance regressions were detected

✅ 2 passed · ⚠️ 1 warning · 🚫 1 blocking signal · ❌ 0 benchmark failures

Advisory: results are reported for review but do not fail the PR.

Why this result is meaningful

The blocked G1/Newton benchmark was 15.30% slower than its baseline. Its compatible historical runs normally varied by about 1.70% (MAD ÷ median), so this slowdown was roughly 9× larger than normal run-to-run variation.

The gate's four-MAD blocking band was approximately 6.79%. The observed slowdown was more than twice that band, making it a clear regression rather than ordinary benchmark noise.

Run context

  • GPU: NVIDIA RTX PRO 6000 Blackwell Server Edition
  • Runtime: cuda=12.8, driver=595.71.05, warp=1.13.0
  • Baseline snapshot: 9c115d3e3f5b

How to read this

Start with BLOCK and HARD FAILURE, then review any WARN rows.

  • ✅ PASS: no meaningful slowdown was detected.
  • ⚠️ WARN: the result is uncertain or performance is in the warning band.
  • 🚫 BLOCK: performance crossed a blocking threshold.
  • ❌ HARD FAILURE: the benchmark did not produce usable FPS.
  • FPS: current throughput; higher is better. Baseline: median of compatible historical runs. Change: + is faster and - is slower.
Task Backend Result FPS Baseline Change Samples What it means
Isaac-Cartpole-Direct newton ⚠️ WARN 983196.0 1010818.0 -2.73% 8 Possible slowdown; review recommended
Isaac-Cartpole-Direct physx ✅ PASS 286334.7 274577.2 +4.28% 8 No meaningful slowdown
Isaac-Velocity-Flat-G1-v0 newton 🚫 BLOCK 28001.9 33058.7 -15.30% 6 About 9× larger than normal 1.70% variation
Isaac-Velocity-Flat-G1-v0 physx ✅ PASS 11686.9 10962.8 +6.61% 6 No meaningful slowdown
Technical details

Threshold values, failure phases, retries, hardware, runtime versions, and diagnostic notes:

Task Backend Verdict FPS Baseline Samples Regression% Floor Threshold Phase Retry GPU Runtime Note
Isaac-Cartpole-Direct newton WARN 983196.0 1010818.0 8 -2.73 N/A rolling_window no NVIDIA RTX PRO 6000 Blackwell Server Edition cuda=12.8, driver=595.71.05, warp=1.13.0 p99/med=1.193; outliers=0
Isaac-Cartpole-Direct physx PASS 286334.7 274577.2 8 4.28 N/A rolling_window no NVIDIA RTX PRO 6000 Blackwell Server Edition cuda=12.8, driver=595.71.05, warp=1.13.0 p99/med=1.242; outliers=0
Isaac-Velocity-Flat-G1-v0 newton BLOCK 28001.9 33058.7 6 -15.30 N/A rolling_window no NVIDIA RTX PRO 6000 Blackwell Server Edition cuda=12.8, driver=595.71.05, warp=1.13.0 baseline MAD=561.0 FPS (1.70%); change=9.0× MAD; four-MAD band=6.79%; p99/med=1.132; outliers=0
Isaac-Velocity-Flat-G1-v0 physx PASS 11686.9 10962.8 6 6.61 N/A rolling_window no NVIDIA RTX PRO 6000 Blackwell Server Edition cuda=12.8, driver=595.71.05, warp=1.13.0 p99/med=1.188; outliers=0

Updated for 38197210 · workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant