[DEMO] WrenchComposer zero-range regression (perf-gate demo) - #5
[DEMO] WrenchComposer zero-range regression (perf-gate demo)#5Neil4561 wants to merge 2 commits into
Conversation
…m#5265 This is a DEMONSTRATION pull request for the performance regression gate. It is NOT intended to merge. It deliberately reproduces the real, documented per-step performance regression that upstream PR isaac-sim#5265 (the WrenchComposer dual-buffer port) caused on locomotion tasks: when an `apply_external_force_torque` event is configured with all-zero `force_range`/`torque_range` (the default for velocity tasks like H1, G1, Anymal-C), the zero wrenches were still routed through the dual-buffer composer every step, paying the full compose-and-apply cost in `write_data_to_sim` for a semantic no-op (~47% throughput hit). The upstream fix added an early return when both ranges are exactly zero. This commit removes that guard to recreate the regression so the gate can catch it end-to-end on a real PR (mirrored to a pull-request/<N> branch), benchmarked from this PR's own source, and report a RED verdict on Isaac-Velocity-Flat-G1-v0 while untouched tasks stay green. Reference: upstream isaac-sim/IsaacLab PR isaac-sim#5265.
Demo-only change: restrict tasks.json to the two tasks that have sufficient ancestry-selectable baselines for this demonstration run (Cartpole as the unaffected GREEN control, Velocity-Flat-G1 as the regressed RED case). Factory-GearMesh and Repose-Cube-Vision are dropped to keep the build-from-source gate run fast and free of insufficient- baseline noise. Not intended to merge.
Performance Regression GateOverall result🚫 One or more blocking-level performance regressions were detected ✅ 2 passed · Advisory: results are reported for review but do not fail the PR. Why this result is meaningfulThe blocked G1/Newton benchmark was 15.30% slower than its baseline. Its compatible historical runs normally varied by about 1.70% (MAD ÷ median), so this slowdown was roughly 9× larger than normal run-to-run variation. The gate's four-MAD blocking band was approximately 6.79%. The observed slowdown was more than twice that band, making it a clear regression rather than ordinary benchmark noise. Run context
How to read thisStart with BLOCK and HARD FAILURE, then review any WARN rows.
Technical detailsThreshold values, failure phases, retries, hardware, runtime versions, and diagnostic notes:
Updated for |
Demonstration PR — not for merge
This PR exists to demonstrate the Performance Regression Gate catching a
real, documented performance regression end-to-end.
What it does
It deliberately reproduces the per-step regression that upstream
isaac-sim/IsaacLab#5265
(the WrenchComposer dual-buffer port) caused on locomotion tasks. When an
apply_external_force_torqueevent is configured with all-zeroforce_range/torque_range(the default for velocity tasks such as H1, G1,Anymal-C), the zero wrenches were still routed through the dual-buffer composer
every step, paying the full compose-and-apply cost in
write_data_to_simfor asemantic no-op (~47% throughput hit). The upstream fix added an early return
when both ranges are exactly zero; this PR removes that guard to recreate the
regression.
Expected gate verdict
Isaac-Velocity-Flat-G1-v0→ RED (regressed; exercises the zero-range event)Isaac-Cartpole-Direct→ GREEN (unaffected control)The gate builds the image from this PR's own source (production-faithful path),
selects baselines by git merge-base ancestry, and posts a sticky verdict table.
The matrix is scoped to these two tasks for a fast, clean demo run.