docs(testing): add an RC end-to-end computer-use checklist#126
Conversation
Turns the maintainer's informal pre-1.5 manual testing checklist into a durable doc, extended with everything shipped since that a unit/browser test can't exercise: native window capture (vs. display-only), stop reliability with all extras disabled, HUD drag tracking, playhead sync during playback, auto-zoom cursor-following, Full Camera, and export-audio verification -- each phrased as a regression check against the specific issue it guards (#60, #115, #100, #111, #72, #108). Meant to be run top to bottom against a release candidate via the computer-use workflow already documented in AGENTS.md § Desktop E2E testing with computer-use, with a results log at the bottom to track what build was tested and what was found.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughThe RC computer-use checklist adds ordered end-to-end checks for core workflows and Windows/macOS behavior, plus a results log template with an example validation entry. ChangesRC end-to-end testing
Estimated code review effort: 1 (Trivial) | ~2 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/testing/rc-e2e-checklist.md`:
- Line 31: Make the `#115` stop-completion checklist item explicitly Windows-only
and require execution on Windows; update docs/testing/rc-e2e-checklist.md lines
31-31 accordingly. Add an OS/platform column to the results log table at
docs/testing/rc-e2e-checklist.md lines 98-100 so each recorded result identifies
its test platform.
- Line 20: Update the “reproduce twice” guidance in the testing checklist so
intermittent behavioral bugs require two reproductions when practical, while
crashes, hangs, data loss, and security issues are always logged after the first
occurrence with reproduction details.
- Line 43: Update the system audio verification checklist item to require two
separate recordings with opposite toggle states—one enabled and one disabled—and
verify playback matches the corresponding state for each recording.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: a26568bc-123f-4619-b070-9c0fcb75de92
📒 Files selected for processing (1)
docs/testing/rc-e2e-checklist.md
First real run of the checklist against release/v1.7.0. Checked off everything actually verified, left unchecked (with an inline note) whatever wasn't exercised this pass rather than marking it done on faith, and logged the two bugs found + fixed (annotation placeholder text, post-reload duration cap -- both in #127).
- Mark the #115 stop-completion item Windows-only and explicit about needing real Windows hardware (the native WGC stop path it guards isn't exercised on macOS/Linux). - Add a Platform column to the results log so each run records which OS it covered. - Loosen "reproduces twice" to only apply to ambiguous/flaky-looking issues; crashes, hangs, data loss, and security issues get logged on first occurrence. - Tighten the system-audio checklist item to require two recordings in opposite toggle states (not just "two separate recordings"), with playback verified to actually match each state.
Summary
Turns the maintainer's informal pre-1.5 manual testing checklist (source/audio/editor/timeline/persistence smoke test) into a durable doc at
docs/testing/rc-e2e-checklist.md, extended with everything shipped since that a unit/browser test can't exercise: native window capture vs. display-only, stop reliability with all extras disabled, HUD drag tracking, playhead sync during playback, auto-zoom cursor-following, Full Camera, and export-audio verification.Each new item is phrased as an explicit regression check against the issue it guards (#60, #115, #100, #111, #72, #108), so a future run makes clear why that step exists, not just what to click.
Meant to be run top to bottom against a release candidate via the computer-use workflow already documented in
AGENTS.md§ Desktop E2E testing with computer-use. Includes a results log at the bottom to track which build was tested and what was found.Doc-only change, no code touched.
Summary by CodeRabbit