Skip to content

docs(testing): add an RC end-to-end computer-use checklist#126

Merged
EtienneLescot merged 3 commits into
mainfrom
docs/rc-e2e-checklist
Jul 19, 2026
Merged

docs(testing): add an RC end-to-end computer-use checklist#126
EtienneLescot merged 3 commits into
mainfrom
docs/rc-e2e-checklist

Conversation

@EtienneLescot

@EtienneLescot EtienneLescot commented Jul 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

Turns the maintainer's informal pre-1.5 manual testing checklist (source/audio/editor/timeline/persistence smoke test) into a durable doc at docs/testing/rc-e2e-checklist.md, extended with everything shipped since that a unit/browser test can't exercise: native window capture vs. display-only, stop reliability with all extras disabled, HUD drag tracking, playhead sync during playback, auto-zoom cursor-following, Full Camera, and export-audio verification.

Each new item is phrased as an explicit regression check against the issue it guards (#60, #115, #100, #111, #72, #108), so a future run makes clear why that step exists, not just what to click.

Meant to be run top to bottom against a release candidate via the computer-use workflow already documented in AGENTS.md § Desktop E2E testing with computer-use. Includes a results log at the bottom to track which build was tested and what was found.

Doc-only change, no code touched.

Summary by CodeRabbit

  • Documentation
    • Added/updated a release-candidate end-to-end manual smoke-test checklist.
    • Expanded validation coverage across capture + launch, audio, editor load + playback, webcam/full camera, timelines, project persistence, and style/other areas.
    • Included step-by-step execution guidance plus a results log template with an example entry and tracking for pass/fail/unexercised items.

Turns the maintainer's informal pre-1.5 manual testing checklist into a
durable doc, extended with everything shipped since that a unit/browser test
can't exercise: native window capture (vs. display-only), stop reliability
with all extras disabled, HUD drag tracking, playhead sync during playback,
auto-zoom cursor-following, Full Camera, and export-audio verification --
each phrased as a regression check against the specific issue it guards
(#60, #115, #100, #111, #72, #108).

Meant to be run top to bottom against a release candidate via the
computer-use workflow already documented in AGENTS.md § Desktop E2E testing
with computer-use, with a results log at the bottom to track what build was
tested and what was found.
@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 58ff9c2d-cab6-4f7a-b504-3cb941d39766

📥 Commits

Reviewing files that changed from the base of the PR and between a5e1858 and ffed2b1.

📒 Files selected for processing (1)
  • docs/testing/rc-e2e-checklist.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • docs/testing/rc-e2e-checklist.md

📝 Walkthrough

Walkthrough

The RC computer-use checklist adds ordered end-to-end checks for core workflows and Windows/macOS behavior, plus a results log template with an example validation entry.

Changes

RC end-to-end testing

Layer / File(s) Summary
Checklist execution and results logging
docs/testing/rc-e2e-checklist.md
Adds execution instructions, detailed checks for capture, audio, playback, camera, timeline, persistence, styling, and platform-specific behavior, plus a results table for recording validation outcomes.

Estimated code review effort: 1 (Trivial) | ~2 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description includes a summary, but it omits most required template sections such as related issue, type, release impact, desktop impact, screenshots, and testing. Add the missing template sections and fill in the related issue, change type, release/desktop impact, screenshots or video, and testing details.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and clearly describes the main change: adding an RC end-to-end computer-use checklist.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/rc-e2e-checklist

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/testing/rc-e2e-checklist.md`:
- Line 31: Make the `#115` stop-completion checklist item explicitly Windows-only
and require execution on Windows; update docs/testing/rc-e2e-checklist.md lines
31-31 accordingly. Add an OS/platform column to the results log table at
docs/testing/rc-e2e-checklist.md lines 98-100 so each recorded result identifies
its test platform.
- Line 20: Update the “reproduce twice” guidance in the testing checklist so
intermittent behavioral bugs require two reproductions when practical, while
crashes, hangs, data loss, and security issues are always logged after the first
occurrence with reproduction details.
- Line 43: Update the system audio verification checklist item to require two
separate recordings with opposite toggle states—one enabled and one disabled—and
verify playback matches the corresponding state for each recording.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a26568bc-123f-4619-b070-9c0fcb75de92

📥 Commits

Reviewing files that changed from the base of the PR and between 44829d6 and 67ef196.

📒 Files selected for processing (1)
  • docs/testing/rc-e2e-checklist.md

Comment thread docs/testing/rc-e2e-checklist.md Outdated
Comment thread docs/testing/rc-e2e-checklist.md Outdated
Comment thread docs/testing/rc-e2e-checklist.md Outdated
First real run of the checklist against release/v1.7.0. Checked off
everything actually verified, left unchecked (with an inline note) whatever
wasn't exercised this pass rather than marking it done on faith, and logged
the two bugs found + fixed (annotation placeholder text, post-reload
duration cap -- both in #127).
- Mark the #115 stop-completion item Windows-only and explicit about needing
  real Windows hardware (the native WGC stop path it guards isn't exercised
  on macOS/Linux).
- Add a Platform column to the results log so each run records which OS it
  covered.
- Loosen "reproduces twice" to only apply to ambiguous/flaky-looking issues;
  crashes, hangs, data loss, and security issues get logged on first
  occurrence.
- Tighten the system-audio checklist item to require two recordings in
  opposite toggle states (not just "two separate recordings"), with playback
  verified to actually match each state.
@EtienneLescot
EtienneLescot merged commit 7013b8b into main Jul 19, 2026
10 checks passed
@EtienneLescot
EtienneLescot deleted the docs/rc-e2e-checklist branch July 19, 2026 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant