Skip to content

tests: speed up slow tests - #23616

Merged
MikeMcQuaid merged 1 commit into
mainfrom
speed-up-slowest-tests
Aug 22, 2026
Merged

tests: speed up slow tests#23616
MikeMcQuaid merged 1 commit into
mainfrom
speed-up-slowest-tests

Conversation

@bevanjkay

@bevanjkay bevanjkay commented Aug 22, 2026

Copy link
Copy Markdown
Member

This PR optimises some of the slowest tests in the suite, without undermining their accuracy.

  1. Migrator Spec - use zip fixture instead of dmg - avoids the need to mount the image.
Benchmark

Benchmark 1: before Time (mean ± σ): 10.638 s ± 1.252 s [User: 1.239 s, System: 1.053 s] Range (min … max): 7.639 s … 12.059 s 10 runs

Benchmark 2: after
Time (mean ± σ): 2.560 s ± 0.094 s [User: 1.172 s, System: 0.933 s]
Range (min … max): 2.497 s … 2.743 s 10 runs

Summary
after ran
4.16 ± 0.51 times faster than before

  1. Tab Spec - use a file:// url, to prevent the spec from trying to use curl.
Benchmark

Benchmark 1: before Time (mean ± σ): 19.561 s ± 0.207 s [User: 4.323 s, System: 0.955 s] Range (min … max): 19.326 s … 19.981 s 10 runs

Benchmark 2: after
Time (mean ± σ): 5.413 s ± 0.108 s [User: 4.274 s, System: 0.895 s]
Range (min … max): 5.308 s … 5.647 s 10 runs

Summary
after ran
3.61 ± 0.08 times faster than before

  1. spec_helper - stub the timeout so there isn't any unnecessary sleep during the test run.
Benchmark

Benchmark 1: before Time (mean ± σ): 4.052 s ± 0.046 s [User: 1.168 s, System: 0.613 s] Range (min … max): 3.986 s … 4.125 s 10 runs

Benchmark 2: after
Time (mean ± σ): 1.933 s ± 0.019 s [User: 1.128 s, System: 0.557 s]
Range (min … max): 1.908 s … 1.964 s 10 runs

Summary
after ran
2.10 ± 0.03 times faster than before


  • Have you followed our Contributing guidelines?
  • Have you checked for other open Pull Requests for the same change?
  • Have you explained what your changes do? Performance claims (e.g. "this is faster") must include brew benchmark results.
  • Have you explained why you'd like these changes included, not just what they do?
  • For bug fixes, have you given step-by-step brew commands to reproduce the bug?
  • Have you written new tests (excluding integration tests)? Here's an example.
  • Have you successfully run brew lgtm (style, typechecking and tests) locally?

  • I did not use AI/LLM to create this PR, or I disclosed the tool/model below and reviewed its output; I did not attribute commits to AI and will answer maintainer questions and review comments myself without AI/LLM.

I used claude-code with Opus 5 to find the slowest tests, and speed them up. I verified the performance improvement and the changes that were made.

Copilot AI lite review requested due to automatic review settings August 22, 2026 09:37

@MikeMcQuaid MikeMcQuaid left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, good catch!

@MikeMcQuaid
MikeMcQuaid enabled auto-merge August 22, 2026 09:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces overall test suite runtime by adjusting a few slow specs to avoid expensive operations (disk image mounting, external network tooling, and real backoff sleeps) while keeping the same behavioral assertions.

Changes:

  • Update the Cask migrator spec to use a ZIP-based cask fixture instead of a DMG-based fixture to avoid DMG mounting overhead.
  • Update the Tab spec to use a file:// URL in the relevant formula fixtures to avoid triggering curl-related download strategy behavior during Tab.create.
  • Stub Utils.sleep in spec_helper so tests don’t actually wait during Utils.exponential_backoff_sleep (and similar) calls.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
Library/Homebrew/test/cask/migrator_spec.rb Switches the “new cask” fixture to a ZIP variant and updates expectations accordingly to avoid DMG mount costs.
Library/Homebrew/test/tab_spec.rb Uses file:// URLs in the specific Tab.create scenarios to avoid curl/tooling overhead during downloader setup.
Library/Homebrew/test/spec_helper.rb Globally stubs Utils.sleep to eliminate real backoff sleeps during the test suite.

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

@MikeMcQuaid
MikeMcQuaid added this pull request to the merge queue Aug 22, 2026
Merged via the queue into main with commit 33fc750 Aug 22, 2026
49 checks passed
@MikeMcQuaid
MikeMcQuaid deleted the speed-up-slowest-tests branch August 22, 2026 10:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants