tests: speed up slow tests - #23616
Merged
Merged
Conversation
MikeMcQuaid
enabled auto-merge
August 22, 2026 09:38
Contributor
There was a problem hiding this comment.
Pull request overview
This PR reduces overall test suite runtime by adjusting a few slow specs to avoid expensive operations (disk image mounting, external network tooling, and real backoff sleeps) while keeping the same behavioral assertions.
Changes:
- Update the Cask migrator spec to use a ZIP-based cask fixture instead of a DMG-based fixture to avoid DMG mounting overhead.
- Update the Tab spec to use a
file://URL in the relevant formula fixtures to avoid triggering curl-related download strategy behavior duringTab.create. - Stub
Utils.sleepinspec_helperso tests don’t actually wait duringUtils.exponential_backoff_sleep(and similar) calls.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| Library/Homebrew/test/cask/migrator_spec.rb | Switches the “new cask” fixture to a ZIP variant and updates expectations accordingly to avoid DMG mount costs. |
| Library/Homebrew/test/tab_spec.rb | Uses file:// URLs in the specific Tab.create scenarios to avoid curl/tooling overhead during downloader setup. |
| Library/Homebrew/test/spec_helper.rb | Globally stubs Utils.sleep to eliminate real backoff sleeps during the test suite. |
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR optimises some of the slowest tests in the suite, without undermining their accuracy.
zipfixture instead ofdmg- avoids the need to mount the image.Benchmark
Benchmark 1: before Time (mean ± σ): 10.638 s ± 1.252 s [User: 1.239 s, System: 1.053 s] Range (min … max): 7.639 s … 12.059 s 10 runs
Benchmark 2: after
Time (mean ± σ): 2.560 s ± 0.094 s [User: 1.172 s, System: 0.933 s]
Range (min … max): 2.497 s … 2.743 s 10 runs
Summary
after ran
4.16 ± 0.51 times faster than before
file://url, to prevent the spec from trying to usecurl.Benchmark
Benchmark 1: before Time (mean ± σ): 19.561 s ± 0.207 s [User: 4.323 s, System: 0.955 s] Range (min … max): 19.326 s … 19.981 s 10 runs
Benchmark 2: after
Time (mean ± σ): 5.413 s ± 0.108 s [User: 4.274 s, System: 0.895 s]
Range (min … max): 5.308 s … 5.647 s 10 runs
Summary
after ran
3.61 ± 0.08 times faster than before
spec_helper- stub the timeout so there isn't any unnecessary sleep during the test run.Benchmark
Benchmark 1: before Time (mean ± σ): 4.052 s ± 0.046 s [User: 1.168 s, System: 0.613 s] Range (min … max): 3.986 s … 4.125 s 10 runs
Benchmark 2: after
Time (mean ± σ): 1.933 s ± 0.019 s [User: 1.128 s, System: 0.557 s]
Range (min … max): 1.908 s … 1.964 s 10 runs
Summary
after ran
2.10 ± 0.03 times faster than before
brew benchmarkresults.brewcommands to reproduce the bug?brew lgtm(style, typechecking and tests) locally?I used
claude-codewith Opus 5 to find the slowest tests, and speed them up. I verified the performance improvement and the changes that were made.