fix(browser): close the pages of a project that has no tests left - #10991
fix(browser): close the pages of a project that has no tests left#10991JCQuintas wants to merge 2 commits into
Conversation
✅ Deploy Preview for vitest-dev ready!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
sheremet-va
left a comment
There was a problem hiding this comment.
I have very strong race condition vibes here, but can't pinpoint it. closePage is never properly awaited which seems dangerous
| return | ||
| } | ||
|
|
||
| debug?.('[%s][%s] closing the page, the project has no tests left', sessionId, this.browserName) |
There was a problem hiding this comment.
how could a function "closePage" know why it was called? "the project has no tests left" seems out of place here
| return orchestrator | ||
| } | ||
|
|
||
| /** |
There was a problem hiding this comment.
A lot of over explanation, everything is obvious from the code
did your agent read AGENTS.md? comments are supposed to be brief
There was a problem hiding this comment.
Yeah, it seems it didn't 😓
It also didn't read my own rules to open PRs in draft by default 😢
78f0b0f to
fd89e72
Compare
Browser pages were only closed by the provider's `close()`, which runs once the whole workspace run is over. Projects run concurrently and each opens up to `maxWorkers` pages, so a run held `projects * maxWorkers` pages at its peak and the projects that finished early kept their page alive until the slowest one was done. Add an optional `closePage` to `BrowserProvider`, implement it for Playwright, and call it from the pool once a session has no test files left. The results of the last test can still be on their way when that happens, so the session waits for them to be handled before its page goes away. In watch mode the pages are kept so that reruns still reuse the session.
7f84bee to
8776457
Compare
Description
Closes #10990.
In browser mode the pages of a project that has finished running its tests stay open until the whole workspace run ends, because they are only closed by the provider's
close(). Projects run concurrently and each opens up tomaxWorkerspages, so a run holdsprojects * maxWorkerspages at its peak and the projects that finish early keep their page — and everything their tests loaded into it — alive until the slowest project is done.This closes a session's pages once its project has no test files left.
BrowserProvidergets an optionalclosePage(sessionId), so a provider that cannot close a single page keeps the current behaviour. Playwright closes the page and leaves the context toclose(), which can still hold pending tracing chunks.BrowserPoolcloses the pages when the last session of a project goes idle, and drops the sessions fromreadySessions/orchestratorsso a later run opens fresh pages. The close is not awaited inside the run — a browser that stopped answering would hold up every other project — but the promise is kept and awaited by the pool'sclose(), which already bounds the wait withPROVIDER_CLOSE_TIMEOUT.BrowserSessionstracks the work a session still has in flight (onTaskUpdate,triggerCommand) so the page cannot be closed while its results are being handled, or while a command it issued is still running.withTimeoutnever cancels the callback it gave up on, so a timed-out hook keeps using the page after its test is over.In watch mode the pages are kept, so reruns still reuse the session.
With the reproduction from #10990 (six projects whose tests finish 500ms apart), before:
after:
For context on why this matters: mui/mui-x has 21 browser projects with
maxWorkers: 2, so its run keeps 42 Chromium tabs open and peaks at ~12.8GB, which is more than its 16GB CI container can hold. A renderer gets OOM-killed and the run fails withBrowser page crashedon whichever project finishes last, with every test passing.The second commit:
specs/runner.test.ts > timeout hooksThat spec failed on
Test: vite@7, browserwhile I was working on this, and it is worth reporting separately even though it turned out not to be caused by the change above — I could not open a second PR for it because of the one-PR limit inAGENTS.md.processTimeoutOptionshanded the providerremaining - 250, so the margin between the provider's error and the task timer was a fixed 250ms whatever the budget. Playwright's timer runs in the Vitest process while the competing task timer runs in the page, so that margin has to cover the wholeprovider -> server -> clientround-trip of the rejection plus any scheduling delay in the Vitest process. Over 918 instrumented provider clicks on a loaded machine the p99 delay is 414ms — larger than the entire margin — and it is worst on the last click of a project's run (p90 138ms, against ≤26ms for the earlier ones).When the task timer wins, the error is
Error: Hook timed out in <n>ms.instead of the descriptive locator error. The spec collects only lines containingTimeoutError:(runner.test.ts:421), so that block silently disappears from the inline snapshot and the assertion fails with one missing entry.So the buffer now scales with the task's budget, and the fixture gets a budget large enough for that to matter — its clicks go from 249ms of margin to 750ms. A 500ms task is unchanged (
min(max(250, 250), 1000) === 250), and the snapshot needs no edit because the value is normalized. The spec takes ~11s instead of ~7s.To be explicit about what I could and could not show: an A/B on the same build with
closePages()gated to a no-op flipped 3 runs out of 9 both with and without closing, it reproduces with a single project running alone, andmainfails the same spec (run 32034308571), so the flakiness is pre-existing. Closing the pages plausibly amplifies it on a 4-vCPU runner by moving page teardown into the slowest project's tail, but I have no CI-side timing that proves that.Testing
lintandtypecheckpass.test/workspaces-browserpasses,specs/playwright-trace.test.tspasses, andspecs/runner.test.ts -t "timeout hooks"passes 4/4 under CPU load where it used to flip.The full
test/browsersuite is not usable as a gate in my environment: it fails 12 specs on a cleanmaincheckout withrolldowndep-scan errors. With this branch it fails the same 12 — no more, no fewer — so I am relying on your CI for the real signal there.