Run E2E tests with two workers - #2489
Draft
ejsmith wants to merge 2 commits into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Why
The required E2E step has grown from six tests in about one minute to 28 tests in about six minutes. The E2E fixtures already include Playwright worker and retry indexes in generated identities, so two workers can use the existing isolation model.
This PR intentionally changes only the worker count so its hosted timing and reliability are attributable.
Local experiment
The existing saved-view failures reproduced with one worker and were more frequent in the serial stress run, so the sample did not show a concurrency-induced failure mode. Those flakes should still be fixed separately.
Verification
npm run checkCI=true npx playwright test --list --project=chromium(28 tests discovered)npx prettier --check playwright.config.tsMeasurement
All three hosted samples completed successfully and executed all 28 tests:
The median Playwright phase is 5m20s versus the recent one-worker median of 6m14s: 54 seconds, or 14%, faster. Browser-install variance is independent of this change and is addressed by #2488.
There were no final failures. The retries were limited to the already-known
list-query-cache,saved-views, andstack-triageflakes, all of which also reproduce serially. However, these samples used seven retries total versus three across the three one-worker preinstalled-Chrome control runs. The local stress experiment showed the opposite direction, so the sample does not establish that concurrency caused the difference, but it does not clear the reliability gate either.This experiment clears the retention gate on hosted test-phase wall time and should remain open as an isolated draft. Do not merge it until the known flakes are fixed or a larger hosted sample shows that the retry rate is not worse than one worker.
Breaking changes
None.