Skip to content

Run E2E tests with two workers - #2489

Draft
ejsmith wants to merge 2 commits into
mainfrom
feature/ci-two-e2e-workers
Draft

Run E2E tests with two workers#2489
ejsmith wants to merge 2 commits into
mainfrom
feature/ci-two-e2e-workers

Conversation

@ejsmith

@ejsmith ejsmith commented Aug 18, 2026

Copy link
Copy Markdown
Member

What changed

  • run the CI E2E suite with two Playwright workers instead of one

Why

The required E2E step has grown from six tests in about one minute to 28 tests in about six minutes. The E2E fixtures already include Playwright worker and retry indexes in generated identities, so two workers can use the existing isolation model.

This PR intentionally changes only the worker count so its hosted timing and reliability are attributable.

Local experiment

Run Runtime Result
Full suite, one worker 6.4m 26 passed, 1 flaky, 1 failed after retries
Full suite, two workers 4.1m 26 passed, 2 flaky, no final failures
Saved-view stress matrix, one worker 4.9m 5 passed, 4 flaky
Saved-view stress matrix, two workers 1.6m 7 passed, 2 flaky

The existing saved-view failures reproduced with one worker and were more frequent in the serial stress run, so the sample did not show a concurrency-induced failure mode. Those flakes should still be fixed separately.

Verification

  • npm run check
  • CI=true npx playwright test --list --project=chromium (28 tests discovered)
  • npx prettier --check playwright.config.ts
  • full local Aspire E2E A/B runs shown above

Measurement

All three hosted samples completed successfully and executed all 28 tests:

Attempt Browser install Playwright tests E2E job Retries
1 29s 5m20s 9m17s 3
2 3m56s 4m17s 11m45s 1
3 1m43s 5m42s 10m55s 3

The median Playwright phase is 5m20s versus the recent one-worker median of 6m14s: 54 seconds, or 14%, faster. Browser-install variance is independent of this change and is addressed by #2488.

There were no final failures. The retries were limited to the already-known list-query-cache, saved-views, and stack-triage flakes, all of which also reproduce serially. However, these samples used seven retries total versus three across the three one-worker preinstalled-Chrome control runs. The local stress experiment showed the opposite direction, so the sample does not establish that concurrency caused the difference, but it does not clear the reliability gate either.

This experiment clears the retention gate on hosted test-phase wall time and should remain open as an isolated draft. Do not merge it until the known flakes are fixed or a larger hosted sample shows that the retry rate is not worse than one worker.

Breaking changes

None.

@github-actions

Copy link
Copy Markdown

Code Coverage

Package Line Rate Branch Rate Complexity Health
Exceptionless.Insulation 37% 35% 286
Exceptionless.Core 75% 67% 10184
Exceptionless.Web 85% 69% 7903
Exceptionless.AppHost 38% 41% 147
Summary 78% (25157 / 32154) 67% (11811 / 17554) 18520

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants