feat(runner): propagate AsyncLocalStorage context from fixtures into tests and hooks - #10858
feat(runner): propagate AsyncLocalStorage context from fixtures into tests and hooks#10858VialFlorian wants to merge 10 commits into
Conversation
✅ Deploy Preview for vitest-dev ready!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
bd71b5b to
323fd01
Compare
5b60230 to
eebbab4
Compare
dd9b368 to
d098c33
Compare
|
Confirming I'm the author and maintaining this PR (built with AI assistance) I'm already using this in a private codebase, where the test suite runs every test concurrently, each inside its own |
|
|
||
| ::: warning | ||
| Only test-scoped fixtures propagate their store. Fixtures with `scope: 'file'` or `scope: 'worker'` still provide their **value**, but tests do not run inside their store. | ||
| To establish one store for a whole suite or file, use [`aroundEach`](/api/hooks#aroundeach) or [`aroundAll`](/api/hooks#aroundall) instead. |
There was a problem hiding this comment.
I am confused how aroundEach establishes a store for a whole suite or a file. It runs once per test, not suite or a file
There was a problem hiding this comment.
Yes indeed, I have updated to make it more clear
To set up a store around each test of a suite, register an [`aroundEach`](/api/hooks#aroundeach) hook; to open a single store spanning every test in a suite or file, use [`aroundAll`](/api/hooks#aroundall).
Do you mean playwright fixture also does something similar now with AsyncLocalStorage? |
d098c33 to
5cafca5
Compare
@hi-ogawa No, I assumed it did and that was wrong. I checked with a small repro: a Playwright fixture doing import { AsyncLocalStorage } from 'node:async_hooks'
import { test as base } from '@playwright/test'
const als = new AsyncLocalStorage<{ id: string }>()
const test = base.extend<{ store: { id: string } }>({
store: async ({}, use) => {
const store = { id: 'from-fixture' }
await als.run(store, () => use(store))
},
})
test('als propagation', async ({ store }) => {
console.log(store.id, als.getStore()) // "from-fixture" undefined
})I've corrected the description: the |
14bec30 to
68fe72a
Compare
hi-ogawa
left a comment
There was a problem hiding this comment.
The idea looks sound to me. Still digesting but I have thought of some tests cases that looks worth recording:
- For non auto fixture, the async context becomes available only after fixture execution, so something like this happens:
const test = base.extend({ fixture: () => ...als.run... })
beforeEach(() => als.getStore()) // undefined
test(({ fixture }) => als.getStore()) // fixture store
afterEach(() => als.getStore()) // fixture store- without explicit inter-fixture dependency, each fixture runs still construct each "chained" async context:
const test = base.extend({
fixture1: () => ...als1.run...,
fixture2: () => ...als2.run..., // since fixture1 runs before fixture2, so als1 store is available
})
test({ fixture1, fixture2 }) => als1.getStore(), als2.getStore()) // both stores available
hi-ogawa
left a comment
There was a problem hiding this comment.
I'm still in favor of this feature but found another test case to illustrate the "non composable" nature of runInAsyncScope itself. So making it look like composable does require Vitest internal to coordinate snapshot/restore properly (like especially for aroundEach). It may look too surgical, but I think it's worth tracking as test case.
Custom runTask example
test('fixture snapshots preserve contexts established by a custom runTask', async () => {
const { errorTree } = await runInlineTests({
'vitest.config.ts': `
export default {
test: {
runner: './runner.ts',
},
}
`,
'storage.ts': `
import { AsyncLocalStorage } from 'node:async_hooks'
const key = Symbol.for('vitest.custom-runner-storage')
export const runnerStorage: AsyncLocalStorage<string>
= (globalThis as any)[key] ??= new AsyncLocalStorage<string>()
`,
'runner.ts': `
import type { RunnerTask } from 'vitest'
import { TestRunner } from 'vitest'
import { runnerStorage } from './storage'
export default class CustomRunner extends TestRunner {
async runTask(test: RunnerTask): Promise<void> {
const fn = CustomRunner.getTestFn(test)
await runnerStorage.run('runner', () => fn())
}
}
`,
'custom-runner.test.ts': `
import { AsyncLocalStorage } from 'node:async_hooks'
import { beforeEach, expect, test as base } from 'vitest'
import { runnerStorage } from './storage'
const fixtureStorage = new AsyncLocalStorage<string>()
const test = base.extend({
fixture: [
async ({}, use) => fixtureStorage.run('fixture', () => use('fixture')),
{ auto: true },
],
})
// This causes the auto fixture snapshot to be captured before runTask.
beforeEach(() => {})
base('preserves the runner context without fixtures', () => {
expect(runnerStorage.getStore()).toBe('runner')
})
test('loses the runner context with fixtures', () => {
expect(fixtureStorage.getStore()).toBe('fixture')
expect(runnerStorage.getStore()).toBe('runner')
})
`,
})
expect(errorTree()).toMatchInlineSnapshot(`
{
"custom-runner.test.ts": {
"loses the runner context with fixtures": [
"expected undefined to be 'runner' // Object.is equality",
],
"preserves the runner context without fixtures": "passed",
},
}
`)
})| // equivalent to `AsyncLocalStorage.snapshot()` but with an explicit resource type | ||
| setAsyncContextSnapshotFactory(() => { | ||
| const resource = new AsyncResource('VITEST_ASYNC_CONTEXT_CHAIN') | ||
| return fn => resource.runInAsyncScope(fn) | ||
| }) |
There was a problem hiding this comment.
Is top level code only way? I think it may be more natural to coordinate some runner related init phase somehow.
There was a problem hiding this comment.
No, I can move it into resolveTestRunner, or do you have something else in mind?
| callbackName: 'runTest()', | ||
| onTimeout: error => abortContextSignal(test.context, error), | ||
| invokeHook: (hook, use) => hook(use, test.context, suite), | ||
| advanceAsyncContext: () => refreshAsyncContextChain(test.context), |
There was a problem hiding this comment.
I don't think this rewriting name advanceAsyncContext/refreshAsyncContextChain is called for. Can be just refreshAsyncContextChain all the way.
| `) | ||
| }) | ||
|
|
||
| test('stacked aroundEach stores and a fixture store all accumulate', async () => { |
There was a problem hiding this comment.
We want also another variant where aroundEach overrides same ALS as fixture. I initially had an mislead impression that refreshAsyncContextChain would make it non overridable but it would actually work fine.
Introduce a small module that stores, per test context, an async context snapshot captured at fixture `use()` time, so later user callbacks can be entered inside that captured frame
An AsyncLocalStorage context established around a fixture's `use()` call was lost before the test ran (upstream vitest-dev#5858). Capture a snapshot inside each test-scoped fixture's `use()` and enter later fixtures, hooks and the test body through it. File/worker-scoped fixtures propagate only their value: a shared snapshot would leak context between tests and erase aroundAll stores. The snapshot is a named AsyncResource ignored by detectAsyncLeaks; browsers never register the factory and are unaffected.
A store set by an aroundEach hook's own `als.run()` was erased when a fixture snapshot captured before the hook body was entered later. Re-capture the chain when the hook calls `runTest()` only if a chain exists, so pipelines without fixtures never enter snapshots and keep byte-identical stack traces.
…chain The callbacks are not fixture-wrapped, so they were invoked outside the chain and could not see the stores fixtures had established.
Cleanup functions returned from beforeEach ran outside the chain while afterEach, being fixture-wrapped, ran inside it.
7896326 to
5700932
Compare
Description
Fixtures created with
test.extenddeliver their value through a deferreduse()handshake, so anAsyncLocalStoragecontext established arounduse()never reached the test body:This PR captures an async context snapshot inside each test-scoped fixture's
use()and enters later fixtures,beforeEach/afterEach,onTestFinished/onTestFailed,beforeEachcleanup callbacks and the test body through the latest capture. The fixture handshake, teardown ordering and caching are untouched.What this enables
Per-test stores become trivial to set up (the pattern that motivated this change)
An app whose code reads a request-scoped store (loggers, tenant config, DB clients) can give every test its own isolated store, with per-test overrides, no visible wrapper, and full support for
.only/.skip/.each/test.concurrent:Each test keeps its own store even under
test.concurrent. Today this requires a customit-wrapper helper that loses the chainable API, or is simply impossible where hooks and dependent fixtures must see the store too.Revisiting #5858
This revisits #5858, which was closed in 2024 as intentional ("Fixtures initialization runs separately from test execution") and redirected to #5728, since resolved by
aroundEach/aroundAllin 4.1.aroundEachwraps each test of a suite in a store andaroundAllcan span one store across a whole suite or file, but the fixture form remains the natural way to model per-test context with overrides and composition. Thetest.extendAPI is modeled on Playwright's fixtures; note that Playwright does not propagate the store from fixtures either (verified with a small repro), so this is new capability rather than parity. The approach here did not exist in that discussion: it does not restructure fixture resolution into call nesting, it only re-enters a frame captured atuse()time, so the change is small and strictly additive.Related: #5858, #5728
Design notes
aroundAllstores. Scoped fixtures keep propagating their value only (tested and documented).aroundEache2e snapshots).AsyncResourcethatdetectAsyncLeaksignores, andaroundEach/aroundAllstores compose with fixture stores (covered by the new e2e suite).Please don't delete this checklist! Before submitting the PR, please make sure you do the following:
aroundEachHook Similar to RSpecaround#5728 — see "Revisiting Async context is not passed into tests from fixtures #5858"; can open a fresh discussion first if preferred.)test/e2e/test/fixture-async-context.test.tssuite fails onmain.)pnpm-lock.yamlunless you introduce a new test example.Tests
pnpm test:ci.Documentation
pnpm run docscommand. (New "Fixtures andAsyncLocalStorage" section in the Test Context guide.)Changesets
feat:,fix:,perf:,docs:, orchore:.