diff --git a/docs/superpowers/specs/2026-07-08-hermes-esm-live-bindings-experiment-runbook.md b/docs/superpowers/specs/2026-07-08-hermes-esm-live-bindings-experiment-runbook.md new file mode 100644 index 0000000000..f66995f0a2 --- /dev/null +++ b/docs/superpowers/specs/2026-07-08-hermes-esm-live-bindings-experiment-runbook.md @@ -0,0 +1,244 @@ +# ESM Live Bindings — Transform & OTA Experiment Runbook + +**Status (2026-08-03):** transform implemented behind a flag; link-time default-import +rewrite implemented and unit-tested; benchmarks and correctness proven. Awaiting +publish + review, then the OTA experiment. + +**Current draft stack** (all `Unpublished`, in dependency order): + +| Diff | What it does | +|---|---| +| D111629268 | `definesESModuleInterop` detector; positive-only `out.isESModule` hint; `unstable_isESModule` threaded to buck | +| D111629272 | Serialize-time rewrite of ESM default imports to `require(dep).default` (+ unit tests) | +| D111629271 | Gates that rewrite on `unstable_staticHermesOptimizedRequire` | +| D111529091 | Opt-in `unstable_liveBindings` transform + live `metroImportDefault` runtime | + +Side branch, **not** on the OTA critical path (see "Do we need the Hermes change?"): + +| Diff | What it does | +|---|---| +| D111629267 | `[hermes][prototype]` `CallRequireImportDefault` opcode + `moduleImportedDefaults_` cache | + +Superseded (do not land): D111143093 → D111148330 (earlier transform, folded into +D111529091), D111629273 (abandoned; its content is folded into D111629268). +D111568389 (`export *` must not re-export `default`) is standalone. + +## What the two halves do + +### Live bindings (D111529091) + +`import-export-plugin` gains a `liveBindings` option, gated behind the +`unstable_liveBindings` custom transform option. When enabled: + +- **Exports (write side):** every reassignment of an exported binding re-publishes + the new value onto `exports` as a plain data-property write (`x = v` → + `exports.x = (x = v)`; `x++` → `(x++, exports.x = x, x)`). Implemented via each + binding's `constantViolations` after a scope crawl, so a shadowing local in a + nested scope is correctly left alone. +- **Re-export forwarding** (`export {x} from`, `export * from`) installs getters, + because the source module can reassign the binding and local assignment tracking + cannot observe that. A module's *own* exports keep the fast data-property path. +- **Imports (read side):** default imports keep the `_$$_IMPORT_DEFAULT` call + shape; liveness comes from the runtime helper caching the source module's + *exports object* rather than the resolved default value, then re-reading + `.default` on each subsequent read. + +Semantics: **live value, relaxed TDZ** — a read before the source module finishes +initialising returns `undefined` rather than throwing. + +### Link-time default-import rewrite (D111629268 → D111629272 → D111629271) + +At serialize time, inside `inlineModuleIds`, default imports of dependencies known +to be ES modules are rewritten: + +```js +_$$_IMPORT_DEFAULT(depMap[0]) → require(depMap[0]).default +``` + +so the read lowers to the Static Hermes `CallRequire` fast path (native +per-`RuntimeModule` export cache) instead of the JS `metroImportDefault` helper. +CJS/unknown targets keep the interop helper, because their default is the whole +`module.exports`, not `.default`. + +The rewrite is padded to the original length so byte offsets and source maps are +unaffected, and it leaves the dependency-map reference intact so the subsequent id +inlining still fires (keeping the resulting `require()` +`CallRequire`-eligible). + +## How the two compose (this is the point) + +`require(dep).default` is a **fresh property read on every evaluation**, so for +ESM-detected targets the rewrite is *inherently live* — and it rides +`CallRequire`. That means: + +- For ESM dependencies, the rewrite gives you liveness **and** the fast path, with + no interop helper. The live-`metroImportDefault` work in D111529091 is redundant + for exactly these sites. +- For CJS/unknown dependencies (not rewritten), D111529091's live + `metroImportDefault` is still what provides default-import liveness. Both paths + are needed. +- Because the rewrite removes helper calls that would otherwise pay the live + helper's cost, it **pays back part of** live bindings' +3.2% bytecode. The two + should be measured together, not independently. + +### Ordering / caching — verified safe + +`inline-requires` runs at **transform** time; the rewrite runs at **serialize** +time on already-cached transform output. So the rewrite cannot be defeated by +use-site memoization, and it cannot poison the transform cache. +`unstable_liveBindings` *is* part of the transform cache key and graph id +(`getGraphId.js`, `transformHelpers.js`, `Transformer.js`), so control and +treatment artifacts cannot be confused. `unstable_esmDefaultImportRewrite` is +derived at bundle time from `unstable_staticHermesOptimizedRequire` and is +deliberately *not* a transform-cache input — it only affects serialization. + +### Known limitation: dev bundles are not rewritten + +In dev builds `keepRequireNames` is on (`keepRequireNames: options.dev`), so +`collectDependencies` appends a debug name: + +```js +_$$_IMPORT_DEFAULT(depMap[0], "./x") +``` + +The rewrite matches only the **single-argument** form, so dev bundles keep the +interop helper. This is intentional — the rewrite targets optimized/production +bundles, which is also the only place `unstable_staticHermesOptimizedRequire` +applies — but it means **you cannot validate the rewrite in a dev bundle.** Pinned +by a test (`does NOT rewrite when a dev-only debug name argument is present`). + +Also note the replacement is always exactly 3 bytes shorter than the helper call +for any dependency-map name and uid suffix, so the "would grow → skip" branch is +unreachable in practice; coverage does not silently drop to zero. + +## Interaction with inline-requires (lazy evaluation) — critical for startup/TTI + +A startup/TTI comparison must credit the inlined baseline for lazy evaluation. +Making bindings live does **not** hoist `require()` to module init: + +```js +// snapshot + inline (today's inlined baseline): +var x; +function use() { return (x || (x = _$$_REQUIRE(dep).x)) + x; } // lazy, memoized VALUE (snapshot) + +// live + inline: +var _dep; +function use() { return (_dep || (_dep = _$$_REQUIRE(dep))).x + _dep.x; } // lazy, memoized MODULE, LIVE member +``` + +The `require()` call sits in the identical position inside the identical lazy +memoization guard in both, so the live variant evaluates the **same module +factories at the same time** as the snapshot baseline. The residual cost is +per-read (a member load vs a cached local) and per-write (republish), not extra +module evaluation. + +Corollary (a *win* for live): for modules with **multiple** named imports, the +snapshot path destructures eagerly at module top, requiring `dep` when the +importer's factory runs. The live path keeps member loads at use sites, so +inline-requires can defer them — live is *lazier* for multi-import modules. + +## Turning it on for a build + +- **Metro CLI / `react-native bundle`:** `--transform-option unstable_liveBindings=true` +- **Programmatic:** `customTransformOptions: {unstable_liveBindings: true}` +- **Metro config:** return it from `transformer.getTransformOptions` + +The rewrite needs no separate switch: it turns on with +`unstable_staticHermesOptimizedRequire`. + +## OTA / Buck-modifier experiment plan + +Goal: ship two OTA bundles of the same app revision and compare HBC size and +startup/TTI on device. + +1. **Pre-reqs.** The target must already build with `experimentalImportSupport` + (live bindings are a no-op otherwise). Turn + `unstable_staticHermesOptimizedRequire` on in **both** arms, so the only delta + between control and treatment is `unstable_liveBindings` — otherwise you are + measuring two changes at once. +2. **Build variants via a Buck modifier** (e.g. `//buck/modifiers:metro_live_bindings`) + that appends `--transform-option unstable_liveBindings=true` to the Metro bundle + action. No source change, so the arms are otherwise identical. +3. **Offline proxy first (fast, deterministic):** per variant, compile to HBC + (`hermes -O -emit-binary -out out.hbc bundle.js`), record `wc -c out.hbc`, then + replay a TTI-marker trace through `hermes_synth` + (`buck run @xplat/mode/hermes/opt hermes_synth -- trace.json out.hbc -marker=`). +4. **Confirm the rewrite actually fired** on the treatment bundle before trusting + any number: grep the optimized bundle for `_$$_IMPORT_DEFAULT` and for + `).default`, and record the ratio. If the helper count is unchanged, the rewrite + did not apply (most likely a dev-mode bundle, or ESM detection returned nothing) + and the experiment is invalid. +5. **On-device confirmation:** the relevant MobileLab TTI test (e.g. + `fb4a.marketplace_cold_start`, metric `hermesTime`) or a QPL-instrumented cold + start, control vs treatment. + +## Measured cost of the transform (2000-module synthetic ESM graph) + +Live bindings only, **without** the default-import rewrite. `hermesc -O -emit-binary`: + +| Bundle | JS bytes | HBC bytes | +|---|---|---| +| snapshot (flag off = today's Metro) | 1,172,027 | 877,541 | +| live (flag on) | 1,200,689 | 905,661 | +| **delta** | **+28,662 (+2.4%)** | **+28,120 (+3.2%)** | + +**Whole-bundle eval time** (real Hermes VM, 25 runs, median, against an +empty-loader baseline): + +| Bundle | total (ms) | graph eval (ms, minus baseline) | +|---|---|---| +| empty loader | 24.90 | — | +| snapshot | 35.29 | 10.40 | +| live | 35.88 | 10.98 | + +Live adds **+0.59 ms (+5.6%)** of graph-eval time for 2000 modules. + +**Heap** (`-gc-print-stats`): total allocated 1,316,808 → 1,348,808 (**+32 KB, ++2.4%**); peak RSS effectively identical (~26 MB, VM-dominated). + +These numbers are an upper-ish bound (every module has a reassigned export) and +predate the rewrite, which should claw back some of the bytecode delta. **Re-measure +the combined stack before quoting a number for the OTA decision.** + +### Unconditional cost when the flag is OFF + +D111529091 is *not* byte-for-byte free at the bundle level, even though the +transform output is unchanged. The live-`metroImportDefault` branch and its +prelude-global read ship in `metro-runtime`'s `require.js` polyfill for **every** +bundle, flag on or off: **~+320 bytes JS / ~+170 bytes** on the metro-buck e2e +fixture bundle. Snapshots updated accordingly. If that matters, the runtime branch +would need to be stripped at build time rather than gated at runtime. + +## Expected impact (from the benchmark suite, D111143093) + +- **Reads:** in the inlined regime live is ~2.3× faster than the `require().x` + baseline; in the non-inlined regime it costs ~+30–40% over the bare-local snapshot. +- **Bytecode size:** the live shape is ~1.8× the non-live bare-local per module, but + smaller than the inlined baseline and ~22% smaller than a Babel-getter design. +- **Startup/init:** the live shape is the cheapest live option to construct, ~3× + cheaper than Babel getters. + +## Do we need the Hermes change (D111629267)? + +Probably **not for this experiment.** Once default imports of ES modules are +rewritten to `require(dep).default`, they already lower to plain `CallRequire` and +hit the existing `moduleExports_` cache in master. A dedicated +`CallRequireImportDefault` opcode would only help the *CJS/unknown* sites that keep +the interop helper. + +Keeping it off the critical path also removes a Hermes **runtime** change from an +OTA experiment, which is much cheaper to ship. The prototype is therefore parked as +a side branch. Before it could land on its own it needs: a real build + test run +(its test plan is still "TODO"), plus JIT (`lib/VM/JIT/arm64/JitEmitter.cpp`) and +SH-native (`lib/BCGen/SH/SH.cpp`, `_sh_ljs_callRequire`) paths — it currently +covers only the interpreter and HBC ISel, so as written it is a bytecode-only +optimization. + +## Remaining follow-ups + +- Publish all four metro diffs and get reviewers (all are `Unpublished`, blocker + `revision_not_accepted`); D111529091 still needs a test plan. +- Namespace imports (`metroImportAll`) remain non-live — separate change. +- Build the Buck modifier and run steps 3–5 above. +- Re-measure size/TTI for the **combined** stack (rewrite + live bindings). +- Decide whether the unconditional runtime cost above is acceptable. diff --git a/packages/metro-transform-plugins/src/import-export-plugin.js b/packages/metro-transform-plugins/src/import-export-plugin.js index e9d1809c46..4153b85628 100644 --- a/packages/metro-transform-plugins/src/import-export-plugin.js +++ b/packages/metro-transform-plugins/src/import-export-plugin.js @@ -36,7 +36,7 @@ export type Options = Readonly<{ importDefault: string, importAll: string, resolve: boolean, - out?: {isESModule: boolean, ...}, + out?: {isESModule?: boolean, ...}, }>; type State = { @@ -570,11 +570,11 @@ export default function importExportPlugin({ state.exportNamed.length ) { body.unshift(esModuleExportTemplate()); + // Only ever set a positive signal: a definite ES module by + // presence of export syntax. if (state.opts.out) { state.opts.out.isESModule = true; } - } else if (state.opts.out) { - state.opts.out.isESModule = false; } }, }, diff --git a/packages/metro-transform-worker/API.md b/packages/metro-transform-worker/API.md index df6d43c32e..6a83af1c3f 100644 --- a/packages/metro-transform-worker/API.md +++ b/packages/metro-transform-worker/API.md @@ -18,6 +18,7 @@ export type JsOutput = Readonly<{ lineCount: number; map: VlqMap; functionMap: null | undefined | FBSourceFunctionMap; + isESModule?: boolean; }>; type: JSFileType; }>; diff --git a/packages/metro-transform-worker/src/__tests__/index-test.js b/packages/metro-transform-worker/src/__tests__/index-test.js index 2387b786ba..5445de1a24 100644 --- a/packages/metro-transform-worker/src/__tests__/index-test.js +++ b/packages/metro-transform-worker/src/__tests__/index-test.js @@ -265,6 +265,159 @@ test('transforms import/export syntax when experimental flag is on', async () => ]); }); +describe('isESModule', () => { + test('is true for an ES module (positive hint from import-export-plugin)', async () => { + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from('export default 42;', 'utf8'), + {...baseTransformOptions, experimentalImportSupport: true}, + ); + + expect(result.output[0].data.isESModule).toBe(true); + }); + + test('is true for ESM already lowered to CJS by Babel (AST fallback)', async () => { + const contents = [ + 'Object.defineProperty(exports, "__esModule", { value: true });', + 'exports.default = 42;', + ].join('\n'); + + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from(contents, 'utf8'), + baseTransformOptions, + ); + + expect(result.output[0].data.isESModule).toBe(true); + }); + + test('is true for the `exports.__esModule = true` assignment form', async () => { + const contents = [ + 'exports.__esModule = true;', + 'exports.default = 42;', + ].join('\n'); + + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from(contents, 'utf8'), + baseTransformOptions, + ); + + expect(result.output[0].data.isESModule).toBe(true); + }); + + test('is true for the sequence-expression assignment form (`@babel/runtime` helpers)', async () => { + // Shape emitted by every helper under `@babel/runtime/helpers/`: + // module.exports = fn, module.exports.__esModule = true, + // module.exports["default"] = module.exports; + const contents = [ + 'function _interopRequireDefault(e) { return e; }', + 'module.exports = _interopRequireDefault,', + ' module.exports.__esModule = true,', + ' module.exports["default"] = module.exports;', + ].join('\n'); + + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from(contents, 'utf8'), + baseTransformOptions, + ); + + expect(result.output[0].data.isESModule).toBe(true); + }); + + test('is unset (never false) for a module with no ESM marker but WITH dependencies', async () => { + // A module with any require call could resolve to an ES module at + // runtime (e.g. `module.exports = require('./esm-thing')`), so we can't + // rule out ESM interop without whole-graph analysis. The provably-not-ESM + // rule requires zero dependencies. + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from("var x = require('./other'); module.exports = x;", 'utf8'), + baseTransformOptions, + ); + + expect(result.output[0].data.isESModule).toBeUndefined(); + }); + + test('is false for a module with no ESM marker and no dependencies', async () => { + // Not ESM: no marker AND no dependencies means the module would have to + // be deliberately obfuscating emission of `__esModule` at runtime, this is + // sufficient proof of non-ESM for our purposes. + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from('module.exports = 42;', 'utf8'), + baseTransformOptions, + ); + + expect(result.output[0].data.isESModule).toBe(false); + }); + + test('is unset (never false) for a CommonJS re-export of an ES module', async () => { + // At runtime this module IS an ES module: it re-exports `./esm`, whose + // `exports.__esModule` is truthy. In isolation, though, the marker isn't + // statically visible here, so we must leave the hint unset rather than + // asserting a misleading `false` (a false negative). + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from("module.exports = require('./esm');", 'utf8'), + baseTransformOptions, + ); + + expect(result.output[0].data.isESModule).toBeUndefined(); + }); + + test('is false for a JSON module (trivially never an ES module)', async () => { + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.json', + Buffer.from('{"foo": 1}', 'utf8'), + baseTransformOptions, + ); + + expect(result.output[0].data.isESModule).toBe(false); + }); + + test('is unset (never false) for a module that only imports (no exports)', async () => { + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from('import "./c";', 'utf8'), + {...baseTransformOptions, experimentalImportSupport: true}, + ); + + expect(result.output[0].data.isESModule).toBeUndefined(); + }); + + test('is unset (never false) for a script', async () => { + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from('doStuff();', 'utf8'), + {...baseTransformOptions, type: 'script'}, + ); + + expect(result.output[0].data.isESModule).toBeUndefined(); + }); +}); + test('does not add "use strict" on non-modules', async () => { const result = await Transformer.transform( baseConfig, diff --git a/packages/metro-transform-worker/src/index.js b/packages/metro-transform-worker/src/index.js index c48f6f8025..70f53ab91a 100644 --- a/packages/metro-transform-worker/src/index.js +++ b/packages/metro-transform-worker/src/index.js @@ -54,6 +54,10 @@ import { } from 'metro-source-map'; import metroTransformPlugins from 'metro-transform-plugins'; import collectDependencies from 'metro/private/ModuleGraph/worker/collectDependencies'; +import { + canDefineESModuleInterop, + definesESModuleInterop, +} from 'metro/private/ModuleGraph/worker/esmClassification'; import generateImportNames from 'metro/private/ModuleGraph/worker/generateImportNames'; import { importLocationsPlugin, @@ -173,6 +177,26 @@ export type JsOutput = Readonly<{ lineCount: number, map: VlqMap, functionMap: ?FBSourceFunctionMap, + // ESM-interop signal. + // + // `true` - definitely an ES module (a truthy top-level + // `exports.__esModule`, as emitted by Metro's own ESM + // transform or by ESM precompiled to CJS by Babel/tsc, i.e. + // a module with a real `.default`). + // `false` - provably NOT an ES module: either trivially (JSON) or + // because the module has no ESM interop marker AND no + // dependencies at all, so it cannot expose ESM interop at + // runtime (nothing to re-export via `module.exports = + // require('./esm')`). Common at FBiOS scale via generated + // Relay fragments and similar build-generated data modules. + // unset - undetermined. A module with `require(...)` calls but no + // ESM marker could still expose ESM interop at runtime, so + // the classifier stays silent. Consumers must fall back to + // the runtime interop helper. + // + // Serialiser-level rewrites use this tri-state to bypass the runtime + // interop helper for definitively-classified reads. + isESModule?: boolean, }>, type: JSFileType, }>; @@ -299,12 +323,18 @@ async function transformJS( // fold requires and perform constant folding (if in dev). const plugins: Array = []; + // Positive-only ESM hint from the import-export-plugin (set to `true` for a + // definite ES module, left unset otherwise). Forwarded to collectDependencies, + // which falls back to AST detection when it is unset. + const importExportOut: {isESModule?: boolean} = {}; + if (options.experimentalImportSupport === true) { plugins.push([ metroTransformPlugins.importExportPlugin, { importAll, importDefault, + out: importExportOut, resolve: false, } as ImportExportPluginOptions, ]); @@ -376,6 +406,19 @@ async function transformJS( let dependencyMapName = ''; let dependencies; + let isESModule = false; + // No ESM marker, no dependencies, and no expression anywhere in the module + // that could define `exports.__esModule` out of view of the top-level scan + // (see `canDefineESModuleInterop`). The dependency check is retained + // separately because a module with dependencies could re-export an ES module + // wholesale - `module.exports = require('./esm.js')` - which is a runtime + // property of the graph rather than of this module's syntax. + // + // Note this establishes the absence of ESM interop, not the presence of + // CommonJS - a script or an empty module qualifies too. Sufficient to cover + // the common FBiOS case: generated Relay fragments and other build-generated + // data modules that literal-export constants and never require anything else. + let hasNoESModuleInterop = false; let wrappedAst; // If the module to transform is a script (meaning that is not part of the @@ -410,6 +453,14 @@ async function transformJS( : null, }; ({ast, dependencies, dependencyMapName} = collectDependencies(ast, opts)); + // Positive-only hint from the import-export-plugin (a definite ES module), + // otherwise infer from the AST (catches ESM already lowered to CJS by + // Babel/tsc, where the plugin saw no ESM syntax). + isESModule = importExportOut.isESModule ?? definesESModuleInterop(ast); + hasNoESModuleInterop = + !isESModule && + dependencies.length === 0 && + !canDefineESModuleInterop(ast); } catch (error) { if (error instanceof InternalInvalidRequireCallError) { throw new InvalidRequireCallError(error, file.filename); @@ -513,6 +564,18 @@ async function transformJS( functionMap: file.functionMap, lineCount, map, + // A tri-state signal (see JsOutput.data.isESModule): + // `true` - definitely an ES module (positive ESM check). + // `false` - definitely no ESM interop: no marker, no dependencies, + // and no expression that could define the marker out of + // view. Not a claim that the module is CommonJS. + // unset - undetermined; consumers must fall back to helper + // behaviour. + ...(isESModule + ? {isESModule: true} + : hasNoESModuleInterop + ? {isESModule: false} + : null), }, type: file.type, }, @@ -638,7 +701,16 @@ async function transformJSON( const outputMap = vlqMapFromTuples(map); const output: Array = [ { - data: {code, functionMap: null, lineCount, map: outputMap}, + data: { + code, + functionMap: null, + lineCount, + map: outputMap, + // JSON is trivially never an ES module, so we can assert a definite + // `false` here (unlike the JS path, where an undetected runtime ESM + // means we must leave the hint unset rather than emit a false negative). + isESModule: false, + }, type: jsType, }, ]; diff --git a/packages/metro/src/DeltaBundler/Serializers/helpers/getSourceMapInfo.js b/packages/metro/src/DeltaBundler/Serializers/helpers/getSourceMapInfo.js index 8513ee4e7b..60151e3c34 100644 --- a/packages/metro/src/DeltaBundler/Serializers/helpers/getSourceMapInfo.js +++ b/packages/metro/src/DeltaBundler/Serializers/helpers/getSourceMapInfo.js @@ -34,8 +34,12 @@ export default function getSourceMapInfo( readonly lineCount: number, readonly isIgnored: boolean, } { + const data = getJsOutput(module).data; return { - ...getJsOutput(module).data, + code: data.code, + functionMap: data.functionMap, + lineCount: data.lineCount, + map: data.map, isIgnored: options.shouldAddToIgnoreList(module), path: options?.getSourceUrl?.(module) ?? module.path, source: options.excludeSource ? '' : getModuleSource(module), diff --git a/packages/metro/src/ModuleGraph/worker/__tests__/esmClassification-test.js b/packages/metro/src/ModuleGraph/worker/__tests__/esmClassification-test.js new file mode 100644 index 0000000000..1e6920d606 --- /dev/null +++ b/packages/metro/src/ModuleGraph/worker/__tests__/esmClassification-test.js @@ -0,0 +1,181 @@ +/** + * Copyright (c) Meta Platforms, Inc. and affiliates. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + * + * @flow strict-local + * @format + * @oncall react_native + */ + +import { + canDefineESModuleInterop, + definesESModuleInterop, +} from '../esmClassification'; +import {parse} from '@babel/parser'; + +const ast = (code: string) => parse(code, {sourceType: 'script'}); + +const defines = (code: string) => definesESModuleInterop(ast(code)); +const canDefine = (code: string) => canDefineESModuleInterop(ast(code)); + +describe('definesESModuleInterop', () => { + test.each([ + ["Object.defineProperty(exports, '__esModule', {value: true});", 'babel'], + ['Object.defineProperty(exports, "__esModule", {value: !0});', 'minified'], + [ + "Object.defineProperty(module.exports, '__esModule', {value: 1});", + 'handwritten wrapper', + ], + ['exports.__esModule = true;', 'loose'], + [ + 'module.exports = f, module.exports.__esModule = true, module.exports["default"] = module.exports;', + '@babel/runtime helper', + ], + ])('detects the marker: %s (%s)', code => { + expect(defines(code)).toBe(true); + }); + + test('does not fire on an unrelated export', () => { + expect(defines('exports.foo = 1;')).toBe(false); + }); + + test('does not fire on a marker nested inside a function', () => { + // Not top-level, so out of scope for this check - which is precisely the + // gap `canDefineESModuleInterop` exists to close. + expect( + defines('function r(e) {Object.defineProperty(e, "__esModule", {});}'), + ).toBe(false); + }); +}); + +describe('canDefineESModuleInterop', () => { + describe('rules a module out', () => { + test('the generated Relay artifact shape', () => { + // Shape emitted by relay-compiler for a fragment, after the Flow types + // (which are comments) are stripped: a single module-scope binding + // initialised from an IIFE returning an object literal, one static + // property write, and a whole-object export. + expect( + canDefine(` + 'use strict'; + var node = (function(){ + var v0 = {"kind": "Literal", "name": "id", "value": 42}; + return { + "argumentDefinitions": [v0], + "kind": "Fragment", + "metadata": null, + "name": "SomeFragment", + "selections": [v0], + "type": "SomeType", + "abstractKey": "__isSomeType" + }; + })(); + if (__DEV__) { + node.hash = "4e3995aa3aa0eb9886c4cfa56381b521"; + } + module.exports = node; + `), + ).toBe(false); + }); + + test('a module with only static named exports', () => { + expect(canDefine('exports.a = 1; exports.b = "two";')).toBe(false); + }); + + test('a module exporting an object literal directly', () => { + expect(canDefine('module.exports = {a: 1, b: 2};')).toBe(false); + }); + + test('an empty module', () => { + expect(canDefine("'use strict';")).toBe(false); + }); + + test('static writes via module.exports.', () => { + expect(canDefine('module.exports.a = 1;')).toBe(false); + }); + }); + + describe('bails out', () => { + test('when the marker is present at the top level', () => { + expect(canDefine('exports.__esModule = true;')).toBe(true); + }); + + test('when the token appears anywhere at all, however nested', () => { + expect( + canDefine( + 'function r(e) {Object.defineProperty(e, "__esModule", {value: 1});}', + ), + ).toBe(true); + }); + + test('when the token appears only as an object key', () => { + expect( + canDefine('module.exports = {__esModule: true, default: 1};'), + ).toBe(true); + }); + + test('on the webpack UMD bundle shape', () => { + // The marker is installed by a helper on a dynamically passed object and + // the exported value is opaque - invisible to a top-level scan, and + // reachable with zero dependencies. This is the real-world case that + // motivates the check (e.g. vendored `*.min.js` bundles). + expect( + canDefine(` + !function(e, t) { + "object" == typeof exports && "object" == typeof module + ? module.exports = t() + : e.math = t(); + }(this, function() { + function i(e) { var t = {exports: {}}; return t.exports; } + i.r = function(e) { + Object.defineProperty(e, "__esModule", {value: !0}); + }; + return i(0); + }); + `), + ).toBe(true); + }); + + test('when exports is passed to a function', () => { + expect(canDefine('makeItESM(exports);')).toBe(true); + }); + + test('when exports is aliased to a local', () => { + expect(canDefine('var e = exports; e.foo = 1;')).toBe(true); + }); + + test('when a property is written under a computed key', () => { + expect(canDefine("exports['__' + 'esModule'] = true;")).toBe(true); + }); + + test('when an object literal uses a computed key', () => { + expect(canDefine("module.exports = {['__' + 'esModule']: true};")).toBe( + true, + ); + }); + + test('when the exported object spreads another value', () => { + expect(canDefine('module.exports = {...someOtherModule};')).toBe(true); + }); + + test('on Object.assign into exports', () => { + expect(canDefine('Object.assign(exports, someOtherModule);')).toBe(true); + }); + + test('on Object.defineProperties', () => { + expect(canDefine('Object.defineProperties(exports, descriptors);')).toBe( + true, + ); + }); + + test('when module is read for something other than exports', () => { + expect(canDefine('module.hot.accept();')).toBe(true); + }); + + test('when the exports object is returned from the module scope', () => { + expect(canDefine('someRegistry.register(module.exports);')).toBe(true); + }); + }); +}); diff --git a/packages/metro/src/ModuleGraph/worker/esmClassification.js b/packages/metro/src/ModuleGraph/worker/esmClassification.js new file mode 100644 index 0000000000..28ade28f49 --- /dev/null +++ b/packages/metro/src/ModuleGraph/worker/esmClassification.js @@ -0,0 +1,289 @@ +/** + * Copyright (c) Meta Platforms, Inc. and affiliates. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + * + * @flow strict-local + * @format + * @oncall react_native + */ + +import type { + CallExpression as BabelNodeCallExpression, + File as BabelNodeFile, + Identifier as BabelNodeIdentifier, + Node as BabelNode, +} from '@babel/types'; + +import * as types from '@babel/types'; + +/** + * Classifies a module's relationship to ESM/CJS interop from two directions: + * + * `definesESModuleInterop` - does this module set `exports.__esModule`? + * `canDefineESModuleInterop` - could it, anywhere we cannot see? + * + * The two are not complements. The first only inspects top-level statements, + * so a false result means "no marker here", not "no marker" - the marker can + * be installed from anywhere the exports object is reachable. The second + * closes that gap, so it takes both to assert that a module has no ESM + * interop at all. + */ + +function isExportsObject(node: BabelNode): boolean { + // `exports` + if (types.isIdentifier(node, {name: 'exports'})) { + return true; + } + // `module.exports` + return ( + types.isMemberExpression(node, {computed: false}) && + types.isIdentifier(node.object, {name: 'module'}) && + types.isIdentifier(node.property, {name: 'exports'}) + ); +} + +function isTruthyConstant(node: BabelNode): boolean { + if (types.isBooleanLiteral(node)) { + return node.value === true; + } + if (types.isNumericLiteral(node)) { + return node.value !== 0; + } + // `!0` (minified `true`) + if (types.isUnaryExpression(node, {operator: '!', prefix: true})) { + return types.isNumericLiteral(node.argument, {value: 0}); + } + return false; +} + +// `Object.defineProperty(exports, "__esModule", { value: })` +function isDefinePropertyESModule(call: BabelNodeCallExpression): boolean { + const callee = call.callee; + if ( + !types.isMemberExpression(callee, {computed: false}) || + !types.isIdentifier(callee.object, {name: 'Object'}) || + !types.isIdentifier(callee.property, {name: 'defineProperty'}) + ) { + return false; + } + const args = call.arguments; + if ( + args.length < 3 || + !isExportsObject(args[0]) || + !types.isStringLiteral(args[1], {value: '__esModule'}) || + !types.isObjectExpression(args[2]) + ) { + return false; + } + return args[2].properties.some( + prop => + types.isObjectProperty(prop, {computed: false}) && + (types.isIdentifier(prop.key, {name: 'value'}) || + types.isStringLiteral(prop.key, {value: 'value'})) && + isTruthyConstant(prop.value), + ); +} + +function expressionSetsESModule(expr: BabelNode): boolean { + if (types.isSequenceExpression(expr)) { + // `a, b, c` at the top level - each subexpression is independently + // observable. Recognise the marker in any position, so patterns like + // `module.exports = fn, module.exports.__esModule = true, ...` from + // `@babel/runtime/helpers/*` are detected. + return expr.expressions.some(expressionSetsESModule); + } + if (types.isCallExpression(expr)) { + return isDefinePropertyESModule(expr); + } + if (!types.isAssignmentExpression(expr) || expr.operator !== '=') { + return false; + } + const left = expr.left; + return ( + types.isMemberExpression(left) && + left.computed !== true && + types.isIdentifier(left.property, {name: '__esModule'}) && + isExportsObject(left.object) && + isTruthyConstant(expr.right) + ); +} + +/** + * Returns whether the given (post-transform) module AST declares ESM/CJS + * interop by setting `exports.__esModule` truthy at the top level. Recognises + * four shapes seen in the wild: + * + * Object.defineProperty(exports, '__esModule', {value: true}) + * - Metro's own ESM transform (`import-export-plugin`) + * - `@babel/plugin-transform-modules-commonjs` (default output) + * - typescript compiler `--module commonjs` + * - rollup with `esModule: true` (default) + * Object.defineProperty(module.exports, '__esModule', {value: true}) + * - handwritten interop wrappers + * exports.__esModule = true // also value 1 or !0 + * - `@babel/plugin-transform-modules-commonjs` with `loose: true` + * - some older tsc output + * - rollup with `esModule: 'if-default-prop'` + * module.exports = fn, module.exports.__esModule = true, ... + * - every helper under `@babel/runtime/helpers/` (sequence expression) + * + * AST-based (not a scan of generated code) so the check is robust to + * whitespace, quoting, and attribute ordering. Intentionally independent of + * whether the import-export-plugin ran: a module already lowered to CJS with + * the marker must still be recognised as an ES module, so this cannot be + * replaced by the plugin's `out.isESModule`. + */ +export function definesESModuleInterop(ast: BabelNodeFile): boolean { + for (const stmt of ast.program.body) { + if ( + types.isExpressionStatement(stmt) && + expressionSetsESModule(stmt.expression) + ) { + return true; + } + } + return false; +} + +// Collects the `exports`/`module` identifier nodes that belong to an export +// write we can fully account for: `module.exports = `, +// `exports. = ` and `module.exports. = `. Any +// occurrence left uncollected is treated as an escape by the caller. +function collectAccountedExportsRefs( + ast: BabelNodeFile, +): Set { + const accounted = new Set(); + + const accountForExportsObject = (node: BabelNode): boolean => { + // `exports` + if (types.isIdentifier(node, {name: 'exports'})) { + accounted.add(node); + return true; + } + // `module.exports` - both identifiers are occurrences of the names we + // track, so both have to be accounted for. Bound to locals so the + // refinements survive the calls that establish them. + if (types.isMemberExpression(node, {computed: false})) { + const object = node.object; + const property = node.property; + if ( + types.isIdentifier(object, {name: 'module'}) && + types.isIdentifier(property, {name: 'exports'}) + ) { + accounted.add(object); + accounted.add(property); + return true; + } + } + return false; + }; + + for (const stmt of ast.program.body) { + if (!types.isExpressionStatement(stmt)) { + continue; + } + const expr = stmt.expression; + if (!types.isAssignmentExpression(expr) || expr.operator !== '=') { + continue; + } + const left = expr.left; + // `module.exports = ` + if (accountForExportsObject(left)) { + continue; + } + // `exports. = ` / `module.exports. = `. A + // computed key is rejected by `hasDynamicPropertyDefinition`. + if (types.isMemberExpression(left, {computed: false})) { + accountForExportsObject(left.object); + } + } + + return accounted; +} + +// Any construct that can define a property whose key is not visible in the +// source text. With the `__esModule` token absent, these are the only +// remaining ways to produce the key (e.g. `exports['__' + 'esModule']`). +function hasDynamicPropertyDefinition(ast: BabelNodeFile): boolean { + let found = false; + types.traverseFast(ast, node => { + if (found) { + return; + } + if ( + // `x[k] = v` + (types.isAssignmentExpression(node) && + types.isMemberExpression(node.left, {computed: true})) || + // `{[k]: v}` + (types.isObjectProperty(node) && node.computed === true) || + // `{...x}` - `x` may carry the key + types.isSpreadElement(node) || + // `Object.assign(target, ...)`, `Object.defineProperties(...)` + (types.isCallExpression(node) && + types.isMemberExpression(node.callee, {computed: false}) && + types.isIdentifier(node.callee.object, {name: 'Object'}) && + (types.isIdentifier(node.callee.property, {name: 'assign'}) || + types.isIdentifier(node.callee.property, { + name: 'defineProperties', + }))) + ) { + found = true; + } + }); + return found; +} + +/** + * Returns whether the module *might* define `exports.__esModule`, i.e. whether + * `definesESModuleInterop` returning false could be a false negative. + * + * `definesESModuleInterop` only inspects top-level statements, so on its own it + * cannot distinguish "no marker" from "marker installed somewhere it can't + * see". A self-contained bundle, for instance, may hand its exports object to a + * helper that sets the key (webpack's `__webpack_require__.r`), which is + * invisible to a statement scan and needs no dependencies to do it. + * + * Returning false is an assertion that no expression in the module can produce + * the key, established by checking that all three hold: + * + * 1. `__esModule` does not occur anywhere, as an identifier or string. + * 2. `exports`/`module` are only ever read as the target of an export write + * we can enumerate - never aliased, passed to a function, or accessed + * with a computed key, any of which would let the key be set out of view. + * 3. No construct can define a property under a key that is not literally + * present in the source (see `hasDynamicPropertyDefinition`), which is + * what closes the gap left by (1). + * + * Deliberately conservative: anything unrecognised returns true. This is not a + * claim that the module is CommonJS - a script or an empty module also + * qualifies - only that it does not opt into ESM interop. + */ +export function canDefineESModuleInterop(ast: BabelNodeFile): boolean { + if (hasDynamicPropertyDefinition(ast)) { + return true; + } + const accounted = collectAccountedExportsRefs(ast); + let unsafe = false; + types.traverseFast(ast, node => { + if (unsafe) { + return; + } + if ( + types.isIdentifier(node, {name: '__esModule'}) || + types.isStringLiteral(node, {value: '__esModule'}) + ) { + unsafe = true; + return; + } + if ( + (types.isIdentifier(node, {name: 'exports'}) || + types.isIdentifier(node, {name: 'module'})) && + !accounted.has(node) + ) { + unsafe = true; + } + }); + return unsafe; +}