From 5c75fa77f0864b83bcab93ebf9bff20e66885d0e Mon Sep 17 00:00:00 2001 From: Vitali Zaidman Date: Wed, 19 Aug 2026 02:34:58 -0700 Subject: [PATCH 1/2] Statically detect transformed ESM and set isESModule on transform results (#1862) Summary: ## What Two pieces of infrastructure that later diffs in the stack build on: AST-based ESM classification in `esmClassification.js`, and a tri-state `isESModule` signal on `JsOutput.data`. The module exports a positive check and a negative one. They are not complements, and it takes both to say a module has no ESM interop. ## `definesESModuleInterop` - the positive check Detects a truthy top-level `exports.__esModule` or `module.exports.__esModule`, where truthy means `true`, `1` or `!0`. Four shapes, covering the common ESM->CJS toolchains: ```js Object.defineProperty(exports, '__esModule', {value: true}); // metro import-export-plugin, babel, tsc, rollup Object.defineProperty(module.exports, '__esModule', {value: true}); // hand-written interop wrappers exports.__esModule = true; // babel loose, older tsc, rollup if-default-prop module.exports = fn, module.exports.__esModule = true, ... // babel/runtime/helpers (sequence expression) ``` AST-based rather than a scan of generated source, so it is robust to whitespace, quoting and property-attribute order. ## `canDefineESModuleInterop` - the negative check The positive check only inspects top-level statements, so a false result means "no marker here", not "no marker". The marker can be installed from anywhere the exports object is reachable: a self-contained bundle can hand its exports to a helper that sets the key (webpack's `__webpack_require__.r`), needing no dependencies and staying invisible to a statement scan. `canDefineESModuleInterop` closes that gap. It returns false - nothing in the module can produce the key - only when all three hold: 1. `__esModule` occurs nowhere, as an identifier or as a string. 2. `exports`/`module` are only ever read as the target of an export write we can enumerate. Never aliased, passed to a function, or accessed with a computed key. 3. No construct can define a property under a key that is not literally in the source: computed member assignment, computed object key, spread, `Object.assign`, `Object.defineProperties`. (1) bars the literal, (3) bars constructing it dynamically, and (2) stops the exports object escaping somewhere the other two cannot see. Anything unrecognised returns true, so the assertion is only ever made on modules whose exports are fully accounted for. This is what lets a `false` be an assertion about the module rather than a statement about where we happened to look. ## Tri-state signal The plugin hint is positive-only: `import-export-plugin` sets `out.isESModule = true` when it processes ESM syntax and leaves the field unset otherwise. A definite `false` there would suppress the AST fallback for ESM already lowered to CJS upstream by babel/tsc/rollup. The worker combines hint and fallback, then widens the result to a tri-state on `JsOutput.data.isESModule`: ```js const isESModule = importExportOut.isESModule ?? definesESModuleInterop(ast); const hasNoESModuleInterop = !isESModule && dependencies.length === 0 && !canDefineESModuleInterop(ast); ``` - `true`: definitely ESM, so it has a real `.default`. - `false`: no ESM interop. No marker, no dependencies, and no expression that could define the marker out of view. This establishes the absence of ESM interop, not the presence of CommonJS - a script or an empty module qualifies too. Common at FBiOS scale via generated Relay fragments. - unset: undetermined. A module with `require(...)` calls but no marker can still expose interop via `module.exports = require('./esm')`, so consumers must fall back to the runtime helper. The dependency clause is kept alongside the new check because re-exporting an ES module wholesale is a property of the graph, not of this module's syntax, so it is not something an AST check can rule out. Detection lives in the worker rather than `collectDependencies` because the worker owns both the plugin hint and the JsOutput. `getSourceMapInfo` stops spreading `JsOutput.data` and names its four fields explicitly, so the new optional field cannot leak into source map info: ```js const data = getJsOutput(module).data; return {code: data.code, functionMap: data.functionMap, lineCount: data.lineCount, map: data.map, ...}; ``` OSS serialisers pick the field up automatically via `module.output[].data`. Metro-Buck bridges it explicitly in a follow-up, since it re-serialises through its own module IR. ## Coverage Against the release FBiOS RN bundle (17,058 modules): 12,475 (73.1%) detected ESM, 4,581 (26.8%) with no `__esModule` at runtime, 2 (0.01%) missed. Both misses are IIFE-wrapped UMD bundles (one of them `whatwg-fetch.umd.js`) with the marker inside an inner factory; resolving `exports` through the IIFE is not worth it for two modules. Those numbers measure the positive check, which is unchanged. The negative check does not affect them - it only narrows which of the unmarked modules are asserted `false` rather than left unset. UMD of exactly that shape is now declined rather than asserted, so the two misses cannot become a wrong `false` if such a module has no dependencies. Reviewed By: huntie Differential Revision: D111629268 --- .../src/import-export-plugin.js | 6 +- packages/metro-transform-worker/API.md | 1 + .../src/__tests__/index-test.js | 153 ++++++++++ packages/metro-transform-worker/src/index.js | 74 ++++- .../Serializers/helpers/getSourceMapInfo.js | 6 +- .../__tests__/esmClassification-test.js | 181 +++++++++++ .../ModuleGraph/worker/esmClassification.js | 289 ++++++++++++++++++ 7 files changed, 705 insertions(+), 5 deletions(-) create mode 100644 packages/metro/src/ModuleGraph/worker/__tests__/esmClassification-test.js create mode 100644 packages/metro/src/ModuleGraph/worker/esmClassification.js diff --git a/packages/metro-transform-plugins/src/import-export-plugin.js b/packages/metro-transform-plugins/src/import-export-plugin.js index e9d1809c46..4153b85628 100644 --- a/packages/metro-transform-plugins/src/import-export-plugin.js +++ b/packages/metro-transform-plugins/src/import-export-plugin.js @@ -36,7 +36,7 @@ export type Options = Readonly<{ importDefault: string, importAll: string, resolve: boolean, - out?: {isESModule: boolean, ...}, + out?: {isESModule?: boolean, ...}, }>; type State = { @@ -570,11 +570,11 @@ export default function importExportPlugin({ state.exportNamed.length ) { body.unshift(esModuleExportTemplate()); + // Only ever set a positive signal: a definite ES module by + // presence of export syntax. if (state.opts.out) { state.opts.out.isESModule = true; } - } else if (state.opts.out) { - state.opts.out.isESModule = false; } }, }, diff --git a/packages/metro-transform-worker/API.md b/packages/metro-transform-worker/API.md index df6d43c32e..6a83af1c3f 100644 --- a/packages/metro-transform-worker/API.md +++ b/packages/metro-transform-worker/API.md @@ -18,6 +18,7 @@ export type JsOutput = Readonly<{ lineCount: number; map: VlqMap; functionMap: null | undefined | FBSourceFunctionMap; + isESModule?: boolean; }>; type: JSFileType; }>; diff --git a/packages/metro-transform-worker/src/__tests__/index-test.js b/packages/metro-transform-worker/src/__tests__/index-test.js index 2387b786ba..5445de1a24 100644 --- a/packages/metro-transform-worker/src/__tests__/index-test.js +++ b/packages/metro-transform-worker/src/__tests__/index-test.js @@ -265,6 +265,159 @@ test('transforms import/export syntax when experimental flag is on', async () => ]); }); +describe('isESModule', () => { + test('is true for an ES module (positive hint from import-export-plugin)', async () => { + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from('export default 42;', 'utf8'), + {...baseTransformOptions, experimentalImportSupport: true}, + ); + + expect(result.output[0].data.isESModule).toBe(true); + }); + + test('is true for ESM already lowered to CJS by Babel (AST fallback)', async () => { + const contents = [ + 'Object.defineProperty(exports, "__esModule", { value: true });', + 'exports.default = 42;', + ].join('\n'); + + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from(contents, 'utf8'), + baseTransformOptions, + ); + + expect(result.output[0].data.isESModule).toBe(true); + }); + + test('is true for the `exports.__esModule = true` assignment form', async () => { + const contents = [ + 'exports.__esModule = true;', + 'exports.default = 42;', + ].join('\n'); + + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from(contents, 'utf8'), + baseTransformOptions, + ); + + expect(result.output[0].data.isESModule).toBe(true); + }); + + test('is true for the sequence-expression assignment form (`@babel/runtime` helpers)', async () => { + // Shape emitted by every helper under `@babel/runtime/helpers/`: + // module.exports = fn, module.exports.__esModule = true, + // module.exports["default"] = module.exports; + const contents = [ + 'function _interopRequireDefault(e) { return e; }', + 'module.exports = _interopRequireDefault,', + ' module.exports.__esModule = true,', + ' module.exports["default"] = module.exports;', + ].join('\n'); + + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from(contents, 'utf8'), + baseTransformOptions, + ); + + expect(result.output[0].data.isESModule).toBe(true); + }); + + test('is unset (never false) for a module with no ESM marker but WITH dependencies', async () => { + // A module with any require call could resolve to an ES module at + // runtime (e.g. `module.exports = require('./esm-thing')`), so we can't + // rule out ESM interop without whole-graph analysis. The provably-not-ESM + // rule requires zero dependencies. + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from("var x = require('./other'); module.exports = x;", 'utf8'), + baseTransformOptions, + ); + + expect(result.output[0].data.isESModule).toBeUndefined(); + }); + + test('is false for a module with no ESM marker and no dependencies', async () => { + // Not ESM: no marker AND no dependencies means the module would have to + // be deliberately obfuscating emission of `__esModule` at runtime, this is + // sufficient proof of non-ESM for our purposes. + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from('module.exports = 42;', 'utf8'), + baseTransformOptions, + ); + + expect(result.output[0].data.isESModule).toBe(false); + }); + + test('is unset (never false) for a CommonJS re-export of an ES module', async () => { + // At runtime this module IS an ES module: it re-exports `./esm`, whose + // `exports.__esModule` is truthy. In isolation, though, the marker isn't + // statically visible here, so we must leave the hint unset rather than + // asserting a misleading `false` (a false negative). + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from("module.exports = require('./esm');", 'utf8'), + baseTransformOptions, + ); + + expect(result.output[0].data.isESModule).toBeUndefined(); + }); + + test('is false for a JSON module (trivially never an ES module)', async () => { + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.json', + Buffer.from('{"foo": 1}', 'utf8'), + baseTransformOptions, + ); + + expect(result.output[0].data.isESModule).toBe(false); + }); + + test('is unset (never false) for a module that only imports (no exports)', async () => { + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from('import "./c";', 'utf8'), + {...baseTransformOptions, experimentalImportSupport: true}, + ); + + expect(result.output[0].data.isESModule).toBeUndefined(); + }); + + test('is unset (never false) for a script', async () => { + const result = await Transformer.transform( + baseConfig, + '/root', + 'local/file.js', + Buffer.from('doStuff();', 'utf8'), + {...baseTransformOptions, type: 'script'}, + ); + + expect(result.output[0].data.isESModule).toBeUndefined(); + }); +}); + test('does not add "use strict" on non-modules', async () => { const result = await Transformer.transform( baseConfig, diff --git a/packages/metro-transform-worker/src/index.js b/packages/metro-transform-worker/src/index.js index c48f6f8025..70f53ab91a 100644 --- a/packages/metro-transform-worker/src/index.js +++ b/packages/metro-transform-worker/src/index.js @@ -54,6 +54,10 @@ import { } from 'metro-source-map'; import metroTransformPlugins from 'metro-transform-plugins'; import collectDependencies from 'metro/private/ModuleGraph/worker/collectDependencies'; +import { + canDefineESModuleInterop, + definesESModuleInterop, +} from 'metro/private/ModuleGraph/worker/esmClassification'; import generateImportNames from 'metro/private/ModuleGraph/worker/generateImportNames'; import { importLocationsPlugin, @@ -173,6 +177,26 @@ export type JsOutput = Readonly<{ lineCount: number, map: VlqMap, functionMap: ?FBSourceFunctionMap, + // ESM-interop signal. + // + // `true` - definitely an ES module (a truthy top-level + // `exports.__esModule`, as emitted by Metro's own ESM + // transform or by ESM precompiled to CJS by Babel/tsc, i.e. + // a module with a real `.default`). + // `false` - provably NOT an ES module: either trivially (JSON) or + // because the module has no ESM interop marker AND no + // dependencies at all, so it cannot expose ESM interop at + // runtime (nothing to re-export via `module.exports = + // require('./esm')`). Common at FBiOS scale via generated + // Relay fragments and similar build-generated data modules. + // unset - undetermined. A module with `require(...)` calls but no + // ESM marker could still expose ESM interop at runtime, so + // the classifier stays silent. Consumers must fall back to + // the runtime interop helper. + // + // Serialiser-level rewrites use this tri-state to bypass the runtime + // interop helper for definitively-classified reads. + isESModule?: boolean, }>, type: JSFileType, }>; @@ -299,12 +323,18 @@ async function transformJS( // fold requires and perform constant folding (if in dev). const plugins: Array = []; + // Positive-only ESM hint from the import-export-plugin (set to `true` for a + // definite ES module, left unset otherwise). Forwarded to collectDependencies, + // which falls back to AST detection when it is unset. + const importExportOut: {isESModule?: boolean} = {}; + if (options.experimentalImportSupport === true) { plugins.push([ metroTransformPlugins.importExportPlugin, { importAll, importDefault, + out: importExportOut, resolve: false, } as ImportExportPluginOptions, ]); @@ -376,6 +406,19 @@ async function transformJS( let dependencyMapName = ''; let dependencies; + let isESModule = false; + // No ESM marker, no dependencies, and no expression anywhere in the module + // that could define `exports.__esModule` out of view of the top-level scan + // (see `canDefineESModuleInterop`). The dependency check is retained + // separately because a module with dependencies could re-export an ES module + // wholesale - `module.exports = require('./esm.js')` - which is a runtime + // property of the graph rather than of this module's syntax. + // + // Note this establishes the absence of ESM interop, not the presence of + // CommonJS - a script or an empty module qualifies too. Sufficient to cover + // the common FBiOS case: generated Relay fragments and other build-generated + // data modules that literal-export constants and never require anything else. + let hasNoESModuleInterop = false; let wrappedAst; // If the module to transform is a script (meaning that is not part of the @@ -410,6 +453,14 @@ async function transformJS( : null, }; ({ast, dependencies, dependencyMapName} = collectDependencies(ast, opts)); + // Positive-only hint from the import-export-plugin (a definite ES module), + // otherwise infer from the AST (catches ESM already lowered to CJS by + // Babel/tsc, where the plugin saw no ESM syntax). + isESModule = importExportOut.isESModule ?? definesESModuleInterop(ast); + hasNoESModuleInterop = + !isESModule && + dependencies.length === 0 && + !canDefineESModuleInterop(ast); } catch (error) { if (error instanceof InternalInvalidRequireCallError) { throw new InvalidRequireCallError(error, file.filename); @@ -513,6 +564,18 @@ async function transformJS( functionMap: file.functionMap, lineCount, map, + // A tri-state signal (see JsOutput.data.isESModule): + // `true` - definitely an ES module (positive ESM check). + // `false` - definitely no ESM interop: no marker, no dependencies, + // and no expression that could define the marker out of + // view. Not a claim that the module is CommonJS. + // unset - undetermined; consumers must fall back to helper + // behaviour. + ...(isESModule + ? {isESModule: true} + : hasNoESModuleInterop + ? {isESModule: false} + : null), }, type: file.type, }, @@ -638,7 +701,16 @@ async function transformJSON( const outputMap = vlqMapFromTuples(map); const output: Array = [ { - data: {code, functionMap: null, lineCount, map: outputMap}, + data: { + code, + functionMap: null, + lineCount, + map: outputMap, + // JSON is trivially never an ES module, so we can assert a definite + // `false` here (unlike the JS path, where an undetected runtime ESM + // means we must leave the hint unset rather than emit a false negative). + isESModule: false, + }, type: jsType, }, ]; diff --git a/packages/metro/src/DeltaBundler/Serializers/helpers/getSourceMapInfo.js b/packages/metro/src/DeltaBundler/Serializers/helpers/getSourceMapInfo.js index 8513ee4e7b..60151e3c34 100644 --- a/packages/metro/src/DeltaBundler/Serializers/helpers/getSourceMapInfo.js +++ b/packages/metro/src/DeltaBundler/Serializers/helpers/getSourceMapInfo.js @@ -34,8 +34,12 @@ export default function getSourceMapInfo( readonly lineCount: number, readonly isIgnored: boolean, } { + const data = getJsOutput(module).data; return { - ...getJsOutput(module).data, + code: data.code, + functionMap: data.functionMap, + lineCount: data.lineCount, + map: data.map, isIgnored: options.shouldAddToIgnoreList(module), path: options?.getSourceUrl?.(module) ?? module.path, source: options.excludeSource ? '' : getModuleSource(module), diff --git a/packages/metro/src/ModuleGraph/worker/__tests__/esmClassification-test.js b/packages/metro/src/ModuleGraph/worker/__tests__/esmClassification-test.js new file mode 100644 index 0000000000..1e6920d606 --- /dev/null +++ b/packages/metro/src/ModuleGraph/worker/__tests__/esmClassification-test.js @@ -0,0 +1,181 @@ +/** + * Copyright (c) Meta Platforms, Inc. and affiliates. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + * + * @flow strict-local + * @format + * @oncall react_native + */ + +import { + canDefineESModuleInterop, + definesESModuleInterop, +} from '../esmClassification'; +import {parse} from '@babel/parser'; + +const ast = (code: string) => parse(code, {sourceType: 'script'}); + +const defines = (code: string) => definesESModuleInterop(ast(code)); +const canDefine = (code: string) => canDefineESModuleInterop(ast(code)); + +describe('definesESModuleInterop', () => { + test.each([ + ["Object.defineProperty(exports, '__esModule', {value: true});", 'babel'], + ['Object.defineProperty(exports, "__esModule", {value: !0});', 'minified'], + [ + "Object.defineProperty(module.exports, '__esModule', {value: 1});", + 'handwritten wrapper', + ], + ['exports.__esModule = true;', 'loose'], + [ + 'module.exports = f, module.exports.__esModule = true, module.exports["default"] = module.exports;', + '@babel/runtime helper', + ], + ])('detects the marker: %s (%s)', code => { + expect(defines(code)).toBe(true); + }); + + test('does not fire on an unrelated export', () => { + expect(defines('exports.foo = 1;')).toBe(false); + }); + + test('does not fire on a marker nested inside a function', () => { + // Not top-level, so out of scope for this check - which is precisely the + // gap `canDefineESModuleInterop` exists to close. + expect( + defines('function r(e) {Object.defineProperty(e, "__esModule", {});}'), + ).toBe(false); + }); +}); + +describe('canDefineESModuleInterop', () => { + describe('rules a module out', () => { + test('the generated Relay artifact shape', () => { + // Shape emitted by relay-compiler for a fragment, after the Flow types + // (which are comments) are stripped: a single module-scope binding + // initialised from an IIFE returning an object literal, one static + // property write, and a whole-object export. + expect( + canDefine(` + 'use strict'; + var node = (function(){ + var v0 = {"kind": "Literal", "name": "id", "value": 42}; + return { + "argumentDefinitions": [v0], + "kind": "Fragment", + "metadata": null, + "name": "SomeFragment", + "selections": [v0], + "type": "SomeType", + "abstractKey": "__isSomeType" + }; + })(); + if (__DEV__) { + node.hash = "4e3995aa3aa0eb9886c4cfa56381b521"; + } + module.exports = node; + `), + ).toBe(false); + }); + + test('a module with only static named exports', () => { + expect(canDefine('exports.a = 1; exports.b = "two";')).toBe(false); + }); + + test('a module exporting an object literal directly', () => { + expect(canDefine('module.exports = {a: 1, b: 2};')).toBe(false); + }); + + test('an empty module', () => { + expect(canDefine("'use strict';")).toBe(false); + }); + + test('static writes via module.exports.', () => { + expect(canDefine('module.exports.a = 1;')).toBe(false); + }); + }); + + describe('bails out', () => { + test('when the marker is present at the top level', () => { + expect(canDefine('exports.__esModule = true;')).toBe(true); + }); + + test('when the token appears anywhere at all, however nested', () => { + expect( + canDefine( + 'function r(e) {Object.defineProperty(e, "__esModule", {value: 1});}', + ), + ).toBe(true); + }); + + test('when the token appears only as an object key', () => { + expect( + canDefine('module.exports = {__esModule: true, default: 1};'), + ).toBe(true); + }); + + test('on the webpack UMD bundle shape', () => { + // The marker is installed by a helper on a dynamically passed object and + // the exported value is opaque - invisible to a top-level scan, and + // reachable with zero dependencies. This is the real-world case that + // motivates the check (e.g. vendored `*.min.js` bundles). + expect( + canDefine(` + !function(e, t) { + "object" == typeof exports && "object" == typeof module + ? module.exports = t() + : e.math = t(); + }(this, function() { + function i(e) { var t = {exports: {}}; return t.exports; } + i.r = function(e) { + Object.defineProperty(e, "__esModule", {value: !0}); + }; + return i(0); + }); + `), + ).toBe(true); + }); + + test('when exports is passed to a function', () => { + expect(canDefine('makeItESM(exports);')).toBe(true); + }); + + test('when exports is aliased to a local', () => { + expect(canDefine('var e = exports; e.foo = 1;')).toBe(true); + }); + + test('when a property is written under a computed key', () => { + expect(canDefine("exports['__' + 'esModule'] = true;")).toBe(true); + }); + + test('when an object literal uses a computed key', () => { + expect(canDefine("module.exports = {['__' + 'esModule']: true};")).toBe( + true, + ); + }); + + test('when the exported object spreads another value', () => { + expect(canDefine('module.exports = {...someOtherModule};')).toBe(true); + }); + + test('on Object.assign into exports', () => { + expect(canDefine('Object.assign(exports, someOtherModule);')).toBe(true); + }); + + test('on Object.defineProperties', () => { + expect(canDefine('Object.defineProperties(exports, descriptors);')).toBe( + true, + ); + }); + + test('when module is read for something other than exports', () => { + expect(canDefine('module.hot.accept();')).toBe(true); + }); + + test('when the exports object is returned from the module scope', () => { + expect(canDefine('someRegistry.register(module.exports);')).toBe(true); + }); + }); +}); diff --git a/packages/metro/src/ModuleGraph/worker/esmClassification.js b/packages/metro/src/ModuleGraph/worker/esmClassification.js new file mode 100644 index 0000000000..28ade28f49 --- /dev/null +++ b/packages/metro/src/ModuleGraph/worker/esmClassification.js @@ -0,0 +1,289 @@ +/** + * Copyright (c) Meta Platforms, Inc. and affiliates. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + * + * @flow strict-local + * @format + * @oncall react_native + */ + +import type { + CallExpression as BabelNodeCallExpression, + File as BabelNodeFile, + Identifier as BabelNodeIdentifier, + Node as BabelNode, +} from '@babel/types'; + +import * as types from '@babel/types'; + +/** + * Classifies a module's relationship to ESM/CJS interop from two directions: + * + * `definesESModuleInterop` - does this module set `exports.__esModule`? + * `canDefineESModuleInterop` - could it, anywhere we cannot see? + * + * The two are not complements. The first only inspects top-level statements, + * so a false result means "no marker here", not "no marker" - the marker can + * be installed from anywhere the exports object is reachable. The second + * closes that gap, so it takes both to assert that a module has no ESM + * interop at all. + */ + +function isExportsObject(node: BabelNode): boolean { + // `exports` + if (types.isIdentifier(node, {name: 'exports'})) { + return true; + } + // `module.exports` + return ( + types.isMemberExpression(node, {computed: false}) && + types.isIdentifier(node.object, {name: 'module'}) && + types.isIdentifier(node.property, {name: 'exports'}) + ); +} + +function isTruthyConstant(node: BabelNode): boolean { + if (types.isBooleanLiteral(node)) { + return node.value === true; + } + if (types.isNumericLiteral(node)) { + return node.value !== 0; + } + // `!0` (minified `true`) + if (types.isUnaryExpression(node, {operator: '!', prefix: true})) { + return types.isNumericLiteral(node.argument, {value: 0}); + } + return false; +} + +// `Object.defineProperty(exports, "__esModule", { value: })` +function isDefinePropertyESModule(call: BabelNodeCallExpression): boolean { + const callee = call.callee; + if ( + !types.isMemberExpression(callee, {computed: false}) || + !types.isIdentifier(callee.object, {name: 'Object'}) || + !types.isIdentifier(callee.property, {name: 'defineProperty'}) + ) { + return false; + } + const args = call.arguments; + if ( + args.length < 3 || + !isExportsObject(args[0]) || + !types.isStringLiteral(args[1], {value: '__esModule'}) || + !types.isObjectExpression(args[2]) + ) { + return false; + } + return args[2].properties.some( + prop => + types.isObjectProperty(prop, {computed: false}) && + (types.isIdentifier(prop.key, {name: 'value'}) || + types.isStringLiteral(prop.key, {value: 'value'})) && + isTruthyConstant(prop.value), + ); +} + +function expressionSetsESModule(expr: BabelNode): boolean { + if (types.isSequenceExpression(expr)) { + // `a, b, c` at the top level - each subexpression is independently + // observable. Recognise the marker in any position, so patterns like + // `module.exports = fn, module.exports.__esModule = true, ...` from + // `@babel/runtime/helpers/*` are detected. + return expr.expressions.some(expressionSetsESModule); + } + if (types.isCallExpression(expr)) { + return isDefinePropertyESModule(expr); + } + if (!types.isAssignmentExpression(expr) || expr.operator !== '=') { + return false; + } + const left = expr.left; + return ( + types.isMemberExpression(left) && + left.computed !== true && + types.isIdentifier(left.property, {name: '__esModule'}) && + isExportsObject(left.object) && + isTruthyConstant(expr.right) + ); +} + +/** + * Returns whether the given (post-transform) module AST declares ESM/CJS + * interop by setting `exports.__esModule` truthy at the top level. Recognises + * four shapes seen in the wild: + * + * Object.defineProperty(exports, '__esModule', {value: true}) + * - Metro's own ESM transform (`import-export-plugin`) + * - `@babel/plugin-transform-modules-commonjs` (default output) + * - typescript compiler `--module commonjs` + * - rollup with `esModule: true` (default) + * Object.defineProperty(module.exports, '__esModule', {value: true}) + * - handwritten interop wrappers + * exports.__esModule = true // also value 1 or !0 + * - `@babel/plugin-transform-modules-commonjs` with `loose: true` + * - some older tsc output + * - rollup with `esModule: 'if-default-prop'` + * module.exports = fn, module.exports.__esModule = true, ... + * - every helper under `@babel/runtime/helpers/` (sequence expression) + * + * AST-based (not a scan of generated code) so the check is robust to + * whitespace, quoting, and attribute ordering. Intentionally independent of + * whether the import-export-plugin ran: a module already lowered to CJS with + * the marker must still be recognised as an ES module, so this cannot be + * replaced by the plugin's `out.isESModule`. + */ +export function definesESModuleInterop(ast: BabelNodeFile): boolean { + for (const stmt of ast.program.body) { + if ( + types.isExpressionStatement(stmt) && + expressionSetsESModule(stmt.expression) + ) { + return true; + } + } + return false; +} + +// Collects the `exports`/`module` identifier nodes that belong to an export +// write we can fully account for: `module.exports = `, +// `exports. = ` and `module.exports. = `. Any +// occurrence left uncollected is treated as an escape by the caller. +function collectAccountedExportsRefs( + ast: BabelNodeFile, +): Set { + const accounted = new Set(); + + const accountForExportsObject = (node: BabelNode): boolean => { + // `exports` + if (types.isIdentifier(node, {name: 'exports'})) { + accounted.add(node); + return true; + } + // `module.exports` - both identifiers are occurrences of the names we + // track, so both have to be accounted for. Bound to locals so the + // refinements survive the calls that establish them. + if (types.isMemberExpression(node, {computed: false})) { + const object = node.object; + const property = node.property; + if ( + types.isIdentifier(object, {name: 'module'}) && + types.isIdentifier(property, {name: 'exports'}) + ) { + accounted.add(object); + accounted.add(property); + return true; + } + } + return false; + }; + + for (const stmt of ast.program.body) { + if (!types.isExpressionStatement(stmt)) { + continue; + } + const expr = stmt.expression; + if (!types.isAssignmentExpression(expr) || expr.operator !== '=') { + continue; + } + const left = expr.left; + // `module.exports = ` + if (accountForExportsObject(left)) { + continue; + } + // `exports. = ` / `module.exports. = `. A + // computed key is rejected by `hasDynamicPropertyDefinition`. + if (types.isMemberExpression(left, {computed: false})) { + accountForExportsObject(left.object); + } + } + + return accounted; +} + +// Any construct that can define a property whose key is not visible in the +// source text. With the `__esModule` token absent, these are the only +// remaining ways to produce the key (e.g. `exports['__' + 'esModule']`). +function hasDynamicPropertyDefinition(ast: BabelNodeFile): boolean { + let found = false; + types.traverseFast(ast, node => { + if (found) { + return; + } + if ( + // `x[k] = v` + (types.isAssignmentExpression(node) && + types.isMemberExpression(node.left, {computed: true})) || + // `{[k]: v}` + (types.isObjectProperty(node) && node.computed === true) || + // `{...x}` - `x` may carry the key + types.isSpreadElement(node) || + // `Object.assign(target, ...)`, `Object.defineProperties(...)` + (types.isCallExpression(node) && + types.isMemberExpression(node.callee, {computed: false}) && + types.isIdentifier(node.callee.object, {name: 'Object'}) && + (types.isIdentifier(node.callee.property, {name: 'assign'}) || + types.isIdentifier(node.callee.property, { + name: 'defineProperties', + }))) + ) { + found = true; + } + }); + return found; +} + +/** + * Returns whether the module *might* define `exports.__esModule`, i.e. whether + * `definesESModuleInterop` returning false could be a false negative. + * + * `definesESModuleInterop` only inspects top-level statements, so on its own it + * cannot distinguish "no marker" from "marker installed somewhere it can't + * see". A self-contained bundle, for instance, may hand its exports object to a + * helper that sets the key (webpack's `__webpack_require__.r`), which is + * invisible to a statement scan and needs no dependencies to do it. + * + * Returning false is an assertion that no expression in the module can produce + * the key, established by checking that all three hold: + * + * 1. `__esModule` does not occur anywhere, as an identifier or string. + * 2. `exports`/`module` are only ever read as the target of an export write + * we can enumerate - never aliased, passed to a function, or accessed + * with a computed key, any of which would let the key be set out of view. + * 3. No construct can define a property under a key that is not literally + * present in the source (see `hasDynamicPropertyDefinition`), which is + * what closes the gap left by (1). + * + * Deliberately conservative: anything unrecognised returns true. This is not a + * claim that the module is CommonJS - a script or an empty module also + * qualifies - only that it does not opt into ESM interop. + */ +export function canDefineESModuleInterop(ast: BabelNodeFile): boolean { + if (hasDynamicPropertyDefinition(ast)) { + return true; + } + const accounted = collectAccountedExportsRefs(ast); + let unsafe = false; + types.traverseFast(ast, node => { + if (unsafe) { + return; + } + if ( + types.isIdentifier(node, {name: '__esModule'}) || + types.isStringLiteral(node, {value: '__esModule'}) + ) { + unsafe = true; + return; + } + if ( + (types.isIdentifier(node, {name: 'exports'}) || + types.isIdentifier(node, {name: 'module'})) && + !accounted.has(node) + ) { + unsafe = true; + } + }); + return unsafe; +} From f04d597f9e28521bf4bc5ee7d404a5cd9dbcba29 Mon Sep 17 00:00:00 2001 From: Vitali Zaidman Date: Wed, 19 Aug 2026 02:34:58 -0700 Subject: [PATCH 2/2] ESM live bindings: rewrite the experiment runbook against the current stack MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Summary: Replaces the stale runbook from the superseded stack (D111149711, which described D111148330's shape) with one written against the diffs that are actually going to land, and documents two things we didn't know when it was first written. What changed vs the old runbook: - **Stack map is current.** D111629268 → D111629272 → D111629271 → D111529091, with the Hermes prototype (D111629267) explicitly parked as a side branch and the superseded diffs listed as do-not-land. - **New section: how the two halves compose.** `require(dep).default` is a fresh property read on every evaluation, so for ESM-detected dependencies the serialize-time rewrite is *inherently live* **and** `CallRequire`-eligible — it supersedes the live `metroImportDefault` helper for exactly those sites, while CJS/unknown dependencies still rely on the live helper from D111529091. Both paths are needed; the size/TTI numbers must be taken on the combined stack. - **New section: ordering / caching.** Records why `inline-requires` (transform time) cannot defeat the rewrite (serialize time), and that `unstable_liveBindings` is in the transform cache key and graph id, so control and treatment artifacts can't be confused. - **New: dev bundles are not rewritten.** `keepRequireNames: options.dev` makes `collectDependencies` emit `_$$_IMPORT_DEFAULT(depMap[k], './x')`, and the rewrite deliberately matches only the single-argument form. This is fine (the rewrite targets optimized bundles) but it means the rewrite cannot be validated in a dev bundle — worth knowing before someone tries. - **New: the transform is not free when the flag is OFF.** The live `metroImportDefault` branch and its prelude-global read ship in `metro-runtime`'s `require.js` polyfill for every bundle regardless of the flag: ~+320 bytes JS on the metro-buck e2e fixture. The old runbook's "zero behaviour change when disabled" framing was wrong. - **Experiment plan is executable.** Explicitly turns `unstable_staticHermesOptimizedRequire` on in *both* arms so the only delta is `unstable_liveBindings`, and adds a step to verify the rewrite actually fired (helper count vs `).default` count) before trusting any measurement. - **Follow-ups list pruned** of items already addressed by D111529091's getter-based re-export forwarding and by D111568389. Docs only, no code. Differential Revision: D114885628 --- ...es-esm-live-bindings-experiment-runbook.md | 244 ++++++++++++++++++ 1 file changed, 244 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-08-hermes-esm-live-bindings-experiment-runbook.md diff --git a/docs/superpowers/specs/2026-07-08-hermes-esm-live-bindings-experiment-runbook.md b/docs/superpowers/specs/2026-07-08-hermes-esm-live-bindings-experiment-runbook.md new file mode 100644 index 0000000000..f66995f0a2 --- /dev/null +++ b/docs/superpowers/specs/2026-07-08-hermes-esm-live-bindings-experiment-runbook.md @@ -0,0 +1,244 @@ +# ESM Live Bindings — Transform & OTA Experiment Runbook + +**Status (2026-08-03):** transform implemented behind a flag; link-time default-import +rewrite implemented and unit-tested; benchmarks and correctness proven. Awaiting +publish + review, then the OTA experiment. + +**Current draft stack** (all `Unpublished`, in dependency order): + +| Diff | What it does | +|---|---| +| D111629268 | `definesESModuleInterop` detector; positive-only `out.isESModule` hint; `unstable_isESModule` threaded to buck | +| D111629272 | Serialize-time rewrite of ESM default imports to `require(dep).default` (+ unit tests) | +| D111629271 | Gates that rewrite on `unstable_staticHermesOptimizedRequire` | +| D111529091 | Opt-in `unstable_liveBindings` transform + live `metroImportDefault` runtime | + +Side branch, **not** on the OTA critical path (see "Do we need the Hermes change?"): + +| Diff | What it does | +|---|---| +| D111629267 | `[hermes][prototype]` `CallRequireImportDefault` opcode + `moduleImportedDefaults_` cache | + +Superseded (do not land): D111143093 → D111148330 (earlier transform, folded into +D111529091), D111629273 (abandoned; its content is folded into D111629268). +D111568389 (`export *` must not re-export `default`) is standalone. + +## What the two halves do + +### Live bindings (D111529091) + +`import-export-plugin` gains a `liveBindings` option, gated behind the +`unstable_liveBindings` custom transform option. When enabled: + +- **Exports (write side):** every reassignment of an exported binding re-publishes + the new value onto `exports` as a plain data-property write (`x = v` → + `exports.x = (x = v)`; `x++` → `(x++, exports.x = x, x)`). Implemented via each + binding's `constantViolations` after a scope crawl, so a shadowing local in a + nested scope is correctly left alone. +- **Re-export forwarding** (`export {x} from`, `export * from`) installs getters, + because the source module can reassign the binding and local assignment tracking + cannot observe that. A module's *own* exports keep the fast data-property path. +- **Imports (read side):** default imports keep the `_$$_IMPORT_DEFAULT` call + shape; liveness comes from the runtime helper caching the source module's + *exports object* rather than the resolved default value, then re-reading + `.default` on each subsequent read. + +Semantics: **live value, relaxed TDZ** — a read before the source module finishes +initialising returns `undefined` rather than throwing. + +### Link-time default-import rewrite (D111629268 → D111629272 → D111629271) + +At serialize time, inside `inlineModuleIds`, default imports of dependencies known +to be ES modules are rewritten: + +```js +_$$_IMPORT_DEFAULT(depMap[0]) → require(depMap[0]).default +``` + +so the read lowers to the Static Hermes `CallRequire` fast path (native +per-`RuntimeModule` export cache) instead of the JS `metroImportDefault` helper. +CJS/unknown targets keep the interop helper, because their default is the whole +`module.exports`, not `.default`. + +The rewrite is padded to the original length so byte offsets and source maps are +unaffected, and it leaves the dependency-map reference intact so the subsequent id +inlining still fires (keeping the resulting `require()` +`CallRequire`-eligible). + +## How the two compose (this is the point) + +`require(dep).default` is a **fresh property read on every evaluation**, so for +ESM-detected targets the rewrite is *inherently live* — and it rides +`CallRequire`. That means: + +- For ESM dependencies, the rewrite gives you liveness **and** the fast path, with + no interop helper. The live-`metroImportDefault` work in D111529091 is redundant + for exactly these sites. +- For CJS/unknown dependencies (not rewritten), D111529091's live + `metroImportDefault` is still what provides default-import liveness. Both paths + are needed. +- Because the rewrite removes helper calls that would otherwise pay the live + helper's cost, it **pays back part of** live bindings' +3.2% bytecode. The two + should be measured together, not independently. + +### Ordering / caching — verified safe + +`inline-requires` runs at **transform** time; the rewrite runs at **serialize** +time on already-cached transform output. So the rewrite cannot be defeated by +use-site memoization, and it cannot poison the transform cache. +`unstable_liveBindings` *is* part of the transform cache key and graph id +(`getGraphId.js`, `transformHelpers.js`, `Transformer.js`), so control and +treatment artifacts cannot be confused. `unstable_esmDefaultImportRewrite` is +derived at bundle time from `unstable_staticHermesOptimizedRequire` and is +deliberately *not* a transform-cache input — it only affects serialization. + +### Known limitation: dev bundles are not rewritten + +In dev builds `keepRequireNames` is on (`keepRequireNames: options.dev`), so +`collectDependencies` appends a debug name: + +```js +_$$_IMPORT_DEFAULT(depMap[0], "./x") +``` + +The rewrite matches only the **single-argument** form, so dev bundles keep the +interop helper. This is intentional — the rewrite targets optimized/production +bundles, which is also the only place `unstable_staticHermesOptimizedRequire` +applies — but it means **you cannot validate the rewrite in a dev bundle.** Pinned +by a test (`does NOT rewrite when a dev-only debug name argument is present`). + +Also note the replacement is always exactly 3 bytes shorter than the helper call +for any dependency-map name and uid suffix, so the "would grow → skip" branch is +unreachable in practice; coverage does not silently drop to zero. + +## Interaction with inline-requires (lazy evaluation) — critical for startup/TTI + +A startup/TTI comparison must credit the inlined baseline for lazy evaluation. +Making bindings live does **not** hoist `require()` to module init: + +```js +// snapshot + inline (today's inlined baseline): +var x; +function use() { return (x || (x = _$$_REQUIRE(dep).x)) + x; } // lazy, memoized VALUE (snapshot) + +// live + inline: +var _dep; +function use() { return (_dep || (_dep = _$$_REQUIRE(dep))).x + _dep.x; } // lazy, memoized MODULE, LIVE member +``` + +The `require()` call sits in the identical position inside the identical lazy +memoization guard in both, so the live variant evaluates the **same module +factories at the same time** as the snapshot baseline. The residual cost is +per-read (a member load vs a cached local) and per-write (republish), not extra +module evaluation. + +Corollary (a *win* for live): for modules with **multiple** named imports, the +snapshot path destructures eagerly at module top, requiring `dep` when the +importer's factory runs. The live path keeps member loads at use sites, so +inline-requires can defer them — live is *lazier* for multi-import modules. + +## Turning it on for a build + +- **Metro CLI / `react-native bundle`:** `--transform-option unstable_liveBindings=true` +- **Programmatic:** `customTransformOptions: {unstable_liveBindings: true}` +- **Metro config:** return it from `transformer.getTransformOptions` + +The rewrite needs no separate switch: it turns on with +`unstable_staticHermesOptimizedRequire`. + +## OTA / Buck-modifier experiment plan + +Goal: ship two OTA bundles of the same app revision and compare HBC size and +startup/TTI on device. + +1. **Pre-reqs.** The target must already build with `experimentalImportSupport` + (live bindings are a no-op otherwise). Turn + `unstable_staticHermesOptimizedRequire` on in **both** arms, so the only delta + between control and treatment is `unstable_liveBindings` — otherwise you are + measuring two changes at once. +2. **Build variants via a Buck modifier** (e.g. `//buck/modifiers:metro_live_bindings`) + that appends `--transform-option unstable_liveBindings=true` to the Metro bundle + action. No source change, so the arms are otherwise identical. +3. **Offline proxy first (fast, deterministic):** per variant, compile to HBC + (`hermes -O -emit-binary -out out.hbc bundle.js`), record `wc -c out.hbc`, then + replay a TTI-marker trace through `hermes_synth` + (`buck run @xplat/mode/hermes/opt hermes_synth -- trace.json out.hbc -marker=`). +4. **Confirm the rewrite actually fired** on the treatment bundle before trusting + any number: grep the optimized bundle for `_$$_IMPORT_DEFAULT` and for + `).default`, and record the ratio. If the helper count is unchanged, the rewrite + did not apply (most likely a dev-mode bundle, or ESM detection returned nothing) + and the experiment is invalid. +5. **On-device confirmation:** the relevant MobileLab TTI test (e.g. + `fb4a.marketplace_cold_start`, metric `hermesTime`) or a QPL-instrumented cold + start, control vs treatment. + +## Measured cost of the transform (2000-module synthetic ESM graph) + +Live bindings only, **without** the default-import rewrite. `hermesc -O -emit-binary`: + +| Bundle | JS bytes | HBC bytes | +|---|---|---| +| snapshot (flag off = today's Metro) | 1,172,027 | 877,541 | +| live (flag on) | 1,200,689 | 905,661 | +| **delta** | **+28,662 (+2.4%)** | **+28,120 (+3.2%)** | + +**Whole-bundle eval time** (real Hermes VM, 25 runs, median, against an +empty-loader baseline): + +| Bundle | total (ms) | graph eval (ms, minus baseline) | +|---|---|---| +| empty loader | 24.90 | — | +| snapshot | 35.29 | 10.40 | +| live | 35.88 | 10.98 | + +Live adds **+0.59 ms (+5.6%)** of graph-eval time for 2000 modules. + +**Heap** (`-gc-print-stats`): total allocated 1,316,808 → 1,348,808 (**+32 KB, ++2.4%**); peak RSS effectively identical (~26 MB, VM-dominated). + +These numbers are an upper-ish bound (every module has a reassigned export) and +predate the rewrite, which should claw back some of the bytecode delta. **Re-measure +the combined stack before quoting a number for the OTA decision.** + +### Unconditional cost when the flag is OFF + +D111529091 is *not* byte-for-byte free at the bundle level, even though the +transform output is unchanged. The live-`metroImportDefault` branch and its +prelude-global read ship in `metro-runtime`'s `require.js` polyfill for **every** +bundle, flag on or off: **~+320 bytes JS / ~+170 bytes** on the metro-buck e2e +fixture bundle. Snapshots updated accordingly. If that matters, the runtime branch +would need to be stripped at build time rather than gated at runtime. + +## Expected impact (from the benchmark suite, D111143093) + +- **Reads:** in the inlined regime live is ~2.3× faster than the `require().x` + baseline; in the non-inlined regime it costs ~+30–40% over the bare-local snapshot. +- **Bytecode size:** the live shape is ~1.8× the non-live bare-local per module, but + smaller than the inlined baseline and ~22% smaller than a Babel-getter design. +- **Startup/init:** the live shape is the cheapest live option to construct, ~3× + cheaper than Babel getters. + +## Do we need the Hermes change (D111629267)? + +Probably **not for this experiment.** Once default imports of ES modules are +rewritten to `require(dep).default`, they already lower to plain `CallRequire` and +hit the existing `moduleExports_` cache in master. A dedicated +`CallRequireImportDefault` opcode would only help the *CJS/unknown* sites that keep +the interop helper. + +Keeping it off the critical path also removes a Hermes **runtime** change from an +OTA experiment, which is much cheaper to ship. The prototype is therefore parked as +a side branch. Before it could land on its own it needs: a real build + test run +(its test plan is still "TODO"), plus JIT (`lib/VM/JIT/arm64/JitEmitter.cpp`) and +SH-native (`lib/BCGen/SH/SH.cpp`, `_sh_ljs_callRequire`) paths — it currently +covers only the interpreter and HBC ISel, so as written it is a bytecode-only +optimization. + +## Remaining follow-ups + +- Publish all four metro diffs and get reviewers (all are `Unpublished`, blocker + `revision_not_accepted`); D111529091 still needs a test plan. +- Namespace imports (`metroImportAll`) remain non-live — separate change. +- Build the Buck modifier and run steps 3–5 above. +- Re-measure size/TTI for the **combined** stack (rewrite + live bindings). +- Decide whether the unconditional runtime cost above is acceptable.