Skip to content

bench(lint): with prefer-direct-reexport rule - #34

Open
mainframev wants to merge 3 commits into
bench/basefrom
bench/lint-with-rule
Open

bench(lint): with prefer-direct-reexport rule#34
mainframev wants to merge 3 commits into
bench/basefrom
bench/lint-with-rule

Conversation

@mainframev

Copy link
Copy Markdown
Owner

Purpose

With-rule side for measuring the lint performance cost of the prefer-direct-reexport ESLint rule.

Baseline to compare against: #33 (bench/lint-baseline).

This branch is #33 plus exactly 4 files and nothing else:

packages/eslint-plugin/src/internal.js             |    1 +   <- rule enabled as 'error'
tools/eslint-rules/index.ts                        |    2 +   <- registration
tools/eslint-rules/rules/prefer-direct-reexport.ts |  433 +   <- the rule
tools/eslint-rules/rules/prefer-direct-reexport.spec.ts | 1098 +

The benchmark CI job is the same commit object as in #33, so the measurement harness is not a variable.

Method

yarn nx run-many -t lint --all --parallel=4 --skipNxCache --excludeTaskDependencies
Flag Why
--all identical project set on both sides; nx affected would skew asymmetrically
--skipNxCache lint is cache: true; otherwise we would time cache replays
--excludeTaskDependencies nx.json sets lint.dependsOn: ["build"]; keeps build time out of the measurement
no --nxBail this side has known violations; bailing would abort early and under-report time

Run 3× per PR; compare the fastest run of each.

A second, runner-independent measurement runs TIMING=25 eslint src on react-components/react-button/library and dumps ESLint's per-rule profiler table into the job summary. Since the config uses projectService: true (type-aware linting), TypeScript program construction likely dominates total lint time — so this table, not the wall-clock delta, is the more reliable attribution of the rule's own cost.

Expected

The lint job exits non-zero here: the rule is enabled as error and the codebase has ~264 pre-existing violations that have never been cleaned up. That is intentional and does not affect timing (no --nxBail; every project is linted on every run). Note that formatting/serialising those violations is itself work the baseline does not do — a small bias in favour of the baseline, inherent to comparing against plain master.

Throwaway branch; not intended to merge.

@mainframev
mainframev force-pushed the bench/lint-with-rule branch from 2222742 to 55fc7dd Compare August 16, 2026 03:52
@mainframev

Copy link
Copy Markdown
Owner Author

Results — lint performance delta is below the noise floor

Full workspace lint, 247 projects, uncached, --parallel=4, ubuntu-latest, 3 runs each.

Run 1 Run 2 Run 3 Min Median Mean Failed tasks
Baseline (#33) 1169s 1148s 1148s 1148s 1148s 1155.0s 0
With rule (#34) 1156s 1149s 1161s 1149s 1156s 1155.3s 19
Delta +1s (+0.09%) +8s (+0.70%) +0.3s (+0.03%)

The spread within the baseline alone (1148→1169 = 21s, 1.8%) is larger than the difference between the two PRs, so the wall-clock delta is not distinguishable from runner noise.

Per-rule attribution (the reliable number)

Wall-clock on shared runners is noisy, so the rule was also profiled directly with ESLint's own profiler on react-components/react-button/library:

Rule                                          | Time (ms) | Relative
:---------------------------------------------|----------:|--------:
@typescript-eslint/no-deprecated              |   570.837 |    54.7%
@nx/workspace-base-hook-no-forbidden-runtime  |    25.372 |     2.4%
@nx/workspace-prefer-direct-reexport          |     0.396 |     0.0%

prefer-direct-reexport costs 0.396 ms — roughly 64× cheaper than the existing sibling custom rule base-hook-no-forbidden-runtime, and ~1400× cheaper than @typescript-eslint/no-deprecated, which alone accounts for over half of all rule time. In CI it did not even reach the top 25 (cutoff 4.6 ms).

This is expected: the rule is purely syntactic + scope-based. It does no type resolution, which is what makes the type-aware rules expensive.

Conclusion

The rule's lint cost is negligible. Total lint time is dominated by TypeScript program construction (projectService: true) and type-aware rules; a syntactic rule is lost in the noise.

Methodology note — first run was invalid, and was discarded

The initial attempt used --excludeTaskDependencies without pre-building, so 213 of 247 projects died instantly with Cannot find module '@fluentui/eslint-plugin-react-components' (required by packages/eslint-plugin/src/configs/react/config.js:11) — including every v9 package the rule actually applies to. That run appeared to show a ~3% delta; it was measuring config-resolution failures, not linting.

Fixed by adding an untimed nx run-many -t build -p eslint-plugin-react-components step before the timed section. The corrected runs report failed_tasks=0 on the baseline, confirming all 247 projects genuinely linted.

Caveats

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant