Target Node 24, remove all runtime dependencies, 2.0.0 - #28
Target Node 24, remove all runtime dependencies, 2.0.0#28expo-tuft[bot] wants to merge 4 commits into
Conversation
Hyperinstall's dependencies had aged into a stream of Dependabot PRs
while the program itself is a few hundred lines that Node can now run on
its own. This drops every runtime dependency and the Babel build step.
- engines: Node >=24; the source is plain ESM that runs directly, so
there is no lib/ build and "npm publish" ships src/
- instapromise -> node:fs/promises
- co -> async/await
- rimraf -> fs.rm({ recursive, force })
- lodash -> language built-ins and util.isDeepStrictEqual
- @exponent/promise-props, await-lock -> a few lines of local code
- yargs, minimist (unused) -> util.parseArgs with hand-written usage
- npm-package-arg -> src/localDeps.js, which answers the one question
hyperinstall asked it: is this specifier a local path?
- fstream-npm (deprecated) -> src/checksum.js, which hashes a local
dependency's files directly, skipping node_modules and VCS metadata
- @babel/*, eslint-config-universe -> eslint 9 flat config + prettier 3
Dev dependencies are now eslint, @eslint/js, and prettier.
Also adds a test suite (node:test, no test framework dependency) and a
CI workflow running lint, format, and tests on Node 24 and 26.
Behavior notes:
- Local "file:" dependency checksums now cover every file in the
dependency except node_modules and VCS metadata, instead of the exact
set of files npm publish would include. This is a superset, so it
never misses a change but may trigger one extra install.
- The generated npm-hyperinstall script now quotes "$@" and $ROOT, and
is chmod'ed 0o755 so the process umask cannot leave it unexecutable.
- hyperinstall clean no longer fails when the state file or the script
is already gone.
Co-authored-by: ide <379606+ide@users.noreply.github.com>
ESM-only and Node >=24 are breaking changes. Co-authored-by: ide <379606+ide@users.noreply.github.com>
The first pass at replacing fstream-npm hashed every file under a local dependency except node_modules and VCS metadata. That is a superset of what npm publishes, so it never misses a change, but it also hashes build output, caches, and logs. Measured against a react-native checkout, one local dependency of ~59k files and 14 GB took 20s to checksum -- hyperinstall's entire point is that an up-to-date run costs a fraction of a second. Files are now selected the way a person thinks about a package: - "git ls-files --cached --others --exclude-standard" lists the files Git knows about, skipping anything ignored. These are hashed by content, so switching branches back and forth doesn't look like a change. - Files that the "files" field of package.json publishes but Git ignores -- how packages ship build output that isn't checked in -- are hashed by size and modification time, which detects a rebuild without reading gigabytes of artifacts. - Outside a Git work tree, or when the package itself is ignored, every file is read as before, since there is nothing to tell source apart from build output. Reads and stats are bounded to 8 at a time, and entries that Git lists but that can't be read (deleted-but-tracked files, submodules) are skipped; because file names are hashed alongside their digests, their disappearance is itself a change. Checksum times for local dependencies, cold cache: spawn-async 12 files 4ms -> 20ms (git adds a ~14ms spawn) expo-modules-core 1031 files 128ms -> 73ms expo-router 2906 files 333ms -> 145ms react-native 58686 files 20159ms -> 936ms Co-authored-by: ide <379606+ide@users.noreply.github.com>
|
Bumped to 2.0.0, and reworked how local The problem. Replacing It also means routine, meaningless churn (a rebuild, a stray The fix. Select files the way a person thinks about a package:
Reads and stats are bounded to 8 at a time. Entries Git lists but that can't be read (deleted-but-tracked files, submodule gitlinks) are skipped; since file names are hashed alongside their digests, a file disappearing is itself a change. Cold-cache checksum of one local dependency:
The 16ms floor is the Known limits, all of which the 5 more tests cover the Git paths: ignored files excluded, tracked and untracked changes detected, published-but-ignored build output detected (including a same-mtime-different-size case via Also verified end to end on a real project: a gitignored |
|
One clarification on the table in my previous comment: "before" there is the intermediate walk-everything commit in this PR, not
So The rework also fixes a blind spot in the original that isn't about speed. It hashed the per-file checksums sorted, with the file names discarded ( A rename in a local dependency therefore never triggered a reinstall. This PR hashes Two smaller things the original did that the rework drops: every file in the package was read concurrently with no bound (EMFILE risk on large packages — the new code caps at 8), and the |
|
Following up on why this doesn't use npm's current implementation, since "just use what npm uses" is the obvious question. npm replaced You can sidestep Arborist by passing a stand-in — The blocker is speed.
30 seconds to enumerate 4,508 files, repeatable across runs, before hashing a single byte — because that package's working tree holds ~59k files of build output that So: 10 more packages to install and 30x slower on the case that matters, to compute a file list that |
Covers the fallback when the git binary is missing entirely, not just when the package is outside a work tree: changes are still detected, and ignored files count toward the checksum rather than being missed. Co-authored-by: ide <379606+ide@users.noreply.github.com>
Hyperinstall's dependency list had aged into a steady stream of Dependabot PRs (
yargs@6,npm-package-arg@4,rimraf@2,fstream-npm@1,instapromise@latest,co) while the program itself is a few hundred lines that Node can now run unassisted. This targets Node 24 and drops every runtime dependency plus the Babel build step.package.json
engines.node:>=16 <=16→>=24"type": "module"— the source is plain ESM that Node runs directly, so there's nolib/build andnpm publishshipssrc/dependencies: 10 → 0devDependencies: 6 → 3 (eslint,@eslint/js,prettier)yarn.lock: 3,213 → 583 linesexponent/hyperinstallrepository/bugs URLsinstapromisenode:fs/promisescoasync/awaitrimraf@2fs.rm({ recursive, force })lodashutil.isDeepStrictEqual@exponent/promise-props,await-locksrc/Lock.js)yargs@6,minimist(unused)util.parseArgs+ hand-written usagenpm-package-arg@4src/localDeps.js— answers the one question hyperinstall asked it: is this specifier a local path?fstream-npm@1(deprecated)src/checksum.js— see below@babel/cli,@babel/core,@babel/preset-enveslint-config-universe@10(pulls in the TypeScript and React plugin trees)js.configs.recommendedHashing local dependencies
fstream-npmwas used for one thing: list a localfile:dependency's files so their contents can be checksummed and a change can trigger a reinstall. Its modern equivalent,npm-packlist, is driven from an@npmcli/arboristtree; it can be called with a stand-in for that tree, but it installs 10 packages and itsignore-walkdescends the whole working tree — 30s to list react-native's 4,508 publishable files, versus 68ms forgit ls-files(measured in a comment below).src/checksum.jsinstead selects files the way a person thinks about a package:git ls-files --cached --others --exclude-standard— the files Git knows about, ignored ones excluded. Hashed by content, so flipping between branches and back doesn't look like a change.filesfield of package.json publishes but Git ignores — how a package ships build output that isn't checked in — hashed by size + mtime, which catches a rebuild without reading gigabytes of artifacts.node_modulesand VCS metadata. Nothing there distinguishes source from build output.Reads and stats are bounded to 8 at a time. File names are hashed alongside their digests, so renames and deletions register as changes.
Cold-cache checksum of one local dependency, versus hashing every file in it:
Known limits, all shared with the
fstream-npmversion: symlinked files aren't followed, file modes aren't hashed, submodule contents aren't descended into, and local dependencies of local dependencies aren't followed transitively.Tests and CI
There were no tests. Added 34 covering the install decision logic (first install, no-op re-run, dependency/lockfile/cache-breaker changes, missing
node_modules, localfile:dep changes,--force/--clean, state pruning), the checksum file selection in and out of a Git work tree, local-specifier detection, and the CLI end to end. They usenode:test, so no test-framework dependency. CI runs lint, format check, and tests on Node 24 and 26.Beyond the suite, verified against a real project:
init, first install (yarn for the package with ayarn.lock, npm for the other), a 50ms no-op re-run, a gitignored log in a local dependency causing no reinstall while a change to its publishedbuild/output and to itssrc/each caused exactly one, the generatednpm-hyperinstallscript, andclean.Other behavior notes
npm-hyperinstallscript now quotes"$@"and$ROOT, and ischmoded 0755 so a restrictive umask can't leave it non-executable.hyperinstall cleanno longer fails when the state file or script is already gone.--clean(previously described as part of--force) and the Node 24 prerequisite.