Skip to content

feat(cli): use upstream @dfinity/pic, deprecate pocket-ic below 9.0.0 - #642

Merged
Kamirus merged 5 commits into
mainfrom
kamil-claude/mops-cli-major-update-f3fccd
Aug 4, 2026
Merged

feat(cli): use upstream @dfinity/pic, deprecate pocket-ic below 9.0.0#642
Kamirus merged 5 commits into
mainfrom
kamil-claude/mops-cli-major-update-f3fccd

Conversation

@Kamirus

@Kamirus Kamirus commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Closes #561.

pocket-ic 9.0.0 and newer now runs through upstream @dfinity/pic 0.23.0. Nothing changes for anyone on a supported pin: the same toolchain-managed binary starts with the same --ttl, and installing the CLI still downloads no extra binary. Pins below 9.0.0 keep working but warn, and stop working in v3.

Why replace the fork

pic-js-mops has no public source repository. It is published to npm from a local checkout of dfinity/pic-js carrying four patches, so the only way to audit it is diffing the published tarball against upstream, and exactly one person can publish a fix.

Two of those patches are now upstream and shipped together in 0.23.0: dfinity/pic-js#276 for an explicit binary path and dfinity/pic-js#278 for a server --ttl. The third needs no upstream change — serverProcess is TS-private but present at runtime, so the client seam casts through it for canister-log streaming. The fourth was a stripped postinstall, handled below.

Why it is a devDependency, not a dependency

@dfinity/pic downloads a ~94 MB pocket-ic binary in a postinstall that throws when it cannot. The published ic-mops package is the unbundled dist/ tree carrying the full dependency list — the bun bundle is a separate artifact that only cli.mops.one installs — so a dependencies entry reaches every user. Installing into a throwaway prefix with egress blocked:

# @dfinity/pic in dependencies
npm error TypeError: fetch failed … connect EPERM 140.82.121.3:443
npm error     at downloadPicBinary (postinstall.mjs:69)
exit 1

# pre-bundled devDependency
added 368 packages in 3s
exit 0

mops manages that binary itself via [toolchain] pocket-ic, so the download was never wanted. Three pieces replace it: vendor/pic-entry.mjs re-exports PocketIc and PocketIcServer explicitly, because pic ships CJS and a plain export * through esbuild yields no named exports at all; vendor:pic bundles that to dist/vendor/pic.mjs (1.1 MB); and fix-dist repoints the dynamic import, throwing if it is ever renamed. import("pic-ic") is deliberately left alone.

Removable later — if pic makes the download lazy, as suggested during review of #276, all three pieces and their smoke test go away.

Deprecating pins below 9.0.0

Before, a pocket-ic = "4.0.0" pin ran with no notice. After:

$ mops test --mode replica
`pocket-ic` is pinned to 4.0.0 in `[toolchain]`. Support for `pocket-ic` below 9.0.0 is deprecated and will be removed in mops v3.
Run `mops toolchain use pocket-ic 12.0.0` to move to a supported version and silence this warning.
 PASS
Tests passed
Done in 3.06s, passed 1

The warning fires where the legacy client is selected rather than per command, so it cannot drift from the selection logic and prints once per run.

One subtlety worth calling out: addCycles takes a number on pic-ic and a bigint upstream. The intersection cast that reconciled the two old clients would still compile here and then hand a number to the modern client at runtime, so the seam exports a wrapper that branches on the same pin that picked the client.

Migration

Bump the pin: mops toolchain use pocket-ic 12.0.0.

The affected set is narrower than the warning implies. pic-ic@0.5.4 only speaks the 4.0.0 server protocol, so 5.x–8.x pins already fail with BinTimeoutError on main today — only 4.0.0 and 9.x+ ever worked, matching the range documented in #412. In practice this affects 4.0.0 pins.

@icp-sdk/core stays on 4.0.2

pic depends on @icp-sdk/core ^5, but as a plain dependency, not a peer — so it does not dictate mops's version. mops stays on 4.0.2 and the vendored bundle carries pic's own 5.x inside it.

That pin is load-bearing, not incidental. @icp-sdk/core 5.x drops the IC HTTP API v2 endpoints, and v2 is the only version the dfx and dfx-pocket-ic replicas serve — dfx 0.29.1 answers /api/v4/…/call with a 400 and an HTML error page, and 404s v3 query/read_state. The SDK's fallback keys on 404 for v4 only, so it never fires, and query/read_state have no fallback at all. Bumping mops to 5.x therefore breaks every replica test and benchmark in a project with no pocket-ic pin, which is how the unpinned path is still wired. That is also what made CI red on an earlier revision of this branch.

Two copies are safe here because nothing crosses the seam: idlFactory is the caller-injected ({ IDL }) => … form, so pic supplies its own IDL, and the canisterId pic returns only ever goes back into pic or through .toText(). Worth keeping in mind for future edits — handing a mops-constructed Principal to pic would be the way to break it. Keeping 4.0.2 also realigns the CLI with frontend/, and leaves the SDK bump with the dfx-replica removal in v3, where it belongs.

What is unchanged

Supported pins behave identically — same binary path, same --ttl, same output. Unpinned projects still fall back to the deprecated dfx replica with its existing warning. pic-ic itself is untouched. The bundled cli.mops.one distribution needed no changes, since it already inlines pic from node_modules.

Verified against a packed CLI in a throwaway prefix rather than a dev checkout, across all three replica paths: the root unpinned mops test sweep passes all 18 files on the dfx replica; pocket-ic = "12.0.0" passes mops test --mode replica and mops bench spawning <cache>/pocket-ic/12.0.0/pocket-ic --port-file … --ttl 60 with no warning and no binary downloaded at install; pocket-ic = "4.0.0" passes the replica test with the warning above, spawning through pic-ic's older --pid style. Legacy mops bench is confirmed only as far as a successful client-to-server round trip — a 2024-era replica rejects modern moc output, and older moc cannot parse persistent actor.

Follow-ups worth flagging

  • A clean npm install in cli/ still pulls the 94 MB binary, because devDependency install scripts run for local dev. The lazy-download change upstream fixes that too.
  • Bundling pic's @icp-sdk/core costs ~500 KB in dist/vendor/pic.mjs. It collapses back to one copy when v3 drops the dfx replica and mops can move to 5.x itself.
  • v3 removes pic-ic, the AnyPocketIc* union types it forces, and turns this warning into a hard error alongside an enforced supported server range.
  • Out of scope: files: ["*"] ships both distributions inside the npm tarball, roughly doubling it. Tracked in TODO.md.

Also here: v3 plan rescope

NEXT-MAJOR.md is now split into a committed v3 scope and a deferred section, with the reasoning recorded inline:

  • The version model stays as it is. Bare 1.2.3 remains exact, conflicts stay max-wins, freshness keeps coming from caret-bounded mops update. Cargo's model needs = pins as an escape hatch, and = in published packages requires backend validator changes plus a window where older CLIs cannot parse what newer ones publish — caret without the escape hatch is the worst of both. That track moves to a future major, gated on moc --override so multi-version coexistence lands with it. v3 keeps the pure wins: a real semver comparator instead of parseInt, and a cross-major conflict warning that always fires.
  • --locked, not a mops ci command. npm's separate command is the ecosystem outlier; cargo, pnpm and yarn all use a flag, and a flag composes with the commands that resolve implicitly. --lock then disappears entirely: check becomes --locked, ignore has no successor, and update becomes self-healing on a corrupt lock.
  • Lockfile v4 leaves the major. Pinning git deps by commit SHA needs a format bump, but cargo shipped v1 through v4 while staying on 1.x by making formats readable long before writing them.
  • dfx keeps one explicit escape hatch. Every implicit fallback goes; --replica dfx stays until our own dev loop is on icp-cli.

@Kamirus Kamirus changed the title v3 plan rescope + spike: switch to @dfinity/pic cli: switch to upstream @dfinity/pic, deprecate pocket-ic < 9.0.0 Aug 3, 2026
Kamirus added 2 commits August 3, 2026 17:42
`pocket-ic` 9.0.0 and newer now runs through upstream `@dfinity/pic`
0.23.0 instead of the `pic-js-mops` fork. That fork has no public source
repository — it is published from a local checkout of `dfinity/pic-js`
with four patches, so the only way to audit it is diffing the published
tarball, and one person can publish a fix. Two of the patches are now
upstream (dfinity/pic-js#276 for an explicit binary path, #278 for a
server `--ttl`), the third needs no upstream change (`serverProcess` is
TS-private but present at runtime, so the client seam casts), and the
fourth is handled here.

`@dfinity/pic` stays a devDependency, pre-bundled into `dist/vendor/`:
its postinstall downloads a ~94 MB pocket-ic binary and throws when it
cannot, and the published CLI is the unbundled `dist/` tree with the full
dependency list, so a `dependencies` entry would make `npm i -g ic-mops`
pull that binary and fail outright with no network. mops manages the
binary itself via `[toolchain] pocket-ic`. Removable later: if pic makes
the download lazy, this becomes an ordinary dependency again.

Pins below 9.0.0 keep working through the unchanged `pic-ic` client but
now warn, and go away in v3 — the same deprecate-then-drop path used for
the dfx replica and vessel. `addCycles` gets a wrapper rather than a
cast, because it takes a `number` on `pic-ic` and a `bigint` upstream and
an intersection type would compile while feeding the wrong one at
runtime.

Verified with a packed CLI installed into a throwaway prefix: with
`pocket-ic = "12.0.0"` both `mops test --mode replica` and `mops bench`
pass, spawning the toolchain-managed binary with `--ttl`, and the install
downloads no binary; with `pocket-ic = "4.0.0"` the replica test still
passes and warns. `@icp-sdk/core` moves to 5.4.0 as a single deduped copy
(pic wants ^5, and 6.x would nest a second copy and give two
`Principal`/`IDL` identities).
Splits `NEXT-MAJOR.md` into a committed v3 scope and a deferred section
so nothing is lost, and records the reasoning inline:

- Keep today's version model. Bare `1.2.3` stays exact, conflicts stay
  max-wins, freshness keeps coming from caret-bounded `mops update`.
  Cargo's model needs `=` pins as an escape hatch, and `=` in published
  packages needs backend validator changes plus a window where older
  CLIs cannot parse what newer ones publish, so that track moves to a
  future major gated on moc `--override`. v3 keeps the pure wins: a real
  semver comparator and an always-on cross-major conflict warning.
- `--locked` instead of a `mops ci` command, and drop `--lock`: `check`
  becomes `--locked`, `ignore` has no successor, `update` becomes
  self-healing on a corrupt lock.
- Lockfile v4 gets its own read-early/write-later track, decoupled from
  the CLI major.
- dfx: remove every implicit fallback, keep `--replica dfx` explicit
  until our own dev loop is on icp-cli.
- The `@dfinity/pic` switch ships in 2.x; only dropping `pic-ic` and
  enforcing a supported server range stay in v3.

TODO.md picks up the `files: ["*"]` packaging bloat found along the way
and drops the stale `mops ci` phrasing.
@Kamirus
Kamirus force-pushed the kamil-claude/mops-cli-major-update-f3fccd branch from 9f1308c to 2e8aff0 Compare August 3, 2026 15:42
@Kamirus
Kamirus marked this pull request as ready for review August 3, 2026 15:43
@Kamirus Kamirus changed the title cli: switch to upstream @dfinity/pic, deprecate pocket-ic < 9.0.0 feat(cli): use upstream @dfinity/pic, deprecate pocket-ic below 9.0.0 Aug 3, 2026
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Cursor AI review

👍 APPROVE — looks safe to merge

Category Assessment Details
Summary Replaces unauditable pic-js-mops with vendored upstream @dfinity/pic 0.23.0 for pocket-ic ≥9.0.0, keeps pic-ic for older pins with a one-shot deprecation warning, and syncs docs/changelog/skills/v3 plan.
Code Quality Seam stays in pocket-ic-client.ts; warning helper mirrors deprecate-dfx-replica.ts; vendor entry + fix-dist rewrite are the minimum needed to avoid pic’s throwing postinstall.
Consistency Sibling replica/bench paths still share startPocketIc; docs (09-mops.toml.md, toolchain overview), ## Next changelog, and mops-cli skill all updated; examples moved off deprecated 1.0.0.
Security Traced startPocketIc / addCycles / serverStderr into replica.ts deploy; @dfinity/pic is a pinned devDependency pre-bundled so published installs do not run its binary download; mops stays on @icp-sdk/core 4.0.2 for dfx replicas.
Tests vendor-pic.test.ts asserts named exports and the dist import rewrite; existing pocket-ic.test.ts still covers pin 12.0.0; CI cli && npm ci runs prepare so those dist checks are not skipped in CI.
Maintainability Temporary vendor machinery is documented with a clear removal path once upstream lazy-downloads; NEXT-MAJOR.md correctly moves fork elimination to 2.x and leaves pic-ic drop for v3.

Verdict

Decision: APPROVE
Risk: Low
Reason: Intended, non-breaking client swap with a compatibility path for legacy pins; not in registry/install/auth/release surfaces, and packaging/docs/tests cover the main failure modes of the vendor seam.


Generated for commit a281d55

@automation-sa-sre automation-sa-sre left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated approval: the AI review verdict for 2e8aff0 is APPROVE. See the "Cursor AI review" comment for details.

Kamirus added 2 commits August 3, 2026 19:56
Moving mops to `@icp-sdk/core` 5.4.0 broke every replica test that runs
on the `dfx` or `dfx-pocket-ic` replica. 5.x calls `/api/v4/.../call`,
`/api/v3/.../query` and `/api/v3/.../read_state`; dfx 0.29.1's HTTP
gateway serves only the `v2` endpoints. It answers `/api/v4` with the
gateway's own 400 "canister id not resolved" HTML page rather than a 404,
so the agent's v4-to-v2 fallback — which keys on 404 — never fires, and
`query`/`read_state` have no fallback at all. The failure surfaced as an
8 KB HTML error page dumped into the test output.

The bump was only ever needed because `@dfinity/pic` depends on
`@icp-sdk/core` ^5, and `vendor:pic` left that dependency external. It
is a plain dependency, not a peer, so the vendored bundle can carry its
own copy instead. Nothing crosses the two copies: `idlFactory` receives
pic's `IDL` as an argument, and the `canisterId` pic returns only goes
back into pic or through `.toText()`. This also realigns the CLI with
`frontend/`, which was left on 4.0.2.

Dropping `--external:@icp-sdk/core` grows `dist/vendor/pic.mjs` from
600 KB to 1.1 MB — one file in the tarball, against the ~94 MB
postinstall download that vendoring avoids.

Verified with the dist build installed into a throwaway prefix: the root
`mops test` sweep passes all 18 files (both `test/storage-actor.test.mo`
and `cli/tests/pocket-ic/test/hello.test.mo` failed before), `pocket-ic
= "12.0.0"` passes on `@dfinity/pic` for both `mops test` and `mops
bench`, `pocket-ic = "4.0.0"` still passes on `pic-ic` and still warns,
and unpinned `mops bench` still runs on the dfx replica with its
deprecation warning.
`toolchain use pocket-ic` still referred to a "bundled pic-js client",
which is now upstream `@dfinity/pic` vendored into `dist/`, and the skill
never stated the supported version floor. Adds the `9.0.0` floor and the
deprecation of lower pins.

Unpinned behavior is unchanged — replica tests and benchmarks still fall
back to the deprecated `dfx` replica with a warning — so the guidance at
the top and under replica tests stays as it is.

@automation-sa-sre automation-sa-sre left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated approval: the AI review verdict for 72a0660 is APPROVE. See the "Cursor AI review" comment for details.

The spike concluded that 4.0.2 -> 5.4.0 was clean. It is not: 5.x drops
the IC HTTP API v2 endpoints, which are all the dfx replicas serve, so
unpinned replica tests and benchmarks break. CI caught it. mops keeps
4.0.2 and bundles pic's own 5.x, and the bump belongs with the
dfx-replica removal.

Follow-ups filed as #651, #652, #655, #657.

@automation-sa-sre automation-sa-sre left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated approval: the AI review verdict for a281d55 is APPROVE. See the "Cursor AI review" comment for details.

@Kamirus
Kamirus merged commit 68096ee into main Aug 4, 2026
32 checks passed
@Kamirus
Kamirus deleted the kamil-claude/mops-cli-major-update-f3fccd branch August 4, 2026 11:49
Kamirus added a commit that referenced this pull request Aug 5, 2026
`decompress` is a critical, unfixable advisory in the CLI's dependency
tree, and the plan doc said dropping vessel would take it with it.
Neither half of that was right, so both are corrected here along with
the parts of the plan that #642 made stale.

## `decompress` has to go, and vessel won't take it

`cli/package.json` pins `decompress` 4.2.1. Both advisories against it
cover `<=4.2.1`, so the newest release is still affected and there is
nothing to upgrade to:

```
$ npm audit
critical  decompress
  Decompress: Archive extraction can create files and links outside of the target directory
  decompress: Arbitrary File Write via Archive Extraction (Zip Slip)
  fixAvailable: false
```


[GHSA-mp2f-45pm-3cg9](GHSA-mp2f-45pm-3cg9),
[GHSA-h39j-r5qq-r9mm](GHSA-h39j-r5qq-r9mm).
Removing the dependency is the only remedy, and there are two call sites
— the toolchain extractor (`toolchain-utils.ts:64`, for
moc/wasmtime/pocket-ic/lintoko archives) and `installFromGithub`
(`vessel.ts:167`, for `repo = "..."` deps). The second one is the
exposed surface: those archives come from arbitrary user-specified
repos, which is precisely the zip-slip threat model.

Filed in `TODO.md` rather than here, because replacing an extractor
breaks nothing and should not wait for a major.

## `installFromGithub` must move, not be deleted

The vessel item listed `installFromGithub` among the things v3 deletes.
It lives in `cli/vessel.ts` for historical reasons only —
`cli/commands/install/install-dep.ts:23` and `cli/commands/add.ts:104`
call it for ordinary GitHub deps, which mops supports and which the
lockfile-v4 track plans to improve with commit-SHA pinning. Deleting it
would break git deps; it moves into `cli/commands/install/` instead.

This is also why dropping vessel does not resolve the advisory on its
own: that `decompress` call site outlives vessel.

## pic migration is history now

#642 shipped the `@dfinity/pic` switch, so the long spike narrative is
replaced by current state and the three follow-up issues it produced
([#651](#651) vendor
bundling, [#652](#652)
`@icp-sdk/core` 5.x,
[#655](#655) log hook). Only
removing the legacy `pic-ic` client is still v3 work, tracked in
[#653](#653). Provenance
notes are kept in condensed form, since the reason the fork existed is
worth not relearning.

## Docs only

No code changes. Nothing here is an argument for cutting v3 sooner — the
decompress work deliberately lands outside it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Switch from pic-js-mops to upstream @dfinity/pic once v13 support is published

2 participants