From 775ab93d133640a35be20f8492eef89fa58df98f Mon Sep 17 00:00:00 2001 From: Jakub Dzikowski Date: Mon, 13 Jul 2026 13:00:00 +0200 Subject: [PATCH 01/85] Fix: cross-platform process-tree termination via killProcessTree MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit jaiph run spawns the workflow leader detached and previously stopped it with process.kill(-pid, signal) — a POSIX-only negative-PID group kill. On win32 that throws, the child.kill() fallback terminated only the leader, and the agent backends / script children it spawned were orphaned. Add src/runtime/kernel/portability.ts exporting killProcessTree(pid, signal) as the single sanctioned home for group kills. On POSIX it preserves prior behavior (negative-PID group kill with a per-process ESRCH fallback); on win32 it force-kills the whole tree with `taskkill /pid /T /F` (spawned, not shelled) via an injectable seam, degrading to a per-process kill if taskkill cannot launch. Because taskkill /F is already forceful, the SIGTERM->SIGKILL escalation is a documented no-op on win32. Repoint every group-kill call site at the helper: run teardown (lifecycle.ts), the prompt watchdog (prompt.ts), and the Docker run-timeout kill (docker.ts). New unit tests stub process.platform to cover both branches (negative-PID kill on POSIX plus ESRCH fallback; taskkill /T argv on win32) and prove win32 never calls process.kill with a negative PID; a src/-wide lint test asserts no production file outside portability.ts matches `process.kill(-`. Docs updated. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 4 + docs/architecture.md | 2 +- docs/configuration.md | 2 +- src/cli/run/lifecycle.ts | 11 +- src/runtime/docker.ts | 18 +-- src/runtime/kernel/portability.test.ts | 204 +++++++++++++++++++++++++ src/runtime/kernel/portability.ts | 78 ++++++++++ src/runtime/kernel/prompt.test.ts | 26 +++- src/runtime/kernel/prompt.ts | 18 +-- 9 files changed, 332 insertions(+), 31 deletions(-) create mode 100644 src/runtime/kernel/portability.test.ts create mode 100644 src/runtime/kernel/portability.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 1c09f9e5..1de9058f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,9 @@ # Unreleased +## All changes + +- **Fix — Portability: cross-platform process-tree termination:** `jaiph run` spawns the workflow leader with `detached: true` (`src/runtime/kernel/workflow-launch.ts`) and previously stopped it via `process.kill(-pid, signal)` — a negative-PID group kill, which signals the leader's whole process group. That primitive is POSIX-only: on `win32` it throws, the previous `child.kill()` fallback terminated only the leader, and the agent backends / script children it spawned were **orphaned**. A new portability module `src/runtime/kernel/portability.ts` exports **`killProcessTree(pid, signal)`** and is now the single sanctioned home for group kills. On POSIX it preserves the prior behavior — `process.kill(-pid, signal)` with a per-process fallback when the group no longer exists (`ESRCH`). On `win32` it force-kills the entire tree with `taskkill /pid /T /F` (spawned via an injectable `_portability.spawn` seam, not shelled), degrading to a per-process `process.kill(pid, signal)` if `taskkill` cannot be launched (reported asynchronously via the child's `error` event). Because `taskkill /F` is already a forceful kill with no graceful phase, a follow-up `SIGKILL` escalation after a `SIGTERM`/`SIGINT` is a **documented no-op** on `win32` — the tree is already gone. Every subprocess-termination call site is repointed at the helper: run teardown `terminateRunProcessGroup` (`src/cli/run/lifecycle.ts`, previously the negative-PID group kill) and the two SIGTERM→SIGKILL escalation paths that previously assumed POSIX signal semantics via `child.kill()` — the prompt watchdog's `killChild` (`src/runtime/kernel/prompt.ts`) and the Docker run-timeout kill (`src/runtime/docker.ts`) — so those escalations now terminate the whole child tree instead of just the leader. POSIX Ctrl-C behavior of `jaiph run` is unchanged (leader and children terminate; exit code and cleanup semantics identical), covered by the existing signal-lifecycle e2e. New unit tests (`src/runtime/kernel/portability.test.ts`) cover both platform branches by stubbing `process.platform` (precedent: `src/runtime/docker.test.ts`) and assert the exact mechanism invoked — negative-PID kill on POSIX (plus the ESRCH per-process fallback and SIGKILL escalation), `taskkill /pid /T /F` argv on `win32` (SIGTERM, SIGINT/Ctrl-C, and the degrade-to-per-process path), a proof that the win32 branch **never** calls `process.kill` with a negative PID, and a `src/`-wide lint test asserting no production file outside `portability.ts` matches `process.kill(-`. The watchdog unit tests in `src/runtime/kernel/prompt.test.ts` are updated to spy on `process.kill` (the watchdog now terminates through `killProcessTree`). Docs updated (`docs/architecture.md`, `docs/configuration.md`). + # 0.10.0 ## Summary diff --git a/docs/architecture.md b/docs/architecture.md index 6dc87d33..d672076b 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -123,7 +123,7 @@ User-visible contracts (banner, hooks, run artifacts, `run_summary.jsonl`, `retu ### CLI responsibilities - Parse, validate, and launch workflows/tests. -- Own **process spawn** for `jaiph run` (detached workflow runner process group for signal propagation). +- Own **process spawn** for `jaiph run` (detached workflow runner process group for signal propagation). Terminating a run means terminating the whole tree — the detached leader plus the agent backends and script children it spawned — routed through **`killProcessTree(pid, signal)`** (`src/runtime/kernel/portability.ts`), the single sanctioned home for group kills. On POSIX it signals the leader's process group with **`process.kill(-pid, signal)`**, falling back to a per-process kill if the group no longer exists (`ESRCH`). On **`win32`** a negative-PID group kill throws and a per-process kill would orphan the children, so it force-kills the tree with **`taskkill /pid /T /F`** (spawned, not shelled), degrading to a per-process kill if `taskkill` cannot be launched. Because `taskkill /F` is already forceful, a follow-up `SIGKILL` escalation after a `SIGTERM`/`SIGINT` is a **documented no-op** on `win32`. All group-kill call sites route through this helper: run teardown (`src/cli/run/lifecycle.ts`), the prompt watchdog (`src/runtime/kernel/prompt.ts`), and the Docker run-timeout kill (`src/runtime/docker.ts`). - Parse live runtime events; render terminal progress; trigger hooks — skipped in **`jaiph run --raw`** (child stdio inherited; see [CLI](cli.md#jaiph-run)). ## Contracts diff --git a/docs/configuration.md b/docs/configuration.md index d1ba8de1..83bdb949 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -236,7 +236,7 @@ The retry backoff above handles a backend that *fails*. A separate set of watchd Set any variable to `0` to disable that layer. The idle timer resets on every chunk of backend output, so a slow-but-active run is bounded only by the absolute cap. -The completion-grace layer specifically addresses the known `claude -p` failure mode where the CLI streams its final answer (and the terminal `result` event) but the process never exits — often because a descendant it spawned is still holding the output pipe open. When a watchdog fires it sends `SIGTERM`, escalating to `SIGKILL` after 5s, and tears down the runtime's handles on the child's stdio so a lingering descendant cannot keep the run alive. Under Docker, `runtime.docker_timeout_seconds` remains the outer backstop for the whole container. +The completion-grace layer specifically addresses the known `claude -p` failure mode where the CLI streams its final answer (and the terminal `result` event) but the process never exits — often because a descendant it spawned is still holding the output pipe open. When a watchdog fires it terminates the backend's whole process tree (via `killProcessTree`; see [Architecture](architecture.md)) with `SIGTERM`, escalating to `SIGKILL` after 5s, and tears down the runtime's handles on the child's stdio so a lingering descendant cannot keep the run alive. On Windows the tree is force-killed with `taskkill /T` on the first signal, so the `SIGKILL` escalation is a no-op. Under Docker, `runtime.docker_timeout_seconds` remains the outer backstop for the whole container. ## Custom agent commands diff --git a/src/cli/run/lifecycle.ts b/src/cli/run/lifecycle.ts index 55f93206..392e90c1 100644 --- a/src/cli/run/lifecycle.ts +++ b/src/cli/run/lifecycle.ts @@ -1,6 +1,7 @@ import { ChildProcess } from "node:child_process"; import { spawnJaiphWorkflowProcess } from "../../runtime/kernel/workflow-launch"; +import { killProcessTree } from "../../runtime/kernel/portability"; export function spawnRunProcess( args: string[], @@ -17,15 +18,7 @@ export function terminateRunProcessGroup( if (!pid) { return; } - try { - process.kill(-pid, signal); - } catch { - try { - child.kill(signal); - } catch { - // no-op - } - } + killProcessTree(pid, signal); } export function setupRunSignalHandlers( diff --git a/src/runtime/docker.ts b/src/runtime/docker.ts index de91f7c8..b281f050 100644 --- a/src/runtime/docker.ts +++ b/src/runtime/docker.ts @@ -5,6 +5,7 @@ import { tmpdir } from "node:os"; import { join, resolve, dirname, relative } from "node:path"; import type { RuntimeConfig } from "../types"; import { OVERLAY_RUN_SH_BASE64, decodeEmbeddedAsset } from "./embedded-assets"; +import { killProcessTree } from "./kernel/portability"; /** Resolved Docker runtime config with defaults applied and env overrides merged. */ export interface DockerRunConfig { @@ -812,17 +813,16 @@ export function spawnDockerProcess(opts: DockerSpawnOptions): DockerSpawnResult let timeoutTimer: NodeJS.Timeout | undefined; if (opts.config.timeoutSeconds > 0) { timeoutTimer = setTimeout(() => { - try { - child.kill("SIGTERM"); - } catch { - // no-op + const pid = child.pid; + if (!pid) { + return; } + // Terminate the `docker run` child and its descendants. On win32 the + // taskkill /T force-kills the tree, so the SIGKILL escalation below is a + // documented no-op there (see killProcessTree). + killProcessTree(pid, "SIGTERM"); setTimeout(() => { - try { - child.kill("SIGKILL"); - } catch { - // no-op - } + killProcessTree(pid, "SIGKILL"); }, 5000); }, opts.config.timeoutSeconds * 1000); } diff --git a/src/runtime/kernel/portability.test.ts b/src/runtime/kernel/portability.test.ts new file mode 100644 index 00000000..6ee257c6 --- /dev/null +++ b/src/runtime/kernel/portability.test.ts @@ -0,0 +1,204 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { EventEmitter } from "node:events"; +import { readFileSync, readdirSync } from "node:fs"; +import { resolve, join } from "node:path"; +import { killProcessTree, _portability } from "./portability"; + +// Precedent for platform stubbing: src/runtime/docker.test.ts. +function withPlatform(platform: string, fn: () => void): void { + const orig = Object.getOwnPropertyDescriptor(process, "platform"); + Object.defineProperty(process, "platform", { value: platform, configurable: true }); + try { + fn(); + } finally { + if (orig) Object.defineProperty(process, "platform", orig); + } +} + +/** Capture every `process.kill(pid, signal)` call while `fn` runs. */ +function withKillSpy( + fn: (calls: Array<{ pid: number; signal: NodeJS.Signals | undefined }>) => void, + behavior?: (pid: number, signal: NodeJS.Signals | undefined) => void, +): void { + const calls: Array<{ pid: number; signal: NodeJS.Signals | undefined }> = []; + const orig = process.kill; + (process as { kill: typeof process.kill }).kill = ((pid: number, signal?: NodeJS.Signals) => { + calls.push({ pid, signal }); + if (behavior) behavior(pid, signal); + return true; + }) as typeof process.kill; + try { + fn(calls); + } finally { + (process as { kill: typeof process.kill }).kill = orig; + } +} + +/** Capture the `_portability.spawn` invocation and return a fake child. */ +function withSpawnSpy( + fn: (calls: Array<{ command: string; args: string[] }>, fakeChild: EventEmitter) => void, +): void { + const calls: Array<{ command: string; args: string[] }> = []; + const fakeChild = new EventEmitter() as EventEmitter & { unref?: () => void }; + fakeChild.unref = () => {}; + const orig = _portability.spawn; + _portability.spawn = ((command: string, args: string[]) => { + calls.push({ command, args }); + return fakeChild as unknown as ReturnType; + }) as typeof _portability.spawn; + try { + fn(calls, fakeChild); + } finally { + _portability.spawn = orig; + } +} + +// --------------------------------------------------------------------------- +// POSIX branch +// --------------------------------------------------------------------------- + +test("POSIX: killProcessTree sends signal to the negative PID (process group)", () => { + withPlatform("linux", () => { + withKillSpy((calls) => { + killProcessTree(1234, "SIGTERM"); + assert.equal(calls.length, 1, "exactly one process.kill call on the happy path"); + assert.deepEqual(calls[0], { pid: -1234, signal: "SIGTERM" }); + }); + }); +}); + +test("POSIX: falls back to per-process kill when the group kill throws (ESRCH)", () => { + withPlatform("linux", () => { + withKillSpy( + (calls) => { + killProcessTree(1234, "SIGINT"); + assert.deepEqual(calls, [ + { pid: -1234, signal: "SIGINT" }, // group kill attempted first + { pid: 1234, signal: "SIGINT" }, // then per-process fallback + ]); + }, + (pid) => { + if (pid < 0) throw new Error("ESRCH"); + }, + ); + }); +}); + +test("POSIX: SIGKILL escalation also uses the negative PID group kill", () => { + withPlatform("linux", () => { + withKillSpy((calls) => { + killProcessTree(999, "SIGKILL"); + assert.deepEqual(calls[0], { pid: -999, signal: "SIGKILL" }); + }); + }); +}); + +// --------------------------------------------------------------------------- +// win32 branch +// --------------------------------------------------------------------------- + +test("win32: killProcessTree spawns `taskkill /pid /T /F`", () => { + withPlatform("win32", () => { + withSpawnSpy((spawnCalls) => { + withKillSpy((killCalls) => { + killProcessTree(4321, "SIGTERM"); + assert.equal(spawnCalls.length, 1, "taskkill spawned once"); + assert.equal(spawnCalls[0].command, "taskkill"); + assert.deepEqual(spawnCalls[0].args, ["/pid", "4321", "/T", "/F"]); + assert.equal(killCalls.length, 0, "no process.kill on the happy win32 path"); + }); + }); + }); +}); + +test("win32: SIGINT also spawns taskkill /T (Ctrl-C path)", () => { + withPlatform("win32", () => { + withSpawnSpy((spawnCalls) => { + killProcessTree(7, "SIGINT"); + assert.deepEqual(spawnCalls[0].args, ["/pid", "7", "/T", "/F"]); + }); + }); +}); + +test("win32: SIGKILL escalation is a documented no-op (tree already force-killed)", () => { + withPlatform("win32", () => { + withSpawnSpy((spawnCalls) => { + withKillSpy((killCalls) => { + killProcessTree(4321, "SIGKILL"); + assert.equal(spawnCalls.length, 0, "no second taskkill on SIGKILL escalation"); + assert.equal(killCalls.length, 0, "no process.kill on SIGKILL escalation"); + }); + }); + }); +}); + +test("win32: degrades to per-process kill when taskkill cannot be spawned", () => { + withPlatform("win32", () => { + withSpawnSpy((_spawnCalls, fakeChild) => { + withKillSpy((killCalls) => { + killProcessTree(555, "SIGTERM"); + // spawn returns a child that reports failure asynchronously via "error". + fakeChild.emit("error", new Error("spawn taskkill ENOENT")); + assert.deepEqual(killCalls, [{ pid: 555, signal: "SIGTERM" }]); + }); + }); + }); +}); + +test("win32: NEVER calls process.kill with a negative PID (SIGTERM, SIGINT, SIGKILL)", () => { + withPlatform("win32", () => { + withSpawnSpy((_spawnCalls, fakeChild) => { + withKillSpy((killCalls) => { + killProcessTree(1234, "SIGTERM"); + fakeChild.emit("error", new Error("no taskkill")); // force the fallback path too + killProcessTree(1234, "SIGINT"); + fakeChild.emit("error", new Error("no taskkill")); + killProcessTree(1234, "SIGKILL"); + for (const call of killCalls) { + assert.ok(call.pid > 0, `win32 must never signal a negative PID (saw ${call.pid})`); + } + }); + }); + }); +}); + +// --------------------------------------------------------------------------- +// Lint contract: only the portability module may group-kill via negative PID. +// --------------------------------------------------------------------------- + +// Tests run from dist/src/runtime/kernel/, so repo root is five levels up. +const REPO_ROOT = resolve(__dirname, "../../../.."); +const SRC_ROOT = join(REPO_ROOT, "src"); + +/** Non-test production source files (excludes *.test.ts / *.acceptance.test.ts). */ +function walkProductionTsFiles(dir: string): string[] { + const out: string[] = []; + for (const entry of readdirSync(dir, { withFileTypes: true })) { + const full = join(dir, entry.name); + if (entry.isDirectory()) { + out.push(...walkProductionTsFiles(full)); + } else if (entry.isFile() && entry.name.endsWith(".ts") && !entry.name.endsWith(".test.ts")) { + out.push(full); + } + } + return out; +} + +test("no production source file outside portability.ts invokes process.kill with a negative PID", () => { + // Match `process.kill(-` allowing whitespace, e.g. `process.kill( -pid`. + const negativeGroupKill = /process\.kill\(\s*-/; + const offenders: string[] = []; + for (const file of walkProductionTsFiles(SRC_ROOT)) { + const rel = file.slice(REPO_ROOT.length + 1); + // The helper is the one sanctioned home for the negative-PID group kill. + if (rel === join("src", "runtime", "kernel", "portability.ts")) continue; + const content = readFileSync(file, "utf8"); + if (negativeGroupKill.test(content)) offenders.push(rel); + } + assert.deepEqual( + offenders, + [], + `negative-PID group kill must be confined to portability.ts; offenders: ${offenders.join(", ")}`, + ); +}); diff --git a/src/runtime/kernel/portability.ts b/src/runtime/kernel/portability.ts new file mode 100644 index 00000000..a012ef90 --- /dev/null +++ b/src/runtime/kernel/portability.ts @@ -0,0 +1,78 @@ +// Cross-platform process-tree termination. +// +// `jaiph run` launches the workflow leader detached (`workflow-launch.ts`), and +// the leader in turn spawns agent backends (`prompt.ts`) and script children, +// or a `docker run` child (`docker.ts`). Terminating a run cleanly means +// terminating that whole tree, not just the leader — otherwise backends and +// script children are orphaned. +// +// POSIX has a single primitive for this: a detached child is its own process +// group leader (pgid == pid), so `process.kill(-pid, signal)` delivers `signal` +// to every process in the group. Windows has no such primitive — a negative pid +// throws, and `child.kill()` / `process.kill(pid)` terminate only the leader. +// `killProcessTree` hides that difference behind one call site. + +import { spawn, type ChildProcess } from "node:child_process"; + +/** + * Test seam for the `taskkill` spawn. Swapped out in unit tests so the win32 + * branch can be exercised on any host without a real `taskkill` binary. + */ +export const _portability = { + spawn(command: string, args: string[]): ChildProcess { + return spawn(command, args, { stdio: "ignore" }); + }, +}; + +/** + * Terminate `pid` and its descendants with `signal`, portably. + * + * POSIX: `process.kill(-pid, signal)` signals the whole process group. If the + * target is not a group leader the group does not exist (ESRCH) and we fall + * back to signaling the single process, matching the pre-portability behavior. + * + * win32: negative-pid group kill is unsupported (it throws) and per-process + * kill orphans children, so we spawn `taskkill /pid /T /F`, which force- + * terminates the entire tree. If `taskkill` cannot be launched we degrade to a + * per-process `process.kill(pid, signal)`. Because `taskkill /F` is already a + * forceful kill with no graceful phase, a follow-up `SIGKILL` escalation after + * a `SIGTERM`/`SIGINT` is a **documented no-op** — the tree is already gone. + */ +export function killProcessTree(pid: number, signal: NodeJS.Signals): void { + if (process.platform === "win32") { + killProcessTreeWin32(pid, signal); + return; + } + try { + process.kill(-pid, signal); + } catch { + killSingleProcess(pid, signal); + } +} + +function killProcessTreeWin32(pid: number, signal: NodeJS.Signals): void { + // `taskkill /F` already force-killed the tree on the first (SIGTERM/SIGINT) + // call, so the SIGKILL escalation has nothing left to terminate. + if (signal === "SIGKILL") { + return; + } + let child: ChildProcess; + try { + child = _portability.spawn("taskkill", ["/pid", String(pid), "/T", "/F"]); + } catch { + killSingleProcess(pid, signal); + return; + } + // `spawn` reports a missing/unspawnable `taskkill` asynchronously via "error", + // not by throwing — degrade to a per-process kill when that fires. + child.once?.("error", () => killSingleProcess(pid, signal)); + child.unref?.(); +} + +function killSingleProcess(pid: number, signal: NodeJS.Signals): void { + try { + process.kill(pid, signal); + } catch { + // no-op: process already gone or not signalable. + } +} diff --git a/src/runtime/kernel/prompt.test.ts b/src/runtime/kernel/prompt.test.ts index fa10db5c..1e042d7c 100644 --- a/src/runtime/kernel/prompt.test.ts +++ b/src/runtime/kernel/prompt.test.ts @@ -1,4 +1,4 @@ -import { describe, it } from "node:test"; +import { describe, it, afterEach } from "node:test"; import * as assert from "node:assert/strict"; import { chmodSync, existsSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; @@ -235,18 +235,40 @@ describe("executePrompt", () => { }); /** Minimal ChildProcess stand-in: an EventEmitter with a recording `kill`. */ +const FAKE_CHILD_PID = 4242; +let origProcessKill: typeof process.kill | undefined; + +// The watchdog terminates via killProcessTree(pid, signal) (portability.ts), +// which on POSIX calls process.kill(-pid|pid, signal). Spy on process.kill and +// record the signals aimed at the fake child's pid (either sign). Restored per +// test by the afterEach below so the spy never leaks into other suites. function makeFakeChild(): { child: ChildProcess; killSignals: string[] } { const emitter = new EventEmitter() as EventEmitter & { pid: number; kill: (s?: string) => boolean }; const killSignals: string[] = []; - emitter.pid = 4242; + emitter.pid = FAKE_CHILD_PID; emitter.kill = (signal?: string) => { killSignals.push(signal ?? "SIGTERM"); return true; }; + origProcessKill = process.kill; + (process as { kill: typeof process.kill }).kill = ((pid: number, signal?: NodeJS.Signals) => { + if (Math.abs(pid) === FAKE_CHILD_PID) { + killSignals.push(signal ?? "SIGTERM"); + return true; + } + return origProcessKill!(pid, signal); + }) as typeof process.kill; return { child: emitter as unknown as ChildProcess, killSignals }; } describe("installPromptWatchdog", () => { + afterEach(() => { + if (origProcessKill) { + (process as { kill: typeof process.kill }).kill = origProcessKill; + origProcessKill = undefined; + } + }); + it("layer 2: terminates and fails when output stalls past the idle timeout", async () => { const { child, killSignals } = makeFakeChild(); const events: Array<{ status: number; reason: string; final: string }> = []; diff --git a/src/runtime/kernel/prompt.ts b/src/runtime/kernel/prompt.ts index c4b26f05..d09c4977 100644 --- a/src/runtime/kernel/prompt.ts +++ b/src/runtime/kernel/prompt.ts @@ -5,6 +5,7 @@ import { writeFileSync, readFileSync, existsSync, accessSync, mkdirSync, cpSync, import { basename, delimiter, join } from "node:path"; import { parseStream, type StreamWriter } from "./stream-parser"; import { consumeNextMockResponse, dispatchMockArms, type MockPromptArm } from "./mock"; +import { killProcessTree } from "./portability"; export type PromptConfig = { backend: string; @@ -452,17 +453,16 @@ export function installPromptWatchdog( }; const killChild = (): void => { - try { - child.kill("SIGTERM"); - } catch { - // no-op + const pid = child.pid; + if (!pid) { + return; } + // Terminate the backend and any descendants it spawned. On win32 this + // taskkill /T already force-kills the tree, so the SIGKILL escalation + // below is a documented no-op there (see killProcessTree). + killProcessTree(pid, "SIGTERM"); const escalate = setTimeout(() => { - try { - child.kill("SIGKILL"); - } catch { - // no-op - } + killProcessTree(pid, "SIGKILL"); }, 5000); escalate.unref?.(); }; From afef44b85a14d860743b35b12c3d1fb1bd2f567d Mon Sep 17 00:00:00 2001 From: Jakub Dzikowski Date: Mon, 13 Jul 2026 13:17:58 +0200 Subject: [PATCH 02/85] Fix: execute script steps via explicit interpreter, not shebang Script steps previously ran by spawning the emitted file directly, relying on the `#!/usr/bin/env ` shebang and the 0o755 exec bit. Windows honors neither, and `noexec` mounts on Linux strip the exec bit, both breaking script execution. The runtime already knows the interpreter since it writes the shebang itself, so `executeScript` now reads the emitted script's shebang, resolves the interpreter through the new `resolveInterpreterFromShebang` (`src/parse/script-bash.ts`), and spawns ` ` explicitly. The shebang line is still written into every emitted script so they remain directly executable by hand on POSIX, but the runtime no longer depends on it or on the exec bit being honored. A spawn ENOENT from a missing interpreter now surfaces as a diagnosable Jaiph error naming the interpreter instead of a raw `spawn ENOENT`. New unit tests assert the resolved interpreter and argv on an injectable `_scriptSpawn` seam, prove a script with the exec bit stripped (0o644) still runs on POSIX, and prove a missing-interpreter shebang produces the diagnosable error. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 1 + QUEUE.md | 104 ++++++++++ docs/architecture.md | 1 + docs/setup.md | 2 +- src/parse/script-bash.test.ts | 62 ++++++ src/parse/script-bash.ts | 39 ++++ .../node-workflow-runtime.script-exec.test.ts | 183 ++++++++++++++++++ src/runtime/kernel/node-workflow-runtime.ts | 63 +++++- 8 files changed, 449 insertions(+), 6 deletions(-) create mode 100644 src/parse/script-bash.test.ts create mode 100644 src/runtime/kernel/node-workflow-runtime.script-exec.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 1de9058f..e6418d99 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,7 @@ ## All changes +- **Fix — Portability: execute script steps via an explicit interpreter, not shebang + exec bit:** Script steps used to run by spawning the emitted file directly (`executeScript` in `src/runtime/kernel/node-workflow-runtime.ts`), relying on the `#!/usr/bin/env ` shebang line and the `0o755` bit set by `src/transpile/build.ts`. Windows honors neither, and `noexec` mounts on Linux strip the exec bit — both break script execution. The runtime already knows the interpreter (it writes the shebang itself), so `executeScript` now reads the emitted script's shebang and resolves the interpreter through the new **`resolveInterpreterFromShebang`** (`src/parse/script-bash.ts`) — `#!/usr/bin/env ` → spawn ``, an absolute-path shebang (`#!/bin/bash`) → spawn that path, an `env -S` split-flag form → the real interpreter after `-S`, a missing/blank shebang → default `bash` — then spawns ` ` explicitly. The shebang line is **still** written into every emitted script (they stay directly executable by hand on POSIX) and the `0o755` bit is still set, but the runtime no longer depends on either being honored by the OS. A spawn `ENOENT` from an interpreter missing on `PATH` now surfaces as a diagnosable Jaiph error naming the interpreter (`script interpreter "" not found — install it or fix the script shebang`) instead of a raw `spawn ENOENT`. New unit tests assert the resolved interpreter + argv on the spawn call itself through an injectable `_scriptSpawn` seam (`src/runtime/kernel/node-workflow-runtime.script-exec.test.ts`), prove that a script with the exec bit stripped (`0o644`) still runs on POSIX, and prove that a missing-interpreter shebang produces the diagnosable error; `resolveInterpreterFromShebang` gets its own unit coverage (`src/parse/script-bash.test.ts`). Docs updated (`docs/architecture.md`, `docs/setup.md`). - **Fix — Portability: cross-platform process-tree termination:** `jaiph run` spawns the workflow leader with `detached: true` (`src/runtime/kernel/workflow-launch.ts`) and previously stopped it via `process.kill(-pid, signal)` — a negative-PID group kill, which signals the leader's whole process group. That primitive is POSIX-only: on `win32` it throws, the previous `child.kill()` fallback terminated only the leader, and the agent backends / script children it spawned were **orphaned**. A new portability module `src/runtime/kernel/portability.ts` exports **`killProcessTree(pid, signal)`** and is now the single sanctioned home for group kills. On POSIX it preserves the prior behavior — `process.kill(-pid, signal)` with a per-process fallback when the group no longer exists (`ESRCH`). On `win32` it force-kills the entire tree with `taskkill /pid /T /F` (spawned via an injectable `_portability.spawn` seam, not shelled), degrading to a per-process `process.kill(pid, signal)` if `taskkill` cannot be launched (reported asynchronously via the child's `error` event). Because `taskkill /F` is already a forceful kill with no graceful phase, a follow-up `SIGKILL` escalation after a `SIGTERM`/`SIGINT` is a **documented no-op** on `win32` — the tree is already gone. Every subprocess-termination call site is repointed at the helper: run teardown `terminateRunProcessGroup` (`src/cli/run/lifecycle.ts`, previously the negative-PID group kill) and the two SIGTERM→SIGKILL escalation paths that previously assumed POSIX signal semantics via `child.kill()` — the prompt watchdog's `killChild` (`src/runtime/kernel/prompt.ts`) and the Docker run-timeout kill (`src/runtime/docker.ts`) — so those escalations now terminate the whole child tree instead of just the leader. POSIX Ctrl-C behavior of `jaiph run` is unchanged (leader and children terminate; exit code and cleanup semantics identical), covered by the existing signal-lifecycle e2e. New unit tests (`src/runtime/kernel/portability.test.ts`) cover both platform branches by stubbing `process.platform` (precedent: `src/runtime/docker.test.ts`) and assert the exact mechanism invoked — negative-PID kill on POSIX (plus the ESRCH per-process fallback and SIGKILL escalation), `taskkill /pid /T /F` argv on `win32` (SIGTERM, SIGINT/Ctrl-C, and the degrade-to-per-process path), a proof that the win32 branch **never** calls `process.kill` with a negative PID, and a `src/`-wide lint test asserting no production file outside `portability.ts` matches `process.kill(-`. The watchdog unit tests in `src/runtime/kernel/prompt.test.ts` are updated to spy on `process.kill` (the watchdog now terminates through `killProcessTree`). Docs updated (`docs/architecture.md`, `docs/configuration.md`). # 0.10.0 diff --git a/QUEUE.md b/QUEUE.md index 64f890c3..d41cd418 100644 --- a/QUEUE.md +++ b/QUEUE.md @@ -13,3 +13,107 @@ Process rules: 7. **Acceptance criteria are non-negotiable.** A task is not done until every acceptance bullet is verified by a test that fails when the contract is violated. "It works on my machine" or "the existing tests pass" is not acceptance. *** + +## Portability: single `resolveShell()` seam for inline shell lines and hooks #dev-ready + +Inline workflow shell lines run via hardcoded `spawn("sh", ["-c", ...])` (`executeShLine` in `src/runtime/kernel/node-workflow-runtime.ts`) and hooks do the same (`src/cli/run/hooks.ts`). Jaiph's language semantics require POSIX `sh` on all platforms — inline lines must NOT be translated to cmd/PowerShell, or workflows stop being portable. + +Add `resolveShell(): string` to the portability module: + +* On POSIX: `sh`. +* On `win32`: locate `sh.exe` on `PATH`, then in the standard Git for Windows locations (`/bin/sh.exe`, `/usr/bin/sh.exe`). If none found, throw a Jaiph error with a stable code (e.g. `E_NO_POSIX_SHELL`) telling the user to install Git for Windows. +* Resolution is memoized per process. + +Both call sites go through `resolveShell()`. No other `spawn("sh", ...)` remains in `src/`. + +Acceptance: + +* Unit tests stub `process.platform` and `PATH` lookup: POSIX returns `sh`; win32 returns a discovered `sh.exe` path; win32 with no shell available throws `E_NO_POSIX_SHELL` and the message names Git for Windows. +* A grep-style test asserts no literal `spawn("sh"` / `spawn('sh'` call sites exist in `src/` outside the portability module. +* Inline shell-line semantics on POSIX are unchanged (existing e2e passes). + +## Portability: home directory, Docker gating, and ANSI on win32 #dev-ready + +Remaining small POSIX assumptions for a host-only Windows runtime: + +1. `prepareClaudeEnv` (`src/runtime/kernel/prompt.ts`) reads `execEnv.HOME || process.env.HOME`. Use `os.homedir()` as the final fallback so `USERPROFILE`-only environments resolve; an explicit `HOME` in `execEnv` still wins. +2. Docker sandboxing (`src/runtime/docker.ts`) hardcodes POSIX socket paths and Linux/macOS workspace-presentation branches. On `win32`, the Docker sandbox is out of scope: `resolveDockerConfig` must resolve to host-only mode with a one-line notice (same UX as an explicit `JAIPH_UNSAFE=true`), never attempt `docker` probing, and never hard-fail because Docker is missing. +3. Live status rendering already gates SGR colors on `isTTY` + `NO_COLOR` and uses a single erase/cursor-up sequence (`src/cli/run/stderr-handler.ts`). Node enables VT processing on Windows 10+; no rendering change required. Add a `canUseAnsi()` helper to the portability module and route the existing gates through it so the policy lives in one place. + +Acceptance: + +* Unit test: with `HOME` unset and `os.homedir()` stubbed, `prepareClaudeEnv` resolves the config dir from `os.homedir()`; with `execEnv.HOME` set, it wins. +* Unit test: with `process.platform` stubbed to `win32`, Docker resolution returns host-only mode, emits the notice once, and performs zero `docker` invocations (spawn spied). +* Unit test: `canUseAnsi()` is false when `isTTY` is false or `NO_COLOR` is set; all color/erase emission sites consume it (grep test: no direct `isTTY && NO_COLOR` gating outside the portability module). + +## Distro: build and release `jaiph-windows-x64.exe` #dev-ready + +The release workflow (`.github/workflows/release.yml`) cross-compiles four assets (`jaiph-{darwin,linux}-{arm64,x64}`) via `bun build --compile` and publishes them with `SHA256SUMS`. Add Windows: + +* New matrix entry: `--target=bun-windows-x64`, asset name `jaiph-windows-x64.exe` (Bun has no Windows arm64 target — do not add one). +* Include the `.exe` in `SHA256SUMS` generation and in both the stable and nightly `gh release` upload lists. +* Add a Windows sanity gate alongside the existing linux-x64 one: run `jaiph-windows-x64.exe --version` on a `windows-latest` job and, for stable releases, assert the output matches the tag. +* Update the "Release asset naming contract" in `docs/contributing.md` to include the new asset name. + +Acceptance: + +* A nightly release run publishes five binaries + `SHA256SUMS`, and the `.exe` checksum entry verifies against the downloaded asset. +* The Windows sanity gate fails the release when `--version` output mismatches the tag (verified by test or a deliberate dry-run with a wrong version). +* `docs/contributing.md` naming contract lists `jaiph-windows-x64.exe`; the e2e installer test's asset-name mapping and the contract stay in sync (grep/parity check). + +## Distro: PowerShell installer at `jaiph.org/install.ps1` #dev-ready + +The bash installer (`docs/install`) downloads a per-platform binary from the GitHub Release for a pinned ref and verifies it against `SHA256SUMS`. Windows users need a native equivalent — the bash script must keep rejecting Windows and point at the PowerShell one. + +Add `docs/install.ps1` (served as `https://jaiph.org/install.ps1`, `irm https://jaiph.org/install.ps1 | iex`): + +* Downloads `jaiph-windows-x64.exe` and `SHA256SUMS` from the pinned release ref (default: current stable tag; overridable via `JAIPH_REPO_REF` / first argument, mirroring the bash installer). +* Verifies the SHA-256 (`Get-FileHash`) against `SHA256SUMS`; mismatch aborts and installs nothing. +* Installs to `%LOCALAPPDATA%\jaiph\bin\jaiph.exe` (overridable via `JAIPH_BIN_DIR`), adds the directory to the user `PATH` if absent, and prints the same try-it hints as the bash installer. +* Non-x64 / ARM Windows: exit with a documented unsupported-platform message. +* Update `docs/install`'s unsupported-platform message to mention the PowerShell installer for Windows, and document the Windows install path in `docs/setup.md` and the main page install tabs. + +Acceptance: + +* Pester (or equivalent) tests run on `windows-latest` in CI, pointing the installer at a `file://`-style local release directory (same technique as `e2e/tests/07_installer_binary.sh`): happy path installs and `jaiph --version` works; checksum mismatch exits non-zero and leaves no binary; unsupported arch exits with the documented message. +* The installed binary runs from a shell with no Node/npm/Bun on `PATH`. +* `docs/setup.md` and the main page reference the PowerShell one-liner (docs parity check passes). + +## Distro: main-page install tabs — Windows variant with platform auto-detect #dev-ready + +The main page (`docs/index.html`) hero has three install tabs (run sample / init project / just install), all showing bash `curl … | bash` one-liners. Tab switching in `docs/assets/js/main.js` is click-only; there is no platform detection. Windows visitors currently see commands that cannot run natively. + +Given a published PowerShell installer at `https://jaiph.org/install.ps1` (`irm https://jaiph.org/install.ps1 | iex`): + +* Add a Windows variant to the install section: at minimum the "Just install" panel must offer the PowerShell one-liner alongside the curl one (separate tab, sub-toggle, or swapped command — implementer's choice, but the PowerShell command must be copy-able via the existing copy button). +* Auto-detect the visitor's platform on load (`navigator.userAgentData?.platform` with `navigator.platform` fallback) and default to the Windows variant for Windows visitors; macOS/Linux visitors see exactly today's default. Manual switching always remains possible; no layout shift for non-Windows users. +* Tabs whose commands have no PowerShell equivalent (run sample / init project) must not show a bash-only command as the Windows default — show the install one-liner plus a short "then run:" `jaiph` command instead, or an equivalent honest fallback. +* Keep the static-render constraint: the page must remain correct with JS disabled (bash commands shown, Windows variant reachable via tab markup). + +Acceptance: + +* Playwright docs tests (same suite as the "Try it out" test) assert: with a Windows platform emulated, the page defaults to the PowerShell command; with macOS/Linux emulated, the default is unchanged from today; manual tab switching works in both. +* The copy button on the Windows variant copies the exact `irm … | iex` line (asserted via clipboard stub). +* With JS disabled, all bash commands render and no panel is blank. +* Docs parity check (`.jaiph/docs_parity.jh`) passes with the new content. + +## Distro: native Windows smoke job in CI #dev-ready + +CI's only Windows coverage runs the e2e suite inside WSL (`e2e-wsl` in `.github/workflows/ci.yml`), which exercises the Linux binary. Developing Jaiph on Windows is out of scope — this job proves *running* Jaiph natively works. + +Add a `windows-native-smoke` job on `windows-latest`: + +* Build the standalone Windows binary from the checkout (`bun build --compile --target=bun-windows-x64`). +* With Git for Windows' `sh.exe` available (preinstalled on the runner), run a sample workflow host-only (`JAIPH_UNSAFE=true`) that covers: an inline shell line, a `script` step with a non-bash lang tag (e.g. ` ```node `), string interpolation, and `log` output. +* Assert the process tree is cleaned up after a mid-run cancellation (spawn `jaiph run`, terminate it, assert no orphaned child processes remain). +* No agent-backend credentials in this job: `prompt`-step coverage is limited to the credential pre-flight failing with the documented error, not a hang. +* Keep `e2e-wsl` as-is; do not gate its removal on this task. + +Acceptance: + +* The smoke job is required for merge (listed in the CI gate alongside `test`/`e2e`/`e2e-wsl`). +* Workflow output assertions run against actual `jaiph.exe` stdout (exit code + expected `log` lines). +* The cancellation assertion fails if any child of the workflow leader survives termination. +* The job completes with no WSL usage (fails if `wsl` is invoked). + +*** diff --git a/docs/architecture.md b/docs/architecture.md index d672076b..b3fd5fa7 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -72,6 +72,7 @@ All orchestration — local `jaiph run`, `jaiph test`, and **Docker `jaiph run`* - **Node Workflow Runtime (`src/runtime/kernel/node-workflow-runtime.ts`)** - `NodeWorkflowRuntime` interprets the AST directly: walks workflow steps, manages scope/variables, delegates prompt and script execution to kernel helpers, handles channels/inbox/dispatch, owns the frame stack and heartbeat, and writes run artifacts. + - **Script steps execute via an explicit interpreter, not the shebang + exec bit.** `executeScript` reads the emitted script's shebang line, resolves the interpreter through **`resolveInterpreterFromShebang`** (`src/parse/script-bash.ts`) — `#!/usr/bin/env ` → spawn ``, an absolute-path shebang → spawn that path, a missing shebang → default `bash` — and spawns ` `. This is portable: it does not depend on the OS honoring the shebang (Windows honors neither shebang nor exec bit) or on the file's `0o755` bit (`noexec` mounts strip it). The shebang line is **still** written into every emitted script (they stay directly executable by hand on POSIX), but the runtime never relies on it being honored. A spawn `ENOENT` from a missing interpreter surfaces as a diagnosable Jaiph error naming the interpreter rather than a raw `ENOENT`. - One private `evaluateExpr(scope, expr, …)` dispatcher handles every value position — `const` / `return` / `send` / `say` step handlers and the body of every `exec` step delegate to it. It switches on `Expr.kind` to run the managed call (`call` / `ensure_call` / `inline_script`) or `prompt`, walks a `match` expression, or interpolates a `literal` value through `interpolateWithCaptures`. There is no fan-out across "managed sidecar vs literal value" because that branch is gone from the AST. - **Prompt transport-failure retry.** `runPromptStep` wraps each `executePrompt` invocation in a retry loop driven by the schedule resolved through `src/runtime/kernel/prompt-retry.ts` (default `15s → 1m → 10m → 30m → 2h`, six total attempts; configurable via `JAIPH_PROMPT_RETRY` / `JAIPH_PROMPT_RETRY_DELAYS`). Only the transport path (non-zero exit from the backend) is retried; invalid JSON and schema-validation failures return `{ ok: false }` on the first attempt. Each attempt emits its own `PROMPT_START` / `PROMPT_END` and `STEP_START` / `STEP_END`; each failure (and the final termination) logs a `LOGERR` through `RuntimeEventEmitter.emitLog`. The backoff sleep is injectable (`sleep` constructor option) and interruptible via `runtime.abort()` / an internal `AbortController` so SIGINT and in-process aborts halt the loop without further backend calls. Retry composes **below** `recover` / `catch` — backoff is exhausted before the failure reaches the recover loop. See [Configuration — Prompt retry on transport failure](configuration.md#prompt-retry-on-transport-failure). - Three sibling modules under `src/runtime/kernel/` carry concerns that used to live inline in the runtime file. Dependency direction is one-way (orchestrator → helpers/emitter/mock); no circular imports back. diff --git a/docs/setup.md b/docs/setup.md index e9c1a9e7..e64585bc 100644 --- a/docs/setup.md +++ b/docs/setup.md @@ -15,7 +15,7 @@ The curl installer downloads a per-platform standalone binary from the current s ## Prerequisites -- A POSIX `sh` (the runtime uses `sh -c` for inline shell lines inside workflows; emitted `script` steps follow their own shebang). +- A POSIX `sh` (the runtime uses `sh -c` for inline shell lines inside workflows). Each emitted `script` step runs under the interpreter named by its shebang (`bash` by default), so that interpreter must be on `PATH`; the runtime spawns it explicitly and does not rely on the file's exec bit, so scripts also work under `noexec` mounts. - For the curl installer (step 1): `curl` and either `shasum` or `sha256sum` on `PATH`. - For the npm alternative (step 1): Node.js and npm on the host. diff --git a/src/parse/script-bash.test.ts b/src/parse/script-bash.test.ts new file mode 100644 index 00000000..03f920fd --- /dev/null +++ b/src/parse/script-bash.test.ts @@ -0,0 +1,62 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { resolveInterpreterFromShebang } from "./script-bash"; + +test("resolveInterpreterFromShebang: #!/usr/bin/env bash resolves to bash", () => { + assert.deepEqual(resolveInterpreterFromShebang("#!/usr/bin/env bash"), { + command: "bash", + prefixArgs: [], + }); +}); + +test("resolveInterpreterFromShebang: #!/usr/bin/env node resolves to node", () => { + assert.deepEqual(resolveInterpreterFromShebang("#!/usr/bin/env node"), { + command: "node", + prefixArgs: [], + }); +}); + +test("resolveInterpreterFromShebang: #!/usr/bin/env python3 resolves to python3", () => { + assert.deepEqual(resolveInterpreterFromShebang("#!/usr/bin/env python3"), { + command: "python3", + prefixArgs: [], + }); +}); + +test("resolveInterpreterFromShebang: absolute-path shebang spawns that path", () => { + assert.deepEqual(resolveInterpreterFromShebang("#!/bin/bash"), { + command: "/bin/bash", + prefixArgs: [], + }); +}); + +test("resolveInterpreterFromShebang: custom interpreter shebang resolves to the named interpreter", () => { + assert.deepEqual(resolveInterpreterFromShebang("#!/usr/bin/env my-lang"), { + command: "my-lang", + prefixArgs: [], + }); + assert.deepEqual(resolveInterpreterFromShebang("#!/opt/tools/customlang"), { + command: "/opt/tools/customlang", + prefixArgs: [], + }); +}); + +test("resolveInterpreterFromShebang: interpreter flags after the interpreter are preserved", () => { + assert.deepEqual(resolveInterpreterFromShebang("#!/usr/bin/env node --experimental-vm-modules"), { + command: "node", + prefixArgs: ["--experimental-vm-modules"], + }); +}); + +test("resolveInterpreterFromShebang: env -S split flag is skipped", () => { + assert.deepEqual(resolveInterpreterFromShebang("#!/usr/bin/env -S deno run"), { + command: "deno", + prefixArgs: ["run"], + }); +}); + +test("resolveInterpreterFromShebang: non-shebang line returns null", () => { + assert.equal(resolveInterpreterFromShebang("echo hi"), null); + assert.equal(resolveInterpreterFromShebang(""), null); + assert.equal(resolveInterpreterFromShebang("#!"), null); +}); diff --git a/src/parse/script-bash.ts b/src/parse/script-bash.ts index c4d69251..70d25329 100644 --- a/src/parse/script-bash.ts +++ b/src/parse/script-bash.ts @@ -9,3 +9,42 @@ export function scriptShebangIsBash(shebang?: string): boolean { if (/^#!\/usr\/bin\/env\s+bash(?:\s|$)/.test(t)) return true; return false; } + +/** The interpreter to spawn for a script, plus any leading interpreter flags. */ +export type ScriptInterpreter = { command: string; prefixArgs: string[] }; + +/** + * Resolve the interpreter a script should be executed with from its shebang + * line, so the runtime can spawn ` ` + * explicitly rather than relying on the OS honoring the shebang or the file's + * exec bit (Windows honors neither; `noexec` mounts break the exec bit too). + * + * Supported forms: + * `#!/usr/bin/env bash` -> { command: "bash", prefixArgs: [] } + * `#!/usr/bin/env python3` -> { command: "python3", prefixArgs: [] } + * `#!/bin/bash` -> { command: "/bin/bash", prefixArgs: [] } + * `#!/usr/bin/env node --foo` -> { command: "node", prefixArgs: ["--foo"] } + * `#!/usr/bin/env -S deno run` -> { command: "deno", prefixArgs: ["run"] } + * + * The interpreter is spawned by name so Node resolves it on PATH; a bare name + * that names a missing interpreter surfaces as an ENOENT the caller can turn + * into a diagnosable error. Returns null when the line is not a shebang. + */ +export function resolveInterpreterFromShebang(shebangLine: string): ScriptInterpreter | null { + const t = shebangLine.trim(); + if (!t.startsWith("#!")) return null; + const rest = t.slice(2).trim(); + if (rest === "") return null; + const tokens = rest.split(/\s+/); + const first = tokens[0]!; + // `#!/usr/bin/env [args...]`: the real interpreter is the token + // after `env` (skipping a leading `-S` split flag). Spawn it directly. + if (first === "env" || first.endsWith("/env")) { + let idx = 1; + if (tokens[idx] === "-S") idx += 1; + if (idx >= tokens.length) return null; + return { command: tokens[idx]!, prefixArgs: tokens.slice(idx + 1) }; + } + // `#!/absolute/path/interp [args...]`: spawn that path directly. + return { command: first, prefixArgs: tokens.slice(1) }; +} diff --git a/src/runtime/kernel/node-workflow-runtime.script-exec.test.ts b/src/runtime/kernel/node-workflow-runtime.script-exec.test.ts new file mode 100644 index 00000000..21a44455 --- /dev/null +++ b/src/runtime/kernel/node-workflow-runtime.script-exec.test.ts @@ -0,0 +1,183 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { EventEmitter } from "node:events"; +import { chmodSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { buildRuntimeGraph } from "./graph"; +import { NodeWorkflowRuntime, _scriptSpawn } from "./node-workflow-runtime"; + +/** Minimal fake ChildProcess that emits `close(exitCode)` on the next tick. */ +function fakeChild(exitCode: number): EventEmitter { + const child = new EventEmitter() as EventEmitter & { stdout: EventEmitter; stderr: EventEmitter }; + const makeStream = (): EventEmitter & { setEncoding: () => void } => { + const s = new EventEmitter() as EventEmitter & { setEncoding: () => void }; + s.setEncoding = () => {}; + return s; + }; + child.stdout = makeStream(); + child.stderr = makeStream(); + setImmediate(() => child.emit("close", exitCode)); + return child; +} + +/** Swap `_scriptSpawn.spawn` for a recording stub while `fn` runs. */ +async function withSpawnSpy( + fn: (calls: Array<{ command: string; args: string[] }>) => Promise, +): Promise { + const calls: Array<{ command: string; args: string[] }> = []; + const orig = _scriptSpawn.spawn; + _scriptSpawn.spawn = ((command: string, args: string[]) => { + calls.push({ command, args }); + return fakeChild(0) as unknown as ReturnType; + }) as typeof _scriptSpawn.spawn; + try { + await fn(calls); + } finally { + _scriptSpawn.spawn = orig; + } +} + +function makeRuntime(root: string): { runtime: NodeWorkflowRuntime; env: NodeJS.ProcessEnv; scriptsDir: string } { + const jh = join(root, "flow.jh"); + writeFileSync(jh, ["workflow default() {", ' log "noop"', "}", ""].join("\n")); + const scriptsDir = join(root, "scripts"); + mkdirSync(scriptsDir, { recursive: true }); + const graph = buildRuntimeGraph(jh); + const env: NodeJS.ProcessEnv = { + ...process.env, + JAIPH_TEST_MODE: "1", + JAIPH_RUNS_DIR: join(root, ".jaiph", "runs"), + JAIPH_SCRIPTS: scriptsDir, + JAIPH_WORKSPACE: root, + }; + const runtime = new NodeWorkflowRuntime(graph, { env, cwd: root, suppressLiveEvents: true }); + return { runtime, env, scriptsDir }; +} + +// AC1: executeScript spawns the resolved interpreter with the script path as +// argv[1] (spawn args[0]) — asserted on the spawn call, not on side effects. +const SHEBANG_CASES: Array<{ label: string; shebang: string; expected: string }> = [ + { label: "bash", shebang: "#!/usr/bin/env bash", expected: "bash" }, + { label: "node", shebang: "#!/usr/bin/env node", expected: "node" }, + { label: "python3", shebang: "#!/usr/bin/env python3", expected: "python3" }, + { label: "custom", shebang: "#!/usr/bin/env my-custom-lang", expected: "my-custom-lang" }, +]; + +for (const c of SHEBANG_CASES) { + test(`executeScript: spawns resolved interpreter "${c.expected}" with the script path as argv[1] (${c.label} shebang)`, async () => { + const root = mkdtempSync(join(tmpdir(), "jaiph-script-exec-")); + try { + const { runtime, env, scriptsDir } = makeRuntime(root); + const scriptName = `run_${c.label}`; + const scriptPath = join(scriptsDir, scriptName); + writeFileSync(scriptPath, `${c.shebang}\necho hi\n`); + await withSpawnSpy(async (calls) => { + const result = await (runtime as unknown as { + executeScript: ( + filePath: string, + scriptName: string, + args: string[], + env: NodeJS.ProcessEnv, + ) => Promise<{ status: number }>; + }).executeScript(join(root, "flow.jh"), scriptName, ["a1", "a2"], env); + assert.equal(result.status, 0); + assert.equal(calls.length, 1, "expected exactly one spawn call"); + assert.equal(calls[0]!.command, c.expected, "spawned command is the resolved interpreter"); + assert.equal(calls[0]!.args[0], scriptPath, "script path is argv[1] (spawn args[0])"); + assert.deepEqual(calls[0]!.args, [scriptPath, "a1", "a2"], "script args follow the script path"); + }); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); +} + +// AC2: a script with the exec bit stripped (0o644) still executes on POSIX, +// because the runtime spawns the interpreter explicitly rather than the file. +test("executeScript: script with exec bit stripped (0o644) still executes through the runtime", { skip: process.platform === "win32" }, async () => { + const root = mkdtempSync(join(tmpdir(), "jaiph-script-noexec-")); + try { + const jh = join(root, "flow.jh"); + writeFileSync( + jh, + [ + "script write_marker = ```", + 'printf "ran-ok" > marker.txt', + "```", + "", + "workflow default() {", + " run write_marker()", + "}", + "", + ].join("\n"), + ); + const scriptsDir = join(root, "scripts"); + mkdirSync(scriptsDir, { recursive: true }); + const scriptPath = join(scriptsDir, "write_marker"); + writeFileSync( + scriptPath, + ["#!/usr/bin/env bash", "set -euo pipefail", 'printf "ran-ok" > marker.txt', ""].join("\n"), + ); + // Strip the exec bit: with shebang+exec-bit execution this would fail EACCES. + chmodSync(scriptPath, 0o644); + + const graph = buildRuntimeGraph(jh); + const env: NodeJS.ProcessEnv = { + ...process.env, + JAIPH_TEST_MODE: "1", + JAIPH_RUNS_DIR: join(root, ".jaiph", "runs"), + JAIPH_SCRIPTS: scriptsDir, + JAIPH_WORKSPACE: root, + }; + const runtime = new NodeWorkflowRuntime(graph, { env, cwd: root, suppressLiveEvents: true }); + const status = await runtime.runDefault([]); + assert.equal(status, 0, "workflow succeeded despite stripped exec bit"); + assert.equal(readFileSync(join(root, "marker.txt"), "utf8"), "ran-ok"); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +// AC3: a shebang naming a missing interpreter fails with a diagnosable Jaiph +// error naming the interpreter, not a raw ENOENT. +test("executeScript: missing interpreter fails with a diagnosable error naming the interpreter (not raw ENOENT)", async () => { + const root = mkdtempSync(join(tmpdir(), "jaiph-script-badinterp-")); + try { + const missing = "jaiph-nonexistent-interp-xyz"; + const jh = join(root, "flow.jh"); + writeFileSync( + jh, + [ + "script run_bad = ```", + 'echo "unreachable"', + "```", + "", + "workflow default() {", + " run run_bad()", + "}", + "", + ].join("\n"), + ); + const scriptsDir = join(root, "scripts"); + mkdirSync(scriptsDir, { recursive: true }); + writeFileSync(join(scriptsDir, "run_bad"), `#!/usr/bin/env ${missing}\necho unreachable\n`); + + const graph = buildRuntimeGraph(jh); + const env: NodeJS.ProcessEnv = { + ...process.env, + JAIPH_TEST_MODE: "1", + JAIPH_RUNS_DIR: join(root, ".jaiph", "runs"), + JAIPH_SCRIPTS: scriptsDir, + JAIPH_WORKSPACE: root, + }; + const runtime = new NodeWorkflowRuntime(graph, { env, cwd: root, suppressLiveEvents: true }); + const result = await runtime.runNamedWorkflow("default", []); + assert.equal(result.status, 1, "workflow failed on missing interpreter"); + const message = `${result.output ?? ""}${result.error ?? ""}`; + assert.match(message, new RegExp(missing), "error names the missing interpreter"); + assert.doesNotMatch(message, /ENOENT/, "error is not a raw ENOENT"); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); diff --git a/src/runtime/kernel/node-workflow-runtime.ts b/src/runtime/kernel/node-workflow-runtime.ts index c2012aa6..42b3305c 100644 --- a/src/runtime/kernel/node-workflow-runtime.ts +++ b/src/runtime/kernel/node-workflow-runtime.ts @@ -1,5 +1,5 @@ import { spawn } from "node:child_process"; -import { appendFileSync, mkdirSync, writeFileSync } from "node:fs"; +import { appendFileSync, mkdirSync, readFileSync, writeFileSync } from "node:fs"; import { basename, dirname, join, resolve as resolvePath } from "node:path"; import { PassThrough } from "node:stream"; import { randomUUID } from "node:crypto"; @@ -27,6 +27,7 @@ import { stripOuterQuotes, type PromptSchemaField, } from "./runtime-arg-parser"; +import { resolveInterpreterFromShebang } from "../../parse/script-bash"; import { RuntimeEventEmitter, type Frame } from "./runtime-event-emitter"; import { executeMockBodyDef, type MockBodyDef, type StepResult } from "./runtime-mock"; import { linesOfDelimitedString } from "../string-lines"; @@ -42,6 +43,13 @@ export type { MockBodyDef } from "./runtime-mock"; const HANDLE_PREFIX = "__JAIPH_HANDLE__"; +/** + * Test seam for the script/shell subprocess spawn. Swapped out in unit tests so + * the resolved interpreter + argv can be asserted on the spawn call itself + * without side effects (mirrors `_portability.spawn` in `portability.ts`). + */ +export const _scriptSpawn = { spawn }; + export function formatInvalidAsyncHandleError(handleId: string): string { return `invalid async handle "${handleId}" — the handle was never created or was already consumed`; } @@ -1479,16 +1487,22 @@ export class NodeWorkflowRuntime { return { status: 0, output: res.output, error: "" }; } - /** Spawn a child process, stream stdout/stderr into io and collect them into the StepResult. */ + /** + * Spawn a child process, stream stdout/stderr into io and collect them into + * the StepResult. When `interpreter` is set, a spawn ENOENT (the interpreter + * binary is missing on PATH) is turned into a diagnosable Jaiph error naming + * the interpreter instead of a raw `spawn ENOENT`. + */ private spawnAndCapture( command: string, args: string[], env: NodeJS.ProcessEnv, cwd: string, io: StepIO | undefined, + interpreter?: string, ): Promise { return new Promise((resolve) => { - const child = spawn(command, args, { cwd, env, stdio: ["ignore", "pipe", "pipe"] }); + const child = _scriptSpawn.spawn(command, args, { cwd, env, stdio: ["ignore", "pipe", "pipe"] }); let output = ""; let error = ""; child.stdout?.setEncoding("utf8"); @@ -1502,7 +1516,10 @@ export class NodeWorkflowRuntime { io?.appendErr(chunk); }); child.on("error", (err) => { - const msg = err instanceof Error ? err.message : String(err); + const code = (err as NodeJS.ErrnoException).code; + const msg = code === "ENOENT" && interpreter + ? `script interpreter "${interpreter}" not found — install it or fix the script shebang` + : err instanceof Error ? err.message : String(err); error += msg; io?.appendErr(msg); resolve({ status: 1, output, error }); @@ -1536,7 +1553,43 @@ export class NodeWorkflowRuntime { if (!scriptsDir) { return { status: 1, output: "", error: "JAIPH_SCRIPTS not set for script execution" }; } - return this.spawnAndCapture(join(scriptsDir, scriptName), args, env, this.scriptCwd(env, filePath), io); + const scriptPath = join(scriptsDir, scriptName); + const interp = this.resolveScriptInterpreter(scriptPath); + if (!interp.ok) return { status: 1, output: "", error: interp.error }; + // Spawn ` ` explicitly. This does not + // depend on the OS honoring the shebang line (Windows) or on the file's + // exec bit (stripped bit / `noexec` mounts); the shebang is still written + // into the file so it stays directly executable by hand on POSIX. + return this.spawnAndCapture( + interp.command, + [...interp.prefixArgs, scriptPath, ...args], + env, + this.scriptCwd(env, filePath), + io, + interp.command, + ); + } + + /** + * Resolve the interpreter to spawn for an emitted script from its shebang + * line. Emitted scripts always carry a shebang (`buildScriptFiles`); a script + * without one falls back to bash (Jaiph's default script language) rather + * than depending on the OS exec bit. + */ + private resolveScriptInterpreter( + scriptPath: string, + ): { ok: true; command: string; prefixArgs: string[] } | { ok: false; error: string } { + let firstLine: string; + try { + const content = readFileSync(scriptPath, "utf8"); + const nl = content.indexOf("\n"); + firstLine = nl === -1 ? content : content.slice(0, nl); + } catch { + return { ok: false, error: `script file not found or unreadable: ${scriptPath}` }; + } + const interp = resolveInterpreterFromShebang(firstLine); + if (!interp) return { ok: true, command: "bash", prefixArgs: [] }; + return { ok: true, command: interp.command, prefixArgs: interp.prefixArgs }; } /** From 7a0f2e4d64953ba9e32ec9d252e2cd89391b2ccb Mon Sep 17 00:00:00 2001 From: Jakub Dzikowski Date: Mon, 13 Jul 2026 13:35:22 +0200 Subject: [PATCH 03/85] Fix: route inline shell lines and hooks through resolveShell() seam Inline workflow shell lines (executeShLine) and CLI hook commands used a hardcoded spawn("sh", ["-c", ...]), which fails on win32 where there is no sh on the default PATH. Add resolveShell() to the portability module as the single seam both call sites go through: on POSIX it returns bare sh; on win32 it locates Git for Windows' bundled sh.exe, first on PATH and then in the standard install layouts (/bin/sh.exe, /usr/bin/sh.exe) under each known root, throwing a diagnosable E_NO_POSIX_SHELL error naming Git for Windows if none is found. Resolution is memoized per process. Inline lines are never translated to cmd/PowerShell, so POSIX shell semantics are unchanged. Unit tests stub process.platform and the PATH/existence lookup across all branches, and a src/-wide lint test asserts no spawn("sh") call site remains outside portability.ts. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 1 + QUEUE.md | 18 --- docs/architecture.md | 1 + src/cli/run/hooks.ts | 3 +- src/runtime/kernel/node-workflow-runtime.ts | 3 +- src/runtime/kernel/portability.test.ts | 140 +++++++++++++++++++- src/runtime/kernel/portability.ts | 63 ++++++++- 7 files changed, 206 insertions(+), 23 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e6418d99..875ce769 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,7 @@ ## All changes +- **Fix — Portability: single `resolveShell()` seam for inline shell lines and hooks:** Inline workflow shell lines (`executeShLine` in `src/runtime/kernel/node-workflow-runtime.ts`) and hook commands (`src/cli/run/hooks.ts`) used to run via a hardcoded `spawn("sh", ["-c", …])`. That works on POSIX but fails on `win32`, where there is no `sh` on the default `PATH`. A new **`resolveShell(): string`** in the portability module (`src/runtime/kernel/portability.ts`) is now the single seam both call sites go through. On POSIX it returns bare `sh`. On `win32` it locates Git for Windows' bundled `sh.exe` — first on `PATH`, then in the standard install layouts (`/bin/sh.exe`, `/usr/bin/sh.exe`) under each known root (`%ProgramFiles%`, `%ProgramFiles(x86)%`, `%ProgramW6432%`, plus the default `C:\Program Files` / `C:\Program Files (x86)`) — and throws a diagnosable Jaiph error with the stable code **`E_NO_POSIX_SHELL`** naming Git for Windows (https://git-scm.com/download/win) as the fix if none is found. Resolution is memoized for the lifetime of the process. Crucially, inline lines are **never** translated to `cmd`/PowerShell — Jaiph's language semantics require POSIX `sh` on every platform, or workflows stop being portable — so the seam only ever resolves *which* `sh` to run, never rewrites the command. POSIX inline-shell semantics are unchanged (existing e2e passes). New unit tests (`src/runtime/kernel/portability.test.ts`) stub `process.platform` and the `PATH` / existence lookup: POSIX returns `sh`, `win32` returns a discovered `sh.exe` path (both the on-`PATH` and Git-for-Windows-fallback branches), `win32` with no shell available throws `E_NO_POSIX_SHELL` with a message naming Git for Windows, and the result is memoized (the existence probe is not consulted on a second call); a `src/`-wide lint test asserts no production file outside `portability.ts` calls `spawn("sh"` / `spawn('sh'` directly. Docs updated (`docs/architecture.md`). - **Fix — Portability: execute script steps via an explicit interpreter, not shebang + exec bit:** Script steps used to run by spawning the emitted file directly (`executeScript` in `src/runtime/kernel/node-workflow-runtime.ts`), relying on the `#!/usr/bin/env ` shebang line and the `0o755` bit set by `src/transpile/build.ts`. Windows honors neither, and `noexec` mounts on Linux strip the exec bit — both break script execution. The runtime already knows the interpreter (it writes the shebang itself), so `executeScript` now reads the emitted script's shebang and resolves the interpreter through the new **`resolveInterpreterFromShebang`** (`src/parse/script-bash.ts`) — `#!/usr/bin/env ` → spawn ``, an absolute-path shebang (`#!/bin/bash`) → spawn that path, an `env -S` split-flag form → the real interpreter after `-S`, a missing/blank shebang → default `bash` — then spawns ` ` explicitly. The shebang line is **still** written into every emitted script (they stay directly executable by hand on POSIX) and the `0o755` bit is still set, but the runtime no longer depends on either being honored by the OS. A spawn `ENOENT` from an interpreter missing on `PATH` now surfaces as a diagnosable Jaiph error naming the interpreter (`script interpreter "" not found — install it or fix the script shebang`) instead of a raw `spawn ENOENT`. New unit tests assert the resolved interpreter + argv on the spawn call itself through an injectable `_scriptSpawn` seam (`src/runtime/kernel/node-workflow-runtime.script-exec.test.ts`), prove that a script with the exec bit stripped (`0o644`) still runs on POSIX, and prove that a missing-interpreter shebang produces the diagnosable error; `resolveInterpreterFromShebang` gets its own unit coverage (`src/parse/script-bash.test.ts`). Docs updated (`docs/architecture.md`, `docs/setup.md`). - **Fix — Portability: cross-platform process-tree termination:** `jaiph run` spawns the workflow leader with `detached: true` (`src/runtime/kernel/workflow-launch.ts`) and previously stopped it via `process.kill(-pid, signal)` — a negative-PID group kill, which signals the leader's whole process group. That primitive is POSIX-only: on `win32` it throws, the previous `child.kill()` fallback terminated only the leader, and the agent backends / script children it spawned were **orphaned**. A new portability module `src/runtime/kernel/portability.ts` exports **`killProcessTree(pid, signal)`** and is now the single sanctioned home for group kills. On POSIX it preserves the prior behavior — `process.kill(-pid, signal)` with a per-process fallback when the group no longer exists (`ESRCH`). On `win32` it force-kills the entire tree with `taskkill /pid /T /F` (spawned via an injectable `_portability.spawn` seam, not shelled), degrading to a per-process `process.kill(pid, signal)` if `taskkill` cannot be launched (reported asynchronously via the child's `error` event). Because `taskkill /F` is already a forceful kill with no graceful phase, a follow-up `SIGKILL` escalation after a `SIGTERM`/`SIGINT` is a **documented no-op** on `win32` — the tree is already gone. Every subprocess-termination call site is repointed at the helper: run teardown `terminateRunProcessGroup` (`src/cli/run/lifecycle.ts`, previously the negative-PID group kill) and the two SIGTERM→SIGKILL escalation paths that previously assumed POSIX signal semantics via `child.kill()` — the prompt watchdog's `killChild` (`src/runtime/kernel/prompt.ts`) and the Docker run-timeout kill (`src/runtime/docker.ts`) — so those escalations now terminate the whole child tree instead of just the leader. POSIX Ctrl-C behavior of `jaiph run` is unchanged (leader and children terminate; exit code and cleanup semantics identical), covered by the existing signal-lifecycle e2e. New unit tests (`src/runtime/kernel/portability.test.ts`) cover both platform branches by stubbing `process.platform` (precedent: `src/runtime/docker.test.ts`) and assert the exact mechanism invoked — negative-PID kill on POSIX (plus the ESRCH per-process fallback and SIGKILL escalation), `taskkill /pid /T /F` argv on `win32` (SIGTERM, SIGINT/Ctrl-C, and the degrade-to-per-process path), a proof that the win32 branch **never** calls `process.kill` with a negative PID, and a `src/`-wide lint test asserting no production file outside `portability.ts` matches `process.kill(-`. The watchdog unit tests in `src/runtime/kernel/prompt.test.ts` are updated to spy on `process.kill` (the watchdog now terminates through `killProcessTree`). Docs updated (`docs/architecture.md`, `docs/configuration.md`). diff --git a/QUEUE.md b/QUEUE.md index d41cd418..c02bca1e 100644 --- a/QUEUE.md +++ b/QUEUE.md @@ -14,24 +14,6 @@ Process rules: *** -## Portability: single `resolveShell()` seam for inline shell lines and hooks #dev-ready - -Inline workflow shell lines run via hardcoded `spawn("sh", ["-c", ...])` (`executeShLine` in `src/runtime/kernel/node-workflow-runtime.ts`) and hooks do the same (`src/cli/run/hooks.ts`). Jaiph's language semantics require POSIX `sh` on all platforms — inline lines must NOT be translated to cmd/PowerShell, or workflows stop being portable. - -Add `resolveShell(): string` to the portability module: - -* On POSIX: `sh`. -* On `win32`: locate `sh.exe` on `PATH`, then in the standard Git for Windows locations (`/bin/sh.exe`, `/usr/bin/sh.exe`). If none found, throw a Jaiph error with a stable code (e.g. `E_NO_POSIX_SHELL`) telling the user to install Git for Windows. -* Resolution is memoized per process. - -Both call sites go through `resolveShell()`. No other `spawn("sh", ...)` remains in `src/`. - -Acceptance: - -* Unit tests stub `process.platform` and `PATH` lookup: POSIX returns `sh`; win32 returns a discovered `sh.exe` path; win32 with no shell available throws `E_NO_POSIX_SHELL` and the message names Git for Windows. -* A grep-style test asserts no literal `spawn("sh"` / `spawn('sh'` call sites exist in `src/` outside the portability module. -* Inline shell-line semantics on POSIX are unchanged (existing e2e passes). - ## Portability: home directory, Docker gating, and ANSI on win32 #dev-ready Remaining small POSIX assumptions for a host-only Windows runtime: diff --git a/docs/architecture.md b/docs/architecture.md index b3fd5fa7..c6f5c6a0 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -73,6 +73,7 @@ All orchestration — local `jaiph run`, `jaiph test`, and **Docker `jaiph run`* - **Node Workflow Runtime (`src/runtime/kernel/node-workflow-runtime.ts`)** - `NodeWorkflowRuntime` interprets the AST directly: walks workflow steps, manages scope/variables, delegates prompt and script execution to kernel helpers, handles channels/inbox/dispatch, owns the frame stack and heartbeat, and writes run artifacts. - **Script steps execute via an explicit interpreter, not the shebang + exec bit.** `executeScript` reads the emitted script's shebang line, resolves the interpreter through **`resolveInterpreterFromShebang`** (`src/parse/script-bash.ts`) — `#!/usr/bin/env ` → spawn ``, an absolute-path shebang → spawn that path, a missing shebang → default `bash` — and spawns ` `. This is portable: it does not depend on the OS honoring the shebang (Windows honors neither shebang nor exec bit) or on the file's `0o755` bit (`noexec` mounts strip it). The shebang line is **still** written into every emitted script (they stay directly executable by hand on POSIX), but the runtime never relies on it being honored. A spawn `ENOENT` from a missing interpreter surfaces as a diagnosable Jaiph error naming the interpreter rather than a raw `ENOENT`. + - **Inline shell lines resolve their shell through one portable seam.** A single-line shell step (`executeShLine`) and CLI hook commands (`src/cli/run/hooks.ts`) both run under POSIX `sh -c`, but the shell itself is resolved through **`resolveShell()`** (`src/runtime/kernel/portability.ts`) rather than a hardcoded `spawn("sh", …)`. On POSIX this is bare `sh`; on **`win32`**, where there is no `sh` on the default `PATH`, it discovers Git for Windows' bundled `sh.exe` — first on `PATH`, then in the standard install layouts (`/bin/sh.exe`, `/usr/bin/sh.exe`) under each known root — memoizes the result for the process, and throws a diagnosable **`E_NO_POSIX_SHELL`** error naming Git for Windows if none is found. Inline lines are **never** translated to `cmd`/PowerShell: Jaiph's shell semantics are POSIX `sh` on every platform, so the seam only ever chooses *which* `sh` to invoke, never rewrites the command — otherwise workflows would stop being portable. `resolveShell()` is the single call site for the POSIX shell; no other `spawn("sh", …)` remains in `src/`. - One private `evaluateExpr(scope, expr, …)` dispatcher handles every value position — `const` / `return` / `send` / `say` step handlers and the body of every `exec` step delegate to it. It switches on `Expr.kind` to run the managed call (`call` / `ensure_call` / `inline_script`) or `prompt`, walks a `match` expression, or interpolates a `literal` value through `interpolateWithCaptures`. There is no fan-out across "managed sidecar vs literal value" because that branch is gone from the AST. - **Prompt transport-failure retry.** `runPromptStep` wraps each `executePrompt` invocation in a retry loop driven by the schedule resolved through `src/runtime/kernel/prompt-retry.ts` (default `15s → 1m → 10m → 30m → 2h`, six total attempts; configurable via `JAIPH_PROMPT_RETRY` / `JAIPH_PROMPT_RETRY_DELAYS`). Only the transport path (non-zero exit from the backend) is retried; invalid JSON and schema-validation failures return `{ ok: false }` on the first attempt. Each attempt emits its own `PROMPT_START` / `PROMPT_END` and `STEP_START` / `STEP_END`; each failure (and the final termination) logs a `LOGERR` through `RuntimeEventEmitter.emitLog`. The backoff sleep is injectable (`sleep` constructor option) and interruptible via `runtime.abort()` / an internal `AbortController` so SIGINT and in-process aborts halt the loop without further backend calls. Retry composes **below** `recover` / `catch` — backoff is exhausted before the failure reaches the recover loop. See [Configuration — Prompt retry on transport failure](configuration.md#prompt-retry-on-transport-failure). - Three sibling modules under `src/runtime/kernel/` carry concerns that used to live inline in the runtime file. Dependency direction is one-way (orchestrator → helpers/emitter/mock); no circular imports back. diff --git a/src/cli/run/hooks.ts b/src/cli/run/hooks.ts index 5cc2a95e..e3af0305 100644 --- a/src/cli/run/hooks.ts +++ b/src/cli/run/hooks.ts @@ -2,6 +2,7 @@ import { existsSync, readFileSync } from "node:fs"; import { homedir } from "node:os"; import { join } from "node:path"; import { spawn } from "node:child_process"; +import { resolveShell } from "../../runtime/kernel/portability"; import type { HookConfig, HookEventName, HookPayload } from "../../types"; import type { RunEmitter } from "./emitter"; @@ -133,7 +134,7 @@ export function runHooksForEvent( for (const cmd of commands) { try { - const child = spawn("sh", ["-c", cmd], { + const child = spawn(resolveShell(), ["-c", cmd], { stdio: ["pipe", "ignore", "pipe"], env: { ...process.env }, }); diff --git a/src/runtime/kernel/node-workflow-runtime.ts b/src/runtime/kernel/node-workflow-runtime.ts index 42b3305c..4e8b923e 100644 --- a/src/runtime/kernel/node-workflow-runtime.ts +++ b/src/runtime/kernel/node-workflow-runtime.ts @@ -28,6 +28,7 @@ import { type PromptSchemaField, } from "./runtime-arg-parser"; import { resolveInterpreterFromShebang } from "../../parse/script-bash"; +import { resolveShell } from "./portability"; import { RuntimeEventEmitter, type Frame } from "./runtime-event-emitter"; import { executeMockBodyDef, type MockBodyDef, type StepResult } from "./runtime-mock"; import { linesOfDelimitedString } from "../string-lines"; @@ -1597,7 +1598,7 @@ export class NodeWorkflowRuntime { * the workspace, matching script cwd semantics. */ private executeShLine(scope: Scope, command: string, io: StepIO): Promise { - return this.spawnAndCapture("sh", ["-c", command], scope.env, this.scriptCwd(scope.env, scope.filePath), io); + return this.spawnAndCapture(resolveShell(), ["-c", command], scope.env, this.scriptCwd(scope.env, scope.filePath), io); } private async executeInlineScript( diff --git a/src/runtime/kernel/portability.test.ts b/src/runtime/kernel/portability.test.ts index 6ee257c6..f95a3476 100644 --- a/src/runtime/kernel/portability.test.ts +++ b/src/runtime/kernel/portability.test.ts @@ -3,7 +3,7 @@ import assert from "node:assert/strict"; import { EventEmitter } from "node:events"; import { readFileSync, readdirSync } from "node:fs"; import { resolve, join } from "node:path"; -import { killProcessTree, _portability } from "./portability"; +import { killProcessTree, resolveShell, _portability } from "./portability"; // Precedent for platform stubbing: src/runtime/docker.test.ts. function withPlatform(platform: string, fn: () => void): void { @@ -163,6 +163,144 @@ test("win32: NEVER calls process.kill with a negative PID (SIGTERM, SIGINT, SIGK }); }); +// --------------------------------------------------------------------------- +// resolveShell(): single POSIX-sh seam for inline shell lines and hooks. +// --------------------------------------------------------------------------- + +/** Stub `_portability.fileExists` and `process.env` while `fn` runs. */ +function withShellEnv( + opts: { exists: (path: string) => boolean; env: Record }, + fn: () => void, +): void { + const origExists = _portability.fileExists; + const savedEnv: Record = {}; + for (const key of Object.keys(opts.env)) { + savedEnv[key] = process.env[key]; + if (opts.env[key] === undefined) delete process.env[key]; + else process.env[key] = opts.env[key]; + } + _portability.fileExists = opts.exists; + _portability.resetShellCache(); + try { + fn(); + } finally { + _portability.fileExists = origExists; + for (const key of Object.keys(savedEnv)) { + if (savedEnv[key] === undefined) delete process.env[key]; + else process.env[key] = savedEnv[key]; + } + _portability.resetShellCache(); + } +} + +test("POSIX: resolveShell returns bare `sh`", () => { + withPlatform("linux", () => { + _portability.resetShellCache(); + try { + assert.equal(resolveShell(), "sh"); + } finally { + _portability.resetShellCache(); + } + }); +}); + +test("win32: resolveShell returns an sh.exe discovered on PATH", () => { + withPlatform("win32", () => { + // Compute the expected path with the same `join` production uses so the + // assertion is host-separator-independent (POSIX CI vs a real win32 host). + const onPath = join("C:\\git\\bin", "sh.exe"); + withShellEnv( + { + env: { PATH: "C:\\tools;C:\\git\\bin", ProgramFiles: undefined, "ProgramFiles(x86)": undefined, ProgramW6432: undefined }, + exists: (p) => p === onPath, + }, + () => { + assert.equal(resolveShell(), onPath); + }, + ); + }); +}); + +test("win32: resolveShell falls back to the standard Git for Windows location", () => { + withPlatform("win32", () => { + // Only the Git usr/bin layout exists; nothing on PATH. + const gitUsrBin = join("C:\\Program Files", "Git", "usr", "bin", "sh.exe"); + withShellEnv( + { + env: { PATH: "C:\\tools", ProgramFiles: "C:\\Program Files", "ProgramFiles(x86)": undefined, ProgramW6432: undefined }, + exists: (p) => p === gitUsrBin, + }, + () => { + assert.equal(resolveShell(), gitUsrBin); + }, + ); + }); +}); + +test("win32: resolveShell throws E_NO_POSIX_SHELL naming Git for Windows when no sh.exe exists", () => { + withPlatform("win32", () => { + withShellEnv( + { + env: { PATH: "C:\\tools", ProgramFiles: "C:\\Program Files", "ProgramFiles(x86)": undefined, ProgramW6432: undefined }, + exists: () => false, + }, + () => { + assert.throws( + () => resolveShell(), + (err: Error) => { + assert.match(err.message, /E_NO_POSIX_SHELL/); + assert.match(err.message, /Git for Windows/); + return true; + }, + ); + }, + ); + }); +}); + +test("resolveShell memoizes: fileExists is not consulted on the second call", () => { + withPlatform("win32", () => { + const onPath = join("C:\\git\\bin", "sh.exe"); + let calls = 0; + withShellEnv( + { + env: { PATH: "C:\\git\\bin", ProgramFiles: undefined, "ProgramFiles(x86)": undefined, ProgramW6432: undefined }, + exists: (p) => { + calls++; + return p === onPath; + }, + }, + () => { + assert.equal(resolveShell(), onPath); + const after = calls; + assert.equal(resolveShell(), onPath); + assert.equal(calls, after, "second call is served from the memoized cache"); + }, + ); + }); +}); + +// --------------------------------------------------------------------------- +// Lint contract: no `spawn("sh", …)` outside the portability module. +// --------------------------------------------------------------------------- + +test("no production source file invokes spawn(\"sh\", …) directly — all go through resolveShell", () => { + // Match spawn("sh" / spawn('sh' with optional whitespace after the paren. + const rawShSpawn = /spawn\(\s*['"]sh['"]/; + const offenders: string[] = []; + for (const file of walkProductionTsFiles(SRC_ROOT)) { + const rel = file.slice(REPO_ROOT.length + 1); + if (rel === join("src", "runtime", "kernel", "portability.ts")) continue; + const content = readFileSync(file, "utf8"); + if (rawShSpawn.test(content)) offenders.push(rel); + } + assert.deepEqual( + offenders, + [], + `inline shell must resolve through resolveShell(); offenders: ${offenders.join(", ")}`, + ); +}); + // --------------------------------------------------------------------------- // Lint contract: only the portability module may group-kill via negative PID. // --------------------------------------------------------------------------- diff --git a/src/runtime/kernel/portability.ts b/src/runtime/kernel/portability.ts index a012ef90..d8e8be76 100644 --- a/src/runtime/kernel/portability.ts +++ b/src/runtime/kernel/portability.ts @@ -13,17 +13,76 @@ // `killProcessTree` hides that difference behind one call site. import { spawn, type ChildProcess } from "node:child_process"; +import { existsSync } from "node:fs"; +import { join } from "node:path"; /** - * Test seam for the `taskkill` spawn. Swapped out in unit tests so the win32 - * branch can be exercised on any host without a real `taskkill` binary. + * Test seams. Swapped out in unit tests so the win32 branches can be exercised + * on any host without a real `taskkill` binary or a real `sh.exe` on disk. */ export const _portability = { spawn(command: string, args: string[]): ChildProcess { return spawn(command, args, { stdio: "ignore" }); }, + /** Existence probe for `resolveShell` PATH / Git-for-Windows lookups. */ + fileExists(path: string): boolean { + return existsSync(path); + }, + /** Reset the memoized shell so a test can re-run resolution under a new platform. */ + resetShellCache(): void { + cachedShell = undefined; + }, }; +let cachedShell: string | undefined; + +/** + * Resolve the POSIX shell used to run inline workflow shell lines and hooks. + * + * Jaiph's language semantics require POSIX `sh` on every platform — inline + * lines are never translated to cmd/PowerShell, or workflows stop being + * portable. On POSIX the answer is simply `sh`. On win32 there is no `sh` on + * the default PATH, so we locate Git for Windows' bundled `sh.exe`, first on + * PATH and then in the standard install locations. The result is memoized for + * the lifetime of the process. + */ +export function resolveShell(): string { + if (cachedShell !== undefined) return cachedShell; + cachedShell = process.platform === "win32" ? resolveWindowsPosixShell() : "sh"; + return cachedShell; +} + +function resolveWindowsPosixShell(): string { + // PATH first: honor an sh.exe the user already put on their PATH. + const path = process.env.PATH ?? ""; + for (const dir of path.split(";")) { + if (!dir) continue; + const candidate = join(dir, "sh.exe"); + if (_portability.fileExists(candidate)) return candidate; + } + // Then the standard Git for Windows layouts under each known install root. + const roots = [ + process.env.ProgramFiles, + process.env["ProgramFiles(x86)"], + process.env.ProgramW6432, + "C:\\Program Files", + "C:\\Program Files (x86)", + ]; + for (const root of roots) { + if (!root) continue; + for (const rel of [join("Git", "bin", "sh.exe"), join("Git", "usr", "bin", "sh.exe")]) { + const candidate = join(root, rel); + if (_portability.fileExists(candidate)) return candidate; + } + } + throw new Error( + "E_NO_POSIX_SHELL Jaiph requires a POSIX `sh` to run inline shell lines and hooks, " + + "but `sh.exe` was not found on PATH or in the standard Git for Windows install " + + "locations. Install Git for Windows (https://git-scm.com/download/win), which " + + "bundles `sh.exe`.", + ); +} + /** * Terminate `pid` and its descendants with `signal`, portably. * From 2b1922df3f659fd0cd20c9f3a31fec96590d5c5c Mon Sep 17 00:00:00 2001 From: Jakub Dzikowski Date: Mon, 13 Jul 2026 13:55:59 +0200 Subject: [PATCH 04/85] Fix: portability home-dir fallback, Docker gating on win32, ANSI policy Close three remaining POSIX assumptions that broke a host-only Windows runtime. prepareClaudeEnv now falls back to os.homedir() when neither execEnv.HOME nor process.env.HOME is set, so USERPROFILE-only environments resolve the .claude config dir; an explicit execEnv.HOME still wins. resolveDockerConfig forces host-only mode on win32 with a one-line notice (same UX as JAIPH_UNSAFE=true), so the CLI never probes docker and never hard-fails on a missing daemon, and JAIPH_DOCKER_ENABLED=true cannot override it. A new canUseAnsi() helper in the portability module centralizes the isTTY + NO_COLOR gate; every color/erase emission site routes through it so the policy lives in one place. Adds unit tests for each and a src-wide lint test asserting no production file outside portability.ts gates directly on isTTY && NO_COLOR. Docs updated. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 1 + QUEUE.md | 14 ------ docs/architecture.md | 4 +- docs/configuration.md | 5 +- docs/env-vars.md | 4 +- docs/sandboxing.md | 4 ++ src/cli/commands/run.ts | 3 +- src/cli/run/progress.ts | 9 ++-- src/cli/shared/errors.ts | 3 +- src/runtime/docker.test.ts | 38 ++++++++++++++++ src/runtime/docker.ts | 34 ++++++++++++-- src/runtime/kernel/portability.test.ts | 63 +++++++++++++++++++++++++- src/runtime/kernel/portability.ts | 16 +++++++ src/runtime/kernel/prompt.test.ts | 46 +++++++++++++++++++ src/runtime/kernel/prompt.ts | 5 +- 15 files changed, 219 insertions(+), 30 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 875ce769..102f19dc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,7 @@ ## All changes +- **Fix — Portability: home-dir fallback, Docker gating on Windows, and a single ANSI policy:** Three remaining POSIX assumptions that broke a host-only `win32` runtime are closed. (1) **Home directory** — `prepareClaudeEnv` (`src/runtime/kernel/prompt.ts`) resolved the `.claude` config dir from `execEnv.HOME || process.env.HOME`, which is empty in a `USERPROFILE`-only Windows environment. It now falls back to `os.homedir()` as a final source (`execEnv.HOME || process.env.HOME || homedir()`); an explicit `HOME` in `execEnv` still wins. (2) **Docker gating** — the Docker sandbox hardcodes POSIX socket paths and Linux/macOS workspace-presentation branches, so it is out of scope on Windows. `resolveDockerConfig` (`src/runtime/docker.ts`) now forces **host-only mode** on `win32` (same UX as an explicit `JAIPH_UNSAFE=true`) with a one-line notice emitted once per process, so the CLI never probes `docker` and never hard-fails on a missing daemon; `JAIPH_DOCKER_ENABLED=true` cannot override this. (3) **ANSI colors** — a new **`canUseAnsi(stream?)`** helper in the portability module (`src/runtime/kernel/portability.ts`) returns `isTTY && NO_COLOR`-unset, and every color/erase emission site (`src/cli/commands/run.ts`, `src/cli/run/progress.ts`, `src/cli/shared/errors.ts`) routes its gate through it so the policy lives in one place; no rendering change is needed because Node enables console VT processing on Windows 10+ automatically, making `isTTY` a sufficient ANSI proxy. New unit tests: `prepareClaudeEnv` resolves from a stubbed `os.homedir()` when `HOME` is unset and prefers `execEnv.HOME` when set (`src/runtime/kernel/prompt.test.ts`); `resolveDockerConfig` under a stubbed `win32` returns host-only, emits the notice once, and performs zero `docker` invocations (`src/runtime/docker.test.ts`); `canUseAnsi()` is false when `isTTY` is false or `NO_COLOR` is set, plus a `src/`-wide lint test asserting no production file outside `portability.ts` gates directly on `isTTY && NO_COLOR` (`src/runtime/kernel/portability.test.ts`). Docs updated (`docs/architecture.md`, `docs/sandboxing.md`, `docs/configuration.md`, `docs/env-vars.md`). - **Fix — Portability: single `resolveShell()` seam for inline shell lines and hooks:** Inline workflow shell lines (`executeShLine` in `src/runtime/kernel/node-workflow-runtime.ts`) and hook commands (`src/cli/run/hooks.ts`) used to run via a hardcoded `spawn("sh", ["-c", …])`. That works on POSIX but fails on `win32`, where there is no `sh` on the default `PATH`. A new **`resolveShell(): string`** in the portability module (`src/runtime/kernel/portability.ts`) is now the single seam both call sites go through. On POSIX it returns bare `sh`. On `win32` it locates Git for Windows' bundled `sh.exe` — first on `PATH`, then in the standard install layouts (`/bin/sh.exe`, `/usr/bin/sh.exe`) under each known root (`%ProgramFiles%`, `%ProgramFiles(x86)%`, `%ProgramW6432%`, plus the default `C:\Program Files` / `C:\Program Files (x86)`) — and throws a diagnosable Jaiph error with the stable code **`E_NO_POSIX_SHELL`** naming Git for Windows (https://git-scm.com/download/win) as the fix if none is found. Resolution is memoized for the lifetime of the process. Crucially, inline lines are **never** translated to `cmd`/PowerShell — Jaiph's language semantics require POSIX `sh` on every platform, or workflows stop being portable — so the seam only ever resolves *which* `sh` to run, never rewrites the command. POSIX inline-shell semantics are unchanged (existing e2e passes). New unit tests (`src/runtime/kernel/portability.test.ts`) stub `process.platform` and the `PATH` / existence lookup: POSIX returns `sh`, `win32` returns a discovered `sh.exe` path (both the on-`PATH` and Git-for-Windows-fallback branches), `win32` with no shell available throws `E_NO_POSIX_SHELL` with a message naming Git for Windows, and the result is memoized (the existence probe is not consulted on a second call); a `src/`-wide lint test asserts no production file outside `portability.ts` calls `spawn("sh"` / `spawn('sh'` directly. Docs updated (`docs/architecture.md`). - **Fix — Portability: execute script steps via an explicit interpreter, not shebang + exec bit:** Script steps used to run by spawning the emitted file directly (`executeScript` in `src/runtime/kernel/node-workflow-runtime.ts`), relying on the `#!/usr/bin/env ` shebang line and the `0o755` bit set by `src/transpile/build.ts`. Windows honors neither, and `noexec` mounts on Linux strip the exec bit — both break script execution. The runtime already knows the interpreter (it writes the shebang itself), so `executeScript` now reads the emitted script's shebang and resolves the interpreter through the new **`resolveInterpreterFromShebang`** (`src/parse/script-bash.ts`) — `#!/usr/bin/env ` → spawn ``, an absolute-path shebang (`#!/bin/bash`) → spawn that path, an `env -S` split-flag form → the real interpreter after `-S`, a missing/blank shebang → default `bash` — then spawns ` ` explicitly. The shebang line is **still** written into every emitted script (they stay directly executable by hand on POSIX) and the `0o755` bit is still set, but the runtime no longer depends on either being honored by the OS. A spawn `ENOENT` from an interpreter missing on `PATH` now surfaces as a diagnosable Jaiph error naming the interpreter (`script interpreter "" not found — install it or fix the script shebang`) instead of a raw `spawn ENOENT`. New unit tests assert the resolved interpreter + argv on the spawn call itself through an injectable `_scriptSpawn` seam (`src/runtime/kernel/node-workflow-runtime.script-exec.test.ts`), prove that a script with the exec bit stripped (`0o644`) still runs on POSIX, and prove that a missing-interpreter shebang produces the diagnosable error; `resolveInterpreterFromShebang` gets its own unit coverage (`src/parse/script-bash.test.ts`). Docs updated (`docs/architecture.md`, `docs/setup.md`). - **Fix — Portability: cross-platform process-tree termination:** `jaiph run` spawns the workflow leader with `detached: true` (`src/runtime/kernel/workflow-launch.ts`) and previously stopped it via `process.kill(-pid, signal)` — a negative-PID group kill, which signals the leader's whole process group. That primitive is POSIX-only: on `win32` it throws, the previous `child.kill()` fallback terminated only the leader, and the agent backends / script children it spawned were **orphaned**. A new portability module `src/runtime/kernel/portability.ts` exports **`killProcessTree(pid, signal)`** and is now the single sanctioned home for group kills. On POSIX it preserves the prior behavior — `process.kill(-pid, signal)` with a per-process fallback when the group no longer exists (`ESRCH`). On `win32` it force-kills the entire tree with `taskkill /pid /T /F` (spawned via an injectable `_portability.spawn` seam, not shelled), degrading to a per-process `process.kill(pid, signal)` if `taskkill` cannot be launched (reported asynchronously via the child's `error` event). Because `taskkill /F` is already a forceful kill with no graceful phase, a follow-up `SIGKILL` escalation after a `SIGTERM`/`SIGINT` is a **documented no-op** on `win32` — the tree is already gone. Every subprocess-termination call site is repointed at the helper: run teardown `terminateRunProcessGroup` (`src/cli/run/lifecycle.ts`, previously the negative-PID group kill) and the two SIGTERM→SIGKILL escalation paths that previously assumed POSIX signal semantics via `child.kill()` — the prompt watchdog's `killChild` (`src/runtime/kernel/prompt.ts`) and the Docker run-timeout kill (`src/runtime/docker.ts`) — so those escalations now terminate the whole child tree instead of just the leader. POSIX Ctrl-C behavior of `jaiph run` is unchanged (leader and children terminate; exit code and cleanup semantics identical), covered by the existing signal-lifecycle e2e. New unit tests (`src/runtime/kernel/portability.test.ts`) cover both platform branches by stubbing `process.platform` (precedent: `src/runtime/docker.test.ts`) and assert the exact mechanism invoked — negative-PID kill on POSIX (plus the ESRCH per-process fallback and SIGKILL escalation), `taskkill /pid /T /F` argv on `win32` (SIGTERM, SIGINT/Ctrl-C, and the degrade-to-per-process path), a proof that the win32 branch **never** calls `process.kill` with a negative PID, and a `src/`-wide lint test asserting no production file outside `portability.ts` matches `process.kill(-`. The watchdog unit tests in `src/runtime/kernel/prompt.test.ts` are updated to spy on `process.kill` (the watchdog now terminates through `killProcessTree`). Docs updated (`docs/architecture.md`, `docs/configuration.md`). diff --git a/QUEUE.md b/QUEUE.md index c02bca1e..b55ccf29 100644 --- a/QUEUE.md +++ b/QUEUE.md @@ -14,20 +14,6 @@ Process rules: *** -## Portability: home directory, Docker gating, and ANSI on win32 #dev-ready - -Remaining small POSIX assumptions for a host-only Windows runtime: - -1. `prepareClaudeEnv` (`src/runtime/kernel/prompt.ts`) reads `execEnv.HOME || process.env.HOME`. Use `os.homedir()` as the final fallback so `USERPROFILE`-only environments resolve; an explicit `HOME` in `execEnv` still wins. -2. Docker sandboxing (`src/runtime/docker.ts`) hardcodes POSIX socket paths and Linux/macOS workspace-presentation branches. On `win32`, the Docker sandbox is out of scope: `resolveDockerConfig` must resolve to host-only mode with a one-line notice (same UX as an explicit `JAIPH_UNSAFE=true`), never attempt `docker` probing, and never hard-fail because Docker is missing. -3. Live status rendering already gates SGR colors on `isTTY` + `NO_COLOR` and uses a single erase/cursor-up sequence (`src/cli/run/stderr-handler.ts`). Node enables VT processing on Windows 10+; no rendering change required. Add a `canUseAnsi()` helper to the portability module and route the existing gates through it so the policy lives in one place. - -Acceptance: - -* Unit test: with `HOME` unset and `os.homedir()` stubbed, `prepareClaudeEnv` resolves the config dir from `os.homedir()`; with `execEnv.HOME` set, it wins. -* Unit test: with `process.platform` stubbed to `win32`, Docker resolution returns host-only mode, emits the notice once, and performs zero `docker` invocations (spawn spied). -* Unit test: `canUseAnsi()` is false when `isTTY` is false or `NO_COLOR` is set; all color/erase emission sites consume it (grep test: no direct `isTTY && NO_COLOR` gating outside the portability module). - ## Distro: build and release `jaiph-windows-x64.exe` #dev-ready The release workflow (`.github/workflows/release.yml`) cross-compiles four assets (`jaiph-{darwin,linux}-{arm64,x64}`) via `bun build --compile` and publishes them with `SHA256SUMS`. Add Windows: diff --git a/docs/architecture.md b/docs/architecture.md index c6f5c6a0..d72c38de 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -92,7 +92,7 @@ All orchestration — local `jaiph run`, `jaiph test`, and **Docker `jaiph run`* - `jaiph format` rewrites `.jh` / `.test.jh` files into canonical style. `emitModule(ast, trivia, opts?)` reads the semantic AST together with the parallel **`Trivia`** store ([Trivia (CST layer)](#trivia-cst-layer)) to round-trip leading comments, top-level order, `config` body sequence, `"""..."""` and `bareSource` forms, the original quotedness of top-level `const` values (`EnvDeclDef.wasQuoted` — `true` for `"…"` / `"""…"""` sources, `undefined` for bare tokens — so a quoted value is never silently rewritten as bare based on whether it contains a space), and prompt / script body discriminators. Step emission switches on `WorkflowStepDef.type` (8 variants) and an `emitExpr` helper switches on `Expr.kind` (8 kinds) — there are no dual code paths for "managed sidecar vs literal value" because that branch was removed from the AST. Call arguments render straight off the typed `Arg[]` — `var` → bare name, `literal` → raw — so the formatter no longer re-parses any args string or consults a `bareIdentifierArgs` shadow field. Pure data→text emitter; no side-effects beyond file writes. Round-trip is bit-for-bit on every fixture under `examples/` and `test-fixtures/golden-ast/fixtures/` — pinned by `src/format/roundtrip.test.ts`, which asserts `parse → format → parse → format` converges in one step on every fixture. - **Docker runtime helper (`src/runtime/docker.ts`)** - - Parses mount specs, resolves Docker config (image, network, timeout), and builds the `docker run` invocation when the CLI enables **Docker sandboxing** for `jaiph run` (environment-driven; there is no `jaiph run --docker` flag — see [Sandboxing](sandboxing.md)). The container runs the same **`jaiph run --raw`** / **`__workflow-runner`** entry as local execution. The default image is the official `ghcr.io/jaiphlang/jaiph-runtime` GHCR image; every selected image must already contain `jaiph` (no auto-install or derived-image build at runtime). Image preparation (`prepareImage`) runs before the CLI banner: it checks whether the image is local, pulls with `--quiet` if needed (short status lines on stderr instead of Docker’s default pull UI), and verifies that `jaiph` exists in the image. `spawnDockerProcess` does not pull or verify — it receives a pre-resolved image. The spawn call uses `stdio: ["ignore", "pipe", "pipe"]` — stdin is ignored so the Docker CLI does not block on stdin EOF, which would stall event streaming and hang the host CLI after the container exits. + - Parses mount specs, resolves Docker config (image, network, timeout), and builds the `docker run` invocation when the CLI enables **Docker sandboxing** for `jaiph run` (environment-driven; there is no `jaiph run --docker` flag — see [Sandboxing](sandboxing.md)). On **`win32`** the Docker sandbox is out of scope: **`resolveDockerConfig`** forces host-only mode (same UX as an explicit **`JAIPH_UNSAFE=true`**) with a one-line notice, so the CLI never probes `docker` and never hard-fails on a missing daemon (`JAIPH_DOCKER_ENABLED=true` cannot override this). The container runs the same **`jaiph run --raw`** / **`__workflow-runner`** entry as local execution. The default image is the official `ghcr.io/jaiphlang/jaiph-runtime` GHCR image; every selected image must already contain `jaiph` (no auto-install or derived-image build at runtime). Image preparation (`prepareImage`) runs before the CLI banner: it checks whether the image is local, pulls with `--quiet` if needed (short status lines on stderr instead of Docker’s default pull UI), and verifies that `jaiph` exists in the image. `spawnDockerProcess` does not pull or verify — it receives a pre-resolved image. The spawn call uses `stdio: ["ignore", "pipe", "pipe"]` — stdin is ignored so the Docker CLI does not block on stdin EOF, which would stall event streaming and hang the host CLI after the container exits. - **Workspace immutability:** By default Docker runs cannot modify the host workspace. The host checkout is mounted read-only (overlay) or as a disposable clone (copy); `/jaiph/workspace` is sandbox-local and discarded on exit. The only host-writable path is `/jaiph/run` (run artifacts). Workflows that need to capture workspace changes should write files (for example a `git diff` into a temp path) and publish them with `artifacts.save()`. The explicit opt-in **inplace** mode (truthy **`JAIPH_INPLACE`** — `1` or `true`, or `jaiph run --inplace`) breaks this contract on purpose — the host workspace itself is bind-mounted read-write so the run's edits persist live on the host, with the rest of the sandbox (caps, env allowlist, mount set) unchanged. See [Sandboxing](sandboxing.md) for the full contract and [Save artifacts](artifacts.md). ## Local module graph @@ -170,7 +170,7 @@ Authoring rules, fixtures, and mock syntax for `*.test.jh` are documented in [Te ## CLI progress reporting pipeline -The progress UI combines a **static** step tree derived from the workflow AST (`src/cli/run/progress.ts`) with **live** updates from the runtime event stream. Event wiring: `src/cli/run/events.ts` and `src/cli/run/stderr-handler.ts` parse `__JAIPH_EVENT__` lines; `src/cli/run/emitter.ts` bridges into the renderer. Line-oriented formatting (`formatStartLine`, `formatHeartbeatLine`, `formatCompletedLine`) lives primarily in `src/cli/run/display.ts`, which shares some display helpers with `progress.ts`. Async branch numbering (subscript ₁₂₃… prefixes) is driven by `async_indices` on step and log events — the runtime propagates a chain of 1-based branch indices through `AsyncLocalStorage`, and the stderr handler renders them at the appropriate indent level. `const` steps whose `Expr` value is `kind: "match"` are walked for nested `run` / `ensure` arms; matched targets appear as child items in the step tree (for example `▸ workflow my_flow` or `▸ rule my_rule` under the `const` row). This pipeline does not apply to **`jaiph run --raw`**. +The progress UI combines a **static** step tree derived from the workflow AST (`src/cli/run/progress.ts`) with **live** updates from the runtime event stream. Event wiring: `src/cli/run/events.ts` and `src/cli/run/stderr-handler.ts` parse `__JAIPH_EVENT__` lines; `src/cli/run/emitter.ts` bridges into the renderer. Line-oriented formatting (`formatStartLine`, `formatHeartbeatLine`, `formatCompletedLine`) lives primarily in `src/cli/run/display.ts`, which shares some display helpers with `progress.ts`. Async branch numbering (subscript ₁₂₃… prefixes) is driven by `async_indices` on step and log events — the runtime propagates a chain of 1-based branch indices through `AsyncLocalStorage`, and the stderr handler renders them at the appropriate indent level. Whether ANSI SGR colors are emitted is a single policy — **`canUseAnsi()`** (`src/runtime/kernel/portability.ts`) returns `isTTY && NO_COLOR` unset — and every color emission site (`src/cli/commands/run.ts`, `src/cli/run/progress.ts`, `src/cli/shared/errors.ts`) routes its gate through it rather than re-deriving `isTTY && NO_COLOR` locally. On Windows 10+ Node enables console VT processing automatically, so `isTTY` is a sufficient ANSI proxy with no extra win32 branch. `const` steps whose `Expr` value is `kind: "match"` are walked for nested `run` / `ensure` arms; matched targets appear as child items in the step tree (for example `▸ workflow my_flow` or `▸ rule my_rule` under the `const` row). This pipeline does not apply to **`jaiph run --raw`**. ## Distribution: Node vs Bun standalone diff --git a/docs/configuration.md b/docs/configuration.md index 83bdb949..d7dc7492 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -83,14 +83,17 @@ In-file `runtime.docker_enabled` is not supported (`E_PARSE`); use the env-only ## Docker enablement +Checks are applied top to bottom; the first match wins. + | Check | Result | |---|---| +| Platform is Windows (`win32`) | Docker off (host-only mode, with a one-line notice). Overrides everything below, including `JAIPH_DOCKER_ENABLED=true`. | | `JAIPH_DOCKER_ENABLED` is set to exact `true` | Docker on. | | `JAIPH_DOCKER_ENABLED` is set to any other value | Docker off. | | `JAIPH_DOCKER_ENABLED` is unset and `JAIPH_UNSAFE=true` | Docker off. | | Default (no env) | Docker on. | -`CI=true` does not change this default. Host `jaiph run --raw` never consults this branch — the workflow runner is local in that path. See [Sandboxing](sandboxing.md) for the full model. +`CI=true` does not change this default. Host `jaiph run --raw` never consults this branch — the workflow runner is local in that path. On Windows the Docker sandbox is out of scope, so `jaiph run` resolves to host-only mode automatically without probing `docker` or failing on a missing daemon — see [Sandboxing — Windows runs host-only](sandboxing.md#windows-runs-host-only) for the full model. ## Precedence {: #precedence} diff --git a/docs/env-vars.md b/docs/env-vars.md index 5f232352..42927c27 100644 --- a/docs/env-vars.md +++ b/docs/env-vars.md @@ -43,7 +43,7 @@ The table below covers every `JAIPH_*` name read from `process.env` / `env` in ` | `JAIPH_CODEX_API_URL` | runtime | string | `https://api.openai.com/v1/chat/completions` | — | Chat-completions endpoint for the `codex` backend. | | `JAIPH_DEBUG` | host, runtime | bool (exact `"true"`) | `false` | `run.debug` | Enable debug tracing for the run. | | `JAIPH_DEBUG_LOCKED` | internal | bool | — | — | Lock flag for `JAIPH_DEBUG`. | -| `JAIPH_DOCKER_ENABLED` | host | bool (exact `true`) | — | — | Force Docker on (`true`) or off (any other value). When unset, Docker is on unless `JAIPH_UNSAFE=true`. | +| `JAIPH_DOCKER_ENABLED` | host | bool (exact `true`) | — | — | Force Docker on (`true`) or off (any other value). When unset, Docker is on unless `JAIPH_UNSAFE=true`. Ignored on Windows (`win32`), where the sandbox is out of scope and runs are always host-only. | | `JAIPH_DOCKER_IMAGE` | host | string | `ghcr.io/jaiphlang/jaiph-runtime:` | `runtime.docker_image` | Container image. Must already contain `jaiph`. | | `JAIPH_DOCKER_KEEP_SANDBOX` | host | bool (`1` / `true`) | `false` | — | Copy mode only — when enabled, leave the host-side `.sandbox-/` clone on disk after exit for debugging. | | `JAIPH_DOCKER_NETWORK` | host | string (`default`, `none`, or named network) | `default` | `runtime.docker_network` | `docker run --network` value. `none` disables egress. | @@ -115,7 +115,7 @@ These error codes surface during Docker-backed `jaiph run` invocations. They are | Code | Trigger | Behaviour | |---|---|---| -| `E_DOCKER_NOT_FOUND` | `docker info` fails (Docker not installed or daemon not running). | Run exits before launch. No fallback to local execution. | +| `E_DOCKER_NOT_FOUND` | `docker info` fails (Docker not installed or daemon not running). | Run exits before launch. No fallback to local execution. Not reachable on Windows, where the CLI resolves to host-only mode without probing `docker`. | | `E_DOCKER_PULL` | `docker pull` fails (network error, image not found, auth failure). | Run exits before launch. | | `E_DOCKER_NO_JAIPH` | Selected image does not contain a `jaiph` CLI. | Run exits before launch. | | `E_DOCKER_RUNS_DIR` | Absolute `JAIPH_RUNS_DIR` points outside the workspace. | Run exits before launch. | diff --git a/docs/sandboxing.md b/docs/sandboxing.md index 054626f2..c1523b4a 100644 --- a/docs/sandboxing.md +++ b/docs/sandboxing.md @@ -75,6 +75,10 @@ A second, equally deliberate choice: **enablement lives entirely in environment The escape hatch — `JAIPH_UNSAFE=true` or `jaiph run --unsafe` — exists because some environments genuinely cannot run Docker (a sandboxed CI without nested virtualization, a developer iterating on the runtime itself). The choice to take that hatch should be visible and ergonomic, which is why it is a single explicit host-side switch rather than an in-file `config` knob. +## Windows runs host-only + +On Windows (`win32`) the Docker sandbox is out of scope: the sandbox modes rely on POSIX socket paths and Linux/macOS-specific workspace presentation, so Jaiph does not attempt them there. `jaiph run` on Windows resolves to **host-only mode automatically** — the same posture as an explicit `JAIPH_UNSAFE=true` — and prints a one-line notice that the run is host-only. The CLI never probes for `docker` and never fails just because a Docker daemon is absent, and `JAIPH_DOCKER_ENABLED=true` cannot force the sandbox back on. Windows workflows therefore run with no OS sandbox; keep the [not-protected-against](#what-docker-does-not-protect-against) list in mind, or run under WSL, where the Linux path (and the full sandbox) applies. + ## Why `jaiph test` does not use Docker The test runner runs in-process on the host. This is intentional: tests are a development feedback loop, they typically mock prompts and replace external calls, and Docker spawn overhead would harm the iteration cycle. Tests already get isolation from the things they care about (prompts, network) through the runtime's mock infrastructure. The Docker boundary is for `jaiph run`, where the workflow is executing real scripts against real resources. diff --git a/src/cli/commands/run.ts b/src/cli/commands/run.ts index 5f143e54..aa2b73c7 100644 --- a/src/cli/commands/run.ts +++ b/src/cli/commands/run.ts @@ -12,6 +12,7 @@ import { basename } from "node:path"; import { parsejaiph } from "../../parser"; import { buildScripts, buildScriptsFromGraph } from "../../transpiler"; import { loadModuleGraph, writeModuleGraph } from "../../transpile/module-graph"; +import { canUseAnsi } from "../../runtime/kernel/portability"; import { metadataToConfig } from "../../config"; import { buildStepDisplayParamPairs, formatNamedParamsForDisplay } from "./format-params.js"; import { @@ -130,7 +131,7 @@ export async function runWorkflow(rest: string[]): Promise { const outDir = target ? resolve(target) : mkdtempSync(join(tmpdir(), "jaiph-run-")); const shouldCleanup = !target; try { - const colorEnabled = process.stdout.isTTY && process.env.NO_COLOR === undefined; + const colorEnabled = canUseAnsi(); const isTTY = !!process.stdout.isTTY; const startedAt = Date.now(); diff --git a/src/cli/run/progress.ts b/src/cli/run/progress.ts index 546c7aac..f3b9b9b8 100644 --- a/src/cli/run/progress.ts +++ b/src/cli/run/progress.ts @@ -1,6 +1,7 @@ import { resolve } from "node:path"; import { jaiphModule, type Expr, type WorkflowStepDef } from "../../types"; import { workflowSymbolForFile } from "../../transpiler"; +import { canUseAnsi } from "../../runtime/kernel/portability"; export type TreeRow = { rawLabel: string; @@ -282,7 +283,7 @@ export function parseLabel(rawLabel: string): { kind: string; name: string } { export function styleKeywordLabel(rawLabel: string): string { const { kind, name } = parseLabel(rawLabel); - const enabled = process.stdout.isTTY && process.env.NO_COLOR === undefined; + const enabled = canUseAnsi(); if (!enabled) { return `${kind} ${name}`; } @@ -290,7 +291,7 @@ export function styleKeywordLabel(rawLabel: string): string { } export function styleDim(text: string): string { - const enabled = process.stdout.isTTY && process.env.NO_COLOR === undefined; + const enabled = canUseAnsi(); if (!enabled) { return text; } @@ -298,7 +299,7 @@ export function styleDim(text: string): string { } export function styleYellow(text: string): string { - const enabled = process.stdout.isTTY && process.env.NO_COLOR === undefined; + const enabled = canUseAnsi(); if (!enabled) { return text; } @@ -306,7 +307,7 @@ export function styleYellow(text: string): string { } export function styleBold(text: string): string { - const enabled = process.stdout.isTTY && process.env.NO_COLOR === undefined; + const enabled = canUseAnsi(); if (!enabled) { return text; } diff --git a/src/cli/shared/errors.ts b/src/cli/shared/errors.ts index 91582395..f7c13888 100644 --- a/src/cli/shared/errors.ts +++ b/src/cli/shared/errors.ts @@ -1,9 +1,10 @@ import { existsSync, readFileSync, readdirSync, statSync } from "node:fs"; import { join } from "node:path"; import { CONTAINER_RUN_DIR } from "../../runtime/docker"; +import { canUseAnsi } from "../../runtime/kernel/portability"; export function colorPalette(): { green: string; red: string; dim: string; reset: string } { - const enabled = process.stdout.isTTY && process.env.NO_COLOR === undefined; + const enabled = canUseAnsi(); if (!enabled) { return { green: "", red: "", dim: "", reset: "" }; } diff --git a/src/runtime/docker.test.ts b/src/runtime/docker.test.ts index a227fac4..ffbdfcb8 100644 --- a/src/runtime/docker.test.ts +++ b/src/runtime/docker.test.ts @@ -22,7 +22,9 @@ import { cleanupDocker, withDockerExitGuard, _dockerExec, + _dockerSpawn, _uidDetect, + _win32Notice, type DockerRunConfig, type DockerSpawnOptions, type DockerSpawnResult, @@ -220,6 +222,42 @@ test("resolveDockerConfig: in-file dockerEnabled is ignored (field removed from assert.equal(cfg.enabled, false, "JAIPH_UNSAFE disables Docker regardless of in-file"); }); +test("resolveDockerConfig: win32 forces host-only mode, notice once, zero docker invocations", () => { + const origPlatform = Object.getOwnPropertyDescriptor(process, "platform"); + Object.defineProperty(process, "platform", { value: "win32", configurable: true }); + // Spy every docker entry point so we can assert win32 never probes. + const origExec = _dockerExec.run; + const origSpawn = _dockerSpawn.run; + const origWrite = _win32Notice.write; + const origEmitted = _win32Notice.emitted; + let execCalls = 0; + let spawnCalls = 0; + const notices: string[] = []; + _dockerExec.run = () => { execCalls += 1; }; + _dockerSpawn.run = () => { spawnCalls += 1; return {} as any; }; + _win32Notice.write = (msg) => { notices.push(msg); }; + _win32Notice.emitted = false; + try { + // Even an explicit JAIPH_DOCKER_ENABLED=true cannot re-enable Docker on win32. + const first = resolveDockerConfig(undefined, { JAIPH_DOCKER_ENABLED: "true" }); + const second = resolveDockerConfig(undefined, {}); + assert.equal(first.enabled, false, "win32 resolves to host-only mode"); + assert.equal(second.enabled, false, "win32 resolves to host-only mode on every call"); + assert.equal(notices.length, 1, "notice is emitted exactly once across calls"); + assert.match(notices[0], /Windows/, "notice mentions Windows"); + assert.match(notices[0], /host-only|no sandbox/, "notice describes host-only mode"); + assert.equal(execCalls, 0, "no docker exec probing on win32"); + assert.equal(spawnCalls, 0, "no docker spawn on win32"); + } finally { + _dockerExec.run = origExec; + _dockerSpawn.run = origSpawn; + _win32Notice.write = origWrite; + _win32Notice.emitted = origEmitted; + if (origPlatform) Object.defineProperty(process, "platform", origPlatform); + else Object.defineProperty(process, "platform", { value: process.platform, configurable: true }); + } +}); + test("checkDockerAvailable: E_DOCKER_NOT_FOUND message mentions JAIPH_UNSAFE", () => { const src = readFileSync(join(__dirname, "docker.ts"), "utf8"); assert.ok( diff --git a/src/runtime/docker.ts b/src/runtime/docker.ts index b281f050..b1e49c04 100644 --- a/src/runtime/docker.ts +++ b/src/runtime/docker.ts @@ -89,9 +89,34 @@ const DEFAULTS: DockerRunConfig = { timeoutSeconds: 14400, }; +/** + * Test seam for the one-time win32 host-only notice. Tests reset `emitted` + * between runs and can spy `write` to assert the notice fires exactly once. + */ +export const _win32Notice = { + emitted: false, + write(message: string): void { + process.stderr.write(message); + }, +}; + +/** Emit the win32 host-only notice at most once per process. */ +function emitWin32HostOnlyNotice(): void { + if (_win32Notice.emitted) return; + _win32Notice.emitted = true; + _win32Notice.write( + "jaiph: Docker sandbox is not supported on Windows; running host-only (no sandbox).\n", + ); +} + /** * Resolve effective Docker config. - * Precedence: env vars (`JAIPH_DOCKER_*`) > unsafe default rule. + * Precedence: platform > env vars (`JAIPH_DOCKER_*`) > unsafe default rule. + * + * On win32 the Docker sandbox is out of scope: resolution is forced to + * host-only mode (same UX as an explicit `JAIPH_UNSAFE=true`) with a one-line + * notice, so the CLI never probes `docker` and never hard-fails on a missing + * daemon. `JAIPH_DOCKER_ENABLED=true` cannot override this. * * Default rule (when no explicit `JAIPH_DOCKER_ENABLED` is set): * - `JAIPH_UNSAFE=true` → Docker off (explicit "run on host" escape hatch) @@ -101,9 +126,12 @@ export function resolveDockerConfig( inFile: RuntimeConfig | undefined, env: Record, ): DockerRunConfig { - // enabled: env JAIPH_DOCKER_ENABLED > unsafe default rule + // enabled: win32 host-only override > env JAIPH_DOCKER_ENABLED > unsafe default rule let enabled: boolean; - if (env.JAIPH_DOCKER_ENABLED !== undefined) { + if (process.platform === "win32") { + emitWin32HostOnlyNotice(); + enabled = false; + } else if (env.JAIPH_DOCKER_ENABLED !== undefined) { enabled = env.JAIPH_DOCKER_ENABLED === "true"; } else { // Default: Docker on unless the user explicitly opts out via JAIPH_UNSAFE. diff --git a/src/runtime/kernel/portability.test.ts b/src/runtime/kernel/portability.test.ts index f95a3476..173d42a2 100644 --- a/src/runtime/kernel/portability.test.ts +++ b/src/runtime/kernel/portability.test.ts @@ -3,7 +3,7 @@ import assert from "node:assert/strict"; import { EventEmitter } from "node:events"; import { readFileSync, readdirSync } from "node:fs"; import { resolve, join } from "node:path"; -import { killProcessTree, resolveShell, _portability } from "./portability"; +import { killProcessTree, resolveShell, canUseAnsi, _portability } from "./portability"; // Precedent for platform stubbing: src/runtime/docker.test.ts. function withPlatform(platform: string, fn: () => void): void { @@ -280,6 +280,67 @@ test("resolveShell memoizes: fileExists is not consulted on the second call", () }); }); +// --------------------------------------------------------------------------- +// canUseAnsi(): single ANSI color/erase policy for the CLI. +// --------------------------------------------------------------------------- + +/** Run `fn` with NO_COLOR forced to a given presence. */ +function withNoColor(value: string | undefined, fn: () => void): void { + const saved = process.env.NO_COLOR; + if (value === undefined) delete process.env.NO_COLOR; + else process.env.NO_COLOR = value; + try { + fn(); + } finally { + if (saved === undefined) delete process.env.NO_COLOR; + else process.env.NO_COLOR = saved; + } +} + +test("canUseAnsi: true only when isTTY and NO_COLOR is unset", () => { + withNoColor(undefined, () => { + assert.equal(canUseAnsi({ isTTY: true }), true); + }); +}); + +test("canUseAnsi: false when isTTY is false", () => { + withNoColor(undefined, () => { + assert.equal(canUseAnsi({ isTTY: false }), false); + assert.equal(canUseAnsi({}), false); + }); +}); + +test("canUseAnsi: false when NO_COLOR is set, even on a TTY", () => { + withNoColor("1", () => { + assert.equal(canUseAnsi({ isTTY: true }), false); + }); + // NO_COLOR honored even when empty (https://no-color.org: any value). + withNoColor("", () => { + assert.equal(canUseAnsi({ isTTY: true }), false); + }); +}); + +// --------------------------------------------------------------------------- +// Lint contract: the `isTTY && NO_COLOR` policy lives only in portability.ts. +// --------------------------------------------------------------------------- + +test("no production source outside portability.ts gates directly on isTTY && NO_COLOR", () => { + // Match either ordering of the combined gate on a single line. + const combinedGate = /(isTTY[^\n]*NO_COLOR)|(NO_COLOR[^\n]*isTTY)/; + const offenders: string[] = []; + for (const file of walkProductionTsFiles(SRC_ROOT)) { + const rel = file.slice(REPO_ROOT.length + 1); + if (rel === join("src", "runtime", "kernel", "portability.ts")) continue; + const content = readFileSync(file, "utf8"); + if (combinedGate.test(content)) offenders.push(rel); + } + assert.deepEqual( + offenders, + [], + `ANSI gating must route through canUseAnsi(); offenders: ${offenders.join(", ")}`, + ); +}); + // --------------------------------------------------------------------------- // Lint contract: no `spawn("sh", …)` outside the portability module. // --------------------------------------------------------------------------- diff --git a/src/runtime/kernel/portability.ts b/src/runtime/kernel/portability.ts index d8e8be76..4f4c6263 100644 --- a/src/runtime/kernel/portability.ts +++ b/src/runtime/kernel/portability.ts @@ -34,6 +34,22 @@ export const _portability = { }, }; +/** + * Whether ANSI SGR colors and cursor/erase sequences may be emitted. + * + * Single policy for the whole CLI: ANSI is on only when writing to a TTY and + * `NO_COLOR` is unset (https://no-color.org). Live status rendering, the + * progress tree, and error formatting route their `isTTY && NO_COLOR` gate + * through here so the rule lives in one place rather than being re-derived at + * each emission site. + * + * On Windows 10+ Node enables VT processing on the console automatically, so + * `isTTY` is a sufficient proxy for ANSI support — no extra win32 branch. + */ +export function canUseAnsi(stream: { isTTY?: boolean } = process.stdout): boolean { + return Boolean(stream.isTTY) && process.env.NO_COLOR === undefined; +} + let cachedShell: string | undefined; /** diff --git a/src/runtime/kernel/prompt.test.ts b/src/runtime/kernel/prompt.test.ts index 1e042d7c..7d875a72 100644 --- a/src/runtime/kernel/prompt.test.ts +++ b/src/runtime/kernel/prompt.test.ts @@ -467,6 +467,52 @@ describe("prepareClaudeEnv", () => { } }); + it("resolves the config dir from os.homedir() when HOME is unset", () => { + // The named `import { homedir }` in prompt.ts reads node:os's raw (mutable) + // require object at call time, so stubbing homedir on that same cached + // object changes what prepareClaudeEnv sees. Also clear process.env.HOME so + // the os.homedir() fallback (not the ambient HOME) is what resolves. + const osRaw = require("node:os") as { homedir: () => string }; + const root = mkdtempSync(join(tmpdir(), "jaiph-claude-env-homedir-")); + const origHomedir = osRaw.homedir; + const savedHome = process.env.HOME; + delete process.env.HOME; + osRaw.homedir = () => root; + try { + const prepared = prepareClaudeEnv({}, join(root, "workspace")); + assert.equal(prepared.error, undefined); + assert.equal(prepared.warning, undefined); + // A writable /.claude means execEnv is returned unchanged and + // session-env is created there — proving os.homedir() was the source. + assert.equal(prepared.env.CLAUDE_CONFIG_DIR, undefined); + assert.ok(existsSync(join(root, ".claude", "session-env"))); + } finally { + osRaw.homedir = origHomedir; + if (savedHome === undefined) delete process.env.HOME; + else process.env.HOME = savedHome; + rmSync(root, { recursive: true, force: true }); + } + }); + + it("prefers execEnv.HOME over os.homedir()", () => { + const osRaw = require("node:os") as { homedir: () => string }; + const homeRoot = mkdtempSync(join(tmpdir(), "jaiph-claude-env-home-wins-")); + const homedirRoot = mkdtempSync(join(tmpdir(), "jaiph-claude-env-homedir-unused-")); + const origHomedir = osRaw.homedir; + osRaw.homedir = () => homedirRoot; + try { + const prepared = prepareClaudeEnv({ HOME: homeRoot }, join(homeRoot, "workspace")); + assert.equal(prepared.error, undefined); + // Config resolves under execEnv.HOME, not the os.homedir() stub. + assert.ok(existsSync(join(homeRoot, ".claude", "session-env"))); + assert.ok(!existsSync(join(homedirRoot, ".claude"))); + } finally { + osRaw.homedir = origHomedir; + rmSync(homeRoot, { recursive: true, force: true }); + rmSync(homedirRoot, { recursive: true, force: true }); + } + }); + it("falls back to workspace-local claude config when default is not writable", () => { const root = mkdtempSync(join(tmpdir(), "jaiph-claude-env-fallback-")); try { diff --git a/src/runtime/kernel/prompt.ts b/src/runtime/kernel/prompt.ts index d09c4977..b124cb79 100644 --- a/src/runtime/kernel/prompt.ts +++ b/src/runtime/kernel/prompt.ts @@ -3,6 +3,7 @@ import { spawn as nodeSpawn, type ChildProcess } from "node:child_process"; import { writeFileSync, readFileSync, existsSync, accessSync, mkdirSync, cpSync, constants as fsConstants } from "node:fs"; import { basename, delimiter, join } from "node:path"; +import { homedir } from "node:os"; import { parseStream, type StreamWriter } from "./stream-parser"; import { consumeNextMockResponse, dispatchMockArms, type MockPromptArm } from "./mock"; import { killProcessTree } from "./portability"; @@ -229,7 +230,9 @@ type ClaudeEnvPreparation = { * Falls back to workspace-local `.jaiph/claude-config` when home config is not writable. */ export function prepareClaudeEnv(execEnv: NodeJS.ProcessEnv, workspaceRoot: string): ClaudeEnvPreparation { - const home = execEnv.HOME || process.env.HOME || ""; + // Final fallback to os.homedir() so USERPROFILE-only environments (Windows) + // resolve; an explicit HOME in execEnv still wins. + const home = execEnv.HOME || process.env.HOME || homedir() || ""; const defaultConfigDir = home ? join(home, ".claude") : ""; const configuredDir = execEnv.CLAUDE_CONFIG_DIR || defaultConfigDir; From 07fb4e2c2576f91e96b33741e62e07bd72083d57 Mon Sep 17 00:00:00 2001 From: Jakub Dzikowski Date: Mon, 13 Jul 2026 14:16:01 +0200 Subject: [PATCH 05/85] Feat: build and release jaiph-windows-x64.exe Add a Windows x64 target to the release workflow so every release ships a fifth standalone binary alongside darwin/linux x arm64/x64. A new bun-windows-x64 matrix entry (with an ext field so the compiled outfile and uploaded artifact carry the .exe suffix) produces jaiph-windows-x64.exe; Bun has no windows-arm64 target so Windows is x64 only. The .exe is included in SHA256SUMS generation and in both the stable and nightly gh release upload lists. A new sanity-windows job runs on windows-latest, downloads the .exe artifact, and runs jaiph-windows-x64.exe --version through a version gate; the publish job now needs [build, sanity-windows], so a Windows version mismatch fails the whole release. The linux-x64 gate's inline comparison is extracted into a shared, testable scripts/release-version-check.sh that both gates delegate to. Update the release asset naming contract in docs/contributing.md and docs/architecture.md, and add integration/release-workflow.test.ts asserting the five-binary matrix, SHA256SUMS and upload-list coverage, the shared gate behavior, and naming-contract/installer parity. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/release.yml | 83 ++++++++--- CHANGELOG.md | 1 + QUEUE.md | 15 -- docs/architecture.md | 2 +- docs/contributing.md | 7 +- integration/release-workflow.test.ts | 201 +++++++++++++++++++++++++++ scripts/release-version-check.sh | 34 +++++ 7 files changed, 304 insertions(+), 39 deletions(-) create mode 100644 integration/release-workflow.test.ts create mode 100755 scripts/release-version-check.sh diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index ab8d24d6..af3850a1 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -57,15 +57,24 @@ jobs: - target: bun-darwin-arm64 os: darwin arch: arm64 + ext: "" - target: bun-darwin-x64 os: darwin arch: x64 + ext: "" - target: bun-linux-x64 os: linux arch: x64 + ext: "" - target: bun-linux-arm64 os: linux arch: arm64 + ext: "" + # Bun has no bun-windows-arm64 target; windows ships x64 only. + - target: bun-windows-x64 + os: windows + arch: x64 + ext: ".exe" steps: - name: Checkout uses: actions/checkout@v4 @@ -88,20 +97,61 @@ jobs: - name: Cross-compile standalone binary for ${{ matrix.target }} run: | set -euo pipefail - bun build --compile --target=${{ matrix.target }} ./src/cli.ts --outfile "jaiph-${{ matrix.os }}-${{ matrix.arch }}" - ls -la "jaiph-${{ matrix.os }}-${{ matrix.arch }}" + out="jaiph-${{ matrix.os }}-${{ matrix.arch }}${{ matrix.ext }}" + bun build --compile --target=${{ matrix.target }} ./src/cli.ts --outfile "${out}" + ls -la "${out}" - name: Upload binary artifact uses: actions/upload-artifact@v4 with: - name: jaiph-${{ matrix.os }}-${{ matrix.arch }} - path: jaiph-${{ matrix.os }}-${{ matrix.arch }} + name: jaiph-${{ matrix.os }}-${{ matrix.arch }}${{ matrix.ext }} + path: jaiph-${{ matrix.os }}-${{ matrix.arch }}${{ matrix.ext }} if-no-files-found: error retention-days: 7 + sanity-windows: + name: Sanity gate (windows-x64 --version) + needs: build + runs-on: windows-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Resolve tag and channel + id: meta + shell: bash + run: | + set -euo pipefail + case "${GITHUB_REF}" in + refs/tags/v*) + tag="${GITHUB_REF_NAME}"; channel="stable" ;; + refs/heads/nightly|refs/tags/nightly) + tag="nightly"; channel="nightly" ;; + *) + echo "Unsupported ref for release: ${GITHUB_REF}" >&2; exit 1 ;; + esac + echo "tag=${tag}" >> "${GITHUB_OUTPUT}" + echo "channel=${channel}" >> "${GITHUB_OUTPUT}" + + - name: Download windows binary artifact + uses: actions/download-artifact@v4 + with: + name: jaiph-windows-x64.exe + path: release-assets + + - name: Sanity gate (windows-x64 --version) + working-directory: release-assets + shell: bash + run: | + set -euo pipefail + got="$(./jaiph-windows-x64.exe --version)" + echo "got: ${got}" + bash ../scripts/release-version-check.sh \ + "${{ steps.meta.outputs.channel }}" "${{ steps.meta.outputs.tag }}" "${got}" + release: name: Publish release assets - needs: build + needs: [build, sanity-windows] runs-on: ubuntu-latest permissions: contents: write @@ -138,7 +188,7 @@ jobs: set -euo pipefail ls -la rm -f SHA256SUMS - sha256sum jaiph-darwin-arm64 jaiph-darwin-x64 jaiph-linux-x64 jaiph-linux-arm64 > SHA256SUMS + sha256sum jaiph-darwin-arm64 jaiph-darwin-x64 jaiph-linux-x64 jaiph-linux-arm64 jaiph-windows-x64.exe > SHA256SUMS cat SHA256SUMS - name: Sanity gate (linux-x64 --version) @@ -148,19 +198,8 @@ jobs: chmod +x jaiph-linux-x64 got="$(./jaiph-linux-x64 --version)" echo "got: ${got}" - if [ "${{ steps.meta.outputs.channel }}" = "stable" ]; then - tag="${{ steps.meta.outputs.tag }}" - expected="jaiph ${tag#v}" - if [ "${got}" != "${expected}" ]; then - echo "Version sanity check failed: expected '${expected}', got '${got}'" >&2 - exit 1 - fi - else - if ! printf '%s\n' "${got}" | grep -Eq '^jaiph [0-9]+\.[0-9]+\.[0-9]+'; then - echo "Version sanity check failed: '${got}' does not look like a jaiph version" >&2 - exit 1 - fi - fi + bash ../scripts/release-version-check.sh \ + "${{ steps.meta.outputs.channel }}" "${{ steps.meta.outputs.tag }}" "${got}" - name: Publish stable release ${{ steps.meta.outputs.tag }} if: steps.meta.outputs.channel == 'stable' @@ -172,13 +211,15 @@ jobs: gh release upload "${tag}" --clobber \ jaiph-darwin-arm64 jaiph-darwin-x64 \ jaiph-linux-x64 jaiph-linux-arm64 \ + jaiph-windows-x64.exe \ SHA256SUMS else gh release create "${tag}" \ --title "${tag}" \ - --notes "Jaiph ${tag} — standalone binaries (darwin/linux × arm64/x64) plus SHA256SUMS." \ + --notes "Jaiph ${tag} — standalone binaries (darwin/linux × arm64/x64, windows x64) plus SHA256SUMS." \ jaiph-darwin-arm64 jaiph-darwin-x64 \ jaiph-linux-x64 jaiph-linux-arm64 \ + jaiph-windows-x64.exe \ SHA256SUMS fi @@ -191,6 +232,7 @@ jobs: gh release upload nightly --clobber \ jaiph-darwin-arm64 jaiph-darwin-x64 \ jaiph-linux-x64 jaiph-linux-arm64 \ + jaiph-windows-x64.exe \ SHA256SUMS else gh release create nightly \ @@ -200,5 +242,6 @@ jobs: --target "${GITHUB_SHA}" \ jaiph-darwin-arm64 jaiph-darwin-x64 \ jaiph-linux-x64 jaiph-linux-arm64 \ + jaiph-windows-x64.exe \ SHA256SUMS fi diff --git a/CHANGELOG.md b/CHANGELOG.md index 102f19dc..32dfc3b2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,7 @@ ## All changes +- **Feature — Distro: build and release `jaiph-windows-x64.exe`:** The release workflow (`.github/workflows/release.yml`) now cross-compiles and publishes a Windows binary alongside the existing darwin/linux × arm64/x64 set. A new matrix entry `bun-windows-x64` (`os: windows`, `arch: x64`) produces the asset **`jaiph-windows-x64.exe`**; Bun has no `bun-windows-arm64` target, so Windows ships x64 only. The matrix gained an `ext` field (`""` for POSIX, `".exe"` for Windows) so the compiled `--outfile` and the uploaded artifact carry the extension. `SHA256SUMS` generation now covers all **five** binaries including the `.exe`, and both the stable (`v*`) and rolling `nightly` `gh release` upload lists ship the `.exe` — so every release publishes **six** assets (five binaries + `SHA256SUMS`). A new **`sanity-windows`** job runs on `windows-latest`, downloads the `.exe` artifact, and runs `jaiph-windows-x64.exe --version` through the version gate; the publish job now `needs: [build, sanity-windows]`, so a Windows version mismatch fails the whole release. The linux-x64 gate's inline version comparison was extracted into a shared, unit-testable **`scripts/release-version-check.sh`** (` `: stable output must equal `jaiph ` exactly; any other channel only has to look like a `jaiph ` banner), and both the linux-x64 and windows-x64 gates delegate to it so the comparison lives in one place. The bash installer (`docs/install`) and its e2e test (`e2e/tests/07_installer_binary.sh`) are unchanged — they resolve darwin/linux names from `{os}×{arch}` and cannot install a Windows `.exe` via `curl … | bash`; Windows users download the `.exe` from the Release directly. New acceptance tests (`integration/release-workflow.test.ts`) assert the five-binary matrix (and the absence of a windows-arm64 target), that `SHA256SUMS` and both upload lists include the `.exe`, that a `.exe` checksum entry round-trips through the installer's own `awk` lookup, that the shared gate fails on a stable tag/version mismatch and passes on a match, that the `windows-latest` job runs `--version` through the shared script and blocks publish, and that the naming contract ↔ release matrix ↔ installer asset names stay in parity. Docs updated (`docs/architecture.md`, `docs/contributing.md`). - **Fix — Portability: home-dir fallback, Docker gating on Windows, and a single ANSI policy:** Three remaining POSIX assumptions that broke a host-only `win32` runtime are closed. (1) **Home directory** — `prepareClaudeEnv` (`src/runtime/kernel/prompt.ts`) resolved the `.claude` config dir from `execEnv.HOME || process.env.HOME`, which is empty in a `USERPROFILE`-only Windows environment. It now falls back to `os.homedir()` as a final source (`execEnv.HOME || process.env.HOME || homedir()`); an explicit `HOME` in `execEnv` still wins. (2) **Docker gating** — the Docker sandbox hardcodes POSIX socket paths and Linux/macOS workspace-presentation branches, so it is out of scope on Windows. `resolveDockerConfig` (`src/runtime/docker.ts`) now forces **host-only mode** on `win32` (same UX as an explicit `JAIPH_UNSAFE=true`) with a one-line notice emitted once per process, so the CLI never probes `docker` and never hard-fails on a missing daemon; `JAIPH_DOCKER_ENABLED=true` cannot override this. (3) **ANSI colors** — a new **`canUseAnsi(stream?)`** helper in the portability module (`src/runtime/kernel/portability.ts`) returns `isTTY && NO_COLOR`-unset, and every color/erase emission site (`src/cli/commands/run.ts`, `src/cli/run/progress.ts`, `src/cli/shared/errors.ts`) routes its gate through it so the policy lives in one place; no rendering change is needed because Node enables console VT processing on Windows 10+ automatically, making `isTTY` a sufficient ANSI proxy. New unit tests: `prepareClaudeEnv` resolves from a stubbed `os.homedir()` when `HOME` is unset and prefers `execEnv.HOME` when set (`src/runtime/kernel/prompt.test.ts`); `resolveDockerConfig` under a stubbed `win32` returns host-only, emits the notice once, and performs zero `docker` invocations (`src/runtime/docker.test.ts`); `canUseAnsi()` is false when `isTTY` is false or `NO_COLOR` is set, plus a `src/`-wide lint test asserting no production file outside `portability.ts` gates directly on `isTTY && NO_COLOR` (`src/runtime/kernel/portability.test.ts`). Docs updated (`docs/architecture.md`, `docs/sandboxing.md`, `docs/configuration.md`, `docs/env-vars.md`). - **Fix — Portability: single `resolveShell()` seam for inline shell lines and hooks:** Inline workflow shell lines (`executeShLine` in `src/runtime/kernel/node-workflow-runtime.ts`) and hook commands (`src/cli/run/hooks.ts`) used to run via a hardcoded `spawn("sh", ["-c", …])`. That works on POSIX but fails on `win32`, where there is no `sh` on the default `PATH`. A new **`resolveShell(): string`** in the portability module (`src/runtime/kernel/portability.ts`) is now the single seam both call sites go through. On POSIX it returns bare `sh`. On `win32` it locates Git for Windows' bundled `sh.exe` — first on `PATH`, then in the standard install layouts (`/bin/sh.exe`, `/usr/bin/sh.exe`) under each known root (`%ProgramFiles%`, `%ProgramFiles(x86)%`, `%ProgramW6432%`, plus the default `C:\Program Files` / `C:\Program Files (x86)`) — and throws a diagnosable Jaiph error with the stable code **`E_NO_POSIX_SHELL`** naming Git for Windows (https://git-scm.com/download/win) as the fix if none is found. Resolution is memoized for the lifetime of the process. Crucially, inline lines are **never** translated to `cmd`/PowerShell — Jaiph's language semantics require POSIX `sh` on every platform, or workflows stop being portable — so the seam only ever resolves *which* `sh` to run, never rewrites the command. POSIX inline-shell semantics are unchanged (existing e2e passes). New unit tests (`src/runtime/kernel/portability.test.ts`) stub `process.platform` and the `PATH` / existence lookup: POSIX returns `sh`, `win32` returns a discovered `sh.exe` path (both the on-`PATH` and Git-for-Windows-fallback branches), `win32` with no shell available throws `E_NO_POSIX_SHELL` with a message naming Git for Windows, and the result is memoized (the existence probe is not consulted on a second call); a `src/`-wide lint test asserts no production file outside `portability.ts` calls `spawn("sh"` / `spawn('sh'` directly. Docs updated (`docs/architecture.md`). - **Fix — Portability: execute script steps via an explicit interpreter, not shebang + exec bit:** Script steps used to run by spawning the emitted file directly (`executeScript` in `src/runtime/kernel/node-workflow-runtime.ts`), relying on the `#!/usr/bin/env ` shebang line and the `0o755` bit set by `src/transpile/build.ts`. Windows honors neither, and `noexec` mounts on Linux strip the exec bit — both break script execution. The runtime already knows the interpreter (it writes the shebang itself), so `executeScript` now reads the emitted script's shebang and resolves the interpreter through the new **`resolveInterpreterFromShebang`** (`src/parse/script-bash.ts`) — `#!/usr/bin/env ` → spawn ``, an absolute-path shebang (`#!/bin/bash`) → spawn that path, an `env -S` split-flag form → the real interpreter after `-S`, a missing/blank shebang → default `bash` — then spawns ` ` explicitly. The shebang line is **still** written into every emitted script (they stay directly executable by hand on POSIX) and the `0o755` bit is still set, but the runtime no longer depends on either being honored by the OS. A spawn `ENOENT` from an interpreter missing on `PATH` now surfaces as a diagnosable Jaiph error naming the interpreter (`script interpreter "" not found — install it or fix the script shebang`) instead of a raw `spawn ENOENT`. New unit tests assert the resolved interpreter + argv on the spawn call itself through an injectable `_scriptSpawn` seam (`src/runtime/kernel/node-workflow-runtime.script-exec.test.ts`), prove that a script with the exec bit stripped (`0o644`) still runs on POSIX, and prove that a missing-interpreter shebang produces the diagnosable error; `resolveInterpreterFromShebang` gets its own unit coverage (`src/parse/script-bash.test.ts`). Docs updated (`docs/architecture.md`, `docs/setup.md`). diff --git a/QUEUE.md b/QUEUE.md index b55ccf29..6f022f58 100644 --- a/QUEUE.md +++ b/QUEUE.md @@ -14,21 +14,6 @@ Process rules: *** -## Distro: build and release `jaiph-windows-x64.exe` #dev-ready - -The release workflow (`.github/workflows/release.yml`) cross-compiles four assets (`jaiph-{darwin,linux}-{arm64,x64}`) via `bun build --compile` and publishes them with `SHA256SUMS`. Add Windows: - -* New matrix entry: `--target=bun-windows-x64`, asset name `jaiph-windows-x64.exe` (Bun has no Windows arm64 target — do not add one). -* Include the `.exe` in `SHA256SUMS` generation and in both the stable and nightly `gh release` upload lists. -* Add a Windows sanity gate alongside the existing linux-x64 one: run `jaiph-windows-x64.exe --version` on a `windows-latest` job and, for stable releases, assert the output matches the tag. -* Update the "Release asset naming contract" in `docs/contributing.md` to include the new asset name. - -Acceptance: - -* A nightly release run publishes five binaries + `SHA256SUMS`, and the `.exe` checksum entry verifies against the downloaded asset. -* The Windows sanity gate fails the release when `--version` output mismatches the tag (verified by test or a deliberate dry-run with a wrong version). -* `docs/contributing.md` naming contract lists `jaiph-windows-x64.exe`; the e2e installer test's asset-name mapping and the contract stay in sync (grep/parity check). - ## Distro: PowerShell installer at `jaiph.org/install.ps1` #dev-ready The bash installer (`docs/install`) downloads a per-platform binary from the GitHub Release for a pinned ref and verifies it against `SHA256SUMS`. Windows users need a native equivalent — the bash script must keep rejecting Windows and point at the PowerShell one. diff --git a/docs/architecture.md b/docs/architecture.md index d72c38de..4e8a7ce8 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -176,7 +176,7 @@ The progress UI combines a **static** step tree derived from the workflow AST (` - **Development / npm:** `npm run build` runs `npm run embed-assets` (regenerates **`src/runtime/embedded-assets.ts`** from `runtime/overlay-run.sh` and `docs/jaiph-skill.md`, and **`src/version.ts`** from `package.json`'s `version` field), then `tsc`, copies **`src/runtime/`** to **`dist/src/runtime/`** (kernel, `docker.ts`, etc.), and copies **`runtime/overlay-run.sh`** from the repo root into **`dist/src/runtime/overlay-run.sh`**. The published `jaiph` bin is **`node dist/src/cli.js`**. - **Standalone:** `npm run build:standalone` runs the same build, copies **`dist/src/runtime`** to **`dist/runtime`** beside the binary, then `bun build --compile ./src/cli.ts --outfile dist/jaiph`. Workflow launch self-spawns via **`process.execPath`** using the internal **`__workflow-runner`** argv marker (`src/runtime/kernel/workflow-launch.ts` + `src/cli/index.ts`): the node build invokes `node dist/src/cli.js __workflow-runner …`; the bun-compiled binary invokes itself, `jaiph __workflow-runner …`. The reserved marker is excluded from `--help`/usage and the file-shorthand path. `overlay-run.sh` and `docs/jaiph-skill.md` are also embedded base64 inside the executable via **`src/runtime/embedded-assets.ts`**, so the standalone artifact is **fully self-contained** — no sibling `runtime/` or `docs/` files required. The displayed `jaiph --version` string is sourced from the generated **`src/version.ts`** (codegen'd from `package.json` by `embed-assets`), so the literal is statically baked into both the `tsc` and the `bun build --compile` outputs without a runtime read of `package.json`. **Bash** (or whatever shebang your `script` steps use) is still required on the host for script subprocesses. Ship **`dist/jaiph`** alone, or with **`dist/runtime`** alongside it for parity with the npm layout (table in [Contributing](contributing.md)). -- **Release artifacts:** `.github/workflows/release.yml` cross-compiles the standalone binary for **darwin/linux × arm64/x64** on **`v*`** tag pushes and on pushes to the **`nightly`** branch, generates a `SHA256SUMS` covering the four binaries, runs a `--version` sanity gate on the linux-x64 output, and uploads the five assets to the matching GitHub Release (stable tag or rolling **`nightly`** prerelease). Asset filenames are fixed by the installer contract — see [Contributing — Release asset naming contract](contributing.md#release-asset-naming-contract). +- **Release artifacts:** `.github/workflows/release.yml` cross-compiles the standalone binary for **darwin/linux × arm64/x64** plus **windows x64** (`jaiph-windows-x64.exe`; Bun has no windows arm64 target) on **`v*`** tag pushes and on pushes to the **`nightly`** branch, generates a `SHA256SUMS` covering the five binaries, runs `--version` sanity gates on the linux-x64 and windows-x64 outputs, and uploads the six assets to the matching GitHub Release (stable tag or rolling **`nightly`** prerelease). Asset filenames are fixed by the installer contract — see [Contributing — Release asset naming contract](contributing.md#release-asset-naming-contract). ## Mermaid architecture diagram diff --git a/docs/contributing.md b/docs/contributing.md index 05119b24..91e422fe 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -212,7 +212,7 @@ The workflow refuses to start when the git tree is dirty or when `v` al Pushing a **`v*`** tag triggers two things in this repo: 1. **Docker image publish** — the `docker-publish` job in `ci.yml` pushes `ghcr.io/jaiphlang/jaiph-runtime:` and `:latest` after the other CI jobs succeed. -2. **Standalone-binary release** — `.github/workflows/release.yml` cross-compiles the Bun-compiled standalone binary for four targets via `oven-sh/setup-bun` and `bun build --compile --target=…`, generates a `SHA256SUMS` file, runs a Linux x64 sanity gate (`./jaiph-linux-x64 --version` must equal `jaiph `), and uploads all five assets to the GitHub Release for the tag (creating it if needed). The release job waits for the `CI` workflow on the same SHA to succeed before publishing. Re-runs are available via `workflow_dispatch`. +2. **Standalone-binary release** — `.github/workflows/release.yml` cross-compiles the Bun-compiled standalone binary for five targets via `oven-sh/setup-bun` and `bun build --compile --target=…`, generates a `SHA256SUMS` file, runs a Linux x64 sanity gate and a `windows-latest` Windows x64 sanity gate (`--version` must equal `jaiph ` for stable tags — both delegate to `scripts/release-version-check.sh`), and uploads all six assets to the GitHub Release for the tag (creating it if needed). The Windows gate is a required dependency of the publish job, so a version mismatch there fails the whole release. The release job waits for the `CI` workflow on the same SHA to succeed before publishing. Re-runs are available via `workflow_dispatch`. Pushes to the **`nightly`** branch follow the same matrix and upload to a **rolling prerelease** tagged `nightly` (`gh release upload nightly --clobber`), so `jaiph use nightly` keeps working under the binary installer. @@ -228,9 +228,10 @@ The installer (`docs/install`) downloads these exact filenames from the release | `bun-darwin-x64` | `jaiph-darwin-x64` | | `bun-linux-x64` | `jaiph-linux-x64` | | `bun-linux-arm64` | `jaiph-linux-arm64` | -| — | `SHA256SUMS` (covers all four binaries) | +| `bun-windows-x64` | `jaiph-windows-x64.exe` | +| — | `SHA256SUMS` (covers all five binaries) | -Every release (stable `v*` and rolling `nightly`) ships exactly these five assets. +Bun has no `bun-windows-arm64` target, so Windows ships x64 only. Every release (stable `v*` and rolling `nightly`) ships exactly these six assets. ### Local docs site (Jekyll) diff --git a/integration/release-workflow.test.ts b/integration/release-workflow.test.ts new file mode 100644 index 00000000..522b3bf5 --- /dev/null +++ b/integration/release-workflow.test.ts @@ -0,0 +1,201 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { readFileSync, mkdtempSync, writeFileSync, rmSync } from "node:fs"; +import { createHash, randomBytes } from "node:crypto"; +import { spawnSync } from "node:child_process"; +import { join } from "node:path"; +import { tmpdir } from "node:os"; + +// Acceptance for "Distro: build and release jaiph-windows-x64.exe". +// +// These guards turn each acceptance bullet into a check that fails when the +// contract is violated: +// 1. The release workflow builds/ships five binaries + SHA256SUMS, and a +// checksum entry for the .exe verifies against the asset (round-trip via +// the installer's own lookup). +// 2. The shared version-check gate (scripts/release-version-check.sh, invoked +// by both the linux-x64 and the windows-latest gate) fails on a +// tag/version mismatch and passes on a match. +// 3. The docs/contributing.md naming contract, the release matrix, the +// installer (docs/install), and the e2e installer test all agree on the +// asset names (grep/parity check). + +const REPO_ROOT = process.cwd(); +const RELEASE_YML = readFileSync(join(REPO_ROOT, ".github/workflows/release.yml"), "utf8"); +const CONTRIBUTING = readFileSync(join(REPO_ROOT, "docs/contributing.md"), "utf8"); +const INSTALLER = readFileSync(join(REPO_ROOT, "docs/install"), "utf8"); +const INSTALLER_TEST = readFileSync(join(REPO_ROOT, "e2e/tests/07_installer_binary.sh"), "utf8"); +const VERSION_CHECK = join(REPO_ROOT, "scripts/release-version-check.sh"); + +// Single source of truth for the assets a release must ship. +const BINARY_ASSETS = [ + "jaiph-darwin-arm64", + "jaiph-darwin-x64", + "jaiph-linux-x64", + "jaiph-linux-arm64", + "jaiph-windows-x64.exe", +]; +// Names the bash installer (and its e2e test) can construct from {os}×{arch}. +const INSTALLER_ASSETS = [ + "jaiph-darwin-arm64", + "jaiph-darwin-x64", + "jaiph-linux-x64", + "jaiph-linux-arm64", +]; + +// Slice a workflow's job/step body out of the YAML by a stable anchor so +// per-section assertions don't accidentally match text from another job. +function sliceBetween(text: string, start: string, end: string | null): string { + const from = text.indexOf(start); + assert.notEqual(from, -1, `expected to find "${start}" in workflow`); + const to = end === null ? text.length : text.indexOf(end, from + start.length); + assert.notEqual(to, -1, `expected to find "${end}" after "${start}"`); + return text.slice(from, to === text.length ? text.length : to); +} + +// ── Acceptance 1: five binaries + SHA256SUMS are built and uploaded ─────────── + +test("release matrix cross-compiles the windows-x64 target (x64 only)", () => { + assert.match(RELEASE_YML, /target:\s*bun-windows-x64/, "windows-x64 target present"); + const winEntry = sliceBetween(RELEASE_YML, "target: bun-windows-x64", "steps:"); + assert.match(winEntry, /os:\s*windows/); + assert.match(winEntry, /arch:\s*x64/); + assert.match(winEntry, /ext:\s*"\.exe"/); + // Bun has no windows arm64 target — do not add one. + assert.doesNotMatch(RELEASE_YML, /target:\s*bun-windows-arm64/, "no windows arm64 target"); + // The four original targets are still built. + for (const target of ["bun-darwin-arm64", "bun-darwin-x64", "bun-linux-x64", "bun-linux-arm64"]) { + assert.match(RELEASE_YML, new RegExp(`target:\\s*${target}\\b`), `${target} still built`); + } +}); + +test("SHA256SUMS generation covers all five binaries including the .exe", () => { + const shaLine = RELEASE_YML.split("\n").find((l) => l.includes("sha256sum ") && l.includes("SHA256SUMS")); + assert.ok(shaLine, "found the sha256sum generation line"); + for (const asset of BINARY_ASSETS) { + assert.ok(shaLine!.includes(asset), `SHA256SUMS covers ${asset}`); + } +}); + +test("both stable and nightly release uploads include the .exe and SHA256SUMS", () => { + const stable = sliceBetween(RELEASE_YML, "Publish stable release", "Publish nightly prerelease"); + const nightly = sliceBetween(RELEASE_YML, "Publish nightly prerelease", null); + for (const section of [stable, nightly]) { + for (const asset of [...BINARY_ASSETS, "SHA256SUMS"]) { + assert.ok(section.includes(asset), `upload list includes ${asset}`); + } + } +}); + +test("a SHA256SUMS entry for the .exe verifies against the asset via the installer's lookup", () => { + // Mirror the release: hash a windows binary, write the SHA256SUMS line, then + // resolve it back with the exact awk lookup docs/install uses. A mismatch + // between "generation" and "verification" would fail here. + const dir = mkdtempSync(join(tmpdir(), "jaiph-sha-")); + try { + const asset = "jaiph-windows-x64.exe"; + const binPath = join(dir, asset); + const bytes = randomBytes(4096); + writeFileSync(binPath, bytes); + const expected = createHash("sha256").update(bytes).digest("hex"); + const sumsPath = join(dir, "SHA256SUMS"); + writeFileSync(sumsPath, `${expected} ${asset}\n`); + + // The installer resolves a checksum with this awk expression. + assert.match(INSTALLER, /awk -v name="\$\{BIN_NAME\}" '\$2 == name \|\| \$2 == "\*"name \{ print \$1 \}'/); + const looked = spawnSync( + "awk", + ["-v", `name=${asset}`, '$2 == name || $2 == "*"name { print $1 }', sumsPath], + { encoding: "utf8" }, + ); + assert.equal(looked.status, 0, looked.stderr); + assert.equal(looked.stdout.trim(), expected, "installer lookup returns the asset's checksum"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +}); + +// ── Acceptance 2: the version sanity gate fails on a tag/version mismatch ────── + +function runVersionCheck(channel: string, tag: string, got: string) { + return spawnSync("bash", [VERSION_CHECK, channel, tag, got], { encoding: "utf8" }); +} + +test("version sanity gate fails when a stable --version mismatches the tag", () => { + const bad = runVersionCheck("stable", "v9.9.9", "jaiph 1.2.3"); + assert.equal(bad.status, 1, "mismatch exits non-zero"); + assert.match(bad.stderr, /Version sanity check failed/); + + const good = runVersionCheck("stable", "v1.2.3", "jaiph 1.2.3"); + assert.equal(good.status, 0, good.stderr); +}); + +test("version sanity gate only requires a version-shaped banner for nightly", () => { + const good = runVersionCheck("nightly", "nightly", "jaiph 0.10.0"); + assert.equal(good.status, 0, good.stderr); + + const bad = runVersionCheck("nightly", "nightly", "not-a-version"); + assert.equal(bad.status, 1, "garbage banner fails even on nightly"); + assert.match(bad.stderr, /Version sanity check failed/); +}); + +test("a windows-latest job runs the .exe --version through the shared gate and blocks publish", () => { + const job = sliceBetween(RELEASE_YML, "sanity-windows:", "\n release:"); + assert.match(job, /runs-on:\s*windows-latest/); + assert.match(job, /jaiph-windows-x64\.exe --version/); + assert.match(job, /release-version-check\.sh/, "windows gate delegates to the shared script"); + // The linux gate uses the same shared script (no duplicated comparison logic). + const linux = sliceBetween(RELEASE_YML, "Sanity gate (linux-x64 --version)", "Publish stable release"); + assert.match(linux, /release-version-check\.sh/); + // A windows gate failure must fail the release: publish depends on it. + assert.match(RELEASE_YML, /needs:\s*\[build, sanity-windows\]/, "release job needs sanity-windows"); +}); + +// ── Acceptance 3: contract ↔ matrix ↔ installer parity ──────────────────────── + +function contractAssets(): string[] { + const section = sliceBetween(CONTRIBUTING, "#### Release asset naming contract", "### Local docs site"); + const names = new Set(); + for (const m of section.matchAll(/`(jaiph-[A-Za-z0-9.\-]+|SHA256SUMS)`/g)) { + names.add(m[1]); + } + return [...names]; +} + +test("the naming contract lists exactly the five binaries plus SHA256SUMS", () => { + const listed = contractAssets().sort(); + const expected = [...BINARY_ASSETS, "SHA256SUMS"].sort(); + assert.deepEqual(listed, expected, "contract asset set matches the release"); + // The prose count stays in sync with the table. + assert.match(CONTRIBUTING, /exactly these six assets/); +}); + +test("release matrix builds exactly the binaries named in the contract", () => { + // Every contract binary maps to a matrix target of the same os/arch, and the + // matrix builds nothing the contract omits. + const matrixTargets = [...RELEASE_YML.matchAll(/target:\s*(bun-[a-z0-9-]+)/g)].map((m) => m[1]); + const built = matrixTargets + .map((t) => t.replace(/^bun-/, "jaiph-")) + .map((n) => (n === "jaiph-windows-x64" ? "jaiph-windows-x64.exe" : n)) + .sort(); + assert.deepEqual(built, [...BINARY_ASSETS].sort(), "matrix binaries == contract binaries"); +}); + +test("installer and its e2e test can only produce asset names the contract lists", () => { + // Installer + e2e test construct names from {os}×{arch}; pin the construction + // so a rename in the contract that isn't mirrored here fails the parity check. + assert.match(INSTALLER, /BIN_NAME="jaiph-\$\{os\}-\$\{arch\}"/); + assert.match(INSTALLER_TEST, /HOST_BIN_NAME="jaiph-\$\{HOST_OS\}-\$\{HOST_ARCH\}"/); + const listed = new Set(contractAssets()); + for (const asset of INSTALLER_ASSETS) { + assert.ok(listed.has(asset), `contract lists installer asset ${asset}`); + // Both bash sources still support the os/arch that produces this name. + const [, os, arch] = asset.split("-"); + for (const src of [INSTALLER, INSTALLER_TEST]) { + assert.ok(src.includes(os), `bash source supports os ${os}`); + assert.ok(src.includes(arch), `bash source supports arch ${arch}`); + } + } + // The .exe is release-only; the bash installer does not download it. + assert.ok(listed.has("jaiph-windows-x64.exe")); +}); diff --git a/scripts/release-version-check.sh b/scripts/release-version-check.sh new file mode 100755 index 00000000..0cd0c5c2 --- /dev/null +++ b/scripts/release-version-check.sh @@ -0,0 +1,34 @@ +#!/usr/bin/env bash +# +# Sanity-check a compiled jaiph binary's `--version` output against the release +# channel/tag. Shared by the linux-x64 and windows-x64 gates in +# `.github/workflows/release.yml` so the comparison lives in one place and is +# unit-testable (see integration/release-workflow.test.ts). +# +# Usage: release-version-check.sh +# channel: "stable" | "nightly" +# tag: release tag (e.g. "v0.10.0" for stable, "nightly" otherwise) +# version-output: the string printed by ` --version` +# +# For a stable release the output must equal "jaiph " exactly. +# For any other channel it only has to look like a jaiph version banner. +set -euo pipefail + +channel="${1:?channel required}" +tag="${2:?tag required}" +got="${3?version output required}" + +if [ "${channel}" = "stable" ]; then + expected="jaiph ${tag#v}" + if [ "${got}" != "${expected}" ]; then + echo "Version sanity check failed: expected '${expected}', got '${got}'" >&2 + exit 1 + fi +else + if ! printf '%s\n' "${got}" | grep -Eq '^jaiph [0-9]+\.[0-9]+\.[0-9]+'; then + echo "Version sanity check failed: '${got}' does not look like a jaiph version" >&2 + exit 1 + fi +fi + +echo "Version sanity check passed: ${got}" From dca4ea52c01a93111ac9ed590285b66d2255773f Mon Sep 17 00:00:00 2001 From: Jakub Dzikowski Date: Mon, 13 Jul 2026 14:44:42 +0200 Subject: [PATCH 06/85] Feat: PowerShell installer at jaiph.org/install.ps1 Add docs/install.ps1 as the Windows counterpart to the POSIX docs/install (run with irm https://jaiph.org/install.ps1 | iex). It downloads jaiph-windows-x64.exe and SHA256SUMS from the pinned release ref (default the current stable tag, overridable via JAIPH_REPO_REF / first arg plus JAIPH_RELEASE_BASE_URL for local file:// dirs used by tests), verifies the SHA-256 with Get-FileHash and installs nothing on mismatch, then installs to %LOCALAPPDATA%\jaiph\bin\jaiph.exe (overridable via JAIPH_BIN_DIR), adds it to the user PATH if absent, and prints the same try-it hints as the bash installer. Non-x64 / ARM Windows exits non-zero with a documented unsupported-platform message. The bash installer now rejects Windows-like uname values and points at the PowerShell one-liner, and prepare_release.jh rewrites the pinned ref in both installers in lockstep. CI gains an installer-powershell job on windows-latest that cross-compiles the real .exe and runs e2e/tests/installer_powershell.ps1 (checksum mismatch, unsupported arch, happy-path install with no Node/npm/Bun on PATH); the Docker publish job now needs it. Host-portable guards live in integration/installer-powershell.test.ts. Docs updated (setup.md, index.html, architecture.md, contributing.md, README). Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/ci.yml | 39 ++++++- .jaiph/prepare_release.jh | 26 +++-- CHANGELOG.md | 1 + QUEUE.md | 18 --- README.md | 6 + docs/architecture.md | 2 +- docs/contributing.md | 5 +- docs/index.html | 4 + docs/install | 7 ++ docs/install.ps1 | 136 ++++++++++++++++++++++ docs/setup.md | 9 ++ e2e/tests/installer_powershell.ps1 | 141 +++++++++++++++++++++++ integration/installer-powershell.test.ts | 111 ++++++++++++++++++ 13 files changed, 472 insertions(+), 33 deletions(-) create mode 100644 docs/install.ps1 create mode 100644 e2e/tests/installer_powershell.ps1 create mode 100644 integration/installer-powershell.test.ts diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c911083d..dfa0a737 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -275,9 +275,46 @@ jobs: $bashScript = $bashScript -replace "`r", "" wsl -d "$distro" -- bash -lc "$bashScript" + installer-powershell: + name: PowerShell installer (windows-x64) + runs-on: windows-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Setup Node + uses: actions/setup-node@v4 + with: + node-version: "20" + cache: npm + + - name: Setup Bun + uses: oven-sh/setup-bun@v2 + + - name: Install dependencies + run: npm ci + + - name: Build TypeScript + embed assets + run: npm run build + + # Same build as release.yml's windows-x64 leg, so the acceptance runs the + # real self-contained binary the release ships. + - name: Cross-compile windows-x64 standalone binary + shell: bash + run: | + set -euo pipefail + bun build --compile --target=bun-windows-x64 ./src/cli.ts --outfile jaiph-windows-x64.exe + ls -la jaiph-windows-x64.exe + + - name: Run PowerShell installer acceptance (docs/install.ps1) + shell: pwsh + run: | + $env:JAIPH_TEST_WINDOWS_EXE = Join-Path $env:GITHUB_WORKSPACE "jaiph-windows-x64.exe" + ./e2e/tests/installer_powershell.ps1 + docker-publish: name: Publish Docker runtime image - needs: [test, e2e, docs-local, e2e-wsl] + needs: [test, e2e, docs-local, e2e-wsl, installer-powershell] if: github.ref == 'refs/heads/nightly' || startsWith(github.ref, 'refs/tags/v') runs-on: ubuntu-latest permissions: diff --git a/.jaiph/prepare_release.jh b/.jaiph/prepare_release.jh index a110429c..a57c7089 100755 --- a/.jaiph/prepare_release.jh +++ b/.jaiph/prepare_release.jh @@ -58,18 +58,20 @@ script npm_version_no_tag = `npm version "$1" --no-git-tag-version --allow-same- script update_install_release_ref = ```python3 import sys old, new = sys.argv[1], sys.argv[2] -path = "docs/install" -with open(path, "r", encoding="utf-8") as f: - src = f.read() needle = f"v{old}" -count = src.count(needle) -if count == 0: - sys.stderr.write(f"docs/install: hardcoded ref v{old} not found\n") - sys.exit(1) -new_src = src.replace(needle, f"v{new}") -with open(path, "w", encoding="utf-8") as f: - f.write(new_src) -print(count) +total = 0 +# Both installers pin the same hardcoded ref; keep them in lockstep. +for path in ("docs/install", "docs/install.ps1"): + with open(path, "r", encoding="utf-8") as f: + src = f.read() + count = src.count(needle) + if count == 0: + sys.stderr.write(f"{path}: hardcoded ref v{old} not found\n") + sys.exit(1) + with open(path, "w", encoding="utf-8") as f: + f.write(src.replace(needle, f"v{new}")) + total += count +print(total) ``` script run_npm_build = `npm run build >&2` @@ -124,7 +126,7 @@ workflow default(arg) { log """ prepare_release: staged release v${version} - package.json + package-lock.json (npm version ${version}) - - docs/install (release ref v${old_version} -> v${version}) + - docs/install + docs/install.ps1 (release ref v${old_version} -> v${version}) - docs/registry (regenerated) - dist/ (rebuilt; jaiph --version == jaiph ${version}) diff --git a/CHANGELOG.md b/CHANGELOG.md index 32dfc3b2..3a6a2f11 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,7 @@ ## All changes +- **Feature — Distro: PowerShell installer at `jaiph.org/install.ps1`:** Windows now has a native install path to match the POSIX `curl … | bash` one. A new **`docs/install.ps1`** (served as `https://jaiph.org/install.ps1`, run with `irm https://jaiph.org/install.ps1 | iex`) is the counterpart to `docs/install`: it downloads **`jaiph-windows-x64.exe`** and `SHA256SUMS` from the pinned release ref (default the current stable tag `v0.10.0`, overridable via `JAIPH_REPO_REF` or the first argument — mirroring the bash installer — plus `JAIPH_RELEASE_BASE_URL` for a local/`file://` release dir used by the tests), verifies the SHA-256 with **`Get-FileHash`** against `SHA256SUMS` and aborts installing nothing on mismatch, then installs to **`%LOCALAPPDATA%\jaiph\bin\jaiph.exe`** (overridable via `JAIPH_BIN_DIR`), adds that directory to the user `PATH` if absent, and prints the same `jaiph --version` / `jaiph --help` try-it hints as the bash installer. Non-x64 / ARM Windows exits non-zero with a documented unsupported-platform message pointing at the from-source instructions (Bun has no Windows arm64 target, so Windows ships x64 only). The bash installer (`docs/install`) now rejects Windows-like `uname` values (`MINGW*`/`MSYS*`/`CYGWIN*`/`Windows_NT`) and points at the PowerShell one-liner instead of the generic build-from-source message; the release-prep workflow (`.jaiph/prepare_release.jh`) rewrites the pinned `vX.Y.Z` ref in **both** `docs/install` and `docs/install.ps1` in lockstep so they can never drift. CI gains an **`installer-powershell`** job on `windows-latest` that cross-compiles the real `jaiph-windows-x64.exe` (same build as the release leg) and runs `e2e/tests/installer_powershell.ps1` against `docs/install.ps1` — checksum mismatch (non-zero, no binary left), unsupported arch (documented message), and a happy-path install where `jaiph --version` works with no Node/npm/Bun on `PATH`; the Docker publish job now `needs` it. Host-portable guards live in `integration/installer-powershell.test.ts` (installer contract, bash↔PowerShell lockstep ref, docs parity). Docs updated (`docs/setup.md`, `docs/index.html`, `docs/architecture.md`, `docs/contributing.md`). - **Feature — Distro: build and release `jaiph-windows-x64.exe`:** The release workflow (`.github/workflows/release.yml`) now cross-compiles and publishes a Windows binary alongside the existing darwin/linux × arm64/x64 set. A new matrix entry `bun-windows-x64` (`os: windows`, `arch: x64`) produces the asset **`jaiph-windows-x64.exe`**; Bun has no `bun-windows-arm64` target, so Windows ships x64 only. The matrix gained an `ext` field (`""` for POSIX, `".exe"` for Windows) so the compiled `--outfile` and the uploaded artifact carry the extension. `SHA256SUMS` generation now covers all **five** binaries including the `.exe`, and both the stable (`v*`) and rolling `nightly` `gh release` upload lists ship the `.exe` — so every release publishes **six** assets (five binaries + `SHA256SUMS`). A new **`sanity-windows`** job runs on `windows-latest`, downloads the `.exe` artifact, and runs `jaiph-windows-x64.exe --version` through the version gate; the publish job now `needs: [build, sanity-windows]`, so a Windows version mismatch fails the whole release. The linux-x64 gate's inline version comparison was extracted into a shared, unit-testable **`scripts/release-version-check.sh`** (` `: stable output must equal `jaiph ` exactly; any other channel only has to look like a `jaiph ` banner), and both the linux-x64 and windows-x64 gates delegate to it so the comparison lives in one place. The bash installer (`docs/install`) and its e2e test (`e2e/tests/07_installer_binary.sh`) are unchanged — they resolve darwin/linux names from `{os}×{arch}` and cannot install a Windows `.exe` via `curl … | bash`; Windows users download the `.exe` from the Release directly. New acceptance tests (`integration/release-workflow.test.ts`) assert the five-binary matrix (and the absence of a windows-arm64 target), that `SHA256SUMS` and both upload lists include the `.exe`, that a `.exe` checksum entry round-trips through the installer's own `awk` lookup, that the shared gate fails on a stable tag/version mismatch and passes on a match, that the `windows-latest` job runs `--version` through the shared script and blocks publish, and that the naming contract ↔ release matrix ↔ installer asset names stay in parity. Docs updated (`docs/architecture.md`, `docs/contributing.md`). - **Fix — Portability: home-dir fallback, Docker gating on Windows, and a single ANSI policy:** Three remaining POSIX assumptions that broke a host-only `win32` runtime are closed. (1) **Home directory** — `prepareClaudeEnv` (`src/runtime/kernel/prompt.ts`) resolved the `.claude` config dir from `execEnv.HOME || process.env.HOME`, which is empty in a `USERPROFILE`-only Windows environment. It now falls back to `os.homedir()` as a final source (`execEnv.HOME || process.env.HOME || homedir()`); an explicit `HOME` in `execEnv` still wins. (2) **Docker gating** — the Docker sandbox hardcodes POSIX socket paths and Linux/macOS workspace-presentation branches, so it is out of scope on Windows. `resolveDockerConfig` (`src/runtime/docker.ts`) now forces **host-only mode** on `win32` (same UX as an explicit `JAIPH_UNSAFE=true`) with a one-line notice emitted once per process, so the CLI never probes `docker` and never hard-fails on a missing daemon; `JAIPH_DOCKER_ENABLED=true` cannot override this. (3) **ANSI colors** — a new **`canUseAnsi(stream?)`** helper in the portability module (`src/runtime/kernel/portability.ts`) returns `isTTY && NO_COLOR`-unset, and every color/erase emission site (`src/cli/commands/run.ts`, `src/cli/run/progress.ts`, `src/cli/shared/errors.ts`) routes its gate through it so the policy lives in one place; no rendering change is needed because Node enables console VT processing on Windows 10+ automatically, making `isTTY` a sufficient ANSI proxy. New unit tests: `prepareClaudeEnv` resolves from a stubbed `os.homedir()` when `HOME` is unset and prefers `execEnv.HOME` when set (`src/runtime/kernel/prompt.test.ts`); `resolveDockerConfig` under a stubbed `win32` returns host-only, emits the notice once, and performs zero `docker` invocations (`src/runtime/docker.test.ts`); `canUseAnsi()` is false when `isTTY` is false or `NO_COLOR` is set, plus a `src/`-wide lint test asserting no production file outside `portability.ts` gates directly on `isTTY && NO_COLOR` (`src/runtime/kernel/portability.test.ts`). Docs updated (`docs/architecture.md`, `docs/sandboxing.md`, `docs/configuration.md`, `docs/env-vars.md`). - **Fix — Portability: single `resolveShell()` seam for inline shell lines and hooks:** Inline workflow shell lines (`executeShLine` in `src/runtime/kernel/node-workflow-runtime.ts`) and hook commands (`src/cli/run/hooks.ts`) used to run via a hardcoded `spawn("sh", ["-c", …])`. That works on POSIX but fails on `win32`, where there is no `sh` on the default `PATH`. A new **`resolveShell(): string`** in the portability module (`src/runtime/kernel/portability.ts`) is now the single seam both call sites go through. On POSIX it returns bare `sh`. On `win32` it locates Git for Windows' bundled `sh.exe` — first on `PATH`, then in the standard install layouts (`/bin/sh.exe`, `/usr/bin/sh.exe`) under each known root (`%ProgramFiles%`, `%ProgramFiles(x86)%`, `%ProgramW6432%`, plus the default `C:\Program Files` / `C:\Program Files (x86)`) — and throws a diagnosable Jaiph error with the stable code **`E_NO_POSIX_SHELL`** naming Git for Windows (https://git-scm.com/download/win) as the fix if none is found. Resolution is memoized for the lifetime of the process. Crucially, inline lines are **never** translated to `cmd`/PowerShell — Jaiph's language semantics require POSIX `sh` on every platform, or workflows stop being portable — so the seam only ever resolves *which* `sh` to run, never rewrites the command. POSIX inline-shell semantics are unchanged (existing e2e passes). New unit tests (`src/runtime/kernel/portability.test.ts`) stub `process.platform` and the `PATH` / existence lookup: POSIX returns `sh`, `win32` returns a discovered `sh.exe` path (both the on-`PATH` and Git-for-Windows-fallback branches), `win32` with no shell available throws `E_NO_POSIX_SHELL` with a message naming Git for Windows, and the result is memoized (the existence probe is not consulted on a second call); a `src/`-wide lint test asserts no production file outside `portability.ts` calls `spawn("sh"` / `spawn('sh'` directly. Docs updated (`docs/architecture.md`). diff --git a/QUEUE.md b/QUEUE.md index 6f022f58..5028cbc8 100644 --- a/QUEUE.md +++ b/QUEUE.md @@ -14,24 +14,6 @@ Process rules: *** -## Distro: PowerShell installer at `jaiph.org/install.ps1` #dev-ready - -The bash installer (`docs/install`) downloads a per-platform binary from the GitHub Release for a pinned ref and verifies it against `SHA256SUMS`. Windows users need a native equivalent — the bash script must keep rejecting Windows and point at the PowerShell one. - -Add `docs/install.ps1` (served as `https://jaiph.org/install.ps1`, `irm https://jaiph.org/install.ps1 | iex`): - -* Downloads `jaiph-windows-x64.exe` and `SHA256SUMS` from the pinned release ref (default: current stable tag; overridable via `JAIPH_REPO_REF` / first argument, mirroring the bash installer). -* Verifies the SHA-256 (`Get-FileHash`) against `SHA256SUMS`; mismatch aborts and installs nothing. -* Installs to `%LOCALAPPDATA%\jaiph\bin\jaiph.exe` (overridable via `JAIPH_BIN_DIR`), adds the directory to the user `PATH` if absent, and prints the same try-it hints as the bash installer. -* Non-x64 / ARM Windows: exit with a documented unsupported-platform message. -* Update `docs/install`'s unsupported-platform message to mention the PowerShell installer for Windows, and document the Windows install path in `docs/setup.md` and the main page install tabs. - -Acceptance: - -* Pester (or equivalent) tests run on `windows-latest` in CI, pointing the installer at a `file://`-style local release directory (same technique as `e2e/tests/07_installer_binary.sh`): happy path installs and `jaiph --version` works; checksum mismatch exits non-zero and leaves no binary; unsupported arch exits with the documented message. -* The installed binary runs from a shell with no Node/npm/Bun on `PATH`. -* `docs/setup.md` and the main page reference the PowerShell one-liner (docs parity check passes). - ## Distro: main-page install tabs — Windows variant with platform auto-detect #dev-ready The main page (`docs/index.html`) hero has three install tabs (run sample / init project / just install), all showing bash `curl … | bash` one-liners. Tab switching in `docs/assets/js/main.js` is click-only; there is no platform detection. Windows visitors currently see commands that cannot run natively. diff --git a/README.md b/README.md index 82fb3d97..bd6dbbe6 100644 --- a/README.md +++ b/README.md @@ -58,6 +58,12 @@ Requires `node` and `curl`. The script installs Jaiph automatically if needed. curl -fsSL https://jaiph.org/install | bash ``` +On Windows, install with PowerShell instead (installs `jaiph-windows-x64.exe` to `%LOCALAPPDATA%\jaiph\bin`): + +```powershell +irm https://jaiph.org/install.ps1 | iex +``` + Or install from npm: ```bash diff --git a/docs/architecture.md b/docs/architecture.md index 4e8a7ce8..e9bd19e9 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -176,7 +176,7 @@ The progress UI combines a **static** step tree derived from the workflow AST (` - **Development / npm:** `npm run build` runs `npm run embed-assets` (regenerates **`src/runtime/embedded-assets.ts`** from `runtime/overlay-run.sh` and `docs/jaiph-skill.md`, and **`src/version.ts`** from `package.json`'s `version` field), then `tsc`, copies **`src/runtime/`** to **`dist/src/runtime/`** (kernel, `docker.ts`, etc.), and copies **`runtime/overlay-run.sh`** from the repo root into **`dist/src/runtime/overlay-run.sh`**. The published `jaiph` bin is **`node dist/src/cli.js`**. - **Standalone:** `npm run build:standalone` runs the same build, copies **`dist/src/runtime`** to **`dist/runtime`** beside the binary, then `bun build --compile ./src/cli.ts --outfile dist/jaiph`. Workflow launch self-spawns via **`process.execPath`** using the internal **`__workflow-runner`** argv marker (`src/runtime/kernel/workflow-launch.ts` + `src/cli/index.ts`): the node build invokes `node dist/src/cli.js __workflow-runner …`; the bun-compiled binary invokes itself, `jaiph __workflow-runner …`. The reserved marker is excluded from `--help`/usage and the file-shorthand path. `overlay-run.sh` and `docs/jaiph-skill.md` are also embedded base64 inside the executable via **`src/runtime/embedded-assets.ts`**, so the standalone artifact is **fully self-contained** — no sibling `runtime/` or `docs/` files required. The displayed `jaiph --version` string is sourced from the generated **`src/version.ts`** (codegen'd from `package.json` by `embed-assets`), so the literal is statically baked into both the `tsc` and the `bun build --compile` outputs without a runtime read of `package.json`. **Bash** (or whatever shebang your `script` steps use) is still required on the host for script subprocesses. Ship **`dist/jaiph`** alone, or with **`dist/runtime`** alongside it for parity with the npm layout (table in [Contributing](contributing.md)). -- **Release artifacts:** `.github/workflows/release.yml` cross-compiles the standalone binary for **darwin/linux × arm64/x64** plus **windows x64** (`jaiph-windows-x64.exe`; Bun has no windows arm64 target) on **`v*`** tag pushes and on pushes to the **`nightly`** branch, generates a `SHA256SUMS` covering the five binaries, runs `--version` sanity gates on the linux-x64 and windows-x64 outputs, and uploads the six assets to the matching GitHub Release (stable tag or rolling **`nightly`** prerelease). Asset filenames are fixed by the installer contract — see [Contributing — Release asset naming contract](contributing.md#release-asset-naming-contract). +- **Release artifacts:** `.github/workflows/release.yml` cross-compiles the standalone binary for **darwin/linux × arm64/x64** plus **windows x64** (`jaiph-windows-x64.exe`; Bun has no windows arm64 target) on **`v*`** tag pushes and on pushes to the **`nightly`** branch, generates a `SHA256SUMS` covering the five binaries, runs `--version` sanity gates on the linux-x64 and windows-x64 outputs, and uploads the six assets to the matching GitHub Release (stable tag or rolling **`nightly`** prerelease). Asset filenames are fixed by the installer contract — see [Contributing — Release asset naming contract](contributing.md#release-asset-naming-contract). Two installers consume these assets: the POSIX **`docs/install`** (`curl … | bash`, darwin/linux; rejects Windows and points at the PowerShell one) and **`docs/install.ps1`** (`irm https://jaiph.org/install.ps1 | iex`, Windows x64), which downloads `jaiph-windows-x64.exe`, verifies it against `SHA256SUMS` with `Get-FileHash`, and installs to `%LOCALAPPDATA%\jaiph\bin`. ## Mermaid architecture diagram diff --git a/docs/contributing.md b/docs/contributing.md index 91e422fe..0c9751f2 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -171,6 +171,8 @@ Tests that span multiple modules, require subprocess/PTY harnesses, exercise pro | `integration/docs-reference-task5.test.ts` | Integration | Reference quadrant — permalinks, nav placement, `env-vars.md` source parity against `src/`, anti-tutorial shape guards | | `integration/docs-tutorials-task6.test.ts` | Integration | Tutorial quadrant — permalinks, `/getting-started` redirect absorption, runnable `first-workflow` snippet with documented output | | `integration/docs-nav-structure-task7.test.ts` | Integration | Nav spine — five Diátaxis section headings in documented order; every published page under its quadrant exactly once | +| `integration/release-workflow.test.ts` | Integration | Release matrix / asset-naming contract — five-binary matrix (no windows-arm64), `SHA256SUMS` + upload lists include `jaiph-windows-x64.exe`, shared version-gate script, naming contract ↔ matrix ↔ installer parity | +| `integration/installer-powershell.test.ts` | Integration | Windows PowerShell installer (`docs/install.ps1`) contract — download/verify/install steps, bash↔PowerShell lockstep release ref, and `docs/setup.md` / main-page one-liner parity | | `integration/sample-build/build.test.ts` | Integration | Build/transpile behavior — `buildScripts`, script extraction | | `integration/sample-build/cli-tree.test.ts` | Integration | CLI tree output rendering for sample workflows | | `integration/sample-build/run-core.test.ts` | Integration | Core runtime execution — workflow runs, step sequencing, artifacts | @@ -187,7 +189,7 @@ The `integration/sample-build/` directory also has a shared `helpers.ts` module ## CI pipeline -The project uses GitHub Actions (`.github/workflows/ci.yml`). The workflow defines **six** jobs; on a typical feature-branch push, **five** of them run. The sixth — **Publish Docker runtime image** — runs only on pushes to **`nightly`** and on **`v*`** version tags, after the test, E2E, docs, and WSL jobs succeed (ShellCheck is not a publish gate). It builds and pushes `ghcr.io/jaiphlang/jaiph-runtime` (the default `runtime.docker_image` / `JAIPH_DOCKER_IMAGE` when Docker sandboxing is on; see **Docker runtime helper** in [Architecture](architecture.md#core-components)). +The project uses GitHub Actions (`.github/workflows/ci.yml`). The workflow defines **seven** jobs; on a typical feature-branch push, **six** of them run. The seventh — **Publish Docker runtime image** — runs only on pushes to **`nightly`** and on **`v*`** version tags, after the test, E2E, docs, WSL, and PowerShell-installer jobs succeed (ShellCheck is not a publish gate). It builds and pushes `ghcr.io/jaiphlang/jaiph-runtime` (the default `runtime.docker_image` / `JAIPH_DOCKER_IMAGE` when Docker sandboxing is on; see **Docker runtime helper** in [Architecture](architecture.md#core-components)). | Job | Runner | Purpose | |-----|--------|---------| @@ -196,6 +198,7 @@ The project uses GitHub Actions (`.github/workflows/ci.yml`). The workflow defin | **E2E** | Matrix: **`ubuntu-latest` twice** + **`macos-latest`** | Job id `e2e`; in the Actions UI each leg appears as **`E2E (,