fix(driver): pin launchd drivers to committed code, and make an unpinned fire LOUD (#558) - #666
Merged
Merged
Conversation
…ned fire LOUD (#558) THE DEFECT — one root, three symptoms. launchd invokes each driver by absolute path into the shared clone, and everything a fire reads hangs off that single root (mission-control.sh:40 `REPO="${MISSION_WORKDIR:-$(dirname $0/../..)}"; cd "$REPO"`): the driver, the skill under .claude/skills/, and the charter under design_docs/. Nothing keeps that clone current and nothing reports the drift, so all three go stale together and every previous repair patched one artefact — a third of a bug. Measured four times now: #556's retired qwen3.5 ran 24/24 two days after retirement (iter-131); a stale skill (iter-128); a stale charter (iter-129); and 2026-08-12, 564cc46's lane-degradation fix inert on V1 at 12 commits behind while ailang-motoko and ailang-world both had it. Two one-time human reconciles, zero durable fixes. THE FIX. tools/launchd/lib/pin-root.sh re-execs the driver out of a worktree pinned to committed origin/dev, which moves script, skill and charter in one step. Three details carry the correctness: * It re-points MISSION_WORKDIR too. mission-motoko.env:8 and mission-world.env:5 pin it, and mission-control.sh:40 reads it AHEAD of $0-relative resolution — so pinning only the script leaves those two on a fresh driver against a stale charter and skill. That half-fix reports green, which is worse than no fix. Sprint worktrees are unaffected: the skill creates them by absolute path (SKILL.md:1662). * A failed pin is LOUD, not fatal. Aborting would make network availability a hard dependency of every fire — trading rare silent staleness for common loud outage. So the fire proceeds and posts "driver ran UNPINNED" on both human channels. Continuing SILENTLY would have rebuilt the exact defect 564cc46 removed one layer up. * Position is load-bearing: after the state block (so a failure has LOG/MSG_FROM/ MISSION_GH_ISSUE to report with), before the probes (a re-exec restarts the script, so pinning later bills every lane twice), before the pidfile write (the re-exec'd copy would read its parent's pid and yield to itself, no-opping every fire). Both notices now share one _mc_notify, extracted from 564cc46's block: two near-identical notifiers is how one of them silently rots. TESTED — first automated coverage tools/launchd has ever had (37 assertions, `make test-launchd-drivers`, /bin/bash 3.2.57 explicitly, since the rig has no newer one): - real origin + deliberately-stale clone: re-exec observed, ROOT moved, FRESH content read, args survive; plus a control proving the clone really was stale (AILANG_DRIVER_PIN=0 reads STALE-CONTENT from the clone path). - no recursion on the second pass; drift carried across the exec. - fetch failure -> STALE + reason + fire still runs, never reports pinned. - driver absent FROM THE REF -> our guard fires, not bash's "No such file" (the first version of that test passed for the wrong reason and was rewritten). - notices awk-extracted from the real file, never retyped: fires when degraded, SILENT when healthy, rc=1 on either channel -> WARNING and still exits 0, unset issue -> WARNING not a silent skip. Lane block regression-tested through the extraction. - `set -u` bug caught by inspection and fixed: PIN_DRIFT was initialised only inside the helper, so the helper-absent branch — the pre-helper clone this branch exists to survive — would have aborted at the DRY RUN line. The fallback crashing only on the fallback path. CHICKEN AND EGG, stated rather than hidden: this cannot take effect until the shared clone receives it once, and that first reconcile is human — Principle 0 forbids unattended branch ops on a shared dirty tree. Wired into mission-control.sh only; the other five entry points follow once this has survived live fires. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous commit added `make test-launchd-drivers` and no job invoked it. A suite that
only runs when someone types the command is decoration — and it is the same shape as the
bug this branch fixes: a protection that is silently not in effect.
macOS runner deliberately. The rig is bash 3.2.57 and GitHub's macOS images still ship 3.2
as /bin/bash; ubuntu ships bash 5, where every `declare -A` and `${v,,}` that would break
the rig passes clean. Testing rig shell portability on bash 5 measures the wrong instrument.
A guard step asserts BASH_VERSINFO[0] == 3 and fails the job if the runner is ever upgraded,
so the coverage cannot silently degrade into a bash-5 run that still reports green.
Also made test_pin_root.sh hermetic: it builds a real git lab, and a CI runner has no
user.name/user.email and may default init.defaultBranch anywhere, so the lab commits would
have failed there and the suite would have reported a git-config problem as a pin failure —
green on the author's box, red everywhere else. Now pins identity and neutralises global and
system git config. Verified by running it with HOME=/nonexistent and the identity env
unset: 20/20.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ther than on a date Adds the five remaining launchd entry points to the V1 queue as a single rollout item, with the gate written as evidence (>=3 consecutive fires logging a successful pin, read from the driver log rather than from the file's presence on disk) instead of a delay. Three things recorded deliberately, because each has already cost an iteration: * INERT UNTIL RECONCILED. #666 changes nothing on the rig until the shared clone receives it once, and that is a human branch op — the standing fast-forward authorisation does not cover it (local dev is 1 ahead, not 0). This is exactly what deferred m-planner-codex-lane at iter-131: a sprint that lands green, passes CI, and changes nothing while the report claims the capability was gained. * ORDER. Merge #666 first, reconcile second. The reverse brings the lane fix without the pin, and the clone starts drifting again immediately. * THE GATE'S LIMIT. Three green fires exercise the re-exec, the root move and the fetch, but not the failure path, which production will not produce on demand. That arm is covered by the test suite alone, so a passing gate is evidence about the happy path and must not be reported as confidence in the whole fix. ailang-world is explicitly out of scope: different repo, hand-synced fork (513 lines vs 671, differently-shaped fallback site), portable only by hand. Handed over as msg_20260812_085746; World's loop decides when. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
sunholo-voight-kampff
added a commit
that referenced
this pull request
Aug 12, 2026
… gate on onboarding (#558) (#667) TWO DEFECTS IN #666, one of which would have taken the V1 loop down on the first fire after the reconcile. Found by reading 76ee405, which landed on dev while #666 was in review. DEFECT 1 — the pin walks into a dead checkout. A pin worktree is BY CONSTRUCTION a path Claude Code has never seen, and an un-onboarded checkout makes headless `claude -p` block on a trust dialog it cannot display. Every model probe then hangs to its timeout (120s x2 x3 models ~= 12 min) and the driver refuses with "NO usable model in prefs" — which reads as a quota outage, not as the harness fault it is. That had just cost the motoko mission its whole first unattended fire (charter V22). Measured here before writing any code: onboarded=true /Users/voightkampff/dev/sunholo-data/ailang onboarded=ABSENT /Users/voightkampff/.ailang-driver-pin/v1 <- the pin target So #666, the moment the shared clone received it, would have converted a working-but-stale loop into a dead one. Stale-but-working strictly beats fresh-and-dead, so the pin now reads projects[<wt>].hasCompletedProjectOnboarding from ~/.claude.json and REFUSES when it is absent — reported on both human channels like every other pin failure, with the exact one-line human fix in the message. Undeterminable (no jq) is treated as un-onboarded on purpose: pinning blind risks 12 minutes of hung probes, refusing costs staleness that is already reported. jq is /usr/bin/jq, inside launchd's default PATH. It gates on hasCompletedProjectOnboarding, NOT the hasTrustDialogAccepted the error text names: 76ee405 measured ailang-world at trust=false/onboarded=true and WORKING, which is the control proving the flag the message points at is not the gate. Test 8 pins both arms so a future edit cannot quietly switch to the wrong flag and stay green. DEFECT 2 — a fix that disables itself after one use. Worktree existence was decided by string-matching `git worktree list`, but git records the RESOLVED realpath, so on any path reached through a symlink (/var -> /private/var, the macOS everyday case) the match silently fails, fire 2 takes the `add` branch against an existing directory, and the pin refuses from then on. Loud, but permanently useless. Now decided by asking the worktree itself. Caught by the new repeat-fire test, not by review — the first suite only ever fired once. CONSEQUENCE FOR DEPLOYMENT: reconciling the shared clone is now safe with no further action — the pin declines and says why. Pinning switches ON when a human runs, once per checkout: cd ~/.ailang-driver-pin/<mission> && claude TESTED: 48 assertions (31 + 17), `make test-launchd-drivers`, bash 3.2.57, CI job on macOS. New: un-onboarded => refuse + fire still runs + never reports pinned; trust-flag-alone does NOT satisfy while onboarding-flag does; no-jq fails safe; second consecutive fire pins. The happy path runs against a synthetic ~/.claude.json in a fake HOME rather than with the check disabled, so a gate regressed to always-refuse fails tests 1-6 and one regressed to always-allow fails 7. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the class behind #558, which has now been measured four times and repaired twice by hand.
The defect: one root, three symptoms
launchd invokes each driver by absolute path into the shared clone, and everything a fire reads hangs off that single root (
mission-control.sh:40):tools/launchd/*.sh$0564cc4640lane fix inert on V1 on 2026-08-12, 12 commits behind, while both sibling missions had it.claude/skills/…design_docs/*mission*.mdNothing keeps that clone current and nothing reports the drift — a fallback whose only witness is a log nobody reads. Every previous repair patched one artefact, which fixes a third of a bug.
The fix
tools/launchd/lib/pin-root.shre-execs the driver out of a worktree pinned to committedorigin/dev, moving script, skill and charter together.MISSION_WORKDIRtoo.mission-motoko.env:8andmission-world.env:5pin it, andmission-control.sh:40reads it ahead of$0-relative resolution — so pinning only the script would leave those two missions on a fresh driver against a stale charter. A half-fix that reports green is worse than none. Sprint worktrees are unaffected (SKILL.md:1662uses absolute paths).564cc4640removed one layer up.LOG/MSG_FROM/MISSION_GH_ISSUEto report with), before the probes (a re-exec restarts the script — pinning later bills every lane twice), before the pidfile write (the re-exec’d copy would read its parent’s pid and yield to itself, no-opping every fire).Both notices now share one
_mc_notify. Two near-identical notifiers is how one of them silently rots.Testing
First automated coverage
tools/launchdhas ever had — 37 assertions,make test-launchd-drivers,/bin/bash3.2.57 explicitly (the rig has no newer one).STALE+ reason + fire still runs, never reports pinned.No such file. The first version of that test passed for the wrong reason and was rewritten.set -ubug caught by inspection:PIN_DRIFTwas initialised only inside the helper, so the helper-absent branch — the pre-helper clone it exists to survive — would have aborted at the DRY RUN line. The fallback crashing only on the fallback path.Two things stated rather than hidden
mission-control.shonly. The other five entry points follow once this has survived live fires.ailang-worldis a separate repo with a forked 513-line driver — it needs a port, not a copy.🤖 Generated with Claude Code