Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@

All notable changes are documented here.

## Unreleased

- Detect and quarantine abort events that are semantically closed but incompatible with Codex desktop projection.
- Repair interrupted turns with Codex-native completion metadata while preserving an explicit non-fabricated tool-output placeholder.
- Publish the current synchronization result in the same device report instead of reporting the previous run.

## 0.1.0 - 2026-07-12

- Initial private repository baseline.
Expand Down
4 changes: 2 additions & 2 deletions SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ Manage project folders through the background CLI/API; do not open the Syncthing
## Safety behavior

- Treat conversation JSONL as append-only. Publish a per-device head and advance the canonical copy only when all heads have a prefix relationship.
- Require semantic completeness in addition to valid JSON: every tool call needs a matching output, and every older turn needs `task_complete` or `turn_aborted`.
- Require semantic and desktop-projection completeness in addition to valid JSON: every tool call needs a matching output, every older turn needs `task_complete` or `turn_aborted`, and repaired aborts need Codex-compatible completion fields.
- Never export the active turn or the tool call currently running Sync2. When possible, publish the semantically complete prefix before the active turn as a stable checkpoint; otherwise skip the task until a healthy checkpoint exists.
- Quarantine incomplete local snapshots, heads, and canonical files from import or promotion while continuing independent skill/project work.
- On divergent conversation history, preserve every head, emit a conflict, and do not invent a merged history.
Expand All @@ -82,6 +82,6 @@ Manage project folders through the background CLI/API; do not open the Syncthing

Read [usage.md](references/usage.md) for daily commands and output interpretation. For first-machine, new-machine, reinstall, or path-change deployment, follow [deployment.md](references/deployment.md). For edge cases, conflict semantics, and storage boundaries, consult [protocol.md](references/protocol.md).

After setup or material changes, run `conversation audit <task>`, `doctor`, `sync --dry-run`, `sync`, `device report`, and `status`. A healthy active task may report one open latest turn and active calls; it must report zero persistent dangling calls and zero stale open turns. Outside maintenance, `doctor` must also require a stable canonical and stable head from every device.
After setup or material changes, run `conversation audit <task>`, `doctor`, `sync --dry-run`, `sync`, `device report`, and `status`. A healthy active task may report one open latest turn and active calls; it must report zero persistent dangling calls, zero stale open turns, and zero projection-unsafe abort events. Outside maintenance, `doctor` must also require a stable canonical and stable head from every device.

If an imported task does not appear immediately in the Codex sidebar, verify its JSONL and `state_5.sqlite` row with `doctor`, then restart Codex once. Do not copy the source device's whole database.
10 changes: 5 additions & 5 deletions references/protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,15 +46,15 @@ Never synchronize local config or base state between devices. They contain devic

Treat Codex rollout JSONL as append-only. Read only complete JSON records; discard an incomplete final line from a live writer snapshot.

Before publishing, validate both syntax and event semantics. Pair every `custom_tool_call`/`function_call` with its output and close every older turn with `task_complete` or `turn_aborted`. Treat calls belonging to the latest open turn as transient. Never publish that active turn; if the prefix before its `task_started` event is independently stable, publish that completed prefix as a checkpoint. This prevents long-running Goals from starving sync when a new turn starts immediately after each completion.
Before publishing, validate syntax, event semantics, and desktop projection compatibility. Pair every `custom_tool_call`/`function_call` with its output and close every older turn with `task_complete` or `turn_aborted`. A repaired abort must match Codex's native projection shape: an associated `<turn_aborted>` developer marker followed by `reason: "interrupted"`, numeric `completed_at`, and numeric `duration_ms`. Treat calls belonging to the latest open turn as transient. Never publish that active turn; if the prefix before its `task_started` event is independently stable, publish that completed prefix as a checkpoint. This prevents long-running Goals from starving sync when a new turn starts immediately after each completion.

Each device publishes its stable local snapshot to `heads/<device>.jsonl`. Reconciliation compares raw bytes:

- when every shorter head is a byte prefix of the longest, promote the longest to `canonical.jsonl`;
- when any pair diverges, retain the existing canonical, preserve all heads, and write `conflict.json`;
- never interleave turns or invent a merged event order.

Ignore semantically incomplete heads and canonical files. Continue skill and project synchronization instead of failing the entire run. Once a valid longer head arrives, it can replace an invalid old canonical without treating the invalid copies as merge candidates.
Ignore semantically incomplete or projection-unsafe heads and canonical files. Continue skill and project synchronization instead of failing the entire run. Once a valid longer head arrives, it can replace an invalid old canonical without treating the invalid copies as merge candidates.

Import only a semantically complete canonical into the platform-appropriate `~/.codex/sessions/YYYY/MM/DD` path. Update that thread in `state_5.sqlite`, `session_index.jsonl`, and an existing local desktop catalog row. Do not copy database files between devices.

Expand Down Expand Up @@ -88,7 +88,7 @@ Case-only file or directory names are unsafe across Windows and default macOS fi

## 5. Device reports and cold start

`device report` publishes content-stable evidence about platform, Node version, protocol revision, Sync2 script SHA-256, vault, selected-task local/vault health, skill counts, scheduler state, and last-run result. `device list` reads all propagated reports. Do not declare a fleet upgraded until every expected device reports the same current protocol revision and script hash.
`device report` publishes content-stable evidence about platform, Node version, protocol revision, Sync2 script SHA-256, vault, selected-task local/vault health, skill counts, scheduler state, and last-run result. A normal sync embeds that same run's result in the report before transport commit; if the transport commit fails, it rewrites the local/vault report as failed instead of leaving a false success. `device list` reads all propagated reports. Do not declare a fleet upgraded until every expected device reports the same current protocol revision and script hash.

On a new device, the first sync usually performs this order:

Expand Down Expand Up @@ -166,8 +166,8 @@ Resolution archives nonchosen vault heads and backs up a differing local history
1. Disable the scheduler and pause the transport folder.
2. Back up the rollout, state database, session index, desktop catalog database, and vault task directory.
3. Run `conversation audit <id>`.
4. Run `conversation repair <id> --title <known-title>`. The repair appends an explicit interrupted-output placeholder and aborts only stale older turns; it does not fabricate the original output or close the currently executing turn.
5. Require zero persistent dangling calls, zero stale open turns, and consistent titles across indexes.
4. Run `conversation repair <id> --title <known-title>`. The repair appends an explicit interrupted-output placeholder and aborts only stale older turns using Codex's native projection-compatible event shape; it does not fabricate the original output or close the currently executing turn.
5. Require zero persistent dangling calls, zero stale open turns, zero projection-unsafe abort events, and consistent titles across indexes.
6. Leave invalid old heads quarantined until a stable local final turn publishes a valid replacement canonical.

### Skill file
Expand Down
88 changes: 73 additions & 15 deletions scripts/sync2.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,7 @@ function auditJsonlBuffer(buffer) {
const startedTurns = [];
const startedTurnEvents = [];
const closedTurns = new Set();
const latestTurnAbortEvents = new Map();
let finalAnswers = 0;
let userMessages = 0;
let lastTimestamp = null;
Expand All @@ -187,9 +188,23 @@ function auditJsonlBuffer(buffer) {
const turnId = payload.turn_id ?? payload.internal_chat_message_metadata_passthrough?.turn_id ?? null;
if (event.type === "event_msg" && payload.type === "task_started" && payload.turn_id) {
startedTurns.push(payload.turn_id);
startedTurnEvents.push({ turnId: payload.turn_id, line: index + 1 });
startedTurnEvents.push({ turnId: payload.turn_id, line: index + 1, startedAt: payload.started_at ?? null });
}
if (event.type === "event_msg" && ["task_complete", "turn_aborted"].includes(payload.type) && payload.turn_id) {
closedTurns.add(payload.turn_id);
if (payload.type === "turn_aborted") {
latestTurnAbortEvents.set(payload.turn_id, {
turnId: payload.turn_id,
line: index + 1,
reason: payload.reason ?? null,
completedAt: payload.completed_at ?? null,
durationMs: payload.duration_ms ?? null,
projectionCompatible: payload.reason === "interrupted"
&& Number.isFinite(payload.completed_at)
&& Number.isFinite(payload.duration_ms),
});
}
}
if (event.type === "event_msg" && ["task_complete", "turn_aborted"].includes(payload.type) && payload.turn_id) closedTurns.add(payload.turn_id);
if ((payload.type === "custom_tool_call" || payload.type === "function_call") && payload.call_id) {
calls.set(payload.call_id, { callId: payload.call_id, kind: payload.type, line: index + 1, turnId });
}
Expand All @@ -204,6 +219,7 @@ function auditJsonlBuffer(buffer) {
const activeDanglingCalls = danglingCalls.filter((item) => latestTurnId && item.turnId === latestTurnId && openTurns.includes(latestTurnId));
const tailDanglingCalls = danglingCalls.filter((item) => item.line === lines.length);
const persistentDanglingCalls = danglingCalls.filter((item) => !activeDanglingCalls.some((active) => active.callId === item.callId));
const projectionUnsafeAbortEvents = [...latestTurnAbortEvents.values()].filter((item) => !item.projectionCompatible);
return {
lines: lines.length,
bytes: buffer.length,
Expand All @@ -216,11 +232,12 @@ function auditJsonlBuffer(buffer) {
activeDanglingCalls,
tailDanglingCalls,
persistentDanglingCalls,
projectionUnsafeAbortEvents,
openTurns,
staleOpenTurns,
latestTurnId,
semanticOk: persistentDanglingCalls.length === 0 && staleOpenTurns.length === 0,
stable: danglingCalls.length === 0 && openTurns.length === 0,
semanticOk: persistentDanglingCalls.length === 0 && staleOpenTurns.length === 0 && projectionUnsafeAbortEvents.length === 0,
stable: danglingCalls.length === 0 && openTurns.length === 0 && projectionUnsafeAbortEvents.length === 0,
};
}

Expand Down Expand Up @@ -1202,7 +1219,7 @@ function reconcileProjectShares(config, dryRun, summary) {
}
}

function buildDeviceReport(config) {
function buildDeviceReport(config, lastRunOverride = undefined) {
const selectedConversations = Object.entries(config.conversations ?? {}).filter(([, event]) => event.selected).map(([id, event]) => {
const rollout = findRollout(config.codexHome, id);
let audit = null;
Expand Down Expand Up @@ -1240,12 +1257,12 @@ function buildDeviceReport(config) {
skillSources,
daemon: daemonStatus(config, config._configFile),
maintenance: { enabled: fs.existsSync(maintenanceFile(config)) },
lastRun: readJson(lastRunFile(config), null),
lastRun: lastRunOverride === undefined ? readJson(lastRunFile(config), null) : lastRunOverride,
};
}

function publishDeviceReport(config, dryRun = false) {
const report = buildDeviceReport(config);
function publishDeviceReport(config, dryRun = false, lastRunOverride = undefined) {
const report = buildDeviceReport(config, lastRunOverride);
writeJson(path.join(config.vault, ".sync2", "device-reports", `${config.deviceId}.json`), report, dryRun);
return report;
}
Expand Down Expand Up @@ -1290,6 +1307,7 @@ function syncCommand(options, direction = "sync") {
const dryRun = Boolean(options["dry-run"]);
const summary = newSummary(direction, dryRun);
const startedAt = nowIso();
let reportPublished = false;
try {
ensureVault(config, dryRun);
if (!dryRun && fs.existsSync(maintenanceFile(config)) && !options.force) {
Expand All @@ -1300,14 +1318,21 @@ function syncCommand(options, direction = "sync") {
syncConversations(config, direction, dryRun, summary);
syncSkills(config, direction, dryRun, summary);
reconcileProjectShares(config, dryRun, summary);
publishDeviceReport(config, dryRun);
const successfulRun = { ok: true, startedAt, finishedAt: nowIso(), summary };
recordLastRun(config, successfulRun, dryRun);
publishDeviceReport(config, dryRun, successfulRun);
reportPublished = !dryRun;
saveConfig(file, config, dryRun);
});
gitPost(config, dryRun);
recordLastRun(config, { ok: true, startedAt, finishedAt: nowIso(), summary }, dryRun);
return summary;
} catch (error) {
recordLastRun(config, { ok: false, startedAt, finishedAt: nowIso(), error: error.message, direction }, dryRun);
const failedRun = { ok: false, startedAt, finishedAt: nowIso(), error: error.message, direction };
recordLastRun(config, failedRun, dryRun);
if (reportPublished) {
try { withVaultLock(config, false, () => publishDeviceReport(config, false, failedRun)); }
catch { /* Preserve the original sync failure. The next successful run republishes health. */ }
}
throw error;
}
}
Expand Down Expand Up @@ -1410,8 +1435,39 @@ function repairConversationCommand(options, selector) {
});
}
}
for (const turnId of before.rollout.staleOpenTurns) {
additions.push({ timestamp: nowIso(), type: "event_msg", payload: { type: "turn_aborted", turn_id: turnId, reason: "recovered_stale_turn" } });
const turnsNeedingProjectionRepair = new Set([
...before.rollout.staleOpenTurns,
...before.rollout.projectionUnsafeAbortEvents.map((item) => item.turnId),
]);
for (const turnId of turnsNeedingProjectionRepair) {
const completedAt = Math.floor(Date.now() / 1000);
const startedAt = before.rollout.startedTurnEvents.find((item) => item.turnId === turnId)?.startedAt;
const durationMs = Number.isFinite(startedAt) ? Math.max(0, (completedAt - startedAt) * 1000) : 0;
additions.push({
timestamp: nowIso(),
type: "response_item",
payload: {
type: "message",
role: "developer",
content: [{
type: "input_text",
text: "<turn_aborted>\nThe previous turn was interrupted before its completion record was persisted. Sync2 recovered the stale turn without reconstructing missing execution output.\n</turn_aborted>",
}],
internal_chat_message_metadata_passthrough: { turn_id: turnId },
},
});
additions.push({
timestamp: nowIso(),
type: "event_msg",
payload: {
type: "turn_aborted",
turn_id: turnId,
reason: "interrupted",
recovery_reason: "recovered_stale_turn",
completed_at: completedAt,
duration_ms: durationMs,
},
});
}
if (additions.length) fs.appendFileSync(rollout, `${additions.map((item) => JSON.stringify(item)).join("\n")}\n`);
const state = threadRow(config.codexHome, before.id) ?? {};
Expand Down Expand Up @@ -1444,9 +1500,11 @@ function repairConversationCommand(options, selector) {
id: before.id,
recovery,
repairedCalls: before.rollout.persistentDanglingCalls.map((item) => item.callId),
closedStaleTurns: before.rollout.staleOpenTurns,
closedStaleTurns: [...turnsNeedingProjectionRepair],
title,
semanticOkAfter: after.rollout.persistentDanglingCalls.filter((item) => !before.rollout.tailDanglingCalls.some((tail) => tail.callId === item.callId)).length === 0 && after.rollout.staleOpenTurns.length === 0,
semanticOkAfter: after.rollout.persistentDanglingCalls.filter((item) => !before.rollout.tailDanglingCalls.some((tail) => tail.callId === item.callId)).length === 0
&& after.rollout.staleOpenTurns.length === 0
&& after.rollout.projectionUnsafeAbortEvents.length === 0,
indexesConsistent: after.indexes.consistent,
};
}
Expand Down
Loading
Loading