Skip to content

Fix stale lane worktree cleanup#642

Merged
arul28 merged 2 commits into
mainfrom
ade/take-look-at-users-admin-f4fc0bc8
Jun 23, 2026
Merged

Fix stale lane worktree cleanup#642
arul28 merged 2 commits into
mainfrom
ade/take-look-at-users-admin-f4fc0bc8

Conversation

@arul28

@arul28 arul28 commented Jun 23, 2026

Copy link
Copy Markdown
Owner

Summary

  • record retryable local cleanup debt when lane delete leaves an unregistered residual worktree directory behind
  • retry residual cleanup safely during lane list refreshes, while skipping Git worktrees, active/archived lane paths, pending creates, symlinks, and unknown non-empty directories
  • document the local-only cleanup table and residual cleanup sweep

Test plan

  • /quality pass completed; Track B extraction applied
  • /test pass completed
  • npx vitest run src/main/services/lanes/laneService.test.ts — 96 passed
  • npx vitest run --shard=1/8 — 49 files, 724 passed
  • npm --prefix apps/desktop run typecheck — passed
  • node scripts/validate-docs.mjs — passed
  • CLI parity: npm --prefix apps/ade-cli run typecheck and npm --prefix apps/ade-cli test — passed
  • TUI parity: npm --prefix apps/ade-cli run typecheck and cd apps/ade-cli && npx vitest run src/tuiClient — passed

Summary by CodeRabbit

  • Bug Fixes
    • Enhanced management of residual Git worktrees to recover from incomplete deletions across application runs.
    • Added retry mechanism for failed worktree cleanup with persistent failure tracking.
    • Improved lane deletion workflow to handle orphaned worktree directories more robustly.

Greptile Summary

This PR introduces a persistent retry mechanism for managed worktree directories that survive lane deletion. When git worktree remove + manual filesystem cleanup both fail, the delete now records the path in a new machine-local local_worktree_residual_cleanups table and defers removal to subsequent laneService.list() calls via a TTL-gated sweep.

  • worktreeResidualCleanup.ts implements the sweep with layered guards (registered git worktrees, active/archived lane paths, in-flight creates, symlinks, unknown non-empty dirs) and exposes recordFailure, deleteRow, and retry to the service.
  • kvDb.ts adds the new table with a (project_id, worktree_path) unique index and correctly excludes it from CRR replication, since absolute paths are machine-local state.
  • laneService.ts wires retry() into the list() repair sequence and calls recordFailure/deleteRow at the appropriate success and non-fatal-failure branches of the delete pipeline.

Confidence Score: 5/5

Safe to merge; the new retry sweep is fully guarded and the failing-cleanup code path falls back gracefully.

The guard sets (git worktrees, lane paths including archived, pending creations) are queried fresh on each sweep and applied at multiple checkpoints. The integration in laneService wraps the sweep in a try-catch so failures cannot break list(). Tests cover the retry-on-list flow, the archived-lane exemption, and the unknown-empty-dir boundary. The one cosmetic gap (attempts not reset to 0 on upsert conflict) has no behavioral impact since there is no retry limit.

No files require special attention.

Important Files Changed

Filename Overview
apps/desktop/src/main/services/lanes/worktreeResidualCleanup.ts New module implementing the residual worktree cleanup sweep; guards (git worktrees, lane paths, pending creates, symlinks, non-empty unknowns) are well-layered and tests cover each exemption path
apps/desktop/src/main/services/lanes/laneService.ts Integrates residualWorktreeCleanup into the delete path (recordFailure on non-fatal cleanup failure, deleteRow on success) and wires retry() into list(); ordering and error-boundary wrapping look correct
apps/desktop/src/main/services/state/kvDb.ts Adds local_worktree_residual_cleanups table with correct NOT NULL constraints, unique index on (project_id, worktree_path), and correctly excludes it from CRR replication
apps/desktop/src/main/services/lanes/laneService.test.ts New tests cover retry-on-next-list, archived-lane exemption, and unknown-empty-dir cleanup; test helpers updated to use worktreesDir-relative paths, aligning with actual lane worktree_path values
docs/features/lanes/worktree-isolation.md Documents the residual cleanup retry sweep and its guard sets accurately; aligns with the implementation

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["lane.delete()"] --> B["git worktree remove"]
    B -->|success| C{"directory\nstill exists?"}
    C -->|no| D["residualCleanup.deleteRow()"]
    C -->|yes| E["removeResidualDirectory()"]
    E -->|success| D
    E -->|failure, not registered| F["residualCleanup.recordFailure()\n→ inserts/upserts DB row\n→ lastSweepAt = 0"]
    F --> G["lane row deleted from DB\ndelete completes with warning"]
    B -->|failure / stale state| E

    H["laneService.list()"] --> I{"TTL elapsed or\nlastSweepAt = 0?"}
    I -->|no| J["return / join activeSweep"]
    I -->|yes| K["residualCleanup.retry()"]
    K --> L["Build guard sets:\ngitWorktreePaths\nlaneWorktreePaths\npendingCreations"]
    L --> M["Load DB rows → rowsByPath"]
    M --> N["Scan worktreesDir for\nunknown empty old dirs"]
    N --> O["For each candidate path"]
    O --> P{"guard\nchecks pass?"}
    P -->|no| Q["skip or deleteRow if\npath no longer exists"]
    P -->|yes| R["removeWorktreeDirectory()"]
    R -->|success| S["deleteRow() + log info"]
    R -->|failure| T["noteRetryFailure()\n→ attempts++, updated_at\nlog warn"]
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A["lane.delete()"] --> B["git worktree remove"]
    B -->|success| C{"directory\nstill exists?"}
    C -->|no| D["residualCleanup.deleteRow()"]
    C -->|yes| E["removeResidualDirectory()"]
    E -->|success| D
    E -->|failure, not registered| F["residualCleanup.recordFailure()\n→ inserts/upserts DB row\n→ lastSweepAt = 0"]
    F --> G["lane row deleted from DB\ndelete completes with warning"]
    B -->|failure / stale state| E

    H["laneService.list()"] --> I{"TTL elapsed or\nlastSweepAt = 0?"}
    I -->|no| J["return / join activeSweep"]
    I -->|yes| K["residualCleanup.retry()"]
    K --> L["Build guard sets:\ngitWorktreePaths\nlaneWorktreePaths\npendingCreations"]
    L --> M["Load DB rows → rowsByPath"]
    M --> N["Scan worktreesDir for\nunknown empty old dirs"]
    N --> O["For each candidate path"]
    O --> P{"guard\nchecks pass?"}
    P -->|no| Q["skip or deleteRow if\npath no longer exists"]
    P -->|yes| R["removeWorktreeDirectory()"]
    R -->|success| S["deleteRow() + log info"]
    R -->|failure| T["noteRetryFailure()\n→ attempts++, updated_at\nlog warn"]
Loading

Comments Outside Diff (1)

  1. apps/desktop/src/main/services/lanes/worktreeResidualCleanup.ts, line 392-397 (link)

    P2 TTL is measured from sweep start, not completion

    lastSweepAt is stamped inside the IIFE so it captures the start time, not the finish time. If a sweep takes, say, 45 s (removing a large worktree via removeWorktreeDirectory), the next call that arrives 16 s after the sweep started would see 61 s − 0 = 61 s > 60 s and launch a second sweep immediately — with activeSweep already null because the first one just finished. Effectively the rest interval can be as short as TTL − sweep_duration. This is unlikely to cause correctness problems in practice (the guard sets are re-queried fresh each sweep), but the comment might mislead future readers into thinking there's a 60 s cool-down between sweep completions.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: apps/desktop/src/main/services/lanes/worktreeResidualCleanup.ts
    Line: 392-397
    
    Comment:
    **TTL is measured from sweep start, not completion**
    
    `lastSweepAt` is stamped inside the IIFE so it captures the start time, not the finish time. If a sweep takes, say, 45 s (removing a large worktree via `removeWorktreeDirectory`), the next call that arrives 16 s after the sweep started would see `61 s − 0 = 61 s > 60 s` and launch a second sweep immediately — with `activeSweep` already null because the first one just finished. Effectively the rest interval can be as short as `TTL − sweep_duration`. This is unlikely to cause correctness problems in practice (the guard sets are re-queried fresh each sweep), but the comment might mislead future readers into thinking there's a 60 s cool-down between sweep completions.
    
    How can I resolve this? If you propose a fix, please make it concise.

    Fix in Claude Code

Reviews (2): Last reviewed commit: "ship: iteration 1 - address review fixtu..." | Re-trigger Greptile

@vercel

vercel Bot commented Jun 23, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
ade Ignored Ignored Preview Jun 23, 2026 5:11pm

@arul28 arul28 changed the title Take Look At Users Admin Fix stale lane worktree cleanup Jun 23, 2026
@arul28

arul28 commented Jun 23, 2026

Copy link
Copy Markdown
Owner Author

@copilot review but do not make fixes

@coderabbitai

coderabbitai Bot commented Jun 23, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Adds a local_worktree_residual_cleanups DB table (excluded from CRR replication) and a new createWorktreeResidualCleanup service that scans for leftover Git worktree directories, records failures with retry metadata, and sweeps them on listLanes. The service is wired into createLaneService for both the delete path and the list bootstrap sequence. Tests are updated to verify the full record-then-retry lifecycle.

Changes

Residual Worktree Cleanup

Layer / File(s) Summary
DB schema: local_worktree_residual_cleanups
apps/desktop/src/main/services/state/kvDb.ts
Creates the local_worktree_residual_cleanups table with fields for project/lane/worktree path, reason, retry attempts, last error, and timestamps; adds unique index on (project_id, worktree_path) and excludes the table from CRR replication.
createWorktreeResidualCleanup service
apps/desktop/src/main/services/lanes/worktreeResidualCleanup.ts
Implements types, configuration constants, path normalization, DB/error helpers, filesystem inspection helpers, and the throttled/deduplicated retry sweep; exports { deleteRow, recordFailure, retry }.
LaneService wiring
apps/desktop/src/main/services/lanes/laneService.ts
Initializes residualWorktreeCleanup with DB/project/worktree context and pendingWorktreeCreationPaths guard; calls retry() in listLanes after legacy repair; replaces ad-hoc handling in git_worktree_remove with recordFailure on failure and deleteRow on success.
Test updates
apps/desktop/src/main/services/lanes/laneService.test.ts
Changes worktreesDir fixture to repoRoot, renames the residual delete test, adds service.list() to exercise the retry path, and asserts the full local_worktree_residual_cleanups row lifecycle.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

  • arul28/ADE#384: Restructures delete()/git_worktree_remove execution for concurrency in the same laneService.ts delete path where this PR adds residual cleanup recording.
  • arul28/ADE#508: Directly modifies lane deletion behavior and tests around residual/unregistered worktrees including manual cleanup failure handling, the same scenario extended here with DB-backed retry.
  • arul28/ADE#562: Modifies the pendingWorktreeCreationPaths in-flight tracker in laneService.ts, which this PR's residual cleanup service consults via isPendingWorktreeCreation to avoid removing directories still being created.

Suggested labels

desktop

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title 'Fix stale lane worktree cleanup' is a concise and clear summary of the main change, which adds residual worktree cleanup logic to handle stale/leftover git worktrees.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ade/take-look-at-users-admin-f4fc0bc8

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
apps/desktop/src/main/services/lanes/laneService.test.ts (1)

3169-3169: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Keep the test worktrees directory separate from repoRoot.

Pointing worktreesDir at the project root makes the sweeper treat arbitrary root children as managed worktrees, which weakens coverage for the direct-child cleanup guard this PR relies on. Prefer creating childPath under a dedicated fixture worktreesDir and seeding the lane with that path. As per coding guidelines, “When working in ADE lanes, all file reads, edits, and writes MUST target paths under the lane worktree, never under the main project-root checkout.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/desktop/src/main/services/lanes/laneService.test.ts` at line 3169, The
test is incorrectly setting worktreesDir to repoRoot, which causes the sweeper
to treat arbitrary root children as managed worktrees and weakens test coverage.
Instead of pointing worktreesDir at repoRoot, create a dedicated fixture
directory separate from repoRoot for worktreesDir, then create childPath as a
subdirectory under this dedicated worktreesDir fixture, and pass this childPath
to the lane configuration. This ensures the test properly validates the
direct-child cleanup guard behavior as intended by the PR.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@apps/desktop/src/main/services/lanes/laneService.test.ts`:
- Line 3169: The test is incorrectly setting worktreesDir to repoRoot, which
causes the sweeper to treat arbitrary root children as managed worktrees and
weakens test coverage. Instead of pointing worktreesDir at repoRoot, create a
dedicated fixture directory separate from repoRoot for worktreesDir, then create
childPath as a subdirectory under this dedicated worktreesDir fixture, and pass
this childPath to the lane configuration. This ensures the test properly
validates the direct-child cleanup guard behavior as intended by the PR.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 074c2749-dd6c-4179-a13c-17181cdbad97

📥 Commits

Reviewing files that changed from the base of the PR and between 84502b7 and 3e16c07.

⛔ Files ignored due to path filters (4)
  • docs/ARCHITECTURE.md is excluded by !docs/**
  • docs/features/lanes/README.md is excluded by !docs/**
  • docs/features/lanes/worktree-isolation.md is excluded by !docs/**
  • docs/features/sync-and-multi-device/crdt-model.md is excluded by !docs/**
📒 Files selected for processing (4)
  • apps/desktop/src/main/services/lanes/laneService.test.ts
  • apps/desktop/src/main/services/lanes/laneService.ts
  • apps/desktop/src/main/services/lanes/worktreeResidualCleanup.ts
  • apps/desktop/src/main/services/state/kvDb.ts

Comment on lines +284 to +294
} catch (error) {
if (row) {
noteRetryFailure(row, error);
}
logger.warn("lane.residual_worktree_cleanup.remove_failed", {
projectRoot,
laneId: row?.lane_id ?? null,
worktreePath: pathKey,
error: error instanceof Error ? error.message : String(error),
});
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Unknown-empty-dir cleanup failures have no backoff or retry record. When removeWorktreeDirectory throws for a dir-scan candidate (no row), the only action is a warn log — the directory will be re-added to candidatePaths on every subsequent sweep as long as it stays empty and old enough, generating a fresh warning each time. On a filesystem where removal consistently fails (e.g., read-only mount, Windows handle lock), this can flood the logs on every list() call. A simple attempt counter in the catch block, or just skipping untracked dirs on a repeating error, would keep noise bounded.

Suggested change
} catch (error) {
if (row) {
noteRetryFailure(row, error);
}
logger.warn("lane.residual_worktree_cleanup.remove_failed", {
projectRoot,
laneId: row?.lane_id ?? null,
worktreePath: pathKey,
error: error instanceof Error ? error.message : String(error),
});
}
} catch (error) {
if (row) {
noteRetryFailure(row, error);
} else {
// Unknown empty dir — no DB record to update. If this keeps failing
// (e.g. read-only filesystem), the warn fires on every sweep.
// Consider recording a row here to gain backoff / attempt counting.
}
logger.warn("lane.residual_worktree_cleanup.remove_failed", {
projectRoot,
laneId: row?.lane_id ?? null,
worktreePath: pathKey,
error: error instanceof Error ? error.message : String(error),
});
}
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/desktop/src/main/services/lanes/worktreeResidualCleanup.ts
Line: 284-294

Comment:
Unknown-empty-dir cleanup failures have no backoff or retry record. When `removeWorktreeDirectory` throws for a dir-scan candidate (no `row`), the only action is a `warn` log — the directory will be re-added to `candidatePaths` on every subsequent sweep as long as it stays empty and old enough, generating a fresh warning each time. On a filesystem where removal consistently fails (e.g., read-only mount, Windows handle lock), this can flood the logs on every `list()` call. A simple attempt counter in the catch block, or just skipping untracked dirs on a repeating error, would keep noise bounded.

```suggestion
        } catch (error) {
          if (row) {
            noteRetryFailure(row, error);
          } else {
            // Unknown empty dir — no DB record to update. If this keeps failing
            // (e.g. read-only filesystem), the warn fires on every sweep.
            // Consider recording a row here to gain backoff / attempt counting.
          }
          logger.warn("lane.residual_worktree_cleanup.remove_failed", {
            projectRoot,
            laneId: row?.lane_id ?? null,
            worktreePath: pathKey,
            error: error instanceof Error ? error.message : String(error),
          });
        }
```

How can I resolve this? If you propose a fix, please make it concise.

Fix in Claude Code

@mintlify

mintlify Bot commented Jun 23, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
ade-ac1c6011 🟢 Ready View Preview Jun 23, 2026, 5:03 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@arul28

arul28 commented Jun 23, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: 692b7525a4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@arul28 arul28 merged commit 93b81a6 into main Jun 23, 2026
27 checks passed
@arul28 arul28 deleted the ade/take-look-at-users-admin-f4fc0bc8 branch June 23, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant