Fast directory listing over the wire for big repos

[karngyan]

Why

The code browser (#127) and the session composer's directory picker (#112) both need to list directories on a remote machine from a phone. The naive version, "send the whole tree", dies on the first real repo: a monorepo has hundreds of thousands of paths, node_modules alone can be tens of thousands, and all of it would stream over the one Noise channel that also carries terminal output, under the relay's 1 MiB frame cap (relay/src/hub.ts, MAX_CLIENT_MESSAGE). This ticket is the listing protocol that stays fast no matter what is on disk.

Rough scope

• A new wire verb, alongside stat and read:

  list{id, path, reqId, cursor?}   ------->  resolve against session cwd, ReadDir
       <-------  listing{reqId, entries[], next?}
  

  One round trip per expanded directory. Nothing is fetched for a directory nobody opened.

• Entries carry {name, kind, size, mtime, ignored, status?}. Sorted on the daemon, directories first, so the browser renders as it receives. Paged at a few hundred entries per frame with a next cursor, so a 50,000-entry directory cannot produce a frame anywhere near the cap or stall a keystroke behind it.

• Caps that mirror file.go: concurrent listings per connection (like maxReads = 2), path length, entries per page.

• gitignore awareness without a per-entry exec: batch the page through git check-ignore --stdin once, or read git ls-files --others --ignored --exclude-standard --directory once per repository root and cache it keyed on .git/index mtime. Non-git directories (home, /tmp) fall back to plain ReadDir with no decorations.

• Git status decorations from one git status --porcelain=v2 per repository root, cached the same way.

• Path resolution, symlink handling, and the no-fence decision are the ones resolvePath and startRead already made; reuse them, do not reinvent.

Later, separate from this ticket

A path index for the fuzzy finder. git ls-files -z streamed as binary frames the way file chunks are (FrameFile, 32 KiB), prefix-compressed, cached in the browser keyed on repository root, HEAD, and index mtime, matched in a worker. Tracked files only, which is what makes it small enough. The index is also what would let @pierre/trees work at all, since it wants the full path list; see #129.

Hard parts, named up front

• Invalidation. The agent is editing the tree while you look at it. Without fsnotify, the honest v1 is a refresh button plus short TTLs on the git caches. Watching is a later ticket.
• Symlink loops and directories with millions of entries. ReadDir with a page size handles the second; the first needs the same EvalSymlinks discipline as startRead.
• Sorting a huge directory on the daemon is a full read before the first page. Acceptable for v1; note it.

Notes

• Blocks the tree in Code browser: file tree and viewer for a session's working directory #127 and the directory picker in Session composer: start agent sessions from any device #112.
• Go: internal/wire/control.go verb table, internal/daemon/conn.go dispatch, internal/daemon/file.go for the resolve and cap patterns. Web: web/src/client/client.ts next to stat() and read(). Spec: spec/protocol.md, "Reading files".