Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion .githooks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,30 @@ never seems to fire, check that setting first.

| Hook | What it does |
|---|---|
| `pre-commit` | Runs `cargo fmt` and stages the result, so CI's fmt check can't fail. |
| `pre-commit` | Runs `cargo fmt` and stages the result, so CI's fmt check can't fail. Then rejects any commit that introduces a root-level `*.md` outside the allowlist (see below). |
| `pre-push` | Blocks direct pushes to `master`; runs the QC gate (skipped when the branch changes no Rust); scans tracked files for customer references. |
| `post-checkout` | Creates `AGENTS.md` from `AGENTS.develop.md` on branch switch, if absent. |

Any hook can be bypassed with `git push --no-verify` / `git commit --no-verify`.

## Root-md allowlist guard (`pre-commit`)

Agents love dropping `*.md` files (diagnoses, plans, worklogs, test scenarios)
at the repo root. The `pre-commit` hook rejects any commit that **introduces**
(added/copied/renamed) a root-level `*.md` outside this allowlist:

- `AGENTS.md`, `AGENTS.develop.md` — agent instructions
- `CLAUDE.md` — one-line pointer to `AGENTS.md`
- `README.md`, `README_CSharp.md` — user-facing docs
- `CHANGELOG.md`, `RELEASING.md` — release infrastructure

Everything else belongs in **`.docs/`** (gitignored, local-only) — see
AGENTS.md, section *Root file hygiene (markdown)*. Only introductions are
checked: modifying an already-tracked stray is only possible after a
deliberate `--no-verify` bypass, where the file itself — not the commit — is
the violation. Dot-folders (`.githooks/`, `.github/`, `.claude/`, …) are out
of scope: a root-level file cannot be inside one.

## `customer-patterns.local`

The `pre-push` leak scan reads its patterns from `.githooks/customer-patterns.local`
Expand Down
47 changes: 45 additions & 2 deletions .githooks/pre-commit
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
#!/bin/bash
# pre-commit hook: format Rust code only.
# pre-commit hook: format Rust code + root-md allowlist guard.
#
# Runs `cargo fmt` and stages any reformatting so CI's fmt-check can't fail.
# 1. Runs `cargo fmt` and stages any reformatting so CI's fmt-check can't fail.
# 2. Rejects commits that introduce a root-level *.md outside the allowlist.
#
# Installed via `git config core.hooksPath .githooks` — see .githooks/README.md.
#
Expand All @@ -27,4 +28,46 @@ if [ -n "$FMT_CHANGED" ]; then
echo "pre-commit: staged rustfmt changes ($FMT_CHANGED)"
fi

# --- root-md allowlist guard -------------------------------------------------
# The repo root keeps only its sanctioned markdown files (AGENTS.md, section
# "Root file hygiene (markdown)"). Anything else — diagnoses, plans, test
# scenarios, worklogs — belongs in .docs/ (gitignored, local-only). Blocking
# here is cheaper than a follow-up cleanup commit after a stray lands on
# develop.
#
# Only introductions are checked (added/copied/renamed). Modifying an already
# tracked stray is only possible after a deliberate --no-verify bypass — the
# file itself is the violation then, and review catches it. Dot-folders
# (.githooks/, .github/, .claude/, ...) are out of scope by construction: a
# root-level file cannot be inside one.
ALLOWED_ROOT_MD="AGENTS.md AGENTS.develop.md CLAUDE.md README.md README_CSharp.md CHANGELOG.md RELEASING.md"

while IFS= read -r staged; do
# root level = no slash anywhere in the staged (post-rename) path
case "$staged" in
*/*) continue ;;
esac
# tr, not ${var,,}: the parameter-expansion lowercase needs bash >= 4 and
# stock macOS still ships bash 3.2, where it is a fatal "bad substitution"
# that would block EVERY commit containing a root-level file.
case "$(printf '%s' "$staged" | tr 'A-Z' 'a-z')" in
*.md) ;;
*) continue ;;
esac
for allowed in $ALLOWED_ROOT_MD; do
[ "$staged" = "$allowed" ] && continue 2
done
echo ""
echo "pre-commit: BLOCKED — root-level '$staged' is not on the md allowlist."
echo " Root markdown is limited to: $ALLOWED_ROOT_MD"
echo " Diagnoses, plans, worklogs and test scenarios belong in .docs/ (gitignored)."
echo " See AGENTS.md, section \"Root file hygiene (markdown)\"."
echo " Deliberate? Use: git commit --no-verify"
echo ""
exit 1
# core.quotePath=false: with the default (true), git C-quotes non-ASCII paths
# (DIAGNOSE_ü.md -> "DIAGNOSE_\303\274.md"), and the quoted trailing `"` makes
# the *.md pattern miss — a stray with a non-ASCII name would sail through.
done < <(git -c core.quotePath=false diff --cached --name-only --diff-filter=ACR)

exit 0
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,12 @@ criterion/
# Testing
/test-repos/

# Local-only markdown (diagnoses, plans, worklogs, test scenarios).
# The repo root keeps only its allowlisted .md files — see AGENTS.md
# "Root file hygiene (markdown)". Also covered by the `.*/` rule above;
# listed explicitly so the intent survives a future edit of that rule.
.docs/

# codesearch database (local index, binary files)
.codesearch.db/
test_tools.jsonl
Expand Down
6 changes: 6 additions & 0 deletions AGENTS.develop.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,12 @@ Single-file native binaries, no runtime dependencies. macOS build is manual-trig
- **Never write separate `AGENTS_xxx.md` sibling files** unless explicitly requested.
OpenCode reads `AGENTS.md` only. Out-of-repo planning goes to
`C:\WorkArea\AI\codesearch\instructions\`.
- **Root file hygiene (markdown)**: the repo root keeps only `AGENTS.md`,
`AGENTS.develop.md`, `CLAUDE.md`, `README.md`, `README_CSharp.md`,
`CHANGELOG.md`, `RELEASING.md`. Any other markdown (diagnoses, plans,
test scenarios, worklogs) goes into `.docs/` (gitignored, local-only),
never committed, and never in a recreated tracked `docs/` folder.
Enforced by the `pre-commit` hook root-md allowlist guard.
- **Path normalization**: all path comparisons must go through a single normalize utility.
Windows UNC prefixes (`\\?\C:\`), backslash/forward-slash mismatches, and worktree
`.git` file resolution have each caused subtle bugs in the past.
Expand Down
15 changes: 14 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Release narratives live in `CHANGELOG.md`; this list keeps only the load-bearing
- **Language coverage** — 17 tree-sitter grammars (table in README). `find_impact` has SCIP symbol precision for **C#** (bundled `scip-csharp`) and **TypeScript** (`npx scip-typescript`, host-resolved). Protobuf is Niveau 1 (text-aware chunking on `message`/`enum`/`service`/`rpc`) only — no `scip-protobuf` emitter exists today.
- **Scale-to-zero-safe federation: a federated peer is NEVER polled on a timer** — ⚠️ **design constraint, do not "improve" this.** Background polling of *local* repos is fine; a *federated* peer must never be contacted on any cadence. The embedded TUI's discovery tick is **config-only** (`REMOTE_ROW_REFRESH_SECS` = 5s, zero HTTP): it rebuilds mounted-remote rows from the `remote_mounts` allowlist so mount/unmount edits and `l` reloads surface, and contacts nobody. A peer is contacted only by (a) an **activity poke** — a real federated tool call just hit it, detected via `remote_peer_activity` in `ServeState`, refreshing that one peer, never a fan-out — or (b) the explicit `i` info-overlay keypress. Idle mounts therefore render activity as `-`, which is the correct steady state, not a fault. **Rejected reasoning (was shipped twice, PR #181/#184, and reverted):** "polling no faster than the host's idle-suspend term is harmless." It is not — each poll *woke* the peer's scale-to-zero replica, which then self-warmed for its own full idle window (~1h), giving ~50% duty cycle on a peer nobody queried (measured: wakes 120/121/120 min apart, zero searches). Not keeping a peer awake past its suspend term is strictly weaker than not waking it, and the two windows are unrelated values anyway (local host vs. remote peer).
- **Standalone remote TUI auth** — `codesearch serve tui --url ...` resolves the API key from `repos.json` (`remotes.*.url` match) or a `--api-key` override and threads the authenticated client through every TUI action, with distinct errors for "no key configured" vs. "key rejected (401)".
- **Keep-warm ping observability + spurious-wake fix** *(branch `fix/federated-silent-poll-diagnosis`)* — the `keep_warm_url` self-ping loop logs every ping (`debug!` on success, `warn!` on failure) instead of discarding both outcomes, and warns at startup when the target host isn't this server's own bind host — **except on a wildcard bind** (`0.0.0.0` / `::`), where our externally-visible host is unknown so the comparison proves nothing; without that carve-out the warning fired on every cold start of the *only* deployment where keep-warm is correct (Azure binds `0.0.0.0`, target is the ingress FQDN), which just trains operators to ignore it. Rule lives in the testable `keep_warm_foreign_target` helper. Keep-warm also **requires a real recorded tool call**: the old `most_recent_tool_call().unwrap_or(start)` fallback meant any wake that wasn't a tool call (`/status` and `/healthz` don't call `record_tool_call`) made the replica self-warm for its whole idle window — reachable *only* when the wake wasn't real work, so its sole practical effect was rewarding spurious wakes (~11× amplification). Full diagnosis, with Azure Log Analytics ground truth: `DIAGNOSE_FEDERATED_KEEP_WARM.md`.
- **Keep-warm ping observability + spurious-wake fix** *(branch `fix/federated-silent-poll-diagnosis`)* — the `keep_warm_url` self-ping loop logs every ping (`debug!` on success, `warn!` on failure) instead of discarding both outcomes, and warns at startup when the target host isn't this server's own bind host — **except on a wildcard bind** (`0.0.0.0` / `::`), where our externally-visible host is unknown so the comparison proves nothing; without that carve-out the warning fired on every cold start of the *only* deployment where keep-warm is correct (Azure binds `0.0.0.0`, target is the ingress FQDN), which just trains operators to ignore it. Rule lives in the testable `keep_warm_foreign_target` helper. Keep-warm also **requires a real recorded tool call**: the old `most_recent_tool_call().unwrap_or(start)` fallback meant any wake that wasn't a tool call (`/status` and `/healthz` don't call `record_tool_call`) made the replica self-warm for its whole idle window — reachable *only* when the wake wasn't real work, so its sole practical effect was rewarding spurious wakes (~11× amplification). Full diagnosis, with Azure Log Analytics ground truth: `.docs/DIAGNOSE_FEDERATED_KEEP_WARM.md`.
- **CLI aliases** — `ls` for `list` (`index`/`groups`/`remote`), `rm` for `remove`. `index rm <alias>` resolves a registered alias before falling back to path interpretation.

> ℹ️ **Remote write verbs** (`add`, `reindex --force`) require a read-write peer; the cloud peer rejects them (`--force` → HTTP 500 "could only be opened read-only; cannot force-reindex"). An **incremental** `reindex` (no `--force`) of an already-registered repo *does* succeed on the cloud peer — that is the custom-kb auto-refresh path. `list` is always safe. `rm` is not durable — the next cold start re-registers from the restored snapshot.
Expand Down Expand Up @@ -65,6 +65,19 @@ Common mistake: a subagent runs `/git pr create` with no explicit `--base`, the
>
> Why the strategy and not the option: against the regressed merge-base, `-X ours` still runs a real three-way merge that treats both sides' content as additions and drags master's stale lines in — a Frankenstein diff (`src/mcp/mod.rs` gained +333 stale lines this way on the v1.2.0 attempt). `-s ours` ignores master's tree entirely and keeps develop's content exactly, which is the desired result here (in this scenario develop's tree already equals master's content); the merge commit only exists to record master as a parent so the merge-base advances. Confirmed empirically on the v1.2.0 release: the `develop → master` PR #185 came back `CONFLICTING`; the throwaway `release/v1.2.0` branch built with `git merge -s ours origin/master` produced an empty content diff and merged clean (#186).

## Root file hygiene (markdown)

Agents love dropping `*.md` files (diagnoses, plans, worklogs, test scenarios) at the repo root. Don't. The repo root keeps **only** these markdown files:

- `AGENTS.md`, `AGENTS.develop.md` — agent instructions (and its develop source)
- `CLAUDE.md` — one-line pointer to `AGENTS.md`, nothing else
- `README.md`, `README_CSharp.md` — user-facing docs
- `CHANGELOG.md`, `RELEASING.md` — release infrastructure

Any other markdown — diagnosis write-ups, implementation plans, test scenarios, worklogs — goes into **`.docs/`** (gitignored, local-only). Never commit it, and never recreate a tracked `docs/` folder; the old `docs/` was dissolved into `.docs/` for this reason.

Enforced by the `pre-commit` hook (root-md allowlist guard — see `.githooks/README.md`): a commit that adds a root-level `*.md` outside the allowlist is rejected. Dot-folders (`.githooks/`, `.github/`, `.claude/`, …) are exempt — this rule polices only loose files at the root.

## Notes for OpenCode / agents

- **Validation:** `cargo check` and `cargo clippy` for iteration. No `--release` builds — always dev/debug until the very end.
Expand Down
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ finalized in place with a date — no renaming/migration step needed.

## [1.3.2]

### Added

- **`pre-commit` hook now enforces the root-md allowlist** (`.githooks/`): a commit that introduces (adds/copies/renames) a root-level `*.md` outside `AGENTS.md`, `AGENTS.develop.md`, `CLAUDE.md`, `README.md`, `README_CSharp.md`, `CHANGELOG.md`, `RELEASING.md` is rejected with a pointer to `.docs/`. Shipped together with the cleanup itself: the stray root mds (`DIAGNOSE_*`, `PLAN_*`, `TEST-SCENARIO-*`) and the tracked `docs/` folder were dissolved into the gitignored `.docs/` folder, and the rule is documented in AGENTS.md, section *Root file hygiene (markdown)*. Requires `git config core.hooksPath .githooks` (already set in existing clones; fresh clones see `.githooks/README.md`).

### Fixed

- **`grep-guard.sh` now resolves Grep coverage from the search target and serve-hub registration instead of the hook's cwd and `.codesearch.db` presence (issue #199).** Two defects fixed together in the POSIX bash hook. (1) The #54 "resolve the repo from the grep target" rewrite never actually worked for POSIX absolute paths: its absolute-detection pattern (`[A-Za-z]:[\\/]*|/[a-zA-Z]/*|//*`) only matched Windows-style roots (drive letters, MSYS `/c/`, UNC `//server`), so a target like `/home/u/projects/my-repo/src` fell into the *relative* branch and resolved against the hook's cwd — with a session cwd that is not itself a git repo (a parent directory holding several repos, the #199 repro) the target resolved to nothing, looked external, and the guard silently allowed every Grep while appearing fully healthy. `/*` now covers every absolute path. (2) Coverage is decided by **registration**: the target's git root must equal one of the repos in the hub's `~/.codesearch/repos.json` (honoring the `CODESEARCH_REPOS_CONFIG` override, mirroring `config_path()`; comparison normalizes `\\?\` prefixes and slash direction, case-insensitive for Windows drive paths — same semantics as the hub's own `/indexing` resolver). A `.codesearch.db` directory at the git root is no longer a signal: it was wrong in both directions (a stale db from a since-unregistered repo denied Grep although the hub could not answer for it — unknown alias — and a registered repo whose db directory was gone slipped through uncovered). Matching the git root rather than a path prefix gives the #199 nested-repo carve-out structurally: an unregistered clone nested inside a registered repo resolves to its own git root, equals no registration, and is correctly treated as uncovered. The resolver fails open (missing/unreadable/malformed `repos.json`, missing `jq` → allow, never deny), and the `CODESEARCH_SERVER` opt-in for pure remote-serve setups is unchanged (its misuse as a coverage signal is tracked separately in #199). The PowerShell twin (`grep-guard.ps1`) keeps the older cwd-tolerant `.codesearch.db` coverage signal for now; porting it is the remaining #199 follow-up.
Expand Down
Loading
Loading