docs: rewrite README to house standard

[steipete]

Summary

• Rewrites the README as a concise front door: pitch, dynamic badges, install, a token-free quickstart, archive-source choices, search, sharing, automation, development, and license.
• Reduces the README from 968 lines to 144 while keeping the banner and pointing deeper readers to the existing documentation site.
• Adds a one-line Unreleased changelog entry, following the repository's precedent for documentation changes.

Documentation disposition

No new reference files were needed. The removed command catalog, bot setup, configuration, storage, security, embedding, sharing, and analytics detail already lives under docs/ and at discrawl.sh; the rewritten README links those canonical pages directly.

No factual feature material was dropped as obsolete or unverifiable. Repeated reference prose was removed from the README, and claims retained in the shorter version were cross-checked against the Go manifest, CLI help, source, workflows, release assets, Homebrew formula, and existing focused docs.

Verification

• make check — passed: tidy/module verification, format, lint, vet, staticcheck, deadcode, gosec, govulncheck, tests, 85.1% coverage, race tests, CLI smoke tests, and six-platform snapshot builds.
• go build -o bin/discrawl ./cmd/discrawl and ./bin/discrawl --version — passed.
• node scripts/build-docs-site.mjs — passed.
• Quickstart transcript against an isolated synthetic Discord Desktop cache — dry-run found one guild/channel/message without writes; import stored it; plain and filtered FTS searches returned the expected message; JSON status/diagnostics/coverage/failures and publish --check passed.
• Bot-backed init/doctor/sync and interactive tui — flags and usage validated against the built binary's real help output; no live Discord credential or interactive terminal was used.
• Homebrew and release channels — verified against openclaw/homebrew-tap and the current GitHub release assets; the host installation was not modified.
• Links — all 32 relative targets resolve; every external URL returned HTTP 200; every Shields SVG rendered without invalid or not found output.
• autoreview --mode local — clean, with no accepted/actionable findings.

[clawsweeper]

Codex review: needs changes before merge. Reviewed August 2, 2026, 8:14 PM ET / August 3, 2026, 00:14 UTC.

ClawSweeper review

What this changes

This branch replaces the long README with a concise installation, archive-source, search, sharing, automation, and documentation front door, and adds an Unreleased changelog entry.

Merge readiness

⚠️ Needs maintainer review before merge - 1 item remains

Keep this PR open for human review because its author is a repository collaborator. The README rewrite is focused and its documented commands, archive modes, and canonical links match the current source and docs; however, the release-owned CHANGELOG.md entry should be removed before merge.

Priority: P3
Reviewed head: 05d909548dd3dea9ca3457aeb8ce0858e45d2047

Review scores

Measure	Result	What it means
Overall readiness	🦐 gold shrimp (3/6)	The documentation rewrite and its reported live CLI validation are strong, but the release-owned changelog edit is a concrete pre-merge correction.
Proof confidence	🦞 diamond lobster (5/6)	Sufficient (terminal): The PR body provides an after-change terminal validation transcript covering the README quickstart against an isolated synthetic Discord Desktop cache, plus command, link, and documentation-site checks.
Patch quality	🦐 gold shrimp (3/6)	1 actionable review finding remain.

Verification

Check	Result	Evidence
Real behavior	Verified	Sufficient (terminal): The PR body provides an after-change terminal validation transcript covering the README quickstart against an isolated synthetic Discord Desktop cache, plus command, link, and documentation-site checks.
Evidence reviewed	4 items	Focused documentation diff: The branch changes only the README and changelog: 66 README lines added, 890 removed, plus four changelog lines. The rewritten README retains links to the existing install, command, configuration, archive, search, and security documentation. README claims match current product documentation: Current source and docs define wiretap, subscribe-cloud, FTS5 search, optional semantic and hybrid search, snapshot exclusions for local direct messages, keyring token handling, and the documented Make targets used by the README. Release-owned changelog change: The only non-README change adds a documentation subsection and entry under Unreleased. Release notes are release-owned under the review policy, so this normal documentation PR should leave the changelog unchanged.
Findings	1 actionable finding	[P2] Remove the release-owned changelog entry
Security	None	None.

How this fits together

The README is Discrawl’s first-run documentation surface: it directs users to installation and local/archive ingestion paths, then hands detailed configuration and command guidance to the docs site. The CLI and its local SQLite archive, Git snapshot, and Worker-backed archive modes supply the behavior the README describes.

flowchart LR
  A[New user] --> B[README front door]
  B --> C[Install and quick start]
  B --> D[Archive source choice]
  D --> E[Discrawl CLI]
  E --> F[Local SQLite archive]
  E --> G[Git or remote archive]
  B --> H[Detailed documentation]

Loading

Before merge

• Remove the release-owned changelog entry (P2) - Leave CHANGELOG.md to the release process. This documentation-only PR can land without an Unreleased release-note entry; retaining it conflicts with the repository review policy for normal PRs.

Findings

• [P2] Remove the release-owned changelog entry — CHANGELOG.md:5-7

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Documentation reduction	2 files affected; 70 added, 890 removed	The PR materially simplifies the repository front door while keeping detailed material in existing canonical docs.
README size	968 lines to 144 lines	The condensed entry point is the principal user-facing change and warrants a link and claim review.

Technical review

Best possible solution:

Land the concise README navigation rewrite after removing the Unreleased changelog entry, leaving release notes to the release-owned process.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this is a documentation-only change. The relevant validation is source and documentation consistency, and the documented CLI/archive paths are present on current main.

Is this the best way to solve the issue?

No: the README consolidation is a maintainable direction, but updating the release-owned changelog is not the best path for a normal documentation PR and should be removed.

Full review comments:

• [P2] Remove the release-owned changelog entry — CHANGELOG.md:5-7
  Leave CHANGELOG.md to the release process. This documentation-only PR can land without an Unreleased release-note entry; retaining it conflicts with the repository review policy for normal PRs.
  Confidence: 0.99

Overall correctness: patch is incorrect
Overall confidence: 0.99

AGENTS.md: not found in the target repository.

Codex review notes: model internal, reasoning high; reviewed against f5074c9bf61f.

Labels

Label changes:

• add P3: This is a low-risk documentation and release-note ownership cleanup with no runtime behavior change.
• add proof: sufficient: Contributor real behavior proof is sufficient. The PR body provides an after-change terminal validation transcript covering the README quickstart against an isolated synthetic Discord Desktop cache, plus command, link, and documentation-site checks.
• add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦞 diamond lobster and patch quality is 🦐 gold shrimp.
• add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (terminal): The PR body provides an after-change terminal validation transcript covering the README quickstart against an isolated synthetic Discord Desktop cache, plus command, link, and documentation-site checks.

Label justifications:

• P3: This is a low-risk documentation and release-note ownership cleanup with no runtime behavior change.
• rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦞 diamond lobster and patch quality is 🦐 gold shrimp.
• status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (terminal): The PR body provides an after-change terminal validation transcript covering the README quickstart against an isolated synthetic Discord Desktop cache, plus command, link, and documentation-site checks.
• proof: sufficient: Contributor real behavior proof is sufficient. The PR body provides an after-change terminal validation transcript covering the README quickstart against an isolated synthetic Discord Desktop cache, plus command, link, and documentation-site checks.

Evidence

Acceptance criteria:

• [P1] git diff --check.
• [P1] git show HEAD:README.md | sed -nE 's/.]((docs/[^)#]+|LICENSE))./\1/p' | while IFS= read -r path; do test -e "$path" || printf 'MISSING %s\n' "$path"; done.

What I checked:

• Focused documentation diff: The branch changes only the README and changelog: 66 README lines added, 890 removed, plus four changelog lines. The rewritten README retains links to the existing install, command, configuration, archive, search, and security documentation. (README.md:34, 05d909548dd3)
• README claims match current product documentation: Current source and docs define wiretap, subscribe-cloud, FTS5 search, optional semantic and hybrid search, snapshot exclusions for local direct messages, keyring token handling, and the documented Make targets used by the README. (docs/install.md:71, f5074c9bf61f)
• Release-owned changelog change: The only non-README change adds a documentation subsection and entry under Unreleased. Release notes are release-owned under the review policy, so this normal documentation PR should leave the changelog unchanged. (CHANGELOG.md:5, 05d909548dd3)
• Current README ownership history: The existing README content traces to the current release commit authored by Peter Steinberger; the submitted rewrite is also authored by the same current-area contributor. (README.md:1, 762f10c51f83)

Likely related people:

• steipete: Peter Steinberger authored the current README baseline in the v0.12.0 release commit and authored this focused rewrite. (role: current README owner and recent documentation contributor; confidence: high; commits: 762f10c51f83, 05d909548dd3; files: README.md, CHANGELOG.md)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

• Remove the Unreleased documentation entry from CHANGELOG.md and retain the README-only documentation change.

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.