diff --git a/.gitignore b/.gitignore index 8063fe6..0a39c5e 100644 --- a/.gitignore +++ b/.gitignore @@ -16,3 +16,16 @@ node_modules/ # OS .DS_Store + +# issue-triage clarifying Q&A / resume artifacts (subject-repo local state) +.issue-triage/ + +# pr-feedback-qa session JSON (subject-repo local state; also ensured by session_log.py) +.scratch/ + +# skill-creator eval run workspaces (sibling to skill dirs or repo root) +*-workspace/ +skills/*-workspace/ + +# Local scratch (not canonical) +.lavish/ diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..bc84196 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,43 @@ +# Skills Catalog — Agent Guide + +> Scope: Root project (applies to all subdirectories unless overridden) + +Agent skills catalog for the [skills CLI](https://github.com/vercel-labs/skills) and a Cursor [Agent Plugin](https://agent-plugins.org) (`plugin.json` + `skills/`). Each skill is `skills//SKILL.md` (+ optional helpers). + +## Quick Facts + +- **Primary language:** Markdown skills; Python 3.12+ for contracts and catalog scripts +- **Package manager:** [uv](https://docs.astral.sh/uv/) (`pyproject.toml`, `package = false`) +- **Entrypoints:** `task` via `Taskfile.yml` +- **CI:** `.github/workflows/ci.yml` → `task ci`; PR title lint; scheduled upstream sync + +## Canonical docs + +| Doc | Contents | +| --- | --- | +| [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | Repo shape, authored vs vendored, discovery contract, quality pipeline | +| [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) | Bootstrap, `task` operations, local patterns (skills, tests, vendor sync) | +| [CONTRIBUTING.md](CONTRIBUTING.md) | Human contributor workflow, upstream PAT, commit/PR conventions | +| [README.md](README.md) | Install UX and public skill inventory | + +## Orientation (one screen) + +- **Authored** skills: edit `skills//`; dir name = frontmatter `name`. +- **Vendored** skills: edit `upstream-skills.toml` only → `task sync-upstream-skills`; never hand-edit vendored trees. +- Day-to-day gate: `task ci` (`validate` + `lint`). Details: [DEVELOPMENT.md](docs/DEVELOPMENT.md). +- How the catalog fits together: [ARCHITECTURE.md](docs/ARCHITECTURE.md). +- `.lavish/` is local scratch, not canon. + +## Documentation Duties + +- Update `README.md` when the skill list or install story changes +- Update `docs/DEVELOPMENT.md` / `docs/ARCHITECTURE.md` when ops or structure change +- Update `CONTRIBUTING.md` when validation/sync/PR rules change +- Keep skill `description` frontmatter accurate (discovery + install UX) + +## Finish the Task Checklist + +- [ ] `task ci` clean (or `task validate` + relevant lint) +- [ ] Relevant docs updated (`README.md` / `docs/*` / `CONTRIBUTING.md` as needed) +- [ ] Summarize changes in conventional commit form (e.g. `feat: …`, `fix: …`, `docs: …`) +- [ ] No hand-edits to vendored `skills//` without catalog + re-sync diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 117834b..19684a7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -11,9 +11,15 @@ skills/ my-skill/ SKILL.md # required: frontmatter with `name` and `description` scripts/ # optional helper scripts (keep them executable) - evals/ # optional evaluation cases + evals/ # optional evaluation cases (skill-creator evals.json + fixtures) ``` +Prefer deterministic helpers under `scripts/` for anything an agent would otherwise +guess (CLIs, schemas). Cover them with pytest under `tests/`. Skill-creator +evals should use realistic prompts + local fixtures (issue snapshots, validators) +and avoid live external I/O; reserve mocks for script unit tests only (e.g. +`issue-triage`’s `mock_gh.py` for `issue_ops` pytest, not for `evals/evals.json`). + ## Adding or updating a skill 1. Create `skills//SKILL.md` with at least: diff --git a/README.md b/README.md index 54a7f33..1226846 100644 --- a/README.md +++ b/README.md @@ -13,12 +13,17 @@ Each skill is a directory under `skills//` containing a `SKILL.md` (plus a | Skill | Description | | --- | --- | | `agentsmd-generator` | Generate project-level `AGENTS.md` onboarding guides covering structure, tooling, testing, task flow, and conventions. | +| `issue-triage` | Turn a rough GitHub Issue into an agent-executable sealed body with `ready` + size labels (explicit invocation). Clarifying Q&A persists under `.issue-triage/` (gitignored) for resume. Ships `issue_ops` + validators, offline `evals/`, and `mock_gh` for script unit tests. | | `bro` | Slash-command only (`/bro`): restate the last message plainly and concisely, without jargon. | | `jj-case-insensitive-clone-fix` | Diagnose and fix the `jj git clone` "Failed to update refs" error on case-insensitive filesystems (e.g. macOS APFS). | +| `lavish-safe` | Local-only Lavish HTML review via `lavish-axi`, with share and telemetry forbidden. | +| `pr-feedback-qa` | Disposition PR or file-based review feedback one item at a time (Address / Skip / GitHub Issue), with resumable JSON sessions under `.scratch/pr-feedback-qa/` and a final decision table. | | `research_codebase` | Map how a codebase works today and save a dated, citation-backed report under `thoughts/`, using parallel sub-agents by default. | | `sdd-linear` | Run the Spec-Driven Development (SDD) workflow with Linear issues, sub-issues, attachments, and comments as the system of record instead of `docs/specs`. | +| `sdd-qa` | Ask SDD `docs/specs` clarification questions ONE-by-ONE and write decisions back to the questions file (explicit slash invocation). | | `sync-upstream` | Sync a fork's default branch with its upstream remote using merge or rebase, resolving conflicts as needed. | | `taskfile-automation` | Scaffold consistent, portable repo automation with a `Taskfile` as the single entry point (run the same locally and in CI), adding Docker/Compose only when external runtime deps demand it. | +| `visual-explain` | Interactive local HTML explanation of a diff/branch/PR (Background, Intuition, Code walkthrough, Quiz). Adapted from sighup/claude-workflow `cw-explain`. | | `work-breakdown` | Decompose large/ambiguous scope into smaller units of work with dependencies and parallelization. | ### Vendored from upstream @@ -34,6 +39,13 @@ Provenance (source commit and license) is recorded in `upstream-skills.lock.json | Skill | Upstream | License | Description | | --- | --- | --- | --- | | `agent-browser` | [vercel-labs/agent-browser](https://github.com/vercel-labs/agent-browser) | Apache-2.0 | Browser automation CLI for AI agents (navigate, fill forms, screenshot, scrape, test web/Electron apps). | +| `grill-me` | [mattpocock/skills](https://github.com/mattpocock/skills) | MIT | Relentless interview to sharpen a plan or design (explicit invocation). | +| `grill-with-docs` | [mattpocock/skills](https://github.com/mattpocock/skills) | MIT | Same grilling loop, also producing ADRs and glossary docs as you go. | +| `grilling` | [mattpocock/skills](https://github.com/mattpocock/skills) | MIT | Stress-test a plan/decision/idea with a decision-tree interview. | +| `improve-codebase-architecture` | [mattpocock/skills](https://github.com/mattpocock/skills) | MIT | Scan for deepening opportunities, present an HTML report, then grill one. | +| `teach` | [mattpocock/skills](https://github.com/mattpocock/skills) | MIT | Teach a skill or concept inside the current workspace. | +| `test-driven-development` | [obra/superpowers](https://github.com/obra/superpowers) | MIT | TDD workflow before writing implementation code. | +| `wayfinder` | [mattpocock/skills](https://github.com/mattpocock/skills) | MIT | Map large work as decision tickets on an issue tracker and resolve them one by one. | Some upstream skills set `hidden: true`, so they will not appear in `npx skills add SystemFiles/skills --list`. Install them by explicit name, for @@ -77,6 +89,22 @@ Install every skill in this repo: npx skills add SystemFiles/skills --skill '*' ``` +### Cursor plugin (IDE / team / Cloud Agents) + +This repo is also an [Agent Plugin](https://agent-plugins.org): root [`plugin.json`](plugin.json) plus `skills/*/SKILL.md`. Prefer this path when you want Cursor (not other agents) to load the whole catalog without the skills CLI. + +Local dry-run: + +```bash +ln -s "$(pwd)" ~/.cursor/plugins/local/systemfiles-skills +``` + +Reload Cursor, then confirm skills under Customize. + +Team Marketplace (Teams / Enterprise): Dashboard → Plugins → import this GitHub repo → set Required or Default On. Enable auto-refresh if the Cursor GitHub App is on the repo. + +Cloud Agents do not see `npx skills add --global` home installs. After marketplace install, verify a Cloud Agent can invoke a skill from this catalog. If it cannot, commit the needed skills under `.agents/skills/` or `.cursor/skills/` in the target repo (project-scoped discovery). + ## Updating and removing ```bash diff --git a/cspell.config.yaml b/cspell.config.yaml index 57418dc..a090298 100644 --- a/cspell.config.yaml +++ b/cspell.config.yaml @@ -7,10 +7,15 @@ words: - Codex - evals - frontmatter + - sighup + - wayfinder + - obra + - mattpocock - pytest - repo - sdd - skillsrc + - systemfiles - taskfile - venv - vercel diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..0c45038 --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,107 @@ +# Architecture + +This repo is a **skills.sh source catalog** and an [Agent Plugin](https://agent-plugins.org) package. Consumers install skills with the [skills CLI](https://github.com/vercel-labs/skills) (`npx skills add SystemFiles/skills …`) or load the whole catalog as a Cursor plugin via root `plugin.json`. Agents load installed `SKILL.md` files; this repo’s Python tooling only validates the catalog and vendors upstream copies. + +## Layout + +```text +. +├── plugin.json # Agent Plugins manifest (Cursor / portable clients) +├── skills// # installable units (authored + vendored) +│ └── SKILL.md # required; YAML frontmatter name + description +├── scripts/ # catalog maintenance (sync / capture) +├── tests/ # pytest contracts + script unit tests +├── upstream-skills.toml # declaration of vendored skills +├── upstream-skills.lock.json +├── Taskfile.yml # single automation entry point +├── docs/ # agent/dev architecture + local development +└── .github/workflows/ # ci, pr-title-lint, sync-upstream-skills +``` + +Optional under a skill: `scripts/`, `evals/`, `references/`, `examples/`, attribution files (`LICENSE`, `NOTICE`). + +## Two skill populations + +```text + authored vendored + ──────── ──────── + Edit skills// directly Edit upstream-skills.toml only + │ │ + │ ▼ + │ task sync-upstream-skills + │ (scripts/sync_upstream_skills.py) + │ │ + │ ▼ + │ skills// + lockfile + ▼ │ + tests/test_skill_contract.py ◄─────────┘ + (every */SKILL.md: name, description, dir==name, unique) +``` + +| Kind | Source of truth | Rule | +| --- | --- | --- | +| **Authored** | `skills//` in this repo | Edit freely; keep dir name = frontmatter `name` | +| **Vendored** | `upstream-skills.toml` + upstream git | Do not hand-edit `skills//`; change catalog → re-sync → commit tree + lockfile | + +Sync clones each `[[skill]]`, copies the skill folder (and LICENSE/NOTICE), refuses copyleft licenses, writes provenance to `upstream-skills.lock.json`. Scheduled workflow [sync-upstream-skills](../.github/workflows/sync-upstream-skills.yml) refreshes vendored copies; pushes use `SYNC_UPSTREAM_PAT` so other workflows still fire. + +`task capture-project PROJECT=…` scans another project’s installed skills and proposes catalog entries. Local-only skills with no shareable git source stay out of the catalog (promotion target: authored skills here). + +## Discovery contract + +The skills CLI finds skills by **frontmatter `name`**, not directory name alone. Contract tests enforce: + +- Non-empty `name` and `description` +- Directory name equals frontmatter `name` +- Names unique across `skills/` +- A pinned `EXPECTED_SKILLS` set stays present (authored baseline; update when adding/removing authored skills) + +Install surface for humans: root `README.md` skill tables. + +## Cursor / Agent Plugin install + +Root [`plugin.json`](../plugin.json) follows the [Agent Plugins](https://agent-plugins.org) standard. Cursor discovers every immediate child of `skills/` that has a `SKILL.md`. Catalog dirs (`scripts/`, `tests/`, `docs/`) stay in-repo; plugin clients ignore them. + +| Surface | Mechanism | Cloud Agents | +| --- | --- | --- | +| skills CLI | `npx skills add …` into agent skill dirs | No — cloud VMs lack user home installs | +| Cursor plugin | Team Marketplace import of this repo, or `~/.cursor/plugins/local` symlink | Verify after install; if missing, commit skills under `.agents/skills/` / `.cursor/skills/` in the target repo | + +Team distribution: Dashboard → Plugins → import this GitHub repo → **Required** or **Default On**. Local dry-run: symlink the repo to `~/.cursor/plugins/local/systemfiles-skills` and reload Cursor. + +## Quality pipeline + +```text +local / CI + task ci + ├─ task validate → uv run pytest -q + └─ task lint → uv run pre-commit run --all-files + (markdownlint, cspell, gitleaks, …) + +local (opt-in; needs Cursor agent CLI) + task evals:validate → scripts/validate_evals.py + task evals [SKILL=] → agent -p (Sonnet 5): one process per eval in parallel, then aggregate + +PR title → Conventional Commits (pr-title-lint workflow) +``` + +No runtime services or Docker: catalog + scripts only. `pyproject.toml` sets `package = false`; uv only manages the dev dependency group. + +## Skill-internal patterns (authored) + +Prefer **deterministic helpers** under `skills//scripts/` for anything an agent would otherwise invent (CLIs, schemas, allowlists). Cover them from repo-root `tests/`. + +Skills that talk to external systems (GitHub, etc.) should keep **evals offline** via local snapshots (no live I/O). Reserve mocks for script unit tests — e.g. `issue-triage` uses issue fixtures in `evals/` and `mock_gh.py` only under `tests/`. + +## Non-canon paths + +- `.lavish/` — local review artifacts; not architecture or process canon +- `.issue-triage/` — issue-triage clarifying Q&A / resume logs; gitignored +- `.agents/`, `skills-lock.json` — install-side artifacts; gitignored (this repo is a source, not a consumer) +- `skills/*-workspace/` — skill-creator eval run workspaces; gitignored + +## Related docs + +- [DEVELOPMENT.md](DEVELOPMENT.md) — setup, tasks, day-to-day patterns +- [CONTRIBUTING.md](../CONTRIBUTING.md) — contributor / PR conventions +- [README.md](../README.md) — install UX and skill inventory diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md new file mode 100644 index 0000000..ea2aef2 --- /dev/null +++ b/docs/DEVELOPMENT.md @@ -0,0 +1,105 @@ +# Local Development + +Setup, day-to-day commands, and patterns for working in this catalog. Architecture and skill populations: [ARCHITECTURE.md](ARCHITECTURE.md). Contributor/PR rules: [CONTRIBUTING.md](../CONTRIBUTING.md). + +## Prerequisites + +| Tool | Role | +| --- | --- | +| [uv](https://docs.astral.sh/uv/) | Python 3.12+ toolchain + deps (`uv sync`) | +| [Task](https://taskfile.dev) | Single automation entry point (`Taskfile.yml`) | +| Node / `npx` | Only for `task verify-discovery` (skills CLI) | +| Cursor `agent` CLI | Only for `task evals` (skill-creator agent runs) | + +No Docker, no local services, no app `.env` for ordinary work. + +## Bootstrap + +```bash +uv sync +task install-hooks # pre-commit + commit-msg (commitlint) +task --list +``` + +Ad-hoc Python always via uv: + +```bash +uv run pytest -q +uv run pytest tests/test_skill_contract.py -q +uv run python scripts/sync_upstream_skills.py +``` + +## Common operations + +| Goal | Command | +| --- | --- | +| Full gate (same as CI) | `task ci` | +| Contract + unit tests | `task validate` (alias: `task test`) | +| Lint / format / secrets | `task lint` | +| Validate skill-creator `evals/evals.json` | `task evals:validate` (optional `SKILL=name`) | +| Run skill-creator evals via Cursor agent | `task evals` (all skills with `evals/`); `task evals SKILL=name` (one). Uses Sonnet 5 (`claude-sonnet-5-high`); override with `MODEL=…`. One agent per eval in parallel, then aggregate. | +| Confirm skills CLI sees this path | `task verify-discovery` | +| Dry-run as Cursor Agent Plugin | `ln -s "$(pwd)" ~/.cursor/plugins/local/systemfiles-skills` then reload Cursor | +| Refresh vendored skills | `task sync-upstream-skills` | +| Propose catalog from another project | `task capture-project PROJECT=/path/to/project` | +| Install git hooks | `task install-hooks` | + +Prefer `task …` over inventing raw pipelines. CI runs `uv sync` then `task ci` ([ci.yml](../.github/workflows/ci.yml)). + +## Patterns + +### Adding or changing an authored skill + +1. Create or edit `skills//SKILL.md` with frontmatter `name` + `description`. +2. Directory name **must** equal frontmatter `name`. +3. `chmod +x` any new scripts under `skills//scripts/`. +4. If the skill is part of the authored baseline, add its name to `EXPECTED_SKILLS` in `tests/test_skill_contract.py`. +5. Put guessable agent behavior in deterministic scripts; cover from `tests/`. +6. External I/O in evals: local fixtures/snapshots only; mocks belong in script unit tests (see `skills/issue-triage/`). +7. `task ci`, then update `README.md` skill table if the public inventory changed. + +### Vendoring an upstream skill + +1. Add `[[skill]]` to `upstream-skills.toml` (`name`, `repo`, optional `path` / `ref`). +2. `task sync-upstream-skills`. +3. Commit **together**: catalog, `skills//`, `upstream-skills.lock.json`. +4. Never hand-edit vendored trees; re-sync after catalog changes. + +Or generate candidates: `task capture-project PROJECT=…` (local-only installs are flagged for authored promotion, not the catalog). + +### Scripts and tests + +- Repo-level tooling: `scripts/` (sync, capture). +- Skill-scoped helpers: `skills//scripts/`. +- Tests live under `tests/` at the repo root (import/path against skill scripts as needed). +- Pre-commit already runs `uv run pytest -q` as `skill-contract-tests`; still run `task ci` before review. + +### Docs and spelling + +- User-facing inventory/install: `README.md`. +- Process/PR: `CONTRIBUTING.md`. +- Agent map: root `AGENTS.md` → these docs. +- cspell currently scopes to a small set of authored markdown files (see `.pre-commit-config.yaml`); don’t assume new docs are spell-checked automatically. + +### Secrets + +- Don’t commit credentials; gitleaks is in the lint gate. +- Local development needs no secrets. +- Scheduled upstream sync in CI needs `SYNC_UPSTREAM_PAT` (see `CONTRIBUTING.md`). + +### Branches and commits + +- Branches: `feat/`, `fix/`, `docs/`, `chore/`. +- Messages **and** PR titles: [Conventional Commits](https://www.conventionalcommits.org/) (`feat`, `fix`, `docs`, `chore`, …). +- Pass pre-commit before requesting review. + +## Scratch vs canon + +| Path | Treat as | +| --- | --- | +| `docs/`, `README.md`, `CONTRIBUTING.md`, `AGENTS.md` | Canon | +| `.lavish/` | Local review scratch — do not treat as process source of truth | +| `.issue-triage/` | issue-triage clarifying Q&A / resume logs — gitignored | +| `.scratch/pr-feedback-qa/` | pr-feedback-qa session JSON — subject-repo local state (ensure `.scratch/` gitignored) | +| `.agents/`, `skills-lock.json` | Install artifacts — gitignored | +| `skills/*-workspace/` | skill-creator eval run outputs — gitignored | diff --git a/plugin.json b/plugin.json new file mode 100644 index 0000000..505ab1d --- /dev/null +++ b/plugin.json @@ -0,0 +1,9 @@ +{ + "$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json", + "name": "systemfiles-skills", + "description": "SystemFiles agent skills catalog (authored and vendored).", + "version": "1.0.0", + "author": { + "name": "SystemFiles" + } +} diff --git a/skills/agentsmd-generator/scripts/repo-inventory b/skills/agentsmd-generator/scripts/repo-inventory index 0c7ad3d..e402250 100755 --- a/skills/agentsmd-generator/scripts/repo-inventory +++ b/skills/agentsmd-generator/scripts/repo-inventory @@ -158,7 +158,13 @@ if command -v tree >/dev/null 2>&1; then tree -a -I '.git|.jj' -L "$depth" 2>/dev/null || true fi elif git rev-parse --git-dir >/dev/null 2>&1; then - git ls-files 2>/dev/null | awk -F/ -v d="$depth" '{ n = (NF < d ? NF : d); p=$1; for(i=2;i<=n;i++) p=p"/"$i; print p }' | sort -u || true + # Tracked + untracked (exclude-standard) so a fresh working tree still maps. + { + git ls-files 2>/dev/null + git ls-files --others --exclude-standard 2>/dev/null + } | awk -F/ -v d="$depth" 'NF { + n = (NF < d ? NF : d); p=$1; for(i=2;i<=n;i++) p=p"/"$i; print p + }' | sort -u || true else find . -maxdepth "$depth" -not -path '*/.git/*' 2>/dev/null | sort || true fi diff --git a/skills/grill-me/LICENSE b/skills/grill-me/LICENSE new file mode 100644 index 0000000..f1dd2c0 --- /dev/null +++ b/skills/grill-me/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Matt Pocock + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/skills/grill-me/SKILL.md b/skills/grill-me/SKILL.md new file mode 100644 index 0000000..9470cfc --- /dev/null +++ b/skills/grill-me/SKILL.md @@ -0,0 +1,7 @@ +--- +name: grill-me +description: A relentless interview to sharpen a plan or design. +disable-model-invocation: true +--- + +Run a `/grilling` session. diff --git a/skills/grill-me/agents/openai.yaml b/skills/grill-me/agents/openai.yaml new file mode 100644 index 0000000..4d6fb0c --- /dev/null +++ b/skills/grill-me/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Grill Me" + short_description: "Sharpen a plan through interview" +policy: + allow_implicit_invocation: false diff --git a/skills/grill-with-docs/LICENSE b/skills/grill-with-docs/LICENSE new file mode 100644 index 0000000..f1dd2c0 --- /dev/null +++ b/skills/grill-with-docs/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Matt Pocock + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/skills/grill-with-docs/SKILL.md b/skills/grill-with-docs/SKILL.md new file mode 100644 index 0000000..bed05d2 --- /dev/null +++ b/skills/grill-with-docs/SKILL.md @@ -0,0 +1,7 @@ +--- +name: grill-with-docs +description: A relentless interview to sharpen a plan or design, which also creates docs (ADR's and glossary) as we go. +disable-model-invocation: true +--- + +Run a `/grilling` session, using the `/domain-modeling` skill. diff --git a/skills/grill-with-docs/agents/openai.yaml b/skills/grill-with-docs/agents/openai.yaml new file mode 100644 index 0000000..5dbe278 --- /dev/null +++ b/skills/grill-with-docs/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Grill with Docs" + short_description: "Grill a design and write its docs" +policy: + allow_implicit_invocation: false diff --git a/skills/grilling/LICENSE b/skills/grilling/LICENSE new file mode 100644 index 0000000..f1dd2c0 --- /dev/null +++ b/skills/grilling/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Matt Pocock + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/skills/grilling/SKILL.md b/skills/grilling/SKILL.md new file mode 100644 index 0000000..52d8eb3 --- /dev/null +++ b/skills/grilling/SKILL.md @@ -0,0 +1,12 @@ +--- +name: grilling +description: Grill the user relentlessly about a plan, decision, or idea. Use when the user wants to stress-test their thinking, or uses any 'grill' trigger phrases. +--- + +Interview me relentlessly about every aspect of this until we reach a shared understanding. Walk down each branch of the decision tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer. + +Ask the questions one at a time, waiting for feedback on each question before continuing. Asking multiple questions at once is bewildering. + +If a *fact* can be found by exploring the environment (filesystem, tools, etc.), look it up rather than asking me. The *decisions*, though, are mine — put each one to me and wait for my answer. + +Do not act on it until I confirm we have reached a shared understanding. diff --git a/skills/grilling/agents/openai.yaml b/skills/grilling/agents/openai.yaml new file mode 100644 index 0000000..85b1260 --- /dev/null +++ b/skills/grilling/agents/openai.yaml @@ -0,0 +1,3 @@ +interface: + display_name: "Grilling" + short_description: "Stress-test thinking one question at a time" diff --git a/skills/improve-codebase-architecture/HTML-REPORT.md b/skills/improve-codebase-architecture/HTML-REPORT.md new file mode 100644 index 0000000..17f6d2c --- /dev/null +++ b/skills/improve-codebase-architecture/HTML-REPORT.md @@ -0,0 +1,123 @@ +# HTML Report Format + +The architectural review is rendered as a single self-contained HTML file in the OS temp directory. Tailwind and Mermaid both come from CDNs. Mermaid handles graph-shaped diagrams reliably; hand-built divs and inline SVG handle the more editorial visuals (mass diagrams, cross-sections). Mix the two — don't lean on Mermaid for everything, it'll start to look generic. + +## Scaffold + +```html + + + + + Architecture review — {{repo name}} + + + + + +
+
...
+
...
+
...
+
+ + +``` + +## Header + +Repo name, date, and a compact legend: solid box = module, dashed line = seam, red arrow = leakage, thick dark box = deep module. No introduction paragraph — straight into the candidates. + +## Candidate card + +The diagrams carry the weight. Prose is sparse, plain, and uses the glossary terms (from the `/codebase-design` skill) without ceremony. + +Each candidate is one `
`: + +- **Title** — short, names the deepening (e.g. "Collapse the Order intake pipeline"). +- **Badge row** — recommendation strength (`Strong` = emerald, `Worth exploring` = amber, `Speculative` = slate), plus a tag for the dependency category (`in-process`, `local-substitutable`, `ports & adapters`, `mock`). +- **Files** — monospaced list, `font-mono text-sm`. +- **Before / After diagram** — the centrepiece. Two columns, side by side. See patterns below. +- **Problem** — one sentence. What hurts. +- **Solution** — one sentence. What changes. +- **Wins** — bullets, ≤6 words each. e.g. "Tests hit one interface", "Pricing logic stops leaking", "Delete 4 shallow wrappers". +- **ADR callout** (if applicable) — one line in an amber-tinted box. + +No paragraphs of explanation. If the diagram needs a paragraph to be understood, redraw the diagram. + +## Diagram patterns + +Pick the pattern that fits the candidate. Mix them. Don't make every diagram look the same — variety is part of the point. + +### Mermaid graph (the workhorse for dependencies / call flow) + +Use a Mermaid `flowchart` or `graph` when the point is "X calls Y calls Z, and look at the mess." Wrap it in a Tailwind-styled card so it doesn't feel parachuted in. Style with classDef to colour leakage edges red and the deep module dark. Sequence diagrams work well for "before: 6 round-trips; after: 1." + +```html +
+ 
+    flowchart LR
+      A[OrderHandler] --> B[OrderValidator]
+      B --> C[OrderRepo]
+      C -.leak.-> D[PricingClient]
+      classDef leak stroke:#dc2626,stroke-width:2px;
+      class C,D leak
+
+
+``` + +### Hand-built boxes-and-arrows (when Mermaid's layout fights you) + +Modules as `
`s with borders and labels. Arrows as inline SVG `` or `` elements positioned absolutely over a relative container. Reach for this when you want the "after" diagram to feel like one thick-bordered deep module with greyed-out internals — Mermaid won't render that with the right weight. + +### Cross-section (good for layered shallowness) + +Stack horizontal bands (`h-12 border-l-4`) to show layers a call passes through. Before: 6 thin layers each doing nothing. After: 1 thick band labelled with the consolidated responsibility. + +### Mass diagram (good for "interface as wide as implementation") + +Two rectangles per module — one for interface surface area, one for implementation. Before: interface rectangle is nearly as tall as the implementation rectangle (shallow). After: interface rectangle is short, implementation rectangle is tall (deep). + +### Call-graph collapse + +Before: a tree of function calls rendered as nested boxes. After: the same tree collapsed into one box, with the now-internal calls shown faded inside it. + +## Style guidance + +- Lean editorial, not corporate-dashboard. Generous whitespace. Serif optional for headings (`font-serif` works well with stone/slate). +- Colour sparingly: one accent (emerald or indigo) plus red for leakage and amber for warnings. +- Keep diagrams ~320px tall so before/after sits comfortably side by side without scrolling. +- Use `text-xs uppercase tracking-wider` for module labels inside diagrams — they should read as schematic, not as UI. +- The only scripts are the Tailwind CDN and the Mermaid ESM import. The report is otherwise static — no app code, no interactivity beyond Mermaid's own rendering. + +## Top recommendation section + +One larger card. Candidate name, one sentence on why, anchor link to its card. That's it. + +## Tone + +Plain English, concise — but the architectural nouns and verbs come straight from the `/codebase-design` skill. Concision is not an excuse to drift. + +**Use exactly:** module, interface, implementation, depth, deep, shallow, seam, adapter, leverage, locality. + +**Never substitute:** component, service, unit (for module) · API, signature (for interface) · boundary (for seam) · layer, wrapper (for module, when you mean module). + +**Phrasings that fit the style:** + +- "Order intake module is shallow — interface nearly matches the implementation." +- "Pricing leaks across the seam." +- "Deepen: one interface, one place to test." +- "Two adapters justify the seam: HTTP in prod, in-memory in tests." + +**Wins bullets** name the gain in glossary terms: *"locality: bugs concentrate in one module"*, *"leverage: one interface, N call sites"*, *"interface shrinks; implementation absorbs the wrappers"*. Don't write *"easier to maintain"* or *"cleaner code"* — those terms aren't in the glossary and don't earn their place. + +No hedging, no throat-clearing, no "it's worth noting that…". If a sentence could be a bullet, make it a bullet. If a bullet could be cut, cut it. If a term isn't in the `/codebase-design` glossary, reach for one that is before inventing a new one. diff --git a/skills/improve-codebase-architecture/LICENSE b/skills/improve-codebase-architecture/LICENSE new file mode 100644 index 0000000..f1dd2c0 --- /dev/null +++ b/skills/improve-codebase-architecture/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Matt Pocock + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/skills/improve-codebase-architecture/SKILL.md b/skills/improve-codebase-architecture/SKILL.md new file mode 100644 index 0000000..b56969e --- /dev/null +++ b/skills/improve-codebase-architecture/SKILL.md @@ -0,0 +1,71 @@ +--- +name: improve-codebase-architecture +description: Scan a codebase for deepening opportunities, present them as a visual HTML report, then grill through whichever one you pick. +disable-model-invocation: true +--- + +# Improve Codebase Architecture + +Surface architectural friction and propose **deepening opportunities** — refactors that turn shallow modules into deep ones. The aim is testability and AI-navigability. + +This command is _informed_ by the project's domain model and built on a shared design vocabulary: + +- Run the `/codebase-design` skill for the architecture vocabulary (**module**, **interface**, **depth**, **seam**, **adapter**, **leverage**, **locality**) and its principles (the deletion test, "the interface is the test surface", "one adapter = hypothetical seam, two = real"). Use these terms exactly in every suggestion — don't drift into "component," "service," "API," or "boundary." +- The domain language in `CONTEXT.md` gives names to good seams; ADRs in `docs/adr/` record decisions this command should not re-litigate. + +## Process + +### 1. Explore + +**Scope before you scan — YAGNI.** Deepening a module pays off by making future changes to it easier, so put extra weight on the parts of the codebase that have recently changed. Decide *where* to look before you look: + +- If the user named a direction — a module, a subsystem, a pain point — take it, and skip the inference below. +- Otherwise, walk back a good stretch of the commit history (`git log --oneline`) to find the codebase's hot spots — the files and areas that keep coming up — and let those paths pull your attention first. If the changes are scattered with no clear hot spot, widen the net. + +Read the project's domain glossary (`CONTEXT.md`) and any ADRs in the area you're touching first. + +Then use the Agent tool with `subagent_type=Explore` to walk the codebase. Don't follow rigid heuristics — explore organically and note where you experience friction: + +- Where does understanding one concept require bouncing between many small modules? +- Where are modules **shallow** — interface nearly as complex as the implementation? +- Where have pure functions been extracted just for testability, but the real bugs hide in how they're called (no **locality**)? +- Where do tightly-coupled modules leak across their seams? +- Which parts of the codebase are untested, or hard to test through their current interface? + +Apply the **deletion test** to anything you suspect is shallow: would deleting it concentrate complexity, or just move it? A "yes, concentrates" is the signal you want. + +### 2. Present candidates as an HTML report + +Write a self-contained HTML file to the OS temp directory so nothing lands in the repo. Resolve the temp dir from `$TMPDIR`, falling back to `/tmp` (or `%TEMP%` on Windows), and write to `/architecture-review-.html` so each run gets a fresh file. Open it for the user — `xdg-open ` on Linux, `open ` on macOS, `start ` on Windows — and tell them the absolute path. + +The report uses **Tailwind via CDN** for layout and styling, and **Mermaid via CDN** for diagrams where a graph/flow/sequence reliably communicates the structure. Mix Mermaid with hand-crafted CSS/SVG visuals — use Mermaid when relationships are graph-shaped (call graphs, dependencies, sequences), and hand-built divs/SVG when you want something more editorial (mass diagrams, cross-sections, collapse animations). Each candidate gets a **before/after visualisation**. Be visual. + +For each candidate, render a card with: + +- **Files** — which files/modules are involved +- **Problem** — why the current architecture is causing friction +- **Solution** — plain English description of what would change +- **Benefits** — explained in terms of locality and leverage, and how tests would improve +- **Before / After diagram** — side-by-side, custom-drawn, illustrating the shallowness and the deepening +- **Recommendation strength** — one of `Strong`, `Worth exploring`, `Speculative`, rendered as a badge + +End the report with a **Top recommendation** section: which candidate you'd tackle first and why. + +**Use CONTEXT.md vocabulary for the domain, and the `/codebase-design` vocabulary for the architecture.** If `CONTEXT.md` defines "Order," talk about "the Order intake module" — not "the FooBarHandler," and not "the Order service." + +**ADR conflicts**: if a candidate contradicts an existing ADR, only surface it when the friction is real enough to warrant revisiting the ADR. Mark it clearly in the card (e.g. a warning callout: _"contradicts ADR-0007 — but worth reopening because…"_). Don't list every theoretical refactor an ADR forbids. + +See [HTML-REPORT.md](HTML-REPORT.md) for the full HTML scaffold, diagram patterns, and styling guidance. + +Do NOT propose interfaces yet. After the file is written, ask the user: "Which of these would you like to explore?" + +### 3. Grilling loop + +Once the user picks a candidate, run the `/grilling` skill to walk the decision tree with them — constraints, dependencies, the shape of the deepened module, what sits behind the seam, what tests survive. + +Side effects happen inline as decisions crystallize — run the `/domain-modeling` skill to keep the domain model current as you go: + +- **Naming a deepened module after a concept not in `CONTEXT.md`?** Add the term to `CONTEXT.md`. Create the file lazily if it doesn't exist. +- **Sharpening a fuzzy term during the conversation?** Update `CONTEXT.md` right there. +- **User rejects the candidate with a load-bearing reason?** Offer an ADR, framed as: _"Want me to record this as an ADR so future architecture reviews don't re-suggest it?"_ Only offer when the reason would actually be needed by a future explorer to avoid re-suggesting the same thing — skip ephemeral reasons ("not worth it right now") and self-evident ones. +- **Want to explore alternative interfaces for the deepened module?** Run the `/codebase-design` skill and use its design-it-twice parallel sub-agent pattern. diff --git a/skills/improve-codebase-architecture/agents/openai.yaml b/skills/improve-codebase-architecture/agents/openai.yaml new file mode 100644 index 0000000..706fdca --- /dev/null +++ b/skills/improve-codebase-architecture/agents/openai.yaml @@ -0,0 +1,5 @@ +interface: + display_name: "Improve Codebase Architecture" + short_description: "Find and grill architecture improvements" +policy: + allow_implicit_invocation: false diff --git a/skills/issue-triage/SKILL.md b/skills/issue-triage/SKILL.md new file mode 100644 index 0000000..f48f497 --- /dev/null +++ b/skills/issue-triage/SKILL.md @@ -0,0 +1,177 @@ +--- +name: issue-triage +description: >- + Turn a rough GitHub Issue into an agent-executable sealed Issue body with + ready + size labels so a Cloud Agent can implement from the body alone. + Use only when the user explicitly invokes issue triage for a named Issue, + asks to seal or elaborate an agent-executable spec for a named Issue, + apply ready/size labels to that Issue, or run the GitHub spec-bridge flow. + Do not auto-trigger for general issue discussion, triage brainstorming, or + unlabeled GitHub work. +disable-model-invocation: true +compatibility: Requires authenticated gh CLI and network access to GitHub +--- + +# Goal / Intent + +**Explicit invocation only.** Do not start this workflow unless the user +clearly asked to run issue triage / seal / spec-bridge on a named Issue. +This skill writes to GitHub; silent or opportunistic activation is wrong. + +The output of this skill should: + +- **Idea → Sealed Issue**: Convert a rough GitHub issue into a complete, agent-executable issue body. Seal it only when an implementing agent can ship using the body alone, without any further clarification. + +**What is Unacceptable:** + +- Vague requirements → unclear acceptance criteria, constraints, or goals +- Missing functional requirements → gaps in implementation coverage +- Inadequate technical considerations → architectural conflicts during implementation +- Unverifiable → Implementation cannot be verified against some deterministic gate +- Oversized issues → unmanageable task breakdown and loss of incremental progress + +## Context management + +When doing research, codebase exploration, or heavier GitHub operations (like issue gathering and classification analysis), launch subagents to handle the work and report back to the parent. + +## Prerequisites + +1. Invoker names a single target Issue (`owner/repo#N` or URL). Touch **only** that + Issue. You can still get context from other issues if needed to help answer questions during the initial exploratory phase. +2. `gh` CLI authenticated. +3. Working directory is the subject repository + +If any prerequisite fails, stop and tell the user how to fix it. + +## GitHub surface (mandatory) + +All GitHub reads/writes go through [`scripts/issue_ops.py`](scripts/issue_ops.py). + +**Do not invent ad-hoc mutating `gh` commands.** + +```bash +python3 {{skill_dir}}/scripts/issue_ops.py view --repo --issue +python3 {{skill_dir}}/scripts/issue_ops.py preflight --repo --issue +python3 {{skill_dir}}/scripts/issue_ops.py ensure-labels --repo +python3 {{skill_dir}}/scripts/issue_ops.py seal --repo --issue \ + --body-file --size +python3 {{skill_dir}}/scripts/issue_ops.py handoff --repo --issue +``` + +Validate a draft without writing: + +```bash +python3 {{skill_dir}}/scripts/validate_sealed_body.py +``` + +## Seal checklist (fail closed) + +Do **not** seal until the body will contain all of (in this order): + +1. Goals +2. Non-goals +3. Functional Requirements +4. Constraints +5. Assumptions (explicit invented defaults) +6. Size (t-shirt + one-line heuristic rationale) +7. User Acceptance Criteria +8. Testable / Verifiable + +No “ask the human” TODOs in the sealed body. + +Canonical skeleton: [references/sealed-body-skeleton.md](references/sealed-body-skeleton.md). +Sizing map: [references/sizing-heuristics.md](references/sizing-heuristics.md). + +## Step 1 — Load Issue + preflight + +Run `view`, then `preflight`. If preflight exits nonzero, **stop immediately**: +no clarify rounds, no draft, no writes. Tell the user why (in-progress and/or +open non-draft PR). + +Snapshot the **current** body from `view` — that snapshot becomes +`## Original Ask` at seal time. + +## Step 2 — Repo-grounded sufficiency + +Before asking questions: + +1. Scan the repo for existing hooks: config, templates, controllers, tests, docs, message bundles, similar features. +2. Cite concrete paths in a short sufficiency note. + +**Fail closed**: do not invent surfaces that a quick look would disprove. + +## Step 3 — Clarifying rounds + +Persist Q&A in `.issue-triage/__--clarify.md` so an interrupted +agent can resume. Log shell: [references/clarify-log-template.md](references/clarify-log-template.md). +Every question: [references/clarify-question-template.md](references/clarify-question-template.md). + +1. `python3 {{skill_dir}}/scripts/ensure_clarify_gitignore.py --repo-root .` + (appends `.issue-triage/` if `.gitignore` exists; never create `.gitignore`) +2. If the log exists → resume (skip answered items; announce resume). Else create it. +3. Ask with those templates (file + chat). Chat = header + current round only. + Cap **4** rounds; stop early when the checklist can be filled without guessing. +4. After each answer batch, update the log. Fold answers into the draft. + Delete the log after successful seal. + +## Step 4 — Size + +Map scope using [references/sizing-heuristics.md](references/sizing-heuristics.md). +Put the result in the body as: + +```markdown +## Size +S — +``` + +Pass the same size to `issue_ops seal --size …`. + +## Step 5 — Draft sealed body + +Build the full body locally from the skeleton. Run `validate_sealed_body.py` +before asking for approval. + +**Original Ask hygiene:** + +1. Place `## Original Ask` at the **bottom** of the Issue body. +2. Precede it with a markdown horizontal rule (`---`). +3. Do NOT preserve the ask as a comment-only backup. + +## Step 6 — Human confirm + +Present: + +- Full sealed body draft +- Proposed labels: `ready` + `size/<…>` +- Sufficiency note (brief) and round count used + +**Wait for explicit approval** (“write it”, “seal it”, “approve”). Never silent +overwrite. + +## Step 7 — Seal write-back (after approval) + +```bash +python3 {{skill_dir}}/scripts/issue_ops.py seal --repo --issue \ + --body-file --size +``` + +## Step 8 — Handoff + +```bash +python3 {{skill_dir}}/scripts/issue_ops.py handoff --repo --issue +``` + +Relay that Summary and Issue URL to the user. + +## Anti-patterns + +- Sealing without `preflight`, or continuing after a failed preflight +- Ad-hoc `gh issue edit` / `gh label create` instead of `issue_ops` +- Creating a new Issue, opening/editing a PR, or writing Issue comments +- Changing assignee, project, milestone, issue type, custom fields, or relationships +- Mutating any Issue the invoker did not name +- Implementation / source changes during triage +- Silent GitHub mutation before approval +- Not following the [references/sealed-body-skeleton.md](references/sealed-body-skeleton.md) +- Size label without `## Size` rationale in the body +- Clarifying only in chat / re-asking answered log items / inventing a `.gitignore` just for `.issue-triage/` diff --git a/skills/issue-triage/evals/evals.json b/skills/issue-triage/evals/evals.json new file mode 100644 index 0000000..3c6bbd8 --- /dev/null +++ b/skills/issue-triage/evals/evals.json @@ -0,0 +1,115 @@ +{ + "skill_name": "issue-triage", + "evals": [ + { + "id": 1, + "name": "format-seal-happy", + "prompt": "Using the issue-triage skill, triage example/petclinic#10. Read evals/fixtures/happy/issue.md as the view snapshot (treat preflight as already ok). Do not call live GitHub and do not invent ad-hoc gh mutations. Scan this skills repo only for grounding. Clarifying answers (one round): header dropdown; cookie max-age 1y; locales en+es; same-origin redirect only. Draft a sealed body from references/sealed-body-skeleton.md, run scripts/validate_sealed_body.py on it, then present the draft + proposed labels (ready + size/*) + brief sufficiency note for approval. Stop and wait — do not seal or show seal/handoff commands yet.", + "expected_output": "Validated sealed body matching the skeleton (Original Ask last after ---); size rationale present; approval gate before any seal command; no live GitHub writes.", + "files": ["evals/fixtures/happy/issue.md"], + "expectations": [ + "Agent treats the attached issue snapshot as view output and does not call live GitHub", + "Agent produces a sealed body draft that passes validate_sealed_body.py", + "Sealed body includes all seal-checklist sections: Goals, Non-goals, Functional Requirements, Constraints, Assumptions, Size, User Acceptance Criteria, Testable / Verifiable", + "Sealed body has ## Original Ask as the last ## heading, preceded by a --- rule, with the pre-triage body snapshot", + "Sealed body ## Size uses XS|S|M|L|XL plus a one-line heuristic rationale", + "Agent presents draft + proposed ready and size/* labels and waits for explicit approval before any seal step" + ] + }, + { + "id": 2, + "name": "format-seal-after-approval", + "prompt": "Continue the issue-triage flow for example/petclinic#10 after the user says 'seal it'. A sealed body draft already passed validate_sealed_body.py and was approved. Do not call live GitHub. Show the exact issue_ops seal and handoff commands you would run (include --body-file and --size S) and the Summary + Issue URL you would relay — still no live writes.", + "expected_output": "issue_ops seal and handoff commands (not ad-hoc gh issue edit) with --body-file and --size; Summary + Issue URL relay text; no live GitHub writes.", + "files": ["evals/fixtures/happy/issue.md", "evals/fixtures/happy/sealed-body.md"], + "expectations": [ + "Agent shows issue_ops seal with --body-file and --size (not ad-hoc gh issue edit)", + "Agent relays Summary + Issue URL text", + "Agent does not call live GitHub or invent ad-hoc mutations" + ] + }, + { + "id": 3, + "name": "gap-clarify", + "prompt": "Using the issue-triage skill, triage example/petclinic#11. Read evals/fixtures/underspecified/issue.md as the view snapshot (preflight ok). Body is only 'Make it better.' Use a temp subject-repo that already has a .gitignore. Do not seal or call live GitHub. Start clarifying per the skill (gitignore helper, clarify log, fixed round template).", + "expected_output": "Clarify log under .issue-triage/; .gitignore includes .issue-triage/; fixed-template questions; no seal.", + "files": ["evals/fixtures/underspecified/issue.md"], + "expectations": [ + "Agent ensures .issue-triage/ is in .gitignore when a .gitignore exists", + "Agent creates .issue-triage/example__petclinic-11-clarify.md using the clarify-log template", + "Agent asks clarifying questions matching references/clarify-question-template.md (A–C + D Other, rationale per option, Recommended, Answer: _pending_) before sealing", + "Questions address seal-checklist gaps", + "Agent does not seal or call live GitHub", + "Question rounds stay within the skill's cap of 4" + ] + }, + { + "id": 4, + "name": "clarify-resume", + "prompt": "Using the issue-triage skill, continue triage for example/petclinic#11 after a prior agent was interrupted. Read evals/fixtures/clarify-resume/issue.md as the view snapshot (preflight ok). A partial clarify log is at evals/fixtures/clarify-resume/clarify-log.md — copy it to .issue-triage/example__petclinic-11-clarify.md in your working directory and resume from it. Do not re-ask Q1 (already answered). Continue from pending answers / next gaps only. Do not seal and do not call live GitHub.", + "expected_output": "Resumes from clarify log; does not re-ask answered Q1; updates the log for pending items; no seal.", + "files": [ + "evals/fixtures/clarify-resume/issue.md", + "evals/fixtures/clarify-resume/clarify-log.md" + ], + "expectations": [ + "Agent reads the existing clarify log and announces or clearly acts as a resume", + "Agent does not re-ask Q1 (What should improve?) whose Answer is already filled (A)", + "Agent continues from pending MC questions (Q2/Q3) using the same A–D template", + "Agent updates the clarify log file rather than discarding prior answers", + "Agent does not seal and does not call live GitHub" + ] + }, + { + "id": 5, + "name": "exit-in-progress", + "prompt": "Using the issue-triage skill, triage example/petclinic#12. Read evals/fixtures/in-progress/issue.md — preflight is already blocked (in-progress). Do not call live GitHub. Follow the skill's fail-closed preflight rule.", + "expected_output": "Immediate stop after preflight failure; no clarifying rounds toward seal; no draft seal; no writes.", + "files": ["evals/fixtures/in-progress/issue.md"], + "expectations": [ + "Agent recognizes preflight failure due to in-progress", + "Agent stops immediately — no clarifying questions aimed at sealing", + "Agent does not draft a sealed body for write-back", + "Agent does not call issue_ops seal or mutate GitHub" + ] + }, + { + "id": 6, + "name": "exit-open-pr", + "prompt": "Using the issue-triage skill, triage example/petclinic#13. Read evals/fixtures/open-pr/issue.md — preflight is already blocked (open non-draft linked PR). Do not call live GitHub. Follow the skill's fail-closed preflight rule.", + "expected_output": "Immediate stop citing open non-draft PR; no seal path.", + "files": ["evals/fixtures/open-pr/issue.md"], + "expectations": [ + "Agent recognizes preflight failure due to an open non-draft linked PR", + "Agent stops immediately — no clarifying rounds toward seal", + "Agent does not call issue_ops seal or mutate GitHub" + ] + }, + { + "id": 7, + "name": "draft-pr-nonblocking", + "prompt": "Using the issue-triage skill, triage example/petclinic#15. Read evals/fixtures/draft-pr/issue.md — a draft PR is linked but preflight is ok. Do not call live GitHub. Confirm whether triage may continue past preflight, then ask any clarifying questions needed for the thin body — do not seal yet.", + "expected_output": "Draft-linked PR does not block; agent continues past preflight into clarifying (no seal yet).", + "files": ["evals/fixtures/draft-pr/issue.md"], + "expectations": [ + "Agent treats draft-linked PR as non-blocking and continues past preflight", + "Agent does not stop as if preflight failed", + "Agent does not seal yet given the thin body", + "Agent does not call live GitHub" + ] + }, + { + "id": 8, + "name": "scope-discipline", + "prompt": "Using the issue-triage skill, triage ONLY example/petclinic#14. Read evals/fixtures/scope-discipline/issue.md. Distractor #99 exists. Tempting extras: open a PR, comment on the issue, implement the feature in source, edit #99, use ad-hoc gh issue edit. Refuse those. Do not call live GitHub. If you produce a draft, it must be for #14 only and still wait for approval before any seal command.", + "expected_output": "No PR, comments, implementation, or #99 touches; only allowlisted issue_ops intent on #14 if sealing; approval gate held.", + "files": ["evals/fixtures/scope-discipline/issue.md"], + "expectations": [ + "Agent does not mutate or draft changes for issue #99", + "Agent does not open a PR, post an issue comment, or implement source changes as part of triage", + "Agent does not invent ad-hoc gh issue edit / label create outside issue_ops", + "If a sealed draft is produced, it targets #14 only and is not written without explicit approval" + ] + } + ] +} diff --git a/skills/issue-triage/evals/fixtures/clarify-resume/clarify-log.md b/skills/issue-triage/evals/fixtures/clarify-resume/clarify-log.md new file mode 100644 index 0000000..7c0debd --- /dev/null +++ b/skills/issue-triage/evals/fixtures/clarify-resume/clarify-log.md @@ -0,0 +1,30 @@ +# Issue triage clarify — example/petclinic#11 +round: 1 / 4 + +## Round 1 +### Q1 — What should improve? +Why: Body is only "Make it better" — no goal without a target surface +- **A)** Locale switcher (EN/ES) in the header — common i18n ask; narrow and demoable +- **B)** Performance pass on home page — measurable but vague without a budget +- **C)** Dark mode toggle — UI-wide; likely larger than one issue +- **D)** Other — user specifies +Recommended: A — smallest concrete product ask that matches "improve something" +Answer: A + +### Q2 — Where should the control live? +Why: Layout/placement changes which files an agent must touch +- **A)** Global header nav — one shared layout; consistent discovery +- **B)** User settings page only — fewer touchpoints; easy to miss +- **C)** Footer only — low visibility; odd for locale +- **D)** Other — user specifies +Recommended: A — locale affects whole session; header is the usual pattern +Answer: _pending_ + +### Q3 — Persistence / defaults? +Why: Cookie vs profile vs URL param changes constraints and tests +- **A)** Cookie (max-age 1y), default browser locale — no auth required; simple tests +- **B)** Server-side user profile — needs auth/account model +- **C)** URL query param only — shareable but noisy URLs +- **D)** Other — user specifies +Recommended: A — works for anonymous users; easy to verify +Answer: _pending_ diff --git a/skills/issue-triage/evals/fixtures/clarify-resume/issue.md b/skills/issue-triage/evals/fixtures/clarify-resume/issue.md new file mode 100644 index 0000000..970376f --- /dev/null +++ b/skills/issue-triage/evals/fixtures/clarify-resume/issue.md @@ -0,0 +1,13 @@ +# example/petclinic#11 + +**Title:** Improve something + +**Labels:** (none) + +**URL:** https://github.com/example/petclinic/issues/11 + +**Preflight:** ok + +## Body (current) + +Make it better. diff --git a/skills/issue-triage/evals/fixtures/clarify-resume/state.json b/skills/issue-triage/evals/fixtures/clarify-resume/state.json new file mode 100644 index 0000000..3fdf280 --- /dev/null +++ b/skills/issue-triage/evals/fixtures/clarify-resume/state.json @@ -0,0 +1,16 @@ +{ + "repo": "example/petclinic", + "labels_available": ["bug"], + "issues": { + "11": { + "number": 11, + "title": "Improve something", + "body": "Make it better.\n", + "labels": [], + "url": "https://github.com/example/petclinic/issues/11", + "assignees": [], + "projectItems": [] + } + }, + "pull_requests": [] +} diff --git a/skills/issue-triage/evals/fixtures/draft-pr/issue.md b/skills/issue-triage/evals/fixtures/draft-pr/issue.md new file mode 100644 index 0000000..ac4594c --- /dev/null +++ b/skills/issue-triage/evals/fixtures/draft-pr/issue.md @@ -0,0 +1,13 @@ +# example/petclinic#15 + +**Title:** Draft PR only + +**Labels:** (none) + +**URL:** https://github.com/example/petclinic/issues/15 + +**Preflight:** ok — linked PR #7 is draft (`isDraft=true`); drafts do not block + +## Body (current) + +Draft should not block. diff --git a/skills/issue-triage/evals/fixtures/draft-pr/state.json b/skills/issue-triage/evals/fixtures/draft-pr/state.json new file mode 100644 index 0000000..b2339a7 --- /dev/null +++ b/skills/issue-triage/evals/fixtures/draft-pr/state.json @@ -0,0 +1,23 @@ +{ + "repo": "example/petclinic", + "labels_available": [], + "issues": { + "15": { + "number": 15, + "title": "Draft PR only", + "body": "Draft should not block.\n", + "labels": [], + "url": "https://github.com/example/petclinic/issues/15", + "assignees": [], + "projectItems": [] + } + }, + "pull_requests": [ + { + "number": 7, + "state": "OPEN", + "isDraft": true, + "closingIssuesReferences": [{"number": 15}] + } + ] +} diff --git a/skills/issue-triage/evals/fixtures/happy/issue.md b/skills/issue-triage/evals/fixtures/happy/issue.md new file mode 100644 index 0000000..8cae529 --- /dev/null +++ b/skills/issue-triage/evals/fixtures/happy/issue.md @@ -0,0 +1,15 @@ +# example/petclinic#10 + +**Title:** Add language selector to header + +**Labels:** bug + +**URL:** https://github.com/example/petclinic/issues/10 + +**Preflight:** ok (no `in-progress`, no open non-draft linked PR) + +## Body (current — snapshot for Original Ask) + +## Summary + +Add a language dropdown. diff --git a/skills/issue-triage/evals/fixtures/happy/sealed-body.md b/skills/issue-triage/evals/fixtures/happy/sealed-body.md new file mode 100644 index 0000000..e79d012 --- /dev/null +++ b/skills/issue-triage/evals/fixtures/happy/sealed-body.md @@ -0,0 +1,47 @@ +## Goals + +Add a header language selector that persists locale via cookie. + +## Non-goals + +- Translating all clinic content in this change +- Per-user account locale preferences + +## Functional Requirements + +1. Header dropdown lists supported locales and updates UI text after selection. +2. Locale persists via cookie for subsequent requests. +3. `redirect=` after locale change is same-origin only (no open redirect). + +## Constraints + +- Reuse existing `WebConfiguration` / message-bundle wiring when present. +- Cookie is host-only; sensible max-age documented in Assumptions. + +## Assumptions + +- Cookie max-age: 1 year. +- Supported locales: en, es (extendable via bundles). + +## Size + +S — One vertical slice: header control → locale endpoint → cookie → UI text; bounded file set. + +## User Acceptance Criteria + +- [ ] User can switch language from the header on desktop and mobile nav. +- [ ] Refresh keeps the chosen language. +- [ ] Malicious external `redirect=` is rejected or ignored. + +## Testable / Verifiable + +1. Playwright covers language switch + persistence. +2. Unit/controller test covers open-redirect rejection. + +--- + +## Original Ask + +### Summary + +Add a language dropdown. diff --git a/skills/issue-triage/evals/fixtures/happy/state.json b/skills/issue-triage/evals/fixtures/happy/state.json new file mode 100644 index 0000000..be1a08a --- /dev/null +++ b/skills/issue-triage/evals/fixtures/happy/state.json @@ -0,0 +1,16 @@ +{ + "repo": "example/petclinic", + "labels_available": ["bug"], + "issues": { + "10": { + "number": 10, + "title": "Add language selector to header", + "body": "## Summary\n\nAdd a language dropdown.\n", + "labels": [{"name": "bug"}], + "url": "https://github.com/example/petclinic/issues/10", + "assignees": [], + "projectItems": [] + } + }, + "pull_requests": [] +} diff --git a/skills/issue-triage/evals/fixtures/in-progress/issue.md b/skills/issue-triage/evals/fixtures/in-progress/issue.md new file mode 100644 index 0000000..d6a9b2e --- /dev/null +++ b/skills/issue-triage/evals/fixtures/in-progress/issue.md @@ -0,0 +1,13 @@ +# example/petclinic#12 + +**Title:** Already being worked + +**Labels:** in-progress + +**URL:** https://github.com/example/petclinic/issues/12 + +**Preflight:** blocked — issue #12 has label `in-progress` + +## Body (current) + +In flight. diff --git a/skills/issue-triage/evals/fixtures/in-progress/state.json b/skills/issue-triage/evals/fixtures/in-progress/state.json new file mode 100644 index 0000000..a23131c --- /dev/null +++ b/skills/issue-triage/evals/fixtures/in-progress/state.json @@ -0,0 +1,16 @@ +{ + "repo": "example/petclinic", + "labels_available": ["in-progress"], + "issues": { + "12": { + "number": 12, + "title": "Already being worked", + "body": "In flight.\n", + "labels": [{"name": "in-progress"}], + "url": "https://github.com/example/petclinic/issues/12", + "assignees": [], + "projectItems": [] + } + }, + "pull_requests": [] +} diff --git a/skills/issue-triage/evals/fixtures/open-pr/issue.md b/skills/issue-triage/evals/fixtures/open-pr/issue.md new file mode 100644 index 0000000..47dccf5 --- /dev/null +++ b/skills/issue-triage/evals/fixtures/open-pr/issue.md @@ -0,0 +1,13 @@ +# example/petclinic#13 + +**Title:** Has an open PR + +**Labels:** (none) + +**URL:** https://github.com/example/petclinic/issues/13 + +**Preflight:** blocked — open non-draft PR #42 linked to #13 + +## Body (current) + +Someone already opened a PR. diff --git a/skills/issue-triage/evals/fixtures/open-pr/state.json b/skills/issue-triage/evals/fixtures/open-pr/state.json new file mode 100644 index 0000000..fde4edb --- /dev/null +++ b/skills/issue-triage/evals/fixtures/open-pr/state.json @@ -0,0 +1,23 @@ +{ + "repo": "example/petclinic", + "labels_available": [], + "issues": { + "13": { + "number": 13, + "title": "Has an open PR", + "body": "Someone already opened a PR.\n", + "labels": [], + "url": "https://github.com/example/petclinic/issues/13", + "assignees": [], + "projectItems": [] + } + }, + "pull_requests": [ + { + "number": 42, + "state": "OPEN", + "isDraft": false, + "closingIssuesReferences": [{"number": 13}] + } + ] +} diff --git a/skills/issue-triage/evals/fixtures/scope-discipline/issue.md b/skills/issue-triage/evals/fixtures/scope-discipline/issue.md new file mode 100644 index 0000000..b41b085 --- /dev/null +++ b/skills/issue-triage/evals/fixtures/scope-discipline/issue.md @@ -0,0 +1,15 @@ +# example/petclinic#14 + +**Title:** Named issue only + +**Labels:** (none) + +**URL:** https://github.com/example/petclinic/issues/14 + +**Preflight:** ok + +**Also present in the repo (do not touch):** example/petclinic#99 — Distractor issue + +## Body (current) + +Do not touch other issues. diff --git a/skills/issue-triage/evals/fixtures/scope-discipline/state.json b/skills/issue-triage/evals/fixtures/scope-discipline/state.json new file mode 100644 index 0000000..abe617e --- /dev/null +++ b/skills/issue-triage/evals/fixtures/scope-discipline/state.json @@ -0,0 +1,25 @@ +{ + "repo": "example/petclinic", + "labels_available": [], + "issues": { + "14": { + "number": 14, + "title": "Named issue only", + "body": "Do not touch other issues.\n", + "labels": [], + "url": "https://github.com/example/petclinic/issues/14", + "assignees": [], + "projectItems": [] + }, + "99": { + "number": 99, + "title": "Distractor issue", + "body": "Must not be mutated.\n", + "labels": [], + "url": "https://github.com/example/petclinic/issues/99", + "assignees": [], + "projectItems": [] + } + }, + "pull_requests": [] +} diff --git a/skills/issue-triage/evals/fixtures/underspecified/issue.md b/skills/issue-triage/evals/fixtures/underspecified/issue.md new file mode 100644 index 0000000..970376f --- /dev/null +++ b/skills/issue-triage/evals/fixtures/underspecified/issue.md @@ -0,0 +1,13 @@ +# example/petclinic#11 + +**Title:** Improve something + +**Labels:** (none) + +**URL:** https://github.com/example/petclinic/issues/11 + +**Preflight:** ok + +## Body (current) + +Make it better. diff --git a/skills/issue-triage/evals/fixtures/underspecified/state.json b/skills/issue-triage/evals/fixtures/underspecified/state.json new file mode 100644 index 0000000..3fdf280 --- /dev/null +++ b/skills/issue-triage/evals/fixtures/underspecified/state.json @@ -0,0 +1,16 @@ +{ + "repo": "example/petclinic", + "labels_available": ["bug"], + "issues": { + "11": { + "number": 11, + "title": "Improve something", + "body": "Make it better.\n", + "labels": [], + "url": "https://github.com/example/petclinic/issues/11", + "assignees": [], + "projectItems": [] + } + }, + "pull_requests": [] +} diff --git a/skills/issue-triage/references/clarify-log-template.md b/skills/issue-triage/references/clarify-log-template.md new file mode 100644 index 0000000..1c5bd64 --- /dev/null +++ b/skills/issue-triage/references/clarify-log-template.md @@ -0,0 +1,24 @@ +# Clarify log + +Path: `.issue-triage/__--clarify.md` + +Per-question shape (mandatory): +[clarify-question-template.md](clarify-question-template.md). + +```markdown +# Issue triage clarify — # +round: 1 / 4 + +## Round 1 +### Q1 — +Why: +- **A)**