qa/checklist: result-issue-per-run + checklist-author/checklist-test skills + list-filename selector (#7304)

baozhoutao · claude · web-flow · commit d475c703967a · 2026-08-10T09:10:45.000Z
* qa/checklist: result issue per run — text-only, repro rules not screenshots Encode the maintainer decision for how a checklist run reports results: - Every completed run (pass or fail alike) files exactly ONE GitHub issue as its durable record, labeled qa-run; add bug/regression when any clause failed so a defect triages straight from the run issue. - The issue is pure text. Screenshots are live judgment aids only — taken to reach a verdict, discarded with the run environment, never uploaded, linked, or committed. This keeps results out of the repo tree AND out of the issue. - Every fail carries a reproduction rule (ordered steps / API calls / selector path + expected-vs-actual) instead of a picture, so a human or fresh agent can re-hit it without any captured image. Updates checklist-run SKILL.md §4 + §2, RUNNER.md (rule 2 + Run records), README.md (release-sweep results), and runs/README.md (format contract). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 * qa/checklist: rename skills to checklist-author / checklist-test pair Give the two checklist skills a shared 'checklist-' namespace so they read as one authoring/execution pair: - coverage-sweep → checklist-author (finds gaps, AUTHORS items) - checklist-run → checklist-test (runs items, produces the report issue) Natural-language triggers ('coverage sweep', '测一下 <功能>') are unchanged; only the skill ids, H1 titles, cross-references and one script comment move. internal: true markers untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 * qa/checklist: resolver accepts a list-directory filename with no prefix checklist-select now takes a prefix-less selector that looks like a path: - `records-forms.json` (or a full `…/areas/records-forms.json` path) → that area, matched by basename — paste a list-directory filename, run the whole area. - a bare path with a `/` or a code extension → resolves as `file:<path>` (test whatever covers that source file), no prefix to remember. Bare ids and bare area names are unchanged. Adds 5 self-test cases (17 total) and documents the conveniences in the header and the checklist-test §0 table. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43 --------- Co-authored-by: Claude <noreply@anthropic.com>
diff --git a/.claude/skills/checklist-author/SKILL.md b/.claude/skills/checklist-author/SKILL.md
@@ -1,5 +1,5 @@
 ---
-name: coverage-sweep
+name: checklist-author
 description: >
   Re-audit the platform test checklist (docs/qa/platform-checklist/) for coverage
   gaps and author the missing items — the five-angle capability sweep. Use whenever
@@ -16,7 +16,7 @@ metadata:
   internal: true
 ---
 
-# Coverage sweep — keep the platform test checklist honest
+# Checklist author — the coverage sweep that keeps the checklist honest
 
 The canonical method lives in **`docs/qa/platform-checklist/SWEEP.md`** — read it
 first and follow it; this skill is the trigger and the orchestration contract, not a
diff --git a/.claude/skills/checklist-test/SKILL.md b/.claude/skills/checklist-test/SKILL.md
@@ -1,13 +1,13 @@
 ---
-name: checklist-run
+name: checklist-test
 description: >
   Execute the platform test checklist (docs/qa/platform-checklist/) against a real
   running app and produce a run record. Use whenever the maintainer says "测一下
   <功能>", "跑这个测试项", "run the checklist for <area/item>", "test this feature
   file", "验证 <功能点>", or points at a framework source file and asks whether it
   still works. Takes a SELECTOR (item id · area · capability kind · priority · a
   release · or a source-file path) and drives every matched item through its steps
-  following RUNNER.md. The companion to `coverage-sweep` (which AUTHORS items); this
+  following RUNNER.md. The companion to `checklist-author` (which AUTHORS items); this
   one RUNS them. NOT a customer-published skill — internal agent tooling (lives in
   .claude/, never in the published `skills/` dir).
 metadata:
@@ -17,7 +17,7 @@ metadata:
   internal: true
 ---
 
-# Checklist run — execute selected items against a live app
+# Checklist test — execute selected items against a live app
 
 You resolve a **selector** to a set of checklist items, boot the app in isolation, drive
 each item's steps in the browser / over the API, and emit a **run record**. The method
@@ -44,11 +44,12 @@ Selectors (one per run):
 |---|---|
 | `platform-core.console-login` (bare id) | that one item |
 | `area:records-forms` (or bare `records-forms`) | every item in the area |
+| **`records-forms.json`** (or the full `…/areas/records-forms.json` path) | **every item in that area — a list-directory filename works with no prefix** |
 | `capability:hook` | items mapped to a metadata kind in `coverage.json` |
 | `priority:P0` | the standing smoke |
 | `surface:api` | every API-surface item (cheap — no browser build needed) |
 | `since:v17` | everything introduced in a release (the release-sweep filter) |
-| **`file:packages/plugins/plugin-approvals/src/approval-service.ts`** | **items whose `source[]` cites that file — "test whatever covers this file"** |
+| **`file:packages/plugins/plugin-approvals/src/approval-service.ts`** | **items whose `source[]` cites that file — "test whatever covers this file"** (a bare path with a `/` or a code extension auto-resolves as `file:` too) |
 | `all` | the whole checklist |
 
 `--json` gives the runnable list (id · priority · surface · revision). **Blocked items are
@@ -83,7 +84,8 @@ Build once, up front, for the whole run.
 - Execute each item's `steps` faithfully; judge each `acceptance` clause and each
   `negative` against its declared `oracle`, capturing the `evidence` the clause names.
   **Server truth outranks pixels; DOM only after a screenshot confirms render; a `fail`
-  needs reproduction ×2 + the automation self-check + a filed issue** (RUNNER §rules).
+  needs reproduction ×2 + the automation self-check + a reproduction rule in the run
+  issue** (RUNNER §rules).
 
 ## 3. When the run teaches you something about the ITEM
 
@@ -94,21 +96,42 @@ a `history` entry, keep `node scripts/check-platform-checklist.mjs` green, and l
 task branch. Product defects found while running go to `FOLLOW-UPS.md` (or a filed issue)
 as expected-fail probes — never tick a clause green over a real defect.
 
-## 4. The run record — results do NOT go in the repo
-
-Write one JSON per run in the shape RUNNER.md defines (env with framework sha +
-`.objectui-sha` + port + db; per-clause verdicts each naming its evidence; derived item
-verdict; issues). **`runs/` is git-ignored** — the record and its screenshots stay in the
-executing environment / the tracking issue / an external QA store, never committed. The
-committed source is the checklist under `areas/`; a run is a dated assertion about one
-build and belongs with that build's artifacts. Report the per-clause verdict table + the
-evidence paths + the env-setup-vs-test time split back to the maintainer.
+## 4. The result issue — one GitHub issue per run, text only
+
+Every completed run — **pass or fail alike** — files exactly one GitHub issue as its
+durable record. Nothing about a run enters the repo tree: not the JSON, not screenshots.
+The JSON run record (RUNNER.md shape) is scratch in the executing environment; `runs/`
+stays git-ignored. The issue is the report.
+
+**The issue is pure text — no images, ever.** Screenshots exist only to let you and your
+subagents reach a verdict *live*; they are a judgment aid, discarded with the run
+environment. The durable report needs the **reproduction rule, not the picture**.
+
+File it with `issue_write` (github MCP):
+
+- **Title** — `QA run · <selector> · <framework-sha[:8]> · <date>`
+- **Labels** — `qa-run` always; add `bug` (and `regression` for a P0/P1) whenever any
+  clause failed, so a real defect is triageable straight from the run issue without a
+  second one.
+- **Body**, in this order:
+  - **Env fingerprint** — framework sha, `.objectui-sha`, port, db, seed, timestamp.
+  - **Scope** — the selector + the `revision` each item ran against.
+  - **Per-clause verdict table** — item · clause · verdict · one line of **text** oracle
+    evidence (the API/network/build/test result — server truth, never a pixel).
+  - **For every `fail`, a reproduction rule** — the exact ordered steps / the API calls
+    (method · path · body) / the ref-targeted selector path to re-hit it on a fresh boot,
+    plus expected-vs-actual from the oracle. Enough for a human or a fresh agent to
+    reproduce it with no screenshot from you.
+  - Derived item verdicts + any fixture-gap list.
+
+Report the same per-clause table + the env-setup-vs-test time split back to the maintainer
+in chat, and link the filed issue.
 
 ## Guardrails
 
 - **Don't fake coverage.** Missing fixture → `blocked(fixture)` with the reason; unbuilt
   console → build it or record `blocked(environment)`; a half-proven item is `partial`,
   not `pass`. A blocked verdict WITH evidence is a successful run; a faked pass is not.
 - **Don't run blocked items as if runnable** — the resolver hides them for this reason.
-- **One selector, one run record.** For a release sweep, run `since:vN` and `priority:P0`
-  as separate records rather than smearing them together.
+- **One selector, one run, one issue.** For a release sweep, run `since:vN` and
+  `priority:P0` as separate runs → separate issues, rather than smearing them together.
diff --git a/docs/qa/platform-checklist/README.md b/docs/qa/platform-checklist/README.md
@@ -11,13 +11,13 @@ acceptance oracle and captured evidence.
 Validated by `pnpm check:platform-checklist` (`scripts/check-platform-checklist.mjs`) —
 a zero-dependency structural + coverage check. **By maintainer decision it runs on a
 periodic MANUAL cadence, not in CI**: run it before a release, after a large platform
-surface lands, or alongside a `coverage-sweep` / `checklist-run`. It is a QA ledger, not
+surface lands, or alongside a `checklist-author` / `checklist-test`. It is a QA ledger, not
 a per-PR code gate, so an unrelated PR is never blocked by checklist drift. Execution
 protocol for agents: [RUNNER.md](./RUNNER.md). Run records: [runs/](./runs/README.md).
 
 **Two internal skills drive this ledger** (`.claude/skills/`, never published):
-`coverage-sweep` **authors** items (find gaps → write them, per
-[SWEEP.md](./SWEEP.md)); `checklist-run` **executes** them (pick items by selector →
+`checklist-author` **authors** items (find gaps → write them, per
+[SWEEP.md](./SWEEP.md)); `checklist-test` **executes** them (pick items by selector →
 drive them → emit a run record, per [RUNNER.md](./RUNNER.md)). The runner resolves what
 to test with `scripts/checklist-select.mjs <selector>` — an item id, an `area:`, a
 `capability:`, a `priority:`, a `since:vN` release, or a **`file:<path>`** that maps a
@@ -179,7 +179,7 @@ checklist drift. It runs on a **manual / periodic cadence** instead. Run
 - **after a large platform surface lands** — a new metadata kind, a new enum, a new area;
 - **whenever you touch the checklist** — the structural + coverage check catches a
   dangling id or a forgotten `revision` bump in your own edit;
-- **alongside a `coverage-sweep`** (find gaps) **or `checklist-run`** (execute items).
+- **alongside a `checklist-author`** (find gaps) **or `checklist-test`** (execute items).
 
 The trade-off of staying out of CI: a new capability kind or enum value that lands on
 `main` between runs is caught at the **next** manual run, not the moment it merged. The
@@ -206,11 +206,12 @@ ever matters more than PR independence, re-adding the one-line CI step
 A release no longer gets a hand-written checklist. The sweep for `vN` is a **filter
 over this ledger**: `since == vN` (the new capabilities) ∪ all `P0` (the standing
 smoke) ∪ any item whose `source` cites a PR in the release. The tracking issue for the
-sweep links here and hosts discussion; results live as a run record kept OUT of the
-repo (in the CI artifact / tracking issue / QA store — `runs/` is git-ignored), plus
-findings filed as issues, one per failure. Item text, fixtures learned, and new
-traps discovered flow **back into the ledger** as revisions — that is the accumulation
-the one-off checklists never had.
+sweep links here and hosts discussion; results stay OUT of the repo — every run files
+one `qa-run` GitHub issue as its record (text only: the verdict table + a reproduction
+rule per failure, never screenshots; `runs/` is git-ignored), and a run that finds a real
+regression carries the `bug` label so it triages straight from that issue. Item text,
+fixtures learned, and new traps discovered flow **back into the ledger** as revisions —
+that is the accumulation the one-off checklists never had.
 
 ## Relationship to what already exists
 
diff --git a/docs/qa/platform-checklist/RUNNER.md b/docs/qa/platform-checklist/RUNNER.md
@@ -50,8 +50,11 @@ test-run output the clause's `evidence` field names.
    - check the `traps` field and rule each listed trap out;
    - for console UI failures, confirm against current objectui source or a fresh build
      — the vendored `/_console` bundle may be stale (skill §2);
-   - then file the issue and cite it in the run record. A `fail` without a filed issue
-     is not a completed verdict.
+   - then capture the **reproduction rule** — ordered steps / API calls (method · path ·
+     body) / the ref-targeted selector path + expected-vs-actual — into the run's result
+     issue, which is labeled `bug`. A `fail` with no reproduction rule in its issue is not
+     a completed verdict. (The screenshot that convinced you is a live judgment aid, not
+     report content — describe what it showed in one line; never attach it.)
 3. **Classify blockers honestly.** Missing seed/persona/fixture → `blocked(fixture)`,
    and *record the gap on the item* (`fixtures.knownGaps` or `blocked`) so the next
    sweep doesn't rediscover it. A defect in the fixture itself (seed silently failing,
@@ -88,13 +91,20 @@ test-run output the clause's `evidence` field names.
 | `wrong-panel` | feature looks missing on a sibling surface | item's `steps` name the exact surface; check it |
 | `wrong-persona` | admin privileges mask a guard | run guard checks as the non-privileged persona |
 
-## Run records
+## Run records — the GitHub issue is the report
 
-One JSON per executed sweep, written to `runs/YYYY-MM-DD-<slug>.json`. **Results are
-NOT committed** — a run record is output about one build, not source; `runs/` is
-git-ignored except its README (the format contract). Keep the record and its evidence
-in the executing environment (CI artifact, runner workspace, the sweep's tracking
-issue, or an external QA store). Shape:
+Every completed run — **pass or fail alike** — files **one GitHub issue** as its durable
+record, labeled `qa-run` (plus `bug` when any clause failed). **Nothing lands in the
+repo** — not the JSON, not screenshots; `runs/` is git-ignored except its README.
+
+**The issue is text only.** Screenshots and DOM dumps are oracles you consult *live* to
+reach a verdict — never report artifacts. What the report carries for a defect is the
+**reproduction rule**: ordered steps / API calls (method · path · body) / the
+ref-targeted selector path + expected-vs-actual from the oracle, enough to re-hit it on a
+fresh boot with no picture. A clause whose oracle was a screenshot is recorded as a
+one-line text description of what it showed, not a link.
+
+The in-environment JSON scratch (RUNNER shape, never committed):
 
 ```jsonc
 {
@@ -114,7 +124,7 @@ issue, or an external QA store). Shape:
       "revision": 1,                     // ← the revision this verdict is valid for
       "verdict": "pass",
       "clauses": [
-        { "clause": 0, "verdict": "pass", "evidence": "…what was captured, where…" }
+        { "clause": 0, "verdict": "pass", "evidence": "…text: what the oracle returned — no image links…" }
       ],
       "issues": [],                      // filed failures / fixture gaps
       "notes": "…"
@@ -123,7 +133,8 @@ issue, or an external QA store). Shape:
 }
 ```
 
-A run summary for humans may additionally go to the sweep's tracking issue or an
-external QA store — but none of it lands in the repo. The durable, version-controlled
-truth is the checklist under `areas/`; a run is a dated assertion about a build that
-belongs wherever that build's other artifacts live.
+The issue body is: env fingerprint · scope (selector + per-item `revision`) · the
+per-clause verdict table (text oracle evidence) · a reproduction rule per `fail` ·
+derived item verdicts + fixture gaps. The durable, version-controlled truth is still the
+checklist under `areas/`; a run is a dated assertion about one build, and it lives in its
+issue, not the tree.
diff --git a/docs/qa/platform-checklist/runs/README.md b/docs/qa/platform-checklist/runs/README.md
@@ -8,22 +8,25 @@ land in the repo (`.gitignore` here tracks only this README). The durable source
 the checklist itself under `../areas/`; a run is a snapshot that goes stale the moment
 the build moves.
 
-**Where results go instead — the canonical home is one tracking issue per release
-sweep** (a `[sweep] vN release test sweep` issue, successor to the #3358 model):
+**Where results go instead — every completed run files one `qa-run` GitHub issue** as its
+durable, **text-only** report (pass or fail alike):
 
-- the issue **body** hosts the human-readable summary — the per-item verdict table
-  (pass / partial / fail / blocked) and the filter that selected them (`since:vN` ∪ all
-  `P0` ∪ items whose `source` cites a release PR);
-- the machine **run-record JSON(s)** (shape below) attach to that issue — pasted in a
-  comment or linked as a CI artifact from the sweep job;
-- every `fail` becomes its **own linked issue** (RUNNER.md makes a filed issue part of a
-  completed `fail` verdict), cross-referenced from the sweep issue.
+- the issue **body** hosts the per-clause verdict table (pass / partial / fail / blocked)
+  and the scope — the selector that chose the items + the `revision` each ran against;
+- **every `fail` carries a reproduction rule** in that same issue — ordered steps / API
+  calls (method · path · body) / the ref-targeted selector path + expected-vs-actual —
+  and the issue gains the `bug` label (a real regression may additionally get its own
+  linked issue) so it triages straight from the run;
+- **screenshots are never part of the report** — they are live judgment aids that die
+  with the run environment, described in one line of text, never attached or linked.
 
-A raw CI artifact or an external QA store is a fine substitute where one exists, but the
-per-release tracking issue is the default so a sweep is never lost. What NEVER lands in
-the repo is the record itself — only the durable ledger under `../areas/` accumulates
-here, through each item's `revision`/`history`. A verdict is interpretable only next to
-the `revision` it names, so the run record stays with its build's artifacts, not in git.
+For a release sweep, the per-run issues roll up under the sweep's `[sweep] vN release test
+sweep` tracking issue (successor to the #3358 model) so a sweep is never lost. The machine
+run-record JSON (shape below) is optional scratch in the executing environment (the
+runner's workspace or a CI artifact) and is git-ignored — **do not commit it.** What NEVER
+lands in the repo is the record itself; only the durable ledger under `../areas/`
+accumulates here, through each item's `revision`/`history`, and a verdict is interpretable
+only next to the `revision` it names.
 
 ## Record shape (write to `YYYY-MM-DD-<slug>.json`, kept out of git)
 
@@ -44,7 +47,7 @@ are only meaningful next to the item `revision` they ran against.
       "id": "approvals.per-group-signoff",
       "revision": 1,                     // ← the revision this verdict is valid for
       "verdict": "pass",                 // derived: pass | partial | fail | blocked | not-run
-      "clauses": [ { "clause": 0, "verdict": "pass", "evidence": "…what was captured, where…" } ],
+      "clauses": [ { "clause": 0, "verdict": "pass", "evidence": "…text: what the oracle returned — no image links…" } ],
       "issues": [],
       "notes": "…"
     }
diff --git a/scripts/checklist-select.mjs b/scripts/checklist-select.mjs
