From 12592aec00027e4e7a2efd66b242922c9ae2dc68 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Mon, 27 Jul 2026 14:23:21 -0400 Subject: [PATCH 01/38] Add testplan artifact to design decompose phase Add a testplan artifact (07-testplan.md) to the design workflow's decompose phase. The testplan provides behavioral test cases anchored to PRD requirement IDs (e.g., TC-FR1-01), structured for independent QA review and eventual ALM export (e.g., Polarion). Key changes: - New Step 7 in decompose generates 07-testplan.md with test cases grouped by requirement, each with ID, preconditions, scenario steps, expected result, and priority - Stories gain a Test Case References section linking to testplan entries - Coverage matrix gains a Test Cases column completing the traceability chain: requirement -> story -> test cases - Artifact renumbering: PR description 07->08, review responses 08->09 - Decomposition review protocol adds Testplan Quality criterion - Revise, publish, and respond phases propagate testplan to docs repo - Sync phase includes Test Case References in Jira story descriptions - Implement and e2e workflows reference testplan as optional context Design: 0.4.0 -> 0.5.0, implement: 0.4.0 -> 0.5.0, e2e: 0.3.1 -> 0.4.0 Assisted-by: Claude Opus 4.6 (1M) --- design/README.md | 15 +-- design/SKILL.md | 2 +- design/decomposition-review.md | 32 ++++-- design/skills/controller.md | 5 +- design/skills/decompose.md | 184 ++++++++++++++++++++++++++++----- design/skills/ingest.md | 3 +- design/skills/publish.md | 32 +++++- design/skills/respond.md | 16 ++- design/skills/revise.md | 33 ++++++ design/skills/sync.md | 7 ++ e2e/SKILL.md | 2 +- e2e/skills/ingest.md | 11 +- implement/SKILL.md | 2 +- implement/skills/ingest.md | 10 +- 14 files changed, 297 insertions(+), 57 deletions(-) diff --git a/design/README.md b/design/README.md index b90d166..700f0ae 100644 --- a/design/README.md +++ b/design/README.md @@ -35,10 +35,10 @@ graph TD | Ingest | `/ingest` | Read PRD, explore codebase | `01-context.md` | | Research | `/research` | Investigate problem space, solutions, standards | `02-research.md` | | Design | `/draft` | Draft design document | `03-design.md` | -| Decompose | `/decompose` | Break into epics and stories | `04-epics.md`, `05-stories/`, `06-coverage.md` | +| Decompose | `/decompose` | Break into epics and stories | `04-epics.md`, `05-stories/`, `06-coverage.md`, `07-testplan.md` | | Revise | `/revise` | Incorporate feedback | Updated design and/or stories | -| Publish | `/publish` | Post design doc as GitHub PR | `07-pr-description.md` | -| Respond | `/respond` | Address reviewer comments | `08-review-responses.md` | +| Publish | `/publish` | Post design doc as GitHub PR | `08-pr-description.md` | +| Respond | `/respond` | Address reviewer comments | `09-review-responses.md` | | Sync | `/sync` | Sync Jira epics and stories | `sync-manifest.json` | ## Typical Flow @@ -63,7 +63,7 @@ graph TD /decompose → breaks design into epics and stories → validates coverage against PRD requirements - → writes 04-epics.md, 05-stories/ (epics + stories), 06-coverage.md + → writes 04-epics.md, 05-stories/ (epics + stories), 06-coverage.md, 07-testplan.md /revise → user reviews, requests changes to design and/or decomposition @@ -73,7 +73,7 @@ graph TD /publish → commits design document to feature branch in docs repo → creates draft GitHub PR - → writes 07-pr-description.md + → writes 08-pr-description.md /respond → fetches PR review comments @@ -109,8 +109,9 @@ Jira issue: epic-2/ story-01-deploy-config.md (→ Jira Story) 06-coverage.md - 07-pr-description.md - 08-review-responses.md + 07-testplan.md + 08-pr-description.md + 09-review-responses.md publish-metadata.json provenance.json sync-manifest.json diff --git a/design/SKILL.md b/design/SKILL.md index 9472adf..f3cbdde 100644 --- a/design/SKILL.md +++ b/design/SKILL.md @@ -1,6 +1,6 @@ --- name: design -version: 0.4.2 +version: 0.5.0 description: >- Design-and-decompose workflow that takes a PRD, researches the problem space, drafts a technical design document, decomposes work into Jira-ready epics and diff --git a/design/decomposition-review.md b/design/decomposition-review.md index e54415c..315bd49 100644 --- a/design/decomposition-review.md +++ b/design/decomposition-review.md @@ -7,8 +7,8 @@ whether the reviewer is a subagent or the decomposer performing inline self-review. The reviewer receives the PRD and the decomposition artifacts (epics, -stories, coverage matrix) but intentionally does **not** receive the -design document. This is the independence property: the reviewer +stories, coverage matrix, testplan) but intentionally does **not** +receive the design document. This is the independence property: the reviewer evaluates what the artifacts actually say, not what the decomposer intended. If a story's acceptance criteria are unclear without cross-referencing the design document, that is a real finding — the @@ -99,14 +99,26 @@ impact: story fall back to "manual validation" or "document procedure for QE" where an automated test could be written? -5. **Integration Stability** — Would implementing all stories in +5. **Testplan Quality** — Does the testplan (`07-testplan.md`) cover + every FR and NFR that has implementing stories? Are test case IDs + correctly anchored to PRD requirement IDs (`TC-FR1-01`, not arbitrary + sequences)? Does each test case have concrete preconditions, numbered + scenario steps, and an observable expected result — or are they vague + restatements of acceptance criteria? Are priorities assigned + consistently (critical for core workflows, not everything marked + high)? Do the `Test Case References` in story files match the + testplan's Story column? Does the coverage matrix's Test Cases column + match the testplan? Are there requirements with stories but no test + cases (testplan gaps)? + +6. **Integration Stability** — Would implementing all stories in dependency order produce a working feature end-to-end? Does any integration work fall between stories — work that is needed but not captured in any story? Are story dependencies documented and complete? Does each story leave the system in a stable state (all tests passing after merge)? -6. **Documentation** — Do `[DOCS]` stories use the `[DOCS]` template +7. **Documentation** — Do `[DOCS]` stories use the `[DOCS]` template (Documentation Scope and Documentation Inputs instead of Implementation Guidance and Testing Approach)? Does the Documentation Scope describe what the reader needs to understand without prescribing @@ -123,7 +135,8 @@ Each finding must include: - **File:** artifact file path (e.g., `05-stories/epic-1/story-02-add-validation.md`) - **Section:** section within the artifact (e.g., "Acceptance Criteria", - "Dependencies", "Testing Approach", "Documentation Scope") + "Dependencies", "Testing Approach", "Test Case References", + "Documentation Scope") - **Severity:** CRITICAL | HIGH | MEDIUM | LOW - **Category:** one of the evaluation criteria above - **Issue:** what the problem is @@ -132,15 +145,16 @@ Each finding must include: Findings that cannot cite a specific file and section in the actual artifacts must be discarded — they indicate hallucinated references. Coverage matrix findings should cite `06-coverage.md` and the specific -row or gap. +row or gap. Testplan findings should cite `07-testplan.md` and the +specific requirement section or test case row. ## Severity Definitions | Severity | Meaning | Action | |----------|---------|--------| -| CRITICAL | Would cause incorrect implementation — missing requirements, contradictory acceptance criteria, broken dependency chain, scope reduction | Must fix before presenting | -| HIGH | Would cause rework — story too large to review in a single PR, missing testing commitment, integration gap between stories, `[QE]` story duplicating `[DEV]` test scope | Should fix before presenting | -| MEDIUM | Would reduce clarity — vague acceptance criteria, imprecise implementation guidance or documentation scope, sizing concerns, minor coverage matrix inconsistencies | Fix if it adds value | +| CRITICAL | Would cause incorrect implementation — missing requirements, contradictory acceptance criteria, broken dependency chain, scope reduction, missing test cases for requirements that have implementing stories | Must fix before presenting | +| HIGH | Would cause rework — story too large to review in a single PR, missing testing commitment, integration gap between stories, `[QE]` story duplicating `[DEV]` test scope, test case with vague preconditions or non-observable expected result | Should fix before presenting | +| MEDIUM | Would reduce clarity — vague acceptance criteria, imprecise implementation guidance or documentation scope, sizing concerns, minor coverage matrix inconsistencies, test case priority inconsistency, minor testplan-to-coverage matrix mismatch | Fix if it adds value | | LOW | Structural improvements — naming consistency, ordering suggestions, documentation clarity | Fix only if clearly valuable | ## Validation Rules diff --git a/design/skills/controller.md b/design/skills/controller.md index e341443..484141c 100644 --- a/design/skills/controller.md +++ b/design/skills/controller.md @@ -58,9 +58,10 @@ the source repo (this directory should be gitignored in the source repo): | Epic files | `05-stories/epic-{N}-{slug}.md` | `/decompose`, `/revise` | | Story files | `05-stories/epic-{N}/story-{NN}-{slug}.md` | `/decompose`, `/revise` | | Coverage matrix | `06-coverage.md` | `/decompose`, `/revise` | -| PR description | `07-pr-description.md` | `/publish` | +| Testplan | `07-testplan.md` | `/decompose`, `/revise` | +| PR description | `08-pr-description.md` | `/publish` | | Publish metadata | `publish-metadata.json` | `/publish` | -| Review responses | `08-review-responses.md` | `/respond` | +| Review responses | `09-review-responses.md` | `/respond` | | Jira sync manifest | `sync-manifest.json` | `/sync` | ### Docs repo configuration diff --git a/design/skills/decompose.md b/design/skills/decompose.md index 6c8eec2..eee6fe5 100644 --- a/design/skills/decompose.md +++ b/design/skills/decompose.md @@ -1,6 +1,6 @@ --- name: decompose -description: Break the design into Jira-ready epics and stories with a coverage matrix. +description: Break the design into Jira-ready epics and stories with a testplan and coverage matrix. --- # Decompose Skill @@ -110,7 +110,7 @@ Write `.artifacts/design/{issue-key}/04-epics.md`: {PRD Requirements column: list the primary requirements for quick reference. If an epic maps to more than ~8 requirements, list the most significant and add "See coverage matrix for full mapping." The coverage matrix - (Step 7) is the authoritative source for requirement-to-story traceability.} + (Step 8) is the authoritative source for requirement-to-story traceability.} ## Dependency Order @@ -212,6 +212,13 @@ Write each story to Epic: Epic {N} — {epic title} PRD Requirements: {FR-1, NFR-1} Design section: {§4.3 API Changes, or specific subsection} + +## Test Case References + +Verified by: {TC-FR1-01, TC-FR1-02, ...} +{Generated during Step 7 (testplan). For stories with no behavioral test + cases (e.g., infrastructure prerequisites): "Verified by: None + (infrastructure — no behavioral test cases)"} ``` **For `[DOCS]` stories** (see `[DOCS]` story requirements below for the @@ -362,7 +369,132 @@ After sizing all epics, verify plausibility: 3. Verify no epic is sized XXL. If any is, stop and require a split before proceeding to Step 7. -### Step 7: Write Coverage Matrix +### Step 7: Generate Testplan + +Generate `.artifacts/design/{issue-key}/07-testplan.md` containing +behavioral test cases anchored to PRD requirements. This artifact serves +dev teams (cross-reference against implemented tests for completeness), +QA teams (independent coverage review without reading every story), and +ALM export (structured for tools like Polarion). + +#### 7a: Derive Test Cases + +For each PRD requirement (FR-N, NFR-N), examine the stories that address +it (from the epic/story files written in Step 5). For each story: + +1. Read the **Acceptance Criteria** — each criterion is a candidate test + case or a grouping of related test cases. +2. Read the **Testing Approach** — use it to understand what scenarios + are expected. +3. Generate test cases at the behavioral/scenario level. Each test case + describes an observable outcome verifiable against a running system. + Do not generate unit-test-level entries — tracing unit tests to + testplan entries is impractical and fragile. + +**Test case ID scheme:** `TC-{requirement-id}-{sequence}`, where +`{requirement-id}` is the PRD requirement ID with the hyphen removed +(e.g., `FR-1` → `FR1`, `NFR-3` → `NFR3`) and `{sequence}` is a +two-digit zero-padded counter within that requirement. Examples: +`TC-FR1-01`, `TC-FR1-02`, `TC-NFR3-01`. + +**Test case fields** (all required): + +| Field | Description | +|-------|-------------| +| Test Case ID | `TC-{req}-{NN}` (e.g., `TC-FR1-01`) | +| Title | One-line scenario description | +| Requirement | PRD requirement ID (e.g., `FR-1`) | +| Story | Which story implements the capability under test (e.g., `Story 1.01`) | +| Preconditions | System state required before the test | +| Scenario / Steps | What the tester does — numbered steps | +| Expected Result | Observable outcome the tester verifies | +| Priority | `critical` / `high` / `medium` / `low` | + +**Priority assignment:** +- `critical` — core user workflows or data integrity +- `high` — important but non-core requirements +- `medium` — edge cases and secondary workflows +- `low` — cosmetic or informational scenarios + +**Coverage target:** Every FR and NFR covered by at least one story +should have at least one test case. A covered requirement with no test +cases is a gap — flag it in the testplan's Gaps section. + +**Negative scenarios:** Include negative/error test cases where the PRD +or design specifies error handling behavior. Do not invent error +scenarios beyond what the requirements and design describe. + +#### 7b: Write the Testplan + +Write `.artifacts/design/{issue-key}/07-testplan.md`: + +```markdown +# Testplan — {issue-key} + +## Overview + +Feature: {feature-key} — {feature-title} +Total test cases: {N} +Requirements covered: {N} of {total FR + NFR count} + +## Test Cases + +### FR-1: {requirement description} + +| ID | Title | Story | Preconditions | Scenario / Steps | Expected Result | Priority | +|----|-------|-------|---------------|------------------|-----------------|----------| +| TC-FR1-01 | {title} | Story 1.01 | {preconditions} | 1. {step} 2. {step} | {expected result} | high | +| TC-FR1-02 | {title} | Story 1.02 | {preconditions} | 1. {step} 2. {step} | {expected result} | medium | + +### FR-2: {requirement description} + +| ID | Title | Story | Preconditions | Scenario / Steps | Expected Result | Priority | +|----|-------|-------|---------------|------------------|-----------------|----------| +| TC-FR2-01 | {title} | Story 2.01 | {preconditions} | 1. {step} 2. {step} | {expected result} | critical | + +### NFR-1: {requirement description} + +| ID | Title | Story | Preconditions | Scenario / Steps | Expected Result | Priority | +|----|-------|-------|---------------|------------------|-----------------|----------| +| TC-NFR1-01 | {title} | Story 1.02 | {preconditions} | 1. {step} 2. {step} | {expected result} | high | + +## Gaps + +{For each PRD requirement with stories but no test cases: why it lacks + coverage and a recommendation. For requirements not covered by any + story (already flagged in the coverage matrix): note "Not testable — + no implementing story." + + If no gaps: "All covered requirements have test cases."} + +## Summary + +| Metric | Count | +|--------|-------| +| Total test cases | {N} | +| Critical | {N} | +| High | {N} | +| Medium | {N} | +| Low | {N} | +| Requirements with test cases | {N} / {total} | +| Requirements without test cases | {N} (see Gaps) | +``` + +Test cases are grouped under requirement headings (not by epic) because +the testplan's purpose is requirement traceability. The Story column +provides the link back to the epic/story structure. + +#### 7c: Add Test Case References to Stories + +After writing the testplan, append a `## Test Case References` section +to each non-`[DOCS]` story file. For each test case in the testplan, +its Story field identifies the implementing story. Collect all TC IDs +for each story and write them into that story's file. + +Re-read each story file before appending to ensure current content. +`[DOCS]` stories do not receive this section. + +### Step 8: Write Coverage Matrix Write `.artifacts/design/{issue-key}/06-coverage.md`: @@ -371,13 +503,13 @@ Write `.artifacts/design/{issue-key}/06-coverage.md`: ## PRD Requirement → Epic/Story Mapping -| PRD Requirement | Epic | Story | Status | -|-----------------|------|-------|--------| -| FR-1: {description} | Epic 1 | Story 1.01, 1.02 | Covered | -| FR-2: {description} | Epic 2 | Story 2.01 | Covered | -| FR-3: {description} | — | — | **GAP** | -| NFR-1: {description} | Epic 1 | Story 1.02 | Covered | -| NFR-2: {description} | Epic 2 | Story 2.01 | Covered | +| PRD Requirement | Epic | Story | Test Cases | Status | +|-----------------|------|-------|------------|--------| +| FR-1: {description} | Epic 1 | Story 1.01, 1.02 | TC-FR1-01, TC-FR1-02 | Covered | +| FR-2: {description} | Epic 2 | Story 2.01 | TC-FR2-01 | Covered | +| FR-3: {description} | — | — | — | **GAP** | +| NFR-1: {description} | Epic 1 | Story 1.02 | TC-NFR1-01 | Covered | +| NFR-2: {description} | Epic 2 | Story 2.01 | TC-NFR2-01 | Covered | ## Gaps @@ -391,7 +523,7 @@ Write `.artifacts/design/{issue-key}/06-coverage.md`: If none: "All stories trace to PRD requirements."} ``` -### Step 8: Verify Artifact Structure +### Step 9: Verify Artifact Structure Quick sanity check before invoking the decomposition review. Verify: @@ -400,11 +532,13 @@ Quick sanity check before invoking the decomposition review. Verify: `05-stories/epic-1-{slug}.md`) 3. Each epic has a corresponding story directory with story files 4. `06-coverage.md` exists and contains at least one mapping row +5. `07-testplan.md` exists and contains at least one test case row +6. Every non-`[DOCS]` story file has a `## Test Case References` section If structural issues are found, fix them before proceeding. Do not invoke a review on incomplete artifacts. -### Step 9: Review Decomposition +### Step 10: Review Decomposition Review the decomposition for structural quality and requirement coverage. This review operates independently from the design document — @@ -422,11 +556,12 @@ subagent for independence. Load it with: `.artifacts/prd/{issue-key}/03-prd.md`) - All decomposition artifacts: `04-epics.md`, all `05-stories/epic-{N}-{slug}.md` files, all - `05-stories/epic-{N}/story-{NN}-{slug}.md` files, `06-coverage.md` + `05-stories/epic-{N}/story-{NN}-{slug}.md` files, `06-coverage.md`, + `07-testplan.md` - NOT the design document (`03-design.md`) — the reviewer evaluates the artifacts on their own merits -Retain the subagent's ID for use in Step 11 — resuming the same +Retain the subagent's ID for use in Step 12 — resuming the same reviewer gives it memory of its previous findings and concerns, producing more coherent follow-up reviews. @@ -441,9 +576,9 @@ severity definitions from the protocol. The subagent path provides stronger independence; the inline path still catches issues by forcing a perspective shift. -### Step 10: Validate and Assess Findings +### Step 11: Validate and Assess Findings -For each finding from Step 9: +For each finding from Step 10: 1. **Validate the reference.** Confirm the cited artifact file and section exist. Discard any finding that references a file or section @@ -466,13 +601,13 @@ For each finding from Step 9: Only fix findings that add real value. Do not make changes for structural preferences not grounded in the evaluation criteria. -### Step 11: Re-Review (if fixes were made) +### Step 12: Re-Review (if fixes were made) -If Step 10 produced changes to the decomposition artifacts: +If Step 11 produced changes to the decomposition artifacts: 1. Obtain a re-review of the updated artifacts: - **If a subagent was used in Step 9 and the runtime supports agent + **If a subagent was used in Step 10 and the runtime supports agent resumption:** Resume the same reviewer agent. Send it the updated artifacts and a summary of fixes applied. This gives the reviewer memory of its original findings and lets it verify they were @@ -492,13 +627,13 @@ If Step 10 produced changes to the decomposition artifacts: - Whether fixes were applied correctly - Whether fixes introduced new issues 2. If new issues are found, fix them following the same validate-and- - assess procedure from Step 10 + assess procedure from Step 11 3. Cap at 2 review-fix rounds total. Decomposition fixes are structural and less likely than code fixes to need multiple iterations. -If no fixes were needed in Step 10, the review passes immediately. +If no fixes were needed in Step 11, the review passes immediately. -### Step 12: Report Review Summary +### Step 13: Report Review Summary ```markdown ## Decomposition Review Summary @@ -513,7 +648,7 @@ If no fixes were needed in Step 10, the review passes immediately. list them with their file, section, and issue description.} ``` -### Step 13: Present to User +### Step 14: Present to User Present the decomposition and highlight: - Number of epics and stories @@ -521,7 +656,7 @@ Present the decomposition and highlight: - Any coverage gaps - Stories that might need size adjustment (too large or too small) - Any assumptions or judgment calls in the decomposition -- Decomposition review summary (from Step 12) +- Decomposition review summary (from Step 13) If the decomposition review gate reported FLAG, present the unfixed CRITICAL/HIGH findings and ask the user to decide how to handle them. @@ -534,6 +669,7 @@ should not resolve them unilaterally. - `.artifacts/design/{issue-key}/05-stories/epic-{N}-{slug}.md` (one per epic) - `.artifacts/design/{issue-key}/05-stories/epic-{N}/story-{NN}-{slug}.md` (one per story) - `.artifacts/design/{issue-key}/06-coverage.md` +- `.artifacts/design/{issue-key}/07-testplan.md` ## When This Phase Is Done diff --git a/design/skills/ingest.md b/design/skills/ingest.md index df9fd10..ea277c2 100644 --- a/design/skills/ingest.md +++ b/design/skills/ingest.md @@ -234,7 +234,8 @@ line-by-line comparison. Then check whether downstream artifacts exist (`02-research.md`, `03-design.md`, `04-epics.md`, `05-stories/`, `06-coverage.md`, -`07-pr-description.md`, `08-review-responses.md`, `sync-manifest.json`). +`07-testplan.md`, `08-pr-description.md`, `09-review-responses.md`, +`sync-manifest.json`). If they do, tell the user: diff --git a/design/skills/publish.md b/design/skills/publish.md index 359595f..6069345 100644 --- a/design/skills/publish.md +++ b/design/skills/publish.md @@ -152,10 +152,28 @@ Read and follow `../../_shared/recipes/render-provenance-footer.md` with `WORKFLOW=design`, `ISSUE_KEY={issue-key}`, `TARGET_FILE="{docs_repo_path}/{release}/{feature}/design.md"`. +If `.artifacts/design/{issue-key}/07-testplan.md` exists, also copy it: + +```bash +cp ".artifacts/design/{issue-key}/07-testplan.md" "{docs_repo_path}/{release}/{feature}/testplan.md" +``` + ```bash git -C "{docs_repo_path}" add "{release}/{feature}/design.md" ``` +If the testplan was copied: + +```bash +git -C "{docs_repo_path}" add "{release}/{feature}/testplan.md" +``` + +```bash +git -C "{docs_repo_path}" commit -m "Add design document and testplan for {issue-key}: {title}" +``` + +If the testplan does not exist, use the original commit message: + ```bash git -C "{docs_repo_path}" commit -m "Add design document for {issue-key}: {title}" ``` @@ -173,7 +191,7 @@ attention: - Key architectural decisions that have significant trade-offs Prepare the PR description and save it to -`.artifacts/design/{issue-key}/07-pr-description.md`: +`.artifacts/design/{issue-key}/08-pr-description.md`: ```markdown ## Design: {title} @@ -189,6 +207,10 @@ Prepare the PR description and save it to markers, or significant trade-offs, list each as a bullet. If none exist, write "General review — no specific items flagged."} +### Documents +- `design.md` — technical design document +- `testplan.md` — behavioral test cases mapped to PRD requirements (included if testplan exists) + ### How to Review - Comment inline on specific sections - Approve when the design accurately reflects a viable implementation approach @@ -199,7 +221,7 @@ If `{issue-key}` is a Jira key, prefix the title with it (`{issue-key}: Design - {title}`); otherwise use `Design: {title}`. ```bash -gh pr create --draft --repo {owner}/{repo} --base {base-branch} --head design/{issue-key} --title "{issue-key}: Design - {title}" --body-file .artifacts/design/{issue-key}/07-pr-description.md +gh pr create --draft --repo {owner}/{repo} --base {base-branch} --head design/{issue-key} --title "{issue-key}: Design - {title}" --body-file .artifacts/design/{issue-key}/08-pr-description.md ``` ### Step 6: Save Publish Metadata @@ -211,11 +233,15 @@ Write `.artifacts/design/{issue-key}/publish-metadata.json`: "release": "{release}", "feature": "{feature}", "design_file_path": "{release}/{feature}/design.md", + "testplan_file_path": "{release}/{feature}/testplan.md", "pr_number": {pr-number}, "branch": "design/{issue-key}" } ``` +Include `testplan_file_path` only if `07-testplan.md` was published. Omit +the field if no testplan exists. + ### Step 7: Report to User Present: @@ -230,7 +256,7 @@ Present: - `.artifacts/design/{issue-key}/publish-metadata.json` - Design document committed and pushed to feature branch in the docs repo - Draft PR created against the docs repo -- `.artifacts/design/{issue-key}/07-pr-description.md` +- `.artifacts/design/{issue-key}/08-pr-description.md` ## When This Phase Is Done diff --git a/design/skills/respond.md b/design/skills/respond.md index 241bd47..535cd3a 100644 --- a/design/skills/respond.md +++ b/design/skills/respond.md @@ -192,6 +192,17 @@ Read and follow `../../_shared/recipes/render-provenance-footer.md` with git -C "{docs_repo_path}" add "{design_file_path}" ``` +If `07-testplan.md` exists and `publish-metadata.json` contains a +`testplan_file_path` field, also copy the testplan: + +```bash +cp ".artifacts/design/{issue-key}/07-testplan.md" "{docs_repo_path}/{testplan_file_path}" +``` + +```bash +git -C "{docs_repo_path}" add "{testplan_file_path}" +``` + ```bash git -C "{docs_repo_path}" commit -m "Design {issue-key}: address review feedback" ``` @@ -238,7 +249,7 @@ rm .artifacts/design/{issue-key}/tmp-reply.md ### Step 5: Update Response Log -Write or update `.artifacts/design/{issue-key}/08-review-responses.md`: +Write or update `.artifacts/design/{issue-key}/09-review-responses.md`: ```markdown # Review Responses — {issue-key} @@ -258,6 +269,7 @@ If design changes were made, check whether they affect the task breakdown: - Did components change? → Epic boundaries may need adjustment - Did APIs or data models change? → Stories may need updating - Did new requirements emerge from review? → Coverage matrix needs checking +- Did requirements or acceptance criteria change? → Testplan may need updating If the decomposition is affected, flag it and recommend `/revise` or re-running `/decompose`. @@ -274,7 +286,7 @@ Summarize: - PR comments posted (with user approval) - `.artifacts/design/{issue-key}/03-design.md` (updated if needed) -- `.artifacts/design/{issue-key}/08-review-responses.md` +- `.artifacts/design/{issue-key}/09-review-responses.md` ## When This Phase Is Done diff --git a/design/skills/revise.md b/design/skills/revise.md index a457fa0..d2f6d25 100644 --- a/design/skills/revise.md +++ b/design/skills/revise.md @@ -37,6 +37,7 @@ Determine which artifacts exist and read them: - `.artifacts/design/{issue-key}/04-epics.md` (epic metadata, if exists) - `.artifacts/design/{issue-key}/05-stories/` (epic and story files, if exist) - `.artifacts/design/{issue-key}/06-coverage.md` (coverage matrix, if exists) +- `.artifacts/design/{issue-key}/07-testplan.md` (testplan, if exists) - `.artifacts/design/{issue-key}/sync-manifest.json` (if exists — means epics/stories have been synced to Jira and filenames are locked) @@ -122,6 +123,19 @@ After applying changes, verify: added, removed, or reassigned. - Do story dependencies still make sense? +**If the testplan exists (`07-testplan.md`):** +- If the design changed: do any test cases reference changed behavior? + Update preconditions, steps, or expected results if the design change + affects what the test validates. +- If stories were added or removed: add or remove test cases as + appropriate. Update `Test Case References` in affected story files. +- If acceptance criteria changed on a story: review the test cases + referencing that story — do they still validate the correct behavior? +- Update the coverage matrix's Test Cases column if test case IDs + changed. +- If requirement IDs changed (rare — requires PRD revision): update + all TC IDs anchored to the changed requirement. + ### Step 5: Update Artifacts Overwrite the affected artifact files. @@ -186,10 +200,23 @@ Read and follow `../../_shared/recipes/render-provenance-footer.md` with `WORKFLOW=design`, `ISSUE_KEY={issue-key}`, `TARGET_FILE="{docs_repo_path}/{design_file_path}"`. +If `07-testplan.md` exists and `publish-metadata.json` contains a +`testplan_file_path` field, also copy the testplan: + +```bash +cp ".artifacts/design/{issue-key}/07-testplan.md" "{docs_repo_path}/{testplan_file_path}" +``` + ```bash git -C "{docs_repo_path}" add "{design_file_path}" ``` +If the testplan was copied: + +```bash +git -C "{docs_repo_path}" add "{testplan_file_path}" +``` + ```bash git -C "{docs_repo_path}" commit -m "Design {issue-key}: revise — {brief description}" ``` @@ -213,6 +240,11 @@ Summarize what changed: - Epic 2: Split Story 2.3 into 2.3 and 2.4 - Coverage matrix: Updated to reflect new story mapping +### Testplan Changes +- {TC-FR1-03 added — new acceptance criterion on Story 1.01} +- {TC-NFR2-01 updated — expected result changed to match revised design} +- {Omit this section if the testplan did not change or does not exist} + ### Consistency Updates - Section 8: Added open question about performance impact of new approach @@ -244,6 +276,7 @@ The following artifacts were modified since the last sync. Re-run - `.artifacts/design/{issue-key}/05-stories/epic-*.md` (updated, if epics changed) - `.artifacts/design/{issue-key}/05-stories/epic-*/story-*.md` (updated, if stories changed) - `.artifacts/design/{issue-key}/06-coverage.md` (updated, if coverage changed) +- `.artifacts/design/{issue-key}/07-testplan.md` (updated, if testplan changed) ## When This Phase Is Done diff --git a/design/skills/sync.md b/design/skills/sync.md index 90484b0..fa58059 100644 --- a/design/skills/sync.md +++ b/design/skills/sync.md @@ -423,6 +423,13 @@ For each new story under each epic, create a Jira issue: {dependencies from story file, with local references resolved to Jira keys per the Reference Resolution section} +## Test Case References + +{test case references from story file, preserving TC IDs as-is. + TC IDs are requirement-anchored (TC-FR1-01) and do not need Jira key + resolution. If the story has no Test Case References section, omit + this section from the Jira description.} + ## Design Reference Design document: {link to design doc PR or file} diff --git a/e2e/SKILL.md b/e2e/SKILL.md index 9f648f5..0f662bd 100644 --- a/e2e/SKILL.md +++ b/e2e/SKILL.md @@ -1,6 +1,6 @@ --- name: e2e -version: 0.3.2 +version: 0.4.0 description: >- Story-to-e2e-test workflow that takes a Jira [QE] Story, discovers the project's e2e testing infrastructure, plans test scenarios, writes e2e diff --git a/e2e/skills/ingest.md b/e2e/skills/ingest.md index 139eba4..92829fb 100644 --- a/e2e/skills/ingest.md +++ b/e2e/skills/ingest.md @@ -151,11 +151,16 @@ Read these from the docs repo: architectural decisions and locked decisions incorporated as content 2. **PRD** (`prd.md`) — the product requirements, with locked decisions reflected in the requirements text +3. **Testplan** (`testplan.md`) — behavioral test cases mapped to PRD + requirements. For `[QE]` stories, this is particularly valuable: the + testplan's test cases for the relevant requirements provide structured + starting points for e2e scenario design. Cross-reference test case + IDs in `02-plan.md` when scenarios align. If the docs repo documents are not found, ask the user for their location -or proceed with only the Jira story content. The design document and PRD -are valuable context but not strictly required — the story's acceptance -criteria are the primary contract. +or proceed with only the Jira story content. The design document, PRD, +and testplan are valuable context but not strictly required — the story's +acceptance criteria are the primary contract. ### Step 6: Explore E2E Test Infrastructure diff --git a/implement/SKILL.md b/implement/SKILL.md index 026d1c9..3bd0eb7 100644 --- a/implement/SKILL.md +++ b/implement/SKILL.md @@ -1,6 +1,6 @@ --- name: implement -version: 0.4.1 +version: 0.5.0 description: >- Story-to-code workflow that takes a Jira Story, plans the implementation, writes contract-based tests and production code via TDD, validates against diff --git a/implement/skills/ingest.md b/implement/skills/ingest.md index ca9b084..2f9d914 100644 --- a/implement/skills/ingest.md +++ b/implement/skills/ingest.md @@ -142,11 +142,15 @@ Read these from the docs repo: architectural decisions and locked decisions incorporated as content 2. **PRD** (`prd.md`) — the product requirements, with locked decisions reflected in the requirements text +3. **Testplan** (`testplan.md`) — behavioral test cases mapped to PRD + requirements. Valuable context for understanding what scenarios the + feature is expected to pass, but not required. The story's acceptance + criteria and testing approach remain the primary contract. If the docs repo documents are not found, ask the user for their location -or proceed with only the Jira story content. The design document and PRD -are valuable context but not strictly required — the story's acceptance -criteria are the primary contract. +or proceed with only the Jira story content. The design document, PRD, +and testplan are valuable context but not strictly required — the story's +acceptance criteria are the primary contract. ### Step 6: Explore the Codebase From fb97179aae7a139672980138269f6674bf6df97f Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 11:07:07 -0400 Subject: [PATCH 02/38] Add Automation field to testplan test cases MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add an Automation column (automated/manual) to the testplan template. This captures whether each behavioral test case will be verified by automated tests or requires manual execution — a standard ALM field that avoids requiring someone to fill it in during export. Assisted-by: Claude Opus 4.6 (1M) --- design/skills/decompose.md | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/design/skills/decompose.md b/design/skills/decompose.md index eee6fe5..62ddb7c 100644 --- a/design/skills/decompose.md +++ b/design/skills/decompose.md @@ -409,6 +409,7 @@ two-digit zero-padded counter within that requirement. Examples: | Scenario / Steps | What the tester does — numbered steps | | Expected Result | Observable outcome the tester verifies | | Priority | `critical` / `high` / `medium` / `low` | +| Automation | `automated` / `manual` | **Priority assignment:** - `critical` — core user workflows or data integrity @@ -416,6 +417,13 @@ two-digit zero-padded counter within that requirement. Examples: - `medium` — edge cases and secondary workflows - `low` — cosmetic or informational scenarios +**Automation assignment:** +- `automated` — the scenario will be verified by automated tests (e2e + or feature-level integration) produced by the implementing story +- `manual` — the scenario requires human verification (visual checks, + hardware interaction, exploratory testing, or scenarios explicitly + scoped as manual in a `[QE]` story) + **Coverage target:** Every FR and NFR covered by at least one story should have at least one test case. A covered requirement with no test cases is a gap — flag it in the testplan's Gaps section. @@ -441,22 +449,22 @@ Requirements covered: {N} of {total FR + NFR count} ### FR-1: {requirement description} -| ID | Title | Story | Preconditions | Scenario / Steps | Expected Result | Priority | -|----|-------|-------|---------------|------------------|-----------------|----------| -| TC-FR1-01 | {title} | Story 1.01 | {preconditions} | 1. {step} 2. {step} | {expected result} | high | -| TC-FR1-02 | {title} | Story 1.02 | {preconditions} | 1. {step} 2. {step} | {expected result} | medium | +| ID | Title | Story | Preconditions | Scenario / Steps | Expected Result | Priority | Automation | +|----|-------|-------|---------------|------------------|-----------------|----------|------------| +| TC-FR1-01 | {title} | Story 1.01 | {preconditions} | 1. {step} 2. {step} | {expected result} | high | automated | +| TC-FR1-02 | {title} | Story 1.02 | {preconditions} | 1. {step} 2. {step} | {expected result} | medium | automated | ### FR-2: {requirement description} -| ID | Title | Story | Preconditions | Scenario / Steps | Expected Result | Priority | -|----|-------|-------|---------------|------------------|-----------------|----------| -| TC-FR2-01 | {title} | Story 2.01 | {preconditions} | 1. {step} 2. {step} | {expected result} | critical | +| ID | Title | Story | Preconditions | Scenario / Steps | Expected Result | Priority | Automation | +|----|-------|-------|---------------|------------------|-----------------|----------|------------| +| TC-FR2-01 | {title} | Story 2.01 | {preconditions} | 1. {step} 2. {step} | {expected result} | critical | automated | ### NFR-1: {requirement description} -| ID | Title | Story | Preconditions | Scenario / Steps | Expected Result | Priority | -|----|-------|-------|---------------|------------------|-----------------|----------| -| TC-NFR1-01 | {title} | Story 1.02 | {preconditions} | 1. {step} 2. {step} | {expected result} | high | +| ID | Title | Story | Preconditions | Scenario / Steps | Expected Result | Priority | Automation | +|----|-------|-------|---------------|------------------|-----------------|----------|------------| +| TC-NFR1-01 | {title} | Story 1.02 | {preconditions} | 1. {step} 2. {step} | {expected result} | high | automated | ## Gaps @@ -476,6 +484,8 @@ Requirements covered: {N} of {total FR + NFR count} | High | {N} | | Medium | {N} | | Low | {N} | +| Automated | {N} | +| Manual | {N} | | Requirements with test cases | {N} / {total} | | Requirements without test cases | {N} (see Gaps) | ``` From d93e1b3ad261ad35f53007db8a262395e226198e Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 13:20:45 -0400 Subject: [PATCH 03/38] Add structured testplan comment handling to design respond phase The respond phase now routes PR review comments targeting testplan.md to testplan-specific handling: - New Testplan feedback category routes line-level comments by path and top-level comments by TC ID pattern detection - Step 4b applies testplan changes (add/modify/remove test cases) with cascade to story Test Case References and coverage matrix - Sync-manifest guard re-applies Story-to-Jira-key resolution when pushing testplan to the docs repo after sync has run - Response log captures testplan changes per round Design: 0.5.0 -> 0.6.0 Assisted-by: Claude Opus 4.6 (1M) --- design/SKILL.md | 2 +- design/skills/respond.md | 77 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 77 insertions(+), 2 deletions(-) diff --git a/design/SKILL.md b/design/SKILL.md index f3cbdde..ef3b438 100644 --- a/design/SKILL.md +++ b/design/SKILL.md @@ -1,6 +1,6 @@ --- name: design -version: 0.5.0 +version: 0.6.0 description: >- Design-and-decompose workflow that takes a PRD, researches the problem space, drafts a technical design document, decomposes work into Jira-ready epics and diff --git a/design/skills/respond.md b/design/skills/respond.md index 535cd3a..303c7c3 100644 --- a/design/skills/respond.md +++ b/design/skills/respond.md @@ -71,8 +71,19 @@ Group comments into categories: | **New requirement** | Flag for user decision — update design or defer | | **Approval / positive** | Acknowledge | | **Open question resolution** | Resolve the open question (see Step 4) | +| **Testplan feedback** | Route to testplan-specific handling (Step 4b) | | **Out of scope** | Draft a reply explaining why | +**Routing testplan comments:** Line-level review comments (from +`gh api .../pulls/{pr-number}/comments`) include a `path` field. Comments +with `path` ending in `testplan.md` are categorized as **Testplan +feedback**. Top-level PR comments (from `gh pr view --json comments`) do +not carry a `path`. For these, inspect the comment body: if it references +test case IDs matching the pattern `TC-` followed by a requirement +identifier, or discusses adding, modifying, or removing test cases, +categorize as **Testplan feedback**. When uncertain, categorize as the +next-best-fit category and let the user reclassify during Step 3. + ### Step 3: Propose Responses Present each comment with a proposed response: @@ -93,6 +104,14 @@ Present each comment with a proposed response: **Category:** Open question resolution **Proposed resolution:** {synthesized answer from reviewer discussion} **Design change needed:** Yes — incorporate into Section {N}, remove open question 8.2 + +### Comment 3 — {reviewer} on testplan.md, TC-FR2-01 +> {quoted comment text} + +**Category:** Testplan feedback +**Proposed response:** {suggested reply} +**Testplan change needed:** {modify TC-FR2-01 expected result / add TC-FR2-03 / remove TC-FR1-02} +**Cascade:** Update Story 2.01 Test Case References, update coverage matrix ``` Wait for the user to approve, modify, or reject each response. @@ -137,6 +156,42 @@ section, synthesize the discussion into a proposed resolution: 7. If the Open Questions section is now empty, remove the entire section (heading and introductory text) from the design document. +#### Applying testplan changes + +When approved changes include testplan modifications (category: Testplan +feedback), apply them in this order: + +1. **Modify `07-testplan.md`.** Add, modify, or remove test cases as + directed by the approved response. For each change: + - **Adding a test case:** Assign the next available sequence number + within the requirement group (e.g., if TC-FR2-01 and TC-FR2-02 + exist, the new case is TC-FR2-03). Fill all required fields (ID, + Title, Requirement, Story, Preconditions, Scenario/Steps, Expected + Result, Priority, Automation). Update the testplan's Overview counts + and Summary table. + - **Modifying a test case:** Update the affected fields. If the Story + assignment changes, update both the old and new story's Test Case + References in step 2 below. + - **Removing a test case:** Delete the row. Update the testplan's + Overview counts and Summary table. + +2. **Cascade to story files.** For each affected story (identified by the + Story column of changed test cases): + - Re-read the story file at + `.artifacts/design/{issue-key}/05-stories/epic-{N}/story-{NN}-{slug}.md`. + - Rewrite the `## Test Case References` section: collect all TC IDs + from the updated testplan where the Story column matches this story, + then write `Verified by: {comma-separated TC IDs}`. + - If a story loses all its test cases, write: + `Verified by: None (no behavioral test cases after testplan revision)`. + +3. **Cascade to coverage matrix.** Re-read + `.artifacts/design/{issue-key}/06-coverage.md`. For each row in the PRD + Requirement mapping table, update the `Test Cases` column to reflect + the current TC IDs from the testplan for that requirement. If a + requirement previously had test cases and now has none, flag it in the + Gaps section. + **Update the local artifact:** Update `.artifacts/design/{issue-key}/03-design.md`. @@ -193,12 +248,25 @@ git -C "{docs_repo_path}" add "{design_file_path}" ``` If `07-testplan.md` exists and `publish-metadata.json` contains a -`testplan_file_path` field, also copy the testplan: +`testplan_file_path` field, also copy the testplan to the docs repo. + +**Sync-manifest guard:** If `.artifacts/design/{issue-key}/sync-manifest.json` +exists, the published testplan's Story column must use Jira keys (resolved +by `/sync`), not local identifiers. Before copying, read the sync manifest +and resolve the Story column in each test case row: replace local +references (e.g., `Story 1.01`) with their Jira keys from the manifest +(e.g., `EDM-1234`). Write the resolved version to the docs repo — do NOT +modify the local `07-testplan.md` (it keeps local identifiers). + +If `sync-manifest.json` does not exist, copy `07-testplan.md` as-is. ```bash cp ".artifacts/design/{issue-key}/07-testplan.md" "{docs_repo_path}/{testplan_file_path}" ``` +(If the sync-manifest guard applied, the file written to the docs repo is +the resolved version, not the literal local copy.) + ```bash git -C "{docs_repo_path}" add "{testplan_file_path}" ``` @@ -261,6 +329,7 @@ Write or update `.artifacts/design/{issue-key}/09-review-responses.md`: - **Category:** {category} - **Response:** {what was replied} - **Design change:** {Yes/No — description if yes} +- **Testplan change:** {Yes/No — TC-FR2-01 modified, TC-FR2-03 added / None} ``` ### Step 6: Assess Decomposition Impact @@ -271,6 +340,12 @@ If design changes were made, check whether they affect the task breakdown: - Did new requirements emerge from review? → Coverage matrix needs checking - Did requirements or acceptance criteria change? → Testplan may need updating +If testplan changes were applied in Step 4b, verify that the cascade +(story Test Case References and coverage matrix Test Cases column) is +consistent. If the cascade reveals an inconsistency not caught during +Step 4b (e.g., a story references a TC ID that was removed), fix it +before proceeding. + If the decomposition is affected, flag it and recommend `/revise` or re-running `/decompose`. From be7035989f49214038cd6416130c29e3faf804fb Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 13:28:07 -0400 Subject: [PATCH 04/38] Address code review findings on testplan commits Fixes from external code review: - Remove Requirement column from field table (derived from section heading, not rendered as table column) - Update controller phase description to mention testplan - Add /respond to testplan Written-by in controller artifact table - Add testplan, story files, coverage matrix to respond output section - Add sync-manifest guard to revise and publish docs-repo copy paths - Remove Test Case References from story template (written by Step 7c, not Step 5) to avoid duplicate sections - Replace "Step 4b" references with section name in respond - Clarify cp command behavior when sync-manifest guard applies - Update respond cascade to also update testplan Gaps section on removal - Add note about unpublished testplan behavior in respond Assisted-by: Claude Opus 4.6 (1M) --- design/skills/controller.md | 4 ++-- design/skills/decompose.md | 15 ++++++--------- design/skills/publish.md | 13 ++++++++++++- design/skills/respond.md | 25 ++++++++++++++++++------- design/skills/revise.md | 13 ++++++++++++- 5 files changed, 50 insertions(+), 20 deletions(-) diff --git a/design/skills/controller.md b/design/skills/controller.md index 484141c..19598f5 100644 --- a/design/skills/controller.md +++ b/design/skills/controller.md @@ -22,7 +22,7 @@ executing phases and handling transitions between them. Draft the design/architecture document using the template and section guidance. 4. **Decompose** (`/decompose`) — `decompose.md` - Break the design into Jira-ready epics and stories with a coverage matrix. + Break the design into Jira-ready epics and stories with a testplan and coverage matrix. 5. **Revise** (`/revise`) — `revise.md` Incorporate user feedback into the design document and/or task breakdown. Repeatable. @@ -58,7 +58,7 @@ the source repo (this directory should be gitignored in the source repo): | Epic files | `05-stories/epic-{N}-{slug}.md` | `/decompose`, `/revise` | | Story files | `05-stories/epic-{N}/story-{NN}-{slug}.md` | `/decompose`, `/revise` | | Coverage matrix | `06-coverage.md` | `/decompose`, `/revise` | -| Testplan | `07-testplan.md` | `/decompose`, `/revise` | +| Testplan | `07-testplan.md` | `/decompose`, `/revise`, `/respond` | | PR description | `08-pr-description.md` | `/publish` | | Publish metadata | `publish-metadata.json` | `/publish` | | Review responses | `09-review-responses.md` | `/respond` | diff --git a/design/skills/decompose.md b/design/skills/decompose.md index 62ddb7c..4433691 100644 --- a/design/skills/decompose.md +++ b/design/skills/decompose.md @@ -212,15 +212,12 @@ Write each story to Epic: Epic {N} — {epic title} PRD Requirements: {FR-1, NFR-1} Design section: {§4.3 API Changes, or specific subsection} - -## Test Case References - -Verified by: {TC-FR1-01, TC-FR1-02, ...} -{Generated during Step 7 (testplan). For stories with no behavioral test - cases (e.g., infrastructure prerequisites): "Verified by: None - (infrastructure — no behavioral test cases)"} ``` +Do NOT include a `## Test Case References` section when writing stories +in Step 5 — that section is added by Step 7c after the testplan is +generated. + **For `[DOCS]` stories** (see `[DOCS]` story requirements below for the Documentation Inputs section): @@ -397,13 +394,13 @@ it (from the epic/story files written in Step 5). For each story: two-digit zero-padded counter within that requirement. Examples: `TC-FR1-01`, `TC-FR1-02`, `TC-NFR3-01`. -**Test case fields** (all required): +**Test case fields** (all required per row; the requirement is identified +by the section heading, not repeated in each row): | Field | Description | |-------|-------------| | Test Case ID | `TC-{req}-{NN}` (e.g., `TC-FR1-01`) | | Title | One-line scenario description | -| Requirement | PRD requirement ID (e.g., `FR-1`) | | Story | Which story implements the capability under test (e.g., `Story 1.01`) | | Preconditions | System state required before the test | | Scenario / Steps | What the tester does — numbered steps | diff --git a/design/skills/publish.md b/design/skills/publish.md index 6069345..cd71c14 100644 --- a/design/skills/publish.md +++ b/design/skills/publish.md @@ -152,12 +152,23 @@ Read and follow `../../_shared/recipes/render-provenance-footer.md` with `WORKFLOW=design`, `ISSUE_KEY={issue-key}`, `TARGET_FILE="{docs_repo_path}/{release}/{feature}/design.md"`. -If `.artifacts/design/{issue-key}/07-testplan.md` exists, also copy it: +If `.artifacts/design/{issue-key}/07-testplan.md` exists, also copy it. + +**Sync-manifest guard:** If `.artifacts/design/{issue-key}/sync-manifest.json` +exists, the published testplan's Story column must use Jira keys. Before +copying, read the sync manifest and resolve the Story column in each test +case row (`Story 1.01` → Jira key from manifest). Write the resolved +version to the docs repo — do NOT modify the local `07-testplan.md`. +If `sync-manifest.json` does not exist, copy `07-testplan.md` as-is. ```bash cp ".artifacts/design/{issue-key}/07-testplan.md" "{docs_repo_path}/{release}/{feature}/testplan.md" ``` +(If the sync-manifest guard applied, write the resolved content to +`{docs_repo_path}/{release}/{feature}/testplan.md` directly instead +of using a literal `cp` of the local file.) + ```bash git -C "{docs_repo_path}" add "{release}/{feature}/design.md" ``` diff --git a/design/skills/respond.md b/design/skills/respond.md index 303c7c3..cbe382d 100644 --- a/design/skills/respond.md +++ b/design/skills/respond.md @@ -71,7 +71,7 @@ Group comments into categories: | **New requirement** | Flag for user decision — update design or defer | | **Approval / positive** | Acknowledge | | **Open question resolution** | Resolve the open question (see Step 4) | -| **Testplan feedback** | Route to testplan-specific handling (Step 4b) | +| **Testplan feedback** | Route to testplan change handling (see Step 4, "Applying testplan changes") | | **Out of scope** | Draft a reply explaining why | **Routing testplan comments:** Line-level review comments (from @@ -190,7 +190,11 @@ feedback), apply them in this order: Requirement mapping table, update the `Test Cases` column to reflect the current TC IDs from the testplan for that requirement. If a requirement previously had test cases and now has none, flag it in the - Gaps section. + coverage matrix Gaps section. + +4. **Update testplan Gaps section.** If test cases were removed and a + requirement that previously had test cases now has none, update the + testplan's own Gaps section to reflect the new gap. **Update the local artifact:** Update `.artifacts/design/{issue-key}/03-design.md`. @@ -249,6 +253,9 @@ git -C "{docs_repo_path}" add "{design_file_path}" If `07-testplan.md` exists and `publish-metadata.json` contains a `testplan_file_path` field, also copy the testplan to the docs repo. +(If the testplan was never published — no `testplan_file_path` in +metadata — testplan changes are applied locally only. Re-run `/publish` +to include the testplan in the docs repo.) **Sync-manifest guard:** If `.artifacts/design/{issue-key}/sync-manifest.json` exists, the published testplan's Story column must use Jira keys (resolved @@ -258,14 +265,15 @@ references (e.g., `Story 1.01`) with their Jira keys from the manifest (e.g., `EDM-1234`). Write the resolved version to the docs repo — do NOT modify the local `07-testplan.md` (it keeps local identifiers). -If `sync-manifest.json` does not exist, copy `07-testplan.md` as-is. +If `sync-manifest.json` does not exist, copy `07-testplan.md` as-is: ```bash cp ".artifacts/design/{issue-key}/07-testplan.md" "{docs_repo_path}/{testplan_file_path}" ``` -(If the sync-manifest guard applied, the file written to the docs repo is -the resolved version, not the literal local copy.) +If the sync-manifest guard applied, write the resolved content to +`{docs_repo_path}/{testplan_file_path}` directly instead of using +a literal `cp` of the unresolved local file. ```bash git -C "{docs_repo_path}" add "{testplan_file_path}" @@ -340,10 +348,10 @@ If design changes were made, check whether they affect the task breakdown: - Did new requirements emerge from review? → Coverage matrix needs checking - Did requirements or acceptance criteria change? → Testplan may need updating -If testplan changes were applied in Step 4b, verify that the cascade +If testplan changes were applied in Step 4, verify that the cascade (story Test Case References and coverage matrix Test Cases column) is consistent. If the cascade reveals an inconsistency not caught during -Step 4b (e.g., a story references a TC ID that was removed), fix it +Step 4 (e.g., a story references a TC ID that was removed), fix it before proceeding. If the decomposition is affected, flag it and recommend `/revise` or @@ -361,6 +369,9 @@ Summarize: - PR comments posted (with user approval) - `.artifacts/design/{issue-key}/03-design.md` (updated if needed) +- `.artifacts/design/{issue-key}/07-testplan.md` (updated if testplan feedback was applied) +- `.artifacts/design/{issue-key}/05-stories/epic-{N}/story-{NN}-{slug}.md` (Test Case References updated if testplan changed) +- `.artifacts/design/{issue-key}/06-coverage.md` (Test Cases column updated if testplan changed) - `.artifacts/design/{issue-key}/09-review-responses.md` ## When This Phase Is Done diff --git a/design/skills/revise.md b/design/skills/revise.md index d2f6d25..f162667 100644 --- a/design/skills/revise.md +++ b/design/skills/revise.md @@ -201,12 +201,23 @@ Read and follow `../../_shared/recipes/render-provenance-footer.md` with `TARGET_FILE="{docs_repo_path}/{design_file_path}"`. If `07-testplan.md` exists and `publish-metadata.json` contains a -`testplan_file_path` field, also copy the testplan: +`testplan_file_path` field, also copy the testplan to the docs repo. + +**Sync-manifest guard:** If `.artifacts/design/{issue-key}/sync-manifest.json` +exists, the published testplan's Story column must use Jira keys. Before +copying, read the sync manifest and resolve the Story column in each test +case row (`Story 1.01` → Jira key from manifest). Write the resolved +version to the docs repo — do NOT modify the local `07-testplan.md`. +If `sync-manifest.json` does not exist, copy `07-testplan.md` as-is. ```bash cp ".artifacts/design/{issue-key}/07-testplan.md" "{docs_repo_path}/{testplan_file_path}" ``` +(If the sync-manifest guard applied, write the resolved content to +`{docs_repo_path}/{testplan_file_path}` directly instead of using +a literal `cp` of the local file.) + ```bash git -C "{docs_repo_path}" add "{design_file_path}" ``` From ee279a5b6a39c1e7eed795ba4e75f544c9abc13f Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 13:35:13 -0400 Subject: [PATCH 05/38] Add testplan Story-column resolution to design sync phase New Step 7 in sync resolves the published testplan's Story column from local identifiers (Story 1.01) to Jira keys (EDM-1234) after syncing stories. The local 07-testplan.md keeps local identifiers; only the docs-repo copy gets Jira keys. This enables downstream workflows (implement, e2e) to filter the testplan by Jira key. Includes idempotency check (skip commit if content unchanged), git safety checks (status, branch verification), and error handling consistent with other sync phase operations. Assisted-by: Claude Opus 4.6 (1M) --- design/skills/sync.md | 82 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 81 insertions(+), 1 deletion(-) diff --git a/design/skills/sync.md b/design/skills/sync.md index fa58059..9a8da85 100644 --- a/design/skills/sync.md +++ b/design/skills/sync.md @@ -595,11 +595,90 @@ Fields: - `synced_at` — top-level only, not per-entry. Updated to the current timestamp at the end of each sync run. -### Step 7: Report to User +### Step 7: Update Published Testplan + +If the testplan has been published to the docs repo, update the +published copy's Story column with Jira keys so downstream workflows +can filter by Jira key. + +**Skip this step entirely if any of these are true:** +- `.artifacts/prd/config.json` does not exist +- `.artifacts/design/{issue-key}/publish-metadata.json` does not exist +- `publish-metadata.json` does not contain a `testplan_file_path` field +- `.artifacts/design/{issue-key}/07-testplan.md` does not exist + +**Resolve Story references:** + +1. Read `.artifacts/design/{issue-key}/07-testplan.md`. +2. For each test case row in the Test Cases tables, resolve the Story + column using the sync manifest: replace local references (e.g., + `Story 1.01`) with their Jira keys (e.g., `EDM-1234`). Use the + reference resolution logic described in the Reference Resolution + section (look up `story-{NN}-*.md` under `epic-{N}/` in the manifest + to find the Jira key). +3. If a Story reference cannot be resolved (not in the manifest), leave + it as-is and note it for the user. + +**Write the resolved testplan to the docs repo:** + +Read `.artifacts/prd/config.json` to get the docs repo path. Read +`publish-metadata.json` to get the `testplan_file_path`. + +Write the resolved testplan content (with Jira keys in the Story column) +to `{docs_repo_path}/{testplan_file_path}`. Do NOT modify the local +`07-testplan.md` — it keeps local identifiers. + +Compare the resolved content against the current content of +`{docs_repo_path}/{testplan_file_path}`. If they are identical, skip +the commit — the published testplan is already up to date. + +```bash +git -C "{docs_repo_path}" fetch origin +``` + +```bash +git -C "{docs_repo_path}" status +``` + +If there are uncommitted changes in the docs repo, ask the user before +continuing. + +```bash +git -C "{docs_repo_path}" branch --show-current +``` + +If not on the PR branch (`design/{issue-key}`), check it out: + +```bash +git -C "{docs_repo_path}" checkout design/{issue-key} +``` + +```bash +git -C "{docs_repo_path}" pull --ff-only +``` + +```bash +git -C "{docs_repo_path}" add "{testplan_file_path}" +``` + +```bash +git -C "{docs_repo_path}" commit -m "Sync {issue-key}: resolve testplan story references to Jira keys" +``` + +```bash +git -C "{docs_repo_path}" push +``` + +If any git operation fails, report the error to the user. The Jira sync +is already complete — this step only affects the docs repo copy. Offer +to retry or skip. + +### Step 8: Report to User Summarize: - How many epics and stories were created, updated, closed, and unchanged - Confirm the hierarchy was verified: every epic has parent = Feature (verified in Step 4), every story has parent = its epic (verified in Step 5) +- Whether the published testplan was updated (and how many Story references were resolved vs. left unresolved), or skipped (no testplan published) - Link to the Feature issue in Jira (which now shows the full hierarchy) **Do not suggest manual parent linking as a next step.** If any parent @@ -624,6 +703,7 @@ local `.artifacts/` files who needs to find the corresponding Jira issue: - Jira epics and stories created, updated, or closed (with user approval) - `.artifacts/design/{issue-key}/sync-manifest.json` (v2 schema) +- Published testplan updated in docs repo with Jira keys (if testplan was published) ## When This Phase Is Done From 8d971839232f86baab38f6181379d886e241c448 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 13:56:37 -0400 Subject: [PATCH 06/38] Add testplan filtering to implement ingest and adopt heading-based format Two changes: 1. Replace horizontal-table testplan format with heading-based structure across all files. Test cases now use H4 headings with a metadata table (Story, Priority, Automation) and H5 sub-sections (Preconditions, Steps, Expected Results). Prose-heavy content is no longer crammed into table cells. All references to "Story column" and "test case row" updated to "Story field" and "test case entry" across decompose, respond, sync, revise, publish, and decomposition-review. 2. Add structured testplan filtering to implement ingest (Step 5d). Replaces the "optional context" mention with: filter the published testplan by Jira key (with TC-ID fallback for pre-sync testplans), three-outcome gate (normal/expected-zero/anomalous-zero), write story-scoped test-plan.md, and Story Test Plan section in the context template. Assisted-by: Claude Opus 4.6 (1M) --- design/decomposition-review.md | 4 +- design/skills/decompose.md | 106 ++++++++++++++++++++++++++------- design/skills/publish.md | 6 +- design/skills/respond.md | 30 +++++----- design/skills/revise.md | 6 +- design/skills/sync.md | 15 +++-- implement/skills/ingest.md | 98 +++++++++++++++++++++++++++--- 7 files changed, 205 insertions(+), 60 deletions(-) diff --git a/design/decomposition-review.md b/design/decomposition-review.md index 315bd49..2e29eac 100644 --- a/design/decomposition-review.md +++ b/design/decomposition-review.md @@ -107,7 +107,7 @@ impact: restatements of acceptance criteria? Are priorities assigned consistently (critical for core workflows, not everything marked high)? Do the `Test Case References` in story files match the - testplan's Story column? Does the coverage matrix's Test Cases column + testplan's Story field? Does the coverage matrix's Test Cases column match the testplan? Are there requirements with stories but no test cases (testplan gaps)? @@ -146,7 +146,7 @@ Findings that cannot cite a specific file and section in the actual artifacts must be discarded — they indicate hallucinated references. Coverage matrix findings should cite `06-coverage.md` and the specific row or gap. Testplan findings should cite `07-testplan.md` and the -specific requirement section or test case row. +specific requirement section or test case entry. ## Severity Definitions diff --git a/design/skills/decompose.md b/design/skills/decompose.md index 4433691..1c65fb9 100644 --- a/design/skills/decompose.md +++ b/design/skills/decompose.md @@ -394,19 +394,16 @@ it (from the epic/story files written in Step 5). For each story: two-digit zero-padded counter within that requirement. Examples: `TC-FR1-01`, `TC-FR1-02`, `TC-NFR3-01`. -**Test case fields** (all required per row; the requirement is identified -by the section heading, not repeated in each row): +**Test case fields** (all required per test case; the requirement is +identified by the parent section heading): | Field | Description | |-------|-------------| -| Test Case ID | `TC-{req}-{NN}` (e.g., `TC-FR1-01`) | -| Title | One-line scenario description | -| Story | Which story implements the capability under test (e.g., `Story 1.01`) | -| Preconditions | System state required before the test | -| Scenario / Steps | What the tester does — numbered steps | -| Expected Result | Observable outcome the tester verifies | -| Priority | `critical` / `high` / `medium` / `low` | -| Automation | `automated` / `manual` | +| Test Case ID and Title | H4 heading: `#### TC-{req}-{NN}: {one-line scenario description}` | +| Story, Priority, Automation | Single metadata table beneath the H4 heading | +| Preconditions | H5 section: system state required before the test | +| Steps | H5 section: what the tester does — numbered steps | +| Expected Results | H5 section: observable outcomes the tester verifies | **Priority assignment:** - `critical` — core user workflows or data integrity @@ -446,22 +443,85 @@ Requirements covered: {N} of {total FR + NFR count} ### FR-1: {requirement description} -| ID | Title | Story | Preconditions | Scenario / Steps | Expected Result | Priority | Automation | -|----|-------|-------|---------------|------------------|-----------------|----------|------------| -| TC-FR1-01 | {title} | Story 1.01 | {preconditions} | 1. {step} 2. {step} | {expected result} | high | automated | -| TC-FR1-02 | {title} | Story 1.02 | {preconditions} | 1. {step} 2. {step} | {expected result} | medium | automated | +#### TC-FR1-01: {scenario title} + +| Story | Priority | Automation | +|-------|----------|------------| +| Story 1.01 | high | automated | + +##### Preconditions + +- {system state required before the test} + +##### Steps + +1. {what the tester does} +2. {next action} + +##### Expected Results + +- {observable outcome the tester verifies} + +#### TC-FR1-02: {scenario title} + +| Story | Priority | Automation | +|-------|----------|------------| +| Story 1.02 | medium | automated | + +##### Preconditions + +- {precondition} + +##### Steps + +1. {step} +2. {step} + +##### Expected Results + +- {expected outcome} ### FR-2: {requirement description} -| ID | Title | Story | Preconditions | Scenario / Steps | Expected Result | Priority | Automation | -|----|-------|-------|---------------|------------------|-----------------|----------|------------| -| TC-FR2-01 | {title} | Story 2.01 | {preconditions} | 1. {step} 2. {step} | {expected result} | critical | automated | +#### TC-FR2-01: {scenario title} + +| Story | Priority | Automation | +|-------|----------|------------| +| Story 2.01 | critical | automated | + +##### Preconditions + +- {precondition} + +##### Steps + +1. {step} +2. {step} + +##### Expected Results + +- {expected outcome} ### NFR-1: {requirement description} -| ID | Title | Story | Preconditions | Scenario / Steps | Expected Result | Priority | Automation | -|----|-------|-------|---------------|------------------|-----------------|----------|------------| -| TC-NFR1-01 | {title} | Story 1.02 | {preconditions} | 1. {step} 2. {step} | {expected result} | high | automated | +#### TC-NFR1-01: {scenario title} + +| Story | Priority | Automation | +|-------|----------|------------| +| Story 1.02 | high | automated | + +##### Preconditions + +- {precondition} + +##### Steps + +1. {step} +2. {step} + +##### Expected Results + +- {expected outcome} ## Gaps @@ -488,8 +548,8 @@ Requirements covered: {N} of {total FR + NFR count} ``` Test cases are grouped under requirement headings (not by epic) because -the testplan's purpose is requirement traceability. The Story column -provides the link back to the epic/story structure. +the testplan's purpose is requirement traceability. The Story field in +each test case's metadata table links back to the epic/story structure. #### 7c: Add Test Case References to Stories @@ -539,7 +599,7 @@ Quick sanity check before invoking the decomposition review. Verify: `05-stories/epic-1-{slug}.md`) 3. Each epic has a corresponding story directory with story files 4. `06-coverage.md` exists and contains at least one mapping row -5. `07-testplan.md` exists and contains at least one test case row +5. `07-testplan.md` exists and contains at least one test case entry 6. Every non-`[DOCS]` story file has a `## Test Case References` section If structural issues are found, fix them before proceeding. Do not diff --git a/design/skills/publish.md b/design/skills/publish.md index cd71c14..99d7796 100644 --- a/design/skills/publish.md +++ b/design/skills/publish.md @@ -155,9 +155,9 @@ Read and follow `../../_shared/recipes/render-provenance-footer.md` with If `.artifacts/design/{issue-key}/07-testplan.md` exists, also copy it. **Sync-manifest guard:** If `.artifacts/design/{issue-key}/sync-manifest.json` -exists, the published testplan's Story column must use Jira keys. Before -copying, read the sync manifest and resolve the Story column in each test -case row (`Story 1.01` → Jira key from manifest). Write the resolved +exists, the published testplan's Story field must use Jira keys. Before +copying, read the sync manifest and resolve the Story field in each test +case's metadata table (`Story 1.01` → Jira key from manifest). Write the resolved version to the docs repo — do NOT modify the local `07-testplan.md`. If `sync-manifest.json` does not exist, copy `07-testplan.md` as-is. diff --git a/design/skills/respond.md b/design/skills/respond.md index cbe382d..b789e93 100644 --- a/design/skills/respond.md +++ b/design/skills/respond.md @@ -165,28 +165,30 @@ feedback), apply them in this order: directed by the approved response. For each change: - **Adding a test case:** Assign the next available sequence number within the requirement group (e.g., if TC-FR2-01 and TC-FR2-02 - exist, the new case is TC-FR2-03). Fill all required fields (ID, - Title, Requirement, Story, Preconditions, Scenario/Steps, Expected - Result, Priority, Automation). Update the testplan's Overview counts - and Summary table. - - **Modifying a test case:** Update the affected fields. If the Story - assignment changes, update both the old and new story's Test Case - References in step 2 below. - - **Removing a test case:** Delete the row. Update the testplan's - Overview counts and Summary table. + exist, the new case is TC-FR2-03). Create the full test case entry: + H4 heading with ID and title, metadata table (Story, Priority, + Automation), and H5 sub-sections (Preconditions, Steps, Expected + Results). Update the testplan's Overview counts and Summary table. + - **Modifying a test case:** Update the affected heading, metadata + table fields, or sub-section content. If the Story assignment + changes, update both the old and new story's Test Case References + in step 2 below. + - **Removing a test case:** Delete the test case entry (heading and + all sub-sections). Update the testplan's Overview counts and + Summary table. 2. **Cascade to story files.** For each affected story (identified by the - Story column of changed test cases): + Story field of changed test cases): - Re-read the story file at `.artifacts/design/{issue-key}/05-stories/epic-{N}/story-{NN}-{slug}.md`. - Rewrite the `## Test Case References` section: collect all TC IDs - from the updated testplan where the Story column matches this story, + from the updated testplan where the Story field matches this story, then write `Verified by: {comma-separated TC IDs}`. - If a story loses all its test cases, write: `Verified by: None (no behavioral test cases after testplan revision)`. 3. **Cascade to coverage matrix.** Re-read - `.artifacts/design/{issue-key}/06-coverage.md`. For each row in the PRD + `.artifacts/design/{issue-key}/06-coverage.md`. For each entry in the PRD Requirement mapping table, update the `Test Cases` column to reflect the current TC IDs from the testplan for that requirement. If a requirement previously had test cases and now has none, flag it in the @@ -258,9 +260,9 @@ metadata — testplan changes are applied locally only. Re-run `/publish` to include the testplan in the docs repo.) **Sync-manifest guard:** If `.artifacts/design/{issue-key}/sync-manifest.json` -exists, the published testplan's Story column must use Jira keys (resolved +exists, the published testplan's Story field must use Jira keys (resolved by `/sync`), not local identifiers. Before copying, read the sync manifest -and resolve the Story column in each test case row: replace local +and resolve the Story field in each test case's metadata table: replace local references (e.g., `Story 1.01`) with their Jira keys from the manifest (e.g., `EDM-1234`). Write the resolved version to the docs repo — do NOT modify the local `07-testplan.md` (it keeps local identifiers). diff --git a/design/skills/revise.md b/design/skills/revise.md index f162667..0967bd7 100644 --- a/design/skills/revise.md +++ b/design/skills/revise.md @@ -204,9 +204,9 @@ If `07-testplan.md` exists and `publish-metadata.json` contains a `testplan_file_path` field, also copy the testplan to the docs repo. **Sync-manifest guard:** If `.artifacts/design/{issue-key}/sync-manifest.json` -exists, the published testplan's Story column must use Jira keys. Before -copying, read the sync manifest and resolve the Story column in each test -case row (`Story 1.01` → Jira key from manifest). Write the resolved +exists, the published testplan's Story field must use Jira keys. Before +copying, read the sync manifest and resolve the Story field in each test +case's metadata table (`Story 1.01` → Jira key from manifest). Write the resolved version to the docs repo — do NOT modify the local `07-testplan.md`. If `sync-manifest.json` does not exist, copy `07-testplan.md` as-is. diff --git a/design/skills/sync.md b/design/skills/sync.md index 9a8da85..f9dc71d 100644 --- a/design/skills/sync.md +++ b/design/skills/sync.md @@ -598,7 +598,7 @@ Fields: ### Step 7: Update Published Testplan If the testplan has been published to the docs repo, update the -published copy's Story column with Jira keys so downstream workflows +published copy's Story field with Jira keys so downstream workflows can filter by Jira key. **Skip this step entirely if any of these are true:** @@ -610,12 +610,11 @@ can filter by Jira key. **Resolve Story references:** 1. Read `.artifacts/design/{issue-key}/07-testplan.md`. -2. For each test case row in the Test Cases tables, resolve the Story - column using the sync manifest: replace local references (e.g., - `Story 1.01`) with their Jira keys (e.g., `EDM-1234`). Use the - reference resolution logic described in the Reference Resolution - section (look up `story-{NN}-*.md` under `epic-{N}/` in the manifest - to find the Jira key). +2. For each test case entry, resolve the Story field in the metadata + table: replace local references (e.g., `Story 1.01`) with their Jira + keys (e.g., `EDM-1234`). Use the reference resolution logic described + in the Reference Resolution section (look up `story-{NN}-*.md` under + `epic-{N}/` in the manifest to find the Jira key). 3. If a Story reference cannot be resolved (not in the manifest), leave it as-is and note it for the user. @@ -624,7 +623,7 @@ can filter by Jira key. Read `.artifacts/prd/config.json` to get the docs repo path. Read `publish-metadata.json` to get the `testplan_file_path`. -Write the resolved testplan content (with Jira keys in the Story column) +Write the resolved testplan content (with Jira keys in the Story field) to `{docs_repo_path}/{testplan_file_path}`. Do NOT modify the local `07-testplan.md` — it keeps local identifiers. diff --git a/implement/skills/ingest.md b/implement/skills/ingest.md index 2f9d914..4a65717 100644 --- a/implement/skills/ingest.md +++ b/implement/skills/ingest.md @@ -143,14 +143,81 @@ Read these from the docs repo: 2. **PRD** (`prd.md`) — the product requirements, with locked decisions reflected in the requirements text 3. **Testplan** (`testplan.md`) — behavioral test cases mapped to PRD - requirements. Valuable context for understanding what scenarios the - feature is expected to pass, but not required. The story's acceptance - criteria and testing approach remain the primary contract. + requirements. If found, proceed to Step 5d for filtering. -If the docs repo documents are not found, ask the user for their location -or proceed with only the Jira story content. The design document, PRD, -and testplan are valuable context but not strictly required — the story's -acceptance criteria are the primary contract. +If the design document or PRD are not found, ask the user for their +location or proceed with only the Jira story content. The design +document and PRD are valuable context but not strictly required — the +story's acceptance criteria are the primary contract. + +#### 5d: Filter Testplan to Story Scope + +If `testplan.md` was found in Step 5c, filter it to the test cases +relevant to this story. The published testplan uses Jira keys in the +Story field in each test case's metadata table (resolved by `/sync`). +Filter by matching the Story field against this story's Jira key +(`{issue-key}`). + +If the Story field still contains local identifiers (e.g., `Story 1.01` +instead of Jira keys), this means `/sync` has not yet been run or the +testplan was published before sync. In this case, look for TC IDs in the +Jira story's Test Case References section (synced from the design +workflow) and match those TC IDs directly against the testplan entries. + +**Three-outcome gate:** + +| Outcome | Condition | Action | +|---------|-----------|--------| +| **Normal** | Matching test cases found | Write `.artifacts/implement/{issue-key}/testplan.md` | +| **Expected zero** | No matches AND story type is `[QE]`, `[DOCS]`, `[UX]`, or `[CI]` | Note in context: "Testplan exists but has no test cases for this story type. This is expected." Do not write `testplan.md`. | +| **Anomalous zero** | No matches AND story type is `[DEV]` or `[UI]` | Warn the user: "Testplan exists but no test cases reference this story. This may indicate a gap in the testplan or an incorrect requirement mapping." This is non-blocking — continue without `testplan.md`. | + +If `testplan.md` was not found in the docs repo, note "No feature-level +testplan available" and continue. This is not an error — the testplan is +a newer feature and older designs may not have one. + +**Write story-scoped testplan (Normal outcome only):** + +Write `.artifacts/implement/{issue-key}/testplan.md`: + +```markdown +# Story Test Plan — {issue-key} + +Source: {docs-repo-path}/testplan.md +Story: {issue-key} — {story-title} +Test cases: {count} + +## TC-FR1-01: {scenario title} + +| Requirement | Priority | Automation | +|-------------|----------|------------| +| FR-1 | high | automated | + +### Preconditions + +- {precondition} + +### Steps + +1. {step} +2. {step} + +### Expected Results + +- {expected outcome} + +## TC-FR1-02: {scenario title} + +{... same structure for each test case ...} +``` + +Each test case becomes an H2 heading with the same structure as the +feature-level testplan (metadata table, Preconditions, Steps, Expected +Results as sub-headings). Heading levels are shifted up by two because +the requirement-grouping layer and the Test Cases section are removed. +The Story field is omitted from the metadata table (redundant — all +entries are for this story). The Requirement ID is included in the +metadata table for traceability back to the feature-level testplan. ### Step 6: Explore the Codebase @@ -269,6 +336,21 @@ If this is a first invocation, write {Which FR-N and NFR-N requirements this story addresses, from the coverage matrix or story metadata.} +### Story Test Plan + +{If testplan.md was written: "Story-scoped test plan with {N} test + cases. See `.artifacts/implement/{issue-key}/testplan.md` for full + details. TC IDs: {comma-separated list}." + + If testplan exists but no matches (expected): "Feature testplan exists + but has no test cases for this {story-type} story (expected)." + + If testplan exists but no matches (anomalous): "Feature testplan exists + but no test cases reference this {story-type} story (anomalous — flagged + during ingest)." + + If testplan not found: "No feature-level testplan available."} + ## Codebase Context ### Affected Components @@ -388,6 +470,7 @@ what changes were found and that the existing context was preserved. ## Output - `.artifacts/implement/{issue-key}/01-context.md` +- `.artifacts/implement/{issue-key}/testplan.md` (if testplan exists with matching test cases) ## When This Phase Is Done @@ -396,6 +479,7 @@ Report your findings: - Affected components and current patterns - Validation profile summary - Dependency warnings (if any) +- Story test plan status (test cases found / expected zero / anomalous zero / no testplan) - Assessment of readiness for `/plan` Then **re-read the controller** (`controller.md`) for next-step guidance. From b4632652c8c3bc98bc884040f19a676c3581b491 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 14:11:23 -0400 Subject: [PATCH 07/38] Add testplan validation gates to implement plan, code, and validate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plan: Test Plan Coverage matrix maps TC IDs to tasks with set-diff completeness gate — every TC ID must be assigned or marked N/A with rationale. Code: Reconciliation gate in Step 3f — after self-review passes, every TC ID mapped to the current task must have a test with assertions matching the Expected Results from testplan.md. Hard gate; floor not ceiling. Validate: Independent Step 7b re-derives TC IDs from testplan.md directly (not from plan's coverage matrix) and verifies each has a test with sufficient assertion depth. Revise: Consistency check now includes Test Plan Coverage mappings when tasks are added, removed, split, or combined. Also renames test-plan.md to testplan.md in ingest for naming consistency. Implement: 0.5.0 -> 0.6.0 Assisted-by: Claude Opus 4.6 (1M) --- implement/SKILL.md | 2 +- implement/skills/code.md | 47 +++++++++++++++++++++++++++++++- implement/skills/plan.md | 22 ++++++++++++++- implement/skills/revise.md | 1 + implement/skills/validate.md | 52 ++++++++++++++++++++++++++++++++++++ 5 files changed, 121 insertions(+), 3 deletions(-) diff --git a/implement/SKILL.md b/implement/SKILL.md index 3bd0eb7..4d90980 100644 --- a/implement/SKILL.md +++ b/implement/SKILL.md @@ -1,6 +1,6 @@ --- name: implement -version: 0.5.0 +version: 0.6.0 description: >- Story-to-code workflow that takes a Jira Story, plans the implementation, writes contract-based tests and production code via TDD, validates against diff --git a/implement/skills/code.md b/implement/skills/code.md index 6570786..ef808cf 100644 --- a/implement/skills/code.md +++ b/implement/skills/code.md @@ -33,7 +33,8 @@ the contract. Commit each logical unit of work independently. Read these files: 1. `.artifacts/implement/{issue-key}/02-plan.md` (implementation plan) 2. `.artifacts/implement/{issue-key}/01-context.md` (story context and validation profile) -3. The project's `AGENTS.md` and/or `CLAUDE.md` (coding conventions) +3. `.artifacts/implement/{issue-key}/testplan.md` (story-scoped testplan, if exists) +4. The project's `AGENTS.md` and/or `CLAUDE.md` (coding conventions) If the plan doesn't exist, tell the user that `/plan` should be run first. @@ -294,6 +295,32 @@ verify the fixes. Only proceed to commit once checks pass. Note any dismissed findings in the implementation report (Discoveries section) so there is a paper trail. +**Test plan reconciliation (if story-scoped testplan exists):** + +After the self-review gate passes, check whether this task has TC IDs +mapped to it in the Test Plan Coverage matrix of `02-plan.md`. If it +does, verify each mapped TC ID before proceeding to commit: + +1. Read the full test case entry for each TC ID from `testplan.md` + (the Preconditions, Steps, and Expected Results sections). +2. Verify that a test exists (written in Step 3b or a prior task) + whose assertions validate the Expected Results described in the + test case. The match is behavioral, not textual — the test must + exercise the described scenario and assert the described outcomes. + The test may use project-specific assertion mechanisms (e.g., + Eventually/Consistently in Ginkgo, polling in pytest). +3. If a TC ID mapped to this task has no corresponding test with + sufficient assertion depth, write the missing test (Step 3b), run + it (Step 3d), re-run the review gate, then re-check. + +This is a hard gate — the task cannot proceed to commit until every +mapped TC ID has coverage. The testplan is a floor, not a ceiling: +tests discovered through TDD that are not tied to any TC ID are +expected and encouraged. + +If no story-scoped testplan exists, or this task has no mapped TC IDs, +skip this check. + #### 3g: Commit The changes are already staged from Step 3f. Create the commit: @@ -443,6 +470,24 @@ After all tasks are complete (or if interrupted), write: {If no integration tests: "No integration tests written — story does not touch component interactions."} +## Test Plan Reconciliation + +{Include only if story-scoped testplan exists. Omit entirely otherwise.} + +| TC ID | Title | Outcome | Notes | +|-------|-------|---------|-------| +| TC-FR1-01 | {title} | verified | Test existed, assertions matched | +| TC-FR1-02 | {title} | written | Test added during task execution | +| TC-NFR1-01 | {title} | N/A | See Deviations from Plan | + +{Every TC ID mapped to a task in the Test Plan Coverage matrix must + appear exactly once. Outcome values: + - verified: test existed and had sufficient assertion depth + - written: test was written to satisfy this TC ID + - strengthened: test existed but lacked assertions; added them + - N/A: test case found inapplicable (explain in Notes and in + Deviations from Plan)} + ## Coverage Notes {Qualitative assessment of what behavioral paths are covered and any diff --git a/implement/skills/plan.md b/implement/skills/plan.md index d1dd03b..4b79be5 100644 --- a/implement/skills/plan.md +++ b/implement/skills/plan.md @@ -30,7 +30,8 @@ review checkpoint before any code is written. Read these files in order: 1. `.artifacts/implement/{issue-key}/01-context.md` (story context) -2. The project's `AGENTS.md` and/or `CLAUDE.md` (coding conventions) +2. `.artifacts/implement/{issue-key}/testplan.md` (story-scoped testplan, if exists) +3. The project's `AGENTS.md` and/or `CLAUDE.md` (coding conventions) If `01-context.md` doesn't exist, tell the user that `/ingest` should be run first. @@ -174,6 +175,24 @@ Write `.artifacts/implement/{issue-key}/02-plan.md` with this structure: {Every AC must appear in at least one task. Flag any gaps.} +## Test Plan Coverage + +{Include this section only if `.artifacts/implement/{issue-key}/testplan.md` + exists. If no story-scoped testplan: omit this section entirely.} + +| TC ID | Title | Covered by Task | Notes | +|-------|-------|-----------------|-------| +| TC-FR1-01 | {title} | Task 2 | | +| TC-FR1-02 | {title} | Task 3 | | +| TC-NFR1-01 | {title} | N/A | {rationale} | + +{Every TC ID from testplan.md must appear. Each must be assigned to a + task or marked N/A with a rationale (e.g., "NFR verified by load test + infrastructure, not unit/integration tests"). This is a set-diff gate: + compute the difference between the set of TC IDs in testplan.md and + the set assigned to tasks or marked N/A. If the difference is + non-empty, the plan is incomplete — resolve before proceeding.} + ## Risk Assessment {Things the plan author is uncertain about. Ordered by impact.} @@ -201,6 +220,7 @@ Before presenting the plan, verify: - [ ] No tasks modify code outside the story's scope - [ ] Task count is reasonable — if you have more than 10 tasks, consider whether the story needs re-scoping - [ ] The plan is achievable — no tasks depend on unavailable infrastructure or unmerged code +- [ ] If story-scoped testplan exists: every TC ID is assigned to a task or marked N/A with rationale (Test Plan Coverage set-diff is clean) ### Step 6: Present to User diff --git a/implement/skills/revise.md b/implement/skills/revise.md index 4a9fd67..cb9c270 100644 --- a/implement/skills/revise.md +++ b/implement/skills/revise.md @@ -87,6 +87,7 @@ After applying changes, verify: - Does the test strategy still align with the tasks? - Do interface definitions match what the tasks describe? - Are commit messages still properly formatted? +- If a Test Plan Coverage section exists: do the TC ID → Task mappings still reflect the current task breakdown? If tasks were added, removed, split, or combined, update the Covered by Task column accordingly. ### Step 5: Update Artifact diff --git a/implement/skills/validate.md b/implement/skills/validate.md index 5a499c2..62cabee 100644 --- a/implement/skills/validate.md +++ b/implement/skills/validate.md @@ -33,6 +33,7 @@ Read: 1. `.artifacts/implement/{issue-key}/01-context.md` (validation profile) 2. `.artifacts/implement/{issue-key}/02-plan.md` (what was implemented) 3. `.artifacts/implement/{issue-key}/04-impl-report.md` (implementation status) +4. `.artifacts/implement/{issue-key}/testplan.md` (story-scoped testplan, if exists) Extract the validation profile's pre-PR checks list. @@ -245,6 +246,36 @@ satisfied: it requires manual verification or describes a UX quality) — note it as "requires manual verification" with an explanation of why +### Step 7b: Test Plan Verification + +If `.artifacts/implement/{issue-key}/testplan.md` exists, independently +verify that every test case has been implemented. This check re-derives +the required TC ID list from `testplan.md` directly — it does NOT rely +on the plan's task-to-TC-ID mappings or `/code`'s per-task reconciliation +for determining which tests exist. The only information carried forward +from the plan is the N/A rationale for test cases marked inapplicable. +This is an intentional independent verification: if the plan's +bookkeeping or the code phase's gate drifted from reality, this step +catches it. + +If `testplan.md` does not exist, skip this step entirely. + +1. Read `testplan.md` and extract all TC IDs. +2. For each TC ID (except those the validate phase agrees are + legitimately N/A based on the rationale in the plan's Test Plan + Coverage matrix): + - Search the test files on the feature branch for a test whose + scenario matches the TC's Steps and whose assertions match the + Expected Results. + - The match is behavioral, not textual — the test must exercise the + described scenario and assert the described outcomes. + - Record the test file and test name for each TC ID. +3. If any TC ID lacks a corresponding test: + - Write the missing test following contract-based testing standards. + - Commit the test following the project's commit format. + - Re-run the relevant checks from Step 3. +4. Record results for the validation report. + ### Step 8: Write Validation Report Write `.artifacts/implement/{issue-key}/05-validation-report.md`: @@ -313,6 +344,26 @@ Write `.artifacts/implement/{issue-key}/05-validation-report.md`: If any gaps: describe what's missing and what was done about it. If any require manual verification: list them with rationale.} +## Test Plan Verification + +{Include only if testplan.md exists. Omit entirely otherwise.} + +| TC ID | Title | Test File | Test Name | Status | +|-------|-------|-----------|-----------|--------| +| TC-FR1-01 | {title} | {file} | {test name} | covered | +| TC-FR1-02 | {title} | {file} | {test name} | gap-filled | +| TC-NFR1-01 | {title} | — | — | N/A | + +{Status values: + - covered: test existed and had sufficient assertion depth + - gap-filled: test was written during validation + - N/A: test case legitimately not applicable (rationale required) + + If all covered: "All test plan cases verified." + If gaps were filled: "{N} test cases required additional tests during + validation." + If any N/A: list rationale for each.} + ## Quality Review Findings {Findings from the code quality review gate (protocol criteria plus @@ -349,6 +400,7 @@ Summarize for the user: - Which checks passed and which failed - Coverage assessment (behavioral, not numeric) - Acceptance criteria status (all satisfied, or which ones have gaps) +- Test plan verification status (all covered / gaps filled / not applicable), if testplan exists - Any tests added during validation - Any regressions found (and whether they were fixed) - Overall verdict: ready for `/publish` or not From 98fe72f16e456ded30ab2bcb702193ce9db22e6f Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 14:19:57 -0400 Subject: [PATCH 08/38] Add testplan filtering and validation gates to e2e workflow Same testplan gate pattern as implement, adapted for e2e semantics: Ingest: Filters testplan by PRD requirement IDs from the [QE] story's Design Reference (not by Story field, since testplan maps test cases to [DEV] stories). Three-outcome gate with [QE] in anomalous-zero. Plan: Test Plan Coverage matrix maps TC IDs to test scenarios (C1/S1) with set-diff completeness gate. Code: Reconciliation gate in Step 3e checks scenario validations against testplan Expected Results before commit. Validate: Independent Step 7b re-derives TC IDs from testplan.md directly and verifies each has a test with sufficient assertion depth. Revise: Consistency check includes Test Plan Coverage mappings. E2e: 0.4.0 -> 0.5.0 Assisted-by: Claude Opus 4.6 (1M) --- e2e/SKILL.md | 2 +- e2e/skills/code.md | 46 +++++++++++++++++++- e2e/skills/ingest.md | 99 ++++++++++++++++++++++++++++++++++++++---- e2e/skills/plan.md | 23 +++++++++- e2e/skills/revise.md | 1 + e2e/skills/validate.md | 50 +++++++++++++++++++++ 6 files changed, 209 insertions(+), 12 deletions(-) diff --git a/e2e/SKILL.md b/e2e/SKILL.md index 0f662bd..fa6a2b8 100644 --- a/e2e/SKILL.md +++ b/e2e/SKILL.md @@ -1,6 +1,6 @@ --- name: e2e -version: 0.4.0 +version: 0.5.0 description: >- Story-to-e2e-test workflow that takes a Jira [QE] Story, discovers the project's e2e testing infrastructure, plans test scenarios, writes e2e diff --git a/e2e/skills/code.md b/e2e/skills/code.md index a45a0f7..c31ad76 100644 --- a/e2e/skills/code.md +++ b/e2e/skills/code.md @@ -32,7 +32,8 @@ each logical unit of work independently. Read these files: 1. `.artifacts/e2e/{issue-key}/02-plan.md` (test plan) 2. `.artifacts/e2e/{issue-key}/01-context.md` (story context and e2e infrastructure) -3. The project's `AGENTS.md` and/or `CLAUDE.md` (coding conventions) +3. `.artifacts/e2e/{issue-key}/testplan.md` (story-scoped testplan, if exists) +4. The project's `AGENTS.md` and/or `CLAUDE.md` (coding conventions) If the plan doesn't exist, tell the user that `/plan` should be run first. @@ -256,6 +257,31 @@ the task-scoped tests (Step 3c) and fast quality checks (Step 3d) to verify the fixes. Only proceed to commit once checks pass. Note any dismissed findings in the implementation report (Discoveries section). +**Test plan reconciliation (if story-scoped testplan exists):** + +After the self-review gate passes, check whether this task's scenario(s) +have TC IDs mapped to them in the Test Plan Coverage matrix of +`02-plan.md`. If so, verify each mapped TC ID before proceeding to +commit: + +1. Read the full test case entry for each TC ID from `testplan.md` + (the Preconditions, Steps, and Expected Results sections). +2. Verify that the test scenario's validations assert the Expected + Results described in the test case. The match is behavioral, not + textual — the test must exercise the described scenario and assert + the described outcomes. +3. If a TC ID mapped to this task has no corresponding validation with + sufficient assertion depth, write the missing validation (Step 3b), + run the tests (Step 3c), re-run the review gate, then re-check. + +This is a hard gate — the task cannot proceed to commit until every +mapped TC ID has coverage. The testplan is a floor, not a ceiling: +validations discovered through test design that are not tied to any +TC ID are expected and encouraged. + +If no story-scoped testplan exists, or this task has no mapped TC IDs, +skip this check. + #### 3f: Commit The changes are already staged from Step 3e. Create the commit: @@ -398,6 +424,24 @@ After all tasks are complete (or if interrupted), write: are self-starting, note that. If tests run against a pre-existing environment: "Tests run against {environment}."} +## Test Plan Reconciliation + +{Include only if story-scoped testplan exists. Omit entirely otherwise.} + +| TC ID | Title | Outcome | Notes | +|-------|-------|---------|-------| +| TC-FR1-01 | {title} | verified | Scenario validation matched | +| TC-FR1-02 | {title} | written | Validation added during task execution | +| TC-NFR1-01 | {title} | N/A | See Deviations from Plan | + +{Every TC ID mapped to a scenario in the Test Plan Coverage matrix must + appear exactly once. Outcome values: + - verified: validation existed and had sufficient assertion depth + - written: validation was written to satisfy this TC ID + - strengthened: validation existed but lacked assertions; added them + - N/A: test case found inapplicable (explain in Notes and in + Deviations from Plan)} + ## Notes {Any qualitative observations about test coverage, gaps, or patterns.} diff --git a/e2e/skills/ingest.md b/e2e/skills/ingest.md index 92829fb..56277b1 100644 --- a/e2e/skills/ingest.md +++ b/e2e/skills/ingest.md @@ -152,15 +152,80 @@ Read these from the docs repo: 2. **PRD** (`prd.md`) — the product requirements, with locked decisions reflected in the requirements text 3. **Testplan** (`testplan.md`) — behavioral test cases mapped to PRD - requirements. For `[QE]` stories, this is particularly valuable: the - testplan's test cases for the relevant requirements provide structured - starting points for e2e scenario design. Cross-reference test case - IDs in `02-plan.md` when scenarios align. + requirements. If found, proceed to Step 5d for filtering. -If the docs repo documents are not found, ask the user for their location -or proceed with only the Jira story content. The design document, PRD, -and testplan are valuable context but not strictly required — the story's -acceptance criteria are the primary contract. +If the design document or PRD are not found, ask the user for their +location or proceed with only the Jira story content. The design +document and PRD are valuable context but not strictly required — the +story's acceptance criteria are the primary contract. + +#### 5d: Filter Testplan to Story Scope + +If `testplan.md` was found in Step 5c, filter it to the test cases +relevant to this story's scope. + +For `[QE]` stories, the testplan's test cases reference `[DEV]` stories +(the testplan maps test cases to implementing stories, not to `[QE]` +stories). Filter by **requirement**: extract the PRD requirement IDs +from the story's Design Reference section (e.g., `FR-1, FR-3, NFR-2`), +then collect all test cases from the testplan whose requirement heading +matches any of those IDs. + +If the story's Design Reference does not list PRD requirements, fall +back to looking for TC IDs in the Jira story's Test Case References +section and matching those directly against the testplan. + +**Three-outcome gate:** + +| Outcome | Condition | Action | +|---------|-----------|--------| +| **Normal** | Matching test cases found | Write `.artifacts/e2e/{issue-key}/testplan.md` | +| **Expected zero** | No matches AND story type is `[DOCS]`, `[UX]`, or `[CI]` | Note in context. Do not write `testplan.md`. | +| **Anomalous zero** | No matches AND story type is `[QE]`, `[DEV]`, or `[UI]` | Warn the user: "Testplan exists but no test cases match this story's requirements. This may indicate a gap in the testplan or the story's requirement mapping." This is non-blocking — continue without `testplan.md`. | + +If `testplan.md` was not found in the docs repo, note "No feature-level +testplan available" and continue. This is not an error — the testplan is +a newer feature and older designs may not have one. + +**Write story-scoped testplan (Normal outcome only):** + +Write `.artifacts/e2e/{issue-key}/testplan.md`: + +```markdown +# Story Test Plan — {issue-key} + +Source: {docs-repo-path}/testplan.md +Story: {issue-key} — {story-title} +Test cases: {count} + +## TC-FR1-01: {scenario title} + +| Requirement | Story | Priority | Automation | +|-------------|-------|----------|------------| +| FR-1 | {DEV story Jira key} | high | automated | + +### Preconditions + +- {precondition} + +### Steps + +1. {step} +2. {step} + +### Expected Results + +- {expected outcome} + +## TC-FR1-02: {scenario title} + +{... same structure for each test case ...} +``` + +Unlike the implement workflow's story-scoped testplan, this version +includes the Story field in the metadata table — it identifies the +`[DEV]` story that implements the behavior this `[QE]` story will +test, which is not redundant here. ### Step 6: Explore E2E Test Infrastructure @@ -433,6 +498,22 @@ If this is a first invocation, write {Which FR-N and NFR-N requirements this story's tests will validate.} +### Story Test Plan + +{If testplan.md was written: "Story-scoped test plan with {N} test + cases covering requirements {FR-1, FR-3, NFR-2}. See + `.artifacts/e2e/{issue-key}/testplan.md` for full details. + TC IDs: {comma-separated list}." + + If testplan exists but no matches (expected): "Feature testplan exists + but has no test cases for this {story-type} story (expected)." + + If testplan exists but no matches (anomalous): "Feature testplan exists + but no test cases match this {story-type} story's requirements + (anomalous — flagged during ingest)." + + If testplan not found: "No feature-level testplan available."} + ## E2E Test Infrastructure ### Framework @@ -628,6 +709,7 @@ what changes were found and that the existing context was preserved. ## Output - `.artifacts/e2e/{issue-key}/01-context.md` +- `.artifacts/e2e/{issue-key}/testplan.md` (if testplan exists with matching test cases) ## When This Phase Is Done @@ -636,6 +718,7 @@ Report your findings: - Dependency status ([DEV] stories merged or not) - E2E infrastructure discovered (framework, test abstractions, reference suite) - Validation profile summary +- Story test plan status (test cases found / expected zero / anomalous zero / no testplan) - Assessment of readiness for `/plan` Then **re-read the controller** (`controller.md`) for next-step guidance. diff --git a/e2e/skills/plan.md b/e2e/skills/plan.md index 86d1a01..1f81410 100644 --- a/e2e/skills/plan.md +++ b/e2e/skills/plan.md @@ -32,8 +32,9 @@ review checkpoint before any test code is written. Read these files in order: 1. `.artifacts/e2e/{issue-key}/01-context.md` (story context and e2e infrastructure) -2. The project's `AGENTS.md` and/or `CLAUDE.md` (coding conventions) -3. Any test-specific documentation referenced in the context (test READMEs, guidelines) +2. `.artifacts/e2e/{issue-key}/testplan.md` (story-scoped testplan, if exists) +3. The project's `AGENTS.md` and/or `CLAUDE.md` (coding conventions) +4. Any test-specific documentation referenced in the context (test READMEs, guidelines) If `01-context.md` doesn't exist, tell the user that `/ingest` should be run first. @@ -276,6 +277,23 @@ Write `.artifacts/e2e/{issue-key}/02-plan.md` with this structure: {Every AC must appear in at least one scenario. Consolidated scenarios will appear in multiple AC rows — this is expected. Flag any gaps.} +## Test Plan Coverage + +{Include this section only if `.artifacts/e2e/{issue-key}/testplan.md` + exists. If no story-scoped testplan: omit this section entirely.} + +| TC ID | Title | Covered by Scenario | Notes | +|-------|-------|---------------------|-------| +| TC-FR1-01 | {title} | C1 | | +| TC-FR1-02 | {title} | S1 | | +| TC-NFR1-01 | {title} | N/A | {rationale} | + +{Every TC ID from testplan.md must appear. Each must be assigned to a + scenario or marked N/A with a rationale. This is a set-diff gate: + compute the difference between the set of TC IDs in testplan.md and + the set assigned to scenarios or marked N/A. If the difference is + non-empty, the plan is incomplete — resolve before proceeding.} + ## Risk Assessment {Things the plan author is uncertain about. Ordered by impact.} @@ -308,6 +326,7 @@ Before presenting the plan, verify: - [ ] Each validation in a consolidated scenario is tagged with its source AC - [ ] Scenario identifiers and titles are unique across the plan (no duplicate C#/S# or repeated names) - [ ] The plan is achievable — no scenarios depend on unmerged features or unavailable test infrastructure methods +- [ ] If story-scoped testplan exists: every TC ID is assigned to a scenario or marked N/A with rationale (Test Plan Coverage set-diff is clean) ### Step 7: Present to User diff --git a/e2e/skills/revise.md b/e2e/skills/revise.md index fc61eca..11052ac 100644 --- a/e2e/skills/revise.md +++ b/e2e/skills/revise.md @@ -96,6 +96,7 @@ After applying changes, verify: - Are scenario identifiers and titles unique across the plan (no duplicate C#/S# or repeated names)? - Does the Scenario Consolidation table still match the current scenario list? - Are commit messages still properly formatted? +- If a Test Plan Coverage section exists: do the TC ID → Scenario mappings still reflect the current scenario breakdown? If scenarios were added, removed, split, or combined, update the Covered by Scenario column accordingly. ### Step 5: Update Artifact diff --git a/e2e/skills/validate.md b/e2e/skills/validate.md index ad7ee53..d4aa562 100644 --- a/e2e/skills/validate.md +++ b/e2e/skills/validate.md @@ -33,6 +33,7 @@ Read: 1. `.artifacts/e2e/{issue-key}/01-context.md` (validation profile and e2e infrastructure) 2. `.artifacts/e2e/{issue-key}/02-plan.md` (what was planned) 3. `.artifacts/e2e/{issue-key}/04-impl-report.md` (implementation status) +4. `.artifacts/e2e/{issue-key}/testplan.md` (story-scoped testplan, if exists) Extract the validation profile's pre-PR checks list and the e2e test execution command. @@ -230,6 +231,34 @@ covered: requires manual measurement) — note it as "not e2e-testable" with an explanation of why +### Step 7b: Test Plan Verification + +If `.artifacts/e2e/{issue-key}/testplan.md` exists, independently verify +that every test case has been covered by the e2e tests. This check +re-derives the required TC ID list from `testplan.md` directly — it does +NOT rely on the plan's task-to-TC-ID mappings or `/code`'s per-task +reconciliation for determining which tests exist. The only information +carried forward from the plan is the N/A rationale for test cases marked +inapplicable. + +If `testplan.md` does not exist, skip this step entirely. + +1. Read `testplan.md` and extract all TC IDs. +2. For each TC ID (except those the validate phase agrees are + legitimately N/A based on the rationale in the plan's Test Plan + Coverage matrix): + - Search the e2e test files on the feature branch for a test + scenario whose validations match the TC's Steps and Expected + Results. + - The match is behavioral, not textual — the test must exercise the + described scenario and assert the described outcomes. + - Record the test file and scenario name for each TC ID. +3. If any TC ID lacks a corresponding test: + - Write the missing test following the reference suite's patterns. + - Commit the test following the project's commit format. + - Re-run the relevant checks from Step 3. +4. Record results for the validation report. + ### Step 8: Write Validation Report Write `.artifacts/e2e/{issue-key}/05-validation-report.md`: @@ -282,6 +311,26 @@ Write `.artifacts/e2e/{issue-key}/05-validation-report.md`: If any gaps: describe what's missing and what was done about it. If any not e2e-testable: list them with rationale.} +## Test Plan Verification + +{Include only if testplan.md exists. Omit entirely otherwise.} + +| TC ID | Title | Test File | Scenario | Status | +|-------|-------|-----------|----------|--------| +| TC-FR1-01 | {title} | {file} | {scenario name} | covered | +| TC-FR1-02 | {title} | {file} | {scenario name} | gap-filled | +| TC-NFR1-01 | {title} | — | — | N/A | + +{Status values: + - covered: test scenario existed and had sufficient assertion depth + - gap-filled: test scenario was written during validation + - N/A: test case legitimately not applicable (rationale required) + + If all covered: "All test plan cases verified." + If gaps were filled: "{N} test cases required additional tests during + validation." + If any N/A: list rationale for each.} + ## Quality Review Findings {Findings from the code quality review gate (protocol criteria plus @@ -318,6 +367,7 @@ Summarize for the user: - Which checks passed and which failed - Anti-pattern check results (clean or what was fixed) - Acceptance criteria coverage (all covered, or which ones have gaps) +- Test plan verification status (all covered / gaps filled / not applicable), if testplan exists - Any regressions found (and whether they were fixed) - Overall verdict: ready for `/publish` or not From 94387ecf6fa718ebcc4f127d559c1d1fe3dd2b2f Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 14:49:08 -0400 Subject: [PATCH 09/38] Address review findings on testplan branch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixes: - Add /respond as writer for story files and coverage matrix in controller artifact table (finding 1) - Specify Verified-by format for Test Case References in decompose Step 7c (finding 2) - Add testplan Gaps section update to revise consistency check (finding 3) - Add independent-verification rationale to e2e validate Step 7b (finding 4) - Add assertion-mechanism note to e2e code reconciliation gate (finding 5) - Update README phases table and typical flow to reflect respond's testplan outputs (finding 6) Finding 7 (e2e version 0.5.0 vs 0.4.0) not applied — the version jump reflects two minor changes (ingest filtering + gate chain across four phases) landing as one atomic feature, not a single minor tweak. Assisted-by: Claude Opus 4.6 (1M) --- design/README.md | 5 +++-- design/skills/controller.md | 4 ++-- design/skills/decompose.md | 18 ++++++++++++++++++ design/skills/revise.md | 2 ++ e2e/skills/code.md | 4 +++- e2e/skills/validate.md | 4 +++- 6 files changed, 31 insertions(+), 6 deletions(-) diff --git a/design/README.md b/design/README.md index 700f0ae..475cf4d 100644 --- a/design/README.md +++ b/design/README.md @@ -38,7 +38,7 @@ graph TD | Decompose | `/decompose` | Break into epics and stories | `04-epics.md`, `05-stories/`, `06-coverage.md`, `07-testplan.md` | | Revise | `/revise` | Incorporate feedback | Updated design and/or stories | | Publish | `/publish` | Post design doc as GitHub PR | `08-pr-description.md` | -| Respond | `/respond` | Address reviewer comments | `09-review-responses.md` | +| Respond | `/respond` | Address reviewer comments | `09-review-responses.md`, updated `07-testplan.md` / stories / `06-coverage.md` | | Sync | `/sync` | Sync Jira epics and stories | `sync-manifest.json` | ## Typical Flow @@ -78,7 +78,8 @@ graph TD /respond → fetches PR review comments → proposes responses (user approves before posting) - → updates design document if needed + → updates design document and testplan if needed + → cascades testplan changes to story Test Case References and coverage matrix → repeatable /sync diff --git a/design/skills/controller.md b/design/skills/controller.md index 19598f5..cb1f51a 100644 --- a/design/skills/controller.md +++ b/design/skills/controller.md @@ -56,8 +56,8 @@ the source repo (this directory should be gitignored in the source repo): | Provenance log | `provenance.json` | `/draft`, `/revise`, `/respond` | | Epic metadata | `04-epics.md` | `/decompose`, `/revise` | | Epic files | `05-stories/epic-{N}-{slug}.md` | `/decompose`, `/revise` | -| Story files | `05-stories/epic-{N}/story-{NN}-{slug}.md` | `/decompose`, `/revise` | -| Coverage matrix | `06-coverage.md` | `/decompose`, `/revise` | +| Story files | `05-stories/epic-{N}/story-{NN}-{slug}.md` | `/decompose`, `/revise`, `/respond` | +| Coverage matrix | `06-coverage.md` | `/decompose`, `/revise`, `/respond` | | Testplan | `07-testplan.md` | `/decompose`, `/revise`, `/respond` | | PR description | `08-pr-description.md` | `/publish` | | Publish metadata | `publish-metadata.json` | `/publish` | diff --git a/design/skills/decompose.md b/design/skills/decompose.md index 1c65fb9..99ef71b 100644 --- a/design/skills/decompose.md +++ b/design/skills/decompose.md @@ -561,6 +561,24 @@ for each story and write them into that story's file. Re-read each story file before appending to ensure current content. `[DOCS]` stories do not receive this section. +The section format is a single `Verified by:` line with comma-separated +TC IDs: + +```markdown +## Test Case References + +Verified by: TC-FR1-01, TC-FR1-02, TC-NFR1-01 +``` + +For stories with no behavioral test cases (e.g., infrastructure +prerequisites): + +```markdown +## Test Case References + +Verified by: None (infrastructure — no behavioral test cases) +``` + ### Step 8: Write Coverage Matrix Write `.artifacts/design/{issue-key}/06-coverage.md`: diff --git a/design/skills/revise.md b/design/skills/revise.md index 0967bd7..29b71fd 100644 --- a/design/skills/revise.md +++ b/design/skills/revise.md @@ -133,6 +133,8 @@ After applying changes, verify: referencing that story — do they still validate the correct behavior? - Update the coverage matrix's Test Cases column if test case IDs changed. +- If test cases were removed and a requirement now has none: update + the testplan's Gaps section to reflect the new gap. - If requirement IDs changed (rare — requires PRD revision): update all TC IDs anchored to the changed requirement. diff --git a/e2e/skills/code.md b/e2e/skills/code.md index c31ad76..aaae90d 100644 --- a/e2e/skills/code.md +++ b/e2e/skills/code.md @@ -269,7 +269,9 @@ commit: 2. Verify that the test scenario's validations assert the Expected Results described in the test case. The match is behavioral, not textual — the test must exercise the described scenario and assert - the described outcomes. + the described outcomes. The test may use project-specific assertion + mechanisms (e.g., Eventually/Consistently in Ginkgo, polling in + pytest). 3. If a TC ID mapped to this task has no corresponding validation with sufficient assertion depth, write the missing validation (Step 3b), run the tests (Step 3c), re-run the review gate, then re-check. diff --git a/e2e/skills/validate.md b/e2e/skills/validate.md index d4aa562..2289978 100644 --- a/e2e/skills/validate.md +++ b/e2e/skills/validate.md @@ -239,7 +239,9 @@ re-derives the required TC ID list from `testplan.md` directly — it does NOT rely on the plan's task-to-TC-ID mappings or `/code`'s per-task reconciliation for determining which tests exist. The only information carried forward from the plan is the N/A rationale for test cases marked -inapplicable. +inapplicable. This is an intentional independent verification: if the +plan's bookkeeping or the code phase's gate drifted from reality, this +step catches it. If `testplan.md` does not exist, skip this step entirely. From 80a799346c003a1d3bee06e611a82db225ae7608 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 15:08:04 -0400 Subject: [PATCH 10/38] Add AC reference field to test case metadata Each test case now maps to specific acceptance criteria on the implementing story (e.g., AC-1, AC-3), enabling fine-grained traceability: requirement -> test case -> story AC. Without this, requirement-level traceability could miss untested ACs when multiple stories cover the same requirement. - AC field added to metadata table in field definitions and all template examples (decompose, implement ingest, e2e ingest) - Coverage target strengthened to include AC-level completeness - Gaps section template updated for AC-level gap reporting - Step 7a adds AC completeness verification during generation - Decomposition review checks AC coverage in Testplan Quality - Respond test case creation mentions AC field Assisted-by: Claude Opus 4.6 (1M) --- design/decomposition-review.md | 4 ++- design/skills/decompose.md | 51 +++++++++++++++++++++++----------- design/skills/respond.md | 2 +- e2e/skills/ingest.md | 6 ++-- implement/skills/ingest.md | 6 ++-- 5 files changed, 45 insertions(+), 24 deletions(-) diff --git a/design/decomposition-review.md b/design/decomposition-review.md index 2e29eac..40b18fc 100644 --- a/design/decomposition-review.md +++ b/design/decomposition-review.md @@ -109,7 +109,9 @@ impact: high)? Do the `Test Case References` in story files match the testplan's Story field? Does the coverage matrix's Test Cases column match the testplan? Are there requirements with stories but no test - cases (testplan gaps)? + cases (testplan gaps)? Does every story AC appear in at least one + test case's AC field? Are there story ACs with no test case mapping + (AC-level gaps)? 6. **Integration Stability** — Would implementing all stories in dependency order produce a working feature end-to-end? Does any diff --git a/design/skills/decompose.md b/design/skills/decompose.md index 99ef71b..c955d60 100644 --- a/design/skills/decompose.md +++ b/design/skills/decompose.md @@ -387,6 +387,10 @@ it (from the epic/story files written in Step 5). For each story: describes an observable outcome verifiable against a running system. Do not generate unit-test-level entries — tracing unit tests to testplan entries is impractical and fragile. +4. After generating test cases for a story, verify that every AC on that + story appears in at least one test case's AC field (see AC mapping + rules below). If an AC has no test case, either add one or flag it + in the Gaps section. **Test case ID scheme:** `TC-{requirement-id}-{sequence}`, where `{requirement-id}` is the PRD requirement ID with the hyphen removed @@ -400,7 +404,7 @@ identified by the parent section heading): | Field | Description | |-------|-------------| | Test Case ID and Title | H4 heading: `#### TC-{req}-{NN}: {one-line scenario description}` | -| Story, Priority, Automation | Single metadata table beneath the H4 heading | +| Story, AC, Priority, Automation | Single metadata table beneath the H4 heading | | Preconditions | H5 section: system state required before the test | | Steps | H5 section: what the tester does — numbered steps | | Expected Results | H5 section: observable outcomes the tester verifies | @@ -418,9 +422,19 @@ identified by the parent section heading): hardware interaction, exploratory testing, or scenarios explicitly scoped as manual in a `[QE]` story) +**AC mapping:** +- `AC` lists the acceptance criteria from the implementing story that + this test case validates (e.g., `AC-1`, `AC-1, AC-3`). +- Use the numbering from the story's Acceptance Criteria section + (first criterion = AC-1, second = AC-2, etc.). +- A test case must map to at least one AC. A single test case may + cover multiple ACs if they describe aspects of the same scenario. + **Coverage target:** Every FR and NFR covered by at least one story -should have at least one test case. A covered requirement with no test -cases is a gap — flag it in the testplan's Gaps section. +should have at least one test case. Within each story, every acceptance +criterion should be referenced by at least one test case's AC field. A +story AC with no test case mapping is a gap — flag it in the testplan's +Gaps section. **Negative scenarios:** Include negative/error test cases where the PRD or design specifies error handling behavior. Do not invent error @@ -445,9 +459,9 @@ Requirements covered: {N} of {total FR + NFR count} #### TC-FR1-01: {scenario title} -| Story | Priority | Automation | -|-------|----------|------------| -| Story 1.01 | high | automated | +| Story | AC | Priority | Automation | +|-------|-----|----------|------------| +| Story 1.01 | AC-1 | high | automated | ##### Preconditions @@ -464,9 +478,9 @@ Requirements covered: {N} of {total FR + NFR count} #### TC-FR1-02: {scenario title} -| Story | Priority | Automation | -|-------|----------|------------| -| Story 1.02 | medium | automated | +| Story | AC | Priority | Automation | +|-------|-----|----------|------------| +| Story 1.02 | AC-2 | medium | automated | ##### Preconditions @@ -485,9 +499,9 @@ Requirements covered: {N} of {total FR + NFR count} #### TC-FR2-01: {scenario title} -| Story | Priority | Automation | -|-------|----------|------------| -| Story 2.01 | critical | automated | +| Story | AC | Priority | Automation | +|-------|-----|----------|------------| +| Story 2.01 | AC-1, AC-3 | critical | automated | ##### Preconditions @@ -506,9 +520,9 @@ Requirements covered: {N} of {total FR + NFR count} #### TC-NFR1-01: {scenario title} -| Story | Priority | Automation | -|-------|----------|------------| -| Story 1.02 | high | automated | +| Story | AC | Priority | Automation | +|-------|-----|----------|------------| +| Story 1.02 | AC-1 | high | automated | ##### Preconditions @@ -530,7 +544,12 @@ Requirements covered: {N} of {total FR + NFR count} story (already flagged in the coverage matrix): note "Not testable — no implementing story." - If no gaps: "All covered requirements have test cases."} + For each story AC with no test case mapping: identify the story, the + uncovered AC, and a recommendation (add a test case, or justify why + the AC does not warrant a behavioral test case). + + If no gaps: "All covered requirements have test cases and all story + ACs are mapped to test cases."} ## Summary diff --git a/design/skills/respond.md b/design/skills/respond.md index b789e93..19ef655 100644 --- a/design/skills/respond.md +++ b/design/skills/respond.md @@ -166,7 +166,7 @@ feedback), apply them in this order: - **Adding a test case:** Assign the next available sequence number within the requirement group (e.g., if TC-FR2-01 and TC-FR2-02 exist, the new case is TC-FR2-03). Create the full test case entry: - H4 heading with ID and title, metadata table (Story, Priority, + H4 heading with ID and title, metadata table (Story, AC, Priority, Automation), and H5 sub-sections (Preconditions, Steps, Expected Results). Update the testplan's Overview counts and Summary table. - **Modifying a test case:** Update the affected heading, metadata diff --git a/e2e/skills/ingest.md b/e2e/skills/ingest.md index 56277b1..1bc7821 100644 --- a/e2e/skills/ingest.md +++ b/e2e/skills/ingest.md @@ -200,9 +200,9 @@ Test cases: {count} ## TC-FR1-01: {scenario title} -| Requirement | Story | Priority | Automation | -|-------------|-------|----------|------------| -| FR-1 | {DEV story Jira key} | high | automated | +| Requirement | Story | AC | Priority | Automation | +|-------------|-------|-----|----------|------------| +| FR-1 | {DEV story Jira key} | AC-1 | high | automated | ### Preconditions diff --git a/implement/skills/ingest.md b/implement/skills/ingest.md index 4a65717..c253b7d 100644 --- a/implement/skills/ingest.md +++ b/implement/skills/ingest.md @@ -189,9 +189,9 @@ Test cases: {count} ## TC-FR1-01: {scenario title} -| Requirement | Priority | Automation | -|-------------|----------|------------| -| FR-1 | high | automated | +| Requirement | AC | Priority | Automation | +|-------------|-----|----------|------------| +| FR-1 | AC-1 | high | automated | ### Preconditions From bee381816a03a81e15450286eb55fbc5fce7fcd4 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 15:11:54 -0400 Subject: [PATCH 11/38] Add Expected Results quality gate with banned vague phrases The generation step now prevents common vague language in Expected Results rather than relying solely on the decomposition review to catch it. Banned phrases include "works correctly", "handles appropriately", "is validated", "completes successfully", and others. Each expected result must state a specific observable outcome. - Quality gate added to decompose Step 7a with banned phrases list - Decomposition review references banned phrases in Testplan Quality - Respond and revise reference the quality gate when modifying test cases, ensuring reviewer-requested changes don't introduce vagueness Assisted-by: Claude Opus 4.6 (1M) --- design/decomposition-review.md | 7 +++++-- design/skills/decompose.md | 18 ++++++++++++++++++ design/skills/respond.md | 3 ++- design/skills/revise.md | 3 ++- 4 files changed, 27 insertions(+), 4 deletions(-) diff --git a/design/decomposition-review.md b/design/decomposition-review.md index 40b18fc..4fc6933 100644 --- a/design/decomposition-review.md +++ b/design/decomposition-review.md @@ -103,8 +103,11 @@ impact: every FR and NFR that has implementing stories? Are test case IDs correctly anchored to PRD requirement IDs (`TC-FR1-01`, not arbitrary sequences)? Does each test case have concrete preconditions, numbered - scenario steps, and an observable expected result — or are they vague - restatements of acceptance criteria? Are priorities assigned + scenario steps, and observable expected results — or do they contain + banned vague phrases ("works correctly", "handles appropriately", "is + validated", etc.)? Are expected results concrete enough to write an + assertion against (specific values, status codes, state changes)? Are + priorities assigned consistently (critical for core workflows, not everything marked high)? Do the `Test Case References` in story files match the testplan's Story field? Does the coverage matrix's Test Cases column diff --git a/design/skills/decompose.md b/design/skills/decompose.md index c955d60..474ee97 100644 --- a/design/skills/decompose.md +++ b/design/skills/decompose.md @@ -440,6 +440,24 @@ Gaps section. or design specifies error handling behavior. Do not invent error scenarios beyond what the requirements and design describe. +**Expected Results quality gate:** The Expected Results section must +describe concrete, observable outcomes — not restatements of the +acceptance criteria in vaguer terms. Banned phrases in Expected Results: +- "works correctly", "works as expected", "works properly" +- "handles appropriately", "handles gracefully" +- "is validated", "is verified", "is processed" +- "behaves as expected", "behaves properly" +- "completes successfully", "responds correctly", "functions as expected" +- "returns the correct value" (state the specific value) +- "no issues", "no problems" +- "appropriate error", "proper error" (name the specific error or code) + +Each expected result must state what the tester observes: a specific +return value, status code, UI state, log message, or data change. If +you cannot state the expected result concretely, the acceptance +criterion or design is underspecified — flag it in the testplan's Gaps +section rather than writing a vague test case. + #### 7b: Write the Testplan Write `.artifacts/design/{issue-key}/07-testplan.md`: diff --git a/design/skills/respond.md b/design/skills/respond.md index 19ef655..b21d026 100644 --- a/design/skills/respond.md +++ b/design/skills/respond.md @@ -170,7 +170,8 @@ feedback), apply them in this order: Automation), and H5 sub-sections (Preconditions, Steps, Expected Results). Update the testplan's Overview counts and Summary table. - **Modifying a test case:** Update the affected heading, metadata - table fields, or sub-section content. If the Story assignment + table fields, or sub-section content. The same Expected Results + quality gate applies — no banned vague phrases. If the Story assignment changes, update both the old and new story's Test Case References in step 2 below. - **Removing a test case:** Delete the test case entry (heading and diff --git a/design/skills/revise.md b/design/skills/revise.md index 29b71fd..a829483 100644 --- a/design/skills/revise.md +++ b/design/skills/revise.md @@ -126,7 +126,8 @@ After applying changes, verify: **If the testplan exists (`07-testplan.md`):** - If the design changed: do any test cases reference changed behavior? Update preconditions, steps, or expected results if the design change - affects what the test validates. + affects what the test validates. The Expected Results quality gate + applies — no banned vague phrases. - If stories were added or removed: add or remove test cases as appropriate. Update `Test Case References` in affected story files. - If acceptance criteria changed on a story: review the test cases From 04f5d52f7f9555f265c470bf337060d91b7f6100 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 15:15:03 -0400 Subject: [PATCH 12/38] Correct version bumps to one per workflow per PR Versions are PR-based (squash merge). Collapse per-commit bumps to a single minor bump per workflow: - Design: 0.4.2 -> 0.5.0 - Implement: 0.4.1 -> 0.5.0 - E2e: 0.3.2 -> 0.4.0 Assisted-by: Claude Opus 4.6 (1M) --- design/SKILL.md | 2 +- e2e/SKILL.md | 2 +- implement/SKILL.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/design/SKILL.md b/design/SKILL.md index ef3b438..f3cbdde 100644 --- a/design/SKILL.md +++ b/design/SKILL.md @@ -1,6 +1,6 @@ --- name: design -version: 0.6.0 +version: 0.5.0 description: >- Design-and-decompose workflow that takes a PRD, researches the problem space, drafts a technical design document, decomposes work into Jira-ready epics and diff --git a/e2e/SKILL.md b/e2e/SKILL.md index fa6a2b8..0f662bd 100644 --- a/e2e/SKILL.md +++ b/e2e/SKILL.md @@ -1,6 +1,6 @@ --- name: e2e -version: 0.5.0 +version: 0.4.0 description: >- Story-to-e2e-test workflow that takes a Jira [QE] Story, discovers the project's e2e testing infrastructure, plans test scenarios, writes e2e diff --git a/implement/SKILL.md b/implement/SKILL.md index 4d90980..3bd0eb7 100644 --- a/implement/SKILL.md +++ b/implement/SKILL.md @@ -1,6 +1,6 @@ --- name: implement -version: 0.6.0 +version: 0.5.0 description: >- Story-to-code workflow that takes a Jira Story, plans the implementation, writes contract-based tests and production code via TDD, validates against From 25e82f646ddfac19432b36ecf148dcf2bf08bc8b Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 15:22:37 -0400 Subject: [PATCH 13/38] Address final code review findings - PR description Documents section now conditionally includes testplan.md bullet only when testplan was published - publish-metadata.json shows two explicit template variants (with and without testplan_file_path) instead of contradictory template-plus-prose-override - README /sync typical flow now mentions testplan story reference resolution Assisted-by: Claude Opus 4.6 (1M) --- design/README.md | 1 + design/skills/publish.md | 19 ++++++++++++++++--- 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/design/README.md b/design/README.md index 475cf4d..4b3b811 100644 --- a/design/README.md +++ b/design/README.md @@ -85,6 +85,7 @@ graph TD /sync → previews all Jira operations (dry run) → syncs epics and stories — creates new, updates changed, closes removed + → resolves testplan story references to Jira keys in docs repo → maintains sync-manifest.json with content hashes ``` diff --git a/design/skills/publish.md b/design/skills/publish.md index 99d7796..adb59ea 100644 --- a/design/skills/publish.md +++ b/design/skills/publish.md @@ -220,7 +220,9 @@ exist, write "General review — no specific items flagged."} ### Documents - `design.md` — technical design document -- `testplan.md` — behavioral test cases mapped to PRD requirements (included if testplan exists) +{If `07-testplan.md` was published, add: +- `testplan.md` — behavioral test cases mapped to PRD requirements +Otherwise, omit the testplan bullet entirely.} ### How to Review - Comment inline on specific sections @@ -239,6 +241,8 @@ gh pr create --draft --repo {owner}/{repo} --base {base-branch} --head design/{i Write `.artifacts/design/{issue-key}/publish-metadata.json`: +If `07-testplan.md` was published: + ```json { "release": "{release}", @@ -250,8 +254,17 @@ Write `.artifacts/design/{issue-key}/publish-metadata.json`: } ``` -Include `testplan_file_path` only if `07-testplan.md` was published. Omit -the field if no testplan exists. +If no testplan was published, omit `testplan_file_path` entirely: + +```json +{ + "release": "{release}", + "feature": "{feature}", + "design_file_path": "{release}/{feature}/design.md", + "pr_number": {pr-number}, + "branch": "design/{issue-key}" +} +``` ### Step 7: Report to User From bf06ad5aa43de8d86d9795b1dce2d04232f469ca Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 15:28:55 -0400 Subject: [PATCH 14/38] Improve sync-manifest guard clarity across publish, respond, revise MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add rationale to publish.md guard explaining the re-publish-after-sync edge case - Restructure cp/guard pattern as explicit if/else branches in all three files — clearer for less capable models than showing a cp command and then overriding it Assisted-by: Claude Opus 4.6 (1M) --- design/skills/publish.md | 14 ++++++++------ design/skills/respond.md | 8 ++++---- design/skills/revise.md | 8 ++++---- 3 files changed, 16 insertions(+), 14 deletions(-) diff --git a/design/skills/publish.md b/design/skills/publish.md index adb59ea..48b4387 100644 --- a/design/skills/publish.md +++ b/design/skills/publish.md @@ -154,20 +154,22 @@ Read and follow `../../_shared/recipes/render-provenance-footer.md` with If `.artifacts/design/{issue-key}/07-testplan.md` exists, also copy it. -**Sync-manifest guard:** If `.artifacts/design/{issue-key}/sync-manifest.json` -exists, the published testplan's Story field must use Jira keys. Before +**Sync-manifest guard:** This guard handles re-publishing after `/sync` +has run (e.g., `decompose → publish → sync → revise → publish`). If +`.artifacts/design/{issue-key}/sync-manifest.json` exists, the published +testplan's Story field must use Jira keys. Before copying, read the sync manifest and resolve the Story field in each test case's metadata table (`Story 1.01` → Jira key from manifest). Write the resolved version to the docs repo — do NOT modify the local `07-testplan.md`. -If `sync-manifest.json` does not exist, copy `07-testplan.md` as-is. +If `sync-manifest.json` does not exist: ```bash cp ".artifacts/design/{issue-key}/07-testplan.md" "{docs_repo_path}/{release}/{feature}/testplan.md" ``` -(If the sync-manifest guard applied, write the resolved content to -`{docs_repo_path}/{release}/{feature}/testplan.md` directly instead -of using a literal `cp` of the local file.) +If `sync-manifest.json` exists, write the resolved content (with Jira +keys in Story fields) to `{docs_repo_path}/{release}/{feature}/testplan.md` +directly — do not `cp` the unresolved local file. ```bash git -C "{docs_repo_path}" add "{release}/{feature}/design.md" diff --git a/design/skills/respond.md b/design/skills/respond.md index b21d026..999088b 100644 --- a/design/skills/respond.md +++ b/design/skills/respond.md @@ -268,15 +268,15 @@ references (e.g., `Story 1.01`) with their Jira keys from the manifest (e.g., `EDM-1234`). Write the resolved version to the docs repo — do NOT modify the local `07-testplan.md` (it keeps local identifiers). -If `sync-manifest.json` does not exist, copy `07-testplan.md` as-is: +If `sync-manifest.json` does not exist: ```bash cp ".artifacts/design/{issue-key}/07-testplan.md" "{docs_repo_path}/{testplan_file_path}" ``` -If the sync-manifest guard applied, write the resolved content to -`{docs_repo_path}/{testplan_file_path}` directly instead of using -a literal `cp` of the unresolved local file. +If `sync-manifest.json` exists, write the resolved content (with Jira +keys in Story fields) to `{docs_repo_path}/{testplan_file_path}` +directly — do not `cp` the unresolved local file. ```bash git -C "{docs_repo_path}" add "{testplan_file_path}" diff --git a/design/skills/revise.md b/design/skills/revise.md index a829483..224ab8e 100644 --- a/design/skills/revise.md +++ b/design/skills/revise.md @@ -211,15 +211,15 @@ exists, the published testplan's Story field must use Jira keys. Before copying, read the sync manifest and resolve the Story field in each test case's metadata table (`Story 1.01` → Jira key from manifest). Write the resolved version to the docs repo — do NOT modify the local `07-testplan.md`. -If `sync-manifest.json` does not exist, copy `07-testplan.md` as-is. +If `sync-manifest.json` does not exist: ```bash cp ".artifacts/design/{issue-key}/07-testplan.md" "{docs_repo_path}/{testplan_file_path}" ``` -(If the sync-manifest guard applied, write the resolved content to -`{docs_repo_path}/{testplan_file_path}` directly instead of using -a literal `cp` of the local file.) +If `sync-manifest.json` exists, write the resolved content (with Jira +keys in Story fields) to `{docs_repo_path}/{testplan_file_path}` +directly — do not `cp` the unresolved local file. ```bash git -C "{docs_repo_path}" add "{design_file_path}" From 804ab5f072ddfb3536cd15e18f798167677be465 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 15:54:23 -0400 Subject: [PATCH 15/38] Address CodeRabbit review findings MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Make Test Case References writes idempotent in decompose Step 7c — replace existing section instead of appending a duplicate - Add git status check to publish before docs-repo modifications - Exclude [DOCS] stories from respond and revise Test Case References cascade - Add Overview/Summary count recomputation to revise consistency check when test cases are added or removed - Cascade renamed TC IDs to story Test Case References and coverage matrix in revise (requirement ID change path) - Reorder sync Step 7 to verify docs-repo state before writing resolved testplan content - Delete stale story-scoped testplan in implement and e2e ingest on non-normal outcomes to prevent downstream gates from enforcing obsolete coverage requirements Assisted-by: Claude Opus 4.6 (1M) --- design/skills/decompose.md | 6 ++++-- design/skills/publish.md | 8 ++++++++ design/skills/respond.md | 3 ++- design/skills/revise.md | 9 +++++++-- design/skills/sync.md | 16 +++++++++------- e2e/skills/ingest.md | 9 +++++++-- implement/skills/ingest.md | 9 +++++++-- 7 files changed, 44 insertions(+), 16 deletions(-) diff --git a/design/skills/decompose.md b/design/skills/decompose.md index 474ee97..57519a7 100644 --- a/design/skills/decompose.md +++ b/design/skills/decompose.md @@ -590,12 +590,14 @@ each test case's metadata table links back to the epic/story structure. #### 7c: Add Test Case References to Stories -After writing the testplan, append a `## Test Case References` section +After writing the testplan, add a `## Test Case References` section to each non-`[DOCS]` story file. For each test case in the testplan, its Story field identifies the implementing story. Collect all TC IDs for each story and write them into that story's file. -Re-read each story file before appending to ensure current content. +Re-read each story file before writing. If the story already has a +`## Test Case References` section (from a prior decomposition), replace +it entirely with the current TC IDs. Do not append a second section. `[DOCS]` stories do not receive this section. The section format is a single `Verified by:` line with comma-separated diff --git a/design/skills/publish.md b/design/skills/publish.md index 48b4387..501ca35 100644 --- a/design/skills/publish.md +++ b/design/skills/publish.md @@ -111,6 +111,14 @@ placed alongside the PRD (`prd.md`) if one was published previously. All git operations run against the **docs repo**. Use `git -C "{docs_repo_path}"` for all commands. +Verify the docs repo is clean before modifying it: + +```bash +git -C "{docs_repo_path}" status +``` + +If there are uncommitted changes, ask the user before continuing. + Check if the branch already exists: ```bash diff --git a/design/skills/respond.md b/design/skills/respond.md index 999088b..7f7ef67 100644 --- a/design/skills/respond.md +++ b/design/skills/respond.md @@ -179,7 +179,8 @@ feedback), apply them in this order: Summary table. 2. **Cascade to story files.** For each affected story (identified by the - Story field of changed test cases): + Story field of changed test cases), skip `[DOCS]` stories (they do + not have a Test Case References section). For non-`[DOCS]` stories: - Re-read the story file at `.artifacts/design/{issue-key}/05-stories/epic-{N}/story-{NN}-{slug}.md`. - Rewrite the `## Test Case References` section: collect all TC IDs diff --git a/design/skills/revise.md b/design/skills/revise.md index 224ab8e..7d0501a 100644 --- a/design/skills/revise.md +++ b/design/skills/revise.md @@ -129,15 +129,20 @@ After applying changes, verify: affects what the test validates. The Expected Results quality gate applies — no banned vague phrases. - If stories were added or removed: add or remove test cases as - appropriate. Update `Test Case References` in affected story files. + appropriate. Update `Test Case References` in affected non-`[DOCS]` + story files. - If acceptance criteria changed on a story: review the test cases referencing that story — do they still validate the correct behavior? - Update the coverage matrix's Test Cases column if test case IDs changed. +- If test cases were added or removed: update the testplan's Overview + counts and Summary table to reflect the current totals. - If test cases were removed and a requirement now has none: update the testplan's Gaps section to reflect the new gap. - If requirement IDs changed (rare — requires PRD revision): update - all TC IDs anchored to the changed requirement. + all TC IDs anchored to the changed requirement, cascade the renamed + IDs to each story's `Test Case References` section and the coverage + matrix's Test Cases column, then revalidate the cross-file mappings. ### Step 5: Update Artifacts diff --git a/design/skills/sync.md b/design/skills/sync.md index f9dc71d..e26f87a 100644 --- a/design/skills/sync.md +++ b/design/skills/sync.md @@ -623,13 +623,7 @@ can filter by Jira key. Read `.artifacts/prd/config.json` to get the docs repo path. Read `publish-metadata.json` to get the `testplan_file_path`. -Write the resolved testplan content (with Jira keys in the Story field) -to `{docs_repo_path}/{testplan_file_path}`. Do NOT modify the local -`07-testplan.md` — it keeps local identifiers. - -Compare the resolved content against the current content of -`{docs_repo_path}/{testplan_file_path}`. If they are identical, skip -the commit — the published testplan is already up to date. +Verify the docs repo state before writing: ```bash git -C "{docs_repo_path}" fetch origin @@ -656,6 +650,14 @@ git -C "{docs_repo_path}" checkout design/{issue-key} git -C "{docs_repo_path}" pull --ff-only ``` +Compare the resolved content against the current content of +`{docs_repo_path}/{testplan_file_path}`. If they are identical, skip +the commit — the published testplan is already up to date. + +Write the resolved testplan content (with Jira keys in the Story field) +to `{docs_repo_path}/{testplan_file_path}`. Do NOT modify the local +`07-testplan.md` — it keeps local identifiers. + ```bash git -C "{docs_repo_path}" add "{testplan_file_path}" ``` diff --git a/e2e/skills/ingest.md b/e2e/skills/ingest.md index 1bc7821..4437a7f 100644 --- a/e2e/skills/ingest.md +++ b/e2e/skills/ingest.md @@ -180,8 +180,13 @@ section and matching those directly against the testplan. | Outcome | Condition | Action | |---------|-----------|--------| | **Normal** | Matching test cases found | Write `.artifacts/e2e/{issue-key}/testplan.md` | -| **Expected zero** | No matches AND story type is `[DOCS]`, `[UX]`, or `[CI]` | Note in context. Do not write `testplan.md`. | -| **Anomalous zero** | No matches AND story type is `[QE]`, `[DEV]`, or `[UI]` | Warn the user: "Testplan exists but no test cases match this story's requirements. This may indicate a gap in the testplan or the story's requirement mapping." This is non-blocking — continue without `testplan.md`. | +| **Expected zero** | No matches AND story type is `[DOCS]`, `[UX]`, or `[CI]` | Note in context. | +| **Anomalous zero** | No matches AND story type is `[QE]`, `[DEV]`, or `[UI]` | Warn the user: "Testplan exists but no test cases match this story's requirements. This may indicate a gap in the testplan or the story's requirement mapping." This is non-blocking. | + +For non-normal outcomes (expected zero, anomalous zero, or no +feature-level testplan): if `.artifacts/e2e/{issue-key}/testplan.md` +exists from a prior ingest run, delete it. A stale story-scoped testplan +would cause downstream gates to enforce obsolete coverage requirements. If `testplan.md` was not found in the docs repo, note "No feature-level testplan available" and continue. This is not an error — the testplan is diff --git a/implement/skills/ingest.md b/implement/skills/ingest.md index c253b7d..2c19218 100644 --- a/implement/skills/ingest.md +++ b/implement/skills/ingest.md @@ -169,8 +169,13 @@ workflow) and match those TC IDs directly against the testplan entries. | Outcome | Condition | Action | |---------|-----------|--------| | **Normal** | Matching test cases found | Write `.artifacts/implement/{issue-key}/testplan.md` | -| **Expected zero** | No matches AND story type is `[QE]`, `[DOCS]`, `[UX]`, or `[CI]` | Note in context: "Testplan exists but has no test cases for this story type. This is expected." Do not write `testplan.md`. | -| **Anomalous zero** | No matches AND story type is `[DEV]` or `[UI]` | Warn the user: "Testplan exists but no test cases reference this story. This may indicate a gap in the testplan or an incorrect requirement mapping." This is non-blocking — continue without `testplan.md`. | +| **Expected zero** | No matches AND story type is `[QE]`, `[DOCS]`, `[UX]`, or `[CI]` | Note in context: "Testplan exists but has no test cases for this story type. This is expected." | +| **Anomalous zero** | No matches AND story type is `[DEV]` or `[UI]` | Warn the user: "Testplan exists but no test cases reference this story. This may indicate a gap in the testplan or an incorrect requirement mapping." This is non-blocking. | + +For non-normal outcomes (expected zero, anomalous zero, or no +feature-level testplan): if `.artifacts/implement/{issue-key}/testplan.md` +exists from a prior ingest run, delete it. A stale story-scoped testplan +would cause downstream gates to enforce obsolete coverage requirements. If `testplan.md` was not found in the docs repo, note "No feature-level testplan available" and continue. This is not an error — the testplan is From 0640c203bc4396b825bf777adb7db7b4322fc1d4 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 16:01:44 -0400 Subject: [PATCH 16/38] Use published branch name in sync testplan step Read the branch field from publish-metadata.json instead of hardcoding design/{issue-key} when checking out the docs repo PR branch. Honors user overrides of the default branch name during /publish. Assisted-by: Claude Opus 4.6 (1M) --- design/skills/sync.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/design/skills/sync.md b/design/skills/sync.md index e26f87a..a66e07f 100644 --- a/design/skills/sync.md +++ b/design/skills/sync.md @@ -640,10 +640,11 @@ continuing. git -C "{docs_repo_path}" branch --show-current ``` -If not on the PR branch (`design/{issue-key}`), check it out: +Read the `branch` field from `publish-metadata.json` to get the docs +repo PR branch name. If not on that branch, check it out: ```bash -git -C "{docs_repo_path}" checkout design/{issue-key} +git -C "{docs_repo_path}" checkout {branch} ``` ```bash From 599b05e8dbe1739e1dae882c515798acffcc2729 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 16:06:13 -0400 Subject: [PATCH 17/38] Address CodeRabbit round 2 findings - Sync Step 7: identical content now skips write, staging, commit, and push (not just commit) to avoid empty-commit failures - Revise: recompute Overview/Summary counts on metadata modifications (Priority, Automation) not just add/remove - Implement/e2e ingest: clarify that Design Reference and Test Case References come from Step 3 story capture; add unknown story-type fallback (ask user) to both three-outcome gates Assisted-by: Claude Opus 4.6 (1M) --- design/skills/revise.md | 5 +++-- design/skills/sync.md | 9 +++++---- e2e/skills/ingest.md | 20 +++++++++++++------- implement/skills/ingest.md | 15 ++++++++++----- 4 files changed, 31 insertions(+), 18 deletions(-) diff --git a/design/skills/revise.md b/design/skills/revise.md index 7d0501a..7ec0ee0 100644 --- a/design/skills/revise.md +++ b/design/skills/revise.md @@ -135,8 +135,9 @@ After applying changes, verify: referencing that story — do they still validate the correct behavior? - Update the coverage matrix's Test Cases column if test case IDs changed. -- If test cases were added or removed: update the testplan's Overview - counts and Summary table to reflect the current totals. +- If test cases were added, removed, or modified (including metadata + changes like Priority or Automation): recompute the testplan's Overview + counts and Summary table from the current test cases. - If test cases were removed and a requirement now has none: update the testplan's Gaps section to reflect the new gap. - If requirement IDs changed (rare — requires PRD revision): update diff --git a/design/skills/sync.md b/design/skills/sync.md index a66e07f..903449e 100644 --- a/design/skills/sync.md +++ b/design/skills/sync.md @@ -653,11 +653,12 @@ git -C "{docs_repo_path}" pull --ff-only Compare the resolved content against the current content of `{docs_repo_path}/{testplan_file_path}`. If they are identical, skip -the commit — the published testplan is already up to date. +the write, staging, commit, and push — the published testplan is +already up to date. Continue to the reporting step. -Write the resolved testplan content (with Jira keys in the Story field) -to `{docs_repo_path}/{testplan_file_path}`. Do NOT modify the local -`07-testplan.md` — it keeps local identifiers. +If the content differs, write the resolved testplan (with Jira keys in +the Story field) to `{docs_repo_path}/{testplan_file_path}`. Do NOT +modify the local `07-testplan.md` — it keeps local identifiers. ```bash git -C "{docs_repo_path}" add "{testplan_file_path}" diff --git a/e2e/skills/ingest.md b/e2e/skills/ingest.md index 4437a7f..9529724 100644 --- a/e2e/skills/ingest.md +++ b/e2e/skills/ingest.md @@ -167,13 +167,15 @@ relevant to this story's scope. For `[QE]` stories, the testplan's test cases reference `[DEV]` stories (the testplan maps test cases to implementing stories, not to `[QE]` stories). Filter by **requirement**: extract the PRD requirement IDs -from the story's Design Reference section (e.g., `FR-1, FR-3, NFR-2`), -then collect all test cases from the testplan whose requirement heading -matches any of those IDs. +from the story's Design Reference section (captured in Step 3 from the +story description, e.g., `FR-1, FR-3, NFR-2`), then collect all test +cases from the testplan whose requirement heading matches any of those +IDs. If the story's Design Reference does not list PRD requirements, fall back to looking for TC IDs in the Jira story's Test Case References -section and matching those directly against the testplan. +section (also captured in Step 3) and matching those directly against +the testplan. **Three-outcome gate:** @@ -183,9 +185,13 @@ section and matching those directly against the testplan. | **Expected zero** | No matches AND story type is `[DOCS]`, `[UX]`, or `[CI]` | Note in context. | | **Anomalous zero** | No matches AND story type is `[QE]`, `[DEV]`, or `[UI]` | Warn the user: "Testplan exists but no test cases match this story's requirements. This may indicate a gap in the testplan or the story's requirement mapping." This is non-blocking. | -For non-normal outcomes (expected zero, anomalous zero, or no -feature-level testplan): if `.artifacts/e2e/{issue-key}/testplan.md` -exists from a prior ingest run, delete it. A stale story-scoped testplan +If the story type prefix is not listed above (unknown type), treat it +as anomalous zero and ask the user how to proceed. + +For non-normal outcomes (expected zero, anomalous zero, unknown type, +or no feature-level testplan): if +`.artifacts/e2e/{issue-key}/testplan.md` exists from a prior ingest +run, delete it. A stale story-scoped testplan would cause downstream gates to enforce obsolete coverage requirements. If `testplan.md` was not found in the docs repo, note "No feature-level diff --git a/implement/skills/ingest.md b/implement/skills/ingest.md index 2c19218..9221fae 100644 --- a/implement/skills/ingest.md +++ b/implement/skills/ingest.md @@ -161,8 +161,9 @@ Filter by matching the Story field against this story's Jira key If the Story field still contains local identifiers (e.g., `Story 1.01` instead of Jira keys), this means `/sync` has not yet been run or the testplan was published before sync. In this case, look for TC IDs in the -Jira story's Test Case References section (synced from the design -workflow) and match those TC IDs directly against the testplan entries. +Jira story's Test Case References section (captured in Step 3 from the +story description) and match those TC IDs directly against the testplan +entries. **Three-outcome gate:** @@ -172,9 +173,13 @@ workflow) and match those TC IDs directly against the testplan entries. | **Expected zero** | No matches AND story type is `[QE]`, `[DOCS]`, `[UX]`, or `[CI]` | Note in context: "Testplan exists but has no test cases for this story type. This is expected." | | **Anomalous zero** | No matches AND story type is `[DEV]` or `[UI]` | Warn the user: "Testplan exists but no test cases reference this story. This may indicate a gap in the testplan or an incorrect requirement mapping." This is non-blocking. | -For non-normal outcomes (expected zero, anomalous zero, or no -feature-level testplan): if `.artifacts/implement/{issue-key}/testplan.md` -exists from a prior ingest run, delete it. A stale story-scoped testplan +If the story type prefix is not listed above (unknown type), treat it +as anomalous zero and ask the user how to proceed. + +For non-normal outcomes (expected zero, anomalous zero, unknown type, +or no feature-level testplan): if +`.artifacts/implement/{issue-key}/testplan.md` exists from a prior +ingest run, delete it. A stale story-scoped testplan would cause downstream gates to enforce obsolete coverage requirements. If `testplan.md` was not found in the docs repo, note "No feature-level From 33856aaed6842b6b8e1144afaaa32bd30a8d8af3 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 16:08:59 -0400 Subject: [PATCH 18/38] Quote branch argument and validate before checkout in sync Shell-quote the {branch} variable in the checkout command to match the quoting treatment of {docs_repo_path}. Validate that the branch field is present and non-empty in publish-metadata.json before executing any git commands. Assisted-by: Claude Opus 4.6 (1M) --- design/skills/sync.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/design/skills/sync.md b/design/skills/sync.md index 903449e..7f19225 100644 --- a/design/skills/sync.md +++ b/design/skills/sync.md @@ -641,10 +641,13 @@ git -C "{docs_repo_path}" branch --show-current ``` Read the `branch` field from `publish-metadata.json` to get the docs -repo PR branch name. If not on that branch, check it out: +repo PR branch name. If `branch` is missing or empty, stop and report +the error — publish-metadata.json is incomplete. + +If not on that branch, check it out: ```bash -git -C "{docs_repo_path}" checkout {branch} +git -C "{docs_repo_path}" checkout "{branch}" ``` ```bash From 318961331f3dcc7041238cb4aa70612b6719b543 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 16:10:31 -0400 Subject: [PATCH 19/38] Move branch validation before git operations in sync Step 7 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Read and validate the branch field from publish-metadata.json at the same time as testplan_file_path — before any git fetch/status/checkout operations, not after them. Assisted-by: Claude Opus 4.6 (1M) --- design/skills/sync.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/design/skills/sync.md b/design/skills/sync.md index 7f19225..455b859 100644 --- a/design/skills/sync.md +++ b/design/skills/sync.md @@ -621,7 +621,9 @@ can filter by Jira key. **Write the resolved testplan to the docs repo:** Read `.artifacts/prd/config.json` to get the docs repo path. Read -`publish-metadata.json` to get the `testplan_file_path`. +`publish-metadata.json` to get the `testplan_file_path` and the +`branch` field. If `branch` is missing or empty, stop and report the +error — publish-metadata.json is incomplete. Verify the docs repo state before writing: @@ -640,11 +642,7 @@ continuing. git -C "{docs_repo_path}" branch --show-current ``` -Read the `branch` field from `publish-metadata.json` to get the docs -repo PR branch name. If `branch` is missing or empty, stop and report -the error — publish-metadata.json is incomplete. - -If not on that branch, check it out: +If not on the published branch, check it out: ```bash git -C "{docs_repo_path}" checkout "{branch}" From a8776f6e5ea83b1083a6053b55dc560bcaf17801 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 16:12:14 -0400 Subject: [PATCH 20/38] Rebuild Gaps section bidirectionally in revise MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Gaps section is now rebuilt from current mappings after any test case addition, removal, or rename — removing stale gaps when coverage is added, not just adding gaps when coverage is removed. Assisted-by: Claude Opus 4.6 (1M) --- design/skills/revise.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/design/skills/revise.md b/design/skills/revise.md index 7ec0ee0..31db7fe 100644 --- a/design/skills/revise.md +++ b/design/skills/revise.md @@ -138,8 +138,10 @@ After applying changes, verify: - If test cases were added, removed, or modified (including metadata changes like Priority or Automation): recompute the testplan's Overview counts and Summary table from the current test cases. -- If test cases were removed and a requirement now has none: update - the testplan's Gaps section to reflect the new gap. +- If test cases were added, removed, or renamed: rebuild the testplan's + Gaps section from the current requirement-to-test-case mappings. + Remove stale gaps for requirements that now have coverage, and add + gaps for requirements that have lost all test cases. - If requirement IDs changed (rare — requires PRD revision): update all TC IDs anchored to the changed requirement, cascade the renamed IDs to each story's `Test Case References` section and the coverage From a285ab4f3bc945a1ba22d2f9be9904269f14432c Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 16:22:55 -0400 Subject: [PATCH 21/38] Fix Overview rendering in testplan templates Add
line breaks to Overview fields in the feature-level and story-scoped testplan templates so they render as separate lines instead of collapsing into one sentence. Assisted-by: Claude Opus 4.6 (1M) --- design/skills/decompose.md | 4 ++-- e2e/skills/ingest.md | 4 ++-- implement/skills/ingest.md | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/design/skills/decompose.md b/design/skills/decompose.md index 57519a7..71ef418 100644 --- a/design/skills/decompose.md +++ b/design/skills/decompose.md @@ -467,8 +467,8 @@ Write `.artifacts/design/{issue-key}/07-testplan.md`: ## Overview -Feature: {feature-key} — {feature-title} -Total test cases: {N} +Feature: {feature-key} — {feature-title}
+Total test cases: {N}
Requirements covered: {N} of {total FR + NFR count} ## Test Cases diff --git a/e2e/skills/ingest.md b/e2e/skills/ingest.md index 9529724..9f7880c 100644 --- a/e2e/skills/ingest.md +++ b/e2e/skills/ingest.md @@ -205,8 +205,8 @@ Write `.artifacts/e2e/{issue-key}/testplan.md`: ```markdown # Story Test Plan — {issue-key} -Source: {docs-repo-path}/testplan.md -Story: {issue-key} — {story-title} +Source: {docs-repo-path}/testplan.md
+Story: {issue-key} — {story-title}
Test cases: {count} ## TC-FR1-01: {scenario title} diff --git a/implement/skills/ingest.md b/implement/skills/ingest.md index 9221fae..da64292 100644 --- a/implement/skills/ingest.md +++ b/implement/skills/ingest.md @@ -193,8 +193,8 @@ Write `.artifacts/implement/{issue-key}/testplan.md`: ```markdown # Story Test Plan — {issue-key} -Source: {docs-repo-path}/testplan.md -Story: {issue-key} — {story-title} +Source: {docs-repo-path}/testplan.md
+Story: {issue-key} — {story-title}
Test cases: {count} ## TC-FR1-01: {scenario title} From 6951ee1de207c00dbb457ef1660df000efa63f83 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 17:45:06 -0400 Subject: [PATCH 22/38] Replace
with bullet lists and broaden Gaps rebuild trigger - Replace
tags in testplan Overview sections with bullet lists across decompose, implement ingest, and e2e ingest templates - Broaden revise Gaps rebuild trigger to include Story, AC, and requirement mapping changes, not just add/remove/rename Assisted-by: Claude Opus 4.6 (1M) --- design/skills/decompose.md | 6 +++--- design/skills/revise.md | 9 +++++---- e2e/skills/ingest.md | 6 +++--- implement/skills/ingest.md | 6 +++--- 4 files changed, 14 insertions(+), 13 deletions(-) diff --git a/design/skills/decompose.md b/design/skills/decompose.md index 71ef418..8be5d4f 100644 --- a/design/skills/decompose.md +++ b/design/skills/decompose.md @@ -467,9 +467,9 @@ Write `.artifacts/design/{issue-key}/07-testplan.md`: ## Overview -Feature: {feature-key} — {feature-title}
-Total test cases: {N}
-Requirements covered: {N} of {total FR + NFR count} +- **Feature:** {feature-key} — {feature-title} +- **Total test cases:** {N} +- **Requirements covered:** {N} of {total FR + NFR count} ## Test Cases diff --git a/design/skills/revise.md b/design/skills/revise.md index 31db7fe..5a5c6c4 100644 --- a/design/skills/revise.md +++ b/design/skills/revise.md @@ -138,10 +138,11 @@ After applying changes, verify: - If test cases were added, removed, or modified (including metadata changes like Priority or Automation): recompute the testplan's Overview counts and Summary table from the current test cases. -- If test cases were added, removed, or renamed: rebuild the testplan's - Gaps section from the current requirement-to-test-case mappings. - Remove stale gaps for requirements that now have coverage, and add - gaps for requirements that have lost all test cases. +- If test cases were added, removed, renamed, or had their Story, AC, + or requirement mapping changed: rebuild the testplan's Gaps section + from the current requirement-to-test-case and AC-to-test-case + mappings. Remove stale gaps for requirements or ACs that now have + coverage, and add gaps for those that have lost coverage. - If requirement IDs changed (rare — requires PRD revision): update all TC IDs anchored to the changed requirement, cascade the renamed IDs to each story's `Test Case References` section and the coverage diff --git a/e2e/skills/ingest.md b/e2e/skills/ingest.md index 9f7880c..c6f7822 100644 --- a/e2e/skills/ingest.md +++ b/e2e/skills/ingest.md @@ -205,9 +205,9 @@ Write `.artifacts/e2e/{issue-key}/testplan.md`: ```markdown # Story Test Plan — {issue-key} -Source: {docs-repo-path}/testplan.md
-Story: {issue-key} — {story-title}
-Test cases: {count} +- **Source:** {docs-repo-path}/testplan.md +- **Story:** {issue-key} — {story-title} +- **Test cases:** {count} ## TC-FR1-01: {scenario title} diff --git a/implement/skills/ingest.md b/implement/skills/ingest.md index da64292..e5c56cd 100644 --- a/implement/skills/ingest.md +++ b/implement/skills/ingest.md @@ -193,9 +193,9 @@ Write `.artifacts/implement/{issue-key}/testplan.md`: ```markdown # Story Test Plan — {issue-key} -Source: {docs-repo-path}/testplan.md
-Story: {issue-key} — {story-title}
-Test cases: {count} +- **Source:** {docs-repo-path}/testplan.md +- **Story:** {issue-key} — {story-title} +- **Test cases:** {count} ## TC-FR1-01: {scenario title} From 60c25943db7c555787716d8fa42dd7b82aeaafb9 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Tue, 28 Jul 2026 17:55:13 -0400 Subject: [PATCH 23/38] Simplify unknown story-type handling and clarify testplan status messages - Fold unknown story-type prefix into anomalous-zero contract in both implement and e2e ingest (same behavior, no separate blocking branch) - Disambiguate feature-level vs story-scoped testplan in context template status messages across both workflows Assisted-by: Claude Opus 4.6 (1M) --- e2e/skills/ingest.md | 29 ++++++++++++++++------------- implement/skills/ingest.md | 29 ++++++++++++++++------------- 2 files changed, 32 insertions(+), 26 deletions(-) diff --git a/e2e/skills/ingest.md b/e2e/skills/ingest.md index c6f7822..3545931 100644 --- a/e2e/skills/ingest.md +++ b/e2e/skills/ingest.md @@ -185,11 +185,11 @@ the testplan. | **Expected zero** | No matches AND story type is `[DOCS]`, `[UX]`, or `[CI]` | Note in context. | | **Anomalous zero** | No matches AND story type is `[QE]`, `[DEV]`, or `[UI]` | Warn the user: "Testplan exists but no test cases match this story's requirements. This may indicate a gap in the testplan or the story's requirement mapping." This is non-blocking. | -If the story type prefix is not listed above (unknown type), treat it -as anomalous zero and ask the user how to proceed. +If the story type prefix is not listed above, treat it as anomalous +zero (warn the user, continue without `testplan.md`). -For non-normal outcomes (expected zero, anomalous zero, unknown type, -or no feature-level testplan): if +For non-normal outcomes (expected zero, anomalous zero, or no +feature-level testplan): if `.artifacts/e2e/{issue-key}/testplan.md` exists from a prior ingest run, delete it. A stale story-scoped testplan would cause downstream gates to enforce obsolete coverage requirements. @@ -511,19 +511,22 @@ If this is a first invocation, write ### Story Test Plan -{If testplan.md was written: "Story-scoped test plan with {N} test - cases covering requirements {FR-1, FR-3, NFR-2}. See - `.artifacts/e2e/{issue-key}/testplan.md` for full details. +{If story-scoped testplan was written: "Story-scoped test plan written + to `.artifacts/e2e/{issue-key}/testplan.md` with {N} test cases + covering requirements {FR-1, FR-3, NFR-2}. TC IDs: {comma-separated list}." - If testplan exists but no matches (expected): "Feature testplan exists - but has no test cases for this {story-type} story (expected)." + If feature-level testplan exists but no matches (expected): + "Feature-level testplan found in docs repo but no test cases match + this {story-type} story (expected). No story-scoped testplan written." - If testplan exists but no matches (anomalous): "Feature testplan exists - but no test cases match this {story-type} story's requirements - (anomalous — flagged during ingest)." + If feature-level testplan exists but no matches (anomalous): + "Feature-level testplan found in docs repo but no test cases match + this {story-type} story's requirements (anomalous — flagged during + ingest). No story-scoped testplan written." - If testplan not found: "No feature-level testplan available."} + If no feature-level testplan in docs repo: "No feature-level testplan + available in docs repo. No story-scoped testplan written."} ## E2E Test Infrastructure diff --git a/implement/skills/ingest.md b/implement/skills/ingest.md index e5c56cd..a05d483 100644 --- a/implement/skills/ingest.md +++ b/implement/skills/ingest.md @@ -173,11 +173,11 @@ entries. | **Expected zero** | No matches AND story type is `[QE]`, `[DOCS]`, `[UX]`, or `[CI]` | Note in context: "Testplan exists but has no test cases for this story type. This is expected." | | **Anomalous zero** | No matches AND story type is `[DEV]` or `[UI]` | Warn the user: "Testplan exists but no test cases reference this story. This may indicate a gap in the testplan or an incorrect requirement mapping." This is non-blocking. | -If the story type prefix is not listed above (unknown type), treat it -as anomalous zero and ask the user how to proceed. +If the story type prefix is not listed above, treat it as anomalous +zero (warn the user, continue without `testplan.md`). -For non-normal outcomes (expected zero, anomalous zero, unknown type, -or no feature-level testplan): if +For non-normal outcomes (expected zero, anomalous zero, or no +feature-level testplan): if `.artifacts/implement/{issue-key}/testplan.md` exists from a prior ingest run, delete it. A stale story-scoped testplan would cause downstream gates to enforce obsolete coverage requirements. @@ -348,18 +348,21 @@ If this is a first invocation, write ### Story Test Plan -{If testplan.md was written: "Story-scoped test plan with {N} test - cases. See `.artifacts/implement/{issue-key}/testplan.md` for full - details. TC IDs: {comma-separated list}." +{If story-scoped testplan was written: "Story-scoped test plan written + to `.artifacts/implement/{issue-key}/testplan.md` with {N} test cases. + TC IDs: {comma-separated list}." - If testplan exists but no matches (expected): "Feature testplan exists - but has no test cases for this {story-type} story (expected)." + If feature-level testplan exists but no matches (expected): + "Feature-level testplan found in docs repo but no test cases match + this {story-type} story (expected). No story-scoped testplan written." - If testplan exists but no matches (anomalous): "Feature testplan exists - but no test cases reference this {story-type} story (anomalous — flagged - during ingest)." + If feature-level testplan exists but no matches (anomalous): + "Feature-level testplan found in docs repo but no test cases reference + this {story-type} story (anomalous — flagged during ingest). No + story-scoped testplan written." - If testplan not found: "No feature-level testplan available."} + If no feature-level testplan in docs repo: "No feature-level testplan + available in docs repo. No story-scoped testplan written."} ## Codebase Context From 55f7bc44793a1f4a7183413d4fd1eae7d8ca42d7 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Wed, 29 Jul 2026 10:13:32 -0400 Subject: [PATCH 24/38] Exclude [DOCS] from testplan derivation and harden respond cascade MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Exclude [DOCS] stories from Step 7a test case derivation — they have documentation outcomes, not behavioral ACs. Record docs-only requirements in Gaps section. - Update decomposition review to exclude [DOCS] from AC completeness - Capture Story/AC mappings before mutating testplan on removal so cascade can update the correct stories - Use max(existing)+1 for TC ID allocation instead of next-available to avoid reusing gaps (ALM ID stability) Assisted-by: Claude Opus 4.6 (1M) --- design/decomposition-review.md | 7 ++++--- design/skills/decompose.md | 10 ++++++++-- design/skills/respond.md | 23 +++++++++++++++-------- 3 files changed, 27 insertions(+), 13 deletions(-) diff --git a/design/decomposition-review.md b/design/decomposition-review.md index 4fc6933..ca34d20 100644 --- a/design/decomposition-review.md +++ b/design/decomposition-review.md @@ -112,9 +112,10 @@ impact: high)? Do the `Test Case References` in story files match the testplan's Story field? Does the coverage matrix's Test Cases column match the testplan? Are there requirements with stories but no test - cases (testplan gaps)? Does every story AC appear in at least one - test case's AC field? Are there story ACs with no test case mapping - (AC-level gaps)? + cases (testplan gaps)? Does every non-`[DOCS]` story AC appear in + at least one test case's AC field? Are there story ACs with no test + case mapping (AC-level gaps)? (`[DOCS]` stories are excluded from + testplan derivation.) 6. **Integration Stability** — Would implementing all stories in dependency order produce a working feature end-to-end? Does any diff --git a/design/skills/decompose.md b/design/skills/decompose.md index 8be5d4f..73d181c 100644 --- a/design/skills/decompose.md +++ b/design/skills/decompose.md @@ -376,8 +376,14 @@ ALM export (structured for tools like Polarion). #### 7a: Derive Test Cases -For each PRD requirement (FR-N, NFR-N), examine the stories that address -it (from the epic/story files written in Step 5). For each story: +For each PRD requirement (FR-N, NFR-N), examine the non-`[DOCS]` stories +that address it (from the epic/story files written in Step 5). Skip +`[DOCS]` stories — they have documentation outcomes, not behavioral +acceptance criteria, and should not generate test cases. If a requirement +is addressed only by `[DOCS]` stories, record it in the Gaps section as +"documentation-only requirement — no behavioral test cases." + +For each non-`[DOCS]` story: 1. Read the **Acceptance Criteria** — each criterion is a candidate test case or a grouping of related test cases. diff --git a/design/skills/respond.md b/design/skills/respond.md index 7f7ef67..66788a1 100644 --- a/design/skills/respond.md +++ b/design/skills/respond.md @@ -161,11 +161,16 @@ section, synthesize the discussion into a proposed resolution: When approved changes include testplan modifications (category: Testplan feedback), apply them in this order: -1. **Modify `07-testplan.md`.** Add, modify, or remove test cases as - directed by the approved response. For each change: - - **Adding a test case:** Assign the next available sequence number - within the requirement group (e.g., if TC-FR2-01 and TC-FR2-02 - exist, the new case is TC-FR2-03). Create the full test case entry: +1. **Modify `07-testplan.md`.** Before making any changes, record the + Story and AC mappings of any test cases that will be removed or + reassigned — these are needed for the cascade in step 2. Then add, + modify, or remove test cases as directed by the approved response. + For each change: + - **Adding a test case:** Assign a sequence number using + `max(existing sequences) + 1` within the requirement group (e.g., + if TC-FR2-01 and TC-FR2-03 exist and TC-FR2-02 was previously + removed, the new case is TC-FR2-04 — do not reuse gaps, as ALM + systems track by ID). Create the full test case entry: H4 heading with ID and title, metadata table (Story, AC, Priority, Automation), and H5 sub-sections (Preconditions, Steps, Expected Results). Update the testplan's Overview counts and Summary table. @@ -178,9 +183,11 @@ feedback), apply them in this order: all sub-sections). Update the testplan's Overview counts and Summary table. -2. **Cascade to story files.** For each affected story (identified by the - Story field of changed test cases), skip `[DOCS]` stories (they do - not have a Test Case References section). For non-`[DOCS]` stories: +2. **Cascade to story files.** For each affected story (identified by + the Story field of changed test cases AND the pre-mutation mappings + captured in step 1 for removed/reassigned cases), skip `[DOCS]` + stories (they do not have a Test Case References section). For + non-`[DOCS]` stories: - Re-read the story file at `.artifacts/design/{issue-key}/05-stories/epic-{N}/story-{NN}-{slug}.md`. - Rewrite the `## Test Case References` section: collect all TC IDs From 9748165335beca6598d7b6cc81c70e243c21f52a Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Wed, 29 Jul 2026 10:16:16 -0400 Subject: [PATCH 25/38] Harden implement/e2e reconciliation and validate gates MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add re-stage (git add) after writing missing tests in reconciliation so new files are included in the staged diff for review and commit - Require non-empty rationale for N/A outcomes in hard gate - Distinguish absent testplan (skip) from broken testplan (plan has TC mappings but testplan.md is missing/unreadable — stop and report) in both code and validate phases Assisted-by: Claude Opus 4.6 (1M) --- e2e/skills/code.md | 19 ++++++++++++------- e2e/skills/validate.md | 5 ++++- implement/skills/code.md | 20 +++++++++++++------- implement/skills/validate.md | 5 ++++- 4 files changed, 33 insertions(+), 16 deletions(-) diff --git a/e2e/skills/code.md b/e2e/skills/code.md index aaae90d..102f60d 100644 --- a/e2e/skills/code.md +++ b/e2e/skills/code.md @@ -274,15 +274,20 @@ commit: pytest). 3. If a TC ID mapped to this task has no corresponding validation with sufficient assertion depth, write the missing validation (Step 3b), - run the tests (Step 3c), re-run the review gate, then re-check. + run the tests (Step 3c), stage the new files (`git add`), re-run + the review gate, then re-check. This is a hard gate — the task cannot proceed to commit until every -mapped TC ID has coverage. The testplan is a floor, not a ceiling: -validations discovered through test design that are not tied to any -TC ID are expected and encouraged. - -If no story-scoped testplan exists, or this task has no mapped TC IDs, -skip this check. +mapped TC ID has coverage or is explicitly marked N/A with a non-empty +rationale. The testplan is a floor, not a ceiling: validations +discovered through test design that are not tied to any TC ID are +expected and encouraged. + +If no story-scoped testplan exists and `02-plan.md` has no Test Plan +Coverage section, skip this check. However, if `02-plan.md` has TC +mappings but `testplan.md` is missing or unreadable, stop and report +the inconsistency — the plan references a testplan that the code phase +cannot access. #### 3f: Commit diff --git a/e2e/skills/validate.md b/e2e/skills/validate.md index 2289978..c8eeeb5 100644 --- a/e2e/skills/validate.md +++ b/e2e/skills/validate.md @@ -243,7 +243,10 @@ inapplicable. This is an intentional independent verification: if the plan's bookkeeping or the code phase's gate drifted from reality, this step catches it. -If `testplan.md` does not exist, skip this step entirely. +If `testplan.md` does not exist and `02-plan.md` has no Test Plan +Coverage section, skip this step entirely. However, if `02-plan.md` +has TC mappings but `testplan.md` is missing or unreadable, stop and +report the inconsistency. 1. Read `testplan.md` and extract all TC IDs. 2. For each TC ID (except those the validate phase agrees are diff --git a/implement/skills/code.md b/implement/skills/code.md index ef808cf..1edc1e0 100644 --- a/implement/skills/code.md +++ b/implement/skills/code.md @@ -311,15 +311,21 @@ does, verify each mapped TC ID before proceeding to commit: Eventually/Consistently in Ginkgo, polling in pytest). 3. If a TC ID mapped to this task has no corresponding test with sufficient assertion depth, write the missing test (Step 3b), run - it (Step 3d), re-run the review gate, then re-check. + it (Step 3d), stage the new files (`git add`), re-run the review + gate, then re-check. This is a hard gate — the task cannot proceed to commit until every -mapped TC ID has coverage. The testplan is a floor, not a ceiling: -tests discovered through TDD that are not tied to any TC ID are -expected and encouraged. - -If no story-scoped testplan exists, or this task has no mapped TC IDs, -skip this check. +mapped TC ID has coverage or is explicitly marked N/A with a non-empty +rationale (e.g., the test case describes behavior that this story's +implementation made inapplicable). The testplan is a floor, not a +ceiling: tests discovered through TDD that are not tied to any TC ID +are expected and encouraged. + +If no story-scoped testplan exists and `02-plan.md` has no Test Plan +Coverage section, skip this check. However, if `02-plan.md` has TC +mappings but `testplan.md` is missing or unreadable, stop and report +the inconsistency — the plan references a testplan that the code phase +cannot access. #### 3g: Commit diff --git a/implement/skills/validate.md b/implement/skills/validate.md index 62cabee..3826d69 100644 --- a/implement/skills/validate.md +++ b/implement/skills/validate.md @@ -258,7 +258,10 @@ This is an intentional independent verification: if the plan's bookkeeping or the code phase's gate drifted from reality, this step catches it. -If `testplan.md` does not exist, skip this step entirely. +If `testplan.md` does not exist and `02-plan.md` has no Test Plan +Coverage section, skip this step entirely. However, if `02-plan.md` +has TC mappings but `testplan.md` is missing or unreadable, stop and +report the inconsistency. 1. Read `testplan.md` and extract all TC IDs. 2. For each TC ID (except those the validate phase agrees are From 2275d98e530970848916c03433ef2703207208e0 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Wed, 29 Jul 2026 10:17:31 -0400 Subject: [PATCH 26/38] Strengthen sync Story-ref warnings and complete revise testplan lifecycle MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Sync: unresolved Story references now produce a specific warning about downstream Jira-key filtering impact (preserve-and-warn, not silent leave-as-is) - Revise: handle first-publish case (testplan exists locally but testplan_file_path not in metadata — advise re-running /publish) - Revise: handle removal case (testplan deleted locally but still in docs repo — git rm from docs repo, clear metadata field) - Revise summary template includes removed TCs example Assisted-by: Claude Opus 4.6 (1M) --- design/skills/revise.md | 21 ++++++++++++++++++++- design/skills/sync.md | 7 +++++-- 2 files changed, 25 insertions(+), 3 deletions(-) diff --git a/design/skills/revise.md b/design/skills/revise.md index 5a5c6c4..1f25201 100644 --- a/design/skills/revise.md +++ b/design/skills/revise.md @@ -212,8 +212,26 @@ Read and follow `../../_shared/recipes/render-provenance-footer.md` with `WORKFLOW=design`, `ISSUE_KEY={issue-key}`, `TARGET_FILE="{docs_repo_path}/{design_file_path}"`. +**Testplan docs-repo sync:** + If `07-testplan.md` exists and `publish-metadata.json` contains a -`testplan_file_path` field, also copy the testplan to the docs repo. +`testplan_file_path` field, copy the testplan to the docs repo (see +sync-manifest guard below). + +If `07-testplan.md` exists but `publish-metadata.json` does NOT contain +`testplan_file_path` (testplan was created after initial publish), note +to the user: "Testplan exists locally but was not included in the +original publish. Re-run `/publish` to include it in the docs repo." + +If `07-testplan.md` does NOT exist but `publish-metadata.json` contains +`testplan_file_path` (testplan was removed during revision), remove the +published testplan from the docs repo: + +```bash +git -C "{docs_repo_path}" rm "{testplan_file_path}" +``` + +Remove `testplan_file_path` from `publish-metadata.json`. **Sync-manifest guard:** If `.artifacts/design/{issue-key}/sync-manifest.json` exists, the published testplan's Story field must use Jira keys. Before @@ -266,6 +284,7 @@ Summarize what changed: ### Testplan Changes - {TC-FR1-03 added — new acceptance criterion on Story 1.01} - {TC-NFR2-01 updated — expected result changed to match revised design} +- {TC-FR2-02 removed — requirement FR-2 no longer in scope} - {Omit this section if the testplan did not change or does not exist} ### Consistency Updates diff --git a/design/skills/sync.md b/design/skills/sync.md index 455b859..3bfc9a9 100644 --- a/design/skills/sync.md +++ b/design/skills/sync.md @@ -615,8 +615,11 @@ can filter by Jira key. keys (e.g., `EDM-1234`). Use the reference resolution logic described in the Reference Resolution section (look up `story-{NN}-*.md` under `epic-{N}/` in the manifest to find the Jira key). -3. If a Story reference cannot be resolved (not in the manifest), leave - it as-is and note it for the user. +3. If a Story reference cannot be resolved (not in the manifest), + preserve the local reference as-is and warn the user: "Story + reference '{local ref}' could not be resolved to a Jira key — + downstream workflows that filter by Jira key will not match this + test case." Include unresolved references in the Step 8 report. **Write the resolved testplan to the docs repo:** From 6b64f2c9cf1f4fd781a1ff2efedf3d6cf2aa10fb Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Wed, 29 Jul 2026 10:20:25 -0400 Subject: [PATCH 27/38] Fix expected-zero status message wording in e2e ingest E2e filtering is requirement-based, so the expected-zero message should say "no test cases match this story's requirements" not "match this story." Assisted-by: Claude Opus 4.6 (1M) --- e2e/skills/ingest.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/e2e/skills/ingest.md b/e2e/skills/ingest.md index 3545931..563cc5d 100644 --- a/e2e/skills/ingest.md +++ b/e2e/skills/ingest.md @@ -518,7 +518,8 @@ If this is a first invocation, write If feature-level testplan exists but no matches (expected): "Feature-level testplan found in docs repo but no test cases match - this {story-type} story (expected). No story-scoped testplan written." + this {story-type} story's requirements (expected). No story-scoped + testplan written." If feature-level testplan exists but no matches (anomalous): "Feature-level testplan found in docs repo but no test cases match From 5cc23153b9be7888e8f4f8a76e43440efa1ae13d Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Wed, 29 Jul 2026 11:28:02 -0400 Subject: [PATCH 28/38] Enforce plan-authorized N/A and handle malformed testplans MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - N/A exemptions in code phase reconciliation now require the plan's Test Plan Coverage matrix to already mark the TC ID as N/A with rationale — code phase cannot invent exemptions - Malformed testplans (readable but no parseable TC IDs) now treated as inconsistent (stop and report) alongside missing/unreadable, in both code and validate phases across implement and e2e Assisted-by: Claude Opus 4.6 (1M) --- e2e/skills/code.md | 16 +++++++++------- e2e/skills/validate.md | 4 ++-- implement/skills/code.md | 17 +++++++++-------- implement/skills/validate.md | 4 ++-- 4 files changed, 22 insertions(+), 19 deletions(-) diff --git a/e2e/skills/code.md b/e2e/skills/code.md index 102f60d..68fa900 100644 --- a/e2e/skills/code.md +++ b/e2e/skills/code.md @@ -278,16 +278,18 @@ commit: the review gate, then re-check. This is a hard gate — the task cannot proceed to commit until every -mapped TC ID has coverage or is explicitly marked N/A with a non-empty -rationale. The testplan is a floor, not a ceiling: validations -discovered through test design that are not tied to any TC ID are -expected and encouraged. +mapped TC ID has coverage. A TC ID may be treated as N/A only if the +plan's Test Plan Coverage matrix already marks it N/A with a non-empty +rationale — the code phase must not invent N/A exemptions that the +plan did not authorize. The testplan is a floor, not a ceiling: +validations discovered through test design that are not tied to any +TC ID are expected and encouraged. If no story-scoped testplan exists and `02-plan.md` has no Test Plan Coverage section, skip this check. However, if `02-plan.md` has TC -mappings but `testplan.md` is missing or unreadable, stop and report -the inconsistency — the plan references a testplan that the code phase -cannot access. +mappings but `testplan.md` is missing, unreadable, or malformed (no +parseable TC IDs), stop and report the inconsistency — the plan +references a testplan that the code phase cannot use. #### 3f: Commit diff --git a/e2e/skills/validate.md b/e2e/skills/validate.md index c8eeeb5..62ebc8c 100644 --- a/e2e/skills/validate.md +++ b/e2e/skills/validate.md @@ -245,8 +245,8 @@ step catches it. If `testplan.md` does not exist and `02-plan.md` has no Test Plan Coverage section, skip this step entirely. However, if `02-plan.md` -has TC mappings but `testplan.md` is missing or unreadable, stop and -report the inconsistency. +has TC mappings but `testplan.md` is missing, unreadable, or malformed +(no parseable TC IDs), stop and report the inconsistency. 1. Read `testplan.md` and extract all TC IDs. 2. For each TC ID (except those the validate phase agrees are diff --git a/implement/skills/code.md b/implement/skills/code.md index 1edc1e0..ea9ecf3 100644 --- a/implement/skills/code.md +++ b/implement/skills/code.md @@ -315,17 +315,18 @@ does, verify each mapped TC ID before proceeding to commit: gate, then re-check. This is a hard gate — the task cannot proceed to commit until every -mapped TC ID has coverage or is explicitly marked N/A with a non-empty -rationale (e.g., the test case describes behavior that this story's -implementation made inapplicable). The testplan is a floor, not a -ceiling: tests discovered through TDD that are not tied to any TC ID -are expected and encouraged. +mapped TC ID has coverage. A TC ID may be treated as N/A only if the +plan's Test Plan Coverage matrix already marks it N/A with a non-empty +rationale — the code phase must not invent N/A exemptions that the +plan did not authorize. The testplan is a floor, not a ceiling: tests +discovered through TDD that are not tied to any TC ID are expected +and encouraged. If no story-scoped testplan exists and `02-plan.md` has no Test Plan Coverage section, skip this check. However, if `02-plan.md` has TC -mappings but `testplan.md` is missing or unreadable, stop and report -the inconsistency — the plan references a testplan that the code phase -cannot access. +mappings but `testplan.md` is missing, unreadable, or malformed (no +parseable TC IDs), stop and report the inconsistency — the plan +references a testplan that the code phase cannot use. #### 3g: Commit diff --git a/implement/skills/validate.md b/implement/skills/validate.md index 3826d69..12e9577 100644 --- a/implement/skills/validate.md +++ b/implement/skills/validate.md @@ -260,8 +260,8 @@ catches it. If `testplan.md` does not exist and `02-plan.md` has no Test Plan Coverage section, skip this step entirely. However, if `02-plan.md` -has TC mappings but `testplan.md` is missing or unreadable, stop and -report the inconsistency. +has TC mappings but `testplan.md` is missing, unreadable, or malformed +(no parseable TC IDs), stop and report the inconsistency. 1. Read `testplan.md` and extract all TC IDs. 2. For each TC ID (except those the validate phase agrees are From eabd1b49643d3ff9128c7081d2220dcd096d0a3c Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Wed, 29 Jul 2026 11:33:30 -0400 Subject: [PATCH 29/38] Fail on incomplete test case entries in reconciliation and validate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Explicitly stop and report when a TC entry is missing Preconditions, Steps, or Expected Results — in both code reconciliation and validate verification, across implement and e2e. Removes ambiguity about what to do with an incomplete test case (previously could be interpreted as "fill in the missing validation"). Assisted-by: Claude Opus 4.6 (1M) --- e2e/skills/code.md | 5 ++++- e2e/skills/validate.md | 5 ++++- implement/skills/code.md | 5 ++++- implement/skills/validate.md | 5 ++++- 4 files changed, 16 insertions(+), 4 deletions(-) diff --git a/e2e/skills/code.md b/e2e/skills/code.md index 68fa900..097fb01 100644 --- a/e2e/skills/code.md +++ b/e2e/skills/code.md @@ -265,7 +265,10 @@ have TC IDs mapped to them in the Test Plan Coverage matrix of commit: 1. Read the full test case entry for each TC ID from `testplan.md` - (the Preconditions, Steps, and Expected Results sections). + (the Preconditions, Steps, and Expected Results sections). If any + of these sections is missing from a test case entry, stop and + report the testplan as malformed — do not attempt to reconcile + against an incomplete test case. 2. Verify that the test scenario's validations assert the Expected Results described in the test case. The match is behavioral, not textual — the test must exercise the described scenario and assert diff --git a/e2e/skills/validate.md b/e2e/skills/validate.md index 62ebc8c..9426f4f 100644 --- a/e2e/skills/validate.md +++ b/e2e/skills/validate.md @@ -248,7 +248,10 @@ Coverage section, skip this step entirely. However, if `02-plan.md` has TC mappings but `testplan.md` is missing, unreadable, or malformed (no parseable TC IDs), stop and report the inconsistency. -1. Read `testplan.md` and extract all TC IDs. +1. Read `testplan.md` and extract all TC IDs. For each TC entry, + verify that Preconditions, Steps, and Expected Results sections + are present. If any entry is missing a required section, stop and + report the testplan as malformed. 2. For each TC ID (except those the validate phase agrees are legitimately N/A based on the rationale in the plan's Test Plan Coverage matrix): diff --git a/implement/skills/code.md b/implement/skills/code.md index ea9ecf3..79d84b0 100644 --- a/implement/skills/code.md +++ b/implement/skills/code.md @@ -302,7 +302,10 @@ mapped to it in the Test Plan Coverage matrix of `02-plan.md`. If it does, verify each mapped TC ID before proceeding to commit: 1. Read the full test case entry for each TC ID from `testplan.md` - (the Preconditions, Steps, and Expected Results sections). + (the Preconditions, Steps, and Expected Results sections). If any + of these sections is missing from a test case entry, stop and + report the testplan as malformed — do not attempt to reconcile + against an incomplete test case. 2. Verify that a test exists (written in Step 3b or a prior task) whose assertions validate the Expected Results described in the test case. The match is behavioral, not textual — the test must diff --git a/implement/skills/validate.md b/implement/skills/validate.md index 12e9577..47a5173 100644 --- a/implement/skills/validate.md +++ b/implement/skills/validate.md @@ -263,7 +263,10 @@ Coverage section, skip this step entirely. However, if `02-plan.md` has TC mappings but `testplan.md` is missing, unreadable, or malformed (no parseable TC IDs), stop and report the inconsistency. -1. Read `testplan.md` and extract all TC IDs. +1. Read `testplan.md` and extract all TC IDs. For each TC entry, + verify that Preconditions, Steps, and Expected Results sections + are present. If any entry is missing a required section, stop and + report the testplan as malformed. 2. For each TC ID (except those the validate phase agrees are legitimately N/A based on the rationale in the plan's Test Plan Coverage matrix): From f78be5c889c6150d67452c27015b860972c7d8ef Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Wed, 29 Jul 2026 11:39:05 -0400 Subject: [PATCH 30/38] Scope testplan publication under explicit existence conditional Restructure publish.md so the sync-manifest guard and testplan copy are clearly nested under the 07-testplan.md existence check with an explicit "skip all testplan steps" instruction when absent. Also moved design.md git-add before the testplan conditional block so it doesn't appear to be part of the testplan path. Assisted-by: Claude Opus 4.6 (1M) --- design/skills/publish.md | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/design/skills/publish.md b/design/skills/publish.md index 7623697..edeed1c 100644 --- a/design/skills/publish.md +++ b/design/skills/publish.md @@ -161,15 +161,23 @@ Read and follow `../../_shared/recipes/render-provenance-footer.md` with `WORKFLOW=design`, `ISSUE_KEY={issue-key}`, `TARGET_FILE="{docs_repo_path}/{release}/{feature}/design.md"`. -If `.artifacts/design/{issue-key}/07-testplan.md` exists, also copy it. +```bash +git -C "{docs_repo_path}" add "{release}/{feature}/design.md" +``` + +**Testplan publication (conditional):** Only if +`.artifacts/design/{issue-key}/07-testplan.md` exists, copy it to the +docs repo. If `07-testplan.md` does not exist, skip all of the +following testplan steps and proceed to the commit. **Sync-manifest guard:** This guard handles re-publishing after `/sync` has run (e.g., `decompose → publish → sync → revise → publish`). If `.artifacts/design/{issue-key}/sync-manifest.json` exists, the published -testplan's Story field must use Jira keys. Before -copying, read the sync manifest and resolve the Story field in each test -case's metadata table (`Story 1.01` → Jira key from manifest). Write the resolved -version to the docs repo — do NOT modify the local `07-testplan.md`. +testplan's Story field must use Jira keys. Before copying, read the sync +manifest and resolve the Story field in each test case's metadata table +(`Story 1.01` → Jira key from manifest). Write the resolved version to +the docs repo — do NOT modify the local `07-testplan.md`. + If `sync-manifest.json` does not exist: ```bash @@ -181,7 +189,7 @@ keys in Story fields) to `{docs_repo_path}/{release}/{feature}/testplan.md` directly — do not `cp` the unresolved local file. ```bash -git -C "{docs_repo_path}" add "{release}/{feature}/design.md" +git -C "{docs_repo_path}" add "{release}/{feature}/testplan.md" ``` If the testplan was copied: From 5d98396fe6d362c5b2330e118eb9a136a37e8b92 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Wed, 29 Jul 2026 11:42:25 -0400 Subject: [PATCH 31/38] Use conditional {pr-title} placeholder in publish gh pr create The prose described two title outcomes (Jira vs non-Jira) but the command always used the Jira format. Now the command uses {pr-title} which is set conditionally above it. Assisted-by: Claude Opus 4.6 (1M) --- design/skills/publish.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/design/skills/publish.md b/design/skills/publish.md index edeed1c..9aba8ae 100644 --- a/design/skills/publish.md +++ b/design/skills/publish.md @@ -248,12 +248,13 @@ Otherwise, omit the testplan bullet entirely.} - Approve when the design accurately reflects a viable implementation approach ``` -Determine `{owner}/{repo}` from `docs_repo_remote`, then create the draft PR. -If `{issue-key}` is a Jira key, prefix the title with it -(`{issue-key}: Design - {title}`); otherwise use `Design: {title}`. +Determine `{owner}/{repo}` from `docs_repo_remote`, then create the +draft PR. Set `{pr-title}` based on whether `{issue-key}` is a Jira +key: if yes, use `{issue-key}: Design - {title}`; otherwise use +`Design: {title}`. ```bash -gh pr create --draft --repo {owner}/{repo} --base {base-branch} --head {branch-name} --title "{issue-key}: Design - {title}" --body-file .artifacts/design/{issue-key}/08-pr-description.md +gh pr create --draft --repo {owner}/{repo} --base {base-branch} --head {branch-name} --title "{pr-title}" --body-file .artifacts/design/{issue-key}/08-pr-description.md ``` ### Step 6: Save Publish Metadata From 2ad55e4240e5133fc04017b066dd95e91444a8f9 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Wed, 29 Jul 2026 11:55:36 -0400 Subject: [PATCH 32/38] Add lint step to reconciliation and clean up publish testplan staging MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add fast quality checks (lint/format) to the reconciliation fix-up flow in both implement and e2e code phases — new test code should pass lint before staging and commit - Clean up publish.md testplan git-add: remove duplicate conditional add, single unconditional add covers both copy and resolved-write paths. Clarify end of testplan-conditional block. Assisted-by: Claude Opus 4.6 (1M) --- design/skills/publish.md | 11 +++++------ e2e/skills/code.md | 5 +++-- implement/skills/code.md | 4 ++-- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/design/skills/publish.md b/design/skills/publish.md index 9aba8ae..96d037f 100644 --- a/design/skills/publish.md +++ b/design/skills/publish.md @@ -188,11 +188,8 @@ If `sync-manifest.json` exists, write the resolved content (with Jira keys in Story fields) to `{docs_repo_path}/{release}/{feature}/testplan.md` directly — do not `cp` the unresolved local file. -```bash -git -C "{docs_repo_path}" add "{release}/{feature}/testplan.md" -``` - -If the testplan was copied: +Regardless of which path was taken (copy or resolved write), stage the +testplan: ```bash git -C "{docs_repo_path}" add "{release}/{feature}/testplan.md" @@ -202,7 +199,9 @@ git -C "{docs_repo_path}" add "{release}/{feature}/testplan.md" git -C "{docs_repo_path}" commit -m "Add design document and testplan for {issue-key}: {title}" ``` -If the testplan does not exist, use the original commit message: +**(End of testplan-conditional block.)** If `07-testplan.md` did not +exist and the testplan steps above were skipped, use the design-only +commit message: ```bash git -C "{docs_repo_path}" commit -m "Add design document for {issue-key}: {title}" diff --git a/e2e/skills/code.md b/e2e/skills/code.md index 097fb01..502e4fa 100644 --- a/e2e/skills/code.md +++ b/e2e/skills/code.md @@ -277,8 +277,9 @@ commit: pytest). 3. If a TC ID mapped to this task has no corresponding validation with sufficient assertion depth, write the missing validation (Step 3b), - run the tests (Step 3c), stage the new files (`git add`), re-run - the review gate, then re-check. + run the tests (Step 3c), run the fast quality checks (Step 3d), + stage the new files (`git add`), re-run the review gate, then + re-check. This is a hard gate — the task cannot proceed to commit until every mapped TC ID has coverage. A TC ID may be treated as N/A only if the diff --git a/implement/skills/code.md b/implement/skills/code.md index 79d84b0..61b58ef 100644 --- a/implement/skills/code.md +++ b/implement/skills/code.md @@ -314,8 +314,8 @@ does, verify each mapped TC ID before proceeding to commit: Eventually/Consistently in Ginkgo, polling in pytest). 3. If a TC ID mapped to this task has no corresponding test with sufficient assertion depth, write the missing test (Step 3b), run - it (Step 3d), stage the new files (`git add`), re-run the review - gate, then re-check. + it (Step 3d), run the fast quality checks (Step 3e), stage the new + files (`git add`), re-run the review gate, then re-check. This is a hard gate — the task cannot proceed to commit until every mapped TC ID has coverage. A TC ID may be treated as N/A only if the From cfd850002607273a28240235f8412b7f4ce795eb Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Wed, 29 Jul 2026 12:01:20 -0400 Subject: [PATCH 33/38] Align N/A reporting, validate mapped TC existence, remove stale publish MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - N/A outcome in report templates now states plan-authorization requirement, consistent with the gate contract - Reconciliation now verifies each mapped TC ID exists in testplan.md before reading its entry — stops and reports if plan references a TC that the testplan doesn't contain - Publish removes stale testplan from docs repo when 07-testplan.md no longer exists locally (re-publish after deletion) Assisted-by: Claude Opus 4.6 (1M) --- design/skills/publish.md | 5 ++++- e2e/skills/code.md | 17 ++++++++++------- implement/skills/code.md | 17 ++++++++++------- 3 files changed, 24 insertions(+), 15 deletions(-) diff --git a/design/skills/publish.md b/design/skills/publish.md index 96d037f..5ce0a5b 100644 --- a/design/skills/publish.md +++ b/design/skills/publish.md @@ -167,7 +167,10 @@ git -C "{docs_repo_path}" add "{release}/{feature}/design.md" **Testplan publication (conditional):** Only if `.artifacts/design/{issue-key}/07-testplan.md` exists, copy it to the -docs repo. If `07-testplan.md` does not exist, skip all of the +docs repo. If `07-testplan.md` does not exist: check whether a +previously published testplan exists at +`{docs_repo_path}/{release}/{feature}/testplan.md`. If it does, remove +it (`git rm`) so the docs repo stays consistent. Then skip all of the following testplan steps and proceed to the commit. **Sync-manifest guard:** This guard handles re-publishing after `/sync` diff --git a/e2e/skills/code.md b/e2e/skills/code.md index 502e4fa..980aa8e 100644 --- a/e2e/skills/code.md +++ b/e2e/skills/code.md @@ -264,11 +264,13 @@ have TC IDs mapped to them in the Test Plan Coverage matrix of `02-plan.md`. If so, verify each mapped TC ID before proceeding to commit: -1. Read the full test case entry for each TC ID from `testplan.md` - (the Preconditions, Steps, and Expected Results sections). If any - of these sections is missing from a test case entry, stop and - report the testplan as malformed — do not attempt to reconcile - against an incomplete test case. +1. For each mapped TC ID, locate its entry in `testplan.md`. If a + mapped TC ID does not exist in the testplan, stop and report the + inconsistency — the plan references a test case that the testplan + does not contain. Read the full test case entry (the Preconditions, + Steps, and Expected Results sections). If any of these sections is + missing, stop and report the testplan as malformed — do not attempt + to reconcile against an incomplete test case. 2. Verify that the test scenario's validations assert the Expected Results described in the test case. The match is behavioral, not textual — the test must exercise the described scenario and assert @@ -452,8 +454,9 @@ After all tasks are complete (or if interrupted), write: - verified: validation existed and had sufficient assertion depth - written: validation was written to satisfy this TC ID - strengthened: validation existed but lacked assertions; added them - - N/A: test case found inapplicable (explain in Notes and in - Deviations from Plan)} + - N/A: test case marked inapplicable in the plan's Test Plan Coverage + matrix with a non-empty rationale (explain in Notes and in + Deviations from Plan — code phase cannot invent N/A exemptions)} ## Notes diff --git a/implement/skills/code.md b/implement/skills/code.md index 61b58ef..1c05422 100644 --- a/implement/skills/code.md +++ b/implement/skills/code.md @@ -301,11 +301,13 @@ After the self-review gate passes, check whether this task has TC IDs mapped to it in the Test Plan Coverage matrix of `02-plan.md`. If it does, verify each mapped TC ID before proceeding to commit: -1. Read the full test case entry for each TC ID from `testplan.md` - (the Preconditions, Steps, and Expected Results sections). If any - of these sections is missing from a test case entry, stop and - report the testplan as malformed — do not attempt to reconcile - against an incomplete test case. +1. For each mapped TC ID, locate its entry in `testplan.md`. If a + mapped TC ID does not exist in the testplan, stop and report the + inconsistency — the plan references a test case that the testplan + does not contain. Read the full test case entry (the Preconditions, + Steps, and Expected Results sections). If any of these sections is + missing, stop and report the testplan as malformed — do not attempt + to reconcile against an incomplete test case. 2. Verify that a test exists (written in Step 3b or a prior task) whose assertions validate the Expected Results described in the test case. The match is behavioral, not textual — the test must @@ -495,8 +497,9 @@ After all tasks are complete (or if interrupted), write: - verified: test existed and had sufficient assertion depth - written: test was written to satisfy this TC ID - strengthened: test existed but lacked assertions; added them - - N/A: test case found inapplicable (explain in Notes and in - Deviations from Plan)} + - N/A: test case marked inapplicable in the plan's Test Plan Coverage + matrix with a non-empty rationale (explain in Notes and in + Deviations from Plan — code phase cannot invent N/A exemptions)} ## Coverage Notes From d21d9019c0e12391f86cb225db0e7362a159ee5b Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Wed, 29 Jul 2026 12:09:42 -0400 Subject: [PATCH 34/38] Use explicit git rm command for stale testplan deletion in publish Provide the full git -C "{docs_repo_path}" rm command instead of an implicit git rm reference, consistent with all other git commands in the publish flow. Assisted-by: Claude Opus 4.6 (1M) --- design/skills/publish.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/design/skills/publish.md b/design/skills/publish.md index 5ce0a5b..d4be259 100644 --- a/design/skills/publish.md +++ b/design/skills/publish.md @@ -169,9 +169,13 @@ git -C "{docs_repo_path}" add "{release}/{feature}/design.md" `.artifacts/design/{issue-key}/07-testplan.md` exists, copy it to the docs repo. If `07-testplan.md` does not exist: check whether a previously published testplan exists at -`{docs_repo_path}/{release}/{feature}/testplan.md`. If it does, remove -it (`git rm`) so the docs repo stays consistent. Then skip all of the -following testplan steps and proceed to the commit. +`{docs_repo_path}/{release}/{feature}/testplan.md`. If it does, remove it: + +```bash +git -C "{docs_repo_path}" rm -- "{release}/{feature}/testplan.md" +``` + +Then skip all of the following testplan steps and proceed to the commit. **Sync-manifest guard:** This guard handles re-publishing after `/sync` has run (e.g., `decompose → publish → sync → revise → publish`). If From b60545e9927fc5c1c10469aa209b28c302f248a4 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Thu, 30 Jul 2026 08:24:31 -0400 Subject: [PATCH 35/38] Bump design version to 0.6.0 after main advanced to 0.5.0 Main's PR #89 bumped design to 0.5.0 for the dependency link fix. Our testplan feature needs its own minor bump above that baseline. Assisted-by: Claude Opus 4.6 (1M) --- design/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/design/SKILL.md b/design/SKILL.md index f3cbdde..ef3b438 100644 --- a/design/SKILL.md +++ b/design/SKILL.md @@ -1,6 +1,6 @@ --- name: design -version: 0.5.0 +version: 0.6.0 description: >- Design-and-decompose workflow that takes a PRD, researches the problem space, drafts a technical design document, decomposes work into Jira-ready epics and From 1bc0b138a8d4502d9e2dcd52b4299a3891dc6b96 Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Sun, 2 Aug 2026 18:52:05 -0400 Subject: [PATCH 36/38] Clarify testplan conditional scoping and decompose guardrails Refactor the testplan docs-repo sync sections in publish, revise, and respond to use explicit skip-list guards (matching the pattern already used in sync.md Step 7) instead of narrative prose with ambiguous nesting. Reframe a negative instruction in decompose as a positive statement, and add a carve-out for all-[DOCS] decompositions in the Step 9 structural check. Assisted-by: Claude Opus 4.6 (1M) --- design/skills/decompose.md | 6 ++++-- design/skills/publish.md | 30 ++++++++++++++++-------------- design/skills/respond.md | 13 ++++++++----- design/skills/revise.md | 28 ++++++++++++++++------------ 4 files changed, 44 insertions(+), 33 deletions(-) diff --git a/design/skills/decompose.md b/design/skills/decompose.md index 73d181c..ca6db10 100644 --- a/design/skills/decompose.md +++ b/design/skills/decompose.md @@ -214,8 +214,8 @@ PRD Requirements: {FR-1, NFR-1} Design section: {§4.3 API Changes, or specific subsection} ``` -Do NOT include a `## Test Case References` section when writing stories -in Step 5 — that section is added by Step 7c after the testplan is +The story template above is complete as shown. Step 7c adds a +`## Test Case References` section to each story after the testplan is generated. **For `[DOCS]` stories** (see `[DOCS]` story requirements below for the @@ -663,6 +663,8 @@ Quick sanity check before invoking the decomposition review. Verify: 3. Each epic has a corresponding story directory with story files 4. `06-coverage.md` exists and contains at least one mapping row 5. `07-testplan.md` exists and contains at least one test case entry + (if the decomposition contains only `[DOCS]` stories, the testplan + may contain only gap entries and this minimum does not apply) 6. Every non-`[DOCS]` story file has a `## Test Case References` section If structural issues are found, fix them before proceeding. Do not diff --git a/design/skills/publish.md b/design/skills/publish.md index d4be259..5fae6fe 100644 --- a/design/skills/publish.md +++ b/design/skills/publish.md @@ -165,17 +165,27 @@ Read and follow `../../_shared/recipes/render-provenance-footer.md` with git -C "{docs_repo_path}" add "{release}/{feature}/design.md" ``` -**Testplan publication (conditional):** Only if -`.artifacts/design/{issue-key}/07-testplan.md` exists, copy it to the -docs repo. If `07-testplan.md` does not exist: check whether a -previously published testplan exists at -`{docs_repo_path}/{release}/{feature}/testplan.md`. If it does, remove it: +**Testplan publication:** + +**Skip all testplan steps below and proceed directly to the design-only +commit if:** +- `.artifacts/design/{issue-key}/07-testplan.md` does not exist + +If `07-testplan.md` does not exist, check whether a previously published +testplan exists at `{docs_repo_path}/{release}/{feature}/testplan.md`. +If it does, remove it before committing: ```bash git -C "{docs_repo_path}" rm -- "{release}/{feature}/testplan.md" ``` -Then skip all of the following testplan steps and proceed to the commit. +Then commit with the design-only message: + +```bash +git -C "{docs_repo_path}" commit -m "Add design document for {issue-key}: {title}" +``` + +**If `07-testplan.md` exists**, copy it to the docs repo: **Sync-manifest guard:** This guard handles re-publishing after `/sync` has run (e.g., `decompose → publish → sync → revise → publish`). If @@ -206,14 +216,6 @@ git -C "{docs_repo_path}" add "{release}/{feature}/testplan.md" git -C "{docs_repo_path}" commit -m "Add design document and testplan for {issue-key}: {title}" ``` -**(End of testplan-conditional block.)** If `07-testplan.md` did not -exist and the testplan steps above were skipped, use the design-only -commit message: - -```bash -git -C "{docs_repo_path}" commit -m "Add design document for {issue-key}: {title}" -``` - ### Step 5: Push and Create PR ```bash diff --git a/design/skills/respond.md b/design/skills/respond.md index a8b4316..e3befd7 100644 --- a/design/skills/respond.md +++ b/design/skills/respond.md @@ -263,11 +263,14 @@ Read and follow `../../_shared/recipes/render-provenance-footer.md` with git -C "{docs_repo_path}" add "{design_file_path}" ``` -If `07-testplan.md` exists and `publish-metadata.json` contains a -`testplan_file_path` field, also copy the testplan to the docs repo. -(If the testplan was never published — no `testplan_file_path` in -metadata — testplan changes are applied locally only. Re-run `/publish` -to include the testplan in the docs repo.) +**Skip testplan docs-repo sync if any of these are true:** +- `07-testplan.md` does not exist +- `publish-metadata.json` does not contain a `testplan_file_path` field + (testplan was never published — changes are applied locally only; re-run + `/publish` to include the testplan in the docs repo) + +**If both `07-testplan.md` exists and `publish-metadata.json` contains +`testplan_file_path`**, copy the testplan to the docs repo: **Sync-manifest guard:** If `.artifacts/design/{issue-key}/sync-manifest.json` exists, the published testplan's Story field must use Jira keys (resolved diff --git a/design/skills/revise.md b/design/skills/revise.md index c063fc3..460921a 100644 --- a/design/skills/revise.md +++ b/design/skills/revise.md @@ -214,18 +214,18 @@ Read and follow `../../_shared/recipes/render-provenance-footer.md` with **Testplan docs-repo sync:** -If `07-testplan.md` exists and `publish-metadata.json` contains a -`testplan_file_path` field, copy the testplan to the docs repo (see -sync-manifest guard below). - -If `07-testplan.md` exists but `publish-metadata.json` does NOT contain -`testplan_file_path` (testplan was created after initial publish), note -to the user: "Testplan exists locally but was not included in the -original publish. Re-run `/publish` to include it in the docs repo." - -If `07-testplan.md` does NOT exist but `publish-metadata.json` contains -`testplan_file_path` (testplan was removed during revision), remove the -published testplan from the docs repo: +**Skip all testplan sync steps below if any of these are true:** +- `07-testplan.md` does not exist and `publish-metadata.json` does not + contain a `testplan_file_path` field (no testplan anywhere) +- `07-testplan.md` exists but `publish-metadata.json` does NOT contain + `testplan_file_path` (testplan was created after initial publish) — + note to the user: "Testplan exists locally but was not included in + the original publish. Re-run `/publish` to include it in the docs + repo." + +**If `07-testplan.md` does NOT exist but `publish-metadata.json` +contains `testplan_file_path`** (testplan was removed during revision), +remove the published testplan from the docs repo: ```bash git -C "{docs_repo_path}" rm "{testplan_file_path}" @@ -233,11 +233,15 @@ git -C "{docs_repo_path}" rm "{testplan_file_path}" Remove `testplan_file_path` from `publish-metadata.json`. +**If `07-testplan.md` exists and `publish-metadata.json` contains +`testplan_file_path`**, copy the testplan to the docs repo: + **Sync-manifest guard:** If `.artifacts/design/{issue-key}/sync-manifest.json` exists, the published testplan's Story field must use Jira keys. Before copying, read the sync manifest and resolve the Story field in each test case's metadata table (`Story 1.01` → Jira key from manifest). Write the resolved version to the docs repo — do NOT modify the local `07-testplan.md`. + If `sync-manifest.json` does not exist: ```bash From 3bafa5a0227db05712c43f92e56c19f48dfde8ca Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Mon, 3 Aug 2026 08:44:51 -0400 Subject: [PATCH 37/38] Add explicit Step 3 capture for testplan-filtering fields, tighten publish branching MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add Test Case References and Design Reference as explicit Step 3 capture bullets in both implement and e2e ingest skills — Step 5d depends on these for testplan filtering but they were only implicitly covered by "Summary and description." Also clarify publish.md's no-testplan commit path so the design-only commit is unambiguously outside the stale-testplan-removal conditional. Assisted-by: Claude Opus 4.6 (1M) --- design/skills/publish.md | 14 ++++++-------- e2e/skills/ingest.md | 2 ++ implement/skills/ingest.md | 2 ++ 3 files changed, 10 insertions(+), 8 deletions(-) diff --git a/design/skills/publish.md b/design/skills/publish.md index 5fae6fe..1c5e52c 100644 --- a/design/skills/publish.md +++ b/design/skills/publish.md @@ -167,19 +167,17 @@ git -C "{docs_repo_path}" add "{release}/{feature}/design.md" **Testplan publication:** -**Skip all testplan steps below and proceed directly to the design-only -commit if:** -- `.artifacts/design/{issue-key}/07-testplan.md` does not exist - -If `07-testplan.md` does not exist, check whether a previously published -testplan exists at `{docs_repo_path}/{release}/{feature}/testplan.md`. -If it does, remove it before committing: +**If `07-testplan.md` does not exist:** check whether a previously +published testplan exists at +`{docs_repo_path}/{release}/{feature}/testplan.md`. If it does, remove +it: ```bash git -C "{docs_repo_path}" rm -- "{release}/{feature}/testplan.md" ``` -Then commit with the design-only message: +Commit with the design-only message (regardless of whether a stale +testplan was removed) and skip to Step 5: ```bash git -C "{docs_repo_path}" commit -m "Add design document for {issue-key}: {title}" diff --git a/e2e/skills/ingest.md b/e2e/skills/ingest.md index 563cc5d..5874bc5 100644 --- a/e2e/skills/ingest.md +++ b/e2e/skills/ingest.md @@ -65,6 +65,8 @@ Fetch the story from Jira. Capture: - Acceptance criteria - Testing approach (if present — this is the primary implementation guidance for [QE] stories) - Implementation guidance (if present — may be sparse for [QE] stories) +- Test Case References (if present — TC IDs used for testplan filtering in Step 5d) +- Design Reference (if present — PRD requirement IDs used for testplan filtering in Step 5d) - Story type prefix — verify it is `[QE]`. If it is `[DEV]`, `[UI]`, or another prefix, warn the user that this workflow is designed for `[QE]` stories and ask whether to proceed. - Parent epic key - Story dependencies (linked issues — "depends on", "is blocked by") diff --git a/implement/skills/ingest.md b/implement/skills/ingest.md index a05d483..e261a22 100644 --- a/implement/skills/ingest.md +++ b/implement/skills/ingest.md @@ -62,6 +62,8 @@ Fetch the story from Jira. Capture: - Acceptance criteria - Implementation guidance (if present) - Testing approach (if present) +- Test Case References (if present — TC IDs used for testplan filtering in Step 5d) +- Design Reference (if present — PRD requirement IDs and design section refs) - Story type prefix (`[DEV]`, `[UI]`, etc.) - Parent epic key - Story dependencies (linked issues — "depends on", "is blocked by") From 731ce64b64ac2e302b961180d0d3df55b24b064b Mon Sep 17 00:00:00 2001 From: Andy Dalton Date: Mon, 3 Aug 2026 14:16:56 -0400 Subject: [PATCH 38/38] Fix respond.md testplan mutation aggregates and stale-removal gap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add Overview/Summary rebuild to the modify branch in respond.md's testplan mutation workflow — changing Priority, Automation, Story, or AC affects aggregate counts. Broaden the Gaps rebuild to cover all mutation types (add/modify/remove/reassign), not just removal. Add stale testplan removal path mirroring revise.md when 07-testplan.md is absent but publish-metadata.json still references a published copy. Assisted-by: Claude Opus 4.6 (1M) --- design/skills/respond.md | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/design/skills/respond.md b/design/skills/respond.md index e3befd7..ecccd76 100644 --- a/design/skills/respond.md +++ b/design/skills/respond.md @@ -179,7 +179,9 @@ feedback), apply them in this order: table fields, or sub-section content. The same Expected Results quality gate applies — no banned vague phrases. If the Story assignment changes, update both the old and new story's Test Case References - in step 2 below. + in step 2 below. If any metadata field changed (Priority, Automation, + Story, or AC), update the testplan's Overview counts and Summary + table. - **Removing a test case:** Delete the test case entry (heading and all sub-sections). Update the testplan's Overview counts and Summary table. @@ -204,9 +206,10 @@ feedback), apply them in this order: requirement previously had test cases and now has none, flag it in the coverage matrix Gaps section. -4. **Update testplan Gaps section.** If test cases were removed and a - requirement that previously had test cases now has none, update the - testplan's own Gaps section to reflect the new gap. +4. **Update testplan Gaps section.** After all mutations, rebuild the + Gaps section from the current testplan state: remove gaps for + requirements or story ACs that now have coverage, and add gaps for + those that lost coverage (from removals or Story/AC reassignment). **Update the local artifact:** Update `.artifacts/design/{issue-key}/03-design.md`. @@ -264,11 +267,22 @@ git -C "{docs_repo_path}" add "{design_file_path}" ``` **Skip testplan docs-repo sync if any of these are true:** -- `07-testplan.md` does not exist +- `07-testplan.md` does not exist AND `publish-metadata.json` does not + contain a `testplan_file_path` field (no testplan anywhere) - `publish-metadata.json` does not contain a `testplan_file_path` field (testplan was never published — changes are applied locally only; re-run `/publish` to include the testplan in the docs repo) +**If `07-testplan.md` does NOT exist but `publish-metadata.json` +contains `testplan_file_path`** (testplan was removed), remove the +published testplan from the docs repo: + +```bash +git -C "{docs_repo_path}" rm "{testplan_file_path}" +``` + +Remove `testplan_file_path` from `publish-metadata.json`. + **If both `07-testplan.md` exists and `publish-metadata.json` contains `testplan_file_path`**, copy the testplan to the docs repo: