Flag provenance sessions with an unverified document origin

[tchughesiv]

Title

[#94]: flag provenance sessions with an unverified document origin

Problem

The provenance footer (## Provenance in published PRD/design docs) is meant to flag documents whose structure was never verified against the template from origin. But provenance_kind() only checks whether every event in the log is commit — it never checks what the first event actually was. A session that starts at /respond, /revise, or a manual-edit record (no prior /draft or commit snapshot) renders provenance_kind: "session", identical to a document that went through /draft correctly — with no disclaimer, no distinguishing signal. Reviewers and automated triage tooling consuming the footer have no way to tell the two apart. Already happened in production: osac-project/enhancement-proposals#124.

Root Cause

provenance_kind() (_shared/scripts/provenance.py) is a binary classifier — commit_only vs session — with no check on events[0]["phase"]. Five reachable call sites can produce a log whose first-ever event is respond, revise, or manual-edit: prd/skills/respond.md, design/skills/respond.md, prd/skills/revise.md, design/skills/revise.md (all call capture-provenance-event unconditionally), and _shared/recipes/record-manual-edit.md (invoked from inside revise.md's manual-edit detection). The existing commit-snapshot safety net in render_footer() only fires when provenance.json doesn't exist yet — but every one of those five paths creates provenance.json via capture before the corresponding render step runs, so the safety net is structurally unreachable from them. Full analysis in .artifacts/bugfix/94/root-cause.md.

Fix

Added an explicit, independently-computed origin_untracked boolean:

def origin_untracked(events: list[dict[str, Any]]) -> bool:
    if not events:
        return False
    if provenance_kind(events) == "commit_only":
        return False
    return events[0].get("phase") != "draft"

Wired into build_metrics_payload() (new origin_untracked field in the machine-readable comment) and build_footer() (new disclaimer line, conditionally appended, mirroring COMMIT_ONLY_NOTE's pattern). Purely additive — no existing branch, field, or test assertion changes shape for any previously-correct case.

An independent self-review round (before this PR was opened) caught a gap in the first-draft version of this function: it originally checked only events[0].get("phase") not in ("draft", "commit"), which meant a document that started as a bare commit safety-net snapshot (never drafted) and later picked up a real authoring event — e.g. [commit, revise] — was still classified as origin-tracked, silently reproducing the same bug via an uncovered path. Corrected to the version above, which treats a log as tracked only if it's still commit_only (covered by its own disclaimer) or its first-ever event was draft.

Files changed:

• _shared/scripts/provenance.py — origin_untracked(), wired into build_metrics_payload()/build_footer()
• _shared/scripts/test_provenance.py — 21 new regression tests
• _shared/provenance-schema.md — documents the new field/disclaimer (0.1.0 → 0.2.0, MINOR per AGENTS.md's versioning rules — this is a behavioral schema addition, not a wording-only change). Also adds a "Limitations" bullet noting that origin_untracked reflects the local session log only: since provenance.json is gitignored, a lost local log (fresh clone, new worktree, cleaned .artifacts/, CI runner) can make a genuinely-drafted document render origin_untracked:true, and the flag doesn't self-heal on a later /draft run. Recovery and a proposed rehydration fix are tracked separately in provenance: recover origin_untracked false positives when the local session log is lost #97 — out of scope here since it's an orthogonal enhancement, not a regression from this fix (the prior behavior for this same scenario was a silent false negative, i.e. exactly issue Provenance has no signal for a session that starts at /respond or /revise with no /draft (or commit-snapshot) origin — affects prd and design #94).
• prd/SKILL.md, design/SKILL.md (both 0.7.0 → 0.7.1) — PATCH cascade bumps only (no content changes) per AGENTS.md's shared-file cascade rule, since both workflows transitively consume provenance.py/provenance-schema.md. Rebased onto main after companion PR Fix /revise and /respond skipping project-level template overrides in prd and design #95 merged (which had already bumped both files to 0.7.0); re-bumped past that to 0.7.1 to preserve the cascade.

No changes to respond.md/revise.md instruction files were needed — the issue's own framing holds: closing the gap in the shared script covers every workflow-level path that can reach it, including /respond's PR-number/file-path fallback.

Testing

• 40/40 unit tests pass (python3 -m unittest discover -s _shared/scripts -p 'test_*.py'), including 21 new regression tests covering: respond/revise/manual-edit as the first event (the reported bug), draft-first and commit-only control cases (unaffected), and the commit-then-authoring composite case caught during self-review.
• Deterministic manual reproduction (isolated scratch repo, not the real .artifacts/) re-run against the fix: bug case now renders the disclaimer and origin_untracked:true; both control cases (draft-first, commit_only) render unchanged with origin_untracked:false.
• Full CI-equivalent validation gate green: structural checks (pre-review-checks.py, 229 pass / 1 pre-existing unrelated warning), skillsaw lint (0 errors), markdownlint-cli2 (0 errors, 229 files), validate-versions.sh (0 errors), lychee link check (0 errors). Re-verified after rebasing onto main post-Fix /revise and /respond skipping project-level template overrides in prd and design #95 merge.
• Verified zero file/logic overlap with companion PR Fix /revise and /respond skipping project-level template overrides in prd and design #95 (issue /revise and /respond don't resolve the project-level template override in either prd or design — only /draft does #93, template-override-resolution) — see Risk Assessment.

Full test report: .artifacts/bugfix/94/verification.md.

Confidence

HIGH — the bug was proven by direct, deterministic reproduction (not inferred from logs), the fix is small and purely additive, and a same-model self-review round independently caught and closed a real gap in the first-draft fix before this PR was opened.

Rollback

Revert this commit. The change is additive-only (one new JSON field, one new conditionally-rendered footer line) — reverting restores the exact prior behavior with no data migration, since provenance.json's persisted schema_version is unchanged.

Risk Assessment

LOW — no changes to workflow instruction files (respond.md/revise.md), only to the shared provenance script, its tests, and its schema doc. Companion PR #95 (issue #93) touched a disjoint file set (respond.md/revise.md/draft.md/controller.md/README.md in prd/design, plus a new template-override-resolution.md recipe) with zero logic overlap — confirmed via git merge-tree simulation before #95 merged. #95 has since merged into main (64ad5c1); this branch was rebased on top of it and the one anticipated conflict — the version lines on prd/SKILL.md/design/SKILL.md, which #95 had already bumped to 0.7.0 — was resolved by re-bumping both to 0.7.1. No other conflicts.

Fixes #94

Summary

• Updated shared provenance handling in _shared/ for PRD and design workflows.
• Added origin_untracked detection for sessions that start with /respond, /revise, manual-edit, or other non-draft events.
• Added machine-readable metadata and a conditional footer disclaimer for untracked origins.
• Preserved existing behavior for empty and commit_only histories.
• Updated the provenance schema from 0.1.0 to 0.2.0.
• Bumped the design skill to 0.6.1 and the PRD skill to 0.6.4.
• Added regression coverage for entry points, fallback behavior, metadata, footer output, drift, and multi-phase histories.
• No workflow instruction files changed. Shared provenance conventions now apply consistently across both workflows.

[coderabbitai]

Walkthrough

Changes

Provenance origin tracking

Layer / File(s)	Summary
Origin classification and footer contract _shared/provenance-schema.md, _shared/scripts/provenance.py	The schema and script define origin_untracked, include it in metrics, and render a footer disclaimer for histories without an initial /draft origin.
Origin classification and rendering tests _shared/scripts/test_provenance.py	Tests cover classification, metrics, footer output, commit-only controls, drift, and multi-phase histories.
Workflow skill metadata updates design/SKILL.md, prd/SKILL.md	The design and PRD skill versions are incremented.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• flightctl/ai-workflows#60: Updates the same design and PRD skill version metadata.
• flightctl/ai-workflows#73: Modifies the shared provenance script, tests, and schema.

Suggested labels: workflow-structure, shared-resources, scripts

Suggested reviewers: eranco74, amir-yogev-gh, adalton

🚥 Pre-merge checks | ✅ 12

✅ Passed checks (12 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The implementation adds origin detection, metadata, and footer signaling for non-draft starts, including respond and revise paths required by issue #94.
Out of Scope Changes check	✅ Passed	The schema update, regression tests, and skill version bumps support the stated provenance change and do not introduce unrelated code changes.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Ai-Attribution	✅ Passed	AI use is explicit in the PR and commit; HEAD includes the acceptable Assisted-by: Claude Opus 4.6 trailer, and no AI Co-Authored-By trailer appears.
No-Absolute-Paths-In-Skills	✅ Passed	The PR only changes version lines in design/SKILL.md and prd/SKILL.md; no absolute filesystem paths were added. Existing /home/user and /workspace examples are exempt fenced examples.
Skill-Md-Under-30-Lines	✅ Passed	The two changed SKILL.md files contain 26 lines (prd) and 28 lines (design), both under the 30-line limit.
Command-Colon-Notation	✅ Passed	All 75 tracked files under commands/ have YAML frontmatter with exactly one name key matching parent workflow:phase notation; the audit found 0 problems.
No-Orphaned-References	✅ Passed	Changed workflow references resolve to existing controller/guideline files; all declared commands and all skill files are referenced, and the schema script reference also exists.
No-Content-Duplication	✅ Passed	Both changed workflow files only bump version metadata. Their instruction prose is unchanged, and the companion guidelines/controller files contain no new verbatim multi-line duplication.
Step-Sequencing	✅ Passed	No changed file matches the skills/*.md scope; only design/SKILL.md and prd/SKILL.md changed, and their visible Quick Start steps are sequential (1, 2).
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: flagging provenance sessions with an unverified document origin.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}