Skip to content

Give CautionDrafter the frame tallies RunDebriefer already sees - #713

Open
xmap wants to merge 1 commit into
mainfrom
fix/caution-drafter-sees-capture-progress
Open

Give CautionDrafter the frame tallies RunDebriefer already sees#713
xmap wants to merge 1 commit into
mainfrom
fix/caution-drafter-sees-capture-progress

Conversation

@xmap

@xmap xmap commented Aug 21, 2026

Copy link
Copy Markdown
Owner

Summary

CautionDrafter proposes standing "Caution" advisories on Assets/Procedures from terminal Run events, but CautionDrafterPayload never carried frame tallies, unlike its sibling RunDebriefer, which already receives them via extract_capture_progress and correctly flags DataSuspect. CautionDrafter was structurally blind to frame-count shortfalls: the data existed on the terminal event, but never reached the LLM call.

  • Add capture_progress: dict[str, int] | None = None to CautionDrafterPayload, mirroring RunDebriefPayload's field of the same name.
  • Wire extract_capture_progress(event) into the subscriber, same helper RunDebriefer already uses.
  • System prompt: a shortfall is a fact about ONE Run; RunDebriefer already records it as DataSuspect; a Caution is a claim about an ASSET, which a single Run cannot establish a pattern for. A shortfall alone must never originate a proposal, and NoAction is correct for an isolated shortfall. Refusing discards nothing: the tallies live permanently on the terminal event and RunDebriefer's verdict is a durable Decision.
  • The existing lookback-then-propose ProposeSupersede branch (shortfall + a matching existing Active Caution) is kept, but demoted to an explicitly secondary path, not the primary mechanism, because the pilot currently has zero Caution streams and it would otherwise be dead on arrival.
  • capture_progress absence stays ordinary (not every Run is a witnessed capture); never compare a saved count with a collected count (verbatim rule from run_debrief.py, different upstream instruments).
  • Module docstring names cross-Run recurrence as a deliberate v1 scope boundary, not a TODO: this agent reads one terminal event at a time and cannot yet distinguish "this Run was short" from "this Asset is short on a third of its Runs by a consistent margin." Grounded in a pilot measurement (2026-08-21): of 974 completions, all declare the same 1541-frame expectation; 616 met it and 364 fell short by ~11 frames on average, a systematic offset, not noise. Sourcing that recurrence signal without a new port/lookup is the next slice.
  • CAUTION_DRAFTER_PROMPT_TEMPLATE_ID is unchanged (per design, pre-PromptTemplate-aggregate); its one-line description in KNOWN_PROMPT_TEMPLATES was updated to note the payload now carries capture_progress. Note: the ID is stable across this change even though the underlying prompt bytes differ before/after, which is the known provenance limitation of the stopgap registry.

Deliberately does not enable origination from a shortfall pattern. This slice is awareness-only by design: the agent sees the tallies and can reason about them in a corroboration role, but nothing in this change lets a shortfall, by itself or by recurrence, produce a new Caution proposal on its own.

Out of scope (unchanged): no new port/lookup/query/projection, no change to RunDebriefer or extract_capture_progress itself, no change to the 5-choice vocabulary/severity anchors/tier quotas, and the unrelated agent_name repr bug in regenerate_run_debrief/handler.py is left alone.

Test plan

  • uv run pytest apps/api/tests/unit apps/api/tests/architecture -q — 45320 passed, 631 skipped
  • uv run pytest apps/api/tests/integration -q — 1273 passed, 1 skipped
  • make lint — clean
  • make typecheck — 0 errors
  • New tests pin payload/request-building behavior only (frame tallies reach the built request when present, None when absent, both the shortfall-with-matching-Caution and shortfall-with-no-existing-Caution payload shapes), never asserting what the model chooses.

🤖 Generated with Claude Code

CautionDrafterPayload never carried capture_progress, so the agent was
structurally blind to frame-count shortfalls even though the terminal
event's tallies were already being extracted and handed to its sibling
RunDebriefer. Wire the same extract_capture_progress helper into the
subscriber and document the tallies on the payload, mirroring
RunDebriefPayload's field of the same name.

The system prompt draws a hard line: a shortfall is a fact about ONE
Run, RunDebriefer already records it as DataSuspect, and this payload
carries only one Run, so a shortfall alone must never originate a
Caution. The lookback-then-propose ProposeSupersede path survives as a
secondary route, since it is genuinely reachable once any Caution
exists for a target, but the pilot currently has zero Caution streams,
so it is explicitly not presented as the primary mechanism.

Cross-Run recurrence, the signal that would make an asset-level claim
sound, is out of scope here: no new port or lookup. The module
docstring names that as a deliberate v1 boundary, not a TODO, grounded
in the pilot's own numbers (364 of 974 completions short, clustered
tightly around an 11-frame offset), and leaves it for the next slice.
@github-actions

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  apps/api/src/cora/agent/prompts
  __init__.py
  caution_drafter.py
  apps/api/src/cora/agent/subscribers
  caution_drafter.py
Project Total  

This report was generated by python-coverage-comment-action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant