Prove assert_tool_call can fail. Fixture: a cassette whose tool-call sequence omits or reorders the required call, plus one whose argument at an asserted dotted path diverges.
Definition of done
- A NEW fixture under
tests/falsifiability/fixtures/ that violates the
property under test.
- A test invoking the real assertion path end-to-end as a library call
(CHARTER §2: every operation is a library call). When a process exit code
must be checked, capture it directly — never through a pipe.
- Two checks at two layers: the FAIL verdict, and the non-zero exit code.
- One appended line in
docs/how-flowproof-tests-flowproof.md naming the
assertion, the fixture, and the defect class it guards.
Hard bars
The Adversary will refuse these even with green CI:
- Do not modify any existing test, committed fixture, assertion implementation,
or anything under scripts/gate/. Committed cassettes are ground truth
(§2) — craft new fixtures; never edit a committed cassette into a violation.
- If your red-path test PASSES where you expected FAIL, you have found a real
false-green defect. That is a RESULT, not a task. Report it with the fixture
attached for the Ledger keeper; the fix is a separate issue you must not
self-authorise.
- Fixtures are deterministic and offline. Replay makes zero LLM calls and
requires no API key.
Charter: Milestone 2 criteria 6–8 (amendment 2026-07-30). Brief:
docs/loop/milestones/m2-falsifiability.md. Parent: #61.
Prove
assert_tool_callcan fail. Fixture: a cassette whose tool-call sequence omits or reorders the required call, plus one whose argument at an asserted dotted path diverges.Definition of done
tests/falsifiability/fixtures/that violates theproperty under test.
(CHARTER §2: every operation is a library call). When a process exit code
must be checked, capture it directly — never through a pipe.
docs/how-flowproof-tests-flowproof.mdnaming theassertion, the fixture, and the defect class it guards.
Hard bars
The Adversary will refuse these even with green CI:
or anything under
scripts/gate/. Committed cassettes are ground truth(§2) — craft new fixtures; never edit a committed cassette into a violation.
false-green defect. That is a RESULT, not a task. Report it with the fixture
attached for the Ledger keeper; the fix is a separate issue you must not
self-authorise.
requires no API key.
Charter: Milestone 2 criteria 6–8 (amendment 2026-07-30). Brief:
docs/loop/milestones/m2-falsifiability.md. Parent: #61.