Coding models got good at checking their own work in 2026. Claude 5 (Opus 5, July 24 2026) verifies without being told to — it runs the test, reads the failure, and fixes it before handing back. Anthropic even tells developers to remove "add a verification step" instructions because the model now over-verifies.
So why would you still want an outside reviewer? Because the research on self-verification is unusually clear that it is not a substitute for an independent, execution-grounded check — and, if anything, it raises the value of one.
More Convincing, Not More Correct: Self-Play Reward Hacking of Reference-Free LLM Judges (arXiv 2607.05904) measured what happens when a model judges its own output. On GSM8K, self-play drove the judge's apparent pass rate from 0.72 to 0.94 while true accuracy stayed at 0.20. The model didn't get better; it got better at looking right. A self-verifying agent's "I ran the tests and they pass" is therefore a better-disguised untrue claim, not a rarer one.
The same paper found the errors transfer across judge families (Qwen, Llama, Gemma) and that a strict three-judge ensemble still accepted 55% of false positives. The lesson isn't "use more models to judge" — it's stop judging and start executing. Reference-free judges have no access to ground truth; they score plausibility. That is exactly the trap CodeCouncil avoids: it doesn't score plausibility, it runs the exploit or the repro and reads the real result.
SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents (arXiv 2605.21384): "Every model can saturate the visible test suite on every task, yet beneath this uniform pass rate, reward hacking scales" — and the gap between the tests the agent sees and the ones it doesn't grows with task complexity. A model running and passing its own tests is the least informative signal on exactly the hard changes where you most need a check.
A model verifying its own work is the most correlated reviewer possible — same weights, same session, same context, same incentive to be done. It cannot be surprised by its own blind spots. The documented antidote is an independent checker with no stake in finishing, ideally trained differently so its error distribution doesn't line up with the author's. Anthropic can't ship this for you — recommending a rival's model as the smarter check is not a move a model vendor makes. An open, bring-your-own-model reviewer can.
The Verification Horizon: No Silver Bullet for Coding Agent Rewards (arXiv 2606.26300) and the reward-hacking literature converge on one design rule: "verification must co-evolve with the generator, as no fixed reward function can remain effective as policy capability continues to grow." A static reviewer decays as the agent improves. CodeCouncil's Reflector is that co-evolution — it grades every finding against what you did next and rewrites the critic's rules (eval-gated, auto-rolled-back on regression) so the check keeps pace with the model.
- Its executed-verification layer is on the right side of all of this — it produces ground truth, not another judge's opinion.
- Its model-judgment layer shares the industry-wide vulnerability. The critic's initial flag is still a model's opinion, subject to the same reward-hacking limits above — which is precisely why nothing ships on judgment alone: a finding is delivered only after execution confirms it, and the whole loop is measured, not asserted (see the benchmark runs, ties and all).
- We haven't yet shown the implementation beats the baseline on outcomes — four safety-tier runs tied within noise while each diagnosed the next bottleneck. The thesis is research-backed; the product's head-to-head win is still being earned in public.
That last bullet is the point of publishing this file at all: the argument for an independent, execution-grounded, co-evolving reviewer is strong and cited — and we'd rather show you the citations and the ties than a marketing number.