fix(plan): score plan selection on the plan as optimised, not as clipped - #4403
Merged
Conversation
clip_export_slots and clip_charge_slots set the charge/export percentage shown on the plan and sent to the inverter. Measured across the random benchmark, export clipping changes the mid-case cost by exactly 0.00 on every call - it is a no-op in the expected world - but it still moves the metric, entirely through the PV10 branch, by an amount that depends on plan shape: 7 of 12 scenarios taxed, 0 improved, up to +4.25. That tax lands between optimisation and should_replace_plan, so plan selection was partly deciding on a difference clipping invented rather than one the plans really have. A plan taxed 4.25 could lose to one taxed 0.0 on that alone. By then the slot is usually hours out and will be re-planned many times before it executes, so constraining its PV10 branch today buys nothing. Clipping is unchanged - the executed and displayed percentages are exactly as before. calculate_plan now keeps a snapshot of the plan taken before clipping and scores selection on that. Both sides fall back to the clipped plans together when either snapshot is missing, so a fresh plan is never scored untaxed against a taxed incumbent. The snapshot is persisted with the plan so a restart does not lose it. An earlier attempt made clipping itself span both traces. That fixed the metric but changed the executed percentages, which is the wrong trade: the percentage should reflect the expected case. Tests: plan_scoring_pair is unit tested including the never-mix fallback, the save/load round trip covers the snapshot, and a new plan_preclip test drives two real recomputes on a scenario where clipping demonstrably taxes the metric (3.49) and asserts the snapshot scores no worse than the executed plan. Taking the snapshot after clipping instead makes that test fail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes plan selection to score plans as optimised (pre-clipping) rather than as executed (post-clipping), avoiding shape-dependent metric changes introduced by clip_export_slots/clip_charge_slots influencing should_replace_plan comparisons.
Changes:
- Add a
plan_preclipsnapshot and use it during plan comparison so scoring reflects the optimised plan before clipping. - Persist/restore the
plan_preclipsnapshot alongside the saved plan to keep comparisons consistent across restarts. - Extend unit tests to cover the scoring-pair fallback rules, persistence round-trip, and an end-to-end scenario where clipping measurably taxes the metric.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| apps/predbat/plan.py | Introduces plan_scoring_pair() and snapshots the pre-clip plan for selection scoring; carries the chosen snapshot forward for next cycle comparisons. |
| apps/predbat/predbat.py | Persists plan_preclip in save_plan() and restores it in load_plan() with basic shape validation. |
| apps/predbat/tests/test_plan_tiebreak.py | Adds targeted unit tests for plan_scoring_pair() behavior (preclip when available; fallback together when missing). |
| apps/predbat/tests/test_plan_preclip.py | Adds an end-to-end scenario test ensuring pre-clip scoring avoids clipping’s metric tax influencing plan selection. |
| apps/predbat/tests/test_plan_persistence.py | Extends persistence tests to include plan_preclip round-trip coverage. |
| apps/predbat/unit_test.py | Registers the new plan_preclip test. |
| .cspell/custom-dictionary-workspace.txt | Adds preclip to the spelling dictionary. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #4402, from investigating why the metric got worse after optimisation finished.
The finding
clip_export_slotssets the export percentage shown on the plan and sent to the inverter. Measured across the 20-scenario random benchmark, it is a perfect no-op in the expected case — mid-case cost delta was exactly+0.00on all 12 calls that reached it. But it still moves the metric, entirely through the PV10 branch:Never once beneficial. The clip decides using only the mid-case
predict_soc, where the battery never reaches the export target so raising it looks free; under PV10 the battery does go that low and the raised target binds.Why that matters
The tax lands after optimisation and before
should_replace_plan. Both sides of that comparison are post-clip, so it isn't a systematic bias — but the tax is shape-dependent, so a plan taxed 4.25 can lose to one taxed 0.0 on a difference clipping invented rather than one the plans really have.And it buys nothing: by the time clipping runs, the affected slot is usually hours out and will be re-planned many times before it executes, so constraining its PV10 branch today has no bearing on what the inverter does.
The fix
Clipping is unchanged. Executed and displayed percentages are exactly as before — the percentage should reflect the expected case, which is what clipping gives.
calculate_plankeeps a snapshot of the plan taken before clipping, and selection scores that. Both sides fall back to the clipped plans together when either snapshot is missing (first recompute after a restart), so a fresh plan is never scored untaxed against a taxed incumbent. The snapshot is persisted alongside the plan so a restart doesn't lose it.Rejected alternative
The first attempt made
clip_export_slotstake its SoC range across both traces, so the clip became a true no-op in both. That fixed the metric (6 better, 1 worse, avg −0.50) but changed the executed percentages — targets moved down, e.g.89→86,12→11. That's the wrong trade: it buys a cleaner metric by making the number sent to the inverter less representative of the expected case, and erodes the "Export level adjustments for safety" intent.Tests
plan_scoring_pairunit tested, including that both sides fall back together and never mix a clipped side with a pre-clip oneplan_precliptest drives two real recomputes on a scenario where clipping demonstrably taxes the metric (3.4897) and asserts the snapshot scores no worse than the plan that executes. Taking the snapshot after clipping instead makes it fail — verified.Note
debug_casesdoes not cover this path:plan_validis False on both of its runs, so the comparison block never executes there. That's why the new test drives the scenario harness instead, on its own PredBat instance sinceread_debug_yamlreconfigures the shared fixture.Full
./run_allgreen (281.66s, 0 failures),./run_pre_commitclean.Still open
clip_charge_slotshas a similar one-directional signature — 9 of 20 scenarios worse, 0 better, +14.44 total — but by a different mechanism: mid cost moves while final SoC does not, so it appears to be clipping away slots that are doing real work rather than the mid-vs-PV10 split above. It needs its own diagnosis and is deliberately not touched here.🤖 Generated with Claude Code