Skip to content

fix(plan): score plan selection on the plan as optimised, not as clipped - #4403

Merged
springfall2008 merged 1 commit into
mainfrom
fix/clip-export-pv10
Jul 31, 2026
Merged

fix(plan): score plan selection on the plan as optimised, not as clipped#4403
springfall2008 merged 1 commit into
mainfrom
fix/clip-export-pv10

Conversation

@springfall2008

Copy link
Copy Markdown
Owner

Follow-up to #4402, from investigating why the metric got worse after optimisation finished.

The finding

clip_export_slots sets the export percentage shown on the plan and sent to the inverter. Measured across the 20-scenario random benchmark, it is a perfect no-op in the expected case — mid-case cost delta was exactly +0.00 on all 12 calls that reached it. But it still moves the metric, entirely through the PV10 branch:

75 windows clipped across 12 scenarios
mid-case cost : +0.00 on every call
PV10 cost     : +287.33 total, avg +23.94, max +140.08, never negative
metric        : +19.73 total, avg +1.64, max +4.25
7 worse    0 better    5 neutral

Never once beneficial. The clip decides using only the mid-case predict_soc, where the battery never reaches the export target so raising it looks free; under PV10 the battery does go that low and the raised target binds.

Why that matters

The tax lands after optimisation and before should_replace_plan. Both sides of that comparison are post-clip, so it isn't a systematic bias — but the tax is shape-dependent, so a plan taxed 4.25 can lose to one taxed 0.0 on a difference clipping invented rather than one the plans really have.

And it buys nothing: by the time clipping runs, the affected slot is usually hours out and will be re-planned many times before it executes, so constraining its PV10 branch today has no bearing on what the inverter does.

The fix

Clipping is unchanged. Executed and displayed percentages are exactly as before — the percentage should reflect the expected case, which is what clipping gives.

calculate_plan keeps a snapshot of the plan taken before clipping, and selection scores that. Both sides fall back to the clipped plans together when either snapshot is missing (first recompute after a restart), so a fresh plan is never scored untaxed against a taxed incumbent. The snapshot is persisted alongside the plan so a restart doesn't lose it.

Rejected alternative

The first attempt made clip_export_slots take its SoC range across both traces, so the clip became a true no-op in both. That fixed the metric (6 better, 1 worse, avg −0.50) but changed the executed percentages — targets moved down, e.g. 89→86, 12→11. That's the wrong trade: it buys a cleaner metric by making the number sent to the inverter less representative of the expected case, and erodes the "Export level adjustments for safety" intent.

Tests

  • plan_scoring_pair unit tested, including that both sides fall back together and never mix a clipped side with a pre-clip one
  • save/load round trip extended to cover the snapshot
  • new plan_preclip test drives two real recomputes on a scenario where clipping demonstrably taxes the metric (3.4897) and asserts the snapshot scores no worse than the plan that executes. Taking the snapshot after clipping instead makes it fail — verified.

Note debug_cases does not cover this path: plan_valid is False on both of its runs, so the comparison block never executes there. That's why the new test drives the scenario harness instead, on its own PredBat instance since read_debug_yaml reconfigures the shared fixture.

Full ./run_all green (281.66s, 0 failures), ./run_pre_commit clean.

Still open

clip_charge_slots has a similar one-directional signature — 9 of 20 scenarios worse, 0 better, +14.44 total — but by a different mechanism: mid cost moves while final SoC does not, so it appears to be clipping away slots that are doing real work rather than the mid-vs-PV10 split above. It needs its own diagnosis and is deliberately not touched here.

🤖 Generated with Claude Code

clip_export_slots and clip_charge_slots set the charge/export percentage shown
on the plan and sent to the inverter. Measured across the random benchmark,
export clipping changes the mid-case cost by exactly 0.00 on every call - it is
a no-op in the expected world - but it still moves the metric, entirely through
the PV10 branch, by an amount that depends on plan shape: 7 of 12 scenarios
taxed, 0 improved, up to +4.25.

That tax lands between optimisation and should_replace_plan, so plan selection
was partly deciding on a difference clipping invented rather than one the plans
really have. A plan taxed 4.25 could lose to one taxed 0.0 on that alone. By
then the slot is usually hours out and will be re-planned many times before it
executes, so constraining its PV10 branch today buys nothing.

Clipping is unchanged - the executed and displayed percentages are exactly as
before. calculate_plan now keeps a snapshot of the plan taken before clipping
and scores selection on that. Both sides fall back to the clipped plans
together when either snapshot is missing, so a fresh plan is never scored
untaxed against a taxed incumbent. The snapshot is persisted with the plan so a
restart does not lose it.

An earlier attempt made clipping itself span both traces. That fixed the metric
but changed the executed percentages, which is the wrong trade: the percentage
should reflect the expected case.

Tests: plan_scoring_pair is unit tested including the never-mix fallback, the
save/load round trip covers the snapshot, and a new plan_preclip test drives
two real recomputes on a scenario where clipping demonstrably taxes the metric
(3.49) and asserts the snapshot scores no worse than the executed plan. Taking
the snapshot after clipping instead makes that test fail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 31, 2026 14:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes plan selection to score plans as optimised (pre-clipping) rather than as executed (post-clipping), avoiding shape-dependent metric changes introduced by clip_export_slots/clip_charge_slots influencing should_replace_plan comparisons.

Changes:

  • Add a plan_preclip snapshot and use it during plan comparison so scoring reflects the optimised plan before clipping.
  • Persist/restore the plan_preclip snapshot alongside the saved plan to keep comparisons consistent across restarts.
  • Extend unit tests to cover the scoring-pair fallback rules, persistence round-trip, and an end-to-end scenario where clipping measurably taxes the metric.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
apps/predbat/plan.py Introduces plan_scoring_pair() and snapshots the pre-clip plan for selection scoring; carries the chosen snapshot forward for next cycle comparisons.
apps/predbat/predbat.py Persists plan_preclip in save_plan() and restores it in load_plan() with basic shape validation.
apps/predbat/tests/test_plan_tiebreak.py Adds targeted unit tests for plan_scoring_pair() behavior (preclip when available; fallback together when missing).
apps/predbat/tests/test_plan_preclip.py Adds an end-to-end scenario test ensuring pre-clip scoring avoids clipping’s metric tax influencing plan selection.
apps/predbat/tests/test_plan_persistence.py Extends persistence tests to include plan_preclip round-trip coverage.
apps/predbat/unit_test.py Registers the new plan_preclip test.
.cspell/custom-dictionary-workspace.txt Adds preclip to the spelling dictionary.

Comment thread apps/predbat/tests/test_plan_preclip.py
@springfall2008
springfall2008 merged commit 565a4e2 into main Jul 31, 2026
3 checks passed
@springfall2008
springfall2008 deleted the fix/clip-export-pv10 branch July 31, 2026 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants