fix(plan): stop tweak/second pass regressing an in-progress force export - #4398
Closed
mbuhansen wants to merge 1 commit into
Closed
fix(plan): stop tweak/second pass regressing an in-progress force export#4398mbuhansen wants to merge 1 commit into
mbuhansen wants to merge 1 commit into
Conversation
During a force export at the highest export price of the horizon, the 20:05 recompute cancelled the export and switched to Demand. The plan handed to tweak_plan cost -16670.26 and the plan it returned cost -16308.52 - 361.74 worse - with the running export moved from "start now at 0%" to "start at 20:40". The whole-plan guard only compares against the previous cycle's plan, so the value tweak had just destroyed was invisible and the worse plan was adopted. The export then flapped Exporting -> Demand -> Exporting -> Demand over the next 16 minutes. Two defects combine: 1. tweak_plan and optimise_full_second_pass write the result of optimise_export/optimise_charge_limit straight back with no check that the whole plan improved. Those functions score candidates against the window being turned off, not against the setting the plan already holds, so they return the best of their own candidate set and can silently downgrade a better choice made by an earlier pass. tweak_plan also resets the window start back to start_orig first, re-opening every later start in the window. 2. The in-progress export commitment applied its metric bonus to every candidate in the window, including ones starting after minutes_now. Giving "stop now, restart later" the same bonus as "keep exporting" cancels it out, so the commitment provided no stickiness - only the cost-gate relaxation was correctly scoped. Add plan_metric_current/plan_window_snapshot/plan_window_restore and keep_plan_change_if_improved, and use them to make both passes monotonic: each window change is re-simulated against the whole plan and reverted when it does not improve. Scope the metric bonus to start <= minutes_now, matching the existing keep_export condition. The sibling swap passes already guard themselves this way (optimise_swap_export, optimise_swap_charge); this brings tweak and the second pass in line. Extends tests/test_export_commitment.py with two cases: an in-progress export must not be restarted later inside its own window, and tweak_plan must revert a window change that makes the plan worse (verified to fail without the guard). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Owner
|
Replace with a new implementation, mainly due to preference on the coding, thanks for the fix |
springfall2008
added a commit
that referenced
this pull request
Jul 31, 2026
* docs(plan): design spec for monotonic tweak and second pass Records the root cause analysis for tweak_plan and optimise_full_second_pass writing back window changes that make the whole plan worse, and the design for guarding both passes. Diagnosis, field logs and performance measurements originate from PR #4398 by @mbuhansen. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(plan): stop tweak and second pass regressing the plan tweak_plan and optimise_full_second_pass call optimise_charge_limit / optimise_export one window at a time and write the result straight back with no check that the whole plan improved. Neither optimiser ranks candidates on the metric the plan is finally judged by: both score against a fixed default (charge to max, or export off) rather than the setting the plan already holds, and both rank on a score carrying adjustments the plan metric does not. The reported symptom was a force export at the best price of the horizon being cancelled mid-flight, then flapping Exporting -> Demand for 16 minutes. should_replace_plan could not catch it because it only compares against the previous cycle's plan, not against the value tweak_plan had just destroyed. Both passes now snapshot the single window they are about to change, and revert it unless the whole-plan metric did not worsen. The metric comes from the optimiser rather than a fresh simulation: it already simulates the entire plan for every option it tries, so the unadjusted metric of the option it selected describes the plan we now hold. Both optimisers therefore return that unadjusted metric alongside the adjusted one they rank on, leaving optimise_detailed_pass - which gates on the adjusted score - untouched. Ties are kept, matching optimise_swap_export. Reverting them was tried first and measured worse: a metric-neutral revert leaves a differently shaped plan of equal value, and the passes after tweak amplify that downstream. Also scopes the in-progress export commitment bonus to candidates that cover the current minute. Applying it to later-starting candidates too cancels it out, which is what let "stop now, restart later" win. Removes the dead best_soc_margin field, which applied after selection and would otherwise mean the SoC written back was not the one simulated. It is assigned 0.0 in fetch.py and 0 in predbat.py and set nowhere else. Tests: the shared harness built its Prediction before soc_kw/soc_max and the rates were set, so it simulated an empty battery, every plan cost 0.0 and any metric assertion was silently a no-op. Fixed, and guarded by a test. Adds coverage for both passes reverting a regression, the in-progress export not being restarted later in its window, and the reported metric matching a fresh whole-plan simulation for both branches. Random benchmark over 20 scenarios against main: 19 unchanged, 1 better, 0 worse. Diagnosis, field logs and the original fix are from PR #4398 by @mbuhansen. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
During a force export at the highest export price of the horizon, the 20:05 recompute cancelled the export and switched the inverter to Demand. The plan handed to
tweak_planhad metric -16218.30 (cost -16670.26) and the plan it returned had metric -15846.94 (cost -16308.52) — 371.36 worse on metric, 361.74 worse on cost — with the running export moved from "start now at 0%" to "start at 20:40".The whole-plan guard (
should_replace_plan) only compares the new plan against the previous cycle's plan, so the valuetweak_planhad just destroyed was invisible and the worse plan was adopted. The export then flappedExporting -> Demand -> Exporting -> Demandover the next 16 minutes, idling the battery through the best-priced hour of the 48h horizon.Two defects combine
1.
tweak_planandoptimise_full_second_passare non-monotonic. Both write the result ofoptimise_export/optimise_charge_limitstraight back with no check that the whole plan improved. Those functions score candidates against the window being turned off, not against the setting the plan already holds, so they return the best of their own candidate set and can silently downgrade a better choice made by an earlier pass.tweak_planalso resets the window start back tostart_origfirst, re-opening every later start in the window.2. The in-progress export commitment did not discriminate by start time. The metric bonus added by #4118 was applied to every candidate in the window, including ones starting after
minutes_now. Giving "stop now, restart later" the same bonus as "keep exporting" cancels it out, so the commitment provided no stickiness at all — only the cost-gate relaxation (keep_export) was correctly scoped.Changes
Adds
plan_metric_current/plan_window_snapshot/plan_window_restore/keep_plan_change_if_improved, and uses them to make both passes monotonic: each window change is re-simulated against the whole plan and reverted when it does not improve on metric — the same objectiveshould_replace_planand the swap passes already use. The comparison is strict, so a change that merely ties is also reverted, which reduces plan churn.Scopes the metric bonus to
start <= minutes_now, matching thekeep_exportcondition that already sits directly below it.The sibling swap passes already guard themselves this way (
optimise_swap_export,optimise_swap_charge); this brings tweak and the second pass in line rather than introducing a new principle.tweak_planloses its now-unusedbest_metric/metric_keepparameters, since it measures the plan it is handed instead of trusting the caller.optimise_full_second_passkeeps its signature.Field verification
Two full days of production logs from the same system (Kostal, 44.79 kWh,
calculate_second_passoff), comparingSwap export optimisation finishedagainstTweak optimisation finishedon every cycle:tweak_planremains useful — 86 improvements and 36 no-ops out of 122 — so the guard removes the losses without neutering the pass.Behaviourally, the forced export then ran unbroken for two hours through the price peak: 25 consecutive cycles
Exporting, with the window start pinned at19:00 - 22:01for the entire period and noset force export to Falsein between. Previously it flapped in and out of Demand. No exceptions or tracebacks across the full day.Note the production run used the C++ prediction kernel, which the unit tests skip when the shared library is not built.
Performance
tweak_planis capped at 8 windows, so it adds at most 8 extrarun_prediction_metriccalls.optimise_full_second_passhas no cap and adds one per window, which is the larger concern — measured onoptimise_windows_kernel(the test that exercisessecond_pass=True), two runs each:No regression. The pass appears to converge slightly faster, plausibly because later windows are no longer optimised on top of a degraded intermediate plan.
Tests
Extends
tests/test_export_commitment.pywith two cases:tweak_planmust revert a window change that makes the plan worseThe second was verified to fail without the guard, reproducing the exact symptom from the log (
tweak_plan kept a delayed export start 780 instead of reverting to 690).optimise_full_second_passis covered by the existingoptimise_windows_kerneltest, which runs it withsecond_pass=True.