Speed up device_scheduler with a recursive stock balance#2282
Merged
Conversation
`_get_stock_change` expressed a device's stock at step j as a running sum over all earlier steps, so the number of model nonzeros grew quadratically with the scheduling horizon (e.g. ~12k nonzeros at 12h, ~349k at 3 days). That dominated solve time on longer horizons. Introduce an explicit `device_stock` variable coupled by a recursive `device_stock_balance` constraint (stock[j] = a*stock[j-1] + b*change[j], with a, b the per-step loss coefficients that apply_stock_changes_and_losses computes). This is mathematically equivalent -- verified identical objectives across horizons and against the planning test suite -- and keeps the number of nonzeros linear in the horizon. Solve time drops ~10x at the 2-day default horizon and ~23x at 3 days. Found while investigating SeitaBV/ems#172. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LeqGFrHfGrBHAJyjdAyr3y
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LeqGFrHfGrBHAJyjdAyr3y
Documentation build overview
37 files changed ·
|
2 tasks
Flix6x
commented
Jul 16, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR optimizes the device scheduler’s stock modeling by replacing a quadratic-sized running-sum stock expression with an explicit device_stock state variable and a per-step recursive stock balance constraint, reducing MILP nonzeros to scale linearly with the scheduling horizon.
Changes:
- Introduce
model.device_stockand a recursivedevice_stock_balanceconstraint to represent stock evolution per time step. - Replace the previous cumulative stock-change computation with a recursive formulation using per-step loss coefficients matching
apply_stock_changes_and_losses(..., how="linear"). - Add a changelog entry documenting the performance impact for longer horizons.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| flexmeasures/data/models/planning/linear_optimization.py | Reworks stock accumulation into a recursive stock balance with a device_stock state variable to reduce model sparsity growth |
| documentation/changelog.rst | Adds an Infrastructure/Support changelog note describing the scheduling performance improvement |
…ce lists Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The recursion formulation lands on a cost-equal alternative vertex (A charges 3, B discharges 1 for free under net site metering). Make device B lossy in both directions so the optimum is unique again. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 17, 2026
Flix6x
added a commit
that referenced
this pull request
Jul 17, 2026
…oup (#2325) * Speed up device_scheduler with a recursive stock balance `_get_stock_change` expressed a device's stock at step j as a running sum over all earlier steps, so the number of model nonzeros grew quadratically with the scheduling horizon (e.g. ~12k nonzeros at 12h, ~349k at 3 days). That dominated solve time on longer horizons. Introduce an explicit `device_stock` variable coupled by a recursive `device_stock_balance` constraint (stock[j] = a*stock[j-1] + b*change[j], with a, b the per-step loss coefficients that apply_stock_changes_and_losses computes). This is mathematically equivalent -- verified identical objectives across horizons and against the planning test suite -- and keeps the number of nonzeros linear in the horizon. Solve time drops ~10x at the 2-day default horizon and ~23x at 3 days. Found while investigating SeitaBV/ems#172. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LeqGFrHfGrBHAJyjdAyr3y * Reference PR #2282 in the changelog entry Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LeqGFrHfGrBHAJyjdAyr3y * Address review: apply changelog suggestion and cache stock-group device lists Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Fix degenerate optimum in alignment regression test after merging main The recursion formulation lands on a cost-equal alternative vertex (A charges 3, B discharges 1 for free under net site metering). Make device B lossy in both directions so the optimum is unique again. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Share storage-efficiency per stock group and index solver stock by group (#2324) - storage-efficiency may now be defined on the entry holding a shared stock's SoC parameters (or on exactly one member device) and applies to all members; conflicting definitions fail fast, mirroring how #2321 treats the other SoC parameters. - device_scheduler now models one stock variable and one balance recursion per stock group instead of per device, dropping the redundant secondary recursions, and validates that grouped devices share their storage efficiency and initial stock. - Fix a latent key collision in device_to_group: an ungrouped device whose index equalled a state-of-charge sensor id was silently merged into that stock group, feeding its flows into the wrong stock. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add changelog entries for PR #2325 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Enable storage losses in the dynamic-capacity test and consolidate changelog The buffer's storage-efficiency (previously ignored, with a 'does not work yet' todo) is now honored, so enable it at 99% per 15 minutes and update the expected schedules: the heater covers the loss-induced remainder beyond the maxed-out boiler around the clock, and the boiler tops up the buffer in the final hour of the cheap-electricity window. Also fold the two changelog entries into the existing shared-storage feature entry for v1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Attach soft SoC commitments to their stock, not to a device index Enabler for scoping #2194's SoC-constraint relaxation per stock: - StockCommitment gains a 'stock' attribute holding the stock key; the StorageScheduler stamps it on all soft SoC commitments (soc-minima, soc-maxima, prefer-full). - The solver couples stock-scoped commitments to the stock group as a whole (via the group's first device, whose stock is the group's stock), regardless of which device index the commitment names. - DeviceInventory.stock_constraint_device(stock_key) exposes the device that applies a stock's SoC constraints, complementing stock_groups for deserialization-time scoping logic. - The solver's grouped-device efficiency guard now tolerates a missing efficiency column (the default, lossless case), which direct callers of device_scheduler commonly omit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Remove dead device_stock_commitment_equalities It was never registered as a model constraint; all device-level commitments route through grouped_commitment_equalities. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Flix6x
added a commit
that referenced
this pull request
Jul 17, 2026
…ters Merging main's group-indexed stock recursion (#2282/#2325) rebuilt group_to_devices from a single-valued device_to_group (last assignment wins), which cannot represent a converter device that belongs to more than one stock group (e.g. a steamer bridging a heat node and a steam node). That silently dropped such devices from all but their last group, orphaning coupled outputs (the CHP dispatched to zero in the cheap-gas merit-order scenario). Keep main's namespaced group keys (the stock-id/device-index collision fix) but build group_to_devices directly from the declared stock groups, preserving full overlapping membership; device_to_group still records a single primary group for per-device stock bounds. Restores test_factory_chp_dispatch (all three merit-order scenarios) on top of the new recursive stock formulation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
_get_stock_changein the device scheduler expressed a device's stock at time step j as a running sum over all earlier steps. The number of model nonzeros therefore grew quadratically with the scheduling horizon, and that dominated solve time on longer horizons.This replaces the running sum with an explicit
device_stockvariable coupled by a recursivedevice_stock_balanceconstraint:where
a,bare the per-step loss coefficients thatapply_stock_changes_and_lossesalready computes (the storage efficiency is a Param, so they're plain floats). This is mathematically identical to the old formulation but keeps the number of nonzeros linear in the horizon.Why it matters
Measured on a HEMS-like MILP (battery + EV with a departure SoC + inflexible load, priced per quarter-hour), solving the exported model directly through HiGHS:
Correctness
1591.100434,-459.702175,-1674.515470,-3855.657177).test_solver.py,test_storage.py,test_commitments.py,test_storage_utils.py) with no new failures.Context
Found while benchmarking the HiGHS solver for SeitaBV/ems#172 — the model build/solve, not the solver choice, turned out to be the real bottleneck on long horizons.
🤖 Generated with Claude Code
https://claude.ai/code/session_01LeqGFrHfGrBHAJyjdAyr3y