Balance internal commodity nodes (heat/steam networks) via first-class balance groups

[Flix6x]

Description

Stacked on #2218. Completes the missing piece of the CHP/factory work: scheduling a multi-commodity factory end-to-end through StorageScheduler.compute().

• device_scheduler gains a balance_groups argument: each group lists the devices of an internal commodity node (a heat or steam network without a grid connection) whose stock-side flows must sum to zero at every time step. This replaces the earlier workaround of overlapping stock groups with a min=max=0 "reference device" (which never worked automatically from a flex-model — see the investigation notes on dev/simulate-factory).
• The StorageScheduler derives balance groups from the flex-config: a non-electricity commodity without energy prices in the flex-context is treated as an internal node (previously this raised Missing consumption price). Electricity still requires a price.
• Converters are described in the flex-model as one entry per commodity port, tied together by a coupling group — so per-commodity balance groups are disjoint and no overlapping group tricks are needed.
• Added changelog item in documentation/changelog.rst
• Docs: a worked converter + internal-node example in the multi-commodity tutorial (tut_converters), cross-referenced from features/scheduling.rst.

Example flex-model

A converter is one flex-model entry per commodity port, tied by a shared coupling name; a commodity with no price in the flex-context becomes an internal node. A CHP (gas → steam + electricity) feeding an internal steam node:

"flex-model": [
  {"sensor": 6, "commodity": "gas",         "coupling": "chp", "coupling-coefficient": 1.0, "power-capacity": "20 kW", "production-capacity": "0 kW"},
  {"sensor": 7, "commodity": "steam",       "coupling": "chp", "coupling-coefficient": 0.5, "power-capacity": "1 MW", "consumption-capacity": "0 kW"},
  {"sensor": 8, "commodity": "electricity", "coupling": "chp", "coupling-coefficient": 0.3, "power-capacity": "1 MW", "consumption-capacity": "0 kW"}
]
"flex-context": [
  {"commodity": "electricity", "consumption-price": "50 EUR/MWh", "production-price": "50 EUR/MWh"},
  {"commodity": "gas",         "consumption-price": "20 EUR/MWh", "production-price": "20 EUR/MWh"},
  {"commodity": "steam",       "inflexible-device-sensors": [10]}
]

Each kW of gas produces 0.5 kW steam + 0.3 kW electricity. steam has no price → internal node: its devices (the CHP/steamer producing, the fixed demand consuming) balance each other every step. The full factory (e-heater + boiler → internal heat, steamer → internal steam, CHP → steam + grid electricity) is in the new docs section and in test_factory_chp_dispatch_through_storage_scheduler.

Converter shorthand — how it would map (follow-up, not in this PR)

A first-class converter shorthand would expand into the per-port + coupling form above, e.g.:

{"converter": "chp",
 "gas":         {"sensor": 6, "power-capacity": "20 kW"},
 "steam":       {"sensor": 7, "coupling-coefficient": 0.5},
 "electricity": {"sensor": 8, "coupling-coefficient": 0.3}}

→ expands to the three coupled entries: the first-listed port (gas) is the import-only coupling reference at coefficient 1.0; the remaining ports (steam, electricity) are export-only at their coefficients. This is noted as a possible future improvement to reduce verbosity; the explicit per-port form is what this PR ships.

How to test

pytest flexmeasures/data/models/planning/tests/test_commitments.py -k factory_chp_dispatch
pytest flexmeasures/data/models/planning/tests/test_storage.py -k factory_chp_dispatch_through_storage_scheduler

The engine-level factory test is parametrized to run in both modes (reference-device stock groups and balance groups) and must produce identical dispatch. The new StorageScheduler test drives the full factory (CHP + gas boiler + e-heater + steamer meeting a fixed 15 kW steam demand from an inflexible sensor) purely from a flex-model and flex-context.

Related Items

Stacked on #2218; continues the work explored on dev/simulate-factory (#2113-era factory simulation).

🤖 Generated with Claude Code

https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B

[Flix6x]

Supersedes #2279 (same content; one commit message reworded — the original couldn't be amended in place since force-pushes are blocked by repository rules). The #2272 interaction note from #2279 applies here unchanged.

🤖 Generated with Claude Code

[read-the-docs-community]

Documentation build overview

📚 flexmeasures | 🛠️ Build #33656784 | 📁 Comparing b0d422d against latest (60cfda8)

  🔍 Preview build  

50 files changed · + 8 added · ± 42 modified

+ Added

• _autodoc/COUPLING.html
• _autodoc/COUPLING_COEFFICIENT.html
• _autodoc/GROUP.html
• _autosummary/flexmeasures.data.models.planning.devices.html
• _autosummary/flexmeasures.data.models.planning.soc_projection.html
• _autosummary/flexmeasures.data.schemas.scheduling.utils.html
• host/multi-tenancy.html
• tut/toy-example-group-constraints.html

± Modified

• changelog.html
• configuration.html
• genindex.html
• getting-started.html
• http-routingtable.html
• index.html
• py-modindex.html
• _autodoc/SOC_MAXIMA.html
• _autodoc/SOC_MINIMA.html
• _autodoc/SOC_TARGETS.html
• and 32 more...

[Flix6x]

This should cross-reference a new docs section (tutorial?) with an example flex-model. Let's have the example flex-model make an appearance in the PR description, too.

Also let the converter shorthand in the PR description get an example of how it would map.

[Flix6x]

Thanks — addressed:

• Docs example: the converter/coupling paragraph in features/scheduling.rst now cross-references a new "Converters between commodities" section in the multi-commodity tutorial (tut_converters), with a worked CHP example flex-model (one entry per commodity port tied by a coupling group) and how an unpriced commodity becomes an internal balance node.
• PR description: now shows the example flex-model + flex-context, and the proposed converter shorthand with how it maps to the per-port + coupling form.
• CI: fixed the failing pre-commit Check (flake8 F811 duplicate GenericAsset import + black formatting in test_storage.py).

The DCO check is red on the pre-existing commits; since force-pushes are blocked by the repo rules I have not rewritten history — it likely needs a maintainer override (the newly added commits are signed off).

[Copilot]

Pull request overview

This PR extends multi-commodity scheduling to support internal commodity nodes (e.g. heat/steam networks without a grid connection) by introducing first-class balance groups that enforce per-timestep flow balance, and wires this through StorageScheduler.compute() so factory-like setups can be scheduled end-to-end from flex-config.

Changes:

• Add balance_groups to device_scheduler and implement per-timestep node balance constraints.
• Teach StorageScheduler to derive balance groups for commodities treated as internal nodes (no energy prices), and pass them through to the optimizer.
• Add end-to-end factory/CHP scheduling tests plus documentation and changelog updates explaining converter (coupling) modeling and internal nodes.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
flexmeasures/data/schemas/scheduling/init.py	Adds a prices_are_defaulted marker intended to distinguish defaulted vs user-specified prices for internal-node detection
flexmeasures/data/models/planning/storage.py	Derives internal-node balance groups for commodities without user-given prices and passes them to device_scheduler
flexmeasures/data/models/planning/linear_optimization.py	Adds balance_groups parameter and enforces per-timestep node balance constraints
flexmeasures/data/models/planning/tests/test_commitments.py	Parametrizes factory scenario to validate both legacy stock-group balancing and new balance-group balancing yield identical dispatch
flexmeasures/data/models/planning/tests/test_storage.py	Adds end-to-end factory dispatch test through StorageScheduler.compute() driven by flex-model/flex-context
documentation/tut/multi-commodity.rst	Documents converter modeling via per-port entries coupled by coupling/coupling-coefficient and introduces internal nodes
documentation/features/scheduling.rst	Cross-references the converter/internal-node concepts from the scheduling feature docs
documentation/changelog.rst	Adds changelog entries for coupled converters and internal-node scheduling behavior

[Flix6x]

Addressed the four review findings in c0eefc5:

1. Capacity-only commodity misclassified as internal node (schemas/scheduling/__init__.py, planning/storage.py). A commodity declaring a grid connection via capacity fields only (site-power/consumption/production-capacity) but no explicit price was flagged as having defaulted prices, so StorageScheduler treated it as an internal node — skipping EMS constraints and forcing a per-step balance. The internal-node determination now also considers the capacity fields: a commodity is an internal node only when the user gave neither prices nor any capacity/grid-connection signal (not any_given). The durable flag was renamed prices_are_defaulted → is_internal_node to match its true meaning. Added test_commodity_flex_context_internal_node_flag asserting a commodity with a capacity field but no price is NOT an internal node.

2. Docstring mismatch (planning/linear_optimization.py). The balance_groups doc claimed a stock-side P_up/eff/stock_delta balance; the implementation enforces the commodity-side sum_d(ems_power[d, j]) == 0. Docstring corrected.

3. Docs (features/scheduling.rst). Clarified that only non-electricity commodities without prices become internal nodes; electricity is always assumed grid-connected, so missing electricity prices raise an error.

4. Docs (tut/multi-commodity.rst). Added the same electricity caveat.

Tests: test_commitments.py + test_scheduling.py — 148 passed (incl. the new internal-node test). No OpenAPI spec change (the flag is a post-load data key, not a declared field).