Refactor validation logic and remove post-edit validation scripts

openspec/changes/archive/2026-07-16-fix-partial-validation-state/.openspec.yaml

@@ -0,0 +1,3 @@
+---
+schema: spec-driven
+created: 2026-07-16

openspec/changes/archive/2026-07-16-fix-partial-validation-state/design.md

@@ -0,0 +1,86 @@
+## Context
+
+`SectionValidator` lazily validates device parameter support before a section or
+hot-water group is read. It currently stores completion as a section-wide set in
+`APIValidator` and a group-wide set in `SectionValidator`. An `include` request
+validates only a subset but records that coarse completion state, so a later
+request for different names or all names can skip validation.
+
+The resulting configuration has not established whether every requested
+parameter is supported. For normal sections, an unvalidated parameter can remain
+in the configuration and later produce a strict response-mapping failure. For
+hot water, the cache contains only the first subset and later reads silently use
+that incomplete cache.
+
+## Goals / Non-Goals
+
+**Goals:**
+
+- Record lazy-validation coverage by parameter ID for sections and hot-water
+  groups.
+- Revalidate only the parameter IDs required by a later request when they have
+  not already been covered.
+- Retain the existing public method signatures, include-filter validation, and
+  per-section/group locking model.
+- Add regression tests for sequential filtered and unfiltered access.
+
+**Non-Goals:**
+
+- Change the public `include` API or make it accept parameter IDs.
+- Alter device-response parsing, model serialization, or unsupported-parameter
+  rules.
+- Retry or cache failed validation requests.
+
+## Decisions
+
+### Track covered parameter IDs per validation owner
+
+`APIValidator` will maintain a mapping from section name to the set of parameter
+IDs whose support was checked. `SectionValidator` will maintain the equivalent
+mapping for hot-water group names. Completion for a request is determined by
+whether its requested IDs are a subset of that owner’s covered IDs.
+
+Parameter IDs are chosen over parameter names because they are the identifiers
+sent to BSB-LAN and remain unambiguous if a configuration ever aliases a name.
+When a response establishes an ID is unsupported, it is removed from the active
+configuration/cache but remains covered, preventing needless repeated probes.
+
+Alternative considered: avoid persisting any completion after an `include`
+request. This avoids the defect but discards useful knowledge and repeatedly
+validates already checked parameters. Parameter-level coverage keeps lazy
+loading efficient while remaining correct.
+
+### Derive requested IDs before the one-time gate
+
+Each ensure method will first resolve the configured IDs for its section or
+group, applying `include` when present. Its lock’s fast and post-lock checks will
+compare that specific requested-ID set to coverage rather than use a coarse
+section/group marker. A full request resolves all currently configured IDs and
+therefore completes only after each is covered.
+
+An include filter that matches no IDs will not add coverage or mark a group or
+section complete. Existing downstream include validation remains responsible for
+returning the appropriate invalid-include error.
+
+Alternative considered: retain a boolean for full validation plus a separate
+partial set. This duplicates state and makes configuration changes harder to
+reason about; one coverage mapping expresses both cases.
+
+### Preserve locks and recheck coverage after waiting
+
+The existing per-section and per-group locks remain in place. The completion
+predicate will calculate coverage for the current requested IDs both before and
+after lock acquisition. Concurrent calls requesting different subsets therefore
+serialize safely, and the second call still validates its uncovered IDs.
+
+## Risks / Trade-offs
+
+- [Configuration mutation during validation] → Derive the requested IDs from
+  the current configuration at each completion check; removed unsupported IDs
+  no longer need validation.
+- [Stale internal tests inspect completion sets] → Update tests to assert
+  parameter coverage and externally observable request/cache behavior instead
+  of obsolete whole-group completion.
+- [Additional first-read requests] → A later request for previously unrequested
+  parameters necessarily makes one validation request; already covered IDs are
+  not re-requested.

openspec/changes/archive/2026-07-16-fix-partial-validation-state/proposal.md

@@ -0,0 +1,38 @@
+## Why
+
+Lazy validation currently records a section or hot-water group as fully
+validated after a request filtered with `include`. This can leave unsupported
+parameters in a section or leave the hot-water cache incomplete, causing later
+unfiltered reads to skip necessary validation and return incomplete data or fail
+while mapping a device response.
+
+## What Changes
+
+- Track validation coverage at the parameter level for sections and hot-water
+  groups instead of treating any successful filtered request as complete.
+- Ensure a later request validates parameter names that an earlier `include`
+  request did not cover.
+- Keep validation completion correct when an include filter has no matching
+  parameters.
+- Add focused regression coverage for filtered-then-unfiltered and
+  differently-filtered reads.
+
+## Capabilities
+
+### New Capabilities
+
+- `lazy-validation-completeness`: Lazy section and hot-water validation covers
+  every parameter required by each request before that request uses the cached
+  configuration.
+
+### Modified Capabilities
+
+- None.
+
+## Impact
+
+- Affected code: `src/bsblan/_validation.py`, `src/bsblan/utility.py`, and the
+  lazy section and hot-water read paths that use their validation state.
+- Affected tests: validation, include-filter, and hot-water regression tests.
+- Public API: no signature changes; `include` calls may perform an additional
+  validation request when they need parameters not checked by a prior call.

openspec/changes/archive/2026-07-16-fix-partial-validation-state/specs/lazy-validation-completeness/spec.md

@@ -0,0 +1,63 @@
+## ADDED Requirements
+
+### Requirement: Section validation covers every requested parameter
+
+Before a lazy section read uses its configured parameters, the client SHALL
+validate support for every parameter ID required by that read. A prior validation
+of a different included subset SHALL NOT make unvalidated parameter IDs appear
+validated. The client SHALL not repeat validation for IDs already covered unless
+validation state is explicitly reset.
+
+#### Scenario: Filtered section read followed by unfiltered read
+
+- **WHEN** a section is first read with `include` selecting one parameter and a
+  later read requests the complete section
+- **THEN** the later read validates every configured parameter ID not checked by
+  the first read before fetching the complete section
+
+#### Scenario: Differently filtered section read
+
+- **WHEN** a section is read with one included parameter and later read with a
+  different included parameter
+- **THEN** the second read validates the newly requested parameter ID
+- **AND** it does not revalidate the parameter ID already covered by the first
+  read
+
+#### Scenario: Unsupported parameter discovered by a later read
+
+- **WHEN** a later section read validates a previously uncovered parameter ID
+  that the device does not support
+- **THEN** the client removes that parameter from the active configuration
+- **AND** subsequent reads do not request that unsupported parameter
+
+### Requirement: Hot-water group validation covers every requested parameter
+
+Before a hot-water group read uses its cached parameters, the client SHALL
+validate and cache every parameter ID required by that read. A prior validation
+of an included subset SHALL NOT make the group complete for other parameter IDs.
+
+#### Scenario: Filtered hot-water read followed by complete group read
+
+- **WHEN** a hot-water group is first read with `include` selecting a subset and
+  later read without `include`
+- **THEN** the later read validates and caches every remaining supported
+  parameter ID in that group before it fetches group data
+
+#### Scenario: Differently filtered hot-water read
+
+- **WHEN** a hot-water group is read with one included parameter and later read
+  with a different included parameter
+- **THEN** the second read validates and caches the newly requested parameter ID
+- **AND** it does not revalidate the parameter ID already covered by the first
+  read
+
+### Requirement: Empty include results do not complete validation
+
+When an include filter selects no parameter IDs in a section or hot-water group,
+the client SHALL NOT record that owner as complete or add parameter coverage.
+
+#### Scenario: Later valid read after no-match include
+
+- **WHEN** an include filter matches no parameters and a later request selects
+  one or more valid parameters from the same section or hot-water group
+- **THEN** the later request validates its selected parameter IDs normally

openspec/changes/archive/2026-07-16-fix-partial-validation-state/tasks.md

@@ -0,0 +1,29 @@
+## 1. Parameter-Level Validation Coverage
+
+- [x] 1.1 Replace coarse section completion tracking in `APIValidator` with
+  per-section parameter-ID coverage, including reset behavior.
+- [x] 1.2 Update lazy section validation to derive the current requested IDs,
+  gate on their coverage, and retain unsupported-ID removal behavior.
+- [x] 1.3 Replace coarse hot-water group completion tracking with per-group
+  parameter-ID coverage and keep the cache synchronized with validated IDs.
+- [x] 1.4 Ensure empty include-filter results add no coverage and do not mark a
+  section or hot-water group complete.
+
+## 2. Regression Coverage
+
+- [x] 2.1 Add section tests for filtered-then-unfiltered and
+  differently-filtered reads, asserting uncovered IDs are validated once.
+- [x] 2.2 Add a section test showing a later-discovered unsupported ID is
+  removed before the complete read is mapped.
+- [x] 2.3 Update hot-water include tests to assert a partial request does not
+  complete the group, then add filtered-then-unfiltered and
+  differently-filtered regression cases.
+- [x] 2.4 Update the empty include-filter test to assert no completion state is
+  recorded and a later valid request performs validation.
+- [x] 2.5 Retain or extend locking coverage to prove sequentially serialized
+  requests recheck their own parameter coverage.
+
+## 3. Validation
+
+- [x] 3.1 Run focused validation and include-filter tests with `--no-cov`.
+- [x] 3.2 Run `uv run prek run --all-files` and resolve all reported failures.