Skip to content

Let the supply trail say "still watching" while the interlock is quiet - #720

Merged
xmap merged 1 commit into
mainfrom
feat/supply-probe-tick
Aug 24, 2026
Merged

Let the supply trail say "still watching" while the interlock is quiet#720
xmap merged 1 commit into
mainfrom
feat/supply-probe-tick

Conversation

@xmap

@xmap xmap commented Aug 24, 2026

Copy link
Copy Markdown
Owner

Why

EPICS CA monitors are change-only, so a healthy BLEPS pushes nothing for hours. The supply probe trail therefore only grew when a reading changed, which left it unable to answer the question it exists for: has this Supply been fine, or did CORA stop looking? That is the question that started the BLEPS work, so the trail failing it is not a gap at the edges.

The enclosure observer has had a sibling poll task since its own probe trail landed. The supply side was specified with one and shipped without.

What

One poller per Supply, gated on a new default-off bleps_supply_probe_tick_seconds.

The shape is adapted rather than copied, because a Supply is fed by many channels where an Enclosure is fed by one PV:

  • Scoped per Supply, not per PV. One row per Supply per tick. Per-PV rows would turn 2 rows a tick into 32 at 2-BM without answering anything the aggregate row leaves open.
  • Reads every channel behind that Supply, the system-wide comms flag included, since that gates the verdict as much as the circuits do.
  • All-or-nothing grading. RELAYED only if every channel answered; otherwise UNREACHED, naming the PV that did not. This is the asymmetry _verdict already applies. A row claiming coverage while one circuit is dark is worse than no row, because it looks identical to a healthy one.
  • No status claim, ever. A tick never touches latest, so only the push path can move a Supply. Refreshing stale values from poll reads is a separate feature with its own open question (what a failed read should do to a value a subscription reported fine a moment earlier), and answering that inside a coverage change is how a fail-open ships unreviewed.

Off by default

Same posture as the enclosure sibling. The reads land on a shared equipment-protection IOC, so a deployment opts in with a cadence rather than inheriting a timer. Not enabled at 2-BM in this PR.

Verification

Full suite 50,477 passed; architecture 31,425 passed; pyright clean; observer file 97.3% covered.

Six mutations run against the new tests, six caught:

mutation tests failed
grade RELAYED unless ALL channels fail 3
attribute a failure to the representative PV 3
drop the comms flag from the poll read set 1 (exactly)
poll the union of all channels per Supply 2
tick republishes the verdict target, on 3 == 1
delete the teardown leftover drain 1 (exactly)

Two of these changed the work. The "a tick carries no status" test first failed by timeout rather than by its own assertion, so the mutation was narrowed until the count assertion caught it. And the teardown leftover-drain branch looked untestable, which nearly justified deleting it; driving the generator by hand made the race deterministic instead.

One unrelated suite flake, test_run_debriefer_seed: it asserts on structlog output via capsys, and the warning it wants is emitted but captured at file level instead. A stream-binding artifact of structlog's global config under xdist. Passes standalone, across its directory, and across all 14,319 unit tests under -n 4. This diff adds no logging configuration.

🤖 Generated with Claude Code

EPICS CA monitors are change-only, so a healthy BLEPS pushes nothing for
hours. The supply probe trail therefore only grew when a reading changed,
which left it unable to answer the question it exists for: whether a
Supply has been fine, or whether CORA stopped looking. That is the same
question that started the BLEPS work, so the trail failing it is not a
gap at the edges.

The enclosure observer has had a sibling poll task since its own probe
trail landed; the supply side was specified with one and shipped without.
This adds it, with the shape changed to fit a Supply rather than copied:
a Supply is fed by many channels, so the tick is scoped per Supply, reads
every PV behind it (the system-wide comms flag included, since that gates
the verdict as much as the circuits do), and writes one row. RELAYED only
if every channel answered; otherwise UNREACHED naming the one that did
not. That all-or-nothing grading is the asymmetry `_verdict` already
applies: a row claiming coverage while a circuit is dark is worse than no
row, because it looks identical to a healthy one.

A tick carries no status claim and never touches the reading cache, so
only the push path can move a Supply. Refreshing stale values from poll
reads is a separate feature with its own open question, what a failed read
should do to a value a subscription reported fine a moment earlier, and
answering that inside a coverage change is how a fail-open ships
unreviewed.

Off by default, like the enclosure sibling: the reads land on a shared
equipment-protection IOC, so a deployment opts in with a cadence rather
than inheriting a timer.
@github-actions

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  apps/api/src/cora/api
  _bleps_supply_observer.py
  main.py
  apps/api/src/cora/infrastructure
  config.py
Project Total  

This report was generated by python-coverage-comment-action

@xmap
xmap merged commit 1fb7b9f into main Aug 24, 2026
32 of 34 checks passed
@xmap
xmap deleted the feat/supply-probe-tick branch August 24, 2026 01:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant