Skip to content

Believe a BLEPS flag that is alarming, because the alarm is the assertion - #719

Merged
xmap merged 1 commit into
mainfrom
fix/bleps-believes-alarming-flags
Aug 23, 2026
Merged

Believe a BLEPS flag that is alarming, because the alarm is the assertion#719
xmap merged 1 commit into
mainfrom
fix/bleps-believes-alarming-flags

Conversation

@xmap

@xmap xmap commented Aug 23, 2026

Copy link
Copy Markdown
Owner

Summary

The BLEPS supply observer discarded every reading whose quality was not Good. On an equipment-protection interlock that is close to backwards: a BLEPS record raises MAJOR precisely when its flag asserts, because putting the trip on an operator's screen is what the alarm is for. CORA's CA adapter collapses MINOR/MAJOR to Uncertain (only INVALID is Bad), so the strict floor did not drop some readings, it dropped exactly the asserted ones and kept the quiet ones. The observer could see a healthy beamline and nothing else.

   circuit CLEAR    quiet, no alarm   ->  Good       ->  believed  OK
   circuit TRIPPED  loud, MAJOR       ->  Uncertain  ->  discarded BUG

One line: the floor moves from != "Good" to == "Bad".

Measured, not assumed

arcturus 2026-08-23, 67 PVs, no exceptions. All eight cooling-water circuits read TRIP with STATE MAJOR, plus three vacuum sections, two ion pumps and six ion gauges. Not one was visible to CORA. Merging the ordinal fix (#718) alone would still have recorded nothing.

This is a precedent, not a new idea

_enclosure_permit_observer was loosened for this exact reason after 2-BM's SecureM read Unknown forever, and _capture_baseline_reader already records that "a MAJOR alarm is still a believable value". Both wrote the principle down. This observer was written later, asked the same question, and took the other answer.

The distinction those files draw is the load-bearing one: "can I believe this value" is not "can I act on it." The Conductor's check / capture / compute-capture gates keep their Good-only floors and are right to. The beam-availability lookup has the same defect (SR-ACIS:2BM:FesPermitM sits at MAJOR while asserting) but is deliberately left for its own slice: it is not enabled at 2-BM, and loosening a start gate is a different argument.

Review findings folded in

An adversarial review found the safety argument was proved for one PV role and applied to three. This function reads the process axis (NO_FAULT/TRIP), the trust axis (fault flags, ""/Present) and the system axis (comms), which do not share a label pair. The measurement did cover all three, but the docstring cited it narrowly. Now stated per axis, with the honest limit: no trust- or system-axis flag was ever seen ASSERTED, so "asserted implies MAJOR" is measured only for the process axis. The direction the code depends on, that a CLEAR reading carries no alarm, is measured for all three.

Tracing the fold showed the loosening is wider than first described, and each widening is now pinned by a test:

axis what it now allows reachable at 2-BM?
process alarmed clear trip counts toward "all clear" no
trust alarmed clear fault flag no longer blinds its channel no
system alarmed clear comms flag no longer reads as a dark feed (gates every Supply) no

The counterweight: the strict floor's conservatism was never free. Any standing alarm on a fault or comms record blinded the observer permanently, the same failure as the process axis one level upstream.

Test plan

  • Full suite: 50,467 passed, 1 pre-existing unrelated flake
  • Six tests fail if the strict floor returns, and nothing else does
  • End-to-end test reproduces the live 2-BM shape (undecodable enum labels, trip at MAJOR, empty-string fault flag, healthy comms) through the whole observer. Both defects must be fixed for it to pass: either alone still yields silence
  • Two review catches: the end-to-end test asserted the FIRST fold's reason, which names only the first channel, so a dropped second channel would have passed; and the Uncertain case removed from test_unreadable_flag_is_unknown_not_low was re-added for the enum shape but not the scalar one it originally covered

Generated with Claude Code

…tion

The supply observer discarded every reading whose quality was not Good.
On an equipment-protection interlock that is close to backwards: a BLEPS
record raises MAJOR precisely WHEN ITS FLAG ASSERTS, since putting the
trip on an operator's screen is what the alarm is for. The CA adapter
collapses MINOR and MAJOR to Uncertain (only INVALID is Bad, because
only INVALID says the value itself cannot be trusted), so the strict
floor did not drop some readings, it dropped exactly the asserted ones
and kept the quiet ones. The observer could see a healthy beamline and
nothing else.

Measured on arcturus 2026-08-23, 67 PVs: all eight cooling-water
circuits read TRIP with STATE MAJOR, along with three vacuum sections,
two ion pumps and six ion gauges. Not one was visible to CORA. Merging
the ordinal fix alone would not have recorded a thing.

The floor is now Bad, matching _enclosure_permit_observer (loosened for
this exact reason after 2-BM's SecureM read Unknown forever) and
_capture_baseline_reader ("a MAJOR alarm is still a believable value").
Both had already written the principle down. This observer was written
later, asked the same question, and took the other answer.

The distinction those files draw is the load-bearing one: "can I believe
this value" is not "can I act on it". The Conductor's check, capture and
compute-capture gates keep their Good-only floors and are right to, as
does the beam-availability lookup pending its own slice.

An adversarial review found the safety argument was proved for one PV
role and applied to three. This function reads the process axis
(NO_FAULT/TRIP), the trust axis (fault flags, ""/Present) and the system
axis (the comms flag), and they do not share a label pair. The
measurement did cover all three, but the docstring cited it narrowly, so
it is now stated per axis, including the honest limit: no trust-axis or
system-axis flag was ever seen ASSERTED, so "asserted implies MAJOR" is
measured only for the process axis. The direction this code depends on,
that a CLEAR reading carries no alarm, is measured for all three.

Tracing the fold showed the loosening is wider than first described, and
each widening now has a test: an alarmed clear trip counts toward "all
clear"; an alarmed clear fault flag no longer blinds its channel; an
alarmed clear comms flag no longer reads as a dark feed, which is the
widest since it gates every Supply at once. None is reachable on
measured 2-BM data. The counterweight is that the strict floor's
conservatism was never free: any standing alarm on a fault or comms
record blinded the observer permanently, the same failure as the process
axis one level upstream.

The _pump diagnostic warning tracks the same floor rather than naming
Good independently, or it would have gone silent for every asserted
flag, the population most worth hearing about.

Six tests fail if the strict floor returns, and nothing else does. Two
review catches folded in: the end-to-end test asserted the FIRST fold's
reason, which names only the first channel, so a dropped second channel
would have passed; and the Uncertain case removed from
test_unreadable_flag_is_unknown_not_low was re-added for the enum shape
but not the scalar one it originally covered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  apps/api/src/cora/api
  _bleps_supply_observer.py
Project Total  

This report was generated by python-coverage-comment-action

@xmap
xmap merged commit f370b90 into main Aug 23, 2026
19 checks passed
@xmap
xmap deleted the fix/bleps-believes-alarming-flags branch August 23, 2026 22:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant