OCE report: align the noise gate to the report's rolling window, Fixes AB#3731628 - #462
Open
Shahzaib (shahzaibj) wants to merge 1 commit into
Open
Conversation
The novelty classifier graded Sunday-aligned CALENDAR weeks while the report displays a ROLLING 7-day window. The two windows drift apart by up to six days, so anything that turned in the last ~6 days -- exactly the period an on-call engineer cares about most -- was structurally invisible to the gate. Measured on the 2026-08-01 run: the gate's "current" week was 07/19-07/26 against a report window of 07/25-08/01, ONE day of overlap. authorization_pending read +63.2% in the report and -37.1% to the classifier; expired_token +26.7% vs -51.0%. Both were filed "ONGOING -- do not re-triage" directly beneath their own rising numbers. On the Authenticator side the same defect surfaced as red scoreboard pills (rolling-derived) above an empty "Needs attention" section (calendar-derived). Fix: bucket the 60-day trend and the sparklines with bin_at(<TIME>, 7d, datetime(<TREND_END>)) instead of startofweek(<TIME>). The final bucket then IS the report's displayed window, every bucket is a complete 7 days, and classifier WoW == displayed WoW by construction. --include-partial-end and TREND_CLASS_END become obsolete and are removed; bucket-trends.js now warns if --end is omitted, because its partial-end auto-drop heuristic would otherwise discard a genuinely complete final bucket. Because 60 is not a multiple of 7 the OLDEST bucket is the partial one -- the safe end to be partial on -- and --start drops it, leaving 8 complete weeks. A/B on real data: attention went 4 -> 5 keys; both mis-filed codes promoted to ACCELERATING; access_denied correctly DEMOTED (actually -53.2%, a former false positive). Alignment adds real signal and removes phantom signal -- it is not merely "more alerts". Adds validate-report.ps1 check 19: every red/amber scoreboard pill must be either promoted into the attention section or named in a .reconcile-note explaining why it is not escalated (below the classification floor / within its own normal band / ONGOING and flat). This closes the reported "red pill above an empty attention section" confusion at the report level, independent of window alignment. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
✅ Work item link check complete. Description contains link AB#3731628 to an Azure Boards work item. |
|
✅ Work item link check complete. Description contains link AB#3731628 to an Azure Boards work item. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
The novelty classifier graded Sunday-aligned calendar weeks. The report displays a rolling 7-day window. Those two windows drift apart by up to six days — so anything that turned in the last ~6 days, exactly the period an on-call engineer cares about most, was structurally invisible to the noise gate.
Measured on the 2026-08-01 run: the gate's "current" week was
07/19 → 07/26against a report window of07/25 → 08/01. One day of overlap.authorization_pendingexpired_tokenBoth were filed "don't look at this" directly beneath their own rising numbers. On the Authenticator side the same defect surfaced differently: red scoreboard pills (rolling-derived) sitting above an empty "Needs attention" section (calendar-derived) — the exact confusion reported.
The fix
Bucket the 60-day trend and the sparklines with:
instead of
startofweek(<TIME>).The final bucket then is the report's displayed window, every bucket is a complete 7 days, and classifier WoW == displayed WoW by construction — not by convention, and not something a future change can quietly break.
--include-partial-endandTREND_CLASS_ENDbecome obsolete and are removed.bucket-trends.jsnow warns if--endis omitted, because its partial-end auto-drop heuristic (if (!endArg …)) would otherwise silently discard a genuinely complete final bucket — under rolling alignment a real 70% collapse could be thrown away as "looks partial".Because 60 isn't a multiple of 7, the oldest bucket is the partial one — the safe end to be partial on — and
--startdrops it, leaving 8 complete weeks.This is not "more alerts"
A/B on real data:
ACCELERATINGaccess_deniedcorrectly demoted — it was actually −53.2%, a false positive the calendar window had been surfacingAlignment adds real signal and removes phantom signal. A controlled A/B over the affected window confirmed the classifier's WoW input matched the displayed WoW on 8 of 8 sampled codes after the change, versus 0 of 8 before.
Also: reconciling red pills
Adds
validate-report.ps1check 19. Scoreboard tables colour a row from its own rolling delta; the attention section is populated from the classifier's novelty verdict. Those answer different questions, so a row can legitimately be red in the table and legitimately absent from attention — but a reader who sees that mismatch unexplained concludes the report is broken.Precedent:
Passkey WebAuthN Registrationshipped carryingtag-bad(−1.27 pts, worst delta in its table) directly above the words "Quiet week — 0 NEW or ACCELERATING". Both statements were true — the scenario peaks at ~732 bad-outcome devices, below the 1,000-device classification floor, so it is structurally excluded and can never appear in attention however sharply it moves.Every
tag-bad/tag-warnrow must now be either promoted into attention or named in a muted.reconcile-notegiving the reason, tested in order: (1) below the classification floor, (2) within its own normal band, (3) ONGOING and flat. Check 19 hard-fails an unreconciled pill.This closes the "red pill above an empty attention section" confusion at the report level, independently of window alignment.
Stack
This is 3 of 3:
masterFixes AB#3731628