OCE report: add novelty noise gate and redesign the Broker report around it, Fixes AB#3733390 - #460
Open
Shahzaib (shahzaibj) wants to merge 1 commit into
Open
OCE report: add novelty noise gate and redesign the Broker report around it, Fixes AB#3733390#460Shahzaib (shahzaibj) wants to merge 1 commit into
Shahzaib (shahzaibj) wants to merge 1 commit into
Conversation
…und it The weekly Broker report had become a browsing exercise rather than a triage tool. The 60-day section rendered 38 charts, ~93% of which duplicated rows in the error tables below it, while the "needs attention this week" section -- the part an on-call engineer actually reads first -- carried 13 volume-ranked rows and zero charts. A flat-but-huge code led the list; the genuinely new ipc_* family sat at positions #6/#9/#10. Root cause: bucket-trends.js reports what MOVED, but nothing decided whether a movement was NEWS. Ranking by device count is not a proxy for novelty. This change adds classify-novelty.js, which labels every series against its own 7-week baseline (NEW / ACCELERATING / ONGOING / VOLATILE / RECOVERY / IMPROVING / STABLE) and emits an `attention` set = NEW + ACCELERATING. That set, plus at most 2 wins, is all that renders visibly with charts; everything still-elevated collapses into a fold with its weeksElevated count. The ACCELERATING/ONGOING split is the whole fix: only "still getting worse" earns a second look. The 60-day section becomes a slow-burn DETECTOR -- it charts only what it promotes (rows rising on 60d and absent from the attention section, typically 0-3, often zero) and folds the full classification with no chart column. Sections 6/7 keep a per-row sparkline as a deliberate exemption: they are lookup tables, not a browsing section. Also guards two measured false positives: a WoW % off an anomalous prior week (429 headlined at +397.8% while sitting 94.5% BELOW its own 60-day median), and a slow drift in block means labelling a flat, WoW-negative series ACCELERATING. validate-report.ps1 gains checks 13-18 to enforce all of this (row-body specificity, no flat top row, no suppressed-ratio chip, mandatory .item-spark, <=8 visible rows, <=6 charts in the 60-day section). Check 12 is left reserved. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
✅ Work item link check complete. Description contains link AB#3733390 to an Azure Boards work item. |
|
✅ Work item link check complete. Description contains link AB#3733390 to an Azure Boards work item. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The 60-day trend section of the weekly Broker report had grown into a catalog. Every code with a 60-day regression got a full row and a chart, whether or not anything had changed recently — so each week the on-call engineer was asked to re-triage the same long-standing known regressions. Meanwhile the section that actually matters, "things that need attention this week", had no charts at all.
That's backwards. The report was generating noise where nothing had changed and withholding evidence where something had.
What this does
Adds
assets/scripts/classify-novelty.js— a novelty classifier that reads abucket-trends.js --json=sidecar and labels each key against its own 7-week baseline:NEWACCELERATINGONGOINGVOLATILERECOVERY/IMPROVINGSTABLEIt also clusters related codes into families, so a single upstream failure doesn't consume six attention slots.
The classifier is the noise gate. Its
attentionset (NEW+ACCELERATING), plus at most 2 wins, is all that renders visibly with charts. Everything else collapses into a fold — still one click away, never deleted, but no longer competing for the reader's attention.The Broker report template is redesigned around that gate:
validate-report.ps1gains hard checks 13–18 so a future run can't quietly regress the gate: row-body specificity, no flat top row, no suppressed-ratio chip, mandatory.item-sparkon attention rows, the 8-row cap, and the 6-chart cap.What this is not
This does not change what the classifier is fed. It grades Sunday-aligned calendar weeks, which is what it did before. That window is misaligned with the rolling 7-day window the report displays — a real bug, but a separate one, fixed in the follow-up PR so it can be reviewed on its own evidence.
Check 12 in
validate-report.ps1is deliberately left as a reserved gap. The prose refers to checks by number, and the Authenticator profile (next PR) fills that slot — keeping the numbering stable across the stack means the follow-up's validator diff is a pure insertion rather than a renumbering.Verification
validate-report.ps1 -Path oncall-wow-report-2026-08-18.html— all hard checks pass, exit 0, including 13–18:Stack
This is 1 of 3. Each PR is reviewable in isolation:
masterSplitting this way keeps the Broker-behaviour changes separate from the purely-additive Authenticator support, so neither has to be reviewed through the other.
Fixes AB#3733390