Skip forks for code scanning, and keep the dashboard tiles under Loki's series cap - #277
Merged
Merged
Conversation
…ki's series cap The three "At a glance" snapshot tiles counted item lines with count(count by (repo, number) (...)), which costs one Loki series per item. Loki caps a metric query at max_query_series (500 by default), so the Code Scanning tile errored outright once the open-alert count crossed it rather than reporting a number. Read the per-scan "scan complete" summary line instead, which already carries open_prs, open_issues and code_alerts. That is one series regardless of how many items exist, so the tiles cannot hit the ceiling at any data volume. Two details make it one series. The json stage extracts only the fields it needs, because a bare `| json` promotes every field and each varying one splits the series. And max_over_time over the snapshot window, not last_over_time, so a single degraded scan reporting 0 cannot blank a tile while a signal that stays blind still reads 0. The two detail tables keep one row per item, which is the right shape for a click-through list, so they remain bounded by the ceiling. Their panel descriptions now say so, name the two traps that put them there, and record that topk() does not evade the limit because it is checked on the inner aggregation.
GitHub reports the alerts of the code a fork inherited as the fork's own, so a fork of a large project surfaces hundreds of findings in code the owner did not write. Measured on one owner: a single fork of a big upstream project carried 500 open alerts against 6 across every first-party repo, which drowns the signal the code-scanning panel exists to raise. CODE_SCANNING_EXCLUDE_REPOS could already silence a fork by name, but only once someone remembered to add it, and the flood arrives the moment a scan is enabled on a new one. CODE_SCANNING_EXCLUDE_FORKS (default true) is the batch form of that rule, so a fork is covered when it is created. Scoped to code scanning only, matching the sibling list: a fork's own pull requests and Actions runs are the owner's work. ghsignal.Repo now carries the API's fork bit and codeScanningExcluded ORs the two conditions, so a fork and a named repo produce the identical skip: no API call, a Debug line, and no entry in the integrity verdict, so neither can mark a scan degraded nor mask a real code-scanning blackout. The two mechanisms stay independent, so the flag can be off while individual forks stay named. Tests cover both directions of the flag, its default, envx's tolerant parsing, and independence from the name list. Red-checked against four mutants: the default flipped to false, the fork bit dropped from the decode, the fork condition dropped from the predicate, and the flag wired always-on, which is what stops the knob silently becoming a hardcoded rule. The README documents the behaviour and how to turn it off. The compose example deliberately does not carry the variable, since a knob that defaults correctly belongs in the configuration reference.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two fixes to the same reported symptom: the Grafana dashboard's code-scanning and failed-runs panels returned
maximum number of series (500) reached for a single queryinstead of data.The tiles no longer count item lines. A metric query costs one Loki series per output row, and Loki caps a query at
max_query_series(500 by default), so the three snapshot tiles errored once the open-alert count crossed it. They now read the per-scanscan completesummary line, which already carriesopen_prs,open_issuesandcode_alerts. That is a single series at any data volume.Forks are skipped for code scanning by default. GitHub reports the alerts of the code a fork inherited as the fork's own. One fork of a large upstream project measured 500 open alerts against 6 across every first-party repo, which is both what drowned the signal and what pushed the panel past the ceiling.
CODE_SCANNING_EXCLUDE_REPOScould silence a fork by name, but only once someone added it;CODE_SCANNING_EXCLUDE_FORKS(defaulttrue) covers a fork the moment it is created.The flag is scoped to code scanning only, matching the sibling list: a fork's own pull requests and Actions runs are the owner's work. A fork and a named repo produce the identical skip, so neither can mark a scan degraded nor mask a real code-scanning blackout.
What is deliberately unchanged
The two detail tables keep one row per item, which is the right shape for a click-through list, so they stay bounded by the datasource ceiling. Their panel descriptions now say so and record that
topk()does not evade the limit, because Loki checks it on the inner aggregation beforetopkreduces it. The compose example does not carry the new variable, since a knob that defaults correctly belongs in the configuration reference.Testing
go test -count=1 -race ./...clean across all 6 packages;golangci-lint run ./...0 issues.envx's tolerant parsing of a malformed value, and independence from the name list.false, the fork bit dropped from the decode, the fork condition dropped from the predicate, and the flag wired always-on (which is what stops the knob silently becoming a hardcoded rule).