Skip to content

fix(sast): ship an explicit gate-4 scan set (no-op today, closes the silent hole) - #120

Open
Prekzursil wants to merge 1 commit into
mainfrom
fix/sast-depth-explicit-scan-set-and-taint-rules
Open

fix(sast): ship an explicit gate-4 scan set (no-op today, closes the silent hole)#120
Prekzursil wants to merge 1 commit into
mainfrom
fix/sast-depth-explicit-scan-set-and-taint-rules

Conversation

@Prekzursil

Copy link
Copy Markdown
Owner

What this is

Gate 4 (SAST) has been scanning a file set it inherited from a vendor default, not one this repo chose — and the difference is invisible in the gate output. This ships an explicit .semgrepignore so the scanned set becomes a deliberate, reviewable decision.

Measured on the pinned CI engine — opengrep v1.22.0 CLI on Linux, exactly the build reusable-quality.yml installs — with a controlled detector and one variable changed at a time. Confidence: almost certain (90-99%).

The hole

file content result
src/inline.go db.Query(fmt.Sprintf(...)) FIRES
tests/inline.go byte-identical to the file above SILENT

paths.scanned listed 3 of 4 files and paths.skipped reported none — nothing in the gate output says a test tree was dropped. Pointing the scanner directly at tests/ returns paths.scanned: [], so an explicit path argument does not override it either.

Enumerated precisely, the built-in template drops *_test.go at any depth, plus the test/ and tests/ directories. Python/TS test filenames (test_app.py, app_test.py, conftest.py, app.test.ts, app.spec.ts, __tests__/) are not dropped — only the two directories. The engine abandons the built-in template as soon as any .semgrepignore exists, so shipping this file is the whole fix.

Effect on this repo, using its own gate-4 scan paths (backend frontend)

No change today: 211 files scanned before and after, 0 findings either way.

Nothing under those paths currently matches *_test.go or a test//tests/ directory, so this file is a no-op against the tree as it stands. It is still worth landing: it makes the scanned set a deliberate choice, so the first test directory added here gets scanned instead of being silently dropped — and it removes the dependence on a vendor default nobody picked.

Both-states proof, against this repo's own curated ruleset

The same eval() file placed in src/ and under tests/:

with this .semgrepignore      -> BOTH fire,  gate exit 1
with it removed (control)     -> src/ fires, tests/ INVISIBLE
clean ast.literal_eval file   -> silent,     gate exit 0

The control is the load-bearing half: a probe that is silent in the broken state measures nothing, so the hole is demonstrated rather than asserted.

Scope discipline

  • No exclusion was added to make a finding go away. The finding count is unchanged (0 before, 0 after).
  • Every directory entry ships both the root-anchored and the nested form, because a bare **/x/ does not match a root-level x/ and fails open with no warning — measured with a positive control (**/vendor/ left a root-level vendor/pkg/b.go scanned; vendor/ excluded both trees).
  • The file carries its own editing rules, including "never add test//tests/", so the hole cannot be quietly reopened.

Related

Control-plane companion — the copyable template plus a non-blocking gate-4 warning for any caller with no explicit scan set: Prekzursil/quality-zero-platform#289. That PR also records, in known-issues/QZ-SAST-001.yml, a second measured gate-4 gap: the curated rules match SQL-injection syntax only, so the same bug written through a local variable escapes them. The obvious mode: taint fix was withdrawn after it produced 58 false positives across momentstudio and Reframe; the entry carries the evidence and the settling experiment.

Gate 4 has been scanning a file set it INHERITED from a vendor default rather
than one this repo chose, and the difference is invisible in the gate output.

Measured 2026-08-11 on the PINNED CI engine (opengrep v1.22.0 CLI on Linux,
exactly the build reusable-quality.yml installs), controlled detector, one
variable changed at a time:

    src/inline.go    db.Query(fmt.Sprintf(...))        -> FIRES
    tests/inline.go  byte-identical to the line above  -> SILENT

`paths.skipped` reported `none` while it happened, so nothing in the gate output
revealed it, and pointing the scanner directly at a test path does not override
it. The dropped set, enumerated with one identical vulnerable file per candidate
path: `*_test.go` at ANY depth, plus the `test/` and `tests/` directories.
The engine abandons the built-in template as soon as ANY `.semgrepignore`
exists, so shipping this file restores the test tree.

EFFECT ON THIS REPO, using its own gate-4 scan paths (backend frontend):

    211 files scanned before, 211 after (+0), 0 findings either way.

BOTH-STATES PROOF, run against this repo's own curated ruleset:

    the same `eval()` file in src/ and under tests/
      with this .semgrepignore   -> BOTH fire,  gate exit 1
      with it removed (control)  -> src/ fires, tests/ INVISIBLE
    a clean `ast.literal_eval` file -> silent, gate exit 0

The control is the load-bearing half: a probe that is silent in the broken state
measures nothing, so the hole is demonstrated rather than asserted.

No exclusion was added to make any finding go away -- the finding count is
unchanged. Every directory entry ships both the root-anchored and the nested
form, because a bare `**/x/` does NOT match a root-level `x/` and fails open
with no warning (measured with a positive control).

Companion control-plane change -- the copyable template plus a non-blocking
gate-4 warning for callers with no explicit scan set:
Prekzursil/quality-zero-platform#289.
@Prekzursil

Copy link
Copy Markdown
Owner Author

The red on this PR is gate-deps, not this change — and gate-sast passed

quality / quality is one job over six gates, so a red badge does not say which gate failed. It was gate 6 (osv-scanner), and this PR adds a single file (.semgrepignore) that osv-scanner never reads — it reads lockfiles and osv-scanner.toml.

The gate this PR actually touches passed:

Scanning 168 files tracked by git with 22 Code rules:
Ran 22 rules on 163 files: 0 findings.

That is the whole point of the change — 163 files scanned where the inherited default scanned fewer, and the finding count did not move.

Why gate-deps is red, with the evidence that it is not this branch:

  • The findings are two npm dev-dependency CVEs in a lockfile this PR does not touch:
    brace-expansion (dev) 5.0.8 (GHSA-rgw5-rvv9-x895, CVSS 7.5) and nanoid (dev) 3.3.16 (GHSA-2v37-7h3g-55p8, CVSS 8.2).
  • Sibling PRs that show CLEAN last ran their checks on 2026-07-31. Their green is stale, not a contrast: the advisory feed refreshed in between, so they have simply not been re-evaluated against today's data.
  • On the other two repos in this series the same gate reds every open PR, including pure Dependabot ones that predate this work.

This is the mechanism already measured in the fleet strategy: pinning the tool does not pin the data. A CVE feed refreshes externally, so gate-deps can turn red with zero code change — the same repo's main was green on 2026-07-25 and the identical directive scans differently today. Fixing it means bumping the dev dependencies (or applying the severity/dev-scope floor being tracked separately), and is out of scope for this PR.

Reviewing this PR on its own terms: read the gate-sast opengrep step, which is green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant