Skip to content

feat(completeness): add --incomplete-policy to control exit behavior on incomplete scan data - #1019

Open
luojiyin1987 wants to merge 3 commits into
OWASP:mainfrom
luojiyin1987:feat/incomplete-scan-policy
Open

feat(completeness): add --incomplete-policy to control exit behavior on incomplete scan data#1019
luojiyin1987 wants to merge 3 commits into
OWASP:mainfrom
luojiyin1987:feat/incomplete-scan-policy

Conversation

@luojiyin1987

@luojiyin1987 luojiyin1987 commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add --incomplete-policy warn|error (default: warn) to control exit code when scan detection data is incomplete.

Behavior

Scenario warn error
Scan complete normal normal
Remediation gap warning, normal exit warning, normal exit
Detection gap warning, normal --fail-on EXIT_ERROR (3)

error only targets detection-impact incomplete data. Remediation gaps are never affected by this flag.

Changes

  • src/types.tsIncompletePolicy type, field on ParsedOptions
  • src/cli/args.ts — default warn, parse --incomplete-policy warn|error
  • src/cli/help.ts — help text + example
  • src/scan/completeness.tsshouldFailForIncompleteScan() helper
  • src/index.ts — single-folder exit code
  • src/scan/multi-folder-scan.ts — multi-folder exit code, shared aggregated completeness
  • action.ymlincomplete-policy input (empty default for old CLI compat)
  • tests/helpers.test.ts — updated expectations

Design decisions

  • Ratchet unaffected. --ratchet detection gap check runs independently. incompletePolicy does not override it.
  • EXIT_ERROR (3) over EXIT_FINDINGS (1). Incomplete scan is more severe than findings.
  • Action backward compat. Empty default means old CLI versions don't see unknown args.

Closes #898

…on incomplete scan data

Add `--incomplete-policy warn|error` (default: warn). When set to error,
EXIT_ERROR (3) is returned on detection-impact incomplete scan data.
Remediation gaps and ratchet behavior are unaffected.

Closes OWASP#1018
- Arg parsing: default warn, parse warn/error, =form, invalid throws
- Single-folder: detection+warn, detection+error, remediation+error, ratchet+warn
- Multi-folder: detection+warn, detection+error, remediation+error, multi-folder detection, ratchet+warn
- Fix action.yml: pass incomplete-policy through Apply security fixes step
- Docs: add --incomplete-policy to CLI reference and workflow integration
@luojiyin1987

Copy link
Copy Markdown
Collaborator Author

Addressed all three review items:

P1 — Behavioral tests (+16 tests, 1446 → 1462):

  • tests/helpers.test.ts: 7 tests — default warn, parse warn/error, = form, invalid values throw
  • tests/cli-integration.test.ts: 4 tests — detection+warn → exit 0, detection+error → EXIT_ERROR, remediation+error → exit 0, ratchet+detection+warn → still EXIT_ERROR
  • tests/multi-folder-scan.test.ts: 5 tests — detection+warn → exit 0, detection+error → EXIT_ERROR, remediation+error → exit 0, multi-folder detection → EXIT_ERROR, ratchet+detection+warn → still EXIT_ERROR

P2 — Action fix step: Added INPUT_INCOMPLETE_POLICY env + conditional arg passing in the Apply security fixes step, so incomplete-policy: error applies to both the scan and the fix rescan.

P3 — Docs:

  • website/docs/cli-reference.md: Added --incomplete-policy row in CI / Automation options table
  • website/docs/workflow-integration.md: Added Fail closed on incomplete detection data subsection with CLI and GitHub Actions examples

@sonukapoor

Copy link
Copy Markdown
Collaborator

Hey @luojiyin1987, thanks for this - the completeness work you've done has been great. There's an existing PR (#941) for the same feature that has priority since it came first. I've asked that contributor to address the outstanding changes within the next 48 hours. If they don't respond we'll close #941 and come back to yours. Will keep you posted!

@luojiyin1987

Copy link
Copy Markdown
Collaborator Author

No worries at all — I'm not in a rush. Happy to wait and see how #941 goes. Thanks for keeping me posted! @sonukapoor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] add a configurable failure policy for incomplete scans

2 participants