Re-validate detect → verify → publish path before re-enabling auto-publish

[Ap6pack]

Closed

The core of this issue — auditing the over-convicting detection that inflated MALICIOUS from 476 to a verified 184 — is done and merged:

• fix(monitor): stop single-rule regex hits from auto-convicting as MALICIOUS #27 installer-host allowlist + fragile-verdict downgrade
• fix(monitor): treat PERSIST-002 and MULTI-001 as high-FP single-rule verdicts #33 PERSIST-002 / MULTI-001 added to the high-FP set
• fix(monitor): extend high-FP set after auditing the residual verdicts (#30) #35 CMD-002, CRED-002, EXFIL-001, EXFIL-003, PI-001, HIJACK-001 added; PERSIST-001 / EXFIL-002 / FRAUD-002 confirmed to stay confident

The remaining items (detector novelty logic, human-approval gate, end-to-end dry-run mode) only matter if the daily auto-publish workflow (threat-research.yml) is re-enabled — and it is currently paused (manual dispatch only). If/when we decide to turn autonomous publishing back on, we'll open a fresh issue scoped to exactly what that needs, rather than keep this one open indefinitely.

Detection quality and enumeration coverage are tracked separately (enumeration fixes in #34/#36).

[Ap6pack]

Progress: residual single-rule MALICIOUS audit (item 1)

Audited the 82 MALICIOUS verdicts that rested on a single non-high-FP rule by running those rules against benign inputs (same method that caught CMD-001/ENV-001).

Breakdown of the 82: PERSIST-002 (31), PERSIST-001 (15), MULTI-001 (12), PI-001 (6), EXFIL-002 (4), EXFIL-001 (4), CMD-002 (4), EXFIL-003 (2), CRED-002 (2), FRAUD-002 (1), HIJACK-001 (1).

Found over-flagging (now fixed in #33):

• MALWAR-PERSIST-002 — matches self-referential file ops on SKILL.md/CLAUDE.md/.claude, including reads (cat SKILL.md | grep) and legit skill-authoring tools that write a CLAUDE.md.
• MALWAR-MULTI-001 — matches benign prose ("applies the patch without showing the full diff").

Both added to HIGH_FP_RULES → verify-or-downgrade instead of auto-convict. Tier shrinks 82 → 39.

Held up (no FP found; stays confident):

• MALWAR-PERSIST-001 (cron/systemd/.bashrc) — tighter patterns, did not over-flag.

Still to check (the remaining 39, lower volume): PI-001, EXFIL-001/002/003, CMD-002, CRED-002, FRAUD-002, HIJACK-001. Next audit pass.

Other checklist items (detector novelty logic, human-approval gate before external disclosure, dry-run mode) unchanged.

---

Generated by Claude Code

[Ap6pack]

Residual single-rule audit — complete (#35)

Ran every remaining rule from the 39 against benign inputs. Six more over-flag legitimate content and now get verify-or-downgrade (PR #35):

• CMD-002 (3 legit npx -y lines), CRED-002 ("enter your API key"), EXFIL-001 (reads of ~/.claude//.cursor/), EXFIL-003 (curl -X POST -d "$(cat file)"), PI-001 (skills quoting injection phrases), HIJACK-001 (roleplay/persona).

Held up, stay confident on a single hit (no FP found): PERSIST-001, EXFIL-002, FRAUD-002.

Tier now: 39 → 20 unverified single-rule MALICIOUS, all on non-over-flagging rules. Audit item done.

Still open on this issue

• Detector novelty logic (scripts/threat-detector.py flags the other bucket every run)
• Human-approval gate before external disclosure
• End-to-end dry-run mode
• (separately, not this issue) baseline coverage restored by fix(monitor): resilient registry enumeration — stop silent truncation of the 66k baseline #34; verification re-scan in progress

---

Generated by Claude Code