Skip to content

feat(redact): path p redact credential redaction - #153

Open
ulmentflam wants to merge 10 commits into
empathic:mainfrom
ulmentflam:evan/redact
Open

feat(redact): path p redact credential redaction#153
ulmentflam wants to merge 10 commits into
empathic:mainfrom
ulmentflam:evan/redact

Conversation

@ulmentflam

@ulmentflam ulmentflam commented Jul 31, 2026

Copy link
Copy Markdown

Adds path p redact, a plumbing command that removes credentials from an
already-derived toolpath document in place, via a reviewable plan-then-apply
flow. New tier-2 crate toolpath-redact holds the engine; path-cli gets a
thin cmd_redact.rs.

Spec: docs/superpowers/specs/2026-07-30-path-redact-command-design.md
Plan: docs/superpowers/plans/2026-07-30-path-redact-command.md

What it does

path p redact --input <cache-id|file> --dry-run          # every surface, findings or not; exit 1 if any
path p redact --input <ref> --dry-run --json > plan.json # reviewable plan
path p redact --input <ref> --plan plan.json             # apply exactly what the plan describes
path p redact --input claude-abc123                      # rewrite the cache entry in place

Detection sits behind a Detector trait, because that is where precision is
worst and the field moves fastest; traversal is stable and shared. The
built-in detector compiles a vendored gitleaks ruleset (224 rules, commit
b58d3f10, MIT). Five transforms, marker the default, which emits a typed
[REDACTED:<rule>:<keyed-fingerprint>].

p cache sync replays a stored RedactionPolicy after a re-derive, so
resuming a redacted session and syncing leaves it redacted with new turns
redacted too. That is the hazard the design was built around:
is_unchanged decides re-derivation from mtime and size and never reads the
document.

Verification

Against a real 46-step Claude session carrying seven credential types:

dry-run exit: 1        21 finding(s) redacted
Valid: Graph (id: path-claude-code-88ad1795, 1 path)
anthropic 0 · aws-id 0 · github 0 · stripe 0 · slack 0 · gitlab 0 · db-uri 0
markers: 21            idempotent: PASS        mode: -rw-------

toolpath-redact 201 tests, path-cli 383 lib + 67 integration. Nothing
#[ignore]d. clippy -D warnings and fmt clean on both crates.

What review caught that tests did not

Every item below was reproduced before being fixed, and none of the 140
tests green at the time would have failed on any of them.

  • The threshold ran after overlap resolution. Resolution is score-blind
    on length, so a 0.6 container spanning a 0.99 match evicted it and the
    threshold then discarded the survivor. Both lost, and the plan reported
    only the low-confidence one. detect.rs's own doc comment forbade this in
    capitals; the code below it did it anyway.
  • verify never compared content, so a plan generated against one
    document was accepted against a mutated one and markers landed at stale
    offsets. Findings now carry a keyed fingerprint that verify recomputes.
  • The audit record republished the credential. Pointers beneath a
    redacted artifact key were recorded from the pre-redaction key, so the
    pass removed the secret from change and wrote it back under
    meta.redaction.
  • Overlapping spans left half a credential in place while the report
    claimed both were replaced.
  • Two of three credentials in a real session went undetected. gitleaks'
    anthropic rule is pinned to a live key's exact length, and
    generic-api-key stopwords ant-, so every sk-ant- key was excluded
    from the one rule that would have caught it.
  • Three rules were excluded as "dialect incompatible" when they only
    needed a larger compiled-size limit. One was generic-api-key, gitleaks'
    catch-all.
  • A zero-finding run still rewrote the cache file, and the flattened
    HashMaps reseed per process, so a second pass was semantically identical
    but byte-different. The integration fixtures' single-key maps, chosen to
    dodge that flake, made them structurally unable to catch it.

Deliberately not closed

Recorded in docs/superpowers/notes/2026-07-30-redaction-known-gaps.md
rather than fixed, because closing them changes behaviour beyond this
feature's scope. Worth reading before relying on this:

  • p import --force, share, p cache rm, and a redact racing a sync each
    write cleartext over a redacted cache entry.
  • The field map is a closed list. ~3100 of 6800 string leaves in a real
    753-step session are never scanned, including environment.vcs_branch and
    step.meta.intent (the git commit message). No credential in the verified
    document sits in one, but this is the most likely next gap.
  • hash is not idempotent: bare 6-hex has no envelope, so each pass
    re-redacts the last one's output.
  • share uploads the un-redacted derivation.

Notes for review

  • Detector test fixtures are split across concat! so the source text does
    not match the pattern each value tests. They are synthetic, but GitHub
    push protection scans the file, not the compiled string.
  • cargo clippy --workspace -- -D warnings and cargo fmt --check remain
    red on findings that predate this branch (cmd_list.rs, cmd_import.rs,
    toolpath-pi/src/reader.rs, rustfmt drift in toolpath-codex). Left
    untouched deliberately.
  • docs/superpowers/notes/2026-07-30-redact-execution-state.md is a
    working checkpoint and should be deleted on merge.

🤖 Generated with Claude Code

https://claude.ai/code/session_012EurfyRkeTRc74v1mDum61


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

ulmentflam and others added 10 commits July 30, 2026 15:17
Design and plan for an explicit post-generation redaction pass over a
generated toolpath document.

Spec covers: why post-generation rather than hook-time (a controlled
experiment shows Claude Code hooks cannot keep a secret off disk), the
schema-aware field map, the `Detector` plug point, five transforms,
plan-then-apply with a dry run that surfaces every field it looked at,
the per-step audit record, in-place cache semantics, and the sync
collision that in-place redaction creates.

Plan is test-driven and organised into waves for parallel execution:
one small blocking vocabulary task, then six independent tracks. Each
task writes failing tests first and is gated on both green tests and an
adversarial review pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Types and trait signatures only; bodies are todo!(). Every Wave 1 track
builds against this contract, so it lands first and alone.

Deviations from the plan's literal snippets, all forced by
`clippy -D warnings` or by a cross-task call site:
- Transform derives Default instead of hand-writing the impl.
- SurfaceCursor.path is pub so the unread field does not trip dead_code.
- RedactError gains BadPredicate for parse_predicate's error path.
- DetectorSet::detectors() so plan generation can run the egress check.
- plan.rs carries todo!() stubs for every function that crosses a task
  boundary, so the contract is fixed before the fan-out.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…docs

T1 replaced the plan's greedy pairwise overlap eviction: on a three-way
overlap (A-B overlap, B-C overlap, A-C disjoint) it dropped A entirely,
leaving a real secret unredacted. Best-first interval selection keeps the
same stated policy without eviction. Scores compare with total_cmp and
NaN scores are dropped, because a NaN sorts above +inf and would win
every overlap before the threshold silently discarded it.

T4 splices edits right-to-left in one pass, validating each span against
the string as spliced so far so a normalisation bug upstream degrades
instead of panicking.

Applies the T0 review: Fingerprint gains Hash/Ord for the audit record's
(rule, fp) aggregation, the unimplemented Transformer trait is gone
rather than shipping as semver debt at 0.1.0, and Transform's doc no
longer claims a leak boundary that Mask contradicts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Applies the T1 review. The one finding worth acting on: overlap
resolution is score-blind on length, so a low-scoring container evicts a
high-scoring finding nested inside it. Thresholding the OUTPUT of
detect_all would then discard the container and take the survivor with
it, publishing the secret. Recorded as an ordering constraint on
detect_all so plan generation filters before normalisation, not after.

Also from that review: three tests covering branches nothing exercised -
a span starting mid-codepoint (the existing test only ever hit the end
boundary), the leftmost tie-break, and two genuinely distinct detectors
contending for the same bytes. The leftmost tie-break is deliberate and
now pinned: breaking on rule name would let an untrusted detector win
contested bytes by naming its rule `aaa`.

Green: detect 23, surface 23, plan 29, transform 16. The internal
detector's vendored-ruleset loading is still red - gitleaks ships rules
that match on path alone and carry no `regex` field.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Completes the engine. T3 vendors the gitleaks ruleset (221 rules load;
1 is path-only with no regex, 3 blow past regex's 10 MiB compiled-size
limit under RE2 semantics, all four excluded with reasons). T7 rewrites
documents from an approved plan. T8 generates plans. T10 replays a
stored policy after a re-derive.

T10 closes the hazard the plan was written around: `is_unchanged` decides
re-derivation from mtime and size and never reads the document, so a
resumed session's re-derive would silently overwrite an in-place
redaction. A missing key now fails that artifact before any write rather
than publishing the secret it was hiding.

`p cache rm` retires the policy alongside the key, not just the key: a
policy whose key is gone would fail that artifact's sync forever with no
recovery path.

Sync replay is unattended, so it refuses to strip signatures or to run a
network detector without a human present.

Applies the T6 and T12 reviews. Two were hard breaks: the toolpath-cli
shim still pinned path-cli 0.16.0, which broke `cargo metadata` on the
shim outright, and release.sh listed toolpath-redact in ALL_CRATES but
not in either tier-2 publish loop, so tier 3 would have failed against
crates.io on an unresolvable dependency. Also adds a threshold range
check - scores clamp to 0.0..=1.0, so `--threshold -1` silently meant
"redact everything" and `5` meant "redact nothing".

Green: toolpath-redact 140, path-cli 379 lib + 56 integration.
Not implemented: T9 CLI dispatch, T11 integration tests, exec.rs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two adversarial reviews found paths that write cleartext over a redacted
cache entry: p import --force, share, p cache rm, and a redact racing a
sync. None is fixed. Closing them means changing shipped behavior, which
is out of scope for this branch, so they are written down instead.

Also records that idempotence is unproven for the hash and partial
transforms, which have no recognisable output form for a re-scan to skip.

The execution-state note is a checkpoint, not documentation. Delete it
when the branch merges.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Wires the CLI, closes the end-to-end test gaps, and fixes the defects
four adversarial reviews and one real user run found. Every defect below
was reproduced before being fixed, and none of the 140 tests passing at
the previous commit would have caught any of them.

The threshold was applied after overlap resolution, which detect.rs's own
doc comment forbids. Resolution is score-blind on length, so a 0.6
container spanning a 0.99 match evicted it and the threshold then dropped
the survivor: both lost, and the plan named only the low-confidence one.
Findings are partitioned on the threshold before resolving, so a
sub-threshold finding can never evict an above-threshold one.

`verify` never compared content, so a plan generated against one document
was accepted against a mutated one and apply spliced markers at stale
offsets. Findings now carry a keyed fingerprint that verify recomputes.

The audit record republished the credential: pointers beneath a redacted
artifact key were recorded from the pre-redaction key, so the pass removed
the secret from `change` and wrote it back under `meta.redaction`.

Overlapping spans left half a credential in place while the report claimed
both were replaced. Overlap is refused before any edit is built, the same
way empty and inverted spans already were.

Detection missed two of three credentials in a real session. gitleaks'
anthropic rule is pinned to a live key's exact length, and `generic-api-key`
stopwords `ant-`, so every sk-ant- key was excluded from the one rule that
would have caught it. Documentation-key allowlist regexes are now suppressed
for redaction while stopwords are kept: gitleaks scans source trees where a
README quoting AWS's example key is noise, but a transcript about to be
published cannot distinguish a copied placeholder from a real key whose
owner ended it in EXAMPLE.

A zero-finding run still rewrote the cache file, and `extra`/`change` are
flattened HashMaps seeded per process, so re-serialising churned key order
and broke byte-identity on a second pass. The integration fixtures' own
single-key maps, chosen to avoid that flake, made them incapable of
catching it.

Detector fixtures are split across `concat!` so the source text does not
match the pattern each value tests. They are synthetic, but GitHub push
protection scans the file rather than the compiled string.

Verified against a real 46-step session carrying seven credential types:
21 typed markers, zero residual, valid against the base and kind schemas,
byte-identical on a second pass, cache mode preserved at 0600.

toolpath-redact 201, path-cli 383 lib + 67 integration. No test ignored.

Known gaps, deliberately not closed, are recorded in
docs/superpowers/notes/2026-07-30-redaction-known-gaps.md.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The field map excluded `actors` on the stated grounds that redaction
"drops wholesale rather than rewrites" it. That is true of `signatures`,
which `guard_signatures` clears, and false of `actors`, which nothing
drops. On a git-derived document the committer's real name and email
address shipped unscanned. Scanning them is the only thing that makes the
doc comment true.

Surfaced: actor `name`, `identities[i]`, and `keys[i].href`. Still
excluded, now with reasons and negative assertions: the actor map's own
keys, because `step.actor` names them and is not rewritten, so renaming
one detaches every step from its actor; `provider`/`model`, which are
harness and API vocabularies; and `keys[i].fingerprint`, a digest of a
public key, where rewriting hides nothing and breaks any signature that
survived.

The completeness test was passing by absence: `fixture_rich` omitted
`step.parents`, `path.graph_ref`, `PathMeta.kind`/`source`,
`VcsSource.change_id`, and both `signatures` vectors, so five of the
exclusions were never proven to be decisions rather than oversights. The
fixture now carries every string-bearing field of the `Path` type tree and
the allowlist grew from 20 entries to 47.

Its phantom check also accepted any pointer resolving to an object, which
would have swallowed a surface aimed at a subtree; it now requires the
artifact-key shape specifically.

Every remaining miss on a 753-step session, two smaller ones, and a
git-derived document is an allowlisted id, enum, or type.

`Graph.meta` is still never redacted: `cmd_redact` iterates `doc.paths`,
so a graph's own title, intent, refs, and actors are unreachable from any
call on its members, and every cached document is a `Graph`. Recorded as a
stated non-goal on `surfaces()` rather than left silent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Upstream added the Claude Code plugin and bumped path-cli to 0.16.1.
Conflicts were bookkeeping only; no source file conflicted.

- path-cli and toolpath-cli take 0.16.2, a patch on top of upstream's
  0.16.1 rather than the 0.17.0 this branch carried.
- CHANGELOG and CLAUDE.md each had both sides appending a new entry at
  the same anchor; both are kept.
- Cargo.lock regenerated rather than hand-merged.

The redaction "Things to know" entry is corrected while resolving: `hash`
is the only non-idempotent transform now that markers are excluded by
span intersection, and `Graph.meta` is named as never scanned, since
`cmd_redact` iterates `doc.paths` and a graph's own title, intent, refs
and actors are unreachable from its members.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`cmd_redact` resolves detectors through `sync::build_detectors` and
records the replay policy through `sync::record_redaction_policy`, but
`sync` is itself `#[cfg(not(target_os = "emscripten"))]`, so both names
were unresolved on that target. `deploy-site.yml` builds
wasm32-unknown-emscripten on every pull request, so this was a red job
rather than a hypothetical.

Gated rather than made portable: redaction needs the document cache and
the on-disk key store, neither of which exists on that target.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant