Skip to content

feat(cognitive-memory): durable fact_snapshot_dedup_ratio graph-memory hygiene self-metric - #4879

Draft
rysweet wants to merge 1 commit into
mainfrom
engineer/continuously-research-and-improve-your-own-cogn-70ab8541-1785153646-a8ebba
Draft

feat(cognitive-memory): durable fact_snapshot_dedup_ratio graph-memory hygiene self-metric#4879
rysweet wants to merge 1 commit into
mainfrom
engineer/continuously-research-and-improve-your-own-cogn-70ab8541-1785153646-a8ebba

Conversation

@rysweet

@rysweet rysweet commented Jul 27, 2026

Copy link
Copy Markdown
Owner

Summary

Adds a durable, regressable graph-memory hygiene self-metric,
fact_snapshot_dedup_ratio, that turns an already-computed snapshot-revision
dedup signal into a comparable metrics.jsonl series. It is the direct sibling
of the existing fact_provenance_coverage metric and follows that precedent
exactly (pure ratio fn + best-effort per-cycle emitter + daemon wiring from the
same graph_stats() snapshot).

This advances the standing cognition-improvement goal on the graph-memory
axis: it makes controlled-forgetting / pruning hygiene observable over time so
a regression is visible rather than silent, without touching any tuned recall,
distillation, or reasoner heuristic (all of which already carry durable metrics
and have deliberate, test-encoded boundaries).

What the metric measures

Snapshot facts (goal-board snapshots and other caller-key streams) are
revisioned: each new revision SUPERSEDES the prior one, and prune_superseded
(controlled forgetting) reclaims the archived tail. graph_stats() already
reports two raw counts for this layer:

Field Meaning
snapshot_facts_total every snapshot revision still held (live + not-yet-pruned superseded)
distinct_snapshot_caller_keys distinct logical snapshot streams behind them
fact_snapshot_dedup_ratio = distinct_snapshot_caller_keys / snapshot_facts_total   ∈ (0, 1]

Higher is healthier (1.0 = one live revision per stream). The ratio falls when
superseded revisions accumulate faster than pruning reclaims them — exactly the
monotonic-growth failure controlled forgetting exists to prevent. Undefined
(None, no sample emitted) on an empty snapshot layer, mirroring the
fact_provenance_coverage / recall_precision_at_k "skip rather than drag the
series to zero" convention. Pure observation — never mutates memory state.

Changed surfaces

File Change
src/cognitive_memory/metrics.rs snapshot_dedup_ratio() (pure), record_snapshot_dedup_ratio_metric() (per-cycle emitter), const, 4 unit tests
src/operator_commands_ooda/daemon/mod.rs one emit call in the existing per-cycle graph_stats() sweep (no extra store read)
tests/bin_simard_memory_cli.rs end-to-end assertion tying the metric to the operator-visible memory stats --json counts
docs/reference/telemetry-metrics.md snapshot-hygiene callout (user-facing metric reference)
docs/reference/cognitive-memory-provenance.md "Observability: snapshot-dedup-hygiene self-metric" section

+217 lines, additive only. No behavior change to any existing path; the emitter
is cfg!(test)-skipped and best-effort (a metrics-write failure is logged,
never propagated).

Merge-ready evidence

(1) qa-team / gadugi scenarios. The operator-visible surface this metric is
derived from is covered by the existing gadugi scenario
tests/gadugi/memory-stats-edges.yaml (+ .sh, drives the real binary at the
process boundary asserting the snapshot dedup section in human + --json
output) and by the extended process-boundary integration test
bin_simard_memory_cli::stats_shows_edges_and_dedup_section_via_direct_open,
which now additionally asserts snapshot_dedup_ratio(distinct, total) == Some(1.0)
over a seeded store — so the metric's numerator/denominator can never silently
diverge from what memory stats renders. Following the fact_provenance_coverage
precedent, the internal per-cycle self-metric (which has no direct CLI trigger)
is proven by unit tests + this seeded integration test rather than a separate
gadugi scenario. NOTE: the external gadugi-test runner
(rysweet/gadugi-agentic-test) is not installed in this engineering
sandbox
, so gadugi-test validate/run was not executed locally; CI runs the
gadugi suite on this PR.

(2) Docs. Updated telemetry-metrics.md and cognitive-memory-provenance.md
(the two user-facing homes of the sibling fact_provenance_coverage metric);
docs_integrity tests pass (4/4, all intra-repo .md links resolve).

(3) quality-audit — 3 SEEK→VALIDATE→FIX cycles, ended clean:

  • Cycle 1 — SEEK: metric direction/None-semantics/clamp consistency across code + tests + both docs. VALIDATE: cargo build --lib, targeted unit tests, cargo clippy --lib all green. FIX: none.
  • Cycle 2 — SEEK: doc-claim accuracy. VALIDATE: confirmed store_fact_with_caller_key archives + adds SUPERSEDES, prune_superseded reclaims the superseded tail (library_adapter.rs), and snapshot_facts_total counts live+superseded (memory_cognitive.rs); confirmed doc anchor slugs match the linked headers; confirmed the exact graph_stats() numerator/denominator (library_adapter.rs:1873-1910). FIX: none needed.
  • Cycle 3 — SEEK: boundary-test completeness. Found the distinct=0, total>0 → Some(0.0) (all-keyless snapshots) boundary was unasserted. FIX: added the assertion (mirroring provenance_coverage(0,4)==Some(0.0)). VALIDATE: unit tests re-run green; full cargo test --lib cognitive_memory:: = 166 passed, 0 failed.

(4) CI. Local equivalents of the CI gates all green: cargo fmt --all --check,
cargo clippy --release --no-deps -- -D warnings (pre-commit), Rust-only gate +
race-subset cargo test (pre-push), plus targeted unit/integration/module/docs
suites above. CI on the PR is the authoritative check — this PR is a draft
until CI is confirmed 100% green
.

(6) Focused diff. 5 files, additive only, no unrelated edits (see
git diff --stat).

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

…f-metric

Add a durable, regressable graph-memory *hygiene* self-metric that turns the
already-computed snapshot-revision dedup counts into a comparable metrics.jsonl
series, mirroring the existing `fact_provenance_coverage` precedent.

Snapshot facts (goal-board snapshots and other caller-key streams) are
revisioned: each new revision SUPERSEDES the prior, and `prune_superseded`
(controlled forgetting) reclaims the archived tail. `graph_stats()` already
reports `snapshot_facts_total` (live + not-yet-pruned superseded revisions) and
`distinct_snapshot_caller_keys` (distinct logical streams), but those raw counts
were only visible via `simard memory stats` / OTel gauges — never as a durable
series. Their ratio is the average *liveness* of the snapshot layer
(distinct / total, in (0,1], higher is healthier); it falls when superseded
revisions accumulate faster than pruning reclaims them — exactly the
monotonic-growth failure controlled forgetting exists to prevent.

- `cognitive_memory::metrics::snapshot_dedup_ratio` — pure ratio fn: None on an
  empty snapshot layer (skip, don't drag the series to a misleading 0.0),
  defensive clamp so a miscount can't exceed 1.0.
- `record_snapshot_dedup_ratio_metric` — best-effort, cfg!(test)-skipped
  per-cycle emitter, sits beside `record_provenance_coverage_metric`.
- Wired into the daemon per-cycle sweep from the SAME `graph_stats()` snapshot
  (no extra store read).
- Unit tests (ratio math, None-for-empty, 0.0-when-keyless, clamp,
  no-op-under-test) + an end-to-end assertion in the seeded
  `bin_simard_memory_cli` integration test tying the metric to the
  operator-visible `memory stats --json` counts so they can't silently diverge.
- Docs: telemetry-metrics + cognitive-memory-provenance observability sections.

Pure observation: never changes memory state.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown

📊 Coverage Summary

Generated by cargo llvm-cov --workspace --summary-only (nightly, excluding test files)

Module Lines Covered Coverage
Total 205598 173104 84.2%

Coverage data from CI run. Test files matching tests?/ are excluded from line counts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant