feat(cognitive-memory): durable fact_snapshot_dedup_ratio graph-memory hygiene self-metric - #4879
Draft
rysweet wants to merge 1 commit into
Conversation
…f-metric Add a durable, regressable graph-memory *hygiene* self-metric that turns the already-computed snapshot-revision dedup counts into a comparable metrics.jsonl series, mirroring the existing `fact_provenance_coverage` precedent. Snapshot facts (goal-board snapshots and other caller-key streams) are revisioned: each new revision SUPERSEDES the prior, and `prune_superseded` (controlled forgetting) reclaims the archived tail. `graph_stats()` already reports `snapshot_facts_total` (live + not-yet-pruned superseded revisions) and `distinct_snapshot_caller_keys` (distinct logical streams), but those raw counts were only visible via `simard memory stats` / OTel gauges — never as a durable series. Their ratio is the average *liveness* of the snapshot layer (distinct / total, in (0,1], higher is healthier); it falls when superseded revisions accumulate faster than pruning reclaims them — exactly the monotonic-growth failure controlled forgetting exists to prevent. - `cognitive_memory::metrics::snapshot_dedup_ratio` — pure ratio fn: None on an empty snapshot layer (skip, don't drag the series to a misleading 0.0), defensive clamp so a miscount can't exceed 1.0. - `record_snapshot_dedup_ratio_metric` — best-effort, cfg!(test)-skipped per-cycle emitter, sits beside `record_provenance_coverage_metric`. - Wired into the daemon per-cycle sweep from the SAME `graph_stats()` snapshot (no extra store read). - Unit tests (ratio math, None-for-empty, 0.0-when-keyless, clamp, no-op-under-test) + an end-to-end assertion in the seeded `bin_simard_memory_cli` integration test tying the metric to the operator-visible `memory stats --json` counts so they can't silently diverge. - Docs: telemetry-metrics + cognitive-memory-provenance observability sections. Pure observation: never changes memory state. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
📊 Coverage Summary
Coverage data from CI run. Test files matching |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a durable, regressable graph-memory hygiene self-metric,
fact_snapshot_dedup_ratio, that turns an already-computed snapshot-revisiondedup signal into a comparable
metrics.jsonlseries. It is the direct siblingof the existing
fact_provenance_coveragemetric and follows that precedentexactly (pure ratio fn + best-effort per-cycle emitter + daemon wiring from the
same
graph_stats()snapshot).This advances the standing cognition-improvement goal on the graph-memory
axis: it makes controlled-forgetting / pruning hygiene observable over time so
a regression is visible rather than silent, without touching any tuned recall,
distillation, or reasoner heuristic (all of which already carry durable metrics
and have deliberate, test-encoded boundaries).
What the metric measures
Snapshot facts (goal-board snapshots and other caller-key streams) are
revisioned: each new revision
SUPERSEDESthe prior one, andprune_superseded(controlled forgetting) reclaims the archived tail.
graph_stats()alreadyreports two raw counts for this layer:
snapshot_facts_totaldistinct_snapshot_caller_keysHigher is healthier (
1.0= one live revision per stream). The ratio falls whensuperseded revisions accumulate faster than pruning reclaims them — exactly the
monotonic-growth failure controlled forgetting exists to prevent. Undefined
(
None, no sample emitted) on an empty snapshot layer, mirroring thefact_provenance_coverage/recall_precision_at_k"skip rather than drag theseries to zero" convention. Pure observation — never mutates memory state.
Changed surfaces
src/cognitive_memory/metrics.rssnapshot_dedup_ratio()(pure),record_snapshot_dedup_ratio_metric()(per-cycle emitter), const, 4 unit testssrc/operator_commands_ooda/daemon/mod.rsgraph_stats()sweep (no extra store read)tests/bin_simard_memory_cli.rsmemory stats --jsoncountsdocs/reference/telemetry-metrics.mddocs/reference/cognitive-memory-provenance.md+217 lines, additive only. No behavior change to any existing path; the emitter
is
cfg!(test)-skipped and best-effort (a metrics-write failure is logged,never propagated).
Merge-ready evidence
(1) qa-team / gadugi scenarios. The operator-visible surface this metric is
derived from is covered by the existing gadugi scenario
tests/gadugi/memory-stats-edges.yaml(+.sh, drives the real binary at theprocess boundary asserting the
snapshot dedupsection in human +--jsonoutput) and by the extended process-boundary integration test
bin_simard_memory_cli::stats_shows_edges_and_dedup_section_via_direct_open,which now additionally asserts
snapshot_dedup_ratio(distinct, total) == Some(1.0)over a seeded store — so the metric's numerator/denominator can never silently
diverge from what
memory statsrenders. Following thefact_provenance_coverageprecedent, the internal per-cycle self-metric (which has no direct CLI trigger)
is proven by unit tests + this seeded integration test rather than a separate
gadugi scenario. NOTE: the external
gadugi-testrunner(
rysweet/gadugi-agentic-test) is not installed in this engineeringsandbox, so
gadugi-test validate/runwas not executed locally; CI runs thegadugi suite on this PR.
(2) Docs. Updated
telemetry-metrics.mdandcognitive-memory-provenance.md(the two user-facing homes of the sibling
fact_provenance_coveragemetric);docs_integritytests pass (4/4, all intra-repo.mdlinks resolve).(3) quality-audit — 3 SEEK→VALIDATE→FIX cycles, ended clean:
cargo build --lib, targeted unit tests,cargo clippy --liball green. FIX: none.store_fact_with_caller_keyarchives + addsSUPERSEDES,prune_supersededreclaims the superseded tail (library_adapter.rs), andsnapshot_facts_totalcounts live+superseded (memory_cognitive.rs); confirmed doc anchor slugs match the linked headers; confirmed the exactgraph_stats()numerator/denominator (library_adapter.rs:1873-1910). FIX: none needed.distinct=0, total>0 → Some(0.0)(all-keyless snapshots) boundary was unasserted. FIX: added the assertion (mirroringprovenance_coverage(0,4)==Some(0.0)). VALIDATE: unit tests re-run green; fullcargo test --lib cognitive_memory::= 166 passed, 0 failed.(4) CI. Local equivalents of the CI gates all green:
cargo fmt --all --check,cargo clippy --release --no-deps -- -D warnings(pre-commit), Rust-only gate +race-subset
cargo test(pre-push), plus targeted unit/integration/module/docssuites above. CI on the PR is the authoritative check — this PR is a draft
until CI is confirmed 100% green.
(6) Focused diff. 5 files, additive only, no unrelated edits (see
git diff --stat).Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com