Wire arm-discovery (Aerial+) → self-directed reasoning: the endgame's ingestion leg#757
Conversation
…-directed reasoning The endgame's INGESTION leg: read a graph's premises OUT OF DATA with no LLM and reason to a conclusion the data only implied. - DISCOVER: the REAL Aerial+ codebook probe (extract_rules, float-free, deterministic) mines 4 is_a premises from 200 co-occurrence rows (socrates/plato->philosopher, philosopher->human, human->mortal; each f=1.00 c=0.98 earned from 50 co-occurrences). - ROUTE: CandidateTriple::from_rule → NARS-truth SPO is_a edges (a custom FeedProjector names items back to entities; filtered to the s->o direction). - REASON: the self-directed transitive-deduction loop (self_directed_graph.rs shape, std-only here) derives `socrates ... mortal` (c=0.94) at a fixed point. Nothing hand-asserted — premises MINED, conclusion REASONED. Two mining mechanics surfaced + handled: extract_rules picks the single nearest consequent per feature (a multi-parent entity drops below the confidence floor → use a single-parent tree, let the reasoner chain); the miner emits both f0->f1 and reversed f1->f0 rules → filter to is_a direction. std-only, zero-dep, default features (honors the crate's isolation); runs via cargo run --manifest-path crates/lance-graph-arm-discovery/Cargo.toml --example discovery_to_reasoning. 4 KILL-gates green. Board: E-ARM-DISCOVERY-REASONING-BRIDGE-1 (the bridge + the RNN-vs-ring-buffer- vs-Markov-NARS memory-model comparison: "the NARS revision arc IS the Markov trajectory"; DeepNSM real-text grounding named as the follow-up). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_1a54af2a-794d-408b-bd2b-2a712bb0591b) |
📝 WalkthroughWalkthroughThis change adds three executable research probes: frequency-based semantic distance, role-based homograph collapse, and association-rule-to-NARS reasoning. Board notes record their measurements, validation commands, boundaries, and outcomes. ChangesCosine-free ingestion probes
Association-rule reasoning bridge
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant CooccurrenceData
participant extract_rules
participant FactProjection
participant NarsReasoning
CooccurrenceData->>extract_rules: mine forward association rules
extract_rules->>FactProjection: provide feature0 -> feature1 rules
FactProjection->>NarsReasoning: create is_a facts
NarsReasoning->>NarsReasoning: derive transitive beliefs to fixed point
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…se operator A surface word form is a SUPERPOSITION of its (lemma, PoS) readings; fixing the SPO slot (S/O→nominal, P→verbal) collapses it to ONE lemma → ONE lemRank (the frequency-centroid address). Same surface string, DIFFERENT centroid by role — homograph disambiguation as superposition collapse, no parser weights. Measured on word_forms.csv (10,239 surface forms, std-only, zero-dep): - 1,166 ambiguous (11.39%); 1,164 CROSS-PoS (collapsible by a role mask); 2 same-PoS residue (found=find.v/found.v — the honest case a mask can't split). - 809 verb∩noun homographs, ALL 809 (100%) collapse to distinct v/n centroids. - thought → P r53 (think.v) vs S/O r692 (thought.n); left → r152 vs r4559 (left.j adjective selected by neither mask); means → r130 vs r1392. All three KILL-gated (each side unique, the two differ). clippy-clean. The front door of the cosine-free ingestion leg: running text arrives inflected+ambiguous; word_forms resolves surface→lemma-centroid and the SPO role collapses the homograph to the one centroid the slot selects. Honest boundary: S/O=noun, P=verb is a simplification — real slot assignment is the parser's job; this shows the collapse MECHANISM, not full parsing. Board: E-SURFACE-FORM-COLLAPSE-1 prepended. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1
…eplacement (ρ=0.762) Tests the operator's claim "the frequency translates to cosine replacement centroid" — not asserted, measured. A word's per-subgenre COCA frequency vector (ln(1+PM) per genre, z-scored across vocab, Fisher-z'd) yields a pairwise distance that reproduces the DeepNSM README's NSM semantic-distance ordering at Spearman ρ=0.762 — with NO embedding and NO cosine on any learned vector. The distance is a property of WHERE the frequency lands, not a comparison. std-only, zero-dep; KILL floor ρ≥0.5; clippy-clean; Rust reproduces the Python probe exactly. This is the empirical ground under the palette256 / Morton / perturbation- shader distance: because the centroid IS the cosine, the distance immaterializes (Fisher-z baked into palette levels, not stored k×k) and amortizes over the gridlake (Morton 4×4 cascade, not O(n²) pairwise). Honest boundary: only the 8 coarse genres in-repo; the 96-subgenre palette (gitignored) is higher-resolution, so ρ=0.762 is a FLOOR not a ceiling. The proven claim is the ordering (rank correlation), not absolute-distance identity. Board: E-FREQ-IS-COSINE-REPLACEMENT-1 prepended. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1
… agents) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1
The agents hand-wrote freq_is_cosine.rs + homograph_collapse.rs without running rustfmt; CI's deepnsm format check (style.yml) flagged them. cargo fmt applied, behaviour unchanged (freq_is_cosine still rho=0.762; homograph_collapse unchanged). No other deepnsm files touched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_489ad75c-38a2-4a31-8b7b-a9b4e6c64eef) |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 804af31bae
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let edges: Vec<(Item, Item, u32)> = pairs | ||
| .iter() | ||
| .map(|&(s, o)| (Item::new(0, s), Item::new(1, o), 1)) | ||
| .collect(); | ||
| let oracle = TopKDistance::new(spec, u32::MAX, &edges); |
There was a problem hiding this comment.
In this example, the TopKDistance oracle is built directly from the same pairs array that encodes the desired is_a premises, so the codebook probe is preloaded with the exact subject→object links before extract_rules runs. With theta: 2, any relation present in the rows but absent from these hard-coded edges is pruned before support/confidence can confirm it, which makes the kill gates unable to support the stated “premises out of data / nothing hand-asserted” claim; derive the oracle independently from the data or make the example/gate explicit that the oracle already supplies the target topology.
Useful? React with 👍 / 👎.
| println!( | ||
| " ambiguous (>1 (lemma,PoS) sense) : {ambiguous} ({pct:.2}% of surfaces)" | ||
| ); | ||
| println!(" ├─ CROSS-PoS (senses span ≥2 PoS letters) : {cross_pos} ← collapsible by a role mask"); |
There was a problem hiding this comment.
With the bundled word_forms.csv, this prints 1,164 cross-PoS surfaces as “collapsible by a role mask”, but the implemented SPO mask only distinguishes nominal n from verbal v, and the code's actual verb∩noun count is 809. Cross-PoS forms outside that n/v split are not shown to collapse to distinct S/O vs P centroids, so the census overstates the measured mechanism; compute this line from the collapse checks or relabel it as merely cross-PoS.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (3)
crates/deepnsm/examples/freq_is_cosine.rs (1)
92-169: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winNo unit tests for the pure math helpers.
zscore_in_place,cosine,fisher_z_distance,ranks, andspearman_rhoare all pure, easily-testable functions but have no#[cfg(test)]coverage (e.g. a known cosine/Fisher-z pair, a tie-handling case forranks, a known Spearman rho for a small fixed vector pair). As per coding guidelines, "Add Rust unit tests alongside implementations via#[cfg(test)]modules; prefer focused scenarios over broad integration tests."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/deepnsm/examples/freq_is_cosine.rs` around lines 92 - 169, Add a #[cfg(test)] module alongside the pure helpers zscore_in_place, cosine, fisher_z_distance, ranks, and spearman_rho. Cover focused cases including z-score normalization, a known cosine/Fisher-z pair, rank ties with average ranks, and a small fixed-vector Spearman correlation with its expected value.Source: Coding guidelines
crates/deepnsm/examples/homograph_collapse.rs (1)
76-113: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winNo unit tests for
collapse/parse_forms.Both are small, pure, and easy to test (e.g. a synthetic multi-sense bucket to exercise
Collapse::Unique/Ambiguous/None, and a hand-written CSV snippet forparse_forms's dedup/skip-malformed-row paths) but ship with no#[cfg(test)]coverage. As per coding guidelines, "Add Rust unit tests alongside implementations via#[cfg(test)]modules; prefer focused scenarios over broad integration tests."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/deepnsm/examples/homograph_collapse.rs` around lines 76 - 113, Add a nearby #[cfg(test)] module covering collapse with synthetic senses that exercises Collapse::Unique, Collapse::Ambiguous, and Collapse::None, and covering parse_forms with a hand-written CSV that verifies identical-sense deduplication and skipping malformed rows. Keep the tests focused on these pure functions and their existing behavior.Source: Coding guidelines
crates/lance-graph-arm-discovery/examples/discovery_to_reasoning.rs (1)
71-73: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd
#[cfg(test)]coverage for the pure reasoning helpers.
concludedand the transitive-deduction math in the fixed-point loop (lines 147-187) are pure and easily testable, but the only validation is the runtime kill-gate at the end ofmain. As per coding guidelines,crates/**/*.rsshould "Add Rust unit tests alongside implementations via#[cfg(test)]modules; prefer focused scenarios over broad integration tests." A couple of focused tests (e.g.concludedtrue/false cases, one deduction step producing the expectedf/c) would guard thef = f1*f2, c = f1*f2*c1*c2formula against silent regressions.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/lance-graph-arm-discovery/examples/discovery_to_reasoning.rs` around lines 71 - 73, 1. Add a #[cfg(test)] module alongside concluded containing focused unit tests for concluded’s matching and non-matching cases. 2. Extract or expose the pure transitive-deduction calculation from the fixed-point loop in main into a testable helper, then add a test verifying one deduction step produces the expected f and c values using f = f1*f2 and c = f1*f2*c1*c2. 3. Keep the tests narrow and preserve the existing runtime reasoning behavior.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@crates/deepnsm/examples/freq_is_cosine.rs`:
- Around line 92-169: Add a #[cfg(test)] module alongside the pure helpers
zscore_in_place, cosine, fisher_z_distance, ranks, and spearman_rho. Cover
focused cases including z-score normalization, a known cosine/Fisher-z pair,
rank ties with average ranks, and a small fixed-vector Spearman correlation with
its expected value.
In `@crates/deepnsm/examples/homograph_collapse.rs`:
- Around line 76-113: Add a nearby #[cfg(test)] module covering collapse with
synthetic senses that exercises Collapse::Unique, Collapse::Ambiguous, and
Collapse::None, and covering parse_forms with a hand-written CSV that verifies
identical-sense deduplication and skipping malformed rows. Keep the tests
focused on these pure functions and their existing behavior.
In `@crates/lance-graph-arm-discovery/examples/discovery_to_reasoning.rs`:
- Around line 71-73: 1. Add a #[cfg(test)] module alongside concluded containing
focused unit tests for concluded’s matching and non-matching cases. 2. Extract
or expose the pure transitive-deduction calculation from the fixed-point loop in
main into a testable helper, then add a test verifying one deduction step
produces the expected f and c values using f = f1*f2 and c = f1*f2*c1*c2. 3.
Keep the tests narrow and preserve the existing runtime reasoning behavior.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 5b0d75ba-e491-4487-b4ec-304e4ce7c139
📒 Files selected for processing (5)
.claude/board/AGENT_LOG.md.claude/board/EPIPHANIES.mdcrates/deepnsm/examples/freq_is_cosine.rscrates/deepnsm/examples/homograph_collapse.rscrates/lance-graph-arm-discovery/examples/discovery_to_reasoning.rs
2 participants
What
crates/lance-graph-arm-discovery/examples/discovery_to_reasoning.rs— wires the two halves the workspace had kept apart: read a graph's premises out of data with no LLM (Aerial+ association-rule discovery), then reason to a conclusion the data only implied. Additive: one std-only example + one board entry, no core change. The dep arrow points away from arm-discovery (it stays zero-dep), so its isolation is untouched.The bridge (measured, no LLM, deterministic)
extract_rules, float-free, deterministic) mines 4is_apremises from 200 co-occurrence rows:socrates/plato → philosopher,philosopher → human,human → mortal(eachf=1.00 c=0.98, earned from 50 co-occurrences).CandidateTriple::from_rule+ a customFeedProjector→ NARS-truth SPOis_aedges (filtered to the subject→object direction).self_directed_graph.rsshape, std-only here) derivessocrates … mortal(c=0.94) at a fixed point.Nothing hand-asserted — the premises were mined, the conclusion reasoned. 4 KILL-gates green.
Two mining mechanics surfaced + handled
extract_rulespicks the single codebook-nearest consequent per feature, so a multi-parent entity (socratesis_a philosopher and greek) yields 0.5 rule-confidence and drops below the floor → use a single-parent tree and let the reasoner do the chaining single-hop mining can't.f0→f1and reversedf1→f0rules → the router filters to theis_adirection.The memory-model comparison (RNN vs ring buffer vs Markov-NARS)
Surfaced by the review question, and it resolves cleanly:
RatificationQueue, 1024)I-SUBSTRATE-MARKOVstates "the NARS revision arc is the Markov trajectory" — so arm-discovery's cross-window "ice-caking" (accreting each rule's truth by NARS revision) and Markov context-building are the same mechanism. The workspace's bet: replace the RNN's opaque learned recurrence with explicit, inspectable, deterministic NARS-revised Markov accretion.Honest boundary + next step
streaming-arm-nars-discovery-v1), not yet wired.TripletGraphloop (kept zero-dep to honor the crate's isolation).Board (same commit)
EPIPHANIES.md— E-ARM-DISCOVERY-REASONING-BRIDGE-1 (the bridge + the memory-model finding).🤖 Generated with Claude Code
https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1
Generated by Claude Code
Summary by CodeRabbit
New Features
Documentation