Skip to content

trace_path format=json drops include_evidence strategy and confidence #1542

Description

@asemraza

Version

codebase-memory-mcp 0.10.0

Platform

macOS (Apple Silicon / arm64)

Install channel

Installed release binary at ~/.local/bin/codebase-memory-mcp

Binary variant

Standard

What happened, and what did you expect?

trace_path(include_evidence=true) correctly returns the public strategy class and confidence in the default tree format, but silently drops both fields when format:"json" is also requested.

The installed tool schema and cli trace_path --help promise that include_evidence adds two columns per hop in addition to name and hop. The JSON response instead contains only:

"cols": ["name", "hop"]

Expected JSON columns:

"cols": ["name", "hop", "strategy", "confidence"]

This affects both the one-shot CLI and MCP because they share the same trace response implementation.

Related to #1355: current provenance support works for default tree output, but the JSON response path still omits it.

Synthetic reproduction

Create repro.php:

<?php

final class EvidenceTarget
{
    public static function answer(): int
    {
        return 42;
    }
}

final class EvidenceCaller
{
    public function run(): int
    {
        return EvidenceTarget::answer();
    }
}

Index it:

codebase-memory-mcp cli --json index_repository \
  --repo-path /path/to/fixture \
  --mode full \
  --name cbm-evidence-json-repro

The fresh fixture index contained 8 nodes, 10 edges, 0 skipped files, and 0 partial parses.

Control: default tree format works

codebase-memory-mcp cli --json trace_path \
  --project cbm-evidence-json-repro \
  --function-name cbm-evidence-json-repro.repro.EvidenceCaller.run \
  --direction outbound \
  --depth 1 \
  --include-evidence true \
  --limit 20

Actual payload:

callees: 1  (rows: name hop strategy confidence; qn = group prefix + "." + name)
cbm-evidence-json-repro.repro.EvidenceTarget:
  answer 1 language_rule 0.95

Failing case: JSON format drops evidence

codebase-memory-mcp cli --json trace_path \
  --project cbm-evidence-json-repro \
  --function-name cbm-evidence-json-repro.repro.EvidenceCaller.run \
  --direction outbound \
  --depth 1 \
  --include-evidence true \
  --format json \
  --limit 20

Actual structured payload:

{
  "function": "cbm-evidence-json-repro.repro.EvidenceCaller.run",
  "direction": "outbound",
  "callees_total": 1,
  "callees": {
    "cols": ["name", "hop"],
    "groups": [
      {
        "qn_prefix": "cbm-evidence-json-repro.repro.EvidenceTarget",
        "rows": [["answer", 1]]
      }
    ]
  }
}

The graph edge contains the evidence

codebase-memory-mcp cli --json query_graph \
  --project cbm-evidence-json-repro \
  --query 'MATCH (a)-[r:CALLS]->(b)
           WHERE a.qualified_name = "cbm-evidence-json-repro.repro.EvidenceCaller.run"
           RETURN b.qualified_name, r.strategy, r.confidence, r.candidates, r.line
           LIMIT 10'

Actual row:

cbm-evidence-json-repro.repro.EvidenceTarget.answer
php_static_resolved  "0.95"  "1"  "15"

Exact code cause in v0.10.0

The handler correctly reads include_evidence:

https://github.com/DeusData/codebase-memory-mcp/blob/v0.10.0/src/mcp/mcp.c#L6008-L6013

The default tree branch forwards it to the evidence-aware emitter:

https://github.com/DeusData/codebase-memory-mcp/blob/v0.10.0/src/mcp/mcp.c#L6268-L6294

However, format:"json" selects the separate JSON branch, which calls bfs_to_tree_json(...) without include_evidence:

https://github.com/DeusData/codebase-memory-mcp/blob/v0.10.0/src/mcp/mcp.c#L6311-L6330

The JSON emitter signature has no evidence parameter:

https://github.com/DeusData/codebase-memory-mcp/blob/v0.10.0/src/mcp/mcp.c#L5832-L5836

So the evidence exists in the graph and traversal result, but is lost specifically during JSON response projection.

Impact

Agents requesting structured JSON receive speculative or heuristic CALLS hops without the strategy/confidence fields explicitly requested for trust decisions. The response succeeds normally, so clients cannot distinguish the contract failure from evidence-free data.

Smallest fix

  • Pass include_evidence into the JSON trace emitter.
  • Add strategy and confidence columns and values using the existing bfs_edge_evidence_for_hop() and public strategy-class mapping.
  • Add regression coverage for format:"json" plus include_evidence:true through both CLI and MCP envelopes.

Confirmations

  • I searched existing issues and did not find this JSON-emitter-specific bug.
  • The reproduction is fully synthetic and contains no proprietary source.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingparsing/qualityGraph extraction bugs, false positives, missing edges

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions