Skip to content

Commit 1eaa2bc

Browse files
committed
fix(dataflow): connect the L4 SDG port layer to the statement ddg (#115)
The SDG assembler wires the binding edges — def stmt → actual_in, actual_out → callsite, formal_in → first use, return → formal_out — into the IR's extra_edges, and the old v1 program_graphs projection emitted them; the v2 emission dropped them, leaving the port lattice an island no end-to-end flows_to walk could cross. emit_l4 now emits the DDG-typed extra edges onto each callable's ddg tagged prov=['reaching-defs'] (the label codeanalyzer-typescript ships for its port-routing edges, keeping the prov vocabulary keystone-shared), deduplicated, deterministically ordered, endpoint-guarded, and idempotent under cache reuse. CDG-typed extras stay unemitted — actual vertices already carry that containment in parent. Nested call vertices (y = f(x)) are likewise anchored: from L3 they carry parent = the enclosing statement's local id, a sanctioned null → value refinement at L2→L3 mirroring callee null → id at L1→L2. Bare-call statements share their key with the CFG node and are untouched. Conformance now admits the third L4 prov value; both decisions are recorded in .claude/SCHEMA_DECISIONS.md.
1 parent 08ab971 commit 1eaa2bc

5 files changed

Lines changed: 310 additions & 7 deletions

File tree

.claude/SCHEMA_DECISIONS.md

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -202,3 +202,31 @@ MERGE collapses legitimately-distinct edges (per-variable dependences; a
202202
conditional's true/false pair) and a live Bolt push then materializes fewer
203203
relationships than the projection produced (caught by the opt-in
204204
`test_neo4j_bolt.py` count gates).
205+
206+
## L4 graph completeness — port wiring + call anchoring (#115, 1.1.1)
207+
208+
Two connectivity gaps closed in the L4 emission (no vocabulary invented; both
209+
decisions use surface the keystone already ships):
210+
211+
1. **Statement ↔ port ddg wiring, `prov:["reaching-defs"]`.** The SDG's
212+
binding edges — `def stmt → actual_in`, `actual_out → callsite`,
213+
`formal_in → first use`, `def/return → formal_out` — existed in the IR
214+
(`fg.extra_edges`, wired by `assemble_sdg`) and were emitted by the old v1
215+
`program_graphs` projection, but the v2 emission dropped them, leaving the
216+
port lattice an island (no end-to-end `flows_to` witness could cross a
217+
call). `emit_l4` now emits them onto each callable's `ddg` tagged
218+
`prov:["reaching-defs"]` — the label codeanalyzer-typescript already ships
219+
for its port-routing edges, so the prov vocabulary stays keystone-shared:
220+
`ssa` (L3 syntactic) ⊂ + `points-to` (L4 alias delta) + `reaching-defs`
221+
(L4 port bindings). Monotonicity: both L4 families are additive over the
222+
untouched ssa set, and every port endpoint exists only at L4.
223+
2. **Call vertices anchor via `parent`, not the CFG spine.** A call nested in
224+
a larger statement (`y = f(x)`) keeps its own `"line:col"` body key and
225+
deliberately stays OFF the cfg spine — calls are dataflow satellites of
226+
their statement, not control-flow steps. From L3 (when statements exist)
227+
such a call carries `parent` = its enclosing statement's local id — the
228+
same anchoring `actual_in`/`actual_out` vertices already use. A bare-call
229+
statement shares its key with the CFG node (no self-parent). This is a
230+
sanctioned `null → value` refinement of `BodyNode.parent` at the L2→L3
231+
boundary, mirroring the `callee: null → id` refinement at L1→L2; the
232+
superset gates compare body keys, so no gate exception was needed.

CHANGELOG.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Fixed
11+
- **L4 SDG port layer is connected to the statement ddg** (#115): the binding
12+
edges `def stmt → actual_in`, `actual_out → callsite`, `formal_in → first use`
13+
and `return → formal_out` were built by the SDG assembler but dropped by the
14+
v2 emission, leaving the interprocedural port lattice an island — no
15+
end-to-end `flows_to(def, callee_formal)` path could cross a call. They are
16+
now emitted on each callable's `ddg` with `prov:["reaching-defs"]` (the same
17+
label codeanalyzer-typescript ships for its port-routing edges). Strictly
18+
additive over the L3 `ssa` set, so `L3 ⊆ L4` monotonicity holds.
19+
- **Nested call vertices are anchored to their statement** (#115): a call inside
20+
a larger statement (`y = f(x)`) sits off the CFG spine by design (a dataflow
21+
satellite); from L3 it now carries `parent` = its enclosing statement's local
22+
id — the same anchoring `actual_in`/`actual_out` vertices use. Sanctioned
23+
`null → value` refinement at L2→L3, mirroring `callee: null → id` at L1→L2;
24+
recorded in `.claude/SCHEMA_DECISIONS.md`.
25+
1026
## [1.1.0] - 2026-07-27
1127

1228
### Changed

codeanalyzer/dataflow/builder.py

Lines changed: 58 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -283,6 +283,23 @@ def _span_of(source: str, node) -> Optional["Span"]:
283283
continue
284284
pycallable.body[local] = BodyNode(kind=node.kind, span=span)
285285

286+
# #115: anchor nested call vertices to their statement. A bare-call
287+
# statement shares its key with its CFG node (handled above); a call
288+
# nested inside a larger statement (`y = f(x)`) has its own key and
289+
# no cfg contact, so it carries `parent` = the enclosing statement's
290+
# local id — the same anchoring actual_in/actual_out vertices use.
291+
for node in pdg.cfg.nodes:
292+
if node.ast_node is None:
293+
continue
294+
stmt_local = im.local(node.id)
295+
for call in _calls_in(node.ast_node):
296+
call_key = f"{call.lineno}:{call.col_offset}"
297+
child = pycallable.body.get(call_key)
298+
if child is None or call_key == stmt_local:
299+
continue
300+
if child.kind == "call":
301+
child.parent = stmt_local
302+
286303
if want_cfg:
287304
pycallable.cfg = [
288305
CfgEdge(
@@ -437,7 +454,12 @@ def emit_l4(
437454
points-to provenance and taint are *not* emitted here (later tasks).
438455
"""
439456
from codeanalyzer.dataflow.identity import IdentityMap
440-
from codeanalyzer.schema.py_schema import BodyNode, ParamEdge, SummaryEdge
457+
from codeanalyzer.schema.py_schema import (
458+
BodyNode,
459+
DdgEdge,
460+
ParamEdge,
461+
SummaryEdge,
462+
)
441463

442464
# L4 emission is additive (it *appends* summary/param edges), so it must
443465
# first clear any L4 state a reused cache left on these live objects —
@@ -517,6 +539,41 @@ def emit_l4(
517539
)
518540
(app.param_in if e.type == "PARAM_IN" else app.param_out).append(edge)
519541

542+
# (e) statement ↔ port ddg wiring (#115). The IR's ``extra_edges`` — the
543+
# def→actual_in, actual_out→callsite, formal_in→use and def→formal_out
544+
# bindings ``assemble_sdg`` wires — are what connect the port lattice to
545+
# the statement-level ddg; without them the SDG is two disconnected
546+
# graphs and no end-to-end flows_to walk can cross a call. Emitted with
547+
# ``prov=["reaching-defs"]`` (the label codeanalyzer-typescript ships for
548+
# its port-routing ddg edges, so the vocabulary stays keystone-shared).
549+
# CDG-typed extras (callsite → actual_in containment) are skipped — the
550+
# actual vertices already carry that anchoring in ``parent``.
551+
for sig, fg in ir.functions.items():
552+
pycallable = sig_to_callable.get(sig)
553+
im = ims.get(sig)
554+
if pycallable is None or im is None:
555+
continue
556+
# Idempotency under cache reuse, mirroring the points-to delta: strip
557+
# any reaching-defs edges a prior run appended before re-emitting.
558+
pycallable.ddg = [e for e in pycallable.ddg if e.prov != ["reaching-defs"]]
559+
seen: set = set()
560+
rows = []
561+
for e in fg.extra_edges:
562+
if e.type != "DDG":
563+
continue
564+
src, dst = im.local(e.source), im.local(e.target)
565+
if src not in pycallable.body or dst not in pycallable.body:
566+
continue
567+
key = (src, dst, e.var)
568+
if key in seen:
569+
continue
570+
seen.add(key)
571+
rows.append(
572+
DdgEdge(src=src, dst=dst, var=e.var, prov=["reaching-defs"])
573+
)
574+
rows.sort(key=lambda r: (r.src, r.dst, r.var or ""))
575+
pycallable.ddg.extend(rows)
576+
520577

521578
def _ddg_local_set(im, pdg) -> Set[Tuple[str, str, Optional[str]]]:
522579
"""The DDG edges of ``pdg`` as a set of ``(local_src, local_dst, var)``.

test/conftest_v2.py

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -68,14 +68,16 @@ def assert_conformant(payload: dict, max_level: int) -> None:
6868
f"L3 ddg edge must have prov ['ssa'], got {e.get('prov')} in {c['id']}"
6969
)
7070
elif max_level >= 4:
71-
# L4 layers an alias-derived (points-to) def-use delta additively on top
72-
# of the unchanged L3 ssa edges, so every ddg edge carries exactly one of
73-
# those two provenances — and no other.
71+
# L4 layers two additive deltas on the unchanged L3 ssa edges: the
72+
# alias-derived points-to def-use delta, and the statement ↔ port
73+
# binding edges (#115) tagged reaching-defs — the keystone-shared label
74+
# codeanalyzer-typescript also emits for its port-routing edges. Every
75+
# ddg edge carries exactly one of those three provenances — no other.
7476
for mod, c in _iter_callables(app):
7577
for e in c.get("ddg", []):
76-
assert e.get("prov") in (["ssa"], ["points-to"]), (
77-
f"L4 ddg edge must have prov ['ssa'] or ['points-to'], "
78-
f"got {e.get('prov')} in {c['id']}"
78+
assert e.get("prov") in (["ssa"], ["points-to"], ["reaching-defs"]), (
79+
f"L4 ddg edge must have prov ['ssa'], ['points-to'] or "
80+
f"['reaching-defs'], got {e.get('prov')} in {c['id']}"
7981
)
8082

8183
if max_level >= 4:

test/test_v2_l4_ports.py

Lines changed: 200 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,200 @@
1+
"""Regression tests for #115: the L4 SDG port layer must be *connected* to the
2+
statement-level ddg, and call vertices must be anchored to their statement.
3+
4+
Before the fix, the interprocedural port lattice (``actual_in → formal_in``,
5+
``formal_out → actual_out`` via param_in/param_out/summary) was an island: no
6+
ddg edge touched any port, so an end-to-end ``flows_to(def, callee_formal)``
7+
walk was inexpressible. The wiring existed in the IR (``fg.extra_edges``,
8+
emitted by the old v1 ``program_graphs`` projection) but the v2 emission
9+
dropped it. The restored edges carry ``prov=["reaching-defs"]`` — the same
10+
label codeanalyzer-typescript ships for its port-routing ddg edges.
11+
"""
12+
from pathlib import Path
13+
14+
from codeanalyzer.dataflow.builder import (
15+
_base_types,
16+
build_function_pdgs,
17+
build_program_graphs,
18+
emit_ddg_pointsto_delta,
19+
emit_l3_body,
20+
emit_l4,
21+
)
22+
from codeanalyzer.dataflow.scalpel_oracle import make_alias_oracle
23+
from codeanalyzer.dataflow.syntactic import SyntacticOracle
24+
from codeanalyzer.schema import PyApplication
25+
from codeanalyzer.schema.assign_ids import assign_ids
26+
from codeanalyzer.schema.l1_body import populate_l1_body
27+
from codeanalyzer.schema.py_schema import PyCallEdge
28+
from codeanalyzer.syntactic_analysis.symbol_table_builder import SymbolTableBuilder
29+
30+
_SOURCE = """\
31+
def build(flag):
32+
result = flag + 1
33+
return result
34+
35+
36+
def main():
37+
x = 5
38+
y = build(x)
39+
z = y * 2
40+
return z
41+
"""
42+
43+
44+
def _build_l4_app(tmp_path: Path):
45+
f = tmp_path / "app.py"
46+
f.write_text(_SOURCE, encoding="utf-8")
47+
mod = SymbolTableBuilder(tmp_path, None).build_pymodule_from_file(f)
48+
app = PyApplication(symbol_table={"app.py": mod})
49+
sig_to_id = assign_ids(app, "portfix")
50+
app.call_graph = [
51+
PyCallEdge(src="app.main", dst="app.build", prov=["jedi"], weight=1)
52+
]
53+
populate_l1_body(app)
54+
syn_infos, _ = build_function_pdgs(
55+
app, k=3, oracle_factory=lambda c, fast: SyntacticOracle()
56+
)
57+
emit_l3_body(app, syn_infos, sig_to_id, graphs={"cfg", "dfg", "pdg"})
58+
ir = build_program_graphs(
59+
app, k=3,
60+
oracle_factory=lambda c, fast: make_alias_oracle(c, fast, _base_types(c)),
61+
)
62+
emit_l4(app, ir, sig_to_id)
63+
emit_ddg_pointsto_delta(app, syn_infos, ir, sig_to_id)
64+
mod = app.symbol_table["app.py"]
65+
return app, mod.functions["build"], mod.functions["main"]
66+
67+
68+
def _edges(c, prov=None):
69+
out = set()
70+
for e in c.ddg or []:
71+
if prov is None or e.prov == prov:
72+
out.add((e.src, e.dst, e.var))
73+
return out
74+
75+
76+
def test_def_stmt_flows_into_actual_in(tmp_path):
77+
"""`x = 5` (7:4) must feed the argument port of the `build(x)` callsite."""
78+
_, _, main = _build_l4_app(tmp_path)
79+
rd = _edges(main, prov=["reaching-defs"])
80+
assert any(
81+
src == "7:4" and dst.endswith("/actual_in:0") for src, dst, _ in rd
82+
), f"missing def→actual_in binding edge; reaching-defs edges: {sorted(rd)}"
83+
84+
85+
def test_actual_out_flows_back_to_callsite(tmp_path):
86+
"""The return-value port must flow back into the callsite statement."""
87+
_, _, main = _build_l4_app(tmp_path)
88+
rd = _edges(main, prov=["reaching-defs"])
89+
assert any(
90+
src.endswith("/actual_out") and dst == "8:4" for src, dst, _ in rd
91+
), f"missing actual_out→use binding edge; reaching-defs edges: {sorted(rd)}"
92+
93+
94+
def test_formal_in_flows_to_first_use(tmp_path):
95+
"""Inside the callee, the parameter port must reach its first-use stmt."""
96+
_, build, _ = _build_l4_app(tmp_path)
97+
rd = _edges(build, prov=["reaching-defs"])
98+
assert any(
99+
src == "@formal_in:0" and dst == "2:4" for src, dst, _ in rd
100+
), f"missing formal_in→use edge; reaching-defs edges: {sorted(rd)}"
101+
102+
103+
def test_return_stmt_flows_into_formal_out(tmp_path):
104+
"""`return result` (3:4) must feed the callee's formal_out port."""
105+
_, build, _ = _build_l4_app(tmp_path)
106+
rd = _edges(build, prov=["reaching-defs"])
107+
assert any(
108+
src == "3:4" and dst == "@formal_out" for src, dst, _ in rd
109+
), f"missing return→formal_out edge; reaching-defs edges: {sorted(rd)}"
110+
111+
112+
def test_port_edges_never_replace_or_retag_l3_edges(tmp_path):
113+
"""The wiring is additive: every ssa edge survives, and no reaching-defs
114+
edge duplicates an (src, dst, var) triple that ssa already carries."""
115+
_, build, main = _build_l4_app(tmp_path)
116+
for c in (build, main):
117+
ssa = _edges(c, prov=["ssa"])
118+
rd = _edges(c, prov=["reaching-defs"])
119+
assert ssa, "L3 ssa edges must still be present"
120+
assert not (ssa & rd), "reaching-defs must not duplicate ssa triples"
121+
122+
123+
def test_port_edge_endpoints_exist_in_body(tmp_path):
124+
_, build, main = _build_l4_app(tmp_path)
125+
for c in (build, main):
126+
for e in c.ddg or []:
127+
assert e.src in c.body, f"dangling ddg src {e.src}"
128+
assert e.dst in c.body, f"dangling ddg dst {e.dst}"
129+
130+
131+
def test_emission_is_idempotent_under_reemit(tmp_path):
132+
"""Re-running emit_l4 + the delta against the same live tree (cache-reuse
133+
shape) must not duplicate the reaching-defs edges."""
134+
f = tmp_path / "app.py"
135+
f.write_text(_SOURCE, encoding="utf-8")
136+
mod = SymbolTableBuilder(tmp_path, None).build_pymodule_from_file(f)
137+
app = PyApplication(symbol_table={"app.py": mod})
138+
sig_to_id = assign_ids(app, "portfix")
139+
app.call_graph = [
140+
PyCallEdge(src="app.main", dst="app.build", prov=["jedi"], weight=1)
141+
]
142+
populate_l1_body(app)
143+
syn_infos, _ = build_function_pdgs(
144+
app, k=3, oracle_factory=lambda c, fast: SyntacticOracle()
145+
)
146+
emit_l3_body(app, syn_infos, sig_to_id, graphs={"cfg", "dfg", "pdg"})
147+
ir = build_program_graphs(
148+
app, k=3,
149+
oracle_factory=lambda c, fast: make_alias_oracle(c, fast, _base_types(c)),
150+
)
151+
emit_l4(app, ir, sig_to_id)
152+
emit_ddg_pointsto_delta(app, syn_infos, ir, sig_to_id)
153+
main = app.symbol_table["app.py"].functions["main"]
154+
first = sorted((e.src, e.dst, e.var, tuple(e.prov)) for e in main.ddg)
155+
emit_l4(app, ir, sig_to_id)
156+
emit_ddg_pointsto_delta(app, syn_infos, ir, sig_to_id)
157+
main = app.symbol_table["app.py"].functions["main"]
158+
second = sorted((e.src, e.dst, e.var, tuple(e.prov)) for e in main.ddg)
159+
assert first == second, "re-emission must be idempotent"
160+
161+
162+
# ----------------------------------------------------------------------------------------------
163+
# #115 part 2: call vertices anchored to their statement via `parent`.
164+
# ----------------------------------------------------------------------------------------------
165+
166+
167+
def test_nested_call_vertex_is_parented_to_its_statement(tmp_path):
168+
"""`y = build(x)`: the call vertex (8:8) floats off the CFG spine; from L3
169+
it must carry `parent` = its enclosing statement's local (8:4)."""
170+
_, _, main = _build_l4_app(tmp_path)
171+
call_nodes = {k: n for k, n in main.body.items() if n.kind == "call"}
172+
assert call_nodes, "fixture must materialize a call vertex"
173+
for key, node in call_nodes.items():
174+
assert node.parent == "8:4", (
175+
f"call vertex {key} must be parented to its statement, "
176+
f"got parent={node.parent!r}"
177+
)
178+
179+
180+
def test_bare_call_statement_needs_no_parent(tmp_path):
181+
"""A bare call (`g(b)`) shares its key with the statement node — no
182+
self-parent is emitted."""
183+
f = tmp_path / "m.py"
184+
f.write_text(
185+
"def g(x):\n return x\n\n\ndef f(a):\n g(a)\n return a\n",
186+
encoding="utf-8",
187+
)
188+
mod = SymbolTableBuilder(tmp_path, None).build_pymodule_from_file(f)
189+
app = PyApplication(symbol_table={"m.py": mod})
190+
sig_to_id = assign_ids(app, "barefix")
191+
populate_l1_body(app)
192+
syn_infos, _ = build_function_pdgs(
193+
app, k=3, oracle_factory=lambda c, fast: SyntacticOracle()
194+
)
195+
emit_l3_body(app, syn_infos, sig_to_id, graphs={"cfg", "dfg", "pdg"})
196+
fcallable = app.symbol_table["m.py"].functions["f"]
197+
call_nodes = {k: n for k, n in fcallable.body.items() if n.kind == "call"}
198+
assert call_nodes, "fixture must materialize the bare call vertex"
199+
for key, node in call_nodes.items():
200+
assert node.parent != key, "a call must never parent to itself"

0 commit comments

Comments
 (0)