Merge pull request #78 from codellm-devkit/fix/issue-46-bolt-symbol-labels

feat!: schema v2 — CanNode CPG, L3 dataflow, TS-prefixed graph vocabulary (1.0.0, graph schema 2.0.0)

Author: rahlk

.github/workflows/ci.yml

@@ -0,0 +1,30 @@
+name: CI
+
+on:
+  pull_request:
+    branches: [main]
+  push:
+    branches: [main]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+
+      - name: Set up Bun
+        uses: oven-sh/setup-bun@v2
+
+      - name: Install dependencies
+        run: bun install --frozen-lockfile
+
+      - name: Typecheck
+        run: bun run typecheck
+
+      - name: Unit + conformance tests
+        run: bun test
+
+      # Container suite is opt-in; ubuntu-latest ships Docker, so testcontainers works out of the box.
+      - name: Neo4j bolt container tests
+        run: RUN_CONTAINER_TESTS=1 bun test test/neo4j-bolt.test.ts

CLAUDE.md

@@ -6,12 +6,46 @@ Agent guidance for `codellm-devkit/codeanalyzer-typescript` (`cants`).
 
 `cants` is a TypeScript/JavaScript static analyzer built on the TypeScript compiler
 (via [ts-morph](https://ts-morph.com/)). It is the CLDK TypeScript backend: it emits
-the canonical CLDK `analysis.json` — a **symbol table** plus a **resolver-based call
-graph** — and can project that same analysis into a **Neo4j** property graph. It
-mirrors its [Python](https://github.com/codellm-devkit/codeanalyzer-python) and
+the **canonical schema v2** — one additive Code Property Graph — in **two projections**,
+`analysis.json` and a **Neo4j** property graph. It mirrors its
+[Python](https://github.com/codellm-devkit/codeanalyzer-python) and
 [Java](https://github.com/codellm-devkit/codeanalyzer-java) sibling analyzers, so
 output-shape parity with them is a first-class concern.
 
+## Schema v2 — the additive CPG (read this before touching output)
+
+The output is **one scale-free structure**: a containment tree of nodes (id / kind /
+`span` / children) with **typed edge overlays** (a CPG). Every classic artifact — symbol
+table, call graph, CFG, PDG, SDG — is a *projection* of that one structure, and analysis
+**levels** are how deeply it is populated (each level only ever *adds*, never rewrites):
+
+- **L1** (`-a 1`): the tree to callable depth — `application → symbol_table{module} →
+  types{}/functions{}/fields{} → callables{}` — plus `call` nodes in each callable's
+  `body{}` (with `callee` unresolved). `source` is stored once per module; every node's
+  text slices off it via `span.bytes`.
+- **L2** (`-a 2`): the `call_graph` edge list (callable→callable) at the application scope,
+  and the `callee` slot on each call node refined `null → id` (the one sanctioned mutation).
+- **L3** (`-a 3`): the rest of `body{}` (statements + `@entry`/`@exit`) and the intra-callable
+  edge lists `cfg`/`cdg`/`ddg` (reaching-definitions, `prov:["reaching-defs"]`) hung on each callable.
+- **L4** (`-a 4`): the synthetic `@formal_in:N`/`@formal_out`/`<L>/actual_in:N`/`<L>/actual_out`
+  vertices, the intra-caller `summary` edges, and the application-scope `param_in`/`param_out`
+  lists (the interprocedural SDG).
+
+**Identity is two-tier**: durable `can://<lang>/<app>/<file>/<type>/<sig>` ids at callable
+depth and above; ordinal `<callable-id>@<line>:<col>` (or `@<tag>`) below. Intra-callable
+edge lists use **bare local ids**; cross-callable lists use **fully-qualified `can://…@local`**
+ids. `L1 ⊆ L2 ⊆ L3 ⊆ L4` is a CI-checkable monotonicity gate (`test/schema-v2.test.ts`). The
+model + every decision live in `.claude/SCHEMA_DECISIONS.md` (§ "Schema v2 migration") and the
+skillset's `canonical-schema.md`.
+
+**Provider/client boundary:** the analyzer is a *pure graph provider* — it emits the graph
+substrate (CFG/PDG/SDG + `summary` edges) and stops. Slicing and taint are reachability
+*queries* over it and belong to the frontend SDK; never add a `taint_flows` section here.
+
+The v1→v2 emitter is a pure transform in **`src/schema/v2/`** (`emit.ts` reshapes the v1
+in-memory model, `dataflow.ts` maps `program_graphs` into the tree) — the parse/resolve/
+dataflow *compute* is untouched; only serialization is v2.
+
 The call graph defaults to the **union** of two backends: the TS compiler's resolver
 and the embedded [Jelly](https://github.com/cs-au-dk/jelly) flow analyzer (which
 recovers higher-order/callback edges the resolver misses). Merged edges keep a
@@ -29,14 +63,23 @@ it first; everything else is a stage it calls, in order:
    JSDoc, with precise source spans.
 3. **call graph** (`src/semantic_analysis`) — `selectProvider()` picks tsc / jelly /
    union; each provider returns edges + external (phantom) symbols.
-4. **cache** (`src/utils/cache.ts`) — content-hash cache under `.codeanalyzer/`, so
-   re-analysis only touches what changed.
-5. **output** (`src/build`, `src/build/neo4j`) — `analysis.json`, a self-contained
-   `graph.cypher` snapshot, or an incremental Bolt push to a live database.
-
-The shape of everything is the **schema** in `src/schema` (`TSApplication` is the top
-type). The Neo4j schema is versioned and enforced by a conformance test — treat it as
-a contract.
+4. **program graphs** (`src/dataflow`) — levels 3–4 (`-a 3`/`-a 4`): CFG → post-dominance/CDG →
+   access-path def-use → PDG → SCC-condensed bottom-up summaries → SDG. This is the *compute*;
+   it produces the internal `program_graphs` model, which `src/schema/v2/dataflow.ts` then maps
+   **into the v2 tree** (`body{}` + `cfg`/`cdg`/`ddg`/`summary` per callable + `param_in`/
+   `param_out`). Decisions: `.claude/SCHEMA_DECISIONS.md`; contract + staged follow-ups: issue #2.
+5. **cache** (`src/utils/cache.ts`) — content-hash cache under `.codeanalyzer/`, so
+   re-analysis only touches what changed (levels 3–4 also record summaries +
+   dependency edges in `graphs_summaries.json`).
+6. **output** (`src/schema/v2`, `src/build/neo4j`) — `src/schema/v2/emit.ts` reshapes the v1
+   compute model into the schema-v2 `analysis.json`; `src/build/neo4j` projects the *same* v2
+   tree into a `graph.cypher` snapshot or an incremental Bolt push. `--emit neo4j` is always
+   **full-depth** (levels gate the JSON path only; combining `-a`/`--graphs` with it is an error).
+
+The **output** shape is schema v2 (`src/schema/v2/model.ts`, `V2Application` the top type); the
+types in `src/schema` (`TSApplication`) are the *internal compute model* the emitter transforms.
+The Neo4j schema (`src/build/neo4j/schema.ts`, v2.0.0) is versioned and enforced by a conformance
+test — treat both as contracts and keep them in lockstep with the JSON.
 
 ## Directory map
 
@@ -47,10 +90,12 @@ a contract.
 | `src/options` | Parsed CLI options / `AnalysisOptions` |
 | `src/syntactic_analysis` | Symbol table (ts-morph traversal) |
 | `src/semantic_analysis` | Call-graph providers (tsc, jelly, union), phantoms |
-| `src/schema` | `TSApplication` types + signatures (the output contract) |
-| `src/build` | Dep materialization + output; `build/neo4j` = graph projection |
-| `src/utils` | fs, caching, logging, serialization, version |
-| `test` | Bun tests + `fixtures/sample-app` |
+| `src/dataflow` | L3/L4 program-graph **compute**: CFG, dominance/CDG, def-use, summaries, SDG |
+| `src/schema` | `TSApplication` — the internal compute model + `signatureOf` + `program_graphs` |
+| `src/schema/v2` | **the schema-v2 emitter**: `model.ts` (target shape) + `emit.ts` (tree/L1/L2) + `dataflow.ts` (L3/L4) |
+| `src/build` | Dep materialization; `build/neo4j` = the v2 graph projection (project/rows/cypher/bolt/schema) |
+| `src/utils` | fs, caching, logging, serialization (`serialize.ts` → `toV2`), version |
+| `test` | Bun tests + `fixtures/sample-app` + `fixtures/dataflow-app`; `schema-v2.test.ts` = the L1–L4 gates |
 
 ## Commands
 

README.md

@@ -4,7 +4,7 @@
 
 # codeanalyzer-typescript (`cants`)
 
-**A TypeScript/JavaScript static-analysis toolkit — the CLDK backend that emits a canonical symbol table and call graph, as `analysis.json` or a Neo4j property graph.**
+**A TypeScript/JavaScript static-analysis toolkit — the CLDK backend that emits the canonical schema-v2 Code Property Graph (symbol table → call graph → intraprocedural dataflow → interprocedural SDG), as `analysis.json` or a Neo4j property graph.**
 
 [![PyPI](https://img.shields.io/pypi/v/codeanalyzer-typescript?style=for-the-badge&logo=pypi&logoColor=white)](https://pypi.org/project/codeanalyzer-typescript/)
 [![Python](https://img.shields.io/pypi/pyversions/codeanalyzer-typescript?style=for-the-badge&logo=python&logoColor=white)](https://pypi.org/project/codeanalyzer-typescript/)
@@ -16,9 +16,10 @@
 ---
 
 `cants` is a static analyzer for TypeScript/JavaScript built on the TypeScript compiler (via
-[ts-morph](https://ts-morph.com/)). It produces the canonical CodeLLM-DevKit (CLDK)
-`analysis.json` — a symbol table plus a resolver-based call graph — and can project that same
-analysis into a **Neo4j property graph**. It is the TypeScript backend behind
+[ts-morph](https://ts-morph.com/)). It produces the canonical CodeLLM-DevKit (CLDK) **schema v2**
+— one additive Code Property Graph, built up level by level (symbol table → call graph →
+intraprocedural dataflow → interprocedural SDG) — as `analysis.json` and can project that same
+structure into a **Neo4j property graph**. It is the TypeScript backend behind
 [CLDK](https://github.com/codellm-devkit/python-sdk), mirroring its
 [Python](https://github.com/codellm-devkit/codeanalyzer-python) and
 [Java](https://github.com/codellm-devkit/codeanalyzer-java) siblings.
@@ -135,8 +136,8 @@ written to `analysis.json` (or `graph.cypher` for `--emit neo4j`) in that direct
 ```text
 Usage: cants [options]
 
-CLDK TypeScript analyzer — emits the canonical analysis.json (symbol table +
-resolver call graph), or a Neo4j graph.
+CLDK TypeScript analyzer — emits the canonical schema-v2 CPG (symbol table →
+call graph → dataflow → SDG) as analysis.json, or a Neo4j graph.
 
 Options:
   -i, --input <path>             project root to analyze (not required for
@@ -157,9 +158,19 @@ Options:
                                  visible in shell history / process list)
                                  (default: "neo4j", env: NEO4J_PASSWORD)
   --neo4j-database <db>          Neo4j database name (env: NEO4J_DATABASE)
-  -a, --analysis-level <n>       analysis depth: 1 = symbol table + tsc resolver
-                                 call graph + RTA (default); 2 = call graph
-                                 (default: "1")
+  -a, --analysis-level <n>       analysis depth: 1 = symbol table (default); 2 =
+                                 + resolver call graph; 3 = + intraprocedural
+                                 dataflow (cfg/cdg/ddg); 4 = + interprocedural
+                                 SDG (param_in/param_out/summary) (default: "1")
+  --graphs <list>                dataflow sections to emit, comma-separated: cfg
+                                 | dfg | pdg (require -a 3) | sdg (requires -a
+                                 4); default: all rungs at or below the level
+  --graph-field-depth <k>        access-path depth bound (k-limit) for level-3
+                                 dataflow (default: "3")
+  -j, --jobs <n>                 worker parallelism for level-3 graphs (default:
+                                 sequential; opt in with N ≥ 2 on large projects
+                                 — each worker loads its own copy of the
+                                 program)
   -t, --target-files <paths...>  restrict analysis to specific files
                                  (incremental)
   --skip-tests                   skip test trees (default)
@@ -213,32 +224,96 @@ Options:
    cants --input ./my-ts-project --eager --cache-dir /path/to/custom-cache
    ```
 
+6. **Program graphs (level 3): CFG/PDG/SDG in `analysis.json`:**
+   ```sh
+   cants --input ./my-ts-project -a 3                    # full program_graphs section
+   cants --input ./my-ts-project -a 3 --graphs cfg,pdg   # scope the emitted graphs
+   ```
+
 ## Output targets
 
 `cants` builds one analysis in memory and can emit it three ways (`--emit`):
 
 ### `analysis.json` (default)
 
-A `TSApplication` document — the canonical CLDK contract the Python SDK parses:
+The **canonical schema v2** — one additive Code Property Graph: a containment tree of nodes
+(`id` / `kind` / `span` / children) with typed edge overlays. Analysis **levels** populate it more
+deeply; each level only ever *adds*.
 
 ```jsonc
 {
-  "symbol_table":     { /* file path → module (classes, interfaces, enums,
-                           type aliases, functions, namespaces, variables, …) */ },
-  "call_graph":       [ /* CALL_DEP edges: { source, target, type, weight,
-                           provenance, tags } keyed by callable signature */ ],
-  "external_symbols": { /* phantom stubs for call targets outside the project */ }
+  "schema_version": "2.0.0", "language": "typescript", "max_level": 4, "k_limit": 3,
+  "application": {
+    "id": "can://typescript/<app>", "kind": "application",
+    "symbol_table": {                     // L1: the tree, keyed by file path
+      "<file>": { "kind": "module", "source": "…",
+        "types":     { /* class | interface | enum | type_alias | namespace nodes */ },
+        "functions": { /* callable nodes: { id, kind, span, body{}, cfg[], cdg[], ddg[], summary[] } */ },
+        "fields":    { /* module-level bindings */ } } },
+    "call_graph": [ /* L2: { src, dst, prov, weight } — callable → callable, can:// ids */ ],
+    "param_in":   [ /* L4: actual_in → formal_in, fully-qualified can://…@local ids */ ],
+    "param_out":  [ /* L4: formal_out → actual_out */ ]
+  }
 }
 ```
 
-Caller- and callee-side identifiers come from a single signature canonicalizer, so call-graph
-`source`/`target` values byte-match the corresponding `symbol_table` / `external_symbols` keys.
+Each callable's `body{}` is keyed by local id (`line:col`, or `@entry`/`@formal_in:N`/… for
+synthetic vertices); intra-callable edge lists (`cfg`/`cdg`/`ddg`/`summary`) use those bare local
+ids, cross-callable lists use fully-qualified `can://…@local` ids. A single signature canonicalizer
+underlies every `can://` id, so call edges, dataflow edges, and tree nodes all join. The full model
+is `.claude/SCHEMA_DECISIONS.md` (§ "Schema v2 migration") and the CLDK `canonical-schema.md`.
+
+### Dataflow (`-a 3` intraprocedural, `-a 4` interprocedural)
+
+Native dependence graphs, built in-process from the same ts-morph AST (no external engine), grown
+**into the tree** (not a separate section):
+
+- **`-a 3`** completes each callable's `body{}` with statement nodes and hangs the intra-callable
+  edge lists `cfg` (exceptional control flow), `cdg` (control dependence), and `ddg` (data
+  dependence via reaching-definitions, `prov:["reaching-defs"]`) on the callable.
+- **`-a 4`** adds the synthetic `@formal_in:N` / `@formal_out` / `<L>/actual_in:N` / `<L>/actual_out`
+  vertices, the intra-caller `summary` edges, and the application-scope `param_in` / `param_out`
+  lists — the whole-program System Dependence Graph.
+
+`-a 3` implies `-a 2`; `-a 4` implies `-a 3`. `--graphs cfg,dfg,pdg,sdg` scopes which rungs emit
+(`cfg`/`dfg`/`pdg` require `-a 3`, `sdg` requires `-a 4`). `L1 ⊆ L2 ⊆ L3 ⊆ L4` is a monotonicity
+gate. Every node is addressed by its `can://…@local` id, so dataflow edges, call edges, and tree
+nodes all join.
+
+**Substrate (locked in [issue #2](https://github.com/codellm-devkit/codeanalyzer-typescript/issues/2)):**
+the CFG and reaching-definitions are hand-built from the ts-morph AST; the call-graph oracle is
+the existing provenance-merged tsc ∪ Jelly graph; aliasing is a flow-insensitive copy-alias MVP
+(Jelly points-to-backed propagation is a staged upgrade). Function summaries are composed
+bottom-up over the SCC condensation of the call graph, with k-limited access paths; module
+globals ride the SDG as extra parameters. The analysis is deliberately sound-leaning and
+over-approximate; known unsoundness (dynamic `eval`, reflection/monkey-patching, npm-internal
+effects) is recorded in `.claude/SCHEMA_DECISIONS.md`. The analyzer is a **pure graph provider**:
+it emits the dependence-graph substrate (CFG/PDG/SDG + `summary` edges) and stops — backward
+slicing and taint are reachability *queries* over the SDG that live in the frontend SDK, not here.
+
+**Parallelism (`-j/--jobs`).** The pipeline implements the level-3 parallel execution model:
+stage-1–4 extraction fans out per callable over a Bun worker pool (partitioned by file) and is
+posted *before* the call-graph solve so the two overlap; summary composition runs as a
+Kahn-style ready-queue wavefront over the SCC condensation DAG (the SCC is the atomic unit).
+`--jobs N` output is **byte-identical** to `--jobs 1` (node ids are span-ordered, all edge lists
+are collect-then-sorted, and the SCC fixpoint is a pure function of its inputs) — enforced by a
+differential test. It is off by default and worth opting into only on large codebases: ts-morph
+ASTs cannot cross the worker boundary, so each extraction worker loads its own copy of the
+program, which dominates the parallelizable graph math on small/mid repos (self-analysis runs
+2.5× slower at `-j 14`). Worker failure at any stage degrades to the sequential path with a
+warning — never to wrong or missing output.
+
+Levels 1/2 are unaffected: nothing in level 3 runs unless `-a 3` is requested.
 
 ### Neo4j graph
 
-`--emit neo4j` projects the same analysis into a labeled property graph (declarations keyed by
-their signature under a shared `:Symbol` label; calls, imports, inheritance, decorators, and call
-sites as relationships):
+`--emit neo4j` projects the **same v2 tree** into a labeled property graph: every node keyed by its
+`can://` id under a shared `:CanNode` merge label (+ a `TS`-prefixed specific kind label, e.g.
+`:TSModule`, `:TSCallable`), containment as `TS_HAS_MODULE`/`TS_DECLARES`/`TS_HAS_METHOD`/
+`TS_HAS_FIELD`/`TS_HAS_BODY_NODE` edges, and the overlays (`TS_CALLS`, `TS_CFG_NEXT`, `TS_CDG`,
+`TS_DDG`, `TS_SUMMARY`, `TS_PARAM_IN`, `TS_PARAM_OUT`) as typed relationships. The graph is
+**always full-depth** — analysis levels gate the JSON path only, so combining `-a`/`--graphs` with
+`--emit neo4j` is an error:
 
 - **Without `--neo4j-uri`** — writes a self-contained `graph.cypher` (constraints + indexes, a
   scoped wipe, then batched `MERGE`s). Load it with `cypher-shell < graph.cypher`.

package.json

@@ -1,15 +1,15 @@
 {
   "name": "codeanalyzer-typescript",
-  "version": "0.5.0",
-  "description": "CLDK TypeScript analyzer — emits the canonical CLDK analysis.json (symbol table + resolver-based call graph) via ts-morph.",
+  "version": "1.0.0",
+  "description": "CLDK TypeScript analyzer — emits the canonical schema v2 (additive CPG: symbol table → call graph → intraprocedural dataflow → interprocedural SDG) as analysis.json and Neo4j, via ts-morph.",
   "type": "module",
   "module": "src/index.ts",
   "bin": {
     "cants": "dist/cants"
   },
   "scripts": {
     "start": "bun run src/index.ts",
-    "build": "bun build ./src/main.ts --compile --external @babel/preset-typescript --outfile dist/cants",
+    "build": "bun build ./src/main.ts ./src/dataflow/worker.ts --compile --external @babel/preset-typescript --outfile dist/cants",
     "gen:schema": "bun run src/index.ts --emit schema > schema.neo4j.json",
     "gen:readme": "bun run scripts/update-readme.ts",
     "test:container": "RUN_CONTAINER_TESTS=1 bun test test/neo4j-bolt.test.ts",