Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
0fbb6ac
docs(brainstorm): telemetry layer coherence for milestone 14
claude Aug 13, 2026
cfd119a
docs(brainstorm): back every per-tool claim with a read source
claude Aug 13, 2026
df1caa8
docs(brainstorm): close the Cursor and Codex verification gaps
blafourcade Aug 14, 2026
a02c5f1
docs(framework): design how a work item links intention, delivery and…
blafourcade Aug 14, 2026
eb996da
docs(framework): drop the fields the backlog already owns
blafourcade Aug 14, 2026
a20c41a
docs(framework): prove the id join on Codex and Copilot without spend…
blafourcade Aug 14, 2026
959d614
docs(framework): record Cursor as a scope question, not a pending check
blafourcade Aug 14, 2026
0a62f21
docs(framework): cursor honours hooks from the command line, measured
blafourcade Aug 14, 2026
78f464f
docs(framework): opencode bundles the otel api but exports nothing
blafourcade Aug 14, 2026
6ab01da
docs(framework): opencode does export, and the join works there
blafourcade Aug 14, 2026
27a76a9
docs(framework): plan a telemetry v1 that ships in a week
blafourcade Aug 14, 2026
bf04869
docs(framework): keeping a local and a remote backlog coherent means …
blafourcade Aug 14, 2026
cda2ccb
docs(framework): per-step cost cannot come from the metrics
blafourcade Aug 14, 2026
fe41cd2
docs(framework): the plan holds sequence, the issues hold content
aidd-bot[bot] Aug 15, 2026
eec6dd4
docs(framework): the join is an exact event correlation, and the sink…
aidd-bot[bot] Aug 15, 2026
a2d437c
docs(brainstorm): plan the run journal on two measured contracts
aidd-bot[bot] Aug 16, 2026
78d0570
docs(brainstorm): vendor_field names the export attribute, not the ho…
aidd-bot[bot] Aug 16, 2026
9448d46
docs(brainstorm): split the run journal into six buildable phases
aidd-bot[bot] Aug 17, 2026
4fed0ee
feat(framework): journal every session so work can be tied to its cost
Aug 18, 2026
8b38d51
fix(docs): the FAQ promised no telemetry while the framework ships it
Aug 18, 2026
d0b35bc
docs(cli): plan turning the provider export on
Aug 18, 2026
117eb92
docs(cli): replan telemetry activation around a switch AIDD owns
Aug 18, 2026
6e469b1
docs(cli): the telemetry switch belongs in .aidd/, not in aidd_docs/
Aug 18, 2026
4779ed1
docs(brainstorm): the end-to-end picture, and what a paid probe settled
Aug 18, 2026
ac70913
docs(brainstorm): nesting is a reporting question, not a recording one
Aug 18, 2026
9c2bfb9
docs(brainstorm): per-step measurement is portable, on five tools mea…
Aug 18, 2026
ced8b80
feat(cli): one AIDD switch the whole framework obeys, and Claude Code…
Aug 18, 2026
2eb4b5a
fix(cli): keep a plugin's hook subdirectories when installing it
Aug 18, 2026
b1bb07a
refactor(cli): name responsibilities instead of helper buckets
Aug 18, 2026
481d67d
build: let the pre-commit hook repair kanban's dependencies again
Aug 18, 2026
13fc4d4
feat(cli): name tools the way their vendors write them, and drop the …
Aug 19, 2026
dbae247
docs(cli): drop the e2e results snapshot
Aug 19, 2026
038384f
refactor(cli): drop an import left dead by the capability rewiring
Aug 19, 2026
e4172dd
refactor(framework): the run journal records facts, not conclusions
Aug 19, 2026
07b5a75
feat(cli): a sink that keeps what a session exported, and nothing else
Aug 19, 2026
ff14acf
build(cli): enforce the rules that were only ever written down
Aug 19, 2026
2208ef2
docs(cli): say the rule, not the reasoning behind it
Aug 20, 2026
cf00253
refactor(cli): name the shared plugin catalog for what it is
Aug 20, 2026
bad878e
feat(cli): let the tool declare how its telemetry config merges
Aug 20, 2026
4c7d186
docs(telemetry): keep the comments that cost money to establish
Aug 20, 2026
be19d88
feat(cli): the sink keeps the number that orders a session
Aug 20, 2026
968e586
feat(telemetry): the journal records which step was running, on four …
Aug 20, 2026
733b0aa
docs(telemetry): the contract and plan #663 was built from
Aug 20, 2026
b58ab16
feat(cli): read what a session cost from the files the tool already w…
Aug 20, 2026
2771dab
docs(telemetry): the contract and plan the local read was built from
Aug 20, 2026
ec64f2f
feat(cli): a stored record names the tool that produced it
Aug 21, 2026
db3c9bd
docs(telemetry): the contract and plan the metrics work is built from
Aug 21, 2026
58d48a3
feat(cli): a record carries the step that was running, and how it knows
Aug 21, 2026
69d7701
docs(telemetry): the contract a service outside this repository consumes
Aug 21, 2026
40195ac
feat(cli): what a period cost, and what each figure is worth
Aug 21, 2026
2eabdda
feat(framework): the plugin measures on its own, with no CLI installed
Aug 21, 2026
8cfce03
fix(framework): a file written through the shell still reaches its task
Aug 21, 2026
bef31ee
docs(framework): what measurement does, what it cannot, and where it …
Aug 21, 2026
38c1dde
refactor(framework): what the plugin ships can be read
Aug 21, 2026
ae40fb5
docs(telemetry): the plan the Copilot journal is built from, and one …
Aug 21, 2026
63e7c8d
refactor(framework): the plugin's measurement is written, not generated
Aug 21, 2026
b5770dd
feat(cli): the variable a tool expands lives on the tool, not in a bu…
Aug 22, 2026
5216305
fix(cli): a tool that runs hooks receives them, and one that does not…
Aug 22, 2026
3577af8
fix(framework): the journal recognises the payload Copilot actually s…
Aug 22, 2026
5a23cf7
feat(framework): a skill that says whether the chain is actually reco…
Aug 22, 2026
525633c
fix(framework): the turn-end walk counts what it scanned and says wha…
Aug 22, 2026
481919c
feat(framework): every tool journals, each proven by a session that ran
Aug 22, 2026
1e1f327
docs(framework): what each tool can measure, and what it cannot
Aug 22, 2026
5c6f154
chore(repo): the tests a change can break run in seconds, not minutes
Aug 22, 2026
7366b74
feat(framework): a stored record names the project it came from
Aug 22, 2026
bff02c9
feat(framework): a period breaks down by day and by project
Aug 22, 2026
584a317
feat(framework): the question chooses the axis, and hands back an art…
Aug 22, 2026
0ac42b6
fix(cli): a skill's entry file keeps the transform its tool asked for
Aug 22, 2026
2867585
test(cli): a temporary repository is its own, even when git spawned t…
Aug 22, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
7 changes: 7 additions & 0 deletions .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,13 @@
"description": "ALPHA, not ready for use. UI and UX concern: design, review, and improve frontend interfaces.",
"strict": true,
"recommended": false
},
{
"name": "aidd-telemetry",
"source": "./plugins/aidd-telemetry",
"description": "Measurement: journals every session so a unit of work can be tied to what it cost. Ships hooks only, and carries no measurement itself.",
"strict": true,
"recommended": false
}
]
}
16 changes: 16 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,22 @@ coverage/
.claude/settings.local.json
.claude/worktrees/

# AIDD CLI's own local state (install manifest, auth): machine-local, never
# part of the project's own tracked content. config.json is the exception:
# it is the committed telemetry switch (see .aidd/config.json and
# aidd_docs/tasks/2026_08/2026_08_20_telemetry-export-enable/phase-1.md),
# tracked so a fresh clone inherits the project's decision.
.aidd/*
!.aidd/config.json

# AIDD run-journal records: where they land once .aidd/config.json turns
# telemetry on (see plugins/aidd-telemetry/hooks/journal.js) - this
# directory being committed is a location, not a permission. The records it
# holds never are.
aidd_docs/runs/*
!aidd_docs/runs/.gitkeep
!aidd_docs/runs/README.md

# SpecStory captures (may contain transcripts / secrets)
.specstory/

Expand Down
1 change: 1 addition & 0 deletions .release-please-manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,6 @@
"plugins/aidd-orchestrator": "2.2.1",
"plugins/aidd-refine": "3.0.0",
"plugins/aidd-ui": "0.2.1-alpha.0",
"plugins/aidd-telemetry": "0.1.0",
"cli": "5.2.1"
}
16 changes: 12 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ _(Already tested on `Legacy` codebases)_
[![Made in France](https://img.shields.io/badge/made%20in-France-0055A4?labelColor=EF4135)](https://www.ai-driven-dev.fr/)

<p>
<!--counts:start--><kbd>7 plugins</kbd> · <kbd>47 skills</kbd> · <kbd>2 agents</kbd><!--counts:end--> · <kbd>MIT</kbd>
<!--counts:start--><kbd>8 plugins</kbd> · <kbd>50 skills</kbd> · <kbd>2 agents</kbd><!--counts:end--> · <kbd>MIT</kbd>
</p>

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
Expand Down Expand Up @@ -55,7 +55,7 @@ Why not just write your own commands? → [FAQ](docs/FAQ.md#-why-aidd-instead-of

### Claude Code

Installs the 6 stable plugins (`aidd-ui` is 🚧 alpha, install separately — see [Plugins](#-plugins)).
Installs the 6 stable plugins (`aidd-ui` and `aidd-telemetry` are 🚧 alpha, install separately — see [Plugins](#-plugins)).

**In the session** (slash commands)

Expand Down Expand Up @@ -211,7 +211,7 @@ flowchart TD

## 🧩 Plugins

Seven plugins covering the whole SDLC — **install all of them**; they work together. (`aidd-ui` is 🚧 **alpha**, off the curated path.)
Eight plugins covering the whole SDLC — **install all of them**; they work together. (`aidd-ui` and `aidd-telemetry` are 🚧 **alpha**, off the curated path.)

<table>
<tr>
Expand Down Expand Up @@ -282,7 +282,15 @@ Synchronous feature flow, async issue-to-PR automation, and product backlog.
UI / UX design — smoke-test only, not ready for use.

</td>
<td width="33%" valign="top"></td>
<td width="33%" valign="top">

### 📈 [aidd-telemetry](plugins/aidd-telemetry/README.md) 🚧

`3 skills` · **alpha**

Answers what a piece of work cost — tokens, models, and which skill spent them. Off unless you turn it on, and nothing leaves your machine.

</td>
<td width="33%" valign="top"></td>
</tr>
</table>
Expand Down
228 changes: 228 additions & 0 deletions aidd_docs/brainstorm/2026_08_13-telemetry-layer.md

Large diffs are not rendered by default.

6 changes: 5 additions & 1 deletion aidd_docs/memory/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,16 @@

## Testing Strategy

- No unit test runner configured at framework level
- The CLI runs vitest in three projects: `unit`, `integration`, `e2e` (`cli/`, ~2,600 tests)
- The plugins' own scripts run under `node --test`, in `scripts/__tests__/`, reaching their subject by path rather than by import
- Skills are validated by running each action's `## Test` end-to-end against a real environment
- Framework correctness validated by running actual skills against a real project (integration)

## Test Execution Process

- **While working, run `pnpm test:changed`** — it runs only the specs a change can break: vitest resolves the CLI's import graph, and the plugin specs are selected by the paths their own text names. Minutes become seconds, and nothing that could break is skipped
- Before declaring work done, run the full suites: `cd cli && pnpm test:unit && pnpm test:integration && pnpm test:e2e`, plus `node --test "scripts/__tests__/*.test.js"`
- Run biome through `rtk proxy` (`rtk proxy npx biome check src/ tests/`): the plain call's output is filtered and reports "no issues" while errors are pending
- Each action declares a `## Test` (a command to run, an artifact check, or an observable side-effect) that must pass before the next action runs
- `scripts/build-dist-verification.md` documents how to verify the build output

Expand Down
181 changes: 181 additions & 0 deletions aidd_docs/product/cost-report-contract.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
# Cost report contract

**Read this if you are writing a skill, or anything else that reports on AIDD work.**
It describes what `aidd telemetry report --json` prints: one object, the same shape
whatever tool did the work, carrying both the figures and a statement of what each tool
could and could not supply.

> If instead you are building a **pricing service or an aggregator** that consumes stored
> records directly, read [`metrics-contract.md`](./metrics-contract.md) — the contract for
> one stored line. The two are deliberately different audiences, and picking the wrong one
> is expensive: the record contract makes you responsible for the two double-count rules,
> the split between the two record kinds, and re-read deduplication. This one has already
> applied all three.

**Never reconstruct these figures from stored records.** One computation in one place is
the whole point: two ways of computing a number is how they start disagreeing.

## Getting the object

```bash
aidd telemetry report --json
aidd telemetry report --from 2026-08-01 --to 2026-08-31 --json
aidd telemetry report --task 2026_08/2026_08_21_cost-reporter --json
```

Prints one JSON object on stdout and exits `0`, including when the period holds nothing.
A period that is not a period — `--from notaday`, `--days 0` — exits `1` naming the flag.

## Determinism

**The same files and the same absolute period produce byte-identical output.** That holds
across repeated calls and across the order records happen to sit in on disk, which differs
between machines because a re-read appends.

It does **not** hold for `--days`, which resolves against today. `--days` is the human
shorthand; anything that stores or compares a figure should ask for `--from` and `--to`.
The object always reports the period **as it resolved**, absolutely, never as it was asked
for — so a figure taken from a `--days` call can still be cited by the days it covered.

## Versioning

Every object carries `cost_report_version`, currently `2` — bumped from `1` when `by_day`
and `by_project` joined `by_step`, `by_model` and `by_tool` as top-level breakdowns.

**Set aside an object whose version you do not recognise rather than guessing its shape.**
The number is bumped when a consumer that understood the previous shape would misread this
one. Adding a field you may ignore is not a bump; changing what an existing field means is.

## The shape

```jsonc
{
"cost_report_version": 2,
"period": { "from_day": "2026-07-01", "to_day": "2026-07-31" },
"task": "2026_08/2026_08_21_cost-reporter", // absent unless --task was given
"sessions": 1,
"totals": { "requests": 2, "input_tokens": 13930, "output_tokens": 4377, "cache_read_tokens": 165632, "cache_creation_tokens": 0 },
"active_time_s": 2820, // absent when no record carried it
"by_step": [{ "step": "aidd-dev:02-implement", "attribution": "journal-interval", "totals": {} }],
"by_model": [{ "model": "gpt-5.6-sol", "totals": {} }],
"by_tool": [{ "tool": "codex", "coverage": "covered", "reason": "…", "capability": {}, "totals": {} }],
"by_project": [{ "project": "acme/widgets", "totals": {} }], // a row with no `project` names none known
"by_day": [{ "day": "2026-07-01", "totals": {} }], // every day in the period, in order, gaps included
"attribution": [{ "attribution": "tool-stated", "totals": {} }],
"read": { "undated_records": 0, "unreadable_lines": 0 }
}
```

### Totals

The same object appears as `totals` everywhere — at the top level and on every row.

| Field | Meaning |
| --- | --- |
| `requests` | Billed requests. Always present. |
| `cost_micro_usd` | Whole micro-dollars. Divide by 1,000,000 for dollars, at the moment of display and not before. |
| `input_tokens`, `output_tokens`, `cache_read_tokens`, `cache_creation_tokens` | The four counters, disjoint — adding all four gives total tokens without counting anything twice. |

**An absent counter means never observed, which is not zero.** A tool whose files carry no
amount has an *unknown* cost, not a free one. Print "unknown", never `$0.00`.

**No amount reaches this object from a local read, on any tool.** Claude Code's `cost_usd`
exists only on its OTLP export. If you are reporting on locally-read sessions, you are
reporting tokens; the rates that turn them into money live outside this repository.

### Breakdowns

`by_step`, `by_model`, `by_tool` and `by_project` are ordered largest first, with a stable
tie-break, so the biggest thing is the first thing you read. `by_day` is the one exception:
it is chronological, one row per day the period spans — a series read out of order is not
a series, and a day nothing ran on is a row of zeros rather than an omitted day.

**Every breakdown sums exactly back to `totals`.** That is asserted, on integers, not
hoped for.

`by_step` is keyed by the step **and** the strength of its attribution: one skill reached
once from the tool's own statement and once from a journal interval is two rows, because
they are two different claims. A row with no `step` carries `attribution: "unattributed"`.

`by_project` carries a row with no `project` for a record stored before this field existed,
or whose session journal named none — never folded into a project the reader happens to be
standing in. A record's project comes from the run journal that covered its session, not
from wherever the report itself happens to run.

### Attribution

`attribution` always has exactly three rows, in this order:

| `attribution` | Means |
| --- | --- |
| `tool-stated` | The tool named the running skill itself, on the line with the counters. Exact. |
| `journal-interval` | Derived from the interval between two boundaries the framework recorded. An inference. |
| `unattributed` | Neither source could say. |

A strength that accounts for nothing is present with `requests: 0`. That zero is a
measurement — the total is known and none of it came from that source.

**`unattributed` does not mean no step ran.** On at least one measured tool the two are
indistinguishable, so the stronger reading would be a fact nobody measured. Do not collapse
it into anything else, and do not call it a residual.

### Capability, per tool

This is the field that makes the contract the same across tools. **Branch on it. Never
infer a tool's limits from whether a number happened to be present** — a tool that cannot
supply an amount and a session that cost nothing look identical in the numbers.

```jsonc
"capability": {
"local_read": { "token_counters": true, "amount": false, "tool_stated_step": false },
"export": { "token_counters": false, "amount": false, "tool_stated_step": false },
"journal_attributable": true,
"task_attributable": false
}
```

| Field | Meaning |
| --- | --- |
| `local_read`, `export` | What that route was **measured** to supply. `null` means the tool declares no such route at all, which is not the same as a declared route supplying nothing. |
| `token_counters` | That route yields the four counters. |
| `amount` | That route yields a figure denominated in currency. Never a credit or a premium request. |
| `tool_stated_step` | The tool names the running step itself. A journal interval is not this. |
| `journal_attributable` | The run journal names this tool's sessions. **False means two things:** no step can come from an interval, *and* a read that sweeps the journal never reaches one of its sessions — so the tool can be perfectly readable and still report nothing until someone names a session by hand. |
| `task_attributable` | This tool's writes can be traced to the task they landed in. |

`coverage` is `"covered"` or `"not-covered"`, and `reason` says why when it is the second,
or what a covered tool's figures cannot be used for.

**Four silences, and only one is a zero.** A tool with `requests: 0` may be: not covered at
all (`coverage: "not-covered"`, read `reason`), covered but unreachable by the sweep
(`journal_attributable: false`), covered and reached and idle (a real zero), or covered and
its reader failed (the human output says so; `aidd telemetry read` reports it per tool).

### What the read could not do

```jsonc
"read": { "undated_records": 3, "unreadable_lines": 2 }
```

`undated_records` are records carrying no moment at all. They belong to **no** period —
the only other moment available is the day the line was stored, which is when AIDD heard
about the work rather than when it happened. `unreadable_lines` are lines no parser could
read.

**Both non-zero means your total is partial.** Say so rather than presenting it as whole.

## Filling it

Records reach storage when someone runs:

```bash
aidd telemetry read # every session the run journal knows
aidd telemetry read --session <id>
```

A period that reports nothing usually means its sessions have not been read yet.

## Known limits

[`docs/telemetry-limits.md`](../../docs/telemetry-limits.md) states what each tool can and
cannot be measured for, and why. Read it before explaining a missing figure.
Loading
Loading