Skip to content

feat(framework): the metrics contract a pricing service consumes #687

Description

@blafourcade

Outcome

A consumer outside this repository — the governor, or anything else — receives a session's metrics complete enough to price and attribute them, without knowing how any tool writes its files.

The gap, measured

Run today against a real 5134-record session, a stored record carries:

{"kind":"request","vendor_id":"2c21d903-…","vendor_field":"sessionId",
 "turn_id":"req_011Ce4ccjP8oZQNsyQ6iKpQi","turn_field":"requestId",
 "model":"claude-opus-5","effort":"high","agent_name":"general-purpose",
 "input_tokens":2,"output_tokens":1,"cache_read_tokens":12127,
 "cache_creation_tokens":19232,"event_timestamp":"",
 "sink_schema_version":2,"provenance":"local-read"}

Good enough to price. Two things are missing, and both are needed by anyone aggregating across a team.

Which tool produced it. There is no tool field. The only clue is vendor_field, which reads sessionId for a locally-read Claude Code record, session.id for an exported one, session_meta.id for Codex — a leaky proxy that encodes the route as much as the tool, and that a consumer would have to reverse-engineer. The run journal has carried tool since its first line; the sink never did, because when it was written there was only one tool.

Which step was running. step_start lives in the run journal and is never joined into the stored record. So the figures know the agent (agent_name separates aidd-dev:executor from the main loop — measured, 2543 calls against 1611 on one session) but not the skill. That join is the entire point of #663, and nothing performs it yet.

Scope

  • Includes: the tool on every stored record, as a fact rather than something inferred from a field name.
  • Includes: joining the run journal's step boundaries onto the cost records they cover, so a shipped figure carries the step it belongs to.
  • Includes: a stated contract for what a consumer receives — the field set, what each field means, which are always present and which are conditional, and what a missing one signifies.
  • Includes: the two double-count rules stated in the contract rather than left for a consumer to rediscover — kind: "request" and kind: "session" measure the same quantities differently and are never summed; a re-read is matched on turn_id.
  • Excludes: pricing. The rates live in the governor.
  • Excludes: transport. Getting the payload out is feat(framework): upload out of band, never at the session's expense #662, and redacting it on the way is feat(framework): redact again on the upload path #655.
  • Excludes: presenting anything to a human, which is feat(framework): a skill that reports cost per task and per step #629.

Done When

  • Every stored record names the tool that produced it, without a consumer parsing vendor_field.
  • A record produced during a step carries that step, joined from the run journal rather than inferred from a timestamp.
  • The contract is written down, and a consumer implementing against it needs nothing from this repository's source.
  • A field that is conditional says under what condition, and what its absence means — an absent counter and a zero counter are different facts.
  • The contract states the two double-count rules explicitly.
  • A tool whose figures cannot be attributed to a step says so, rather than presenting an unattributed figure as attributed.

Why this is the prerequisite

Anything downstream — pricing, per-person aggregation, per-epic reporting — reads this contract. Building those on a shape that cannot name its own tool means every consumer re-derives it, differently, and one of them gets it wrong.

Relations

Field Value
parent #631
related #629, #662, #655, #663, #685

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    Status
    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions