Skip to content

feat(cli): Copilot is readable at session granularity, and reads as not covered #697

Description

@blafourcade

As someone measuring a team that uses Copilot
I want its sessions to report their tokens
So that a tool that consumed 21,000 tokens stops appearing beside the ones that consumed nothing

Context

Copilot is declared telemetryLocalRead: { kind: "unsupported" }, on the ground that its file carries outputTokens per turn and nothing per request. That was measured, and it was measured on the wrong file.

A live session on 1.0.80 handed a Stop hook a transcript_path nothing here had read:

~/.copilot/session-state/<session-id>/events.jsonl

What that file holds

Its session.shutdown event carries all four counters:

{"type":"session.shutdown","data":{
  "totalPremiumRequests": 0.33,
  "totalNanoAiu": 2655750000,
  "tokenDetails": {
    "input":       {"tokenCount": 10},
    "cache_read":  {"tokenCount": 0},
    "cache_write": {"tokenCount": 21070},
    "output":      {"tokenCount": 42}
  }
}}

assistant.message carries model and turnId; session.auto_mode_resolved records which model the router chose.

And per-turn figures genuinely do not exist. assistant.turn_end carries {"turnId":"0"} and nothing else, so the original claim was right about what a turn can say — only about the wrong file.

totalPremiumRequests: 0.33 for one request independently confirms the denomination measured across fourteen sessions: premium requests, not currency, and never read as an amount.

Expected

Copilot reports a session's tokens, as one kind: "session" record. The stored shape already has that kind, and already says a "session" line is not a per-request figure.

Actual

not covered, which reads to a user as a tool the framework cannot see at all — while the numbers sit in a file the host hands us the path to.

Acceptance

  • A Copilot session yields one kind: "session" record carrying the four counters, from session.shutdown.
  • Its capability.local_read says token_counters: true, amount: false, tool_stated_step: false — measured, not assumed.
  • totalPremiumRequests is never stored as cost_usd. It is a count times a per-model multiplier, invariant to consumption.
  • A report says plainly that Copilot's figure is a session total, not a sum of requests — the contract's two-kinds rule already forbids adding it to request lines, and this is the first tool to produce only the second kind.
  • The reader is driven by a captured events.jsonl, redacted like every other fixture here: that file holds the system prompt and the assistant's own messages, and none of it may enter a fixture.

Out of scope

Per-step and per-request attribution for Copilot. Neither exists in the file. The journal would give it a step once #681 lands, and the figure would still be a session total.

Relations

Field Value
parent #631
related #681, #653

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    Status
    Ideation

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions