feat(observability): WS-6a + WS-6c — D1's attribution stamp exists as a substrate

[vjvarada]

WS-6 — Observability wiring + attribution. First dispatchable slice, after #319's remediation cleared the docs gate.

Why only a+c

The re-audit narrowed WS-6a–e down to a + c. The substrate is clear; the three consumers are not, and are HELD:

• WS-6b — its security amendment names no workable mechanism. bind_run_context has exactly one call site (executor.py), contextvars don't cross the HTTP hop into v1_compat, and agent_run rows are written at the run boundary, so a mid-run join finds nothing. The one mechanism the code actually supports at request time is the presence key cc:activity:live:{run_id}, which for the orchestrator path carries a server-established user. §7 must name that (or another) before this dispatches.
• WS-6e — has no token source (build_run_trace_row is pure over events + folded), so it sequences after WS-6b rather than independently.
• WS-6d — additionally waits on the retention/PII answer (Q3), which §7 itself calls a schema decision that is expensive to undo.

What shipped

• instance joins the run context. Resolved once in run_agent_stream via a second additive bind after load_agent — deliberately not by moving the first. The early bind is what correlates a failure during the load; moving it trades five fields for one.
• _emit_usage carries the full four-tuple with zero call-site changes. It arrives by inheritance through activity._INHERIT, which already carried four of the five fields — so this extends an existing mechanism rather than rebuilding it. No production caller changed.
• Shared agents produce an absent key, never '' — double-guarded and pinned, because '' and NULL are different values in the table WS-6d will build.
• refresh_run_presence() patches cc:activity:live:{run_id} after the late bind. Without it, the presence key is written from the start event — which predates the bind — so /observability/active and /roster could never carry instance for any run. Chose a presence patch over re-publishing start: a snapshot overwrite is invisible and idempotent, a duplicate stream entry is permanent and reads as a second activation.
• Interim by_instance cost dimension on the Redis rollup, so the substrate answers a per-partition question before WS-6d lands.
• Drift gate pinning set(_INHERIT) == set(_RUN_CONTEXT_KEYS) — the invariant was prose-only, and the two tuples must extend together or attribution silently half-lands.

Nothing durable is written. The stamp reaches logs and the Redis feed only.

Verification

pytest test_observability + test_activity_bus + test_llm_usage_telemetry
       + test_v1_compat_telemetry + test_instance_wiring   → 89 passed (+17 new)
ruff check . --select F821,F601,F602,F502,F7,B006          → All checks passed!
xenon --max-absolute F --max-modules F --max-average B      → exit 0

cost_summary's complexity went down (22 → 20) while gaining a dimension — the identical source/agent/instance arms collapsed into one branch. I ran the gates myself rather than accepting them as reported.

Reviewer findings

ws-verifier PASSED (re-deriving acceptance for four instancing modes, and probing request-path safety with no run context, no event loop, in a bare thread, and with four malformed responses — no exception, response unmutated). diff-reviewer then returned 2×P1 + 2×P2, no P0; all four were fixed in one repair round, including the presence-key gap above, which is why that fix is in this PR at all.

Two asymmetries recorded rather than papered over:

1. The phase="start" event predates the bind. The presence patch fixes the live view; a start↔end join still must not assume both ends carry the field.
2. A delegated sub-run inherits the caller's partition while its blobs key to ''. So instance is not a foreign key onto agent_blob.instance — WS-6d is warned explicitly. Fixing it needs unbind_contextvars + restore around the sub-run, plus a product call on whether a delegated shared agent should be billed to the caller's tenant at all.

Known coverage gap, disclosed

The single junction line _bind_run_instance(_agent_instance, run_id) inside run_agent_stream is covered by no executing test — test_run_agent_stream_e2e.py hangs on this box. Both halves either side of it are tested, and the verifier reproduced the composition independently for personal / team / shared / cron and confirmed by reading that the bind is unconditional, correctly ordered, exception-proof, and ahead of every model call. Stating it rather than letting the green count imply otherwise.

Deferred

Giving delegated sub-runs their own partition; run_agent's missing run boundary (it resolves an instance but binds no context — a pre-existing gap, and adding one would newly correlate that path's logs); surfacing by_instance in the Control Plane cost view; everything in WS-6b/6d/6e and the owner-gated WS-6f–i.

🤖 Generated with Claude Code