Skip to content

feat(runtime): ExecCtx.onSandboxEvent per-event tee for live output#378

Draft
Tjemmmic wants to merge 1 commit into
mainfrom
feat/exec-ctx-on-sandbox-event-tee
Draft

feat(runtime): ExecCtx.onSandboxEvent per-event tee for live output#378
Tjemmmic wants to merge 1 commit into
mainfrom
feat/exec-ctx-on-sandbox-event-tee

Conversation

@Tjemmmic

Copy link
Copy Markdown
Contributor

Overview

The loop kernel buffers every SandboxEvent from each iteration's streamPrompt into a private array and only exposes iteration-boundary trace events (with a ~280-char output preview). A host that wants to stream the agent's live, token-by-token output has no seam to observe the raw event stream the kernel is already iterating.

What changed

  • Added an optional onSandboxEvent?(event, { iterationIndex, agentRunName }) to ExecCtx.
  • The run-loop collect loop calls it for each event as it arrives, before buffering, wrapped in a try/catch so a throwing observer can never break the run's own event collection.

Compatibility

Additive and opt-in: when onSandboxEvent is unset (every existing caller) behavior is byte-identical. No change to buffering, parsing, cost/token extraction, or the existing traceEmitter.

Testing

src/runtime/run-loop.ts and src/runtime/types.ts typecheck clean and the ESM build succeeds. Note: this repo's full pnpm typecheck / .d.ts build currently fails on pre-existing unrelated issues (@tangle-network/agent-interface not installed, @tangle-network/agent-eval export drift) — not introduced by this change. A unit test asserting the tee fires per event is a sensible follow-up.

@Tjemmmic

Copy link
Copy Markdown
Contributor Author

🤖 AI Code Review (ensemble)

Summary

The PR adds an optional onSandboxEvent callback to ExecCtx that tees raw sandbox stream events to a host observer. The approach is purely additive and lightweight, but the implementation fails to actually isolate async observer failures (leading to potential unhandled promise rejections) and the significant new hot-loop behavior lacks tests.

Issues Found

2 total — 0 P1 (blocking) · 2 P2 (should fix) · 0 P3 (nice to have)

⚠️ P2 — Async onSandboxEvent failures can become unhandled promise rejections

  • File: src/runtime/run-loop.ts:715-723
  • Problem: The try/catch only isolates synchronous throws. The callback type returns void, but TypeScript allows callers to pass an async function where a void callback is expected. If that observer rejects, the surrounding try/catch will not catch it because the promise is neither awaited nor given a .catch, causing an unhandled promise rejection that violates the isolation guarantee.
  • Fix: Capture the return value and defensively handle thenables. For example: const result = args.ctx.onSandboxEvent(...); if (result && typeof (result as PromiseLike<void>).then === 'function') { void (result as PromiseLike<void>).then(undefined, () => {}); }. Update the type to void | PromiseLike<void> if async observers are intentionally supported.

⚠️ P2 — Missing tests for new onSandboxEvent stream tee contract

  • File: src/runtime/run-loop.ts:712-724
  • Problem: Significant new behavior was added to the hot stream loop in executeIteration (callback dispatch, try/catch isolation, metadata construction) but there are no tests verifying the contract. Without tests, a future refactor could silently break the tee or reintroduce the risk of observer exceptions crashing the run.
  • Fix: Add tests asserting: (1) every streamed event is forwarded to onSandboxEvent with the expected iterationIndex and agentRunName; (2) a throwing observer is silently caught and does not break event collection; and (3) rejected async observers do not fail the run.

❌ CHANGES REQUESTED

The async rejection handling issue is a concrete runtime bug that violates the PR's own stated isolation guarantee. Combined with the complete lack of test coverage for significant new hot-loop behavior, the PR needs another pass before merge.

Quick Reference

  • P2: Async onSandboxEvent failures can become unhandled promise rejections
  • P2: Missing tests for new onSandboxEvent stream tee contract

Synthesized by Sokuza AI from multiple independent reviewers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant