Open-source AI gateway for local agent clients: publish fusion-style multi-model workflows as OpenAI/Claude-compatible virtual models with traces, tokens, latency, and cost visibility.

MemTrace: Tracing and Attributing Errors in Large Language Model Memory Systems

OpenAI Agents Tracing | Open-Source Tracing Dashboard including Costs & Usages

Evaluation of Multi-Agent Systems on Cloud Run with the GenAI Client in Vertex AI SDK

Local-first agent debugger with replay, failure memory, smart highlights, and drift detection.

Substructure is an engine for building durable, long-running AI agents using only a stateless HTTP endpoint hosted on your infrastructure, in your code.

Comprehensive agent analytics suite for AI agents built with the Google Agent Development Kit (ADK) , LangChain or other popular frameworks, powered by BigQuery Agent Analytics plugin and Grafana

Open Reflection Protocol — Turn agent failures into regression tests, reusable lessons, and measurable improvements. Built on OpenTelemetry.

Local replay debugger for Browser Use failures with screenshots, model I/O, failed-step timelines, and public-safe HTML exports.

Claude Code session capture and analysis. Trace tool calls, detect backtracks, analyze decisions, track edit chains, and measure plan drift. Local-first, zero-dependency observability.

Shared context substrate for AI agents. Retrieval that learns what's useful. Runs local or cloud.

Observability and governance SDK for AI Agents

ai agent trace, Open-source runtime tracing and diagnostics for AI agent execution flows

AI coding agent observability and causal tracing for Claude Code, Codex CLI, OpenCode, and Python workflows

Homebrew tap for AgentTap - network-level AI agent observability

Capture and analyze Claude Code sessions locally to track every tool call, decision, and reasoning step without external dependencies.

A local LLM proxy tool that intercepts and visualizes all OpenAI API calls, providing a real-time web monitoring dashboard.

Agent evaluation framework: run LLM agents against datasets, capture execution traces, score with rubric-based LLM judges, and view regressions in a web dashboard. Local-first, no external infra required.

Open-source evaluation framework for AI agents. Define test suites with rubrics, run your agent, get LLM-as-judge scores against criteria, inspect full execution traces, and diff runs to catch behavioral regressions.

Causal failure attribution + Cladder-style benchmark + A/B testing harness with power analysis, CUPED, propensity matching, and DiD