feat(mcp): add native OpenTelemetry observability

[fahreddinozcan]

Summary

• instrument individual dispatched MCP requests and notifications at the public SDK transport boundary, including operations inside valid JSON-RPC batches
• emit the development-status OpenTelemetry MCP server metrics mcp.server.operation.duration and mcp.server.session.duration, plus SERVER spans with standard MCP/GenAI/JSON-RPC/network attributes
• extract traceparent, tracestate, and baggage from params._meta per SEP-414, using the extracted context as parent and linking ambient HTTP context
• add bounded Context7 operation, tool-outcome, upstream, authentication, in-flight, and official Node runtime signals
• expose Prometheus/OpenMetrics on a dedicated HTTP-only :9464/metrics listener while supporting externally preloaded OpenTelemetry providers
• provide OTEL_SDK_DISABLED=true as a true hard-off boundary: no telemetry provider modules, transport wrappers, async scopes, spans, metrics, or runtime collectors are loaded
• document and verify the Envoy/application signal ownership boundary so HTTP and backend-proxy metrics are not duplicated

Why this integration

The MCP TypeScript SDK v2.0.0 does not ship a turnkey OpenTelemetry plugin, tracer/meter, exporter, or completed-operation middleware. It does expose SEP-414 propagation constants and the public transport boundary. Instrumenting that boundary observes protocol operations across HTTP and stdio without subclassing protected SDK internals or mistaking an HTTP envelope for a single MCP operation.

The cluster already has gateway HTTP and Kubernetes/container signals, but those cannot identify MCP methods, individual tool health, authentication outcomes, Context7 API dependency failures, or Node event-loop/heap pressure. This adds those missing bounded semantic signals without duplicating gateway request telemetry.

Envoy non-duplication audit

The checked-in production data path is Envoy Gateway -> HTTPRoute/mcp -> Service/mcp-svc -> MCP pods. Envoy is not a sidecar in the MCP pod.

• keep downstream HTTP envelope totals, status classes, latency, active requests/connections, resets, and gateway timeouts in envoy_http_*_downstream_*
• keep Envoy-to-MCP request/status/latency, pending/active work, retries, resets, connection failures, timeouts, and circuit-breaker overflow in envoy_cluster_upstream_*
• keep pod/container CPU, memory, network, and restarts in the Kubernetes monitoring stack
• the application deliberately registers no generic inbound HTTP server/request instruments
• mcp_server_operation_duration_count is a JSON-RPC operation count after SDK dispatch, not an HTTP request count: one valid batch is one Envoy request and multiple MCP operations; pre-dispatch HTTP rejections are Envoy-only
• context7_mcp_upstream_* observes MCP-to-Context7 API calls, which the ingress gateway cannot see

Signals

• mcp_server_operation_duration — standard MCP server operation histogram; _count is the operation count. Tool calls use standard gen_ai.tool.name plus bounded context7.mcp.tool.outcome (success, not_found, or error), avoiding a duplicate tool counter/histogram.
• mcp_server_session_duration — standard MCP session histogram for real stateful stdio sessions; stateless HTTP request transports are excluded
• standard MCP SERVER spans with SEP-414 parent extraction and ambient transport links
• context7_mcp_operations_active
• context7_mcp_upstream_requests_total, context7_mcp_upstream_request_duration, and context7_mcp_upstream_requests_active
• context7_mcp_authentication_attempts_total, context7_mcp_authentication_duration, and context7_mcp_authentication_active
• official OpenTelemetry Node runtime metrics: nodejs_eventloop_*, v8js_gc_duration, v8js_memory_heap_*, and v8js_resource_active
• standard OpenTelemetry target_info resource metadata

Upstream outcomes distinguish HTTP, response-decoding, network, timeout, and cancellation failures; they include both bounded status class and exact numeric HTTP status. Authentication distinguishes accepted, missing, invalid, and unexpected-error outcomes.

Labels exclude API keys, IPs, client versions, queries, library IDs, session IDs, tool arguments/results, and raw error text. Server-side JSON-RPC caller faults remain visible through rpc.response.status_code without setting error.type; internal, transport, and tool failures set bounded error types.

Operations

• HTTP: embedded Prometheus reader defaults to 0.0.0.0:9464/metrics
• stdio: no telemetry listener; the single outer transport owns session lifecycle/version/error capture across SDK modern-probe -> legacy fallback
• graceful EOF/close/SIGHUP awaits SDK close and best-effort flushes an external provider before exit
• provider flush is bounded at 4 seconds and process shutdown independently bounds it at 5 seconds
• OTEL_EXPORTER_PROMETHEUS_HOST and OTEL_EXPORTER_PROMETHEUS_PORT configure the listener
• OTEL_METRICS_EXPORTER=none disables only the embedded exporter, allowing a preloaded provider to receive signals
• OTEL_SDK_DISABLED=true entirely bypasses instrumentation and provider bootstrap
• bind/config/runtime-collector failures are logged and fail open
• an already registered global MeterProvider/TracerProvider takes precedence; external instrumentation owns Node runtime registration in that mode

The deployment repository still needs to expose port 9464 internally and add the Prometheus scrape/ServiceMonitor; it should not route this port through public MCP ingress.

Resource-overhead benchmark

Local Docker benchmark against the merge-base image, using an in-memory/stubbed upstream to make telemetry cost maximally visible: 8 concurrent MCP query-docs clients, 10-second warm-up, 30-second measurement, three repetitions, rotated mode order, and 15-second Prometheus scraping when enabled. Every measured call succeeded.

Mode vs baseline	CPU time / operation (paired median)	Idle container memory	Mean memory under load	Throughput
Current image, OTEL_SDK_DISABLED=true	-0.17% (noise; paired mean +0.78%)	+1.74 MiB	-1.18 MiB (no measurable increase)	+1.04% (noise)
Current image, telemetry enabled + scrape	+12.92%	+7.28 MiB	+22.70 MiB	-0.11% (noise)

Median absolute CPU cost was 0.546 ms/op for baseline, 0.549 ms/op disabled, and 0.639 ms/op enabled. Median throughput was 1815, 1801, and 1803 operations/second respectively. The enabled result is intentionally a worst-case CPU-saturated local test with essentially no upstream latency; production's network-bound requests should have a lower relative percentage, while the absolute memory and CPU measurements remain the useful capacity-planning bounds.

The production image grew from 115,663,904 to 118,468,819 bytes: +2,804,915 bytes (+2.43%).

Validation

• pnpm typecheck — passed
• pnpm lint:check — passed
• pnpm format:check — passed
• pnpm build — passed
• pnpm test — 9 files / 112 tests passed
• frozen-lockfile production Docker build passed
• focused coverage includes disabled-path bypass, SEP-414 propagation, server/caller error semantics, cancellation/abort/close lifecycle, one stateful session across SDK fallback, modern/legacy protocol capture, recoverable vs terminal transport errors, bounded stdio close/flush, nested Undici timeout causes, body-phase timeout/cancellation, exact upstream status, not_found, authentication timing/concurrency, runtime metrics, batch counts, active-gauge cleanup, and exporter collision
• local Envoy v1.38.0, Prometheus v3.5.0, and Grafana 12.1.0 stack is healthy
• live scrape validation confirms canonical tools/call operation series exist and removed context7_mcp_tool_calls_* duplicate series do not
• final thermo-nuclear maintainability review approved with no remaining high-confidence findings
• no production configuration, workloads, credentials, or telemetry data were mutated

References

• OpenTelemetry MCP semantic conventions
• MCP SEP-414 trace context propagation
• Envoy HTTP connection manager statistics
• Envoy upstream cluster statistics

[linear-code]

CTX7-2004

[Copilot]

Pull request overview

This PR adds first-class OpenTelemetry observability to the @upstash/context7-mcp server by instrumenting MCP operations at the SDK transport boundary (including batched JSON-RPC messages), exporting MCP server spans/metrics, and exposing Prometheus/OpenMetrics on a dedicated internal listener.

Changes:

• Add MCP operation/session instrumentation (SERVER spans + mcp.server.operation.duration / mcp.server.session.duration) with SEP-414 context extraction from params._meta.
• Add bounded application metrics for upstream Context7 API calls, tool outcomes, authentication outcomes, and Node runtime signals, plus an embedded Prometheus exporter on :9464/metrics.
• Add shutdown handling for stdio to close/flush reliably, and add tests covering lifecycle, disabled-path behavior, batching, and exporter behavior.

Reviewed changes

Copilot reviewed 19 out of 20 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
pnpm-lock.yaml	Locks new OpenTelemetry dependencies used by the MCP package.
packages/mcp/package.json	Adds OpenTelemetry API/SDK/exporter and runtime instrumentation dependencies.
packages/mcp/Dockerfile	Exposes the dedicated Prometheus metrics port (9464) in the container image.
packages/mcp/src/index.ts	Wires telemetry into HTTP + stdio serving paths, adds auth/upstream observation, and conditional embedded exporter bootstrapping.
packages/mcp/src/lib/api.ts	Wraps Context7 upstream calls with observeUpstreamRequest and adds tool outcome classification for context fetches.
packages/mcp/src/lib/types.ts	Extends ContextResponse with a bounded outcome for tool telemetry.
packages/mcp/src/lib/tool-names.ts	Centralizes tool names and defines bounded ToolCallOutcome.
packages/mcp/src/lib/telemetry.ts	Implements bounded upstream/auth metrics, tool outcome tagging, and upstream error classification.
packages/mcp/src/lib/telemetry-config.ts	Adds env-based switches for hard-off telemetry and embedded Prometheus enablement.
packages/mcp/src/lib/telemetry-provider.ts	Implements embedded Prometheus MetricReader + Node runtime metrics bootstrap with “fail open” behavior.
packages/mcp/src/lib/mcp-telemetry.ts	Adds transport-boundary MCP operation/session instrumentation and span/metric classification.
packages/mcp/src/lib/mcp-operation-scope.ts	Adds AsyncLocalStorage-based per-operation scope for tool outcome/error marking.
packages/mcp/src/lib/stdio-shutdown.ts	Adds idempotent stdio shutdown with bounded flush behavior.
packages/mcp/README.md	Documents signals, env configuration, scrape endpoint, and gateway non-duplication guidance.
packages/mcp/test/telemetry.test.ts	Unit tests for method/tool normalization, SEP-414 extraction, config flags, and span parenting/linking.
packages/mcp/test/telemetry-disabled.test.ts	Verifies OTEL_SDK_DISABLED=true yields no exported application metrics.
packages/mcp/test/stdio-shutdown.test.ts	Tests idempotent shutdown, error handling, and flush timeout behavior.
packages/mcp/test/mcp-telemetry-lifecycle.test.ts	Exercises MCP lifecycle classification, cancellations, transport errors, and session metric behavior.
packages/mcp/test/integration.test.ts	End-to-end validation of exported Prometheus series (batch counts, bounded labels, runtime metrics, exporter collision).
.changeset/clean-otters-observe.md	Publishes a minor release note for the new observability features.

Files not reviewed (1)

• pnpm-lock.yaml: Generated file

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.