From 8b009f171dd30979f2b48749a56e71e80781b242 Mon Sep 17 00:00:00 2001 From: Eric Evans <194135482+ericevans-nv@users.noreply.github.com> Date: Mon, 17 Aug 2026 19:03:17 -0500 Subject: [PATCH 1/3] docs: document Event metadata injection Signed-off-by: Eric Evans <194135482+ericevans-nv@users.noreply.github.com> --- docs/about-nemo-relay/concepts/middleware.mdx | 110 ++++++++++++++---- 1 file changed, 88 insertions(+), 22 deletions(-) diff --git a/docs/about-nemo-relay/concepts/middleware.mdx b/docs/about-nemo-relay/concepts/middleware.mdx index 8fd94c1ef..0d8fd8c03 100644 --- a/docs/about-nemo-relay/concepts/middleware.mdx +++ b/docs/about-nemo-relay/concepts/middleware.mdx @@ -9,12 +9,13 @@ import { MermaidStyles } from "@/components/MermaidStyles"; SPDX-License-Identifier: Apache-2.0 */} This page explains the runtime behavior that runs around managed tool and LLM -calls and sanitizes emitted mark and scope events. +calls, injects Event metadata, and sanitizes emitted mark and scope events. ## What Middleware Is -Middleware controls or transforms tool and LLM execution and sanitizes emitted -events. NeMo Relay applies each middleware type at a specific lifecycle point. +Middleware controls or transforms tool and LLM execution and enriches or +sanitizes emitted events. NeMo Relay applies each middleware type at a specific +lifecycle point. Middleware is organized by lifecycle meaning rather than as one undifferentiated hook system. @@ -27,8 +28,9 @@ registrations accept callbacks that return a value or an awaitable when invoked through an asynchronous Relay API or queued event publication. Worker and native-plugin middleware can also complete asynchronously. Within each middleware chain, Relay awaits entries sequentially in priority order so later -callbacks observe earlier middleware output. Payload and event sanitizer chains -run on the queued publication path and do not delay managed execution. +callbacks observe earlier middleware output. Payload sanitizers, Event metadata +injectors, and Event sanitizer chains run on the queued publication path and do +not delay managed execution. The experimental raw C FFI and Go binding retain synchronous middleware callbacks. Relay invokes each callback on a native thread and waits for it to @@ -49,10 +51,11 @@ sanitization and publication rather than awaiting it. Manual lifecycle APIs synchronous and create or close their handle immediately. -Event sanitizers, conditional-execution guardrails, request intercepts, -execution intercepts, and subscribers are not re-entrant. These callbacks must -not invoke another NeMo Relay API that runs middleware, flushes subscriber -delivery, waits on an exporter, or clears plugins. Scope APIs remain supported: +Event metadata injectors, Event sanitizers, conditional-execution guardrails, +request intercepts, execution intercepts, and subscribers are not re-entrant. +These callbacks must not invoke another NeMo Relay API that runs middleware, +flushes subscriber delivery, waits on an exporter, or clears plugins. Scope +APIs remain supported: callbacks may create, push, or pop scopes at any nesting level and may replace the active scope stack with an arbitrary stack. Emitting a new event is the only supported operation that can enqueue additional callback work; Relay @@ -83,8 +86,8 @@ separately from this application-visible result. ## Registration Levels -Middleware and subscribers can be registered at different levels depending on their -lifetime and visibility. +Middleware and subscribers can be registered at different levels depending on +their lifetime and visibility. ### Global Registrations @@ -107,10 +110,11 @@ everything in application code. ## Middleware Families -NeMo Relay has two major middleware families with three distinct purposes: +NeMo Relay has three major middleware families with four distinct purposes: - **Intercepts** change the real request or callback execution path. - **Conditional-execution guardrails** decide whether the real work runs. +- **Event metadata injectors** add flat metadata to emitted events. - **Sanitize guardrails** change emitted observability without changing the real request or result. @@ -122,6 +126,8 @@ Choose the middleware type that matches the behavior you need: rejected. - Use a **request intercept** when the real request must change before the call. - Use an **execution intercept** when code must run before or after the callback. +- Use an **Event metadata injector** when every emitted Event should receive + additional flat metadata before sanitization. - Use a **sanitize guardrail** when only subscribers and exporters should see rewritten data. - Use a **mark or scope event sanitizer** when the sensitive fields are in @@ -144,6 +150,10 @@ execution intercept cannot undo work from a `next` continuation that it already invoked. This fail-closed contract is the same whether the callback completes directly or asynchronously. +Event metadata injectors fail open. If an injector fails, panics, or returns an +invalid attribute set, Relay logs the failure, omits all metadata from that +injector for the current Event, and continues with the original Event. + ## Intercepts Intercepts are middleware that change the real request or execution path. @@ -177,6 +187,57 @@ LLM streaming has a stream execution path for wrappers that need to run around chunk delivery and finalization rather than only around a single response object. +## Event Metadata Injection + +Event metadata injectors add OTel-compatible values to the existing Event +`metadata` object. They run on every delivered Event and receive the current +Event as immutable context, so a callback can inspect the Event kind and name +before deciding what to return. + +The v1 contract accepts keys made of ASCII letter, number, underscore, or +hyphen segments separated by optional single dots. Empty keys, whitespace, +unsupported punctuation, leading or trailing dots, and repeated dots are +invalid. Relay stores accepted keys literally; dots do not create nested +objects. Values may be strings, numbers, booleans, or homogeneous lists of +those primitive types. Nested objects, `null`, and mixed-type lists are rejected +as one atomic callback result. + +Injection is insert-only. Existing Event metadata is never overwritten. +Injectors run in ascending numeric priority and then by registration name, so +the first injector to insert a missing key wins. A later injector that returns +the same key cannot replace it. + +The following Rust registration adds one string to matching Events: + +```rust +use std::collections::BTreeMap; +use std::sync::Arc; + +use nemo_relay::api::registry::register_event_metadata_injector; +use serde_json::json; + +register_event_metadata_injector( + "machine-profile", + 10, + Arc::new(|event| { + Box::pin(async move { + let attributes = if event.name().starts_with("tool") { + BTreeMap::from([("nv.machine.profile".into(), json!("dgx"))]) + } else { + BTreeMap::new() + }; + Ok(attributes) + }) + }), +)?; +``` + +Register injectors globally, on an owning scope, or through a plugin +registration context. Injectors execute on the queued serial publication path, +before Event sanitizers, so they do not delay the managed tool or LLM callback. +Sanitizers retain the final decision to preserve, rewrite, or remove injected +metadata before subscriber and exporter delivery. + ## Guardrails Guardrails are middleware that block execution or sanitize observability payloads. @@ -229,16 +290,17 @@ arguments passed to the callback or the real value returned to the caller. ## Queued Event Publication Scope operations, marks, and manual or managed tool/LLM lifecycle calls do not -await observability sanitizers. At emission time Relay snapshots the event-only -payload, visible sanitizer chains, and subscribers, then places the work on a -serial dispatcher. The dispatcher awaits the specialized tool or LLM payload -sanitizers, then the event sanitizers, and publishes the event later in FIFO -order. +await observability middleware. At emission time Relay snapshots the event-only +payload, visible Event metadata injectors, sanitizer chains, and subscribers, +then places the work on a serial dispatcher. The dispatcher awaits specialized +tool or LLM payload sanitizers, applies Event metadata injectors, runs Event +sanitizers, and publishes the Event later in FIFO order. Subscriber and exporter delivery is therefore delayed, while start/end/mark order is preserved. Closing a scope or deregistering middleware after emission -does not affect queued snapshots. Sanitizer failures fail closed: Relay records -the callback failure and withholds the governed observability payload. +does not affect queued snapshots. Injector failures fail open and omit that +injector's output. Sanitizer failures fail closed: Relay records the callback +failure and withholds the governed observability payload. ## Managed Execution Order @@ -273,10 +335,13 @@ flowchart LR subgraph PublicationPath[Serial Publication Path] RequestSanitizers[Request Sanitizers] + StartInjection[Event Metadata Injectors] StartEvent[Scope-Start Sanitizers and Deliver Start] ResponseSanitizers[Response Sanitizers] + EndInjection[Event Metadata Injectors] EndEvent[Scope-End Sanitizers and Deliver End] - RequestSanitizers --> StartEvent --> ResponseSanitizers --> EndEvent + RequestSanitizers --> StartInjection --> StartEvent + StartEvent --> ResponseSanitizers --> EndInjection --> EndEvent end QueueStart -.-> RequestSanitizers @@ -292,8 +357,8 @@ The phases run as follows: 3. **Response phase:** Relay snapshots and enqueues the end event's observability copy, then returns the real result. 4. **Publication path:** The serial dispatcher runs request or response - sanitizers, then the matching scope-event sanitizers, and finally delivers - the event to subscribers and exporters. + sanitizers, applies Event metadata injectors, runs the matching scope-event + sanitizers, and finally delivers the Event to subscribers and exporters. The start event is submitted before execution begins, but its sanitizers run later on the publication path and may overlap application execution. The serial @@ -303,6 +368,7 @@ for queued sanitization and delivery when a caller needs that barrier. This ordering preserves the distinction between the families: - Use an intercept to change real execution. +- Use an Event metadata injector to add observability context. - Use a sanitize guardrail to change only emitted observability. ### Rejection Path From 59f8ffb2ed44db9092501eac0d179d1307c10210 Mon Sep 17 00:00:00 2001 From: Eric Evans <194135482+ericevans-nv@users.noreply.github.com> Date: Wed, 19 Aug 2026 01:07:44 -0500 Subject: [PATCH 2/3] docs: clarify event metadata injector semantics Signed-off-by: Eric Evans <194135482+ericevans-nv@users.noreply.github.com> --- docs/about-nemo-relay/concepts/middleware.mdx | 24 ++++++++++--------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/docs/about-nemo-relay/concepts/middleware.mdx b/docs/about-nemo-relay/concepts/middleware.mdx index 0d8fd8c03..a323314e5 100644 --- a/docs/about-nemo-relay/concepts/middleware.mdx +++ b/docs/about-nemo-relay/concepts/middleware.mdx @@ -110,7 +110,7 @@ everything in application code. ## Middleware Families -NeMo Relay has three major middleware families with four distinct purposes: +NeMo Relay provides four middleware purposes: - **Intercepts** change the real request or callback execution path. - **Conditional-execution guardrails** decide whether the real work runs. @@ -150,9 +150,10 @@ execution intercept cannot undo work from a `next` continuation that it already invoked. This fail-closed contract is the same whether the callback completes directly or asynchronously. -Event metadata injectors fail open. If an injector fails, panics, or returns an -invalid attribute set, Relay logs the failure, omits all metadata from that -injector for the current Event, and continues with the original Event. +Event metadata injectors fail open. If an injector fails, panics, or returns +invalid attributes, Relay logs the failure, ignores only that injector's output +for the current Event, and continues with later injectors using the Event as it +stood before the failed injector. ## Intercepts @@ -194,13 +195,14 @@ Event metadata injectors add OTel-compatible values to the existing Event Event as immutable context, so a callback can inspect the Event kind and name before deciding what to return. -The v1 contract accepts keys made of ASCII letter, number, underscore, or -hyphen segments separated by optional single dots. Empty keys, whitespace, -unsupported punctuation, leading or trailing dots, and repeated dots are -invalid. Relay stores accepted keys literally; dots do not create nested -objects. Values may be strings, numbers, booleans, or homogeneous lists of -those primitive types. Nested objects, `null`, and mixed-type lists are rejected -as one atomic callback result. +The v1 contract accepts keys made of non-empty segments containing only ASCII +letters, digits, underscores, or hyphens. Segments may be separated by single +dots, so keys such as `region.zone.name` are valid. Leading dots, trailing +dots, consecutive dots, whitespace, and unsupported punctuation are invalid. +Relay stores accepted keys literally; dots do not create nested objects. Values +may be strings, numbers, booleans, or homogeneous lists of those primitive +types. Nested objects, `null`, and mixed-type lists are rejected as one atomic +callback result. Injection is insert-only. Existing Event metadata is never overwritten. Injectors run in ascending numeric priority and then by registration name, so From 78d2e87d2b52dcda3c2e16c4700279f7605fb511 Mon Sep 17 00:00:00 2001 From: Eric Evans <194135482+ericevans-nv@users.noreply.github.com> Date: Wed, 19 Aug 2026 01:47:01 -0500 Subject: [PATCH 3/3] docs: use neutral metadata injection example Signed-off-by: Eric Evans <194135482+ericevans-nv@users.noreply.github.com> --- docs/about-nemo-relay/concepts/middleware.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/about-nemo-relay/concepts/middleware.mdx b/docs/about-nemo-relay/concepts/middleware.mdx index a323314e5..69d0d64b5 100644 --- a/docs/about-nemo-relay/concepts/middleware.mdx +++ b/docs/about-nemo-relay/concepts/middleware.mdx @@ -219,12 +219,12 @@ use nemo_relay::api::registry::register_event_metadata_injector; use serde_json::json; register_event_metadata_injector( - "machine-profile", + "tool-category", 10, Arc::new(|event| { Box::pin(async move { let attributes = if event.name().starts_with("tool") { - BTreeMap::from([("nv.machine.profile".into(), json!("dgx"))]) + BTreeMap::from([("example.tool.category".into(), json!("catalog"))]) } else { BTreeMap::new() };