Problem
LLM agents accumulate context quickly. A single long-running task can carry thousands of tokens of conversation history, tool outputs, intermediate reasoning, retrieved documents, repeated state, and superseded build/test runs — most of which the model does not need at each step. This drives four compounding costs:
- Direct token cost — every input token is billed; context grows across turns.
- Latency — larger requests take longer to prefill.
- Context-limit risk — long sessions eventually hit the model's context window ceiling.
- Quality risk — an oversized, noisy context can degrade the model's focus on what actually matters.
Goal
Apply context engineering transparently, at the infrastructure layer (AuthBridge / Kagenti sidecar), so that every agent running through the platform benefits automatically — without changes to agent code.
The objective is: reduce the token footprint of LLM requests as much as possible, with minimal impact on task accuracy.
Approach
A composable, provider-agnostic pipeline of context-reduction strategies intercepts LLM-bound requests. Reductions are either lossless or reversible — any dropped content can be recovered on demand. The pipeline is fail-open: any strategy that errors or grows the request is automatically reverted; the original is always a valid fallback.
Integration into Kagenti happens as an AuthBridge in-process plugin, applied transparently around LLM calls.
Scope of this epic
| Track |
Description |
| Plugin integration |
AuthBridge context engineering plugin — see #641 |
| Observability |
Expose per-request and aggregate savings metrics through AuthBridge's StatsSource |
| Configuration |
YAML-driven pipeline config, preset selection, per-strategy tuning, documentation |
| Evaluation |
Benchmark coverage confirming reward parity and token savings on Kagenti agent workloads |
Success criteria
- Measurable reduction in tokens billed per agent session.
- Minimal reduction in accuracy.
- No agent code changes required to benefit.
- Plugin is configurable and observable.
Sub-issues
Problem
LLM agents accumulate context quickly. A single long-running task can carry thousands of tokens of conversation history, tool outputs, intermediate reasoning, retrieved documents, repeated state, and superseded build/test runs — most of which the model does not need at each step. This drives four compounding costs:
Goal
Apply context engineering transparently, at the infrastructure layer (AuthBridge / Kagenti sidecar), so that every agent running through the platform benefits automatically — without changes to agent code.
The objective is: reduce the token footprint of LLM requests as much as possible, with minimal impact on task accuracy.
Approach
A composable, provider-agnostic pipeline of context-reduction strategies intercepts LLM-bound requests. Reductions are either lossless or reversible — any dropped content can be recovered on demand. The pipeline is fail-open: any strategy that errors or grows the request is automatically reverted; the original is always a valid fallback.
Integration into Kagenti happens as an AuthBridge in-process plugin, applied transparently around LLM calls.
Scope of this epic
StatsSourceSuccess criteria
Sub-issues