Large tool results bypass auto-compaction and context_length_exceeded is retried unchanged

[danshapiro]

Summary

A delegated session can exceed the model context after one or more very large tool results even when context-simple has max_tokens: 300000, compact_threshold: 0.8, and auto_compact: true. The resulting OpenAI context_length_exceeded error is then retried five times with an unchanged oversized payload.

Environment

• Amplifier CLI: 2026.07.11-c150034
• amplifier-core: 1.6.0
• Bundle: anchors 0.1.0
• Provider: OpenAI (gpt-5.6-sol, long context enabled)
• Delegation: fresh child with context_depth: none
• OS: Linux / WSL2

Reproduction

1. Start a fresh delegated child with context_depth: none.
2. Have the child run content searches over JSONL files whose individual matching lines are very large.
3. In the observed run, two search tool results were approximately 614,299 and 735,454 characters.
4. Observe provider input usage grow from 57,519 tokens to 441,952 and then 827,426 despite the bundle's 300,000-token context setting and auto-compaction.
5. After one more iteration, OpenAI returns invalid_request_error / context_length_exceeded.
6. Observe five provider:retry events replaying the same deterministic failure.

No raw session contents are attached because they may contain private data; the sizes and usage counts above come from event metadata only.

Expected behavior

Before every provider submission, Amplifier should ensure the serialized request is within the effective provider/model context budget. Large tool results should be truncated, artifactized, or trigger compaction before the next request. A context_length_exceeded response should be non-retryable unless Amplifier first reduces the payload.

Actual behavior

Tool results can push a delegated session far beyond the configured context budget without compaction. The oversized provider request fails, then the same payload is retried five times.

Suggested safeguards

• Enforce a configurable byte/token cap on every tool result, independent of result line count.
• Run a final provider-specific token-budget check immediately before submission.
• Compact or artifactize oversized tool results before sending.
• Classify context_length_exceeded as non-retryable unless the request is changed.
• Add a regression test with a single very large tool result and another with multiple medium-large results.

[danshapiro]

Root-cause follow-up

Debugged this further; the two behaviors reported here have concrete, identified causes in the module code. File references are from the current module cache (provider-openai module, single squashed commit 25b9f b9 era; paths relative to each module's package).

1. Why context_length_exceeded is retried 5x unchanged

The provider does have a ContextLengthError classifier, but it no longer matches OpenAI's current error shape. In amplifier_module_provider_openai/__init__.py:1514-1518:

if (
    "context length" in raw_msg
    or "too many tokens" in raw_msg
    or "maximum context" in raw_msg
):
    raise kernel_errors.ContextLengthError(...)

OpenAI's rejection now reads "Your input exceeds the context window of this model" with stable code context_length_exceeded. None of the three substrings match.

Worse, on the streaming path the error doesn't arrive as openai.BadRequestError at all — it surfaces as a generic SDK exception and falls into the catch-all at __init__.py:1629-1639, which hardcodes retryable=True:

except Exception as e:
    ...
    raise kernel_errors.LLMError(error_msg, provider=self.name, retryable=True) from e

That is exactly the observed LLMError + 5 provider:retry events on a deterministic failure.

The same stale substring list is duplicated in amplifier-core (amplifier_core/utils/retry.py:147, classify_error) and the anthropic provider.

Suggested fix: match on the stable error code, not prose, in both the BadRequestError handler and the catch-all:

err_code = (body.get("error") or body).get("code") if isinstance(body, dict) else None
if (
    err_code == "context_length_exceeded"
    or "context_length_exceeded" in raw_msg
    or "exceeds the context window" in raw_msg
    or "context length" in raw_msg
    or "too many tokens" in raw_msg
    or "maximum context" in raw_msg
):
    raise kernel_errors.ContextLengthError(error_msg, provider=self.name, status_code=400) from e

and apply the same check in the except Exception catch-all before wrapping as retryable LLMError. Same patterns should go into classify_error in amplifier-core.

2. Why proactive compaction misses the budget

Compaction is purely proactive (nothing consumes ContextLengthError reactively — verified by grep across the module set), and the budget check uses a chars // 4 token estimator (context-simple/__init__.py:1299-1301). Code/JSON-heavy content tokenizes at ~3–3.5 chars/token, so real usage runs 15–30% above the estimate.

This slop was harmless when the advertised window (272K default) was far below the model's true limit. With enable_long_context on a 1M-class model (e.g. gpt-5.6-sol), the provider advertises the full 1,050,000-token window (_capabilities.py, get_info() at provider-openai/__init__.py:478-499), leaving only ~8–14% headroom below the hard limit — smaller than the estimator error. The pre-submission provider-specific budget check suggested in the issue description would close this; alternatively a ~3.5 chars/token estimate or a larger safety_margin for long-context windows.

(A third contributing factor — enable_response_chaining: auto sending both previous_response_id and the full re-serialized history — is a distinct defect; filing separately.)

[danshapiro]

Additional footgun affecting the workaround

For anyone mitigating this by disabling long context: beware that enable_long_context (and several sibling flags) are read as a raw truthy check, not parsed as booleans. amplifier_module_provider_openai/__init__.py:391:

self.enable_long_context = self.config.get("enable_long_context", False)

Settings values commonly arrive as strings (e.g. enable_long_context: 'true' in settings.yaml). Setting the workaround as enable_long_context: 'false' therefore silently keeps long context enabled, because the non-empty string 'false' is truthy in Python — the config does the opposite of what it says.

Workaround that actually works: remove the key entirely (falls back to the False default), or ensure the value reaches the provider as a real boolean.

Suggested fix: coerce these flags through a string-aware bool parser (str(v).lower() in ("1", "true", "yes") or equivalent) at config load. The same pattern likely affects other string-configured boolean flags in this provider (e.g. use_streaming, and enable_1m_context / enable_prompt_caching in provider-anthropic if they use the same idiom).

[danshapiro]

Precision corrections after re-verification

We adversarially re-verified every claim in this thread against the source (openai SDK v2.45.0 + the current module cache). Three corrections and two strengthenings below. Issue #321 was re-checked the same way and stands as filed.

1. Correction — how the error reaches the retryable catch-all.
Earlier we said that on the streaming path the error "doesn't arrive as openai.BadRequestError at all." More precisely, there are two paths, and both misclassify:

• A 400 rejected at request-open DOES raise openai.BadRequestError even when streaming — the SDK's AsyncResponseStreamManager.__aenter__ awaits the HTTP POST (openai/lib/streaming/responses/_responses.py:212-213). That hits the substring classifier, misses, and becomes InvalidRequestError — non-retryable, but still not ContextLengthError.
• A failure surfacing mid-stream arrives as base openai.APIError (raw SSE error events; openai/_streaming.py:70,87) or as RuntimeError("Didn't receive a response.completed event.") for response.failed (_responses.py:187). Neither matches any typed handler in the provider, so both land in the catch-all at provider-openai/__init__.py:1629-1639 → LLMError(retryable=True).

The observed signature — LLMError with JSON body plus 5 provider:retry events — identifies the mid-stream path (RetryConfig(max_retries=5) at :401-402; retry_with_backoff loops max_retries + 1 = 6 attempts total). Bottom line unchanged: neither path produces ContextLengthError, and the mid-stream one retries a deterministic failure.

2. Correction — core helper name and relevance.
The amplifier-core fallback is classify_error_message (not classify_error), amplifier_core/utils/retry.py:148-149. Same stale substrings; it would also miss the new message (it does map HTTP 413 → ContextLengthError first, but OpenAI returns 400). Priority caveat: per grep, no cached provider currently calls it — it's documented as a fallback for providers that can't type errors themselves. Fixing it is hygiene, not the hot path.

3. Correction/addition — anthropic provider.
Anthropic's classifier (provider-anthropic/__init__.py:2836-2837) checks only two of the three substrings ("context length", "too many tokens"). Notably, Anthropic's own current overflow message ("prompt is too long: X tokens > Y maximum") matches neither — so the anthropic provider likely has the same misclassification bug against its own API, independent of OpenAI. May deserve its own issue.

4. Strengthening — why the configured max_tokens: 300000 didn't help (exact mechanism for the numbers in the OP).
In context-simple's _calculate_budget (__init__.py:1272-1297), the provider-advertised context_window doesn't merely take precedence — configured max_tokens is a last-resort fallback, used only when the provider reports nothing. There is no min(provider_window, max_tokens). With enable_long_context truthy on gpt-5.6-sol:

budget = 1,050,000 − (128,000 × 0.5) − 4,096 = 981,904 tokens

The OP's observed 827K usage never approaches the compaction trigger (0.8 × 981,904 ≈ 785K estimated — and the chars//4 estimate of the local conversation was far below even that), exactly reproducing "compaction never fired despite max_tokens: 300000". Suggested additional safeguard: treat configured max_tokens as a cap (min()), not a fallback.

5. Clarifications on the truthiness footgun (previous comment).

• Confirmed end-to-end: YAML safe_load keeps quoted strings as str, the app-cli config layer (runtime/config.py:806-826) only substitutes ${VAR} patterns, and the dict reaches the provider verbatim. No layer coerces 'true'/'false'.
• Scope caveat: get_info() declares these fields field_type="boolean", so configs written via interactive setup store real booleans — only hand-edited YAML hits this.
• The omission is conspicuous in-file: the provider coerces neighboring flags (bool(raw_chain) at :377, bool(...retry_jitter...) at :405) but not enable_long_context (:391) or use_streaming (:398).
• The azure provider's own _get_bool helper (provider-azure-openai/__init__.py:504-510) does bool(config[key]) — same footgun for its azure-specific keys.