CodexCont

English · 中文

Continue-thinking middleware for Codex / OpenAI Responses-compatible APIs.

Tested and applicable to the gpt-5.6, gpt-5.5, and gpt-5.4 model series. (Even the latest gpt-5.6 series exhibits a similar reasoning-truncation mechanism.)

This project is a small Starlette proxy that sits between a coding agent and an upstream Responses endpoint. It detects a known reasoning-truncation fingerprint (usage.output_tokens_details.reasoning_tokens == 518 * n - 2), silently asks the model to continue thinking, and folds multiple upstream streaming responses into one coherent downstream response. The agent may speak either HTTP (POST + SSE) or WebSocket to the proxy; either way, the upstream leg is always a plain HTTP+SSE round.

Coding agent  ->  CodexCont  ->  Codex / Responses API

Installing via an AI agent? Hand it INSTALL-GUIDE-AGENT/AGENT.md — a step-by-step runbook written for an AI agent to execute on your machine.

Disclaimer

This project explicitly bypasses the observed OpenAI Codex reasoning-truncation behavior. If your use of this middleware is considered abusive, violates service terms, increases costs unexpectedly, or causes any other adverse consequences, you are solely responsible for those consequences.

What it does

• Streams reasoning items to the agent live.
• Buffers tentative final output (message and function_call) until the upstream terminal event reveals whether the round was truncated.
• If the round is truncated, discards the tentative output and opens a continuation round with the prior reasoning replayed.
• If the round finishes cleanly or a safety cap is reached, flushes the final round output and emits one reconstructed terminal response.
• Leaves non-matching traffic as a transparent passthrough.
• Accepts either HTTP (POST /v1/responses) or WebSocket (ws(s)://.../v1/responses) from the agent -- Codex (>= ~0.140) tries WebSocket first and otherwise burns several retries falling back to HTTP on every turn.
• Forwards server-side context compaction (POST /v1/responses/compact, used by Codex auto-compaction) as a streaming passthrough with the same generous timeout as /responses -- a compaction routinely thinks for over a minute, which would 502 through a plain buffered proxy hop.
• Answers GET /v1/models (and /v1/models/{id}, /health) with a minimal placeholder list instead of 404, since several clients (Codex included) poll it. Any other /v1/* call is forwarded to the real upstream unchanged.

The default continuation method is a hidden phase: "commentary" assistant message ("Continue thinking..."). A legacy synthetic tool-pair mode is also available.

Requirements

• Python >= 3.12
• uv recommended

Runtime dependencies are declared in pyproject.toml:

• httpx
• starlette
• uvicorn[standard] (the standard extra pulls in websockets, needed for the WebSocket route below)

Quick start

One command

Install and run the CLI (no clone required):

uvx --from git+https://github.com/ZhenHuangLab/CodexCont codexcont

From source

git clone https://github.com/ZhenHuangLab/CodexCont.git
cd CodexCont

./codexcont install   # writes ~/.codexcont/config.toml interactively
./codexcont start       # proxy at http://127.0.0.1:8787/v1/responses

Other useful commands: ./codexcont logs -f, ./codexcont stop, ./codexcont wire-codex (point Codex at the proxy). Run ./codexcont --help for the full list.

Or use uv:

uv sync && cp config.example.toml config.toml && uv run python run.py

Point your client at the proxy

Use the proxy URL instead of the real upstream URL.

Example:

http://127.0.0.1:8787/v1/responses

The example default configuration (config.example.toml, copied to config.toml) uses:

[upstream]
url = "https://chatgpt.com/backend-api/codex/responses"
mode = "header"

With mode = "header", a Responses-API-Base request header overrides the configured url; when the header is absent, requests fall back to the configured Codex URL.

For example, to target a generic Responses-compatible endpoint, send:

Responses-API-Base: https://api.openai.com/v1

The middleware appends /responses unless the supplied value already ends with /responses. This control header is stripped before forwarding upstream.

WebSocket transport

Codex (>= ~0.140, the responses_websockets feature) tries a WebSocket upgrade at ws(s)://.../v1/responses before falling back to HTTP; without WebSocket support a proxy 405s on every turn and Codex burns several retries before it falls back (visible as Reconnecting... N/5 and Unsupported upgrade request noise in server logs). CodexCont answers both:

• POST /v1/responses -- the original HTTP + SSE transport.
• ws(s)://.../v1/responses -- speaks the documented WebSocket mode wire shape: the agent sends {"type": "response.create", ...body...}, the proxy answers with individual response.* event frames (or one {"type": "error", ...} frame if a round can't be opened at all).

Internally, every turn -- regardless of which transport the agent used -- still opens one plain HTTP+SSE round upstream and runs it through the identical continuation-folding logic. There is no persistent upstream WebSocket connection, so this buys "no fallback noise" rather than the extra latency win a true end-to-end WebSocket bridge would; folding correctness is identical either way.

Codex (>= ~0.142) chains turns on this transport with previous_response_id -- both tool-loop steps and later user turns send only the new delta plus that id, assuming a persistent, stateful session. Upstream can never resolve it against one of this proxy's disconnected per-round requests (400 Unsupported parameter: previous_response_id), so the proxy keeps a small process-wide cache (response id -> that turn's effective full input) and resolves the chain locally: it splices the cached history in front of the delta and always drops previous_response_id before forwarding. A cache miss (e.g. right after a proxy restart, or the id fell out of the TTL/size-bounded cache) degrades to forwarding just the delta rather than failing the request outright.

Set enable_websocket = false under [server] in config.toml to disable the WebSocket route and force HTTP-only (e.g. while debugging transport issues).

Authentication

config.toml supports three auth modes. The example default is passthrough:

[auth]
mode = "passthrough"               # passthrough | inject | passthrough_then_inject
access_token = ""                  # sent as Authorization: Bearer <access_token>
chatgpt_account_id = ""            # sent as chatgpt-account-id when non-empty

Modes:

• passthrough: forward the caller's auth headers only; inject nothing.
• inject: override/set auth headers from config.
• passthrough_then_inject: keep caller auth when present, otherwise inject from config.

Security guard: if a request supplies Responses-API-Base, the middleware will not leak configured credentials to that request-supplied URL. If the current auth mode would inject configured credentials for that request, it is rejected with 400. To use per-request upstream overrides with credentials, pass the caller's own Authorization and use mode = "passthrough" or mode = "passthrough_then_inject".

Do not commit secrets. rt.json and free_rt.json are ignored by .gitignore, and tokens in config.toml should be handled carefully.

When continuation is applied

The middleware folds only when all of the following are true:

• [continue].enabled = true
• request body is a JSON object
• stream is truthy
• reasoning is not explicitly disabled ("reasoning": false disables folding)
• when using method = "tool_pair", the request does not declare a real tool with the same name as [continue].continue_tool_name

All other requests are proxied unchanged as passthrough streams.

Continuation logic

For each upstream round:

1. Reasoning item events are forwarded live with rewritten sequence_number and output_index.
2. Message and function-call events are buffered as tentative output.
3. On the terminal event, the middleware reads usage.output_tokens_details.reasoning_tokens.
4. If the token count matches 518 * n - 2, is within the configured tier window, has encrypted reasoning content, and safety caps allow it, the middleware:
   • drops the buffered tentative output,
   • appends the round's reasoning plus a continuation marker to the next request input,
   • opens another upstream streaming round.
5. Otherwise it flushes the final buffered output and emits a reconstructed terminal event.

The downstream agent sees one response, while metadata includes details about the hidden rounds.

Response metadata

The final reconstructed response includes proxy metadata such as:

• metadata.proxy_rounds: per-round reasoning token counts and detected tier n.
• metadata.proxy_billed_usage: summed upstream token usage across hidden rounds.
• metadata.proxy_stopped_reason: present when a guard or error stops continuation.

Agent-facing usage is reconstructed to look like one response: round-1 input/cached tokens, summed reasoning tokens, and the final round's non-reasoning output.

Tests

The test suite is self-contained and does not require pytest:

uv run python tests/test_middleware.py
# or
.venv/Scripts/python.exe tests/test_middleware.py

Current offline coverage includes:

• truncation math
• incremental SSE parsing
• fold/rewrite behavior with captured SSE fixtures
• commentary and tool-pair continuation payloads
• header transparency
• upstream URL resolution
• auth safety guard
• EOF/upstream-error behavior
• GET /v1/models / /v1/models/{id} / /health
• the WebSocket route (fold, passthrough, and round-1-error relaying)
• the codexcont CLI's config.toml text-surgery and Codex openai_base_url wiring helpers

Project layout

middleware/
  app.py       # Starlette app: HTTP (POST) and WebSocket route handlers
  cli.py       # `codexcont` CLI: guided installer + background service manager
  codex.py     # truncation math and continuation payload builders
  config.py    # config.toml loader and dataclasses
  creds.py     # upstream header/auth construction
  proxy.py     # fold_stream state machine
  sse.py       # incremental SSE parser/serializer
  store.py     # in-memory ID store for optional stateful repair

tests/
  test_middleware.py
  fixtures/

run.py         # uvicorn entrypoint
codexcont      # one-click POSIX entrypoint for the CLI (codexcont.bat on Windows)
config.example.toml # example runtime configuration; copy to config.toml for local use

Limitations

• Final answer text is buffered until the terminal round proves it is not truncated, so final-answer first-token latency can be higher than a normal stream.
• Non-streaming requests are currently passed through rather than folded.
• The truncation detector is intentionally specific to the observed 518 * n - 2 fingerprint.
• Optional repair_followup = "stateful" uses in-memory process-local state; it is not shared across multiple proxy instances.
• The WebSocket route bridges to a plain HTTP+SSE upstream round per turn; it keeps no persistent upstream WebSocket connection, so it buys "no fallback noise" rather than the extra latency win a true end-to-end WebSocket mode promises. previous_response_id chaining is resolved from a process-wide, TTL/size-bounded local cache rather than a real upstream session -- a cache miss (proxy restart, expired entry) forwards just the delta input instead of the full history.

Acknowledgements

This project would not exist without the discussions in the LINUX DO community. Special thanks to @shinorochi and @dskdkj of the LINUX DO community for jointly pinning down the truncation mechanism and GPT's thinking model, and to @shinorochi for proposing the better approach based on commentary input rather than faked tool calls.