Skip to content

Add rate-limit / back-pressure for the tool-bridge slot pool waiter queue #72

Description

@KochC

In index.js acquireBridgeSlot() (lines ~530-535) appends a Promise resolver to bridgeState.waiters whenever all pool slots are busy. There is no upper bound on waiters.length, so under sustained parallel load (e.g. many concurrent tool-calling requests from Open WebUI or n8n) the queue can grow without limit, consuming memory and eventually causing all queued requests to time out silently from the caller's perspective after the HTTP request deadline. The fix is to cap waiters.length at a configurable maximum (e.g. OPENCODE_LLM_PROXY_TOOL_BRIDGE_MAX_QUEUE) and immediately reject excess requests with HTTP 429 Too Many Requests so callers can apply their own retry logic. The pool size is already configurable via OPENCODE_LLM_PROXY_TOOL_BRIDGE_POOL_SIZE; a companion queue-depth env var would complete the resource-bound picture.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions