Improve error transparency and fix connection pool timeout#119
Merged
Conversation
Drop httpx keepalive_expiry from 20 minutes to 45 seconds so pooled provider connections are not reused after upstream load balancers or gvproxy's NAT have silently dropped them — the source of intermittent RemoteProtocolError failures on the first request after an idle gap. Also include the real exception message in the image-generation SSE error frame (matching the chat streaming path) instead of a hardcoded generic string, so clients can see the actual cause. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012NXSZbLdC1sEJNmTcCszXc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR makes two targeted fixes to improve reliability and debuggability:
Surface actual exception details in image generation error responses — instead of a generic "Stream processing failed" message, the client now receives the real exception string, matching the chat streaming error path for consistency.
Reduce HTTP connection pool keepalive expiry — lower the timeout from 20 minutes to 45 seconds to prevent reuse of stale connections that may have been silently dropped by the enclave's egress NAT layer.
Key Changes
image_generation.py: Modified the exception handler in thegenerate()streaming function to includestr(e)in the error payload sent to clients. This provides better visibility into actual failures (e.g., provider API errors, network issues) in browser-side logs, rather than an opaque generic string. The change aligns error reporting with the chat completion streaming path.llm_backend.py: Reducedkeepalive_expiryfrom60 * 20(1200 seconds / 20 minutes) to45.0seconds in the httpx connection pool limits. Added detailed comment explaining the rationale: gvproxy's NAT flow expiry and the enclave's silent connection termination (no FIN packet) can cause httpx to reuse dead connections, leading toRemoteProtocolErroron the first request after an idle gap. A shorter expiry mitigates this by forcing periodic reconnection.Implementation Details
str(e) or "Stream processing failed"to gracefully fall back if the exception has no string representation.https://claude.ai/code/session_012NXSZbLdC1sEJNmTcCszXc