feat(ai-proxy): support phase-specific HTTP timeouts - #13801
Draft
ruanweihong-aaa wants to merge 2 commits into
Draft
feat(ai-proxy): support phase-specific HTTP timeouts#13801ruanweihong-aaa wants to merge 2 commits into
ruanweihong-aaa wants to merge 2 commits into
Conversation
Add optional connect, send, and read timeout settings for ai-proxy and ai-proxy-multi while preserving the existing timeout fallback behavior.
nic-6443
reviewed
Aug 11, 2026
|
|
||
| __DATA__ | ||
|
|
||
| === TEST 1: ai-proxy resolves phase timeout fallbacks at the transport call site |
Member
There was a problem hiding this comment.
These test cases implemented with mock code are of little value and can be removed.
Author
There was a problem hiding this comment.
Removed the mock-only call-site test in 84c4343. The remaining coverage uses the plugin schema tests, the transport timeout tests, and the Test::Nginx delayed-upstream cases, including partial configuration with only read_timeout set. The focused suite passes: 3 files, 63 tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
ai-proxyandai-proxy-multicurrently expose a singletimeoutvalue that is applied to the HTTP connect, send, and read phases. This makes it difficult to fail fast during connection establishment or request upload while allowing a longer timeout for LLM response generation.This PR adds three optional phase-specific timeout fields:
connect_timeout: timeout for establishing the upstream connection;send_timeout: timeout for sending the request to the upstream;read_timeout: timeout for each socket read from the upstream.All three fields are expressed in milliseconds and accept integer values from
1to600000.The change preserves backward compatibility:
set_timeout(timeout)path;timeoutvalue;Implementation details:
ai-proxyandai-proxy-multischemas;ai-proxytransport call site;lua-resty-http'sset_timeouts(connect, send, read)for phase-specific configuration while retaining the originalset_timeout(timeout)compatibility path;Compatibility
The new fields are optional. Existing users who only configure
timeout, or rely on its default value, continue to use the original code path. The upper bound of each new field is the same as the existingtimeoutupper bound.read_timeoutis a per-socket-read timeout and is not a total wall-clock limit for streaming responses. Existingmax_stream_duration_msandmax_response_bytescontrols remain responsible for total streaming duration and response-size limits.Verification
Targeted, source-matched Docker tests:
ai-proxyandai-proxy-multi, including valid boundaries and invalid values;set_timeouts(connect, send, read)and compatibility with the numericset_timeout(timeout)path;read_timeoutconfigured, exercising fallback to the existingtimeoutvalue for the omitted phases;Files=3, Tests=63, Result: PASS.Additional local dual-Gateway A/B verification used identical etcd, routes, Docker network, and controlled upstream fixtures for the pre-fix and fixed images:
200after about250 ms; fixed returned504after about50 mswithread_timeout=50;1 s; fixed timed out after about50 mswithconnect_timeout=50;60 MiBrequest and an upstream that accepted headers but did not consume the body, the fixed image timed out significantly earlier withsend_timeout=50;These results cover the tests related to this change; they are not a claim that the complete APISIX test suite was executed locally.
Which issue(s) this PR fixes:
Fixes #12072
Checklist