feat(auth,config): configurable quota cooldown floor and per-status transient cooldown - #217
Open
warelik wants to merge 14 commits into
Open
feat(auth,config): configurable quota cooldown floor and per-status transient cooldown#217warelik wants to merge 14 commits into
warelik wants to merge 14 commits into
Conversation
Providers answering 429 for an exhausted daily quota can attach a RetryInfo hint far shorter than the real recovery window. Gemini and Antigravity were observed returning 479417207ns while the key stayed dead for the rest of the day. Both quota paths took that hint verbatim, so an exhausted credential returned to the pool half a second later and BackoffLevel never advanced past its current step: every retry recomputed the same level and immediately overwrote the deadline with the sub-second hint. Compute the escalating ladder first and let a provider hint only push the deadline further out, never pull it in. A genuine long hint still wins; a sub-second one can no longer undercut the ladder. Covered by TestMarkResultSubSecondQuotaHintStillEscalates and TestApplyAuthFailureStateSubSecondQuotaHintStillEscalates in sdk/cliproxy/auth/cooldown_backoff_test.go.
Do not apply the escalating quota ladder floor when a 429 response explicitly specifies a zero or non-positive RetryAfter duration (e.g. transient websocket connection limit errors). Escalating quota cooldown remains gated to positive retry hints and default quota exhaustion.
decideAntigravity429 classifies a RATE_LIMIT_EXCEEDED 429 whose retry hint is shorter than three seconds as an instant retry on the same credential rather than an exhausted quota. The unconditional ladder floor still replaced that hint with a quota cooldown step, parking a still-usable credential for up to the full ladder window. Carry the executor classification through statusErr and Result so the ladder floor only applies to 429s that were not decisively classified as a short-lived rate limit. Exhausted quota and unclassified bodies keep the floor and keep escalating exactly as before.
The transient rate-limit flag only reached the conductor from the non-stream execution path. Streaming failures built their results from retryAfterFromError alone, and the Antigravity token-count path built statusErr by hand, so both still floored a provider-classified short-lived 429 at the quota ladder and parked a usable credential. Set TransientRateLimit next to every RetryAfter assignment in the streaming pool, and build the token-count errors through newAntigravityStatusErr so they inherit the same classification.
The Antigravity executor raises a synthetic 429 while an auth sits in a short cooldown. That cooldown is a local, self-imposed pause of at most a few minutes, but the error carried only a positive retryAfter hint and no classification, so isTransientRateLimitError() returned false and MarkResult()/applyAuthFailureState() read it as an exhausted upstream quota. BackoffLevel then escalated toward the 30 minute ceiling and parked an account that was never throttled upstream. Set transientRateLimit on all three cooldown short-circuits (Execute, executeClaudeNonStream, ExecuteStream) so the conductor rotates to the next auth instead of escalating backoff. Covered by TestAntigravityShortCooldownErrorIsTransient, which asserts the classification on all three entry points.
classifyClaudeUpstreamError built ordinary (non-unified) Claude 429s as claudeRateLimitError wrapping a statusErr with no transientRateLimit flag, so isTransientRateLimitError() returned false and MarkResult() treated an ordinary model-level throttle as exhausted quota, escalating BackoffLevel toward the 30 minute ceiling and parking a credential that was only briefly throttled. Mark the ordinary path transient. Unified 5h/7d rejections keep the quota ladder untouched. Covered by TestClassifyClaudeUpstreamError_OrdinaryRateLimitIsTransient and TestClassifyClaudeUpstreamError_UnifiedRejectionNotTransient.
Both 429 cooldown branches (MarkResult's per-model state and applyAuthFailureState's credential state) only kept a short-lived rate limit out of the quota ladder when a parseable retryAfter hint was present. A transient 429 with no hint fell through to quotaCooldownAfterFailure and advanced BackoffLevel toward the 30 minute ceiling, parking a credential that was only briefly throttled. Transient 429s now bypass the ladder regardless of hint presence: the provider-supplied retryAfter is kept verbatim when present, and without a hint the cooldown falls back to nextTransientErrorRetryAfter (the standard ~60s transient-error cooldown). Covered by TestManager_MarkResult_Transient429WithoutHintBypassesQuotaLadder, which asserts both the per-model and credential quota ladders stay at level 0 and both NextRetryAfter values land at the transient cooldown.
The transient-429 fallback (nextTransientErrorRetryAfter) returns a zero time when transient cooldowns are disabled (transientErrorCooldownSeconds < 0). Both 429 cooldown branches still stored that zero alongside Unavailable=true and Quota.Exceeded=true with an empty NextRecoverAt, and availabilityBlock read the zero-time quota block as an indefinite park. When the transient fallback yields a zero time the 429 handling now leaves the model and the credential available: no quota mark, no suspension, no retry time. A pre-existing quota block is preserved in both paths. Covered by TestManager_MarkResult_Transient429WithoutHintRespectsDisabledCooldown.
…ient cooldown The transient-cooldown-off skip in applyAuthFailureState restored the status message and quota fields but left auth.Unavailable=true (set at the top of the function) and auth.NextRetryAfter untouched, so a credential hit by a transient 429 without a hint stayed indefinitely blocked anyway. Capture the prior availability fields and restore them in the skip. Covered by TestManager_MarkResult_Transient429WithoutHintRespectsDisabledCooldownAuthLevel, which drives an auth-level Result (empty Model) through applyAuthFailureState.
…o as transient When Google returns RESOURCE_EXHAUSTED with an ErrorInfo reason of RATE_LIMIT_EXCEEDED but omits the RetryInfo detail, the decision table downgrades to SoftRetry and the resulting error reported TransientRateLimit() == false. The conductor then read a plain per-minute throttle as exhausted quota and escalated BackoffLevel toward the 30 minute ceiling instead of rotating. newAntigravityStatusErr now also marks the soft rate limit transient, but only when the classification came from the ErrorInfo reason: the bare "too many requests" message heuristic stays on the quota ladder. Covered by a new case in TestNewAntigravityStatusErrMarksTransientRateLimit.
…ng test TestAntigravityConcurrentRequestsReusePooledConnections failed in CI with 9 distinct connections for 3 waves of 8, even though pooling works: a wave boundary can cost one extra dial when an idle connection is retired at exactly the wrong moment. Allow one stray dial per later wave. The MaxIdleConnsPerHost=2 regression still fails loudly: it would open roughly totalConns - 2*(waves-1) distinct connections (20 here), far above the new allowance of 10. Verified with 10 consecutive local runs on this commit: 10/10 pass.
TestExecuteStream_PublishesUsageRecordFromStreamUsage timed out in CI at its 5s cap while waiting for the asynchronously published usage record. The cap only bounds the failure case — a matching record returns immediately — so raising it to 30s costs nothing on success and absorbs scheduler starvation on loaded runners. Verified with 3/3 consecutive local runs on this commit.
…ldown-by-status knobs
warelik
added a commit
to warelik/CLIProxyAPIPlus
that referenced
this pull request
Aug 21, 2026
…wn floor and per-status transient cooldown
- Compare collapsed per-status cooldown maps on reload - Clamp quota-cooldown floor to quotaBackoffMax - Report new cooldown knobs in config diff - Add tests for floor clamp, per-status 0 fallback, and map equality
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the first two missing knobs from
reports/recommended-config.md:quota-cooldown-floor-seconds(default1): makes the quota-ladder base configurable, generalizing the hardcoded floor introduced in fix(auth): floor quota cooldown at the escalating ladder #198 /router-for-me/CLIProxyAPI#5130. Operators can now raise the floor for noisy providers without rebuilding.transient-cooldown-by-status: per-HTTP-status overrides for408/500/502/503/504cooldowns, falling back totransient-error-cooldown-secondsfor any status not listed. This is the missing-knob companion to fix(auth): lower transient error cooldown default to 10s #205 /router-for-me/CLIProxyAPI#5140.Stacking
This branch is based on
warelik/fix/quota-backoff-hint-floor(#198) because the newquota-cooldown-floor-secondsvalue is consumed inside the refactorednextQuotaCooldownpath introduced there. It will remain a draft until #198 lands, or it can be reviewed as a stack.Changes
ConfiggainsQuotaCooldownFloorSecondsandTransientCooldownByStatus.parse.goandconfig_load.godefaultQuotaCooldownFloorSecondsto1.sdk/cliproxy/auth/conductor_cooldown.goloads the floor and per-status map at runtime;nextQuotaCooldownandnextTransientErrorRetryAfteruse them.service_auth.go,internal/api/server.go,internal/api/server_reload.go, andcmd/server/main.gowire the new setters on startup and config reload.config.example.yamldocuments both knobs and their relationship to fix(auth): floor quota cooldown at the escalating ladder #198/fix(auth): lower transient error cooldown default to 10s #205.internal/config/cooldown_config_test.goand two new cases insdk/cliproxy/auth/cooldown_backoff_test.go.Cross-links
router-for-me/CLIProxyAPI#5130(quota ladder floor),router-for-me/CLIProxyAPI#5140(lower transient cooldown default)Test plan
go build ./... go test ./sdk/cliproxy/auth/... ./internal/config/...