fix(auth): lower transient error cooldown default to 10s - #205
Conversation
A single 5xx/408 transient blip was sidelining genuinely-live accounts and models for a full minute. Lower the legacy fallback from 60s to 10s, keep the transient-error-cooldown-seconds knob configurable, and update the example config and the matching default-cooldown test.
warelik
left a comment
There was a problem hiding this comment.
Review verdict: APPROVED (posted as COMMENT; GitHub rejects APPROVE on your own PR).
The PR lowers the built-in transient-error cooldown default from 60s to 10s by changing the single transientErrorCooldown constant, which feeds all three use sites: the nextTransientErrorRetryAfter default path (conductor_cooldown.go:94) and the two ErrorCodeForceCooldown fallbacks (conductor_cooldown.go:928 and :2029). The transient-error-cooldown-seconds knob semantics (0 = built-in default, -1 = disable, N = explicit seconds) are untouched. The one duration-sensitive test is updated with a ±5s window around the new 10s target; the measured value lands just under 10s, so no flake risk. I found no other code, tests, or docs pinning the old 60s value. I did not re-run the test plan; the diff is consistent with it.
Non-blocking notes:
- See the inline comment on conductor_refresh.go: the request-scoped
stop-and-cooldown/continue-and-cooldownforce-cooldown path shares this constant and ignores the config knob, so those rules drop from 60s to 10s with no operator override. Worth confirming as intended. - config.example.yaml now says
keep the legacy 10-second cooldown— 10s is the new default rather than a legacy behavior, souse the default 10-second cooldownwould read better (it does at least match theSetTransientErrorCooldownSecondsdoc comment terminology).
Mirror PR router-for-me/CLIProxyAPI#5140 is identical; the same review applies there.
Add a code-level comment noting that transientErrorCooldown also backs the ErrorCodeForceCooldown fallbacks for request-scoped stop-and-cooldown / continue-and-cooldown rules, and that those paths use the constant directly without consulting the transient-error-cooldown-seconds knob.
warelik
left a comment
There was a problem hiding this comment.
Review verdict: APPROVED — re-review of head 07edc93 (posted as COMMENT; GitHub rejects APPROVE on your own PR).
The new docs commit fully addresses my previous inline feedback, in two places:
- The code comment on
transientErrorCooldowndocuments exactly the dual role I flagged: default for 408/500/502/503/504 transient errors (vianextTransientErrorRetryAfter) and the enforced fallback for request-scoped stop-and-cooldown / continue-and-cooldown rules, with the latter not consulting thetransient-error-cooldown-secondsknob. I verified this against conductor_cooldown.go (:94, :928, :2029) — the comment is accurate. Const-block formatting stays gofmt-clean (the comment line starts a new alignment run, so the single-space=is correct). - The PR description now has an explicit Note on ErrorCodeForceCooldown section calling out the 60s-to-10s behavior change on that path and stating it is intentional — exactly the call-out I asked for.
Nothing else changed since my prior approval. The remaining notes from that review (the config.example.yaml legacy wording, and the force-cooldown path having no operator override) were non-blocking and stand as recorded. Ready to merge.
Summary
Lowers the effective default transient-error cooldown from 60 seconds to 10 seconds.
A single transient 5xx/408 blip was sidelining a genuinely-live account or model for a full minute. With several auths sharing the same upstream model, one gateway hiccup could cause a service blackout. The legacy
transientErrorCooldownconstant is now10s, and the example config documents the new default. Thetransient-error-cooldown-secondsknob remains fully configurable for the normal transient-error path.Note on
ErrorCodeForceCooldowntransientErrorCooldownalso backs the twoErrorCodeForceCooldownfallbacks inconductor_cooldown.go(model state at line ~928 and auth state at line ~2029:now.Add(transientErrorCooldown)). These enforce request-scopedstop-and-cooldown/continue-and-cooldownrules. That path uses the same constant directly and is intentionally not gated bytransient-error-cooldown-seconds, so those forced cooldowns also drop from 60s to 10s. This is consistent with the blackout-avoidance rationale: a force-cooldown rule should not sideline a live account for a full minute either. A code comment has been added toconductor_refresh.goto make this explicit.Cross-link
Mirror PR in upstream
CLIProxyAPI: #5140Related work
Test plan
sdk/cliproxy/auth/conductor_overrides_test.gopins the new default at ~10s.go test ./sdk/cliproxy/auth/...andgo build ./...pass.