Summary
The plugin-host streaming route reports empty_completion for streams that actually carried an in-band provider error, so a dead key behind a plugin executor is never classified as such and the request is not rotated.
internal/pluginhost/executor_route.go — wrapStreamEmptyCompletion — has two gaps:
- No in-band error surfacing at all. The detector's parsed provider error is never consulted, neither while streaming nor at stream close. A
401 invalid_api_key or 429 that arrives inside the SSE body is reported downstream as empty_completion, which is not routable to the auth that produced it.
- Ordering at stream close. Even once the error is consulted, judging emptiness first hides it.
flushData() runs only on a blank SSE separator line or from finish(), so an error event whose data line is newline-terminated but never followed by that blank line stays buffered in dataLines until Finish() runs. Finish() therefore both parses the error and returns isEmptyCompletion() == true; reporting emptiness on its return value before calling StreamError() replaces the routable invalid_api_key with a generic empty_completion.
Reproduction
Feed the wrapped stream two chunks and close it:
chunks <- coreexecutor.StreamChunk{Payload: []byte("data: {\"choices\":[{\"delta\":{\"role\":\"assistant\"}}]}\n\n")}
chunks <- coreexecutor.StreamChunk{Payload: []byte("event: error\ndata: {\"error\":{\"code\":\"invalid_api_key\",\"message\":\"invalid api key\"}}\n")}
close(chunks)
The first frame is recognized but carries no content, so acc.empty() is true; the second never receives its blank separator line. Observed downstream chunk:
empty_completion: upstream returned an empty completion
Expected: a *coreauth.Error with Code == "invalid_api_key".
Fix
Both halves are already implemented and verified upstream in router-for-me/CLIProxyAPI#4881:
- surfacing the detected error while streaming and at close — commit
139aae6d
- finalizing before judging emptiness — commit
4a001bf1, i.e.
terminalEmpty := detector.Finish()
if streamErr := detector.StreamError(); streamErr != nil {
_ = forward(coreexecutor.StreamChunk{Err: streamErr})
return
}
if terminalEmpty {
_ = forward(coreexecutor.StreamChunk{Err: coreauth.EmptyCompletionError()})
return
}
Regression test: TestWrapStreamEmptyCompletion_PrefersDetectedErrorOverTerminalEmptiness. Reverse bite-check, with the ordering restored to Finish()-first:
--- FAIL: TestWrapStreamEmptyCompletion_PrefersDetectedErrorOverTerminalEmptiness (0.00s)
executor_route_close_order_test.go:42: expected the provider error to survive terminal emptiness, got code "empty_completion" (empty_completion: upstream returned an empty completion)
FAIL
FAIL github.com/router-for-me/CLIProxyAPI/v7/internal/pluginhost 0.459s
FAIL
Why an issue and not a PR yet
The fix needs StreamBootstrapDetector.StreamError(), which needs the in-band error state that #195 adds to sdk/cliproxy/auth/empty_completion.go. Opening this as a PR against main today would duplicate roughly 313 lines of #195 in a second branch and guarantee a conflict. The branch is prepared locally and will be opened as a small, additive PR (export accessor plus the two executor_route.go call sites plus the test) as soon as #195 lands.
Cross-references
Summary
The plugin-host streaming route reports
empty_completionfor streams that actually carried an in-band provider error, so a dead key behind a plugin executor is never classified as such and the request is not rotated.internal/pluginhost/executor_route.go—wrapStreamEmptyCompletion— has two gaps:401 invalid_api_keyor429that arrives inside the SSE body is reported downstream asempty_completion, which is not routable to the auth that produced it.flushData()runs only on a blank SSE separator line or fromfinish(), so an error event whose data line is newline-terminated but never followed by that blank line stays buffered indataLinesuntilFinish()runs.Finish()therefore both parses the error and returnsisEmptyCompletion() == true; reporting emptiness on its return value before callingStreamError()replaces the routableinvalid_api_keywith a genericempty_completion.Reproduction
Feed the wrapped stream two chunks and close it:
The first frame is recognized but carries no content, so
acc.empty()is true; the second never receives its blank separator line. Observed downstream chunk:Expected: a
*coreauth.ErrorwithCode == "invalid_api_key".Fix
Both halves are already implemented and verified upstream in router-for-me/CLIProxyAPI#4881:
139aae6d4a001bf1, i.e.Regression test:
TestWrapStreamEmptyCompletion_PrefersDetectedErrorOverTerminalEmptiness. Reverse bite-check, with the ordering restored toFinish()-first:Why an issue and not a PR yet
The fix needs
StreamBootstrapDetector.StreamError(), which needs the in-band error state that #195 adds tosdk/cliproxy/auth/empty_completion.go. Opening this as a PR againstmaintoday would duplicate roughly 313 lines of #195 in a second branch and guarantee a conflict. The branch is prepared locally and will be opened as a small, additive PR (export accessor plus the twoexecutor_route.gocall sites plus the test) as soon as #195 lands.Cross-references
fix(auth): rotate on in-stream provider errors during bootstrap), whose EOF-finalization half is commitb3f4bc19