Skip to content

Plugin-host streams report empty_completion instead of the in-band provider error #201

Description

@warelik

Summary

The plugin-host streaming route reports empty_completion for streams that actually carried an in-band provider error, so a dead key behind a plugin executor is never classified as such and the request is not rotated.

internal/pluginhost/executor_route.gowrapStreamEmptyCompletion — has two gaps:

  1. No in-band error surfacing at all. The detector's parsed provider error is never consulted, neither while streaming nor at stream close. A 401 invalid_api_key or 429 that arrives inside the SSE body is reported downstream as empty_completion, which is not routable to the auth that produced it.
  2. Ordering at stream close. Even once the error is consulted, judging emptiness first hides it. flushData() runs only on a blank SSE separator line or from finish(), so an error event whose data line is newline-terminated but never followed by that blank line stays buffered in dataLines until Finish() runs. Finish() therefore both parses the error and returns isEmptyCompletion() == true; reporting emptiness on its return value before calling StreamError() replaces the routable invalid_api_key with a generic empty_completion.

Reproduction

Feed the wrapped stream two chunks and close it:

chunks <- coreexecutor.StreamChunk{Payload: []byte("data: {\"choices\":[{\"delta\":{\"role\":\"assistant\"}}]}\n\n")}
chunks <- coreexecutor.StreamChunk{Payload: []byte("event: error\ndata: {\"error\":{\"code\":\"invalid_api_key\",\"message\":\"invalid api key\"}}\n")}
close(chunks)

The first frame is recognized but carries no content, so acc.empty() is true; the second never receives its blank separator line. Observed downstream chunk:

empty_completion: upstream returned an empty completion

Expected: a *coreauth.Error with Code == "invalid_api_key".

Fix

Both halves are already implemented and verified upstream in router-for-me/CLIProxyAPI#4881:

  • surfacing the detected error while streaming and at close — commit 139aae6d
  • finalizing before judging emptiness — commit 4a001bf1, i.e.
terminalEmpty := detector.Finish()
if streamErr := detector.StreamError(); streamErr != nil {
	_ = forward(coreexecutor.StreamChunk{Err: streamErr})
	return
}
if terminalEmpty {
	_ = forward(coreexecutor.StreamChunk{Err: coreauth.EmptyCompletionError()})
	return
}

Regression test: TestWrapStreamEmptyCompletion_PrefersDetectedErrorOverTerminalEmptiness. Reverse bite-check, with the ordering restored to Finish()-first:

--- FAIL: TestWrapStreamEmptyCompletion_PrefersDetectedErrorOverTerminalEmptiness (0.00s)
    executor_route_close_order_test.go:42: expected the provider error to survive terminal emptiness, got code "empty_completion" (empty_completion: upstream returned an empty completion)
FAIL
FAIL	github.com/router-for-me/CLIProxyAPI/v7/internal/pluginhost	0.459s
FAIL

Why an issue and not a PR yet

The fix needs StreamBootstrapDetector.StreamError(), which needs the in-band error state that #195 adds to sdk/cliproxy/auth/empty_completion.go. Opening this as a PR against main today would duplicate roughly 313 lines of #195 in a second branch and guarantee a conflict. The branch is prepared locally and will be opened as a small, additive PR (export accessor plus the two executor_route.go call sites plus the test) as soon as #195 lands.

Cross-references

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions