Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions src/adapters/openai-chat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -744,6 +744,18 @@ export function createOpenAIChatAdapter(provider: OcxProviderConfig): ProviderAd
messages,
stream: parsed.stream,
};
// Preserve a caller-selected service tier for OpenAI-compatible chat gateways. The
// request pipeline deliberately does not inject fast mode for this adapter, but dropping
// an explicit value here makes the Responses parser's serviceTier projection ineffective.
//
// Opt-in, like `prompt_cache_key` directly below: `service_tier` is an OpenAI-specific
// extension and 66 registry providers share this adapter, several of which reject
// unknown body fields. Forwarding unconditionally would turn a caller-supplied
// `service_tier` into an upstream 400 on those routes. `supportsServiceTier` is the
// Responses-wire flag (applyServiceTierGate) and deliberately does not gate this path.
if (provider.chatServiceTier && parsed.options.serviceTier !== undefined) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Document the chat service-tier opt-in

When an operator configures an openai-chat custom gateway according to docs-site/src/content/docs/reference/configuration/providers.md:70, setting the documented supportsServiceTier: true still causes this condition to drop every caller-supplied tier because the new, distinct chatServiceTier option is undocumented and absent from the GUI payload. Add chatServiceTier to the provider documentation and translated references (and expose or preserve it in provider configuration flows), or make this adapter honor the existing documented capability for the chat wire.

AGENTS.md reference: AGENTS.md:L234-L235

Useful? React with 👍 / 👎.

body.service_tier = parsed.options.serviceTier;
Comment on lines +756 to +757

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve tiers from the Chat Completions ingress

When an OpenAI-compatible client sends service_tier to POST /v1/chat/completions, chatCompletionsToResponsesBody forwards neighboring fields such as parallel_tool_calls and prompt_cache_key but omits service_tier (src/chat/inbound.ts:274-286). Consequently the Responses parser leaves parsed.options.serviceTier undefined and this new serializer branch never runs, even for a provider with chatServiceTier: true. Copy the validated string into the translated body and add an endpoint-level regression test.

Useful? React with 👍 / 👎.

}
if (modelInList(provider.reasoningSplitModels, parsed.modelId)) body.reasoning_split = true;
const maxTokens = resolveMaxTokens(provider, parsed);
const openRouterRouting = resolveOpenRouterRouting(provider, parsed.modelId);
Expand Down
2 changes: 2 additions & 0 deletions src/providers/derive.ts
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,7 @@ export function providerConfigSeed(entry: ProviderRegistryEntry): OcxProviderCon
...(entry.noPenaltyModels ? { noPenaltyModels: [...entry.noPenaltyModels] } : {}),
...(entry.parallelToolCalls !== undefined ? { parallelToolCalls: entry.parallelToolCalls } : {}),
...(entry.promptCacheKey !== undefined ? { promptCacheKey: entry.promptCacheKey } : {}),
...(entry.chatServiceTier !== undefined ? { chatServiceTier: entry.chatServiceTier } : {}),
...(entry.responsesPath !== undefined ? { responsesPath: entry.responsesPath } : {}),
...(entry.statelessResponses !== undefined ? { statelessResponses: entry.statelessResponses } : {}),
...(entry.requiresAdjacentResponsesToolResults !== undefined
Expand Down Expand Up @@ -419,6 +420,7 @@ export function enrichProviderFromRegistry(name: string, prov: OcxProviderConfig
if (!prov.noPenaltyModels && seed.noPenaltyModels) prov.noPenaltyModels = [...seed.noPenaltyModels];
if (prov.parallelToolCalls === undefined && seed.parallelToolCalls !== undefined) prov.parallelToolCalls = seed.parallelToolCalls;
if (prov.promptCacheKey === undefined && seed.promptCacheKey !== undefined) prov.promptCacheKey = seed.promptCacheKey;
if (prov.chatServiceTier === undefined && seed.chatServiceTier !== undefined) prov.chatServiceTier = seed.chatServiceTier;
// Fill-only: a hand-edited path must survive, and a config saved before the registry
// learned this route still gets backfilled.
if (prov.responsesPath === undefined && seed.responsesPath !== undefined) prov.responsesPath = seed.responsesPath;
Expand Down
6 changes: 6 additions & 0 deletions src/providers/registry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,12 @@ export interface ProviderRegistryEntry {
parallelToolCalls?: boolean;
/** Opt this provider into forwarding prompt_cache_key (OpenAI-specific; strict backends reject it). */
promptCacheKey?: boolean;
/**
* Opt-in: forward `service_tier` on the `/chat/completions` wire. Same hazard as
* `promptCacheKey` — an OpenAI-specific extension that strict gateways reject. Distinct from
* `supportsServiceTier`, which governs the Responses wire.
*/
chatServiceTier?: boolean;
autoToolChoiceOnlyModels?: string[];
preserveReasoningContentModels?: string[];
requiresReasoningPlaceholderModels?: string[];
Expand Down
1 change: 1 addition & 0 deletions src/router.ts
Original file line number Diff line number Diff line change
Expand Up @@ -350,6 +350,7 @@ export function routedProviderConfig(providerName: string, provider: OcxProvider
// opt-in, while an explicit user `false` keeps overriding registry `true`.
...(provider.parallelToolCalls === undefined && registryEntry.parallelToolCalls !== undefined ? { parallelToolCalls: registryEntry.parallelToolCalls } : {}),
...(provider.promptCacheKey === undefined && registryEntry.promptCacheKey !== undefined ? { promptCacheKey: registryEntry.promptCacheKey } : {}),
...(provider.chatServiceTier === undefined && registryEntry.chatServiceTier !== undefined ? { chatServiceTier: registryEntry.chatServiceTier } : {}),
...(provider.reasoningWireFormat === undefined && registryEntry.reasoningWireFormat !== undefined
? { reasoningWireFormat: registryEntry.reasoningWireFormat }
: {}),
Expand Down
9 changes: 9 additions & 0 deletions src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1438,6 +1438,15 @@ export interface OcxProviderConfig {
* fields. Default off; only enable for providers that document this parameter.
*/
promptCacheKey?: boolean;
/**
* Opt-in: forward `service_tier` to the upstream `/chat/completions` body.
* OpenAI-specific extension with the same hazard as `promptCacheKey` — strict backends
* reject unknown fields, and 66 registry providers share the `openai-chat` adapter, so a
* caller-supplied `service_tier` would otherwise turn working requests into upstream 400s.
* `supportsServiceTier` is the Responses-wire flag and does not apply here.
* Default off; only enable for providers that document this parameter on the chat wire.
*/
chatServiceTier?: boolean;
/**
* Provider-local passthrough SSE repair for broken openai-responses gateways that reuse exact
* placeholder message/reasoning ids or omit the terminal id after a stable added event.
Expand Down
31 changes: 31 additions & 0 deletions tests/openai-chat-hardening.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,37 @@ describe("openai-chat credential hardening", () => {
expect(body).not.toHaveProperty("prompt_cache_key");
});

test("preserves a caller-supplied service tier when the provider opts in", () => {
const adapter = createOpenAIChatAdapter(provider({ chatServiceTier: true }));
const req = parsed();
req.options.serviceTier = "priority";

const body = JSON.parse(adapter.buildRequest(req).body);

expect(body.service_tier).toBe("priority");
});

// `service_tier` is an OpenAI-specific extension and this adapter serves 66 registry
// providers, several of which reject unknown body fields. Forwarding it by default would
// turn a caller-supplied tier into an upstream 400 on those routes, so absence of the
// opt-in must mean the field is dropped — the same contract `prompt_cache_key` uses.
test("drops a caller-supplied service tier when the provider has not opted in", () => {
for (const p of [provider(), provider({ chatServiceTier: false })]) {
const req = parsed();
req.options.serviceTier = "priority";

const body = JSON.parse(createOpenAIChatAdapter(p).buildRequest(req).body);

expect(body).not.toHaveProperty("service_tier");
}
});

test("an opted-in provider without a caller tier still sends no service_tier", () => {
const body = JSON.parse(createOpenAIChatAdapter(provider({ chatServiceTier: true })).buildRequest(parsed()).body);

expect(body).not.toHaveProperty("service_tier");
});

test("canonical Kimi Coding Plan routes forward Codex prompt_cache_key", () => {
for (const [providerName, authMode] of [
["kimi", "oauth"],
Expand Down
Loading