test(cache): replay-alias provenance doctrines - #223
Closed
warelik wants to merge 56 commits into
Closed
Conversation
A Gemini part carrying only a thoughtSignature is not a usable completion; the client still needs visible text, a tool call, or positive token usage. This prevents empty responses from being rotated into the next request and masking provider failures.
Route Claude thinking signatures through signature compatibility logic before copying them into Gemini thoughtSignature fields. Compatible signatures are normalized; foreign or opaque ones use the documented Gemini bypass sentinel instead of crashing the Gemini upstream.
Emit Claude signature_delta events as Gemini thoughtSignature carriers and preserve them in the non-stream path, replaying through Gemini compatibility or the bypass sentinel. Prevents signature loss when Claude responses are translated for Gemini.
…tput Mirror the gemini/claude non-stream signature handling for the gemini-cli path: emit signature_delta SSE events and attach the thoughtSignature to non-stream thinking blocks. Keeps cross- provider thinking continuity for the CLI provider.
Preserve the block shape in compat mode while still running the signature through compatibility logic. Foreign or opaque signatures are stripped or emulated; decodable short Claude shapes (e.g. replay cache synthetic signatures) are preserved to avoid breaking compat thinking replay.
The compat-mode fallback that preserves decodable Claude E/R-shaped signatures only applies to unprefixed, non-foreign values now. Foreign provider prefixes (e.g. gemini#...) and signatures detected as foreign no longer slip through the fallback and fail Claude validation. Addresses P1 review on router-for-me/CLIProxyAPI#5150.
- Preserve the required "signature" member on empty compat thinking placeholders instead of deleting it. - Reject any signature containing an unrecognized vendor prefix in the short-signature fallback, not only known foreign prefixes. - Add regression tests for empty placeholder preservation and vendor-prefixed signature rejection.
When a "claude#" prefix is present on a short E/R-shaped signature, strip the prefix and re-run foreign-provider detection on the payload before allowing the fallback. This prevents a genuine Gemini E-prefixed signature mislabeled as "claude#" from being forwarded to Claude. The unprefixed value is emitted when the payload is safe. Adds TestSanitizeClaudeMessagesForClaudeUpstreamStripsMislabeledClaudePrefixInCompatMode.
After stripping a "claude#" prefix, reject the payload if it still contains a '#'. This prevents nested prefixes like "claude#vendor#EgI=" from slipping through: HasDecodableClaudeThinkingSignature would otherwise strip the inner "vendor#" and validate the bare E/R shape, while provider detection returned unknown on the nested payload. Now any residual provider delimiter is an immediate reject. Adds TestSanitizeClaudeMessagesForClaudeUpstreamStripsNestedClaudePrefixInCompatMode.
…ng blocks When compat mode rejects a thinking signature, it now sets the field to an empty string instead of deleting the member. This keeps the block shape intact while matching the placeholder-preservation semantics, and avoids strict Claude-compatible validation rejecting the continuation for a missing signature. Updates compat tests to assert that rejected foreign/Gemini/opaque signatures on non-empty thinking blocks are cleared to an empty string rather than the field being removed.
The short-signature fallback now accepts only signatures that DetectSignatureProviderForBlock identifies as Claude. Unknown or unprovenanced E/R-shaped opaque payloads (e.g. Grok/xAI encrypted_content that happens to base64-encode to 'E' or 'R') no longer pass the fallback. Adds TestSanitizeClaudeMessagesForClaudeUpstreamRejectsGrokOpaqueERInCompatMode and keeps the earlier vendor#/nested regression tests pinned.
…hetic isClaudeReplayableShortSignature now requires either: - a signature DetectSignatureProviderForBlock identifies as Claude, or - a minimal short E-prefixed synthetic used by the thinking replay cache (e.g. "EgI="), where the decoded payload is <= 2 bytes. This rejects longer opaque blobs such as Grok/xAI encrypted_content that happens to base64-encode to 'E' or 'R', while still preserving the known short synthetic shape. Adds TestSanitizeClaudeMessagesForClaudeUpstreamRejectsGrokOpaqueERInCompatMode and keeps the earlier vendor#/nested regression tests pinned.
… gate isClaudeReplayableShortSignature is the final compat fallback and is now only responsible for the 1-2 byte E-prefixed synthetic used by the Claude thinking replay cache (e.g. "EgI="). It no longer calls DetectSignatureProviderForBlock or HasDecodableClaudeThinkingSignature, which are redundant here: longer valid Claude signatures are already preserved by DecideSignatureCompatibilityForModel, and the dead detector branch only added cost for large rejected blobs. isShortClaudeSyntheticSignature rejects any base64 longer than 4 characters before decoding, so it does not allocate when handling multi-kilobyte opaque ciphertext.
…tion, honor cacheControlDisabled - prepareClaudeThinkingReplayRequest now loads cached assistant contents without modifying req.Payload. The restore is applied to bodyForUpstream after sanitizeClaudeMessagesForClaudeUpstreamWithDebug, so cache-provenanced signatures are not cleared by the compat sanitizer. - ClaudeExecutor.Execute and ExecuteStream now skip cache_control placement entirely when the embedder sets cacheControlDisabled (e.g. Kimi reusing the Claude path). - Adds TestClaudeExecutorCompatThinkingReplayRestoresOpaqueOmittedBlock to verify that long opaque replay signatures survive the sanitizer.
…izer Reverted the after-sanitize restore workaround. prepareClaudeThinkingReplayRequest now restores cached assistant content into req.Payload and marks each restored thinking part with _cliproxy_replay_provenance. The compat sanitizer detects that marker, preserves the signature, and strips the marker before sending upstream. Unprovenanced client signatures still pass through Decide and the short-synthetic fallback and are rejected when not recognized.
…ient markers prepareClaudeThinkingReplayRequest now strips client-supplied _cliproxy_replay_provenance from the request, loads cache content, and defers restore until after signature sanitization. The executor then restores before prepareClaudeOAuthToolNamesForUpstream so cached caller-facing tool names match the current body and get remapped for upstream. The compat sanitizer no longer treats _cliproxy_replay_provenance as a trust signal; it removes any incoming marker and subjects the signature to normal compatibility validation. Added TestClaudeExecutorCompatThinkingReplayRestoresBeforeMCPToolNameRemap and TestSanitizeClaudeMessagesForClaudeUpstreamStripsClientReplayProvenanceMarkerInCompatMode.
isShortClaudeSyntheticSignature now returns the trimmed, normalized value so whitespace-padded short synthetics like " EgI= " are forwarded upstream as their canonical form. Callers use the returned normalized string instead of the raw input. Added TestSanitizeClaudeMessagesForClaudeUpstreamNormalizesWhitespacePaddedShortSignatureInCompatMode.
prepareClaudeThinkingReplayRequest now strips tool-use provenance fields from cached assistant content before the replay match. This lets the canonical comparison match an echo'd tool_use even after the upstream sanitizer removes the same fields. The restored content still keeps the trusted thinking signature, and the sanitizer strips any residual tool provenance from the upstream body. Added TestClaudeExecutorCompatThinkingReplayRestoresOmittedThinkingWithToolProvenance.
…ocks restoreKimiThinkingReplayContent now restores cached assistant content even when the current turn contains a thinking block. Non-thinking parts must still match the cached non-thinking parts, and the thinking parts must match after removing signature fields, so a sanitized echoed turn gets its trusted signature back while a modified thinking text does not. Added TestClaudeExecutorCompatThinkingReplayRestoresEchoedSignedThinking.
…de clients claudeThinkingReplayScopeFromRequest now falls back to a credential-scoped session key when the request provides no execution session, X-Claude-Code- Session-Id, prompt cache key, or other session metadata. This preserves same- upstream opaque signatures for standard Claude Messages clients that do not supply session scoping, while still isolating the cache by credential identity through the model family. Added TestClaudeExecutorCompatThinkingReplayRestoresSessionlessSameUpstreamSignature.
…ients claudeThinkingReplayScopeFromRequest no longer falls back to a bare credential+model key for sessionless clients. When no execution session or session header is present, it derives a conversation-scoped key from the first message and the system prompt. This keeps distinct conversations through the same credential isolated while still allowing replay within the same conversation. Added TestClaudeExecutorCompatThinkingReplayIsConversationScopedForSessionlessClients.
…dders stripCacheControls previously only deleted top-level cache_control fields in tools, system, and messages. A Claude tool_result with structured content blocks that themselves carry cache_control would still forward the Anthropic-only field to Kimi and cause rejection. - Make stripCacheControls only target protocol-level cache_control markers on system/tool/message blocks and nested content arrays (e.g. tool_result content). It leaves arbitrary JSON like tool input_schema properties named "cache_control" untouched. - Add stripContentCacheControls to recurse into nested content arrays without wandering into sibling objects such as tool input_schema or tool_use input. - Add TestStripCacheControls and TestStripCacheControls_NestedToolResultContent. Refs: router-for-me/CLIProxyAPI#5154
claudeThinkingReplayConversationSessionKey now mixes caller identity signals into the conversation key: credential id, caller-scope and derived session metadata, and selected headers (User-Agent, X-App, X-Codex-Client-Id). It also includes tools in the content hash. This prevents two sessionless callers sharing a credential and the same first message from colliding on the same replay cache entry. Added TestClaudeExecutorCompatThinkingReplayIsCallerScopedForSessionlessClients.
Moves claudeThinkingReplayConversationSessionKey to helps.ClaudeThinkingReplayConversationSessionKey per the repo layout that places executor support helpers under internal/runtime/executor/helps/. The helper now lives alongside other session/identity helpers and uses a small local canonical JSON routine to avoid an executor import cycle.
ClaudeThinkingReplayConversationSessionKey now writes every caller field (length-prefixed with an 8-byte big-endian length) before hashing. This prevents distinct caller tuples whose concatenated raw values look the same (e.g. auth.ID="ab" vs auth.ID="a" + caller-scope="bc") from producing the same replay key. Added TestClaudeThinkingReplayConversationSessionKey_DelimitsConcatenatedFields.
…eusePooledConnections
When cloak_cache_user_id is false (documented default), injectFakeUserID now uses helps.GenerateRandomFakeUserIDForSession to produce a fresh random device_id each request while keeping the stable session_id from CachedSessionIDRequired, preserving header/body session alignment. cache-user-id:true continues to use CachedUserIDRequired for a deterministic, credential-scoped user_id. Added TestApplyCloaking_DeterministicUserID, TestApplyCloaking_NonCachedUserIDIsRandom, TestInjectFakeUserID_CacheEnabledIsDeterministic, and TestInjectFakeUserID_CacheDisabledIsRandomPerRequest.
…er-id contract Reconciles the cloak_cache_user_id flag with its documented contract: - cache-user-id:false (default) gives a fresh random device_id per request while keeping the stable credential session_id, preserving header/body session alignment and the legacy per-request random identity. - cache-user-id:true produces a deterministic, credential-scoped user_id for operators who want cache warmth. - claudeCredentialSeed falls back to auth.ID/Index/FileName/Label/Provider when apiKey is empty, so distinct custom-header-only credentials get distinct sessions and device IDs. - CachedSessionIDRequired, CachedUserIDRequired, generateFakeUserID, and injectFakeUserID now take *cliproxyauth.Auth. - injectFakeUserID uses CachedUserIDRequired when cache-user-id is true and GenerateRandomFakeUserIDForSession when false. - Add TestApplyCloaking_DeterministicUserID, TestApplyCloaking_NonCachedUserIDIsRandom, TestInjectFakeUserID_CacheEnabledIsDeterministic, TestInjectFakeUserID_CacheDisabledIsRandomPerRequest, and TestCachedSessionIDRequiredDistinctEmptyAPIKeyCredentials.
…sponses, and cloaking - ClaudeThinkingReplayConversationSessionKey no longer hashes tools, so changing or reordering the tools list between turns does not churn the conversation-scoped replay key. - Identity headers (User-Agent, X-App, X-Codex-Client-Id) are now read case-insensitively to avoid missing lowercase SDK-supplied header keys. - kimiThinkingReplayContentIsReplayable caches signed thinking responses even when they do not contain a tool_use block, and kimiNonThinkingContentParts returns a true validity flag (not hasToolUse) so non-tool assistant turns can be restored. - applyCloaking-sensitive-word obfuscation is now applied to cached replay contents before the post-sanitizer restore, so the replay match compares like-for-like obfuscated bytes and restores trusted signatures after sensitive words are masked. Tests added for each scenario.
Unsigned or non-replayable assistant responses no longer clear the Claude-compatible thinking replay cache. Only turns that carry signed thinking are appended; earlier signed turns stay available for later requests that echo them. Added regression test: TestClaudeExecutorCompatThinkingReplayRetainsSignedTurnAfterUnsignedResponse.
restoreClaudeThinkingReplayContents now anchors the match window to the first assistant message present in the incoming request. Cached turns older than the client's oldest echoed assistant message are marked consumed before matching, so compacted or truncated history does not mis-align replayed thinking signatures onto later turns or leak dropped turns. Added regression tests for truncated history and for a leading unsigned assistant with later signed assistants.
…mpaction Sessionless clients that compact older messages change messages.0, which until now recomputed the Claude replay session key and orphaned the cache. The fallback scope now registers a per-caller message hash for every user and assistant message it sees, and resolves the original conversation key from any remaining message in a compacted request. Cached assistant turns are also registered by their client-visible non-thinking shape. This keeps replay state reachable after history truncation. Added TestClaudeExecutorCompatThinkingReplayRetainsScopeAfterHistoryCompaction.
- Home KV: RegisterClaudeThinkingReplayAlias stores a per-message KV entry so different instances can resolve the original conversation scope. ResolveClaudeThinkingReplaySessionKey looks up the same alias keys in home mode. - Local map: aliases are now timestamped, TTL-enforced, and bounded by ClaudeThinkingReplayCacheMaxAliases; expired aliases are purged during the periodic cache cleanup and on registration. - Alias caller hash uses length-prefixed field writes to prevent concatenation collisions (e.g. auth.ID "ab" vs "a" + caller-scope "bc").
…ed-alias scopes - Bound Home KV replay aliases by credential with an LRU-capped per-modelFamily index (ClaudeThinkingReplayCacheMaxAliasesPerCredential). Evicted alias keys are deleted from KV alongside the index. - Store each replay alias as a multi-session list instead of a single mapping so two sessionless conversations that share a visible message do not overwrite each other. - Tag each alias entry with the conversation's first user hash and score alias resolution by message role weight plus a first-user-match bonus. This keeps shared tool_result/user messages from resolving the wrong conversation. - Add internal/cache tests for alias scoring, expiry, home multi-session resolve, and per-credential cap.
…ate turns - Update Home KV replay alias values with compare-and-swap retries so concurrent registrations for the same message hash merge instead of overwriting each other's session list. - Refuse to resolve ambiguous alias ties; a session only wins when it has the unique highest score. This prevents a compacted request with only a shared message from restoring the wrong conversation's hidden thinking. - Anchor replay restore to the latest cached suffix that matches the request's full assistant sequence, so duplicate assistant turns resolve to the correct retained signature after truncation or compaction. - Add regression tests for atomic alias list updates, tie rejection, and duplicate-turn suffix anchoring.
- Key the Home KV alias index by the credential hash embedded in modelFamily instead of the full modelFamily, so the 256-entry per-credential cap is enforced across all model names a caller uses. - Collect evicted alias keys and only delete them after the index KVCompareAndSwap succeeds, preventing a concurrent refresh from being erased before the index update wins. - Add regression tests for cross-model per-credential cap and for eviction remaining atomic with the index CAS.
…iases, recheck evicted aliases - Require an explicit conversation nonce (client_metadata.conversation_id, conversation_id, or X-Conversation-Id) for the sessionless replay fallback so identical conversation openings no longer share one cache. - Roll back committed alias values when the per-credential index update fails, preventing half-registered unindexed aliases. - Re-read the alias index before deleting evicted aliases and skip any that have already been re-registered by a concurrent worker. Tests added/updated for all three behaviors.
rollBackClaudeThinkingReplayAliasHome now treats the value match as the authoritative condition. An unreadable index no longer blocks rollback of an orphan alias; the index is still consulted as a best-effort guard when readable. Add TestClaudeThinkingReplayAliasHomeRollbackConditionalOnCommittedValue to verify rollback succeeds even when the per-credential index cannot be read.
…/caller hash headerFirstValue in both claude_thinking_replay.go and claude_thinking_replay_session.go now trims the header value and treats whitespace-only values as missing. This prevents whitespace-only User-Agent/X-App/X-Codex-Client-Id from churning the replay key and stops whitespace-only X-Conversation-Id from being accepted as a conversation nonce. Tests added in both packages.
registerClaudeThinkingReplayAliasHome now keeps the evicted index record (timestamp) and, before deleting an evicted alias value, re-reads the alias value and skips deletion if it has been repopulated by a concurrent worker (newer session timestamp than the evicted index record). This closes the window where a worker re-registers an alias and updates its value but has not yet updated the index when the eviction pass runs. Add TestClaudeThinkingReplayAliasHomeRechecksEvictedAliasValue to guard the regression.
If the alias value KVCompareAndSwap returns an error, the command may have been applied before the error reached the client. rollBackClaudeThinkingReplayAliasHome is now called with the attempted raw so an orphan value is removed if it matches the last attempted write. Add TestClaudeThinkingReplayAliasHomeRollBackOnFailedRegistration to guard the regression.
…e clients - ClaudeThinkingReplayConversationSessionKey now returns (key, usedNonce). - With a conversation nonce, the key is derived from stable caller fields only, so it survives history compaction and gives identical openings with different nonces distinct scopes. - Without a nonce, it falls back to a content-derived key (first user message + system), preserving replay for stateless clients. - claudeThinkingReplayScopeFromRequest resolves content-derived keys through alias lookup so history compaction does not orphan the replay cache. - Tests added for nonce-stable keys, no-nonce content keys, distinct scopes, and end-to-end no-nonce compaction replay.
- Observe the upstream Claude SSE in ExecuteStream before translation and cache the accumulated content when the message completes, so cross-format (e.g. OpenAI) streams can replay signed thinking turns. - Validate Claude thinking signatures before caching using signature.HasDecodableClaudeThinkingSignature, so only provenanced signed turns are retained and malformed signatures do not evict earlier replay state. - Add end-to-end test for cross-format stream replay and update existing tests to use decodable test signatures.
claudeThinkingReplayFindStartIndex now only accepts a full assistant sequence match. Partial prefixes are rejected, so an unsigned assistant turn that was intentionally not cached no longer causes an earlier cached turn to be restored with the wrong hidden thinking/signature. Add TestClaudeThinkingReplayFindStartIndex_RefusesPartialAnchor.
kimiThinkingReplayStreamAccumulator now recognizes citations_delta events and preserves the citation objects on the cached text block instead of abandoning the stream. This lets cross-format streams that include web-search citations be cached and replayed without losing signed thinking. Add TestKimiThinkingReplayStreamAccumulator_PreservesCitations.
Set fallbackKey only when the conversation session key is content-derived (usedNonce is false). Nonce-based scopes are stable across compaction and do not need alias resolution, so registering every message as an alias for them only wastes the shared alias cap and can evict aliases needed by no-nonce conversations. Add TestClaudeExecutorCompatThinkingReplayNoAliasForNonceScope.
…tle purge scans - Add ClaudeThinkingReplayCacheMaxAliasBytes (64 MiB) and enforce it in enforceClaudeThinkingReplayAliasLimitsLocked. - Maintain claudeThinkingReplayAliasCount so enforce does not rescan the map just to count entries. - Evict the oldest-by-timestamp alias when a per-key list exceeds ClaudeThinkingReplayCacheMaxAliasesPerKey, not the FIFO list[0]. - Throttle purgeExpiredClaudeThinkingReplayAliasesLocked to at most once per minute, so long histories do not scan the full alias map per message. Add tests for byte-limit enforcement and per-key LRU eviction.
…lay streams The replay accumulator in the cross-format stream branch now observes the Claude SSE line after MCP tool names have been restored, so the cached replay matches the caller-visible tool names and the restored opaque thinking/signature is not lost on the next continuation.
claudeThinkingReplayAliasBytes now counts each map key only once and each session/firstUser entry once, instead of adding len(key) on every alias insertion. This prevents the byte counter from inflating on per-key churn and correctly evicts under the 64 MiB cap.
…miss claudeThinkingReplayFindStartIndex now finds the longest cached suffix that matches a contiguous block of assistant turns, so leading/trailing unsigned or new turns do not break an otherwise valid replay anchor. When no cached suffix matches at all, restoreClaudeThinkingReplayContents refuses the fallback match loop instead of restoring individual turns that could pair with the wrong hidden signature.
enforceClaudeThinkingReplayAliasLimitsLocked now deletes aliases by session key identity instead of the pre-mutation slice index, so multiple candidates from the same key in one batch no longer shift indexes and evict the wrong alias.
Pin airouters-11 replay-alias rules in registry tests: (a) compaction-stable scopes, (b) Home-KV per-credential alias cap, (c) atomic alias eviction with index update, (d) identical vs distinct conversation scope determinism. Tests skip with Plus kaitranntt#209 / stock #5150 references when the alias registry behavior is not active (e.g. on main before kaitranntt#209 lands).
warelik
force-pushed
the
ao/airouters-21-replay-alias-doctrine
branch
from
August 22, 2026 05:00
e8e2182 to
ad95d40
Compare
warelik
added a commit
to warelik/CLIProxyAPIPlus
that referenced
this pull request
Aug 22, 2026
warelik
added a commit
to warelik/CLIProxyAPIPlus
that referenced
this pull request
Aug 22, 2026
…es (updated head)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test-only PR pinning airouters-11's replay-alias rules in the alias registry:
Each doctrine test calls
requireReplayAliasSupport()andt.Skips withPlus #209/stock #5150references when the replay-alias registry is not active.This branch was rebuilt to contain only the focused test delta on top of Plus #209 (
ao/airouters-11-fix-signature-boundary-plus), removing the 42-commit/33-file history baggage. Once Plus #209 lands inmain, the PR diff will show onlyinternal/cache/replay_alias_doctrine_test.go.Cross-links
Test plan
go test -run TestReplayAliasDoctrine ./internal/cache(6 passed)