Skip to content

fix(translator): use deterministic tool call IDs in Gemini request translators - #194

Open
warelik wants to merge 2 commits into
kaitranntt:mainfrom
warelik:fix/deterministic-tool-call-ids
Open

fix(translator): use deterministic tool call IDs in Gemini request translators#194
warelik wants to merge 2 commits into
kaitranntt:mainfrom
warelik:fix/deterministic-tool-call-ids

Conversation

@warelik

@warelik warelik commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Problem

In Gemini API multi-turn conversations, the client re-sends the entire conversation history in contents on each turn. Gemini functionCall and functionResponse objects do not carry mandatory ID fields.

When translating Gemini requests to Claude (ConvertGeminiRequestToClaude) and Codex (ConvertGeminiRequestToCodex), missing tool/call IDs were generated using random number generators (crypto/rand). Consequently, historical tool calls were assigned different random IDs on every subsequent turn, completely invalidating upstream LLM prompt caches (Anthropic Prompt Caching and OpenAI Prefix Caching) and resulting in a 100% cache miss rate, higher latencies, and increased token costs.

Fix

  • internal/translator/claude/gemini/claude_gemini_request.go:89: Pre-scans payload contents for existing explicit tool IDs and generates deterministic sequential IDs (toolu_1, toolu_2, ...) while avoiding collisions with explicit IDs.
  • internal/translator/codex/gemini/codex_gemini_request.go:92: Pre-scans payload contents for existing explicit call IDs and generates deterministic sequential IDs (call_1, call_2, ...) while avoiding collisions with explicit IDs.

Tests

  • internal/translator/claude/gemini/claude_gemini_request_test.go:
    • TestConvertGeminiRequestToClaude_DeterministicToolIDsAcrossRepeatedTranslations: Verifies identical multi-turn history produces byte-for-byte identical tool IDs across multiple translations.
    • TestConvertGeminiRequestToClaude_ToolIDsUniqueWithinRequest: Verifies tool IDs within a single request are distinct.
    • TestConvertGeminiRequestToClaude_ExplicitIDWinsOverGenerated: Verifies explicit tool IDs in Gemini payload are preserved.
    • TestConvertGeminiRequestToClaude_ExplicitIDCollisionAvoided: Verifies generated IDs skip existing explicit IDs.
    • TestConvertGeminiRequestToClaude_ExplicitIDAfterGeneratedCollisionAvoided: Verifies collision avoidance regardless of payload ordering.
  • internal/translator/codex/gemini/codex_gemini_request_test.go:
    • TestConvertGeminiRequestToCodex_DeterministicCallIDsAcrossRepeatedTranslations: Verifies repeated translation produces identical call IDs.
    • TestConvertGeminiRequestToCodex_CallIDsUniqueWithinRequest: Verifies unique call IDs per request.
    • TestConvertGeminiRequestToCodex_ExplicitIDWinsOverGenerated: Verifies explicit call IDs are preserved.
    • TestConvertGeminiRequestToCodex_ExplicitIDCollisionAvoided: Verifies generated IDs avoid collisions with explicit call IDs.
    • TestConvertGeminiRequestToCodex_ExplicitIDAfterGeneratedCollisionAvoided: Verifies collision avoidance across mixed ordering.

Reverse bite-check

Reverting internal/translator/claude/gemini/claude_gemini_request.go and internal/translator/codex/gemini/codex_gemini_request.go to random ID generation produces immediate test failures across both test suites:

Claude Request Translator:

=== RUN   TestConvertGeminiRequestToClaude_DeterministicToolIDsAcrossRepeatedTranslations
    claude_gemini_request_test.go:250: tool_use IDs are not deterministic across calls:
        out1 calls: [toolu_GBXaebiBnT2nWvOPEvJeYpVD, toolu_UqDNQQjDLrdtAErZSyJsUytF, toolu_DyF6gGM0PQZqX6bwugXOfjXp]
        out2 calls: [toolu_b9hCGM05swi08thosO2D8IvQ, toolu_PHERlbkrnw43EcT5UrylPjqO, toolu_TVPx9Yn9YE2KAp41vQcL58BF]
--- FAIL: TestConvertGeminiRequestToClaude_DeterministicToolIDsAcrossRepeatedTranslations (0.00s)
FAIL
FAIL	github.com/router-for-me/CLIProxyAPI/v7/internal/translator/claude/gemini	0.387s
FAIL

Codex Request Translator:

=== RUN   TestConvertGeminiRequestToCodex_DeterministicCallIDsAcrossRepeatedTranslations
    codex_gemini_request_test.go:165: call_ids are not deterministic across calls:
        out1 calls: [call_pFVszrqhUIaFkXfjRutkkjav, call_YcHreuho35BMa0ITEYB2mmfU, call_at1qfq2g9gP51I0frhMwA4qW]
        out2 calls: [call_dlzqOPHpsgGLMAnibWEn8UHr, call_lbJ2dL232dlm7CNv7y14qUpf, call_i0HJeoEYHsfO2Og7l1aoRdSY]
--- FAIL: TestConvertGeminiRequestToCodex_DeterministicCallIDsAcrossRepeatedTranslations (0.00s)
FAIL
FAIL	github.com/router-for-me/CLIProxyAPI/v7/internal/translator/codex/gemini	0.388s
FAIL

Verification

TMPDIR=/Users/warelik/.cache/gotmp GOCACHE=/Users/warelik/.cache/gocache go build ./...
TMPDIR=/Users/warelik/.cache/gotmp GOCACHE=/Users/warelik/.cache/gocache go vet ./internal/translator/claude/gemini/... ./internal/translator/codex/gemini/...
gofmt -l internal/translator/claude/gemini/claude_gemini_request.go internal/translator/claude/gemini/claude_gemini_request_test.go internal/translator/codex/gemini/codex_gemini_request.go internal/translator/codex/gemini/codex_gemini_request_test.go
TMPDIR=/Users/warelik/.cache/gotmp GOCACHE=/Users/warelik/.cache/gocache go test -v -run "TestConvertGeminiRequestToClaude_DeterministicToolIDsAcrossRepeatedTranslations|TestConvertGeminiRequestToCodex_DeterministicCallIDsAcrossRepeatedTranslations" ./internal/translator/claude/gemini/... ./internal/translator/codex/gemini/...

Output:

=== RUN   TestConvertGeminiRequestToClaude_DeterministicToolIDsAcrossRepeatedTranslations
--- PASS: TestConvertGeminiRequestToClaude_DeterministicToolIDsAcrossRepeatedTranslations (0.00s)
PASS
ok  	github.com/router-for-me/CLIProxyAPI/v7/internal/translator/claude/gemini	0.748s
=== RUN   TestConvertGeminiRequestToCodex_DeterministicCallIDsAcrossRepeatedTranslations
--- PASS: TestConvertGeminiRequestToCodex_DeterministicCallIDsAcrossRepeatedTranslations (0.00s)
PASS
ok  	github.com/router-for-me/CLIProxyAPI/v7/internal/translator/codex/gemini	0.407s

Tooling note

The jbcontext semantic-search CLI is installed but non-functional in this environment
(jbcontext search fails with the OS keychain is not accessible). The equivalent
review passes were performed with the repository's own tooling and manual inspection
instead; this is disclosed for transparency about how the change was reviewed.

W ARELIK added 2 commits August 20, 2026 13:29
…anslators

Generate sequential deterministic tool and call IDs for Gemini requests without explicit IDs to prevent prompt cache misses across multi-turn conversations.

Refs router-for-me/CLIProxyAPI#5107
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant