agentHost: preserve BYOK state through tool continuations - #331749
agentHost: preserve BYOK state through tool continuations#331749sgent-epic wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Preserves BYOK provider state through Copilot SDK tool continuations, addressing #329283.
Changes:
- Tracks pending response IDs by session, vendor, and model.
- Recovers matching tool-output continuations without overriding explicit state.
- Adds unit and SDK integration coverage.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
byokLmProxyService.ts |
Implements continuation recovery and scoped state tracking. |
byokLmProxyService.test.ts |
Tests recovery, scoping, explicit IDs, and cleanup. |
copilotByokResponses.integrationTest.ts |
Verifies SDK tool continuation behavior end-to-end. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| } | ||
| } | ||
| if (calls.size) { | ||
| state.set(key, { responseId: result.responseId, calls }); |
There was a problem hiding this comment.
Fixed in e547f44 with a 256-entry recency cap and oldest-entry eviction. Regression coverage verifies resumable continuations are preserved.
98a0a9a to
500a305
Compare
e547f44 to
9886540
Compare
|
Hi Vritant Bhardwaj (@vritant24), I hope you're doing well. Is there any way this could be prioritized? It's a blocker for wider piloting of agents in VS Code at Epic. I'm personally really liking the VS Code environment over other options, and I'd like to enable other folks to give it a try as well. Thanks for your time! |
Summary
previous_response_idand forwarding only the matching tool outputsWhy
The bundled Copilot SDK currently replays the full Responses history after a tool call and omits
previous_response_id. Extension-provided stateful models then receive nostateful_markerand reject the continuation.The integration test reproduces the SDK request shape directly. Before the proxy change, it failed because the second request replayed the message, reasoning, and tool call with no response ID. It now reaches the provider with the original response ID and only the tool output.
Fixes #329283
Testing
npm run compile-clientscripts\test.bat --run src\vs\platform\agentHost\test\node\byokLmProxyService.test.tsscripts\test-integration.bat --run src\vs\platform\agentHost\test\node\providerIntegration\copilotByokResponses.integrationTest.tsgit diff --check upstream/main...HEAD