feat(web-search): opt-in live streaming of routed-model output + Dashboard toggle - #1500
Open
RobinBially wants to merge 11 commits into
Open
feat(web-search): opt-in live streaming of routed-model output + Dashboard toggle#1500RobinBially wants to merge 11 commits into
RobinBially wants to merge 11 commits into
Conversation
With the sidecar engaged, runWithWebSearch buffers every semantic adapter event of an iteration before scanning for web_search calls, so clients see nothing until the turn ends — 6-50s of silence, then the whole answer as one burst, on every routed-model turn (Codex sends the hosted web_search tool on every real turn). New config option webSearchSidecar.streamRoutedModelOutput (default false, behavior unchanged without opt-in): stream each iteration's leading text/thinking deltas live; the live window closes permanently at the first buffer-only event (tool calls above all), so web_search interception stays atomic, live events are exactly the first N passthrough entries, and the terminal replay skips them by count — nothing is delivered twice. Scanner semantics (thinking extraction, forced-answer output check) are unchanged. Verified: bun run test — 11197 pass / 0 fail (691 files); 4 new tests including a gated adapter proving live delivery mid-turn; tsc clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RobinBially
requested review from
Ingwannu,
Wibias and
lidge-jun
as code owners
August 11, 2026 23:02
Contributor
|
✅ Deterministic PR hygiene checks passed. |
Contributor
|
Important Review skippedNo new commits to review since the last review. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe PR adds an opt-in ChangesWeb-search sidecar live streaming
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Dashboard
participant SettingsAPI
participant WebSearchPlan
participant WebSearchLoop
participant SSEClient
Dashboard->>SettingsAPI: Save streamRoutedModelOutput
SettingsAPI->>WebSearchPlan: Persisted streaming setting
WebSearchPlan->>WebSearchLoop: Resolved streaming flag
WebSearchLoop->>SSEClient: Leading allowlisted text/thinking events
WebSearchLoop->>WebSearchLoop: Buffer events for web-search scanning
WebSearchLoop->>SSEClient: Deduplicated replay after tool-call boundary
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
✅ READY
Review readiness checklist
✅ 4/4 boxes ticked. This pull request is already Ready for Review. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@devlog/_plan/260812_websearch_sidecar_live_streaming/000_findings_and_design.md`:
- Around line 42-44: Update the final sentence in the documented accepted
tradeoff to qualify the reasoning-first model claim: state that these models may
avoid pre-search text repetition, while acknowledging that leading
thinking_delta and reasoning events still become client-visible when the option
is enabled.
In `@docs-site/src/content/docs/guides/sidecars.md`:
- Around line 37-43: Update the streamRoutedModelOutput documentation near its
description to explicitly cover Kiro behavior: sidecar commentary may stream
before the terminal event, while only search-decision events remain buffered.
Preserve the existing explanation of the live window, atomic web_search
interception, and terminal replay.
In `@tests/web-search.test.ts`:
- Around line 2191-2198: Update tests/web-search.test.ts lines 2191-2198 to gate
the adapter after “prefix ”, use frameReader.readUntil to confirm that delta
reaches the client before releasing the tool call, and assert the function-call
frame precedes the replayed “suffix”. Update lines 2233-2237 to gate the first
pass after “Let me check. ”, require client receipt before allowing web_search,
and assert the post-search answer is emitted exactly once.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 52d2cbcb-0b07-40b2-a9a6-868a3ab47950
📒 Files selected for processing (7)
devlog/_plan/260812_websearch_sidecar_live_streaming/000_findings_and_design.mddocs-site/src/content/docs/guides/sidecars.mdsrc/server/responses/core.tssrc/types.tssrc/web-search/index.tssrc/web-search/loop.tstests/web-search.test.ts
…ient receipt Address review: both tests previously asserted only final frames, which a fully buffered implementation also produces. They now withhold the tool call until the test has observed the leading delta on the wire (buffered delivery deadlocks the gate), and the tool-boundary test additionally asserts wire order: prefix delta -> function_call item -> suffix delta. Docs: note that Kiro commentary streaming is independent of the new option; devlog: qualify the reasoning-first-model claim (their leading reasoning becomes visible too — that visibility is the point). Verified: bun run test tests/web-search.test.ts — 55 pass / 0 fail; tsc clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Author
|
Addressed all three review findings in 2e9e514:
Re-verified: |
RobinBially
marked this pull request as ready for review
August 11, 2026 23:36
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/web-search.test.ts`:
- Around line 2165-2167: Update both live-delivery tests around releaseToolCall
and releaseWebSearch so their 5-second timeout rejects or otherwise fails the
readUntil wait instead of invoking the release gate. Ensure each release
function is called only after readUntil observes the prefix, preventing buffered
terminal replay from satisfying the test.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 87c79557-80a7-4bdf-a411-f8600dc7c482
📒 Files selected for processing (3)
devlog/_plan/260812_websearch_sidecar_live_streaming/000_findings_and_design.mddocs-site/src/content/docs/guides/sidecars.mdtests/web-search.test.ts
Contributor
|
Tip For best results, initiate chat on the files or code changes.
🧠 Learnings usedYou are interacting with an AI system. |
Contributor
|
|
…ad of releasing the gate The previous 5s guard called the adapter's release function on timeout, so a fully buffered implementation could still pass: the timer opens the gate, the terminal replay delivers the leading delta, and readUntil observes the replayed copy. The deadline now rejects the readUntil wait; the gate opens only after the client has genuinely observed the live delta. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
|
…oggle on the overview page GET/PUT /api/sidecar-settings now carry webSearch.streamRoutedModelOutput (boolean; false is the default and removes the key so config files stay minimal), and the web-search sidecar card on the Dashboard overview gains a "Stream answers live" switch so the option is discoverable without editing config.json. Strings added to all eight locales. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…notes Swept in by a bulk add; the follow-up devlog in 260812_websearch_sidecar_live_streaming stands on its own. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RobinBially
changed the title
RobinBially
added a commit
to RobinBially/opencodex
that referenced
this pull request
Aug 12, 2026
RobinBially
added a commit
to RobinBially/opencodex
that referenced
this pull request
Aug 12, 2026
Contributor
|
|
Contributor
|
Tip For best results, initiate chat on the files or code changes.
🧠 Learnings usedYou are interacting with an AI system. |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@gui/src/i18n/en.ts`:
- Around line 284-285: Update the dash.webSearchStreamHint translation to
explicitly state that only leading text and reasoning stream until the first
buffer-only event, while remaining output is buffered for scanning and terminal
replay; keep the wording consistent with the actual runtime behavior.
In `@gui/src/i18n/ru.ts`:
- Around line 281-282: Update the dash.webSearchStreamHint translation to
clarify that only the initial allowlisted text and reasoning are streamed live,
stopping at the first non-streamable event, while subsequent output is buffered
for scanning and replay; leave dash.webSearchStream unchanged.
In `@tests/sidecar-settings-web-search-stream.test.ts`:
- Around line 70-83: Update the test around putSidecarSettings to call
loadConfig() after each PUT, then assert the reloaded configuration preserves
streamRoutedModelOutput as true after enabling and omits the key after disabling
it. Retain the existing response status and echoed-value assertions.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 4c888ac5-ab19-4fcd-9d05-183fe841f7b0
📒 Files selected for processing (19)
devlog/_plan/260812_websearch_sidecar_live_streaming/000_findings_and_design.mddocs-site/src/content/docs/guides/sidecars.mdgui/src/i18n/de.tsgui/src/i18n/en.tsgui/src/i18n/ja.tsgui/src/i18n/ko.tsgui/src/i18n/ru.tsgui/src/i18n/tr.tsgui/src/i18n/zh-TW.tsgui/src/i18n/zh.tsgui/src/pages/dashboard-overview-sections.tsxgui/src/pages/dashboard-shared.tssrc/server/management/config-routes.tssrc/server/responses/core.tssrc/types.tssrc/web-search/loop.tstests/sidecar-settings-web-search-stream.test.tstests/vision-anthropic.test.tstests/web-search.test.ts
Contributor
|
Tip For best results, initiate chat on the files or code changes.
🧠 Learnings usedYou are interacting with an AI system. |
…rt durable persistence CodeRabbit round 4: the dashboard hint implied the whole turn streams — reworded in all eight locales to say leading text/reasoning streams until the model decides on a tool call, with the rest buffered for search interception. The sidecar-settings test now reloads the config from disk after each PUT and asserts the flag survives (true persists, false removes the key); the fixture gained a schema-valid provider because loadConfig() discards invalid files wholesale, which would have voided the reload assertions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
Tip For best results, initiate chat on the files or code changes.
You are interacting with an AI system. |
Contributor
|
🧠 Learnings used✅ Action performedReview finished.
|
Contributor
|
Tip For best results, initiate chat on the files or code changes.
🧠 Learnings usedYou are interacting with an AI system. |
Contributor
|
Tip For best results, initiate chat on the files or code changes.
🧠 Learnings usedYou are interacting with an AI system. |
Contributor
|
Tip For best results, initiate chat on the files or code changes.
🧠 Learnings usedYou are interacting with an AI system. |
Contributor
|
Tip For best results, initiate chat on the files or code changes.
🧠 Learnings usedYou are interacting with an AI system. |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/sidecar-settings-web-search-stream.test.ts`:
- Around line 33-34: Update the comment near the schema-valid provider setup to
accurately describe loadConfig(): it merges defaults with preserved providers
first, then uses backup/defaults only if the merged configuration fails
validation, rather than discarding every invalid file wholesale.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: c2319d3b-984c-42d6-875c-67cc5f46f236
📒 Files selected for processing (9)
gui/src/i18n/de.tsgui/src/i18n/en.tsgui/src/i18n/ja.tsgui/src/i18n/ko.tsgui/src/i18n/ru.tsgui/src/i18n/tr.tsgui/src/i18n/zh-TW.tsgui/src/i18n/zh.tstests/sidecar-settings-web-search-stream.test.ts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
🧠 Learnings used✅ Action performedReview finished.
|
Contributor
|
Tip For best results, initiate chat on the files or code changes.
🧠 Learnings usedYou are interacting with an AI system. |
Contributor
|
Tip For best results, initiate chat on the files or code changes.
🧠 Learnings usedYou are interacting with an AI system. |
Contributor
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat(web-search): opt-in live streaming of routed-model output (
streamRoutedModelOutput) + Dashboard toggleProblem
With the web-search sidecar engaged (Codex sends a hosted
web_searchtool on every real turn, anda usable ChatGPT credential exists),
runWithWebSearchfully buffers every semantic adapter eventof an iteration before scanning for
web_searchcalls. The client sees nothing until the turnfinishes — on reasoning-heavy turns 6–50 s of silence, then the whole answer as one burst. This is
the "chat doesn't stream through opencodex" experience for every routed model.
Reproduced end-to-end: a byte-identical replay of a captured real
codex execrequest buffers on asidecar-enabled instance and streams on an identical instance without ChatGPT auth; a field bisect
shows removing only the
web_searchtool from the request restores streaming (448 deltas, first at3.5 s), while removing the other hosted tools does not.
Fix
New config option
webSearchSidecar.streamRoutedModelOutput(defaultfalse— behavior unchangedwithout opt-in). When enabled, each iteration's leading output streams live and the live window
closes permanently at the first buffer-only event:
loop.ts:text_delta,thinking_delta,reasoning_raw_delta,thinking_signature,redacted_thinking,kiro_redacted_reasoning—exactly what the sidecar-less path would deliver identically. Anything else (tool calls above
all) closes the window, so the
web_searchinterception decision stays atomic and live eventsare exactly the first N passthrough entries.
streams live again.
extractIterationThinkingand theforced-answer output check ([Bug]: Hosted web-search forced-answer pass accepts a malformed tool call and completes without an assistant message #1001) are unchanged.
Documented tradeoff (docs-site/guides/sidecars.md): text the model emits before deciding to
search — which buffered mode silently drops — becomes visible and may partially repeat in the
post-search answer. Reasoning-first models avoid the text-repetition case (their leading reasoning
deltas become client-visible — that visibility is the point of the option).
Discoverability: Dashboard toggle
A config-file-only flag would be hard to find for anyone hitting "why doesn't chat stream?", so the
option is also surfaced in the UI:
GET/PUT /api/sidecar-settingsnow carrywebSearch.streamRoutedModelOutput(boolean;non-boolean values are rejected with 400;
falseremoves the key so config files stay minimal).switch (strings in all eight locales).
Tests
bun test tests/web-search.test.ts— 55 pass (4 new). The live-delivery tests are gated: thefake adapter withholds its next event until the test has observed the leading delta on the wire,
and the 5 s deadline rejects instead of releasing the gate, so a buffered implementation
fails rather than passing via the terminal replay. Covered: live first delta mid-turn; default
(flag unset) still buffers; window closes at
tool_call_startwith exactly-once, in-order replay(prefix delta → function_call → suffix); search loop delivers pre-search text and final answer
exactly once.
bun test tests/sidecar-settings-web-search-stream.test.ts— 4 pass (new): GET reflection,PUT true/false round-trip with key removal, 400 on non-boolean, PUT without the flag leaves an
enabled value untouched.
bun test tests/web-search-*.test.ts— 78 pass. GUI:cd gui && bun test tests— 764 pass;eslint .clean.bun x tsc --noEmitclean in root and gui (TypeScript 7.0.2).bun run testsuite (after merging currentdev): 11310 pass / 0 fail / 8 skip,699 files, 509 s.
opencode-go/deepseek-v4-flash, identical text-forcing request):without restart via
PUT /api/sidecar-settings.Devlog:
devlog/_plan/260812_websearch_sidecar_live_streaming/000_findings_and_design.mdcarriesthe full investigation (bit-identical-instance comparison, capture, bisect).
🤖 Generated with Claude Code
Review readiness checklist
This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:
All CI tests are green on my local testing.
I pushed my PR to the latest dev commit.
I resolved all correct Codex and CodeRabbit findings.
My PR is ready for review.
Summary by CodeRabbit
New Features
Documentation
Tests