feat: add OpenRouter and Together AI provider support#43
Merged
Conversation
Replace 17+ hardcoded match/case blocks across 5 files with a ProviderDescriptor registry, making new provider addition a 2-3 file operation instead of 5+. Registry refactor: - ProviderDescriptor case class holds all per-provider metadata - ProviderRegistry singleton with @volatile safe publication - ProviderRegistrations centralizes all provider registration - ConfigStore, ProviderFactory, ModelRegistry, LLMExtension, and ReasoningModelDetector now read from the registry OpenAICompatibleProvider base class: - Extracts shared Chat Completions wire format from OpenAIProvider - Three extension hooks: extraHeaders, applyReasoningFields, extractThinking - OpenAIProvider slimmed from 108 to ~25 lines OpenRouter provider: - Vendor-prefixed model names (openai/gpt-4o, anthropic/claude-3.5-sonnet) - HTTP-Referer and X-Title headers for attribution - Unified reasoning object instead of reasoning_effort - Thinking text extraction from response message.reasoning field - 17 models in registry covering OpenAI, Anthropic, Google, Meta, DeepSeek Net result: -264 lines, 29/29 tests passing, fat JAR builds clean.
Extends OpenAICompatibleProvider with Together-specific reasoning and
thinking extraction. Vendor-prefixed model names passed through unchanged.
- TogetherProvider: hybrid reasoning via { reasoning: { enabled: true } }
- DeepSeek-R1 thinking: extracts <think> tags from content field,
falls back to message.reasoning for other models
- 10 models in registry: Llama, DeepSeek, Qwen, Gemma, Mistral
- ReasoningModelDetector updated for together reasoning models
- Provider count now 6 (30/30 tests pass)
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2a1b889811
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| s"Config loaded but $providerName provider requires an API key. Set '$keyName' in config. For help: print llm:provider-help \"$providerName\"" | ||
| ) | ||
| } | ||
| ProviderRegistry.get(providerName.toLowerCase.trim).foreach { desc => |
There was a problem hiding this comment.
llm:load-config now validates with ProviderRegistry.get(...).foreach, so an unrecognized provider= value silently skips readiness checks instead of failing early. In configs where the provider name is mistyped (for example with only provider-specific keys present), loading succeeds and the user only sees an error later on first request when provider creation fails, which regresses the immediate-validation behavior expected from load-config.
Useful? React with 👍 / 👎.
Comment on lines
+73
to
+75
| val content = message("content").str | ||
| val thinkPattern = """(?s)<think>(.*?)</think>""".r | ||
| thinkPattern.findFirstMatchIn(content).map(_.group(1).trim).filter(_.nonEmpty) |
There was a problem hiding this comment.
For Together models that embed reasoning as <think>...</think> in message.content (the exact fallback handled here), this code extracts thinking text but does not remove that block from the returned assistant content. Because OpenAICompatibleProvider.parseProviderResponse uses message("content") as the answer, llm:chat-with-thinking can return reasoning text in both outputs (answer and thinking), leaking chain-of-thought instead of separating it.
Useful? React with 👍 / 👎.
… Together content Addresses two P2 findings from the Codex review on #43. 1. LoadConfigCommand now fails fast when `provider=` is unknown. Previously the .foreach pattern silently skipped validation, deferring the error to the first chat request. Model names still warn-only (preserves flexibility for unlisted models). 2. Add cleanContent hook in OpenAICompatibleProvider (default identity). TogetherProvider overrides it to strip <think>...</think> blocks so llm:chat-with-thinking no longer leaks reasoning into the answer slot for DeepSeek-R1. Tests: 31/31 pass. New LLMLoadConfigRejectsUnknownProvider fixture verifies the typo'd provider name surfaces in the error message.
Second round of fixes for Codex review on #43. 1. Revert the cleanContent hook and Together <think> tag stripping. Faithful pass-through of API responses is more important than cosmetic cleanup. Users who want thinking separated can use llm:chat-with-thinking, which still extracts thinking via the existing extractThinking hook without mutating content. 2. llm:load-config now validates the provider name against the raw config map before touching configStore, and snapshots + rolls back on readiness-check failure. A rejected load no longer leaves the session with a half-loaded config that breaks llm:active. Tests: 31/31 pass.
Rewrite demos/tests/tests.nlogox with linear, readable test procedures that exercise all 17 primitives plus provider-specific behavior: - providers + provider-help registry coverage (6 providers) - invalid-provider rejection (validates f539a73 fix) - config-rollback on bad path (validates d9a33de fix) - sync/async chat, choose, history - thinking-config + chat-with-thinking - openrouter-vendor-prefix (skip-aware: only when openrouter active) - together-thinking (skip-aware: only when together + DeepSeek) - reasoning-marker visibility in list-models Drop heavy `carefully` defensive wrappers — keep them only where asserting an error is the test. Final summary prints "RESULTS: N passed, M failed" via globals. Add demos/tests/config.txt.example as a sanitized template covering all 6 providers with REPLACE_ME placeholders. Real config.txt remains gitignored. Update README to point users at the template.
Bring all top-level docs in line with the data-driven provider registry that now supports 6 providers (openai, anthropic, gemini, ollama, openrouter, together). - README.md: lead description and Quick Links cover all 6 providers - SETUP.md: add OpenRouter and Together AI setup sections, update parameter table and provider-specific defaults; note DeepSeek-R1 thinking-tag behavior + max_tokens guidance - USAGE.md: provider list in `set-provider` reference; add a reasoning/thinking example block (`set-thinking`, `set-reasoning-effort`, `set-thinking-budget`, `chat-with-thinking`); expand discovery examples - API-REFERENCE.md: add quick-table rows for the 4 reasoning primitives, add a full Reasoning / Thinking Primitives section, extend Valid Providers list, readiness-check table, and provider-status example to include OpenRouter and Together - EXAMPLES.md: include OpenRouter and Together in the compare-providers list; add Reasoning Models and OpenRouter one-key-many-models examples - TESTING.md: document `demos/tests/config.txt.example` workflow, enumerate the 13 test procedures (including the two skip-aware provider-specific ones), list OPENROUTER_API_KEY / TOGETHER_API_KEY for future smoke-test secrets
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ProviderRegistryto make future provider additions a 2-3 file operation instead of 5+What changed
Registry refactor (addresses #31 partially)
ProviderDescriptorcase class holds all per-provider metadataProviderRegistrysingleton replaces 17+ hardcoded match/case blocks across 5 filesOpenAICompatibleProviderbase class extracted from OpenAIProvider — shared by OpenAI, OpenRouter, Together AIOpenRouter provider
openai/gpt-4o,anthropic/claude-3.5-sonnet)reasoningobject for thinking modelsmessage.reasoningfieldTogether AI provider
meta-llama/Llama-3.3-70B-Instruct-Turbo){ reasoning: { enabled: true } }<think>tags in contentProvider count: 4 → 6
OpenAI, Anthropic, Gemini, Ollama, OpenRouter, Together AI
Test plan
sbt compile— clean, zero warningssbt test— 30/30 pass (was 28 on main)sbt assembly— fat JAR buildsLLMOpenRouterProvider,LLMTogetherProvider(set provider, default model, help text, set-model)Closes #40, closes #41