From 7359a2f788b6366142d572cd4abd35eefbae7666 Mon Sep 17 00:00:00 2001 From: JNK234 Date: Wed, 15 Apr 2026 18:28:21 -0500 Subject: [PATCH 1/6] feat: add OpenRouter provider support with data-driven registry (#40) Replace 17+ hardcoded match/case blocks across 5 files with a ProviderDescriptor registry, making new provider addition a 2-3 file operation instead of 5+. Registry refactor: - ProviderDescriptor case class holds all per-provider metadata - ProviderRegistry singleton with @volatile safe publication - ProviderRegistrations centralizes all provider registration - ConfigStore, ProviderFactory, ModelRegistry, LLMExtension, and ReasoningModelDetector now read from the registry OpenAICompatibleProvider base class: - Extracts shared Chat Completions wire format from OpenAIProvider - Three extension hooks: extraHeaders, applyReasoningFields, extractThinking - OpenAIProvider slimmed from 108 to ~25 lines OpenRouter provider: - Vendor-prefixed model names (openai/gpt-4o, anthropic/claude-3.5-sonnet) - HTTP-Referer and X-Title headers for attribution - Unified reasoning object instead of reasoning_effort - Thinking text extraction from response message.reasoning field - 17 models in registry covering OpenAI, Anthropic, Google, Meta, DeepSeek Net result: -264 lines, 29/29 tests passing, fat JAR builds clean. --- .gitignore | 1 + docs/CONFIGURATION.md | 17 +- docs/PROVIDER-GUIDE.md | 65 +++ src/main/LLMExtension.scala | 166 +++----- src/main/config/ConfigStore.scala | 60 +-- src/main/providers/ModelRegistry.scala | 20 +- .../providers/OpenAICompatibleProvider.scala | 123 ++++++ src/main/providers/OpenAIProvider.scala | 101 +---- src/main/providers/OpenRouterProvider.scala | 67 ++++ src/main/providers/ProviderDescriptor.scala | 51 +++ src/main/providers/ProviderFactory.scala | 373 ++++-------------- .../providers/ProviderRegistrations.scala | 174 ++++++++ src/main/providers/ProviderRegistry.scala | 71 ++++ .../providers/ReasoningModelDetector.scala | 25 +- src/main/resources/config/models.yaml | 29 ++ tests.txt | 15 +- 16 files changed, 790 insertions(+), 568 deletions(-) create mode 100644 src/main/providers/OpenAICompatibleProvider.scala create mode 100644 src/main/providers/OpenRouterProvider.scala create mode 100644 src/main/providers/ProviderDescriptor.scala create mode 100644 src/main/providers/ProviderRegistrations.scala create mode 100644 src/main/providers/ProviderRegistry.scala diff --git a/.gitignore b/.gitignore index 447b3f5..40f86b9 100644 --- a/.gitignore +++ b/.gitignore @@ -62,3 +62,4 @@ __pycache__/ # Config files with real API keys (for local testing) *-config-with-keys* *-with-keys* +.rocketride/ diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 8fbc92c..516a1b2 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -17,7 +17,7 @@ This allows you to: ## Immediate Validation Starting from this version, configuration is validated immediately: - When you call `llm:load-config` or `llm:set-provider`, the extension checks if the provider is ready -- **Cloud providers** (OpenAI, Anthropic, Gemini) require API keys +- **Cloud providers** (OpenAI, Anthropic, Gemini, OpenRouter) require API keys - **Ollama** requires the server to be running and reachable - If validation fails, you get a clear error message with setup instructions - Use `print llm:provider-help "provider-name"` to get detailed setup guidance @@ -28,8 +28,8 @@ Starting from this version, configuration is validated immediately: - No quotes required; avoid trailing spaces around `=`. - **Supported keys**: - Common: `provider`, `model`, `temperature`, `max_tokens`, `timeout_seconds` - - Provider-specific API keys: `openai_api_key`, `anthropic_api_key`, `gemini_api_key` - - Provider-specific base URLs: `openai_base_url`, `anthropic_base_url`, `gemini_base_url`, `ollama_base_url` + - Provider-specific API keys: `openai_api_key`, `anthropic_api_key`, `gemini_api_key`, `openrouter_api_key` + - Provider-specific base URLs: `openai_base_url`, `anthropic_base_url`, `gemini_base_url`, `ollama_base_url`, `openrouter_base_url` - Legacy (still supported): `api_key`, `base_url` (applies to current provider) ## Where to Save the File @@ -75,6 +75,16 @@ max_tokens=2048 timeout_seconds=30 ``` +### OpenRouter (200+ models, one API key) +``` +provider=openrouter +openrouter_api_key=sk-or-REPLACE_ME +model=openai/gpt-4o-mini +temperature=0.7 +max_tokens=1000 +timeout_seconds=30 +``` + ### Local (Ollama, no API key) ``` provider=ollama @@ -92,6 +102,7 @@ You can configure multiple providers at once using provider-specific keys: openai_api_key=sk-REPLACE_ME anthropic_api_key=sk-ant-REPLACE_ME gemini_api_key=REPLACE_ME +openrouter_api_key=sk-or-REPLACE_ME # Set the active provider provider=openai diff --git a/docs/PROVIDER-GUIDE.md b/docs/PROVIDER-GUIDE.md index 3a416fe..5c064a5 100644 --- a/docs/PROVIDER-GUIDE.md +++ b/docs/PROVIDER-GUIDE.md @@ -236,6 +236,71 @@ candidate_count=1 # Default (recommended) # Higher values increase cost but may improve quality ``` +## OpenRouter Configuration + +### API Setup + +1. **Get API Key**: Visit [openrouter.ai/keys](https://openrouter.ai/keys) +2. **Check Usage**: Monitor credits at [openrouter.ai/credits](https://openrouter.ai/credits) +3. **Browse Models**: Explore 200+ models at [openrouter.ai/models](https://openrouter.ai/models) + +### Configuration Parameters + +```ini +# Required Parameters +provider=openrouter +openrouter_api_key=sk-or-your-openrouter-key-here +model=openai/gpt-4o-mini + +# Optional Parameters +openrouter_base_url=https://openrouter.ai/api/v1 +temperature=0.7 +max_tokens=1000 +timeout_seconds=30 +``` + +### Available Models + +Model names use vendor prefixes (`vendor/model-name`): + +| Model | Description | Context | +|-------|-------------|---------| +| `openai/gpt-4o` | OpenAI GPT-4o via OpenRouter | 128K | +| `openai/gpt-4o-mini` | Fast, cost-effective GPT-4 | 128K | +| `openai/o3-mini` | OpenAI reasoning model | 128K | +| `anthropic/claude-3.5-sonnet` | Anthropic Claude 3.5 Sonnet | 200K | +| `anthropic/claude-sonnet-4-20250514` | Latest Claude Sonnet 4 | 200K | +| `google/gemini-2.5-flash` | Google Gemini Flash | 1M | +| `google/gemini-2.5-pro` | Google Gemini Pro | 1M | +| `meta-llama/llama-3.3-70b-instruct` | Meta Llama 3.3 70B | 128K | +| `deepseek/deepseek-r1` | DeepSeek reasoning model | 64K | +| `deepseek/deepseek-chat-v3-0324` | DeepSeek Chat V3 | 64K | + +**Recommended for NetLogo**: `openai/gpt-4o-mini` (fast, cheap, capable) + +### Why OpenRouter? + +- **One API key** for 200+ models from OpenAI, Anthropic, Google, Meta, DeepSeek, and more +- **Automatic fallback**: if one provider is down, OpenRouter routes to another +- **Pay-per-use credits** across all providers +- **Easy model comparison**: switch between vendors without managing separate API keys + +### Hyperparameters + +Same as OpenAI: `temperature`, `max_tokens`, `top_p`. OpenRouter passes these through to the underlying provider. + +### Thinking/Reasoning Models + +OpenRouter supports reasoning across vendors. Set thinking mode and the extension +translates automatically: + +```netlogo +llm:set-provider "openrouter" +llm:set-model "deepseek/deepseek-r1" +llm:set-thinking true +llm:set-reasoning-effort "high" +``` + ## Ollama (Local) Configuration ### Setup Requirements diff --git a/src/main/LLMExtension.scala b/src/main/LLMExtension.scala index 7d9a9af..6c0bfcb 100644 --- a/src/main/LLMExtension.scala +++ b/src/main/LLMExtension.scala @@ -3,7 +3,7 @@ package org.nlogo.extensions.llm import org.nlogo.api._ import org.nlogo.core.{LogoList, Syntax} import org.nlogo.extensions.llm.config.{ConfigLoader, ConfigStore} -import org.nlogo.extensions.llm.providers.{LLMProvider, ProviderFactory, ModelRegistry, OllamaProvider} +import org.nlogo.extensions.llm.providers.{LLMProvider, ProviderFactory, ProviderRegistry, ProviderRegistrations, ModelRegistry, OllamaProvider, ReadinessCheck} import org.nlogo.extensions.llm.models.{ChatMessage, ChatResponse} import scala.collection.mutable.{ArrayBuffer, WeakHashMap} import scala.concurrent.{Await, ExecutionContext, Future} @@ -82,6 +82,9 @@ class LLMExtension extends DefaultClassManager { * Called when the extension is loaded to register primitives */ override def load(manager: PrimitiveManager): Unit = { + // Initialize provider registry before anything else + ProviderRegistrations.registerAll() + // Configuration primitives manager.addPrimitive("set-provider", SetProviderCommand) manager.addPrimitive("set-api-key", SetApiKeyCommand) @@ -189,13 +192,16 @@ class LLMExtension extends DefaultClassManager { } /** - * Check if a provider is ready to use + * Check if a provider is ready to use. + * Uses the readinessCheck from the provider's descriptor. */ private def isProviderReady(providerName: String): Boolean = { - providerName.toLowerCase.trim match { - case "ollama" => isOllamaReachable - case "openai" | "anthropic" | "gemini" => hasApiKey(providerName) - case _ => false + ProviderRegistry.get(providerName) match { + case Some(desc) => desc.readinessCheck match { + case ReadinessCheck.ServerReachable => isOllamaReachable + case ReadinessCheck.ApiKey => hasApiKey(providerName) + } + case None => false } } @@ -287,24 +293,25 @@ class LLMExtension extends DefaultClassManager { // Set provider configStore.set(ConfigStore.PROVIDER, providerName) - // Validate immediately - providerName match { - case "ollama" => - if (!isOllamaReachable) { - val baseUrl = configStore.get(ConfigStore.OLLAMA_BASE_URL) - .orElse(configStore.get(ConfigStore.BASE_URL)) - .getOrElse(ConfigStore.DEFAULT_OLLAMA_BASE_URL) - throw new ExtensionException( - s"Ollama not reachable at $baseUrl. Please start Ollama server or change ollama_base_url. For help: print llm:provider-help \"ollama\"" - ) - } - case _ => - if (!hasApiKey(providerName)) { - val keyName = ConfigStore.getProviderApiKeyName(providerName) - throw new ExtensionException( - s"$providerName provider requires an API key. Set '$keyName' in config or call llm:set-api-key. For help: print llm:provider-help \"$providerName\"" - ) - } + // Validate immediately using descriptor + ProviderRegistry.get(providerName).foreach { desc => + desc.readinessCheck match { + case ReadinessCheck.ServerReachable => + if (!isOllamaReachable) { + val baseUrl = configStore.get(desc.baseUrlConfigKey) + .orElse(configStore.get(ConfigStore.BASE_URL)) + .getOrElse(desc.defaultBaseUrl) + throw new ExtensionException( + s"${desc.displayName} not reachable at $baseUrl. Please start the server or change ${desc.baseUrlConfigKey}. For help: print llm:provider-help \"$providerName\"" + ) + } + case ReadinessCheck.ApiKey => + if (!hasApiKey(providerName)) { + throw new ExtensionException( + s"${desc.displayName} provider requires an API key. Set '${desc.apiKeyConfigKey}' in config or call llm:set-api-key. For help: print llm:provider-help \"$providerName\"" + ) + } + } } // Force re-initialization with new provider @@ -370,25 +377,26 @@ class LLMExtension extends DefaultClassManager { } - // Validate provider after loading config + // Validate provider after loading config using descriptor val providerName = configStore.getOrElse(ConfigStore.PROVIDER, ConfigStore.DEFAULT_PROVIDER) - providerName.toLowerCase.trim match { - case "ollama" => - if (!isOllamaReachable) { - val baseUrl = configStore.get(ConfigStore.OLLAMA_BASE_URL) - .orElse(configStore.get(ConfigStore.BASE_URL)) - .getOrElse(ConfigStore.DEFAULT_OLLAMA_BASE_URL) - throw new ExtensionException( - s"Config loaded but Ollama not reachable at $baseUrl. Please start Ollama server or change ollama_base_url in config. For help: print llm:provider-help \"ollama\"" - ) - } - case _ => - if (!hasApiKey(providerName)) { - val keyName = ConfigStore.getProviderApiKeyName(providerName) - throw new ExtensionException( - s"Config loaded but $providerName provider requires an API key. Set '$keyName' in config. For help: print llm:provider-help \"$providerName\"" - ) - } + ProviderRegistry.get(providerName.toLowerCase.trim).foreach { desc => + desc.readinessCheck match { + case ReadinessCheck.ServerReachable => + if (!isOllamaReachable) { + val baseUrl = configStore.get(desc.baseUrlConfigKey) + .orElse(configStore.get(ConfigStore.BASE_URL)) + .getOrElse(desc.defaultBaseUrl) + throw new ExtensionException( + s"Config loaded but ${desc.displayName} not reachable at $baseUrl. Please start the server or change ${desc.baseUrlConfigKey} in config. For help: print llm:provider-help \"${desc.name}\"" + ) + } + case ReadinessCheck.ApiKey => + if (!hasApiKey(providerName)) { + throw new ExtensionException( + s"Config loaded but ${desc.displayName} provider requires an API key. Set '${desc.apiKeyConfigKey}' in config. For help: print llm:provider-help \"${desc.name}\"" + ) + } + } } currentProvider = None // Force re-initialization with new config @@ -803,80 +811,10 @@ class LLMExtension extends DefaultClassManager { right = List(Syntax.StringType), ret = Syntax.StringType ) - + override def report(args: Array[Argument], context: Context): AnyRef = { val providerName = args(0).getString.toLowerCase.trim - - providerName match { - case "ollama" => - """Ollama Setup Instructions: - | - |1. Install Ollama: - | - Visit https://ollama.ai/download - | - Download and install for your platform - | - |2. Start Ollama server: - | - Open terminal and run: ollama serve - | - Or start Ollama app (it runs in background) - | - |3. Pull a model: - | - Run: ollama pull llama3.2 - | - Or try: ollama pull deepseek-r1:1.5b (smaller) - | - |4. Verify installation: - | - Check llm:provider-status for "reachable: true" - | - |5. Custom server URL: - | - In config: ollama_base_url=http://your-server:11434 - | - Default: http://localhost:11434 - | - |For more models: ollama.ai/library""".stripMargin - - case "openai" => - s"""OpenAI Setup Instructions: - | - |1. Get an API key: - | - Visit https://platform.openai.com/api-keys - | - Create a new API key - | - |2. Set the key: - | - In config file: ${ConfigStore.OPENAI_API_KEY}=sk-your-key-here - | - Or at runtime: llm:set-api-key "sk-your-key-here" - | - |3. Verify: - | - Check llm:provider-status for "has-key: true"""".stripMargin - - case "anthropic" => - s"""Anthropic (Claude) Setup Instructions: - | - |1. Get an API key: - | - Visit https://console.anthropic.com/ - | - Create a new API key - | - |2. Set the key: - | - In config file: ${ConfigStore.ANTHROPIC_API_KEY}=sk-ant-your-key-here - | - Or at runtime: llm:set-api-key "sk-ant-your-key-here" - | - |3. Verify: - | - Check llm:provider-status for "has-key: true"""".stripMargin - - case "gemini" => - s"""Google Gemini Setup Instructions: - | - |1. Get an API key: - | - Visit https://makersuite.google.com/app/apikey - | - Create a new API key - | - |2. Set the key: - | - In config file: ${ConfigStore.GEMINI_API_KEY}=your-key-here - | - Or at runtime: llm:set-api-key "your-key-here" - | - |3. Verify: - | - Check llm:provider-status for "has-key: true"""".stripMargin - - case _ => - s"Unknown provider: $providerName. Supported providers: ${ProviderFactory.getSupportedProviders.mkString(", ")}" - } + ProviderRegistry.helpText(providerName) } } diff --git a/src/main/config/ConfigStore.scala b/src/main/config/ConfigStore.scala index 1aa46ce..2209651 100644 --- a/src/main/config/ConfigStore.scala +++ b/src/main/config/ConfigStore.scala @@ -202,49 +202,23 @@ object ConfigStore { } /** - * Get the provider-specific API key constant name - * - * @param provider Provider name - * @return API key config constant - */ - def getProviderApiKeyName(provider: String): String = { - provider.toLowerCase.trim match { - case "openai" => OPENAI_API_KEY - case "anthropic" => ANTHROPIC_API_KEY - case "gemini" => GEMINI_API_KEY - case _ => API_KEY - } - } - + * Get the provider-specific API key constant name. + * Delegates to ProviderRegistry for registered providers, falls back to generic API_KEY. + */ + def getProviderApiKeyName(provider: String): String = + org.nlogo.extensions.llm.providers.ProviderRegistry.apiKeyConfigKey(provider) + /** - * Get the provider-specific base URL constant name - * - * @param provider Provider name - * @return Base URL config constant - */ - def getProviderBaseUrlName(provider: String): String = { - provider.toLowerCase.trim match { - case "openai" => OPENAI_BASE_URL - case "anthropic" => ANTHROPIC_BASE_URL - case "gemini" => GEMINI_BASE_URL - case "ollama" => OLLAMA_BASE_URL - case _ => BASE_URL - } - } - + * Get the provider-specific base URL constant name. + * Delegates to ProviderRegistry for registered providers, falls back to generic BASE_URL. + */ + def getProviderBaseUrlName(provider: String): String = + org.nlogo.extensions.llm.providers.ProviderRegistry.baseUrlConfigKey(provider) + /** - * Get default base URL for a provider - * - * @param provider Provider name - * @return Default base URL - */ - def getDefaultBaseUrl(provider: String): String = { - provider.toLowerCase.trim match { - case "openai" => DEFAULT_OPENAI_BASE_URL - case "anthropic" => DEFAULT_ANTHROPIC_BASE_URL - case "gemini" => DEFAULT_GEMINI_BASE_URL - case "ollama" => DEFAULT_OLLAMA_BASE_URL - case _ => "" - } - } + * Get default base URL for a provider. + * Delegates to ProviderRegistry for registered providers. + */ + def getDefaultBaseUrl(provider: String): String = + org.nlogo.extensions.llm.providers.ProviderRegistry.defaultBaseUrl(provider) } diff --git a/src/main/providers/ModelRegistry.scala b/src/main/providers/ModelRegistry.scala index afe7baf..8b16965 100644 --- a/src/main/providers/ModelRegistry.scala +++ b/src/main/providers/ModelRegistry.scala @@ -35,7 +35,8 @@ object ModelRegistry { "claude-3-5-haiku-20241022", "claude-3-5-haiku-latest" ), isCustom = false), "gemini" -> ProviderModels(Set("gemini-1.5-pro", "gemini-1.5-flash", "gemini-2.0-flash-exp"), isCustom = false), - "ollama" -> ProviderModels(Set("llama3.2", "llama3.1", "mistral", "phi4"), isCustom = false) + "ollama" -> ProviderModels(Set("llama3.2", "llama3.1", "mistral", "phi4"), isCustom = false), + "openrouter" -> ProviderModels(Set("openai/gpt-4o", "openai/gpt-4o-mini", "anthropic/claude-3.5-sonnet", "deepseek/deepseek-r1"), isCustom = false) ) /** @@ -201,23 +202,16 @@ object ModelRegistry { } /** - * Get default model for a provider + * Get default model for a provider. * - * These defaults are hardcoded as they represent stable, recommended models. - * They do not change based on YAML configuration. + * Delegates to ProviderRegistry so defaults are defined in one place + * (ProviderRegistrations.scala) rather than duplicated here. * * @param providerName Provider name (case-insensitive) * @return Default model name */ - def defaultModel(providerName: String): String = { - providerName.toLowerCase.trim match { - case "openai" => "gpt-4o-mini" - case "anthropic" => "claude-3-5-haiku-latest" - case "gemini" => "gemini-1.5-flash" - case "ollama" => "llama3.2" - case _ => throw new IllegalArgumentException(s"Unknown provider: $providerName") - } - } + def defaultModel(providerName: String): String = + ProviderRegistry.defaultModel(providerName) /** * Check if a model is valid for a provider diff --git a/src/main/providers/OpenAICompatibleProvider.scala b/src/main/providers/OpenAICompatibleProvider.scala new file mode 100644 index 0000000..dbf18f9 --- /dev/null +++ b/src/main/providers/OpenAICompatibleProvider.scala @@ -0,0 +1,123 @@ +// ABOUTME: Abstract base for providers using OpenAI-compatible Chat Completions API +// ABOUTME: Shared by OpenAI, OpenRouter, and Together AI — subclasses override hooks for headers, reasoning, and thinking +package org.nlogo.extensions.llm.providers + +import org.nlogo.extensions.llm.models.{ChatMessage, ChatRequest, ChatResponse, Choice} +import org.nlogo.extensions.llm.config.ConfigStore +import sttp.client4._ +import sttp.model.Uri +import ujson._ +import scala.concurrent.ExecutionContext + +/** + * Abstract base for providers that use the OpenAI Chat Completions wire format. + * + * Handles the standard /chat/completions request/response. Subclasses override + * three hooks for provider-specific behavior: + * + * - extraHeaders: additional HTTP headers (e.g. OpenRouter's HTTP-Referer) + * - applyReasoningFields: how thinking/reasoning config maps to the request body + * - extractThinking: how to extract thinking text from the response + */ +abstract class OpenAICompatibleProvider(implicit ec: ExecutionContext) extends BaseHttpProvider { + + /** Additional headers to merge into every request. Override in subclasses. */ + protected def extraHeaders: Map[String, String] = Map.empty + + /** + * Apply reasoning/thinking fields to the request body. + * Default implementation uses OpenAI's reasoning_effort at the top level. + * OpenRouter overrides this to use the { reasoning: { effort: ... } } object. + */ + protected def applyReasoningFields(baseObj: ujson.Obj, request: ChatRequest): Unit = { + request.thinkingConfig.flatMap(_.reasoningEffort).foreach { effort => + baseObj("reasoning_effort") = effort + } + } + + /** + * Extract thinking/reasoning text from a response message object. + * Default returns None (OpenAI hides reasoning tokens). + * OpenRouter overrides to read the "reasoning" field. + */ + protected def extractThinking(message: ujson.Value): Option[String] = None + + override protected def buildApiUrl(baseUrl: String): Uri = + uri"$baseUrl/chat/completions" + + override protected def buildHeaders(apiKey: Option[String]): Map[String, String] = + Map( + "Authorization" -> s"Bearer ${apiKey.get}", + "Content-Type" -> "application/json" + ) ++ extraHeaders + + override protected def createProviderRequest(request: ChatRequest): ujson.Value = { + val isReasoning = request.thinkingConfig.exists(_.enabled) + + val messages = ujson.Arr( + request.messages.map { msg => + // For reasoning models, convert system role to developer role + val role = if (isReasoning && msg.role == "system") "developer" else msg.role + ujson.Obj( + "role" -> role, + "content" -> msg.content + ) + }* + ) + + val baseRequest = ujson.Obj( + "model" -> request.model, + "messages" -> messages + ) + + if (isReasoning) { + // Reasoning models: use max_completion_tokens, no temperature + request.maxTokens.foreach { maxTokens => + baseRequest("max_completion_tokens") = maxTokens + } + // Apply provider-specific reasoning fields + applyReasoningFields(baseRequest, request) + } else { + // Standard models: use max_tokens and temperature + request.maxTokens.foreach { maxTokens => + baseRequest("max_tokens") = maxTokens + } + request.temperature.foreach { temp => + baseRequest("temperature") = temp + } + } + + baseRequest + } + + override protected def parseProviderResponse(responseBody: String, model: String): ChatResponse = { + try { + val parsed = ujson.read(responseBody) + + val id = parsed("id").str + val created = parsed("created").num.toLong + val choices = parsed("choices").arr.zipWithIndex.map { case (choice, index) => + val message = choice("message") + val role = message("role").str + val content = message("content").str + val finishReason = choice("finish_reason").str + + Choice( + index = index, + message = ChatMessage(role, content), + finishReason = finishReason + ) + }.toArray + + // Extract thinking text via the provider-specific hook + val thinking = parsed("choices").arr.headOption + .map(_("message")) + .flatMap(extractThinking) + + ChatResponse(id, created, model, choices, thinking) + } catch { + case e: Exception => + throw new RuntimeException(s"Failed to parse ${providerName} response: ${e.getMessage}\nResponse: $responseBody", e) + } + } +} diff --git a/src/main/providers/OpenAIProvider.scala b/src/main/providers/OpenAIProvider.scala index 52f8f68..326c0b9 100644 --- a/src/main/providers/OpenAIProvider.scala +++ b/src/main/providers/OpenAIProvider.scala @@ -1,107 +1,28 @@ // ABOUTME: OpenAI provider implementation for GPT models -// ABOUTME: Extends BaseHttpProvider with OpenAI-specific request/response formatting - +// ABOUTME: Extends OpenAICompatibleProvider — all request/response logic is inherited package org.nlogo.extensions.llm.providers -import org.nlogo.extensions.llm.models.{ChatMessage, ChatRequest, ChatResponse, Choice} -import org.nlogo.extensions.llm.config.ConfigStore -import sttp.client4._ -import sttp.model.Uri -import ujson._ import scala.concurrent.ExecutionContext /** - * OpenAI provider implementation for GPT models + * OpenAI provider implementation. + * + * Inherits all Chat Completions wire format from OpenAICompatibleProvider. + * Only declares provider-specific config values. */ -class OpenAIProvider(implicit ec: ExecutionContext) extends BaseHttpProvider { +class OpenAIProvider(implicit ec: ExecutionContext) extends OpenAICompatibleProvider { override def providerName: String = "openai" - override def defaultModel: String = ConfigStore.DEFAULT_OPENAI_MODEL + override def defaultModel: String = "gpt-4o-mini" - override protected def defaultBaseUrl: String = ConfigStore.DEFAULT_OPENAI_BASE_URL + override protected def defaultBaseUrl: String = "https://api.openai.com/v1" - override protected def baseUrlConfigKey: String = ConfigStore.OPENAI_BASE_URL + override protected def baseUrlConfigKey: String = "openai_base_url" - override protected def apiKeyConfigKey: String = ConfigStore.OPENAI_API_KEY + override protected def apiKeyConfigKey: String = "openai_api_key" - override protected def defaultMaxTokens: String = ConfigStore.DEFAULT_MAX_TOKENS + override protected def defaultMaxTokens: String = "1000" override protected def requiresApiKey: Boolean = true - - override protected def buildApiUrl(baseUrl: String): Uri = { - uri"$baseUrl/chat/completions" - } - - override protected def buildHeaders(apiKey: Option[String]): Map[String, String] = Map( - "Authorization" -> s"Bearer ${apiKey.get}", - "Content-Type" -> "application/json" - ) - - override protected def createProviderRequest(request: ChatRequest): ujson.Value = { - val isReasoning = request.thinkingConfig.exists(_.enabled) - - val messages = ujson.Arr( - request.messages.map { msg => - // For reasoning models, convert system role to developer role - val role = if (isReasoning && msg.role == "system") "developer" else msg.role - ujson.Obj( - "role" -> role, - "content" -> msg.content - ) - }* - ) - - val baseRequest = ujson.Obj( - "model" -> request.model, - "messages" -> messages - ) - - if (isReasoning) { - // Reasoning models: use max_completion_tokens, no temperature - request.maxTokens.foreach { maxTokens => - baseRequest("max_completion_tokens") = maxTokens - } - // Add reasoning_effort if specified - request.thinkingConfig.flatMap(_.reasoningEffort).foreach { effort => - baseRequest("reasoning_effort") = effort - } - } else { - // Standard models: use max_tokens and temperature - request.maxTokens.foreach { maxTokens => - baseRequest("max_tokens") = maxTokens - } - request.temperature.foreach { temp => - baseRequest("temperature") = temp - } - } - - baseRequest - } - - override protected def parseProviderResponse(responseBody: String, model: String): ChatResponse = { - try { - val parsed = ujson.read(responseBody) - - val id = parsed("id").str - val created = parsed("created").num.toLong - val choices = parsed("choices").arr.zipWithIndex.map { case (choice, index) => - val message = choice("message") - val role = message("role").str - val content = message("content").str - val finishReason = choice("finish_reason").str - - Choice( - index = index, - message = ChatMessage(role, content), - finishReason = finishReason - ) - }.toArray - - ChatResponse(id, created, model, choices) - } catch { - case e: Exception => - throw new RuntimeException(s"Failed to parse OpenAI response: ${e.getMessage}\nResponse: $responseBody", e) - } - } } diff --git a/src/main/providers/OpenRouterProvider.scala b/src/main/providers/OpenRouterProvider.scala new file mode 100644 index 0000000..6f6dd75 --- /dev/null +++ b/src/main/providers/OpenRouterProvider.scala @@ -0,0 +1,67 @@ +// ABOUTME: OpenRouter provider — access 200+ models through one API key +// ABOUTME: Extends OpenAICompatibleProvider with OpenRouter-specific headers, reasoning format, and thinking extraction +package org.nlogo.extensions.llm.providers + +import org.nlogo.extensions.llm.models.ChatRequest +import scala.concurrent.ExecutionContext + +/** + * OpenRouter provider implementation. + * + * OpenRouter proxies to 200+ models from multiple vendors (OpenAI, Anthropic, + * Google, Meta, DeepSeek, etc.) using an OpenAI-compatible API format. + * + * Key differences from direct OpenAI: + * - Model names are vendor-prefixed (e.g. "openai/gpt-4o", "anthropic/claude-3.5-sonnet") + * - Extra headers: HTTP-Referer and X-Title for attribution + * - Reasoning uses { reasoning: { effort: "..." } } instead of top-level reasoning_effort + * - Thinking text is exposed in the response "reasoning" field + */ +class OpenRouterProvider(implicit ec: ExecutionContext) extends OpenAICompatibleProvider { + + override def providerName: String = "openrouter" + + override def defaultModel: String = "openai/gpt-4o-mini" + + override protected def defaultBaseUrl: String = "https://openrouter.ai/api/v1" + + override protected def baseUrlConfigKey: String = "openrouter_base_url" + + override protected def apiKeyConfigKey: String = "openrouter_api_key" + + override protected def defaultMaxTokens: String = "1000" + + override protected def requiresApiKey: Boolean = true + + /** OpenRouter recommends HTTP-Referer and X-Title for attribution/ranking. */ + override protected def extraHeaders: Map[String, String] = Map( + "HTTP-Referer" -> "https://ccl.northwestern.edu/netlogo/", + "X-Title" -> "NetLogo LLM Extension" + ) + + /** + * OpenRouter uses a unified reasoning object: + * { "reasoning": { "effort": "high" } } + * instead of OpenAI's top-level reasoning_effort. + */ + override protected def applyReasoningFields(baseObj: ujson.Obj, request: ChatRequest): Unit = { + request.thinkingConfig.flatMap(_.reasoningEffort).foreach { effort => + baseObj("reasoning") = ujson.Obj("effort" -> effort) + } + } + + /** + * OpenRouter exposes thinking text in the response message's "reasoning" field + * for models that support it (Anthropic, DeepSeek, etc.). + */ + override protected def extractThinking(message: ujson.Value): Option[String] = { + try { + message.obj.get("reasoning").flatMap { v => + val text = v.str.trim + if (text.nonEmpty) Some(text) else None + } + } catch { + case _: Exception => None + } + } +} diff --git a/src/main/providers/ProviderDescriptor.scala b/src/main/providers/ProviderDescriptor.scala new file mode 100644 index 0000000..a9dee6c --- /dev/null +++ b/src/main/providers/ProviderDescriptor.scala @@ -0,0 +1,51 @@ +// ABOUTME: Data class describing a provider's metadata, config keys, defaults, and factory +// ABOUTME: Replaces scattered hardcoded match/case blocks with a single registration point per provider +package org.nlogo.extensions.llm.providers + +import scala.concurrent.ExecutionContext + +/** + * Describes how a provider should be ready-checked. + * ApiKey: check that the provider-specific API key is set and non-empty. + * ServerReachable: check that the provider's server responds (e.g. Ollama). + */ +enum ReadinessCheck: + case ApiKey + case ServerReachable + +/** + * Immutable descriptor holding all metadata for a single LLM provider. + * + * One descriptor is registered per provider at extension load time. + * Consumers (ProviderFactory, LLMExtension, ModelRegistry, etc.) read + * from the registry instead of maintaining per-provider match/case blocks. + * + * @param name Lowercase canonical name (e.g. "openai", "openrouter") + * @param displayName Human-readable name for UI/help text + * @param apiKeyConfigKey Config key for this provider's API key (e.g. "openai_api_key") + * @param baseUrlConfigKey Config key for this provider's base URL (e.g. "openai_base_url") + * @param defaultBaseUrl Default API base URL + * @param defaultModel Default model name + * @param defaultMaxTokens Default max_tokens value as string + * @param requiresApiKey Whether an API key is mandatory + * @param apiKeyPrefix Optional prefix for API key validation hint (e.g. Some("sk-")) + * @param readinessCheck How to determine if the provider is ready to use + * @param exposesThinking Whether thinking/reasoning text appears in API responses + * @param helpText Multi-line setup instructions shown by llm:provider-help + * @param factory Function that creates a new LLMProvider instance given an ExecutionContext + */ +case class ProviderDescriptor( + name: String, + displayName: String, + apiKeyConfigKey: String, + baseUrlConfigKey: String, + defaultBaseUrl: String, + defaultModel: String, + defaultMaxTokens: String, + requiresApiKey: Boolean, + apiKeyPrefix: Option[String], + readinessCheck: ReadinessCheck, + exposesThinking: Boolean, + helpText: String, + factory: ExecutionContext => LLMProvider +) diff --git a/src/main/providers/ProviderFactory.scala b/src/main/providers/ProviderFactory.scala index 5a2ce00..c722df1 100644 --- a/src/main/providers/ProviderFactory.scala +++ b/src/main/providers/ProviderFactory.scala @@ -1,3 +1,5 @@ +// ABOUTME: Factory for creating LLM provider instances from the ProviderRegistry +// ABOUTME: Delegates creation, validation, and defaults to ProviderDescriptor — no per-provider match/case package org.nlogo.extensions.llm.providers import org.nlogo.extensions.llm.config.ConfigStore @@ -5,63 +7,34 @@ import scala.concurrent.ExecutionContext import scala.util.{Try, Success, Failure} /** - * Factory for creating LLM provider instances + * Factory for creating LLM provider instances. * - * This factory implements the Factory pattern to create provider instances - * based on configuration. It supports easy extension for new providers. + * All provider-specific knowledge lives in ProviderDescriptor and the + * ProviderRegistry. This factory provides creation, validation, and + * default-config methods that delegate to the registry. */ object ProviderFactory { - // Supported provider names - val OPENAI = "openai" - val ANTHROPIC = "anthropic" - val GEMINI = "gemini" - val OLLAMA = "ollama" - - // Set of all supported providers - val SUPPORTED_PROVIDERS: Set[String] = Set(OPENAI, ANTHROPIC, GEMINI, OLLAMA) - /** - * Create a provider instance by name - * - * @param providerName Name of the provider to create - * @param ec ExecutionContext for async operations - * @return Try containing the provider instance + * Create a provider instance by name. */ def createProvider(providerName: String)(implicit ec: ExecutionContext): Try[LLMProvider] = { val normalizedName = providerName.toLowerCase.trim - normalizedName match { - case OPENAI => - Try(new OpenAIProvider()) - - case ANTHROPIC => - Try(new ClaudeProvider()) - - case GEMINI => - Try(new GeminiProvider()) - - case OLLAMA => - Try(new OllamaProvider()) - - case unknown => + ProviderRegistry.get(normalizedName) match { + case Some(desc) => Try(desc.factory(ec)) + case None => Failure(new IllegalArgumentException( - s"Unknown provider: '$unknown'. Supported providers: ${SUPPORTED_PROVIDERS.mkString(", ")}" + s"Unknown provider: '$normalizedName'. Supported providers: ${ProviderRegistry.allNames.mkString(", ")}" )) } } /** - * Create a provider instance with configuration - * - * @param providerName Name of the provider to create - * @param config Configuration map to apply to the provider - * @param ec ExecutionContext for async operations - * @return Try containing the configured provider instance + * Create a provider instance with configuration. */ def createProvider(providerName: String, config: Map[String, String])(implicit ec: ExecutionContext): Try[LLMProvider] = { createProvider(providerName).map { provider => - // Apply configuration to the provider config.foreach { case (key, value) => provider.setConfig(key, value) } @@ -70,305 +43,115 @@ object ProviderFactory { } /** - * Create a provider from a ConfigStore - * - * @param configStore ConfigStore containing provider configuration - * @param ec ExecutionContext for async operations - * @return Try containing the configured provider instance + * Create a provider from a ConfigStore. */ def createProviderFromConfig(configStore: ConfigStore)(implicit ec: ExecutionContext): Try[LLMProvider] = { val providerName = configStore.getOrElse(ConfigStore.PROVIDER, ConfigStore.DEFAULT_PROVIDER) val config = configStore.toMap - createProvider(providerName, config) } /** - * Validate that a provider name is supported - * - * @param providerName Name to validate - * @return true if the provider is supported + * Validate that a provider name is supported. */ - def isSupported(providerName: String): Boolean = { - SUPPORTED_PROVIDERS.contains(providerName.toLowerCase.trim) - } + def isSupported(providerName: String): Boolean = + ProviderRegistry.isSupported(providerName) /** - * Get list of supported provider names - * - * @return Set of supported provider names + * Get set of supported provider names. */ - def getSupportedProviders: Set[String] = SUPPORTED_PROVIDERS + def getSupportedProviders: Set[String] = ProviderRegistry.allNames /** - * Get list of currently implemented providers - * - * @return Set of implemented provider names + * Get set of currently implemented providers. */ - def getImplementedProviders: Set[String] = Set(OPENAI, ANTHROPIC, GEMINI, OLLAMA) + def getImplementedProviders: Set[String] = ProviderRegistry.allNames /** - * Check if a provider is implemented - * - * @param providerName Name to check - * @return true if the provider is implemented + * Check if a provider is implemented. */ - def isImplemented(providerName: String): Boolean = { - getImplementedProviders.contains(providerName.toLowerCase.trim) - } + def isImplemented(providerName: String): Boolean = + ProviderRegistry.isSupported(providerName) /** - * Validate provider configuration + * Validate provider configuration. * - * @param providerName Provider name - * @param config Configuration map - * @return Try[Unit] - Success if valid, Failure with error if invalid + * Generic validation using descriptor metadata: + * - Checks API key presence for providers that require it + * - Optionally validates API key prefix + * - Warns if model is not in the known registry (but allows it) */ def validateProviderConfig(providerName: String, config: Map[String, String]): Try[Unit] = { - if (!isSupported(providerName)) { - return Failure(new IllegalArgumentException( - s"Unsupported provider: '$providerName'. Supported: ${SUPPORTED_PROVIDERS.mkString(", ")}" - )) - } - - if (!isImplemented(providerName)) { - return Failure(new UnsupportedOperationException( - s"Provider '$providerName' is planned but not yet implemented" - )) - } - - // Provider-specific validation - providerName.toLowerCase.trim match { - case OPENAI => - validateOpenAIConfig(config) - case ANTHROPIC => - validateClaudeConfig(config) - case GEMINI => - validateGeminiConfig(config) - case OLLAMA => - validateOllamaConfig(config) - case _ => - Success(()) // Should not reach here given earlier validation - } - } - - /** - * Validate OpenAI-specific configuration - */ - private def validateOpenAIConfig(config: Map[String, String]): Try[Unit] = { - // Check for provider-specific key first, then generic key - val apiKey = config.get(ConfigStore.OPENAI_API_KEY).orElse(config.get(ConfigStore.API_KEY)) - - if (apiKey.isEmpty) { - return Failure(new IllegalArgumentException( - s"OpenAI provider requires an API key. Set '${ConfigStore.OPENAI_API_KEY}' in config or call llm:set-api-key" - )) - } - - val key = apiKey.get - if (key.trim.isEmpty) { - return Failure(new IllegalArgumentException("OpenAI API key cannot be empty")) - } - - if (!key.startsWith("sk-")) { - return Failure(new IllegalArgumentException("OpenAI API key should start with 'sk-'")) - } - - // Warn if model is not in the known list, but allow it anyway - config.get(ConfigStore.MODEL) match { - case Some(model) => - if (!isValidOpenAIModel(model)) { - System.err.println(s"WARNING: Model '$model' is not in the known model list for 'openai'. It will be used anyway — if the model name is wrong, the API will return an error.") - } - case None => // Model is optional, will use default - } - - Success(()) - } - - /** - * Validate Claude-specific configuration - */ - private def validateClaudeConfig(config: Map[String, String]): Try[Unit] = { - // Check for provider-specific key first, then generic key - val apiKey = config.get(ConfigStore.ANTHROPIC_API_KEY).orElse(config.get(ConfigStore.API_KEY)) - - if (apiKey.isEmpty) { - return Failure(new IllegalArgumentException( - s"Anthropic provider requires an API key. Set '${ConfigStore.ANTHROPIC_API_KEY}' in config or call llm:set-api-key" - )) - } - - val key = apiKey.get - if (key.trim.isEmpty) { - return Failure(new IllegalArgumentException("Anthropic API key cannot be empty")) - } - - // Warn if model is not in the known list, but allow it anyway - config.get(ConfigStore.MODEL) match { - case Some(model) => - if (!isValidClaudeModel(model)) { - System.err.println(s"WARNING: Model '$model' is not in the known model list for 'anthropic'. It will be used anyway — if the model name is wrong, the API will return an error.") - } - case None => // Model is optional, will use default - } - - Success(()) - } - - /** - * Validate Gemini-specific configuration - */ - private def validateGeminiConfig(config: Map[String, String]): Try[Unit] = { - // Check for provider-specific key first, then generic key - val apiKey = config.get(ConfigStore.GEMINI_API_KEY).orElse(config.get(ConfigStore.API_KEY)) - - if (apiKey.isEmpty) { - return Failure(new IllegalArgumentException( - s"Gemini provider requires an API key. Set '${ConfigStore.GEMINI_API_KEY}' in config or call llm:set-api-key" - )) - } + val normalizedName = providerName.toLowerCase.trim - val key = apiKey.get - if (key.trim.isEmpty) { - return Failure(new IllegalArgumentException("Gemini API key cannot be empty")) - } + ProviderRegistry.get(normalizedName) match { + case None => + Failure(new IllegalArgumentException( + s"Unsupported provider: '$normalizedName'. Supported: ${ProviderRegistry.allNames.mkString(", ")}" + )) - // Warn if model is not in the known list, but allow it anyway - config.get(ConfigStore.MODEL) match { - case Some(model) => - if (!isValidGeminiModel(model)) { - System.err.println(s"WARNING: Model '$model' is not in the known model list for 'gemini'. It will be used anyway — if the model name is wrong, the API will return an error.") + case Some(desc) => + val keyValidation: Try[Unit] = if (desc.requiresApiKey) { + val apiKey = config.get(desc.apiKeyConfigKey).orElse(config.get(ConfigStore.API_KEY)) + + if (apiKey.isEmpty) { + Failure(new IllegalArgumentException( + s"${desc.displayName} provider requires an API key. Set '${desc.apiKeyConfigKey}' in config or call llm:set-api-key" + )) + } else if (apiKey.get.trim.isEmpty) { + Failure(new IllegalArgumentException( + s"${desc.displayName} API key cannot be empty" + )) + } else { + // Validate API key prefix if specified + desc.apiKeyPrefix match { + case Some(prefix) if !apiKey.get.startsWith(prefix) => + Failure(new IllegalArgumentException( + s"${desc.displayName} API key should start with '$prefix'" + )) + case _ => Success(()) + } + } + } else { + Success(()) } - case None => // Model is optional, will use default - } - Success(()) - } - - /** - * Validate Ollama-specific configuration - */ - private def validateOllamaConfig(config: Map[String, String]): Try[Unit] = { - // Ollama typically doesn't require API key, just base URL - - // Warn if model is not in the known list, but allow it anyway - config.get(ConfigStore.MODEL) match { - case Some(model) => - if (!isValidOllamaModel(model)) { - System.err.println(s"WARNING: Model '$model' is not in the known model list for 'ollama'. It will be used anyway — if the model name is wrong, the API will return an error.") + keyValidation.map { _ => + // Warn if model is not in the known list, but allow it anyway + config.get(ConfigStore.MODEL).foreach { model => + if (!ModelRegistry.isValidModel(normalizedName, model)) { + System.err.println( + s"WARNING: Model '$model' is not in the known model list for '$normalizedName'. " + + "It will be used anyway — if the model name is wrong, the API will return an error." + ) + } + } } - case None => // Model is optional, will use default } - - Success(()) - } - - /** - * Check if an OpenAI model is supported - */ - private def isValidOpenAIModel(model: String): Boolean = { - getOpenAISupportedModels.contains(model) - } - - /** - * Get list of supported OpenAI models - */ - private def getOpenAISupportedModels: Set[String] = { - ModelRegistry.getSupportedModels("openai") - } - - /** - * Check if a Claude model is supported - */ - private def isValidClaudeModel(model: String): Boolean = { - ModelRegistry.isValidModel("anthropic", model) - } - - /** - * Get list of supported Claude models - */ - private def getClaudeSupportedModels: Set[String] = { - ModelRegistry.getSupportedModels("anthropic") - } - - /** - * Check if a Gemini model is supported - */ - private def isValidGeminiModel(model: String): Boolean = { - ModelRegistry.isValidModel("gemini", model) - } - - /** - * Get list of supported Gemini models - */ - private def getGeminiSupportedModels: Set[String] = { - ModelRegistry.getSupportedModels("gemini") - } - - /** - * Check if an Ollama model is supported - */ - private def isValidOllamaModel(model: String): Boolean = { - ModelRegistry.isValidModel("ollama", model) } /** - * Get list of supported Ollama models - */ - private def getOllamaSupportedModels: Set[String] = { - ModelRegistry.getSupportedModels("ollama") - } - - /** - * Get provider-specific configuration requirements - * - * @param providerName Provider name - * @return Set of required configuration keys + * Get provider-specific configuration requirements. */ def getRequiredConfigKeys(providerName: String): Set[String] = { - providerName.toLowerCase.trim match { - case OPENAI => Set(ConfigStore.API_KEY) - case ANTHROPIC => Set(ConfigStore.API_KEY) - case GEMINI => Set(ConfigStore.API_KEY) - case OLLAMA => Set() // Ollama typically doesn't require API key + ProviderRegistry.get(providerName.toLowerCase.trim) match { + case Some(desc) if desc.requiresApiKey => Set(ConfigStore.API_KEY) case _ => Set() } } /** - * Get provider-specific default configuration - * - * @param providerName Provider name - * @return Map of default configuration values + * Get provider-specific default configuration. */ def getDefaultConfig(providerName: String): Map[String, String] = { - providerName.toLowerCase.trim match { - case OPENAI => Map( - ConfigStore.MODEL -> ModelRegistry.defaultModel("openai"), - ConfigStore.BASE_URL -> ConfigStore.DEFAULT_OPENAI_BASE_URL, - ConfigStore.TEMPERATURE -> ConfigStore.DEFAULT_TEMPERATURE, - ConfigStore.MAX_TOKENS -> ConfigStore.DEFAULT_MAX_TOKENS - ) - case ANTHROPIC => Map( - ConfigStore.MODEL -> ModelRegistry.defaultModel("anthropic"), - ConfigStore.BASE_URL -> ConfigStore.DEFAULT_ANTHROPIC_BASE_URL, - ConfigStore.TEMPERATURE -> ConfigStore.DEFAULT_TEMPERATURE, - ConfigStore.MAX_TOKENS -> "4000" - ) - case GEMINI => Map( - ConfigStore.MODEL -> ModelRegistry.defaultModel("gemini"), - ConfigStore.BASE_URL -> ConfigStore.DEFAULT_GEMINI_BASE_URL, - ConfigStore.TEMPERATURE -> ConfigStore.DEFAULT_TEMPERATURE, - ConfigStore.MAX_TOKENS -> "2048" - ) - case OLLAMA => Map( - ConfigStore.MODEL -> ModelRegistry.defaultModel("ollama"), - ConfigStore.BASE_URL -> ConfigStore.DEFAULT_OLLAMA_BASE_URL, + ProviderRegistry.get(providerName.toLowerCase.trim) match { + case Some(desc) => Map( + ConfigStore.MODEL -> desc.defaultModel, + ConfigStore.BASE_URL -> desc.defaultBaseUrl, ConfigStore.TEMPERATURE -> ConfigStore.DEFAULT_TEMPERATURE, - ConfigStore.MAX_TOKENS -> "2048" + ConfigStore.MAX_TOKENS -> desc.defaultMaxTokens ) - case _ => Map() // Should not reach here given earlier validation + case None => Map() } } } diff --git a/src/main/providers/ProviderRegistrations.scala b/src/main/providers/ProviderRegistrations.scala new file mode 100644 index 0000000..993f2f1 --- /dev/null +++ b/src/main/providers/ProviderRegistrations.scala @@ -0,0 +1,174 @@ +// ABOUTME: Registers all built-in provider descriptors with the ProviderRegistry +// ABOUTME: Adding a new provider = adding one block here (plus the provider class and models.yaml) +package org.nlogo.extensions.llm.providers + +import scala.concurrent.ExecutionContext + +/** + * Central registration of all built-in providers. + * + * Called once during LLMExtension.load() to populate the ProviderRegistry. + * To add a new provider, add one register() block here. + */ +object ProviderRegistrations { + + def registerAll(): Unit = { + + ProviderRegistry.register(ProviderDescriptor( + name = "openai", + displayName = "OpenAI", + apiKeyConfigKey = "openai_api_key", + baseUrlConfigKey = "openai_base_url", + defaultBaseUrl = "https://api.openai.com/v1", + defaultModel = "gpt-4o-mini", + defaultMaxTokens = "1000", + requiresApiKey = true, + apiKeyPrefix = Some("sk-"), + readinessCheck = ReadinessCheck.ApiKey, + exposesThinking = false, + helpText = + """OpenAI Setup Instructions: + | + |1. Get an API key: + | - Visit https://platform.openai.com/api-keys + | - Create a new API key + | + |2. Set the key: + | - In config file: openai_api_key=sk-your-key-here + | - Or at runtime: llm:set-api-key "sk-your-key-here" + | + |3. Verify: + | - Check llm:provider-status for "has-key: true"""".stripMargin, + factory = ec => new OpenAIProvider()(using ec) + )) + + ProviderRegistry.register(ProviderDescriptor( + name = "anthropic", + displayName = "Anthropic (Claude)", + apiKeyConfigKey = "anthropic_api_key", + baseUrlConfigKey = "anthropic_base_url", + defaultBaseUrl = "https://api.anthropic.com/v1", + defaultModel = "claude-3-5-haiku-latest", + defaultMaxTokens = "4000", + requiresApiKey = true, + apiKeyPrefix = None, + readinessCheck = ReadinessCheck.ApiKey, + exposesThinking = true, + helpText = + """Anthropic (Claude) Setup Instructions: + | + |1. Get an API key: + | - Visit https://console.anthropic.com/ + | - Create a new API key + | + |2. Set the key: + | - In config file: anthropic_api_key=sk-ant-your-key-here + | - Or at runtime: llm:set-api-key "sk-ant-your-key-here" + | + |3. Verify: + | - Check llm:provider-status for "has-key: true"""".stripMargin, + factory = ec => new ClaudeProvider()(using ec) + )) + + ProviderRegistry.register(ProviderDescriptor( + name = "gemini", + displayName = "Google Gemini", + apiKeyConfigKey = "gemini_api_key", + baseUrlConfigKey = "gemini_base_url", + defaultBaseUrl = "https://generativelanguage.googleapis.com/v1beta", + defaultModel = "gemini-1.5-flash", + defaultMaxTokens = "2048", + requiresApiKey = true, + apiKeyPrefix = None, + readinessCheck = ReadinessCheck.ApiKey, + exposesThinking = true, + helpText = + """Google Gemini Setup Instructions: + | + |1. Get an API key: + | - Visit https://makersuite.google.com/app/apikey + | - Create a new API key + | + |2. Set the key: + | - In config file: gemini_api_key=your-key-here + | - Or at runtime: llm:set-api-key "your-key-here" + | + |3. Verify: + | - Check llm:provider-status for "has-key: true"""".stripMargin, + factory = ec => new GeminiProvider()(using ec) + )) + + ProviderRegistry.register(ProviderDescriptor( + name = "ollama", + displayName = "Ollama", + apiKeyConfigKey = "ollama_api_key", + baseUrlConfigKey = "ollama_base_url", + defaultBaseUrl = "http://localhost:11434", + defaultModel = "llama3.2", + defaultMaxTokens = "2048", + requiresApiKey = false, + apiKeyPrefix = None, + readinessCheck = ReadinessCheck.ServerReachable, + exposesThinking = true, + helpText = + """Ollama Setup Instructions: + | + |1. Install Ollama: + | - Visit https://ollama.ai/download + | - Download and install for your platform + | + |2. Start Ollama server: + | - Open terminal and run: ollama serve + | - Or start Ollama app (it runs in background) + | + |3. Pull a model: + | - Run: ollama pull llama3.2 + | - Or try: ollama pull deepseek-r1:1.5b (smaller) + | + |4. Verify installation: + | - Check llm:provider-status for "reachable: true" + | + |5. Custom server URL: + | - In config: ollama_base_url=http://your-server:11434 + | - Default: http://localhost:11434 + | + |For more models: ollama.ai/library""".stripMargin, + factory = ec => new OllamaProvider()(using ec) + )) + + ProviderRegistry.register(ProviderDescriptor( + name = "openrouter", + displayName = "OpenRouter", + apiKeyConfigKey = "openrouter_api_key", + baseUrlConfigKey = "openrouter_base_url", + defaultBaseUrl = "https://openrouter.ai/api/v1", + defaultModel = "openai/gpt-4o-mini", + defaultMaxTokens = "1000", + requiresApiKey = true, + apiKeyPrefix = None, + readinessCheck = ReadinessCheck.ApiKey, + exposesThinking = true, + helpText = + """OpenRouter Setup Instructions: + | + |1. Get an API key: + | - Visit https://openrouter.ai/keys + | - Create a new API key + | + |2. Set the key: + | - In config file: openrouter_api_key=sk-or-your-key-here + | - Or at runtime: llm:set-api-key "sk-or-your-key-here" + | + |3. Set a model (vendor-prefixed): + | - llm:set-model "openai/gpt-4o" + | - llm:set-model "anthropic/claude-3.5-sonnet" + | - llm:set-model "deepseek/deepseek-r1" + | + |4. Verify: + | - Check llm:provider-status for "has-key: true" + | + |Browse 200+ models: https://openrouter.ai/models""".stripMargin, + factory = ec => new OpenRouterProvider()(using ec) + )) + } +} diff --git a/src/main/providers/ProviderRegistry.scala b/src/main/providers/ProviderRegistry.scala new file mode 100644 index 0000000..43183e4 --- /dev/null +++ b/src/main/providers/ProviderRegistry.scala @@ -0,0 +1,71 @@ +// ABOUTME: Central registry of provider descriptors, populated at extension load time +// ABOUTME: Single source of truth for provider metadata — replaces scattered match/case lookups +package org.nlogo.extensions.llm.providers + +/** + * Singleton registry holding ProviderDescriptor for every supported provider. + * + * Populated once during extension initialization via ProviderRegistrations.registerAll(). + * All consumer code (ProviderFactory, LLMExtension, ConfigStore helpers, etc.) + * reads from this registry instead of maintaining per-provider match/case blocks. + * + * Thread-safety: registration happens on the main thread at load time before + * any NetLogo model code runs. After init the map is effectively read-only. + */ +object ProviderRegistry { + + @volatile private var descriptors: Map[String, ProviderDescriptor] = Map.empty + + /** + * Register a provider descriptor. Call during extension init only. + * Uses @volatile to ensure the full map is visible to any thread that + * reads after registration completes (safe-publication guarantee). + */ + def register(desc: ProviderDescriptor): Unit = { + descriptors = descriptors + (desc.name.toLowerCase.trim -> desc) + } + + /** Look up a descriptor by provider name (case-insensitive). */ + def get(name: String): Option[ProviderDescriptor] = + descriptors.get(name.toLowerCase.trim) + + /** All registered provider names. */ + def allNames: Set[String] = descriptors.keySet + + /** Whether a provider name is registered. */ + def isSupported(name: String): Boolean = + descriptors.contains(name.toLowerCase.trim) + + // --- Convenience accessors that replace scattered match/case --- + + def apiKeyConfigKey(provider: String): String = + get(provider).map(_.apiKeyConfigKey).getOrElse("api_key") + + def baseUrlConfigKey(provider: String): String = + get(provider).map(_.baseUrlConfigKey).getOrElse("base_url") + + def defaultBaseUrl(provider: String): String = + get(provider).map(_.defaultBaseUrl).getOrElse("") + + def defaultModel(provider: String): String = + get(provider).map(_.defaultModel) + .getOrElse(throw new IllegalArgumentException(s"Unknown provider: $provider")) + + def requiresApiKey(provider: String): Boolean = + get(provider).exists(_.requiresApiKey) + + def helpText(provider: String): String = + get(provider).map(_.helpText) + .getOrElse(s"Unknown provider: $provider. Supported: ${allNames.mkString(", ")}") + + def exposesThinking(provider: String): Boolean = + get(provider).exists(_.exposesThinking) + + /** Whether any providers have been registered. */ + def isInitialized: Boolean = descriptors.nonEmpty + + /** Reset registry — for testing only. */ + def reset(): Unit = { + descriptors = Map.empty + } +} diff --git a/src/main/providers/ReasoningModelDetector.scala b/src/main/providers/ReasoningModelDetector.scala index 495bcf9..df982b2 100644 --- a/src/main/providers/ReasoningModelDetector.scala +++ b/src/main/providers/ReasoningModelDetector.scala @@ -29,6 +29,13 @@ object ReasoningModelDetector { case "openai" => val m = model.toLowerCase m.startsWith("o1") || m.startsWith("o3") || m.startsWith("o4") + case "openrouter" => + // Vendor-prefixed: check if the model after the prefix is an OpenAI o-series + val m = model.toLowerCase + if (m.startsWith("openai/")) { + val modelName = m.stripPrefix("openai/") + modelName.startsWith("o1") || modelName.startsWith("o3") || modelName.startsWith("o4") + } else false case _ => false } } @@ -69,15 +76,11 @@ object ReasoningModelDetector { /** * Whether a provider exposes thinking text in its API response. * - * OpenAI Chat Completions API hides reasoning tokens — thinking text is not - * returned. Other providers (Anthropic, Gemini, Ollama) expose it. + * Delegates to ProviderRegistry.exposesThinking() so this knowledge + * is defined once per provider in the descriptor. */ - def providerExposesThinking(provider: String): Boolean = { - provider.toLowerCase.trim match { - case "openai" => false - case _ => true - } - } + def providerExposesThinking(provider: String): Boolean = + ProviderRegistry.exposesThinking(provider) /** * Check if a model is a known reasoning model (for display purposes). @@ -96,6 +99,12 @@ object ReasoningModelDetector { case "ollama" => val m = model.toLowerCase m.contains("deepseek-r1") || m.contains("qwen3") || m.contains("qwq") + case "openrouter" => + // Vendor-prefixed model names: detect reasoning models across vendors + val m = model.toLowerCase + m.startsWith("openai/o1") || m.startsWith("openai/o3") || m.startsWith("openai/o4") || + m.contains("claude-3-7") || m.contains("claude-4") || + m.contains("deepseek-r1") || m.contains("qwq") case _ => false } } diff --git a/src/main/resources/config/models.yaml b/src/main/resources/config/models.yaml index 3373d78..6d86df3 100644 --- a/src/main/resources/config/models.yaml +++ b/src/main/resources/config/models.yaml @@ -72,6 +72,35 @@ gemini: - gemini-2.5-flash-lite-latest - gemini-2.5-flash-lite-preview-09-2025 +openrouter: + # OpenAI models via OpenRouter + - openai/gpt-4o + - openai/gpt-4o-mini + - openai/gpt-4.1 + - openai/gpt-4.1-mini + - openai/o3-mini + - openai/o4-mini + + # Anthropic models via OpenRouter + - anthropic/claude-sonnet-4-20250514 + - anthropic/claude-3.5-sonnet + - anthropic/claude-3.5-haiku + + # Google models via OpenRouter + - google/gemini-2.5-flash + - google/gemini-2.5-pro + + # Meta Llama models via OpenRouter + - meta-llama/llama-3.3-70b-instruct + - meta-llama/llama-3.1-405b-instruct + + # DeepSeek models via OpenRouter + - deepseek/deepseek-r1 + - deepseek/deepseek-chat-v3-0324 + + # Qwen models via OpenRouter + - qwen/qwen-max + ollama: # Top reasoning models - deepseek-r1:70b diff --git a/tests.txt b/tests.txt index ca9fff5..85c1c04 100644 --- a/tests.txt +++ b/tests.txt @@ -10,17 +10,18 @@ LLMConfigPrimitives LLMProvidersAll extensions [llm] - length llm:providers-all => 4 + length llm:providers-all => 5 member? "openai" llm:providers-all => true member? "anthropic" llm:providers-all => true member? "gemini" llm:providers-all => true member? "ollama" llm:providers-all => true + member? "openrouter" llm:providers-all => true LLMProviderStatus extensions [llm] O> llm:set-api-key "test-key" O> llm:set-provider "openai" - length llm:provider-status => 4 + length llm:provider-status => 5 LLMListModels extensions [llm] @@ -248,6 +249,16 @@ LLMMultiAgentChat empty? [agent-result] of turtle 0 => false empty? [agent-result] of turtle 1 => false +LLMOpenRouterProvider + extensions [llm] + O> llm:set-api-key "test-key" + O> llm:set-provider "openrouter" + item 0 llm:active => "openrouter" + item 1 llm:active => "openai/gpt-4o-mini" + empty? (llm:provider-help "openrouter") => false + O> llm:set-model "openai/gpt-4o" + item 1 llm:active => "openai/gpt-4o" + LLMConfigLoading extensions [llm] O> llm:load-config "demos/config" From 2a1b889811706ddfc9fadbb2cabfc4f290596476 Mon Sep 17 00:00:00 2001 From: JNK234 Date: Wed, 15 Apr 2026 18:50:13 -0500 Subject: [PATCH 2/6] feat: add Together AI provider support (#41) Extends OpenAICompatibleProvider with Together-specific reasoning and thinking extraction. Vendor-prefixed model names passed through unchanged. - TogetherProvider: hybrid reasoning via { reasoning: { enabled: true } } - DeepSeek-R1 thinking: extracts tags from content field, falls back to message.reasoning for other models - 10 models in registry: Llama, DeepSeek, Qwen, Gemma, Mistral - ReasoningModelDetector updated for together reasoning models - Provider count now 6 (30/30 tests pass) --- docs/CONFIGURATION.md | 17 +++- docs/PROVIDER-GUIDE.md | 59 ++++++++++++++ src/main/providers/ModelRegistry.scala | 3 +- .../providers/ProviderRegistrations.scala | 35 ++++++++ .../providers/ReasoningModelDetector.scala | 3 + src/main/providers/TogetherProvider.scala | 80 +++++++++++++++++++ src/main/resources/config/models.yaml | 21 +++++ tests.txt | 15 +++- 8 files changed, 227 insertions(+), 6 deletions(-) create mode 100644 src/main/providers/TogetherProvider.scala diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 516a1b2..02c3e02 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -17,7 +17,7 @@ This allows you to: ## Immediate Validation Starting from this version, configuration is validated immediately: - When you call `llm:load-config` or `llm:set-provider`, the extension checks if the provider is ready -- **Cloud providers** (OpenAI, Anthropic, Gemini, OpenRouter) require API keys +- **Cloud providers** (OpenAI, Anthropic, Gemini, OpenRouter, Together AI) require API keys - **Ollama** requires the server to be running and reachable - If validation fails, you get a clear error message with setup instructions - Use `print llm:provider-help "provider-name"` to get detailed setup guidance @@ -28,8 +28,8 @@ Starting from this version, configuration is validated immediately: - No quotes required; avoid trailing spaces around `=`. - **Supported keys**: - Common: `provider`, `model`, `temperature`, `max_tokens`, `timeout_seconds` - - Provider-specific API keys: `openai_api_key`, `anthropic_api_key`, `gemini_api_key`, `openrouter_api_key` - - Provider-specific base URLs: `openai_base_url`, `anthropic_base_url`, `gemini_base_url`, `ollama_base_url`, `openrouter_base_url` + - Provider-specific API keys: `openai_api_key`, `anthropic_api_key`, `gemini_api_key`, `openrouter_api_key`, `together_api_key` + - Provider-specific base URLs: `openai_base_url`, `anthropic_base_url`, `gemini_base_url`, `ollama_base_url`, `openrouter_base_url`, `together_base_url` - Legacy (still supported): `api_key`, `base_url` (applies to current provider) ## Where to Save the File @@ -85,6 +85,16 @@ max_tokens=1000 timeout_seconds=30 ``` +### Together AI (open-source models) +``` +provider=together +together_api_key=REPLACE_ME +model=meta-llama/Llama-3.3-70B-Instruct-Turbo +temperature=0.7 +max_tokens=1000 +timeout_seconds=30 +``` + ### Local (Ollama, no API key) ``` provider=ollama @@ -103,6 +113,7 @@ openai_api_key=sk-REPLACE_ME anthropic_api_key=sk-ant-REPLACE_ME gemini_api_key=REPLACE_ME openrouter_api_key=sk-or-REPLACE_ME +together_api_key=REPLACE_ME # Set the active provider provider=openai diff --git a/docs/PROVIDER-GUIDE.md b/docs/PROVIDER-GUIDE.md index 5c064a5..a7ca7ca 100644 --- a/docs/PROVIDER-GUIDE.md +++ b/docs/PROVIDER-GUIDE.md @@ -301,6 +301,65 @@ llm:set-thinking true llm:set-reasoning-effort "high" ``` +## Together AI Configuration + +### API Setup + +1. **Get API Key**: Visit [api.together.ai/settings/api-keys](https://api.together.ai/settings/api-keys) +2. **Check Usage**: Monitor at [api.together.ai/settings/billing](https://api.together.ai/settings/billing) +3. **Browse Models**: Explore at [api.together.ai/models](https://api.together.ai/models) + +### Configuration Parameters + +```ini +# Required Parameters +provider=together +together_api_key=your-together-key-here +model=meta-llama/Llama-3.3-70B-Instruct-Turbo + +# Optional Parameters +together_base_url=https://api.together.xyz/v1 +temperature=0.7 +max_tokens=1000 +timeout_seconds=30 +``` + +### Available Models + +Model names use vendor prefixes (`vendor/model-name`): + +| Model | Description | Context | +|-------|-------------|---------| +| `meta-llama/Llama-3.3-70B-Instruct-Turbo` | Fast Llama 3.3 70B | 128K | +| `meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo` | Largest Llama | 128K | +| `meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo` | Small, fast Llama | 128K | +| `deepseek-ai/DeepSeek-R1` | DeepSeek reasoning model | 64K | +| `deepseek-ai/DeepSeek-V3` | DeepSeek V3 chat | 64K | +| `Qwen/Qwen2.5-72B-Instruct-Turbo` | Qwen 2.5 72B | 128K | +| `google/gemma-3-27b-it` | Google Gemma 3 | 128K | +| `mistralai/Mistral-Small-24B-Instruct-2501` | Mistral Small | 32K | + +**Recommended for NetLogo**: `meta-llama/Llama-3.3-70B-Instruct-Turbo` (fast, capable, good value) + +### Why Together AI? + +- **Fast inference** on popular open-source models +- **Pay-per-token** pricing, often cheaper than proprietary APIs +- **Open-source models** — Llama, DeepSeek, Qwen, Gemma, Mistral +- **Reasoning support** — DeepSeek-R1 with thinking output + +### Thinking/Reasoning Models + +Together AI supports reasoning models like DeepSeek-R1: + +```netlogo +llm:set-provider "together" +llm:set-model "deepseek-ai/DeepSeek-R1" +llm:set-thinking true +let result llm:chat-with-thinking "What is 15 * 17?" +; result is [answer thinking-text] +``` + ## Ollama (Local) Configuration ### Setup Requirements diff --git a/src/main/providers/ModelRegistry.scala b/src/main/providers/ModelRegistry.scala index 8b16965..cb11a52 100644 --- a/src/main/providers/ModelRegistry.scala +++ b/src/main/providers/ModelRegistry.scala @@ -36,7 +36,8 @@ object ModelRegistry { ), isCustom = false), "gemini" -> ProviderModels(Set("gemini-1.5-pro", "gemini-1.5-flash", "gemini-2.0-flash-exp"), isCustom = false), "ollama" -> ProviderModels(Set("llama3.2", "llama3.1", "mistral", "phi4"), isCustom = false), - "openrouter" -> ProviderModels(Set("openai/gpt-4o", "openai/gpt-4o-mini", "anthropic/claude-3.5-sonnet", "deepseek/deepseek-r1"), isCustom = false) + "openrouter" -> ProviderModels(Set("openai/gpt-4o", "openai/gpt-4o-mini", "anthropic/claude-3.5-sonnet", "deepseek/deepseek-r1"), isCustom = false), + "together" -> ProviderModels(Set("meta-llama/Llama-3.3-70B-Instruct-Turbo", "deepseek-ai/DeepSeek-R1", "Qwen/Qwen2.5-72B-Instruct-Turbo"), isCustom = false) ) /** diff --git a/src/main/providers/ProviderRegistrations.scala b/src/main/providers/ProviderRegistrations.scala index 993f2f1..0294c96 100644 --- a/src/main/providers/ProviderRegistrations.scala +++ b/src/main/providers/ProviderRegistrations.scala @@ -170,5 +170,40 @@ object ProviderRegistrations { |Browse 200+ models: https://openrouter.ai/models""".stripMargin, factory = ec => new OpenRouterProvider()(using ec) )) + + ProviderRegistry.register(ProviderDescriptor( + name = "together", + displayName = "Together AI", + apiKeyConfigKey = "together_api_key", + baseUrlConfigKey = "together_base_url", + defaultBaseUrl = "https://api.together.xyz/v1", + defaultModel = "meta-llama/Llama-3.3-70B-Instruct-Turbo", + defaultMaxTokens = "1000", + requiresApiKey = true, + apiKeyPrefix = None, + readinessCheck = ReadinessCheck.ApiKey, + exposesThinking = true, + helpText = + """Together AI Setup Instructions: + | + |1. Get an API key: + | - Visit https://api.together.ai/settings/api-keys + | - Create a new API key + | + |2. Set the key: + | - In config file: together_api_key=your-key-here + | - Or at runtime: llm:set-api-key "your-key-here" + | + |3. Set a model (vendor-prefixed): + | - llm:set-model "meta-llama/Llama-3.3-70B-Instruct-Turbo" + | - llm:set-model "deepseek-ai/DeepSeek-R1" + | - llm:set-model "Qwen/Qwen2.5-72B-Instruct-Turbo" + | + |4. Verify: + | - Check llm:provider-status for "has-key: true" + | + |Browse models: https://api.together.ai/models""".stripMargin, + factory = ec => new TogetherProvider()(using ec) + )) } } diff --git a/src/main/providers/ReasoningModelDetector.scala b/src/main/providers/ReasoningModelDetector.scala index df982b2..77c0b0b 100644 --- a/src/main/providers/ReasoningModelDetector.scala +++ b/src/main/providers/ReasoningModelDetector.scala @@ -105,6 +105,9 @@ object ReasoningModelDetector { m.startsWith("openai/o1") || m.startsWith("openai/o3") || m.startsWith("openai/o4") || m.contains("claude-3-7") || m.contains("claude-4") || m.contains("deepseek-r1") || m.contains("qwq") + case "together" => + val m = model.toLowerCase + m.contains("deepseek-r1") || m.contains("qwq") || m.contains("qwen3") case _ => false } } diff --git a/src/main/providers/TogetherProvider.scala b/src/main/providers/TogetherProvider.scala new file mode 100644 index 0000000..cce7e6a --- /dev/null +++ b/src/main/providers/TogetherProvider.scala @@ -0,0 +1,80 @@ +// ABOUTME: Together AI provider — fast open-source model inference +// ABOUTME: Extends OpenAICompatibleProvider with Together-specific reasoning and thinking extraction +package org.nlogo.extensions.llm.providers + +import org.nlogo.extensions.llm.models.ChatRequest +import scala.concurrent.ExecutionContext + +/** + * Together AI provider implementation. + * + * Together AI provides fast inference for open-source models (Llama, DeepSeek, + * Qwen, Gemma, Mistral, etc.) via an OpenAI-compatible API. + * + * Key differences from direct OpenAI: + * - Model names are vendor-prefixed (e.g. "meta-llama/Llama-3.3-70B-Instruct-Turbo") + * - Hybrid reasoning models use { reasoning: { enabled: true } } + * - DeepSeek-R1 embeds thinking in ... tags within content + */ +class TogetherProvider(implicit ec: ExecutionContext) extends OpenAICompatibleProvider { + + override def providerName: String = "together" + + override def defaultModel: String = "meta-llama/Llama-3.3-70B-Instruct-Turbo" + + override protected def defaultBaseUrl: String = "https://api.together.xyz/v1" + + override protected def baseUrlConfigKey: String = "together_base_url" + + override protected def apiKeyConfigKey: String = "together_api_key" + + override protected def defaultMaxTokens: String = "1000" + + override protected def requiresApiKey: Boolean = true + + // No extra headers needed (unlike OpenRouter) + + /** + * Together hybrid models use { reasoning: { enabled: true } }. + * Adjustable-effort models use top-level reasoning_effort (the default from base class). + * We send both when thinking is enabled — the API ignores unrecognized fields. + */ + override protected def applyReasoningFields(baseObj: ujson.Obj, request: ChatRequest): Unit = { + // Enable reasoning for hybrid models + baseObj("reasoning") = ujson.Obj("enabled" -> true) + // Also pass effort level if specified (for adjustable-effort models) + request.thinkingConfig.flatMap(_.reasoningEffort).foreach { effort => + baseObj("reasoning_effort") = effort + } + } + + /** + * Extract thinking text from Together AI responses. + * + * Two extraction paths: + * 1. message.reasoning field (most reasoning models) + * 2. ... tags in message.content (DeepSeek-R1) + */ + override protected def extractThinking(message: ujson.Value): Option[String] = { + // Path 1: check message.reasoning field + val fromReasoning = try { + message.obj.get("reasoning").flatMap { v => + val text = v.str.trim + if (text.nonEmpty) Some(text) else None + } + } catch { + case _: Exception => None + } + + if (fromReasoning.isDefined) return fromReasoning + + // Path 2: parse ... tags from content (DeepSeek-R1) + try { + val content = message("content").str + val thinkPattern = """(?s)(.*?)""".r + thinkPattern.findFirstMatchIn(content).map(_.group(1).trim).filter(_.nonEmpty) + } catch { + case _: Exception => None + } + } +} diff --git a/src/main/resources/config/models.yaml b/src/main/resources/config/models.yaml index 6d86df3..ca4c077 100644 --- a/src/main/resources/config/models.yaml +++ b/src/main/resources/config/models.yaml @@ -101,6 +101,27 @@ openrouter: # Qwen models via OpenRouter - qwen/qwen-max +together: + # Meta Llama + - meta-llama/Llama-3.3-70B-Instruct-Turbo + - meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo + - meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo + - meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo + + # DeepSeek + - deepseek-ai/DeepSeek-R1 + - deepseek-ai/DeepSeek-V3 + + # Qwen + - Qwen/Qwen2.5-72B-Instruct-Turbo + - Qwen/Qwen2.5-7B-Instruct-Turbo + + # Google Gemma + - google/gemma-3-27b-it + + # Mistral + - mistralai/Mistral-Small-24B-Instruct-2501 + ollama: # Top reasoning models - deepseek-r1:70b diff --git a/tests.txt b/tests.txt index 85c1c04..dced5d5 100644 --- a/tests.txt +++ b/tests.txt @@ -10,18 +10,19 @@ LLMConfigPrimitives LLMProvidersAll extensions [llm] - length llm:providers-all => 5 + length llm:providers-all => 6 member? "openai" llm:providers-all => true member? "anthropic" llm:providers-all => true member? "gemini" llm:providers-all => true member? "ollama" llm:providers-all => true member? "openrouter" llm:providers-all => true + member? "together" llm:providers-all => true LLMProviderStatus extensions [llm] O> llm:set-api-key "test-key" O> llm:set-provider "openai" - length llm:provider-status => 5 + length llm:provider-status => 6 LLMListModels extensions [llm] @@ -259,6 +260,16 @@ LLMOpenRouterProvider O> llm:set-model "openai/gpt-4o" item 1 llm:active => "openai/gpt-4o" +LLMTogetherProvider + extensions [llm] + O> llm:set-api-key "test-key" + O> llm:set-provider "together" + item 0 llm:active => "together" + item 1 llm:active => "meta-llama/Llama-3.3-70B-Instruct-Turbo" + empty? (llm:provider-help "together") => false + O> llm:set-model "deepseek-ai/DeepSeek-R1" + item 1 llm:active => "deepseek-ai/DeepSeek-R1" + LLMConfigLoading extensions [llm] O> llm:load-config "demos/config" From f539a735340b0836c544e58ce506606f485041e8 Mon Sep 17 00:00:00 2001 From: JNK234 Date: Fri, 17 Apr 2026 15:28:41 -0500 Subject: [PATCH 3/6] fix: reject unknown providers on config load, strip tags from Together content Addresses two P2 findings from the Codex review on #43. 1. LoadConfigCommand now fails fast when `provider=` is unknown. Previously the .foreach pattern silently skipped validation, deferring the error to the first chat request. Model names still warn-only (preserves flexibility for unlisted models). 2. Add cleanContent hook in OpenAICompatibleProvider (default identity). TogetherProvider overrides it to strip ... blocks so llm:chat-with-thinking no longer leaks reasoning into the answer slot for DeepSeek-R1. Tests: 31/31 pass. New LLMLoadConfigRejectsUnknownProvider fixture verifies the typo'd provider name surfaces in the error message. --- demos/test-config-bad-provider | 4 ++ src/main/LLMExtension.scala | 42 ++++++++++--------- .../providers/OpenAICompatibleProvider.scala | 15 ++++++- src/main/providers/TogetherProvider.scala | 20 ++++++++- tests.txt | 7 ++++ 5 files changed, 65 insertions(+), 23 deletions(-) create mode 100644 demos/test-config-bad-provider diff --git a/demos/test-config-bad-provider b/demos/test-config-bad-provider new file mode 100644 index 0000000..4da23d6 --- /dev/null +++ b/demos/test-config-bad-provider @@ -0,0 +1,4 @@ +# Test fixture: unknown provider name should raise an error on load. +provider=opanai +openai_api_key=test-key +model=gpt-4o-mini diff --git a/src/main/LLMExtension.scala b/src/main/LLMExtension.scala index 6c0bfcb..6d07eda 100644 --- a/src/main/LLMExtension.scala +++ b/src/main/LLMExtension.scala @@ -377,26 +377,30 @@ class LLMExtension extends DefaultClassManager { } - // Validate provider after loading config using descriptor + // Validate provider after loading config using descriptor. + // Unknown provider names fail fast — typos shouldn't defer errors to first chat. val providerName = configStore.getOrElse(ConfigStore.PROVIDER, ConfigStore.DEFAULT_PROVIDER) - ProviderRegistry.get(providerName.toLowerCase.trim).foreach { desc => - desc.readinessCheck match { - case ReadinessCheck.ServerReachable => - if (!isOllamaReachable) { - val baseUrl = configStore.get(desc.baseUrlConfigKey) - .orElse(configStore.get(ConfigStore.BASE_URL)) - .getOrElse(desc.defaultBaseUrl) - throw new ExtensionException( - s"Config loaded but ${desc.displayName} not reachable at $baseUrl. Please start the server or change ${desc.baseUrlConfigKey} in config. For help: print llm:provider-help \"${desc.name}\"" - ) - } - case ReadinessCheck.ApiKey => - if (!hasApiKey(providerName)) { - throw new ExtensionException( - s"Config loaded but ${desc.displayName} provider requires an API key. Set '${desc.apiKeyConfigKey}' in config. For help: print llm:provider-help \"${desc.name}\"" - ) - } - } + val desc = ProviderRegistry.get(providerName.toLowerCase.trim).getOrElse { + throw new ExtensionException( + s"Unknown provider '$providerName' in config. Supported: ${ProviderRegistry.allNames.toList.sorted.mkString(", ")}" + ) + } + desc.readinessCheck match { + case ReadinessCheck.ServerReachable => + if (!isOllamaReachable) { + val baseUrl = configStore.get(desc.baseUrlConfigKey) + .orElse(configStore.get(ConfigStore.BASE_URL)) + .getOrElse(desc.defaultBaseUrl) + throw new ExtensionException( + s"Config loaded but ${desc.displayName} not reachable at $baseUrl. Please start the server or change ${desc.baseUrlConfigKey} in config. For help: print llm:provider-help \"${desc.name}\"" + ) + } + case ReadinessCheck.ApiKey => + if (!hasApiKey(providerName)) { + throw new ExtensionException( + s"Config loaded but ${desc.displayName} provider requires an API key. Set '${desc.apiKeyConfigKey}' in config. For help: print llm:provider-help \"${desc.name}\"" + ) + } } currentProvider = None // Force re-initialization with new config diff --git a/src/main/providers/OpenAICompatibleProvider.scala b/src/main/providers/OpenAICompatibleProvider.scala index dbf18f9..6a7a442 100644 --- a/src/main/providers/OpenAICompatibleProvider.scala +++ b/src/main/providers/OpenAICompatibleProvider.scala @@ -42,6 +42,16 @@ abstract class OpenAICompatibleProvider(implicit ec: ExecutionContext) extends B */ protected def extractThinking(message: ujson.Value): Option[String] = None + /** + * Clean the assistant response content before returning it to the user. + * Default is identity — no transformation. + * + * Together AI's DeepSeek-R1 embeds reasoning inside ... + * tags in the content field; TogetherProvider overrides this hook to + * strip those tags so the answer slot doesn't duplicate the thinking slot. + */ + protected def cleanContent(content: String, message: ujson.Value): String = content + override protected def buildApiUrl(baseUrl: String): Uri = uri"$baseUrl/chat/completions" @@ -99,12 +109,13 @@ abstract class OpenAICompatibleProvider(implicit ec: ExecutionContext) extends B val choices = parsed("choices").arr.zipWithIndex.map { case (choice, index) => val message = choice("message") val role = message("role").str - val content = message("content").str + val rawContent = message("content").str + val cleaned = cleanContent(rawContent, message) val finishReason = choice("finish_reason").str Choice( index = index, - message = ChatMessage(role, content), + message = ChatMessage(role, cleaned), finishReason = finishReason ) }.toArray diff --git a/src/main/providers/TogetherProvider.scala b/src/main/providers/TogetherProvider.scala index cce7e6a..ea921a0 100644 --- a/src/main/providers/TogetherProvider.scala +++ b/src/main/providers/TogetherProvider.scala @@ -71,10 +71,26 @@ class TogetherProvider(implicit ec: ExecutionContext) extends OpenAICompatiblePr // Path 2: parse ... tags from content (DeepSeek-R1) try { val content = message("content").str - val thinkPattern = """(?s)(.*?)""".r - thinkPattern.findFirstMatchIn(content).map(_.group(1).trim).filter(_.nonEmpty) + TogetherProvider.ThinkTagPattern.findFirstMatchIn(content) + .map(_.group(1).trim) + .filter(_.nonEmpty) } catch { case _: Exception => None } } + + /** + * Strip ... blocks from the assistant content so the answer + * slot of llm:chat-with-thinking doesn't duplicate the thinking slot. + * Applies to DeepSeek-R1 responses; no-op for other Together models. + */ + override protected def cleanContent(content: String, message: ujson.Value): String = { + val stripped = TogetherProvider.ThinkTagPattern.replaceAllIn(content, "").trim + if (stripped.isEmpty) content else stripped + } +} + +object TogetherProvider { + // (?s) = dot matches newlines; non-greedy to support multiple blocks. + private val ThinkTagPattern = """(?s)(.*?)""".r } diff --git a/tests.txt b/tests.txt index dced5d5..44596b3 100644 --- a/tests.txt +++ b/tests.txt @@ -275,3 +275,10 @@ LLMConfigLoading O> llm:load-config "demos/config" item 0 llm:active => "openai" member? "openai" llm:providers => true + +LLMLoadConfigRejectsUnknownProvider + extensions [llm] + globals [err] + O> carefully [llm:load-config "demos/test-config-bad-provider"] [set err error-message] + is-string? err => true + (position "opanai" err) != false => true From d9a33de580b063b524836af3117c7143407a536b Mon Sep 17 00:00:00 2001 From: JNK234 Date: Fri, 17 Apr 2026 15:37:50 -0500 Subject: [PATCH 4/6] fix: don't mutate Together content, roll back config on load failure Second round of fixes for Codex review on #43. 1. Revert the cleanContent hook and Together tag stripping. Faithful pass-through of API responses is more important than cosmetic cleanup. Users who want thinking separated can use llm:chat-with-thinking, which still extracts thinking via the existing extractThinking hook without mutating content. 2. llm:load-config now validates the provider name against the raw config map before touching configStore, and snapshots + rolls back on readiness-check failure. A rejected load no longer leaves the session with a half-loaded config that breaks llm:active. Tests: 31/31 pass. --- src/main/LLMExtension.scala | 63 +++++++++++-------- .../providers/OpenAICompatibleProvider.scala | 15 +---- src/main/providers/TogetherProvider.scala | 20 ++---- 3 files changed, 42 insertions(+), 56 deletions(-) diff --git a/src/main/LLMExtension.scala b/src/main/LLMExtension.scala index 6d07eda..862caad 100644 --- a/src/main/LLMExtension.scala +++ b/src/main/LLMExtension.scala @@ -366,8 +366,18 @@ class LLMExtension extends DefaultClassManager { ConfigLoader.loadFromFile(filename, modelDir) match { case Success(config) => - configStore.loadFromMap(config) + // Validate the provider name against the raw config BEFORE mutating + // configStore, so a rejected load leaves the session state unchanged. + val providerName = config.getOrElse(ConfigStore.PROVIDER, ConfigStore.DEFAULT_PROVIDER) + val desc = ProviderRegistry.get(providerName.toLowerCase.trim).getOrElse { + throw new ExtensionException( + s"Unknown provider '$providerName' in config. Supported: ${ProviderRegistry.allNames.toList.sorted.mkString(", ")}" + ) + } + // Snapshot old config so we can roll back on readiness-check failure. + val previousConfig = configStore.toMap + configStore.loadFromMap(config) // Load model override file if available modelDir.foreach { dir => @@ -376,33 +386,32 @@ class LLMExtension extends DefaultClassManager { } } - - // Validate provider after loading config using descriptor. - // Unknown provider names fail fast — typos shouldn't defer errors to first chat. - val providerName = configStore.getOrElse(ConfigStore.PROVIDER, ConfigStore.DEFAULT_PROVIDER) - val desc = ProviderRegistry.get(providerName.toLowerCase.trim).getOrElse { - throw new ExtensionException( - s"Unknown provider '$providerName' in config. Supported: ${ProviderRegistry.allNames.toList.sorted.mkString(", ")}" - ) - } - desc.readinessCheck match { - case ReadinessCheck.ServerReachable => - if (!isOllamaReachable) { - val baseUrl = configStore.get(desc.baseUrlConfigKey) - .orElse(configStore.get(ConfigStore.BASE_URL)) - .getOrElse(desc.defaultBaseUrl) - throw new ExtensionException( - s"Config loaded but ${desc.displayName} not reachable at $baseUrl. Please start the server or change ${desc.baseUrlConfigKey} in config. For help: print llm:provider-help \"${desc.name}\"" - ) - } - case ReadinessCheck.ApiKey => - if (!hasApiKey(providerName)) { - throw new ExtensionException( - s"Config loaded but ${desc.displayName} provider requires an API key. Set '${desc.apiKeyConfigKey}' in config. For help: print llm:provider-help \"${desc.name}\"" - ) - } + // Readiness check uses the now-loaded config (needs apiKeyConfigKey lookup). + // Roll back on failure so llm:active and friends keep the prior state. + try { + desc.readinessCheck match { + case ReadinessCheck.ServerReachable => + if (!isOllamaReachable) { + val baseUrl = configStore.get(desc.baseUrlConfigKey) + .orElse(configStore.get(ConfigStore.BASE_URL)) + .getOrElse(desc.defaultBaseUrl) + throw new ExtensionException( + s"Config loaded but ${desc.displayName} not reachable at $baseUrl. Please start the server or change ${desc.baseUrlConfigKey} in config. For help: print llm:provider-help \"${desc.name}\"" + ) + } + case ReadinessCheck.ApiKey => + if (!hasApiKey(providerName)) { + throw new ExtensionException( + s"Config loaded but ${desc.displayName} provider requires an API key. Set '${desc.apiKeyConfigKey}' in config. For help: print llm:provider-help \"${desc.name}\"" + ) + } + } + } catch { + case e: ExtensionException => + configStore.loadFromMap(previousConfig) + throw e } - + currentProvider = None // Force re-initialization with new config case Failure(e) => throw new ExtensionException(s"Failed to load configuration from '$filename': ${e.getMessage}") diff --git a/src/main/providers/OpenAICompatibleProvider.scala b/src/main/providers/OpenAICompatibleProvider.scala index 6a7a442..dbf18f9 100644 --- a/src/main/providers/OpenAICompatibleProvider.scala +++ b/src/main/providers/OpenAICompatibleProvider.scala @@ -42,16 +42,6 @@ abstract class OpenAICompatibleProvider(implicit ec: ExecutionContext) extends B */ protected def extractThinking(message: ujson.Value): Option[String] = None - /** - * Clean the assistant response content before returning it to the user. - * Default is identity — no transformation. - * - * Together AI's DeepSeek-R1 embeds reasoning inside ... - * tags in the content field; TogetherProvider overrides this hook to - * strip those tags so the answer slot doesn't duplicate the thinking slot. - */ - protected def cleanContent(content: String, message: ujson.Value): String = content - override protected def buildApiUrl(baseUrl: String): Uri = uri"$baseUrl/chat/completions" @@ -109,13 +99,12 @@ abstract class OpenAICompatibleProvider(implicit ec: ExecutionContext) extends B val choices = parsed("choices").arr.zipWithIndex.map { case (choice, index) => val message = choice("message") val role = message("role").str - val rawContent = message("content").str - val cleaned = cleanContent(rawContent, message) + val content = message("content").str val finishReason = choice("finish_reason").str Choice( index = index, - message = ChatMessage(role, cleaned), + message = ChatMessage(role, content), finishReason = finishReason ) }.toArray diff --git a/src/main/providers/TogetherProvider.scala b/src/main/providers/TogetherProvider.scala index ea921a0..d3df53e 100644 --- a/src/main/providers/TogetherProvider.scala +++ b/src/main/providers/TogetherProvider.scala @@ -69,28 +69,16 @@ class TogetherProvider(implicit ec: ExecutionContext) extends OpenAICompatiblePr if (fromReasoning.isDefined) return fromReasoning // Path 2: parse ... tags from content (DeepSeek-R1) + // (?s) makes dot match newlines. Content is returned unmodified — we only + // mirror the reasoning into a separate field for llm:chat-with-thinking. try { val content = message("content").str - TogetherProvider.ThinkTagPattern.findFirstMatchIn(content) + val pattern = """(?s)(.*?)""".r + pattern.findFirstMatchIn(content) .map(_.group(1).trim) .filter(_.nonEmpty) } catch { case _: Exception => None } } - - /** - * Strip ... blocks from the assistant content so the answer - * slot of llm:chat-with-thinking doesn't duplicate the thinking slot. - * Applies to DeepSeek-R1 responses; no-op for other Together models. - */ - override protected def cleanContent(content: String, message: ujson.Value): String = { - val stripped = TogetherProvider.ThinkTagPattern.replaceAllIn(content, "").trim - if (stripped.isEmpty) content else stripped - } -} - -object TogetherProvider { - // (?s) = dot matches newlines; non-greedy to support multiple blocks. - private val ThinkTagPattern = """(?s)(.*?)""".r } From c8ef2aff033d36ca16aaf70d6dd657d211e4ab59 Mon Sep 17 00:00:00 2001 From: JNK234 Date: Fri, 1 May 2026 01:30:41 -0500 Subject: [PATCH 5/6] test: simplify integration test model and add config template MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rewrite demos/tests/tests.nlogox with linear, readable test procedures that exercise all 17 primitives plus provider-specific behavior: - providers + provider-help registry coverage (6 providers) - invalid-provider rejection (validates f539a73 fix) - config-rollback on bad path (validates d9a33de fix) - sync/async chat, choose, history - thinking-config + chat-with-thinking - openrouter-vendor-prefix (skip-aware: only when openrouter active) - together-thinking (skip-aware: only when together + DeepSeek) - reasoning-marker visibility in list-models Drop heavy `carefully` defensive wrappers — keep them only where asserting an error is the test. Final summary prints "RESULTS: N passed, M failed" via globals. Add demos/tests/config.txt.example as a sanitized template covering all 6 providers with REPLACE_ME placeholders. Real config.txt remains gitignored. Update README to point users at the template. --- demos/tests/README.md | 9 +- demos/tests/config.txt.example | 47 ++++ demos/tests/tests.nlogox | 462 ++++++++++++--------------------- 3 files changed, 215 insertions(+), 303 deletions(-) create mode 100644 demos/tests/config.txt.example diff --git a/demos/tests/README.md b/demos/tests/README.md index dff426d..b3fddbc 100644 --- a/demos/tests/README.md +++ b/demos/tests/README.md @@ -15,12 +15,17 @@ This suite is not part of default `sbt test`. ## Files - `tests.nlogox` - Manual test model -- `config.txt` - Local config used by the model +- `config.txt.example` - Template with placeholders for all 6 providers +- `config.txt` - Local config used by the model (gitignored, create from template) ## Prerequisites 1. Build and install the extension -2. Configure valid credentials in `config.txt` for cloud providers, or run Ollama locally +2. Create a local config from the template: + ``` + cp config.txt.example config.txt + ``` + Then edit `config.txt`, set `provider=` to your chosen provider, and replace the matching `*_api_key=REPLACE_ME` line with a real key. Or run Ollama locally for a no-key option. 3. Ensure network access for cloud providers ## How To Run diff --git a/demos/tests/config.txt.example b/demos/tests/config.txt.example new file mode 100644 index 0000000..4408119 --- /dev/null +++ b/demos/tests/config.txt.example @@ -0,0 +1,47 @@ +# NetLogo LLM Extension — Test Configuration (Template) +# +# Copy this file to `config.txt` in the same directory and fill in +# the API key for whichever provider you want to test: +# +# cp config.txt.example config.txt +# +# `config.txt` is gitignored so your real key stays local. +# +# To switch providers: change `provider=` and `model=` below, then +# uncomment the matching api_key line. Only one provider runs at a time. + +provider=openrouter +model=openai/gpt-4o-mini + +temperature=0.7 +max_tokens=200 +timeout_seconds=60 + +# --- OpenRouter (https://openrouter.ai/keys) --- +openrouter_api_key=REPLACE_ME +# Sample models: +# openai/gpt-4o-mini +# anthropic/claude-3.5-haiku +# deepseek/deepseek-r1 + +# --- Together AI (https://api.together.ai/settings/api-keys) --- +# together_api_key=REPLACE_ME +# Sample models: +# meta-llama/Llama-3.3-70B-Instruct-Turbo +# deepseek-ai/DeepSeek-R1 + +# --- OpenAI --- +# openai_api_key=sk-REPLACE_ME +# model: gpt-4o-mini + +# --- Anthropic (Claude) --- +# anthropic_api_key=sk-ant-REPLACE_ME +# model: claude-3-5-haiku-latest + +# --- Google Gemini --- +# gemini_api_key=REPLACE_ME +# model: gemini-1.5-flash + +# --- Ollama (local) --- +# ollama_base_url=http://localhost:11434 +# model: llama3.2 diff --git a/demos/tests/tests.nlogox b/demos/tests/tests.nlogox index cd07b98..bcd32be 100644 --- a/demos/tests/tests.nlogox +++ b/demos/tests/tests.nlogox @@ -1,274 +1,185 @@ - = 4) "at least 4 supported providers" - - let ready llm:providers - assert-true (is-list? ready) "providers returns a list" - - ; Every ready provider must appear in the full list - let ok true - foreach ready [ p -> - if not member? p all-providers [ set ok false ] + assert-true (length all-providers >= 6) "providers-all has >= 6 providers" + assert-true (member? "openrouter" all-providers) "openrouter registered" + assert-true (member? "together" all-providers) "together registered" + assert-true (member? "openai" all-providers) "openai registered" + assert-true (member? "anthropic" all-providers) "anthropic registered" + assert-true (member? "gemini" all-providers) "gemini registered" + assert-true (member? "ollama" all-providers) "ollama registered" + + foreach all-providers [ p -> + let help llm:provider-help p + assert-true (is-string? help and not empty? help) (word "provider-help " p) ] - assert-true ok "ready providers are a subset of all providers" let status llm:provider-status - assert-true (is-list? status) "provider-status returns a list" - assert-true (length status >= 4) "status covers all providers" - - print "" -end - -;; --------------------------------------------------------------------------- -;; 2. Provider help -;; --------------------------------------------------------------------------- - -to test-provider-help - print "--- test-provider-help ---" - foreach ["openai" "anthropic" "gemini" "ollama"] [ p -> - let help llm:provider-help p - assert-true (is-string? help and not empty? help) (word "help for " p) - ] - print "" + assert-true (length status >= 6) "provider-status covers all providers" end ;; --------------------------------------------------------------------------- -;; 3. Invalid provider name +;; 2. Invalid provider rejected (PR fix f539a73) ;; --------------------------------------------------------------------------- to test-invalid-provider - print "--- test-invalid-provider ---" + section "invalid-provider" carefully [ llm:set-provider "not-a-provider" test-fail "invalid provider should be rejected" ] [ test-pass (word "invalid provider rejected: " error-message) ] - print "" end ;; --------------------------------------------------------------------------- -;; 4. Active config & model listing +;; 3. Load config + active state ;; --------------------------------------------------------------------------- -to test-active-config - print "--- test-active-config ---" - load-config +to test-load-config + section "load-config" + llm:load-config "config.txt" + test-pass "config.txt loaded" let active llm:active - assert-true (is-list? active and length active = 2) "active returns [provider model]" + assert-true (length active = 2) "active returns [provider model]" + print (word " active provider: " (item 0 active)) + print (word " active model: " (item 1 active)) let cfg llm:config - assert-true (is-string? cfg and not empty? cfg) "config returns non-empty string" + assert-true (not empty? cfg) "config string is non-empty" let models llm:list-models assert-true (member? "=== Available Models ===" models) "list-models has header" - assert-true (member? "[ACTIVE]" models) "list-models shows [ACTIVE] marker" - - print "" + assert-true (member? "[ACTIVE]" models) "list-models marks [ACTIVE]" end ;; --------------------------------------------------------------------------- -;; 5. Model validation +;; 4. Config rollback on failure (PR fix d9a33de) ;; --------------------------------------------------------------------------- -to test-model-validation - print "--- test-model-validation ---" - load-config - - ; Current model should be accepted - let active llm:active - let current-model item 1 active - carefully [ - llm:set-model current-model - test-pass (word "valid model '" current-model "' accepted") - ] [ - test-fail (word "valid model rejected: " error-message) - ] - - ; Unknown model is allowed with a warning (by design — API validates it) +to test-config-rollback + section "config-rollback" + let before llm:active carefully [ - llm:set-model "nonexistent-model-xyz" - test-pass "unknown model accepted (warns to stderr, API will validate)" + llm:load-config "/tmp/__nonexistent_llm_config__.txt" + test-fail "load-config with bad path should error" ] [ - test-fail (word "set-model should not throw: " error-message) + test-pass (word "bad config rejected: " error-message) ] - - ; Restore valid model so subsequent tests work - llm:set-model current-model - - print "" + let after llm:active + assert-true (before = after) "active config unchanged after failed load" end ;; --------------------------------------------------------------------------- -;; 6. Synchronous chat +;; 5. Sync chat (backward compat: returns string, not list) ;; --------------------------------------------------------------------------- to test-sync-chat - print "--- test-sync-chat ---" - load-config + section "sync-chat" llm:clear-history - - carefully [ - let result llm:chat "Reply with exactly: hello" - assert-string result "sync chat result" - assert-true (not empty? result) "sync chat result non-empty" - ] [ - test-fail (word "sync chat error: " error-message) - ] - - print "" + let result llm:chat "Reply with exactly: hello" + assert-true (is-string? result) "llm:chat returns string" + assert-true (not empty? result) "result non-empty" + print (word " reply: " result) end ;; --------------------------------------------------------------------------- -;; 7. Async chat +;; 6. Async chat ;; --------------------------------------------------------------------------- to test-async-chat - print "--- test-async-chat ---" - load-config + section "async-chat" llm:clear-history - - carefully [ - let reporter-task llm:chat-async "Reply with exactly: async hello" - test-pass "async request started" - let result runresult reporter-task - assert-string result "async chat result" - assert-true (not empty? result) "async chat result non-empty" - ] [ - test-fail (word "async chat error: " error-message) - ] - - print "" + let task llm:chat-async "Reply with exactly: async hello" + test-pass "async request started" + let result runresult task + assert-true (is-string? result) "async result is string" + assert-true (not empty? result) "async result non-empty" + print (word " reply: " result) end ;; --------------------------------------------------------------------------- -;; 8. Choose +;; 7. Choose ;; --------------------------------------------------------------------------- to test-choose - print "--- test-choose ---" - load-config + section "choose" llm:clear-history - - carefully [ - let choices ["red" "blue" "green"] - let chosen llm:choose "Pick a color" choices - assert-true (member? chosen choices) (word "chose valid option: " chosen) - ] [ - test-fail (word "choose error: " error-message) - ] - - print "" + let choices ["red" "blue" "green"] + let chosen llm:choose "Pick a color" choices + assert-true (member? chosen choices) (word "chose: " chosen) end ;; --------------------------------------------------------------------------- -;; 9. History +;; 8. History ;; --------------------------------------------------------------------------- to test-history - print "--- test-history ---" - load-config + section "history" llm:clear-history + assert-true (empty? llm:history) "history empty after clear" - let h0 llm:history - assert-true (is-list? h0 and empty? h0) "history empty after clear" - - carefully [ - let chat-result llm:chat "Say ok" - let h1 llm:history - assert-true (length h1 = 2) "history has 2 entries after one exchange" - - let user-msg item 0 h1 - assert-true (item 0 user-msg = "user") "first entry is user role" - - let asst-msg item 1 h1 - assert-true (item 0 asst-msg = "assistant") "second entry is assistant role" - ] [ - test-fail (word "history test error: " error-message) - ] - + let _ llm:chat "Say ok" + let h llm:history + assert-true (length h = 2) "history has 2 entries after one exchange" + assert-true (item 0 (item 0 h) = "user") "entry 0 role = user" + assert-true (item 0 (item 1 h) = "assistant") "entry 1 role = assistant" llm:clear-history - print "" end ;; --------------------------------------------------------------------------- -;; 10. Thinking / reasoning primitives (config only, no API call) +;; 9. Thinking config primitives (no API call) ;; --------------------------------------------------------------------------- to test-thinking-config - print "--- test-thinking-config ---" + section "thinking-config" - ; set-thinking true/false - carefully [ llm:set-thinking true test-pass "set-thinking true" ] [ - test-fail (word "set-thinking true: " error-message) - ] - carefully [ llm:set-thinking false test-pass "set-thinking false" ] [ - test-fail (word "set-thinking false: " error-message) - ] + llm:set-thinking true test-pass "set-thinking true" + llm:set-thinking false test-pass "set-thinking false" - ; reasoning-effort valid values foreach ["low" "medium" "high"] [ effort -> - carefully [ - llm:set-reasoning-effort effort - test-pass (word "reasoning-effort " effort) - ] [ - test-fail (word "reasoning-effort " effort ": " error-message) - ] + llm:set-reasoning-effort effort + test-pass (word "reasoning-effort " effort) ] - ; reasoning-effort invalid carefully [ llm:set-reasoning-effort "invalid" test-fail "invalid reasoning-effort should be rejected" @@ -276,200 +187,149 @@ to test-thinking-config test-pass "invalid reasoning-effort rejected" ] - ; thinking-budget valid - carefully [ - llm:set-thinking-budget 4096 - test-pass "thinking-budget 4096" - ] [ - test-fail (word "thinking-budget 4096: " error-message) - ] + llm:set-thinking-budget 4096 + test-pass "thinking-budget 4096 accepted" - ; thinking-budget too low carefully [ llm:set-thinking-budget 512 - test-fail "thinking-budget 512 should be rejected (< 1024)" + test-fail "thinking-budget 512 should be rejected" ] [ - test-pass "thinking-budget 512 rejected" + test-pass "thinking-budget 512 rejected (< 1024)" ] llm:set-thinking false - print "" end ;; --------------------------------------------------------------------------- -;; 11. chat-with-thinking return format (requires API call) +;; 10. chat-with-thinking returns [answer thinking] ;; --------------------------------------------------------------------------- to test-chat-with-thinking - print "--- test-chat-with-thinking ---" - load-config + section "chat-with-thinking" llm:clear-history llm:set-thinking true - carefully [ - let result llm:chat-with-thinking "What is 2 + 2?" - - assert-true (is-list? result) "chat-with-thinking returns a list" - assert-true (length result = 2) "result has 2 elements [answer thinking]" - - let answer item 0 result - let thinking item 1 result - - assert-string answer "answer element" - assert-string thinking "thinking element" - assert-true (not empty? answer) "answer is non-empty" - - ; Verify thinking text is NOT in history - let history llm:history - if length history >= 2 [ - let asst-content item 1 (item 1 history) - if not empty? thinking [ - assert-true (not member? thinking asst-content) "thinking excluded from history" - ] - ] - ] [ - test-fail (word "chat-with-thinking error: " error-message) - ] + let result llm:chat-with-thinking "What is 2 + 2?" + assert-true (is-list? result) "result is a list" + assert-true (length result = 2) "result has 2 elements" + + let answer item 0 result + let thinking item 1 result + assert-true (is-string? answer) "answer is string" + assert-true (is-string? thinking) "thinking is string" + assert-true (not empty? answer) "answer non-empty" + print (word " answer: " answer) + print (word " thinking: " (substring thinking 0 (min (list 80 length thinking))) "...") llm:set-thinking false llm:clear-history - print "" end ;; --------------------------------------------------------------------------- -;; 12. Backward compatibility: regular chat returns string +;; 11. OpenRouter — vendor-prefixed model names +;; (only runs if active provider is openrouter) ;; --------------------------------------------------------------------------- -to test-backward-compat - print "--- test-backward-compat ---" - load-config - llm:clear-history +to test-openrouter-vendor-prefix + section "openrouter-vendor-prefix" + if not (item 0 llm:active = "openrouter") [ + print " SKIP: active provider is not openrouter" + stop + ] - carefully [ - let result llm:chat "Say hello" - assert-true (is-string? result) "llm:chat returns string, not list" - ] [ - test-fail (word "backward compat error: " error-message) + let original-model item 1 llm:active + llm:set-model "openai/gpt-4o-mini" + test-pass "set-model openai/gpt-4o-mini accepted" + llm:set-model "anthropic/claude-3.5-haiku" + test-pass "set-model anthropic/claude-3.5-haiku accepted" + llm:set-model original-model +end + +;; --------------------------------------------------------------------------- +;; 12. Together AI — DeepSeek-R1 thinking extraction ( tags or reasoning field) +;; (only runs if active provider is together AND model contains DeepSeek) +;; --------------------------------------------------------------------------- + +to test-together-thinking + section "together-thinking" + if not (item 0 llm:active = "together") [ + print " SKIP: active provider is not together" + stop + ] + let model item 1 llm:active + if not (member? "DeepSeek" model or member? "deepseek" model) [ + print (word " SKIP: active model " model " is not a DeepSeek reasoning model") + stop ] - print "" + llm:clear-history + llm:set-thinking true + let result llm:chat-with-thinking "What is 17 * 23? Show your reasoning." + let answer item 0 result + let thinking item 1 result + assert-true (not empty? answer) "answer non-empty" + assert-true (not empty? thinking) "thinking extracted (proves tag or reasoning field handled)" + print (word " answer: " answer) + + llm:set-thinking false + llm:clear-history end ;; --------------------------------------------------------------------------- -;; 13. [reasoning] marker in model list +;; 13. [reasoning] marker visible in list-models ;; --------------------------------------------------------------------------- to test-reasoning-marker - print "--- test-reasoning-marker ---" - load-config - + section "reasoning-marker" let output llm:list-models ifelse member? "[reasoning]" output [ - test-pass "[reasoning] marker present in model list" + test-pass "[reasoning] marker present" ] [ - print "INFO: no [reasoning] markers found (depends on provider/registry)" + print " INFO: no [reasoning] markers (depends on registry)" ] - - print "" end ;; --------------------------------------------------------------------------- -;; Run all tests +;; Entry point — click the run-all-tests button ;; --------------------------------------------------------------------------- to run-all-tests + set pass-count 0 + set fail-count 0 + print "========================================" print " LLM EXTENSION TEST SUITE" print "========================================" - print "" - ; Offline / discovery tests - test-provider-discovery - test-provider-help + ; offline tests + test-providers test-invalid-provider - ; Config-dependent tests - test-active-config - test-model-validation + ; load config once — every test below relies on it + test-load-config + test-config-rollback - ; API tests + ; live API tests test-sync-chat test-async-chat test-choose test-history - test-backward-compat - ; Thinking config (no API) + ; thinking test-thinking-config - - ; Thinking API tests test-chat-with-thinking + + ; provider-specific + test-openrouter-vendor-prefix + test-together-thinking test-reasoning-marker + print "" print "========================================" - print " ALL TESTS COMPLETE" + print (word " RESULTS: " pass-count " passed, " fail-count " failed") print "========================================" end - -;; --------------------------------------------------------------------------- -;; Optional: Ollama qwen3 thinking test (requires local Ollama) -;; --------------------------------------------------------------------------- - -to test-ollama-qwen3-thinking - print "--- test-ollama-qwen3-thinking (requires Ollama) ---" - - carefully [ - llm:set-provider "ollama" - ] [ - print (word "Skipping: cannot set Ollama provider: " error-message) - stop - ] - - llm:set-model "qwen3:0.6b" - llm:clear-history - - ; Regular chat (thinking off) - llm:set-thinking false - carefully [ - let result llm:chat "What is 7 * 8? Answer with just the number." - assert-string result "ollama regular chat returns string" - ] [ - test-fail (word "ollama regular chat: " error-message) - ] - - ; Chat with thinking - llm:set-thinking true - llm:clear-history - carefully [ - let result llm:chat-with-thinking "Solve step by step: 120 / 2" - - assert-true (is-list? result and length result = 2) "ollama thinking returns [answer thinking]" - - let answer item 0 result - let thinking item 1 result - - assert-true (not empty? answer) "ollama thinking answer non-empty" - print (word " Answer: " answer) - - ifelse not empty? thinking [ - print (word " Thinking (" length thinking " chars): " - substring thinking 0 min list 100 length thinking) - ] [ - print " (no thinking text returned)" - ] - ] [ - test-fail (word "ollama thinking chat: " error-message) - print " Ensure qwen3:0.6b is installed: ollama pull qwen3:0.6b" - ] - - ; Verify history has clean entries only - let history llm:history - print (word " History entries: " length history) - - llm:set-thinking false - llm:clear-history - print "" -end]]> +]]> From 835a12f2357aca4d865dfd90c842e2203c839ac0 Mon Sep 17 00:00:00 2001 From: JNK234 Date: Fri, 1 May 2026 16:15:11 -0500 Subject: [PATCH 6/6] docs: update for 6-provider architecture and reasoning primitives Bring all top-level docs in line with the data-driven provider registry that now supports 6 providers (openai, anthropic, gemini, ollama, openrouter, together). - README.md: lead description and Quick Links cover all 6 providers - SETUP.md: add OpenRouter and Together AI setup sections, update parameter table and provider-specific defaults; note DeepSeek-R1 thinking-tag behavior + max_tokens guidance - USAGE.md: provider list in `set-provider` reference; add a reasoning/thinking example block (`set-thinking`, `set-reasoning-effort`, `set-thinking-budget`, `chat-with-thinking`); expand discovery examples - API-REFERENCE.md: add quick-table rows for the 4 reasoning primitives, add a full Reasoning / Thinking Primitives section, extend Valid Providers list, readiness-check table, and provider-status example to include OpenRouter and Together - EXAMPLES.md: include OpenRouter and Together in the compare-providers list; add Reasoning Models and OpenRouter one-key-many-models examples - TESTING.md: document `demos/tests/config.txt.example` workflow, enumerate the 13 test procedures (including the two skip-aware provider-specific ones), list OPENROUTER_API_KEY / TOGETHER_API_KEY for future smoke-test secrets --- docs/API-REFERENCE.md | 89 ++++++++++++++++++++++++++++++++++++++----- docs/EXAMPLES.md | 52 ++++++++++++++++++++++++- docs/README.md | 4 +- docs/SETUP.md | 64 ++++++++++++++++++++++++++++++- docs/TESTING.md | 26 ++++++++++++- docs/USAGE.md | 22 +++++++++-- 6 files changed, 237 insertions(+), 20 deletions(-) diff --git a/docs/API-REFERENCE.md b/docs/API-REFERENCE.md index b445eec..dff8b04 100644 --- a/docs/API-REFERENCE.md +++ b/docs/API-REFERENCE.md @@ -11,12 +11,16 @@ The NetLogo Multi-LLM Extension provides a unified interface for multiple Large | `llm:chat text` | Chat | Send synchronous chat message, returns response | | `llm:chat-async text` | Chat | Send asynchronous chat message, returns awaitable reporter | | `llm:chat-with-template file vars` | Chat | Send templated prompt with variable substitution | +| `llm:chat-with-thinking text` | Chat | Returns `[answer thinking]` for reasoning-capable models | | `llm:choose prompt choices` | Chat | Force selection from provided options | +| `llm:set-thinking bool` | Reasoning | Enable/disable reasoning mode for current provider | +| `llm:set-reasoning-effort level` | Reasoning | Set effort: `"low"`, `"medium"`, `"high"` | +| `llm:set-thinking-budget n` | Reasoning | Token budget for thinking (min 1024; Anthropic + Gemini) | | `llm:history` | History | Get current agent's conversation history | | `llm:set-history list` | History | Set conversation history for current agent | | `llm:clear-history` | History | Clear conversation history for current agent | | `llm:load-config filename` | Configuration | Load settings from file | -| `llm:set-provider name` | Configuration | Set active provider (openai, anthropic, gemini, ollama) | +| `llm:set-provider name` | Configuration | Set active provider (openai, anthropic, gemini, ollama, openrouter, together) | | `llm:set-api-key key` | Configuration | Set API key for current provider | | `llm:set-model name` | Configuration | Set model to use for current provider | | `llm:providers` | Discovery | List ready providers with configured keys/servers | @@ -70,6 +74,8 @@ llm:load-config "models/gpt4-config.txt" - `"anthropic"` - Anthropic Claude models - `"gemini"` - Google Gemini models - `"ollama"` - Local Ollama models +- `"openrouter"` - OpenRouter (200+ models from many vendors via one API key) +- `"together"` - Together AI (fast open-source model inference) **Example**: @@ -229,6 +235,65 @@ set color read-from-string color-choice - Useful for agent decision-making in models - Maintains conversation context +## Reasoning / Thinking Primitives + +For models that expose intermediate reasoning (Anthropic Claude, Google Gemini, Ollama qwen3/deepseek-r1, OpenRouter, Together AI). OpenAI o-series uses internal reasoning that the API does not return. + +### llm:chat-with-thinking + +**Syntax**: `llm:chat-with-thinking text` + +**Description**: Same as `llm:chat`, but returns both the final answer and the model's reasoning text as a 2-element list. + +**Returns**: `[answer thinking]` — both strings. `thinking` will be `""` if the provider/model does not expose reasoning tokens. + +**Example**: + +```netlogo +llm:set-thinking true +let result llm:chat-with-thinking "What is 17 * 23?" +let answer item 0 result +let thinking item 1 result +print (word "Answer: " answer) +print (word "Reasoning: " thinking) +``` + +**Notes**: + +- Only the final answer is added to conversation history (not the thinking text). +- For DeepSeek-R1 on Together AI, thinking is parsed from `...` tags in the content. +- For Anthropic, OpenRouter, and Gemini, thinking comes from a dedicated reasoning field in the API response. + +### llm:set-thinking + +**Syntax**: `llm:set-thinking enabled?` + +**Description**: Enable or disable reasoning mode for the current provider. When enabled, the request includes provider-specific reasoning fields. + +**Parameters**: + +- `enabled?` (boolean): `true` to enable, `false` to disable + +### llm:set-reasoning-effort + +**Syntax**: `llm:set-reasoning-effort level` + +**Description**: Set the reasoning effort hint for models that support it (OpenAI o-series, OpenRouter, Together AI hybrid models). + +**Parameters**: + +- `level` (string): `"low"`, `"medium"`, or `"high"` + +### llm:set-thinking-budget + +**Syntax**: `llm:set-thinking-budget tokens` + +**Description**: Maximum tokens the model may spend on reasoning before producing the final answer. Used by Anthropic and Gemini. + +**Parameters**: + +- `tokens` (number): Minimum 1024. For Anthropic, the value is clamped to `[1024, max_tokens-1]`. + ## History Management ### llm:history @@ -320,12 +385,14 @@ if member? "ollama" llm:providers [ **Readiness Checks**: -| Provider | Check Performed | -| --------- | --------------------------------------------------- | -| OpenAI | Has `openai_api_key` or `api_key` configured | -| Anthropic | Has `anthropic_api_key` or `api_key` configured | -| Gemini | Has `gemini_api_key` or `api_key` configured | -| Ollama | Server reachable at `ollama_base_url` (1s timeout) | +| Provider | Check Performed | +| ----------- | ----------------------------------------------------- | +| OpenAI | Has `openai_api_key` or `api_key` configured | +| Anthropic | Has `anthropic_api_key` or `api_key` configured | +| Gemini | Has `gemini_api_key` or `api_key` configured | +| Ollama | Server reachable at `ollama_base_url` (1s timeout) | +| OpenRouter | Has `openrouter_api_key` configured | +| Together AI | Has `together_api_key` configured | ### llm:providers-all @@ -339,7 +406,7 @@ if member? "ollama" llm:providers [ ```netlogo let all-providers llm:providers-all -print all-providers ; ["openai" "anthropic" "gemini" "ollama"] +print all-providers ; ["openai" "anthropic" "gemini" "ollama" "openrouter" "together"] ``` ### llm:provider-status @@ -359,7 +426,9 @@ print status ; [["openai" ["ready" true] ["has-key" true]] ; ["anthropic" ["ready" false] ["has-key" false]] ; ["gemini" ["ready" false] ["has-key" false]] -; ["ollama" ["ready" true] ["reachable" true] ["base-url" "http://localhost:11434"]]] +; ["ollama" ["ready" true] ["reachable" true] ["base-url" "http://localhost:11434"]] +; ["openrouter" ["ready" false] ["has-key" false]] +; ["together" ["ready" false] ["has-key" false]]] ; Check specific provider status foreach llm:provider-status [ provider-info -> @@ -394,7 +463,7 @@ foreach llm:provider-status [ provider-info -> **Parameters**: -- `provider-name` (string): Provider to get help for (`openai`, `anthropic`, `gemini`, `ollama`) +- `provider-name` (string): Provider to get help for (`openai`, `anthropic`, `gemini`, `ollama`, `openrouter`, `together`) **Returns**: String - Multi-line setup instructions diff --git a/docs/EXAMPLES.md b/docs/EXAMPLES.md index aaf2594..91c4bbd 100644 --- a/docs/EXAMPLES.md +++ b/docs/EXAMPLES.md @@ -306,7 +306,7 @@ extensions [llm] globals [providers-list current-question] to setup - set providers-list ["openai" "anthropic" "gemini"] + set providers-list ["openai" "anthropic" "gemini" "openrouter" "together"] set current-question "" end @@ -479,6 +479,56 @@ end ## Advanced Usage Examples +### Reasoning Models — Show the Model's Work + +Use `llm:chat-with-thinking` to get both the final answer and the model's intermediate reasoning. Works with Anthropic Claude, Gemini 2.x, Ollama qwen3 / deepseek-r1, OpenRouter reasoning models, and Together AI's DeepSeek-R1. + +```netlogo +extensions [llm] + +to setup-reasoning + llm:load-config "config.txt" ;; e.g. provider=together, model=deepseek-ai/DeepSeek-R1 + llm:set-thinking true + llm:set-reasoning-effort "high" ;; OpenAI o-series + OpenRouter + Together hybrid + llm:set-thinking-budget 4096 ;; Anthropic + Gemini +end + +to ask-with-reasoning [ question ] + let result llm:chat-with-thinking question + let answer item 0 result + let thinking item 1 result + print (word "Q: " question) + print (word "Reasoning: " thinking) + print (word "Answer: " answer) +end + +;; ask-with-reasoning "If a train leaves Chicago at 3pm going 60 mph, ..." +``` + +Note: bump `max_tokens` to 2000+ in your config for reasoning models; they often spend most of their token budget on the thinking phase before producing the final answer. + +### OpenRouter — One Key, Many Models + +Switch between vendors without juggling separate API keys: + +```netlogo +extensions [llm] + +to setup + llm:set-provider "openrouter" + llm:set-api-key "sk-or-your-key" +end + +to compare-vendors-via-openrouter + let prompt "Summarize agent-based modeling in one sentence." + foreach ["openai/gpt-4o-mini" "anthropic/claude-3.5-haiku" "meta-llama/llama-3.3-70b-instruct"] [ m -> + llm:set-model m + print (word m ": " llm:chat prompt) + llm:clear-history + ] +end +``` + ### Async Processing with Multiple Requests Handle multiple LLM requests simultaneously: diff --git a/docs/README.md b/docs/README.md index 4ad557a..ca79e79 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,6 +1,6 @@ # NetLogo Multi-LLM Extension -Unified LLM capabilities for NetLogo with multi-provider support (OpenAI, Anthropic/Claude, Google/Gemini, and local Ollama), per‑agent memory, async requests, and simple configuration. +Unified LLM capabilities for NetLogo with multi-provider support — OpenAI, Anthropic/Claude, Google/Gemini, local Ollama, OpenRouter (200+ models via one key), and Together AI (fast open-source inference) — plus per‑agent memory, async requests, reasoning/thinking models, and simple configuration. ## Quick Links - Usage Guide: `docs/USAGE.md` @@ -20,7 +20,7 @@ extensions [ llm ] ``` llm:load-config "config.txt" ``` -See `docs/CONFIGURATION.md` for ready-to-copy examples (OpenAI, Anthropic, Gemini, Ollama). For inline setup: +See `docs/CONFIGURATION.md` for ready-to-copy examples (OpenAI, Anthropic, Gemini, Ollama, OpenRouter, Together AI). For inline setup: ``` llm:set-provider "openai" llm:set-api-key "sk-REPLACE_ME" diff --git a/docs/SETUP.md b/docs/SETUP.md index dbc29b4..9e59f1b 100644 --- a/docs/SETUP.md +++ b/docs/SETUP.md @@ -101,13 +101,63 @@ timeout_seconds=60 **Pull command**: `ollama pull [model-name]` +### OpenRouter (200+ models via one key) + +OpenRouter routes requests to many model vendors (OpenAI, Anthropic, Google, Meta, DeepSeek, etc.) through a single API. + +1. **Get API Key**: Visit [openrouter.ai/keys](https://openrouter.ai/keys) +2. **Create config.txt**: +``` +provider=openrouter +model=openai/gpt-4o-mini +openrouter_api_key=sk-or-your-key-here +temperature=0.7 +max_tokens=1000 +``` + +**Available Models** (vendor-prefixed): +- `openai/gpt-4o-mini` - Fast, cost-effective (recommended) +- `openai/gpt-4o` - GPT-4o through OpenRouter +- `anthropic/claude-3.5-sonnet` - Claude 3.5 Sonnet +- `anthropic/claude-3.5-haiku` - Fast Claude +- `google/gemini-2.0-flash-exp` - Latest Gemini +- `meta-llama/llama-3.3-70b-instruct` - Llama 3.3 70B +- `deepseek/deepseek-r1` - Reasoning model + +Browse the full catalog: [openrouter.ai/models](https://openrouter.ai/models) + +### Together AI (open-source models) + +Together AI provides fast inference for open-source models (Llama, DeepSeek, Qwen, Mistral, Gemma) via an OpenAI-compatible API. + +1. **Get API Key**: Visit [api.together.ai/settings/api-keys](https://api.together.ai/settings/api-keys) +2. **Create config.txt**: +``` +provider=together +model=meta-llama/Llama-3.3-70B-Instruct-Turbo +together_api_key=your-together-key-here +temperature=0.7 +max_tokens=1000 +``` + +**Available Models** (vendor-prefixed): +- `meta-llama/Llama-3.3-70B-Instruct-Turbo` - Fast Llama 3.3 (recommended) +- `meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo` - Llama 3.1 70B +- `deepseek-ai/DeepSeek-R1` - Reasoning model with `` tag output +- `Qwen/Qwen2.5-72B-Instruct-Turbo` - Qwen 2.5 72B +- `mistralai/Mixtral-8x22B-Instruct-v0.1` - Mistral mixture-of-experts + +Browse the full catalog: [api.together.ai/models](https://api.together.ai/models) + +**Reasoning model note:** DeepSeek-R1 emits its thinking inside `...` tags (sometimes filling the entire response). Use `llm:chat-with-thinking` to get the answer and reasoning split out, and bump `max_tokens` to 2000+ so the model has room to finish its answer after thinking. + ## Configuration Parameters ### Core Settings | Parameter | Description | Required | Default | |-----------|-------------|----------|---------| -| `provider` | LLM provider (`openai`, `anthropic`, `gemini`, `ollama`) | Yes | - | +| `provider` | LLM provider (`openai`, `anthropic`, `gemini`, `ollama`, `openrouter`, `together`) | Yes | - | | `model` | Model identifier | Yes | Provider-specific | | `api_key` | API authentication key | Yes* | - | | `temperature` | Response randomness (0.0-1.0) | No | 0.7 | @@ -141,6 +191,16 @@ timeout_seconds=60 - `max_tokens`: 2048 - `timeout_seconds`: 60 +**OpenRouter**: +- `base_url`: `https://openrouter.ai/api/v1` +- `max_tokens`: 1000 +- API key config field: `openrouter_api_key` + +**Together AI**: +- `base_url`: `https://api.together.xyz/v1` +- `max_tokens`: 1000 +- API key config field: `together_api_key` + ## Testing Your Setup 1. **Create Test Model**: @@ -173,7 +233,7 @@ end **"Provider not found"** - Check `provider=` line in config.txt -- Verify spelling: `openai`, `anthropic`, `gemini`, `ollama` +- Verify spelling: `openai`, `anthropic`, `gemini`, `ollama`, `openrouter`, `together` **"API key invalid"** - Verify API key is correct and active diff --git a/docs/TESTING.md b/docs/TESTING.md index e564a70..1023011 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -82,9 +82,31 @@ These tests are **excluded from default `sbt test`** and from CI. Use `demos/tests/tests.nlogox` when you want to verify real providers end-to-end. These tests do require: -- valid provider credentials for cloud providers, or +- valid provider credentials for cloud providers (OpenAI, Anthropic, Gemini, OpenRouter, Together AI), or - a running Ollama server for local provider tests +### Test setup + +1. Copy the template: `cp demos/tests/config.txt.example demos/tests/config.txt` +2. Edit `config.txt`: set `provider=` to your chosen provider and replace the matching `*_api_key=REPLACE_ME` line with a real key. `config.txt` is gitignored, so the key stays local. +3. Open `demos/tests/tests.nlogox` in NetLogo. +4. In the Command Center: `run-all-tests` + +### What the suite covers + +The suite runs the same 13 test procedures regardless of provider, plus two provider-specific procedures that auto-skip if not applicable: + +- `test-providers` — registry has all 6 providers and `provider-help` returns text for each +- `test-invalid-provider` — bogus provider name is rejected +- `test-load-config` / `test-config-rollback` — config file loads cleanly and rolls back on failure +- `test-sync-chat`, `test-async-chat`, `test-choose`, `test-history` — core chat flow +- `test-thinking-config`, `test-chat-with-thinking` — reasoning primitives and `[answer thinking]` return shape +- `test-openrouter-vendor-prefix` — vendor-prefixed model names (skips unless `provider=openrouter`) +- `test-together-thinking` — DeepSeek-R1 `` tag extraction (skips unless `provider=together` AND model contains "DeepSeek") +- `test-reasoning-marker` — `[reasoning]` marker visible in `llm:list-models` + +To exercise everything, run the suite twice — once with `provider=openrouter`, once with `provider=together` (use `model=deepseek-ai/DeepSeek-R1` and `max_tokens=2000+` for the Together reasoning test). + Suggested usage: - run automated tests first (`sbt test`) - run manual integration checks before releases or when changing provider adapters @@ -125,5 +147,5 @@ For cloud-provider smoke tests you will need real API keys in CI secrets. Typical setup later: - separate workflow (manual trigger and/or nightly schedule) -- secrets like `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY` +- secrets like `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, `OPENROUTER_API_KEY`, `TOGETHER_API_KEY` - not required for normal PR merges (to avoid flaky/costly gating) diff --git a/docs/USAGE.md b/docs/USAGE.md index 671752b..48d7511 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -35,7 +35,7 @@ llm:set-model "gpt-4o-mini" extensions [ llm ] llm:load-config "config.txt" ``` - - See full instructions and templates in `docs/CONFIGURATION.md` (OpenAI, Anthropic, Gemini, and Ollama). + - See full instructions and templates in `docs/CONFIGURATION.md` (OpenAI, Anthropic, Gemini, Ollama, OpenRouter, Together AI). ### Using Ollama (no API key) - Start Ollama and pull a model (see quick start in `docs/CONFIGURATION.md`). @@ -56,7 +56,7 @@ show llm:chat "List two agent-based modeling use cases." All primitives are under the `llm:` namespace. - Configuration - - `llm:set-provider "openai|anthropic|gemini|ollama"` + - `llm:set-provider "openai|anthropic|gemini|ollama|openrouter|together"` - `llm:set-api-key "..."` - `llm:set-model "model-name"` - `llm:load-config "path/to/config.txt"` @@ -104,10 +104,26 @@ Each agent (observer/turtles/patches) maintains its own conversation state. - Discovery ``` -show llm:providers ;; available providers +show llm:providers ;; ready providers (with key/server) +show llm:providers-all ;; all 6 supported providers +show llm:provider-status ;; detailed readiness per provider +show llm:provider-help "openrouter" ;; setup instructions for any provider show llm:list-models ;; models supported by current provider +show llm:active ;; [provider model] currently active ``` +- Reasoning / thinking models +``` +llm:set-thinking true ;; turn on reasoning for models that support it +llm:set-reasoning-effort "high" ;; "low" | "medium" | "high" +llm:set-thinking-budget 4096 ;; min 1024, used by Anthropic + Gemini +let r llm:chat-with-thinking "What is 17 * 23?" +print item 0 r ;; final answer +print item 1 r ;; reasoning text (empty for providers that hide thinking, e.g. OpenAI) +``` +Thinking is exposed by Anthropic, Gemini, Ollama, OpenRouter, and Together AI. +OpenAI o-series uses internal reasoning that the API does not return. + ## Common Patterns - Per‑turtle conversations ```