Skip to content

fix: rename semantic drift to lexical drift, update model names #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JNK234 merged 1 commit into from
Mar 19, 2026
Merged

fix: rename semantic drift to lexical drift, update model names #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 9 additions & 12 deletions demos/provider-sensitivity/config-multi-provider.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
# Demo 3: Provider Sensitivity - Multi-provider configuration
# Fill in keys for the providers you want to compare.
# Provider Sensitivity - Multi-provider config
# Fill in your API keys for each provider you want to compare.
# The demo will automatically skip providers that are not ready.

# Default provider/model on startup
provider=ollama
model=llama3.2:3b
provider=openai

@chatgpt-codex-connector chatgpt-codex-connector Bot Mar 19, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep this template from hard-failing without OpenAI keys

Using openai as the active provider breaks the advertised “skip providers that are not ready” flow. setup first does llm:load-config "config-multi-provider.txt" (demos/provider-sensitivity/provider-sensitivity.nlogox:43-57), and LoadConfigCommand aborts if the active provider lacks credentials (src/main/LLMExtension.scala:373-391). That means a user who only fills anthropic_api_key, gemini_api_key, or only has Ollama running never reaches llm:providers; the demo falls back to config instead of using the ready providers from this file. In practice this change makes OpenAI credentials mandatory for a multi-provider template that is supposed to tolerate missing providers.

Useful? React with 👍 / 👎.

model=gpt-5.4-mini
temperature=0.4
max_tokens=250
max_tokens=1000
timeout_seconds=45

# Cloud API keys (leave empty to skip provider, fill in to enable)
Expand All @@ -15,12 +14,10 @@ anthropic_api_key=
gemini_api_key=

# Ollama local endpoint (no key required)
# Start with: ollama serve
# Ensure your selected model is pulled, for example: ollama pull llama3.2
ollama_base_url=http://localhost:11434

# Optional per-provider overrides
openai_model=gpt-4o-mini
anthropic_model=claude-3-5-haiku-latest
gemini_model=gemini-2.0-flash
# Per-provider model overrides
openai_model=gpt-5.4-mini
anthropic_model=claude-haiku-4-5-20251001
gemini_model=gemini-2.5-flash
ollama_model=llama3.2:3b
16 changes: 8 additions & 8 deletions demos/provider-sensitivity/provider-sensitivity.nlogox
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<model version="NetLogo 7.0.3" snapToGrid="true">
<code><![CDATA[; ABOUTME: Telephone Game demo — parallel provider chains paraphrase a seed message, revealing semantic drift.
<code><![CDATA[; ABOUTME: Telephone Game demo — parallel provider chains paraphrase a seed message, revealing lexical drift.
; ABOUTME: Uses turtles, directed links, Jaccard-based drift scoring, optional thinking mode.

extensions [llm]
Expand All @@ -12,7 +12,7 @@ turtles-own [
chain-position ;; 0 = seed, 1..N-1 = paraphrase positions
current-message ;; message this turtle holds
original-message ;; seed message (for drift comparison)
drift-score ;; [0,1] semantic drift from original
drift-score ;; [0,1] lexical drift from original (Jaccard distance on word sets)
length-ratio ;; response length / original length
processed? ;; has this turtle completed its paraphrase?
error? ;; did this turtle's LLM call fail?
Expand Down Expand Up @@ -478,9 +478,9 @@ end
;; ===== DEFAULT MODELS =====

to-report get-default-model [prov]
if prov = "openai" [ report "gpt-4o-mini" ]
if prov = "anthropic" [ report "claude-3-5-haiku-latest" ]
if prov = "gemini" [ report "gemini-2.0-flash" ]
if prov = "openai" [ report "gpt-5.4-mini" ]
if prov = "anthropic" [ report "claude-haiku-4-5-20251001" ]
if prov = "gemini" [ report "gemini-2.5-flash" ]
if prov = "ollama" [ report "llama3.2:3b" ]
report "unknown"
end
Expand Down Expand Up @@ -644,8 +644,8 @@ end
<slider x="15" step="1" y="170" max="10" width="230" display="chain-length" height="50" min="3" direction="Horizontal" default="6.0" variable="chain-length"></slider>
<switch x="15" y="225" height="40" on="true" variable="show-labels?" width="230" display="show-labels?"></switch>
<switch x="15" y="265" height="40" on="false" variable="thinking-mode?" width="230" display="thinking-mode?"></switch>
<plot x="45" autoPlotX="true" yMax="1.0" autoPlotY="true" yAxis="Drift" y="505" xMin="0.0" height="165" legend="true" xMax="10.0" yMin="0.0" width="550" xAxis="Chain Position" display="Semantic Drift by Provider">
<setup>set-current-plot "Semantic Drift by Provider"</setup>
<plot x="45" autoPlotX="true" yMax="1.0" autoPlotY="true" yAxis="Drift" y="505" xMin="0.0" height="165" legend="true" xMax="10.0" yMin="0.0" width="550" xAxis="Chain Position" display="Lexical Drift by Provider">
<setup>set-current-plot "Lexical Drift by Provider"</setup>
<update><![CDATA[let idx 0
foreach ready-providers-list [ prov ->
create-temporary-plot-pen prov
Expand Down Expand Up @@ -680,7 +680,7 @@ foreach ready-providers-list [ prov ->
</widgets>
<info>## WHAT IS IT?

A Telephone Game for LLMs. Parallel chains of turtles pass a seed message through repeated paraphrasing by different LLM providers. Each position in the chain asks its provider to rephrase what it received. As the message travels, semantic drift accumulates — and different providers drift at different rates and in different ways.
A Telephone Game for LLMs. Parallel chains of turtles pass a seed message through repeated paraphrasing by different LLM providers. Each position in the chain asks its provider to rephrase what it received. As the message travels, lexical drift accumulates — vocabulary shifts as each provider substitutes its own word choices, and different providers drift at different rates.

## HOW IT WORKS

Expand Down
Loading