feature:Merge chatbot config with new retrieval strategies by ju3ouz4n · Pull Request #22 · CentraleDigitaleLab/canar

ju3ouz4n · 2026-07-23T07:13:22Z

Summary

Unifies chatbot behavior from original repositiory Romanovytch#24 across the app and benchmark so configuration, retrieval, and generation remain consistent as assistants evolve.

Closes #

Changes

Added validated chatbot configuration for prompts, profiles, uploads, exports, and display metadata.
Replaced agent-specific handlers with a shared retrieval, context assembly, and generation flow.
Added chat profiles pairing generation settings with retrieval strategies, including prompt-only assistants.
Aligned end-to-end benchmarks with the production chatbot flow and expanded tests and documentation.

Modified files and purpose

.github/copilot-instructions.md — Documents the configuration-driven architecture.
.gitignore — Ignores local environments and logs.
ARCHITECTURE.md — Documents profiles, chatbot loading, message assembly, and migration requirements.
benchmark/CONFIG.md — Explains chatbot and retrieval profile selection.
benchmark/README.md — Updates benchmark guidance.
benchmark/config.yaml — Selects r_helpdesk as the benchmark chatbot.
benchmark/e2e/README.md — Documents production-aligned benchmark behavior.
benchmark/e2e/eval_e2e.py — Reuses application prompts, profiles, generation settings, and message assembly.
benchmark/harness/bench_config.py — Loads chatbot configuration and resolves benchmark profiles.
benchmark/harness/tests/test_bench_config.py — Tests chatbot loading and profile selection.
canar/app/agents/generic_agent.py — Removes the superseded generic handler.
canar/app/agents/r_helpdesk.py — Removes the superseded R helpdesk handler.
canar/app/agents/sas_to_r.py — Removes the superseded SAS-to-R handler.
canar/app/api/llm_client.py — Adds synchronous requests and optional tool schemas.
canar/app/chatbots/chatbot_config.py — Defines and persists validated chatbot configuration.
canar/app/chatbots/chatbotconfig.exemple.yaml — Provides an example chatbot configuration.
canar/app/chatbots/chatbotconfig.yaml — Defines production chatbots and prompts.
canar/app/main.py — Routes chatbots through profiles and shared message assembly.
canar/app/profiles.py — Defines and resolves retrieval and generation profiles.
canar/app/tests/__init__.py — Marks application tests as a package.
canar/app/tests/test_chatbot_config.py — Tests chatbot validation and persistence.
canar/app/tests/test_llm_client.py — Tests synchronous and streaming requests.
canar/app/tests/test_llm_utils.py — Tests shared context and message assembly.
canar/app/tests/test_profiles.py — Tests profile construction and resolution.
canar/app/tests/test_r_helpdesk_retrieval_hits.py — Removes tests for the deleted agent handler.
canar/app/tests/test_yaml_loader.py — Tests atomic chatbot loading and rollback.
canar/app/ui/sidebar.py — Builds chatbot labels from loaded configuration.
canar/app/utils/llm_utils.py — Adds shared citation context and prompt assembly.
canar/app/utils/utils_logger.py — Adds file-based application logging.
canar/app/yaml_loader.py — Loads and persists chatbot YAML at startup.
pyproject.toml — Adds PyYAML and packages chatbot YAML files.
requirements.txt — Adds the PyYAML runtime dependency.

How to test

For an existing database:

docker exec canar-postgres psql -U canar -d canar -c \
  'ALTER TABLE chatbotconfig ADD COLUMN IF NOT EXISTS profile_name VARCHAR;'

cd infra
docker compose up -d
cd ..
pytest
make run

Verify both chatbots appear, R helpdesk responses show retrieved sources, and SAS-to-R accepts .sas uploads and exports .R files.

Screenshots (if UI changes)

Not included — verify the chatbot selector, upload control, and export control manually.

Checklist

Tests added/updated
Documentation updated (README/docs)
No secrets in commits or logs
Backward compatibility considered

ju3ouz4n · 2026-07-29T13:48:57Z

@codex review

chatgpt-codex-connector

💡 Codex Review

canar/canar/app/ui/sidebar.py

Lines 35 to 37 in f52f2a3

    
           if st.button("Créer", use_container_width=True, key="create_conv"): 
        
               cid = db.create_conversation(user_id, default_title, agent) 
        
               st.session_state["conv_id"] = cid

Select chatbot IDs without reversing display names

If two valid chatbots share the same display name, labels.index(picked_label) always resolves both selectbox entries to the first ID, making the later chatbot impossible to select or create a conversation with. The schema does not require names to be unique, and the header selector repeats this mapping, so the widgets should retain IDs as their values and use names only for formatting.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-07-29T13:58:57Z

 run:
  dataset: datasets/benchmark_dicovarg21_v0.yaml   # path relative to benchmark/
-  agent: generic_agent
+  agent: r_helpdesk


Keep the DicoVar benchmark on its generic prompt

This benchmark still targets the DicoVar dataset and dico collection, but selecting r_helpdesk now feeds every generated answer a CoachR prompt that requires R examples and SAS→R comparisons. That changes answer relevance and faithfulness for reasons unrelated to retrieval, corrupting comparisons with prior runs and between retrieval profiles; retain a generic YAML chatbot with the previous neutral prompt and select it here.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-29T13:58:57Z

+                response = chat.sync_chat(
+                    messages,
+                    temperature=temperature,
+                    top_p=generation_params.top_p,
+                    max_tokens=max_tokens,


Pass configured tool schemas before entering the tool loop

When a chatbot has a non-empty allowed_tools, this branch calls sync_chat without allowed_tools_schemas because the schema and registry wiring is commented out. The API is therefore never told which tools are available, and the execution block also currently returns None, so enabling this supported YAML field silently produces ordinary synchronous chat instead of the configured tool-assisted behavior.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-29T13:58:57Z

+            existing_bot.profile_name = self.profile_name
+            existing_bot.model = self.model
+            existing_bot.top_k = self.top_k
+            existing_bot.score_threshold = self.score_threshold
+            existing_bot.max_context_tokens = self.max_context_tokens


Persist every YAML-controlled field during chatbot updates

On any restart after a chatbot row already exists, this update path copies only a subset of the validated YAML object and omits accepted_file_types, export_extension, and allowed_tools. Changes to those YAML fields therefore never reach the database, so the UI continues using stale upload, export, and tool settings even though boot reports that the configuration was loaded successfully.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-29T13:58:57Z

+            for bot_data in list_bot:
+                if ChatbotConfig.createChatbot(bot_data):
+                    bot = ChatbotConfig.createChatbot(bot_data)
+                    success = bot.saveChatbot(session, False)


Remove chatbots deleted from the YAML source

The boot loader only upserts entries present in the current YAML and never reconciles rows that are no longer present. After an administrator removes a chatbot and restarts the app, fetch_chatbot_list still selects that stale database row, leaving the removed bot available in the UI or stopping startup if its old profile is no longer registered.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-29T13:58:57Z

+    chatbot_profiles = resolve_chatbot_profiles(chatbot_list, chat_profiles)
+except ValueError as exc:
+    st.error(str(exc))
+    st.stop()
+agent_profiles = build_agent_profile_mapping(chatbot_profiles)


Keep agent retrieval mappings authoritative in profiles.py

This resolution makes the database/YAML profile_name authoritative and passes the resulting mapping into RetrievalService, bypassing the repository's agent mapping in canar/app/retrieval/profiles.py. A configuration edit can therefore move r_helpdesk away from its required simple_vector strategy or give sas_to_r retrieval despite its prompt-only invariant; retain those mappings in the retrieval profile registry rather than deriving them from persisted chatbot metadata.

AGENTS.md reference: AGENTS.md:L31-L34

Useful? React with 👍 / 👎.

canar/app/retrieval/service.py Should use configuration for search.

chatgpt-codex-connector · 2026-07-29T13:58:57Z

            if "```r" in content:
                code = content.split("```r", 1)[1].split("```", 1)[0]


Extract the code fence matching the configured export type

The export UI is now enabled for any chatbot with any export_extension, but the extraction remains hard-coded to an r fence. For example, a chatbot configured with export_extension: py and a python code block downloads the full Markdown response instead of the code, so extraction must be based on the configured export language rather than always searching for ```r.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-29T13:58:57Z

+    except Exception as e:
+        print(f"[Erreur] Une erreur s'est produite : {e}")
+
+    list_bot = data.get("chatbots", [])


Validate the YAML root before reading chatbots

If the configuration file is empty, yaml.safe_load returns None; if its root is a list or scalar, it returns another non-mapping value. In each case this unguarded .get raises outside the exception handler and crashes application startup instead of producing the intended no-valid-chatbot error, so validate that the decoded document is a mapping containing a chatbot list first.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-29T13:58:58Z

+    with Session(_engine) as session:
+        return session.exec(select(ChatbotConfig)).all()


Move chatbot reads behind the DB boundary

The Streamlit entrypoint now imports SQLModel and opens a Session directly to query chatbot rows, bypassing the project-owned DB interface. This couples UI orchestration to the storage implementation and means replacing or changing SQLModel requires editing high-level application code; expose a typed chatbot-list method from the persistence layer instead.

AGENTS.md reference: AGENTS.md:L47-L51

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-29T13:58:58Z

+            api_args["tools"] = allowed_tools_schemas
+            api_args["tool_choice"] = "auto"
+
+        return self.client.chat.completions.create(**api_args)


Convert synchronous LLM responses inside ChatClient

sync_chat returns the OpenAI SDK's raw ChatCompletion, and main.py consequently depends on vendor-specific members such as .choices, .tool_calls, and .model_dump. An otherwise compatible backend or client replacement can therefore break the Streamlit flow solely because its Python response object differs; convert the response to a project-owned message/tool-call type inside this adapter, as the streaming method already does by yielding plain strings.

AGENTS.md reference: AGENTS.md:L47-L50

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-29T13:58:58Z

+chatbots:
+  - id: "r_helpdesk"
+    profile_name: "simple_vector"
+    name: "Assistant R modifié2"


Remove development suffixes from production labels

The bundled production configuration displays Assistant R modifié2 to every user, and the new-user login path similarly creates Nouvelle conversation modif. These are development markers rather than the established French labels, so shipping this configuration visibly exposes test wording in the main selector and every new account's first conversation.

Useful? React with 👍 / 👎.

feature:Merge chatbot config with new retrieval strategies

f52f2a3

chatgpt-codex-connector Bot reviewed Jul 29, 2026

View reviewed changes

	if st.button("Créer", use_container_width=True, key="create_conv"):
	cid = db.create_conversation(user_id, default_title, agent)
	st.session_state["conv_id"] = cid

		if "```r" in content:
		code = content.split("```r", 1)[1].split("```", 1)[0]

		with Session(_engine) as session:
		return session.exec(select(ChatbotConfig)).all()

Uh oh!

Conversation

ju3ouz4n commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

How to test

Screenshots (if UI changes)

Checklist

Uh oh!

ju3ouz4n commented Jul 29, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

ju3ouz4n Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

ju3ouz4n Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ju3ouz4n commented Jul 23, 2026 •

edited

Loading