Merge feat/rag-eval-v2 into master

[PacemakerG]

Merge the completed RAG evaluation v2 work into master, including RAG/routing/Redis evaluation datasets, evaluation scripts, semantic cache updates, tests, and documentation.

[Copilot]

Pull request overview

This PR merges the “RAG evaluation v2” work into master, updating the system architecture docs and implementation to reflect (a) three independent evaluation datasets/pipelines (RAG/routing/Redis semantic cache), (b) Redis Stack + Elasticsearch as the default retrieval/cache infrastructure, and (c) a refactor from tenant_id + user_id + session_id isolation to user_id + session_id isolation.

Changes:

• Replace the previous LangSmith v1 evaluation artifacts with a consolidated v2 evaluation suite (datasets, scripts, docs, and tests) covering RAG (150), routing (50), and Redis semantic cache (50 pairs).
• Introduce/standardize Redis Stack semantic cache + Elasticsearch BM25 (+ RRF) defaults, including local docker-compose services and updated backend config/env defaults.
• Remove tenant-scoping across frontend/backed APIs, state, models, and tests; update docs accordingly (workflow nodes, identity, and evaluation descriptions).

Reviewed changes

Copilot reviewed 103 out of 109 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
README.zh-CN.md	Updates product/architecture description to match v2 workflow (semantic cache + keyword_router) and v2 evaluation summary.
README.md	English README alignment with v2 architecture, Redis semantic cache, and evaluation suite.
frontend/src/App.test.jsx	Adjusts frontend integration tests to remove tenant_id from auth/me expectations.
frontend/src/App.jsx	Removes tenant UI/storage/headers/payloads; identity is now user_id + session_id.
docs/evaluation/评测结果.md	Adds v2 evaluation results report (RAG/routing/Redis).
docs/evaluation/评测方案.md	Adds v2 evaluation protocol/spec (datasets/contracts/metrics).
docs/evaluation/rag_eval_report_20260322.md	Removes older v1 RAG report doc.
docs/evaluation/rag_eval_method.md	Removes older v1 evaluation method doc.
docs/evaluation/rag_agent_profile_report_20260322.md	Removes older v1 profiling report doc.
docs/evaluation/langsmith_eval_report_v1.md	Removes older v1 LangSmith full-chain report doc.
docs/evaluation/langsmith_eval_dataset_v1_summary.md	Removes older v1 dataset summary doc.
docs/competition/technical_solution.md	Updates technical solution doc to reflect semantic cache and binary intent routing.
docker-compose.yml	Adds Redis Stack + Elasticsearch services and wires backend defaults for eval-ready infra.
CLAUDE.md	Updates repo guidance to reflect v2 workflow, identity model, test inventory, and evaluation scripts.
backend/tests/test_workflow.py	Updates routing tests after removal of nutrition/fitness/sleep domains.
backend/tests/test_workflow_routing.py	Updates routing tests and asserts new nodes exist / old nodes removed.
backend/tests/test_services.py	Fixes service import paths and updates conversation-state key format without tenant.
backend/tests/test_semantic_cache_service.py	Replaces prior semantic-cache tests with Redis Stack vector/entity-filter contract tests.
backend/tests/test_semantic_cache_redis_integration.py	Adds optional real Redis Stack integration test (guarded by env var).
backend/tests/test_profile_service.py	Updates profile persistence/isolation rules to user-only scope; adjusts expected filenames.
backend/tests/test_langsmith_eval_tools.py	Removes v1 LangSmith eval tools tests (superseded by v2 suite).
backend/tests/test_evaluation_tools.py	Adds unit tests for v2 dataset building and core eval helpers (rrf/metrics/routing).
backend/tests/test_ecg_monitor_service.py	Removes tenant from ECG monitor worker/status isolation.
backend/tests/test_ecg_api.py	Updates request-context mock to remove tenant.
backend/tests/test_database.py	Updates DB user creation/lookup to be user-scoped (no tenant).
backend/tests/test_auth_api.py	Updates auth API tests to remove tenant fields from requests/responses.
backend/tests/test_api.py	Fixes service imports and removes tenant assertions in session deletion tests.
backend/tests/test_api_edge_cases.py	Fixes service imports after service export refactor.
backend/tests/conftest.py	Updates service imports to explicit modules.
backend/scripts/upload_langsmith_dataset.py	Removes v1 upload script (replaced by v2 uploader).
backend/scripts/evaluation/upload_langsmith.py	Adds v2 uploader for three datasets (rag/routing/redis) with per-kind payload contracts.
backend/scripts/evaluation/evaluate_routing.py	Adds v2 full-chain routing evaluation runner (compiled workflow + live deps).
backend/scripts/evaluation/evaluate_redis_cache.py	Adds v2 Redis semantic cache evaluation (decision accuracy + latency reduction).
backend/scripts/build_rag_gold_dataset.py	Removes v1 gold dataset curation script.
backend/scripts/build_rag_eval_dataset.py	Removes v1 deterministic RAG dataset generator script.
backend/pyproject.toml	Updates Python requirement range, dependencies (numpy/redis), and packaging include settings.
backend/data/knowledge/医学知识库官方下载来源.md	Adds official-source manifest for locally kept PDFs (not committed).
backend/data/eval/redis/result.json	Adds/commits v2 Redis evaluation result artifact.
backend/data/eval/rag_eval_samples.jsonl	Removes legacy sample file.
backend/data/eval/rag_eval_result_v1.json	Removes legacy evaluation result file.
backend/data/eval/langsmith_upload_status_v1.json	Removes legacy upload status file.
backend/app/tools/vector_store.py	Adds persisted embedding-model metadata and stale-store rebuild behavior.
backend/app/tools/llm_client.py	Removes tenant from model routing/LLM instance caching keys and config resolution.
backend/app/tools/keyword_retriever.py	Adds optional “search all departments” mode and cache-key scoping for it.
backend/app/tools/es_keyword_retriever.py	Adds “search all departments” mode and removes tenant metadata usage.
backend/app/tools/es_client.py	Removes tenant field from ES mapping and indexed payload.
backend/app/services/task_queue_service.py	Removes tenant fields from job creation/submission metadata.
backend/app/services/redis_service.py	Makes Redis client binary-safe (decode_responses=False) and decodes JSON payloads explicitly.
backend/app/services/profile_service.py	Scopes profile persistence by user_id only; includes legacy filename migration.
backend/app/services/langsmith_service.py	Removes tenant from LangSmith runnable config metadata/configurable fields.
backend/app/services/greeting_service.py	Removes tenant from greeting flow (history/profile/LLM calls).
backend/app/services/flow_trace_service.py	Removes safety-level fields from flow-trace notes/signature.
backend/app/services/ecg_report_service.py	Removes tenant from ECG report generation/persistence/profile updates.
backend/app/services/ecg_monitor_service.py	Removes tenant from ECG monitor lifecycle and status isolation.
backend/app/services/auth_service.py	Refactors auth to be user-scoped; removes tenant token fields and DB scoping.
backend/app/services/init.py	Switches to lazy service exports to avoid import cycles.
backend/app/schemas/chat.py	Removes tenant_id from job status response schema.
backend/app/schemas/auth.py	Removes tenant_id from login request and auth status response schema.
backend/app/models/user.py	Removes tenant column and uniqueness constraint; makes user_id globally unique.
backend/app/models/message.py	Removes tenant column from message storage model.
backend/app/models/ecg_report.py	Removes tenant column from ECG report storage model.
backend/app/main.py	Removes tenant metadata from PDF ingestion metadata map.
backend/app/core/state.py	Removes tenant_id and safety_level from AgentState defaults/fields.
backend/app/core/langsmith_service.py	Removes tenant from LangSmith runnable config (core copy).
backend/app/core/langgraph_workflow.py	Replaces health_concierge with keyword_router and updates routing edges/logic.
backend/app/core/config.py	Changes defaults: embedding model, query rewriter disabled, ES enabled by default, semantic-cache settings expanded.
backend/app/api/v1/request_context.py	Removes tenant resolution; identity is token/cookie/header-based user+session.
backend/app/api/v1/endpoints/session.py	Removes tenant from session/history endpoint DB calls.
backend/app/api/v1/endpoints/ecg.py	Removes tenant from ECG endpoints and service calls.
backend/app/api/v1/endpoints/chat.py	Removes tenant from rate-limit identity and service calls; tightens job access check to user-only.
backend/app/api/v1/endpoints/auth.py	Removes tenant from auth responses and login rate-limit identity.
backend/app/agents/retriever.py	Adds “search all departments” capability when general-medical is selected; simplifies medical scope resolution.
backend/app/agents/query_rewriter.py	Tightens query list when query rewriter is disabled; removes tenant from LLM calls; adds language-preservation constraint.
backend/app/agents/planner.py	Replaces LLM-based domain/safety triage with deterministic binary medical-intent keyword routing.
backend/app/agents/memory.py	Removes tenant from profile read/write calls.
backend/app/agents/medical_router.py	Improves LLM prompt with taxonomy boundary hints; removes tenant from light-LLM call.
backend/app/agents/judge_need_rag.py	Removes tenant from light-LLM call.
backend/app/agents/executor.py	Removes safety-level shortcut logic and moves ECG service import to lazy import; removes tenant from LLM calls.
backend/.python-version	Sets local Python version to 3.11.
backend/.env.example	Updates env defaults to match v2 (ES/Redis enabled, semantic cache settings, embedding model).
.gitignore	Ignores all local medical PDFs under backend/data/knowledge/**/*.pdf.

Suppressed comments (1)

backend/pyproject.toml:62

• requires-python is set to ">=3.11,<3.13", but Black is configured with target-version = ['py310']. This mismatch can lead to formatting decisions that don’t match the supported runtime (e.g., newer syntax formatting). Update Black’s target-version to match the project’s supported Python (e.g., ['py311']) or adjust requires-python to keep them consistent.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.