diff --git a/CLAUDE.md b/CLAUDE.md index 7fb39c4e..4fc4be3c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -3,49 +3,118 @@ This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. ## Project Overview -TelegramSearchBot is a .NET 9.0 console application that provides Telegram bot functionality for group chat message storage, search, and AI processing. It supports traditional keyword search via Lucene.NET and semantic search via FAISS vectors. +TelegramSearchBot is a .NET 10.0 console application that provides Telegram bot functionality for group chat message storage, search, and AI processing. It supports traditional keyword search via Lucene.NET and semantic search via FAISS vectors. ## Architecture -- **Message Processing Pipeline**: MediatR-based event handling with async message processing -- **Storage**: SQLite + EF Core 9.0 for data, Lucene.NET for full-text search, FAISS for vectors -- **AI Services**: OCR (PaddleOCR), ASR (Whisper), LLM (Ollama/OpenAI/Gemini) +- **Message Processing Pipeline**: IOnUpdate-based controller pattern with dependency-based topological sorting +- **Storage**: SQLite + EF Core 10.0 for data, Lucene.NET for full-text search, FAISS for vectors +- **AI Services**: OCR (PaddleOCR), ASR (Whisper), LLM (Ollama/OpenAI/Gemini/Anthropic) - **Multi-Modality**: Handles text, images, audio, video with automatic content extraction -- **Background Tasks**: Coravel scheduler for periodic indexing and processing +- **Background Tasks**: IScheduledTask-based scheduler with heartbeat monitoring +- **MCP Integration**: Model Context Protocol support for external tool servers (24+ built-in tools) +- **Multi-Process**: OCR/ASR run as separate processes managed by AppBootstrap (Windows JobObject) -## Build Commands +## Solution Structure +``` +TelegramSearchBot.sln (8 projects) +├── TelegramSearchBot # Main console app (entry: Program.cs) +├── TelegramSearchBot.Common # Shared config (Env.cs), attributes, models +├── TelegramSearchBot.Database # EF Core DbContext, migrations +├── TelegramSearchBot.Search # Lucene.NET search engine封装 +├── TelegramSearchBot.LLM # LLM services, MCP client/manager +├── TelegramSearchBot.Test # Core unit/integration tests +├── TelegramSearchBot.Search.Test # Search engine tests +└── TelegramSearchBot.LLM.Test # LLM service tests +``` + +## WHERE TO LOOK +| Task | Location | Notes | +|------|----------|-------| +| Bot commands | `Controller/` | Implement IOnUpdate, declare Dependencies | +| Search logic | `Service/Search/` + `TelegramSearchBot.Search/` | Lucene + vector hybrid | +| AI/LLM | `Service/AI/LLM/` + `TelegramSearchBot.LLM/` | Ollama/OpenAI/Gemini | +| Vector search | `Service/Vector/` | FaissVectorService, ConversationVectorService | +| Config | `TelegramSearchBot.Common/Env.cs` | Config.json at %LOCALAPPDATA%/TelegramSearchBot/ | +| MCP tools | `TelegramSearchBot.LLM/Service/Tools/` | BuiltInToolAttribute-marked methods | +| Scheduled tasks | `Service/Scheduler/` | Implement IScheduledTask with heartbeat | + +## KEY PATTERNS + +### Controller (IOnUpdate) Pattern +```csharp +public class MyController : IOnUpdate { + public List Dependencies => new() { typeof(DependencyController) }; + + public async Task OnUpdate(Update update, PipelineContext context) { + // Use context.PipelineCache to share data across controllers + } +} +``` + +### Service Registration +- DI via Scrutor scanning: `IOnUpdate`, `IService`, `IView` +- Use `[Injectable(ServiceLifetime.Singleton)]` attribute +- Services should be namespace under TelegramSearchBot for scanning + +### MCP Tool Definition +```csharp +[BuiltInTool(Name = "tool_name", Description = "...")] +public async Task ToolMethod([BuiltInParameter(Name = "param")] string value) { + // Tool implementation +} +``` + +### Background Task +```csharp +[Injectable(ServiceLifetime.Singleton)] +public class MyTask : IScheduledTask { + public string CronExpression => "0 * * * *"; // hourly + + public async Task ExecuteAsync(CancellationToken ct) { + SetHeartbeatCallback(() => { /* keep alive */ }); + // Task logic + } +} +``` + +## BUILD COMMANDS ```bash -# Restore dependencies +# Restore & Build dotnet restore TelegramSearchBot.sln - -# Build solution dotnet build TelegramSearchBot.sln --configuration Release # Run tests dotnet test +dotnet test --filter "Category=Vector" # Vector-specific tests +pwsh TelegramSearchBot.Test/RunVectorTests.ps1 -# Run specific test category -dotnet test --filter "Category=Vector" - -# Publish for Windows (current target) +# Publish dotnet publish -r win-x64 --self-contained ``` -## Key Configuration -- **Config Location**: `%LOCALAPPDATA%/TelegramSearchBot/Config.json` -- **Required**: `BotToken`, `AdminId` -- **AI Models**: Configurable via `OllamaModelName`, `OpenAIModelName` -- **Features**: `EnableAutoOCR`, `EnableAutoASR`, `EnableVideoASR` - ## Development Notes -- **Platform**: Currently Windows-only due to runtime identifiers and native dependencies -- **Tests**: xUnit with Moq, EF Core InMemory for database testing -- **Logging**: Serilog with console, file, and OpenTelemetry sinks -- **Database**: EF Core migrations for SQLite schema management - -## Important Paths -- **Main Entry**: `TelegramSearchBot/Program.cs` -- **Configuration**: `TelegramSearchBot/Env.cs` -- **Controllers**: `TelegramSearchBot/Controller/` - Handle bot commands -- **Services**: `TelegramSearchBot/Service/` - Core business logic -- **Models**: `TelegramSearchBot/Model/` - EF entities and DTOs -- **Tests**: `TelegramSearchBot.Test/` - xUnit tests \ No newline at end of file +- **Platform**: Windows primary, Linux partial (OCR/ASR limited) +- **Config**: `%LOCALAPPDATA%/TelegramSearchBot/Config.json` - DO NOT use appsettings.json +- **Logging**: Serilog (console + file + OpenTelemetry) +- **Database**: SQLite at `Env.WorkDir/Data.sqlite`, EF Core migrations in `Migrations/` +- **Vector indexes**: `Env.WorkDir/faiss_indexes/` + +## Anti-Patterns (THIS PROJECT) +1. **Don't hardcode ports** - Use `Env.SchedulerPort` for Garnet/Redis connections +2. **Don't use static vars for cross-controller data** - Use `PipelineContext.PipelineCache` +3. **Don't forget Dependencies** - Missing declarations cause "Circular dependency detected" +4. **Don't skip heartbeat in scheduled tasks** - Will be marked as stuck +5. **Don't modify vector structure without rebuilding** - Run `/faiss重建` after changes + +## Common Gotchas +- New controllers need namespace in TelegramSearchBot for DI scanning to work +- Static Env properties require Config.json in correct location +- MCP tools must be in same assembly for auto-registration +- Long-running tasks (OCR/download) should go to Service/Scheduler or separate process +- When modifying config, update corresponding Docs/README_*.md files + +## Existing Documentation +- `.github/copilot-instructions.md` - Detailed Chinese dev guide (read this first) +- `Docs/Bot_Commands_User_Guide.md` - User-facing command reference +- `Docs/Architecture_Overview.md` - System architecture +- `Docs/Build_and_Test_Guide.md` - Build/test instructions \ No newline at end of file diff --git a/Docs/Architecture_Overview.md b/Docs/Architecture_Overview.md index b45fa715..b6b00512 100644 --- a/Docs/Architecture_Overview.md +++ b/Docs/Architecture_Overview.md @@ -51,7 +51,7 @@ TelegramSearchBot 是一个功能丰富的 Telegram 机器人,提供消息搜 ## 技术栈 ### 核心技术 -- **运行时**: .NET 9.0 +- **运行时**: .NET 10.0 - **框架**: ASP.NET Core 托管模型 - **数据库**: SQLite (Entity Framework Core) - **搜索引擎**: Lucene.NET @@ -148,7 +148,7 @@ Telegram 更新 → 命令解析 → 控制器分发 → 业务处理 → 结果 ### 容器化部署 - **Docker**: 支持 Docker 容器化 -- **镜像**: 基于 .NET 9.0 运行时 +- **镜像**: 基于 .NET 10.0 运行时 - **编排**: 支持 Docker Compose ### 高可用部署 (未来规划) @@ -248,7 +248,7 @@ TelegramSearchBot/ ## 版本兼容性 ### .NET 版本 -- **当前**: .NET 9.0 +- **当前**: .NET 10.0 - **最低**: .NET 8.0 (LTS) - **升级策略**: 跟随 LTS 版本 diff --git a/Docs/Build_and_Test_Guide.md b/Docs/Build_and_Test_Guide.md index 5cbe934e..bd493c5c 100644 --- a/Docs/Build_and_Test_Guide.md +++ b/Docs/Build_and_Test_Guide.md @@ -3,7 +3,7 @@ ## 快速开始 ### 环境要求 -- **.NET SDK**: 9.0 或更高版本 +- **.NET SDK**: 10.0 或更高版本 - **操作系统**: Windows 10/11 (完整功能)、Linux (部分功能限制)、macOS (实验性) - **数据库**: SQLite (内置支持) @@ -98,7 +98,7 @@ dotnet test --logger trx --results-directory TestResults dotnet tool install --global coverlet.console # 运行带覆盖率的测试 -coverlet TelegramSearchBot.Test/bin/Release/net9.0/TelegramSearchBot.Test.dll --target "dotnet" --targetargs "test --no-build" +coverlet TelegramSearchBot.Test/bin/Release/net10.0/TelegramSearchBot.Test.dll --target "dotnet" --targetargs "test --no-build" ``` ### 开发环境设置 @@ -161,7 +161,7 @@ $env:DATABASE_PATH="./data/bot.db" #### 常见构建问题 -**问题**: 找不到 .NET 9.0 SDK +**问题**: 找不到 .NET 10.0 SDK ```bash # 检查已安装的 SDK dotnet --list-sdks @@ -218,7 +218,7 @@ jobs: - name: Setup .NET uses: actions/setup-dotnet@v3 with: - dotnet-version: '9.0.x' + dotnet-version: '10.0.x' - name: Restore dependencies run: dotnet restore - name: Build diff --git a/Docs/Windows_Dependencies_Analysis.md b/Docs/Windows_Dependencies_Analysis.md index 2d99db24..2f9a1b1d 100644 --- a/Docs/Windows_Dependencies_Analysis.md +++ b/Docs/Windows_Dependencies_Analysis.md @@ -102,10 +102,10 @@ ### Dockerfile 示例 ```dockerfile -FROM mcr.microsoft.com/dotnet/aspnet:9.0 AS base +FROM mcr.microsoft.com/dotnet/aspnet:10.0 AS base WORKDIR /app -FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build +FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build WORKDIR /src COPY ["TelegramSearchBot/TelegramSearchBot.csproj", "TelegramSearchBot/"] RUN dotnet restore "TelegramSearchBot/TelegramSearchBot.csproj" diff --git a/README.md b/README.md index 480c698b..b301613b 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ ![Build Status](https://github.com/ModerRAS/TelegramSearchBot/actions/workflows/push.yml/badge.svg) ## 功能列表 -1. 群聊消息存储并支持中文分词搜索 (Lucene) +1. 群聊消息存储并支持中文分词搜索 (Lucene.NET) 2. **向量搜索功能 (FAISS)**: 基于对话段的语义搜索,无需额外服务依赖 3. 群聊消息中多媒体内容自动处理: - 图片自动下载并OCR存储 (PaddleOCR) @@ -13,8 +13,9 @@ - 发送图片附带`打印`指令时自动OCR回复 4. 大语言模型集成: - Ollama本地模型 - - OpenAI API - - Gemini API + - OpenAI API (GPT-4o, GPT-4o-mini等) + - Google Gemini API + - Anthropic Claude API - 可配置多模型通道管理 - **MCP (Model Context Protocol) 工具支持** 5. 高级功能: @@ -23,6 +24,8 @@ - 记忆图谱功能 - **内置Telegram Bot API服务支持(2GB大文件/50MB云端)** - 群组黑名单/设置管理 + - Sequential Thinking 思考链支持 + - Brave 搜索集成 详细功能说明请参考: [Docs/Bot_Commands_User_Guide.md](Docs/Bot_Commands_User_Guide.md) @@ -139,11 +142,32 @@ graph TD E --> N[Ollama] E --> O[OpenAI] E --> P[Gemini] + E --> PA[Anthropic] Q --> R[MCP Servers] R --> S[External Tools] ``` -详细架构设计: [Docs/Existing_Codebase_Overview.md](Docs/Existing_Codebase_Overview.md) +详细架构设计: [Docs/Architecture_Overview.md](Docs/Architecture_Overview.md) + +## 技术栈 +- **运行时**: .NET 10.0 +- **数据库**: SQLite + EF Core 10.0 +- **搜索**: Lucene.NET (全文) + FAISS (向量) +- **AI**: Ollama, OpenAI, Gemini, Anthropic + +## 构建与运行 +```bash +# 构建 +dotnet build TelegramSearchBot.sln --configuration Release + +# 运行 +dotnet run --project TelegramSearchBot + +# 发布 +dotnet publish -r win-x64 --self-contained +``` + +详细文档: [Docs/Build_and_Test_Guide.md](Docs/Build_and_Test_Guide.md) ## License 这里曾经是一个FOSSA Status的,但是因为经常报错烦了,遂删之。 diff --git a/TelegramSearchBot.LLM/AGENTS.md b/TelegramSearchBot.LLM/AGENTS.md new file mode 100644 index 00000000..b8857b97 --- /dev/null +++ b/TelegramSearchBot.LLM/AGENTS.md @@ -0,0 +1,54 @@ +# LLM Layer + +LLM services, MCP client/manager, and built-in tool implementations. + +## OVERVIEW +Manages LLM providers (OpenAI, Ollama, Gemini, Anthropic) and MCP tool integration. + +## STRUCTURE +``` +TelegramSearchBot.LLM/ +├── Interface/ # I*Service interfaces +├── Model/ # DTOs and models +└── Service/ + ├── AI/LLM/ # LLM provider implementations + ├── Mcp/ # MCP client & server manager + └── Tools/ # Built-in tool services +``` + +## WHERE TO LOOK +| Task | Location | Notes | +|------|----------|-------| +| LLM factory | `Service/AI/LLM/LLMFactory.cs` | Create LLM instances | +| MCP client | `Service/Mcp/McpClient.cs` | MCP protocol client | +| Tool registration | `Service/AI/LLM/McpToolHelper.cs` | Scan [BuiltInTool] | +| OpenAI service | `Service/AI/LLM/OpenAIService.cs` | OpenAI API wrapper | + +## KEY PATTERNS + +### Built-in Tool Definition +```csharp +[BuiltInTool(Name = "tool_name", Description = "...")] +public async Task ToolMethod( + [BuiltInParameter(Name = "param")] string value) { + // Implementation +} +``` + +### LLM Provider +```csharp +public class MyLLMService : ILLMService { + public async Task GenerateAsync(string prompt, ...) { } +} +``` + +## CONVENTIONS +- Tools must be in same assembly for auto-registration +- Use `[BuiltInTool]` / `[BuiltInParameter]` (NOT deprecated `[McpTool]`) +- LLM services implement interface from `Interface/` folder +- MaxToolCycles defaults to 25 to prevent infinite loops + +## MCP INTEGRATION +- MCP servers managed by `McpServerManager` +- Tools exposed via MCP protocol to LLM +- External servers configured via bot commands diff --git a/TelegramSearchBot.Search/AGENTS.md b/TelegramSearchBot.Search/AGENTS.md new file mode 100644 index 00000000..15ed50be --- /dev/null +++ b/TelegramSearchBot.Search/AGENTS.md @@ -0,0 +1,31 @@ +# Search Layer + +Lucene.NET full-text search engine and query processing. + +## OVERVIEW +Encapsulates Lucene.NET for keyword search, with support for advanced query syntax. + +## STRUCTURE +``` +TelegramSearchBot.Search/ +├── Model/ # DTOs (SearchMessageDTO, SearchType) +├── Service/ # Search service implementations +├── Tokenizer/ # Text tokenization +└── Tool/ # LuceneManager, query builders +``` + +## WHERE TO LOOK +| Task | Location | Notes | +|------|----------|-------| +| Lucene wrapper | `Tool/LuceneManager.cs` | Index read/write | +| Search service | `Service/SimpleSearchService.cs` | Query execution | +| Query building | `Tool/*QueryBuilder.cs` | Query construction | + +## CONVENTIONS +- Index stored in `Env.WorkDir/lucene_index/` +- Use `DocumentMessageMapper` to convert EF entities to Lucene docs +- Search results cached via `SearchPageCacheCleanupTask` + +## ANTI-PATTERNS +- Don't modify index structure without rebuilding (run `/重建索引`) +- Don't call LuceneManager from controllers - use Service/Search layer diff --git a/TelegramSearchBot.Test/AGENTS.md b/TelegramSearchBot.Test/AGENTS.md new file mode 100644 index 00000000..430e2c11 --- /dev/null +++ b/TelegramSearchBot.Test/AGENTS.md @@ -0,0 +1,46 @@ +# Test Layer + +Unit and integration tests for TelegramSearchBot. + +## OVERVIEW +Three test projects: TelegramSearchBot.Test, TelegramSearchBot.Search.Test, TelegramSearchBot.LLM.Test + +## STRUCTURE +``` +TelegramSearchBot.Test/ +├── Service/ # Service layer tests +├── View/ # View tests +├── Helper/ # Utility tests +└── RunVectorTests.ps1 # Vector test runner + +TelegramSearchBot.Search.Test/ # Search-specific tests +TelegramSearchBot.LLM.Test/ # LLM service tests +``` + +## FRAMEWORKS +- **Test framework**: xUnit ([Fact], [Theory]) +- **Mocking**: Moq for interfaces +- **Database**: EF Core InMemory for isolated DB tests +- **Categories**: `[Trait("Category", "Vector")]` for filtered execution + +## WHERE TO LOOK +| Task | Location | Notes | +|------|----------|-------| +| Vector tests | `Service/Vector/*.cs` | FAISS integration | +| DB context | `Service/Database/DataDbContextTests.cs` | Entity mapping | +| LLM tests | `TelegramSearchBot.LLM.Test/` | Service mocking | + +## CONVENTIONS +- Use unique database names per test: ` $"InMemoryDb_{Guid.NewGuid()}" ` +- Clean up temp files/directories in `[Fact]` cleanup +- Mock `ILogger` and `IServiceProvider` for service tests + +## RUN COMMANDS +```bash +# All tests +dotnet test + +# Vector tests only +dotnet test --filter "Category=Vector" +pwsh TelegramSearchBot.Test/RunVectorTests.ps1 +``` diff --git a/TelegramSearchBot/Service/AGENTS.md b/TelegramSearchBot/Service/AGENTS.md new file mode 100644 index 00000000..fa2af2bc --- /dev/null +++ b/TelegramSearchBot/Service/AGENTS.md @@ -0,0 +1,47 @@ +# Service Layer + +Core business logic implementations for TelegramSearchBot. + +## OVERVIEW +Service layer handles all business logic. Controllers delegate to services - services should be stateless. + +## STRUCTURE +``` +Service/ +├── AI/ # OCR, ASR, QR processing +├── BotAPI/ # Telegram API integration +├── Bilibili/ # B站视频/动态处理 +├── Common/ # Shared utilities (URL, config) +├── Manage/ # Admin configuration services +├── Scheduler/ # Background tasks & cron jobs +├── Search/ # Search query processing +├── Storage/ # Message persistence +├── Tools/ # MCP tool implementations +└── Vector/ # FAISS vector operations +``` + +## WHERE TO LOOK +| Task | Location | Notes | +|------|----------|-------| +| Send messages | `BotAPI/SendService.cs` | Reply/edit messages | +| Message storage | `Storage/MessageService.cs` | EF Core operations | +| Vector search | `Vector/FaissVectorService.cs` | FAISS index management | +| Background tasks | `Scheduler/SchedulerService.cs` | Cron-based execution | +| MCP tools | `Tools/*.cs` | 24+ built-in tools | + +## KEY INTERFACES +```csharp +IScheduledTask // Background task (implement with heartbeat) +IService // Scanned by Scrutor for DI +``` + +## CONVENTIONS +- Services should be injectable via `[Injectable]` +- Use `PipelineContext.PipelineCache` to receive data from controllers +- Don't hardcode ports - use `Env.SchedulerPort` +- Long-running operations should be async or offloaded to Scheduler + +## ANTI-PATTERNS +- Don't store state in services - they may be singleton +- Don't call controllers from services (circular dependency) +- Don't block on async operations without CancellationToken diff --git a/claude-swarm.yml b/claude-swarm.yml index 62cdb1d0..298a4264 100644 --- a/claude-swarm.yml +++ b/claude-swarm.yml @@ -25,7 +25,7 @@ swarm: 5. 跨模块技术问题解决 技术栈关注: - - .NET 9.0 + C# 现代开发实践 + - .NET 10.0 + C# 现代开发实践 - AI服务集成架构 (OCR/ASR/LLM) - 搜索系统优化 (Lucene.NET + FAISS) - 高性能消息处理管道 @@ -58,7 +58,7 @@ swarm: 技术关注点: - MediatR事件驱动架构 - - EF Core 9.0 数据访问层 + - EF Core 10.0 数据访问层 - Lucene.NET 全文搜索优化 - FAISS 向量搜索集成 - 异步消息处理管道 @@ -232,7 +232,7 @@ swarm: 你是TelegramSearchBot的数据库专家,负责SQLite数据库设计、EF Core优化和索引管理。 核心职责: - 1. EF Core 9.0 数据模型设计 + 1. EF Core 10.0 数据模型设计 2. 数据库迁移和版本管理 3. 查询性能优化 4. 数据库索引设计