Skip to content

feat(recall): bilingual keyword search for cross-language recall - #302

Merged
jeff-r2026 merged 2 commits into
Tencent:mainfrom
m0Nst3r873:feature/recall-cross-language
Aug 20, 2026
Merged

feat(recall): bilingual keyword search for cross-language recall#302
jeff-r2026 merged 2 commits into
Tencent:mainfrom
m0Nst3r873:feature/recall-cross-language

Conversation

@m0Nst3r873

@m0Nst3r873 m0Nst3r873 commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Problem

Team knowledge recall is purely lexical (BM25 token matching over a
shared tokenize() on both the index and query sides). The knowledge
base mixes Chinese and English — a learning may be titled in English
while a task is described in Chinese, or vice versa. Because Chinese
query tokens and English document tokens are different strings, they
never match: a Chinese-only query cannot recall an English-only
document, and vice versa.

Approach

Rather than maintain a hardcoded synonym dictionary (zero coverage of
long-tail terms, ongoing maintenance burden), this leverages the LLM's
own translation ability at keyword-selection time. The recall agent is
instructed to expand every meaningful domain term into both
languages
when building its search keywords, so a single query
carries tokens that can match documents in either language.

  • Chinese task → also add the English term (重启推理集群restart inference cluster)
  • English task → also add the Chinese term (restart inference cluster重启 推理 集群)
  • Any language-neutral token — anything written the same way in both
    languages or with no meaningful translation (brand/product names,
    code identifiers and symbols, error codes, version strings, file paths,
    CLI commands, API names, e.g. AppID, Kubernetes, GPU,
    RuntimeError, v2.1.3, oauth) — is kept as-is, since translating it
    only hurts matching.

No code-path or index changes, no SEARCH_INDEX_VERSION bump, no
rebuild required — this is purely a prompt/rule refinement.

Changes

  • agents/teamai-recall.md — Step 2 (keyword extraction) now instructs bilingual expansion, keeping language-neutral tokens untranslated.
  • src/builtin-rules.ts — the built-in teamai-recall rule's Bash-command usage note gets the same bilingual guidance, so the direct teamai recall path benefits too.

Test Plan

  • npx tsc --noEmit — passes.
  • Behavioral check: fed tasks through the new Step 2 prompt and verified the generated teamai recall command expands domain terms bilingually while keeping language-neutral tokens intact:
    • Chinese 升级到 v2.1.3 后 nginx 报 502 Bad Gateway,如何排查v2.1.3 nginx 502 Bad Gateway 排查 troubleshoot 升级 upgrade (version / brand / error code kept as-is)
    • English the oauth token refresh throws InvalidGrantError in productionoauth token refresh 刷新 InvalidGrantError production 生产环境 (protocol / error code kept as-is)
    • Path/command 修改 src/utils/tokenizer.ts 的分词逻辑导致索引失效src/utils/tokenizer.ts 分词 tokenizer 索引 index 失效 invalidation (file path kept as-is)
    • Mixed 为什么 kubectl apply 部署推理服务时 pod 一直 CrashLoopBackOffkubectl apply 推理 inference 服务 service CrashLoopBackOff pod 部署 deploy (CLI command / pod state kept as-is)

🤖 Generated with Claude Code

m0Nst3r873 and others added 2 commits August 20, 2026 14:47
The knowledge base mixes Chinese and English, but recall is purely
lexical (BM25 token matching) — a Chinese-only query never matches an
English-only document, and vice versa. Instead of a hardcoded synonym
dictionary, instruct the recall agent to expand each domain term into
both languages when picking search keywords, leveraging the LLM's own
translation. Proper nouns and identifiers (AppID, GPU, HAI) are kept
as-is.

Updates the teamai-recall subagent prompt (Step 2) and the built-in
teamai-recall rule (Bash-command usage note) so both the subagent and
direct `teamai recall` paths search bilingually.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the closed list of "proper nouns, acronyms, identifiers" with a
general "language-neutral token" principle: keep as-is anything written
the same way in both languages or with no meaningful translation — brand
and product names, code identifiers and symbols, error codes, version
strings, file paths, CLI commands, and API names. This covers boundary
tokens the earlier enumeration missed (error codes, versions, paths,
commands) instead of relying on a fixed set of examples.

Also swap the business-specific product name in the examples for a
neutral, widely-known one so the built-in prompt/rule stays generic
across teams.

Applied symmetrically to the teamai-recall subagent prompt and the
built-in teamai-recall rule.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jeff-r2026
jeff-r2026 merged commit f047b2f into Tencent:main Aug 20, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants