Skip to content

Create llm_enhance repo: extract LLM enhancement from phpMan with Source/Transform/Sink abstraction #222

Description

@chedong

Context

Following 2026-07-14 architectural simplification (docs/01-PRODUCT.md §2.12 + docs/05-PLAN.md §"Migration Plan"), LLM enhancement moves out of phpMan into a standalone repo. Must serve multiple content sources, not just phpMan:

  • phpMan: SQLite cache → MD files → enhanced MD
  • myblog: MySQL (Movable Type) → MD files → enhanced MD
  • future: filesystem, HTTP API, etc.

Goal

llm_enhance repo with source-agnostic abstraction. Same LLM transforms + task manager serves all projects.

Key design (full spec docs/05-PLAN.md §1.2-1.5)

Three ABCs:

  • Source: yields Entry objects. Impls: MysqlExportSource (myblog), PhpmanCacheSource (phpMan), FilesystemSource, HttpApiSource.
  • Transform: Entry → enhanced Entry. Versioned (e.g. emoji@v2). Impls: EmojiTransform, OkfTransform, TranslationTransform.
  • Sink: persists results. Impls: FilesystemSink, SqliteSink (phpMan compat), HttpApiSink.

Plus:

  • TaskManager: SQLite-backed queue, resume, rate limiting, cost tracking
  • LLMClient: multi-provider (OpenAI-compat, Anthropic), retry, truncation detection
  • Prompt loader: versioned prompts in src/llm_enhance/prompts/

Migration source map (Phase B)

From phpMan.phpllm_enhance:

From To
enhanceManPage() transform/emoji.py:EmojiTransform.apply()
callLLM() llm/client.py:LLMClient.call()
cleanEmojiHtml() transform/emoji.py:post_process()
cli/batch-enhance.php cli.py:run_command()
LLM config (5 keys) per-project YAML configs

Deleted (only existed for LLM path):

  • formatMarkdownToHTML(), formatInlineMarkdown()
  • renderTocSidebar()

Steps (Phase A+B+C+D, ~3-4 weeks)

  1. W1: Create repo, pyproject.toml, src layout, ABCs + tests
  2. W2: Port cli/batch-enhance.php → Python CLI (queue/resume/rate-limit)
  3. W2-3: Implement MysqlExportSource for myblog MT 4.x/5.x schema
  4. W3: Implement PhpmanCacheSource reading ~/.phpman/cache/db.sqlite
  5. Test: llm_enhance run --config=myblog.yaml --limit=10 --dry-run and same for phpMan

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions